Flexible Python Automated Testing with RST

The problem with test.py

The test.py method of Python automated testing described at BitterSuitePythonTesting is sufficient for simple assignments. However, when the test.py file is run, the student's Python program has already been loaded, so it gives no control over how the student's Python program is evaluated. It's also hard to run code before running the student's program. The test.py files are not good for automatically testing certain Python assignments.

This page contains an example of how to do Python automated testing when test.py doesn't work well.

Our fake Python assignment

Question 1

Create a Python file alternate.py that defines a function called alternate. The function alternate consumes a Str and returns a Str obtained by alternating the casing of all letters (a-z) in the consumed Str. The first letter of the returned Str should be capital.

Example: alternate("hello! How Are You? Doing OK") should return "HeLlO! hOw ArE yOu? DoInG oK"

Question 2

Create a Python script alternate_keyboard.py that when run, reads lines from the keyboard (standard input) until EOF is reached. The script should print out each line it has read, but with the cases alternating. Each line should be treated independently, ie apply the alternate function from Q1 on each line that's read.

Example:

$ cat input.txt
hello! How Are You? Doing OK
this is my example!!!
I hope you do Well!!
$ python3 alternate-driver.py < input.txt
HeLlO! hOw ArE yOu? DoInG oK
ThIs Is My ExAmPlE!!!
I hOpE yOu Do WeLl!!

Question 3

Create a Python script alternate_keyboard_save.py that consumes one optional command line argument. The script behaves the same as alternate-keyboard.py, except that it saves the output to a file named output.txt in the current directory if no command line argument was provided. If a command line argument was provided, use it as the path to the output file.

Example:

$ cat input.txt
hello! How Are You? Doing OK
this is my example!!!
I hope you do Well!!
$ python3 alternate-driver.py < input.txt
$ cat output.txt
HeLlO! hOw ArE yOu? DoInG oK"
ThIs Is My ExAmPlE!!!
I hOpE yOu Do WeLl!!
$ python3 alternate-driver.py /tmp/my-output.txt < input.txt
$ cat /tmp/my-output.txt
HeLlO! hOw ArE yOu? DoInG oK"
ThIs Is My ExAmPlE!!!
I hOpE yOu Do WeLl!!

Sample solutions

Question 1 - alternate.py

import check

def alternate(mystr):
    answer = ""
    makeNextLetterUppercase = True
    for i in range(len(mystr)):
        char = mystr[i]
        if char.isalpha():
            if makeNextLetterUppercase:
                answer += char.upper()
            else:
                answer += char.lower()
            makeNextLetterUppercase = not makeNextLetterUppercase
        else:
            answer += char
    return answer

# Tests
if __name__ == "__main__":
    check.expect("empty", alternate(""), "")
    check.expect("example", alternate("hello! How Are You? Doing OK"),\
        "HeLlO! hOw ArE yOu? DoInG oK")
    check.expect("test1", alternate("this is my example!!!"),\
        "ThIs Is My ExAmPlE!!!")
    check.expect("test2", alternate("I hope you do Well!!"),\
        "I hOpE yOu Do WeLl!!")

Question 2 - alternate_keyboard.py

import sys
from alternate import alternate

for line in sys.stdin:
    print(alternate(line), end='')

Question 3 - alternate_keyboard_save.py

import sys

output_file_path = "output.txt"

if len(sys.argv) >= 2:
    output_file_path = sys.argv[1]

sys.stdout = open(output_file_path, "w")
import alternate_keyboard

Test cases the instructor wants to check

Students should be able to get credit for Q2 and Q3 even if they haven't done the previous questions.

Question 1

Check the test cases in the sample solutions.

Question 2

In the testing, use Q1 sample solutions in case the student couldn't do Q1.

Use the test input in the question, and also this one:

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do
eiusmod tempor incididunt ut labore et dolore magna aliqua. Tortor
aliquam nulla facilisi cras fermentum. Venenatis urna cursus eget
nunc scelerisque. Urna id volutpat lacus laoreet non. Dolor purus
non enim praesent elementum. Dui nunc mattis enim ut tellus eleme.

Question 3

In the testing, use Q1 and Q2 sample solutions in case the student couldn't do the previous questions.

Same test cases as in Q2. First test should not pass any command-line arguments. The second test should use a command-line argument.

Our test.pt (or test.0 or test.1) test suite

To keep things organized, we will create a brand new folder for each test case. We'll copy all the files we need into that folder and work from there.

Please keep in mind that providing solutions in test suites is a security risk. Even using pyc files is useless because pyc files can be easily decompiled.

A few notes about the test.exe environment (more details at BitterSuiteExternal):

• The test.exe is ran as csNNNt account from a temporary folder that's something like /tmp/.csNNN.a01.t.pt.t01/userid/66923sDZOaI
• The submitdir environment variable is path to a folder containing the student's submissions (it is NOT the /u/csNNN/handin/aXX/userid folder).
• The testdir environment variable is path to test suite in marking folder, ex /u/csNNN/marking/aXX/test.pt
• Anything printed to standard output will be available in the same directory as OUTPUT.txt
• If you create a file in test.exe that you want to keep, use KeepFile
• Set the test case score using echo 100 >&3 where 100 is the percentage that the student gets.
• Set the test case feedback using echo message >&4 where message is what you want to say to the student.

Download (attached below): https://cs.uwaterloo.ca/twiki/pub/ISG/BitterSuiteExternalPythonTesting/sample_test_suite.zip

• test.pt/
  • answers/
  • in/
    • 1/
      • t01/
        • options.rkt
        • test.exe
      • t02/
        • options.rkt
        • test.exe
      • t03/
        • options.rkt
        • test.exe
      • t04/
        • options.rkt
        • test.exe
    • 2/
      • t01/
        • options.rkt
        • test.exe
      • t02/
        • options.rkt
        • test.exe
    • 3/
      • t01/
        • options.rkt
        • test.exe
      • t02/
        • options.rkt
        • test.exe
    • options.rkt
  • solutions/
    • alternate.py
    • alternate_keyboard.py
    • check.py
  • test_data/
    • test_case_long_answer.txt
    • test_case_long_input.txt
    • test_case_short_answer.txt
    • test_case_short_input.txt

(args "q1_t01" "" "")

#!/bin/bash

readonly testname="$1"
readonly testinput="$2"
readonly correctoutput="$3"

mkdir "$testname"
cd "$testname"

if [ ! -f "${submitdir}/alternate.py" ]; then
    echo 0 >&3
    echo "File alternate.py was not submitted." >&4
    exit 0
fi

cp "${submitdir}/alternate.py" .
cp "${testdir}/solutions/check.py" .

timeout 5s /usr/bin/python3 -c "from alternate import alternate; print(alternate('${testinput}'))" > test_stdout.txt 2> test_stderr.txt
exit_code=$?
if [ $exit_code -eq 0 -a ! -s test_stderr.txt ]; then
    echo "$correctoutput" | diff - test_stdout.txt
    if [ $? -eq 0 ]; then
        echo 100 >&3
        exit 0
    fi
fi

echo 0 >&3

(args "q1_t02" "hello! How Are You? Doing OK" "HeLlO! hOw ArE yOu? DoInG oK")

#!/bin/bash

readonly testname="$1"
readonly testinput="$2"
readonly correctoutput="$3"

mkdir "$testname"
cd "$testname"

if [ ! -f "${submitdir}/alternate.py" ]; then
    echo 0 >&3
    echo "File alternate.py was not submitted." >&4
    exit 0
fi

cp "${submitdir}/alternate.py" .
cp "${testdir}/solutions/check.py" .

timeout 5s /usr/bin/python3 -c "from alternate import alternate; print(alternate('${testinput}'))" > test_stdout.txt 2> test_stderr.txt
exit_code=$?
if [ $exit_code -eq 0 -a ! -s test_stderr.txt ]; then
    echo "$correctoutput" | diff - test_stdout.txt
    if [ $? -eq 0 ]; then
        echo 100 >&3
        exit 0
    fi
fi

echo 0 >&3

(args "q1_t03" "this is my example!!!" "ThIs Is My ExAmPlE!!!")

#!/bin/bash

readonly testname="$1"
readonly testinput="$2"
readonly correctoutput="$3"

mkdir "$testname"
cd "$testname"

if [ ! -f "${submitdir}/alternate.py" ]; then
    echo 0 >&3
    echo "File alternate.py was not submitted." >&4
    exit 0
fi

cp "${submitdir}/alternate.py" .
cp "${testdir}/solutions/check.py" .

timeout 5s /usr/bin/python3 -c "from alternate import alternate; print(alternate('${testinput}'))" > test_stdout.txt 2> test_stderr.txt
exit_code=$?
if [ $exit_code -eq 0 -a ! -s test_stderr.txt ]; then
    echo "$correctoutput" | diff - test_stdout.txt
    if [ $? -eq 0 ]; then
        echo 100 >&3
        exit 0
    fi
fi

echo 0 >&3

(args "q1_t04" "I hope you do Well!!" "I hOpE yOu Do WeLl!!")

#!/bin/bash

readonly testname="$1"
readonly testinput="$2"
readonly correctoutput="$3"

mkdir "$testname"
cd "$testname"

if [ ! -f "${submitdir}/alternate.py" ]; then
    echo 0 >&3
    echo "File alternate.py was not submitted." >&4
    exit 0
fi

cp "${submitdir}/alternate.py" .
cp "${testdir}/solutions/check.py" .

timeout 5s /usr/bin/python3 -c "from alternate import alternate; print(alternate('${testinput}'))" > test_stdout.txt 2> test_stderr.txt
exit_code=$?
if [ $exit_code -eq 0 -a ! -s test_stderr.txt ]; then
    echo "$correctoutput" | diff - test_stdout.txt
    if [ $? -eq 0 ]; then
        echo 100 >&3
        exit 0
    fi
fi

echo 0 >&3

(args "q2_t01" "test_case_short_input.txt" "test_case_short_answer.txt")

#!/bin/bash

readonly testname="$1"
readonly inputfile="${testdir}/test_data/$2"
readonly answerfile="${testdir}/test_data/$3"

mkdir "$testname"
cd "$testname"

if [ ! -f "${submitdir}/alternate_keyboard.py" ]; then
    echo 0 >&3
    echo "File alternate_keyboard.py was not submitted." >&4
    exit 0
fi

cp "${submitdir}/alternate_keyboard.py" .
cp "${testdir}/solutions/check.py" .
cp "${testdir}/solutions/alternate.py" .

timeout 5s /usr/bin/python3 alternate_keyboard.py < "${inputfile}" > test_stdout.txt 2> test_stderr.txt
exit_code=$?
if [ $exit_code -eq 0 -a ! -s test_stderr.txt ]; then
    diff "$answerfile" test_stdout.txt
    if [ $? -eq 0 ]; then
        echo 100 >&3
        exit 0
    fi
fi

echo 0 >&3

(args "q2_t02" "test_case_long_input.txt" "test_case_long_answer.txt")

#!/bin/bash

readonly testname="$1"
readonly inputfile="${testdir}/test_data/$2"
readonly answerfile="${testdir}/test_data/$3"

mkdir "$testname"
cd "$testname"

if [ ! -f "${submitdir}/alternate_keyboard.py" ]; then
    echo 0 >&3
    echo "File alternate_keyboard.py was not submitted." >&4
    exit 0
fi

cp "${submitdir}/alternate_keyboard.py" .
cp "${testdir}/solutions/check.py" .
cp "${testdir}/solutions/alternate.py" .

timeout 5s /usr/bin/python3 alternate_keyboard.py < "${inputfile}" > test_stdout.txt 2> test_stderr.txt
exit_code=$?
if [ $exit_code -eq 0 -a ! -s test_stderr.txt ]; then
    diff "$answerfile" test_stdout.txt
    if [ $? -eq 0 ]; then
        echo 100 >&3
        exit 0
    fi
fi

echo 0 >&3

(args "q3_t01" "test_case_short_input.txt" "test_case_short_answer.txt" "")

#!/bin/bash

readonly testname="$1"
readonly inputfile="${testdir}/test_data/$2"
readonly answerfile="${testdir}/test_data/$3"
output_file="$4"

mkdir "$testname"
cd "$testname"

if [ ! -f "${submitdir}/alternate_keyboard_save.py" ]; then
    echo 0 >&3
    echo "File alternate_keyboard_save.py was not submitted." >&4
    exit 0
fi

cp "${submitdir}/alternate_keyboard_save.py" .
cp "${testdir}/solutions/check.py" .
cp "${testdir}/solutions/alternate.py" .
cp "${testdir}/solutions/alternate_keyboard.py" .

timeout 5s /usr/bin/python3 alternate_keyboard_save.py $output_file < "${inputfile}" > test_stdout.txt 2> test_stderr.txt
exit_code=$?
if [ -z "$output_file" ]; then
    output_file="output.txt"
fi
if [ $exit_code -eq 0 -a ! -s test_stderr.txt -a ! -s test_stdout.txt -a -s "$output_file" ]; then
    diff "$answerfile" "$output_file"
    if [ $? -eq 0 ]; then
        echo 100 >&3
        exit 0
    fi
fi

echo 0 >&3

(args "q3_t02" "test_case_long_input.txt" "test_case_long_answer.txt" "my-output-file.txt")

#!/bin/bash

readonly testname="$1"
readonly inputfile="${testdir}/test_data/$2"
readonly answerfile="${testdir}/test_data/$3"
output_file="$4"

mkdir "$testname"
cd "$testname"

if [ ! -f "${submitdir}/alternate_keyboard_save.py" ]; then
    echo 0 >&3
    echo "File alternate_keyboard_save.py was not submitted." >&4
    exit 0
fi

cp "${submitdir}/alternate_keyboard_save.py" .
cp "${testdir}/solutions/check.py" .
cp "${testdir}/solutions/alternate.py" .
cp "${testdir}/solutions/alternate_keyboard.py" .

timeout 5s /usr/bin/python3 alternate_keyboard_save.py $output_file < "${inputfile}" > test_stdout.txt 2> test_stderr.txt
exit_code=$?
if [ -z "$output_file" ]; then
    output_file="output.txt"
fi
if [ $exit_code -eq 0 -a ! -s test_stderr.txt -a ! -s test_stdout.txt -a -s "$output_file" ]; then
    diff "$answerfile" "$output_file"
    if [ $? -eq 0 ]; then
        echo 100 >&3
        exit 0
    fi
fi

echo 0 >&3

(language external)
(value 1)

import check

def alternate(mystr):
    answer = ""
    makeNextLetterUppercase = True
    for i in range(len(mystr)):
        char = mystr[i]
        if char.isalpha():
            if makeNextLetterUppercase:
                answer += char.upper()
            else:
                answer += char.lower()
            makeNextLetterUppercase = not makeNextLetterUppercase
        else:
            answer += char
    return answer

# Tests
if __name__ == "__main__":
    check.expect("empty", alternate(""), "")
    check.expect("example", alternate("hello! How Are You? Doing OK"),\
        "HeLlO! hOw ArE yOu? DoInG oK")
    check.expect("test1", alternate("this is my example!!!"),\
        "ThIs Is My ExAmPlE!!!")
    check.expect("test2", alternate("I hope you do Well!!"),\
        "I hOpE yOu Do WeLl!!")

import sys
from alternate import alternate

for line in sys.stdin:
    print(alternate(line), end='')

# Last updated Feb 4, 2019
"""
    This module is used for testing python code in CS 116 and CS 234

    The useful functions in this module are:
    * check.within, for testing functions whose output includes floats
    * check.expect, for testing all other functions
    * check.set_screen, for testing screen output (print statements)
    * check.set_input, for testing keyboard input (raw_input)
    * check.set_file, for testing file output

    For details on using these functions, please read
    the Python Style guide from the CS 116 website,
    www.student.cs.uwaterloo.ca/~cs116/styleGuide
"""

import sys, os, builtins
backup_stdin = sys.stdin
backup_stdout = sys.stdout
old_input = builtins.input


# redefine input to hide prompts
def blank_input(prompt = None):
    return old_input()

class InputError(Exception):
    """
    Not enough parameters given to set_input.
    """
    def __init__(self):
        self.val = "not enough parameters to set_input"
    def __str__(self):
        return self.val
    

class redirect_input:
    """
    Keyboard input is redirected from this class
    whenever set_input is called.
    """
    def __init__(self, inp):
        self.lst = inp
    def readline(self):
        if self.lst:
            return self.lst.pop(0)
        else:
            raise InputError()

class redirect_output:
    """
    Screen output is redirected to this class
    whenever set_screen is called.
    """
    def __init__(self):
        self.screen = ""
    def __str__(self):
        return self.screen
    def __nonzero__(self):
        return bool(self.screen)
    def write(self, string):
        self.screen += string
    def reset(self):
        self.screen = ""

expected_screen = ""
actual_screen = redirect_output()
test_output = redirect_output()
input_list = []
file_list = []
dir_list = []
exact_screen = False


def set_screen(string):
    """
    Consumes a description of the expected screen output
    for the next call to check.expect or check.within.
    """
    global expected_screen
    expected_screen = str(string)
    sys.stdout = actual_screen
    
    builtins.input = blank_input

def set_print_exact(*strlst):
    """
        Consumes a list of strings in the expected exact output
        for the next call to check.expect or check.within.
        """
    global expected_screen, exact_screen
    expected_screen = '\n'.join(strlst)
    exact_screen = True
    sys.stdout = actual_screen
    
    builtins.input = blank_input
    
    

def set_input(*inputs):
    """
    Consumes a variable amount of strings representing keyboard input for
    the next call to check.expect or check.within. Python treats *inputs
    as a tuple.
    """
    global input_list
    #if type(lst) != list:
    #    raise TypeError("set_input must consume a list")
    for i in inputs:
        if type(i) != str:
            raise TypeError("all parameters must be strings")
    input_list = list(map(lambda s: s+"\n", inputs))
    
    
    sys.stdin = redirect_input(input_list)
    sys.stdout = actual_screen

def set_file(resulting_file, expected_file):
    """
    Consumes two strings: resulting_file (the name
    of a file that will be produced by the function)
    and expected_file (the name of a file to whose
    contents will match resulting_file if the function
    works correctly). Checks that the files contain the
    same text, ignoring whitespace, on the next call
    to check.expect or check.within.
    """
    global file_list, dir_list
    dir_list = os.listdir(os.getcwd())
    file_list.append((resulting_file, expected_file, False))

def set_file_exact(resulting_file, expected_file):
    """
    Consumes two strings: resulting_file (the name
    of a file that will be produced by the function)
    and expected_file (the name of a file to whose
    contents will match resulting_file if the function
    works correctly). Checks that the files contain the
    same text, including whitespace, on the next call
    to check.expect or check.within.
    """
    global file_list, dir_list
    dir_list = os.listdir(os.getcwd())
    file_list.append((resulting_file, expected_file, True))

def expect(label, function_call, expected_value):
    """
    Testing function for equality. Will also print the
    description given to set_screen, use the keyboard
    input given to set_input, and compare files given
    to set_files.
    """
    run_test(label, function_call, expected_value, None)

def within(label, function_call, expected_value, acceptable_tolerance):
    """
    Testing function for floating point numbers. Will also
    print the description given to set_screen, use the
    keyboard input given to set_input, and compare files
    given to set_files.
    """
    run_test(label, function_call, expected_value, acceptable_tolerance)

def run_test(label, result, expected, tolerance):
    """
    Performs the tests given to check.within and check.expect.
    Do not use run_test in your code for CS 116.
    """
    global actual_screen, expected_screen, input_list, file_list, dir_list, exact_screen
    sys.stdout = test_output
    
    
    if input_list:
        print ("{0}: FAILED; not all input strings were used\n".format(label))
    elif type(result) != type(expected):
        rtype = str(type(result)).strip(' ')
        etype = str(type(expected)).strip(' ')
        print ("{0}: FAILED; {1} is a {2}, whereas {3} is a {4}\n".format(label,expected,etype,result,rtype))
    elif tolerance and not approx(result, expected, tolerance):
        print ("{0}: FAILED; expected {1}, saw {2}\n".format(label, expected, result))
    elif not(tolerance) and result != expected:
        print ("{0}: FAILED; expected {1}, saw {2}\n".format(label, expected, result))
    else:
        print ("{0}: PASSED\n".format(label))
    
    if file_list:
        new_files = []
        for tup in file_list:
            new_label = "{0} {1}".format(label, tup[0:2])
            compare_files(new_label, new_files, tup[0], tup[1], tup[2])
        extra_files = list(set(os.listdir(os.getcwd())) ^ set(dir_list + new_files) )
        if extra_files:
            print ("{0}: The following additional files were created: {1}".format(label, ", ".join(extra_files)))
    
    if exact_screen:
        actual_screen = str(actual_screen)[:-1]
        if expected_screen != actual_screen:        
            print("{0} - print output: FAILED; expected\n{1}\nsaw\n{2}\n".format(label, expected_screen, actual_screen))
        else:
            print("{0} - print output: PASSED\n".format(label))
        actual_screen = redirect_output()
    elif expected_screen:
        print ("{0} (expected screen output):".format(label))
        print (expected_screen)
        print ("")
        print ("{0} (actual screen output):".format(label))
        print (actual_screen)

    input_list, file_list, dir_list = [], [], []
    expected_screen = ""
    exact_screen = False
    actual_screen.reset()
    
    sys.stdin = backup_stdin
    sys.stdout = backup_stdout
    builtins.input = old_input
    if test_output:
        print (str(test_output).strip())
        print ("-----")
    test_output.reset()

def approx(result, expected, tolerance):
    """
    Performs approximate equality comparisons for check.within.
    Do not use approx in your code for CS 116.
    """
    if type(result) != type(expected):
        return False
    
    tp = type(result)
    
    if tp == float:
        return abs(result - expected) <= tolerance
    elif tp == complex:
        return abs(result.real - expected.real) <= tolerance and \
               abs(result.imag - expected.imag) <= tolerance
    elif tp in (list, tuple): # sequences that can contain floats
        return len(result) == len(expected) and \
               all(map(approx, result, expected, [tolerance]*len(result)))
    elif tp in (dict, type(redirect_output()), set, frozenset):
    # unordered containers that can contain floats
    # for whatever reason, 'instance' is not a type, so type(redirect_output) is used instead
        if tp == dict:
            result = result.items()
            expected = expected.items()
        elif tp == type(redirect_output()):
            # check that both instances are of the same class
            if result.__class__ != expected.__class__:
                return False
            result = result.__dict__.items()
            expected = expected.__dict__.items()
        # if tp in (set, frozenset) then no action required
        return approx(sorted(result), sorted(expected), tolerance)
    else: # anything that hits else cannot contain floats
        return result == expected

def compare_files(label, new_files, fname1, fname2, exact):
    """
    Performs file comparisons for check.within and check.expect.
    Do not use compare_files in your code for CS 116.
    """
    try:
        f = open(fname1, 'r')
        lines1 = list(map(lambda x: x.strip(), f.readlines()))
        f.close()
        new_files.append(fname1)
    except IOError:
        print ("{0}: File {1} does not exist\n".format(label, fname1))
        return None
    try:
        f = open(fname2, 'r')
        lines2 = list(map(lambda x: x.strip(), f.readlines()))
        f.close()
        new_files.append(fname2)
    except IOError:
        print ("{0}: File {1} does not exist\n".format(label, fname2))
        return None
    
    if lines1 == [] and lines2 == []:
        return None
    elif lines1 == []:
        print ("{0}: {1} is empty but {2} is not.".format(label, fname1, fname2))
        print ("{0} (line 1): {1}\n".format(fname2, lines2[0]))
        return None
    elif lines2 == []:
        print ("{0}: {1} is empty but {2} is not.".format(label, fname2, fname1))
        print ("{0} (line 1): {1}\n".format(fname1, lines1[0]))
        return None
    
    while lines1[-1].isspace() or lines1[-1] == "":
        lines1.pop()
    while lines2[-1].isspace() or lines2[-1] == "":
        lines2.pop()
    
    if len(lines1) != len(lines2):
        print ("{0}: {1} and {2} do not contain the same number of lines.".format(label, fname1, fname2))
    
    n = min(len(lines1), len(lines2))
    bad_lines = []
    
    for i in range(n):
        if exact:
            line1 = lines1[i].rstrip()
            line2 = lines2[i].rstrip()
        else:
            line1 = "".join(lines1[i].split())
            line2 = "".join(lines2[i].split())
        if line1 != line2:
            bad_lines.append(i+1)
    
    if bad_lines:
        first_line = bad_lines[0]
        bad_lines = ", ".join(map(lambda x: str(x), bad_lines))
        print ("{0}: The following lines in {1} and {2} do not match: {3}".format(label, fname1, fname2, bad_lines))
        
        extra_spaces = " " * abs(len(fname1) - len(fname2))
        if len(fname1) < len(fname2):
            fname1 += extra_spaces
        else:
            fname2 += extra_spaces
        
        print ("{0} (line {1}): {2}".format(fname1, first_line, lines1[first_line-1]))
        print ("{0} (line {1}): {2}".format(fname2, first_line, lines2[first_line-1]))
    
    if len(lines1) != len(lines2) or bad_lines:
        print ("")

LoReM iPsUm DoLoR sIt AmEt, CoNsEcTeTuR aDiPiScInG eLiT, sEd Do
EiUsMoD tEmPoR iNcIdIdUnT uT lAbOrE eT dOlOrE mAgNa AlIqUa. ToRtOr
AlIqUaM nUlLa FaCiLiSi CrAs FeRmEnTuM. vEnEnAtIs UrNa CuRsUs EgEt
NuNc ScElErIsQuE. uRnA iD vOlUtPaT lAcUs LaOrEeT nOn. DoLoR pUrUs
NoN eNiM pRaEsEnT eLeMeNtUm. DuI nUnC mAtTiS eNiM uT tElLuS eLeMe.

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do
eiusmod tempor incididunt ut labore et dolore magna aliqua. Tortor
aliquam nulla facilisi cras fermentum. Venenatis urna cursus eget
nunc scelerisque. Urna id volutpat lacus laoreet non. Dolor purus
non enim praesent elementum. Dui nunc mattis enim ut tellus eleme.

HeLlO! hOw ArE yOu? DoInG oK
ThIs Is My ExAmPlE!!!
I hOpE yOu Do WeLl!!

hello! How Are You? Doing OK
this is my example!!!
I hope you do Well!!