Remember that when it comes to testing three things should
come to mind and should be explained to the reader of the tests:
   0) How can I perform an intelligent test with the least amount of
      output possible?
   1) What is the expected output?
   2) What is the observed output?
   3) Is the observed output the same as the expected output?
      Why and/or why not?

Here are some general rules of thumb that you should consider when
designing, implementing and running your tests (e.g. tester.dem).

0)  Think. 
   Think about how to automate as many test procedures as possible.
   Ideally you should be able to write test programs that only 
   produce one line of output - either stating that the test succeeded
   or that it failed and where it failed.

   One way of doing this is to make extensive use of the Turing assert
   statement.  Recall that an assert statement in OOT automatically
   stops the program with useful information as to which statement
   failed and the cause of the failure.

   You must reason about the state of the testing program and the
   state of underlying modules being tested.  Assuming that the
   modules being tested are functioning correctly, what statements must
   be must be true at key points in the testing program and in the underlying
   modules. Add assert statements at these points in these programs
   to test these assumptions. The goal is
   to insert enough useful assert statements so that if all the
   assertions pass, then we have strong evidence that the program
   is correct. Note that we can never be sure that we have not missed a
   bug.

1) Someone should be able to spend about 1 minute looking at the output of
   each test program to determine whether all of the tests in the program
   failed or succeeded.

   If the reader of the program is required to trace through pages and
   pages of output then the test is essentially useless.

2) Consider to whom you are trying to demonstrate your program (i.e.,
   think about the marker).  What type of and how much information will
   they want to see in your testing?

   For example, pretend that you are the project leader working in a large
   company. Once a month there is a large meeting of technical staff in
   which each of the project leaders has 10 minutes to give a status report
   on their project. This means that you have 10 minutes to demonstrate
   using the output of your testing what parts of your project do and
   do not work correctly.  (Remember showing what does NOT work is just as
   important as showing what does work - perhaps more so).

3) Be smart about your testing. If the program you write which performs
   your tests (your test program) is more than 3 or 4 pages long it is
   likely too long. If the output produced from running the test program is
   more than 2 or 3 pages your test program is probably
   not designed very well. Think about the number of different things
   that need to be tested and how much time it will take to look at the
   output. A good test program should demonstrate simply and easily
   that the program does or does not work using minimal output.

4) Even though the test program is written only for the purpose of
   testing the implementation of other modules, classes or programs, it
   still needs to be clear, concise, and documented. In particular, the
   test program should specify what values you expect it to return
   (this can be done using comments where needed).

5) The test program should be bug free. Take some time when writing the
   test program to ensure that it is correct. Too often people waste a
   bunch of time trying to fix the modules they are working on when in fact
   the modules are fine but there is a bug in the test program (or they
   misunderstand how the test works or the return values that are
   expected).

6) Use multiple test programs to test different aspects of your system.
   (In assignment 1 these different tests are performed during the 
   different task. For example, your interactive tests performed with
   driver.dem should differ from the tests you perform using tester.dem.)

7) Test special cases. Ideally a production quality program should be
   completely bullet proof (not that many really are :-).  No user, even
   malicious ones, should be able to crash the program.

8) Check the return values of function calls. That is check that the call
   has completed successfully - if it has not, handle the error
   intelligently.

Updated for 1030: January 2, 1998