Testing Scheme with Mutation

Whenever an evaluator is created, its state is mutated and passed along with the evaluator. So, if you have an evaluator created at the level in/1/ and its state is mutated by a result expression at in/1/1, that mutated state will be carried forward to in/1/2. While it's rare for a question to want students to mutate global state, you won't want to fail every test but one because of this (in fact, you should instead define particular tests that check for it explicitly; for example, (result (let ([c1 (make-counter)][c2 (make-counter)]) (c1) (c1) (c2) (c1) (c1) (c1) (c2)))). The only option is to reload the evaluator state for each new test.

One way to do this is specify a loadcode in every test directory. This is error-prone and mind-numbing. Instead, you should make use of the evaluator-reset option. The safest option is to pass before-expression as the option, as this will reload the evaluator before every new expression that's passed to it. However, this is slow, and in most cases should be unnecessary; the expected and equal expressions should not be dependent on the evaluator state in this fashion. Instead, the more useful one is before-test, which will reset the evaluator's state before evaluating the result, but then not before the following expected and equal evaluations.