Miscellaneous notes

This section has been moved to General FAQ under Common Issues HERE.

Outdated

Why is there a message "Evaluator creation error: FAILED - Error creating evaluator, Timeout of 60 seconds reached; assuming infinite recursion." when the function clearly should not time out?

Be careful when banning string functions. For some reason, what should say "FAILED - Error creating evaluator, Disallowed function reverse called" becomes a timeout message when the function called format is not allowed. Make sure to allow format to prevent this issue; it's a fairly trivial function which seems to be used by the autotesting scripts.

How would I account for students who already have a (require ...) line because it's actually part of the assignment?

This most likely happens when we provide a library (like 2htdp/image) that the students need to do the questions. Simply make sure a copy of the provided file is in our test.0/provided/ folder and add the name of the file to the (modules ...) line, so it's made available.

If the students are allowed to require their own files, it gets more complicated. In order to use their version of the required file, you do not put anything in test.0/provided/ and only add the name to the (modules ...) line. In general, allowing students to require their own files is not recommended, as it can cause issues with names being re-defined when running the check-testcases script. Be careful asking for similar types of questions to be submitted in different files in this case, as they may have names repeated.

Can students keep their check-expects in their files to get marks for test case coverage without interfering with correctness?

Although <a href="https://cs.uwaterloo.ca/twiki/edit/ISG/DrRacket?topicparent=ISG.TestingGuide;nowysiwyg=0" rel="nofollow" title="DrRacket (this topic does not yet exist; you can create it)"> DrRacket </a> is an interpreted (actually, JIT-compiled) language, there is still some "compilation" that takes place. The application performs an initial syntax check for things like missing brackets without running anything. Then, it runs the code sequentially, interrupting mid-execution if an error pops up. Finally, check-expect is a special form which is only evaluated after these first two steps are completed. In short, the correctness tests will run fine even if there are incorrect check-expects in the file, because the functions can be called as long as they've been defined at some point. Of course, the tests will still fail if the check-expect has an important character missing or calls a function that the student didn't even write.

How do I fix and re-run test cases if TA marking has already started?

The fastest way is probably to use the set_marks_rst command to re-assign the marks on MarkUs. You can create a folder separate from test.pt and test.0, like test.1.fix_q2_test_002, with only the Q2 folder present to make rst/distrst run faster. Then you can use set_marks_rst with the test.1.fix_q2_test_002.AUTOTESTRESULTS folder targeted.

How can I allow list outputs to be considered correct regardless of order?

There are lots of ways to do this. The simplest is to use sort or quicksort on both the student's output and the expected output, and compare with equal?. You can provide a my-sort or my-quicksort if they aren't allowed, which hides the call to sort/quicksort in a provided file. If you like ALFs, you can check that the student's list is a subset of the expected list using andmap and member?. Then do it again vice-versa, because two sets that are subsets of each other are necessarily equivalent. Finally, the "true" way to store unordered sets is with Racket's hash table type; you can convert both lists into hash tables using hash or make-hash (i.e. value-frequency pairs), and simply compare using equal? (it will ignore ordering once it recognizes you are comparing hash tables).

-- DustinFirman - 2017-12-21