RST/BitterSuite Testing Using Arbitrary Scripts

RST/BitterSuite can use scripts that you write for testing. This allows for a lot of flexibility.

Similar to Racket and Python testing, you'll need to create a test.pt or test.1 folder in the marking/aXX folder of the course account. The test.pt folder is for public tests, which are run when students submit. test.1 contains test cases that you want to run after the assignment's due date. The folders test.pt and test.1 have the same structure. The main difference is that test.1 usually contains more and harder test cases.

RST will run these scripts in this order:

  1. test.pt/runTests
  2. test.pt/computeMarks
  3. Any test scripts in the test.pt/in/ folder.
  4. test.pt/computeMarks-postprocess

Running tests from runTests

The runTests script is run in a testing account named csNNNt with a 't' at the end. This is safer than running the test script in the course account ( csNNN). After you create the runTests script, make sure it is executable using the chmod command.

Here is an example test suite that checks if students submitted a C program that prints "Hello". In this example, students submit their code in a file named main.c.

Running tests from computeMarks

After RST runs runTests, it will run computeMarks. The computeMarks script is run in the course account. This lets computeMarks access files in the course account, but it can also be dangerous (ex. if a student submits code that deletes files, and you run the student's code, it can delete files in the course account).

Examples where you can use computeMarks for testing are:

  • You want to check if the students have registered their clicker ID on the course website
  • You want to check if the students have completed tracing exercises on the course website

Running tests from test.pt/in/ folder

After RST runs computeMarks, it will run your tests in the in folder. You can use the "External" language to make RST run scripts in the in folder. Name your scripts test.exe. For example, the following test suite checks that students submitted a C program that counts characters read in from the keyboard.

Clean Up Work in test.pt/computeMarks-postprocess

If there's anything you need to do before RST finishes, you can create a script called computeMarks-postprocess and RST will run it at the end before quitting.

Topic revision: r3 - 2019-01-01 - YiLee
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback