ISG Testing

A home for testing software used by ISG.

Some documentation for testing tools is available at ISGScriptsManPages

The ImprovePit contains suggestions for how each of these pieces of software may be improved.

Brief discussion of testing software

RST is a testing framework that is designed to run user-provided scripts to do testing for it, and handles the details like setting up a temporary directory, switching to a testing account, and generating a results file automatically. This is what's run from the command-line by the tutors. It is solely a testing system; the submission details are handled by MarkUs or submit (ISGScriptsManPages#Submit). It is often used for running secret autotests after submissions have been closed, and can be accessed via the RSTPublicTests front-end to provide unlimited public test runs to students.

Marmoset is an alternate to RST that uses a website frontend to control both submission and testing. It is oriented to allowing time windows in which students can request all eligible tests be run on their code; these tests are divided into public, request, and secret tests, and passing all tests in one category unlocks the next category.

BitterSuite is a system to designed to hide the testing details in a particular domain (ie, testing in Scheme or in C) and allow the test designer to specify particular testing information. For example, in Scheme a file and associated modules are specified to load in a particular language, and then a number of (expected ...) and (result ...) expressions are given for particular test cases; the tutor/TA/IA/... doesn't need to know how the tests are done. It has hooks that allow it to run in RST or in Marmoset.

Topic revision: r24 - 2018-09-05 - YiLee
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback