(r23) Testing < ISG

ISG Web>ISGScripts>Testing (revision 23) (raw view)~~EditAttach~~

---+ ISG Testing

A home for testing software used by ISG.
   * ISG.RST 
      * RSTPublicTests
      * [[ISG.RSTA][RSTA]]
      * DistributedRST
   * [[ISG.Marmoset][Marmoset]]
   * ISG.BitterSuite
Some documentation for testing tools is available at ISGScriptsManPages

The ImprovePit contains suggestions for how each of these pieces of software may be improved.

---++ Brief discussion of testing software

RST is a testing framework that is designed to run user-provided scripts to do testing for it, and handles the details like setting up a temporary directory, switching to a testing account, and generating a results file automatically. This is what's run from the command-line by the tutors. It is solely a testing system; the submission details are handled by the (Dec/15 update) SubmitScripts [pre Dec/15: [[http://www.student.cs.uwaterloo.ca/~isg/scripts/submit/][submit command]]]. It is often used for running secret autotests after submissions have been closed, and can be accessed via the RSTPublicTests front-end to provide unlimited public test runs to students.

Marmoset is an alternate to RST that uses a website frontend to control both submission and testing. It is oriented to allowing time windows in which students can request all eligible tests be run on their code; these tests are divided into public, request, and secret tests, and passing all tests in one category unlocks the next category.

BitterSuite is a system to designed to hide the testing details in a particular domain (ie, testing in Scheme or in C) and allow the test designer to specify particular testing information. For example, in Scheme a file and associated modules are specified to load in a particular language, and then a number of (expected ...) and (result ...) expressions are given for particular test cases; the tutor/TA/IA/... doesn't need to know *how* the tests are done. It has hooks that allow it to run in RST or in Marmoset.

Topic revision: r23 - 2018-09-04 - YiLee

ISG Web

ISG Web Home
- Changes
- Index
- Search

Webs
- AIMAS
- CERAS
- CF
- CrySP
- External
- Faqtest
- HCI
- Himrod
- ISG
- Main
- Multicore
- Sandbox
- TWiki
- TestNewSandbox
- TestWebS
- UW

My links
- People
- CERAS
- WatForm
- Tetherless lab
- Ubuntu Main.HowTo
- eDocs
- RGG NE notes
- RGG
- CS infrastructure
- Grad images

Edit

Instructional Support Group, David R. Cheriton School of Computer Science, University of Waterloo