BitterSuite Filter Language Documentation

This language is intended to be used as the first test run for a given testing suite where applicable. It provides a submission filter which removes files that violate filter conditions, and returns a score representing the percentage of submitted files that were not filtered. This is useful for submission filters that do not allow flexible filtering at the point of submission, such as with MarkUs.

The Filter language should eventually be expanded to allow arbitrary ways to filter submitted files via provided Racket functions. For now, though, it only supports running the same set of filters that are applied via the ISG SubmitFileFilter, or another executable script.

Relevant Options

The language is selected by specifying (language filter) in an options.ss file.

(test-action action)

Determines what action should be taken to filter files. For now, the only option is submit-style-filter.

(external-cmd path-to-cmd)

Specifies which command should be used to filter when test-action is set to submit-style-filter. This command will be passed four command-line arguments: the name of the file being tested, the directory containing the submitted files, the directory containing any relevant configuration files, and a unique key that represents the current submission and not any subsequent runs. A non-zero return value is used to signal that the file should be filtered. This defaults to the same behaviour as the default submit file filter.

(delete-filtered-files? bool)

If bool is #t, then any files that violate the filter conditions will be removed from the submission directory; otherwise, the files will not be deleted.

Relevant files

No files in the testing hierarchy will be recognized by the filter language.

Special Notes

Take extra care about the number of marks you assign to a filter test. While for RSTPublicTests a non-zero value is certainly appropriate, it is likely never appropriate for the autotests that actually mark the students, in which case you should make sure (value 0) is a specified option.