Marking Scheme Walkthrough

Introduction

The marking scheme is kept in a file called mark-scheme in the same directory as runTests and computeMarks. It will be the first thing shown to students, and should contain a brief summary of how marks are allocated for the assignment and what the student earned on each portion (typically filled in by hand by a marking TA). Some general style pointers to take into consideration to facilitate readability are given in MarkingScheme; note that in order to make marking more effective, some additional marking information should be provided to the TAs.

The information contained in this walkthrough up to and including the Simple substitution section is sufficient to design decent-looking and informative marking schemes. The information afterwards is primarily of interest to power-users who wish to take advantage of scripting capabilities to reduce the amount of time required to create marking schemes and/or to reduce the potential for making errors when constructing the marking scheme.

Using nroff

If the -n flag is passed to BitterSuite's computeMarks script, then it will be directed to format the provided marking scheme with nroff. You can then use nroff formatting commands to format the output for the TAs, as discussed in MarkingScheme. BitterSuite will automatically provide the following nroff formatting commands before the start of the marking scheme processing as a courtesy:

.ll ${linelength}
.ls 1
.pl 1

These commands set the length of each line appropriately (dependent on whether or not RST is generating printouts), directs the output to be single-spaced, and sets the page length to 1 to prevent nroff from generating a large amount of unnecessary trailing whitespace.

Using shell interpretation

Another flag that can be passed to computeMarks is -i, which instructs Bittersuite to have the shell interpret the marking scheme as it is being output. Specifically, it is run through the folowing command:

eval echo "`cat ${provided_mark_scheme}`"

The shell that will be used to evaluate the marking scheme is bash.

Simple substitution

If the marking scheme is entirely enclosed within double-quotes, then the effect of this command is that the shell will substitute values for any specified variables in the marking scheme. The reason this is potentially useful is that the autotesting results are exported for use in the marking scheme in a series of variables whose names map to the testing directory hierarchy. This allows, for example, the question 1 autotesting results to be included as a subcomponent of all of question 1's marks, making the marks earned for each question much more clean.

The total earned marks are available in the environment variable $te, and the total possible marks ("out of") are stored in the environment variable $to. To obtain any subdirectories, the directories are listed in order with underscore as a separator in the variable name. So, the earned mark for a test located at in/1 would be stored in $t1e, a test at in/2/a would be in $t2_ae, a test at in/3/treetest/emptyleafnodes would be in $t3_treetest_emptyleafnodese, and so on.

A sample mini-scheme would be as follows:

"--- Marking scheme for 1-question assignment ---

Q1: ____ / 10
  Design recipe: ____ / 4
  Autotesting:    $t1e / $t1o"

Scripting

A more powerful option is to provide a script to be interpreted to produce a marking scheme. This could, for example, store the marks each component is out of and sum them automatically, reducing the likelihood that anybody on course staff will make an arithmetic error and provide the wrong total for the marking scheme.

Instead of being enclosed in double-quotes as in the simple substitution example, the script should begin with $( and end with ).

Marking Scheme Math

If, for example, the components of a mark for question 1 were the autotesting results and the design recipe, then a total mark for question 1 could be pre-calculated by having the shell perform the arithmetic automatically:

q1tot=$(( q1o + q1dr ))

However, the shell typically does not support floating point arithmetic (and as of the time of this writing, the student.cs environment does not). Because of this, it is much safer to pass the work to bc. But, bc does not always format numbers very nicely; as such, they should then be passed to some sort of postprocessor. This process can be simplified by defining the following function before doing any calculations:

         domath () {
            # Restrict to two trailing decimal points for the floating point value,
            # then chop off trailing zeroes, then chop off a trailing decimal point.
            echo "scale = 2; $1" | bc | sed -e 's/\(\.[1-9]*\)\(0*\)$/\1/g' | sed 's/\.$//g'
         }

Then the calculation of q1tot may be expressed as:

q1tot=$( domath "q1o + q1dr" )

Marking Scheme Output

Due to the nature of the echo statements that the marking scheme passes through, newlines will be swallowed. As such, it is necessary either to make the output marking scheme less readable initially by using character codes, or to provide some post-processing instructions.

The following code allows more natural expression of the marking scheme itself:

echo "
This is a multi-line marking scheme.
When it is used in the student output,
the newlines contained within it
will be preserved properly.
" | perl -ne 'chomp; print "$_\\n";'

A Combined Example

The following example taken from the man page illustrates a marking scheme combining the use of nroff and scripting facilities:

 #!/bin/bash
$(

domath () {
   # Restrict to two trailing decimal points for the floating point value,
   # then chop off trailing zeroes, then chop off a trailing decimal point.
   # Note: because of truncation, do not accumulate domath results.
   echo "scale = 2; $1" | bc | sed -e 's/\(\.[1-9]*\)\(0*\)$/\1/g' | sed 's/\.$//g'
}

q1dr=4
q1tot=$( domath "$t1o + $q1dr" )

q2dr=5
q2tot=$( domath "$t2o + $q2dr" )

alltot=$( domath "$q1dr + $q2dr + $to" )

# The innermost echo allows preservation of space for evaluation later
# by replacing them with hex codes.
# This delayed evaluation is handled by the echo that is echoed by
# 'eval echo'
# There really must be a simpler way to do this...
tempvar=$(eval echo "echo -e \"$(echo "
.ti 5
This is a marking scheme that incorporates both nroff
directives and shell interpretation, including the pre-assignment of variables
that are used internally.  This should help ensure, for example, that course staff
doesn't make any arithmetic errors when creating assignment totals.

.ad c
=========================================

Total Mark Assigned:   _______  / $alltot

Marked By:  ________

=========================================

.ad l
.in 8
.ti -8
Q1: _____ / $q1tot

.ti -4
Design recipe: $(printf "%11s" "_____ / $q1dr")

.ti -4
Autotesting: $(printf "%13s" "$t1e / $t1o")

.ti -8
Q2: _____ / $q2tot

.ti -4
Design recipe: $(printf "%11s" "_____ / $q2dr")

.ti -4
Autotesting: $(printf "%13s" "$t2e / $t2o")
.ti 12
" | sed 's/ /\\x20/g')\"" | perl -ne 'chomp; print "$_\\n";' )

# Debug
#echo $tempvar >& 2
#echo -e $tempvar >& 2

echo $tempvar
)