Plotting Scripts

Created by KyleSpaans for the Winter 2009 offering of CS136.

There are scripts that can generate an EPS imagine suitable for inclusion into OUTPUT.ps using GnuPlot.

Code

Currently the plotting is done by a shell script called from a test.sh. That is, you will need to create a seperate, 0-valued test just for the plot script. Ideally this would be moved into runTests-preprocess. To get the plot graphics include into the student's OUTPUT.ps file some lines need to be added to computeMarks-postprocess:

if [ -s $PLOTFILE.eps ]; then
        keepFile $PLOTFILE.eps 128 -e 'width=0.7\textwidth,keepaspectratio'
fi

A note of caution: /usr/bin/time or possibly the timeout script don't always seem to fail reliably. For example, sometimes when student code segfaults, time will catch this and print "program terminated abnormally", yet the return value gets changed to 0, which can muck up the script.

#!/bin/bash

# Compile then plot student code.
export PATH=`/bin/showpath gnu standard`
readonly PROGPATH=`type $0 | awk '{print $3}'`
readonly THISDIR=`dirname $PROGPATH`


FILEBASE="uncompress"
OUTPUT="$tmpdir/uncompress"
PLOTFILE="$tmpdir/${FILEBASE}_timings.eps"
PLOTOUTPUT="$tmpdir/plotable2.dat"
TIMES="times2.txt"

######################

if [ -f "$FILEBASE.c" ]; then
   # No need for compiler error messages, those can be left for another test
   gcc -std=c99 -w -o $OUTPUT "$FILEBASE.c" 
fi

# Check that $OUTPUT actually got compiled, and exit if it didn't.
if [ ! -f "$OUTPUT" ]; then
   exit 0
fi

if [ -f "xcoords.txt" ]; then
   rm -f xcoords.txt
fi

# Time it!
$tmpdir/timeout 10 "/usr/bin/time -p $OUTPUT < 1000words.txt > /dev/null" 2>> $TIMES
if [ "$?" -eq "0" ]; then
   echo 1000 > "xcoords.txt"
fi
$tmpdir/timeout 30 "/usr/bin/time -p $OUTPUT < 2000words.txt > /dev/null" 2>> $TIMES
if [ "$?" -eq "0" ]; then
   echo 2000 >> "xcoords.txt"
fi
$tmpdir/timeout 45 "/usr/bin/time -p $OUTPUT < 4000words.txt > /dev/null" 2>> $TIMES
if [ "$?" -eq "0" ]; then
   echo 4000 >> "xcoords.txt"
fi
$tmpdir/timeout 60 "/usr/bin/time -p $OUTPUT < 8000words.txt > /dev/null" 2>> $TIMES
if [ "$?" -eq "0" ]; then
   echo 8000 >> "xcoords.txt"
fi

# Filter and format the timing data
grep user $TIMES > "temptimes2.txt"

# Make sure that there actually were times
if [ -s "temptimes2.txt" -o -s "xcoords.txt" ]; then

   sed -e 's/user //' "temptimes2.txt" | paste "xcoords.txt" - > $PLOTOUTPUT

   # Use a "Here document" to shovel all of the commands to gnuplot.
   gnuplot <<EOF
set term postscript eps enhanced     # Set the output type to EPS
set output "$PLOTFILE"               # Call it $PLOTFILE
set nokey                            # Remove the legend from the output
set title "Running Time of $student calculating n input"
set xlabel "Size of Input n"
set ylabel "Time in Seconds"
plot "$PLOTOUTPUT" with linespoints # plot the contents of the file with
                                    # points that are connected by lines
EOF

fi

GNUPLOT

The code piped into the gnuplot program at the end of the script is worth explaining. Set term tells GNUPLOT what kind of output to generate, in this case an Enhanced PostScript image. Set output gives the name of a file to use for the output. Next the legend is removed, the title is set, and axis labels are created. The final command plots the data in $PLOTOUTPUT using the linespoints style (points joined by lines).

Cautions

It is a very involved process to create timing tests. In the included example, the appropriate timeout values and input sizes had to be determined. (The above script was used for A8Q1 and A8Q2 from Winter 2009). You have to know about the current assignment in order to craft tests that take a measurable amount of time. You have to handcraft the testing code so that it can be called from the timing script. You then have to hand edit the xcoords that will optionally be used for each timing. Hopefully this process will be made easier, but it's still quite involved right now.

As of end of Winter 2009, the script is only useful if the the student code is functional enough to run through all the tests within the script without any kind of error, if there is errors in several part of the plot, it will make the resulting plot either incorrect and inconsistent or won't even appear all together (as there aren't any data to be used to plot). In times like that, hand checking should be used instead.

Security

Do not put any testing or plotting code inside of computeMarks-postprocess. That script is run by the course account's user. Student code should only be run under a seperate account, cs136-t. This is the purpose of the runTests scripts, and why (if possible) the plotting functionality should be moved there.

Current Progress

As of now, the graph is not robust enough to deal with the possible high error in the student codes, meaning that the possibly for the graphs to mess up very high. This would most likely provide more harm for the markers than they helped. That is why more time should be spend in making the plotting script more robust before they should be used for assignment marking.

-- KyleSpaans - 18 Mar 2009