CS135 Marking Schemes

For every assignment you will have to create/modify three files. These three files should completed before the marking meeting and .

Many versions of these files are archived, so take advantage of this by copying the marking scheme files of previous terms or assignments and modifying these files to suit your needs. This is an especially good idea if your assignment builds off a previous term's assignment or utilizes questions from previous assignments.

Marking Scheme

The marking scheme refers to the document that you will distribute and cover with the graders during the marking meeting. This file should contain more detailed criteria explantions and marking intructions for graders than the marking rubric that will uploaded to MarkUs. This is usually the first document you write during the pre-marking process. Most of the previous marking schemes found in the archive are written using Microsoft Word and converted to pdf for distribution to markers.

Style criteria are often maintained from assignment to assignment and require very few changes unless the style guide for the course gets major changes between assignments.

Contracts

Especially in later assignments, there may be many acceptable contracts. Students may define their own types, or may use literal definitions for compound types (Eg. (listof (list Nat Str)) instead of AL). You must be careful to cover as many of these as possible when writing the marking scheme. The Fall 2014 marking schemes have good examples of the most current format used for this.

It may be useful to look through student's submissions in the handin folder, or egrep for incorrect contracts that you may suspect students will use. This will give you a more complete idea of what needs to be mentioned in your marking scheme (no pun intended).

To find (mostly) all contracts for a single function: (run from handin/a0x directory)

egrep -r --include="FILENAME.rkt" -m 1 -i "^.*;; *FUNCTION-NAME: *" *

Replace FILENAME and FUNCTION-NAME appropriately. All options after -r are optional but greatly increase the speed of the search.

Further, if you are interested in a more concise output, this variation will be more helpful:

egrep -r --include="FILENAME.rkt" -m 1 -i "^.*;; *FUNCTION-NAME: *" * | tr A-Z a-z | perl -pi -e 's/:/: /g' | perl -pi -e 's/->/ -> /g' | tr -s " -" | sed 's/ $//' | sort | uniq

It attempts to remove all duplicates by standardizing spacing and capitalization. Keep in mind that this converts all characters to lowercase, so types such as "Nat" and "Int" will appear to be incorrect.

To find and sort all contracts for all functions: (run from handin/a0x directory)

egrep -r --include="*.rkt" -i -h "^.*;; *(\w|-)+: *\w*.*->" * | sort

The "| sort" can be removed to view the unsorted output in real time, but this is generally not useful.

For concise output, as above:

egrep -r --include="*.rkt" -i -h "^.*;; *(\w|-)+: *\w*.*->" * | tr A-Z a-z | perl -pi -e 's/:/: /g' | perl -pi -e 's/->/ -> /g' | tr -s " -" | sed 's/ $//' | sort | uniq

Bonus Questions

Most bonus questions will have the same marking criteria as a normal question, but should not be included in the normal marking criteria as these questions are optional. To help students differentiate you should label the bonus criteria of the marking rubric differently, such as Bonus: Q4 Correctness rather than just Q4: Correctness. Whether or not you choose to do this is not as important as being consistent with the marking schemes.

Marking Rubric

For generating the marking rubric to upload to MarkUs, the buildrubric script can be utilized to do the bulk of the work for making a new rubric.

This rubric is a csv file containing the criteria outlined in the marking scheme as well as the weighting and marking levels. A general criteria in this file has the format

Criteria Name,Weight,Very Poor,Weak,Passable,Good,Excellent,Very Poor level description.,Weak level description.,Passable level description.,Good level description.,Excellent level description.

and will have multiple lines such as the one above separated by a newline.

A marking rubric can be uploaded directly to MarkUs by navigating to the proper assignment under the assignment tab, clicking the marking scheme tab, and finally clicking the upload link and selecting the rubric file. As long as this file is in the proper format it will upload properly. Any criteria already on MarkUs with the same name as a criteria in the uploaded file will be overwritten so you either modify the csv and upload again or make changes directly to MarkUs.

Since students only have access to the description of the level they achieve for a given criteria, make sure that these descriptions are meaningful on their own. For questions with banned functions, the level 0 should be similar to "0 of 14 tests passed, or banned functions used" in order to avoid confusion when the student sees that their mark differs from the result in GRADED_ASSIGNMENT.SS.

Annotations

Annotations, similar to the marking rubric, will be uploaded to MarkUs for graders to use and students to eventually see. When coming up with annotations it is necessary to think of useful feedback for common errors in the assignment. This requires some thought as most of these annotations will need to be finished before you have marked any assignments. Some commonly useful annotations include correct contracts, reasons for losing marks in a given criteria, and any errors you expect to see due to your time fixing those errors in office hours. Markers can add annotations during marking so the initial annotations only cover the more obvious and frequently used annotations.

Most markers found it useful to have the annotations separated into groups by question with addition categories for general style annotations.

Instructor Input

Different instructors will have differing levels of involvement with the marking schemes. Typically in Fall 2014, we would have a draft written on Thursday, sent to the instructor for input, and then meet with them on Monday or Tuesday to discuss the assignment in more detail. The mark meetings were then on Wednesday.

-- AndrewGemmel - 2014-12-19