How to Prepare an Assignment for Marking in MarkUs

Courses where students submit to MarkUs (including through Open edX)

Some courses need to upload files to MarkUs for graders to see. Students would also see these files after the marks are released/published. An example is a course that runs automated testing after the due date, and the testing results need to be uploaded onto MarkUs. However, not every course needs to do this, so this section is broken up into two cases.

Students submit to MarkUs and ISA does not need to upload any files to MarkUs

If you just need to see the files that students submitted on-time when marking:

1. After the assignment is due, go to the Submissions tab.
2. Click the "Collect All Submissions" link near the top-right.
3. Choose "Mark the files that were in the students' accounts at due date". Wait for all the rows to become green, and then you can start marking.

Students submit on MarkUs and ISA needs to upload files to MarkUs

Here are steps you can use for courses that need to upload files to MarkUs for graders to see. Run all the commands from the course account in a Terminal. The course account is usually named csNNN, and the server to use is linux.student.cs.uwaterloo.ca.

Note for assignments with questions that don't involve submitting on Markus

Step 1: After the assignment is due, go to Submissions tab and click "Collect All Submissions" link near the top-right. Choose "Mark the files that were in the students' accounts at due date". Wait for all the rows to become green.

If you want to give students a few extra minutes after the due date, you can enter that when collecting the submissions. For example, if an assignment is due at 9:00PM but you want to accept submissions until 9:10PM, enter 9:10PM when you collect all the submissions.

Markers can then start marking if they want. Alternatively, you can make the markers wait until the test results are uploaded to begin marking. Some courses combine the student's submissions into one big file called something like GRADED_ASSIGNMENT.ss (or .py). In this case, the TAs will have to wait until this file is uploaded to begin marking.

Step 2: Download the collected submissions from MarkUs to the course account by going to Submissions tab, then clicking the "Download all submissions" link. Save the zip file in a new folder in the course account, and extract the zip. You'll find all the student's collected submissions.

Alternatively, you can download the collected submissions from the Terminal. Replace ASSN with the assignment short ID or name, and SAVE_FOLDER with the path to the folder where you want submissions to be saved in the course account.

/u/isg/bin/markus.py download_marking ASSN SAVE_FOLDER

For example, to download the collected submissions for a01 to the ~/handin/a01_autotest/ folder, run:

/u/isg/bin/markus.py download_marking a01 ~/handin/a01_autotest/

Explanation for step 2: When students submit to MarkUs (doesn't matter if it's through Open edX), MarkUs will automatically copy the student's files to the course account's ~/handin/ folder, even if it's late. For example, if the assignment in MarkUs has repository folder set to a01, then MarkUs will updated ~/handin/a01/ folder every time a student submits, even past the due date. That is why you cannot use ~/handin/a01/ for grading, and why you need to create a separate copy (ie ~/handin/a01_autotest/) which only contains on-time submissions and which MarkUs will not update randomly.

Step 3: Run whatever automated testing scripts you want to run for this assignment (ex. run rst, scripts to check student's test cases, MarkUsScripts#MakeGradedAssignment_py, etc). Put the test results in the folder where you saved the submissions in Step 2. Place each student's test results in his or her folder.

Step 4: Run this command, which will upload the test results to MarkUs for graders to see. Replace ASSN and SAVE_FOLDER as described in Step 2. Replace FILE_TO_UPLOAD with the name of the test results file (ex. GRADED_ASSIGNMENT.ss in some courses).

/u/isg/bin/markus.py upload_marking --match 'FILE_TO_UPLOAD' ASSN SAVE_FOLDER

You can run this command multiple times if you have multiple files you need to upload. Replace FILE_TO_UPLOAD with each file you want to upload.

Students can see their test results when marks have been released. If you want students to see their test results earlier, use the markus.py upload_svn sub-command in addition to upload_marking. Students can see files uploaded to their SVN repository right away, but those files will not appear when marking unless you chose the correct revision when collecting submissions.

Step 5: Check in MarkUs that the test results are now visible when marking. It should appear in the drop-down menu of files.

Courses where students do NOT submit on MarkUs

This section is for courses where students do NOT submit on MarkUs. For example, students may submit through Marmoset or the submit command line tool. MarkUs is used only for grading. Run all the commands from the course account in a Terminal.

Step 1: If you haven't already, add all the students to MarkUs in the Users => Students tab. You can use the "Upload from course account" link near the top-right. In the assignment's settings, uncheck "Allow web submits" since students are not submitting on MarkUs, and hide the assignment. There is no need for students to see the assignment until marking is done.

Step 2: After the assignment is due, create a folder containing the student's submissions which you want to upload to Markus. Put each student's files inside their own folder. For example, the folder structure might look like this (one folder for each student):

a01_files
 |- a9smith
 |   |- helper.cc
 |   |- mycode.cc
 |- b9wang
 |   |- mycode.cc
 |- dkelly
 |   |- test.cc
 |   |- main.cc

The submit command line tool creates a folder like this inside the ~/handin/ folder in the course account.

For Marmoset, if your course has a marm_sql script and you use it, you will get a folder structure like this. Alternatively, if you chose to download submissions from Marmoset web site using the "Download all students' best submissions" link, you'll get a similar folder structure but with "-On-time" at the end. You can remove the "-On-time" by running these 3 commands, where EXTRACTED_FOLDER is the folder containing the student folders.

bash
cd EXTRACTED_FOLDER
for dir in *; do mv "$dir" "${dir%%-On-time*}"; done

Step 3: Use the markus.py create_group sub-command to create a group for each student in your folder of submissions. Markus expects students be in groups, but each group can contain just one student if students are working alone.

You can create groups with the following commands, replacing EXTRACTED_FOLDER with the folder from previous step, and replacing ASSN with the assignment short ID or name.

bash # This will make sure you are using the Bash shell.
cd EXTRACTED_FOLDER  # Go inside the folder you created from previous step
for student in *; do /u/isg/bin/markus.py create_group --repo-name $student ASSN $student $student; done

The last command uses Bash for loops to loop over every student, saving the student's username in the $student variable.

Unfortunately creating groups can take a while, but you can let it run in the background while you do other things. When this is finished, check the Groups and Submissions tab in MarkUs that the students are now listed.

Step 4: Run this command to upload each student's submission. Replace ASSN with the assignment short ID or name. Replace PATH_TO_FOLDER with the path to the folder you created in Step 2.

/u/isg/bin/markus.py upload_svn ASSN PATH_TO_FOLDER

Unfortunately this command will take a while, so try running it when you're doing something else. If the wait is unacceptable, you can use the upload_marking sub-command instead.

If you get errors saying "svn: E170013: Unable to connect to a repository" it might be because your course's Markus is deployed so that you do not have direct access to the repository. In this case, you should click Collect All Submissions, choose the first option "Mark the files that are in the students' accounts right now", and wait for rows to all become green. Then use upload_marking sub-command instead: /u/isg/bin/markus.py upload_marking ASSN PATH_TO_FOLDER
Then check that the files are visible when grading, and you can skip next step (ie you don't need to collect submissions again).

Step 5: In the MarkUs website, go to Submissions tab for the assignment. Click "Collect All Submissions" link near the top-right. Choose "Mark the files that were in the students' accounts right now". Wait for all the rows to become green. When the rows are all green, marking can start.