Rollback request, feedback request & Gracket conversion

Rollback requests

Rollback/Roll Forward Policy

Rollback/roll forward requests policy changes from term to term (possibly several times during a term). Hence, please confirm with your instructor(s) about the current rollback/roll forward policy before proceeding with handling these requests. Using the Fall 2021 term as an example, the policies are:

For A01:

• The latest submission before the deadline + 100 minutes will be marked, and 1% is deducted from the entire assignment for each minute the assignment is late.
• Students may request us to "roll forward" to a later submission after the deadline, or "roll back" to an earlier version. If the student asks us to roll forward to a later version, the 1% per minute late penalty will be applied to the entire assignment. If the student asks us to roll backward to an earlier version, there will be no penalty.
• Only one timestamp can be accepted, taking separate timestamps for separate questions is not permitted. All the latest versions of the students’ files they submitted before that timestamp will be remarked.
• The deadline for submitting roll forward requests is one week from the assignment deadline.

From A02 to A05:

• Automatically mark the last submission made before the assignment due date (typically Tuesdays at 5:00 pm), unless the student explicitly asks us to do otherwise
• All the other policies of A01 carry forward

From A06 to A11:

• The Tuesday 5:00 PM deadline stays the same as in previous assignments
• Rollback/roll forward requests that are received before 10:00 am Wednesday will be treated more favourably than requests received after that.
• The student’s grade for the question (there is no longer an assignment-wide penalty) will be based on:
  • Hand-marking of the files we’ve been told about before 10:00 am Wednesday (No additional hand-marking after the deadline).
  • Automatic marking/testing (correctness) of the files you’ve requested.
  • Minus a penalty of 1% per minute late.
  • If the request comes after 10AM Wednesday, a deduction of one rubric level on the correctness marks.

Using the script

Step 1: Go to the assignment submission list on MarkUs, and find the student's repository. This can be done through your admin account.

Step 2: Select the appropriate date, and click "Find by Revision Number"

Step 3: Open Putty and create an SSH session with cs135.

Step 4: Enter the command "regrade.sh" and follow the script's instruction.

Usage: ./markus/bin/regrade.sh <rollback|feedback> <-A|-a|-h> [-R] [-f=file1,file2,...] questid aXX ['YYYY-MM-DD hh:mm:ss']

Options "<rollback|feedback>": Select “rollback” for rollback/roll forward requests.

Flags "<-A|-a|-h>":

• Flag “-A” will make the script generate both the rollback autotesting result file “rollback/AUTOGRADING.ss” and the file that will be used for handmarking, “rollback/HANDMARKING.ss”
• Flag “-a” will only generate the rollback autotesting result file “rollback/AUTOGRADING.ss"
• Flag “-h” will only generate the file that will be used for handmarking “rollback/HANDMARKING.ss”
• Flag “-R” will *REPLACE* files in the student's submission instead of uploading to a separate folder

Optional flag “-f”:

• If the request only wants to rollback/roll forward specific question file(s), enter the names of all required question files separated by commas. This will make the script generate all the question files “rollback/aXXqXX.rkt” and upload them to MarkUs.

• If the flag “-f” is omitted in the command, there will be no question files “rollback/aXXqXX.rkt” generated. Instead, only “rollback/AUTOGRADING.ss” or “rollback/HANDMARKING.ss” or both will be generated and uploaded according to the previous flag (-A, -a or -h).
• Example: “-f=a09p02.rkt, a09p03.rkt” means that the request only wants to rollback/roll forward questions 2 and 3 in assignment 9. Correspondingly, files “rollback/a09p02.rkt” and “rollback/a09p03.rkt” will be generated by the script and being uploaded onto MarkUs.

Timestamp "['YYYY-MM-DD hh:mm:ss']": adding +1 second to the time will guarantee a successful rollback.

Example:

For a request that wants to rollback questions 1 and 2 in assignment 10 that has the timestamp 12/06/2021, 4:23:03 PM, the following command will address the rollback request:

./markus/bin/regrade.sh rollback -a -f=a10q01.rkt,a10q02.rkt questid a10 '2021-12-06 16:23:04'

Step 6: Enter the command "y", "n" at the end of the script when it prompts you to do so.

Step 7: Update their correctness mark on MarkUs according to the current rollback/roll forward policy, based on the student's mark after the rollback/roll forward.

Doing it Manually

It is highly recommended that you use the script to perform feedbak requests since this is a very cumbersome process. Only use this method if there is an issue with the script.

Performing a rollback request manually is essentially the same thing as Collecting Assignments, steps 1-6, except you are performing it for one student instead of all students. Here are the modified steps from Collecting Assignments for performing a rollback request manually.

Steps 1-3: Repeat steps from Using the Script.

Step 4: Backup the student's handin directory in handin/a##_autotest by using

mv studentId studentId_prerollback

Where a## is the assignment that you're performing the rollback in.

It's important to back up the student's handin directory in case you need to revert the rollback.

Step 5: Download the student's submission in MarkUs to the course account. You can do this by running the command:

/u/isg/bin/markus.py download --time 'YYYY-MM-DD hh:mm:ss' a## studentId /u/cs135/handin/a##_autotest/

Where 'YYYY-MM-DD hh:mm:ss' is the year, month, date, hour, minutes, seconds of the submission that the student would like to rollback to, a## is the assignment that you're performing the rollback in, studentId is the student you're performing the rollback on, and the last parameter is the path to where you want to save the student's files (which should always be inside of handin/a##_autotest).

Step 6: run rst to run correctness tests on the student submission e.g.

rst -t t -s studentId a##_autotest 0 studentId_rollback

This will save the correctness tests results inside of marking/a##/test.0.studentId_rollback

Step 7: run check-testcases by using the run-student.sh script inside of the check-testcases directory e.g.

./run-student.sh a## studentId

The results of this command will be saved in check-testcases/a##/test-cases/individual-results/

This can be done in parallel with step 5 if you have two terminals open.

Step 8: We can now make the AUTOGRADING.ss file. Go to ~/markus/bin. Run the command: ./make_autotesting --student userid axx >> ~/handin/axx_autotest/userid/AUTOGRADING.ss to create AUTOGRADING.ss file.

If you need to use MakeGradedAssignment,py, I found it to be easiest to go inside of the handin/a##_autotest/studentId directory for this step. Run the command: python ~/markus/bin/MakeGradedAssignment.py ## path_to_correctness_test_results path_to_check-testcases_results path_to_file1 path_to_file2 ... This command can be huge and cumbersome, which is why it's easiest to work inside of the student's handin directory so that you can just use the file names instead of the path of the files. This command will produce the contents of what the GRADED_ASSIGNMENT.ss file should look like. Double-check that this is what it's supposed to look like before you create the file, and fix any errors that might have arised. Once you have confirmed that it looks correct, run the same command but with > GRADED_ASSIGNMENT.ss appended at the end of the command. This will create the GRADED_ASSIGNMENT.ss file in the current directory that you're working in.

Step 9: Once the AUTOGRADING.ss file has been created, we can now upload the AUTOGRADING.ss file onto MarkUs. To do this, run

/u/isg/bin/markus.py upload_marking a## studentId path_to_AUTOGRADING.ss name_of_file_on_markus

name_of_file_on_markus is the name that will show up on MarkUs when students use the drop down menu to select files.

Step 10: Check on MarkUs that the AUTOGRADING.ss file that was uploaded is the correct one.

Feedback request

Students who submit their work late may want feedback on their submission, even though it is not worth marks. Students are told to email cs135@uwaterloo.ca so the ISAs know to look for it (ISAs are not required to automatically look for lates) and mark for feedback. Students requesting feedback may be students who were excused from an assignment but, as per instructions given by the ISC or instructor when excused, highly recommending they do and submit the assignment for feedback so they know how they are doing and to help avoid falling behind.

Using the script

If the student did not submit anything before the deadline, (see a) click on the user's name in the *Repository column. Use the pull down to the left of Find by Revision Number to find the last revision. Click Collect and Grade This Revision.

Follow step 1 - 6 in Using the script under Rollback Request above. Make sure to choose "feedback" for feedback requests in Step 4.

Usage: ./markus/bin/regrade.sh <rollback|feedback> <-A|-a|-h> [-R] [-f=file1,file2,...] questid aXX ['YYYY-MM-DD hh:mm:ss']

Options "<rollback|feedback>" : Select "feedback" for feedback requests

Step 7: Grade as usual and provide the results in an annotation as well as filling in the marks for those tests. Make sure there are annotations to explain where the student lost marks.

• In the Summary tab for this student, you will see a Bonus/Deductions section (scroll down). Subtract off the feedback marks (for Comment put "Feedback marks").
• Change the marking status to Complete and release.
• Email the student and tell them the feedback is now available in MarkUs and to let you know if they have any questions.

** if something was submitted by the deadline and hence, marked, we need to preserve the original marking in case it counts toward their grade. Therefore, run the script with the "-f" option and keep all the files that already existed. Follow the steps above and make sure you keep track of the original mark/marking so you don't subtract that off. Put comments and marks in the master marking as needed. Remember to subtract off the feedback marks only.

Doing it Manually

It is highly recommended that you use the script to perform feedback requests since this is a very cumbersome process. Only use this method if there is an issue with the script.

The key steps are, collecting just that student's late submission in MarkUs, grading as normal, then, in MarkUs, subtracting off the mark their submission would have received, had it been submitted on time (so as not to accidentally give them the marks). The reason for doing it this way is so that instructors or ISC can go in and see the student's work in case the student's grade at end of term is borderline passing or borderline for being admitted to CS 136 or there is an INC request or grade appeal.

Step 1: Logon to MarkUs with your admin access.

Step 2: In the appropriate assignment, Submissions tab, find the student's submission (you will likely have to View All first).

Step 3: Unrelease the marking for this student (assuming this feedback is occurring after the release of marking for the assignment)

Step 4: If the student did not submit anything before the deadline, click on the user's name in the *Repository column. Use the pull down to the left of Find by Revision Number to find the last revision. Collect and Grade This Revision.

• Grade as usual (you can run the correctness tests on the code for an individual student, if necessary) and provide the results in an annotation as well as filling in the marks for those tests). Make sure there are annotations to explain where the student lost marks.
• In the Summary tab for this student, you will see a Bonus/Deductions section (scroll down). Subtract off the feedback marks (for Comment put "Feedback marks".
• Change the marking status to Complete and release.
• Email the student and tell them the feedback is now available in MarkUs and to let you know if they have any questions.

If something was submitted by the deadline and hence, marked, we need to preserve the original marking in case it counts toward their grade. Therefore do the Steps above but instead of clicking Collect and Grade This Revision, download the files individually from the repository (steps shown in Rollback Requests -> Doing it Manutally) then continue with the remaining points (make sure you keep track of the original mark/marking so you don't subtract that off). Put comments and marks in the master marking as needed. Remember to subtract off the feedback marks only.

Gracket conversion

Gracket conversion requires manual conversion.

Step 1: SSH to CS135

Step 2: run: grep -wHirl gracket ~/handin/aXX_autotest/*/*.ss

Where "aXX" indicates the assignment.

Example: grep -wHirl gracket ~/handin/a05_autotest/*/*.ss

This will create a list of students that have Gracket issues in A05.

Step 3: Logon to the student's MarkUs, then find the assignment that is affected, and download the Racket file that needs to be manually converted.

Step 4: Run the following script on the downloaded Gracket file gracketfile.rkt:

/u/isg/racket7.9/bin/racket ./u/yc2lee/wxme_converter.rkt gracketfile.rkt > plainfile.rkt

This should save the resulted plain text Racket file into plainfile.rkt.

Step 5: Upload the converted racket file to the appropriate assignment, either manually through the student’s account on MarkUs or using the script:

~/markus/bin/make_autotesting.sh -s studentid aXX

Step 6: Perform a rollback request (in this case it would be a rollforward) towards the newly uploaded repository.