Marking Meetings

All information about prepping for, holding, and concluding marking meetings will be found on this page.

Pre-meeting Tasks

Prior to the meeting, you must:

1. Create the Assignment Marking Channel
2. Schedule the meeting on Teams
3. Create all appropriate question threads
4. Finalize the Marking Schemes
5. Prepare for your Presentation

The first task is to create the appropriate marking channel. To do this, you must have Owner Level access to the Graders team. If you do not, please contact your supervisor. Channels can most easily be created by clicking on the three dots next to the team name and clicking "Add Channel". Enter the name "Assignment XX Marking" with the relevant assignment number, and check the box next to "Automatically show this channel in everyone’s channel list."

Next we must schedule the meeting on Teams. To do this, go to the appropriate marking channel and click on the bottom-arrow next to "Meet" in the page's top-right corner. Click on "Schedule Meeting". Title the meeting "CS135 AXX Marking Meeting", and fill in the appropriate time.

Now, question threads must be created. Question threads must be created for each criteria being marked on the assignment. Additionally, the following threads must be created:

1. Problems with Correctness & Check Tests/Cases Marks – graders leave the userid's of students whose correctness or testing rubric levels have not been filled.
2. General Marking Questions – general marking clarifications which don't fit in any of the other threads should be asked in this thread
3. Common Errors – graders should report common student errors and deductions during and after the completion of their marking. All graders must include common errors.

Once the channel is fully prepared, it's time to prepare for the meeting. Remember that you are presenting to TAs, and should therefore be well-prepared. First, conduct a final proofread of the Marking Schemes and correct any mistakes. Next, attempt to mark at least a few solutions before the conducting the meeting. This could help identify some flaws or mistakes in the marking schemes or annotations so that you can make changes in time.

Finally, you should go over your role for the meeting. In Fall 2021, two ISAs led the handmarking team; one would go over the marking schemes with the TAs and the other would walkthrough a few submissions on MarkUs. Make sure you have gone over these a couple times to ensure you are giving a clear presentation to the TAs. You may even want to do a practice presentation on your own.

Running the Meeting

In marking meetings, ISAs must walk TAs through the marking schemes and mark example submissions. Any and all general or assignment-specific expectations must be communicated. Go through each criterion in detail. The marking scheme for A01 should be carefully explained to markers because the Racket language and CS135 course are completely new to most of them. As the markers get more familiar with the marking schemes, you can pick out what is new and important in the marking scheme only. Most of your focus later in the term will be going through the specific marking scheme. Try to keep the meeting reasonably short, most definitely under 1 hour and ideally ~30 minutes.

Ensure that you record the meeting, as well, for graders who miss it. Inform all attendees that the meeting is being recorded. The ISA who set-up the meeting on Teams should click “End Meeting” instead of “Leave” when it is ending.

Post-meeting Tasks

The following must be addressed after the meeting:

1. Meeting Attendance Summary
2. Marking Meeting Summary Post
3. Communication with TAs who missed the meeting

Following the meeting, the ISA who clicked "End Meeting" should see a downloadable .csv file containing the Attendance Report of the meeting. Information about who was present and who was absent should be noted down by comparing the attendance to the full TA List.

All TAs who missed the meeting should be messaged immediately. The message should first ask them to watch the recording and read over the meeting summary post, then to schedule a time for a 1-1 follow-up meeting to ensure they have understood what is expected of them.

Finally, a new thread should be created in the channel named "AXX Marking Meeting Summary." This post should contain all relevant information about marking for the assignment as well as a link to the meeting recording. It should also indicate to the TAs to "thumbs up" the post and respond once their first 5 marking submissions are ready for feedback. The post should contain numerous links to other threads, as well. Here is an example:

"CS135_Graders_F21 Please read this as there may be some info here (including for solutions) that you don't know about. Reminder the spot-checking deadline is Thursday, December 9th, 12pm if you attended the marking meeting, and the usual, Friday, December 10th, 10:00 am, if you did not.

• What if I missed the marking meeting? Please watch the marking meeting video here: (insert link to marking meeting recording) and then and an ISA will plan a brief ~ 10 minutes chat with you to address any special instructions or doubts. We will be following up on this.
• Where do I go to mark?
  • Only start marking if you either attended the live marking meeting or watched the video later AND did a debrief with an ISA after watching the video, it is important you understand how the bibliography marking works
  • To see the submissions, please go to (insert link to MarkUs page) and log in with your WATIAM userid and password. If you then click Assignment 11, you will see all the submissions. If you click on the group name (NOT repository) for a student then you will be able to start marking their submission.
• Where do I find the marking schemes and solutions?
  • The marking schemes are in the Files tab at the top of this channel. We have a general and a specific marking scheme, please use BOTH for marking.
  • All required solutions are embedded in the Specific Marking Scheme, please use those.
  • All other solutions are in the files tab.
• What do I do once I have marked my first 5 submissions? The deadline for these is Thursday 12 PM, but if you finish before, please reply to this thread to get feedback earlier.
• What do I do if I cannot finish marking on time for some reason or the other? Contact Bani Singh on Teams immediately as soon as you know, or email the course account (cs135@uwaterloo.ca) as specified by Karen Anderson.
• Students are doing weird stuff or autotesting just seems to be done wrong - what to do? If this is related to correctness and check test cases then please reply under (insert link to appropriate thread). If not then use the General Questions thread (insert link to appropriate thread).
• Make sure to always find the right thread for your concerns to we can keep track of all clarifications properly.
• Remember when you are done marking send an email to cs135@uwaterloo.ca with the number of hours you took. And please don't forget to post your common errors in the Common Errors thread (insert link to appropriate thread) then do that as well.
• If you have any concerns related to your hours then please contact Karen Anderson. One thing to keep in mind is, we aim to have your total hours add up for the term properly. However per assignment it can vary, some assignments may take more than 5 hours but then other assignments will take much less and it will balance out.
• What do I do after reading this post? Please react with a thumbs up so that we know you have seen this and that you know your way around."

-- Adam Abdulhamid - 2021-12-13

Comments