Resources and Information for CS 135 Tutors

NOTE: THIS PAGE IS OUTDATED AND MAY HAVE MISLEADING INFO. PLEASE SEE THIS INSTEAD: CompSci135TutorDuties

This twiki is for technical details only. Any other duty information can be found in Waterloo LEARN (D2L): CS Instructional Support Group -> Course Admin & Duties for ISAs.

Key Links

• ApacheSubversion (SVN)
• CompSci135TutorDuties
• TestingGuide
• CheckTestcases
• PiazzaTips
• ProcmailSetup
• MossScript
• WebTemplate
• MarkUs
• CS135MarkingSchemes
• Clickers
• ClickerDebuggingTips
• ceconv
• Scripts
• lintR (a marking tool under development by bwbecker; not yet ready to use)

READ ME - Approximate Twiki timeline

Before the term starts

Read the CompSci135TutorDuties page; it has a good overview of all the tasks that need to be done throughout the term. Make sure an ISA(s) is assigned to each major duty and is aware of which Twiki pages to read. Practice navigating the folders in the course account with Samba and/or Bash.

Make sure the cs135@uwaterloo.ca course e-mail is set up and everyone knows the log-in information. You may even have e-mails already, so check this as soon as possible.

Set up Piazza/MarkUs and update the course webpage with the new information for the term.

Decide office hours tutorial slots, and attended lecture sections for the ISAs. If you are in MC 4065, add our consulting hours to the shared ISG calendar as well, so you can book the workstations you need.

Set up a communication channel for the office for contact outside of work. Something like Slack, Discord, Facebook, etc. works well. Discuss approximate working hours for the team.

First week of term

Get to know the Twiki pages and scripts associated with your duties. The important pages are the ones at the top of this page and which are mentioned in CompSci135TutorDuties.

Get office hours and photos from instructors and add them to the Personnel page.

Early weeks of term

Practice using the scripts needed for collecting, testing, marking, etc. each assignment. There are commands you can use without making any of your testing stuff live - one example is to create a testing assignment which is hidden on MarkUs. Try creating fake testing folders and running rst/distrst, run-all.sh, and the other scripts.

Consolidating MarkUs marks with Stepping problem marks

Some CS135 assignments have Stepping problems. Stepping problems don't require any file submissions, so there's a potential problem when using it with MarkUs.

The problem: Suppose a student, Stewdant, completes some Stepping problems, but he does not go to MarkUs to submit anything. Since Stewdant has no files submitted in MarkUs, he'll get a grade of 0 in MarkUs. However, Stewdant should not be getting 0 on the assignment because he did complete some Stepping problems.

How to fix:

The problem is tricky to fix because if Stewdant does not visit the assignment in MarkUs at all, he won't have a repository for the assignment. You'll have to create one manually for him. The easiest way is to do a role switch in MarkUs into Stewdant's account, then click on the assignment link. After clicking the link, you can switch back to an admin.

First, complete Steps 1 to 4 in the procedure at MarkUsGroupsSVNRepos#Auto_testing_Procedure. After Step 4 (checking out), you should have a folder aXX_autotest in /u/cs135/handin.

This Bash script will find all the students who finished at least one Stepping problem, but does not have a folder in aXX_autotest (see MarkUsGroupsSVNRepos for more details on autotesting). Copy and paste this code into a file, and save it somewhere on the course account.

#!/bin/bash

assn="a03"
# assignment 3 has stepping problems 1a to 1f
steppingprob="assignment3q1[a-f]"

course=`whoami`

cd /u/$course/course/stepping

for stud in *; do
        # check if stepping done
        if egrep -q "${steppingprob}.*(value|err) c$" ${stud}; then
                # check if in classlist
                if egrep -q "${stud}" /u/$course/.classlist; then
                        # make sure he has a folder in autotest
                        if [ ! -d /u/$course/handin/${assn}_autotest/${stud} ]; then
                                echo ${stud}
                        fi
                fi
        fi
done

You can run this script by opening a terminal, and navigating to the folder the script is in (using Bash command cd). Then run bash script.sh where script.sh is whatever you named the script file.

If the script displays no students, then nobody is affected and you can continue with autotesting.

Otherwise, for each student displayed, we have to create a repository manually for him. Role switch into the student's account, click the assignment link, then role switch back to admin. Repeat this for each student.

Once the repositories are created, check them out manually to the aXX_autotest folder.

1. In a terminal, using cd, navigate to the handin/aXX_autotest folder.
2. For each student, lookup their repository name in MarkUs 's Submissions tab. The name will be something like group_XXXX where the XXXX is a four digit number.
3. In handin/aXX_autotest folder, run a command similar to the following:

svn co svn+ssh://markus@ubuntu1404.student.cs.uwaterloo.ca/u/markus/markus_cs135_w/svn-repos-root/group_4234/a03 s9stud

You have to replace markus_cs135_w with your course and term, and the group_4234 with the repository name you looked up. Replace a9stud with the student's Quest ID.

Run a similar command for each student printed by script above.

Afterwards, you can continue with autotesting procedure.

Note: If an affected student also has late submissions, make sure to give him 0 for everything submitted late. (The checkout command will pull late submission files which will then be autotested, so you can also delete these files from the student's handin directory before autotesting.)

Running Correctness Tests

You can run this anywhere in the course account.

For single submissions:

rst -t t -s userid aXX testdir outputdir

• userid is the id of the user you want to test (it is the name of their handin directory)
• aXX is the assignment number
• testdir specifies the directory of tests you want to test, i.e. if you want to test from test.pt, you need to use pt as this argument
• outputdir specifies the name of the directory in which the test results will go

rst -t t -s crdong a06 0 finalize

This will run the correctness tests from test.0 and run them on crdong for a06. His test results will be in cs135/marking/a06/test.0.finalize

If you want to run it on a lot of students:

distrst -t t -s '*' aXX testdir outputdir

'*' can be replaced with any other regular expression if you want a specific subset of students

Assign submissions to graders (Winter 2016)

In Winter 2016, I used some scripts that are in /cs135/markus/bin/assign_graders (not those mentioned on MarkUsScripts )

python distribute.py gradermap.txt file2 > file3

Checking Assignment Averages by Grader Using the avgbygrader.py Script

The purpose of the avgbygrader.py script is to help enforce marking consistency between different TAs. It reads a grade report for an assignment as well as a grader-student distribution file and produces a summary of each grader and the assignment average of the students they marked.

This script can be found in cs135/markus/bin/assign_graders

[Note]: The results of this script does not conclusively show whether marking inconsistency exists or not. Rather, its purpose is to identify the TAs whose assignment averages are abnormally high or low, so that the ISAs can double check whether these TAs marked correctly prior to releasing the grades.

How to Use (for assignment aXX):

1.First we need the grader-student distribution map. To download it, go on MarkUs -> aXX -> Graders and press “download” at the top-right corner. Rename this file aXXmap.csv.

2.Ideally, every student in aXXmap.csv should be paired to a grader. However, this is often not the case and sometimes there are students in the file that are not assigned to any graders. So open aXXmap.csv and delete any student that has not been assigned to a grader (ie. Lines that are not in the form of studentID,graderID are fine, but delete lines that only have a studentID).

3.Now we need the grade report for aXX. To access it, go on MarkUs -> aXX -> Submissions and press “Detailed CSV Report” (NOT “CSV Report”!) in order to download it. Rename this file to aXXgrades.csv.

4.Move aXXmap.csv and aXXgrades.csv to the same folder as avgbygrader.py (by default it is cs135/markus/bin/assign_graders).

5.Open avgbygrader.py. Set the following variables:

a.grader_dist_file = ‘aXXmap.csv’

b.assn_report = ‘aXXgrades.csv’

c.summary_file = open(‘aXXsummary.csv’, ‘a’)

(this will save the result of the script to aXXsummary.csv in the same folder).

6.Run the script with python avgbygrader.py.

7.Voila! You now have an Excel sheet summarizing the average assignment grade by each grader.

Also: To check grading progress for TA/graders, go to Axx -> Dashboard and see Grader Distribution under the mark distribution graph.

Non-Technical (To be moved to D2L)

Outdated

No Longer Relevant