Resources and Information for CS 146 ISAs

This twiki is for technical details only. Any other duty information can be found in Waterloo LEARN (D2L): CS Instructional Support Group -> Course Admin & Duties for ISAs.

The old Twiki page is attached as summary.pdf (scroll all the way down to the bottom). There's some more details about things there, though you can already find most of that information on other pages in the Twiki.

Onboarding Timeline (Last updated Winter 2023)

The following are tasks that should be completed within the first week of the term so that the course may run smoothly.

• Meet with the instructor and ISC to discuss expectations.
• Ensure the course website is updated:
  • All mentions of the term/year (Home, Grading, Lectures, Marmoset)
  • Update the Exams page to indicate that nothing has yet been scheduled.
    
  • Check with the instructor that the grading scheme on the Grading page is correct.
  • Restrict access to the lecture and tutorial summaries on the Handouts page (you can still access them by typing in the literal urls)
  • Update the Lectures page to reflect your term's information.
  • Update the Personnel page to reflect your term's information, and check with your instructors that their emails and office hours are listed correctly.
• Import the previous year's Learn quizzes into the current year's, and adjust the dates on the quizzes according to your instructor's instructions.
  • Import: Course Admin > Import/Export > Copy Components
    • Select "Select Components"
    • Import the Question Library, the Quizzes, and Grade Items.
• Book the CS Consulting Centre for regular office hours. See the ISG Learn course or ask your ISC for instructions on booking the CS Consulting Centre. Try to choose varied times that don't conflict with common first year math courses, especially advanced offerings. Update the ISA consulting schedule on the personnel section of the course website. It is recommended to also allow for consulting by appointment. Four hours a week were booked in Winter 2020.
• Familiarize yourself with the Marmoset assignment project structure. ssh into the course testing account (as described below) and navigate into the marmoset subdirectory. Here you will find testing setups from pervious course offerings, including all relevant test scripts. Previous assignment testing setups can usually be easily modified to work for current assignments.
• Ensure you have access to:
  • the Marmoset, as an instructor (otherwise contact your ISC)
  • the course E-Mail, with forwarding (otherwise contact your ISC)
  • the course Piazza, as an instructor (otherwise contact your ISC)
• On the course account, create ~/.coursestaff.[TERM], following the template of previous terms, so .htaccess scripts will include the current stuff.
• Make a Piazza post describing the quintile system for Marmoset.
• Check out the archives from the previous years. You can find grading schemes used, relevant Piazza posts, and more there.

These are tasks which should be done on a regular/rolling basis

• Ensure that everything using the classlist is updated daily until a week past the add deadline:
  • Marmoset, MarkUs, Piazza, and cs146's handouts/.htaccess file (the ~/bin/export_classlist.bash script will do a bunch of the work for you)
  • Append the cs146 handouts/.htaccess file to cs145's handouts/.htaccess file so newly-enrolled students can look at the CS145 course notes, and send a notice on Piazza (if unsure about this, discuss with instructor and/or ISC)
  • Update the handbacks/.htaccess file
• Tutorials, if you are delivering them:
  • Consult the archived tutorial material from the previous course offering in the archives folder of the course account.
  • Discuss the timeline with your instructor(s). In Winter 2020, we observed that it may be good to give instructors the second and third week to discuss material to move some assignments ahead and the tutorial material closer to when it becomes relevant in lecture.
  • If you make any changes to the tutorial material, make sure updated versions are saved on the course account for archival purposes.
  • After each tutorial, update the relevant handouts to the tutorials section of the handouts page on the course website.
• Assignments:
  • Ensure that assignment projects are ready to be made public on Marmoset as soon as the instructor posts the corresponding assignment instructions. You can look at the old assignments using the direct urls to see what batches they come in.
  • There is a link on the course webpage (created 2023) that has an A-PRIMPL assembler. The link is currently disabled. When/If the A-PRIMPL Assembler assignment question is used again, you can give studens access to the site.
  • Maintain an Excel spreadsheet of assignment marks. Once an assignment deadline passes, download student scores from Marmoset and add them to the Excel sheet.
  • When due dates roll over, run MOSS on the assignments and report any plagiarism to your ISC. Run Racket/MMIX questions over ascii and C questions over C and ascii. Use discretion; lots of problems have very simple solutions and many students will likely use similar solutions. Look for damning evidence (exact structures, nonsensical renaming).
  • Handmarking:
    • Set things up on Markus for assignments that will be handmarked. Reference previous years' marking schemes, in their archive folder (ex. /u/cs146/archives/1221/...), to base your marking schemes on. Scripts to help set things up are in ~/markus/.
    • Return assignments through Markus; the handbacks system has security flaws. You may want to write a script that uploads a second file with a mark breakdown and calculation to each student's repository on Markus if you're using a marking scheme that isn't purely summing everything, such as was done in W22. You can find previous such scripts for W22 in ~/handback/.
• Quizzes:
  • Every week, check that the relevant material has been covered for the quiz (Quizzes > [quiz options] > Preview). If not, move the offending question to the next quiz.
  • When the quiz is released, announce it on Piazza.
  • Once the quiz deadline has passed, download the results (Grades > Enter Grades > Export), adjust it to make sure it's all-or-nothing, and add it to the marks spreadsheet.
• Piazza:
  • Make announcements here. When an announcement is made, forward the email to the course account for the audit trail.
  • Ensure no public questions are about specific release tests, as this gives up too much information about specific approaches to problems.
• Email:
  • Always use the course email when emailing students, and only email students at their uwaterloo emails. All other communication with students should be redirected to the course email.
  • Prefix emails with [cs146] for easy filtering for everyone receiving emails.
  • You may send emails from your personal email (for instance to instructors or your ISC) for course-related reasons, but cc the course email for the audit trail.
• Midterm:
  • If the midterm resubmission happens again in 2024, you should have the students submit through MarkUs, and not Marmoset. This will save you a lot of work.
• File Management:
  • For uploading/downloading files, you can use scp (secure copy), w3m (text web browser), or the smb files on the mac computers.

These are tasks to be done at the end of the term or at the end of other tasks as clean-up and preparation for the next time CS146 runs.

• Archive everything. Zip everything important (recursively) and add them to the ~/archives folder under both accounts:
  $ zip -r ~/archives/1201/public_html.zip ~/public_html/
  etc.
  • Include notes for assignment handmarking, including
    • Marking schemes (their rationale, and comments on their effectiveness)
    • Lists of annotations
• When assignments are complete, go through its README file and note anything that might help future ISAs.
• After a tutorial is given, note anything that might help future ISAs under the README in the tutorial summaries folder.

In the more recent years this course has been run, we have used All-or-Nothing LEARN Quizzes as part of student's grade. As of Winter 2023, LEARN has changed their quiz setup view, so here is how to set up quizzes with the new interface.

One-Time Setup:

• Import the quizzes from the previous year. This can be done by going to “Course Admin > Import/Export/Copy Components”, and searching from a previous Learn Course. Make sure you choose “Select Components” and only copy the quizzes.
• Create a grading scheme in “Grades > Schemes > New Scheme”, to set up the All-or-Nothing Grading Scheme. The scheme should look like this:

• Symbol, Start %, Colour, Assigned Value Not Perfect, 0, Red, 0 Perfect!, 100, Green, 100 

• Make sure that All-or-Nothing scheme is the default grading scheme.
• On the Grades page, delete all the default items by selecting “More Actions > Delete”
• Create a category called Quizzes (“New > Category”), set its weight to the weight of the quizzes (5% as of Winter 2023)
• In the category:
  • Select “Distribute weight evenly across all items”
  • Select “Override Display options for this item”
  • Ensure that “Points Grade” and “Weighted Grade” are both unchecked, while “Grade scheme symbol” and “Grade scheme colour” are both checked.

Per Quiz Setup:

• Change “Not in Gradebook” to “Edit or Link to Existing”, settings are:
  • Grade Category = Quizzes
  • Grade Scheme = All-or-Nothing
• Change Dates to match the term
  • Quizzes should start Mondays @ 12PM, end at Fridays @ 10PM
• Go to the grades page, select the newly created quiz item
  • Override display options for both managerial view and student view to be the same as the settings mentioned earlier
• In “Evaluation and Feedback > Customize Quiz Results Displays”, ensure the Additional view’s effect date is correct -> This wont update unless you physically click the “Update” button so be careful (closing the window or pressing OK wont change this).
• In “Evaluation and Feedback”, ensure that:
  • Auto-publish attempt results immediately upon completion is checked
  • Synchronize to grade book on publish is checked
  • Attempt Grade is checked
  • “No questions” is selected
• In “Timing and Display”, select “Allow Hints”
• You can preview the quiz to see if the settings are correct. On submission, you should not see any questions, and only the Perfect/Not Perfect grade.

WINTER 2017 COMMENTS

1. I asked Nick from CSCF to make some changes to the marmoset configuration to cs146. I'm not sure if this will persist through to future terms. These changes are: have the result table show these columns in this order: name, type, points, outcome, long result; show each row, even if the test passed. I recommend keeping these changes.

2. Some testing computers are faster than others. All the assignment tests in cs146 are timed, so some submissions may pass or fail depending on which server it ends up being tested on. The different machines are listed on the cscf linux server webpage. Ask Nick to put all the marmoset testing servers on computers with the same speeds, if possible, if not already so.

Where to Find Things

These are useful locations on the accounts on

• The course account
  • From your account on the CSCF server,
    $ ssh cs146@localhost (or $ ssh cs146@linux.student.cs.uwaterloo.ca)
  • If prompted for a password, you have not been authorized yet; contact your ISC asap.
  • You should have access to a pseudocommand called README, which will let you read the README file in each directory.
• The testing account
  • From the cs146 account,
    $ ssh cs146t (or $ ssh cs146t@linux.student.cs.uwaterloo.ca)
  • You should have access to a pseudocommand called README, which will let you read the README file in each directory.
• Source pages of the course website
  • /u/cs146/public_html/
  • There will be a number of *.shtml files, which contain the HTML source of the course website.
• Source pages under handouts
  • /u/cs146/public_html/handouts/*/
  • You need to update tutorial summaries; the instructor will update assignments.
• Test setups for assignments
  • /u/cs146t/marmoset/assignments/*/
  • ./sol/ contains the solution file (if there are multiple, make an archive folder and put everything but the correct solution into that folder or the test script will screw up)
  • ~/marmoset/utilities/ contails two scripts:
    • mkstubs: reads the ~/marmoset/assignments/ folder and creates dynamic test stubs in the ~/marmoset/stubs/ directory.
    • testCheck.sh: simulates the dynamic test setup for a single assignment using the solution file in the test directory.
  • See Marmoset or search the README files for details about revising test setups.
• MOSS
  • /u/cs146/handin/
  • Download submissions from Marmoset, copy it into this folder, and unzip.
  • ./Scripts/auto_moss.bash will run MOSS on submissions in parallel and produce the results for you.
• Handbacks
  • /u/cs146/handbacks/
  • ./cutEnd.sh truncates folder names in an assignment to just the name of the student (you need to do this for the handbacks to work)
  • ./handbacks.sh will create/update the relevant folders and grant them the relevant access for an entire assignment.
  • You should probably use Markus for assignments that will be fine with a raw grade.
• Markus
  • /u/cs146/markus/ has scripts
  • For direct interfacing with MarkUs, use the scripts detailed in MarkUsScripts.
• Midterm Practice
  • You can find a PDF containing midterm practice questions under /u/cs146/tutors/midtermPractice.tex