The responsibilities of a CS135 Tutor may vary over academic terms depending on the number of students and instructor preferences. The general weekly responsibilities will include preparing marking rubrics, holding marking meetings, office hours, tutorials, posting solutions and running moss. It is important to attend lectures and prepare your own assignment solutions in order to maximize your abilities over the course material and potential to help all students. If you are not involved in tutorials, it is still useful to attend one of the tutorial sections.

There is some information available at CompSci135TutorDutiesOld that is currently outdated, but may become relevant in future terms. Mainly, it consists of pre-MarkUs assignment information.

)

The responsibilities of all tutors can be found on the ISG webpage.

See WebTemplate for details on formatting and keeping the course webpage up to date.

## Course Account Structure

It is strongly recommended that you get familiar with the course account structure early in the term before things get hectic. Here are some of the major directories you should know about.

### Marking Directory

Under the ~/marking/ directory, there are directories for each assignment

~/marking/${assignment} (Dec/15 update) Under each assignment directory, there are two major directories. • test.pt: contains test suite for basic tests • test.0: contains test suite for correctness tests There may also exist folders such as test.anyname that ISAs may have created to experiment before changing the .pt or .0 folders. Also, you may see test.0.[<timestamp>/<username>/AUTOTESTRESULTS] folders which are created when rst is run. (pre Dec/15: Under each assignment directory, there are three major directories. • solutions: The folder is needed for the ISG script rst to run properly, but the contents of the folder do not affect the scripts. • testm.tc: contains scripts for generating Public Tests • testp.0: contains scripts for generating the actual test results There may also exist folders such as testp.anyname that tutors may have created to experiment before changing the .tc or .0 folders. Also, you may see testp.0.timestamp folders which are created when rst is run. ~/marking/${assignment}/solutions

This folder is usually used to store the solutions to each assignment in a convenient location, but as mentioned above, the contents are not ever used by any of our scripts.

For a more detailed description on the other folders and files in the marking directory, see the Marking Suite Usage PDF located in ./suite-docs.

In Fall 2008, there were two extra scripts created to assist in the automarking process, which will not be described in the PDF. These scripts are genRST and greedyRST. We will describe these in the Assignment Preparation section of this document.

)

### Student Submission Directory

~/handin/\${assignment}

When students submit on MarkUs, their submitted files get placed in a directory named for their username that gets created automatically. If a student having a username abc submits assignment a03, the submitted files are saved under the ~/handin/a03/abc directory. Note that for running practice tests when constructing the automarking tests, it may be beneficial to construct a test student in the directory. When choosing a name, often using a number as the first character is a good idea, as it will keep things organized, and can be filtered out easily when using RST. You can also use your MarkUs student account [username]-student, which you should set up.

(pre Dec/15:

When students use the submit command, their submitted files get placed under a directory of their username that gets created automatically. If a student having a username abc submits assignment 3, the submitted files are saved under the ~/handin/3/abc directory. Note that for running practice tests when constructing the automarking tests, it may be beneficial to construct a test student in the directory. When choosing a name, often using a number as the first character is a good idea, as it will keep things organized, and can be filtered out easily when using genRST

)

### Snapshot Directory

If you have mistakenly deleted any important files, don't panic. The deleted files can be recovered from ~cs135/.snapshot directory. This directory keeps copies of old contents of the course account. In order to recover a file, copy the file from this ~cs135/.snapshot directory to the correct location.

### Public_html Directory

The public_html directory contains all the contents for the course webpage. Here are a few specifics that may be helpful

Course Pages - these should all be put into the main public_html directory (ie. public_html/assignments.shtml)

• assign_info.shtml: General assignment info - this is a standard page that likely will not need to be updated
• assignments.shtml: Assignments page - will be updated weekly to post new assignments; ensure dates are correct for the given term
• calendar.shtml: Calendar page for the course - will need to be updated once at the beginning of term
• designRecipe.shtml: temporary solution the missing information from the style guide - will not need to be updated until the style guide is updated
• emailAndPiazza.shtml: Contains the communication information for the course - this will not need to be updated
• enhancements.shtml: Contains the enhancements for the course, you shouldn't need to update this page - read it over and check to make sure it is current
• exam.shtml: Contains exam information - you will need to update the dates at the beginning of term and add information about a week or two before each exam
• grading.shtml: Contains the grading and marks information - You should only need to update this once at the beginning of the term
• handout.shtml: Contains the handouts and errata page - this is currently standardized between terms and will not need to be changed
• images/: Contains most if not all the images for the course webpage. Pictures of course staff go in this folder and are then linked on the course webpage. Other images used in webpages are also in this folder
• index.shtml: Main page for the course - will need to be updated at the beginning of term
• MarkUshelp.shtml: Markus information page - check and modify at the beginning of term
• personnel.shtml: Contains information for the course personnel. Will need to be updated at the beginning of term
• philosophy.shtml: Contains course philosophy. Check at the beginning of every term, but it will likely not need to be updated
• postmorts.shtml: Contains the postmortems for the assignments and midterms. Will need to be updated weekly.
• prospective.shtml: Page intended for prospective students. Check to make sure it is up to date at the beginning of term.
• requestPublicTest.html: Not used when using MarkUs
• toc.shtml: Contains the table of contents on the left side bar. Should only need to edit if you rename a page or create a new page.
• tutorial.shtml: Contains the weekly tutorials. Will need to be updated weekly either before or after tutorial
• TutorOfficeHours.shtml: Contains a table of the weekly tutor office hours. Should only need to be updated at the beginning of term and during exam period.
Style Pages - Not to be updated unless otherwise noted. These should all be placed into the main public_html folder (i.e. public_html/bottom.shtml)

• bottom.shtml: Contains the bottom of webpages
• lastModifiedFooter.php/shtml: Used to display the last modified date for each webpage
• redirect.html: used to redirect
• standard_fini.php/shtml: Used to display the standard finish to each webpage
• standard_init.php/shtml: A standard initializer for each page
• standard_style.shtml: Standard style for each webpage
• style.css: The style file for each webpage
• template.shtml: A template for each webpage (with standard init and fini)
Folders - standard folders that should be in the public_html directory (ie. public_html/exams/...)

• announce/: The announcement folder for the course (including the announce script)
• assns/: Where to store the assignments to be posted to the course webpage
• exams/: Where to post midterm and final exam information, including the first page if posted on the exams page
• handouts/: All of the course slides for each module, as well as for the full course in both 1up and 3up format
• postmortems/: Where to post each postmortem after each assignment has been marked
• protect/: This folder has the watermarking scripts for the solutions. It also holds requestAssgnSolnCommon.php, which is where you will post each assignment's solutions
• stepping/: All of the stepping questions used for assignments. Should not be edited unless creating new stepping questions. Posting steppers is found at stepping/questions/problem-list.php
• tutorials/: Holds the tutorial slides for each tutorial
Webpage Templates - Templates to be used at the beginning and end of each term, with blank copies of webpages.
--Main.RyanCampagnola - 2013-12-05

## Assignment Preparation

Please see the MarkUs page for more instructions. If the course is not using MarkUs, visit CompSci135TutorDutiesOld for relevant information.

### Finalize the Questions

An instructor is in charge of making up questions for a written assignment. Work on a solution set as the instructor should circulate a draft version of the assignment. Try to make constructive suggestions.

### Prepare for Testing

(Apr/29 update)

This is a short summary of what needs to be done for Testing. If you need more details see the TestingGuide.

If assignment questions are re-used, you may be able to reuse tests from the archives. Be sure to update them appropriately, as older ones may be set up to run on older servers.

Basic Tests: Design a test suite with a simple test case for each function, and to test that structures are correctly defined. The test-connect line in config.ss should be commented out for the new ubuntu1404 servers. ( test.pt in the assignment's marking directory)

Correctness Tests: Design a set of test cases ranging from trivial to tricky tests, testing as many areas as possible. One test case should only test for only one specific area. Correctness tests do not have to be complete before the assignment is posted, only basic tests. ( test.0 in the assignment's marking directory)

If you want to see examples of past tests look through the archives for examples of how the test cases work. You can copy the directory over as a starting point (but be sure they follow formatting of the most recent term). For basic tests, be sure to update the header file if needed. For stepper questions, be sure to update the due date in the test.exe file (this is a text file, not a binary). I always added 10 minutes to the due time as a buffer.

The test files go into the in subdirectory. Provided files go in the provided subdirectory.

In the options files (in the appropriate subdirectories to which they apply) you need to specify the language, default timeout, and the name of the student file. This is also where you load provided files/modules. Be sure to list all necessary modules in a single module option to avoid conflicts.

Provided files are written in full Racket, while Racket code in the test files should be in the language of the assignment. See the TestingGuide if you wish to provide test constants or extra helper functions.

You can use ceconv to convert Racket check-expects into test directories that can be added to the in subdirectory.

Use the disallowed.rkt file to automate the disallowance of restricted functions. This must be the LAST file listed in the modules option. You must comment out allowed functions as necessary. Note, this will only cause failure if that portion of the student's code is run, so basic tests cannot guarantee that a student has not used a disallowed function somewhere in their code. This also means that only individual correctness tests that rely on those portions of the student's code will fail.

Be sure to test your tests before they go live!

(pre Apr/29: Prepare for Correctness Testing

Design a set of test cases ranging from trivial to tricky tests, testing as many areas as possible. One test case should only test for only one specific area. Make sure the total of the test values add up to the amount of correctness marks specified by the instructor. If you want to see examples of how it was done in the past look through the archives (starting 2006) for examples of how the test cases work.

)

### Set up the Assignment on MarkUs

See MarkUs instructions.

### Post the Specifications

In terms of maintaining the website as a whole, there will be some areas that tutors are responsible for updating (such as the tutorials page and the Tutor Office Hours page). (outdated: Generally the instructors take care of posting the specifications but) make sure you verify what the instructors are going to manage.

### Prepare a Solution Set and a Marking Scheme

Attempt the assignment and think about the different ways that students can approach the given problems. Ask other ISA's for their own approach and try to make the marking scheme flexible for all types of solutions. You should complete a draft of the marking scheme at the very latest a day before the assignment is due so that you can send it to the instructor in charge of the assignment for feedback. Depending on the term, you may not need to send it to the instructor for feedback. Talk to the instructors and your ISC about what your responsibilities are for working on the marking schemes on the assignment.

It is useful to provide a detailed breakdown of mark assignment on the rubric. Next to each mark should be an explanation of when the marks are to be given or withheld. When sending the marking scheme to markers, make sure that you send the official solutions as well. Further marking details can include expected common mistakes (although markers should watch out for those too) or possible variations in correct answers may also be provided with solutions (such as variations on contracts).

### Collecting Assignments and Running the Marking Scripts

(Dec 19, 2019 update)

The main goal for assignment collection is to:

- Isolate the assignments that students submit before the due date from the ones that are submitted after the due date. Students can still submit their assignment after the due date, but we do not want to run our marking scripts on those submissions.

- Run our marking scripts on the assignments and generate a GRADED_ASSIGNMENT.ss file for each student.

- Run the automated grading scripts for correctness tests/check-testcases

Contained in each GRADED_ASSIGNMENT.ss file is each student's correctness test result, their test coverage results, and all of the files that they've submitted. The GRADED_ASSIGNMENT.ss file makes it really easy for TAs to grade each assignment, and it is a convenient place for students to receive their feedback. Generating this file will be the bulk of what we'll need to do during the assignment collection process.

The assignment collection process can be done anytime after the assignment is due, but you'll want to finish it before your first marking meeting so that TAs can get started on their marking load as soon as possible. Be sure to ask your ISC if you need any clarifications on the steps below!

Step 1:

Log onto MarkUs and go to the assignment that you want to collect. Go to the submissions tab for that assignment. Click on Collect All Submissions to prepare MarkUs for marking. When you click on Collect All Submissions, you'll be greeted with three different options for how you want to collect the assignments. Typically we have a 10 minute buffer window for any issues that MarkUs might have at the due date. Check with your ISC if there is a buffer window for assignment collection during your term! If there is a buffer window, you'll want to select "Mark the latest files that were submitted before: ..." and select the appropriate time (i.e. 2019-12-03 9:10 PM. This will collect all assignment submissions before or at 9:10PM on December 3rd, 2019). If there is no buffer window, then you can just select "Mark the latest files that were submitted before the due date."

Once you've chosen how you want to collect the assignments, MarkUs will begin the collection process. Wait until all rows turn green. Once all rows have turned green, TAs could technically start marking each assignment in MarkUs, but we still need to generate the GRADED_ASSIGNMENT.ss file to make marking easier, and to provide supplemental information to help with the marking process.

Step 2:

From this point onward we'll be working on the terminal/command line. Make sure that you have ssh'd into the course account at this point. Our next goal is to generate the GRADED_ASSIGNMENT.ss file. In order to do that, we need to take the assignments that are in MarkUs 's repository and download them into the course account. To download all student submissions that were submitted by the due date into the course account, run the following command in the terminal (you can run this command anywhere in the course account; you don't need to be in any specific directory to run it).

/u/isg/bin/markus.py download ASSN SAVE_FOLDER

Where ASSN is the assignment you're collecting and SAVE_FOLDER is the location where you want to save the submissions. Typically we save the submissions under handin/ASSN_autotest. As an example,

/u/isg/bin/markus.py download a01 ~/handin/a01_autotest/

This will download all student submissions that were submitted by the due date for a01 into the handin/a01_autotest/ directory. If you used a buffer window in step 1, you'll have to make sure you download all submissions that were submitted within that buffer window. To do that, run

/u/isg/bin/markus.py download --time 'YYYY-MM-DD HH:MM:SS' ASSN SAVE_FOLDER

This will download all student submissions before or on the specified date and time. As an example,

/u/isg/bin/markus.py download --time '2019-09-16 21:10:00' a01 ~/handin/a01_autotest/

This will download all student submissions for a01 that were submitted before or at 9:10pm on September 16, 2019 into the handin/a01_autotest/ directory.

Step 3:

Once you have ran the download command and have finished downloading all of the submissions into the specified handin directory, it's time to do some processing on each student submission. First, we're going to run our correctness test on all of the submissions. The first thing you'll need to do is create a symlink from your testing folder to a new one that has _autotest appended onto it. To do this, cd into the directory where you created your correctness tests for the assignment. This should be the marking directory (i.e. cd ~/marking/). Once you're in the marking directory, do

ln -s ASSN ASSN_autotest

Where ASSN is the assignment you're collecting. For example,

ln -s a01 a01_autotest

Will create a new folder called a01_autotest inside of the marking directory which links to the a01 folder where you created correctness tests for a01. a01_autotest is like an alias for a01, so anything you do to a01_autotest (like deleting files inside of this directory), it will also happen inside of the a01 directory. They're basically the same directory. Why do we need to do this? We need a folder called a01_autotest in the marking directory, which has the same name as our handin folder, in order for our correctness tests to run. Once you have created the symlink, it's time to run the correctness tests! To do this, run

distrst -t t -s '*' ASSN_autotest TEST_SUITE SAVED_RESULTS

Where ASSN is the assignment that you're collecting, TEST_SUITE is the type of tests you want to run (can either be pt for public test, or 0 for correctness test), and SAVED_RESULTS will be the name of the folder where your correctness test results will be saved to. For example,

distrst -t t -s '*' a01_autotest 0 AUTOTESTRESULTS

This will run the correctness tests on the submissions located in the handin/a01_autotest/ using the marking/a01_autotest directory that we created earlier, using test suite 0 (for correctness tests) and saving the results inside a folder called test.0.AUTOTESTRESULTS (we typically use AUTOTESTRESULTS as the SAVED_RESULTS name) located inside of the marking/a01_autotest directory. At this point, multiple servers will be used to run correctness tests on students, all in parallel to make things faster. This might take a while depending on the number of students that you have during your term. While this is running, you should run the next step in parellel.

Step 4:

For assignments where the design recipe is required, we typically run some scripts to test students' test coverage (how well students' check-expects test their code). If you or other ISAs have written any check-testcases scripts earlier, now is the time to run them. Go into the check-testcases directory in the course account (cd ~/check-testcases/). Once you're there, run

./run-all.sh ASSN

Where ASSN is the assignment that you're collecting. This will run the check-testcases scripts that were made earlier on each student submission in the handin/ASSN_autotest/ directory. This might take a while depending on the number of students that you have during your term. Make sure you run this in parallel with the correctness tests (step 3) to be efficient! At this point you can go and do something else until steps 3 and 4 are done (sleep, netflix, league).

Step 5:

At this point you have all of the files necessary to create a GRADED_ASSIGNMENT.ss file for each student. To do this, open up the MakeGradedAssignment.py file located in ~/markus/bin. (vi ~/markus/bin/MakeGradedAssignment.py). Once the file is open, fill in the required fields inside of the file for the assignment that you're collecting. Once that's done, save the file and run it using:

python ~/markus/bin/MakeGradedAssignment.py

This will generate the GRADED_ASSIGNMENT.ss file for each student and save the file inside of each student's directory inside of the handin/ASSN_autotest/ directory. You should pick some random students inside of the handin directory and double check that the GRADED_ASSIGNMENT.ss file looks correct. You need to catch any mistakes in the correctness test or the test coverage scripts now before you send it off for TAs to mark, otherwise it'll most likely be too late to fix.

Step 6:

Once you have confirmed that the GRADED_ASSIGNMENT.ss files look correct, you now need to upload each GRADED_ASSIGNMENT.ss file into each student's MarkUs repository so that TAs and students can see the file on the MarkUs interface. All you need to do is run

/u/isg/bin/markus.py upload_marking --filter 'GRADED_ASSIGNMENT.ss' ASSN SAVE_FOLDER

Where ASSN is the assignment you're collecting and SAVE_FOLDER is where you saved the assignment submissions in step 2. For example,

/u/isg/bin/markus.py upload_marking --filter 'GRADED_ASSIGNMENT.ss' a01 ~/handin/a01_autotest/

will upload all of the GRADED_ASSIGNMENT.ss files for each student from the handin/a01_autotest/ directory into MarkUs for a01. Once this has finished running, you should go back to MarkUs and double check that the GRADED_ASSIGNMENT.ss file is present for a couple of students.

Step 7:

At this point TAs could start marking, but we can be a bit nicer to them by automating the grading for correctness tests and coverage tests for each student. We'll first automate the correctness test marking. Go into the markus directory in the course account and go into the assignment folder that you're collecting (i.e. cd ~/markus/a01/). In this directory you can store any files that you'll be using to process things in MarkUs. One file that we'll want to create is a csv file that tells MarkUs how to assign rubric levels according to each student's correctness mark score. For example, creating a file called rubric_automarking.csv, we'll want to format this file as follows,

1,Q1: Correctness,1,3,4,5

bonus,Bonus:Correctness,,,,1

Where:

1. First column is the name of the question as it appears on the GRADED_ASSIGNMENT.ss file (i.e. Question 1: 5/5, the 1 is what you want to put into the first column)
2. Second column is the name of the criterion as it appears in MarkUs
3. Third column is the minimum mark needed to get Level 1.
4. Fourth column is the minimum mark needed to get Level 2.
5. Fifth column is the minimum mark needed to get Level 3.
6. Sixth column is the minimum mark needed to get Level 4.
Once you're done creating this file, process it with the following command.
/u/isg/bin/markus.py set_marks_rst ASSN TEST_RESULT_LOCATION rubric_converter.csv

Where ASSN is the assignment that you're collecting and TEST_RESULT_LOCATION is the location of where you saved the correctness test results in step 3. For example,

/u/isg/bin/markus.py set_marks_rst a01 ~/marking/a01_autotest/test.0.AUTOTESTRESULTS rubric_converter.csv

will start the autograding process in MarkUs for a01. Once it's done, double check on MarkUs that the students have been assigned the correct rubric level according to their correctness test results. This step can be done anytime after step 3 is complete and can be done in parellel with step 8. Make sure that you go onto MarkUs and double-check that the correct rubric levels are being assigned.

Step 8:

To automate the coverage test marking, first go to the test-cases directory for the assignment that you are collecting in the check-testcases directory. For example,

cd ~/check-testcases/a02/test-cases/

Once you're in the test-cases directory, you should see a csv file called output with a timestamp appended to it. This file was created after running step 4 and will be used to automate the grading on MarkUs. As an example, run this file by using the following command:

/u/isg/bin/markus.py set_marks_csv a02 output-20191126_22:47:51.csv

This will assign rubric levels on markus for all of the test coverage criterions. This step can be done anytime after step 4 is complete and can be done in parallel with step 7. Make sure that you go onto MarkUs and double-check that the correct rubric levels are being assigned.

At this point you should be completely done the assignment collection process and TAs can now begin to grade assignments!

(Apr/29 update) Outdated! Use if necessary!

Follow steps 1-9 of the autotesting procedure: MarkUsGroupsSVNRepos#Auto_testing_Procedure. MarkUs scripts are in ~/markus/bin/.

Step 3: I always added 10 minutes to the due time as a buffer. Use test.0.AUTOTESTRESULTS.

Step 5: It's up to the instructor if they want you to use WxmeConverter. You might use it for the first few assignments. I didn't use it at all in my second term, but make sure students know to not introduce non-text into their files. Also, basic tests should catch this: stress the importance of checking basic tests and not leaving submissions until last minute! Moss can be run later. See following about the check-testcases script.

Step 6: Remember, we're using test suite 0, not 1.

You shouldn't need Step 10, since the checkout script should check out versions that are on time. Just be sure to mark submissions in ~/handin/aXX_autotest , where the checkout occurs. Those in ~/handin/aXX could be late.

check-testcases: You can use /u/cs135/archives/1161/assns/a09/tests/check-testcases/ as a starting point. This version will count up the number of examples and tests (number of tests INCLUDES examples). The run.sh file (run this after setting the assignment specifics to run the tests) is also easier to customize. With this version, you need 2 files for each student file you want to test: aXX[filename].rkt and aXX[filename]-base.rkt. Edit the assignment specifics in each as appropriate.

(pre Apr/29:

After everything is ready to go, run the following command,

~isg/bin/rst 3 p 0 �regex� timestamp

The option �p� stands for �print�. The �0� (zero) goes with the zero in testp.0.

�regex� should be replaced by a regular expression which will match all user IDs of the students whose submissions you want to run the tests against. For example, if you wish to run the marking script for all students with their userID starting with �a� through �c�, use the following command:

~isg/bin/rst 3 p 0 �[a-c]*� testRun

The timestamp can be anything and will be become part of the name of the folder which will store the outputs. Note that running rst twice with the same timestamp will keep results under the same directory. For example,

   ~isg/bin/rst 3 p 0 �nthibode� tutor
~isg/bin/rst 3 p 0 �zhli� tutor


will create folders named nthibode and zhli under marking/3/testp.0.tutor. These folders will not be overwritten if the same timestamps are used, you have to delete the old outputs if you want to run the tests against the same student again.

It is a good idea to test run the marking scripts for a few students/tutors before the assignment is due, just in case an unexpected error comes up. As soon as the assignment is due, execute the following command,

~isg/bin/rst 3 p 0 �*� firstRun New Note: Whether using the new scripts or running rst the �old fashioned way,� occasionally rst will segfault for no good reason (if anyone ever figures out why, you are my hero). This means that the student�s output will not have had all the tests run properly on him/her. To try and find any of these segfault cases, the easiest method is to use the command grep.

Navigate your way to the folder testp.0.timestamp and run the following command

grep �/0   Automarking� ./*/Autotesting

This will search every student�s autotesting output and look for any which contain the phrase written above. This is the list of student who rst will have to be rerun on. There are two things that need to be done before this. First, a file called MARKS will have been created in testp.0.timestamp. Open this file in a text editor and remove any of these students from the file (find is a very handy tool). Second, remove the student�s entire folder from testp.0.timestamp. Then you can rerun rst on the student.

)

(outdated: Send print job to Graphics Services

(note: this information is also included in the marking suite usage PDF)

After the auto-marking is complete, the assignments must be printed out for the TAs to grade the documentation and design. The following command,

~isg/bin/printOut 3 0 firstRun tut �g

will send a print job to Graphics Services that contains all Assignment 3 output files found in the ~/marking/3/testp.0.firstRun directory. The tut specifies how the assignments will be sorted. Here the assignments will be sorted into piles by tutorial sections. You could change tut to N to have the assignments in only N piles (where N is some number). The �g specifies that the assignments will be printed to Graphics Services. If instead you had a small pile to print, you could change this to �7, and instead the assignments would be printed to lp7 in DC3116. (Note: Do not print to DC3116 unless it is a very small number of assignments)

Make sure that you test the command before sending the output to graphics. The -o option will output to the screen the list of postscript files that are to be printed:

~isg/bin/printOut 3 0 firstRun tut �o

Make sure to allow at least 24 hours for the job to finish and for the papers to be delivered to the DC Copy Centre. You should be notified by email when the print jobs are completed.

)

### Collect Assignments and Run Marking Meeting

Send out emails to TAs to remind them to get familiar with the assignment and ready for the meeting. Keep a record of each TA�s attendance and a brief comment on the quality of their work for TA evaluation at the end of the term.

Bring copies of the assignment and marking scheme to the meeting. Encourage the TAs to ask questions during the meeting, ask for their opinion on certain problems. Correct any views that are inconsistent with the CS 135 curriculum. It is also a good idea to have one or two assignments already marked to use as an example of different issues that may come up.

### Post Solutions

Solutions are available to students using Solution Request described below. The solutions are watermarked with their QuestID and are meant for their personal use only.

### Hand Assignments Back

The assignments are returned releasing the marks on MarkUs. It is advisable to release the previous marks and annotations before the next assignment is due. In Fall 2011, because our assignments were due on Monday night, we generally handed back our assignments by 10:00 am (our deadline was 8:00 am) on Monday. This gave the graders a weekend to grade and the students a chance to review their submissions.

### Running Moss

Moss is used to autodetect potential plagiarism cases. The scripts to run Moss can be found in /u/cs135/moss/bin. This program should be run, and then the output HTML file can be analyzed by hand to determine which cases seem likely and which appear to be false positives. Any suspicious pairs/groups should have particularly suspicious code portions highlighted, and then the assignments should be passed to the Instructional Support Coordinator.

## Rollbacks

You can use the following procedure for rollbacks (when a student wants an earlier submission marked):

In the ~/handin/aXX_autotest/ directory, rename the student's folder to have _prerollback appended. In ~/check-testcases/aXX/test-results/, append _prerollback to both of the student's results files. In ~/marking/aXX/test.0.AUTOTESTRESULTS/, rename the student's folder in the same way.

Do the following MODIFIED versions of steps 4-8 of the autotesting procedure MarkUsGroupsSVNRepos#Auto_testing_Procedure (MarkUs scripts are in ~/markus/bin/):

Step 4: Run only the viewcmds option to get a sample checkout command, and modify it for the correct student (you also need the student's group number from MarkUs), to checkout the correct revision (which you can determine on Markus in the student's group on the Submissions tab; the student's basic test email will also include the revision number). If you are not rolling back all of the student's files, then copy the files you are not rolling back from the _prerollback directory.

Step 5: Make sure original results are backed up. Re-run the appropriate scripts. For check-testcases, be sure to copy the run.sh file and modify the copy so it only runs for the rollback student(s).

Step 6: Re-run RST, specifying the student(s) with the -s option.

Step 7: Run python MakeGradedAssignment.py - (note the dash). You should see instructions on how to run the script on an individual student. DO NOT press 'y' to continue if it asks you to run for all students. You can run it once to make sure the output on stdout is correct, then run it again redirecting to the student's handin autotest directory to a new file named GRADED_ASSIGNMENT_rollback.ss.

Step 8: Run only the viewcmds option to get sample add and commit commands, and modify them to add and commit the new GRADED_ASSIGNMENT_rollback.ss file. It will tell you the new revision number. On MarkUs, go to the student's group on the Submissions tab and select this revision number. Click "Collect and Grade This Revision".

## Approximate Breakdown of Work for CS 135

• 5 Hours of office hours
• 6 Hours of lab supervision
• 4 Hours of tutorials
• 3 Hours of lectures
• 0.5 Hours of marking meeting
• 2 Hours of reviewing next assignment
• 8 Hours for assignment marking
• 4 Hours for answering emails
• Miscellaneous tasks like assignment remarks, Coffee chats, writing midterm/final, etc.

## Tutorials

### Initial Preparations

To prepare tutorials, it is a good idea to use tutorials from previous terms as a starting point. Often these will contain questions that you can use. In addition, you will want to look at the assignment for the next week and see of the current tutorial relates or if it contains any questions (and thus solutions) that are in the assignment (this can happen, especially when you are dealing with topics like trees). It is also very helpful to ask the professor(s) if they have any ideas for the upcoming tutorial or if there is anything they would like to cover. Since I used the past tutorials as a base, I wrote down which of the tutorial questions should be removed and those that should be added.

### Writing Tutorials

During Fall 2011, I would start with the material from one of the previous terms and remove some material and add more material. If you are starting with tutorials created prior to Fall 2011, you will notice they are of a different format than those during/after that term. I would suggest using the format of Fall 2011 since it is more consistent with the format of the lecture slides.

First of all I updated the tutorial (changed the date etc.). Then I would comment out the unneeded questions (the questions I wished to remove). This allows tutors in the future to see questions from previous terms even if they were not used in the previous term (although this could lead to excessively long text files). I would then add questions to the tutorial.

As already mentioned. Taking material from past terms is often helpful, but there are a few things you will want to watch:

• Make sure that the style is consistent with what is being taught that term. There may be slight changes in the style guide between terms
• Make sure the references to the lecture slides are up to date. These can change between terms so make sure they are the same or you will have confused students.
• Always double check the solution and everything else on the slide. Do not just assume that is is right.

• Wikibooks - LaTeX
• For general questions use google, there are a lot of forums that most probably have an answer to your question
Much of the information below can be gleaned from looking over the Fall 2011 slides.

Included in the Fall 2011 tutorial directories will be all the required files to make the tutorial slides (with answers) as well as the slides that will be posted to the course website (slides without answers). To make the tutorial, type the make command into terminal. You should ensure the following files are in the directory with .tex file:

• 135slides.cls
• pause.sty
• Makefile
• pp4p.jar
• schemesymbols.tex
• slatex.sty
If you do not reuse the tutorials from previous years you will want to copy the <noautolink>LaTeX</noautolink> preamble from previous terms.

-- HumaZafar - 10 Sep 2012

It would be more convenient for you, if you convert your latex files to pdfs using the command make from the course account directory where you should have your make [0-1][1-9]-*-slides.tex and the six files mentioned above. You need to name your files in this particular way for the Makefile script containing the make command to run on them. If you are curious about the make command, read Makefile using a text editor. This command will automatically generate the following five pdf files:

• *-show.pdf: This is the version you need to send to the IA and use in your own tutorial
• *-post.pdf: This version contains all the slides except the solution slides. This is the file that we post on the course webpage.
• *-post3up.pdf: This is same as the *-post.pdf except it is in 3-up.
• *-click.pdf: This version contains only the clicker questions.
• *-prof-work.tex: You don't need to use this pdf for the tutorials.
For the make command to successfully filter out the solution slides, remember to use \begin{slides} and \end{slides} to for all your question slides including the clicker questions. Use \begin{clicker} and \end{clicker} for all your solution slides including the clicker slides.

-- XixiHu - 21 Feb 2013

A slide environment begins with the \begin{slide} command and the solution slides environment begin with the \begin{clicker} command. The titles for each slide are as follows: {\Large\textbf{Title Here}}. It is also helpful to change the font size for the rest slide to \small. If you cannot fit all of the solution on one slide you can go to two slides or use a smaller font size ( footnotesize, scriptsize, and tiny). Instead of using \begin{verbatim} environment or \texttt{} command to display scheme code, use one of the following"

• \begin{schemedisplay}: I used this most frequently out of any of the commands. Great for showing blocks of code and it places a newline character before and after the block of code. It gives more space to blank lines in your code segments.
• \scheme{}: I generally used this for short segments of code such as function names or answers to multiple choice questions. This is intended for one line of code or less.
• \begin{schemebox}: Similar to schemedisplay except it does not have a newline character before or after and the spacing in the text is smaller
For the list structure, use \begin{alphalist} (with item for the list items and enumi as the counter).

When in one of the aforementioned 'scheme' modes, you can use the follow special characters. \to produces a right arrow similar to -> and yields produces a double arrow =>.

### Clickers

I found clickers to be a good learning tool in tutorial, although the lack of marks assigned to clickers in tutorials decreased students' incentive to bring their clickers.

To use clickers, you will need to ensure the clicker software is downloaded from i>clicker Instructor downloads page onto the computer you intend to present from (whether your course account - using the podium computer, or a laptop). Also make sure that you download version 5.5 (I can't remember why, but one of the profs told us to make sure we had the correct version).

It is recommended that you have a separate 'course' set up for each section. To create a new 'course' click "Add Course" and input information. To reduce confusion, especially if you teach multiple tutorials, you could input the time of the tutorial in the Course number field. To set up the details for the course select it and select choose. In tutorial you will click "Start Session" which will open the small white clicker application box but for now click on "Your Settings and Preferences" which will open another window. In the first tab (General): Input your clicker ID (the eight digit hex code on the back of the clicker) where it says "Instructor's Remote ID", Set the frequency for the room that you are doing the tutorial in and change the alert message to "Not at all" (This is a pop up that will appear every time you ask a question). The default settings in the "Registration/Export" and "Question and Polling tabs are fine (You may want to double check that your Polling timer under "Question and Polling" is set to count up). Under "Scoring" change the session participation marks to 0, and under performance change both "Points for Responding" and "Points for Correct Response" to 1. And ensure that there are no limits to the total performance points earned in a session. When you are done click "Set for this Course"

Don't forget to remind your students of the frequency for at least the first few classes.

## Solution Request

The preferred mechanism for releasing solutions, old exams, and other information that shouldn't be shared (even though it probably will) is via watermarked PDF files. Steps for adding a new document:

• Add a new directory in /u/cs135/protectPDF
• Place a pdf of the file to be served in the directory. The file must have the same name as the directory except that it ends in '.pdf'.
• Check the permissions on both the directory and the pdf to make sure they aren't broadly readable.
• Edit /u/cs135/public_html/protect/requestAssgnSolnCommon.php to include the new document.
• Redisplay the request page and check that the new document is available and correctly delivered. There is extra processing that happens the first time (splitting the PDF up into images, one image per page) so it takes even longer than usual.
To create a PDF for the individual assignments, I opened the file in Racket then I went to the print screen. In the bottom left corner of the print screen there is a button that says PDF. If you click it and select save as PDF, you can easily save the solution as a PDF and it will also have the Racket colouring. Then follow the steps above.

Notes:

• The process relies on a program named <noautolink>ImageMagick</noautolink> (free). The installation on the solaris machines is out of date, so we ssh to a linux machine to do the actual work.
• The system tries to clean up temporary files, etc. after itself but if students leave the page before the work is done, stuff may get left behind. Clean up the cs135/protectPDF directory periodically. Everything with a long random name can go.
• The script has recently been updated to be able to handle PDFs up to 9999 pages long, making it suitable for releasing exam review slides. However, doing so may overtax the system and cause errors if many students are requesting at the same time.
• In Fall 2014, the system was overwhelmed by students requesting in excess of 4500 review pdfs (120 slides) in a period of several hours. There has since been a 5 minute request cooldown implemented to prevent this.
-- HumaZafar - 10 Sep 2012
Topic attachments
I Attachment History Action Size Date Who Comment
rkt disallowed.rkt r1 manage 1.4 K 2016-04-29 - 02:01 LeonardZgrablic Disallows restricted functions
txt webpage_template_guide.txt r2 r1 manage 4.1 K 2013-12-05 - 13:10 RyanCampagnola
Topic revision: r18 - 2019-12-20 - JimmyPham
• ISG Web

• Webs