Resources and Information for CS 241e ISAs

This TWiki is for technical details only. Any other duty information can be found in Waterloo LEARN (D2L): CS Instructional Support Group -> Course Admin & Duties for ISAs.

Introduction

General Notes and Warnings

CS241e is an enriched version of CS241, but it's quite different from the standard version of the course. This TWiki focuses on the technical side of being an ISA, but if you haven't taken CS241e before, be sure to keep up with the material and stay ahead of students.

Since 2015, CS241e has been consistently taught by one instructor, Ondřej Lhoták. Much of the infrastructure for the course was created by Ondřej and he is always updating and improving it between terms. As such, the information on this TWiki can easily become outdated! IMPORTANT: before running any of the scripts described on this TWiki page, you should read the source code and check that they actually do what you expect. If you notice any incorrect information on the TWiki, please update it.

Ondřej is familiar with Marmoset, and usually he will be the one responsible for creating and maintaining the Marmoset test setups, canonical solutions, and libraries. The ISA will generally not need to do much Marmoset setup. This TWiki page attempts to document all aspects of Marmoset setup for completeness, but this documentation could quickly become incorrect if Ondřej changes his workflow. In general, if there is an issue with the Marmoset scripts, notify Ondřej about it rather than trying to follow the methods on this page.

Updating The TWiki

• If any information you read on the TWiki during this term was inaccurate or out of date, please update it.
• If any of the explanations of how to do certain tasks were unclear, please rewrite them.
• If there are any tasks requiring technical knowledge that you had to do throughout the term, and they aren't documented on this page, please add them.
• If you think of a better way to organize the CS 241E section of the TWiki than having most of the information on this giant page, feel free to reorganize it.
• If you made any modifications to any of the existing scripts documented on this page, please update the documentation on this page.
• If you wrote any new scripts or tools for future ISAs to use, please add documentation for it on the TWiki explaining what they are and how to use them.
• If you think of anything else that should be on this list of things to update, please add it.

“Beginning of Term” Tasks

Note: some of these tasks (such as setting up Piazza, some of the course website updates, updating the assignment code viewer, and creating Marmoset projects) may have already been done by the instructor or previous course staff before you arrived. Information about them is kept here for completeness's sake.

1: Access the Course Email

The course email is cs241e@uwaterloo.ca. See the AccessingCourseM365Email page to learn how to access it.

2: Access the Course Account

See the AccessingCourseAccount page to learn how to access the cs241e course account.

Note: you'll primarily be accessing the account via SSH during the term.

3: Clone the Course Git Repository

Note: if you are unfamiliar with Git, see A Brief Overview of Git in this page's Appendix first.

The course Git repository stores course material, which includes:

• Assignment code
• The course website
• Lecture notes and past tutorial material

It is up to you if you want to clone the course Git repository to your own machine or to the school's Linux environment (or both). There are scripts contained within the repository and documented on this TWiki page that require prerequisite installations (e.g., sbt) in order to run. These prerequisite installations should already be on the school's Linux environment. However, it is unlikely you will need to run these scripts yourself, so it may be more convenient to have the repository stored locally.

To clone the course Git repository:

1. Ensure your public SSH key is in ~/.ssh/authorized_keys in the course account.
2. Run the following command on your own computer or account on the school's Linux environment.
   

   git clone cs241e@linux.student.cs.uwaterloo.ca:private/cs241e.git

Once you have cloned the repository, read The Course Git Repository and its subsections for more information about its contents.

4: Create the Course Staff File

See the Create Course Staff File For Current Term section on the CompSci241ISADuties page.

5: Update Term Variables in MOSS Scripts

In the course account, there is a script called moss241e.sh in the ~/moss/Scripts/ directory which is used to run MOSS on student submissions.

Read the documentation for the TERM#NUM variables within the script and update them accordingly.

The scripts marm_sql, archive, and moss+archive+update in the ~/bin/ directory are also used to run MOSS on student submissions. Update the CURRENT_TERM variables in archive and moss+archive+update and lines 21 and 23 in marm_sql.

6: Set Up Piazza

See the Set Up Piazza section on the CompSci241ISADuties page.

7: Update the Course Website

The course website is located at https://student.cs.uwaterloo.ca/~cs241e/.

To edit the course website, edit the files in the ~/web/ directory of the course Git repository. Pushed changes will get copied automatically to the ~/public_html/ directory in the course account, where the website is built from. Note: do not edit website files in the course account directly – they will get overwritten after a Git push.

Things to update on the course website:

• All mentions of the term and year of the current course offering (e.g., “Fall 2022”).
• All links to the course Piazza.
• Course personnel information at personnel.shtml. Leave office hours as “TBA” if not yet known.
• Midterm and final exam information at exam.shtml. Leave final exam information as “TBA.” Comment out extraneous information (e.g., past exam review sessions).
• Remove all tutorial links at tutorialNotes.shtml.
• Remove all assignment links at assignmentpost.shtml and update their due dates.
• The !IntelliJ IDEA setup page at idea.shtml. Check the required JDK version, latest IDEA version, and update the pictures if the IDEA interface has changed a lot.
• The assignment info page at assignment.shtml. Check all information mentioned above and the path for Marmoset submissions (will change if Scala version has changed).

8: Update the Course Google Calendar

The course website displays an embedded Google Calendar. Keep it up-to-date throughout the term. It should contain:

• Lecture times
• Tutorial times
• Office hours
• Assignment deadlines
• Exam times

To update the calendar:

1. Visit the course website and click the Google Calendar logo in the bottom right of the calendar.
   
2. Log into the course Google account.
   • Use the course email (cs241e@uwaterloo.ca) to log in. The password is in ~/README.txt in the course account. Do not post the password on the TWiki.
3. Add all the events to the Google Calendar.
   • If you aren't familiar with using Google Calendar, view the help page here. The main thing to know is how to create "repeating events" that reoccur every week, which is helpful for adding lectures, tutorials, and office hours. Make sure that the "end date" is set appropriately.

9: Update the Assignment Code HTML Viewer

Assignment code can be viewed in HTML on the course website at https://www.student.cs.uwaterloo.ca/~cs241e/current/scalasrc/A1.html.

These website pages need to be updated if there is a new version of the assignment code to be used that term.

To update the assignment code HTML viewer:

1. In the course Git repository, generate the “problem source code” by running the following command.

   sbt clean problems

2. Navigate to ~/scripts/ and run the following command.

   ./mkcodehtml

10: Add Users to Marmoset

Note: before you can set up Marmoset, CSCF has to create the Marmoset instance for the course and add you with instructor privileges.

See the Adding users sections on the MarmosetSummary page. If you are asked whether or not to limit user ID length, choose to limit it to 8 characters in length.

Note: for the first few weeks of the term, you may have students who enroll into the course late. Watch for emails to the course email account from Waterloo Odyssey regarding changes to the classlist, and add students to Marmoset accordingly.

11: Create Marmoset Projects

In CS241e, there is one Marmoset project for each assignment.

To create a project for an assignment, click the "create new project" link on the course's Marmoset homepage. Then fill out the fields as follows:

• project Number and project title should be the same. Set them to "Ax", where "x" is the assignment number (e.g., "A1" for assignment 1, "A10" for assignment 10).
• on-time deadline and late deadline should be set according to the current late policy.
• URL can be set to the assignment page URL on the course website, but this is not necessary.
• description should be set blank.
• stack trace policy should be set to "the entire stack trace for Java or full output printed to stdout for C".
• canonical class account can be set to your account, but this is not necessary. (It indicates who created the project.)
• All other fields should be left with their default values.

Note: late penalties are applied through a separate script, not through Marmoset.

If you would like to know more details on how Marmoset test scripts are created, uploaded, and executed, see Setting up Marmoset Tests in this page's Appendix.

“During The Term” Tasks

Using moss+archive+update

moss+archive+update is a relatively new script (Fall 2020) that automates all of the “During The Term” tasks:

1. downloading filtered submissions and running MOSS for a given assignment;
2. updating the archives with the MOSS results;
3. updating unofficial marks on the course website; and
4. downloading original submissions into the archives.

If you use this script, you should not need to follow the instructions in the following sections – they are kept for completeness, although it would be a good idea to read them anyways to understand what exactly moss+archive+update is doing.

Note: the script does not update $moss_folder_paths in check_auth.php, so that will still need to be done manually, as described in the next section.

The script is located at ~/bin/moss+archive+update. To use it, run the following command:

moss+archive+update assignment# [-v]

Options:
-v: Enables verbose mode

For example, if you are running moss+archive+update on A1, the command would be moss+archive+update 1 [-v]

Without Using moss+archive+update

Run MOSS to Check For Plagiarism

To run MOSS for an assignment:

1. In the course account, navigate to ~/handin/. Run the following command to download student submissions from Marmoset for the given assignment.
   

   marm_sql -d Ax

   (This will download submissions to ~/handin/Ax/.)
   
   
2. Navigate to ~/moss/Scripts and run the following command.

   ./moss241e.sh Ax

   When the command finishes, the MOSS results will be in ~/moss/Ax/ and there will be a subdirectory for each of the previous terms that were compared against.
   
   
3. To make the MOSS results viewable in a web browser, edit $moss_folder_paths in check_auth.php (located in ~/web/moss/ on the course Git repository) to add the necessary redirection paths. The lines should follow the form:

'Ax' => '/u/cs241e/moss/Ax/html_sc',
'Ax_[term#1]' => '/u/cs241e/moss/Ax/html_sc_[term#1]',
'Ax_[term#2] => '/u/cs241e/moss/Ax/html_sc_[term#2]',
'Ax_[term#3]' => '/u/cs241e/moss/Ax/html_sc_[term#3]',

Update Unofficial Marks

Students may view their unofficial marks on the course website at: https://www.student.cs.uwaterloo.ca/~cs241e/cgi-bin/displayMarks.py.

The marks shown on the website are read from the file ~/marks/public-marks.txt, stored on the course account.

There is a relatively new script (Fall 2020) called update.sh located in ~/marks/scratch/ which automates the process of updating the official marks.

To update the unofficial marks using update.sh:

1. In the course account, navigate to ~/marks/scratch/. Run the following command, where X is a number rather than an assignment name (e.g., 1 vs A1). All marks from assignment 1 to X will be updated. Midterm marks will be updated if the midterm-crowdmark.csv file is present in ~/marks/scratch/.

   ./update.sh X

To update the unofficial marks without using update.sh:

1. In the course account, navigate to ~/marks/scratch/ and run the following command.

   marm_sql -v -m AX

2. If there has been a midterm and the grades are on Crowdmark, download the CSV file from Crowdmark and save it in ~/marks/scratch/ as midterm-crowdmark.csv. Then run the following command.

   ./crowdmark.sh midterm-crowdmark.csv > midterm.csv

3. Combine the marks from Steps 1 and 2 by running the following command.

   ./combine.sh > public-marks.txt

4. To post the marks to the website, run the following command.

   cp public-marks.txt ..

Note: if the course's late policy for grade calculations has changed for the current term, ~/bin/marm_sql will need to be updated to reflect the changes in how grades are computed.

Note: if the assignments change and/or the number of marks per question have changed for the current term, ~/marks/scratch/combine.sh and ~/marks/scratch/base.txt will probably need to be updated.

Note: CS 241E typically uses only public tests on Marmoset. If release tests or secret tests are ever used, be careful to wait after the late deadline to update the unofficial marks, as updating the unofficial marks prematurely can leak the results of the release/secret tests to students. (It is good practice to wait until after the deadline to update unofficial marks even without release/secret tests anyway.)

Archive MOSS Results and Original Student Submissions

Archives for each term are kept in the course account at ~/archives/. A single term's archive contains:

1. MOSS results for each assignment.
2. original student submissions for each assignment.
3. assignment pages from the course website.

It's suggested to archive MOSS results and original submissions soon after you run MOSS on a given assignment and to archive the assignment pages at the end of the course.

To archive MOSS results:

1. Copy the MOSS results stored in ~/moss/ to ~/archives/[CURRENT_TERM]/moss/.

To archive original student submissions:

1. Navigate to ~/archives/[CURRENT_TERM]/handin/.
2. Run the following command.

   marm_sql -d Ax

“End Of Term” Tasks

Archive Assignment Pages

To archive assignment pages:

1. Make a directory called assignments/ in ~/archives/[CURRENT_TERM]/, navigate to it, and run the following command.

   cp ~/public_html/current/a*.html . 

Appendix

A Brief Overview Of Git

#ABriefOverviewOfGit

You will need a small amount of familiarity with the Git version control system to work as a CS241e ISA. Many Git tutorials jump quickly into subjects like branching, which are extremely important but not really necessary for CS241e. For this reason, we cover a small fragment of Git that should be sufficient for basic tasks like updating the course website.

A Git repository is a collection of files, together with metadata that tracks changes between different versions of the files. The git clone command allows you to create a local copy of the repository on your computer. You can then freely edit the files. The original repository, as well as other people's local copies, will not be affected. When you are satisfied with your changes, the git push command allows you to upload your changes to the original repository. This uploading process doesn't permanently overwrite the files ? a history of versions is maintained using the aforementioned change-tracking metadata. Other users of the repository can use the git pull command to download your changes to their local copy. Because a version history is maintained, if your changes are terrible, people can revert to an earlier version without your changes.

The above paragraph leaves out some steps. Git's change-tracking metadata isn't magic ? you need to tell Git which files it should care about. This is done using the git add command. For example, git add important.txt will tell Git that it should care about the file called important.txt. Often you make changes to a lot of files at once, and using git add =on each one individually would be tedious. In this case, you can use the command =git add -A to add all changed files. There are other options for git add that may be useful in scenarios where you want to add some changed files, but leave others out. You may also find it useful to use a graphical user interface for Git such as SourceTree, which lets you choose which files to add visually.

Once you have added the changed files, you are still not done. You need to tell Git to finalize or "commit" the changes, so that the changes officially get recorded in the version history. This is done with the git commit command. A commit generally includes a short but informative message describing what was changed. The git commit command will open up a text editor in which you can write your commit message. After the commit is finished, you can finally use git push to push your changes to the original repository.

A problem can occur if two people try to make different changes to the same file. A merge conflict occurs if you try to pull changes from the original repository that conflict with the changes in your local copy. Merge conflicts are less scary than many Git beginners think they are, but too scary to explain how to resolve in this little guide. Try to coordinate with other course staff to avoid merge conflicts altogether. If they occur, look for a tutorial on resolving them, or ask for help from someone more experienced with Git.

The Course Git Repository

#TheCourseGitRepository

CS241e currently uses a Git repository to store course material. The term "course material" encompasses:

• All assignment-related code: the "handout code" given to students, the instructor's solutions, the Marmoset test cases, and the private libraries distributed with the handout code.
• The course website. The repository is set up so that changes made to the course website in the repository, once pushed will be copied to the public_html directory on the course account and made immediately visible to students!
• Lecture notes and past tutorial material.

The Git repository has a lot of folders. Here are the most important ones:

• assignments – contains the base source code for both assignment problems and instructor's solutions. The problems and solutions are combined in a single file, and a build script is used to generate separate "problem source code" and "solution source code". The Marmoset tests for each assignment are also stored here under the test subdirectory.
• problems – the "problem source code" generated by the build script is stored here. When you first clone the repository, this will be empty, and you must run the build script.
• solutions – the "solution source code" generated by the build script is stored here. When you first clone the repository, this will be empty, and you must run the build script.
• library – contains the source code for the libraries that are distributed to students as part of the handout code.
• testutil – contains some utility code used by the Marmoset tests.
• documents – the assignment specifications and questions are stored here as Markdown files. Past tutorials are also stored here.
• lectures – contains lecture notes.
• marmoset – contains scripts for constructing the Marmoset test setup and canonical solution files.
• scripts – likely intended to hold general scripts for use in the course, but currently just contains one script (which generates the HTML viewer for the handout code on the course website).
• web – contains the HTML source for the course website. Modifications made on your local copy of the repository will not be reflected on the course website. However, once you push your modifications to the original repository, the course website will be immediately updated with them!

As an ISA, you'll probably only be modifying the contents of the web folder. However, the other folders will certainly be useful to look at. You might want to peek at the solutions to get a better understanding of something, you might want to look at what certain Marmoset tests actually do, or you might want to look at the lecture notes to remind yourself of course concepts.

The Repository Build Script (doall.sh)

In the root of the repository, there is a shell script called doall.sh. This shell script runs a number of sbt tasks (sbt is a build tool for Scala) to generate the "problem" and "solution" source code, compile the handout code, run the Marmoset tests against the solutions to check them for correctness, and package the handout code. It also copies the current version of the handout code to the course website's HTML viewer.

You will need to have sbt installed to use the script. If you are working on the school's Linux environment, it should be installed there, but if you are working from your own computer you might have to install it yourself.

The doall.sh script will take a very long time to run because of the fact that it runs all the Marmoset tests. If you just want to generate the "problem" and "solution" source code, you can just run sbt clean problems solutions.

If doall.sh fails, the problem might be differences in the version of Java or the version of some required dependency between your computer and the instructor's computer. Report the problem to the instructor.

The Git Repository and IntelliJ / Testing Students' Code

An interesting aspect of the course Git repository is that it actually doubles as an IntelliJ project! It works similarly to opening the handout code. Follow the instructions on https://www.student.cs.uwaterloo.ca/~cs241e/current/idea.shtml for opening the handout code in IntelliJ, but instead select the folder containing the Git repository. IntelliJ will create some project metadata files in the Git repository directory, but the repository is actually set up to ignore these extra files, so it shouldn't cause any problems.

This is mostly useful to the instructor for developing the handout code, but it can be a nice way to browse the repository as an ISA. Also, there is one great benefit of this functionality for ISAs: you can use it to run the Marmoset tests directly against student code! This is useful when a student complains about failing a test and has no idea why – you can narrow down the possible reasons by actually running the test case yourself, and give more pointed help.

The idea is to create a second copy of the repository. To test a student's code, download it as a ZIP file from Marmoset, then extract the src directory from the ZIP file to the assignment directory of the copied repository. This will overwrite the assignment code with the student's solutions (which is why we make a copy of the repository). Then, you can run the Marmoset tests (under assignments/test) and they will run against the student's code. When another student asks for help, just replace the previous student's code with the new student's code.

Setting up Marmoset tests

#SettingupMarmosetTests

Initial set-up steps, which only need to be done once.

1. If you haven't previously set up a copy of the repository for yourself, create a sub-directory for yourself under /u/cs241e/u, using your UW userid as the directory name, and then clone a copy of the course repository underneath.
2. Go to the marmoset/ subdirectory. In there, there is a shell script mktests.sh that creates the ZIP files (as .jar files) to upload to Marmoset, using the testing .scala files in assignments/test/src/cs241e/.
3. Run mktests.sh. Note that the line for assign in `seq 11` 6A 6B; do in mktests.sh is used to create tests for all 11 assignments thanks to the seq command, and then splits the tests for assignment 6 into a 6A and a 6B (assignment was split due to the size for the online offerings of the course).

For each project:

1. Create the Marmoset project in Marmoset. See the previous section on creating a Marmoset project.
2. Upload the test cases ZIP file for that particular assignment. You can find the test cases ZIP file in the sub-directory marmoset/target. For example, the file name will look something like: test-a1-2009.01-1831856a5f2ecf19d20130e9c84ef88ed739ab3c.jar
3. Upload the "canonical" solution for that assignment as a ZIP file. You will need to create a ZIP file for this. There may be easier ways, but the simplest approach is to:
   1. Start with the provided student base case. Download it, and set up IntelliJ as described on the course website.
   2. Add in the solution code for that particular assignment. You can find the solutions in your copy of the repository under solutions/src/cs241e/. There is a .scala file for each assignment, and you'll need to check the assignment to see which other files will also need modification. Replace all occurrences of "???" with the necessary code fragments.
   3. Use sbt to create the appropriate ZIP file. It's probably a good idea to rename it to include the assignment name in case you need to go back.
   4. Upload that ZIP file to Marmoset as the "canonical" solution. Note that tests cannot be run until the solution is set up.
4. Check with the instructor when the project should be made visible to the students, and when the results should be made available.
5. The tests will need to be made active. To do this, click on the "assign points" link found in the "activate/inactivate" column of the "Testing setups" section of the project in Marmoset. Ensure that marks are assigned to each test (defaults to 1 point per test, which is what we want) and click the button labelled "Assign points, edit test-setup comment and activate test setup.

The workflow is to tweak the tests within IntelliJ IDEA in assignments/test/src/cs241e/, run/debug/test them in IntelliJ, then when you're happy with them, run the mktests.sh script and upload the .jar file for the relevant assignment to Marmoset.

Additional Course Account Scripts

Documentation for potentially useful scripts created in Fall 2020 that are not mentioned earlier on this page.

marm_why

marm_why is a modified version of the marm_why script from CS 246. It basically mimics the instructor view on Marmoset, minus the web interface.

Given a student ID and assignment name, marm_why creates a directory ~/localtest/[STUDENT_ID].[ASSIGNMENT_NAME]/ containing the test results from each Marmoset test as well as the submitted source code.

marm_why is located at ~/bin/marm_why. To use it, run the following command:

marm_why student_id assignment [-s sub_num] [-t test name] [-v] [-h]

Options:
-s sub_num: This allows the testing of specific submissions. sub_num should be the submission number seen on Marmoset.
-t test: When testing marmoset submission, will only check the specified test instead of checking all of the tests for the submission.
-v: Outputs details as the script runs. This can be useful if the script is slow.
-h: Display long-form help message

filter_archived_files

filter_archived_files will "filter" the files of archived student submissions for a given assignment, isolating the specific files that get modified in the course of completing that assignment. The purpose of this is to reduce the total number of files that MOSS checks, since students resubmit their whole codebase for each assignment.

More specifically, for a given assignment, the script will copy over the necessary files for each student from ~/archives/[TERM#]/handin/ to ~/archives/[TERM#]/moss/[ASSIGNMENT]/sc/.

Note: there should typically be no need to run filter_archived_files as, going forward, the archives will be filtered in this manner already. You should only need to run this script if the archives are somehow missing the filtered files or if assignment specifications have changed in your term.

filter_archived_files is located at ~/bin/filter_archived_files. To use it:

1. Update the PAST_TERMS variable within the script.
2. Navigate to ~/archives/ and run the following command. (Ax refers to the name of the directory you want to create in ~/archives/[TERM#]/moss/, while archived_assignment refers to the directory name in ~/archives/[TERM#]/handin/ where the student source code is stored.)

   filter_archived_files Ax [archived_assignment]

filter_moss_files

filter_moss_files is located at ~/bin/filter_moss_files. It is used in moss241e.sh to do the file-filtering process on student submissions from the current term. You should not need to run this script directly.