Resources and Information for CS 241e ISAs

This twiki is for technical details only. Any other duty information can be found in Waterloo LEARN (D2L): CS Instructional Support Group -> Course Admin & Duties for ISAs.

Under Construction!

Sylvie (F19 CS241e ISA) is in the process of writing a new version of this page. Some information might be temporarily duplicated or omitted as the page is edited.

Introduction

General Notes and Warnings

CS241e is an enriched version of CS241, but it's quite different from the standard version of the course. This Twiki focuses on the technical side of being an ISA, but if you haven't taken CS241e before, be sure to keep up with the material and stay ahead of students.

Since 2015, CS241e has been consistently taught by one instructor, Ondřej Lhoták. Much of the infrastructure for the course was created by Ondřej and he is always updating and improving it between terms. As such, the information on this Twiki can easily become outdated! Before running any of the scripts described on this Twiki page, you should read the source code and check that they actually do what you expect. If you notice any incorrect information on the Twiki, please update it.

Ondřej is familiar with Marmoset, and usually he will be the one responsible for creating and maintaining the Marmoset test setups, canonical solutions, and libraries. The ISA will generally not need to do much Marmoset setup. This Twiki page attempts to document all aspects of Marmoset setup for completeness, but this documentation could quickly become incorrect if Ondřej changes his workflow. In general, if there is an issue with the Marmoset scripts, notify Ondřej about it rather than trying to follow the methods on this page.

The Course Git Repository

In what follows, familiarity with the Git version control system is assumed. There is a brief introduction to Git in the Appendix.

CS241e currently uses a Git repository to store course material. The term "course material" encompasses:

  • All assignment-related code: the "handout code" given to students, the instructor's solutions, the Marmoset test cases, and the private libraries distributed with the handout code.
  • The course website. The repository is set up so that changes made to the course website in the repository, once pushed will be copied to the public_html directory on the course account and made immediately visible to students!
  • Lecture notes and past tutorial material.
The Git repository has a lot of folders. Here are the most important ones:
  • assignments – contains the base source code for both assignment problems and instructor's solutions. The problems and solutions are combined in a single file, and a build script is used to generate separate "problem source code" and "solution source code". The Marmoset tests for each assignment are also stored here under the test subdirectory.
  • problems – the "problem source code" generated by the build script is stored here. When you first clone the repository, this will be empty, and you must run the build script.
  • solutions – the "solution source code" generated by the build script is stored here. When you first clone the repository, this will be empty, and you must run the build script.
  • library – contains the source code for the libraries that are distributed to students as part of the handout code.
  • testutil – contains some utility code used by the Marmoset tests.
  • documents – the assignment specifications and questions are stored here as Markdown files. Past tutorials are also stored here.
  • lectures – contains lecture notes.
  • marmoset – contains scripts for constructing the Marmoset test setup and canonical solution files.
  • scripts – likely intended to hold general scripts for use in the course, but currently just contains one script (which generates the HTML viewer for the handout code on the course website).
  • web – contains the HTML source for the course website. Modifications made on your local copy of the repository will not be reflected on the course website. However, once you push your modifications to the original repository, the course website will be immediately updated with them!
As an ISA, you'll probably only be modifying the contents of the "web" folder. However, the other folders will certainly be useful to look at. You might want to peek at the solutions to get a better understanding of something, you might want to look at what certain Marmoset tests actually do, or you might want to look at the lecture notes to remind yourself of course concepts.

Repository Build Script

In the root of the repository, there is a shell script called doall.sh. This shell script runs a number of sbt tasks (sbt is a build tool for Scala) to generate the "problem" and "solution" source code, compile the handout code, run the Marmoset tests against the solutions to check them for correctness, and package the handout code. It also copies the current version of the handout code to the course website's HTML viewer.

You will need to have sbt installed to use the script. If you are working on the school's Linux environment, it should be installed there, but if you are working from your own computer you might have to install it yourself.

The doall.sh script will take a very long time to run because of the fact that it runs all the Marmoset tests. If you just want to generate the "problem" and "solution" source code, you can just run sbt clean problems solutions.

If doall.sh fails, the problem might be differences in the version of Java or the version of some required dependency between your computer and the instructor's computer. Report the problem to the instructor.

The Git Repository and IntelliJ

An interesting aspect of the course Git repository is that it actually doubles as an IntelliJ project! It works similarly to opening the handout code. Follow the instructions on https://www.student.cs.uwaterloo.ca/~cs241e/current/idea.shtml for opening the handout code in IntelliJ, but instead select the folder containing the Git repository. IntelliJ will create some project metadata files in the Git repository directory, but the repository is actually set up to ignore these extra files, so it shouldn't cause any problems.

This is mostly useful to the instructor for developing the handout code, but it can be a nice way to browse the repository as an ISA. Also, there is one great benefit of this functionality for ISAs: you can use it to run the Marmoset tests directly against student code! This is useful when a student complains about failing a test and has no idea why – you can narrow down the possible reasons by actually running the test case yourself, and give more pointed help.

The idea is to create a second copy of the repository. To test a student's code, download it as a ZIP file from Marmoset, then extract the src directory from the ZIP file to the assignment directory of the copied repository. This will overwrite the assignment code with the student's solutions (which is why we make a copy of the repository). Then, you can run the Marmoset tests (under assignments/test) and they will run against the student's code. When another student asks for help, just replace the previous student's code with the new student's code.

Beginning of Term Tasks

Updating the Course Website

The course website is stored on the course account under the "public_html" directory. However, you should never edit this directory manually. Instead, you should modify the files under the "web" directory of the course Git repository. When you push your changes, the changes will automatically be copied above to the public_html directory. This means if you edit public_html manually, your changes will be overwritten next time someone pushes changes using Git!

At the start of the term, you should go through each page of the course website and update the pages as necessary. Some of the steps below might be already done for you by the instructor, the ISC, or the previous term's ISA.

  • Update any places where the term and year appears (e.g. "Fall 2019") to reflect the current term and year.
  • Update the "Piazza Link" on the sidebar to the one for the current term, after Piazza has been set up.
  • Update the “Assignment Info” page. There are a several things to look at here:
    • If the minimum version of Java required has changed, update the version number in the Java JDK link.
    • Update the version number in the IntelliJ link to the latest version.
    • Make sure the Java JDK link and IntelliJ link still point to the correct places (Oracle or Jetbrains might have changed their websites around).
    • Update the assignment skeleton code link once the instructor has prepared the skeleton code for the term.
    • Look over the Marmoset submission guide. If IntelliJ’s interface has significantly changed and the guide is no longer correct, update the guide.
    • At the bottom there is a mention of the version number of Scala in a file path. If a newer version of Scala is being used, update this number.
  • Look at the "!IntelliJ IDEA Setup" page. If IntelliJ’s interface has significantly changed since this page was written, you may need to update the setup guide.
  • Remove all links from the “Assignments” page and update the assignment deadlines to reflect the ones for the current term.
  • If this offering of the course has exams, update the “Exam Information” page with information about the current term’s midterm exam. Leave the section about the final exam on the page, but put “TBA” (To Be Announced) for the date, time and location. Comment out any irrelevant information (e.g., information about midterm remarking or midterm and final review sessions). If there are no exams, the "Exam Information" page can be removed.
  • Remove all links from the “Tutorials” page.
  • Update the “Personnel” page with information about the current term’s course staff. Update the page again once everyone’s office hours are determined (you can write “TBA” if they are not yet known).
  • On the “Resources” page, comment out the “Code from Lectures” section and anything else similar that shouldn't be there at the start of the term.

Updating The Google Calendar

The front page of the course website contains an embedded Google Calendar. It contains:

  • Lecture and tutorial times
  • Office hours
  • Assignment deadlines
  • Midterm and final exam times
This calendar should be updated with the necessary information at the start of the term. It should be kept up to date throughout the term if any information changes.

To update the calendar, first click the Google Calendar logo in the bottom right:

calendar.png

Log in to the course Google Account using the course email address cs241e@uwaterloo.ca (even though it's a Google Account, it's associated with the course email rather than a Gmail address). The password should be the same as the course email account.

If the password doesn't work, you can use the "forgot your password" option to reset it. If you change it to something different from the course email account password, make sure to tell your ISC the new password so that they can record it. (Don't post the password on the TWiki!)

Once you're logged in, you can add all the events for the current term to the calendar. If you aren't familiar with how to use Google Calendar, the help page can be found here. The main thing you should know is that it is possible to create "repeating events" that happen on certain days every week. This is very useful for things like lectures, tutorials and office hours that will probably happen the same time every week; you can just create a repeating event for these instead of adding them one at a time.

Updating the Assignment Code HTML Viewer

The handout code can be viewed online at the following page: https://www.student.cs.uwaterloo.ca/~cs241e/current/scalasrc/A1.html

Whenever the handout code is updated, the HTML version of the code needs to be updated accordingly. This is done using the mkcodehtml script under the "scripts" directory of the course Git repo.

Be careful with this script. In one past term, an ISA accidentally uploaded all the assignment solutions to the course website due to incorrect use of the script. The script has been updated since then to make this harder to accidentally do. Nonetheless, before running the script, you should look at the source code and make sure you understand exactly what files are going to be copied over to the course website. You might want to just leave this task up the instructor to be safe.

That being said, the script is straightforward to use. Make sure you have generated the "problem source code" by running sbt clean problems. Then change to the script directory and run ./mkcodehtml, and this will generate HTML files for the most recent versions of the problem source code and library source code and put them under the "web" directory. The script is written in Lisp, so you will probably need to run it on the Linux environment, unless you happen to have a Lisp interpreter installed or feel like installing one. Once the script finishes, you can push the changes to the course website.

Setting Up Piazza

The instructions under the "Set Up Piazza" header on the CompSci241ISADuties page should work for CS241e as well.

Setting Up Marmoset

Before you can begin setting up Marmoset, CSCF must create a Marmoset instance for your course, and you must be added with instructor privileges. Once this is done, when you log in to Marmoset, you should be presented with a screen where you can select from three accounts: "userid", "userid-canonical", and "userid-student". The first one, "userid", is the main account you will be using with instructor privileges. The second, "userid-canonical", is used internally by Marmoset, but there's basically no reason to ever log on to it. The third, "userid-student", is an account with the exact same privileges as a student. Instructor accounts can switch between "instructor view" and "student view", so this is not very useful, but there may be some times where you want to see exactly what a student sees.

After logging in with your "userid" account, you will need to click the "Instructor View" link on the top bar to get access to most instructor privileges.

Adding Students and Staff

Follow the instructions on MarmosetSummary under the "Adding users" header to batch-add students and to add other members of course staff. If asked whether or not to limit user ID length, you should choose to limit it to 8 characters in length.

Early on in the term, new students will be adding and dropping the course every day. You will need to keep adding students to Marmoset until the class roster stabilizes, or else newly added students will not be able to access Marmoset. The easiest way to do this is probably to just repeat the batch-add process once a day. The batch-add feature is smart enough to not create duplicate students. If a student contacts you saying they can't access Marmoset, you can use the feature for adding a single student to make sure they get added immediately.

Creating Marmoset Projects

A "project" on Marmoset refers to a dropbox of sorts where students can submit their code and have it automatically run against a set of test scripts. In CS241e, there is one project for each assignment (unlike ordinary CS241, which has one project for each assignment question).

Creating projects is pretty straightforward. At the bottom of the "Projects" table (which will be empty at the start of the term), click the "create new project" link and fill out the fields.

  • By convention, the "Project Number" and "Project Title" are set to be the exact same string. This allow scripts that work with Marmoset projects to refer to either one interchangeably. In CS241e, this string is usually just "AN" where N is the assignment number ("A1" for Assignment 1, "A2" for Assignment 2, "A11" for Assignment 11, etc.)
  • Fill in the on-time deadline and late deadline according to the late policy for the current term.
  • For "URL" you can put a link to the assignment page, but it's not really necessary.
  • The "description" is usually left blank.
  • Important! Change the "stack trace policy" to "the entire stack trace for Java or full output printed to stdout for C". With the default setting, tests will not show any output.
  • Everything else can be left as the default. The stuff related to "late penalties" can be ignored because late penalties are applied by a separate script.
  • This is not essential, but you might want to change the "canonical class account" to your account, to indicate that you are the one who set up the project.
This is all you should need to know about setting up Marmoset as an ISA. The Appendix of this Twiki page covers some of the finer details of how the Marmoset test scripts for CS241e are actually created, uploaded and executed.

Setting Up The Course Staff File

There is a special file in the course account that some scripts use to determine which user IDs are members of course staff. If this has not been set up for you by someone else, you should set it up. Follow the instructions on CompSci241ISADuties under the "Create Course Staff File For Current Term" header.

"During The Term" Tasks

Updating The Unofficial Marks

There is a page on the course website where students can check their "unofficial marks" for the course: https://www.student.cs.uwaterloo.ca/~cs241e/cgi-bin/displayMarks.py

The marks shown on this page are read from the file ~/marks/public-marks.txt stored on the course account. You will have to update this file throughout the term.

Some helpful scripts for this are stored under ~/marks/scratch. You will also make use of MarmSql for downloading the marks. Here is the process for updating the marks:

  1. Navigate to the ~/marks/scratch directory and run the following command:
    marm_sql -v -m AX
    Where X is the assignment number. The -v is optional; it makes marm_sql display a progress counter. If you just want to run it in the background with nothing printed to the screen, you can do:
    marm_sql -m AX &
    The & is Bash syntax for creating a job that runs in the background. You can do other things in the shell while the marks download, and you will get a message when the job finishes.
    Note that if the late policy for the course has changed in your term, you will need to modify marm_sql. Computation of late marks is done in marm_sql, not on Marmoset. The script is stored at ~/bin/marm_sql if you need to edit it.
  2. If there has been a midterm, and the grades are on Crowdmark, download the CSV file from Crowdmark and save it as midterm-crowdmark.csv. Then run:
    ./crowdmark.sh midterm-crowdmark.csv > midterm.csv
    This processes the Crowdmark file into an easier-to-handle format. If the midterm results aren't out yet or there is no midterm for the course, you can ignore this step.
  3. Run the command:
    ./combine.sh > public-marks.txt
    This will combine the files downloaded in Step 1, and the midterm marks from Step 2 (if applicable) into a single file formatted for use on the "unofficial marks" page. Look at the file and mark sure the marks look correct.
  4. When you're confident and ready to post the marks, run the command:
    cp public-marks.txt ..
    Now the marks will be visible on the webpage!
There is a script in ~/marks/scratch called update.sh which does all of the above steps in sequence, including copying the final marks to the main marks directory so they will appear on the webpage. Run ./update.sh Number, where Number is a number like 1, 2, 10, etc. rather than an assignment name like A1, A2 or A10. This will update the marks for Assignments 1 through Number, so for example, ./update.sh 5 will update the marks for Assignments 1 through 5. It will also update the midterm marks if the midterm-crowdmark.csv file is present.

At the time of writing, CS241e only uses public tests on Marmoset. If the course ever starts using release or secret tests, be careful when updating the marks. Updating the marks for an assignment that is currently running can "leak" the results of release and secret tests to students. Wait until after the deadline to update the marks. Even if the course only uses public tests, it's probably good practice to wait until after the deadline to post marks for an assignment, rather than letting students see their "marks so far" on the unofficial marks page.

Running Moss To Check For Plagiarism

....

"End Of Term" Tasks

Preparing The Grades Spreadsheet

....

Archiving Course Material

....

Updating The Twiki

....

Appendix

A Brief Overview Of Git

You will need a small amount of familiarity with the Git version control system to work as a CS241e ISA. Many Git tutorials jump quickly into subjects like branching, which are extremely important but not really necessary for CS241e. For this reason, we cover a small fragment of Git that should be sufficient for basic tasks like updating the course website.

A Git repository is a collection of files, together with metadata that tracks changes between different versions of the files. The git clone command allows you to create a local copy of the repository on your computer. You can then freely edit the files. The original repository, as well as other people's local copies, will not be affected. When you are satisfied with your changes, the git push command allows you to upload your changes to the original repository. This uploading process doesn't permanently overwrite the files – a history of versions is maintained using the aforementioned change-tracking metadata. Other users of the repository can use the git pull command to download your changes to their local copy. Because a version history is maintained, if your changes are terrible, people can revert to an earlier version without your changes.

The above paragraph leaves out some steps. Git's change-tracking metadata isn't magic – you need to tell Git which files it should care about. This is done using the git add command. For example, git add important.txt will tell Git that it should care about the file called important.txt. Often you make changes to a lot of files at once, and using git add on each one individually would be tedious. In this case, you can use the command git add -A to add all changed files. There are other options for git add that may be useful in scenarios where you want to add some changed files, but leave others out. You may also find it useful to use a graphical user interface for Git such as SourceTree, which lets you choose which files to add visually.

Once you have added the changed files, you are still not done. You need to tell Git to finalize or "commit" the changes, so that the changes officially get recorded in the version history. This is done with the git commit command. A commit generally includes a short but informative message describing what was changed. The git commit command will open up a text editor in which you can write your commit message. After the commit is finished, you can finally use git push to push your changes to the original repository.

A problem can occur if two people try to make different changes to the same file. A merge conflict occurs if you try to pull changes from the original repository that conflict with the changes in your local copy. Merge conflicts are less scary than many Git beginners think they are, but too scary to explain how to resolve in this little guide. Try to coordinate with other course staff to avoid merge conflicts altogether. If they occur, look for a tutorial on resolving them, or ask for help from someone more experienced with Git.

Marmoset Scripts

This section is intended to clarify how the Marmoset test scripts work in CS241e and to document the process of uploading the scripts to Marmoset. Usually the instructor will handle this, but it still might be useful for understand how it works as an ISA. For example, if something goes wrong, you might be able to help the instructor identify the problem if you know a bit about the inner workings of the scripts.

If this is your first time working with Marmoset, first read the page MarmosetSummary, and maybe also Marmoset. (The second page is outdated, but still potentially useful to read since it has more detailed explanations of some concepts.)

To create Marmoset tests that run against student code, you need to create a "test setup", which can be either "static" or "dynamic". You need to then upload this test setup to the Marmoset project, along with a "canonical solution" which passes all the test cases, serving as proof that the test setup is not broken. More details on this later, but for now remember that you need these two components: test setup and canonical solution.

It is common for courses using Marmoset to use dynamic test setups, and to store the test scripts on the course's "t" account (e.g. cs241t for CS241, cs246t for CS246). However, if you go looking for the Marmoset tests on the cs241et account you will just find a bunch of outdated scripts; the CS241e Marmoset tests are in the Git repo. Additionally, the test setups for CS241e are static, which means that if any changes are made, the test setup must be reuploaded.

Generating the Test Setup and Canonical Solution Files

The Marmoset test cases are defined as Scala tests, stored under the assignments/test directory of the repo. Of course, Marmoset test scripts have a particular structure and you cannot just upload a bunch of Scala files as your test setup. To generate proper test setups from these Scala files, a custom script is used, stored under the "marmoset" folder of the Git repo.

The script is called mktests.sh. To use it, simply run it with no arguments. As with any unfamiliar script, read the source code to see what it actually does before running it.

Canonical solutions can be found under the "solutions" folder in the repository after running sbt clean solutions. Just zip up the contents of the "src" subfolder. Alternatively, run sbt clean solutions packageSrc and look under "solutions/target/scala-2.13" for the packaged solution sources.

Uploading the Test Setup and Canonical Solution Files

....

Copying Libraries To The cs241et Account

....


Old CS241e Twiki Page Starts Below

Key Information

The vast majority of the information required is posted in CompSci241ISADuties. The remainder of this page deals with things specific to CS241e.

Updating the Website

Note that the CS241e website is updated using git, in order to clone the git repository, ask your prof/ISC for the git clone link (Posting it here may be ill-advised). Create a new folder in the private directory, and clone the git repository in that folder (or even rename of one no longer in-use folders in the ~/private/ directory to avoid wasting a ton of space). After doing this, you may use the following commands in the folder (without the quotation marks):

  • "git pull": this pulls the current changes from the git server
  • "git add -file-": adds a new/updated file to the git repository
  • "git commit": commits the current changes to the git repository. By default this will bring up an emacs file in which you should briefly describe what you changed.
  • "git push": uploads the git repository with your changes
Typically you will use the commands in the order "git pull" -> make edits to the site -> "git add -files-" -> git commit -> git push

Updating the Google Calendar

The front page of the course website contains an embedded Google Calendar. It contains:

  • Lecture and tutorial times
  • Office hours
  • Assignment deadlines
  • Midterm and final exam times
This calendar should be updated with the necessary information at the start of the term, as well as when the final exam schedule is released.

To update the calendar, first click the Google Calendar logo in the bottom right:

calendar.png

Log in to the course Google Account using the course email address cs241e@uwaterloo.ca (even though it's a Google Account, it's associated with the course email rather than a Gmail address). The password should be the same as the course email account.

If the password doesn't work, you can use the "forgot your password" option to reset it. If you change it to something different from the course email account password, make sure to tell your ISC the new password so that they can record it. (Don't post the password on the TWiki!)

Once you're logged in, you can add all the events for the current term to the calendar. If you aren't familiar with how to use Google Calendar, the help page can be found here. The main thing you should know is that it is possible to create "repeating events" that happen on certain days every week. This is very useful for things like lectures, tutorials and office hours that will probably happen the same time every week; you can just create a repeating event for these instead of adding them one at a time.

Posting Syntax-Highlighted Code

If the starter code has changed you may need to update the syntax-highlighted code available on the website. Starting in ~/private/<yourdir>/

  1. Update the files in res/src-nosols/ and library/src/cs241e with their new versions.
  2. cd scripts/
  3. ./mkcodehtml
You should be ready to go. Remember to commit all of res/, library/ and web/ afterwards.

Running Moss

The moss for CS241e is in the moss folder in the home directory, you should only have to change the term numbers in the moss241e.sh script in order to run it for the current term, everything else should be fine without edits. See RunningMoss for moss documentation.

Downloading Marks

When downloading marks, MarmSql should automatically apply extensions and calculate marks according to the formula (.25 * best on-time submission + .75 * best on-time or late submission). Double-check a few at random to make sure it's working correctly.

Topic attachments
I Attachment History Action Size Date Who Comment
PNGpng calendar.png r1 manage 4.0 K 2019-09-04 - 10:42 SylvieDavies  
Edit | Attach | Watch | Print version | History: r14 < r13 < r12 < r11 < r10 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r14 - 2020-09-21 - SylvieDavies
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback