Course Evaluation System Sysadmin Documentation

This page is meant to include all useful instructions for maintaining the course evaluations by CSCF sysadmins

Table of Contents

Code

The production code is located in core.cs:/u/cs-eval/evals . It is a checkout of a git repository from https://git.uwaterloo.ca/cscf/course-evaluation/ containing all source code revisions. See GitUWaterlooCSCF for notes on usage.

See CourseEvaluationData for more detail of the data.

Scheduled Tasks

Once a Term, early in the term

The timing of this is set by looking up the official faculty release-date- see below under "Make RT item to address the following term's work"

Send Reports to Users

1. we should send command-line users (in core group cs-eval) an email update that the data is available. An (older, more verbose) sample email is in CourseEvaluationUsers. A more recent email can be found in ST#100066
2. any individual users requesting their data should be sent data; this has tapered off since results are available via the web, but if any special requests come, see "Queries on a Particular Instructor" below.
3. Math chairs are to receive the appropriate outputs for their areas, as follows:

Evals are produced and sent using the following steps (checking the output for sensibleness):

cd data
T=S2014
../report.py --term $T -f
../report.py --avg_terms 15 --term $T --subject 'CS,SE' --tag grad | tail -n +2 > GRAD
../report.py --avg_terms 15 --term $T --subject 'CS,SE' > CS  
../report.py --avg_terms 15 --term $T --subject 'AMATH,MATH' > AM
../report.py --avg_terms 15 --term $T --subject 'CO,MATH' > CO
../report.py --avg_terms 15 --term $T --subject 'COMM,MATBUS' > COMM
../report.py --avg_terms 15 --term $T --subject 'PMATH,MATH' > PM
../report.py --avg_terms 15 --term $T --subject 'ACTSC,STAT,MTHEL' > STAT
../report.py --avg_terms 15 --term $T --subject 'ACTSC,STAT,MTHEL' --tag grad | tail -n +2  >> STAT
../report.py --avg_terms 15 --term $T --subject 'MATH,MTHEL,ENGL' > MATH

The first step will take a few minutes (and will output progress reports) during which it will erase and re-fill the local cache database for the specified term. This may be re-run at any time if necessary for some reason. If there are errors with the tool connecting to the server (evaluate.uwaterloo.ca), check with Mirko Vucicevich (mvucicev@uwaterloo.ca).

The second and further steps will run quickly, and won't output progress reports.

Review each file to make sure it has data as expected, and then run:

cat CS GRAD|mail -s "$T Term Evaluations" cs-director@uwaterloo.ca
cat AM|mail -s "$T Term Evaluations" am-chair@uwaterloo.ca
cat CO|mail -s "$T Term Evaluations" co-chair@uwaterloo.ca
cat COMM|mail -s "$T Term Evaluations" ilham.akhundov@uwaterloo.ca
cat PM|mail -s "$T Term Evaluations" pm-chair@uwaterloo.ca
cat STAT|mail -s "$T Term Evaluations" sas-chair@uwaterloo.ca
cat MATH|mail -s "$T Term Evaluations" mat_assd@math.uwaterloo.ca

Clean up:

rm GRAD MATH STAT PM COMM CO AM CS

Make RT item to address the following term's work

When you've finished this term's reports, make a RT ticket to address producing the next term's reports- with no due-date. In that ticket, set a reminder a month into the next term to check the course release date. When that reminder comes due, check the official course release date, which is listed in https://evaluate.uwaterloo.ca/administration/review-courses/ (and if that's not available, it's probably somewhere in the campus undergrad calendar). Use the release-date to set the due-date for the ticket.

Guides to the Completion of Common Requests

Top Instructors for CS|SE Courses

The first month in a new term, the CS Director may ask for a Top Instructors report (mwg has been able to run the report himself). To produce it, run:

./report.py --term $TERM --top_instructors

./report.py --term $TERM --top_instructors --tag grad

The output has the Top Instructors report, followed by full course data for those courses. As described, top instructors have 70% response-rate or 25+ replies; and all ratings must be 4.1 or higher. The results should all be CS and SE courses (or cross-listed as CS and SE courses); and/or grad courses. If a course shows up as a non-CS course, but is cross-listed as a CS course in the full course data, you may need to adjust the output to list the CS course instead of the non-CS course.

Queries on a particular instructor

An instructor may request prior term results for themselves. We can run ./report.py --userid $USERID --term $TERM --avg_terms --out=table (where TERM can be, for Fall 2013: F2013 or 2013.3 or 2013.9 or the always-favourite format 1139).

• The output is in column / separated table format. If the instructor would like CSV output, simply leave off the --out=table parameter.

A Math Chair (or CS DIrector) may request all available data for a particular instructor who is undergoing review for tenure or promotion. We will run: ./report.py --userid $USERID --avg_terms and send the results to the chair.

If they request a batch, this can be accomplished with: date; rm tandp.csv; for i in user1 user2 user3 user4 user5 ; do ../report.py --userid $i --avg_terms >> tandp.csv; ../report.py --userid $i --tag grad --avg_terms >> tandp.csv; done; date

• check output carefully that all instructors are included- a mistyped userid will skip them
• userid must be 8-character short form.
• The Chair or Director may have additional requests for data formatting: see ST#102146 for iterations of a CS Director request (a process which will probably be simplified for the next request, and so I've not fully documented it here).

Requests to change the users in group 'cs-eval'

When the CS/SE faculty members involved with tenure/promotion change, there will be a request to remove/add users from the system. This is done by removing and adding the userid(s) from/to unix group 'cs-eval' in the Core region via the Sponsors Data Accounts system. (The specific file controlling cs-eval is /software/accounts-master/data/sponsors/School/cs.admin )

Users should also get CS-wide access to the evaluate.uwaterloo.ca data; this involves visiting https://http://evaluate.uwaterloo.ca/ and choosing the Administrative menu from the left-hand side; selecting Department Acess in CS ( http://evaluate.uwaterloo.ca/administration/department-access/#cs ) and adding each user (8-character version of username) with "read" and "promote" permissions. To remove a user, simply remove their permissions. There are user instructions, if needed, at https://cs.uwaterloo.ca/cscf/teaching/evaluate/

-- DanielAllen - 2014-01-30