Course Evaluation System Sysadmin Documentation

This page is meant to include all useful instructions for maintaining the course evaluations by CSCF sysadmins

Table of Contents

Code

The production code is located in core.cs:/u/cs-eval/evals . It is a checkout of a git repository from https://git.uwaterloo.ca/cscf/course-evaluation/ containing all source code revisions. See GitUWaterlooCSCF for notes on usage.

See CourseEvaluationData for more detail of the data.

Scheduled Tasks

Once a Term, early in the term

The timing of this is set by looking up the official faculty release-date- see below under "Make RT item to address the following term's work"

Send Reports to Users

1. we should send command-line users (in core group cs-eval) an email update that the data is available. An (older, more verbose) sample email is in CourseEvaluationUsers. A more recent email can be found in ST#100066
2. any individual users requesting their data should be sent data; this has tapered off since results are available via the web, but if any special requests come, see "Queries on a Particular Instructor" below.
3. Math chairs are to receive the appropriate outputs for their areas, as follows:

Evals are produced and sent using the following steps (checking the output for sensibleness):

su - cs-eval
cd evals/data
T=S2014
../report.py --term $T -f
../report.py --avg_terms 15 --term $T --subject 'CS,SE' --tag grad | tail -n +2 > GRAD
../report.py --avg_terms 15 --term $T --subject 'CS,SE' > CS  
../report.py --avg_terms 15 --term $T --subject 'AMATH,MATH' > AM
../report.py --avg_terms 15 --term $T --subject 'CO,MATH' > CO
../report.py --avg_terms 15 --term $T --subject 'COMM,MATBUS' > COMM
../report.py --avg_terms 15 --term $T --subject 'PMATH,MATH' > PM
../report.py --avg_terms 15 --term $T --subject 'ACTSC,STAT,MTHEL' > STAT
../report.py --avg_terms 15 --term $T --subject 'ACTSC,STAT,MTHEL' --tag grad | tail -n +2  >> STAT
../report.py --avg_terms 15 --term $T --subject 'MATH,MTHEL,ENGL' > MATH

The first step will take a few minutes (and will output progress reports) during which it will erase and re-fill the local cache database for the specified term. This may be re-run at any time if necessary for some reason. If there are errors with the tool connecting to the server (evaluate.uwaterloo.ca), check with Mirko Vucicevich (mvucicev@uwaterloo.ca).

The second and further steps will run quickly, and won't output progress reports.

Review each file to make sure it has data as expected, and then run:

cat CS GRAD|mail -s "$T Term Evaluations" cs-director@uwaterloo.ca
cat AM|mail -s "$T Term Evaluations" am-chair@uwaterloo.ca
cat CO|mail -s "$T Term Evaluations" co-chair@uwaterloo.ca
cat COMM|mail -s "$T Term Evaluations" ilham.akhundov@uwaterloo.ca
cat PM|mail -s "$T Term Evaluations" pm-chair@uwaterloo.ca
cat STAT|mail -s "$T Term Evaluations" sas-chair@uwaterloo.ca
cat MATH|mail -s "$T Term Evaluations" mat_assd@math.uwaterloo.ca

Clean up:

rm GRAD MATH STAT PM COMM CO AM CS

Make RT item to address the following term's work

When you've finished this term's reports, make a RT ticket to address producing the next term's reports- with no due-date. In that ticket, set a reminder a month into the next term to check the course release date. When that reminder comes due, check the official course release date, which is listed in https://evaluate.uwaterloo.ca/administration/review-courses/ (and if that's not available, it's probably somewhere in the campus undergrad calendar). Use the release-date to set the due-date for the ticket.

Guides to the Completion of Common Requests

Top Instructors for CS|SE Courses

The first month in a new term, the CS Director may ask for a Top Instructors report (mwg has been able to run the report himself). To produce it, run:

./report.py --term $TERM --top_instructors

./report.py --term $TERM --top_instructors --tag grad

The output has the Top Instructors report, followed by full course data for those courses. As described, top instructors have 70% response-rate or 25+ replies; and all ratings must be 4.1 or higher. The results should all be CS and SE courses (or cross-listed as CS and SE courses); and/or grad courses. If a course shows up as a non-CS course, but is cross-listed as a CS course in the full course data, you may need to adjust the output to list the CS course instead of the non-CS course.

Queries on a particular instructor, and "Tenure and Promotion" requests

An instructor may request prior term results for themselves. We can run ./report.py --userid $USERID --term $TERM --avg_terms --out=table (where TERM can be, for Fall 2013: F2013 or 2013.3 or 2013.9 or the always-favourite format 1139).

• The output is in column / separated table format. If the instructor would like CSV output, simply leave off the --out=table parameter.

A Math Chair (or CS DIrector) may request all available data for a particular instructor who is undergoing review for tenure or promotion. We will run: ./report.py --userid $USERID --avg_terms and send the results to the chair.

The CS Director or admin assistant will usually ask for a batch around the end of June. If they request a batch, this can be accomplished with the following steps:

Make a ticket; with subscription code sc-evals.with title in the form "course-evaluation: 2020 tenure and promotion requests"; see-also the last year's.

rm tandp.csv; for i in user1 user2 user3 user4 user5 ;  do ../report.py --userid $i --avg_terms >> tandp.csv ; echo ",Average undergrad">> tandp.csv; echo >> tandp.csv; ../report.py --userid $i --tag grad --avg_terms >> tandp.csv; echo ",Average grad">> tandp.csv; echo >> tandp.csv; done

• check output carefully that all instructors are included- a mis-typed userid will skip them
• userid must be 8-character short form.
• (Under each prof, the descriptive phrase "Average undergrad" should appear in the second column, because we will be deleting the first column.)
• The Chair or Director may have additional requests for data formatting: see ST#102146 for iterations of a CS Director request.

A sample excel spreadsheet for tenure and promotion will be attached to this wiki page.

Processing of the CSV is quickly done in excel (the following was tested with Mac Excel 2016):

• import tandp.csv into a new worksheet - specifically telling it to skip three columns: course_id, userid, and section
• in the empty line labeled "Average undergrad" directly under each prof's undergrad courses, highlight the seven columns from rresp to summary (should be rows E through K). In the tool strip, hit the arrow next to "Autosum" and choose "Average". It will produce averages of the preceding data, which should be an average of that prof's course results. Repeat for grad courses.
• for each prof, highlight each header row and average row and hit command-B (or ctrl-B on windows) to boldface them.
• check the results for accuracy and completeness. You will probably need to auto-fill the name column where we imported data with just the userid. There may be some missing "grad" column data, though it will be sorted accurately.

Requests to change the users in group 'cs-eval'

When the CS/SE faculty members involved with tenure/promotion change, there will be a request to remove/add users from the system. This is done by removing and adding the userid(s) from/to unix group 'cs-eval' in the Core region via the Sponsors Data Accounts system. (The specific file controlling cs-eval is /software/accounts-master/data/sponsors/School/cs.admin )

Users should also get CS-wide access to the evaluate.uwaterloo.ca data; this involves visiting https://http://evaluate.uwaterloo.ca/ and choosing the Administrative menu from the left-hand side; selecting Department Acess in CS ( http://evaluate.uwaterloo.ca/administration/department-access/#cs ) and adding each user (8-character version of username) with "read" and "promote" permissions. To remove a user, simply remove their permissions. There are user instructions, if needed, at https://cs.uwaterloo.ca/cscf/teaching/evaluate/

-- DanielAllen - 2014-01-30

• tandp-sample.xlsx: Tenure and Promotion randomized sample