Backlinks to Testing in ISG Web (Search all webs)
Results from ISG web retrieved at 12:55 (GMT)
A00 Setup Releasing to Students Students will receive a PDF describing the assignment and what is to be submitted. Students will also receive any starter files to...
Assignment Info The following is a list of pages on TWiki that relate to dealing with student assignments. AssignmentZero AssignmentManagement AssignmentRemarkPolicies...
Assignment Remark Policies The remarking policies for courses tend to vary depending on the course. Often, courses will accept remarks for up to two weeks after the...
Rollback request, feedback request Gracket conversion Rollback requests Rollback/Roll Forward Policy Rollback/roll forward requests policy changes from term to...
Walkthrough For Everything to do with Assignments (with lots of reference to BitterSuiteWalkthrough ) For This Walkthrough, we will assume that all work is being...
Automatically Assigning Correctness Marks in MarkUs This page is about a program that can automatically assign correctness marks on MarkUs. It assumes that your...
Automated Testing Suite Creation and Execution A script was created in Summer 2010 to assist tutors in the creation of autotesting folders for the assignments. An...
Basic Course Account Setup Initial Account Creation Initial access and testing accounts The person requesting creation of the course should be in the .rhosts of...
Basic Tests The basic tests are designed to help students detect any simple issues with their code. If the solution`s output is in the wrong format, the name of the...
See also : bittersuite3 man page: ISGScriptsManPages#Bittersuite BitterSuiteImprovePit BitterSuite BitterSuite is a collection of code currently designed...
BitterSuite3 Proposal NOTE : Several suggested features have not been implemented. Please see the official documentation for the actual features of this testing...
Acknowledgement of the Academic Integrity Statement The following test can be placed in a subfolder of in that has (language external) applied to it. #!/usr/bin...
BitterSuite C Language Documentation The C tests work in two stages. Any required files are compiled to object files and then linked together to form an executable...
BitterSuite C Testing This page will give helpful hints for how to do C testing. Verifying C code compiles without generating warnings Testing Hierarchy Solutions...
Configuring BitterSuite To configure BitterSuite, add a file config.ss to the suite directory. This ability was added to remove the need for obscure command line...
BitterSuite: Counting Function Calls The attached is a scheme file which can be used as a filter when rst is run. For instructions on how to implement the filtering...
BitterSuite Development This page gives a brief description of the history of the development of BitterSuite. The code for the latest versions of all previous versions...
BitterSuite External Language Documentation The External language is designed to be relatively simple, gaining most of its flexibility by supplying a myriad of environment...
Flexible Python Automated Testing with RST The problem with test.py The test.py method of Python automated testing described at BitterSuitePythonTesting is sufficient...
BitterSuite Script Testing This page will give helpful hints for testing in the `external` language, which is used when the builtin facilities do not suffice to test...
BitterSuite Filter Language Documentation This language is intended to be used as the first test run for a given testing suite where applicable. It provides a submission...
BitterSuite Filter language: Reject files until A0 is submitted The text below should be placed in a file on the course account; for example, in the test suite`s...
BitterSuite: Filtering student code prior to execution Warning: This has not yet been tested in BitterSuite 3. This needs to be tested, modified as needed, and verified...
Note: Read BitterSuiteWalkthrough before reading this document. BitterSuite I/0 Walkthrough Tutors of courses like CS 115 and CS 135 likely do not need to worry about...
BitterSuite Suggested Improvements Creation of testing conventions All of the language modules in particular should have a rigorous series of tests written for them...
BitterSuite Initial Setup The following options should be in the file /u/csXXX/.rstrc : default runTests /u/isg/bittersuite3/runTests default computeMarks /u/isg...
Marking Scheme Walkthrough Introduction One simple and easily overlooked element is a marking scheme. This will be a plain text file which will be dumped verbatim...
Plotting Scripts Created by KyleSpaans for the Winter 2009 offering of CS136. There are scripts that can generate an EPS imagine suitable for inclusion into OUTPUT...
Related Pages BitterSuitePythonModules BitterSuitePythonTesting HumaZafar 29 Mar 2011
BitterSuite Python Modules This is a list of what you can do with Python modules. Note that any restrictions or changes made by these modules can be overridden by...
BitterSuite Python Testing Getting Started: test.pt and test.1 folder layout Python testing using RST/Bittersuite is similar to Racket testing. You have to create...
BitterSuite Scheme Language Documentation Scheme in BitterSuite has a relatively complex back end, as it attempts to support a number of languages (the languages and...
BitterSuite Scheme Testing This page describes hints for testing in Scheme. Also see BitterSuiteScheme for a formal description of the language options. Related Pages...
RST/BitterSuite Testing Using Arbitrary Scripts RST/BitterSuite is a software testing framework which can use scripts that you write for testing. This allows for a...
Surveying Students A simple survey can be completed in BitterSuite. For example, the file provided to students may begin with: ;; Most of the editing you need to...
Testing Scheme with Mutation Whenever an evaluator is created, its state is mutated and passed along with the evaluator. So, if you have an evaluator created at the...
Validating Files Submitted by Students Note: Performing filtering in this fashion is no longer recommended. The BitterSuiteFilter language should be used instead...
Verifying C Code compiles without warnings This page discusses how to design tests with appropriate feedback which will verify that code submitted by students will...
Verifying Students have Posted to the Newsgroup A utility that will download posts from a newsgroup, newsarchiver.py , is attached to this document (the Wiki insists...
Verifying Submission Method This can be done with some support from the submit command. As there is no other convenient builtin method, it is possible to misuse submit...
Verifying that students have used the public tests This utilizes the algorithm described in AssignmentZero to force students to run public tests more than once: first...
Walkthrough for BitterSuite This walkthrough will go through the creation of a simple marking suite to illustrate the creation of a Scheme suite using BitterSuite...
Name bittersuite3 A framework for running tests via RST Description BitterSuite3 is a testing framework that works on top of rst . Instead of providing full...
Name bittersuite A framework for running tests via RST Synopsis rst Description BitterSuite is a testing framework that works on top of rst . Instead of providing...
Walkthrough For Everything to do with Assignments (with lots of reference to BitterSuiteWalkthrough ) For This Walkthrough, we will assume that all work is being...
CS246 Assignment 5 (Group Project) Walkthrough In recent terms, CS246 has had a partner final project. This means some extra handling for this final project. The following...
nop CheckTestcases In Fall 2012, CS 135 tried using a script to check the students` check expects. The script checks if the student has thoroughly tested their code...
Checking Helper Functions and Recursion This page assumes knowledge of how to do rst test setups for cs135 (TestingGuide), how options.rkt files work (OptionsRkt)...
Assignment Collection The main goal for assignment collection is to: Isolate the assignments that students submit before the due date from the ones that are submitted...
Resources and Information for CS 116 ISAs This twiki is for technical details only. Any other duty information can be found in Waterloo LEARN (D2L): CS Instructional...
Resources and Information for CS 135 Tutors NOTE: THIS PAGE IS OUTDATED AND MAY HAVE MISLEADING INFO. PLEASE SEE THIS INSTEAD: CompSci135TutorDuties This twiki...
CS135 Main Page The responsibilities of a CS135 Tutor may vary over academic terms depending on the number of students and instructor preferences. The general weekly...
Resources and Information for CS 138 ISAs This TWiki is for technical details only. Any other duty information can be found in Waterloo LEARN (D2L): CS Instructional...
Resources and Information for CS 145 ISAs This twiki is for technical details only. Any other duty information can be found in Waterloo LEARN (D2L): CS Instructional...
Resources and Information for CS 146 ISAs This twiki is for technical details only. Any other duty information can be found in Waterloo LEARN (D2L): CS Instructional...
Resources and Information for CS 231 IAs This twiki is for technical details only. WIP: The `Key Links` and `Course Webpage and Piazza` sections are currently just...
Resources and Information for CS 234 IAs This twiki is for technical details only. Any other duty information can be found in Waterloo LEARN (D2L): CS Instructional...
Resources and Information for CS 240 ISAs This twiki is for technical details only. Any other duty information can be found in Waterloo LEARN (D2L): CS Instructional...
CS 241 ISA Duties This page describes important tasks CS 241 ISAs must do throughout the term. The focus of this page is mostly on tasks that require technical knowledge...
Resources and Information for CS 241e ISAs This TWiki is for technical details only. Any other duty information can be found in Waterloo LEARN (D2L): CS Instructional...
Resources and Information for CS 246 ISA Please refer to the handbook for up to date information regarding CS 246. Please ask your ISC if you can`t access that page...
CS 247 ISA Duties This page describes important tasks CS 247 ISAs must do throughout the term. The focus of this page is mostly on tasks that require technical knowledge...
Name computeMarks Produce an output file for a given rst suite Synopsis computeMarks Description computeMarks is a program that must be written for each rst...
Connecting to Course Accounts The course account is where many files for the term are stored, and you likely will be making use of it on a regular basis. Often, it...
Providing Libraries and Helper Files This section should be your go to reference if there are any issues with providing a library or creating helper functions for...
Correctness Tests Correctness tests are used to completely test every students code. The mark they get on an assignment are heavily dependent on the results of the...
Course Account Training Materials, Fall 2010 Summary of Changes The old slides had a tendency to focus on particular testing/submission technologies. With a rapid...
Maintaining the Course Webpage General Tips Pick the best way to present information. Just because previous ISAs used a particular form of presentation doesn...
Name distrst Automatically distribute an rst run across course specfied servers. Synopsis Version 1: distrst {assignment} p f t c {suite} {`studentglob...
KarenAnderson 13 Nov 2009 (Made more general) KarenAnderson 18 Aug 2009 Here are a list of end of term tutor tasks (Feel free to add as appropriate) Anyone:...
Name file filter Perform filtering on files students submit Synopsis file filter Usage Inside of the course`s /u/csXXX/.submitrc there should be the following...
File Layout This diagram presents an overview of the file structure in the course account. Most of what you need is under the marking directory in the course account...
~/markus/bin/getstats.py Running getstats.py on an assignment will return a summary of the correctness results, test by test. Examples from Fall 2020, a10: Using...
ISG Account ISGAccountScriptConventions ISGAccountWebsiteConventions ISGAccountUpdatingProtocol Subversion The main portions of the ISG account are...
ISG Account Updating Protocol Because the account is maintained in a subversion repository, the main checkout should not be altered during the term. This requirement...
ISG Info Below is a list of pages on the TWiki pertaining to ISG related stuff. General ISGTechnicalDirection ISGWebUsageStatistics ISGFrameworkImprovePit...
ISG Scripts This should be the place to look for information about the various scripts supported by ISG, particularly for situations that aren`t covered by the information...
ISG Scripts Man Pages This page contains man pages for some of ISG`s scripts. These man pages used to be at https://www.student.cs.uwaterloo.ca/~isg/, but that website...
This document is an archive of a proposal that was developed early in Winter 2010. Implemented portions of it are reflected in ISGAccountScriptConventions. See also...
ISG Web Usage Statistics, 2009 Month: Topic views: Topic saves: File uploads: Most popular topic views: Top contributors for topic save...
Possible Improvements This is a general catalogue of suggestions for how this software may be improved in the future. Some pieces of software are given their own pages...
Name keepFile Request printing of a file generated by runTests or computeMarks . Synopsis keepFile {file} { maxlines option } option... Description...
Name mailOut Send students e mail with the results of an rst run. Synopsis mailOut {assignment} {suite} {run id} Usage The first parameter specifies which...
The script maketests was designed by Sean Harrap in the fall term of 2015. Minor edits were made by Ten Bradley in winter 2016. The goal of the script is to make developing...
Mark Schemes for RST public tests While a formal `marking scheme` is normally not needed for public tests as no handmarking of these tests will be done, there can...
MarkUs MarkUs is an open source software that lets students submit assignments online. Graders can mark the assignments online, and students can view their marked...
MarkUs: SVN Repositories, Groups, and Auto testing MarkUs manages the student`s submissions with Subversion (SVN) repositories. If you`re using MarkUs, someone...
MarkUs Installation Issues See also: MarkUs Modified Files The following files have been modified in our local installation: markus/app/views/main/login.html...
How to Prepare an Assignment for Marking in MarkUs Courses where students submit to MarkUs (including through Open edX) Some courses need to upload files to MarkUs...
MarkUs Scripts Please list scripts that have been developed for working with MarkUs, including where to find them and how to use them. If you cannot find the script...
Attempt #1 Command run: export RAILS ENV `production` cd ~/f10 dev/markus 0.9.0 rake db:create (in /u7/markus/f10 dev/markus 0.9.0) FATAL...
Markers Mimicking Marmoset A system to have markers provide initial online handmarking feedback while assignments were being submitted so they would be able to re...
Marking Meetings All information about prepping for, holding, and concluding marking meetings will be found on this page. Pre meeting Tasks Prior to the meeting,...
Marking Scheme The marking scheme should be in very simple language and concise. It will be read by TA`s for marking and students for feedback. Any detailed or verbose...
Documentation for the marm sql Script The marm sql script allows common Marmoset tasks to be performed from the command line. Users can download all the best on...
Name marmocreate Create a zip file for Marmoset that wraps an RST assignment Synopsis marmocreate o zip outfile {assignment} Description marmocreate...
Marmocreate Improve Pit Marmocreate is a program that wraps a collection of RST test suites in a fashion that can be understood by Marmoset . It has only been used...
Marmoset Related Pages MarmosetCourseConfiguration LocalMarmosetSubmissionMirror MarkersMimickingMarmoset PrintingMarmosetAssignments PostingMarks...
Marmoset Marmoset is a website that lets students submit their code and assignment online. When students submit, Marmoset will test the student`s submissions and tell...
Marmoset Testing Script Guide In the cs246t account, each project is usually organized as ~ bin/ this is where all the scripts are marmoset...
Basics of Running Moss This page shows the basic steps for running Moss. Your course may have scripts that combine these steps. Step 1: Put files somewhere in the...
Test Constants and Helper Functions (old) When a student has to define a structure in their submitted file it is hard to write test constants and helper functions...
Old MarkUs Material MarkUs has evolved quite rapidly. The material below accumulated during that evolutionary phase. There may be information that is still useful...
Uploading Marks to the course Webpage. Current script version: 1.3.4 The displayMarks.cgi file should be used to display current marks to students. It has been...
options.rkt There are two different kinds of options.rkt files. The one directly under the in directory contains the global options for the entire assignment. It will...
Preparing Public Tests See also: PreparingSubmitAtStartOfTerm PreparingPublicTestsForEachAssignment Listing allowable servers (required) The undergraduate...
Preparing Public Tests for Each Assignment See also PreparingPublicTestsAtStartOfTerm. This testing suite can be constructed and tested the same as any other RST testing...
Preparing submit/RST based assignments There are several steps required to set up an assignment in order to have logs kept properly and to allow students to receive...
Preparing Submit for Each Assignment See also PreparingSubmitAtStartOfTerm. Creating a submission log subdirectory Before each individual assignment is opened for...
Name printOut Print postscripts with the results of an rst run. Synopsis printOut {assignment} {suite} {run id} {piles} {where} Usage The first parameter specifies...
Printing Marmoset Assignments There are three steps to printing out assignments 1. Download the students code and grades csv files from marmoset; 1. Unpack...
Proofreading Assignments At least one week prior to the release of an assignment to students, instructors will release a draft to ISAs for feedback. Ideally, each...
ProofreadingTestSetups This is a checklist to go through after finishing test setups and before running tests on every student`s submission. Check Autotesting Setups...
Providing Libraries and Helper Files This section has been moved to the top of the ConstantsAndHelpers page. Outdated You should be aware of how the various language...
Explanation The following is a good example of a marking scheme for the RSTPublicTests because of the extra information it automatically pushes to students. It also...
Name pub test web A web interface for pub test logger Synopsis /u/isg/bin/pub test web Usage This is a front end for public test requests that is explicitly...
RST .stepheader { background color: #DCDCDC; font weight: bold; font size: larger; padding right: 15px; } How to Run RST rst is a command that you need...
RST Suggested Improvements See also: RSTVersionTwo MarmocreateImprovePit All RST related programs (rst, rsta, distrst, testcompRST, pub test runner, pub...
RSTPublicTests Suggested Improvements There seems to be an implicit nice applied when public tests are launched, probably due to nohup; however, this nice should...
RST Test Servers Initially, RSTPublicTests and DistributedRST unconditionally targetted only the Solaris 8 CPU servers. However, with an explosion of server families...
Remark Requests Opening remark requests This page will explain how to open the remark requests form on the website for a new assignment. 1 Navigate to ~/web/content...
Name rsta Generate answers to a solution before running rst . Synopsis Version 1: rst {assignment} p f t c {suite} `studentglob` run id Version...
Name rstrc Specify configuration parameters for rst Synopsis rstrc Description Configuration for rst may be provided in the files /u/csXXX/.rstrc , /u...
Markus Rubric How To Set Up the Rubric on Markus Log into Markus in the admin account (the account that you log in with your email). Go to the current assignment...
Name runTests Run a series of tests for a given rst suite Synopsis runTests Description runTests is a program that may be written for each rst suite. If...
#!/usr/bin/env bash # Assumes $testdir has a directory `in` containing a sub directory # for each test case. Each subdirectory has files: # .desc description...
Sample .submitrc The following are contents of a sample .submitrc file: submit version 2 # Account information homeac `cs125` testac `cs125t` # Directories and...
Providing code to capture and report exceptions The following module provides a single function whose purpose is to capture and report exceptions. This should be...
Comparing Inexact Values Before discussing this, the first rule is: avoid dealing with inexact values if at all possible. Scheme has the ability to deal with rationals...
Comparing Unknown Structs Sometimes students are asked to write their own structure definitions instead of importing them from a module/teachpack. This can cause...
Creating Test Data from Input Files There are cases in which you wish to test a large or complex data structure, and it is difficult or just irritating to supply it...
Creating Instances of Student Defined Structures If you are going to require students to use a particular structure (for example, to represent nodes of a tree), the...
Disallowing Specific Functions It would be handy to be able to set! a function such as reverse to an expression which raises an error in a module. PLT Scheme does...
Comparing lists, ignoring internal ordering The case frequently arises where you want to compare two lists, but don`t care about internal ordering within the lists...
Limiting CPU Time BitterSuite already has a mechanism for limiting the amount of real time a test takes. However, sometimes a finer grain of control is desired on...
ISG Script and student.cs Environment Information for the Fall 2010 Term This page is meant to give brief information to ISCs and instructors about programs and/or...
Scripts Here are the scripts we use in CS 135. Test Setup Scripts Deep Copy Dan`s Disallowed Cameron`s Whitelist cs135 tracing library Marker...
Explanation The following mark scheme is meant to be interpreted via bash, as is done optionally in BitterSuite. It also contains directives that are intended to be...
Special Cases and Common Issues This covers miscellaneous helpful topics and general tips that can be used while setting up public/private tests. CommonIssues Many...
KarenAnderson 22 Jul 2010 General end of term tasks for Spring 2010 (feel free to add) IOI setup help Fri. Aug. 13 (contact: Barbara (bmzister)) IOI pack...
Stepper Questions This page contains information about how to set up stepper questions from edX or the course website for autotesting purpose. Steppers Through edX...
Submit course information This page gives administrative details about courses that make use of Submit/Odyssey for submission, and some combination of RST, BitterSuite...
Submit Permissions Hook This is available via the final script option in .submitrc , and is located at /u/isg/bin/submit/submit permissions hook . It simply changes...
Avoiding certain hosts for any sort of work Do not use student.cs for any sort of development or testing. Invoke hostselect listup undergrad to get a list of...
Setting up TA MarkUs and Assignments To help TAs be familiar with each assignment and ensure high quality marking, a subset of each assignment...
TestTests This script was written in Fall 2012 and first used extensively in Winter 2013. It automates the evaluation of student test cases so that grad TAs do not...
Testing Guide If this is your first time reading through the testing guide, please read the full summary below. Quick Links CorrectnessTests information on creating...
Miscellaneous notes This section has been moved to General FAQ under Common Issues HERE. Outdated Why is there a message `Evaluator creation error: FAILED Error...
Instructional Support Group Wiki Welcome to the home of TWiki.ISG . This is a web based collaboration area primarily targeted for ISA use, which will allow better...
Statistics for ISG Web Month: Topic views: Topic saves: File uploads: Most popular topic views: Top contributors for topic save and...
This page is meant for feedback about ISG`s scripts at the end of Winter 2010 to help ensure that the modified versions are ready to go in Spring 2010. The goal is...
Testing ISG Twiki website Testing Added a logo uwlogo.png:
Overview Marmoset is a testing suite operated through a web interface (for both ISG and students); it provides an easy way to create programming problems and assign...
Number of topics: 153
 |
|
Copyright © 2008-2025 by the contributing authors. All material on this collaboration platform is the property of the contributing authors. Ideas, requests, problems regarding TWiki? Send feedback