Survey Analysis

In the Fall 2012 term, the CSCF Technical Writer conducted a survey to research the requirements that CSCF staff had for Twiki. This page summarizes the results of the survey and lists other suggestions and opinions that were given.

Approaches

Text

Levels of Details

• FAQs: common failures, emergency type questions, etc.
• Tutorials and How-Tos
• Architecture-Design-Type Documentation
• Reference manuals
• FAQs can link to tutorials, tutorials can link to architecture-design-type documentation, etc.

• E.g. Markus
• Cover page
• Sub-categories

Web Interface and Text

• Includes a list of all components, the status of each component, etc.
• Provides a quick way to see if the system is operating properly, and if not, which parts.
• E.g. telephone lab: the components include servers, daemons, switches and all the telephones
• Requires CGI (computer generated images) web info

• Ping the machines (desired result: echo response) http://watcher.cscf.uwaterloo.ca/cgi-bin/smokeping.cgi?target=DRCSCS.ICMPEchoPing.RegionalMasters.Core
• Network test

• E.g. https://cs.uwaterloo.ca/cscf/services/

Information Sources and Current Content

Documentation Locations

• Twiki
• ISG guides and tutorials: https://cs.uwaterloo.ca/twiki/viewauth/ISG/MarkUs
• UserSupport: https://cs.uwaterloo.ca/twiki/view/CF/UserSupport
• eDocs: Almost all the contents are outdated https://cs.uwaterloo.ca/cscf/internal/edocs/machines/markus002.student.cs.uwaterloo.ca/ (it would be nice if all of the links on that page were clickable) https://cs.uwaterloo.ca/cscf/internal/edocs/machines/toc.shtml
• CSCF Public https://cs.uwaterloo.ca/cscf/ https://cs.uwaterloo.ca/twiki/view/Trash/CFMarkUs
• CSCF Internal https://cs.uwaterloo.ca/cscf/internal/
• IST training material: http://ist.uwaterloo.ca/
• Sharepoint
• S.T.: https://cs.uwaterloo.ca/cscf/internal/ST/beta/

Personnel

• Points of contact
• Teaching staff
• All USG members
• Daniel, Trevor

Current knowledge of Marmoset/Markus

• Nothing
• The information from the "show and tell" by Omar
• Web server parts of Markus
• Markus is used for submitting assignments and auto-testing

Format

Source

Drupal

• Blocks input of mistakes
• Allows for easy expression of complicated things and display of HTML tables across multiple webpages
• Automatically applies changes to all instances

• Easily publish content as HTML
• Hard to search
• Limitation: requires special software
• Easily display images

• WYSIWYG - what you see is what you get, including text and pictures
• Supports source code view and webpage view

Komposer

Twiki

• Priority for the documentation is information sharing to train staff and to avoid duplicating work
• No strict requirements for hardware, servers, software, browsers, or any other tools, e.g. Even PDA's are compatible.
• Easy to learn markup language: minimal stylistic elements, e.g. verbatim, creating tables and new pages, total page history
• Has file upload function. Avoid attaching files if you can because opening them significantly slows the readers down.
• "Manage" and "Make table visible" functions allow for revisions without starting from scratch
• Will not be affected by the move to Drupal

HTML

• Flexibility: can be displayed as other formats and no browsers are required
• Search: meets all search criteria

• Easy to move around, but hard to search

Output:

HTML

• Flexible: can be displayed as other formats, e.g. PPT

Organization

Determine Relevance

Synopsis

• I.e. "rhetoric of arrival"
• Provides the context, the purpose, and a summary of the documentation
• Indicates the intended audience: anyone who is interested; in the past, students have learned about various systems from the documentation, reported problems, and suggested solutions.
• Before the table of contents on each page

• General and non-technical info
• E.g. the answer to "what does Markus do?"
• Nice example: https://cs.uwaterloo.ca/twiki/view/CF/OwnCloud#Background

• Reveals the author's approach to organization; helps readers find the topic they need
• Twiki's tables of contents created with "% T O C %" suffice
• Create "hover-reveal" tables of contents in HTML and hyperlink each word

• Consistent: e.g. problems and solutions
• Descriptive: leverage CF "Topic Title" search option

• Descriptive hyperlinked text e.g. ISG's detailed notes, CSCF, and developper
• A comment next to each link that specifies the context, the intended audience (anyone who is interested) and a summary of the documentation
• Unique URLs for disambiguation: e.g. https://cs.uwaterloo.ca/twiki/view/CF/OwnCloud#Windows_client is better than https://cs.uwaterloo.ca/twiki/view/CF#Windows_client

Quick Access

Cross References

• Between tasks sharing common portions
• Between documentation in different locations: EDOCS, CF Internal, etc.
• Having references on both pages helps prevent broken links.

• Can be useful, but currently too many topics are misplaced and there is not much depth to Twiki topics.
• This feature is already available in Twiki. Breadcrumbs are automatically created with the addition of new Twiki pages.
• Might be useful because they can indicate if you are close to the topic you need.

• Top of the Page: Place troubleshooting information at the top of the page
• Emails: Notify colleagues of new documentation about difficult problems

• Indexes' function is keyword search; something Google is better at anyway. Creating an index for an entire website is too time consuming.
• As an alphabetical keyword search function, indexes are useful. Actually call indexes "indexes"; not keyword search.

Google Search

• Google search replaces the need for an organization standard
• Descriptive keywords facilitate Google Search

• By making the primary audience very specific: teaching support, faculty support, courses, etc.

• Only use broad categories: authentication and time - private vs. public and current vs. archive (conflicts with the above).
• Contributors are encouraged to make as much information public as possible. One benefit is that makes the information searchable by Google. However, sensitive information should be placed in CF Private.
• All information that involves troubleshooting or requires passwords which the general public does not have access to should be placed in CF Private.
• Minimize the number of clicks (this and the above may conflict: make judgement calls). Search a keyword, e.g. Markus, and place links to the results on the main Markus page.

• CF / Twiki Search: https://cs.uwaterloo.ca/twiki/view/CF/WebSearch?search=markus&scope=text
• Advanced CF / Twiki Search: https://cs.uwaterloo.ca/twiki/view/CF/WebSearchAdvanced
• Crucial for trouble-shooting because it allows for the quickest access to relevant info
• Documentation would not be of any use if it wasn't searchable
• Should be complemented by bubble webs

Brevity

• Without sacrificing the content, don't use more words than you have to. E.g. "The command is LS" is appropriate, whereas "Type LS, hit enter" is excessive since it would be tacitly understood anyway by a technical audience.
• Bubble webs facilitate simplicity
• Priority: providing information needed to solve the problems

Needless Features

• Top ten troubleshooting issues
• Progress bars
• Checkboxes: recording the progress in text files, the S.T., or the web browsing history would be more convenient
• Sitemaps: sitemaps are supposed to be dynamically generated, but this is difficult in practice.

Maintenance

Content

Tasks

Troubleshooting

Problem Scenarios

• Symptoms, e.g. error messages

• Questions to ask the client requesting service: lab`s room number, student ID, machine name and version, date and time when the problem occurred
• Is there a testing environment I can use to help diagnose the problem?

• A server, an app, etc.

• Rebooting and other solutions
• Detailed steps including scripts and commands if applicable, similar to the content of S.T. Thoroughness is very important: if a few key details are missing, that could render the entire document worthless and the reader would have to figure out the solution from scratch.
• Warnings

Deployment

Software Installation and Configuration

• Locations of the software and the source code; locations of the download files if applicable
• Credentials: open source? license location?
• Authors: was it created by internal staff?
• Admins: who is responsible?
• Programming language
• How to update it
• Detailed how-to articles with detailed steps
• Info about deployment is not needed because it only needs to be done once

Hardware

• Machine description: exact name, version, date of retirement (if a/p)
• How to start the machine

Components for All Tasks

• Detailed steps: commands if applicable
• Warnings
• Notes: Estimated time and independence of tasks

Services

Apps

• General printing and monitoring
• Exam seating and printing system, e.g. how does the master copy of the exam go into the system? How does info get merged? Describe interactions with other departments: ISG, graphic services, etc. with a diagram.
• Email services for admin and students
• Admin support
• Mac, Windows suport
• Graphics lab
• Web
• Software packages
• Outlook and Exchange
• Creating mailing lists for admin and students
• Numerical support
• Maple
• Matlab
• Setting up accounts: Undergrad enrollment and adding students and instructors to groups
• Encryption keys
• Authentication: how to look up, change, and reset the usernames and passwords
• Various places in the system
• For the exam seating system

• The existing documentation is very outdated

• Postgre SQL
• MySQL
• DBL

Most Important Purpose/ Function of the Documentation

1. Provide solutions to emergency-type problems
2. Share information: therefore it is crucial that accessing and editing the documentation is convenient and easy to learn. Twiki's lack of fancy stylistic elements and nice pictures is not a problem.
3. We have enough information; the biggest concern is to convert the data into a more visual form so that the desired information will be easier to find.

Most Useful Improvement

Content Addition

• Document previously undocumented knowledge
• Troubleshooting (problems that are expected to recur: frequently appear on S.T.): what software is running, how to become an admin, and how to add an account etc.

• There is sufficient information; we just need to use data visualization to make desired info easier to find.

• A starting point is to create a web of bubbles with topic titles in their centers. Use this web to give a comprehensive overview of the content of the documentation in Twiki. In this vision, each bubble is clickable and links to a new web, where the original bubble becomes the central bubble branching off (lines) to subtopics.
• Cross-referencing: there should also be lines linking bubbles with related topics.
• Breadcrumbs function: a bubble labelled as "back" in each web, leading to the upper level web
• Shapes: can be used to indicate the categories of info, e.g. hardware, software
• Colours: e.g. green and read, can be used to indicate whether the content under this topic has been edited recently.
• Sizes: can indicate the frequency of access for each topic, similar to what word clouds do. There is a Linux based software that creates word clouds. Facebook's word cloud app may be open source.

Explore different ways to visualize the info in the documentation. Make a note about these methods in the documentation as "future considerations".

• CSCF periodic table of documentation: each entry in a given row should share a common property, so should each entry in a given column. E.g. Separate rows for staff, faculty, and clients. There are other organization tricks embedded in the Periodic table that should be applied to the CSCF documentation.
• A soccer-ball-like rotatable sphere with pentagon faces. Each pentagon should contain a topic title and be surrounded by related topics.

Up to Date

Display

• Last five edits for each document
• What changed when: last week, last month, and last year
• version history's flaws: only the update time is needed, not the differences between the versions.
• Information that is not changed often could either be perfect or irrelevant: discretion is needed.

Authors

• Display a particular person's work: creations and edits

Other Improvements

S.T.

• Place into Twiki a duplicate of the information in S.T.without the false starts: only descriptions of the problems and solutions
• Condensing an entire duplicate of the content of S.T. will be too much work. If you decide to duplicate and then rework parts of the content from the S.T., it is crucial that you leave the original as is.

Format Conversion

• Convert all EDOCS files to HTML using a PHP script: reduces time spent opening files. Warning: leave the originals as backup.

Communication

Wording

• Avoid uncommon or undefined abbreviations and jargon
• An arrow "->" replaces the word "click"

Visuals

Screenshots

• Needed only if a visual feature is tricky to find
• Don't need to be large or appear frequently for a technical audience
• Nice example: http://ist.uwaterloo.ca/cs/wireless.html#toc-connecting-ubuntu

Flowcharts

• Useful for troubleshooting if many decisions are required and the text is short
• Too high maintenance
• Are less effective than using a table of contents (indented structure) and many good headings
• Numbered steps suffice for troubleshooting, and especially for deployment, which is a very linear process
• Not as useful as bubble webs

Contributors

For Clients

Clients should feel free to participate in documentation writing

Content

• Provide only limited configuration info: setting preferences
• Policy info: forbidden activities, access to shared accounts, etc.

• Word
• PDF
• HTML webpages are effective
• Follow CS formatting standards: information for, information about

Survey Questions

I would like to start off by telling you a little bit about why I'm doing this survey. The goal of this survey is to find out how to best write documentation for the teaching-related services supported by the User Support Group. Our focus is going to be on Markus, Marmoset, and ISG Scripts. So the information that you provide will just be used to develop a set of criteria for the documentation, and not for any other purpose, and all your answers will be completely anonymous when we put them up on Twiki.

1. With this first question, I want to find out if the existing documentation is covering the right topics and the right amount of details. So if one of the people that you are a point of contact for is having a problem with one of the systems, and someone who usually works with them, let's say Omar isn't around to help, where would you look to find the necessary information? Be specific.

2. In the scenario that we just talked about, are there any particular people that you would ask for more information? Who would they be?

3. What sort of things about these services would you like for the documentation to contain?

4. With the next question, I want to get a sense of how effective our current documentation is. Could you tell me what do you already know about Markus, Marmoset and ISG's ~isg/bin scripts (on linux.students.cs) ? Could you tell me where you currently get this information?

5. Besides instructions for specific tasks and trouble-shooting, what kinds of background information would you like to see about these systems ? How in-depth should this background information be?+

6. I know that we already have some documentation on Twiki, and I would like to know which parts are useful, and which parts are not, so could you tell me in your experience, which types of the information on Twiki do you consult often and which types do you rarely consult?

a. Installation and configuration procedures

b. Troubleshooting

c. Areas of improvement for Markus and Marmoset

d. Auto-testing

e. Adding assignments

f. Creating and editing marking schemes/rubric's criteria

7. What do you feel is the best sort of format or media for the documentation to be available in? (e.g. PDF. Microsoft Word, HTML)? How to do you feel that the documentation should be written (e.g. MS Word, HTML, Latex)

8. How do you prefer for the documentation to be organized? Should the topics about installation and configuration tasks follow a chronological order? Should topics about troubleshooting be grouped by the features to be troubleshot?

9. In terms of organization, would you prefer for the contents to be organized using a comprehensive table of contents or a sitemap? In your experience, which of the two helps locate information more quickly?

10. What organizational elements do you find are the most useful for helping you navigate documents? Should the documentation have breadcrumbs to help improve navigation? What about internal links for cross-referencing information about related tasks? Should it have a section that contains the ten most frequent tasks to be performed and issues to be troubleshot?

11. Would an index be good for navigating the document? In your experience, are flowcharts beneficial for illustrating long procedures?

12. Should the documentation be searchable?

13. Could you give me an idea of what sort of interaction with reading materials help you engage with them? Would check boxes and progress bars make it easier to follow through the procedures instructions? To what extent do you think snapshots of Markus and Marmoset should be used to illustrate the setup procedures?

14. How brief would you like the documentation to be?