This is a repository for useful information for programmers who are new to CSCF.

Overview

Welcome to CSCF. Here are some hints, tips, and guidelines.

First Day

After your supervisor hires you in Workday, you should be able to log on and fill out any required forms as indicated.

Also, complete the following training modules:

• Employee Safety Orientation
• Workplace Violence Awareness
• Accessibility Training

Basic Setup

Microsoft Teams

• Log in to Microsoft Teams at https://teams.microsoft.com/ using your userid@uwaterloo.ca account.
• Let your supervisor know, and they can then request you're added to the CSCF team.

TWiki

Lots of our documentation is stored in this twiki.

• Create a TWiki account with the display name FirstnameLastname, and login username as your Quest ID.
• If you're unable to edit CSCF pages (such as this one), ask your supervisor to grant your account access.

Request Tracker (RT)

• Log in to RT with your Quest login information.
• You should be able to see CSCF tickets in the queue list. If not, ask your supervisor to grant you access.
• We'll begin with an interactive session / demo and you can further familiarize yourself with the RT documentation here.
• The following setting changes are useful (in the top-right menu "Logged in as User -> Settings -> Preferences"). Change the following to "yes" from "Default (No)"
  • Remember default queue
  • Use autocomplete to find owners?
  • WYSIWYG message composer
  • Display plain-text attachments in fixed-width font

git.uwaterloo.ca

SSH Key

• Log in to GitLab with your Quest login information
• Click on your user icon and go to 'Settings'.
• In the left sidebar, click on 'SSH Keys' and follow the linked instructions on generating an SSH key (under 'Add an SSH key').
• On the same page, follow the instructions under 'Key' to add your public key. The key should then be visible at the bottom of the page.
• If you're working on odyssey, give your supervisor your public key so they can grant you access to the odyssey-test server.

git setup and campus security

• Most of our projects involve Personally Identifiable Information (PII) in some form or another, which is critical we keep private.
  • examples: client names, emails, student and staff ID numbers.
• Best practice is not storing PII locally at all, and using fake data where possible. If this isn't possible, you have a duty to keep the PII private.
• If you receive a .sql file with data to import, and it isn't part of a git repo, there is probably a good reason for it staying out of git.
  • Similarly for config files containing secrets such as passwords.
• You must ensure that PII and secrets are not checked into a git repo and pushed to git.uwaterloo.ca
  • Best practice is deleting the .sql once it's been loaded. If you need to store it, keep .sql containing PII out of any directory kept in a git repo
  • If the PII needs to go within a directory that is checked in to git, you must add it to .gitignore
  • every time you git add, confirm that you're avoiding PII data
• If you add PII to a git repo, or you discover PII or other secrets in a git repo, let your manager know.
  • the remedy will probably involve replacing the repo with a copy that does not include that commit; or if it's secrets such as passwords, changing them where ever they are used.
  • And if there is any chance PII has been downloaded outside our control, we have to have conversations with the campus privacy office

SCS "CS-General" account

• A linux account has been provided on linux.cs.uwaterloo.ca - you can set the password by following https://cs.uwaterloo.ca/password/ (and the "Change my password" link)

SCS VPN

Connecting to the SCS VPN allows you access to campus resources as if you were on the CS network.

• Setup instructions can be found here: https://vpn.cs.uwaterloo.ca/
• If you're working on odyssey, and if your SSH connection request or browser connection to odyssey-test.uwaterloo.ca is timing out, check that you're connected to the VPN.

authman public key management

• once you have CS VPN working, and you have a SSH public key generated for your userid, you can upload it to https://authman.uwaterloo.ca/, which will allow us to grant you access to various linux servers or accounts.
• Follow the instructions on that page to upload your public key
• read-access to the key will be available at the URL https://authman.uwaterloo.ca/keys/your_userid

Other training

• We've collected training materials for Linux and related topics that you may want to review.
• Some of the code bases we are working on are complex. Your supervisor will be happy to give you a "code tour", after you've had a chance to take a look. If he forgets to suggest it, please feel free to ask!

VS Code

Install VS Code. Add these extensions:

• Remote Development Extension Pack (includes 3 extensions)
• Git Extension Pack (includes 5 extensions)

And add whatever language-specific extensions it asks you about.

SSH and Virtual Machines

If you're going to be developing remotely, you will need to select Remote-SSH: Connect to Host... from the VS code command palette (f1 or ctrl+shift+P). Then, you will need to input something along the lines of user@hostname. The hostname can be the IP address of the VM, and the user will be the name of the user you're using on the VM.

For example, if you are working on the SAT, the VM's IP address (at the time of writing) is 192.168.10.10 and the user is vagrant. So, you would type vagrant@192.168.10.10 and VS code will ssh into the VM.

If you've set up a VM in Virtuabox with networking via NAT (simple default), with a port forward from SSH on the virtual machine, the VM's hostname is localhost (IP address 127.0.0.1) additionally specifying a forwarded port, which will require additional configuration.

You should verify that you can ssh into the VM before you try to connect in VS code. You can do this by running ssh user@hostname from a terminal, replacing user and hostname with the appropriate info as explained previously. Specify a forwarded port with '=ssh -p nnnn user@hostname=' If the VM doesn't let you connect, make sure you've added your public ssh key (from your local machine) to the VM.

See this documentation for more details.

Additional notes:

• If a SSH connection cannot be established between your local machine and virtual machine, try tweaking the firewall settings by following up to step 4 from the following link: https://www.digitalocean.com/community/tutorials/how-to-set-up-a-firewall-with-ufw-on-ubuntu-16-04.

• Remember a 64 bit linux machine is required for remote SSH connections. A 32 bit linux virtual machine does not support these features.

• If an authorized_keys file is not automatically generated when a key pair is created, you must create your own to be able to add your local machines public key. To do so, follow these steps.

inside your .ssh directory :

• touch ~/.ssh/authorized_keys
• chmod 600 ~/.ssh/authorized_keys
• echo "insert_public_key_here">> ~/.ssh/authorized_keys

• For remote debugging in VSCode, you must customize the launch.json file for first time use. In this file, set your port numbers correctly. If you are having trouble with this, ask your supervisor as to what a correctly configured launch.json file should include.

Mac Environment Setup

You'll need a package manager. We recommend Homebrew. Follow the instructions on their site for installing into your user directory:

   /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

To install third-party libraries easily, install pip by running the following command:

   curl https://bootstrap.pypa.io/get-pip.py | python

Depending on the projects you'll be working on, you may be installing a Linux virtual machine, which means you'll proceed as directed by your manager (probably under "PHP" or "Python" below).

Linux Setup

Ubuntu 20.04 (LTS) should be mostly "good to go." However, for video services, we need to install ffmpeg packages.

$ sudo -s
[sudo] password for (user): 
root@RSG-PC280:~# apt-get install ffmpeg
[ many packages are displayed]
After this operation, 179 MB of additional disk space will be used.
Do you want to continue? [Y/n]
[say 'y']
[...]

Additionally:

• apt-get install openssh-server
• You should set up a firewall. A simple option is 'ufw'.

Commands you should run below come after the '$' character, and is bold. The command output, not bold, should match as below. Any differences should be confirmed with your manager, as they might open connections from the wider internet.

The commands set up a default deny (DROP) firewall for incoming connections, with all outbound connections allowed with state tracking. It adds SSH connections from the wired campus public network, and web access to both HTTP and HTTPS from the three campus networks (public and the two private).

• More "Uncomplicated Firewall" (ufw) options are described in Ubuntu firewall Server-Guide.

$ sudo ufw default deny incoming
$ sudo ufw default allow outgoing
$ sudo ufw allow from 129.97.0.0/16 to any port ssh
$ sudo ufw allow proto tcp from 129.97.0.0/16 to any port 80,443
$ sudo ufw allow proto tcp from 172.16.0.0/16 to any port 80,443
$ sudo ufw allow proto tcp from 10.0.0.0/8 to any port 80,443
$ sudo ufw logging on
$ sudo ufw enable
Command may disrupt existing ssh connections. Proceed with operation (y|n)? y
Firewall is active and enabled on system startup

$ sudo ufw status verbose
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), disabled (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
22                         ALLOW IN    129.97.0.0/16             
80,443/tcp                 ALLOW IN    129.97.0.0/16             
80,443/tcp                 ALLOW IN    172.16.0.0/16             
80,443/tcp                 ALLOW IN    10.0.0.0/8                
$

The rest of the Linux setup is broken into whether you're going to be principally working on PHP or python applications.

PHP

See detailed notes about setting up Linux, Apache, MySQL, PHP, PHPMyAdmin, and a developer environment complete with debugging tools.

• If you're working on "Inventory" also see InventoryInstallation
• If you're working on Faculty Recruiting, also see FacultyRecruitingSystem

Python

If you're working on the Odyssey web application, see OdysseyCoopGuide .

If you're working on the Standardized Administration Tool (SAT), you'll be installing a virtual machine from instructions starting with the co-op guide at https://git.uwaterloo.ca/cscf/people-app

For other python dev, the following ubuntu packages will be a good start for developing in Python.

• Install VS Code https://code.visualstudio.com/docs/setup/linux#_debian-and-ubuntu-based-distributions

• Install git, psql, mysql, python3-pip, phpmyadmin, phppgadmin, pyscopg2, python3-memcache

  
          sudo apt-get update && sudo apt-get install git postgresql phppgadmin mysql-server phpmyadmin python3-pip python3-psycopg2 python3-memcache

  - You will receive prompts to setup mysql root user and phpmyadmin server type (apache2) and admin user
• Install python modules django 1.6, tastypie, south, requests, yaml, prettyTable, numpy

          sudo pip install django==1.6 django-tastypie south pyyaml prettyTable numpy
  
          cd django-cas && sudo python setup.py install 
  
          git clone https://github.com/joequery/requests-sslv3
  
          cd requests-sslv3 && sudo python setup.py install 

References

• HelpDeskGuide
• NewCoopStudents
• Useful Linux training materials
• InventoryInstallation - walks through setup as a new programmer on Inventory (PHP) with many tips on programmer environment.
• OdysseyDocumentation - front page for Isaac's projects/framework.
• ProgrammingStyleGuidelines
• StaffStuff

Chirag's TODO list:

• A good explanation of what MVC how it should be structured. A few good posts to reference:
  • http://stackoverflow.com/questions/2630758/whats-the-difference-between-application-layer-and-business-logic-layer
  • http://stackoverflow.com/a/391562/1406888
  • http://stackoverflow.com/a/1023548/1406888

Extras:

• explanation/guide how to set up pages to be viewable to others--very useful knowledge to be able to show clients the current progress of the software.
• Very brief section describing the network (i.e. what has access to the outside world, doesn't need to be detailed, but is helpful)

-- DanielAllen - 2015-01-30