Odyssey Introduction session (DRAFT)

Prerequisites

You will need the following:

• Git
• Python 3.6
• PostgreSQL
• pip
• JDK (if you're working with the java code)

Note: for simplicity, there's a Docker container that will have everything installed for you already!

Documentation

You can find documentation for local python modules through the following steps:

1. Go to https://odyssey.uwaterloo.ca/system/
2. Click "Overview"
3. In the Details section, under Coding Standards, click "Python"
4. Finally, click "Sphinx" for formatted documentation of most python modules

The final link should be the following: https://odyssey.uwaterloo.ca/system/overview/lang-python/sphinx/

Setup

Checkout the codebase with git (you will need access to the cscf group on git.uwaterloo.ca):

git checkout https://git.uwaterloo.ca/cscf/odyssey.git

Python setup:

1. Download requirements.txt
2. Run pip install -r requirements.txt to install all required external python modules

Postgres setup:

1. Get access to a database instance
2. Create a .pg_service.conf file in your root. A sample file can be found here
3. Create a .pgpass file in your root. A sample file can be found here
4. Set your environment variable $PGSERVICE to a service name in your .pg_service.conf file (such as dev_db or prod_db in the sample)

Java setup (optional):

1. Copy the jar files on production in /u/odyssey/a/java-jar
   • The jar files you should copy are iText-2.1.7.jar, postgresql-9.2-1002.jdbc4.jar, odyssey.jar
2. Add the jar files to your $CLASSPATH

Running

Running the app:

1. Download the following script to run python modules: python-run
2. Set your environment variable $DEMO_MODE to yes
3. Inside trunk/python, run [path to above script] uw.local.teaching.bin.main_web
4. Go to http://0.0.0.0:8080/your_userid/

In demo mode, always place your userid after the domain name and type the path after like so: http://0.0.0.0:8080/your_userid/schedule

Python-ism

Python has unique syntax that is not common among other programming languages such as Java, JavaScript, or C. The following section explains code that may be confusing to developers not too familiar with Python.

Inheritance

Python classes can inherit from a base class. The syntax looks like the following:

class ClassName(BaseClassName):
    ...

You can read more about inheritance here.

First-class Functions

In Python, functions are first class objects. They can be passed as arguments just like numbers or strings, or defined inside other functions. In addition, Python allows us to return functions from other functions.

Decorators

Decorators are functions that wrap a function and modify it's behaviour. Consider the following code:

def decorator(any_function):
   def wrapper():
      print("Something is happening before any_function is called")
      any_function()
      print("Something is happening after any_function is called")
   return wrapper

def main_function():
   print("This is the main function!")

main_function = decorator(main_function)

The code above modifies the behaviour of main_function. If you now call main_function, instead of printing:

This is the main function!

It will now print:

Something is happening before any_function is called.
This is the main function!.
Something is happening after any_function is called

Python offers a short syntax notation for using decorators. The code above can be simplified to:

@decorator
def main_function():
   print("This is the main function!")

The python runtime environment provides built in decorators. Some of the most common decorators are @classmethod and @staticmethod. The @classmethod decorator is used to define class methods, and the @staticmethod decorator is used to define static methods.

Web Framework

WSGI

We use a custom web framework that supports WSGI, an interface specification between web servers and python web applications that promotes portability across a variety of web servers. A WSGI application is a python object that gets passed two parameters: an environment which is a dictionary containing environment variables and a function that starts a response. The function that starts a response accepts two required positional arguments, and one optional argument. The two required positional parameters deal with the response status and response headers, while the optional parameter deals with sending error messages to the browser. We use the package flup to start a WSGI server, and the package web for local development.

A more detailed explanation on WSGI can be found in the official spec.

Usage

To add another endpoint, add a handler in the handler dictionary found in wsgifunc.py. The key in the dictionary is the path, and the value is a handler (function) that returns (1) the title for the webpage and (2) html that is inserted in the middle of the page. Parameters of the handler are read from the environment, by default cursor is a required parameter. Handlers are typically annotated with @delegate_get_post, @use_remote_identity, and @return_html - though these may change depending on the usage.

Impersonating Users

To impersonate a user in the ISA application head to http://localhost:8080/ where a website with an input box named Userid will be rendered. Upon entering the ID of the user, the browser will redirect to the most appropriate menu depending on the type of user logging in.

To impersonate a staff member, the staff user id is required. An example of a staff member id is "fesdunba". Impersonating this user will let you access the staff main menu, which differs from the main menu viewed from a student account.

Note: Web server configuration must be on Demo Mode.

Gotchas

Common gotchas and pitfalls when working with odyssey:

• When running locally do not include /teaching in the path, in production the URL will be https://odyssey.uwaterloo.ca/teaching/schedule but locally it will be http://0.0.0.0:8080/schedule (but in demo mode the URL will be http://0.0.0.0:8080/your_userid/schedule)
• On a new database instance, you might see no tables shown due to your search_path. Check the list of schemas in pql with the command \dn, and update the search_path with the following SQL statement ALTER ROLE your_role SET search_path TO schema1,schema2,...;. Restart psql to see your changes
• On Mac you will have to add .uwaterloo.ca to your hosts in .pg_service.conf and .pgpass, but on Linux you can optionally remove these (due to DNS search-rule differences in Linux)

Security

Restricting Port Access (For non-Mac OS)

If your webserver is on an unrestricted network (such as 129.97.171.x ) you need to lock things down, or the rest of campus can make requests to the web server on port 8080. In order to limit access to only a subset of the university we must execute the following code on the terminal:

sudo iptables -A INPUT -i lo -j ACCEPT
sudo iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
sudo iptables -A INPUT -p tcp --dport ssh -j ACCEPT
sudo iptables -A INPUT -p tcp -s 129.97.15.0/24 --dport 8080 -j ACCEPT
sudo iptables -A INPUT -p tcp -s 129.97.170.0/23 --dport 8080 -j ACCEPT
sudo iptables -A INPUT -j DROP 

Now, to make the changes permanent: 1. Run the following command on the command line:

sudo sh -c "iptables-save > /etc/iptables.rules" 

2. Run the following command on the command line:

gksudo gedit /etc/NetworkManager/dispatcher.d/01firewall

This will open a text editor, where you will paste the following commands, then save and exit:

 
if [ -x /usr/bin/logger ]; then
        LOGGER="/usr/bin/logger -s -p daemon.info -t FirewallHandler"
else
        LOGGER=echo
fi

case "$2" in
        up)
                if [ ! -r /etc/iptables.rules ]; then
                        ${LOGGER} "No iptables rules exist to restore."
                        return
                fi
                if [ ! -x /sbin/iptables-restore ]; then
                        ${LOGGER} "No program exists to restore iptables rules."
                        return
                fi
                ${LOGGER} "Restoring iptables rules"
                /sbin/iptables-restore -c < /etc/iptables.rules
                ;;
        down)
                if [ ! -x /sbin/iptables-save ]; then
                        ${LOGGER} "No program exists to save iptables rules."
                        return
                fi
                ${LOGGER} "Saving iptables rules."
                /sbin/iptables-save -c > /etc/iptables.rules
                ;;
        *)
                ;;
esac

3. Run the following command on the command line to finalize:

sudo chmod +x /etc/NetworkManager/dispatcher.d/01firewall

Reboot the computer to test the firewall is still active. Run the following command to check:

'sudo iptables -L'

It will return 7 or more lines in the INPUT section if it was successful. If INPUT is empty, there was a problem requiring debugging.

SSH Port Forwarding

To access the database from outside the work network (such as from home), you can use SSH port forwarding to gain access to the database from a server that does - like your linux.cs account. Simply run the following (modifying credentials appropriately):

ssh -L 5432:odyssey-dev-1.private.uwaterloo.ca:5432 dj2walte@linux.cs.uwaterloo.ca

And then access the database like so:

psql -h localhost -d dj2walte_dev -U dj2walte