Update Local Database

We often connect OAT to a local instance of postgres for testing, etc.

Requirements for the local database:

  • Same schema as production.
  • Lots of the same data as production.
  • A much-reduced set of students.
  • Students that are fixed as much as possible (if they change, tests break).
  • Ability to add new students to test time-sensitive features (eg we added degree audit but didn’t have plans defined for the ancient students already in the DB; added some new students).
  • Obfuscate the students so a co-op student can work with them.

We periodically extract specific students from the database using code in oat-extractdata. The extracted data is placed in ${OAT_HOME}/oat-testdata. Both of these repositories should be checked out from git.uwaterloo.ca:oat. Their ReadMe documents are helpful.

Workflow

  1. Make sure the current tests in OAT all pass. This process will break tests; you want to be in as good a position when you start as possible. Don’t change OAT before correcting the tests with the new data!
  2. Update the config file in oat-testdata/12_dev (most likely; there are others for specific purposes that are mostly unused).
  3. In oat-extracttestdb, run mill run --service oat-prod 12_dev
    • The service is the database you want to copy the data from.
    • 12_dev identifies the config file to read and where to place the data. It’s assumed to be a directory in ${OAT_HOME}/oat-testdata.
  4. In oat-testdata/12_dev, run the load script.

Note: This assumes you’ve had the dev environment working previously. If not, follow the steps in Dev Setup.

Debugging

  • Running the load script will sometimes crash on a new role: psql:schema_mod.sql:50830: ERROR: role "_quest_monitor" does not exist.
    Edit the file oat-testdata/createRolesForLocalDB.sql to include the new role. Then psql service=local-dev < createRolesForLocalDB.sql.