Here we discuss what to do if a Researcher wishes to sponsor an account ("resources") for an ex-grad-student.

---

IncludeSponsorsDataPreamble

These notes relate to adding users to groups and servers using the Xhiered Accounts packages. This controls access to "xhiered" machines as well as those using the CS Active Directory for accounts management.

IncludeSponsorsDataRequiredAuthorization

Required authorization

• Users who need to update accounts need to be in group "sponsor" and group "accounts" on cs-xh-admin.cs.private.uwaterloo.ca
  • That is, in those groups in the CS-GENERAL AD
• Users who need to manipulate accounts, for example set passwords, in the cs-general region to be in the "accounts" group on cs-general.cs.private. Similarly for any other region.

IncludeSponsorsDataCreatingUserids

Creating Truly New Users (Userids)

Before you can use the sponsors data software to create new accounts for a userid, that userid must be present in the file /software/accounts-userids/data/Userids on the machine cs-xh-admin.cs.private.uwaterloo.ca. Often, but not always, that happens automatically.

If the userinfo command gives results for the userid, then the userid is appropriately present in this file, and you can proceed. (And use the Id Number: indicated by the output as the id number to associate with the userid). If userinfo gives no output for the particular user, more work is needed to get the userid into the file.

The Id Number: will be a student id, employee id, HR id, employee id, or WatIAM "P-number", in that order. In fact, you will see other obscure values listed too. But only the one shown can be used. Users with a lengthy history at UW sometimes have several other values shown as the Other Ids: value; you cannot use any of those values.

In most cases user names and identification numbers already exist, precreated in WatIAM. In other limited cases, the correct thing to do is create a userid which really should, for various reasons, not be put in WatIAM. And occasionally you need to do things to cause the WatIAM account creation.

WatIAM account creation (precreation) is discussed at this link which requires appropriate authentication.

After the WatIAM account has been created, you normally need to wait until the next business day for the new userid to be available for sponsors data. Various technical personnel, including Adrian Pepper, can accelerate that process, however.

If you need to do that yourself, perhaps see Discussion of the Userids File for Experts.

And bear in mind Information about Userids in the Sponsors Data.

IncludeSponsorsDataCreatingUserids

Creating Truly New Groups (gids)

If you are creating a new group to appear in the Active Directory (eg: users_researchgroup), you will need to follow the instructions in CreatingGroupAccounts

IncludeSponsorsDataLocation

Location of Accounts Sponsor files

The accounts files for CS are on the UNIX computer cs-xh-admin.cs.private.uwaterloo.ca

cd /software/accounts-master/data/sponsors

IncludeSponsorsDataDirectoryTableStart

Directory/File Name	Purpose
Research/ IncludeSponsorsDataDirectoryResearch
Research/	files under Research correspond to an individual researcher or research group. Here resources, or extra resources, for users specifically affiliated with particular researchers can be sponsored. These might be extra disk space (or theoretically printing) for a grad student, or entire accounts for associated or visiting researchers.
REGISTRAR-YYYYMM/ IncludeSponsorsDataDirectoryREGISTRARyy
REGISTRAR-YYYYMM/	Files under REGISTRAR-YYYYMM, e.g. REGISTRAR-201001 implement the predictable long-term expiry of sponsored resources by freezing the indicated previous terms class (program/year) memberships. Another file there ensures any account created stays until after the drop-add deadline. Occasionally (usually only in fall terms) you will see REGISTRAR-YYYYMMext directories. Read the file .README in such a directory for an explanation of its purpose and how it was created.

Graduate Students would previously have had resources allocated from files in the subdirectories named REGISTRAR-YYYYMMDD arising from previous allocations under REGISTRAR.

It will be necessary to add appropriate sponsorship information for them to the appropriate per-researcher file beneath /software/accounts-master/data/sponsors/Research.

For example, perhaps you received an email message like:

From wbcowan@cs.uwaterloo.ca Wed Jan 20 16:02:51 2010
Message-ID: <4B576F6A.1030300@bagend.uwaterloo.ca>
Date: Wed, 20 Jan 2010 13:02:34 -0800
From: William Cowan 
To: accounts@cs.uwaterloo.ca
Subject: Re: CS Grad computer account expiring

Please retain the account for Samwise Gamgee.

accounts@cs.uwaterloo.ca wrote:
> CS computing accounts for grad students who have left are
> sponsored by the School for a few terms.  After this time
> they are disabled unless the supervisor wishes to sponsor
> them.
>
> The accounts for:
>
>     Samwise Gamgee (sgamgee) 
>     Peregin Took (ptook) 
>     Meriadoc Brandibuck (mbrandibuck)
>
> on the core CS computing environment will be disabled in
> early February 2010 and the files removed shortly thereafter.
>
> Warning email has already been sent to the students at their
> @cs and @uwaterloo.ca addresses.
>
> If you have any questions please contact accounts@cs.uwaterloo.ca.

In this case you might first consider the file

/software/accounts-master/data/sponsors/Research/Cowan

but further inspection would reveal that, because of a surname conflict,

/software/accounts-master/data/sponsors/Research/WCowan

is the one to change. (Note that student names in this example are fictional, but we've used two real researcher names for a realistic example).

Making the changes

On cs-xh-admin.cs.private.uwaterloo.ca use an appropriate text editor you are comfortable with.

A normal resource sponsorship to be added to the file (WCowan in our example), to extend resource sponsorship for a the named graduate student would look like

====
# Created January 2010 to perpetuate core.cs Grad account sgamgee
Computing: core.cs
SponsorshipEnds: 2015/Jan/21
Quota: 1G
AssignTo: sgamgee
====
Computing: serverus.cs
SponsorshipEnds: 2015/Jan/21
Groups: general_cs
AssignTo: sgamgee

But if you have lots of similar cases to handle, it can be more convenient to do them all together, perhaps even adding to previously existing sections, and ending up with something like:

====
# Perpetuate core.cs Grad accounts
Computing: core.cs
SponsorshipEnds: 2015/Jan/21
Quota: 1G
AssignTo: bbaggins sgamgee ptook mbrandybuck
====
Computing: serverus.cs
SponsorshipEnds: 2015/Jan/21
Groups: general_cs
AssignTo: bbaggins sgamgee ptook mbrandybuck

You can even do differing SponsorshipEnds dates (sometimes called "Expiry Dates"), as in.

====
# Perpetuate core.cs Grad accounts
Computing: core.cs
SponsorshipEnds: 2015/Jan/21
Quota: 1G
AssignTo: sgamgee
SponsorshipEnds: 2013/Oct/31
AssignTo: ptook
SponsorshipEnds: 2012/Dec/25
AssignTo: mbrandybuck
SponsorshipEnds: 2010/Jan/10
AssignTo: bbaggins
====
Computing: serverus.cs
SponsorshipEnds: 2015/Jan/21
Groups: general_cs
AssignTo: sgamgee
SponsorshipEnds: 2013/Oct/31
AssignTo: ptook
SponsorshipEnds: 2012/Dec/25
AssignTo: mbrandybuck
SponsorshipEnds: 2010/Jan/10
AssignTo: bbaggins

If it was decided all such students should get more diskquota, there would be only one number to change.

And yet you could later add resources for only one.

====
# Perpetuate core.cs Grad accounts
Computing: core.cs
SponsorshipEnds: 2015/Jan/21
Quota: 1G
AssignTo: sgamgee
SponsorshipEnds: 2013/Oct/31
AssignTo: ptook
SponsorshipEnds: 2012/Dec/25
AssignTo: mbrandybuck
SponsorshipEnds: 2010/Jan/10
AssignTo: bbaggins
====
Computing: serverus.cs
SponsorshipEnds: 2015/Jan/21
Groups: general_cs
AssignTo: sgamgee
SponsorshipEnds: 2013/Oct/31
AssignTo: ptook
SponsorshipEnds: 2012/Dec/25
AssignTo: mbrandybuck
SponsorshipEnds: 2010/Jan/10
AssignTo: bbaggins

====
# Samwise needs more room for mpegs
Computing: core.cs
SponsorshipEnds: 2015/Jan/21
Quota: 20G
AssignTo: sgamgee

For more of a tutorial about editing sponsors data files perhaps see SponsorsDataEditingTutorial .

Cautionary Notes About Editing the Sponsors Data

IncludeSponsorsDataEditingCaution

In general you should avoid making easy-to-make mistakes.

Nearly all files found in the directory

   /software/accounts-master/data/sponsors/

and all directories beneath it, will be processed by the sponsor_resources command. Exceptions are files whose name begins with . (dot/period) and files in sub-directories which sub-directories are named RCS. If a subdirectory name begins with . (dot/period) then everything beneath it will be ignored (unless referred to explicitly by other files).

The upshot of that is that you cannot place arbitrary files in these directories, or sponsor_resources will stop working correctly.

When editing the sponsors data files, it can often be convenient to copy and modify previously existing lines to create your new additions. If you do that, make sure you correctly change all relevant SponsorshipEnds dates, or remove them as appropriate. Also make sure you delete or change any comments which are irrelevant in the new context. It's better to leave no comments than leave confusing comments.

Consider that the files are intended to help later readers understand the sponsorship situation; use a few comment lines ("#"), and, in general, put a blank line before each ==== line.

Note that id numbers need to be associated with userids. This can be done directly in the AssignTo line as in

      AssignTo: sgamgee:02020202

Or it can be done in the Userids: section at the top of the file. Only one specific id number can be used for a particular userid. See IncludeSponsorsDataIdNumbers for details.

To help make sure we don't slip up and put a real userid:studentid in one of these pages, we by convention omit :studentid in all examples.

Finally, make sure you remember to use the ci -u command so you leave the file editable by others.

IncludeSponsorResourcesAccountsClient

Run the sponsor_resources program to process the changes

IncludeSponsorsDataRunSponsorResources

Note: everyone who runs the sponsor_resources command should be aware that it is really just a portion of the accounts-master command. If only one procedure needed to be chosen, the correct thing to do is run accounts-master, not sponsor_resources. accounts-master does a superset of what sponsor_resources does. However, after editing sponsors data, sponsor_resources alone is usually sufficient.

sponsor_resources takes the data under /software/accounts-master/data/sponsors and produces per-user requirements in per-machine (actually per-region) files under /software/accounts-master/data/resources. Along the way, it might detect errors in the changes you made. Fix any problems that are reported and keep rerunning sponsor_resources until all your errors have gone away.

In particular, never leave the sponsors data in a state where sponsor_resources does not produce the three lines which begin with FYI:. (The first being FYI: ... computings ...). The error, warnings, notes line can actually have non-zero errors if you can assert they are not a result of your changes. (Corollary; run sponsor_resouces before you even start making changes so you can be sure what noise you did not cause).

Typically you can simply use the command sponsor_resources with no arguments. However, it is often prudent to redirect standard output and error output.

Here is an example of a "bad run":

@cs-xh-admin[140]% sponsor_resources
Error: /software/accounts-master/data/sponsors/Research/Terry line 63: Userid 'bjlafren' is not a standard userid
FYI:  137970(29794) computings, 1292(1274) printers, 530(403) aliases, 0(0) ppps
FYI:  1 error, 0 warnings, (0 notes)
FYI: expired sponsorship entries:  17888 computings, 51 printers, 0 aliases, 0 ppps
handle_group (group_id=1): Found 2 group description lines, expected <= 1
handle_group (group_id=11): Found 2 group description lines, expected <= 1
found 157201 userinfos, 14368 distinct
administration = cscf
psql::2: NOTICE:  truncate cascades to table "sponsor_billcode"
psql::2: NOTICE:  truncate cascades to table "sponsor_class"
psql::2: NOTICE:  truncate cascades to table "sponsor_member"
psql::2: NOTICE:  truncate cascades to table "sponsor_computing"
psql::2: NOTICE:  truncate cascades to table "sponsor_printing"
psql::2: NOTICE:  truncate cascades to table "sponsor_computing_group"
psql::2: NOTICE:  truncate cascades to table "sponsor_mailalias"
TRUNCATE TABLE
@cs-xh-admin[141]%

In this case, sponsor_resources complained because we used the short version of the userid ("bjlafren"), but it always requires the long version (ie: "bjlafreniere").

The following is an example of a "good run":

@cs-xh-admin[143]% sponsor_resources
FYI:  137973(29797) computings, 1292(1274) printers, 530(403) aliases, 0(0) ppps
FYI:  0 errors, 0 warnings, (0 notes)
FYI: expired sponsorship entries:  17888 computings, 51 printers, 0 aliases, 0 ppps
handle_group (group_id=1): Found 2 group description lines, expected <= 1
handle_group (group_id=11): Found 2 group description lines, expected <= 1
found 157201 userinfos, 14368 distinct
administration = cscf
psql::2: NOTICE:  truncate cascades to table "sponsor_billcode"
psql::2: NOTICE:  truncate cascades to table "sponsor_class"
psql::2: NOTICE:  truncate cascades to table "sponsor_member"
psql::2: NOTICE:  truncate cascades to table "sponsor_computing"
psql::2: NOTICE:  truncate cascades to table "sponsor_printing"
psql::2: NOTICE:  truncate cascades to table "sponsor_computing_group"
psql::2: NOTICE:  truncate cascades to table "sponsor_mailalias"
TRUNCATE TABLE
@cs-xh-admin[144]%

Above we have been assuming a small, non-fatal error directly related to the change you made and which you were able to correct.

In extreme cases ("an extremely bad run?") you might end up with something like...

@cs-xh-admin[145]% sponsor_resources
Fatal error: "/software/accounts-master/data/sponsors/Research/yuying" line 89: not SponsorshipStarts, SponsorshipEnds, Quotas, Groups, or AssignTo:  Computing: serverus.cs
@cs-xh-admin[146]%

The Fatal error diagnostic, corroborated by the lack of FYI: ... computings, ... indicate that the sponsors data is severely broken to the extent that the data under /software/accounts-master/data/resources will not have been updated at all. Intended changes will not have taken effect, and, unless someone fixes the problem, future changes will not take effect either. (However, the resource data will remain in the state left by the last successful_sponsor_resources_ ).

Corollary: run sponsor_resources before you begin making changes and make sure the sponsors data was not already in a state like this; if it is, try to track down the most recent change, fix it if it is easy and obvious and in any case alert the person who left the broken state (you may in any case need their help to make some decisions related to the fix).

The diagnostics indicate which file you should look at. Frequently problems occur because of missing ==== lines. But also, having two such lines in a row, even if separated by comments and blank lines, is a guaranteed fatal error, although there's no real reason why it should be. (That is to say, it ought to have been possible at one time to modify the grammar to expect the ==== token and recognize (and mostly ignore) an empty section, but that was never done, and so ==== must always be followed by one of the appropriate keywords as listed in the diagnostic.

Unavoidable extraneous noise

It would be nice if we could tell you to always keep modifying until you have 0 errors but because of details how some automatic data is produced from IST information, that is not always possible. Some things have reached the point where they are actually unresolvable by us. This results in errors which must be ignored, even though you must continue to look for your own errors, and especially for the absence of Fatal error:.

Consequently noise like the following is sometimes unavoidable, and while things may need to be done to work around the indicated problems, the majority of resource data will reflect the most recent changes made.

@cs-xh-admin[145]% sponsor_resources
Error: "/software/accounts-master/data/sponsors/REGISTRAR/cs" line 845: Userid 'ktfrog' should have id number 'hr999007' not '74123456'
Error: "/software/accounts-master/data/sponsors/REGISTRAR/cs" line 868: Userid 'ktfrog' should have id number 'hr999007' not '74123456'
FYI:  137974(29801) computings, 1294(1276) printers, 530(403) aliases, 0(0) ppps
FYI:  2 errors, 0 warnings, (0 notes)
FYI: expired sponsorship entries:  17888 computings, 51 printers, 0 aliases, 0 ppps
handle_group (group_id=1): Found 2 group description lines, expected <= 1
handle_group (group_id=11): Found 2 group description lines, expected <= 1
found 157201 userinfos, 14368 distinct
administration = cscf
psql::2: NOTICE:  truncate cascades to table "sponsor_billcode"
psql::2: NOTICE:  truncate cascades to table "sponsor_class"
psql::2: NOTICE:  truncate cascades to table "sponsor_member"
psql::2: NOTICE:  truncate cascades to table "sponsor_computing"
psql::2: NOTICE:  truncate cascades to table "sponsor_printing"
psql::2: NOTICE:  truncate cascades to table "sponsor_computing_group"
psql::2: NOTICE:  truncate cascades to table "sponsor_mailalias"
TRUNCATE TABLE
@cs-xh-admin[146]%

Path to the "sponsor_resources" command

If you don't happen to have the maintenance commands in your path, the path to the sponsor_resources command is:
/software/accounts-master/maintenance/sponsor_resources

Run the userinfo program to verify your changes

IncludeSponsorsDataRunUserinfo

Run the userinfo command before and after you make your changes to verify that its output reflects your intended changes. More details are here.

Run the accounts-client program

IncludeSponsorsDataRunAccountsClient

accounts-client {hostname, eg:cs-general.cs.private} >& ~/hostname-date &

This will cause the desired changes to actually happen on the appropriate machine (hostname; what is described as "Computing:" in the sponsor file, which may in turn affect a region of machines). /etc/passwd and /etc/group file will be updated if necessary, as will system quota files, and home directories will be created for any newly-created users. The diagnostic output from the job will be written to the given filename in your home directory. eg:

   accounts-client softbase.cs >& ~/softbase-20100208 &
   

If your changes will cause changes on multiple regions, you will need to run the command for each. eg:

   accounts-client student.cs >& ~/student-20100208 &
   accounts-client cs-general.cs.private >& ~/cs-general-20100208 &
   

If you run accounts-client with no name, all known regions are updated. eg:

   accounts-client >& ~/ac-all-20100208 &
   

That can take a long time to finish.

Path to the "accounts-client" command

If you don't happen to have the maintenance commands in your path, the path to the accounts-client command is:
/software/accounts-master/maintenance/accounts-client

Updating Directory Services Domains

Once the sponsor_resources process has run and files under /software/accounts-master/data/resources/computing/have been updated, CSCF Directory Services domains are subsequently updated by cron job. User and group information are revised based upon the information stored in specific files found in the resources/computing subdirectory. At time of writing (Feb 2017), the Directory Services updates are run following the schedule below.

1. CS-GENERAL domain: 20 minutes past the hour
   • Using the cs-general.cs.private and serverus.cs files.
2. CS-TEACHING domain: 50 minutes past the hour
   • Using the cs-teaching.cs.private and canadenis.student.cs files.

To force a domain update to run immediately, run the following commands on the xhier admin machine (cs-xh-admin) as root user:

1. For CS-GENERAL domain:# /software/local_cs-xh-admin.cs.private.uwaterloo.ca/servers/cscf_ad_domain_update cs.uwaterloo.ca
2. For CS-TEACHING domain:# /software/local_cs-xh-admin.cs.private.uwaterloo.ca/servers/cscf_ad_domain_update student.cs.uwaterloo.ca

Documentation

IncludeSponsorsDataDocumentation

The following traditional UNIX man page documentation describes the sponsors data base in a technical fashion. As you become familiar with how things work in general, you might find this documentation good for checking specific details.

This stopped working a while ago. For now see SponsorsDataAccountsDocumentationBig and search for the command in question. When I get time I'll change the following references to be simply that.

• man sponsors
• man sponsor_resources
• man accounts-client
• man userinfo
• man accounts-master (command)
• man accounts-master (package)
• list of man pages in the accounts-master package

-- AdrianPepper - 15 Mar 2010