Sponsors Data Overview

This is a "backup" copy. The original is currently available at http://www.math.uwaterloo.ca/MFCF/info/overviews/resource.shtml

Please use that if it is available.

Well how about that. It died as part of the great web friendliness improvement. Or something. So you're probably stuck with my copy here.

Introduction

For many years, with only a few machines to administer, MFCF managed its resources individually on each machine. As the number of machines, the types of machines, and the types of resources increased, such methods could no longer adequately handle our requirements. Keeping track of who had what resource where turned into an impossible task. Given a particular account on a machine, no one could tell why the account existed, nor whether or not one could safely remove it. Things got even worse when professors and research groups began getting their own machines and creating arbitrary accounts on them. Eventually we discovered bizarre situations, such as former grad students still using accounts on MFCF machine several years after they had graduated and left the university. To solve these problems, we used an existing crude mechanism, which could create and delete the undergraduate student UNIX accounts required for specific courses, as the basis for our resource management system. The resulting system allows us to administer all resources on all types of machines for all types of users simply by manipulating a database on one central machine. New machines join the system with relatively little work, and even completely new situations fit easily into the mechanism, requiring changes to very few parts of the system.

Requirements

In order to simplify the problem of keeping track of resource assignments, we have imposed some conditions on what the resource management system must handle.

Users must have unique reliable identifiers

If a user has one userid on one machine and a different userid on another, or if two users have the same userid on two different machines, far too much confusion results. To prevent this, MFCF assigns each person (or specialized software or group account) exactly one userid, normally from the userid database maintained by IST.

Resource allocations must have someone responsible for them

If one cannot easily determine exactly why each user had a particular resource, confusion results, and resource usage goes unaccounted for. To prevent this, all resource allocations must have an associated sponsorship, which includes both a reason for the allocation and a person responsible (not necessarily financially) for it.

Separation of ownership, administration, and authorization

The owner or administrator of a resource will not necessarily know or care which users have access to that resource. Thus the idea of sponsorship must not depend upon anything related to the ownership or administration of the equipment.

Multiplicity of sponsorship of a user's resource

A user may have a resource for more than one reason at the same time (e.g. a secretary using a computer for more than one professor's work, or a student using a computer for more than one course). The loss of one sponsorship should not affect other sponsorships.

Userids

MFCF

A central database contains records of all userids, each record including such permanent information as studentID, SIN, employee ID, and name. More transient information, such as a student's current registration or a professor's office number does not go into this database. This ensures that what simple information it does contain has reliable and up-to-date values without any human intervention. Other databases, not necessarily maintained by MFCF, provide the more transient information, accessed by their reliable userid or ID-number keys.

IST

Several years ago IST started a similar userid database for the University. This UWdir database contains more detailed information such as office numbers, phone extensions, current email addresses, home URLs, and department affiliation. MFCF goes to considerable effort to keep the userids in the two databases in sync, but due to the conflicting philosophies involved (e.g. MFCF like to retain information while IST likes to purge non-current entries), this synchronization consumes far more time than it should.

Sponsorship

MFCF assigns to each resource allocation two different attributes that together provide sufficient information to determine exactly why users have their particular resources.

Sponsors

Whatever mechanisms and reasons may exist for providing a user with a resource, ultimately the responsibility for that assignment belongs to some individual. So each resource assignment must trace back to an associated sponsor. A sponsor has the responsibility of authorizing such things as the amount and duration of the resource allocated to a user. Sponsorship may, but does not necessarily, imply financial obligation.

Classes

The idea of a class of resources comes from the original scheme of assigning resources to students registered in specific courses. (E.g. all students currently taking CS234 get 50 Mbytes of disk space and an account on the machine poppy.) This has generalized to having arbitrary sets of users receiving arbitrary sets of resources, with a class name providing some indication of what distinguishes this assignment from another. (E.g. within a professor's sponsorship, all grad students doing research for him might belong to one class, while that professor's visitor from another university might belong to another class.) This enables sponsors with large numbers of resources or sponsored users to better remember their reasons for allocating each resource to each user.

Sponsors Database

MFCF maintains a central resource sponsorship database through which all resource assignments occur.

Files

Currently the sponsors database consists of several directories (one per department), each containing several files (one per sponsor), though one could arbitrarily rearrange this filesystem organization without requiring any software changes. Our administrative staff maintain the data itself using simple text editors (e.g. VI), preventing conflicts via the lockedit command. Should the need arise for a different interface though, such as replacing the text files with an SQL database, we could switch to that without worrying about any of the existing software other than the internal function that gathers the data. A single C library function provides a read-only programming interface to the data, sucking the entire database into in-memory structures.

Data Structure

One can consider the sponsors database as a tree structure. A typical database entry for a sponsor contains one or more resource classes, with each resource class containing one or more resources, and each resource containing zero or more allocations to users. A class can also sponsor equipment support that does not depend upon who actually uses the equipment. For instance, the Dean of Math would sponsor classes such as CS100 and STAT230. The STAT230 class would provide resources such as a computing account in the student.math region, a Nexus account, and 2 Mbytes of disk space. And all students currently registered in that course would receive those resources. Another class would sponsor equipment support for terminals, independent of who has access to them. Similarly, a professor might sponsor classes for his or her own research, for grad students's research, and for visitors. Again, each class would provide various resources (computer accounts, printing quotas, UNIX group membership, etc.) to various users. Note that the separation of a sponsor's data into classes simply provides a convenient way of organizing things. Sponsors may wish to have a single class (effectively no class).

Processing

Compilation

A program regularly takes the sponsors data and reorganizes it by resource. Under the current implementation, this means that the program sponsor_resources produces a file for each resource and places it under one of several directories, one for each type of resource. For instance, all allocations for computing on the machine mfcf.math end up in a file called resources/computing/mfcf.math. Each resulting resource contains one record for each user, listing any required attributes and resource restrictions. (E.g. For UNIX computing accounts it includes finger information, disk quota, and group membership.)

Remote Processing

Another program takes each of the resource files and, depending upon the type of resource, passes it to another process or machine that knows how to allocate the requested resource. For instance, the mfcf.math computing file goes to the machine mfcf.math itself, where a program compares that file with the current state of the machine. If the resource file lists an account that doesn't exist on the machine, the mechanism immediately creates such an account. Similarly if an account exists on the machine but doesn't appear in the file, that account no longer has any sponsors and so an account deletion mechanism takes over the account. For sponsored accounts that do exist, it adjusts disk quotas and group memberships as necessary, including handling cases of multiple sponsorship. For Nexus accounts, the resource file goes to student.math, where a program compares it with a dump of the existing Nexus accounts. Any differences result in a MS-DOS batch file containing add and delete commands that the perseus.math PC then processes. For Active Directory accounts, the resource file goes to mfcf.math where a program queries our AD server and compares the results with these requests. LDAP commands immediately correct any differences (e.g. new accounts, expired accounts, group membership). Note that in each case, quite different mechanisms handle the compiled resource request files, sometimes by the accounts-master machine itself, sometimes by a second machine, and sometimes by the target machine via an intermediate machine. The exact mechanisms depend upon the type of resource but not upon the sponsorship data itself, allowing implementation changes and reconfiguration without affecting or requiring changes to anything else.

Remote Results

Wherever possible, each machine that controls access to a resource ends up with information that allows one to determine why each allocation exists. The lpquota command lists the sponsoring class as the Account associated with the printer quota entry. (The differing terminology predates the sponsorship mechanism.) Similarly, the userinfo command lists one's Classes, including group and diskquota limits. In no case does a resource exist without a corresponding value indicating its sponsorship. Even in cases where resources haven't fully gone away, a sponsor will still exist. (E.g. we grant a grace period for computing accounts to allow users time to move their files etc., and during this period MFCF itself (and not any previous sponsor of the resource) sponsors any resource usage.)

Summary

The central sponsors database defines the allocation of resources to users on all the machines that MFCF administers. This database operates independently of the type and location of the resources controlled. The information passes to each resource's maintenance mechanism with instructions to make-it-so. This mechanism operates independently of the source of the information, neither process needing to know anything about how the other works. One can easily track every resource allocation to a responsible sponsor, and every sponsor to the resources for which they have responsibility.

Future Directions

This mechanism could easily extend to manage other kinds of resources, such as xhiered software packages.

-- RayButterworth - 01 Oct 2008
-- AdrianPepper - 15 Mar 2011