SponsorsDataAccounts < CF

CF Web>Accounts>SponsorsDataAccounts (2019-07-30, AdrianPepper) (raw view)
---+ Sponsors Data

%TOC{depth="5"}%

The hierarchical text files under the directory
_/software/accounts-master/data/sponsors_
on the machine _cs-xh-admin.cs.private.uwaterloo.ca_
are a means to describe resources which should be allocated to users
on various other machines.
Computer programs are used to update remote machines to reflect the information there. 

---+++ History
The system was designed by [[http://www.math.uwaterloo.ca/mfcf/][MFCF]]
to facilitate charging real
money for resources, and is
based heavily upon the concept that all resources must have a _sponsor_
(although sponsors do not need to provide real money, nor do sponsors
even need to correspond to identifiable people).
A long-time maintainer wrote an [[SponsorsDataAccountsOverview][overview]].

---+++ Overview
The text files can loosely be thought of as a database.
In other respects their contents are really a configuration language,
approaching
the complexity (although not the generalized power) of
a programming language.
In fact, they are configuration files which are used to generate
a text file database under the corresponding
_/software/accounts-master/data/resources_ directory.
Those files are text tables, with identifiable fields or columns, but
are not in normal relational form because some of the fields contain lists
containing an arbitrary number of multiple values.

<!-- Modified Jun 18, 2013 to reflect fact that Ray Butterworth, a little
time before that, changed things so "Membership" was no longer effectively
a textual macro, but generated internal records similar to Sponsorship.  -->
Furthermore, in fact, there is data embodied in the sponsors data which
is not reflected in any particular version of the generated resource
data.
Resources can have start and end ("expiration") dates which are not
present in the generated data;
shown by SponsorshipEnds and
MembershipEnds (also corresponding Starts).
In the internal model used by several programs, _[[%CFMANURL%?page=sponsor_resources][sponsor_resources]]_
among them,
an End date is recorded internally, corresponding to SponsorshipEnds
(similarly for SponsorshipStarts, MembershipStarts and MembershipEnds), but those dates are not shown in
any output.

So a "proper" database representation of resources embodied in the
sponsors data might include all start and end dates, listing some
resource data which is not in effect on the current day.

The author is beginning to conclude the sponsors data is not a database,
but, rather, a configuration language to generate a database state
corresponding to resource allocation desired for any particular day.

Leaving aside the quibble about what is a database,
people sometimes refer to these files (either sponsors or resources
directory) as the
_accounts database_
but
_computer accounts_
(working login names for particular computer systems)
are created only as a side-effect of sponsorship.
Furthermore,
those
_accounts_
are  only one type of resource which can be controlled by these
files.

The types of resources which can currently be allocated are
=Computing=, =Printing=, and =MailAlias=.

When doing actual _accounts maintenance_, it is usually the =Computing=
type of resource we will be dealing with.
But allocating =Printing= quota, or even setting up a =MailAlias= 
can be considered accounts maintenance too.

Historically _ppp(dialin)_ access
was maintained as a separate type, and there appears to be under
development some attempt
to incorporate some sort of maintenance of a _licence_ resource,
but those are currently irrelevant.

=Computing= resources will generate login userids for some computer system; 
currently CSCF generates login userids for either a specific UNIX system
(which can be the NFS server for a region of related machines), or a
_Active Directory_ domain.
For historical reasons, the Active Directory domains are referred to by
the names of specific servers.

Resources which trigger the generation of login userids are actually of
type =Quota= or =Groups=.
If the _sponsors database_ indicates that some valid login
userid should receive (disk) =Quota=, or be made a member of some =Group=
on some controlled system,
that will cause the corresponding login userid to be created on that
system, if it did not already exist. <!-- (when [[%CFMANURL%?page=accounts_client][accounts_client]] is
run for that specific system)-->

<!-- Notes about Expiry and base quota organization should be detailed elsewhere -->
Resource allocation or removal can be configured ahead of time by the use of =Start= and =End= dates.
System configuration on the controlled machines will define how soon after sponsorship ends an account will actually be removed.
This cannot be modified by users who can otherwise modify the sponsors data.
In fact, it's even difficult for them to determine.
When an account is no longer sponsored, but has not been removed, 
the sponsors data software considered it to be already
"Expired", but "in grace period".
This terminology can be confusing.

A similar confusion can occur because other
system configuration on the controlled machines will define
a base quota for all users.
On some systems, this is so huge that most users are typically given
zero quota.
This base quota similarly cannot be modified
and is difficult to determine for users who can otherwise modify the
sponsors data.


As an example, perhaps see [[IncludeSponsorsDataDirectoryOrganization][this]] attempt
at an explanation of how the data is organized on host =cs-xh-admin.cs.private.uwaterloo.ca=.


<!--
   The following general types of _accounts_ are maintained.
   * [[RegistrarSponsoredAccounts][Registrar Sponsored Accounts]] -
        driven primarily by data received from the Registrar's Office
        (graduate and undergraduate are not fundamentally different)
   * [[ResearchSponsoredAccounts][Research Sponsored Accounts]] -
        created and maintained at the request of, or on behalf
        of, researchers
   * [[AdministrativeAccounts][Administrative Accounts]] -
        created and maintained for School administration purposes
-->

---+++ Strengths and Weaknesses
The system's fort&eacute; is assigning lots of similar resources to
large groups of users based on userid (user name) and student id
(number) and course or program
registration as received from the registrar.
In contrast, the system does not readily provide means for
mass maintenance of information about users whose
resource needs are otherwise arbitrarily varied.
Note, however, that the fact the sponsors data is composed purely
of text files does allow arbitrary tools to be used when massive
changes are required.

---+++ Philosophy
The system is not really designed to make the creation of _computer accounts_
an easy task.
Rather, it is designed to make it difficult to allocate resources without
creating an audit trail for those resources.
Even without any special software, allocation of resources on computer systems tends to be naturally relatively
easy.
But several years later it can be difficult to tell why those resources
were allocated and whether their allocation is still legitimate.
The extra pain the sponsors database requires to attribute resource requests
to sponsors
can help solve that future problem, and allow the reclamation of
resources whose allocation has become unjustified.
And that said, the sponsors database does make some
aspects of resource allocation easy, as a few lines of
configuration can cause the allocation of hundreds of computer
logins (or other resources) across several machines.

---+++ Omissions
The sponsors data can contain lots of information specific to classes
which is not used in the generation of computer accounts.
The documentation here largely ignores that; it is really only relevant
for RegistrarSponsoredAccounts, and, even then, the accuracy or inaccuracy
of any such configuration does not actually affect the accounts generated.

---+++ Getting Started
A good way to learn what can be done in the _sponsors database_
is to simply read portions of it.
SponsorsDataEditingTutorial purports to be a tutorial.

---++ Tools

Various
Unix command-line tools,
run on _cs-xh-admin.cs.private.uwaterloo.ca_, allow privileged users
to view and manipulate the sponsors data.
<!-- <pre>
%SCRIPTURL%
%BASEWEB%
 </pre> -->

%RED%2019-07-30: Only recently updated to refer to links which actually work.%ENDCOLOR%

   * _[[AccountsUserinfoCommand][userinfo]]_ 
   * _[[IncludeSponsorsDataRunAccountsMaster][accounts-master]]_ or _[[IncludeSponsorsDataRunSponsorResources][sponsor_resources]]_
   * _[[IncludeSponsorsDataRunAccountsClient][accounts-client]]_

When adjusting resources for one user,
the usual cycle is "userinfo" to see their current state, followed
by manipulation of the _sponsors database_ with a standard text
editor, followed by "sponsor_resources", and then appropriate use of
"accounts-client" to actually send the changes to the desired machine(s).
For example, you might manipulate the _sponsors database_ to give
an existing user extra disk space (disk quota), or to cause an
entirely new account (login name) to appear on a particular computer.


---++ Documentation

   * [[SponsorsDataAccountsDocumentationRaw][historical documentation, webified, not organized]]  (%RED%no longer works; look things up in the next link%ENDCOLOR%)
   * [[SponsorsDataAccountsDocumentationBig][historical documentation, webified, not organized, one file]] (after dynamic man page presentation broke, this is really the only way to see the old man pages through this TWiki interface)

---++ See Also

   * [[PrintQuota][Printer Quota]] is another type of resource the database attempts to manage
   * [[TemplateSponsorDataAccounts][Sponsors Data Accounts Maintenance in general]]  (or  TemplateSponsorDataAccountsCollapsed  )
   * SponsorsDataEditingTutorial


-- Main.AdrianPepper - 21 Jul 2011
Topic revision: r27 - 2019-07-30 - AdrianPepper
Information in this area is meant for use by CSCF staff and is not official documentation, but anybody who is interested is welcome to use it if they find it useful.
Other Webs
My links
- People
- CERAS
- WatForm
- Tetherless lab
- Ubuntu Main.HowTo
- eDocs
- RGG NE notes
- RGG
- CS infrastructure
- Grad images
Edit