IncludeSponsorsDataRunSponsorResources < CF

CF Web>IncludeSponsorResourcesAccountsClient>IncludeSponsorsDataRunSponsorResources (2021-06-12, AdrianPepper) (raw view)
EditAttach
---+ sponsor_resources command
<!--<pre>
   //
   // Aside: TWiki H1 style is yucky.
   // 
   // CF.IncludeSponsorsDataRunSponsorResources
   // 
   // This page was primarily designed for inclusion where appropriate,
   // but might work well stand-alone, especially with the careful
   // use of the STARTINCLUDE and STOPINCLUDE "variables" which
   // allow a header and footer, although not generalized creation
   // of differences in included and non-included forms.
   // (Actually IncludeSponsorResourcesAccountsClient was stolen from
   // something lfolland had written).
   //
   // Note the line after header after  STARTINCLUDE. 
   // The CFADRIANGADGETINCLUDE variable renders as a link which 
   // makes it easy for a reader to get to the inclusion to edit it,
   // although the rendered presentation does not seem fully intuitive yet.
   //
</pre>-->
<!-- Bah!  You don't seem to be able to have multiple STOP/STARTINCLUDE -->
%STARTINCLUDE%
---++ Run the sponsor_resources program to process the changes
%CFADRIANGADGETINCLUDE%

Note: everyone who runs the =sponsor_resources= command
should be aware that it is really just a portion of the
[[IncludeSponsorsDataRunAccountsMaster][ =accounts-master= ]] command.
If only one procedure needed to be chosen, the correct thing to do is
run =accounts-master=, not =sponsor_resources=.
=accounts-master= does a superset of what =sponsor_resources= does.
However, after editing sponsors data, =sponsor_resources=
alone is usually sufficient.

_[[%CFMANURL%?page=sponsor_resources][sponsor_resources]]_ takes the data under
=/software/accounts-master/data/sponsors= and produces per-user requirements
in per-machine (actually per-region) files under =/software/accounts-master/data/resources=.
Along the way, it might detect errors in the changes you made.
Fix any problems that are reported and keep rerunning _sponsor_resources_
until all your errors have gone away.

In particular, never leave the sponsors data in a state where _sponsor_resources_ does not produce the three lines which begin with =FYI:=.
(The first being =FYI: ... computings ...=).  The =error, warnings, notes= line can actually have non-zero errors if you can assert they are not
a result of your changes.  (Corollary; run _sponsor_resouces_ before you even start making changes so you can be sure what noise you did not cause).

Typically you can simply use the command =sponsor_resources= with no arguments.  However, it is often prudent to redirect standard output and error output.

Here is an example of a "bad run":

<pre>
@cs-xh-admin[140]% sponsor_resources
Error: /software/accounts-master/data/sponsors/Research/Terry line 63: Userid 'bjlafren' is not a standard userid
FYI:  137970(29794) computings, 1292(1274) printers, 530(403) aliases, 0(0) ppps
FYI:  1 error, 0 warnings, (0 notes)
FYI: expired sponsorship entries:  17888 computings, 51 printers, 0 aliases, 0 ppps
handle_group (group_id=1): Found 2 group description lines, expected <= 1
handle_group (group_id=11): Found 2 group description lines, expected <= 1
found 157201 userinfos, 14368 distinct
administration = cscf
psql:<stdin>:2: NOTICE:  truncate cascades to table "sponsor_billcode"
psql:<stdin>:2: NOTICE:  truncate cascades to table "sponsor_class"
psql:<stdin>:2: NOTICE:  truncate cascades to table "sponsor_member"
psql:<stdin>:2: NOTICE:  truncate cascades to table "sponsor_computing"
psql:<stdin>:2: NOTICE:  truncate cascades to table "sponsor_printing"
psql:<stdin>:2: NOTICE:  truncate cascades to table "sponsor_computing_group"
psql:<stdin>:2: NOTICE:  truncate cascades to table "sponsor_mailalias"
TRUNCATE TABLE
@cs-xh-admin[141]%</pre>
In this case, =sponsor_resources= complained because we used the short version
of the userid ("bjlafren"), but it always requires the long version
(ie: "bjlafreniere").

The following is an example of a "good run":
<pre>
@cs-xh-admin[143]% sponsor_resources
FYI:  137973(29797) computings, 1292(1274) printers, 530(403) aliases, 0(0) ppps
FYI:  0 errors, 0 warnings, (0 notes)
FYI: expired sponsorship entries:  17888 computings, 51 printers, 0 aliases, 0 ppps
handle_group (group_id=1): Found 2 group description lines, expected <= 1
handle_group (group_id=11): Found 2 group description lines, expected <= 1
found 157201 userinfos, 14368 distinct
administration = cscf
psql:<stdin>:2: NOTICE:  truncate cascades to table "sponsor_billcode"
psql:<stdin>:2: NOTICE:  truncate cascades to table "sponsor_class"
psql:<stdin>:2: NOTICE:  truncate cascades to table "sponsor_member"
psql:<stdin>:2: NOTICE:  truncate cascades to table "sponsor_computing"
psql:<stdin>:2: NOTICE:  truncate cascades to table "sponsor_printing"
psql:<stdin>:2: NOTICE:  truncate cascades to table "sponsor_computing_group"
psql:<stdin>:2: NOTICE:  truncate cascades to table "sponsor_mailalias"
TRUNCATE TABLE
@cs-xh-admin[144]%</pre>

Above we have been assuming a small, non-fatal error directly related to the change you made and which you were able to correct.

In extreme cases ("an extremely bad run?") you might end up with something like...

<pre>
@cs-xh-admin[145]% sponsor_resources
Fatal error: "/software/accounts-master/data/sponsors/Research/yuying" line 89: not SponsorshipStarts, SponsorshipEnds, Quotas, Groups, or AssignTo:  Computing: serverus.cs
@cs-xh-admin[146]%</pre>

The =Fatal error= diagnostic, corroborated by the lack of =FYI: ... computings, ...=
indicate that the sponsors data is severely broken to the extent that
the data under =/software/accounts-master/data/resources= will not have been
updated at all.
Intended changes will not have taken effect, and, unless someone fixes the
problem, future changes will not take effect either.  (However, the
resource data will remain in the state left by the last
successful_sponsor_resources_ ).

Corollary: run _sponsor_resources_
before you begin making changes and make sure the sponsors data was not already
in a state like this; if it is, try to track down the most recent change,
fix it if it is easy and obvious and in any case alert the person who
left the broken state (you may in any case need their help to make some
decisions related to the fix).

The diagnostics indicate which file you should look at.  Frequently problems occur because of missing =&#61;&#61;&#61;&#61;= lines.  But also, having two such lines in a row, even if separated by comments and blank lines, is a guaranteed fatal error, although there's no real reason why it should be.  (That is to say,
it ought to have been possible at one time to modify the grammar to expect the =&#61;&#61;&#61;&#61;= token and recognize (and mostly ignore) an empty section,
but that was never done, and so =&#61;&#61;&#61;&#61;= must always be followed
by one of the appropriate keywords as listed in the diagnostic.

---++++ Unavoidable extraneous noise

It would be nice if we could tell you to always keep modifying until you have =0 errors=
but because of details how some automatic data is produced from IST information,
that is not always possible.  Some things have reached the point where they are
actually unresolvable by us.  This results in errors which must be ignored, even
though you must continue to look for your own errors, and especially for
the absence of =Fatal error:=.

Consequently noise like the following is sometimes unavoidable,
and while things may need to be done to work around the indicated problems,
the majority of resource data will reflect the most recent changes made.

<pre>
@cs-xh-admin[145]% sponsor_resources
Error: "/software/accounts-master/data/sponsors/REGISTRAR/cs" line 845: Userid 'ktfrog' should have id number 'hr999007' not '74123456'
Error: "/software/accounts-master/data/sponsors/REGISTRAR/cs" line 868: Userid 'ktfrog' should have id number 'hr999007' not '74123456'
FYI:  137974(29801) computings, 1294(1276) printers, 530(403) aliases, 0(0) ppps
FYI:  2 errors, 0 warnings, (0 notes)
FYI: expired sponsorship entries:  17888 computings, 51 printers, 0 aliases, 0 ppps
handle_group (group_id=1): Found 2 group description lines, expected <= 1
handle_group (group_id=11): Found 2 group description lines, expected <= 1
found 157201 userinfos, 14368 distinct
administration = cscf
psql:<stdin>:2: NOTICE:  truncate cascades to table "sponsor_billcode"
psql:<stdin>:2: NOTICE:  truncate cascades to table "sponsor_class"
psql:<stdin>:2: NOTICE:  truncate cascades to table "sponsor_member"
psql:<stdin>:2: NOTICE:  truncate cascades to table "sponsor_computing"
psql:<stdin>:2: NOTICE:  truncate cascades to table "sponsor_printing"
psql:<stdin>:2: NOTICE:  truncate cascades to table "sponsor_computing_group"
psql:<stdin>:2: NOTICE:  truncate cascades to table "sponsor_mailalias"
TRUNCATE TABLE
@cs-xh-admin[146]%</pre>


---+++ Path to the "sponsor_resources" command
If you don't happen to have the maintenance commands in your path, the path to the sponsor_resources command is:<br>
=/software/accounts-master/maintenance/sponsor_resources=

%STOPINCLUDE%

-- Main.AdrianPepper - 04 Feb 2011

-- Main.AdrianPepper - 2018-02-15
---
---+++See Also
   * [[IncludeSponsorsDataRunAccountsMaster][ =accounts-master= ]] command
   * [[IncludeSponsorsDataRunAccountsClient][ =accounts-client= ]] command

%INCLUDE{CF.IncludeAdrianReferers}%
Topic revision: r14 - 2021-06-12 - AdrianPepper
Information in this area is meant for use by CSCF staff and is not official documentation, but anybody who is interested is welcome to use it if they find it useful.
Other Webs
My links
- People
- CERAS
- WatForm
- Tetherless lab
- Ubuntu Main.HowTo
- eDocs
- RGG NE notes
- RGG
- CS infrastructure
- Grad images
Edit