Here we describe common problems and approaches to solving them. All diagnosis and testing should start with the debug version.
The CGI does much of the work.
The usual approach to diagnosing a problem cause is to first reproduce it, using the debug version. Instead of "standard error" messages ending up in the WWW server logs, the debug version appends them to the resulting pages.
The next approach is to add "debugging printfs" to the debug code, relying upon the above behaviour. The "printfs" are most easily done via:
require st::debug;
&st::debug::it("VarName",VarRef);
where VarRef is a reference to the data to be displayed, and VarName is the name to use in the display.
In some cases, it can be handy to display messages when dealing with only the serial number of a test record (e.g. 72834), to avoid disturbing early adopters using the debug version.
And in some cases, it's useful to see what invoked the problem code, which can be done via
require st::debug; &st::debug::pr(&st::debug::traceback());
The RunOn Sentence can be copied elsewhere for testing.
The equivalent of a "debugging printf" is the
window.alert function.
Although note that each invocation stops execution and produces a popup
that has to be answered.
It can be faster to diagnose a problem (that requires repeated tests)
using command line tools.
In general, the available options can be seen via the "-v -h" (verbose help)
options.
`st-qlist` can be quite handy for testing queue listings.
Most of the tools accept a "-v" option, for verbose output,
and a "-Z" option, for a debugging level of verbosity.
They set the variable $main::babble:
Options $main::babble-v 1 -Z 2 -v -Z 3
The `st-find` command searches the entire main code base, used for both the CGI and command line tools. It doesn't search outboard tools such as the ACL CGI.
It is especially handy for looking for error messages, and for references to suspect fields.
ST (sadly) doesn't have an automated test suite. The closest there is is a page with references to the various functions that the CGI can perform. The page is an include file, so that it can be added to whatever else is found handy, e.g. via
<!--#include virtual="/cscf/internal/ST/debug/doc/internals/testing.ihtml" -->
The "ShowOnly" button (in the tests) is handy when repeated tests would cause repeated changes that would have to be (repeatedly) undone. The "MaxPerms" selection is good for testing problems that only happen for other than privileged users.
We occasionally have seen generated queries causing the database server to spin. On database.cs, as database administrator, use
mysqladmin processlist
If it reports "rt" (or "ST") copying to temp file, and ST seems unresponsive,
then the odds are the offending query has to be killed
(via mysqladmin kill ...).
ST #93842
provides an example.
In theory one should never have to edit the database directly,
if only because the `st-sql` command provides the
same database access that CGI and command line tools use.
However, record deletion can't be done that way.
That's rarely needed, however it can be handy when debugging
inadvertently or unavoidably creates unwanted records or transactions.
That (and other edits) can be done via direct access as database
administrator, executing
In addition, there'sdelete from each_req where serial_num = SerialNumber; delete from transactions where serial_num = SerialNumber;
https://cs.uwaterloo.ca/phpMyAdmin/
which can be handy if there's a need to edit something in place.
We have two email gateways, one of which processes email and remails it to the other.
Both write attachments to
/software/odyssey_apache/data/vhosts/cs.uwaterloo.ca/cscf/internal/edocs/rtattach/
However the needed permissions are different, as described below.
This gateway was made to handle attachments, and delete included messages. It runs only on the @cs mail server, there is no MFCF version.
The incoming mail is logged via `procmail` into
/u/rt-em-gw/procmail.log
The output of the gateway program itself
/u/rt-em-gw/emailgw/bin/emailgw.plis written (via the .procmailrc) to
/u/rt-em-gw/emailgw/log
There appears to be no provision to roll either of the logs, so beware of disk quota problems with the "rt-em-gw" account.
The main gateway
/software/rt-math-1/servers/st-mail-gateway
resides on the @cs MX handler (see `host -t MX cs.uwaterloo.ca.`). It logs to
/software/rt-math-1/logs/mailgateThat's local to the mail servers (perhaps it should be regional). It can be read by those in (the very historical) group "meeting". That should probably change to a better name.
As an aid to diagnosing gateway problems, test messages can be provided to the gateway via the command line, e.g.
log=SomeLogFile cp /dev/null $log chown rt $log /software/rt-math-1/servers/st-mail-gateway_debug -v -Z -n -e $log <$TestMessage
This is similar to the @cs gateway, although it resides on the @math MX handler. The catch is that it doesn't handle attachments, as nobody has picked a spot for it. And doing so could run the risk of attachments being in unexpected places for MFCF/CSCF collaboration on a single record.
The "rt-em-gw" gateway requires permissions for userid "rt-em-gw" on the attachments directories. The "ST-comment" gateway requires permissions for userid "rt". In theory that's setup properly by both gateways. However we've seen cases where people manually create the directories, and get the permissions wrong. The common need is handled by both userids being in group "www_cf", with the containing directory being setgid to propagate the group, and the gateways providing group read/write/execute.
/software/odyssey_apache/data/vhosts/cs.uwaterloo.ca/cscf/internal/edocs/rtattach/.ReadMeprovides advice for those who create the directories manually.