RSTPublicTests Suggested Improvements

• There seems to be an implicit nice applied when public tests are launched, probably due to nohup; however, this nice should be explicit, as in nohup nice +6 ....
• Request logs are currently not implemented. This should be appended to the files by the runners by caching the timestamp of the request file before it is deleted.
• The public interface pub_test_request can be bypassed and the course account interface utilized directly because it requires permission 6555. There should be a cs-marks group guard on the course-account-specific version to prevent this. So, the public interface could initially be a stub script that is world-executable; this in turn would fork to executables on the different architectures which are setgid cs-marks. These would do intensive input checking and verification of the course-account files. They would then execute the course account executables (now 4550 instead of 6555), which would pick up the course-specific effective userid, and then call the main request logging code as is currently done.
• It's arguable that the public test runner should only e-mail something like Public test runner $identifier reported errors, which may have resulted in incorrect public test results or prevented tests from completing or being e-mailed to you. Please try again, and contact course staff if the errors persist. instead of dumping the actual error output, which can be retrieved from the log files by course staff on request. * Add the following documentation instruction for pub_test_request: Make sure that the course account bin is world executable (as well as every parent directory) so the executable can be found. This is only necessary without the permission change in the point above; if that is implemented, these directories only need to be executable by the cs-marks group.

CSCF involvement

CSCF has already copied a particular state of pub_test_request so that it will automatically be available in the standard PATH for all students on student.cs systems. The question of how to push future updates to this command has still not been answered.

The question is whether or not they should take on more involvement. Right now, pub_test_request calls a command that must be put on the course accounts manually to get setuid status before calling pub_test_logger. Instead, it should be possible for this to run as root and drop down to course account permissions to run pub_test_logger without an executable being placed on the course accounts. This would simplify use of the command from the instructional standpoint.

The other potential involvement would be for the pub_test_runner executables. Right now, every course launches N of these on each server, where N is the number of processors on that server. Most of the time, these processes are idling and doing unnecessary polling. Instead, it should be possible for root to launch this pool of processes, and for each of them to drop down to the appropriate course to search for requests and service them if necessary. Again, this decreases the burden on the course accounts in terms of monitoring daemon status; however, it does mean that the pollers need to do more work (read configuration files on each account dropdown, check appropriate directory vs. reading configuration once at startup; the advantage though is automatic configuration refresh). There is also the issue that the intent is for each course to choose a particular platform on which to launch the test runners. Root would have to run them on all platforms, and then the course configuration would need a way to state which server family it wanted requests serviced on so only the appropriate runner would take action.

The issue is there would be a weird interplay between the portion of the public tests under CSCF's umbrella and the portion under ISG's umbrella; however, this already exists with the absorption of pub_test_request.