CF Web>Software>MediaWiki (2013-06-06, DrewPilcher)

EditAttach

MediaWiki

This page is intended to document the aspects of MediaWiki that USG members are interested in learning about.

With that said, anyone who finds it interesting is welcome to read it.

MediaWiki

What is MediaWiki?

General

"MediaWiki is a free software open source wiki package written in PHP, originally for use on Wikipedia. It is now also used by several other projects of the non-profit Wikimedia Foundation and by many other wikis, including the...home of MediaWiki."

"MediaWiki is server software."

"... There may be bugs or security problems. Do not install MediaWiki unless you intend to keep up with security upgrades (please subscribe to receive announcements of security updates)."

"The program is primarily developed to run on a large server farm for Wikipedia and its sister projects. Features, performance, configurability, ease-of-use, etc. are designed in this light..."

"No guarantee or warranty of any kind is provided."

University of Waterloo

MFCF has a MediaWiki server on which we can set up wikis for people.
As of Oct 4, 2010, USG supported MediaWiki version 1.16.0, which was used for CS 100.
As of Oct 13 2010, MediaWiki was hosted on the www.student.cs server

Section's last update : Jul 14 2012

Dependencies

Twiki Upgrade

"Intall recent version of Twiki on new www.cs web server"
"Service: General Computing Environment - WWW"
"Component: WWW - Editor"
"http://twiki.org/cgi-bin/view/TWiki04x02/TWikiUpgradeGuide indicates the upgrade [to the newer version of Twiki] will require some planning. It may be a good opportunity to review other Wiki software, including

MediaWiki. "

See also : Thu, Mar 26 2009 10:25 entry of https://cs.uwaterloo.ca/cscf/internal/request/UpdateRequest?56585

MediaWiki Database

"There is a custom table "watiam_user" in the MediaWiki database which maps

the WatIAM userid, MediaWiki username and realname, UW ID, and WatIAM names together. " -- Isaac Morland

See also : Mon, Mar 8 2010 13:59 entry of https://cs.uwaterloo.ca/cscf/internal/request_beta/UpdateRequest?67706

Who are the MediaWiki clients?

USG

The term's CS100 instructors, staff, and students.
Students use MediaWiki to work on modules and assignments

Research

USG Tasks Instructions

Setup for CS 100

Note : In some of the following instructions, the nicknames are used for creating accounts. However, the Instructional Support Coordinator of Dec. 2012 has requested that we avoid using nicknames for CS 100. Click on the following link for an example of instructions where nicknames are not used: https://cs.uwaterloo.ca/twiki/view/CF/MediaWiki#Final_Solution

1. Install MediaWiki -

Set up Web Server with MediaWiki for CS 100

1. Need to setup vmservers with VMWare ESXi https://www.vmware.com/products/esxi/

"These three hosts are to provide redundant services for identified essential CS Teaching services (web, database smtp, imaps, ...)

It is important to have the web server with media-wiki setup for CS100 ASAP. "

vmserver002.cs - Sun 4600 in MC 3015 rack ?? vmserver004.cs - Sun 4440 in MC 3015 rack ?? (currently in DC 2560g) vmserver006.cs - Sun 4600 in DC 2303a rack G2

lom-vmserver006.cs 10.15.18.105 NetMgnt dc2303a-cs1a, port 47 vmserver006.cs 172.19.34.142 ETH0 dc2303a-cs1 , port a-18 (VLAN??)

lom-vmserver004.cs 10.15.28.34 NetMgnt mc-cs2, port L16 vmserver004.cs 172.19.34.141 ETH0 mc-cs2, port L17 (VLAN??)

lom-vmserver002.cs 10.15.28.35 NetMgnt mc3015-cs2a, port 7 vmserver002.cs 172.19.34.140 ETH0 mc-cs2, port I22 (VLAN??)

1834 dc-cs-vm1net 172.19.34.0/25

lom-vmserver006.cs 10.15.18.105 NetMgnt dc2303a-cs1a, port 47 vmserver006.cs 172.19.34.142 ETH0 dc2303a-cs1 , port a-18 (vlan 1834)

lom-vmserver004.cs 10.15.28.34 NetMgnt mc-cs2, port L16 vmserver004.cs 172.19.34.141 ETH0 mc-cs2, port L17 (vlan 1834)

lom-vmserver002.cs 10.15.28.35 NetMgnt mc3015-cs2a, port 7 vmserver002.cs 172.19.34.140 ETH0 mc-cs2, port I22 (vlan 1834)

"Having problems with: lom-vmserver006.cs 10.15.18.105 NetMgnt dc2303a-cs1a, port 47 The switch see's it, but not accessible. However lom-vmserver106.cs is accessible.

I will look into it tomorrow morning. " -- pbeldows

"I reset the SP and re-entered the networking information: lom-vmserver006.cs is now accessible. root password changed to our standard pw for LOM. " -- pbeldows

"Figured out 802.1q tagging networking probles I was having.

In the networking tab on the hypervisor host, click the near top right corner add button.

Add a "Virtual Machine" Network (for each Vlan you are using. Leave the Vlan ID blank for the untagged vlan and enter the number for the tagged vlans) giving this "Virutal Machine" Network a "Network label".

Then go to your virtual host, and edit settings -> Harware -> Network Adapter and set the "Network Connection" to the "Network label" setup in the previous step.

I should have said the networking problems I was having with VMware ESXi-4.0 hypervisor setup.

See also : https://cs.uwaterloo.ca/cscf/internal/request_beta/UpdateRequest?68493 Setup vmserver002.cs vmserver004.cs and vmserver006.cs

2. Configure - Automatic Authentication and Account Creation (Done once)

Desired Outcome

"We need to decide how to handle user accounts and authentication for the new CS 100 MediaWiki. Wiki users must be able to be matched to WatIAM users and/or student IDs, and probably ought to exist automatically without students having to create them separately. "

See also : Wed, May 20 2009 11:54 entry of https://cs.uwaterloo.ca/cscf/internal/request_debug/UpdateRequest?67706

Steps

Authentication

http://wiki.case.edu/CaseWiki:External_Authentication seems to have information

that is on-point for configuring MediaWiki to use $REMOTE_USER, which is desirable if feasible.

"Information that I think is better refined is at: "

http://www.mollerus.net/tom/blog/2008/09/single_signon_to_mediawiki_113_using_active_direct.html

http://www.mediawiki.org/wiki/Extension:AutomaticREMOTE_USER

See also : Tue, Sep 22 2009 14:58 entry of https://cs.uwaterloo.ca/cscf/internal/request_debug/UpdateRequest?67706

Success Indicator

"Indeed the above seems to work nicely. I simply followed the instructions

exactly and when I reloaded the MediaWiki front page I was logged in as "Ijmorlan". A row in the mwuser table was created with user_name set to "Ijmorlan" i.e. my userid with the first character capitalized, and user_email set to ijmorlan@example.com. Unfortunately MediaWiki doesn't seem to have a concept of mapping the login userid to a MediaWiki username, which really ought to be something like "IsaacMorland" for me."

http://www.mollerus.net/tom/blog/2008/09/single_signon_to_mediawiki_113_using_active_direct.html

http://www.mediawiki.org/wiki/Extension:AutomaticREMOTE_USER

See also : Wed, Sep 23 2009 14:37 entry of https://cs.uwaterloo.ca/cscf/internal/request_debug/UpdateRequest?67706

Account Creation/Auto-loading Users

1. For auto-loading users, it appears we want to set the following fields:

user_id - might as well use pdbid user_name - capitalized login userid, unless I can figure out an easy way to insert a mapping from WatIAM userid to MediaWiki name user_real_name - get from WatIAM data fields user_email - userid@uwaterloo.ca user_email_authenticated - maybe set to registration time user_options - probably should initialize same as auto-registration user_touched - probably doesn't matter user_registration - registration time user_editcount - set to 0 by auto-registration

It wouldn't be a bad idea to modify the preferences setting to reject changes to name and email address as those are supposed to be handled institutionally by WatIAM/Sun Identity Suite.

See also : Wed, Sep 23 2009 14:37 entry of https://cs.uwaterloo.ca/cscf/internal/request_debug/UpdateRequest?67706

2. Making autoCreate return false also works.

"It seems the "@example.com" comes from the initUser function. "

See also : Wed, Sep 23 2009 15:04 entry of https://cs.uwaterloo.ca/cscf/internal/request_debug/UpdateRequest?67706

"The Auth_remote_user_hook which is registered to intercept the authentication check grabs the username from the $REMOTE_USER and then passes it along - we should be able to map the name right there. "

We also need to look up the MediaWiki name from the userid whenever we use $REMOTE_USER, which also occurs in authenticate() and initUser().

"This assumes we let MediaWiki create the MediaWiki user upon first login.

It might be simpler to turn off auto-creation and simply load the mwuser table ourselves. Then I think we just need to translate the $REMOTE_USER in Auth_remote_user_hook() and in authenticate(), and we don't have to access our watiam data from PHP in order to populate the real name and email address.

We want to do this with a minimum of PHP code and a maximum of standard database operations.

5. On the other hand, auto-creation seems to create pages per user and might do other account setup stuff that is worthwhile, so the small extra effort may be worth it.

I've updated Auth_remoteuser.php to auto-create accounts but to translate the $REMOTE_USER. The translation returns the boolean false value in the event the userid is unknown and this causes a "Login Required" notice to appear, which is tidy enough for our purposes. "

6. At present I have the following extra table in the MediaWiki schema to record legitimate potential users:

Note

The format needed to be changed because it clashed with the Twiki language.
See the Mon, Nov 16 2009 11:51 entry of https://cs.uwaterloo.ca/cscf/internal/request_debug/UpdateRequest?67706, or the "mediawiki.watiam_user" attachment for the original information documented in S.T.
mediawiki.watiam_user.pdf: mediawiki.watiam_user mediawiki.watiam_user

cs100=> \d watiam_user Table "mediawiki.watiam_user"

Column	Type	Modifiers

pdbid	integer	not null
userid	text	not null
mw_username	text	not null
mw_realname	text
uw_id	text

Indexes: "watiam_user_pkey" PRIMARY KEY, btree (pdbid) "watiam_user_mw_username_key" UNIQUE, btree (mw_username) "watiam_user_userid_key" UNIQUE, btree (userid)

A query to obtain the basic class list information:

odyssey_prod=> set search_path to uwdata; odyssey_prod=> select pdbid, userid, nicknames, givennames, surname, uw_id from cls_offering natural join cls_section natural join regdata_student_class join uwdir_person_by_student_id using (uw_id) where (course_subject, course_catalog) = ('CS', '100') and term_id='1099' and section_type='E' and section_session <> 'PCS';

Note :

The format needed to be changed because it clashed with the Twiki language.
See the Fri, Sep 25 2009 10:48 entry of https://cs.uwaterloo.ca/cscf/internal/request_debug/UpdateRequest?67706, or the "A Query for the Basic Class List Information" attachment for the original information documented in S.T.
AQueryfortheBasicClassListInformation.pdf: A Query for the Basic Class List Information

pdbid	userid	nicknames	givennames	surname	uw_id

[....]

A query to obtain staff registration information:

odyssey_prod=> select pdbid, userid, nicknames, givennames, surname, null as uw_id from uwdata.uwdir_person where userid in ('ijmorlan', 'fwtompa', 'bmzister', 'dkeenan', 'echrzano');

Note :

The format needed to be changed because it clashed with the Twiki language.
See the Fri, Sep 25 2009 10:48 entry of https://cs.uwaterloo.ca/cscf/internal/request_debug/UpdateRequest?67706, or the "A Query for the Basic Class List Information" attachment for the original information documented in S.T.
* Aquerytoobtainstaffregistrationinformation.pdf: A query to obtain staff registration information: A query to obtain staff registration information

pdbid	userid	nicknames	givennames	surname	uw_id

26427	fwtompa	Frank	Frank W	Tompa
15156	ijmorlan	Isaac	Isaac	Morland
76235	dkeenan	Dawn	Dawn E	Keenan
70169	bmzister	Barbara	Barbara M	Daly
89020	echrzano	Edward	Edward F	Chrzanowski

(5 rows)

The mw_username needs to be computed by taking the person's "first name" (whatever that is) and concatenating it with their "last name" (whatever that is), with no spaces and beginning with an uppercase letter. We'll have to do a bit of work to decide how to compute those from the nicknames, givennames, and surname field in WatIAM.

The mw_realname needs to be the person's full name written out in some sort of reasonable way, with spaces between the names.

We can use \copy in psql to get the data out of odyssey_prod and into cs100.

7. I have loaded staff and student data into the watiam_user table. I decided to combine the raw name data from WatIAM with the mw_username and mw_realname fields in the same table, so we now have this table definition:

The format needed to be changed because it clashed with the Twiki language.
See the Wed, Sep 30 2009 14:17 entry of https://cs.uwaterloo.ca/cscf/internal/request_debug/UpdateRequest?67706, or the "mediawiki.watiam_user-2" attachment for the original information documented in S.T.
mediawiki.watiam_user.pdf: mediawiki.watiam_user-2

cs100=> \d watiam_user Table "mediawiki.watiam_user"

Column	Type	Modifiers
pdbid	integer	not null
userid	text	not null
mw_username	text	.
mw_realname	text	.
uw_id	text	.
nicknames	text	.
givennames	text	.
surname	text	.

Indexes: "watiam_user_pkey" PRIMARY KEY, btree (pdbid) "watiam_user_mw_username_key" UNIQUE, btree (mw_username) "watiam_user_userid_key" UNIQUE, btree (userid)

To obtain staff data :

odyssey_prod=> \copy (select pdbid, userid, nicknames, givennames, surname, null as uw_id from uwdata.uwdir_person where userid in ('zdjarnol', 'dbleskie', 'jtbrown', 'd4chow', 'kjtpond', 'nmavande', 'arkane', 'j6weber', 'fwtompa', 'ssafay', 'bmzister', 'ijmorlan', 'dkeenan', 'echrzano')) to cs100.txt

To obtain student data :

odyssey_prod=> set search_path to uwdata; SET odyssey_prod=> \copy (select pdbid, userid, nicknames, givennames, surname, uw_id from cls_offering natural join cls_section natural join regdata_student_class join uwdir_person_by_student_id using (uw_id) where (course_subject, course_catalog) = ('CS', '100') and term_id='1099' and section_type='E' and section_session <> 'PCS') to cs100.txt

And each time, to load into the mediawiki database:

cs100=> \copy watiam_user (pdbid, userid, nicknames, givennames, surname, uw_id) from cs100.txt

3. Configure - Restrict Access (Done once)

Desired Outcome

"An issue is how to handle people who aren't supposed to have access. Right now I just have "require valid-user" in the .htaccess, but that lets in lots of people, and MediaWiki by default will create an account for somebody it hasn't seen. "

See also : Wed, Sep 23 2009 14:59 entry of https://cs.uwaterloo.ca/cscf/internal/request/UpdateRequest?67706

8. Created function to compute "real name" for MediaWiki purposes:

CREATE OR REPLACE FUNCTION compute_mw_realname ( nicknames text, givennames text, surname text ) RETURNS text AS $$ DECLARE first text; last text; BEGIN last := replace (trim (surname), '-', ''); IF nicknames IS NOT NULL THEN first := replace (trim (nicknames), '-', ''); IF substring (first from 1 + length (first) - length (last)) = last THEN first := trim (substring (first from 1 for length (first) - length (last))); IF first = '' THEN first = replace (trim (givennames), '-', ''); END IF; END IF; ELSE first := replace (trim (givennames), '-', ''); END IF; first = replace (first, '-', ''); RETURN upper (substring (first from 1 for 1)) || substring (first from 2) || ' ' || last; END; $$ LANGUAGE PLPGSQL IMMUTABLE;

Not perfect but does a pretty good job of dealing with messy data. Then:

cs100=> update watiam_user set mw_realname = compute_mw_realname (nicknames, givennames, surname); UPDATE 265 cs100=> update watiam_user set mw_username = replace (mw_realname, ' ', ''); UPDATE 265

Then, since students should not go in yet:

cs100=> delete from watiam_user where uw_id is not null; DELETE 251

(but unless somebody whose record causes a problem adds the course this week, we should be fine)

9. cs100=> \copy watiam_user (pdbid, userid, nicknames, givennames, surname, uw_id) from cs100.txt cs100=> select uw_id is null, count(*) from watiam_user group by uw_id is null;

?column?	count

	238
t	14

(2 rows)

cs100=> update watiam_user set mw_realname = compute_mw_realname (nicknames, cs100(> givennames, surname) where mw_realname is null; UPDATE 238 cs100=> update watiam_user set mw_username = replace (mw_realname, ' ', '') where mw_username is null; UPDATE 238 cs100=>

Steps

In Auth_remoteuser.php it seems I can adjust the userExists function to return false rather than true. When I do so it lets me in but only if I already have an mwuser entry. So we can initialize mwuser from registration data and WatIAM, use this setting to disallow automatic creation of other accounts.

See also : Wed, Sep 23 2009 14:59 entry of https://cs.uwaterloo.ca/cscf/internal/request_debug/UpdateRequest?67706

4. Ask the Instructional Support Coordinator (ISC) (Done every term)

Which instance do you wish to use? E.g. "For this term, we will be using the wiki-b instance."
Which users need to be added?
Format of usernames to expect from the Instructional Support Coordinator: e.g., ijmorlan (for Isaac Morland); bmzister (for Babara Daly)
When should which users be added? Students? Tutors? Instructors?
Are additional instances required for development?

See also :

https://cs.uwaterloo.ca/cscf/internal/request_beta/UpdateRequest?85213 (Set up CS 100 MediaWiki for W13)

https://cs.uwaterloo.ca/cscf/internal/request_beta/UpdateRequest?79903 (Set up CS 100 MediaWiki for W12)

https://cs.uwaterloo.ca/cscf/internal/request_beta/UpdateRequest?83263 (Set up CS 100 MediaWiki for F12)

https://cs.uwaterloo.ca/cscf/internal/request_beta/UpdateRequest?81210 (Set up CS 100 MediaWiki for S12)

Section's last update : Oct. 9, 2012 (F12)

5. Set Up a Regular Instance (Done every term)

Staff and Students Together (If so requested)

When was this done? Fall 2012 procedure

Note : the Instructional Support Coordinator may request that staff and students be added together.

1. Start by copying source test instance data to 1129 schema:

Procedure used for Fall 2012 cs100=> alter schema mediawiki_b rename to mediawiki_1129; ALTER SCHEMA

06:05 root@services024.student.cs# cd ~cs100/Wiki\ Dumps/ 06:05 root@services024.student.cs# setuid cs100 pg_dump -h postgres.student.cs --file=cs100_wiki_1129.dmp --format=c --schema=mediawiki_1129

cs100=> alter schema mediawiki_1129 rename to mediawiki_b; ALTER SCHEMA

06:06 root@services024.student.cs# setuid cs100 pg_restore -h postgres.student.cs --dbname=cs100 --single-transaction cs100_wiki_1129.dmp

06:08 root@services024.student.cs# cd ~cs100/public_html/ 06:11 root@services024.student.cs# mkdir wold 06:11 root@services024.student.cs# mv w wold 06:11 root@services024.student.cs# /bin/cp -rp wtest/b w

See also : Tue, Oct 9 2012 06:24 entry of https://cs.uwaterloo.ca/cscf/internal/request/UpdateRequest?83263

2. Then edit w/LocalSettings.php until:

06:21 root@services024.student.cs# diff wold/w/LocalSettings.php w/LocalSettings.php 72c72 < $wgDBmwschema = "mediawiki_1125"; --- > $wgDBmwschema = "mediawiki_1129";

3. Now populate users.

4. Now set up staff:

odyssey_prod=> \copy (select pdbid, userid, nicknames, givennames, surname, null as uw_id from uwdata.uwdir_person where userid in ('jwbeare','njrocha','acczhang','k36zhang','yczhong','cribeiro','swforsyt')) to cs100-staff.txt

cs100=> set search_path to mediawiki_1129, public; SET cs100=> delete from watiam_user where userid not in ('bmzister', 'ijmorlan'); DELETE 4 cs100=> \copy watiam_user (pdbid, userid, nicknames, givennames, surname, uw_id) from ./cs100-staff.txt cs100=> update watiam_user set mw_realname = compute_mw_realname (null, givennames, surname); UPDATE 9 cs100=> update watiam_user set mw_username = replace (mw_realname, ' ', ''); UPDATE 9 cs100=> select * from watiam_user;

Note :

The format needed to be changed because it clashed with the Twiki language.
The following names may be term-specific
See the Oct 9 2012 06:32 entry of https://cs.uwaterloo.ca/cscf/internal/request_beta/UpdateRequest?83263, or the "MediaWiki Tables" attachment for the original information documented in S.T.
MediaWikiSetupTables.pdf: MediaWiki Setup Tables

pdbid	userid	mw_username	mw_realname	uw_id	nicknames	givennames	surname

70169	bmzister	BarbaraMDaly	Barbara M Daly	.	Barbara	Barbara M		Daly
15156	ijmorlan	IsaacMorland	Isaac Morland	.	Isaac	Isaac		Morland
170235	cribeiro	CristinaRibeiro	Cristina Ribeiro	.		Cristina		Ribeiro
944044548	yczhong	YiChenZhong	Yi Chen Zhong	.	Annie	Yi Chen		Zhong
49739	swforsyt	SimonForsyth	Simon Forsyth	.		Simon		Forsyth
165946	jwbeare	JamesWalterBeare	James Walter Beare	.		James Walter		Beare
526259415	k36zhang	KaijieZhang	Kaijie Zhang	.	Crystal	Kaijie		Zhang
125493788	njrocha	NivenJosephRocha	Niven Joseph Rocha	.	Niven	Niven Joseph		Rocha
394279936	acczhang	AndyChienChiangZhang	Andy Chien Chiang Zhang	.	Andy	Andy Chien	Chiang	Zhang

(9 rows)

5. Set up students:

odyssey_prod=> set search_path to uwdata, public; SET odyssey_prod=> \copy (select pdbid, userid, nicknames, givennames, surname, uw_id from cls_offering natural join cls_section natural join regdata_student_class join uwdir_person_by_student_id using (uw_id) where (course_subject, course_catalog) = ('CS', '100') and term_id='1129' and section_type='E' and section_session <> 'PCS') to cs100-students.txt

cs100=> \copy watiam_user (pdbid, userid, nicknames, givennames, surname, uw_id) from cs100-students.txt cs100=> update watiam_user set mw_realname = compute_mw_realname (null, givennames, surname) where mw_realname is null; UPDATE 241 cs100=> update watiam_user set mw_username = replace (mw_realname, ' ', '') where mw_username is null; UPDATE 241

See also : Tue, Oct 9 2012 06:38 entry of https://cs.uwaterloo.ca/cscf/internal/request_beta/UpdateRequest?83263 Original S.T. ticket

Section's last update : Oct. 9, 2012

6. Additional Instances If applicable, set up more instances authorized for the staff to carry out development.

Staff First, Then students (If so requested)

Staff

Staff: Example One

odyssey_prod=> \copy (select pdbid, userid, nicknames, givennames, surname, null as uw_id from uwdata.uwdir_person where userid in ('fwtompa','d4chow','sfortune','r2fox','njrocha','acczhang','bmzister','ijmorlan')) to cs100-staff.txt

cs100=> alter SCHEMA mediawiki_1119 rename to mediawiki_1119_temp_renamed; ALTER SCHEMA

10:29 root@student.cs# cd /u2/cs100/Wiki\ Dumps/ 10:33 root@student.cs# export -n PGSERVICE 10:33 root@student.cs# setuid cs100 pg_restore -h postgres.student.cs --dbname=cs100 --single-transaction cs100_wiki_1119.dmp

cs100=> alter SCHEMA mediawiki_1119 rename to mediawiki_1121; ALTER SCHEMA cs100=> alter SCHEMA mediawiki_1119_temp_renamed rename to mediawiki_1119; ALTER SCHEMA cs100=> set search_path to mediawiki_1121; SET cs100=> delete from watiam_user; DELETE 8 cs100=> \copy watiam_user (pdbid, userid, nicknames, givennames, surname, uw_id) from ~/cs100-staff.txt cs100=> update watiam_user set mw_realname = compute_mw_realname (null, givennames, surname); UPDATE 8 cs100=> update watiam_user set mw_username = replace (mw_realname, ' ', ''); UPDATE 8 cs100=> select * from watiam_user;

Note :

The format needed to be changed because it clashed with the Twiki language.
The following names may be term-specific.
See the Fri, Jan 27 2012 10:42 entry of https://cs.uwaterloo.ca/cscf/internal/request_beta/UpdateRequest?79903, or the "MediaWiki Tables" attachment for the original information documented in S.T.
MediaWikiSetupTables.pdf: MediaWiki Setup Tables

pdbid	userid	mw_username	mw_realname	uw_id	nicknames	givennames	surname

125493788	njrocha	NivenJosephRocha	Niven Joseph Rocha	.	Niven	Niven Joseph	Rocha
147066300	r2fox	RebeccaFox	Rebecca Fox	.	Becca	Rebecca	Fox
394279936	acczhang	AndyChienChiangZhang	Andy Chien Chiang Zhang	.	Andy	Andy Chien	Chiang Zhang
106457	d4chow	DarrickChow	Darrick Chow	.		Darrick	Chow
26427	fwtompa	FrankWTompa	Frank W Tompa	.	Frank	Frank W	Tompa
70169	bmzister	BarbaraMDaly	Barbara M Daly	.	Barbara	Barbara M	Daly
560369125	sfortune	ScottFortune	Scott Fortune	.	Scott	Scott Fortune
15156	ijmorlan	IsaacMorland	Isaac Morland	.	Isaac	Isaac	Morland

(8 rows)

cs100=>

Then updated ~cs100/public_html/w/LocalSettings.php changing 1119 to 1121 in one place.

See also : Fri, Jan 27 2012 10:42 entry of https://cs.uwaterloo.ca/cscf/internal/request/UpdateRequest?79903

Staff: Example Two from Feb.2010

List of tutors/TAs from the Instructional Support Coordinator

James Beare jwbeare Janice Brown jtbrown Darrick Chow d4chow Busra Hacioglu bhaciogl Candace Lau c25lau Nick Maldonado jnmaldon Sonica Sharma s34sharm Claire Grison cegrison Robert Fraser r3fraser Scott Maclean smaclean Crisina Ribeiro cribeiro Andrew Kane arkane

"Updated editors for this term, in both "mediawiki" schema (restore of dump from before last term's students were loaded) and "mediawiki_1099" schema (last term's Wiki):"

odyssey_prod=> \copy (select pdbid, userid, nicknames, givennames, surname, null as uw_id from uwdata.uwdir_person where userid in ('bmzister', 'fwtompa', 'jwbeare', 'jtbrown', 'd4chow', 'bhaciogl', 'c25lau', 'jnmaldon', 's34sharm', 'cegrison', 'r3fraser', 'smaclean', 'cribeiro', 'arkane')) to cs100.txt

cs100=> delete from watiam_user; DELETE 14 cs100=> \copy watiam_user (pdbid, userid, nicknames, givennames, surname, uw_id) from cs100.txt cs100=> update watiam_user set mw_realname = compute_mw_realname (nicknames, givennames, surname); UPDATE 14 cs100=> update watiam_user set mw_username = replace (mw_realname, ' ', ''); UPDATE 14 cs100=> set search_path to mediawiki_1099; SET cs100=> delete from watiam_user; DELETE 253 cs100=> \copy watiam_user (pdbid, userid, nicknames, givennames, surname, uw_id) from cs100.txt cs100=> update watiam_user set mw_realname = compute_mw_realname (nicknames, givennames, surname); UPDATE 14 cs100=> update watiam_user set mw_username = replace (mw_realname, ' ', ''); UPDATE 14

"Add the Instructional Support Coordinator on as editors for the restored dump and for last term's renamed Wiki." -- Isaac Morland

See also : Thu, Jan 28 2010 14:15 entry of https://cs.uwaterloo.ca/cscf/internal/request/UpdateRequest?67706

Students: No Duplicate Names

08:03 root@student.cs# cd /u2/cs100/Wiki\ Dumps/ 08:04 root@student.cs# export -n PGSERVICE 08:04 root@student.cs# setuid cs100 /software/postgresql-8.4/bin/pg_dump -h postgres.student.cs --file=cs100_wiki_1121.dmp --format=c --schema=mediawiki_1121

odyssey_prod=> set search_path to uwdata; SET odyssey_prod=> \copy (select pdbid, userid, nicknames, givennames, surname, uw_id from cls_offering natural join cls_section natural join regdata_student_class join uwdir_person_by_student_id using (uw_id) where (course_subject, course_catalog) = ('CS', '100') and term_id='1121' and section_type='E' and section_session <> 'PCS') to cs100-students.txt odyssey_prod=>

cs100=> \copy watiam_user (pdbid, userid, nicknames, givennames, surname, uw_id) from cs100-students.txt cs100=> update watiam_user set mw_realname = compute_mw_realname (null, givennames, surname) where mw_realname is null; UPDATE 336 cs100=> update watiam_user set mw_username = replace (mw_realname, ' ', '') where mw_username is null; UPDATE 336

See also : Tue, Jan 31 2012 08:11 entry of https://cs.uwaterloo.ca/cscf/internal/request/UpdateRequest?67706

Students: Duplicate Names

11:38 root@student.cs# setuid cs100 /software/postgresql-8.4/bin/pg_dump -h postgres.student.cs --file=cs100_wiki_1109.dmp --format=c --schema=mediawiki_1109

odyssey_prod=> \copy (select pdbid, userid, nicknames, givennames, surname, uw_id from cls_offering natural join cls_section natural join regdata_student_class join uwdir_person_by_student_id using (uw_id) where (course_subject, course_catalog) = ('CS', '100') and term_id='1109' and section_type='E' and section_session <> 'PCS') to cs100-students.txt

cs100=> set search_path to mediawiki_1109; SET cs100=> \copy watiam_user (pdbid, userid, nicknames, givennames, surname, uw_id) from cs100-students.txt cs100=> update watiam_user set mw_realname = compute_mw_realname (null, givennames, surname) where mw_realname is null; UPDATE 236 cs100=> update watiam_user set mw_username = replace (mw_realname, ' ', '') where mw_username is null; UPDATE 236

"The Instructional Support Coordinator requested to add students, so did same procedure as indicated in step one (except with [the] term updated to current term). Had a duplicate name, so replaced last step with:

cs100=> update watiam_user set mw_username='SamKim' where uw_id='20389285'; UPDATE 1 cs100=> update watiam_user set mw_username = replace (mw_realname, ' ', '') where mw_username is null; UPDATE 738

(using WatIAM nickname of one student instead of their official givennames) " -- Isaac Morland

See also : Tue, Feb 1 2011 14:55 entry of https://cs.uwaterloo.ca/cscf/internal/request_debug/UpdateRequest?67706

4. Inform the Instructional Support Coordinator of the account names

"There are two students "Min Tae Kim" so one of them (mt2kim, 20389285) has "SamKim" for username (based on WatIAM nickname). Other than that MediaWiki names are based on real names. Dump file is in the same "Wiki Dumps" directory as previous terms'. "

Explanation: How to Deal with Duplicate Names

"This term there are some students with duplicate names. There are two named Kathy Nguyen; I handled this by inserting the digits from the UW Userid into their MediaWiki realnames and usernames. Additionally, there are two named Hanna Lee. One of them has "Hanna" as a nickname so I inserted her 'givenname' into her realname and used it instead of 'Hanna' for her username. " -- Isaac Morland

Note : This method of dealing with duplicate names causes problems. Click on this link for the solution: https://cs.uwaterloo.ca/twiki/view/CF/MediaWiki#Caused_by_Duplicate_Names

See also : Thu, Feb 11 2010 10:19 entry of https://cs.uwaterloo.ca/cscf/internal/request/UpdateRequest?67706

By Re-loading the "Blank" Database from the Previous Term

Add the staff first, and then add the students.

1. Ask the Instructional Support Coordinator if the database should be set by re-loading the `blank` database from [the previous term].` -- Isaac Morland

See also : The Mon, May 31 2010 10:34 entry of https://cs.uwaterloo.ca/cscf/internal/request_beta/UpdateRequest?67706

2. If the Instructional Support Coordinator agrees, proceed with:

setuid cs100 /software/postgresql-8.4/bin/pg_restore -d cs100 -h postgres.student.cs cs100_wiki_1101.dmp

MediaWiki now comes up.

Ask the Instructional Support Coordinator for the "list of individuals requiring admin access". Before step 3 is done, "the previous term's admin people have admin access". --Isaac Morland

E.g.

Darrick Chow Janice Brown Robert Fraser Andrew Kane Crisitna Ribiero Suzana Pinto

d4chow jtbrown r3fraser arkane cribeiro s2pinto

4. Add the list of administrators, and the ISC (Instructional Support Coordinator)

cs100=> delete from watiam_user ; DELETE 14 cs100=> \copy watiam_user (pdbid, userid, nicknames, givennames, surname, uw_id) from cs100-1105-staff.txt cs100=> update watiam_user set mw_realname = compute_mw_realname (nicknames, givennames, surname); UPDATE 8 cs100=> update watiam_user set mw_username = replace (mw_realname, ' ', ''); UPDATE 8 cs100=> select * from watiam_user;

pdbid	userid	mw_username	mw_realname	uw_id	nicknames	givennames	surname
--------	----------	-----------------	------------------	-------	-----------	-------------------------	---------
57799	arkane	ArkaneKane	Arkane Kane		arkane	Andrew Robertson Joseph	Kane
170235	cribeiro	CristinaRibeiro	Cristina Ribeiro			Cristina	Ribeiro
106457	d4chow	DarrickChow	Darrick Chow			Darrick	Chow
15156	ijmorlan	IsaacMorland	Isaac Morland		Isaac	Isaac	Morland
65144	jtbrown	JaniceBrown	Janice Brown		Janice	Janice Teresa	Brown
6197	s2pinto	SuzanaPinto	Suzana Pinto		Suzana	Suzana	Pinto
99404	r3fraser	BobFraser	Bob Fraser		Bob	Robert	Fraser
70169	bmzister	BarbaraDaly	Barbara Daly		Barbara	Barbara M	Daly

(8 rows)

5. Students:

08:50 root@student.cs# setuid cs100 /software/postgresql-8.4/bin/pg_dump -h postgres.student.cs --file=cs100_wiki_1105.dmp --format=c --schema=mediawiki

odyssey_prod=> set search_path to uwdata; SETodyssey_prod=> \copy (select pdbid, userid, nicknames, givennames, surname, uw_id from cls_offering natural join cls_section natural join regdata_student_class join uwdir_person_by_student_id using (uw_id) where (course_subject, course_catalog) = ('CS', '100') and term_id='1105' and section_type='E' and section_session <> 'PCS') to cs100b.txt

cs100=> \copy watiam_user (pdbid, userid, nicknames, givennames, surname, uw_id) from cs100.txt cs100=> update watiam_user set mw_realname = compute_mw_realname (nicknames, cs100(> givennames, surname); UPDATE 139 cs100=> update watiam_user set mw_username = replace (mw_realname, ' ', ''); UPDATE 139

"This should enable all the CS 100 students to get into the MediaWiki. "

This section's last update : Jun. 2010

See also: Mon, May 31 2010 10:34 entry of https://cs.uwaterloo.ca/cscf/internal/request_debug/UpdateRequest?67706

Set Up an Instance for Testing & Development (If requested for a term)

Ask the Instructional Support Coordinator

Which wikis do you want users to be authorized for? E.g. current instances may include test instances a, b, and c.

Set Up an Instance

Success Indicator

Examples of schemas mediawiki_[abc] and corresponding installations that have been set up :

These live in a directory ~cs100/public_html/wtest.

Authorize the same staff for them as for the main instance. "All are

copies of what we started from for this term's main instance. "

See also :

Fri, Jun 8 2012 13:00 entry of https://cs.uwaterloo.ca/cscf/internal/request_beta/UpdateRequest?83169 original S.T. item
https://cs.uwaterloo.ca/cscf/internal/request_beta/UpdateRequest?81210 set up CS100 MediaWiki for S12

Section's last update : Aug. 28, 2012

Set Up an Account for Testing on the CS100 Wiki (If requested)

An account used for testing is also known as a "plain account". This account allows "course staff [to] log into the MediaWiki and pretend to be a student for testing purposes." -- Isaac Morland

See also : Mon, Nov 16 2009 11:58 entry of https://cs.uwaterloo.ca/cscf/internal/request_debug/UpdateRequest?67706

Ask the Instructional Support Coordinator

What should the account name and password be?

Setup

1. If there is no WatIAM account for testing, create one. Also, tell the Instructional Support Coordinator that in the meantime, testing can be done "by changing a tutor, a TA, or the Instructional Support Coordinator temporarily to have no special privileges." -- Isaac Morland Note : An Instructional Support Coordinator has declined this suggestion in the past.

2. Request Adrian Pepper to create a WatIAM userID.

Why is this step needed? "Authentication itself is directly tied to WatIAM. In turn, Wiki identities are identified with the external identificaition, so there is no way to say [a particular person] is allowed to pretend to be "Test Student", which is essentially what we need." -- Isaac Morland
cs100=> insert into watiam_user values (289588667, 'cs-t000', 'TestStudent',

'Test Student', '20054321', 'Test', 'Test', 'Student'); INSERT 0 1

4. Communicate specifications for the account

"The account is 'hidden' for Uwdir purposes" -- Adrian Pepper
Specify the account is only to allow a WatIAM login. No student.cs account is needed.

3. Determine the test student account name

The account created in Nov. 2009 was cs-t000.
Later accounts should be 001,002, ...

4. Request Adrian Pepper to add a comment on the account.

The comment should be: "CS Test Account; currently used by CS 100 MediaWiki"

5. Create the MediaWiki eligible user record so if they log into the MediaWiki [using the current account name, for example, CS-t000], it should work properly." -- Isaac Morland

6. Request Adrian Pepper to activate the account, which includes setting the password

"Something simple which we can pass along to the CS 100 staff" - Isaac Morland

7. Request that Adrian Pepper send the password to you, or to the CS 100 staff.

See also : Wed, Nov 18 2009 14:38 entry of https://cs.uwaterloo.ca/cscf/internal/request_debug/UpdateRequest?67706

Success Indicator

Note : The following usernames and ID numbers may be specific to the account created in Nov. 2009 "If the Instructional Support Coordinator logs in using the password, the MediaWiki should recognize the instructor as 'Test Student' with ID number xxxxxxxx". (The ID number has been anonymized for security purposes. In reality, give the instructor the actual ID number.)

See also : Fri, Nov 20 2009 11:12 entry of https://cs.uwaterloo.ca/cscf/internal/request_debug/UpdateRequest?67706

Get Feedback (every term)

Let the client know that staff should test out MediaWiki before opening it up to the students.

Setup for an Individual

Note : USG members have been responsible for this task in the past.

1. Ask Client

1. If clients ask for Wikis set up on their own accounts, ask if they mind for it to be set up on the MFCF account instead. Reason "I don't think PHP is supported on the webspace provided to grad students by default. You can check if you can run PHP on your account or not by creating a simple PHP file with the content."

< ?php echo "Hello World!"; ? >

and opening it with the browser. Most probably it will show an error.

This page: http://www.math.uwaterloo.ca/mfcf/faq/www_author.html#www/scripts_php.faq seems to contain information about how to make PHP work.

See also : Fri, Jul 13 2012 15:26 entry of https://cs.uwaterloo.ca/cscf/internal/request/UpdateRequest?83826

Original Request "I'm trying to set up MediaWiki(http://www.mediawiki.org/wiki/MediaWiki) on my Waterloo Unix account, username: mkotowsk. However, I get an "internal server error" when I try to access the directory where I extracted the tar.gz package with MediaWiki software (http://www.math.uwaterloo.ca/~mkotowsk/mediawiki/mediawiki-1.19.1/). According to [the] installation guide, [it] should work all right.

I'm not sure if something is misconfigured on the server side or maybe running MediaWiki under SunOS requires some extra footwork, anyway this is the install page: http://www.mediawiki.org/wiki/Installation, which lists the requirements (I think they should be satisfied here, as faras PHP and SQL are concerned, but you probably know better)."

See also : Fri, Jul 13 2012 14:48 entry of https://cs.uwaterloo.ca/cscf/internal/request/UpdateRequest?83826

2. Wiki's name

What would you like to call your wiki?

2. Actual Setup

1. Following steps at Installing_MediaWiki...

- created "affine-connections" under /var/www/html/mediawiki/wikis/ - created symlinks to /var/www/corefiles/mediawiki/mediawiki-1.13.2 since that's what research-jk wiki has - removed config and images symlinks and made directories - copied sample index.php to config/ and chmodded - updated /etc/httpd/conf/httpd.conf to include alias for affine-connections

Now it says to use a browser and go to the new wiki instance. Sounds like that's the part that Chris said doesn't work. Yep, "not found". Need to kick Apache to notice httpd.conf change? Yes, "/etc/init.d/httpd reload" made the page appear, and it gives the expected link to "please set up the wiki first".

Filled out the web form after staring at some other configurations. Need various userids and passwords from the password vault. [Misinterpreted] which ones were supposed to be for this wiki instance and which for permission to act. Will have to fix that, so did not move LocalSettings.php up a level yet.

Went on to set up the skins. That's it for now.

See also : Fri, Oct 5 2012 17:23 entry of https://cs.uwaterloo.ca/cscf/internal/request/UpdateRequest?83826

2. It appears that the userid/password I set in the web interface that then appeared in the LocalSettings.php file as $wgDBuser and $wgDBpassword resulted in essentially a no-op in the MySQL database because they were already there.

They're not associated with the affine-connections* tables in the database -- those tables have the mkotowksi userid/password that I set elsewhere in the web form.

They are (will be) what the MediaWiki back-end will use when communicating with the MySQL database, and there seems to be no real need for them all to be different across the different MediaWikis, as they're not what the people use, they're what the MediaWiki back end uses. But we'll try to follow the pattern of using different ones nevertheless.

So we can probably just create another userid/password pair in MySQL with the same permissions as all the others, and edit those into the LocalSettings.php file, and then proceed with the rest of the set-up steps.

3. Userid:

mysql> grant usage on mediawikidb.* to acwikiuser@localhost; Query OK, 0 rows affected (0.00 sec)

mysql> grant usage on mediawikidb.* to acwikiuser@localhost.localdomain; Query OK, 0 rows affected (0.00 sec)

mysql> grant usage on mediawikidb.* to acwikiuser@'%'; Query OK, 0 rows affected (0.00 sec)

mysql> grant select,insert,update,delete,create temporary tables on mediawikidb. * to acwikiuser@localhost; Query OK, 0 rows affected (0.00 sec)

mysql> grant select,insert,update,delete,create temporary tables on mediawikidb. * to acwikiuser@localhost.localdomain; Query OK, 0 rows affected (0.00 sec)

mysql> grant select,insert,update,delete,create temporary tables on mediawikidb. * to acwikiuser@'%'t; Query OK, 0 rows affected (0.00 sec)

Password (read the config file to see what it is) :

mysql> set password for acwikiuser = password('*******'); Query OK, 0 rows affected (0.00 sec)

mysql> set password for acwikiuser@localhost = password('*******'); Query OK, 0 rows affected (0.00 sec)

mysql> set password for acwikiuser@localhost.localdomain = password('*******'); Query OK, 0 rows affected (0.00 sec)

4. Resuming the configuration steps...

Moved the LocalSettings.php file into place. Moved the config dir aside. Verified ScriptPath.

Finally, "Instance Server Authentication" This section talks about expected permissions on the images and skins directories. Looks like people have not followed this when setting up other mediawikis. Maybe I'll leave this as-is until Marcin indicates a need for it.

https://wiki.math.uwaterloo.ca/affine-connections/

5. Give the client the URL

https://wiki.math.uwaterloo.ca/affine-connections/

6. Give the client the UserID and password

Arrange a time to bring the client the userid and password.

See Also : Tue, Oct 16 2012 17:49 entry of https://cs.uwaterloo.ca/cscf/internal/request/UpdateRequest?83826

Update MediaWiki

"Setup is almost done. I successfully ran the MediaWiki update script to update the database schema to 1.16.0, starting from the re-loaded data from the beginning of last term. I need to adjust local settings bit and copy over the non-DB data (files and images) from the core region installation.

I'm also waiting for the list of userids of staff members who need access. I can finish the setup using just myself for testing but it won't help the actual CS 100 staff until I put them on. " -- Isaac Morland

Section's Last Update : Oct. 2010

See also : Fri, Oct 1 2010 15:56 entry of https://cs.uwaterloo.ca/cscf/internal/request_beta/UpdateRequest?67706

Delete Older Wiki

Request

Delete the older wiki from a previous term.

Steps

1. I saved a dump file in ~cs100@student.cs. 2. Then, I added the 682 current students to the MediaWiki and removed access to the old one.

See also : Thu, Feb 11 2010 10:36 entry of https://cs.uwaterloo.ca/cscf/internal/request/UpdateRequest?67706

Reset Password

Evidently I misremembered the password I set for his account. Verified that the userid is at least there:

mysql> use mediawikidb; mysql> desc affine_connections_user; mysql> select user_name,password from affine_connections_user;

Reset password using advice at

http://www.mediawiki.org/wiki/Manual:Resetting_passwords http://www.mediawiki.org/wiki/Manual:Maintenance_scripts

While cd to the affine-connections directory,

php /var/www/corefiles/mediawiki/mediawiki-1.13.2/maintenance/changePassword.php --conf ./LocalSettings.php --user=mkotowsk --password=

Visited his office again to verify he can now log in.

See Also : Wed, Oct 17 2012 14:36 entry of https://cs.uwaterloo.ca/cscf/internal/request/UpdateRequest?83826

Server-Upgrade Related

Server Upgrade Info

We are planning to upgrade www.student.cs similarly to www.cs (RT#68417). To do this, we need a student.cs region analogue to services120.cs (RT#44713). All the same considerations will apply. In particular, the odyssey-3_apache package will need to be present (RT#64487), which will require a custom xh-alter-requests setup unless a better solution can be found.
"[T]his is needed for MediaWiki (https://cs.uwaterloo.ca/cscf/internal/request_debug/UpdateRequest?68423) "

See also : Wed, Jul 22 2009 14:30 entry of https://cs.uwaterloo.ca/cscf/internal/request/UpdateRequest?68422

Web-Server Related

"We needed to move to the new www.student.cs and stop hosting teaching materials on the 'core region' www.cs web server."

Change of Web Servers

1. Provide the URL for the new MediaWiki installation.

2. Provide authentication information. E.g. "The new installation is authenticated using CAS, which requires the same WatIAM userid/password people should already be used to using."

3. Install a Redirect

"I mean to install a redirect so that requests for the old location will get redirected to the new location. I would suggest updating course materials in the fullness of time but it does not need to be done immediately as students will be smoothly redirected to the proper place. I have to run right now but I can put this in place next time I have a moment, this evening even; please let me know if there is any other issue besides the change in URL for the students. "

See also : Wed, Oct 6 2010 10:30 entry of https://cs.uwaterloo.ca/cscf/internal/request/UpdateRequest?67706

Note : Redirects can be removed.

See also : Tue, Jan 18 2011 14:42 entry of https://cs.uwaterloo.ca/cscf/internal/request_debug/UpdateRequest?67706

Purpose of This Step

"The new installation has meant that all [the course materials for CS 110] needed to be modified." -- Instructional Support Coordinator

Troubleshooting

MediaWiki Stopped Working

Final Solution

"It was working and then stopped working; the setting I adjusted is the maximum amount of memory allowed for the program /usr/bin/convert which is used to resize images (e.g. to make thumbnails). I adjusted it from an amount which I would characterize as "more than ample" to an amount which I would consider "absurdly excessive" (at least, for the size of most of the images we're dealing with). So why did I need to do that, and why did it work before? "

Warning : "This isn't really a solved problem and you'll need to be alert for recurrences - I can't really predict whether this is going to be a problem again. "

See also : Wed, Oct 13 2010 10:28 entry of https://cs.uwaterloo.ca/cscf/internal/request_debug/UpdateRequest?67706

Broken Links and Missing Pages

Problem Scenario

See also :

Service: Student Computing Environment

OS: Linux

Server: services024.student.cs

Requestor: ISG

> It would seem that the links or other pages are broken or missing. Is this the
> snapshot of the wiki that was done at the beginning of last term? We would
> like for it to simply be a rollover of the wiki prior to the students
> getting at it.
>
> The first image is missing with an error and none of the 6 or 7 random
> pages that I chose will open. It just sits and spins.

2. /usr/bin/convert won`t run

Diagnosis Questions

Possible Causes

Failure Modes

Some samples of the varying failure modes:

09:58 root@services024.student.cs# setuid cs100 ./ulimit4.sh.original 180 102400 102400 \'/usr/bin/convert\'\ -quality\ 80\ -background\ white\ -size\ 450\ \'/u2/cs100/public_html/w/images/4/44/Bamberger_Dom.jpg\'\ -thumbnail\ \'450x600\!\'\ -set\ comment\ \'File\ source:\ https://www.student.cs.uwaterloo.ca/~cs100/wiki/File:Bamberger_Dom.jpg\'\ -depth\ 8\ -sharpen\ \'0x0.4\'\ \'/u2/cs100/public_html/w/images/thumb/4/44/Bamberger_Dom.jpg/450px-Bamberger_Dom.jpg\'\ 2\>\&1 convert: Memory allocation failed `Cannot allocate memory' @ cache.c/AcquirePixelCacheNexus/294. 09:58 root@services024.student.cs# setuid cs100 ./ulimit4.sh.original 180 102400 102400 \'/usr/bin/convert\'\ -quality\ 80\ -background\ white\ -size\ 450\ \'/u2/cs100/public_html/w/images/4/44/Bamberger_Dom.jpg\'\ -thumbnail\ \'450x600\!\'\ -set\ comment\ \'File\ source:\ https://www.student.cs.uwaterloo.ca/~cs100/wiki/File:Bamberger_Dom.jpg\'\ -depth\ 8\ -sharpen\ \'0x0.4\'\ \'/u2/cs100/public_html/w/images/thumb/4/44/Bamberger_Dom.jpg/450px-Bamberger_Dom.jpg\'\ 2\>\&1 convert: MemoryAllocationFailed `Cannot allocate memory' @ cache.c/AcquirePixelCacheNexus/294. convert: MemoryAllocationFailed `Cannot allocate memory' @ hashmap.c/NewLinkedList/1411. ^C 09:59 root@services024.student.cs# setuid cs100 ./ulimit4.sh.original 180 102400 102400 \'/usr/bin/convert\'\ -quality\ 80\ -background\ white\ -size\ 450\ \'/u2/cs100/public_html/w/images/4/44/Bamberger_Dom.jpg\'\ -thumbnail\ \'450x600\!\'\ -set\ comment\ \'File\ source:\ https://www.student.cs.uwaterloo.ca/~cs100/wiki/File:Bamberger_Dom.jpg\'\ -depth\ 8\ -sharpen\ \'0x0.4\'\ \'/u2/cs100/public_html/w/images/thumb/4/44/Bamberger_Dom.jpg/450px-Bamberger_Dom.jpg\'\ 2\>\&1 convert: MemoryAllocationFailed `Cannot allocate memory' @ cache.c/AcquirePixelCacheNexus/294. convert: MemoryAllocationFailed `Cannot allocate memory' @ cache.c/AcquirePixelCacheNexus/294. Unexpected error. ./ulimit4.sh.original: line 4: 29334 Aborted '/usr/bin/convert' -quality 80 -background white -size 450 '/u2/cs100/public_html/w/images/4/44/Bamberger_Dom.jpg' -thumbnail '450x600!' -set comment 'File source: https://www.student.cs.uwaterloo.ca/~cs100/wiki/File:Bamberger_Dom.jpg' -depth 8 -sharpen '0x0.4' '/u2/cs100/public_html/w/images/thumb/4/44/Bamberger_Dom.jpg/450px-Bamberger_Dom.jpg' 2>&1 09:59 root@services024.student.cs# setuid cs100 ./ulimit4.sh.original 180 102400 102400 \'/usr/bin/convert\'\ -quality\ 80\ -background\ white\ -size\ 450\ \'/u2/cs100/public_html/w/images/4/44/Bamberger_Dom.jpg\'\ -thumbnail\ \'450x600\!\'\ -set\ comment\ \'File\ source:\ https://www.student.cs.uwaterloo.ca/~cs100/wiki/File:Bamberger_Dom.jpg\'\ -depth\ 8\ -sharpen\ \'0x0.4\'\ \'/u2/cs100/public_html/w/images/thumb/4/44/Bamberger_Dom.jpg/450px-Bamberger_Dom.jpg\'\ 2\>\&1 ^C 10:00 root@services024.student.cs# setuid cs100 ./ulimit4.sh.original 180 102400 102400 \'/usr/bin/convert\'\ -quality\ 80\ -background\ white\ -size\ 450\ \'/u2/cs100/public_html/w/images/4/44/Bamberger_Dom.jpg\'\ -thumbnail\ \'450x600\!\'\ -set\ comment\ \'File\ source:\ https://www.student.cs.uwaterloo.ca/~cs100/wiki/File:Bamberger_Dom.jpg\'\ -depth\ 8\ -sharpen\ \'0x0.4\'\ \'/u2/cs100/public_html/w/images/thumb/4/44/Bamberger_Dom.jpg/450px-Bamberger_Dom.jpg\'\ 2\>\&1 convert: Memory allocation failed `Cannot allocate memory' @ cache.c/AcquirePixelCacheNexus/294. 10:00 root@services024.student.cs#

Other Possible Causes

"There seems to be a problem with the "convert" program which generates some of the images (specifically, thumbnails, as well as the world map on the front page). I tested and was able to click around pages just fine until I happened to hit one that had a "convert" image on it; then all my page requests hung until I killed the "convert" process on the web server. Then they worked again. So, we're working on figuring out what is really going wrong. Essentially, MediaWiki was working fine yesterday but now it's running into trouble when it tries to invoke /usr/bin/convert. I've left an example running on services024.student.cs." We have:

CGI, pid 1136 index.php5, waiting to read a pipe, node 1552446 child of 1136, pid 1164, sh, child of 1164, pid 1165, ulimit4.sh (part of MediaWiki), child of 1165, pid 1166, convert, stdout/stderr are node 1552446, waiting on "futex(0x7f316cf5fd80, FUTEX_WAIT_PRIVATE, 2, NULL"; using strace -f I get four threads, pid 1166/7/8/9, all waiting on a similar futex call; no child processes

In other words, convert is completely wedged for some reason. If I "HUP" it it stops quickly enough and the page comes up with a weird error apparently from convert:

Error creating thumbnail: convert: MemoryAllocationFailed `Cannot allocate memory' @ cache.c/AcquirePixelCacheNexus/294. convert: MemoryAllocationFailed `Cannot allocate memory' @ exception.c/AcquireExceptionInfo/108.

I think there may be some caching going on because I've been able to load a page with such error message on it without causing wedging.

Further detail, all activity by a particular user gets wedged all at once because any other MediaWiki (index.php5) requests end up waiting on /var/lib/php5/sess_[session id visible in session cookie]. This is where the spinning described by [Barb, the Instructional Support Coordinator,] comes from.

Error creating thumbnail: convert: Memory allocation failed `Cannot allocate memory' @ cache.c/AcquirePixelCacheNexus/294. convert: Memory allocation failed `Cannot allocate memory' @ hashmap.c/NewLinkedList/1411. convert: magick/semaphore.c:272: DestroySemaphoreInfo: Assertion `(*semaphore_info)->signature == 0xabacadabUL' failed.

2. I'm a little suspicious of ~cs100/public_html/w/bin/ulimit4.sh - maybe limits are too low? But it was working before and it feels wrong to fix every problem by just raising limits higher than they were when things were working fine. Although it appears the actual limits come from MediaWiki when it invokes ulimit4.sh:

11:51 root@services024.student.cs# less bin/ulimit4.sh #!/bin/bash

ulimit -t $1 -v $2 -f $3 eval "$4" /usr/bin/convert won't run on services024.student.cs with memory limited to 100MiB even for small files;

http://www.imagemagick.org/discourse-server/viewtopic.php?f=3&t=17302

The "convert" problem sounds like an open ImageMagick bug: https://bugs.launchpad.net/ubuntu/+source/imagemagick/+bug/597469

4. Failure Modes Some samples of the varying failure modes:

Possible Solutions

Similar to Solaris "truss" functionality being available in Linux "strace", the Solaris "ptree" functionality should largely be available in "pstree".

Thanks, pstree gives some nice output, which also provides more details of exact parameters for further digging:

12:45 root@services024.student.cs# pstree -l -p -a 1136 index.php5,1136 index.php5 â””â”€sh,1164 -c '/u2/cs100/public_html/w/bin/ulimit4.sh'\040180\040102400\040102400\040''\''/usr/bin/convert'\''\040-quality\04080\040-background\040white\040-size\040100\040'\''/u2/cs100/public_html/w/images/e/e1/Fenderdr.jpg'\''\040-thumbnail\040'\''100x75!'\''\040-set\040comment\040'\''File\040source:\040https://www.student.cs.uwaterloo.ca/~cs100/wiki/File:Fenderdr.jpg'\''\040-depth\0408\040-sharpen\040'\''0x0.4'\''\040'\''/u2/cs100/public_html/w/images/thumb/e/e1/Fenderdr.jpg/100px-Fenderdr.jpg'\''\0402>&1' â””â”€ulimit4.sh,1165 /u2/cs100/public_html/w/bin/ulimit4.sh 180 102400 102400 '/usr/bin/convert'\040-quality\04080\040-background\040white\040-size\040100\040'/u2/cs100/public_html/w/images/e/e1/Fenderdr.jpg'\040-thumbnail\040'100x75!'\040-set\040comment\040'File\040source:\040https://www.student.cs.uwaterloo.ca/~cs100/wiki/File:Fenderdr.jpg'\040-depth\0408\040-sharpen\040'0x0.4'\040'/u2/cs100/public_html/w/images/thumb/e/e1/Fenderdr.jpg/100px-Fenderdr.jpg'\0402>&1 â””â”€convert,1166 -quality 80 -background white -size 100 /u2/cs100/public_html/w/images/e/e1/Fenderdr.jpg -thumbnail 100x75! -set comment File\040source:\040https://www.student.cs.uwaterloo.ca/~cs100/wiki/File:Fenderdr.jpg -depth 8 -sharpen 0x0.4 /u2/cs100/public_html/w/images/thumb/e/e1/Fenderdr.jpg/100px-Fenderdr.jpg â”œâ”€{convert},1167 â”œâ”€{convert},1168 â””â”€{convert},1169 (The "&9500;"-type stuff is RT double-escaping line-drawing characters)

13:08 root@services024.student.cs# identify images/e/e1/Fenderdr.jpg images/e/e1/Fenderdr.jpg JPEG 1024x768 1024x768+0+0 8-bit DirectClass 302kb 13:08 root@services024.student.cs# convert -quality 80 -background white -size 100 /u2/cs100/public_html/w/images/e/e1/Fenderdr.jpg -thumbnail 100x75! -set comment File\040source:\040https://www.student.cs.uwaterloo.ca/~cs100/wiki/File:Fenderdr.jpg -depth 8 -sharpen 0x0.4 test.jpg 13:09 root@services024.student.cs# ls -l test.jpg -rw-r----- 1 root root 3291 2010-10-06 13:09 test.jpg

The symptom looks similar to the noted bug, but the underlying issue seems different - I don't see any 2^32-1 or 2^32-2 in the dimensions anywhere as in that bug.

I kill -HUP'd a bunch of "convert" processes; some new ones came into existence almost immediately, I'm guessing because their parent process was actually trying to convert several images and so moved on to the next one as soon as I killed one. I repeated the process (except for the one I deliberately kept yesterday). Eventually all the backed-up index.php5 processes went away.

Then I requested another page and got the spinning behaviour back.

Taking into account the parameters with which it is called, ulimit4.sh runs:

ulimit -t 180 -v 102400 -f 102400

and then uses eval to run:

convert -quality 80 -background white -size 120 /u2/cs100/public_html/w/images/2/24/Altes_Rathaus_Regnitz.jpg -thumbnail 120x90! -set comment File\040source:\040https://www.student.cs.uwaterloo.ca/~cs100/wiki/File:Altes_Rathaus_Regnitz.jpg -depth 8 -sharpen 0x0.4 /u2/cs100/public_html/w/images/thumb/2/24/Altes_Rathaus_Regnitz.jpg/120px-Altes_Rathaus_Regnitz.jpg

The ulimit parameters mean 180s maximum CPU time, and 100MiB maximum virtual memory and size of written files. It seems clear that these should not be a problem.

I remembered seeing something on the MarkUs blog about dealing with convert gobbling memory. [The] error they were encountering was solved by using convert's limit option. http://blog.markusproject.org/?p=1623

Final Solution

Summary

1. Running the "original" version with the maximum virtual memory set to 1000MiB works. 2. Added:

$wgMaxShellMemory = 1024000;

to LocalSettings.php, and now everything works.

But it was working before with the lower setting, which should be perfectly sufficient.

So the immediate problem is sidestepped, but there is a non-web problem related to running /usr/bin/convert limited to a reasonable amount of memory.

Explanation

Increasing limit to 1000MiB makes it work, at least enough to run MediaWiki.

So this makes me think that Terry's suggestion might actually be on point. I had assumed that converting some of the little files we were working with couldn't possibly need 100MiB but apparently this is wrong. This also explains why I didn't jump to experiment with limits. Still, some simple setting of -limit options doesn't seem to produce useful results.

Should we consider rolling our own ImageMagick build, given the Ubuntu community package has known resource issues?

5 I set up a script to use strace to monitor the convert process. I replaced ulimit4.sh with:

Slow Response (non-emergency)

Problem Scenario

" The TAs noted slow response from MediaWiki last term, when 200 students were registered in the course. This term there are 700 students in CS200 and the MediaWiki component of the course will be released in the next couple of weeks. "

Diagnosis Questions

Possible Causes

"Should we consider rolling our own ImageMagick build, given the Ubuntu community package doesn`t work well?"

http://www.imagemagick.org/discourse-server/viewtopic.php?f=3&t=17302

Possible Solutions

At a minimum, we should check whether this is still a problem once we have upgraded to Ubuntu 12. I can't really say whether we should be building our own ImageMagick. Not a current project, however. --------------------- #!/bin/bash

outfile=~cs100/strace/$$

echo "--------" >>$outfile echo $(date '+%Y-%m-%d %T') $$ starting >>$outfile echo "cwd=$(pwd)" >>$outfile printf "args: %q %q %q %q\n" "$@" >>$outfile env >>$outfile echo "--------" >>$outfile

env -u HOME strace -f -ff -o $outfile ~cs100/public_html/w/bin/ulimit4.sh.original "$@"

echo "--------" >>$outfile echo $(date '+%Y-%m-%d %T') $$ finished >>$outfile echo "--------" >>$outfile ---------------------

This worked fine but didn't reveal anything really obvious.

Part of the difficulty is that there seems to be two failure modes: sometimes it reports an error and exits right away; other times it gets stuck and doesn't exit until it receives a signal (HUP or ^C are enough).

The original ulimit4.sh was renamed to ulimit4.sh.original. I also created ulimit4.sh.fake which was identical except the ulimit line was commented out.

From the command line I tried:

setuid cs100 ./ulimit4.sh.original 180 102400 102400 \'/usr/bin/convert\'\ -quality\ 80\ -background\ white\ -size\ 450\ \'/u2/cs100/public_html/w/images/4/44/Bamberger_Dom.jpg\'\ -thumbnail\ \'450x600\!\'\ -set\ comment\ \'File\ source:\ https://www.student.cs.uwaterloo.ca/~cs100/wiki/File:Bamberger_Dom.jpg\'\ -depth\ 8\ -sharpen\ \'0x0.4\'\ \'/u2/cs100/public_html/w/images/thumb/4/44/Bamberger_Dom.jpg/450px-Bamberger_Dom.jpg\'\ 2\>\&1

and:

setuid cs100 ./ulimit4.sh.fake 180 102400 102400 \'/usr/bin/convert\'\ -quality\ 80\ -background\ white\ -size\ 450\ \'/u2/cs100/public_html/w/images/4/44/Bamberger_Dom.jpg\'\ -thumbnail\ \'450x600\!\'\ -set\ comment\ \'File\ source:\ https://www.student.cs.uwaterloo.ca/~cs100/wiki/File:Bamberger_Dom.jpg\'\ -depth\ 8\ -sharpen\ \'0x0.4\'\ \'/u2/cs100/public_html/w/images/thumb/4/44/Bamberger_Dom.jpg/450px-Bamberger_Dom.jpg\'\ 2\>\&1

"The 'original' version doesn't work, although it seems unpredictable which of the two failure modes will apply. The 'fake' version just works, completing in no time."

See also : https://cs.uwaterloo.ca/cscf/internal/request_beta/UpdateRequest?74581 Section's last update :Apr 25 2012

Created Accounts Do Not Appear to the Instructional Support Coordinator

Problem Scenario

"It doesn't look like there are any student accounts created."

Final Solution

Explain the following to the Instructional Support Coordinator:

"Upon first login, MediaWiki will automatically create a MediaWiki account based upon the data in a custom table that I have populated. What I did [earlier] was to put the students in this custom table. Until they log in, however, they will not have actual MediaWiki accounts. "
"This method involved the least amount of custom code but allows us to use our standard authentication methods rather than having something MediaWiki-specific." -- Isaac Morland

This section was last updated on: Nov. 18, 2009

Incorrect Wiki Usernames

Caused by Duplicate Names

Problem Scenario

The following are people who have notified the CS instructors with concerns for their Wiki usernames:

Hanna Lee (h65lee)

* Could not find anything wrong with the username. She did not mention the name used but there is a wiki page for HannaLee. She may have thought it was supposed to be her WatIAM username.

See also : Tue, Mar 2 2010 03:59 entry of https://cs.uwaterloo.ca/cscf/internal/request/UpdateRequest?67706

Possible Causes

I discovered that in manually overriding the names for the two "Hanna Lee"s I switched the usernames, so the non-Jyy-Harn Lee was confused to see that in her username.

There are two Hanna Lee's. One has a WatIAM given name of "Hanna" while the other has "Hanna Lee" as nickname. This required a manual override and it appears I did it backwards - so Jyy-Harn (Hanna Lee) Lee had username "HannaLee" and Hanna Lee has username "JyyHarnLee".

See also : Tue, Mar 2 2010 04:03 entry of https://cs.uwaterloo.ca/cscf/internal/request/UpdateRequest?67706

Final Solution

So: swap the usernames back to how they should be.

cs100=> update watiam_user set mw_username='temp' where pdbid=198939; UPDATE 1 cs100=> update watiam_user set mw_username='HannaLee' where pdbid=191105; UPDATE 1 cs100=> update watiam_user set mw_username='JyyHarnLee' where pdbid=198939; UPDATE 1 cs100=> update mwuser set user_name='temp' where user_id=262; UPDATE 1 cs100=> update mwuser set user_name='JyyHarnLee' where user_id=410; UPDATE 1 cs100=> update mwuser set user_name='HannaLee' where user_id=262; UPDATE 1

(Ideally I would change the unique constraint to be deferrable and be able to update each table in one update but this gets the job done.)

See also : Tue, Mar 2 2010 04:03 entry of https://cs.uwaterloo.ca/cscf/internal/request/UpdateRequest?67706

Other Cases of Incorrect Usernames (non-emergency)

Problem Scenarios

The following are people who have notified the CS instructors with concerns for their Wiki usernames:

Lily Wu (l32wu)

* username on the wiki is: StephanieWu;

Ishani Malthi Sriskandha (misriska)

* username on the wiki is: MaliSriskandha;

See also : Tue, Mar 2 2010 03:59 entry of https://cs.uwaterloo.ca/cscf/internal/request_debug/UpdateRequest?67706

Possible Causes

For Lily Wu, I'm seeing "StephenWu". This is the WatIAM nickname at the time I set up

the accounts plus WatIAM surname.

For Ishani Malthi Sriskandha, again, this is WatIAM nickname at the time plus WatIAM surname.

See also : Tue, Mar 2 2010 04:03 entry of https://cs.uwaterloo.ca/cscf/internal/request/UpdateRequest?67706

Possible Solutions

Although MediaWiki is designed to use the nicknames, the WatIAM nicknames create confusion for the tutors. That's because names "don't necessarily match up with the names the [instructors and tutors] have on their lists (they don't get a copy of the students' WatIAM names)".
Do not use formal names because some people use their middle names.
"I would have been tempted to invent Wiki style names that contained people's WatIAM userid. That's the one name that everyone is expected to know. E.g. I could be UwcwinceU." -- Bill Ince

Final Solution

For the current term, where the Wiki has already been set up: give the tutors convenient access to the mapping table. "There is a custom table 'watiam_user' in the MediaWiki database which maps the WatIAM userid, MediaWiki username and realname, UWID, and WatIAM names together. " -- Isaac Morland

I've attached a dump of this table for your information. If you load it in a spreadsheet you should get a nice searchable table. This will at least allow you to figure out who everybody is; if you find this inconvenient then we can still re-think the name conversion process for next term. Of course, there will always be issues with this process since names aren't unique. " -- Isaac Morland

See also : Mon, Mar 8 2010 13:59 entry of https://cs.uwaterloo.ca/cscf/internal/request/UpdateRequest?67706

For future terms, use the userIDs as the usernames. The following is an example where the nicknames are not used.

odyssey_prod=> \copy (select pdbid, userid, nicknames, givennames, surname, null as uw_id from uwdata.uwdir_person where userid in ('ssafay','cegrison','d4chow','s34sharm','rkielstr','jnmaldon','bmzister')) to cs100-staff.txt odyssey_prod=>

cs100=> delete from watiam_user where userid<>'ijmorlan'; DELETE 7 cs100=> \copy watiam_user (pdbid, userid, nicknames, givennames, surname, uw_id) from ./cs100-staff.txt cs100=> update watiam_user set mw_realname = compute_mw_realname (null, givennames, surname); UPDATE 8 cs100=> update watiam_user set mw_username = replace (mw_realname, ' ', ''); UPDATE 8 cs100=> select * from watiam_user;

See also : Mon, Oct 4 2010 11:43 entry of https://cs.uwaterloo.ca/cscf/internal/request/UpdateRequest?67706

Note

The format needed to be changed because it clashed with the Twiki language.
See the Mon, Oct 4 2010 11:43 entry of https://cs.uwaterloo.ca/cscf/internal/request_debug/UpdateRequest?67706, or the "mediawiki.watiam_user" attachment for the original information documented in S.T.

pdbid	userid	mw_username	mw_realname	nicknames	givennames	surname
15156	ijmorlan	IsaacMorland	Isaac Morland		Isaac	Morland
1300	ssafay	SuzanneSafayeni	Suzanne Safayeni	Suzanne	Suzanne	Safayeni
18165	rkielstr	RonaldKielstra	Ronald Kielstra	Ron	Ronald	Kielstra
159718	cegrison	ClaireEvelynGrison	Claire Evelyn Grison		Claire Evelyn	Grison
152876	s34sharm	SonicaSharma	Sonica Sharma		Sonica	Sharma
109564	jnmaldon	JaimeNicolasMaldonado	Jaime Nicolas Maldonado	Nick	Jaime Nicolas	Maldonado
106457	d4chow	DarrickChow	Darrick Chow		Darrick	Chow
70169	bmzister	BarbaraMDaly	Barbara M Daly	Barbara	Barbara M	Daly

(8 rows)

cs100=>

Cannot Log in

Problem Scenario

"We have several students who cannot log into the Wiki."

Diagnosis Questions

What are their userids?
Are they able to log into CAS (i.e.if they go to https://cas.uwaterloo.ca/cas/login do they get a "log in successful" message?)?
Do they get some weird error message or does MediaWiki just claim they're not logged in (or something else)? )

See also : Thu, Oct 21 2010 11:37 entry of https://cs.uwaterloo.ca/cscf/internal/request_debug/UpdateRequest?67706

Possible Causes

No student accounts have been added.

Final Solution

If the student or staff member is authorized by the Instructional Support Coordinator, then add the account to the list.

See also : Wed, Jun 15 2011 11:35 entry of https://cs.uwaterloo.ca/cscf/internal/request_debug/UpdateRequest?67706

I	Attachment	History	Action	Size	Date	Who	Comment
docx	AQueryfortheBasicClassListInformation.docx	r1	manage	12.9 K	2012-11-30 - 12:02	LilySong	A Query for the Basic Class List Information
pdf	AQueryfortheBasicClassListInformation.pdf	r1	manage	109.1 K	2012-11-30 - 12:02	LilySong	A Query for the Basic Class List Information
docx	Aquerytoobtainstaffregistrationinformation.docx	r1	manage	13.2 K	2012-11-30 - 12:08	LilySong	A query to obtain staff registration information
pdf	Aquerytoobtainstaffregistrationinformation.pdf	r1	manage	117.9 K	2012-11-30 - 12:08	LilySong	A query to obtain staff registration information
docx	MediaWikiSetupTables.docx	r1	manage	14.8 K	2012-11-28 - 10:34	LilySong	Media Wiki Setup Tables
pdf	MediaWikiSetupTables.pdf	r1	manage	129.8 K	2012-11-28 - 10:33	LilySong	Media Wiki Setup Tables
docx	mediawiki.watiam_user.docx	r1	manage	12.9 K	2012-11-30 - 11:56	LilySong	mediawiki.watiam_user
pdf	mediawiki.watiam_user.pdf	r1	manage	109.5 K	2012-11-30 - 11:56	LilySong	mediawiki.watiam_user
docx	mediawiki.watiam_user2.docx	r1	manage	13.0 K	2012-12-06 - 10:34	LilySong	mediawiki.watiam_user2
pdf	mediawiki.watiam_user2.pdf	r1	manage	110.5 K	2012-12-06 - 10:35	LilySong	mediawiki.watiam_user2