Linux Working Group

Meeting Date

• TEAMS: 2024-01-10

Invited

Anthony (group leader), Lori, Dave, O, Clayton, Guoxiang, Nathan, Nick, Todd, Ed, Devon

Attendees

Anthony (group leader), Clayton, Nick, Todd, Ed, Devon, Fraser

Review and accept previous meeting minutes.

CsLWGMeeting20231213

Review last meeting's Action Items

CS Mailservers are going away (eventually)

• Mail servers persist due to CS Advisors. Will begin soft shutdown by blocking all hosts other than IST's mail appliance (which forwards the advising mail to mx.cs) and dropping all non-advising mail.
• os upgrades needed
• csadviso@cs.uwaterloo.ca special forwarding will continue working, consulting with IST and Brad Lushman (a2brenna) * Nick is looking to replace the forwarding script with Microsoft Power Automate (M365) * Demo what has been completed to advisors, IST has been notified, advisors need to provide feedback * No feedback at this time

Ongoing problems with Inventory and IPAM are hobbling Infrastructure operations - RT1285291

• Inventory is unaware of this IP / domain limitations in IPAM as well as DHCP and MAC address requirements
  • Invalid records were imported from Infoblox that work until they are edited
  • public domain that resolved to private IP, modifying this record will break the record
• Some CSCF do not have access to create manual DNS entries (Devon, Lori, Guoxiang, Todd have access. Dave?)
• Inventory bug: Changing room field on a record with IPAM DNS & DHCP causes DHCP to break
• Anthony to reach out to IST for clarification regarding is this a policy vs technological limitation.
  • Delayed due to lack of staff time
• Need CSCF management to take over this ticket

What's still using old MySQL?

NextCloud (Vault) migration completed

Web server (includes Inventory)

• needs OS (whole LAMP stack) to be updated (Nathan, Isaac)
  • When?

Retire CS-GENERAL and associated domain controllers

• Last user is Vault
  • Vault upgrade needs to be performed -upgrade NextCloud(?), then fix AD
  • Vault migration from GENERAL to CS-GENERAL may take place after upgrade. Nathan to determine what is priority
  • Why can't vault switch domains to GENERAL? (a2brenna) - File space in vault is mapped to the user's UUID. Clayton has provided a mapping from GENERAL to CS-GENERAL.

Ongoing problems with Ganesha service RT#1303795

• needs further enhancements to monitoring service?
  • Devon and Anthony to preparing doc for help desk
    • Delayed due to lack of staff time
  • More comprehensive monitoring of NFS performance is in the works (a2brenna, dmerner) ~ January
    • Delayed due to lack of staff time

Monitoring Services

• Number of false alerts is a concern.
• Container networking will not survive a reboot
• Lack of Service Maintenance outside of standard working hours has been more of a problem lately.
  • Management is aware and need to review this.

More reliable usage data needed for labs (Mac and Linux) RT #1284635

• • Other faculties are interested in using labs
    • Need to document this in an accessible spot
  • Are there instructors / events (i.e. contests, workshops, WICS) that require the use of these labs that we can report on?

linux.cscf.uwaterloo.ca

• New linux.cscf.uwaterloo.ca running Ubuntu 22.04 is almost ready
  • ready for CSCF testing shortly
  • Switch over first week of Jan
    • Delayed due to lack of staff time

Incremental backups of block devices

• Possible solutions include rsync and borg but neither is ideal
• gxshen to investigate Legato NetWorker backups of block devices

Snapshots are still disabled

• a2brenna to enable snapshots on a file system to test performance - hoping to be done before start of next term
  • Delayed due to lack of staff time
  • There are new performance anomalies that make this less likely to happen until they are understood and/or fixed (a2brenna).
• communication should be sent at the beginning of the term to inform users of the current status
• In the meantime, more frequent backups of select directories (course accounts) are being arranged, see RT #1312411

Postmortem of chilled water outage

• Needs to be scheduled - CSCF Management
  • Did not happen. At this point any attempt would be increasingly difficult/pointless as too much time has passed, memories fade, the timeline of events becomes impossible to accurately construct etc. I'm not even disappointed because the word disappointment would require that I expected a different outcome (a2brenna).

SE undergrad users to be created under CS-TEACHING (RT?)

• Relationship between SE and CSCF needs to be better understood (bring up at next staff meeting)
• Any custom tools / software needed?

Comments