Linux Working Group

Meeting Date

• TEAMS: 2022-01-12

Invitees - Attendees

• Adrian, Anthony (group leader), Clayton, Guoxiang, Lori, Fraser, Devon, Nathan, Nick, Todd, Dave, Lawrence, Omar

Review and accept previous meeting minutes.

• CsLWGMeeting20211215

Review last meeting's Action Items

• Anthony/Adrian - work on new postfix recipe to have servers send mail out directly - RT#1204074
  • Adrian and Anthony have been discussing new recipe
  • some concerns about mail forwarding from back-end systems
  • question re: originating as "userid@uwaterloo.ca"
  • what about some of the csi-xxx@cs emails? csi-haproxy@cs csi-dfsc@cs csi-software@cs could setup: csi-cs@uwaterloo.ca csi-cs+software@uwaterloo.ca
    • for receiving "rsg-xxx-yyy@cs" emails, we created cs-rsg{+address}@uwaterloo.ca
  • does IST have a relay?
    • bulkmail.uwaterloo.ca if have stored passwords
  • question about authenticated email from the command line
• Anthony/Guoxiang - initiate warranty - RT#1196085 - under warranty until 2025-03-30
  • https://rt.uwaterloo.ca/Ticket/Display.html?id=1196085
  • Guoxiang has updated the firmware
  • no problems recently,
  • still to-do
• Lawrence to create a ticket for Devon with a big list of Host Groups and contacts - needs to be done manually by Devon
  • RT#1144232
  • Lawrence to follow-up with Devon
• RT#1194157 - /opt/csw
  • Guoxiang - tell us the size of the filesystem -> 9GB
  • Nathan - create an appropriate CephFS
    • done, it is now on DFSc
  • Clayton - create the NFS share
    • to-do
• Devon - create ticket to document power issues to show Plant Operations
  • done?
  • Anthony will follow-up with Devon
• Lori/Nathan - consider whether we can recover 960GB Optane cards from 422 systems
  • Was for OdysseyDB setup
  • because hardware slow to arrive?
  • current machines have enough space, so we don't need to do this
• Guoxiang/Lori - create or report# ticket for Storage option catalogue (low priority)
  • still to-do
• Clayton - document process of adding hosts to AD and move to a generally accessible place - create a ticket
  • still to-do
• Lawrence to send out email to SCS everybody starting at time 1pm, expected end time 5pm
  • email sent
  • updates and reboots happened
  • Zpools behaved poorly during the process
  • RT#1201464 documents the problems "Extreme ZFS performance degradation"
    • kudos to Fraser for his assistance with this issue

Proposed Agenda Items

Issues with DFSc

• approaching the issue on 3 fronts
  • test system on a block Rados device as a possible alternative solution
  • possibly use the previous NetApp for one of the home filesystems as a comparison
    • Guoxiang - would require some updates of the NetApp system
    • Anthony - would like to do some initial testing
    • Anthony - create a ticket to request Guoxiang to create a test volume on the NetApp
  • continuing to tweak Ceph parameters to avoid lockups / performance issues
    • are all NFS/Ganesha servers updated?
    • Clayton says yes, but there is an issue with one of the machines
    • old ones need to be shutdown/decommissioned
    • testing / comparison of performance in various scenarios - see RT#1194172 for home directory performance in CS-General and CS-Teaching

Action Items for next meeting

• Anthony - create a ticket for documentation of AD process -> assign to Clayton
• Anthony - create a ticket to request Guoxiang to create a test volume on the NetApp
• Anthony - create a ticket to document experiments with network file performance
• Anthony/Adrian - work on new postfix recipe to have servers send mail out directly - RT#1204074
• Anthony/Guoxiang - initiate warranty - RT#1196085 - under warranty until 2025-03-30
• Clayton - decommission the older NFS / Ganesha servers
• Clayton - document process of adding hosts to AD and move to a generally accessible place
• Clayton - create NFS share /opt/csw - RT#1194157 - /opt/csw
• Devon - create ticket to document power issues to show Plant Operations - done? Anthony will follow-up
• Guoxiang/Lori - create or report# ticket for Storage option catalogue (low priority)
• Lawrence - discuss continued use of the NetApp with MFCF
• Lawrence / RSG - update jerusalem and graceland to mount new NFS share