DFSc Working Group



Meeting Date

  • TEAMS: 2022-06-16

Invitees - Attendees

  • Dave, Gouxiang, Lori, Nathan, Nick, Lawrence

Review and accept previous meeting minutes.

Proposed Agenda Items

Old business

Action items for this meeting 2022-06-16

  • ldpaniak/nfish: Ceph upgrade on the 902s
  • Omar: Fraser - create a ticket for the plans for local storage option -> https://rt.uwaterloo.ca/Ticket/Display.html?id=1209206
  • Anthony: Any plans to deal with cephfs client crashes on teaching systems? - yc2lee/gxshen - RT#1214857 (revert to 5.4 kernel)
  • Clayton: NFS servers to ganesha 4.0
  • Guoxiang: need to get a full backup before the Ceph upgrade
  • Lori: disable/reduce new snapshots in anticipation of Ceph upgrade (?)

New business

  • Access to DFSc performance counters, RT#1211673 dlgawley
    • Dave's not here - leave for next meeting

Upgrades

Start with 902 systems. Practice upgrades, work out bugs nfish/a2brenna/ldpaniak

  • in progress

Server side

  • Want to upgrade to Pacific by end of summer at the latest: strays, upgrade path, less OSD spill from RocksDB(sharding) currently:3, 30, 300GB..., mclock scheduler, graphana daemons. Octopus out of support 2022-06-01: Early May?
  • One MDS problem
  • Remove all snapshots?
    • in preparation for the upgrade
    • Anthony: we should ensure we have complete backups before that
    • Guoxiang: running low on disk space for index files, backups are slow, new hardware has problems, doesn't know when he can do the full backup (pretty soon?)
  • Real downtime with low/no cluster load
  • ceph deploy deprecated: Migrate to ceph-adm (docker containerization) then upgrade
    • maybe not as deprecated as originally thought
    • probably can't use ceph-adm
    • will try on the 902s
  • splitting cs-teaching into multiple real filesystems: u0-u19(?)

Client side

Scratch(ish) drives on 211 systems (fhgunn)

  • ZFS sends for sync

Action items for next meeting

  • Back up email on DFSC/CS systems on August 5 (dlgawley , gxshen)
  • Start retiring pre-current term snaps (before May) (ldpaniak, nfish) https://rt.uwaterloo.ca/Ticket/Display.html?id=1228804
  • New snapshots continue for now as required by courses (a2brenna, yc2lee)
  • Revisit snapshot removal schedule at next DFSC meeting for the purposes of mid-August DFSC upgrade
Edit | Attach | Watch | Print version | History: r4 < r3 < r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r4 - 2022-06-16 - LoriPaniak
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback