tml.cs, tml2.cs and tml3 hardware build notes and software configuration

Contacts

• These machines are owned by Yaoliang Yu
• Managed by Mike Gore

Remote Admin Access

• ssh cscf-adm@tml.cs.uwaterloo.ca (F20 password)
• ssh cscf-adm@tml2.cs.uwaterloo.ca (F20 password)
• ssh cscf-adm@tml3.cs.uwaterloo.ca (F19 password)

Local Admin Access

• tml.cs
  • cscf-adm (2020 password)
• tml2.cs
  • cscf-adm (2019 password)
• tml2.cs
  • cscf-adm (2019 password)

Inventory Records

• Inventory lookup: tml.cs.uwaterloo.ca
• Inventory lookup: tml2.cs.uwaterloo.ca
• Inventory lookup: tml3.cs.uwaterloo.ca

TML hardware inventory

• Motherboard: ASUS WS X299 SAGE
• Intel(R) Core(TM) i9-7920X CPU @ 2.90GHz
• Memory: 64GB (4 x 16GB Corsair CMK32GX4M2A2666C16 2133MHZ)
• Disks: Samsung NVME 2TB SSD
• GPU GeForce RTX 2080 and GeForce GTX 1080

TML2 hardware inventory

• Motherboard: ASUS X399 Taichi
• CPU: AMD Ryzen Threadripper 2920X 12-Core Processor
• Memory: 48GB (3 x 16GB Corsair CMK32GX4M2A2666C16 2133MHZ)
• Disks: Samsung NVME 1TB SSD
• GPU GeForce RTX 2080

TML3 hardware inventory

• Motherboard: ASUS WS X299 SAGE
• CPU: Intel(R) Core(TM) i9-7940X CPU @ 3.10GHz
• Memory: 64GB (4 x 16GB Corsair CMK32GX4M2A2666C16 2133MHZ)
• Disks: Samsung NVME 1TB SSD
• GPU
  • NVIDIA Corporation GV100 TITAN V
  • NVIDIA Corporation GV100 TITAN V

Notes on the purchase of original TML

• The main constraint in the design of this machine was to permit up to 4 liquid cooled GPU cards in one chassis.
• The biggest problem we faced is that most GPU cards are air cooled from the side - putting 4 every other slot would obstruct the fans
• I9 System with 64G ram (expandable to 128G)
• Mother board needs at least 7 slots and the chassis must have 8 slots (GPU cards are two slots wide) the
  • GPU cards plug into slots: 1,3,5,7
• Build is only cost effective if you plan to add the 4 GPU's - currently the CPU is 35% of the overall cost
• Overall this system runs quiet as all of the fans ar 120mm and because of the large radiators they do not need to run fast

Parts for original TML

• GPU - liquid cooled
  • EVGA GeForce GTX 1080 Ti FTW3 Hybrid
  • https://www.amazon.ca/EVGA-GeForce-Hybrid-Gaming-Technology/dp/B074D7S8HR
  • Notes: This card came with the liquid cooler - however many of the EVGA GPU cards can been be upgraded with EVGA liquid cooler add ons
  • FYI: Any 11G ram GPU car is going to be very hard to find in stock
• Case - lots of room for liquid cooling radiators
  • Item#: N82E16811133338
  • Thermaltake Core X5 Tempered Glass Edition E-ATX Stackable Black Cube Chassis CA-1E8-00M1WN-02
  • https://www.newegg.ca/Product/Product.aspx?reviews=all&Item=N82E16811133338
  • Notes: This case is stack-able - is wide enough that the motherboard is mounted horizontally
  • I really liked working with this case for this configuration - there are a number of places to put liquid cooling radiators
• Motherboard - 7 slot, lots of PCIe lanes and two NVMe disk slots
  • MFG #: WS X299 SAGE - Asus WS X299 SAGE Workstation Motherboard
  • http://www.directdial.com/WS%20X299%20SAGE.html
• CPU - I9
  • MFG #: BX80673I97940X
  • Intel Core i9 i9-7940X Tetradeca-core (14 Core) 3.10 GHz Processor
  • http://www.directdial.com/BX80673I97940X.html
• CPU cooler - liquid cooled
  • MFG #: CW-9060032-WW
  • Corsair H115i PRO Cooling Fan/Radiator
  • http://www.directdial.com/CW-9060032-WW.html
• RAM - must be purchased in groups of 4
  • Corsair Vengeance LPX 32GB (2x16GB) DDR4 DRAM 2666MHz C16 Memory Kit
  • MFG #: CMK32GX4M2A2666C16
  • http://www.directdial.com/CMK32GX4M2A2666C16.html
• NVMe disk
  • MFG #: MZ-V7E2T0BW
  • Samsung 970 EVO MZ-V7E2T0BW 2 TB Internal Solid State Drive PCI Express - M.2 2280
  • http://www.directdial.com/MZ-V7E2T0BW.html
• Power Supply
  • MFG #: 120-G2-1600-X1
  • EVGA SuperNOVA 1600 G2 Power Supply
  • http://www.directdial.com/120-G2-1600-X1.html
  • For up to 4 GPUs at max workload

Software Documentation

• *Note - I moved the GPU setup scripts to CF/GPUInstallScripts TWIKI has they are updated frequently

TML Pictures

• TML - with covers off power supply side view:
  

• TMP with covers off rear top view:
  

• TML covers off CPU side view:
  

Install scripts

• *Note - I will be moving the GPU setup scrips to CF/GPUInstallScripts

GPU install scripts

• GPUInstallScripts