TML_GPU < CF

CF Web>Research>GPUComputerBuilds>TML_GPU (2023-01-09, TomCauduro) (raw view)

-- Main.MikeGore - 2018-12-03
%TOC%
---++ tml.cs,  tml2.cs and tml3 hardware build notes and software configuration

---++ Contacts
   * These machines are owned by Yaoliang Yu
   * Managed by Mike Gore

---++ Remote Admin Access
   * ssh cscf-adm@tml.cs.uwaterloo.ca (F20 password)
   * ssh cscf-adm@tml2.cs.uwaterloo.ca (F20 password)
   * ssh cscf-adm@tml3.cs.uwaterloo.ca (F19 password)

---++ Local Admin Access
   * tml.cs
      * cscf-adm (2020 password) 
   * tml2.cs
      * cscf-adm (2019 password) 
   * tml2.cs
      * cscf-adm (2019 password) 

---++ Inventory Records
   * [[https://www.cs.uwaterloo.ca/cscf/internal/inventory/search.php?&submit=Search&shostDomainName=tml.cs.uwaterloo.ca][Inventory lookup: tml.cs.uwaterloo.ca]]
   * [[https://www.cs.uwaterloo.ca/cscf/internal/inventory/search.php?&submit=Search&shostDomainName=tml2.cs.uwaterloo.ca][Inventory lookup: tml2.cs.uwaterloo.ca]]
   * [[https://www.cs.uwaterloo.ca/cscf/internal/inventory/search.php?&submit=Search&shostDomainName=tml3.cs.uwaterloo.ca][Inventory lookup: tml3.cs.uwaterloo.ca]]

---++ TML hardware inventory
   * Motherboard: ASUS WS X299 SAGE
   * Intel(R) Core(TM) i9-7920X CPU @ 2.90GHz
   * Memory: 64GB (4 x 16GB Corsair CMK32GX4M2A2666C16 2133MHZ)
   * Disks: Samsung NVME 2TB SSD 
   * GPU *GeForce RTX 2080* and *GeForce GTX 1080*

---++ TML2 hardware inventory
   * Motherboard: ASUS X399 Taichi
   * CPU: AMD Ryzen Threadripper 2920X 12-Core Processor
   * Memory: 48GB (3 x 16GB Corsair CMK32GX4M2A2666C16 2133MHZ)
   * Disks: Samsung NVME 1TB SSD 
   * GPU *GeForce RTX 2080* 

---++ TML3 hardware inventory
   * Motherboard: ASUS WS X299 SAGE
   * CPU: Intel(R) Core(TM) i9-7940X CPU @ 3.10GHz
   * Memory: 64GB (4 x 16GB Corsair CMK32GX4M2A2666C16 2133MHZ)
   * Disks: Samsung NVME 1TB SSD 
   * GPU
      * NVIDIA Corporation GV100 TITAN V
      * NVIDIA Corporation GV100 TITAN V

---+++ Notes on the purchase of original TML
   * The main constraint in the design of this machine was to permit up to 4 liquid cooled GPU cards in one chassis.
   * The biggest problem we faced is that *most GPU cards are air cooled from the side* - putting 4 every other slot would obstruct the fans
   * I9 System with 64G ram (expandable to 128G)
   * Mother board needs at least 7 slots and the chassis must have 8 slots (GPU cards are two slots wide) the 
      * GPU cards plug into slots: 1,3,5,7
   * Build is only cost effective if you plan to add the 4 GPU's - currently the CPU is 35% of the overall cost
   * Overall this system runs quiet as all of the fans ar 120mm and because of the large radiators they do not need to run fast
   
---++ Parts for original TML
   * *GPU - liquid cooled*
      * EVGA GeForce GTX 1080 Ti FTW3 Hybrid
      * https://www.amazon.ca/EVGA-GeForce-Hybrid-Gaming-Technology/dp/B074D7S8HR
      * Notes: This card came with the liquid cooler - however many of the EVGA GPU cards can been be upgraded with EVGA liquid cooler add ons
      * FYI: Any 11G ram GPU car is going to be very hard to find in stock
   * *Case* - lots of room for liquid cooling radiators 
      * Item#: N82E16811133338
      * Thermaltake Core X5 Tempered Glass Edition E-ATX Stackable Black Cube Chassis CA-1E8-00M1WN-02
      * https://www.newegg.ca/Product/Product.aspx?reviews=all&Item=N82E16811133338
      * Notes: This case is stack-able - is wide enough that the motherboard is mounted horizontally
      * I really liked working with this case for this configuration - there are a number of places to put liquid cooling radiators 
   * *Motherboard* - 7 slot, lots of PCIe lanes and two NVMe disk slots
      * MFG #: WS X299 SAGE - Asus WS X299 SAGE Workstation Motherboard
      * http://www.directdial.com/WS%20X299%20SAGE.html
   * *CPU - I9*
      * MFG #: BX80673I97940X
      * Intel Core i9 i9-7940X Tetradeca-core (14 Core) 3.10 GHz Processor
      * http://www.directdial.com/BX80673I97940X.html
   * *CPU cooler - liquid cooled*
      * MFG #: CW-9060032-WW
      * Corsair H115i PRO Cooling Fan/Radiator
      * http://www.directdial.com/CW-9060032-WW.html
   * *RAM* - must be purchased in groups of 4
      *  Corsair Vengeance LPX 32GB (2x16GB) DDR4 DRAM 2666MHz C16 Memory Kit
      * MFG #: CMK32GX4M2A2666C16
      * http://www.directdial.com/CMK32GX4M2A2666C16.html
   * *NVMe disk*
      * MFG #: MZ-V7E2T0BW
      * Samsung 970 EVO MZ-V7E2T0BW 2 TB Internal Solid State Drive PCI Express - M.2 2280
      * http://www.directdial.com/MZ-V7E2T0BW.html
   * *Power Supply*
      * MFG #: 120-G2-1600-X1
      * EVGA SuperNOVA 1600 G2 Power Supply
      * http://www.directdial.com/120-G2-1600-X1.html
      * For up to 4 GPUs at max workload

---++ Software Documentation
*18Feb2020  - Mike Gore*
   * *Note - I moved the GPU setup scripts to [[CF/GPUInstallScripts]] TWIKI has they are updated frequently

---+++ TML Pictures
   * TML - with covers off power supply side view: <br />
     <img src="%PUBURLPATH%/%WEB%/%TOPIC%/IMG_20181203_102802.jpg" alt="IMG_20181203_102802.jpg" width="1008" height="756" />

   * TMP with covers off rear top view: <br />
     <img src="%PUBURLPATH%/%WEB%/%TOPIC%/IMG_20181203_102818.jpg" alt="IMG_20181203_102818.jpg" width="756" height="1008" />

   * TML covers off CPU side view: <br />
     <img src="%PUBURLPATH%/%WEB%/%TOPIC%/IMG_20181203_102831.jpg" alt="IMG_20181203_102831.jpg" width="1008" height="756" />

---++ Install scripts
*14Feb2020  - Mike Gore*
   * *Note - I will be moving the GPU setup scrips to [[CF/GPUInstallScripts]] 
---++ GPU install scripts
   * GPUInstallScripts

Attachments

Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
jpg	IMG_20181203_102802.jpg	r1	manage	949.7 K	2018-12-03 - 10:38	MikeGore	TML - with covers off power supply side view
jpg	IMG_20181203_102818.jpg	r1	manage	1014.8 K	2018-12-03 - 10:39	MikeGore	TMP with covers off rear top view
jpg	IMG_20181203_102831.jpg	r1	manage	997.0 K	2018-12-03 - 10:40	MikeGore	TML covers off CPU side view

Topic revision: r12 - 2023-01-09 - TomCauduro

Information in this area is meant for use by CSCF staff and is not official documentation, but anybody who is interested is welcome to use it if they find it useful.

Other Webs

My links
- People
- CERAS
- WatForm
- Tetherless lab
- Ubuntu Main.HowTo
- eDocs
- RGG NE notes
- RGG
- CS infrastructure
- Grad images

Edit