m160 Bioinformatics Research Cluster

  • This document describes Ming Li's CCF research cluster m160.cs

EDOCS

Quotas

  • See UbuntuQuota System has quota setup but not used yet

Basic Health Test

  • check-nodes
    • ping test of nodes
  • check-ssh-nodes
    • SSH test of nodes
  • check-ilom-nodes
    • ping test of node iloms

List Drive Serial Numbers using smartmontools

  • /cscf-adm/src/disk-utils
  • These scripts are presently not working

Syncing Packages between nodes

  • Uses m160-1 as a reference node and duplicates all of its packages on each node
  • Script: sync-packages

Syncing userid or password updates between nodes

  • Provides: Sync account entries,passwords and SSH keys across nodes
    • Adds user SSH keys to their .ssh/authorized_keys file
  • Script: sync-users

Remounting nodes

  • fix-mount - run this on m160 as root Used if the head node restarts

Sync user accounts and ssh keys

  • sync-users
    • syncs user accounts SSH keys and various scripts to all nodes

Rebooting Nodes

  • reboot-nodes

DNSMASQ - PXE BOOT, DHCP, DNS

  • 24 Feb 2022 - we switched to the updated PXE,BOOT/DNS/DHCP setup
  • See /cscf-adm/src/dnsmasq
  • Provides: PXE BOOT, DNS, DHCP
  • Updating DHCP and DNS information for nodes
    • cd /cscf-adm/src/dnsmasq
    • vi /cscf-adm/src/hosts/m160/dnsmasq.common
    • Restart: make
  • PXE BOOT /tftpboot/pxes
    • Config: /tftpboot/pxes/pxelinux.cfg/default
    • Mounted ISO boot images: /tftpboot/pxes/iso
    • Clonezilla Images: /tftpboot/pxes/images
      • make_clonezilla rebuilds and mounts any NODE images

Firewall NAT

  • 24 Feb 2022 - we switched to the updated PXE,BOOT/DNS/DHCP setup
* This critical script allows the nodes to talk to the outside world and limits access to the head node
  • Startup script: /etc/init.d/firewall start|stop|status|restart|install
  • Support scripts:
    • /usr/local/bin/common_host, /usr/local/bin/common_vars,/usr/local/bin/common_functions

Imaging Nodes

  • See Remote Management
  • PXE BOOT the node you want to Image by powering on or resetting the node
    • Press F12 during BIOS boot phase when you first see the BIOS screen - keep pressing the F12 until
      • - or - from m160
    • *ipmipxeboot lom-m160-NN (NN = 1..16)
      • Run this on m160
      • (cscf-adm, 2020 password when prompted)
    • ipmipoweron or ipmireset lom-m160-NN (NN = 1 .. 16)
      • Run this on m160
      • (cscf-adm, 2020 password when prompted)
  • Pick Clonezilla

Image Archive Locations and Access

  • tftpboot/pxes/images/m160-16-2021-05-18-04-img
    • Userid: cscf-adm - 2020 password in safe

Networking

IPMI View utility - Linux

  • Provides: remote console and power managment of the cluster
  • Notes: See IPMI for Ubuntu Linux utility
  • Head Node: lom-m160.cs
  • Nodes: lom-m160-N.cs
  • Access: cscf-adm - password in safe - 2020
  • Documentation: TWIKI page IPMI
  • Start IPMI View utility
  • All LOM interfaces live on network 172.19.96
    • Start IPMI View Search* using addresses 172.19.96.227 to 172.19.96.246
    • Save the LOM interfaces the utility finds
    • OK to exit
  • Double click on the node you wish to manage - on left hand side under IPMI Domain
  • Login: cscf-adm - password in safe - 2020
  • Open KVM Console
  • Open the "Soft Keyboard* so you can send special characters that your local OS may intercept

MegaCli Utility

Not working on 20.04LTS

Hardware and Inventory Section

Not working on 20.04LTS
  • MegaCli64 -PDList -aALL

List All drive serial Numbers

Not working on 20.04LTS
  • MegaCli64 -PDList -aALL | grep "Inquiry Data"
Updated 3 Nov 2011
root@m160:~/utils# MegaCli64 -PDList -aALL | grep "Inquiry Data"
   Inquiry Data: SEAGATE ST32000444SS    00069WM33GWT            
   Inquiry Data: SEAGATE ST32000444SS    00069WM33KW5            
   Inquiry Data: SEAGATE ST32000444SS    00069WM33LG7            
   Inquiry Data: SEAGATE ST32000444SS    00069WM33CX1            
   Inquiry Data: SEAGATE ST32000444SS    00069WM33GSG            
   Inquiry Data: SEAGATE ST32000444SS    00069WM34F8D            
   Inquiry Data: SEAGATE ST32000444SS    00069WM33G6P            
   Inquiry Data: SEAGATE ST32000444SS    00069WM33LY8            
   Inquiry Data: SEAGATE ST32000444SS    00069WM33M19            
   Inquiry Data: SEAGATE ST32000444SS    00069WM33L2C            
   Inquiry Data: SEAGATE ST32000444SS    00069WM345BT            
   Inquiry Data: SEAGATE ST32000444SS    00069WM31XJP            
   Inquiry Data: SEAGATE ST32000444SS    00069WM34FS4            
   Inquiry Data: SEAGATE ST32000444SS    00069WM33KXB            
   Inquiry Data: SEAGATE ST32000444SS    00069WM33KTK            
   Inquiry Data: SEAGATE ST32000444SS    00069WM33LN4            
   Inquiry Data: SEAGATE ST32000444SS    00069WM34DNT            
   Inquiry Data: SEAGATE ST32000444SS    00069WM31W2Y            
   Inquiry Data: SEAGATE ST32000444SS    00069WM34FGR            
   Inquiry Data: SEAGATE ST32000444SS    00069WM33LHQ            
   Inquiry Data: SEAGATE ST32000444SS    00069WM34AAN            
   Inquiry Data: SEAGATE ST32000444SS    00069WM345E6            
   Inquiry Data: SEAGATE ST32000444SS    00069WM34BEK            
   Inquiry Data: SEAGATE ST32000444SS    00069WM345FX            
   Inquiry Data: SEAGATE ST32000444SS    00069WM347EN            
   Inquiry Data: SEAGATE ST32000444SS    00069WM31Y3G            
   Inquiry Data: SEAGATE ST32000444SS    00069WM34AWW            
   Inquiry Data: SEAGATE ST32000444SS    00069WM33E8F            
   Inquiry Data: SEAGATE ST32000444SS    00069WM34C3J            
   Inquiry Data: SEAGATE ST32000444SS    00069WM33LWS            
   Inquiry Data: SEAGATE ST32000444SS    00069WM34H9D            
   Inquiry Data: SEAGATE ST32000444SS    00069WM2XK5C            
   Inquiry Data: SEAGATE ST32000444SS    00069WM33D50            
   Inquiry Data: SEAGATE ST32000444SS    00069WM34K29            
   Inquiry Data: SEAGATE ST32000444SS    00069WM33LXA            
   Inquiry Data: SEAGATE ST32000444SS    00069WM33CH4            
   Inquiry Data: SEAGATE ST2000NM0001    0001Z1P068XV            
   Inquiry Data: SEAGATE ST2000NM0001    0001Z1P06ZSJ            
   Inquiry Data: SEAGATE ST2000NM0001    0001Z1P05DS9            
   Inquiry Data: SEAGATE ST2000NM0001    0001Z1P073CP            
   Inquiry Data: SEAGATE ST2000NM0001    0001Z1P06EV3            
   Inquiry Data: SEAGATE ST2000NM0001    0001Z1P06ZTV            
   Inquiry Data: SEAGATE ST2000NM0001    0001Z1P06ZTC            
   Inquiry Data: SEAGATE ST2000NM0001    0001Z1P05DLA            
   Inquiry Data: SEAGATE ST2000NM0001    0001Z1P072H2            
   Inquiry Data: SEAGATE ST2000NM0001    0001Z1P06EVR            
   Inquiry Data: SEAGATE ST2000NM0001    0001Z1P04Z2Z            
   Inquiry Data: SEAGATE ST2000NM0001    0001Z1P06JDV            
   Inquiry Data: SEAGATE ST2000NM0001    0001Z1P067YK            
   Inquiry Data: SEAGATE ST2000NM0001    0001Z1P0600E        
   

PO with parts summary

* quote_1510258_4-1.pdf: PO with parts summary

APC SUA3000XLT UPS

Note installed presently after move to MC3015

AP5017 APC LCD/KVM

Hardware and Documents

Switches

  • MC3015 cs2d
  • MC3015 cs2e

Compute Nodes - Supermicro 6016TT-TF Twin 1U Intel Xeon 5600/5500 Series

Hardware

Inventory and Access and LOM

Hardware and Documents

HP ProCurve 2910al-24G Ethernet Switch

Hardware and Documents

DES-1024D 24-port 10/100 Desktop/Rackmount Switch

  • Summary.ods: MAC address inventory for switches and m160 cluster ports
Topic attachments
I Attachment History Action Size Date Who Comment
Unknown file formatods Summary.ods r1 manage 15.3 K 2021-02-03 - 12:38 MikeGore MAC address inventory for switches and m160 cluster ports
PDFpdf quote_1510258_4-1.pdf r1 manage 1628.5 K 2011-03-29 - 15:04 MikeGore PO with parts summary
Edit | Attach | Watch | Print version | History: r33 < r32 < r31 < r30 < r29 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r33 - 2022-02-24 - MikeGore
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback