Note - cluster is no longer in service RT#1039321

• The data for all of novo1,novo2 and novo10 including /u is now located on root@asimov:/backups/research/novo

Novo BladeCenter Management Notes

Novo BladeCenter Management Interface - Remote Control, Remote Console, Network, Configuration, etc

• All Blade Managment tasks and remote console access cab be done via this Web Interface
• See: Novo Management Interface NOVOMGT
• Quick Summary:
  • https://novo-mgmt.cs.uwaterloo.ca
    • Userid: cscfadm, password (2017 version same as cscf-adm- see password safe)
    • Remote Video KVM requires IE under windows with Sun Java
  • Console:
    • Blade Tasks
    • Remote Control
    • Start Remote Control BUTTON
    • (Pick NODE from dropdown to right of Video Icon - on top center toolbar)
      • Double Click in screen to activate the window
  • Power:
    • Blade Tasks
    • Remote Control
    • Start Remote Control BUTTON (A new windows Will Pop Up - say continue if you see a security warning)
    • (Pick NODE from dropdown to right of Video Icon - on top center toolbar)
      • Click on Power Control Icon dropdown
      • Choose Power ON or OFF
        • Note: The following will NOT work: Restart, Shutdown, Shutdown with NMI (MNI does not waits for the OS to respond)

Network Connections

• Notes: All this Documentation assumes we are looking at the back of the NOVO Cluster

Management Connection

• dc2303a-cs1a Port 44, vlan 1896 - connects to rear bottom center network jack on Novo Cluster
  • Note: This connection is Hard Wired Internally to all switches and devices (can not be undone by rule sets)

Blade Center Switches and Connections

• Network Connections Left Side of Rack, Top Down
• novo1.cs.uaterloo.ca Top set of 4 ports EXT1-4 (top/down)
  • Hard Wired Internally to ETH0 on ALL NODES
  • Programmed Use: UW Vlans 7 and 170
• novo2.cs.uwaterloo.ca Bottom set of 4 ports EXT1-4 (top/down)
  • Hard Wired Internally to ETH1 on ALL NODES
  • Programmed Use: Private Network 192.168.8.0/24

Switch Password Change

• DASHBOARD -> BNT Layer 2/3 Copper Gigabit Ethernet Switch Module -> System ->User Table
• Note: For some totally bizarre reason the password can *NOT be changed in the CONFIGURE* tab!!!

novo-sw1.cs port connections to Campus Network

• Network Connections Left Side of Rack, Top Down
• Login: admin (cscf-adm 2013 password version)
• Connections

  dc2303a-cs1a Port	Blade Center Switch 1 Port	Port Name	Vlan
  NC	1	EXT1	1
  36	n2	EXT2	170
  35	3	EXT3	7
  NC	4	EXT4	1

novo-sw2.cs port connections to Campus Network

• Network Connections Left Side of Rack, Top Down
• Login admin (cscf-adm 2013 password version)
• Connections

  dc2303a-cs1a Port	Blade Center Switch 2 Port	Port Name	Vlan
  NC	1	EXT1	1668
  NC	2	EXT2	1
  NC	3	EXT3	1
  NC	4	EXT4	1

Vlan Management

• CONFIGURE -> BNT Layer 2/3 Copper Gigabit Ethernet Switch Module -> Layer 2 -> Virtual Lans
• Vlan: 1 - default lan that attached to everything that does NOT have a configuration
• Vlan 4095 Hardwired Management Vlan - we have it attached to 1896 via novo-mgmt.cs.uwaterloo.ca on Port 44

Port Management

• CONFIGURE -> BNT Layer 2/3 Copper Gigabit Ethernet Switch Module -> Layer 2 -> Switch Ports
• INT1-INT10 connects to NODE1 .. NODE10 respectively
• nono-sw1.cs.uwaterloo.ca manages eth0 on the nodes - this is FIXED and can NOT be changed!
• nono-sw2.cs.uwaterloo.ca manages eth1 on the nodes - this is FIXED and can NOT be changed!

Management Interface Navigation Screenshots

• novo-mgmt-cs.uwaterloo.ca
  • Login: cscfadm
  • Password: (2013 version password)

Blade_Tasks

Power/Restart

1. ) Power_Restart

Remote Control

1. ) Remote_Control
2. ) Host_Select
3. ) Host_Login

MM Control

1. ) Login_Profiles

IO Module Tasks

• Configuration -> Slot 1 Address for novo-sw1.cs.uwaterloo.ca
  • Switch Interface for UW Lans
• Configuration -> Slot 1 Address for novo-sw2.cs.uwaterloo.ca
  • Switch Interface for Private Lans
• Configuration -> Slot 1 Address for novo-fc1.cs.uwaterloo.ca
  • SAS Interface
• Configuration -> Slot 1 Address for novo-fc1.cs.uwaterloo.ca
  • SAS* interface

1. ) Configuration

SAS Disk Configuration

• novo-fc1.cs.uwaterloo.ca
  • Login: USERID (yes I know, the userid is USERID and can not be altered)
  • Password: (2013 version password for cscf-adm)
• novo-fc2.cs.uwaterloo.ca
  • Login: USERID (yes I know, the userid is USERID and can not be altered)
  • Password: (2013 version password for cscf-adm)

Storage management (BIF and Ceras) IBM blades center.

• Raid 5 (2 spares disks) 20 TB. as follow:
  • 1 array (raid 5)
  • 1 volume for /home (/u) 5 TB backed up
  • 1 /scratch volume 10 TB
  • Free space (reserve) 5 TB.
  • Block writing I/O in OS.

1. ) From storage management client, connect to Storage module
   • Create an array:
     • Raid x ==> x disks + 1 or 2 spares (min 3)
     • Raid 1 ==> 2 disks, 1 can be used with two partitions.

1. ) Create a volume ==> name
2. ) Assign to a group
   • Add Host(s) ===> HBa/HBb and LUN (lun may be automaticlay detected).

1. ) From MM or switch connected to the storage manager
   • Create zone set * Create alias for blades (see naming) * In zone set: * Create zone (s) * Insert alias * Apply * Save * Activate, Change from list "new zone set".

Documenation

• NortelL23Switch4eservers.pdf: Nortel Networks L2/3 Ethernet Switch Module for IBM server BladeCenter

• NortelLayer2-3SwitchCmdRef.pdf: Command Reference Layer 2/3 GbE Switch Module for IBM BladeCenter Version 1.5

• NortelLayer2-3SwitchInstallGuide.pdf: Installation Guide Nortel Networks Layer 2/3 GbE Switch Module for IBM ERserver BladeCenter

• NortelLayer2-3SwitchInstallGuide.pdf: User’s Guide BladeCenter Open Fabric Manager with Nortel/BNT Extensions for Nortel Layer 2/3 GbE Switch Module Version 40.0

• WP_8021s_SpanningTree_Cisco.pdf: BLADE 802.1s Spanning Tree and Interoperability with Cisco

• ModuleManagementInstallGuide.pdf: BladeCenter Management Module Installation Guide

• IBM_GbE_L2-7_Applicat_Guide.pdf: BLADE OS™ 21.0 Application Guide BNT Layer 2-7 GbE Switch Module for IBM BladeCenter® Version 21.0

• BladeCenterInstallGuide.pdf: BladeCenter E Type 8677 Installation and User’s Guide

• ApplicationsL2-7GbESM.pdf: Application Switching with Nortel Networks Layer 2-7 Gigabit Ethernet Switch Module for IBM BladeCenter

• ApplicationGuideL2-3_AG_5-1.pdf: BLADEOS™ Application Guide BNT Layer2/3 GbE Switch Module for IBM BladeCenter® Version 5.1