CSCF Application Server (Asimov) notes

22 July 2015 - Updates to new server notes are in progress - mostly done

Summary

Purpose

  • File system for CSCF image deployment and misc services
  • Prototype for next generation Teir 2 File Server

Hardware

  • SuperMicro CSE-847E16-R1K28LPB case
  • SuperMicro X9DRD-7LN4F-JBOD-O motherboard, 2xIntel Xeon E5-2630 V2 (6 Core) 2.6Ghz, 128GB 1600Mhz DDR3 ECC RAM (16x8GB)
  • 2xIntel 100GB SSD (OS RAID 1)
  • 24x Seagate Constellation 4TB SAS (ST4000NM0023)
  • 800G PXIE flash disk - zcache for zfs

Ubuntu OS

  • We are now running Ubuntu 14.04.2 LTS on new hardware

Data migration - old to new Asimov

Index

Feature Summary - high level

  • Ubuntu Linux OS
  • Software and CD installation repository shared on the network using SAMBA
    • 285 CD's, Windows and Linux, that can be installed over the network
      • CD's are in RAW ISO format - easy to burn
      • CD's ram images are also mounted online so they are all available
    • Common Windows Applications and License Keys
  • SAMBA server - see CSApplicationServer#File_Shares
  • System Imager - see SystemImager
  • Machine image repository - Clonezilla, Acronis and GHOST images of fully installed machines
    • See ImageDeploy
    • ie "Syspreped" Windows machine with Applications fully configured/patched and/or fully setup Linux images
      • Most images are created and maintained by the CORE infrastructure group - Phil Beldowski
  • Has private NAT firewalled network - VLAN 420
    • DHCP with local name server - See PXE server
    • PXE tftpboot functions - See PXE server
      • Boot DOS and Linux repair and imaging utilities or Linux network installers
    • DNS name server for NAT - See PXE server
    • SAMBA file shares
    • SSH and SSHFS - we can ssh mount resources
    • NFS file shares
    • APACHE Web service
      • Many of the SAMBA shares can be accessed here as http://asimov:8080/exports or /iso
      • DHCP reporting tool for private network activity
  • External network
    • SAMBA file shares - to all of UofW
    • SSH and SSHFS - we can ssh mount resources
    • NFS file shares - limited networks
    • APACHE Web service

Image deployment over the network from ASIMOV

Drive and File System

  • asimov-pool/images 57T 1.2T 56T 2% /images
  • asimov-pool/coregroup 60T 3.7T 56T 7% /coregroup
  • asimov-pool/backups 57T 1.3T 56T 3% /backups

SMART Disk Monitoring

  • TBD

File Shares

  • To see a list of shares you can run
    • NET VIEW command from a command prompt in Windows, for example, NET VIEW \\ASIMOV
    • smbclient -L asimov.cscf -U cscf-adm from Linux (provided by the smbclient package in Ubuntu).

root@asimov-new:~# smbclient -L localhost -U cscf-adm
Enter cscf-adm's password: 
Domain=[CS_GENERAL] OS=[Unix] Server=[Samba 4.1.6-Ubuntu]

   Sharename       Type      Comment
   ---------       ----      -------
   homes           Disk      Home Directories
   print$          Disk      Printer Drivers
   PXE             Disk      PXE Boot home directory
   PXEIMAGES       Disk      PXE Boot share for Network Book Images
   PXEISO          Disk      Mounted CDROM Images,some may be Licensed Software, Requires License form IST
   PXERAWISO       Disk      RAW ISO CDROM Images, some may be Licensed Software, Requires License form IST
   OPT             Disk      OPT utilities
   research        Disk      Research Share
   install         Disk      Lawrence Folland - Research Group - work on file reorg.
   CORE            Disk      CORE GROUP
   CORE_OLD        Disk      CORE GROUP OLD
   IMAGES          Disk      Deployment images for all of CSCF
   PROJECTS        Disk      CSCF Projects
   iso             Disk      Mounted CDROM Images,Licensed Software, Requires License form IST
   rawiso          Disk      RAW ISO CDROM Images, Licensed Software, Requires License form IST
   exports         Disk      ALL EXPORTS, Microsoft OS Dependent, Some site license software
   software        Disk      SOFTWARE not provided on CDROM images, Some licensed software
   sysprep         Disk      SYSPREP INSTALL IMAGES - Should move to IMAGES share
   SUPERMICRO      Disk      SUPERMICRO Utilities IPMI, Fimware,etc
   OFFICE10        Disk      Alias Office XP, network install, Note: needs license forms
   OFFICE11        Disk      Alias Office 2003, network install, Note: needs license forms
   vs.net2003      Disk      Alias VISUAL STUDIO NET 2003, Note: needs license forms
   vs.net2005      Disk      Alias VISUAL STUDIO NET 2005, Note: needs license forms
   backup          Disk      CSCF - RESEARCH USE - GENERAL TEMP BACKUP FOR ANY COMPUTER
   MAC             Disk      MAC software
   CLAYTON         Disk      Clayton Server Images and scratch area
   ADBACKUPS       Disk      AD Backups
   nexus           Disk      STEVE NICKERSON - NEXUS
   nexus_adm       Disk      STEVE NICKERSON - NEXUS ADMIN
   SNICKERS        Disk      Steve Nickerson image storage area
   IPC$            IPC       IPC Service (asimov-new server (Samba, Ubuntu))
   cscflp01        Printer   cscflp01
   xrxpr003        Printer   xrxpr003
   xrxpr004-monochrome Printer   xrxpr004-monochrome
   ljc-alg         Printer   ljc-alg
   dbpr001         Printer   dbpr001
   dbpr004         Printer   dbpr004
   xrxpr005        Printer   xrxpr005
   xrxpr004-colour Printer   xrxpr004-colour
   xrxpr002        Printer   xrxpr002
   cs006437        Printer   cs006437
   ljc-dc3334      Printer   ljc-dc3334
   lj-m425dn-dc2313 Printer   lj-m425dn-dc2313
   lj-dc3624       Printer   lj-dc3624
   lp2-dc3507      Printer   lp2-dc3507
   chilpr01        Printer   chilpr01
   mfp102          Printer   mfp102
   lj-cs-simplex   Printer   lj-cs-simplex
   lj-dc2582       Printer   lj-dc2582
   cs006417        Printer   cs006417
   slush           Printer   slush
   ljp-maple       Printer   ljp-maple
   xrxpr001        Printer   xrxpr001
   lp7-cs          Printer   lp7-cs
   cs007854        Printer   cs007854
   docutech6180    Printer   docutech6180
   graphics-dc-xerox-5775 Printer   graphics-dc-xerox-5775
   lj-dc3334       Printer   lj-dc3334
   cs007582        Printer   cs007582
   graphics-gsc-ricoh Printer   graphics-gsc-ricoh
   lp-cscf-private Printer   lp-cscf-private
   lj-dc2306       Printer   lj-dc2306
   xrxpr006        Printer   xrxpr006
   graphics-dc-ricoh Printer   graphics-dc-ricoh
   ljp-plg         Printer   ljp-plg
   lj-cs           Printer   lj-cs
   lj1-isg         Printer   lj1-isg
   lj2-watform     Printer   lj2-watform
   lp-5136a        Printer   lp-5136a
   cscf-adm        Disk      Home Directories
Domain=[CS_GENERAL] OS=[Unix] Server=[Samba 4.1.6-Ubuntu]

   Server               Comment
   ---------            -------
   ASIMOV-NEW           asimov-new server (Samba, Ubuntu)

   Workgroup            Master
   ---------            -------
   CS_GENERAL           


Common Sources

Source files are located in three directories

Common System Services

Notes System services can be controlled with the service command which is a script that simply invokes command in /etc/init.d . Each script has standard options such as start, stop, restart, status
  • mount_cds - mounts /iso images created from /images/rawiso
  • samba - samba service
  • apache2 - Web Server
  • dnsmasq - - Local Name server,DHCP server,PXE server * See DNSMASQ for details * See PXE server
  • firewall NatMasqFirewallExample
  • tftpboot structure See PXEBootFilesExample
  • Examples
    • service mount_cds stop - removes /iso
    • service mount_cds start - creates /iso loopback mounts from /images/rawiso

List Changed Configuration files

TODO UPDATE THIS
  • debsums -ec 2>/dev/null
       debsums -ec 2>/dev/null 
       /etc/apache2/sites-available/default
       /etc/apache2/ports.conf
       /etc/apcupsd/apcupsd.conf
       /etc/init.d/apcupsd
       /etc/default/apcupsd
       /etc/crontab
       /etc/cups/cupsd.conf
       /etc/default/debsums
       /etc/denyhosts.conf
       /etc/dnsmasq.conf
       /etc/e2fsck.conf
       /etc/fai/fai.conf
       /etc/fai/make-fai-nfsroot.conf
       /etc/fai/apt/sources.list
       /etc/qt3/qt_plugins_3.3rc
       /etc/manpath.config
       /etc/mercurial/hgrc.d/hgext.rc
       /etc/rpc
       /etc/default/nfs-common
       /etc/exports
       /etc/default/nfs-kernel-server
       /etc/ntp.conf
       /etc/ssh/ssh_config
       /etc/sysctl.conf
       /etc/smartd.conf
       /etc/default/smartmontools
       /etc/systemimager/getimage.exclude
       /etc/systemimager/rsync_stubs/10header
       /etc/systemimager/systemimager.conf
       /etc/systemimager/bittorrent.conf
       /etc/vsftpd.conf
       /etc/default/xinetd
       

Modifying service run levels

  • update-rc.d
  • old method
    • chkconfig -help
    • Notes* chkconfig uses comment tags in service files located in /etc/init.d

System Config Files and Utilities

Important system config files

  • File List cscf-adm/scripts/save.list
  • Notes see comment in this file for descriptions

Samba

  • Config /etc/samba/smb.conf
  • Service /etc/inetd.d/samba

Backups and Mirroring

  • Script directory cscf-adm/scripts
  • Documents cscf-adm/scripts/README.TXT
  • Dependencies /etc/crontab - see comments

Local system utilities

  • Source directory cscf-adm/src/src
  • Documents cscf-adm/src/README.TXT

Firewall

  • Directory cscf-adm/scripts/remote
  • Documents cscf-adm/scripts/remote/README.TXT
  • Service etc/inetd.d/iptables - calls cscf-adm/scripts/remote/firewall

Network

  • Configuration Files
    • /etc/sysconfig/
    • /etc/network/
    • /etc/hostname
    • /etc/hosts
    • /etc/resolv.conf

Name Server

  • Configuration Files
    • /cscf-adm/src/dnsmasq - see DNSMASQ

APC UPS

Note: currently disabled
  • Config: /etc/apcupsd/apcupsd.conf

Web Server

Web Server Utils

  • Source Directory cscf-adm/src.web
  • Public Directory cscf-adm/public_html

Httpd server update

  • On our server we include /etc/httpd/conf/local.conf from /etc/httpd/conf/httpd.conf for all local settings
Example /etc/httpd/conf/local.conf - permit access to iso images
Alias /iso "/iso"
<Directory /iso>
    order deny,allow
    deny from all
    allow from all
    Options All
</Directory>
  • Restart web server: service httpd restart
  • Restart mounted iso images: service mount_cds restart

PXE Server

  • DNSMASQ Configuration files /root/dnsmasq - See DNSMASQ
  • Home directory /tftpboot/pxes is the PXE root - see PXEBootFilesExample
  • Firewall NAT MASQ private network - see CF/NatMasqFirewallExample
  • See ~/readme.txt for up to date detailed file and directory descriptions

Adding PXE image to the CS Application Server

Notes Outlines how to setup a PXE boot environment for a new system

Installing PXE boot image

/tftpboot/pxes

/tftpboot/pxes/pxelinux.cfg/dos_menu

PXE related Applications and Documents

Hardware update research 16 Oct 2014

TODO ONLY RELATES TO THE OLD ASIMOV

Asimov OLD provides the following services

  • Private network nat and firewall for VLAN 420 DNS and DHCP server
  • File storage /image server for research and infrastructure
    • For all research and infrastructure Grad desktop images (new and past)
    • Software repository - well over 260 CD/DVD images of installation media
    • VMware images
    • Nexus images
    • Windows Server images for CS
  • CS CAD drawings
  • Documentation repository
  • Site license repository
  • Temporary backup space for machines undergoing repair and re-installation
  • Deploy studio repository for Macs
  • Special projects
  • PXE boot server with
    • Live Ubuntu images
    • RIP recovery tools
    • Diagnostic and disk wipe utils
    • All Ubuntu netboot installers - all version
    • Acronis images of machines (see file server)
  • Common installation scripts used by research and infrastructure in deploying machines "ubuntu-scripts"

Asimov CSCF Infrastructure Uses:

  • DHCP/NAT/Firewall/Network utility server for vlan 420
  • PC Imaging in DC (SE Lab, School Pool Laptops and Workstations, RSG workstations/loaner laptops.etc)
  • PC Image storage/archival
  • PC Image creation/prep (ubuntu-scripts)
  • PC PXE Boot Utilities (DBAN, Acronis.etc)
  • ISO/Application storage/archival (iso/rawiso/exports) (at least 300 or so current ISOs)
  • License storage (site_licenses.txt)
  • Macintosh DeployStudio repository for devonshire.cscf (imaging admin workstations, diagnosing Macs.etc)
  • Macintosh image storage/archival
  • Macintosh package storage/archival

Asimov's current problems/limitations

  • Imaging performance
    • Network/Disk speed is too slow for imaging multiple machines (one machine on our current core image may take upwards of a few hours to complete)
      • Available RAM also plays a factor in this
    • Macs imaging over DeployStudio seem to have speed and timeout issues when transferring large images such as Ed's undergrad image (about 80GB)
  • Storage Space
    • We like to backup machines that come in for service before making changes and before redeploying them (incase of future issues) but lack of space means we can't do this for most machine
    • We currently have to store backups on an external USB RAID box due to lack of available drive bays
    • When making new images, we have to shuffle a lot of files around due to the lack of a RAID setup
  • Reliability/Expandability
    • Current 'server' has only one desktop power supply installed (one point of failure)
    • No LOM (have to manage console over the remote KVM)
    • No free drive bays for future expansion

Hardware Requirements

  • Large hardware RAID in RAID 60 w/ battery backup on controller
    • At least 10TB of usable storage with room for expansion
    • At least 8 drives required, extras configured as hot spares
  • 10G SFP+ connection to both vlan 420 and vlan 15
  • 16GB+ of RAM
  • Redundant hotswap power supplies
  • LOM interface
  • Hardware warranty
  • Ubuntu 14.04.1 LTS server
Topic attachments
I Attachment History Action Size Date Who Comment
Unknown file formatext acronis_menu r1 manage 0.7 K 2011-12-09 - 13:07 MikeGore  
Unknown file formatext add_ghost r1 manage 6.2 K 2011-12-09 - 13:11 MikeGore  
Unknown file formatext default r1 manage 2.3 K 2011-12-09 - 13:07 MikeGore  
Unknown file formatext dos_menu r1 manage 0.5 K 2011-12-09 - 13:08 MikeGore  
Unknown file formatconf menu.conf r1 manage 3.7 K 2011-12-09 - 13:13 MikeGore  
Unknown file formatext mk_ubuntu r1 manage 2.8 K 2011-12-09 - 13:11 MikeGore  
Unknown file formatext rip_menu r1 manage 0.5 K 2011-12-09 - 13:08 MikeGore  
Edit | Attach | Watch | Print version | History: r57 | r55 < r54 < r53 < r52 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r53 - 2015-07-22 - MikeGore
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2025 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback