CSCF Application Server (Asimov) notes
22 July 2015 - Updates to new server notes are in progress - mostly done
Summary
Purpose
- File system for CSCF image deployment and misc services
- Prototype for next generation Teir 2 File Server
Hardware
- SuperMicro CSE-847E16-R1K28LPB case
- SuperMicro X9DRD-7LN4F-JBOD-O motherboard, 2xIntel Xeon E5-2630 V2 (6 Core) 2.6Ghz, 128GB 1600Mhz DDR3 ECC RAM (16x8GB)
- 2xIntel 100GB SSD (OS RAID 1)
- 24x Seagate Constellation 4TB SAS (ST4000NM0023)
- 800G PXIE flash disk - zcache for zfs
Ubuntu OS
- We are now running Ubuntu 14.04.2 LTS on new hardware
Data migration - old to new Asimov
Index
Feature Summary - high level
- Ubuntu Linux OS
- Software and CD installation repository shared on the network using SAMBA
- 285 CD's, Windows and Linux, that can be installed over the network
- CD's are in RAW ISO format - easy to burn
- CD's ram images are also mounted online so they are all available
- Common Windows Applications and License Keys
- SAMBA server - see CSApplicationServer#File_Shares
- System Imager - see SystemImager
- Machine image repository - Clonezilla, Acronis and GHOST images of fully installed machines
- See ImageDeploy
- ie "Syspreped" Windows machine with Applications fully configured/patched and/or fully setup Linux images
- Most images are created and maintained by the CORE infrastructure group - Phil Beldowski
- Has private NAT firewalled network - VLAN 420
- DHCP with local name server - See PXE server
- PXE tftpboot functions - See PXE server
- Boot DOS and Linux repair and imaging utilities or Linux network installers
- DNS name server for NAT - See PXE server
- SAMBA file shares
- SSH and SSHFS - we can ssh mount resources
- NFS file shares
- APACHE Web service
- Many of the SAMBA shares can be accessed here as http://asimov:8080/exports
or /iso
- DHCP reporting tool for private network activity
- External network
- SAMBA file shares - to all of UofW
- SSH and SSHFS - we can ssh mount resources
- NFS file shares - limited networks
- APACHE Web service
Image deployment over the network from ASIMOV
Drive and File System
- asimov-pool/images 57T 1.2T 56T 2% /images
- asimov-pool/coregroup 60T 3.7T 56T 7% /coregroup
- asimov-pool/backups 57T 1.3T 56T 3% /backups
SMART Disk Monitoring
File Shares
- To see a list of shares you can run
- NET VIEW command from a command prompt in Windows, for example, NET VIEW \\ASIMOV
- smbclient -L asimov.cscf -U cscf-adm from Linux (provided by the smbclient package in Ubuntu).
root@asimov-new:~# smbclient -L localhost -U cscf-adm
Enter cscf-adm's password:
Domain=[CS_GENERAL] OS=[Unix] Server=[Samba 4.1.6-Ubuntu]
Sharename Type Comment
--------- ---- -------
homes Disk Home Directories
print$ Disk Printer Drivers
PXE Disk PXE Boot home directory
PXEIMAGES Disk PXE Boot share for Network Book Images
PXEISO Disk Mounted CDROM Images,some may be Licensed Software, Requires License form IST
PXERAWISO Disk RAW ISO CDROM Images, some may be Licensed Software, Requires License form IST
OPT Disk OPT utilities
research Disk Research Share
install Disk Lawrence Folland - Research Group - work on file reorg.
CORE Disk CORE GROUP
CORE_OLD Disk CORE GROUP OLD
IMAGES Disk Deployment images for all of CSCF
PROJECTS Disk CSCF Projects
iso Disk Mounted CDROM Images,Licensed Software, Requires License form IST
rawiso Disk RAW ISO CDROM Images, Licensed Software, Requires License form IST
exports Disk ALL EXPORTS, Microsoft OS Dependent, Some site license software
software Disk SOFTWARE not provided on CDROM images, Some licensed software
sysprep Disk SYSPREP INSTALL IMAGES - Should move to IMAGES share
SUPERMICRO Disk SUPERMICRO Utilities IPMI, Fimware,etc
OFFICE10 Disk Alias Office XP, network install, Note: needs license forms
OFFICE11 Disk Alias Office 2003, network install, Note: needs license forms
vs.net2003 Disk Alias VISUAL STUDIO NET 2003, Note: needs license forms
vs.net2005 Disk Alias VISUAL STUDIO NET 2005, Note: needs license forms
backup Disk CSCF - RESEARCH USE - GENERAL TEMP BACKUP FOR ANY COMPUTER
MAC Disk MAC software
CLAYTON Disk Clayton Server Images and scratch area
ADBACKUPS Disk AD Backups
nexus Disk STEVE NICKERSON - NEXUS
nexus_adm Disk STEVE NICKERSON - NEXUS ADMIN
SNICKERS Disk Steve Nickerson image storage area
IPC$ IPC IPC Service (asimov-new server (Samba, Ubuntu))
cscflp01 Printer cscflp01
xrxpr003 Printer xrxpr003
xrxpr004-monochrome Printer xrxpr004-monochrome
ljc-alg Printer ljc-alg
dbpr001 Printer dbpr001
dbpr004 Printer dbpr004
xrxpr005 Printer xrxpr005
xrxpr004-colour Printer xrxpr004-colour
xrxpr002 Printer xrxpr002
cs006437 Printer cs006437
ljc-dc3334 Printer ljc-dc3334
lj-m425dn-dc2313 Printer lj-m425dn-dc2313
lj-dc3624 Printer lj-dc3624
lp2-dc3507 Printer lp2-dc3507
chilpr01 Printer chilpr01
mfp102 Printer mfp102
lj-cs-simplex Printer lj-cs-simplex
lj-dc2582 Printer lj-dc2582
cs006417 Printer cs006417
slush Printer slush
ljp-maple Printer ljp-maple
xrxpr001 Printer xrxpr001
lp7-cs Printer lp7-cs
cs007854 Printer cs007854
docutech6180 Printer docutech6180
graphics-dc-xerox-5775 Printer graphics-dc-xerox-5775
lj-dc3334 Printer lj-dc3334
cs007582 Printer cs007582
graphics-gsc-ricoh Printer graphics-gsc-ricoh
lp-cscf-private Printer lp-cscf-private
lj-dc2306 Printer lj-dc2306
xrxpr006 Printer xrxpr006
graphics-dc-ricoh Printer graphics-dc-ricoh
ljp-plg Printer ljp-plg
lj-cs Printer lj-cs
lj1-isg Printer lj1-isg
lj2-watform Printer lj2-watform
lp-5136a Printer lp-5136a
cscf-adm Disk Home Directories
Domain=[CS_GENERAL] OS=[Unix] Server=[Samba 4.1.6-Ubuntu]
Server Comment
--------- -------
ASIMOV-NEW asimov-new server (Samba, Ubuntu)
Workgroup Master
--------- -------
CS_GENERAL
Common Sources
Source files are located in three directories
- /cscf-adm/src/dnsmasq - DHCP,PXE,DNS configuration files and scripts
- /cscf-adm/src - program sources
- /cscf-adm/scripts - backup and other scripts
- /cscf-adm/src.web - web based reporting utils
Common System Services
Notes System services can be controlled with the
service command which is a script that simply invokes command in
/etc/init.d . Each script has standard options such as
start,
stop,
restart,
status
- mount_cds - mounts /iso images created from /images/rawiso
- samba - samba service
- apache2 - Web Server
- dnsmasq - - Local Name server,DHCP server,PXE server * See DNSMASQ for details * See PXE server
- firewall NatMasqFirewallExample
- tftpboot structure See PXEBootFilesExample
- Examples
- service mount_cds stop - removes /iso
- service mount_cds start - creates /iso loopback mounts from /images/rawiso
List Changed Configuration files
TODO UPDATE THIS
Modifying service run levels
- update-rc.d
- old method
- chkconfig -help
- Notes* chkconfig uses comment tags in service files located in /etc/init.d
System Config Files and Utilities
Important system config files
- File List cscf-adm/scripts/save.list
- Notes see comment in this file for descriptions
Samba
- Config /etc/samba/smb.conf
- Service /etc/inetd.d/samba
Backups and Mirroring
- Script directory cscf-adm/scripts
- Documents cscf-adm/scripts/README.TXT
- Dependencies /etc/crontab - see comments
Local system utilities
- Source directory cscf-adm/src/src
- Documents cscf-adm/src/README.TXT
Firewall
- Directory cscf-adm/scripts/remote
- Documents cscf-adm/scripts/remote/README.TXT
- Service etc/inetd.d/iptables - calls cscf-adm/scripts/remote/firewall
Network
- Configuration Files
- /etc/sysconfig/
- /etc/network/
- /etc/hostname
- /etc/hosts
- /etc/resolv.conf
Name Server
- Configuration Files
- /cscf-adm/src/dnsmasq - see DNSMASQ
APC UPS
Note: currently disabled
- Config: /etc/apcupsd/apcupsd.conf
Web Server
Web Server Utils
- Source Directory cscf-adm/src.web
- Public Directory cscf-adm/public_html
Httpd server update
- On our server we include /etc/httpd/conf/local.conf from /etc/httpd/conf/httpd.conf for all local settings
Example /etc/httpd/conf/local.conf - permit access to iso images
Alias /iso "/iso"
<Directory /iso>
order deny,allow
deny from all
allow from all
Options All
</Directory>
- Restart web server: service httpd restart
- Restart mounted iso images: service mount_cds restart
PXE Server
- DNSMASQ Configuration files /root/dnsmasq - See DNSMASQ
- Home directory /tftpboot/pxes is the PXE root - see PXEBootFilesExample
- Firewall NAT MASQ private network - see CF/NatMasqFirewallExample
- See ~/readme.txt for up to date detailed file and directory descriptions
Adding PXE image to the CS Application Server
Notes Outlines how to setup a
PXE boot environment for a new system
Installing PXE boot image
/tftpboot/pxes
/tftpboot/pxes/pxelinux.cfg/dos_menu
PXE related Applications and Documents
Hardware update research 16 Oct 2014
TODO ONLY RELATES TO THE OLD ASIMOV
Asimov OLD provides the following services
- Private network nat and firewall for VLAN 420 DNS and DHCP server
- File storage /image server for research and infrastructure
- For all research and infrastructure Grad desktop images (new and past)
- Software repository - well over 260 CD/DVD images of installation media
- VMware images
- Nexus images
- Windows Server images for CS
- CS CAD drawings
- Documentation repository
- Site license repository
- Temporary backup space for machines undergoing repair and re-installation
- Deploy studio repository for Macs
- Special projects
- PXE boot server with
- Live Ubuntu images
- RIP recovery tools
- Diagnostic and disk wipe utils
- All Ubuntu netboot installers - all version
- Acronis images of machines (see file server)
- Common installation scripts used by research and infrastructure in deploying machines "ubuntu-scripts"
Asimov CSCF Infrastructure Uses:
- DHCP/NAT/Firewall/Network utility server for vlan 420
- PC Imaging in DC (SE Lab, School Pool Laptops and Workstations, RSG workstations/loaner laptops.etc)
- PC Image storage/archival
- PC Image creation/prep (ubuntu-scripts)
- PC PXE Boot Utilities (DBAN, Acronis.etc)
- ISO/Application storage/archival (iso/rawiso/exports) (at least 300 or so current ISOs)
- License storage (site_licenses.txt)
- Macintosh DeployStudio repository for devonshire.cscf (imaging admin workstations, diagnosing Macs.etc)
- Macintosh image storage/archival
- Macintosh package storage/archival
Asimov's current problems/limitations
- Imaging performance
- Network/Disk speed is too slow for imaging multiple machines (one machine on our current core image may take upwards of a few hours to complete)
- Available RAM also plays a factor in this
- Macs imaging over DeployStudio seem to have speed and timeout issues when transferring large images such as Ed's undergrad image (about 80GB)
- Storage Space
- We like to backup machines that come in for service before making changes and before redeploying them (incase of future issues) but lack of space means we can't do this for most machine
- We currently have to store backups on an external USB RAID box due to lack of available drive bays
- When making new images, we have to shuffle a lot of files around due to the lack of a RAID setup
- Reliability/Expandability
- Current 'server' has only one desktop power supply installed (one point of failure)
- No LOM (have to manage console over the remote KVM)
- No free drive bays for future expansion
Hardware Requirements
- Large hardware RAID in RAID 60 w/ battery backup on controller
- At least 10TB of usable storage with room for expansion
- At least 8 drives required, extras configured as hot spares
- 10G SFP+ connection to both vlan 420 and vlan 15
- 16GB+ of RAM
- Redundant hotswap power supplies
- LOM interface
- Hardware warranty
- Ubuntu 14.04.1 LTS server