Server Setup tools Summary

• Autmatically setup a new DNS/TFTPBOOT/NFS/Liveboot server with utilities
• Web Page: https://cs.uwaterloo.ca/twiki/view/CF/ImageServer?cover=print

Overview

• This documentation is a very brief outline for software used on himrod shiraz m160 and other clusters
• Specifically files found on */cscf-adm/src on these servers
• For more Documentation on scripts and programs documented here please visit http://www.cs.uwaterloo.ca/~magore/ImageServer/ids_index.html
• For detail on PXE booting multiple architectures see https://cs.uwaterloo.ca/twiki/view/CF/DNSMASQ-MULTIARCH
• Older Documentation that is still of value
  • See https://cs.uwaterloo.ca/twiki/view/CF/WatITis2013 eventually those pagess will be updates to this system

Configuration

• Uses only THREE config files and a few fixed template files (for UofW)
  1. ) /cscf-adm/src/hosts/common_hosts.
     • Defines network interfaces rolls, ie NATing, forwarding
  2. ) /cscf-adm/src/dnsmasq/dnsmasq.common.
     • Uses vastly simplified syntax sort-forms
  3. ) /etc/network/interfaces
     • Defines the network interfaces

Template Files

• dnsmasq/dnsmasq.template.private
  • default DNSMASQ settings for ONE private network
• dnsmasq/dnsmasq.template.private_two
  • default DNSMASQ settings for TWO private networks
• dnsmasq/dnsmasq.template.private_three
  • default DNSMASQ settings for THREE private networks
• dnsmasq/hosts.template
  • Default /etc/host beginning
• dnsmasq/resolv.template
  • Default external DNS servers - should be moved into common_host
• samba/smb.conf.template
  • Default SAMBA configuration with SAMBA shares for PXE and Live Images

Firewall

• Automatic NAT firewall
  • Builds NAT rules from /cscf-adm/src/common_host
  • Firewall NAT self installs
    • fireall install
  • Options start/stop/restart/status/insstall/remove

Syslinux

• Install syslinux and dependencies

Automatic PXE menu creation

• All installers create corresponding PXE menus

Liveboot and Ubuntu installers

• Fetch and setup specified Ubuntu Network and Liveboot images
  • Example: fetch_ubuntu_releases 12.04 14.04

ACRONIS

• Install ACRONIS imaging tool with PXE menus

RIP recovery is Possible Diagnostics

• RIP recovery tools via PXE boot with local customizations, kernels and modules

Clonezilla Imaging tool

• Clonezilla via PXE boot

Samba

• Install and setup SAMBA

Accounts

• Create local Linux and SAMBA administrative accounts

NFS

• Install and setup NFS if needed
• Automatic export or update of bootable images

Backup with legato Networker

• Install Legato Networker Backup

CSCF specific

• CSCF root public key

USB KEY bootable replica of PXE boot environment

• Replicates PXE boot images to standalone USE KEY for offline use
• pxe2usb.sh replicates the server PXE boot images to a USB key
  • USB boot all of the same tools, installers and utilies

DNSMASQ

• Creates DNS, reverse DNS, and CNAMEs records from simplified file
  • /cscf-adm/src/dnsmasq/dnsmasq.common.
  • is the local hostname - example: himrod
  • Defines interfaces and hosts for dnsmasq
• Install: /cscf-adm/src/dnsmasq/install_dnsmasq
  • Installs all related packages if missing
  • Creates:
    • /etc/hosts adds all hostnames known to dnsmasq for all interfaces
  • /etc/resolv.conf network manager trigger script if needed * Writes a custom resolv.conf with local DNS and default UW DNS from templates
  • Start/Restarts any services
  • Backup related config files

• Example:

                  iface=em1
                  subnet=129.97.119
                  254,c8:1f:66:dd:f0:ca,himrod.cs.uwaterloo.ca,himrod.cs
                  ...
  
                  iface=em3
                  subnet=192.168.1
                  1,c8:1f:66:dd:f0:cc,himrod
                  10,c8:1f:66:df:69:7c,himrod-storage
                  11,c8:1f:66:dd:d8:ed,himrod-big-1
                  ...
                  

Makefile

• make all installs everything on a clean system
• However at minimum make must be installed
• The Makefile invokes all installer scripts

BASH Scripts summary

• common_functions 80 shell functions
• common_vars Directory locations and layout
• common_host CSCF, UofW Specific HOST Network definitions
  • Created from hosts/common_hosts.
  • is the local hostname
• wakeup uses the dnsmasq MAC definitions to invoke etherwake on a host
• accounts Encrypted passwords for admin accounts
• install_samba installs SAMBA
  • Uses common_vars for the userids
  • Uses samba/smb.conf.template
• install_admin_users installs admin users for Linux and SAMBA
  • Uses common_vars for the userids
• dnsmasq/install_dnsmasq installs dnsmasq and configuration files
• instll_pxe installs DNSMASQ, TFTPBOOT framework, Utilities and configuration files
• install_syslinux Downloads, Compiles and installs syslinux
• fixperms update permission and ownership of shared files for SAMBA shares
• fetch_ubuntu_releases downloads any specified Ubuntu release
  • The script only needs the release number like 14.04, 6.06 or beta
  • It figures out which of the Three Mirrors sites to use and
  • Calls pxe_liveboot_nfs restart to update NFS mounts and PXE boot menus
• fetch_ubuntu_installers downloads Ubuntu Network Install images
• fetch_rip downloads RIP Recovery is possible images
• fetch_clonezilla Downloads CloneZilla and makes a Live Image with PXE menu
• pxe_liveboot_nfs
  • NFS export any Ubuntu Live images
• update_rip Modifies RIP adding local tools, kernels and modules
• remove_localizations Convert the server image into a client node
  • This is intended to be run on a clone of the server

C Programs

• basestr.c
  • :Match substring
  • Returns: return( strncmp(argv[1],argv[2],len) == 0 ? 0 : 1);

• fixname.c
  • Escape strings so they can be safely passed to a shell
  • Names may be quoted with ''s
• lc.c
  • University of Waterloo List Catalogue command
• sh2twiki.c
  • Convert BASH and C markup to HTML or TWIKI
• testfree.c
  • Usage: testfree device -v -f threshold(in Megabytes)

Documentation

• Automatically extract BASH and C markups
  • Creates TWIKI or HTML Documentation from Markup
• Features
  • Uses half the markup required the doxygen
  • Understands SHELL Variables and Functions

Markup Rules

• Heading: Markup syntax notes
• Comments:
  • Comments start with // or #
  • Examples
  • # this is a comment
  • // this is a comment
• Tokens:
  • Tokens are any word following a comment that ends with ':'
  • Examples
  • # Notes:
  • // Arguments: userid result
  • Notes: the token and any following comments are part of the token
• Reserved:
  • The following tokens have special meaning
  • Title: Creates a title for the document.
  • This keyword and any unbroken chain of comments and unreserved
  • tokens are part of the title and will be listed in the index file.
  • Author: Document Author, also Authour works
  • Heading: Create a heading
• Variables:
  • The following code objects have special meaning
  • VAR=value Variable assignments
  • This variable if following a token will become a comment
  • # ======== This a a section break
  • // ======= This is a section break
• Blank_Lines:
  • Blank Comments lines end a token or grouping
  • Examples:
  • //
  • #
  • C enum are processed
  • enum {
  • value1,
  • value2
  • };
• Functions: are processed along with the contents by counting brace levels
• Note: ANY unbroken preceding group of tokens/comments are part of the
  • function documentation.
  • Functions names in the documentation are listed BEFORE the preceding tokens
• Examples:
  • // This is a custom printf command
  • // Arguments: format string, variable number of arguments
  • // Returns: void
  • void myprintf(char *str, ...)
  • {
  • printf code goes here ...
  • }
• Special_Keywords:
• Processed keywords that create documentation sections
  • pause message
  • pause_yes message
  • pause_no message
  • must be followed by if,then,else,fo - optionly end with a section
  • Notes:
  • This is used in the ASIMOV updates script to create TWIKI documentation for
  • That script. The compatibility is included here