Overview

• Updates to hostname, MAC address, and IP address in the inventory web-interface will trigger corresponding updates to our DHCP service. All three elements must be filled in on a host for the update/addition to occur.
• DHCP services are hosted on watcher202 (master) watcher204, and watcher right now
• If the master server is not responsive for more than 5 minutes, a new master server is selected automatically based on the server list in the config file
• DHCP data is automagically regenerated from the database.cs MySQL equipment database using DIPAAS
• make sure that the same SSH keys exist on the 'cscfdhcp' account on all DHCP and backup servers

Firewalling (on DHCP/DIPAAS server)

• Incoming+Outgoing UDP 67, 68
• Incoming+Outgoing SSH (TCP 22)
• Outgoing MySQL (TCP 3306)

DIPAAS (Dynamic IP Address Assignment System)

• requires PHP 4 or higher with MySQL extensions, RCS, SSH, rsync
• manages DHCP services on a server by generating isc-dhcpv3 style static dhcp config files, reloads dhcp, and provides automatic failover
• logs warnings (inconsistent database information, other non-critical problems) and errors (critical problems that cause DIPAAS to abort) to file
• modular data source architecture (included is the database.cs MySQL interface)
  • this module will only include items that have a hostname, IP address and MAC address
• easy to edit/configure config.inc.php file, see that file for configuration options
• supports manual adding of hosts in the config file
• supports CIDR / non-aligned subnets
• supports custom settings per host/per subnet
• automagic failover

Installing DIPAAS and ISC DHCP server

1. Install PHP4 with MySQL extensions, RCS, SSH, rsync, and dhcpd on the host server
   • FreeBSD: (todo)
   • Debian: apt-get dhcp3-server php4-mysql rcs rsync ssh
   • Fedora: yum install rcs rsync openssh-clients dhcp php-mysql
2. Make a user called cscfdhcp (with password disabled)
   • FreeBSD: pw user add cscfdhcp
   • Linux: useradd cscfdhcp
3. (New cluster only) Generate an SSH key for it to use with the backup server ssh-keygen -t dsa, make sure it has no password, accept the defaults
4. (New cluster only) Add the contents of id_dsa.pub to ~/.ssh/authorized_keys on the backup server
5. Copy the DIPAAS current active package from backup to the new server e.g. su cscfdhcp 'rsync -e ssh -a cscf.cs:~/backup/dipaas ~'
6. Add a cronjob for cscfdhcp to run every minute running php dipaas.php, e.g. * * * * * /usr/local/bin/php dipaas/dipaas.php
7. Only make changes to the master server, any changes made anywhere else will result in loss of your changes, run 'whoismaster' or 'whatami' in the DIPAAS package to have it tell you the appropriate information
8. (Preferred) Give the DIPAAS user sudo to reload the DHCP server or (not as preferred) set up a root cronjob to reload the DHCP server at preset intervals
   • (currently used on watcher202) Line to add to sudoers using visudo
     • sudo: (FreeBSD) cscfdhcp ALL=NOPASSWD:/usr/local/etc/rc.d/isc-dhcpd restart
     • sudo: (Debian) cscfdhcp ALL=NOPASSWD:sudo /etc/init.d/dhcp3-server restart
     • sudo: (Redhat) cscfdhcp ALL=NOPASSWD:/sbin/service dhcpd restart
   • root crontab entry to reload DHCP server:
     • 0,15,30,45 * * * * /usr/local/etc/rc.d/isc-dhcpd restart > /dev/null 2>&1
9. Run php dipaas/dipaas.php, verify and accept SSH host keys

Editing the config

1. Find master server by logging into any server as cscfdhcp, then do cd ~cscfdhcp/dipaas; ./whoismaster
2. Log in to master server as cscfdhcp and do cd ~cscfdhcp/dipaas; ./master.lock
3. Edit config.inc.php
4. RCS the changes
5. Optionally force configs to sync immediately by doing ./forceupdate
6. Do ./master.unlock

Adding a new server

1. Set up the server (see above)
2. Edit config.inc.php on the master server, add an entry for the new server, be aware that the servers are listed in order of priority (i.e. servers at the top will become masters before servers at the bottom)
3. (If applicable) Add server to the Netscreens' DHCP relaying lists
4. Wait for a config update to go through (1-2 minutes) or force and update with ./forceupdate

Removing a server

1. Edit config.inc.php on the master server, remove the entry for the server that's being removed
2. (If applicable) Remove the server that's being removed from the Netscreens' DHCP relaying lists
3. Wait for a config update to go through (1-2 minutes)
4. Take down the server to be removed

Changing master servers

1. Edit config.inc.php on the current master server, reoder the servers appropriately
2. Run php dipaas.php dofullupdate

SSH key"> Generating a new SSH key

1. On the master server, do: ssh-keygen -t dsa overwriting the default ~/.ssh/id_dsa
2. Copy id_dsa.pub to authorized_keys cp ~/.ssh/id_dsa.pub ~/.ssh/authorized_keys

• You can also use RSA keys (i.e. id_rsa, id_rsa.pub, ssh-keygen -t rsa), but please remove the DSA keys first
• The keys will automatically sync over to slave servers on the next update

Configuration

• edit config.inc.php to your satisfaction
  • the subnet that the DHCP server is on must be listed as a net where DHCP is enabled (limitation of DHCP protocol)
  • if the subnet that the DHCP server is on does not contain any hosts, create a dummy host for any ip on that subnet
  • make sure all DHCP servers managed by this install of DIPAAS are listed
  • list all backup servers
  • this file is well commented and has examples
• /etc/dhcpd.conf should contain a single line including the global config file for DIPAAS, e.g. include "/home/cscfdhcp/dipaas/etc/dhcpd-global.conf";
• the global DIPAAS config file contains all global dhcpd config parameters, make sure there's a section that includes the file generated by DIPAAS, e.g. shared-network CSNets { include "/home/cscfdhcp/dipaas/etc/dhcpd-csnets.conf"; }

Troubleshooting (DHCP isn't working!)

• On Windows, try:
  • ipconfig /release *
  • ipconfig /renew *
• Check the file output by DIPAAS to make sure the MAC/hostname/IP address are present and correct (dhcpd-csnets.conf), if it's not present, check the DIPAAS log for any warnings about that host. Also check in the equipment database to see if the MAC/hostname/IP addresses are all present and correct.
• Check that DHCP relaying is set up properly for that VLAN

Troubleshooting (specific to DIPAAS/DHCPD setup)

• If Sudo-based dhcp restart is enabled, DHCP is refreshed any time there is a change within DIPAAS, currently set to every minute.
• If cron-based dhcp restart is used, Our crontab-based DHCP refresh was done every 15 minutes on the quarter-hour.
  • If you're in a hurry, log in as root on cscf, then ssh watcher202 'su cscfdhcp -c "/usr/local/bin/php /home/cscfdhcp/dipaas/dipaas.php dofullupdate"'
• Check the log file in for DIPAAS in /home/cscfdhcp/dipaas/log/dipaas.log-[hostname]
• Check for DHCPD messages in /var/log/messages
• Check that only one firewall (the one that is the .1 IP for the subnet the DHCP server is on) is doing DHCP relaying

Known problems

• None...

DIPAAS files

• DIPAAS is installed on watcher202.cs:~cscfdhcp/dipaas
• Backup archive located on cscf.cs:~cscfdhcp/backups/dipaas
• Files (directly executable):
  • dipaas.php - DIPAAS exec wrapper, to use, do php dipaas/dipaas.php
  • master.lock / master.unlock - locks/unlocks current server as master, use before editing config
  • whatami - identifies whether current server is master or slave
  • whoismaster - identifies current master server
  • forceupdate - force all servers to sync with this server regardless of master/slave
• Files (everything else):
  • config.inc.php - config file, edit to your heart's content
  • main.inc.php - main DIPAAS codebase
  • lib.inc.php - some useful somewhat generic functions used by DIPAAS
  • modules/db_mysql_equipment.inc.php - module to acquire data from database.cs inventory MySQL database
  • log/* - DIPAAS logs
  • etc/dhcpd-global.conf - global dhcpd options
  • etc/dhcpd-csnets.conf - subnet/host declarations, autogenerated by DIPAAS

-- SevernTsui - 04 Mar 2005