The DHCP information system in Computer Science

Overview

CSCF-CSI provides a DHCP infrastructure for computers, printers and other devices on its networks. This infrastructure consists of multiple redundant DHCP servers available to respond to DHCP requests from clients, coupled with a reservation-generating system (DIPAAS) that extracts the necessary reservation information from the CS Inventory system.

There are three distinct topics for discussion:

1. how clients should configure their computers to use DHCP, including known OS-specific issues
2. troubleshooting ("I'm not getting an address!")
3. details of our DHCP server systems (we use ISC-DHCP "isc-dhcpd-V3.1.3"), including details of DIPAAS -- the reservation-builder application, with operational and developer information

By and large, general CSCF staff should not need to interact with the DHCP servers. The reservation-generator checks for updates every minute, and if changes are found, a new reservation file is activated and all of the servers are restarted.

For those interested in a detailed description of the DIPAAS program for the purpose of code development or debugging, see the DIPAAS-plm document.

---

Client configuration and OS quirks

We support three major OS platforms for clients: Ubuntu, Windows and MacOSX. In most cases the client software on these systems does what is expected and there are no known issues for basic functionality (getting a name and IP address).

There are some issues with the new "domain-search" directive that the DHCP server provides to clients in order to five a list of domain-names that the client's DNS resolver can use to resolve names. Some systems do not support this at all (MacOSX, Windows), and some will support it as long as the client configuration is appropriate.

Ubuntu

For "modern" versions of dhclient (v3.1.3 and up) will support the domain-search directive if the following option is present in the dhclient.conf file (typically /etc/dhcp3/dhclient.conf):

option domain-search code 119 = string;

This statement should be placed before any of the "request" or "send" directives.

The "domain-search" directive modifies the behaviour of the "resolver library" (the OS code that resolves names into IP addresses). The idea is that the domain-search can provide multiple domain-names, and the resolver library can try each name in turn. However, Ubuntu in our environment has a more general problem with the (non-)interaction of the standard NetworkManager application and the traditional "dhclient with /etc/network/interfaces file", and their effect on the resolver library.

The resolver library depends on the file /etc/resolv.conf, and both the methods mentioned above re-write the file. Modifications to the file by hand will be lost. If the "dhclient with interface file" method is used, there resolv.conf rewrite is determined solely by the response from the DHCP server and thus the option statement above is important. If the NetworkManager method is used, the end-user can modify what the software will write into resolv.conf -- in essence the user can blend the DHCP server information with local custom settings.

Ubuntu NetworkManager customization

To make domain-search modifications with the NetworkManager:

1. use the GUI and open "Edit Connections..."
2. select the relevant interface, and "Edit..."
3. choose the "IPv4 Settings" tab
4. choose "Automatic (DHCP) addresses only" from the "Method" box
5. fill in both the "Search domains:" box (space-delimited) and the "DNS Servers:" box
6. save your settings
7. disconnect and reconnect the connection to make the new settings active

Windows

To be determined, connection-specific settings can be used.

MacOSX

The MacOSX DHCP client does not expose the same kind of option settings as Ubuntu, so the only choice for MacOSX users is to override the search-domain setting:

It appears that you can choose to over-ride the search domains you receive from DHCP:

• System Preference
  • Network
    • Advanced
      • DNS -- may see greyed entries in the Search Domains list. Click "+" and add as many entries as you'd like. Add them in reverse order, so that you end up top-to-bottom with your prioritized list.

---

Our base DHCP server installation

We have four DHCP server running in our network, two per building. They run on the same systems as the DNS servers:

• Davis Centre

  DC DNS/DHCP servers

  System	IP	Location
  dc-dns-1	172.19.32.5	Linux container on dc-cloud102-c2.cs.uwaterloo.ca
  dc-dns-2	172.19.32.6	Linux container on dc-cloud104-c2.cs.uwaterloo.ca

• Math and Computer

  DC DNS/DHCP servers

  System	IP	Location
  mc-dns-1	172.19.47.5	Linux container on mc-cloud002-c1.cs.uwaterloo.ca
  mc-dns-2	172.19.47.6	Linux container on mc-cloud002-c2.cs.uwaterloo.ca

The basic install is ISC latest (as of December 2011) stable DHCP Ubuntu package, which is v3.1.3. To install:

apt-get install --install-recommends dhcp3-server

We use all the installation defaults. Of note:

• /etc/dhcp3 contains the configuration file dhcpd.conf. In our setup, this is a simple include of the DIPAAS-generated configuration file (located in /etc/dhcp3/cscfdhcp/dipaas/etc).
• logging goes to /var/log/daemonlog. A really useful command for monitoring DHCP activity in the log is:

      tail -f /var/log/daemon.log | grep dhcpd

• the service is controlled by /etc/init.d/dhcp3-server, which supports the usual arguments (status, restart, start, stop)

Local login to these systems is not supported. Log in as root from cscf.cs.uwaterloo.ca. When necessary to become userid "cscfdhcp" (see following discussions), use `su - cscfdhcp`.

Information for developers of DIPAAS

For those interested in a detailed description of the DIPAAS program for the purpose of code development or debugging, see the DIPAAS-plm ("program logic manual") document.

---

Troubleshooting

Network configuration

If a client workstation is failing to be assigned an address, and the network definition is new (which includes existing networks for which the CIDR block (netmask) has changed), make sure that the network is correctly defined in the DIPAAS config.inc.php file (see the DIPAAS-plm ("program logic manual") for details.

Status dashboard

https://www.cs.uwaterloo.ca/cscf/internal/networks/DHCP/index.php

From the client

Linux

It has been observed that the Ubuntu NetworkManager and the old-style /etc/network/interfaces methods of network management are incompatible. Decide which one you are going to use, and stick with it.

Useful commands:

dhclient -r
dhclient

Useful files:

/var/lib/dhcp3/dhclient.leases
/etc/resolv.conf

Windows

Useful commands:

ipconfig /all
ipconfig /release
ipconfig /renew

Note that the "release" followed by the "renew" is important. Just a "renew" will attempt to send a release message to the current DHCP server. If that server isn't there (we have multiple servers that can grant identical leases), then the client has to time out on the release before it can renew. "Release" by itself doesn't care if the release is acknowledged -- it just discards the lease.

MacOSX

At the DIPAAS server

*** DIPASS must be configured to provide service for the vlan(s) that the server(s) nic is using or else the DHCP3 service will not listen for any DHCP request broadcasts on that external NIC. (dlg 2015-09-25) ***

(December 2011) We currently have four DHCP servers running. Dc-dns-2.cs.uwaterloo.ca is the master server where DIPAAS runs, and dc-dns-1.cs.uwaterloo.ca, mc-dns-1.cs.uwaterloo.ca and mc-dns-2.cs.uwaterloo.ca are its slaves. Make sure that one of them is running. If the master is not running, then updates via inventory will not be working.

Useful commands:

/etc/init.d/dhcp3-server status 
/etc/init.d/dhcp3-server restart 

# Chech for warning that dhcp3 is not listening for broadcast requests
grep "dhcpd: Not configured to listen on any interfaces!" /var/log/syslog
# Check if server is even seeing requests
grep DHCPREQUEST /var/log/syslog
# Check if server is offerring leases to requests
grep DHCPOFFER /var/log/syslog

# other places to look for this information.
tail -f /etc/dhcp3/cscfdhcp/dipaas/log/dipass.log-*
tail -f /var/log/daemon.log | grep dhcpd

Lastly for the log entries you discover above, double check that the time stamp at the beginning of the line, is within the time frame that you're concerned about.

For specific client failure:

• look at /etc/dhcp3/cscfdhcp/dipaas/etc/dhcpd-csnets.conf and find the reservation stanza for the client.
• look at the inventory system to confirm the MAC address

Previous version (historical interest only)