This page is obsolete as of October 2010 and is retained only for historical interest.

Network Load Balancing

Starting in 2008, CSCF will be experimenting with network load balancing of services such as http, smtp, imap, pop, and ipp. The purpose of load balancing is not so much for performance improvements as it is for high availability of CSCF services.

Technology

We'll be using two Sun servers (lb1.cscf and lb2.cscf) running OpenBSD 4.2. No additional software is needed, as we'll be using hoststated, pf(sync) and CARP.

Each server has three network interfaces:

• A public interface for connections from the Internet and the general UW campus
• A private interface to communicate with servers hosting services to be load balanced
• A private interface connecting the servers to each other, to allow for state synchronization.

One server will provide load balancing services, with the second as a backup. We may experiment with an "active/active" configuration as the number of services to be load balanced grows.

How it works (an example with www)

Each server is configured with a "carp" virtual network interface. The carp interface is given the IP address associated with the 'A' record of www.cs.uwaterloo.ca. The carp interfaces are configured in such a way that both servers know about each other and know who's got priority on being the "master" server.

hoststated is configured to listen on the carp interface, relaying connections (using a pf rdr rule) to a pool of web servers on the private network. hoststated also monitors the availability of servers in the pool, so hosts deemed unavailable are removed from the pool.

When a connection to www.cs.uwaterloo.ca arrives, pf essentially does a "reverse NAT" to redirect the connection to one of the webservers. pf can be configured through hoststated to have sticky connections, to ensure a client will maintain a connection to the same server for the duration of the "visit".

Topology Diagram

Migration Strategy

Currently, all services are on a group of hosts with public IP addresses. Since all of these hosts have multiple interfaces, the idea is to connect the second interface on each host to the private "load balanced services network". We can arrange for the services to listen on all interfaces and the load balanced traffic will be on the private network. Eventually, we can stop the service from listening on the "public interface". For this to work properly, it's essential that only the load balancers be configured to forward IP traffic and that no clients be on hosts that span both networks.

References

• CARP/pfsync FAQ
• hoststated man page
• pf.conf man page
• carp man page

-- JasonTestart - 19 Dec 2007