Load Balancing with HAProxy and Apache

Posted by in Tech on Mar 30th, 2011
Load Balancing with HAProxy and Apache


What you need to know:


1. Familiar with linux text editors
2. Have a basic understanding of load balancing
3. Familiarity with Apache, MySQL, and rsync
4. The ability to create at least 4 virtual machines


What does this set-up provide?


This set-up will provide you with some very basic failover at the web servers by using a load balancer. Keep in mind I will not be discussing HA at the load balancer level so you will still have a single point of failure with this guide, as it is only intended to provide some fundamental knowledge on how to set-up a replicated and load balanced pair of web servers.


Why is this useful?


If your website needs some redundancy as well as the need to be able to handle large volumes of traffic then this will give you a good starting point. This solution will not fit every website’s needs so you must design to fit your application, performance requirements, disaster recovery plan, redundancy and scalability. This guide will however provide you with the basics to configure a small web server farm that will handle most ‘run of the mill’ websites.


Getting Started..


First and foremost we need four virtual machines running CentOS 5.5. Just about any hypervisor will work. If you do not have one readily available you can download VMWare Server, or Virtual box for free. Once your hypervisor is installed you need to proceed with creating your virtual machines. These will not require much disk space for this “how to”. Creating these virtual machines with 256MB RAM and 10GB of disk space should be sufficient for a testing environment (not for a production website). When creating these virtual machines keep in mind to set their host names respective to their roles for easy identification. For example:

Web Server 1 – WWW1
Web Server 2 – WWW2
Load Balancer – LB1
Database Server – DB1

Feel free to set these to whatever you would like as long as you can remember their roles. For this example we will use the host names listed above. Once you have all virtual machines installed go ahead and ensure they have network connectivity and that they are up to date. To verify this run the following commands:

yum clean all
yum -y update
yum -y upgrade

Now that we have our environment ready to go we can get started with configuring the virtual machines.


LB1 – HAProxy Load Balancer


Before we get started with the installation of the load balancer we need to ensure that we have the correct YUM repositories installed. With that said the only additional repository that we will need is EPEL. Here is a link for the EPEL repository that will outline all the details for the repository: http://fedoraproject.org/wiki/EPEL

To install the EPEL repository simply run the following:

rpm -Uvh http://download.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm

Now that our repository is ready go ahead and install HAProxy:

yum -y install haproxy

Now comes the configuration. Fortunately HAProxy is fairly simple to configure however we need some basic information before we can proceed.

1. eth0 IP of the load balancer
2. eth0 IP of both web servers

Once you have this information you can go ahead and edit the configuration file located at /etc/haproxy/haproxy.cfg

Example:

global
 
    #logging is designed to work with syslog facility's due to chrooted environment
    #log loghost    local0 info - By default this is commented out
 
    #chroot directory
    chroot /usr/share/haproxy
 
    #user/group id
    uid 99
    gid 99
 
    #running mode
    daemon
defaults
 
    #HTTP Log format
    mode http
 
    #number of connection retries for the session
    retries 3
 
    #try another webhead if retry fails
    option redispatch
 
    #session settings - max connections, and session timeout values
    maxconn 10000
    contimeout 10000
    clitimeout 50000
    srvtimeout 50000
 
listen webfarm 127.0.0.5:80
 
    #HTTP Log format
    mode http
    #stats uri /haproxy
    stats uri /haproxy
 
    #balance roundrobin - Typical Round Robin
    #balance leastconn - Least Connections
    #balance static-rr - Static Round Robin - Same as round robin, but weights have no effect
    balance roundrobin
 
    #cookie <COOKIENAME> prefix - Used for cookie-based persistence
    cookie webpool insert
 
    #option httpclose - http connection closing
    option  httpclose
 
    #option forwardfor - best stated as "Enable insertion of the X-Forwarded-For header to requests sent to the web heads" aka send EU IP
    option forwardfor
 
    #Web Servers (Examples)
    server WWW1 127.0.0.1:80 check port 80 # Active in rotation   
    server WWW2 127.0.0.2:80 check port 80 # Active in rotation

HAProxy is now configured. As you can see this was a pretty simple process, however there are a few things left to do..

1. Ensure that HAProxy starts on reboot

chkconfig haproxy on

2. Configure iptables to allow HTTP access to port 80

iptables -I INPUT -m tcp -p tcp --dport 80 -j ACCEPT

3. Start the service

/etc/init.d/haproxy start

At this point you should be able to open a browser and view the HAProxy status page. If your eth0 IP address for your load balancer was 127.0.0.5 you could browse to http://127.0.0.5/haproxy to view the current status.


WWW1/2 – Apache Web Servers


This part isnt much more difficult than setting up your load balancer however the apache configuration file syntax is a little more cryptic. So first things first.. Install apache and configure iptables.

yum -y install httpd
chkconfig httpd on

This will provide you with a basic installation of apache that will work for the purpose of this example. At the bottom of this portion I will provide an example of a more advanced configuration file for use with virtual hosts. As with our load balancer we will need to configure iptables to allow HTTP traffic access to port 80.

iptables -I INPUT -m tcp -p tcp --dport 80 -j ACCEPT

By default without any configuration changes apache sets your document root to /var/www/html. This is where you will store your website content. This can be changed to fit your needs when creating virtual hosts, or for security concerns. To create an example web page simply run the following command. This will allow us to ensure the load balancer is working later on.

On WWW1:

echo "WWW1" > /var/www/html/index.html

On WWW2:

echo "WWW2" > /var/www/html/index.html

Now that you have installed a fake temporary website we should be able to start apache and test our load balancer. Starting apache is very similar to starting HAProxy. Simply run:

/etc/init.d/httpd start

If you type http://127.0.0.5/ into your browser you should be greeted by one of your web servers with a response of “WWW1″ or “WWW2″ on the screen. If you press F5 on your keyboard (command + R for mac) to refresh the page multiple times you should see the text change back and forth. This is because we configured the load balancer with ’round robin’ which sends one request to WWW1 the second to WWW2 the third to WWW1 and so on. At this point once you verify your load balancer worked as expected you can configure a basic cron job using rsync to replicate your data.

In this example we are only using basic one way replication provided by rsync. In order to do this we need to setup a key pair for our web servers. Simple way to do this is as follows:

[root@WWW1 ~]$ ssh-keygen -t rsa -b 2048

Once you have saved the key you need to copy the public key over to WWW2 in /root/.ssh/authorized_keys. This file may not exist so you will need to create it. After setting our public key on WWW2 verify you can login as the root user from WWW1 to WWW2. If this is successful add the following to /etc/crontab:

05 * * * * rsync -aPSv -e 'ssh' /var/www/html 127.0.0.2:/var/www/html

Make sure you edit the IP addresses listed in this article to be the same as those of your virtual machines. The ones used here will not work and are only for examples. At this point we have set up and configured our load balancer as well as our web servers and replication. To test replication simply login to WWW1 and run

touch /var/www/html/text.txt

Give it a few minutes and you should see this file show up on WWW2. If all is well then move on to the next section to configure your database server.

Example Virtual Host:

<VirtualHost 127.0.0.2:80>
	ServerName	example.com
	ServerAlias	example.com
	UseCanonicalName Off
	DocumentRoot	/home/user/public_html/example.com
	ErrorLog	/var/log/httpd/example.com-error_log
	CustomLog	/var/log/httpd/example.com-access_log combined
	<Directory /home/user/public_html/example.com>
		Options Indexes FollowSymLinks
		IndexOptions FancyIndexing
		AllowOverride All
		Order allow,deny
		Allow from all
	</Directory>
		php_admin_value open_basedir "/home/user/public_html/example.com:/tmp"
		php_admin_flag safe_mode On
</VirtualHost>


DB1 – MySQL Database Server


This section is going to be pretty brief as the exact configuration for MySQL is dependant on your application. With that in mind here we go..

First and foremost install the service:

yum -y install mysql mysql-server
chkconfig mysqld on

Once installed you can proceed with any configuration changes in /etc/my.cnf or simply start the service as is by running:

/etc/init.d/mysqld start

As with apache and HAProxy we must configure the firewall to be able to communicate with our web servers, however this time not on port 80.

iptables -I INPUT -m tcp -p tcp --dport 3306 -j ACCEPT

And thats it! If you are planning on using a database driven application such as wordpress you will want to set your wp-config.php to point to the eth0 IP of your DB1 server during the configuration portion of the wordpress setup.

I hope this helps anyone looking for a similar configuration, as well as provide some ideas on how to set up something like this. If you have questions please feel free to ask by posting here!