This tutorial is about creating a highly available HTTP load balancer using HAProxy. The setup can be slightly complicated but you will appreciate the result – you will get a load-balanced + highly available web service in your network. Basic linux skill is assumed.
Scenario:
Imagine we have 2 physical machines. In each machine, I have 2 virtual machines. All 4 virtual machines will be in the same subnet, ie 10.1.1.0/24 in this case.
Steps:
1. create 4 vm, centos1.dev (10.1.1.111), centos2.dev (10.1.1.112), centos3.dev (10.1.1.113) and centos4.dev (10.1.1.114). These 4 vm should have the bare min. packages installed.
unless using DNS, add this to /etc/hosts on all virtual machines
10.1.1.111 centos1.dev 10.1.1.112 centos2.dev 10.1.1.113 centos3.dev 10.1.1.114 centos4.dev
2. leave firewall and selinux on. allow port 80 for all 4 vm.
3. centos1.dev and centos2.dev will be the load balancer and centos3.dev and centos4.dev will be the 2 http servers. In centos3 and centos4,
yum groupinstall "web server" chkconfig httpd on
4. Then in centos3.dev and centos4.dev again, edit /etc/httpd/conf/httpd.conf, in order to capture the real IP of the user, replace %h to %{X-Forwarded-For}i. We also add a virtual host.
#LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined ... ... NameVirtualHost *:80 <VirtualHost *:80> DocumentRoot /var/www/html ServerName * SetEnvIf Request_URI "^/haproxy\.txt$" dontlog CustomLog /var/log/httpd/access.log combined env=!dontlog </VirtualHost>
This virtual config is just for the sake of testing, you need to change it in the real environment.
5. In both centos3 and centos 4 again,
cd /var/www/html, echo "centos3" > index.html
(in centos4, echo “centos4” > index.html)
then create haproxy.txt in the same dir for both http servers. Without the check file, haproxy will fail.
touch haproxy.txt
Restart apache (httpd) in both servers
Installing HAProxy:
1. ssh into centos1.dev and centos2.dev and install Haproxy. Someone has compiled the rpm for us. Download it from the rpmbone website and install it.
http://rpm.pbone.net/index.php3/stat/4/idpl/13437166/com/haproxy-1.3.22-1.el5.x86_64.rpm.html
After installing it,
chkconfig haproxy on
2. edit /etc/haproxy/haproxy.cfg
global log 127.0.0.1 local0 log 127.0.0.1 local1 notice maxconn 4096 user haproxy group haproxy defaults log global mode http option httplog option dontlognull option redispatch retries 3 maxconn 2000 contimeout 5000 clitimeout 50000 srvtimeout 50000 listen ha-http 10.1.1.110:80 mode http stats enable stats auth user:password balance roundrobin cookie JSESSIONID prefix option httpclose option forwardfor option httpchk HEAD /haproxy.txt HTTP/1.0 server apache1 centos3.dev:80 cookie A check server apache2 centos4.dev:80 cookie B check
3. To allow HAProxy to bind to the shared IP address, we add the following line to /etc/sysctl.conf:
net.ipv4.ip_nonlocal_bind=1
then reload sysctl config,
sysctl -p
Installing Heartbeat:
1. Heartbeat is necessary for any highly available systems. In both centos1 and centos2, to install heartbeat
yum install heartbeat
2. after that in centos1.dev, cd /etc/ha.d, edit /etc/ha.d/authkeys
auth 2 2 sha1 loadbalancing-ha
3. edit /etc/ha.d/ha.cf
keepalive 2 deadtime 10 udpport 694 bcast eth0 mcast eth0 225.0.0.1 694 1 0 ucast eth0 centos2.dev udp eth0 logfacility local0 node centos1.dev node centos2.dev
node needs to be the machine name, ie type “hostname” in command line to see. Now we want centos1 to be highly available, so edit /etc/ha.d/haresources:
centos1.dev 10.1.1.110
4. if firewall is turned on, remember to allow 694:udp (do it for both centos1.dev and centos2.dev)
5. after setting everything in centos1, copy the files over to centos2, ie
scp {authkeys,haresource,ha.cf} 10.1.1.112:/etc/ha.d
6. now in centos2, edit ha.cf
keepalive 2 deadtime 10 udpport 694 bcast eth0 mcast eth0 225.0.0.1 694 1 0 ucast eth0 centos1.dev udp eth0 logfacility local0 node centos1.dev node centos2.dev
Noticed the difference in ucast
7. Now we want to start heartbeat in both machines upon reboot
echo "service heartbeat start" >> /etc/rc.local
Testing
The ip 10.1.1.110:80 is now load balanced and highly available. To test it, shutdown 10.1.1.111 and the load balancer will still function. If 10.1.1.113 HTTP is down, 10.1.1.114 will take over and vice version.
Viewing Haproxy Stats
1. The options “stats enable” and “stats auth” in the HAProxy configuration allow the admin to view the stats, just go to http://10.1.1.110/haproxy?stats and type in username as user and password as password
Conclusion
I hope you follow me so far and appreciate what HAproxy can offer. I certainly enjoy blogging about it and I hope you it useful.
I’m newbie in linux.. Can you describe, what “centos.dev” mean?
I can’t found *.dev in my VMWare..
Thank you.. This posting are very useful..
To Bayu Stormy:
centos?.dev are privately defined domain names.
You may use centos?.mylab, or whatever you want.
Just make sure not to conflict with the internet domain names.
centos1.dev have 2 interface right?
If I want to set eth0 : Publick IP and eth1 : 10.1.1.111
what the haproxy.cfg look like?
Btw Nice Artikel.. I’m Newbie in CentOS…
Can you explain me why “listen ha-http 10.1.1.110:80” in haproxy.cfg? Why not “listen ha-http 10.1.1.111:80”?
Thanks..