How to build an Asterisk Cluster

How to build an Asterisk Cluster

Intuit is conducting a first online-workshop via WebEx about building a Asterisk-Cluster.
This first workshop focuses on a Active-Passive setup.
(The next workshop will cover a large Active-Active setup with multiple Asterisk, DB and Load Balance servers.)

Details:
Date/Time: Friday, 20. May 2009 at 8 AM, GMT +8 (Kuala Lumpur Time... )
Duration: 2 hours (depends on how many questions we have..)
Topics covered:
- How to choose your hardware (Servers and PSTN equipment, we will use a FoneBridge from Redfone)
- How to prepare your OS for cluster (we use Debian 5 in the workshop)
- Asterisk standard vanilla installation (we will use our install script)
- Explain the concept of DRBD and Heartbeat
- Configure DRBD and Heartbeat
- Test the cluster
- Discussion
Cost: FOC (you can buy me a beer if you are in Kuala Lumpur)
Please e-mail me or Marco if you want to join the seminar and we will send you the meeting-invite!
Marco: marco@intuitinnovations.com
Daniel: daniel@intuitinnovations.com
Cheers
Daniel

Asterisk High Availability HOWTO with Heartbeat and Redfone fonebridge

Overview
The following is a brief HOWTO for installing High-Availability Asterisk using Open Source tools combined with fail-over capable & intelligent hardware (the fonebridge).
The heartbeat utility is used in a 'Passive-Active' scenario but could easily be modified to do 'Active-Active'.

Background
Some of our more demanding customers in the Call Center and Banking Industry are loathe to accept an implementation with no mechanism for fail-over and high-availability so this is the hardware/software combination we are using to meet their demands.

Client Background
The following scenario was used for a medium sized call center operation with about 60 analog stations, and a single T1 PRI.

Hardware

• 2 x 1U Supermicro Servers (P4, 512Mb, Dual Gig Eth, Dual SATA with RAID 0)
• 1 x Redfone Quad T1 fonebridge to terminate PRI connectivity, power channel banks and provide fail-over capability between the two Supermicros.
• 1 x T1 PRI
• 3 x Adtran 750 FXS channel banks to drive analog phones
• 2 x UPS/Surge Protectors

Software

• Fedora Core 4
• Asterisk, zaptel, libpri from CVS head
• Linux HA software suite from Ultramonkey. They have RPMs for RHE3 that install fine on Fedora Core 4
• Each server is a mirror image of the other in terms of Asterisk configs and software.

Software Install
After a standard install of FC4, Asterisk, zaptel, libpri we installed all of the packages from Ultramonkey pretty much following their guidelines: http://www.ultramonkey.org/3/installation-rh.el.3.html
You may have a few dependencies issues, mainly perl libs, but we were able to satisfy all of them by using Yum. If you are running Apt you should be able to accomplish the same thing.

Configuring Hearbeat
After installing heartbeat there are only three files that need to be modified for your environment. They are ha.cf, haresources and authkeys. They should all be placed in the /etc/ha.d/ directory. The files should be absolutely identical on all machines that are part of your Asterisk high-availability cluster. We only have two servers running but you could easily scale to more using the exact same configurations. These are our config files. All comment lines have been removed but as you can see they are short and simple.

ha.cf
debugfile /var/log/ha-debug
logfile /var/log/ha-log
logfacility local0
keepalive 200ms
deadtime 2
warntime 1
initdead 120
udpport 694
bcast eth0
node asterisk1
node asterisk2

haresources
asterisk1 10.10.10.110 fonulator asterisk

authkeys
auth 1
1 sha1 SuPerS&cretP@$$werd

Operation
Each Asterisk server has a unique IP address which is part of the LAN segment. This could be a NATed network or Internet facing with public IP addresses. Heartbeat manages the monitoring of the hardware state of each machine over Ethernet or serial port or a combination of both (recommended) and assigns the Virtual IP to the Asterisk server which is currently in an active state. Example;

Asterisk1= 10.10.10.100
Asterisk2= 10.10.10.120
Virtual IP= 10.10.10.110 (see haresources)

With Heartbeat it is important that your node names are identical to the host names reflected in #uname -n. You also may need to manually add IP/hosts statements to your /etc/hosts file so each machine knows how to reach the other via IP.

Following the rules in haresources, Heartbeat will assign machine name asterisk1 as the primary server when both systems start up. It will then start the following scripts; fonulator (this is the little script that configures the fonebridge) and asterisk which starts the Asterisk server. These are both standard startup scripts placed in /etc/init.d/ .
If the Primary server suffers a hardware fault or simply stops responding to the heartbeats going between the two nodes asterisk2 will execute /etc/init.d/fonulator start to reconfigure the fonebridge on the fly and begin redirecting traffic to asterisk2 followed by /etc/init.d/asterisk start to start the Asterisk server.

Results
With heartbeat, IP takeover occurs in under a second. The fonulator utility re-configures the fonebridge in just about the same amount of time and then depending on your hardware platform and the complexity of apps running in Asterisk it can take between 5-15 seconds for Asterisk to start up on your secondary server, load all config files, clear alarms and be ready to process calls. Total fail-over time about 15-20 seconds.

Resources
Ultramonkey http://www.ultramonkey.org (High Avail software packages)
Linux HA http://www.linux-ha.org (The High Availability Linux Project)
Redfone http://www.red-fone.com (Maker of the Quad T1/E1 fonebridge)

High Availability Cluster (Heartbeat) On CentOS

Configuring A High Availability Cluster (Heartbeat) On CentOS

This guide shows how you can set up a two node, high-availability HTTP cluster with heartbeat on CentOS. Both nodes use the Apache web server to serve the same content.

Pre-Configuration Requirements

1. Assign hostname node01 to primary node with IP address 172.16.4.80 to eth0.
2. Assign hostname node02 to slave node with IP address 172.16.4.81.

Note: on node01

uname -n

must return node01.
On node02

uname -n

must return node02.
172.16.4.82 is the virtual IP address that will be used for our Apache webserver (i.e., Apache will listen on that address).

Configuration

1. Download and install the heartbeat package. In our case we are using CentOS so we will install heartbeat with yum:

yum install heartbeat

or download these packages:

heartbeat-2.08
heartbeat-pils-2.08
heartbeat-stonith-2.08

2. Now we have to configure heartbeat on our two node cluster. We will deal with three files. These are:

authkeys
ha.cf
haresources

3. Now moving to our configuration. But there is one more thing to do, that is to copy these files to the /etc/ha.d directory. In our case we copy these files as given below:

cp /usr/share/doc/heartbeat-2.1.2/authkeys /etc/ha.d/
cp /usr/share/doc/heartbeat-2.1.2/ha.cf /etc/ha.d/
cp /usr/share/doc/heartbeat-2.1.2/haresources /etc/ha.d/

4. Now let's start configuring heartbeat. First we will deal with the authkeys file, we will use authentication method 2 (sha1). For this we will make changes in the authkeys file as below.

vi /etc/ha.d/authkeys

Then add the following lines:
auth 2
2 sha1 test-haChange the permission of the authkeys file:

chmod 600 /etc/ha.d/authkeys

5. Moving to our second file (ha.cf) which is the most important. So edit the ha.cf file with vi:

vi /etc/ha.d/ha.cf

Add the following lines in the ha.cf file:
logfile /var/log/ha-log
logfacility local0
keepalive 2
deadtime 30
initdead 120
bcast eth0
udpport 694
auto_failback on
node node01
node node02Note: node01 and node02 is the output generated by

uname -n

6. The final piece of work in our configuration is to edit the haresources file. This file contains the information about resources which we want to highly enable. In our case we want the webserver (httpd) highly available:

vi /etc/ha.d/haresources

Add the following line:
node01 172.16.4.82 httpd7. Copy the /etc/ha.d/ directory from node01 to node02:

scp -r /etc/ha.d/ root@node02:/etc/

8. As we want httpd highly enabled let's start configuring httpd:

vi /etc/httpd/conf/httpd.conf

Add this line in httpd.conf:
Listen 172.16.4.82:809. Copy the /etc/httpd/conf/httpd.conf file to node02:

scp /etc/httpd/conf/httpd.conf root@node02:/etc/httpd/conf/

10. Create the file index.html on both nodes (node01 & node02):
On node01:

echo "node01 apache test server" > /var/www/html/index.html

On node02:

echo "node02 apache test server" > /var/www/html/index.html

11. Now start heartbeat on the primary node01 and slave node02:

/etc/init.d/heartbeat start

12. Open web-browser and type in the URL:

http://172.16.4.82

It will show node01 apache test server.
13. Now stop the hearbeat daemon on node01:

/etc/init.d/heartbeat stop

In your browser type in the URL http://172.16.4.82 and press enter.
It will show node02 apache test server.
14. We don't need to create a virtual network interface and assign an IP address (172.16.4.82) to it. Heartbeat will do this for you, and start the service (httpd) itself. So don't worry about this.
Don't use the IP addresses 172.16.4.80 and 172.16.4.81 for services. These addresses are used by heartbeat for communication between node01 and node02. When any of them will be used for services/resources, it will disturb hearbeat and will not work. Be carefull!!!