Introduction

Below you'll find a step by step setup for installing FS as a SBC. The LCR engine is provided by
Kamailio and its module carrierroute. Kamailio is an opensource SIP Proxy (not a B2BUA).
In this setup, the dialplan is detailed only for inbound to outbound traffic, but it could be easily
extended for outbound to inbound traffic (or DID).
The design is the following:
FS is configured with an internal and an external profile, each profile listening on a different
network interface.
Kamailio is listening on the loopback interface, and is not used by any other process than FS.
Internal--------FS--------External
|
|
Kamailio

When FS receives an INVITE, FS fires this event in the default context, where it always matches
the extension named LOOKUP_ROUTE. In this extension, FS sends the INVITE to Kamailio, that
will replies with a 302 Redirect SIP message that contains the route FS has to use to reach the
number dialed. The call is then transfered in the context ROUTING where FS will match the
associated Gateway (in this setup it is an associated code) that will route the call.

If Kamailio has no route for the dialed number, it replies with a "604 unable to route the call" that
FS will relay.
So Kamailio is used only for route lookup by FS, and FS is the only equipment that is seen in the
SIP transaction.

This design has been choosen cause I needed a way to store big LCR table (it's now more than
450 000 routes on the production server) and LCR module was not available when I began playing
with FS (version 1.0.1).
I extensively tested this setup on an Intel Quad Core server (Q9550 running at 2.83Ghz with 8GB
RAM. 4GB should have been far enough).
I ran some tests using SIPP with 750 simultaneous calls and between 50 to 100 calls per second
(CPS) for more than 10 days with 25%-30% CPU ressource free. FS version was 1.2 at this time.

Advantages
Stable with quite huge routing table: Kamailio was using less than 3% CPU when doing the SIPP
tests with 160 000 entries in LCR table.
Scalable: some people are using carrierroute module with LCR table up to 1 million entries.
Correct call per second rate: up to 100cps.

Drawbacks
Each INVITE you send to FS will first send an INVITE to Kamailio, and once FS receives the
Kamailio answer, FS will bridge the call to the Peer that will process the call.
I'm feeling that if you want to process more call per seconds (more than 100 CPS), the fact that the
Sofia library is monothreaded and that it has to deal with one more session (FS to Kamailio) before
processing the call to the Peer could prevent FS to reach a higher CPS rate. (For everybody that
has a better internal knowledge of FS than me, feel free to correct this assumption).

Other ways to go to achieve the same (not tested)

Try mod_lcr that has been introduced in FS 1.0.3 and report on the list and Wiki.
Use the XML CURL module to process the call routing using your preferred script language
running on yout HTTP server.
Preliminaries
Install Debian stable netinstall (64 bits version, take care you need a 64bit processor)
http://cdimage.debian.org/debian-cd/5.0.2/amd64/iso-cd/debian-502-amd64-netinst.iso
Upgrade debian distribution
apt-get update
apt-get dist-upgrade

Install SSH Server for remote administration

apt-get install openssh-server

Install Vlan packages

apt-get install vlan

Install Bridge packages

apt-get install bridge-utils

Install Packet Capture packages

apt-get install ngrep
apt-get install tshark
apt-get install tcpdump

Install text editor

apt-get install emacs22-nox

Install screen
apt-get install screen

Disable useless services

/etc/init.d/exim4 stop
update-rc.d -f exim4 remove
/etc/init.d/portmap stop
update-rc.d -f portmap remove
/etc/init.d/nfs-common stop
update-rc.d -f nfs-common remove

Restart server
reboot

Check TCP or UDP services running

netstat -tnlp
only SSH (port 22) should be listening
netstat -unlp
only dhclient3 (port 68) should be listening (I need dhcp for my tests and to get an IP for
administration of the test server)

Install utils for compilation

apt-get install build-essential
Installation of FreeSWITCH
Install dependencies
apt-get install subversion subversion-tools automake1.9 gcc-4.1 autoconf make wget libtool g++
libncurses5 libncurses5-dev
apt-get install debhelper automake1.9 autoconf libtool unixodbc-dev libasound2-dev libcurl4-
openssl-dev libssl-dev libogg-dev libvorbis-dev libperl-dev libgdbm-dev libdb-dev libgnutls-dev
libspandsp-dev libtiff4-dev

Download FreeSWITCH latest Git and put them in /opt/sources/freewitch

cd /opt/sources/
git clone <git://git.freeswitch.org/freeswitch.git> freeswitch-git
cd freeswitch-git
./bootstrap.sh
Edit modules.conf depending on your needs
./configure -prefix=/opt/freeswitch-git --enable-core-odbc-support
make
make install
make sounds-install
make moh-install

To upgrade to latest Git

cd /opt/sources/freeswitch-git
make current
and that's it you get the latest Git in /opt/freeswitch-git

Prepare Network interface (I'm using a switch with vlans, useful for network isolation)
* Load linux vlan module
modprobe 8021q

* add 2 vlans (one for internal profile ex: vlan 100, the other for external profile ex: vlan 200)
vconfig add eth0 100
vconfig add eth0 200

* bring these interfaces up

ip link set up dev eth0.100
ip link set up dev eth0.200

* bring them IP
ip addr add 10.10.10.254/24 dev eth0.100
ip addr add 10.10.20.254/24 dev eth0.200

Edit sip_profiles to use these IP

cd /opt/freeswitch/conf/sip_profiles
edit internal.xml and replace $${local_ip_v4} with internal IP (10.10.10.254)
edit external.xml and replace $${local_ip_v4} and $${external_rtp_ip} with external IP
(10.10.20.254)

Optimizations before launching FS

ulimit -c unlimited
ulimit -d unlimited
ulimit -f unlimited
ulimit -i unlimited
ulimit -n 999999
ulimit -q unlimited
ulimit -u unlimited
ulimit -v unlimited
ulimit -x unlimited
ulimit -s 240
ulimit -l unlimited
ulimit -a

Start FreeSWITCH 

/opt/freeswitch/bin/freeswitch -nc -hp

-nc stands for no console
-hp stands for high priority

Connect on FreeSWITCH via fs_cli

/opt/freeswitch/bin/fs_cli
with this tool you'll be able to get a console connection to FS
the log level has not been changed so it could be very verbose
to disable log, type /nolog in console
Check if FS is running
netstat -unlp
you should have this
udp 0 0 10.10.10.254:5060 0.0.0.0:* 9240/freeswitch
udp 0 0 10.10.20.254:5060 0.0.0.0:* 9240/freeswitch
Installation of Kamailio
Installation of Bison, a parser generator
apt-get install bison

Installation of Flex, a fast lexical analyzer generator

apt-get install flex

Installation of MySQL and MySQL dev libraries

apt-get install mysql-server-5.0
You'll be prompt to set the password of your database, let's test with password "1234" or whatever
you want

apt-get install libmysql++-dev

Installation of libconfuse, needed for carrierroute module compilation

apt-get install libconfuse-dev

Get sources of Kamailio and put them in /opt/sources

cd /opt/sources
wget http://www.kamailio.org/pub/kamailio/1.5.0/src/kamailio-1.5.0-notls%5Fsrc.tar.gz
tar -xvzf kamailio-1.5.0-notls_src.tar.gz
cd kamailio-1.5.0-notls

If you need a LCR route table of approximatively 50 000 routes, then bypass this step.
For those who'd like to load a bigger LCR route table in carrierroute (for my needs: 160 000 routes)
emacs /opt/sources/kamailio-1.5.0-notls/config.h
#define PKG_MEM_POOL_SIZE 32*1024*1024 /*!< Used only if PKG_MALLOC is defined*/
#define SHM_MEM_SIZE 4*32 /*!< Used if SH_MEM is defined*/

Note: these values have been set by empirical means and are maybe too high even for 160 000
routes

Compile Kamailio in /opt/kamailio-1.5.0

make prefix=/opt/kamailio-1.5.0
make prefix=/opt/kamailio-1.5.0 modules=modules/db_mysql modules
make prefix=/opt/kamailio-1.5.0 modules=modules/carrierroute modules
make prefix=/opt/kamailio-1.5.0 install
make prefix=/opt/kamailio-1.5.0 modules=modules/carrierroute install
make prefix=/opt/kamailio-1.5.0 modules=modules/db_mysql install

Add a symbolic link for /opt/kamailio-1.5.0

ln -sf /opt/kamailio-1.5.0/ /opt/kamailio

Edit Kamailio Database Settings

cd /opt/kamailio/etc/kamailio/
emacs kamctlrc
# this parameter...
DBENGINE=MYSQL
## database host
DBHOST=localhost
## database name (for ORACLE this is TNS name)
DBNAME=openser
# database path used by dbtext or db_berkeley
# DB_PATH="/usr/local/etc/kamailio/dbtext"
## database read/write user
DBRWUSER=openser
## password for database read/write user
DBRWPW="1234"
## database read only user
DBROUSER=openserro
## password for database read only user
DBROPW="1234"
## database super user (for ORACLE this is 'scheme-creator' user)
DBROOTUSER="root"
# user name column
USERCOL="username"

Then launch Kamailio utils to create database

/opt/kamailio/sbin/kamdbctl create
You'll be prompted for root sql password (remember you used 1234 or ...) and don't forget to set
yes for carrierroute table creation

Populate carrierroute database with some routes

Connect to mysql
mysql -u root -p
Enter your root password
In mysql console:
use openser;
INSERT INTO carrier_name (carrier) VALUES ("default");
INSERT INTO domain_name (domain) VALUES ("default");
INSERT INTO carrierroute
(carrier,domain,scan_prefix,flags,mask,prob,strip,rewrite_host,rewrite_prefix,rewrite_suffix,descriptio
VALUES("1","1","1000","0","0","1","0","PEER-01","","","FRANCE");
exit

You have inserted your first route, that will match prefix 1000 and route this to PEER_01 with a
description, FRANCE in this example. PEER_01 will be defined later in FS dialplan. The
description parameter could be exported, for billing purpose for example. In this setup, I'll export
the description to an AREA variable in FS.

Configuration of Kamailio
cd /opt/kamailio/etc/kamailio
mv kamailio.cfg kamailio.cfg.default
emacs kamailio.cfg

and put the config below:

# KAMAILIO basic configuration script for use with carrierroute and FS
####### Global Parameters #########
debug=1
log_stderror=no
log_facility=LOG_LOCAL0
fork=yes
children=8
/* uncomment the next line to disable TCP (default on) */
disable_tcp=yes
/* uncomment the next line to disable the auto discovery of local aliases
based on revers DNS on IPs (default on) */
auto_aliases=no
/* uncomment and configure the following line if you want opensips to
bind on a specific interface/port/proto (default bind on all available) */
listen=udp:127.0.0.1:5062
####### Modules Section ########
#set module path
mpath="/opt/kamailio/lib64/kamailio/modules/"
/* uncomment next line for MySQL DB support */
loadmodule "db_mysql.so"
loadmodule "sl.so"
loadmodule "tm.so"
loadmodule "rr.so"
loadmodule "maxfwd.so"
loadmodule "textops.so"
loadmodule "mi_fifo.so"
loadmodule "xlog.so"
loadmodule "carrierroute.so"
loadmodule "pv.so"
loadmodule "avpops.so"
#----------------- setting module-specific parameters ---------------
# ----- mi_fifo params -----
modparam("mi_fifo", "fifo_name", "/tmp/kamailio_fifo")
# ------ Carrierroute ------
modparam("carrierroute", "config_source", "db")
modparam("carrierroute", "db_url", "mysql://openser:1234@localhost/openser")
modparam("carrierroute", "carrierroute_table", "carrierroute")
modparam("carrierroute", "default_tree", "default")
modparam("carrierroute", "fetch_rows", 2000)
####### Routing Logic ########
# main request routing logic
route{
if (!mf_process_maxfwd_header("10")) {
sl_send_reply("483","Too Many Hops");
exit;
}
t_check_trans();
if ($rU==NULL) {
# request with no Username in RURI
sl_send_reply("484","Address Incomplete");
exit;
}
# LOOKUP ROUTE TABLE WHEN ASKED BY HEADER: X-ROUTE:LOOKUP
if (is_method("INVITE") && $hdr(X-ROUTE)=="LOOKUP"){
if(!cr_route("default", "default", "$rU", "$rU", "call_id","$avp(s:route_desc)")){
#xlog("ROUTING FAILED: no route found for $rU");
sl_send_reply("604", "Unable to route this call");
exit;
} else {
#xlog("LOOKUP FOUND: $rd $avp(s:route_desc)");
avp_pushto("$ru/username", "$avp(s:route_desc)");
sl_send_reply("302", "$rd");
exit;
}
}
}

Start Kamailio

/opt/kamailio/sbin/kamctl start
Check Kamailio is running
netstat -unlp
You should see something like this
udp 0 0 127.0.0.1:5062 0.0.0.0:* 16269/kamailio

Check your route table

Seems broken in Kamailio 1.5.0, check SQL table

/opt/kamailio/sbin/kamctl cr show
You should see a route with PEER_01
Configuration of FreeSWITCH
Dialplan configuration for FS
cd /opt/freeswitch/conf/dialplan
emacs default.xml

Put this
<?xml version="1.0" encoding="utf-8"?>
<!-- http://wiki.freeswitch.org/wiki/Dialplan_XML -->
<include>
<context name="default">
<extension name="LOOKUP_ROUTE">
<condition field="destination_number" expression="(\d+)$">
<action application="set" data="hangup_after_bridge=true"/>
<action application="set" data="continue_on_fail=true"/>
<action application="export" data="sip_h_X-ROUTE=LOOKUP"/>
<action application="bridge" data="sofia/internal/${destination_number}@127.0.0.1:5062"/>
<action application="set" data="ROUTE_GW=${sip_redirect_contact_host_0}"/>
<action application="set" data="AREA=${sip_redirect_contact_user_0}"/>
<action application="transfer" data="${destination_number} XML ROUTING"/>
</condition>
</extension>
</context>
<context name="ROUTING">

<extension name="PEER_01">
<condition field="${ROUTE_GW}" expression="PEER_01">
<action application="set" data="hangup_after_bridge=true"/>
<action application="set"
data="continue_on_fail=NORMAL_TEMPORARY_FAILURE,TIMEOUT,NO_ROUTE_DESTINATION
<action application="set" data="PEER=1.1.1.1"/>
<action application="bridge" data="sofia/external/${destination_number}@1.1.1.1"/>
<action application="set" data="PEER=2.2.2.2"/>
<action application="bridge" data="sofia/external/${destination_number}@2.2.2.2"/>
<action application="set" data="PEER=3.3.3.3"/>
<action application="bridge" data="sofia/external/${destination_number}@3.3.3.3"/>
</condition>
</extension>
</context>
</include>

In fact PEER_01 is used as an index for a combination of peers. I do this to have a failover
mechanism to a backup peer if the 1st peer is down.

You can load balance to different PEER combinations by using probability in carrierroute table. Ex:
using 2 routes pointing to PEER_01 and PEER_02 with a probability of 0.5 and you will share the
load between these 2 peers combination.
Test connectivity between FreeSWITCH and Kamailio
Test that FS and Kamailio are talking to each others
Register an IP phone using credentials in /opt/freeswitch/conf/directory/default/1001 for example
to dump traffic on the loopback interface for SIP messages between FS and Kamailio
ngrep -d lo -nn -W byline

When your phone is registered, dial 1000

You should see this in the ngrep trace
U 10.10.10.254:5060 -> 127.0.0.1:5062
INVITE sip:[email protected]:5062 SIP/2.0.
Via: SIP/2.0/UDP 10.10.10.254;rport;branch=z9hG4bKZjpQ3tQ8SN7gc.
Max-Forwards: 69.
From: "1001" <sip:[email protected]>;tag=pZ4egZ0m43Q3c.
To: <sip:[email protected]:5062>.
Call-ID: 7015d499-6c86-122c-80be-001b21009ccd.
CSeq: 110711449 INVITE.
Contact: <sip:[email protected]:5060>.
User-Agent: FreeSWITCH-mod_sofia/1.0.trunk-11592M.
Allow: INVITE, ACK, BYE, CANCEL, OPTIONS, PRACK, MESSAGE, SUBSCRIBE, NOTIFY,
REFER, UPDATE, REGISTER, INFO.
Supported: timer, precondition, path, replaces.
Allow-Events: talk, refer.
Min-SE: 120.
Content-Type: application/sdp.
Content-Disposition: session.
Content-Length: 333.
X-ROUTE: LOOKUP.
Remote-Party-ID: "1001" <sip:[email protected]>;screen=yes;privacy=off.
.
v=0.
o=FreeSWITCH 3223313286554167397 4405097820346134292 IN IP4 10.10.10.254.
s=FreeSWITCH.
c=IN IP4 10.10.10.254.
t=0 0.
m=audio 31686 RTP/AVP 9 0 8 3 101 13.
a=rtpmap:9 G722/8000.
a=rtpmap:0 PCMU/8000.
a=rtpmap:8 PCMA/8000.
a=rtpmap:3 GSM/8000.
a=rtpmap:101 telephone-event/8000.
a=fmtp:101 0-16.
a=rtpmap:13 CN/8000.
a=ptime:20.
#
U 127.0.0.1:5062 -> 10.10.10.254:5060
SIP/2.0 302 PEER_01.
Via: SIP/2.0/UDP 10.10.10.254;rport=5060;branch=z9hG4bKZjpQ3tQ8SN7gc.
From: "1001" <sip:[email protected]>;tag=pZ4egZ0m43Q3c.
To: <sip:[email protected]:5062>;tag=458fb4012080e656b6742c09466dabcd.1740.
Call-ID: 7015d499-6c86-122c-80be-001b21009ccd.
CSeq: 110711449 INVITE.
Contact: sip:1000@PEER_01.
Server: Kamailio (1.4.3-notls (x86_64/linux)).
Content-Length: 0.
.
#
U 10.10.10.254:5060 -> 127.0.0.1:5062
ACK sip:[email protected]:5062 SIP/2.0.
Via: SIP/2.0/UDP 10.10.10.254;rport;branch=z9hG4bKZjpQ3tQ8SN7gc.
Max-Forwards: 69.
From: "1001" <sip:[email protected]>;tag=pZ4egZ0m43Q3c.
To: <sip:[email protected]:5062>;tag=458fb4012080e656b6742c09466dabcd.1740.
Call-ID: 7015d499-6c86-122c-80be-001b21009ccd.
CSeq: 110711449 ACK.
Content-Length: 0.
.

Optimizations
Stop FreeSWITCH
/opt/freeswitch/bin/freeswitch -stop

Lowering FS Log Level

emacs /opt/freeswitch/conf/autoload_configs/switch.conf.xml
<!-- Default Global Log Level - value is one of debug,info,notice,warning,err,crit,alert -->
<param name="loglevel" value="err"/>

Set Call Admission Control

emacs /opt/freeswitch/conf/autoload_configs/switch.conf.xml
<!--Most channels to allow at once -->
<param name="max-sessions" value="1000"/>
<!--Most channels to create per second -->
<param name="sessions-per-second" value="100"/>

With these settings, we have defined a maximum of 500 bridged calls (2*500 channels) and a
maximum of 100 new sessions per second. You should set these parameters according to your
hardware

Creation of a ramdisk for the FS database

mv /opt/freeswitch/db /opt/freeswitch/db_old
mkdir /opt/freeswitch/db
mount -t tmpfs tmpfs /opt/freeswitch/db

Use Proxy Media to keep FS in the RTP media path (topology hiding) but without analyzing RTP or
DTMF
emacs /opt/freeswitch/conf/sip_profiles/internal.xml
<param name="inbound-late-negotiation" value="true"/>

Add directive proxy media in dialplan

<extension name="PEER_01">
<condition field="${sip_h_X-ROUTE}" expression="PEER_01">
<action application="set" data="proxy_media=true"/>
<action application="set" data="hangup_after_bridge=true"/>
<action application="set"
data="continue_on_fail=NORMAL_TEMPORARY_FAILURE,TIMEOUT,NO_ROUTE_DESTINATION
<action application="set" data="PEER=1.1.1.1"/>
<action application="bridge" data="sofia/external/${destination_number}@1.1.1.1"/>
<action application="set" data="PEER=2.2.2.2"/>
<action application="bridge" data="sofia/external/${destination_number}@2.2.2.2"/>
<action application="set" data="PEER=3.3.3.3"/>
<action application="bridge" data="sofia/external/${destination_number}@3.3.3.3"/>
</condition>
</extension>
Disable Presence Support in SIP_Profiles
emacs /opt/freeswitch/conf/sip_profiles/internal.xml
<param name="manage-presence" value="false"/>

emacs /opt/freeswitch/conf/sip_profiles/external.xml
<param name="manage-presence" value="false"/>

Check that your Network Interfaces Cards (NIC) are not sharing IRQ

cat /proc/interrupts
ideally, your ethernet cards should be put on different IRQ, and if you have more than one CPU,
you can force affinity of a NIC with a specific CPU (I will add a note later on this)
Check CPU usage
Install some useful utils for checking CPU consumptions
apt-get install sysstat
You can check the load of your CPUs using
mpstat -P ALL 1
13:13:13 CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
13:13:14 all 0,00 0,00 0,00 0,00 0,00 0,00 0,00 100,00 162,00
13:13:14 0 0,00 0,00 0,00 0,00 0,00 0,00 0,00 100,00 162,00

The %idle represents the idle percentage of the CPU, higher is better as it means that you have
free CPU ressources

vmstat 1
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
1 0 80 234312 93496 1273928 0 0 1 32 55 65 2 3 95 0
0 0 80 234296 93496 1273928 0 0 0 0 106 1299 0 0 100 0

The last 2 columns are interesting:

id: idle, same as before for mpstat, the higher is better

wa: waiting IO, when this number is high it means that your CPU is waiting for IO (ex: HDD with too
much access) and it will have a negative impact on performance

top
I think that I do not have to introduce this tool