Using mod_xml_curl In FreeSWITCH - How Hard Can It Be? (Part 1)

I've been playing with FreeSWITCH for more than three years. In all that time I've never really had a good reason to use mod_xml_curl. Until now, that is. The good reason is nothing more than my own edification. I like to answer questions on the mailing list and finally I got sick of telling people to use xml_curl without having used it my self.

MOD_XML_CURL: The Magic Module For Dynamic FreeSWITCH Configuration
Okay, here's a quick overview of mod_xml_curl: it's a module in FreeSWITCH that allows parts of the FreeSWITCH configuration to be fetched from a web server that can, in turn, grab the configuration information from a database (or anywhere else for that matter). The primary ways to use xml_curl are for the dialplan and the user directory, but you can also use it for configuration options and for phrase macros. The specific question on the mailing list was about pulling user information from a database, so that's where I decided to start.

NOW WHAT?
I've read about it, I've heard people talk about it, I know lots of people who use it, but I've never done it myself, so the first question I had to answer is this: what do I do first? Well, the specific question was about pulling user information from a database, so I decided the first thing to do would be to look at the sample XML files in the default configuration to get an idea of what kind of XML config stuff I would need to generate and send back to FreeSWITCH. I also poked around in the contrib folder to see what others had done. Finally, I consulted the oft-maligned but generally underappreciated FreeSWITCH wiki. Amazingly, with just these meager resources I was able to get a POC working in just a few hours. Now mind you, I did this in my spare time while avoiding my wife and kids, and with limited resources. (I don't have a stack of VoIP phones on my home LAN like some of you people do.)

BTW, I'm not a Web-programming guru, nor am I a database admin. I'm familiar with these from having tinkered with some LAMP setups. I'm decent with Linux, decent with Apache, decent with MySQL, and somewhat advanced with Perl. Small problem: I don't have MySQL on my laptop which is where I needed to run everything. However, I had installed PostgreSQL version 8.3 for another project. I know almost nothing about PG. Fortunately, I had Mr. William King (quentusrex on IRC - #freeswitch on irc.freenode.net) around and he gave me a few pointers. ("In MySQL I did it this way, how do I do that in PG?") A tip o' the cap to him. So at this point I was all set.

My configuration at the start looked like this:
MacBook Pro with OS X Leopard
Latest SVN of FreeSWITCH
Apache 2.2
Perl 5.8.8
PostgreSQL 8.3

So the first thing I decided to do was create a new database in PG, then create a users table. With William's help and the handy pgadmin3 program I was in business. For kicks, I went back to my table and added an extra column: context. I didn't need this column but I thought it would be a nice little exercise to add this value for each user even though I would usually just want "default" for the context. My table ended up looking like this:

CREATE TABLE users (
uid bigserial NOT NULL,
lastname character varying(15),
firstname character varying(15),
username character varying(11) NOT NULL,
mailbox character varying(11) NOT NULL,
"password" character varying(11) NOT NULL,
context character varying(11) DEFAULT 'default'::character varying,
CONSTRAINT users_pk PRIMARY KEY (uid)
)

Here's a nice little pic of the data I put in there thanks to the "edit data" widget in pgadmin3:

I put first and last names in separate fields because I figured that would be common in a real world scenario. I have the username field which is what I'm kinda using for everything: userid, auth. username and username. (Yeah, I'm cheating, get over it.) I have also a password and a context field. That's it. At this point I was feeling confident. I had the database, a users table, a login role for accessing the database, and a little momentum on my side.

Now I needed to decide which task to handle next: handling the CGI call or figuring out how to query the database. I decided to tackle what I felt was the tougher of the two: handling the xml_curl call and returning information that FreeSWITCH could use. It turns out that it wasn't actually all that tough: I opened up conf/autoload_configs/xml_curl.conf.xml and found this near the top:

<!-- The url to a gateway cgi that can generate xml similar to what's in this file only on-the-fly (leave it commented if you dont need it) -->
<!-- one or more |-delim of configuration|directory|dialplan -->
<!-- <param name="gateway-url" value="http://www.freeswitch.org/gateway.xml" bindings="dialplan"/> -->

I'm no genius but to me that's pretty clear information: I needed a 'binding' for directory. For kicks I left in the 'directory' binding as well. I ended up with this:

<param name="gateway-url" value="http://127.0.0.1/cgi-bin/xmlcurl_sample.pl" bindings="directory|dialplan"/>

I saved everything, started up FS and promptly realized that I didn't actually build mod_xml_curl. No worries. A quick trip to the FS source directory did the trick:

cd /usr/src/freeswitch.trunk
make mod_xml_curl-install

Gotta love FreeSWITCH - way easy. (I added mod_xml_curl to modules.conf and modules.conf.xml so that it would auto build and auto load, respectively.) I loaded up mod_xml_curl and saw no errors or warnings. I saw that it added a new app named "xml_curl" so I looked at the syntax. (Just type 'xml_curl help' to see the syntax.) Basically it just allows you to turn on debug for xml_curl, which dumps a copy of the information returned from the webserver when xml_curl makes a call. The FS CLI shows the file name and path when it creates this debug file.

The next order of business was to observe what my CGI script would see in terms of information coming from xml_curl. Ah Perl, my lovely Swiss-army chainsaw. That was easy:

#!/usr/bin/perl
#
# xmlcurl_sample.pl
#
use strict;
use warnings;
use Data::Dump qw(dump); # I prefer this to Data::Dumper
use CGI.

my $q = CGI->new;
open(FILEOUT,'>','/tmp/cgidump.txt'); ## look in this file to see what's up
print FILEOUT dump($q);
close(FILEOUT);

But where to put this lovely file? I poked around to find my default Apache configs and it turns out that there's an alias for "cgi-bin" so that's where I put my Perl script to handle the CGI call. That's it. Now I can see what's coming in. All I need to do is make a phone call or have a user try to register to FS. (Recall that I set up an xml_curl binding for both 'directory' and 'dialplan'.) Dialing out was easier since I didn't have a phone handy. Naturally since my script didn't return anything useful there were errors on the FS CLI, but I was more interested in what was in my cgidump.txt file:

bless({
".charset" => "ISO-8859-1",
".fieldnames" => {},
".parameters" => [
"hostname",
"section",
"tag_name",
"key_name",
"key_value",
"Event-Name",
"Core-UUID",
"FreeSWITCH-Hostname",
"FreeSWITCH-IPv4",
"FreeSWITCH-IPv6",
"Event-Date-Local",
"Event-Date-GMT",
"Event-Date-Timestamp",
"Event-Calling-File",
"Event-Calling-Function",
"Event-Calling-Line-Number",
"Channel-State",
"Channel-State-Number",
"Channel-Name",
"Unique-ID",
"Call-Direction",
"Presence-Call-Direction",
"Answer-State",
"Channel-Read-Codec-Name",
"Channel-Read-Codec-Rate",
"Channel-Write-Codec-Name",
"Channel-Write-Codec-Rate",
"Caller-Dialplan",
"Caller-Caller-ID-Name",
"Caller-Caller-ID-Number",
"Caller-Network-Addr",
"Caller-Destination-Number",
"Caller-Unique-ID",
"Caller-Source",
"Caller-Context",
"Caller-Channel-Name",
"Caller-Screen-Bit",
"Caller-Privacy-Hide-Name",
"Caller-Privacy-Hide-Number",
"variable_channel_name",
"variable_endpoint_disposition",
"variable_read_codec",
"variable_read_rate",
"variable_write_codec",
"variable_write_rate",
"variable_zrtp_secure_media",
"variable_playback_seconds",
"variable_playback_ms",
"variable_playback_samples",
"variable_current_application_data",
"variable_current_application",
"Hunt-Dialplan",
"Hunt-Caller-ID-Name",
"Hunt-Caller-ID-Number",
"Hunt-Network-Addr",
"Hunt-Destination-Number",
"Hunt-Unique-ID",
"Hunt-Source",
"Hunt-Context",
"Hunt-Channel-Name",
"Hunt-Screen-Bit",
"Hunt-Privacy-Hide-Name",
"Hunt-Privacy-Hide-Number",
],
"Answer-State" => ["answered"],
"Call-Direction" => ["inbound"],
"Caller-Caller-ID-Name" => ["FreeSWITCH"],
"Caller-Caller-ID-Number" => ["0000000000"],
"Caller-Channel-Name" => ["portaudio/9999"],
"Caller-Context" => ["features"],
"Caller-Destination-Number" => ["is_zrtp_secure"],
"Caller-Dialplan" => ["XML"],
"Caller-Network-Addr" => ["192.168.1.110"],
"Caller-Privacy-Hide-Name" => ["false"],
"Caller-Privacy-Hide-Number" => ["false"],
"Caller-Screen-Bit" => ["true"],
"Caller-Source" => ["mod_portaudio"],
"Caller-Unique-ID" => ["718c69cd-48cf-4701-90b1-ab2e471cfb6d"],
"Channel-Name" => ["portaudio/9999"],
"Channel-Read-Codec-Name" => ["L16"],
"Channel-Read-Codec-Rate" => [8000],
"Channel-State" => ["CS_EXECUTE"],
"Channel-State-Number" => [4],
"Channel-Write-Codec-Name" => ["L16"],
"Channel-Write-Codec-Rate" => [8000],
"Core-UUID" => ["9b4e5fe9-9bc9-4bed-915a-3c277cbd760b"],
"Event-Calling-File" => ["mod_dialplan_xml.c"],
"Event-Calling-Function" => ["dialplan_xml_locate"],
"Event-Calling-Line-Number" => [368],
"Event-Date-GMT" => ["Thu, 29 Oct 2009 05:18:05 GMT"],
"Event-Date-Local" => ["2009-10-28 22:18:05"],
"Event-Date-Timestamp" => ["1256793485008872"],
"Event-Name" => ["REQUEST_PARAMS"],
"FreeSWITCH-Hostname" => ["michael-collinss-macbook-pro.local"],
"FreeSWITCH-IPv4" => ["10.15.0.136"],
"FreeSWITCH-IPv6" => ["::1"],
"Hunt-Caller-ID-Name" => ["FreeSWITCH"],
"Hunt-Caller-ID-Number" => ["0000000000"],
"Hunt-Channel-Name" => ["portaudio/9999"],
"Hunt-Context" => ["features"],
"Hunt-Destination-Number" => ["is_zrtp_secure"],
"Hunt-Dialplan" => ["XML"],
"Hunt-Network-Addr" => ["192.168.1.110"],
"Hunt-Privacy-Hide-Name" => ["false"],
"Hunt-Privacy-Hide-Number" => ["false"],
"Hunt-Screen-Bit" => ["true"],
"Hunt-Source" => ["mod_portaudio"],
"Hunt-Unique-ID" => ["718c69cd-48cf-4701-90b1-ab2e471cfb6d"],
"Presence-Call-Direction" => ["inbound"],
"Unique-ID" => ["718c69cd-48cf-4701-90b1-ab2e471cfb6d"],
escape => 1,
hostname => ["michael-collinss-macbook-pro.local"],
key_name => [""],
key_value => [""],
section => ["dialplan"],
tag_name => [""],
"variable_channel_name" => ["portaudio/9999"],
"variable_current_application" => ["execute_extension"],
"variable_current_application_data" => ["is_zrtp_secure XML features"],
"variable_endpoint_disposition" => ["ANSWER"],
variable_playback_ms => [0],
"variable_playback_samples" => [0],
"variable_playback_seconds" => [0],
variable_read_codec => ["L16"],
variable_read_rate => [8000],
variable_write_codec => ["L16"],
variable_write_rate => [8000],
"variable_zrtp_secure_media" => ["true"],
}, "CGI")


Looking closely you can probably guess that I just used the pa (portaudio) command to dial 9999. I was in business! That's all for part 1 of this subject.

To recap: I thought I would try setting up a proof-of-concept script for using mod_xml_curl in FreeSWITCH. I already had Apache, Perl, and PostgreSQL installed on my laptop, and the latest FreeSWITCH SVN. I configured xml_curl.conf.xml to call my CGI script: xmlcurl_sample.pl. I did a very simple data dump of the params sent into the script from xml_curl. This allowed me to verify that communications from xml_curl to my CGI script were functional.

In part 2 I'll talk about sending a response back to the server and setting up the DBI to query a database.