1. SER Developer's Guide

Jan Janak

Jiri Kuthan

Revision History
Revision $Revision$ $Date$

Abstract

The document describes the SIP Express Router internals and algorithms. It describes overall server architecture, request processing, configuration, memory management, interprocess locking, module interface and selected modules in detail.

The document is intended mainly for module developers wishing to implement a new module for the server. Other people like developers of SIP related software or students might be interested too.


1.1. The Server Startup
1.1.1. Installation Of New Signal Handlers
1.1.2. Processing Command Line Parameters
1.1.3. Parser Initialization
1.1.4. Malloc Initialization
1.1.5. Timer Initialization
1.1.6. FIFO Initialization
1.1.7. Built-in Module Initialization
1.1.8. Server Configuration
1.1.8.1. Lexical Analysis
1.1.8.2. Syntactical Analysis
1.1.8.3. Config File Structure
1.1.9. Interface Configuration
1.1.10. Turning into a Daemon
1.1.11. Module Initialization
1.1.12. Routing List Fixing
1.1.13. Statistics Initialization
1.1.14. Socket Initialization
1.1.15. Forking
1.1.15.1. dont_fork variable is set (not zero)
1.1.15.2. dont_fork is not set (zero)
1.2. Main Loop
1.2.1. receive_msg Function
1.3. The Server Showdown
1.4. Data Structures
1.4.1. Type str
1.4.2. Structure hdr_field
1.4.3. Structure sip_uri
1.4.4. Structure via_body
1.4.5. Structure ip_addr
1.4.6. Structure lump
1.4.7. Structure lump_rpl
1.4.8. Structure msg_start
1.4.9. Structure sip_msg
1.4.10. Structure to_body
1.4.11. Structure cseq_body
1.5. The Routing Engine
1.5.1. do_action Function
1.6. The SIP Message Parser
1.6.1. Structure of a SIP Message
1.6.2. Parser Structure
1.6.2.1. The First Line Parser
1.6.2.2. The Header Field Name Parser
1.6.2.3. To Header Field Parser
1.6.2.4. From Header Field Parser
1.6.2.5. CSeq Header Field Parser
1.6.2.6. Event Body Parser
1.6.2.7. Expires HF Body Parser
1.6.2.8. Via HF Body Parser
1.6.2.9. Contact Header Field Parser
1.6.2.10. Digest Body Parser
1.7. Module Interface
1.7.1. Shared Objects
1.7.2. Exporting Functions
1.7.3. Exporting Parameters
1.7.4. Module Initialization
1.7.5. Module Clean-up
1.7.6. Module Callbacks
1.7.7. exports Structure - Assembling the Pieces Together
1.7.8. Example - Simple Module Interface
1.7.9. Module Interface Internals
1.7.9.1. Structure sr_module
1.7.9.2. Structure module_exports
1.7.9.3. Module Loading
1.7.9.4. Module Configuration
1.7.9.5. Looking Up an Exported Function
1.7.9.6. Additional Functions
1.8. The Database Interface
1.8.1. Data types
1.8.1.1. Type db_con_t
1.8.1.2. Type db_key_t
1.8.1.3. Type db_type_t
1.8.1.4. Type db_val_t
1.8.1.5. Type db_row_t
1.8.1.6. Type db_res_t
1.8.2. Functions
1.8.2.1. bind_dbmod
1.8.2.2. db_init
1.8.2.3. db_close
1.8.2.4. db_query
1.8.2.5. db_free_result
1.8.2.6. db_insert
1.8.2.7. db_delete
1.8.2.8. db_update
1.8.2.9. db_use_table
1.9. Locking Interface
1.9.1. Why use it ?
1.9.2. How to use it ?
1.9.3. Simple Locks
1.9.3.1. Allocation And Initialization
1.9.3.2. Destroying And Deallocating the Locks
1.9.3.3. Locking And Unlocking
1.9.3.4. Lock Sets
1.10.
1.10.1. Extend select framework with module related functions
1.10.2. Define your working functions
1.10.3. Create module's select parsing table
1.10.4. Register the created table
1.10.5. Example - module defining select extension

1.1. The Server Startup

Revision History
Revision $Revision$ $Date$

The main function in file main.c is the first function called upon server startup. It's purpose is to initialize the server and enter main loop. The server initialization will be described in the following sections.

Particular initialization steps are described in order in which they appear in main function.

1.1.1. Installation Of New Signal Handlers

The first step in the initialization process is the installation of new signal handlers. We need our own signal handlers to be able to do graceful shutdown, print server statistics and so on. There is only one signal handler function which is function sig_usr in file main.c.

The following signals are handled by the function: SIGINT, SIGPIPE, SIGUSR1, SIGCHLD, SIGTERM, SIGHUP and SIGUSR2.

1.1.2. Processing Command Line Parameters

SER utilizes the getoptfunction to parse command line parameters. The function is extensively described in the man pages.

1.1.3. Parser Initialization

SER contains a fast 32-bit parser. The parser uses pre-calculated hash table that needs to be filled in upon startup. The initialization is done here, there are two functions that do the job. Function init_hfname_parser initializes hash table in header field name parser and function init_digest_parser initializes hash table in digest authentication parser. The parser's internals will be described later.

1.1.4. Malloc Initialization

To make SER even faster we decided to re-implement memory allocation routines. The new malloc better fits our needs and speeds up the server a lot. The memory management subsystem needs to be initialized upon server startup. The initialization mainly creates internal data structures and allocates memory region to be partitioned.

Important

The memory allocation code must be initialized BEFORE any of its function is called !

1.1.5. Timer Initialization

Various subsystems of the server must be called periodically regardless of the incoming requests. That's what timer is for. Function init_timer initializes the timer subsystem. The function is called from main.c and can be found in timer.c The timer subsystem will be described later.

Warning

Timer subsystem must be initialized before config file is parsed !

1.1.6. FIFO Initialization

SER has built-in support for FIFO control. It means that the running server can accept commands over a FIFO special file (a named pipe). Function register_core_fifo initializes FIFO subsystem and registers basic commands, that are processed by the core itself. The function can be found in file fifo_server.c.

The FIFO server will be described in another chapter.

1.1.7. Built-in Module Initialization

Modules can be either loaded dynamically at runtime or compiled in statically. When a module is loaded at runtime, it is registered [1] immediately with the core. When the module is compiled in statically, the registration[1] must be performed during the server startup. Function register_builtin_modules does the job.

1.1.8. Server Configuration

The server is configured through a configuration file. The configuration file is C-Shell like script which defines how incoming requests should be processed. The file cannot be interpreted directly because that would be very slow. Instead of that the file is translated into an internal binary representation. The process is called compilation and will be described in the following sections.

Note

The following sections only describe how the internal binary representation is being constructed from the config file. The way how the binary representation is used upon a request arrival will be described later.

The compilation can be divided in several steps:

1.1.8.1. Lexical Analysis

Lexical analysis is process of converting the input (the configuration file in this case) into a stream of tokens. A token is a set of characters that 'belong' together. A program that can turn the input into stream of tokens is called scanner. For example, when scanner encounters a number in the config file, it will produce token NUMBER.

There is no need to implement the scanner from scratch, it can be done automatically. There is a utility called flex. Flex accepts a configuration file and generates scanner according to the configuration file. The configuration file for flex consists of several lines - each line describing one token. The tokens are described using regular expressions. For more details, see flex manual page or info documentation.

Flex input file for the SER config file is in file cfg.lex. The file is processed by flex when the server is being compiled and the result is written in file lex.yy.c. The output file contains the scanner implemented in the C language.

1.1.8.2. Syntactical Analysis

The second stage of configuration file processing is called syntactical analysis. Purpose of syntactical analysis is to check if the configuration file has been well formed, doesn't contain syntactical errors and perform various actions at various stages of the analysis. Program performing syntactical analysis is called parser.

Structure of the configuration file is described using grammar. Grammar is a set of rules describing valid 'order' or 'combination' of tokens. If the file isn't conformable with it's grammar, it is syntactically invalid and cannot be further processed. In that case an error will be issued and the server will be aborted.

There is a utility called yacc. Input of the utility is a file containing the grammar of the configuration file, in addition to the grammar, you can describe what action the parser should do at various stages of parsing. For example, you can instruct the parser to create a structure describing an IP address every time it finds an IP address in the configuration file and convert the address to its binary representation.

For more information see yacc documentation.

yacc creates the parser when the server is being compiled from the sources. Input file for yacc is cfg.y. The file contains grammar of the config file along with actions that create the binary representation of the file. Yacc will write its result into file cfg.tab.c. The file contains function yyparse which will parse the whole configuration file and construct the binary representation. For more information about the bison input file syntax see bison documentation.

1.1.8.3. Config File Structure

The configuration file consist of three sections, each of the sections will be described separately.

  • Route Statement - The statement describes how incoming requests will be processed. When a request is received, commands in one or more "route" sections will be executed step by step. The config file must always contain one main "route" statement and may contain several additional "route" statements. Request processing always starts at the beginning of the main "route" statement. Additional "route" statements can be called from the main one or another additional "route" statements (It it similar to function calling).

  • Assign Statement - There are many configuration variables across the server and this statement makes it possible to change their value. Generally it is a list of assignments, each assignment on a separate line.

  • Module Statement - Additional functionality of the server is available through separate modules. Each module is a shared object that can be loaded at runtime. Modules can export functions, that can be called from the configuration file and variables, that can be configured from the config file. The module statement makes it possible to load modules and configure them. There are two commands in the statement - loadmodule and modparam. The first can load a module. The second one can configure module's internal variables.

In the following sections we will describe in detail how the three sections are being processed upon server startup.

1.1.8.3.1. Route Statement

The following grammar snippet describes how the route statement is constructed

route_stm = "route" "{" actions "}"  
{ 
    $$ = push($3, &rlist[DEFAULT_RT]); 
}
		   
actions = actions action { $$ = append_action($1, $2}; }
| action { $$ = $1; }

action = cmd SEMICOLON { $$ = $1; }
| SEMICOLON { $$ = 0; }

cmd = "forward" "(" host ")" { $$ = mk_action(FORWARD_T, STRING_ST, NUMBER_ST, $3, 0)
| ...

A config file can contain one or more "route" statements. "route" statement without number will be executed first and is called the main route statement. There can be additional route statements identified by number, these additional route statements can be called from the main route statement or another additional route statements.

Each route statement consists of a set of actions. Actions in the route statement are executed step by step in the same order in which they appear in the config file. Actions in the route statement are delimited by semicolon.

Each action consists of one and only one command (cmd in the grammar). There are many types of commands defined. We don't list all of them here because the list would be too long and all the commands are processed in the same way. Therefore we show only one example (forward) and interested readers might look in cfg.y file for full list of available commands.

Each rule in the grammar contains a section enclosed in curly braces. The section is the C code snippet that will be executed every time the parser recognizes that rule in the config file.

For example, when the parser finds forward command, mk_action function (as specified in the grammar snippet above) will be called. The function creates a new structure with type field set to FORWARD_T representing the command. Pointer to the structure will be returned as the return value of the rule.

The pointer propagates through action rule to actions rule. Actions rule will create linked list of all commands. The linked list will be then inserted into rlist table. (Function push in rule route_stm). Each element of the table represents one "route" statement of the config file.

Each route statement of the configuration file will be represented by a linked list of all actions in the statement. Pointers to all the lists will be stored in rlist array. Additional route statements are identified by number. The number also serves as index to the array.

When the core is about to execute route statement with number n, it will look in the array at position n. If the element at position n is not null then there is a linked list of commands and the commands will be executed step by step.

Reply-Route statement is compiled in the same way. Main differences are:

  • Reply-Route statement is executed when a SIP REPLY comes (not ,SIP REQUEST).

  • Only subset of commands is allowed in the reply-route statement. (See file cfg.y for more details).

  • Reply-route statement has it's own array of linked-lists.

1.1.8.3.2. Assign Statement

The server contains many configuration variables. There is a section of the config file in which the variables can be assigned new value. The section is called The Assign Statement. The following grammar snippet describes how the section is constructed (only one example will be shown):

assign_stm = "children" '=' NUMBER { children_no=$3; }
| "children" '=' error  { yyerror("number expected"); } 
...

The number in the config file is assigned to children_no variable. The second statement will be executed if the parameter is not number or is in invalid format and will issue an error and abort the server.

1.1.8.3.3. Module Statement

The module statement allows module loading and configuration. There are two commands:

  • loadmodule - Load the specified module in form of a shared object. The shared object will be loaded using dlopen.

  • modparam - It is possible to configure a module using this command. The command accepts 3 parameters: module name, variable name and variable value.

The following grammar snippet describes the module statement:

module_stm = "loadmodule" STRING 
{ 
    DBG("loading module %s\n", $2);
    if (load_module($2)!=0) {
        yyerror("failed to load module");
    }
}

| "loadmodule" error	 { yyerror("string expected");  }
| "modparam" "(" STRING "," STRING "," STRING ")" 
{
    if (set_mod_param($3, $5, PARAM_STR|PARAM_STRING, $7) != 0) {
        yyerror("Can't set module parameter");
    }
}

| "modparam" "(" STRING "," STRING "," NUMBER ")" 
{
    if (set_mod_param($3, $5, PARAM_INT, (void*)$7) != 0) {
        yyerror("Can't set module parameter");
    }
}

| MODPARAM error { yyerror("Invalid arguments"); }

When the parser finds loadmodule command, it will execute statement in curly braces. The statement will call load_module function. The function will load the specified filename using dlopen. If dlopen was successful, the server will look for exports structure describing the module's interface and register the module. For more details see module section.

If the parser finds modparam command, it will try to configure the specified variable in the specified module. The module must be loaded using loadmodule before modparam for the module can be used ! Function set_mod_param will be called and will configure the variable in the specified module.

1.1.9. Interface Configuration

The server will try to obtain list of all configured interfaces of the host it is running on. If it fails the server tries to convert hostname to IP address and will use interface with the IP address only.

Function add_interfaces will add all configured interfaces to the array.

Try to convert all interface names to IP addresses, remove duplicates...

1.1.10. Turning into a Daemon

When configured so, SER becomes a daemon during startup. A process is called daemon when it hasn't associated controlling terminal. See function daemonize in file main.c for more details. The function does the following:

  • chroot is performed if necessary. That ensures that the server will have access to a particular directory and its subdirectories only.

  • Server's working directory is changed if the new working directory was specified (usually it is /).

  • If command line parameter -g was used, the server's group ID is changed to that value.

  • If command line parameter -u was used, the server's user ID is changed to that value.

  • Perform fork, let the parent process exit. This ensures that we are not a group leader.

  • Perform setsid to become a session leader and drop the controlling terminal.

  • Fork again to drop group leadership.

  • Create a pid file.

  • Close all opened file descriptors.

1.1.11. Module Initialization

The whole config file was parsed, all modules were loaded already and can be initialized now. A module can tell the core that it needs to be initialized by exporting mod_init function. mod_init function of all loaded modules will be called now.

1.1.12. Routing List Fixing

After the whole routing list was parsed, there might be still places that can be further processed to speed-up the server. For example, several commands accept regular expression as one of their parameters. The regular expression can be compiled too and processing of compiled expression will be much faster.

Another example might be string as parameter of a function. For example if you call append_hf("Server: SIP Express Router\r\n") from the routing script, the function will append a new header field after the last one. In this case, the function needs to know length of the string parameter. It could call strlen every time it is called, but that is not a very good idea because strlen would be called every time a message is processed and that is not necessary.

Instead of that the length of the string parameter could be pre-calculated upon server startup, saved and reused later. The processing of the request will be faster because append_hf doesn't need to call strlen every time, I can just reuse the saved value.

This can be used also for string to int conversions, hostname lookups, expression evaluation and so on.

This process is called Routing List Fixing and will be done as one of last steps of the server startup.

Every loaded module can export one or more functions. Each such function can have associated a fixup function, which should do fixing as described in this section. All such fixups of all loaded modules will be called here. That makes it possible for module functions to fix their parameters too if necessary.

1.1.13. Statistics Initialization

If compiled-in, the core can produce some statistics about itself and traffic processed. The statistics subsystem gets initialized here, see function init_stats.

1.1.14. Socket Initialization

UDP socket initialization depends on dont_fork variable. If this variable is set (only one process will be processing incoming requests) and there are multiple listen interfaces, only the first one will be used. This mode is mainly for debugging.

If the variable is not set, then sockets for all configured interfaces will be created and initialized. See function udp_init in file udp_server.c for more details.

1.1.15. Forking

The rest of the initialization process depends on value of dont_fork variable. dont_fork is a global variable defined in main.c. We will describe both variants separately.

1.1.15.1. dont_fork variable is set (not zero)

If dont_fork variable is set, the server will be operating in special mode. There will be only one process processing incoming requests. This is very slow and was intended mainly for debugging purposes. The main process will be processing all incoming requests itself.

The server still needs additional children:

  • One child is for the timer subsystem, the child will be processing timers independently of the main process.

  • FIFO server will spawn another child if enabled. The child will be processing all commands coming through the fifo interface.

  • If SNMP support was enabled, another child will be created.

The following initialization will be performed in dont fork mode. (look into function main_loop in file main.c.

  • Another child will be forked for the timer subsystem.

  • Initialize the FIFO server if enabled, this will fork another child. For more info about the FIFO server, see section The FIFO server.

  • Call init_child(0). The function performs per-child specific initialization of all loaded modules. A module can be initialized though mod_init function. The function is called BEFORE the server forks and thus is common for all children.

    If there is anything, that needs to be initialized in every child separately (for example if each child needs to open its own file descriptor), it cannot be done in mod_init. To make such initialization possible, a module can export another initialization function called init_child. The function will be called in all children AFTER fork of the server.

    And since we are in "dont fork" mode and there will no children processing requests (remember the main process will be processing all requests), the init_child wouldn't be called.

    That would be bad, because child_init might do some initialization that must be done otherwise modules might not work properly.

    To make sure that module initialization is complete we will call init_child here for the main process even if we are not going to fork.

That's it. Everything has been initialized properly and as the last step we will call udp_rcv_loop which is the main loop function. The function will be described later.

1.1.15.2. dont_fork is not set (zero)

dont_fork is not set. That means that the server will fork children and the children will be processing incoming requests. How many children will be created depends on the configuration (children variable). The main process will be sleeping and handling signals only.

The main process will then initialize the FIFO server. The FIFO server needs another child to handle communication over FIFO and thus another child will be created. The FIFO server will be described in more detail later.

Then the main process will perform another fork for the timer attendant. The child will take care of timer lists and execute specified function when a timer hits.

The main process is now completely initialized, it will sleep in pause function until a signal comes and call handle_sigs when such condition occurs.

The following initialization will be performed by each child separately:

Each child executes init_child function. The function will sequentially call child_init functions of all loaded modules.

Because the function is called in each child separately, it can initialize per-child specific data. For example if a module needs to communicate with database, it must open a database connection. If the connection would be opened in mod_init function, all the children would share the same connection and locking would be necessary to avoid conflicts. On the other hand if the connection was opened in child_init function, each child will have its own connection and concurrency conflicts will be handled by the database server.

And last, but not least, each child executes udp_rcv_loop function which contains the main loop logic.

1.2. Main Loop

Revision History
Revision $Revision$ $Date$

Upon startup, all children execute recvfrom function. The process will enter the kernel mode. When there is no data to be processed at the moment, the kernel will put the process on list of processes waiting for data and the process will be put asleep.

When data to be processed was received, the first process on the list will be removed from the list and woken up. After the process finished processing of the data, it will call recvfrom again and will be put by the kernel at the end of the list.

When next data arrives, the first process on the list will be removed, processes the data and will be put on the end of the list again. And so on...

The main loop logic can be found in function udp_rcv_loop in file udp_server.c.

The message is received using recvfrom function. The received data is stored in buffer and zero terminated.

If configured so, basic sanity checks over the received message will be performed.

The message is then processed by receive_msg function and recvfrom is called again.

1.2.1. receive_msg Function

The function can be found in receive.c file.

  • In the server, a request or response is represented by sip_msg structure. The structure is allocated in this function. The original message is stored in buf attribute of the structure and is zero terminated. Then, another copy of the received message will be created and the copy will be stored in orig field. The original copy will be not modified during the server operation. All changes will be made to the copy in buf field. The second copy of the message will be removed in the future.

  • The message will be parsed (function parse_msg). We don't need the whole message header to be parsed at this stage. Only the first line and first Via header need to be parsed. The server needs to know if the message is request or response - hence the first line. The server also needs the first Via to be able to add its own Via - hence the first Via. Nothing else will be parsed at the moment - this saves time. (Message parser as well as sip_msg structure will be described later).

  • A module may register callbacks. Each callback have associated an event, that will trigger the callback. One such callback is pre-script callback. Such callback will be called immediately before the routing part of the config file will be executed. If there are such callbacks registered, they will be executed now.

  • As the next step we will determine type of the message. If the message being processed is a REQUEST then basic sanity checks will be performed (make sure that there is the first Via and parsing was successful) and the message will be passed to routing engine. The routing engine is one of the most complicated parts of the server and will be in detail described in chapter The Routing Engine.

  • If the message is a RESPONSE, it will be simply forwarded to its destination.

  • After all, post-script callbacks will be executed if any and the structure representing the message will be released.

  • Processing of the message is done now and the process is ready for another SIP message.

1.3. The Server Showdown

Revision History
Revision $Revision$ $Date$

The server shutdown can be triggered by sending a signal to the server. The server will behave differently upon receiving various types of signals, here is a brief summary:

  • SIGINT, SIGPIPE, SIGTERM, SIGCHLD will terminate the server.

  • SIGUSR1 will print statistics and let the server continue.

  • SIGHUP, SIGUSR2 will be ignored.

There is only one common signal handler for all signals - function sig_usr in file main.c.

In normal mode of operation (dont_fork variable is not set), the main server is not processing any requests, it calls pause function and will be waiting for signals only. What happens when a signal arrives is shown in the previous paragraph.

When in normal mode (dont_fork is not set), the signal handler of the main process will only store number of the signal received. All the processing logic will be executed by the main process outside the signal handler (function handle_sigs) The function will be called immediately after the signal handler finish. The main process usually does some cleanup and running such things outside the signal handler is much more safe than doing it from the handler itself. Children only print statistics and exit or ignore the signal completely, that is quite safe and can be done directly from the signal handler of children.

When dont_fork is set, all the cleanup will be done directly from the signal handler, because there is only one process - the main process. This is not so safe as the previous case, but this mode should be used for debugging only and such shortcoming doesn't harm in that case.

Upon receipt of SIGINT, SIGPIPE or SIGTERM destroy_modules will be called. Each module may register so-called destroy function if it needs to do some cleanup when the server is terminating (flush of cache to disk for example). destroy_modules will call destroy function of all loaded modules.

If you need to terminate the server and all of its children, the best way how to do it is to send SIGTERM to the main process, the main process will in turn send the same signal to its children.

The main process and its children are in the same process group. Therefore the main process can kill all its children simply by sending a signal to pid 0, sending to pid 0 will send the signal to all processes in the same process group as the sending process. This is how the main process will terminate all children when it is going to shut down.

If one child exited during normal operation, the whole server will be shut down. This is better than let the server continue - a dead child might hold a lock and that could block the whole server, such situation cannot be avoided easily. Instead of that it is better to shutdown the whole server and let it restart.

1.4. Data Structures

Revision History
Revision $Revision$ $Date$

There are some data structures that are important and widely used in the server. We will describe them in detail in this section.

Note

There are many more structures and types defined across the server and modules. We will describe only the most important and common data structure here. The rest will be described in other sections if needed.

1.4.1. Type str

Revision History
Revision $Revision$ $Date$

One of our main goals was to make SER really fast. There are many functions across the server that need to work with strings. Usually these functions need to know string length. We wanted to avoid using of strlen because the function is relatively slow. It must scan the whole string and find the first occurrence of zero character. To avoid this, we created str type. The type has 2 fields, field s is pointer to the beginning of the string and field len is length of the string. We then calculate length of the string only once and later reuse saved value.

Important

str structure is quite important because it is widely used in SER (most functions accept str instead of char*).

str Type Declaration

struct _str{
    char* s;
    int len;
};

typedef struct _str str;

The declaration can be found in header file str.h.

Warning

Because we store string lengths, there is no need to zero terminate them. Some strings in the server are still zero terminated, some are not. Be careful when using functions like snprintf that rely on the ending zero. You can print variable of type str this way:

printf("%.*s", mystring->len, mystring->s);

That ensures that the string will be printed correctly even if there is no zero character at the end.

1.4.2. Structure hdr_field

Revision History
Revision $Revision$ $Date$

The structure represents a header field of a SIP message. A header field consist of name and body separated by a double colon. For example: "Server: SIP Express Router\r\n" is one header field. "Server" is header field name and "SI Express Router\r\n" is header field body.

The structure is defined in file hf.h under parser subdirectory.

Structure Declaration

struct hdr_field {   
    int type;                /* Header field type */
    str name;                /* Header field name */
    str body;                /* Header field body */
    void* parsed;            /* Parsed data structures */
    struct hdr_field* next;  /* Next header field in the list */
};

Field Description:

  • type - Type of the header field, the following header field types are defined (and recognized by the parser):

    HDR_VIA1, HDR_VIA2, HDR_TO, HDR_FROM, HDR_CSEQ, HDR_CALLID, HDR_CONTACT, HDR_MAXFORWARDS, HDR_ROUTE, HDR_RECORDROUTE, HDR_CONTENTTYPE, HDR_CONTENTLENGTH, HDR_AUTHORIZATION, HDR_EXPIRES, HDR_PROXYAUTH, HDR_WWWAUTH, HDR_SUPPORTED, HDR_REQUIRE, HDR_PROXYREQUIRE, HDR_UNSUPPORTED, HDR_ALLOW, HDR_EVENT, HDR_OTHER.

    Their meaning is self explanatory. HDR_OTHER marks header field not recognized by the parser.

  • name - Name of the header field (the part before colon)

  • body - body of the header field (the part after colon)

  • parsed - Each header field body can be further parsed. The field contains pointer to parsed structure if the header field was parsed already. The pointer is of type void* because it can point to different types of structure depending on the header field type.

  • next - Pointer to the next header field in linked list.

1.4.3. Structure sip_uri

Revision History
Revision $Revision$ $Date$

This structure represents parsed SIP URI.

struct sip_uri {
    str user;     /* Username */
    str passwd;   /* Password */
    str host;     /* Host name */
    str port;     /* Port number */
    str params;   /* Parameters */
    str headers;  
};

Field Description:

  • user - Username if found in the URI.

  • passwd - Password if found in the URI.

  • host - Hostname of the URI.

  • params - Parameters of the URI if any.

  • headers - See the SIP RFC.

1.4.4. Structure via_body

Revision History
Revision $Revision$ $Date$

The structure represents parsed Via header field. See file parse_via.h under parser subdirectory for more details.

struct via_body { 
    int error;
    str hdr;                      /* Contains "Via" or "v" */ 
    str name;
    str version;   
    str transport;
    str host;
    int port;
    str port_str;
    str params;
    str comment;
    int bsize;                    /* body size, not including hdr */
    struct via_param* param_lst;  /* list of parameters*/
    struct via_param* last_param; /*last via parameter, internal use*/

    /* shortcuts to "important" params*/
    struct via_param* branch;
    struct via_param* received;

    struct via_body* next;        /* pointer to next via body string if
    compact via or null */
};

Field Description:

  • error - The field contains error code when the parser was unable to parse the header field.

  • hdr- Header field name, it can be "Via" or "v" in this case.

  • name - Protocol name ("SIP" in this case).

  • version - Protocol version (for example "2.0").

  • transport - Transport protocol name ("TCP", "UDP" and so on).

  • host - Hostname or IP address contained in the Via header field.

  • port - Port number as integer.

  • port_str - Port number as string.

  • params - Unparsed parameters (as one string containing all the parameters).

  • comment - Comment.

  • bsize - Size of the body (not including hdr).

  • param_lst - Linked list of all parameters.

  • last_param - Last parameter in the list.

  • branch - Branch parameter.

  • received - Received parameter.

  • next - If the Via is in compact form (more Vias in the same header field), this field contains pointer to the next Via.

1.4.5. Structure ip_addr

Revision History
Revision $Revision$ $Date$

The structure represents IPv4 or IPv6 address. It is defined in ip_addr.h.

struct ip_addr{
     unsigned int af;     /* address family: AF_INET6 or AF_INET */
     unsigned int len;    /* address len, 16 or 4 */
		
     /* 64 bits aligned address */
     union {
         unsigned int   addr32[4];
         unsigned short addr16[8];
         unsigned char  addr[16];
     }u;
};

1.4.6. Structure lump

Revision History
Revision $Revision$ $Date$

The structure describes modifications that should be made to the message before the message will be sent.

The structure will be described in more detail later in chapter SIP Message Modifications.

1.4.7. Structure lump_rpl

Revision History
Revision $Revision$ $Date$

The structure represents text that should be added to reply. List of such data is kept in the request and processed when the request is being turned into reply.

The structure will be described in more detail later in chapter SIP Message Modifications.

1.4.8. Structure msg_start

Revision History
Revision $Revision$ $Date$

The structure represents the first line of a SIP request or response.

The structure is defined in file parse_fline.h under parser subdirectory.

Structure Declaration

struct msg_start {
    int type;                        /* Type of the Message - Request/Response */
    union {
        struct {
            str method;              /* Method string */
            str uri;                 /* Request URI */
            str version;             /* SIP version */
            int method_value;        /* Parsed method */
        } request;

        struct {
            str version;             /* SIP version */
            str status;              /* Reply status */
            str reason;              /* Reply reason phrase */
            unsigned int statuscode; /* Status code */
        } reply;
    }u;
};

Description of Request Related Fields:

  • type - Type of the message - REQUEST or RESPONSE.

  • method - Name of method (same as in the message).

  • uri - Request URI.

  • version - Version string.

  • method_value - Parsed method. Field method which is of type str will be converted to integer and stored here. This is good for comparison, integer comparison is much faster then string comparison.

Description of Response Related Fields:

  • version - Version string.

  • status - Response status code as string.

  • reason - Response reason string as in the message.

  • statuscode - Response status code converted to integer.

1.4.9. Structure sip_msg

Revision History
Revision $Revision$ $Date$

This is the most important structure in the whole server. This structure represents a SIP message. When a message is received, it is immediately converted into this structure and all operations are performed over the structure. After the server finished processing, this structure is converted back to character array buffer and the buffer is sent out.

Structure Declaration:

struct sip_msg {
    unsigned int id;               /* message id, unique/process*/
    struct msg_start first_line;   /* Message first line */
    struct via_body* via1;         /* The first via */
    struct via_body* via2;         /* The second via */
    struct hdr_field* headers;     /* All the parsed headers*/
    struct hdr_field* last_header; /* Pointer to the last parsed header*/
    int parsed_flag;               /* Already parsed header field types */

    /* Via, To, CSeq, Call-Id, From, end of header*/
    /* first occurrence of it; subsequent occurrences 
     * saved in 'headers' 
     */

    struct hdr_field* h_via1;
    struct hdr_field* h_via2;
    struct hdr_field* callid;
    struct hdr_field* to;
    struct hdr_field* cseq;
    struct hdr_field* from;
    struct hdr_field* contact;
    struct hdr_field* maxforwards;
    struct hdr_field* route;
    struct hdr_field* record_route;
    struct hdr_field* content_type;
    struct hdr_field* content_length;
    struct hdr_field* authorization;
    struct hdr_field* expires;
    struct hdr_field* proxy_auth;
    struct hdr_field* www_auth;
    struct hdr_field* supported;
    struct hdr_field* require;
    struct hdr_field* proxy_require;
    struct hdr_field* unsupported;
    struct hdr_field* allow;
    struct hdr_field* event;

    char* eoh;        /* pointer to the end of header (if found) or null */
    char* unparsed;   /* here we stopped parsing*/

   struct ip_addr src_ip;
   struct ip_addr dst_ip;
	
   char* orig;       /* original message copy */
   char* buf;        /* scratch pad, holds a modified message,
	              *  via, etc. point into it 
	              */
   unsigned int len; /* message len (orig) */
                     /* modifications */

   str new_uri;       /* changed first line uri*/
   int parsed_uri_ok; /* 1 if parsed_uri is valid, 0 if not */
   struct sip_uri parsed_uri; /* speed-up > keep here the parsed uri*/
	
   struct lump* add_rm; /* used for all the forwarded 
                         * requests */
   struct lump* repl_add_rm;    /* used for all the forwarded replies */
   struct lump_rpl *reply_lump; /* only for locally generated replies !!!*/
   char add_to_branch_s[MAX_BRANCH_PARAM_LEN];
   int add_to_branch_len;

       /* index to TM hash table; stored in core to avoid unnecessary calcs */
   unsigned int  hash_index;
		
       /* allows to set various flags on the message; may be used for 
	* simple inter-module communication or remembering processing state
	* reached 
	*/
   flag_t flags;	
};

Field Description:

  • id - Unique ID of the message within a process context.

  • first_line - Parsed first line of the message.

  • via1 - The first Via - parsed.

  • via2 - The second Via - parsed.

  • headers - Linked list of all parsed headers.

  • last_header - Pointer to the last parsed header (parsing is incremental, that means that the parser will stop if all requested headers were found and next time it will continue at the place where it stopped previously. Therefore this field will not point to the last header of the message if the whole message hasn't been parsed yet).

  • parsed_flag - Already parsed header field types (bitwise OR).

The following fields are set to zero if the corresponding header field was not found in the message or hasn't been parsed yet. (These fields are called hooks - they always point to the first occurrence if there is more than one header field of the same type).

  • h_via1 - Pointer to the first Via header field.

  • h_via2 - Pointer to the second Via header field.

  • callid - Pointer to the first Call-ID header field.

  • to - Pointer to the first To header field.

  • cseq - Pointer to the first CSeq header field.

  • from - Pointer to the first From header field.

  • contact - Pointer to the first Contact header field.

  • maxforwards - Pointer to the first Max-Forwards header field.

  • route - Pointer to the first Route header field.

  • record_route - Pointer to the first Record-Route header field.

  • content_type - Pointer to the first Content-Type header field.

  • content_length - Pointer to the first Content-Length header field.

  • authorization - Pointer to the first Authorization header field.

  • expires - Pointer to the first Expires header field.

  • proxy_auth - Pointer to the first Proxy-Authorize header field.

  • www_auth - Pointer to the first WWW-Authorize header field.

  • supported - Pointer to the first Supported header field.

  • require - Pointer to the first Require header field.

  • proxy_require - Pointer to the first Proxy-Require header field.

  • unsupported - Pointer to the first Unsupported header field.

  • allow - Pointer to the first Allow header field.

  • event - Pointer to the first Event header field.

The following fields are mostly used internally by the server and should be modified through dedicated functions only.

  • eoh - Pointer to the End of Header or null if not found yet (the field will be set if and only if the whole message was parsed already).

  • unparsed - Pointer to the first unparsed character in the message.

  • src_ip - Sender's IP address.

  • dst_ip - Destination's IP address.

  • orig - Original (unmodified) message copy, this field will hold unmodified copy of the message during the whole message lifetime.

  • buf - Message scratch-pad (modified copy of the message) - All modifications made to the message will be done here.

  • len - Length of the message (unmodified).

  • new_uri - New Request-URI to be used when forwarding the message.

  • parsed_uri_ok - 1 if parsed_uri is valid, 0 if not.

  • parsed_uri - The original parsed Request URI, sometimes it might be necessary to revert changes made to the Request URI and therefore we store the original URI here.

  • add_rm - Linked list describing all modifications that will be made to REQUEST before it will be forwarded. The list will be processed when the request is being converted to character array (i.e. immediately before the request will be send out).

  • repl_add_rm - Linked list describing all modifications that will be made to REPLY before it will be forwarded. the list will be processed when the reply is being converted to character array (i.e. immediately before the request will be send out).

  • reply_lump - This is list of data chunks that should be appended to locally generated reply, i.e. when the server is generating local reply out of the request. A local reply is reply generated by the server. For example, when processing of a request fails for some reason, the server might generate an error reply and send it back to sender.

  • add_to_branch_s - String to be appended to branch parameter.

  • add_to_branch_len - Length of the string.

  • hash_index - Index to a hash table in TM module.

  • flags - Allows to set various flags on the message. May be used for simple inter-module communication or remembering processing state reached.

1.4.10. Structure to_body

Revision History
Revision $Revision$ $Date$

The structure represents parsed To body. The structure is declared in parse_to.h file.

Structure Declaration:

struct to_param {
    int type;              /* Type of parameter */
    str name;              /* Name of parameter */
    str value;             /* Parameter value */
    struct to_param* next; /* Next parameter in the list */
};

struct to_body{
    int error;                    /* Error code */
    str body;                     /* The whole header field body */
    str uri;                      /* URI */
    str tag_value;                /* Value of tag */
    struct to_param *param_lst;   /* Linked list of parameters */
    struct to_param *last_param;  /* Last parameter in the list */
};

Structure to_param is a temporary structure representing a To URI parameter. Right now only TAG parameter will be marked in type field. All other parameters will have the same type.

Field Description:

  • error - Error code will be put here when parsing of To body fails.

  • body - The whole header field body.

  • uri - URI of the To header field.

  • tag_value - Value of tag parameter if present.

  • param_lst - Linked list of all parameters.

  • last_param - Pointer to the last parameter in the linked list.

1.4.11. Structure cseq_body

Revision History
Revision $Revision$ $Date$

The structure represents parsed CSeq body. The structure is declared in parse_cseq.h file.

Structure Declaration:

struct cseq_body{
    int error;  /* Error code */
    str number; /* CSeq number */
    str method; /* Associated method */
};

Field Description:

  • error - Error code will be put here when parsing of CSeq body fails.

  • number - CSeq number as string.

  • method - CSeq method.

1.5. The Routing Engine

Revision History
Revision $Revision$ $Date$

In a previous section we discussed how routing part of a config file gets translated into binary representation. In this section, we will discuss how the binary representation is used during message processing.

Upon a SIP message receipt, the server performs some basic sanity checks and converts the message into sip_msg structure. After that the Routing Engine will start processing the message.

The routing engine can be found in file action.c.

The main function is run_actions. The function accepts two parameters. The first parameter is list of actions to be processed (Remember, the config file gets translated into array of linked lists. Each linked list in the array represents one "route" part of the config file). The second parameter is sip_msg structure representing the message to be processed.

Upon a receipt of a request, the linked list representing the main route part will be processed so the first parameter will be rlist[0]. (The linked list of main route part is always at index 0).

The function will then sequentially call do_action function for each element of the linked list. Return value of the function is important. If the function returns 0, processing of the list will be stopped. By returning 0 a command can indicate that processing of the message should be stopped and the message will be dropped.

Modules may export so-called on_break handlers. on_break handler is a function, that will be called when processing of the linked-list is interrupted (ret == 0). All such handlers will be called when processing of the linked-list is finished and ret == 0.

1.5.1. do_action Function

do_action function is core of the routing engine. There is a big switch statement. Each case of the statements is one command handled by the server core itself.

The following commands are handled by the SER core itself: drop, forward, send, log, append_branch, len_gt, setflag, resetflag, isflagset, setavpflag, resetavpflag, isavpflagset, error, route, exec, revert_uri, set_host, set_hostport, set_user, set_userpass, set_port, set_uri, prefix, strip, if, module.

Each of the commands is represented by a case statement in the switch. (For example, if you are interested in implementation of drop command, look at "case DROP_T:" statement in the function.

The respective commands will be described now.

  • drop - This command is very simple, it simply returns 0 which will result in abortion of processing of the request. No other commands after drop will be executed.

  • forward - The function will forward the message further. The message will be either forwarded to the Request URI of the message or to IP or host given as parameter.

    In the first case, host in the Request URI must be converted into corresponding IP address. Function mk_proxy converts hostname to corresponding IP address. The message is then sent out using forward_request function.

    In the second case, hostname was converted to IP address in fixup i.e. immediately after the config file was compiled into its binary representation. The first parameter is pointer to proxy structure created in the fixup and therefore we only need to call forward_request here to forward the message further.

  • send - This functions sends the message to a third-party host. The message will be sent out as is - i.e. without Request URI and Via altering.

    Hostname or IP address of the third-party host is specified as a parameter of the function.

    The message will be sent out using udp_send directly.

  • log - The message given as a parameter will be logged using system logger. It can be either syslog or stderr (depends on configuration). The message is logged using LOG which is a macro defined in dprint.h header file.

  • append_branch - Append a new URI for forking.

    More than one destinations may be associated with a single SIP request. If the server was configured so, it will use all the destinations and fork the request.

    The server keeps an array of all destinations, that should be used when forking. The array and related functions can be found in file dset.c. There is function append_branch which adds a new destination to the set.

    This command simply calls append_branch function and adds a new destination to the destination set.

  • len_gt - The command accepts one number as a parameter. It then compares the number with length of the message. If the message length is greater or equal then the number then 1 will be returned otherwise the function returns -1.

  • setflag - Sets a flag in the message. The command simply calls setflags function that will set the flag. Fore more information see file flag.c.

  • resetflag - Same as command setflag - only resetflag will be called instead of setflag.

  • isflagset - Test if the flag is set or not.

  • setavpflag(avp, flag_id) - Sets a flag in the AVP(s). The command simply set custom flag of AVP. The flags may be used in script using isavpflagset or in a module to perform specific operation on marked AVPs. Flag identifier must be declared via avpflags statement.

  • resetavpflag(avp, flag_id) - Same as command setavpflag - only resetavpflag will be called instead of setavpflag.

  • isavpflagset(avp, flag_id) - Test if the avp flag is set or not.

  • error - Log a message with NOTICE log level.

  • route - Execute another route statement.

    As we have mentioned already, there can be more that one route statement in the config file. One of them is main (without number), the other are additional. This command makes it possible to execute an additional route statement.

    The command accepts one parameter which is route statement number. First sanity checks over the parameter will be performed. If the checks passed, function run_actions will be called. The function accepts two parameters. The first one is linked list to execute, the second one is sip_msg structure representing the message to be processed.

    As you might remember, each route statement was compiled into linked list of commands to be executed and head of the linked list was stored in rlist array. For example, head of linked list representing route statement with number 4 will be stored at position 4 in the array (position 0 is reserved for the main route statement).

    So the command will simply call run_actions(rlist[a->p1.number], msg) and that will execute route statement with number given as parameter.

  • exec - Execute a shell command.

    The command accepts one parameter of type char*. The string given as parameter will be passed to system function which will in turn execute /bin/sh -c <string>.

  • revert_uri - Revert changes made to the Request URI.

    If there is a new URI stored in new_uri of sip_msg structure, it will be freed. The original Request URI will be used when forwarding the message.

    If there is a valid URI in parsed_uri field of sip_msg structure (indicated by parsed_uri_ok field), it will be freed too.

  • set_host - Change hostname of Request URI to value given as parameter.

    If there is a URI in new_uri field, it will be modified, otherwise the original Request URI will be modified.

  • set_hostport - change hostname and port of Request URI to value given as string parameter.

    If there is a URI in new_uri field, it will be modified, otherwise the original Request URI will be modified.

  • set_user - Set username part of Request URI to string given as parameter.

    If there is a URI in new_uri field, it will be modified, otherwise the original Request URI will be modified.

  • set_userpass - Set username and password part of Request URI to string given as parameter.

    If there is a URI in new_uri field, it will be modified, otherwise the original Request URI will be modified.

  • set_port - Set port of Request URI to value given as parameter.

    If there is a URI in new_uri field, it will be modified, otherwise the original Request URI will be modified.

  • set_uri - Set a new Request URI.

    If there is a URI in new_uri field, it will be freed. If there is a valid URI in parsed_uri field, it will be freed too.

    Then URI given as parameter will be stored in new_uri field. (If new_uri contains a URI it will be used instead of Request URI when forwarding the message).

  • prefix - Set the parameter as username prefix.

    The string will be put immediately after "sip:" part of the Request URI.

    If there is a URI in new_uri field, it will be modified, otherwise the original Request URI will be modified.

  • strip - Remove first n characters of username in Request URI.

    If there is a URI in new_uri field, it will be modified, otherwise the original Request URI will be modified.

  • if - if Statement.

    There is an expression associated with the command and one or two linked lists of commands. The expression is a regular expression compiled into binary form in the fixup when the config file was compiled.

    The expression will be evaluated now. If the result is > 0, the first linked list will be executed using run_action function. The linked list represents command enclosed in curly braces of if command.

    Otherwise, if there is the second list, it will be executed in the same way. The second list represents commands of else statement.

  • module - Execute a function exported by a module.

    When a command in a route statement is not recognized by the core itself (i.e. it is not one of commands handled by the core itself), list of exported functions of all loaded modules will be searched for a function with corresponding name and number of parameters.

    If the function was found, module command (this one) will be created and pointer to the function will be stored in p1.data field.

    So, this command will simply call function whose pointer is in p1.data field and will pass 2 parameters to the function. If one or both of the parameters were not used, 0 will be passed instead.

    Return value of the function will be returned as return value of module command.

    This command makes SER pretty extensible while the core itself is still reasonably small and clean. Additional functionality is put in modules and loaded only when needed.

1.6. The SIP Message Parser

Revision History
Revision $Revision$ $Date$

In this section we will discuss internals of the SIP message header parser implemented in the server. Message parsing is very important and one of the most time consuming operations of a SIP server. We have been trying to make the parser as fast as possible.

A header field parser can be either in the server core or in a module. By convention, parser that is needed by the core itself or is needed by at least two modules will be in the core. Parsers contained in modules will be not described in this section.

There is a parser subdirectory that contains all the parsers and related stuff.

The following parsers can be found under parser subdirectory:

1.6.1. Structure of a SIP Message

A SIP message consists of message header and optional message body. The header is separated from the body with a empty line (containing CRLF only).

Message header consists of the first line and one or more header fields. The first line determines type of the message. Header fields provide additional information that is needed by clients and servers to be able to process the message.

Each header field consists of header field name and header field body. Header field name is delimited from header field body by a colon (":"). For example, "Server: SIP Express Router" - in this case "Server" is header field name and "SIP Express Router" is header field body.

1.6.2. Parser Structure

The server implements what we call incremental parsing. It means that a header field will be not parsed unless it is really needed. There is a minimal set of header that will be parsed every time. The set includes:

  • The first line - the server must know if the message is request or response

  • Via header field - Via will be needed for sure. We must add ourself to Via list when forwarding the message.

1.6.2.1. The First Line Parser
Revision History
Revision $Revision$ $Date$

Purpose of the parser is to parse the first line of a SIP message. The first line is represented by msg_start structure define in file parse_fline.h under parser subdirectory.

The main function of the first line parser is parse_first_line, the function will fill in msg_start structure.

Follow inline comments in the function if you want to add support for a new message type.

1.6.2.2. The Header Field Name Parser
Revision History
Revision $Revision$ $Date$

The purpose of the header field type parser is to recognize type of a header field. The following types of header field will be recognized:

Via, To, From, CSeq, Call-ID, Contact, Max-Forwards, Route, Record-Route, Content-Type, Content-Length, Authorization, Expires, Proxy-Authorization, WWW-Authorization, supported, Require, Proxy-Require, Unsupported, Allow, Event.

All other header field types will be marked as HDR_OTHER.

Main function of header name parser is parse_hname2. The function can be found in file parse_hname.c. The function accepts pointers to begin and end of a header field and fills in hdf_field structure. name field will point to the header field name, body field will point to the header field body and type field will contain type of the header field if known and HDR_OTHER if unknown.

The parser is 32-bit, it means, that it processes 4 characters of header field name at time. 4 characters of a header field name are converted to an integer and the integer is then compared. This is much faster than comparing byte by byte. Because the server is compiled on at least 32-bit architectures, such comparison will be compiled into one instruction instead of 4 instructions.

We did some performance measurement and 32-bit parsing is about 3 times faster for a typical SIP message than corresponding automaton comparing byte by byte. Performance may vary depending on the message size, parsed header fields and header fields type. Test showed that it was always as fast as corresponding 1-byte comparing automaton.

Since comparison must be case insensitive in case of header field names, it is necessary to convert it to lower case first and then compare. Since converting byte by byte would slow down the parser a lot, we have implemented a hash table, that can again convert 4 bytes at once. Since set of keys that need to be converted to lowercase is known (the set consists of all possible 4-byte parts of all recognized header field names) we can pre-calculate size of the hash table to be synonym-less. That will simplify (and speed up) the lookup a lot. The hash table must be initialized upon the server startup (function init_hfname_parser).

The header name parser consists of several files, all of them are under parser subdirectory. Main file is parse_hname2.c - this files contains the parser itself and functions used to initialize and lookup the hash table. File keys.h contains automatically generated set of macros. Each macro is a group of 4 bytes converted to integer. The macros are used for comparison and the hash table initialization. For example, for Max-Forwards header field name, the following macros are defined in the file:

#define _max__ 0x2d78616d   /* "max-" */
#define _maX__ 0x2d58616d   /* "maX-" */
#define _mAx__ 0x2d78416d   /* "mAx-" */
#define _mAX__ 0x2d58416d   /* "mAX-" */
#define _Max__ 0x2d78614d   /* "Max-" */
#define _MaX__ 0x2d58614d   /* "MaX-" */
#define _MAx__ 0x2d78414d   /* "MAx-" */
#define _MAX__ 0x2d58414d   /* "MAX-" */

#define _forw_ 0x77726f66   /* "forw" */
#define _forW_ 0x57726f66   /* "forW" */
#define _foRw_ 0x77526f66   /* "foRw" */
#define _foRW_ 0x57526f66   /* "foRW" */
#define _fOrw_ 0x77724f66   /* "fOrw" */
#define _fOrW_ 0x57724f66   /* "fOrW" */
#define _fORw_ 0x77524f66   /* "fORw" */
#define _fORW_ 0x57524f66   /* "fORW" */
#define _Forw_ 0x77726f46   /* "Forw" */
#define _ForW_ 0x57726f46   /* "ForW" */
#define _FoRw_ 0x77526f46   /* "FoRw" */
#define _FoRW_ 0x57526f46   /* "FoRW" */
#define _FOrw_ 0x77724f46   /* "FOrw" */
#define _FOrW_ 0x57724f46   /* "FOrW" */
#define _FORw_ 0x77524f46   /* "FORw" */
#define _FORW_ 0x57524f46   /* "FORW" */

#define _ards_ 0x73647261   /* "ards" */
#define _ardS_ 0x53647261   /* "ardS" */
#define _arDs_ 0x73447261   /* "arDs" */
#define _arDS_ 0x53447261   /* "arDS" */
#define _aRds_ 0x73645261   /* "aRds" */
#define _aRdS_ 0x53645261   /* "aRdS" */
#define _aRDs_ 0x73445261   /* "aRDs" */
#define _aRDS_ 0x53445261   /* "aRDS" */
#define _Ards_ 0x73647241   /* "Ards" */
#define _ArdS_ 0x53647241   /* "ArdS" */
#define _ArDs_ 0x73447241   /* "ArDs" */
#define _ArDS_ 0x53447241   /* "ArDS" */
#define _ARds_ 0x73645241   /* "ARds" */
#define _ARdS_ 0x53645241   /* "ARdS" */
#define _ARDs_ 0x73445241   /* "ARDs" */
#define _ARDS_ 0x53445241   /* "ARDS" */

As you can see, Max-Forwards name was divided into three 4-byte chunks: Max-, Forw, ards. The file contains macros for every possible lower and upper case character combination of the chunks. Because the name (and therefore chunks) can contain colon (":"), minus or space and these characters are not allowed in macro name, they must be substituted. Colon is substituted by "1", minus is substituted by underscore ("_") and space is substituted by "2".

When initializing the hash table, all these macros will be used as keys to the hash table. One of each upper and lower case combinations will be used as value. Which one ?

There is a convention that each word of a header field name starts with a upper case character. For example, most of user agents will send "Max-Forwards", messages containing some other combination of upper and lower case characters (for example: "max-forwards", "MAX-FORWARDS", "mAX-fORWARDS") are very rare (but it is possible).

Considering the previous paragraph, we optimized the parser for the most common case. When all header fields have upper and lower case characters according to the convention, there is no need to do hash table lookups, which is another speed up.

For example suppose we are trying to figure out if the header field name is Max-Forwards and the header field name is formed according to the convention (i.e. "Max-Forwards"):

  • Get the first 4 bytes of the header field name ("Max-"), convert it to an integer and compare to "_Max__" macro. Comparison succeeded, continue with the next step.

  • Get next 4 bytes of the header field name ("Forw"), convert it to an integer and compare to "_Forw_" macro. Comparison succeeded, continue with the next step.

  • Get next 4 bytes of the header field name ("ards"), convert it to an integer and compare to "_ards_" macro. Comparison succeeded, continue with the next step.

  • If the following characters are spaces and tabs followed by a colon (or colon directly without spaces and tabs), we found Max-Forwards header field name and can set type field to HDR_MAXFORWARDS. Otherwise (other characters than colon, spaces and tabs) it is some other header field and set type field to HDR_OTHER.

As you can see, there is no need to do hash table lookups if the header field was formed according to the convention and the comparison was very fast (only 3 comparisons needed !).

Now lets consider another example, the header field was not formed according to the convention, for example "MAX-forwards":

  • Get the first 4 bytes of the header field name ("MAX-"), convert it to an integer and compare to "_Max__" macro.

    Comparison failed, try to lookup "MAX-" converted to integer in the hash table. It was found, result is "Max-" converted to integer.

    Try to compare the result from the hash table to "_Max__" macro. Comparison succeeded, continue with the next step.

  • Compare next 4 bytes of the header field name ("forw"), convert it to an integer and compare to "_Max__" macro.

    Comparison failed, try to lookup "forw" converted to integer in the hash table. It was found, result is "Forw" converted to integer.

    Try to compare the result from the hash table to "Forw" macro. Comparison succeeded, continue with the next step.

  • Compare next 4 bytes of the header field name ("ards"), convert it to integer and compare to "ards" macro. Comparison succeeded, continue with the next step.

  • If the following characters are spaces and tabs followed by a colon (or colon directly without spaces and tabs), we found Max-Forwards header field name and can set type field to HDR_MAXFORWARDS. Otherwise (other characters than colon, spaces and tabs) it is some other header field and set type field to HDR_OTHER.

In this example, we had to do 2 hash table lookups and 2 more comparisons. Even this variant is still very fast, because the hash table lookup is synonym-less, lookups are very fast.

1.6.2.3. To Header Field Parser
Revision History
Revision $Revision$ $Date$

Purpose of this parser is to parse body of To header field. The parser can be found in file parse_to.c under parser subdirectory.

Main function is parse_to but there is no need to call the function explicitly. Every time the parser finds a To header field, this function will be called automatically. Result of the parser is to_body structure. Pointer to the structure will be stored in parsed field of hdr_field structure. Since the pointer is void*, there is a convenience macro get_to in file parse_to.h that will do the necessary type-casting and will return pointer to to_body structure.

The parser itself is a finite state machine that will parse To body according to the grammar defined in RFC3261 and store result in to_body structure.

The parser gets called automatically from function get_hdr_field in file msg_parser.c. The function first creates and initializes an instance of to_body structure, then calls parse_to function with the structure as a parameter and if everything went OK, puts the pointer to the structure in parsed field of hdr_field structure representing the parsed To header field.

The newly created structure will be freed when the message is being destroyed, see function clean_hdr_field in file hf.c for more details.

See Section 1.4.10, “Structure to_body for detailed description of the data structure produced by this parser.

1.6.2.4. From Header Field Parser
Revision History
Revision $Revision$ $Date$

This parser is only a wrapper to the To header field parser. Since bodies of both header fields are identical, From parser only calls To parser.

The wrapper can be found in file parse_from.c under parser subdirectory. There is only one function called parse_from_header. The function accepts one parameter which is pointer to structure representing the From header field to be parsed. The function creates an instance of to_body structure and initializes it. It then calls parse_to function and if everything went OK, the pointer to the newly created structure will be put in parsed field of the structure representing the parsed header field.

The newly created structure will be freed when the whole message is being destroyed. (See To header field parser description for more details).

From parser must be called explicitly !

If the main parser finds a From header field, it will not parse the header field body automatically. It is up to you to call the parse_from_header when you want to parse a From header field body.

1.6.2.5. CSeq Header Field Parser
Revision History
Revision $Revision$ $Date$

Purpose of this parser is to parse body of CSeq header field. The parser can be found in file parse_cseq.c under parser subdirectory.

Main function is parse_cseq but there is no need to call the function explicitly. Every time the parser finds a CSeq header field, this function will be called automatically. Result of the parser is cseq_body structure. Pointer to the structure will be stored in parsed field of hdr_field structure. Since the pointer is void*, there is a convenience macro get_cseq in file parse_cseq.h that will do the necessary type-casting and will return pointer to cseq_body structure.

The parser will parse CSeq body according to the grammar defined in RFC3261 and store result in cseq_body structure.

The parser gets called automatically from function get_hdr_field in file msg_parser.c. The function first creates and initializes an instance of cseq_body structure, then calls parse_cseq function with the structure as a parameter and if everything went OK, puts the pointer to the structure in parsed field of hdr_field structure representing the parsed CSeq header field.

The newly created structure will be freed when the message is being destroyed, see function clean_hdr_field in file hf.c for more details.

See Section 1.4.11, “Structure cseq_body for description of the cseq_body structure.

1.6.2.6. Event Body Parser
Revision History
Revision $Revision$ $Date$

Purpose of this parser is to parse body of an Event Header field. The parser can be found in file parse_event.c under parser subdirectory.

Note

This is NOT fully featured Event body parser ! The parser was written for Presence Agent module only and thus can recognize Presence package only. No subpackages will be recognized. All other packages will be marked as "OTHER".

The parser should be replace by a more generic parser if subpackages or parameters should be parsed too.

Main function is parse_event in file parse_event.c. The function will create an instance of event_t structure and call the parser. If everything went OK, pointer to the newly created structure will be stored in parsed field of hdr_field structure representing the parsed header field.

As usually, the newly created structure will be freed when the whole message is being destroyed. See function clean_hdr_field in file hf.c.

The parser will be not called automatically when the main parser finds an Event header field. It is up to you to call the parser when you really need the body of the header field to be parsed (call parse_event function).

1.6.2.6.1. Structure event_t
Revision History
Revision $Revision$ $Date$

The structure represents parsed Event body. The structure is declared in parse_event.h file.

Structure Declaration:

#define EVENT_OTHER    0
#define EVENT_PRESENCE 1

typedef struct event {
    str text;   /* Original string representation */
    int parsed; /* Parsed variant */
} event_t;

Field Description:

  • text - Package name as text.

  • parsed - Package name as integer. It will be EVENT_PRESENCE for presence package and EVENT_OTHER for rest.

1.6.2.7. Expires HF Body Parser
Revision History
Revision $Revision$ $Date$

The parser parses body of Expires header field. The body is very simple, it consists of number only. so the parser only removes any leading tabs and spaces and converts the number from string to integer. That's it.

The parser can be found in file parse_expires.c under parser subdirectory. Main function is parse_expires. The function is not called automatically when an Expires header field was found. It is up to you to call the function if you need the body to be parsed.

The function creates a new instance of exp_body_t structure and calls the parser. If everything went OK, pointer to the newly created structure will be saved in parsed field of the hdr_field structure representing the parsed header field.

1.6.2.7.1. Structure exp_body_t
Revision History
Revision $Revision$ $Date$

The structure represents parsed Expires body. The structure is declared in parse_expires.h file.

Structure Declaration:

typedef struct exp_body {
    str text;   /* Original text representation */
    int val;    /* Parsed value */
} exp_body_t;

Field Description:

  • text - Expires value as text.

  • val - Expires value as integer.

1.6.2.8. Via HF Body Parser
Revision History
Revision $Revision$ $Date$

Purpose of this parser is to parse body of Via header field. The parser can be found in file parse_via.c under parser subdirectory.

Main function is parse_via but there is no need to call the function explicitly. Every time the parser finds a Via header field, this function will be called automatically. Result of the parser is via_body structure. Pointer to the structure will be stored in parsed field of hdr_field structure representing the parsed header field.

The parser itself is a finite state machine that will parse Via body according to the grammar defined in RFC3261 and store result in via_body structure.

The parser gets called automatically from function get_hdr_field in file msg_parser.c. The function first creates and initializes an instance of via_body structure, then calls parse_via function with the structure as a parameter and if everything went OK, puts the pointer to the structure in parsed field of hdr_field structure representing the parsed Via header field.

The newly created structure will be freed when the message is being destroyed, see function clean_hdr_field in file hf.c for more details.

Structure via_body is described in Section 1.4.4, “Structure via_body.

1.6.2.9. Contact Header Field Parser
Revision History
Revision $Revision$ $Date$

The parser is located under parser/contact subdirectory. The parser is not called automatically when the main parser finds a Contact header field. It is your responsibility to call the parser if you want a Contact header field body to be parsed.

Main function is parse_contact in file parse_contact.c. The function accepts one parameter which is structure hdr_field representing the header field to be parsed. A single Contact header field may contain multiple contacts, the parser will parse all of them and will create linked list of all such contacts.

The function creates and initializes an instance of contact_body structure. Then function contact_parser will be called. If everything went OK, pointer to the newly created structure will be stored in parsed field of the hdr_field structure representing the parsed header field.

Function contact_parser will then check if the contact is star, if not it will call parse_contacts function that will parse all contacts of the header field.

Function parse_contacts can be found in file contact.c. It extracts URI and parses all contact parameters.

The Contact parameter parser can be found in file cparam.c.

The following structures will be created during parsing:

Note

Mind that none of string in the following structures is zero terminated ! Be very careful when processing the strings with functions that require zero termination (printf for example) !

typedef struct contact_body {
    unsigned char star;    /* Star contact */
    contact_t* contacts;   /* List of contacts */
} contact_body_t;

This is the main structure. Pointer to instance of this structure will be stored in parsed field of structure representing the header field to be parsed. The structure contains two field:

  • star field - This field will contain 1 if the Contact was star (see RFC3261 for more details).

  • contacts field - This field contains pointer to linked list of all contacts found in the Contact header field.

typedef struct contact {
    str uri;              /* contact uri */
    cparam_t* q;          /* q parameter hook */
    cparam_t* expires;    /* expires parameter hook */
    cparam_t* method;     /* method parameter hook */
    cparam_t* params;     /* List of all parameters */
    struct contact* next; /* Next contact in the list */
} contact_t;

This structure represents one Contact (Mind that there might be several contacts in one Contact header field delimited by a comma). Its fields have the following meaning:

  • uri - This field contains pointer to begin of URI and its length.

  • q - This is a hook to structure representing q parameter. If there is no such parameter, the hook contains 0.

  • expires - This is a hook to structure representing expires parameter. If there is no such parameter, the hook contains 0.

  • method - This is a hook to structure representing method parameter. If there is no such parameter, the hook contains 0.

  • params - Linked list of all parameters.

  • next - Pointer to the next contact that was in the same header field.

typedef enum cptype {
    CP_OTHER = 0,  /* Unknown parameter */
    CP_Q,          /* Q parameter */
    CP_EXPIRES,    /* Expires parameter */
    CP_METHOD      /* Method parameter */
} cptype_t;

This is an enum of recognized types of contact parameters. Q parameter will have type set to CP_Q, Expires parameter will have type set to CP_EXPIRES and Method parameter will have type set to CP_METHOD. All other parameters will have type set to CP_OTHER.

/*
 * Structure representing a contact
 */
typedef struct cparam {
    cptype_t type;       /* Type of the parameter */
    str name;            /* Parameter name */
    str body;            /* Parameter body */
    struct cparam* next; /* Next parameter in the list */
} cparam_t;

This structure represents a contact parameter. Field description follows:

  • type - Type of the parameter, see cptype enum for more details.

  • name - Name of the parameter (i.e. the part before "=").

  • body - Body of the parameter (i.e. the part after "=").

  • next - Next parameter in the linked list.

1.6.2.10. Digest Body Parser
Revision History
Revision $Revision$ $Date$

Purpose of this parser is to parse digest response. The parser can be found under parser/digest subdirectory. There might be several header fields containing digest response, for example Proxy-Authorization or WWW-Authorization. The parser can be used for all of them.

The parser is not called automatically when by the main parser. It is your responsibility to call the parser when you want a digest response to be parsed.

Main function is parse_credentials defined in digest.c. The function accepts one parameter which is header field to be parsed. As result the function will create an instance of auth_body_t structure which will represent the parsed digest credentials. Pointer to the structure will be put in parsed field of the hdr_field structure representing the parsed header field. It will be freed when the whole message is being destroyed.

The digest parser contains 32-bit digest parameter parser. The parser was in detail described in section Header Field Name Parser. See that section for more details about the digest parameter parser algorithm, they work in the same way.

Description of digest related structures follows:

typedef struct auth_body {
    /* This is pointer to header field containing
     * parsed authorized digest credentials. This
     * pointer is set in sip_msg->{authorization,proxy_auth}
     * hooks.
     *
     * This is necessary for functions called after
     * {www,proxy}_authorize, these functions need to know
     * which credentials are authorized and they will simply
     * look into 
     * sip_msg->{authorization,proxy_auth}->parsed->authorized
     */
    struct hdr_field* authorized;
    dig_cred_t digest;           /* Parsed digest credentials */
    unsigned char stale;         /* Flag is set if nonce is stale */
    int nonce_retries;           /* How many times the nonce was used */
} auth_body_t;

This is the "main" structure. Pointer to the structure will be stored in parsed field of hdr_field structure. Detailed description of its fields follows:

  • authorized - This is a hook to header field containing authorized credentials.

    A SIP message may contain several credentials. They are distinguished using realm parameter. When the server is trying to authorize the message, it must first find credentials with corresponding realm and than authorize the credentials. To authorize credentials server calculates response string and if the string matches to response string contained in the credentials, credentials are authorized (in fact it means that the user specified in the credentials knows password, nothing more, nothing less).

    It would be good idea to remember which credentials contained in the message are authorized, there might be other functions interested in knowing which credentials are authorized.

    That is what is this field for. A function that successfully authorized credentials (currently there is only one such function in the server, it is function authorize in auth module) will put pointer to header field containing the authorized credentials in this field. Because there might be several header field containing credentials, the pointer will be put in authorized field in the first header field in the message containing credentials. That means that it will be either header field whose pointer is in www_auth or proxy_auth field of sip_msg structure representing the message.

    When a function wants to find authorized credentials, it will simply look in msg->www_auth->parsed->authorized or msg->proxy_auth->parsed->authorized, where msg is variable containing pointer to sip_msg structure.

    To simplify the task of saving and retrieving pointer to authorized credentials, there are two convenience functions defined in digest.c file. They will be described later.

  • digest - Structure containing parsed digest credentials. The structure will be described in detail later.

  • stale - This field will be set to 1 if the server received a stale nonce. Next time when the server will be sending another challenge, it will use "stale=true" parameter. "stale=true" indicates to the client that username and password used to calculate response were correct, but nonce was stale. The client should recalculate response with the same username and password (without disturbing user) and new nonce. For more details see RFC2617.

  • nonce_retries - This fields indicates number of authorization attempts with same nonce.

/*
 * Errors returned by check_dig_cred
 */
typedef enum dig_err {
    E_DIG_OK = 0,        /* Everything is OK */
    E_DIG_USERNAME  = 1, /* Username missing */
    E_DIG_REALM = 2,     /* Realm missing */
    E_DIG_NONCE = 4,     /* Nonce value missing */
    E_DIG_URI = 8,       /* URI missing */
    E_DIG_RESPONSE = 16, /* Response missing */
    E_DIG_CNONCE = 32,   /* CNONCE missing */
    E_DIG_NC = 64,       /* Nonce-count missing */
} dig_err_t;

This is enum of all possible errors returned by check_dig_cred function.

  • E_DIG_OK - No error found.

  • E_DIG_USERNAME - Username parameter missing in digest response.

  • E_DIG_REALM - Realm parameter missing in digest response.

  • E_DIG_NONCE - Nonce parameter missing in digest response.

  • E_DIG_URI - Uri parameter missing in digest response.

  • E_DIG_RESPONSE - Response parameter missing in digest response.

  • E_DIG_CNONCE - Cnonce parameter missing in digest response.

  • E_DIG_NC - Nc parameter missing in digest response.

/* Type of algorithm used */
typedef enum alg {
    ALG_UNSPEC = 0,   /* Algorithm parameter not specified */
    ALG_MD5 = 1,      /* MD5 - default value*/
    ALG_MD5SESS = 2,  /* MD5-Session */
    ALG_OTHER = 4     /* Unknown */
} alg_t;

This is enum of recognized algorithm types. (See description of algorithm structure for more details).

  • ALG_UNSPEC - Algorithm was not specified in digest response.

  • ALG_MD5 - "algorithm=MD5" was found in digest response.

  • ALG_MD5SESS - "algorithm=MD5-Session" was found in digest response.

  • ALG_OTHER - Unknown algorithm parameter value was found in digest response.

/* Quality Of Protection used */
typedef enum qop_type { 
    QOP_UNSPEC = 0,   /* QOP parameter not present in response */
    QOP_AUTH = 1,     /* Authentication only */
    QOP_AUTHINT = 2,  /* Authentication with integrity checks */
    QOP_OTHER = 4     /* Unknown */
} qop_type_t;

This enum lists all recognized qop parameter values.

  • QOP_UNSPEC - qop parameter was not found in digest response.

  • QOP_AUTH - "qop=auth" was found in digest response.

  • QOP_AUTHINT - "qop=auth-int" was found in digest response.

  • QOP_OTHER - Unknown qop parameter value was found in digest response.

/* Algorithm structure */
struct algorithm {
     str alg_str;       /* The original string representation */
     alg_t alg_parsed;  /* Parsed value */
};

The structure represents "algorithm" parameter of digest response. Description of fields follows:

  • alg_str - Algorithm parameter value as string.

  • alg_parsed - Parsed algorithm parameter value.

/* QOP structure */
struct qp {
    str qop_str;           /* The original string representation */
    qop_type_t qop_parsed; /* Parsed value */
};

This structure represents "qop" parameter of digest response. Description of fields follows:

  • qop_str - Qop parameter value as string.

  • qop_parsed - Parsed "qop" parameter value.

/*
 * Parsed digest credentials
 */
typedef struct dig_cred {
    str username;         /* Username */
    str realm;            /* Realm */
    str nonce;            /* Nonce value */
    str uri;              /* URI */
    str response;         /* Response string */
    str algorithm;        /* Algorithm in string representation */
    struct algorithm alg; /* Type of algorithm used */
    str cnonce;           /* Cnonce value */
    str opaque;           /* Opaque data string */
    struct qp qop;        /* Quality Of Protection */
    str nc;               /* Nonce count parameter */
} dig_cred_t;

This structure represents set of digest credentials parameters. Description of field follows:

  • username - Value of "username" parameter.

  • realm - Value of "realm" parameter.

  • nonce - Value of "nonce" parameter.

  • uri - Value of "uri" parameter.

  • response - Value of "response" parameter.

  • algorithm - Value of "algorithm" parameter as string.

  • alg - Parsed value of "algorithm" parameter.

  • cnonce - Value of "cnonce" parameter.

  • opaque - Value of "opaque" parameter.

  • qop - Value of "qop" parameter.

  • nc - Value of "nc" parameter.

1.6.2.10.1. Other Functions Of the Digest Parser

There are some other mainly convenience functions defined in the parser. The function will be in detail described in this section. All the functions are defined in digest.c file.

dig_err_t check_dig_cred( _c);  
dig_cred_t* _c;
 

This function performs some basic sanity check over parsed digest credentials. The following conditions must be met for the checks to be successful:

  • There must be non-empty "username" parameter in the credentials.

  • There must be non-empty "realm" parameter in the credentials.

  • There must be non-empty "nonce" parameter in the credentials.

  • There must be non-empty "uri" parameter in the credentials.

  • There must be non-empty "response" parameter in the credentials.

  • If qop parameter is set to QOP_AUTH or QOP_AUTHINT, then there must be also non-empty "cnonce" and "nc" parameters in the digest.

Note

It is recommended to call check_dig_cred before you try to authorize the credentials. If the function fails, there is no need to try to authorize the credentials because the authorization will fail for sure.

int mark_authorized_cred( _m,  
  _h);  
struct sip_msg* _m;
struct hdr_field* _h;
 

This is convenience function. The function saves pointer to the authorized credentials. For more info see description of authorized field in auth_body structure.

int get_authorized_cred( _m,  
  _h);  
struct sip_msg* _m;
struct hdr_field** _h;
 

This is convenience function. The function will retrieve pointer to authorized credentials previously saved using mark_authorized_cred function. If there is no such credentials, 0 will be stored in variable pointed to by the second parameter. The function returns always zero. For more information see description of authorized field in auth_body structure.

1.7. Module Interface

Revision History
Revision $Revision$ $Date$

Abstract

SER features modular architecture which allows us to split SER's functionality across several modules. This approach gives us greater flexibility, only required set of functions can be loaded upon startup which minimizes the server's memory footprint. Modules can be also provided by 3rd party developers and distributed separately from the main server. Most of the functionality that SER provides is available through modules, the core itself contains only minimum set of functions that is essential for proper server's behavior or that is needed by all modules.

This chapter provides detailed information on module interface of SER, which is used to pass information on available functions and parameters from the modules to the core.

1.7.1. Shared Objects

Abstract

First it would be good to know how SER loads and uses modules before we describe the module interface in detail. This section gives a brief overview of SER's module subsystem.

SER modules are compiled as "shared objects". A file containing a shared object has usually .so suffix. All modules (shared objects) will be stored in one directory after installation. For example tm module, which contains code essential for stateful processing, will be stored in file named tm.so. By default these files are stored in /usr/local/lib/ser/modules directory.

You can later load the modules using loadmodule command in your configuration file. If you want to load previously mentioned tm.so module, you can do it using loadmodule "/usr/local/lib/ser/modules/tm.so" in your configuration file. This command invokes dynamic linker provided by the operating system which opens tm.so file, loads it into memory and resolves all symbol dependencies (a module might require symbols from the core, for example functions and variables).

As the last step of the module loading the core tries to find variable named exports, which describes all functions and parameters provided by the module. These functions and parameters are later available to the server and can be used either in the configuration file or by other modules.

1.7.2. Exporting Functions

Abstract

Each module can provide zero or more functions, which can be used in the configuration file or by other modules internally. This section gives a detailed description of structure describing exported functions and passing this information to the core through the module interface.

Each function exported by a module must be described by cmd_export_t structure. Structures describing all exported functions are arranged into an array and pointer to the array is then passed to the core. The last element of the array must contain 0 in all it's fields, this element serves as the mark telling the core that this is the very last element and it must stop scanning the array.

Each exported function is described by the following structure:

struct cmd_export_ {
	char* name;             /* null terminated command name */
	cmd_function function;  /* pointer to the corresponding function */
	int param_no;           /* number of parameters used by the function */
	fixup_function fixup;   /* pointer to the function called to "fix" the parameters */
	int flags;              /* Function flags */
};	   

typedef struct cmd_export_ cmd_export_t;

Meaning of the fileds:

  • char* name

    This is the name under which the function will be visible to the core. Usually it is the same as the name of the corresponding function.

  • cmd_function function

    cmd_function type is defined as follows:

    typedef int (*cmd_function)(struct sip_msg*, char*, char*);

    The first parameter is a SIP message being processed, the other 2 parameters are given from the configuration file.

    Note

    From time to time you might need to export a function that has different synopsis. This can happen if you export functions that are supposed to be called by other modules only and must not be called from the configuration script. In this case you will have to do type-casting otherwise the compiler will complain and will not compile your module.

    Simply put (cmd_function) just before the function name, for example (cmd_function)my_function. Don't use this unless you know what are you doing ! The server might crash if you pass wrong parameters to the function later !

  • int param_no

    Number of parameters of the function. It can be 0, 1 or 2. The function will be not visible from the configuration script if you use another value.

  • fixup_function fixup

    This is the function that will be used to "fixup" function parameters. Set this field to 0 if you don't need this.

    If you provide pointer to a fixup function in this field, the fixup function will be called for each occurrence of the exported function in the configuration script.

    The fixup function can be used to perform some operation on the function parameters. For example, if one of the parameters is a regular expression, you can use the fixup to compile the regular expression. The fixup functions are called only once - upon the server startup and so the regular expression will be compiled before the server starts processing messages. When the server calls the exported function to process a SIP message, the function will be given the already compiled regular expression and doesn't have to compile it again. This is a significant performance improvement.

    Fixup functions can also be used to convert string to integer. As you have might noticed, the exported functions accept up to 2 parameters of type char*. Because of that it is not possible to pass integer parameters from the script files directly. If you want to pass an integer as a parameter, you must pass it as string (i.e. enclosed in quotes).

    Fixup function can be used to convert the string back to integer. Such a conversion should happend only once because the string parameter doesn't change when the server is running. Fixup is therefore ideal place for the conversion, it will be converted upon the server startup before the server starts processing SIP messages. After the conversion the function will get directly the converted value. See existing modules for example of such a fixup function.

  • int flags

    Usage of each function can be restricted. You may want to write a function that can be used by other modules but cannot be called from the script. If you write a function that is supposed to process SIP requests only, you may want to restrict it so it will be never called for SIP replies and vice versa. That's what is flags field for.

    This field is OR value of different flags. Currently only REQUEST_ROUTE and REPLY_ROUTE flags are defined and used by the core. If you use REQUEST_ROUTE flag, then the function can be called from the main route block. If you use REPLY_ROUTE flag, then the function can be called from reply route blocks (More on this in the SER User's Guide). If this field is set to 0, then the function can be called internally (i.e. from other modules) only. If you want to make your function callable anywhere in the script, you can use REQUEST_ROUTE | REPLY_ROUTE.

1.7.3. Exporting Parameters

Abstract

Each module can provide zero or more parameters, which can affect the module's behavior. This section gives a detailed description of structures describing exported parameters and passing this information to the core through the module interface.

Each parameter exported by a module must be described by param_export_t structure. Structures describing all exported parameters are arranged into an array and pointer to the array is then passed to the core. The last element of the array must contain 0 in all it's fields, this element serves as the mark telling the core that this is the very last element and it must stop scanning the array (This is same as in array of exported functions).

Each exported parameter is described by the following structure:

struct param_export_ {
	char* name;             /* null terminated param. name */
	modparam_t type;        /* param. type */
	void* param_pointer;    /* pointer to the param. memory location */
};

typedef struct param_export_ param_export_t;

Meaning of the fields:

  • char* name

    This is null-terminated name of the parameters as it will be used in the scripts. Usually this is the same as the name of the variable holding the value.

  • modparam_t type

    Type of the parameter. Currently only two types are defined. PARAM_INT for integer parameters (corresponding variable must be of type int), PARAM_STR for str parameters (corresponding variable must be of type char*) and PARAM_STRING for string parameters (corresponding variable must be of type char*).

  • void* param_pointer

    Pointer to the corresponding variable (stored as void* pointer, make sure that the variable has appropriate type depending on the type of the parameter !).

1.7.4. Module Initialization

If you need to initialize your module before the server starts processing SIP messages, you should provide initialization function. Each module can provide two initialization functions, main initialization function and child-specific initialization function. Fields holding pointers to both initialization functions are in main export structure (will be described later). Simply pass 0 instead of function pointer if you don't need one or both initialization functions.

The main initialization function will be called before any other function exported by the module. The function will be called only once, before the main process forks. This function is good for initialization that is common for all the children (processes). The function should return 0 if everything went OK and a negative error code otherwise. Server will abort if the function returns a negative value.

Per-child initialization function will be called after the main process forks. The function will be called for each child separately. The function should perform initialization that is specific for each child. For example each child process might open it's own database connection to avoid locking of a single connection shared by many processes. Such connections can be opened in the per-child initialization function. The function accepts one parameter which is rank (integer) of child for which the function is being executed. This allows developers to distinguish different children and perform different initialization for each child. The meaning of return value is same as in the main initialization function.

1.7.5. Module Clean-up

A module can also export a clean-up function that will be called by the main process when the server shuts down. The function accepts no parameters and return no value.

1.7.6. Module Callbacks

TBD.

1.7.7. exports Structure - Assembling the Pieces Together

We have already described how a module can export functions and parameters, but we haven't yet described how to pass this information to the core. Each module must have variable named exports which is structure module_exports. The variable will be looked up by the core immediately after it loads the module. The structure contains pointers to both arrays (functions, parameters), pointers to both initialization functions, destroy function and the callbacks. So the structure contains everything the core will need.

The structure looks like the follows:

struct module_exports{
    char* name;                     /* null terminated module name */
    cmd_export_t* cmds;             /* null terminated array of the exported commands */
    param_export_t* params;         /* null terminated array of the exported module parameters */
    init_function init_f;           /* Initialization function */
    response_function response_f;   /* function used for responses, returns yes or no; can be null */
    destroy_function destroy_f;     /* function called when the module should be "destroyed", e.g: on ser exit; can be null */
    onbreak_function onbreak_f;
    child_init_function init_child_f;  /* function called by all processes after the fork */
};

Field description:

  • char* name

    Null terminated name of the module

  • cmd_exports* cmds

    Pointer to the array of exported functions

  • param_export_t* params

    Pointer to the array of exported parameters

  • init_function init_f

    Pointer to the module initialization function

  • response_function response_f

    Pointer to function processing responses

  • destroy_function destroy_f

    Pointer to the module clean-up function

  • onbreak_function onbreak_f

    TBD

  • child_init_function init_child_f

    Pointer to the per-child initialization function

1.7.8. Example - Simple Module Interface

Let's suppose that we are going to write a simple module. The module will export two functions - foo_req which will be processing SIP requests and foo_int which is an internal function that can be called by other modules only. Both functions will take 2 parameters.

/* Prototypes */
int foo_req(struct sip_msg* msg, char* param1, char* param2);
int foo_res(struct sip_msg* msg, char* param1, char* param2);

static cmd_export cmds[] = {
    {"foo_req", foo_req, 2, 0, ROUTE_REQUEST},
    {"foo_int", foo_int, 2, 0, 0            },
    {0, 0, 0, 0}
};

The module will also have two parameters, foo_bar of type integer and bar_foo of type string.

int foo_bar = 0;
char* bar_foo = "default value";
str bar_bar = STR_STATIC_INIT("default");

static param_export params[] = {
    {"foo_bar", PARAM_INT, &foo_bar},
    {"bar_foo", PARAM_STRING, &bar_foo},
    {"bar_bar", PARAM_STR, &ar_bar.s},
    {0, 0, 0}
};

We will also create both initialization functions and a clean-up function:

static int mod_init(void)
{
    printf("foo module initializing\n");
}

static int child_init(int rank)
{
    printf("child nr. %d initializing\n", rank);
    return 0;
}

static void destroy(void)
{
    printf("foo module cleaning up\n");
}

And finally we put everything into the exports structure:

struct module_exports exports = {
    "foobar",   /* Module name */
    cmds,       /* Exported functions */
    params,     /* Exported parameters */
    mod_init,   /* Module initialization function */
    0,          /* Response function */
    destroy,    /* Clean-up function */
    0,          /* On Cancel function */
    child_init  /* Per-child init function */
};

And that's it.

1.7.9. Module Interface Internals

All the data structures and functions mentioned in this section can be found in files sr_module.h and sr_module.c.

1.7.9.1. Structure sr_module
Revision History
Revision $Revision$ $Date$

Each loaded module is represented by an instance of sr_module structure. All the instances are linked. There is a global variable modules defined in file sr_module.c which is head of linked-list of all loaded modules.

Detailed description of the structure follows:

struct sr_module{
    char* path;
    void* handle;
    struct module_exports* exports;
    struct sr_module* next;
};

Fields and their description:

  • path - Path to the module. This is the path you pass as parameter to loadmodule function in the config file.

  • handle - Handle returned by dlopen.

  • exports - Pointer to structure describing interface of the module (will be described later).

  • next - Pointer to the next sr_module structure in the linked list.

1.7.9.2. Structure module_exports
Revision History
Revision $Revision$ $Date$

This structure describes interface that must be exported by each module. Every module must have a global variable named exports which is of type struct module_exports.

Immediately after dlopen the server will try to find symbol named exports in the module to be loaded. This symbol is a structure describing interface of the module. Pointer to the symbol will be then put in exports field of sr_module structure representing the module in the server.

Detailed description of the structure follows:

struct module_exports{
    char* name;                     /* null terminated module name */
    char** cmd_names;               /* cmd names registered 
                                     * by this modules */
    cmd_function* cmd_pointers;     /* pointers to the 
                                     * corresponding functions */
    int* param_no;                  /* number of parameters used by 
                                     * the function */
    fixup_function* fixup_pointers; /* pointers to functions 
                                     * called to "fix"
                                     * the params, e.g: precompile 
                                     * a re */
    int cmd_no;                     /* number of registered commands
                                     * (size of cmd_{names,pointers} 
                                     */
    char** param_names;      /* parameter names registered 
                              * by this modules */
    modparam_t* param_types; /* Type of parameters */
    void** param_pointers;   /* Pointers to the corresponding 
                              * memory locations */
    int par_no;              /* number of registered parameters */
    init_function init_f;         /* Initialization function */
    response_function response_f; /* function used for responses,
                                   * returns yes or no; can be null 
                                   */
    destroy_function destroy_f;   /* function called when the module 
                                   * should be "destroyed", e.g: on 
                                   * ser exit;
                                   * can be null */
    onbreak_function onbreak_f;
    child_init_function init_child_f;  /* function called by all 
                                        * processes after the fork */
};

Fields and their description:

  • name - Name of the module.

  • cmd_names - Array of names of exported commands.

  • cmd_pointers - Array of pointers to functions implementing commands specified in cmd_names array.

    Function Prototype:

    int cmd_function( msg,  
      param1,  
      param2);  
    struct sip_msg* msg;
    char* param1;
    char* param2;
     

    The first parameter is sip_msg currently being processed. Remaining parameters are parameters from the config file. If the function accepts only one parameter, param2 will be set to zero, if the function accepts no parameters, param1 and param2 will be set to zero.

    The function should return number > 0 if everything went OK and processing of the message should continue. The function should return 0 if processing of the message should be stopped. The function should return number < 0 on an error.

  • param_no - Array of number of parameters of exported commands.

  • fixup_pointer - Array of pointers to fixup functions, each fixup function for one exported command. If there is no fixup function for a particular exported function, corresponding field in the array will contain zero.

    Function Prototype:

    int fixup_function( param,  
      param_no);  
    void** param;
    int param_no;
     

    The first parameter is pointing to variable to be fixed. The second parameter is order of the variable.

    The function should return 0 if everything went OK and number < 0 on an error.

  • cmd_no - Number of exported commands.

    Important

    cmd_names, cmd_pointers, param_no and fixup_pointer arrays must have at least cmd_no elements ! (It might even kill your cat if you fail to fulfill this condition).

  • param_names - Array of names of exported parameters.

  • param_types - Array of types of parameters, each field of the array can be either PARAM_STR/PARAM_STRING or PARAM_INT (currently only three parameter types are defined).

  • param_pointers - Array of pointers to variables, that hold values of the parameters.

  • param_no - Number of exported parameters.

    Important

    param_names, param_types and param_pointers arrays must have at least param_no elements ! (Remember the previous note about your cat ? The same might happen to your dog if you fail to fulfill the condition second time !).

  • init_f - Pointer to module's initialization function, 0 if the module doesn't need initialization function.

    Function Prototype:

    int init_function( );  
     

    The function should return 0 if everything went OK and number < 0 on an error;

  • response_f - If a module is interested in seeing responses, it will provide pointer to a function here. The function will be called when a response comes. The field will contain 0 if the module doesn't want to see responses.

    Function Prototype:

    int response_function( msg);  
    struct sip_msg* msg;
     

    The function accepts one parameter which is structure representing the response currently being processed.

    The function should return 0 if the response should be dropped.

  • destroy_f - Destroy function. The function will be called when the server is shutting down. Can be 0 if the module doesn't need destroy function.

    Function Prototype:

    void destroy_function( );  
     
  • onbreak_f - On break function. The function will be called when processing of a route statement was aborted. Can be 0 if module doesn't need this function.

    Function Prototype:

    void onbreak_function( msg);  
    struct sip_msg* msg;
     

    The function accepts one parameter which is message currently being processed.

  • init_child_f - Child initialization function. This is an additional initialization function. init_f will be called from the main process BEFORE the main process forks children. init_child_f will be called from all children AFTER the fork.

    Per-child specific initialization can be done here. For example, each child can open its own database connection in the function, and so on.

    Function Prototype:

    int child_init_function( rank);  
    int rank;
     

    The function accepts one parameter, which is rank (starting from 0) of child executing the function.

    The function should return 0 if everything went OK and number < 0 on an error.

1.7.9.3. Module Loading

Modules are compiled and stored as shared objects. Shared objects have usually appendix ".so". Shared objects can be loaded at runtime.

When you instruct the server to load a module using loadmodule command in the config file, it will call function load_module. The function will do the following:

  • It will try to open specified file using dlopen. For example if you write loadmodule "/usr/lib/ser/modules/auth.so" in the config file, the server will try to open file "/usr/lib/ser/modules/auth.so" using dlopen function.

    If dlopen failed, the server will issue an error and abort.

  • As the next step, list of all previously loaded modules will be searched for the same module. If such module is found, it means, that user is trying to load the same module twice. In such case an warning will be issued and server will abort.

  • The server will try to find pointer to "exports" symbol using dlsym in the module. If that fails, server will issue an error and abort.

  • And as the last step, function register_module will register the module with the server core and loading of the module is complete.

Function register_module registers a module with the server core. By registration we mean the following set of steps (see function register_module in file sr_module.c for more details):

  • The function creates and initializes new instance of sr_module structure.

  • path field will be set to path of the module.

  • handle field will be set to handle previously returned by dlopen.

  • exports field will be set to pointer to module's exports structure previously obtained through dlsym in load_module function.

  • As the last step, the newly created structure will be inserted into linked list of all loaded modules and registration is complete.

1.7.9.4. Module Configuration

In addition to set of functions each module can export set of configuration variables. Value of a module's configuration variable can be changed in the config file using modparam function. Module configuration will be described in this section.

1.7.9.4.1. Function modparam

modparam function accepts three parameters:

  • module name - Name of module as exported in name field of exports global variable.

  • variable name - Name of variable to be set - it must be one of names specified in param_names field of exports variable of the module.

  • value - New value of the variable. There are two types of variables: string and integer. If the last parameter (value) of modparam function is enclosed in quotes, it is string parameter and server will try to find the corresponding variable among string parameters only.

    Otherwise it is integer parameter and server will try to find corresponding variable among integer parameters only.

1.7.9.4.2. Function set_mod_param

When the server finds modparam function in the config file, it will call set_mod_param function. The function can be found in modparam.c file. The function will do the following:

  • It tries to find corresponding variable using find_param_export function.

  • If it is string parameter, a new copy of the string will be obtained using strdup function and pointer to the copy will be stored in the variable.

    If it is integer parameter, its value will be simply copied in the variable.

1.7.9.4.3. Function find_param_export

This function accepts 3 parameters:

  • module - Name of module.

  • parameter - Name of parameter to be found.

  • type - Type of the parameter.

The function will search list of all modules until it finds module with given name. Then it will search through all module's exported parameters until it finds parameter with corresponding name and type. If such parameter was found, pointer to variable holding the parameter's value will be returned. If the function failed to find either module or parameter with given name and type then zero will be returned.

1.7.9.5. Looking Up an Exported Function

If you need to find exported function with given name and number of parameters, find_export function is what you need. The function is defined in sr_module.c file. The function accepts two parameters:

  • name - Name of function to be found.

  • param_no - Number of parameters of the function.

The function will search through list of all loaded modules and in each module through array of all exported functions until it finds function with given name and number of parameters. If such exported function was found, find_exported will return pointer to the function, otherwise zero will be returned.

1.7.9.6. Additional Functions

There are several additional functions defined in file sr_module.c. There functions are mostly internal and shouldn't be used directly by user. We will shortly describe them here.

  • register_builtin_modules - Some modules might be linked statically with main executable, this is handy for debugging. This function will register all such modules upon server startup.

  • init_child - This function will call child-initialization function of all loaded modules. The function will be called by the server core immediately after the fork.

  • find_module - The function accepts pointer to an exported function and number of parameters as parameters and returns pointer to corresponding module that exported the function.

  • destroy_modules - The function will call destroy function of all loaded modules. This function will be called by the server core upon shut down.

  • init_modules - The function will call initialization function of all loaded modules. The function will be called by the server before the fork.

1.8. The Database Interface

Revision History
Revision $Revision$ $Date$

This is a generic database interface for modules that need to utilize a database. The interface should be used by all modules that access database. The interface will be independent of the underlying database server.

Note

If possible, use predefined macros if you need to access any structure attributes.

For additional description, see comments in sources of mysql module.

If you want to see more complicated examples of how the API could be used, see sources of dbexample, usrloc or auth modules.

1.8.1. Data types

There are several data types. All of them are defined in header files under db subdirectory, a client must include db.h header file to be able to use them.

1.8.1.1. Type db_con_t
Revision History
Revision $Revision$ $Date$

This type represents a database connection, all database functions (described below) use a variable of this type as one argument. In other words, variable of db_con_t type serves as a handle for a particular database connection.

typedef struct db_con {
    char* table;     /* Default table to use */
    void* con;       /* Database connection */
    void* res;       /* Result of previous operation */
    void* row;       /* Internal, not for public use */
   int connected;    /* 1 if connection is established */
} db_con_t;

There are no macros defined for db_con_t type.

1.8.1.2. Type db_key_t
Revision History
Revision $Revision$ $Date$

This type represents a database key. Every time you need to specify a key value, this type should be used. In fact, this type is identical to const char*.

typedef const char* db_key_t;

There are no macros defined (they are not needed).

1.8.1.3. Type db_type_t
Revision History
Revision $Revision$ $Date$

Each cell in a database table can be of a different type. To distinguish among these types, the db_type_t enumeration is used. Every value of the enumeration represents one data-type that is recognized by the database API. This enumeration is used in conjunction with db_type_t. For more information, see the next section.

typedef enum {
    DB_INT,       /* Integer number */
    DB_DOUBLE,    /* Decimal number */
    DB_STRING,    /* String */
    DB_STR,       /* str structure */
    DB_DATETIME   /* Date and time */
    DB_BLOB       /* Binary large object */
} db_type_t;

There are no macros defined.

1.8.1.4. Type db_val_t
Revision History
Revision $Revision$ $Date$

This structure represents a value in the database. Several data-types are recognized and converted by the database API:

  • DB_INT - Value in the database represents an integer number.

  • DB_DOUBLE - Value in the database represents a decimal number.

  • DB_STRING - Value in the database represents a string.

  • DB_STR - Value in the database represents a string.

  • DB_DATETIME - Value in the database represents date and time.

  • DB_BLOB - Value in the database represents binary large object.

These data-types are automatically recognized, converted from internal database representation and stored in a variable of corresponding type.

typedef struct db_val {
    db_type_t type;              /* Type of the value */
    int nul;                     /* NULL flag */
    union {                      
        int int_val;             /* Integer value */
        double double_val;       /* Double value */
        time_t time_val;         /* Unix time_t value */
        const char* string_val;  /* Zero terminated string */
        str str_val;             /* str structure */
        str blob_val;            /* Structure describing blob */
    } val;
} db_val_t;

Note

All macros expect pinter to db_val_t variable as a parameter.

  • VAL_TYPE(value) Macro.

    Use this macro if you need to set/get the type of the value

    Example 1. VAL_TYPE Macro

    ...
    VAL_TYPE(val) = DB_INT;
    if (VAL_TYPE(val) == DB_FLOAT)
    ...

  • VAL_NULL(value) Macro.

    Use this macro if you need to set/get the null flag. Non-zero flag means that the corresponding cell in the database contained no data (NULL value in MySQL terminology).

    Example 2. VAL_NULL Macro

    ...
    if (VAL_NULL(val) == 1) {
        printf("The cell is NULL");
    }
    ...

  • VAL_INT(value) Macro.

    Use this macro if you need to access integer value in db_val_t structure.

    Example 3. VAL_INT Macro

    ...
    if (VAL_TYPE(val) == DB_INT) {
        printf("%d", VAL_INT(val));
    }
    ...

  • VAL_DOUBLE(value) Macro.

    Use this macro if you need to access double value in the db_val_t structure.

    Example 4. VAL_DOUBLE Macro

    ...
    if (VAL_TYPE(val) == DB_DOUBLE) {
        printf("%f", VAL_DOUBLE(val));
    }
    ...

  • VAL_TIME(value) Macro.

    Use this macro if you need to access time_t value in db_val_t structure.

    Example 5. VAL_TIME Macro

    ...
    time_t tim;
    if (VAL_TYPE(val) == DB_DATETIME) {
        tim = VAL_TIME(val);
    }
    ...

  • VAL_STRING(value) Macro.

    Use this macro if you need to access string value in db_val_t structure.

    Example 6. VAL_STRING Macro

    ...
    if (VAL_TYPE(val) == DB_STRING) {
        printf("%s", VAL_STRING(val));
    }
    ...

  • VAL_STR(value) Macro.

    Use this macro if you need to access str structure in db_val_t structure.

    Example 7. VAL_STR Macro

    ...
    if (VAL_TYPE(val) == DB_STR) {
        printf("%.*s", VAL_STR(val).len, VAL_STR(val).s);
    }
    ...

  • VAL_BLOB(value) Macro.

    Use this macro if you need to access blob value in db_val_t structure.

    Example 8. VAL_STR Macro

    ...
    if (VAL_TYPE(val) == DB_BLOB) {
        printf("%.*s", VAL_BLOB(val).len, VAL_BLOB(val).s);
    }
    ...

1.8.1.5. Type db_row_t
Revision History
Revision $Revision$ $Date$

This type represents one row in a database table. In other words, the row is an array of db_val_t variables, where each db_val_t variable represents exactly one cell in the table.

typedef struct db_row {
    db_val_t* values;    /* Array of values in the row */
    int n;               /* Number of values in the row */
} db_val_t;
  • ROW_VALUES(row) Macro.

    Use this macro to get pointer to array of db_val_t structures.

    Example 9. ROW_VALUES Macro

    ...
    db_val_t* v = ROW_VALUES(row);
    if (VAL_TYPE(v) == DB_INT)
    ...

  • ROW_N(row) Macro.

    Use this macro to get number of cells in a row.

    Example 10. ROW_N Macro

    ...
    db_val_t* val = ROW_VALUES(row);
    for(i = 0; i < ROW_N(row); i++) {
        switch(VAL_TYPE(val + i)) {
            case DB_INT: ...; break;
            case DB_DOUBLE: ...; break;
            ...
        }
    }
    ...

1.8.1.6. Type db_res_t
Revision History
Revision $Revision$ $Date$

This type represents a result returned by db_query function (see below). The result can consist of zero or more rows (see db_row_t description).

Note

A variable of type db_res_t returned by db_query function uses dynamically allocated memory, don't forget to call db_free_result if you don't need the variable anymore. You will encounter memory leaks if you fail to do this !

In addition to zero or more rows, each db_res_t object contains also an array of db_key_t objects. The objects represent keys (names of columns).

typedef struct db_res {
    struct {
        db_key_t* keys;    /* Array of column names */
        db_type_t* types;  /* Array of column types */
        int n;             /* Number of columns */
    } col;
    struct db_row* rows;   /* Array of rows */
    int n;                 /* Number of rows */
} db_res_t;
  • RES_NAMES(res) Macro.

    Use this macro if you want to obtain pointer to an array of cell names.

    Example 11. RES_NAMES Macro

    ...
    db_key_t* column_names = ROW_NAMES(row);
    ...

  • RES_COL_N(res) Macro.

    Use this macro if you want to get the number of columns in the result.

    Example 12. RES_COL_N Macro

    ...
    int ncol = RES_COL_N(res);
    for(i = 0; i < ncol; i++) {
        /* do something with the column */
    }
    ...

  • RES_ROWS(res) Macro.

    Use this macro if you need to obtain pointer to array of rows.

    Example 13. RES_ROWS Macro

    ...
    db_row_t* rows = RES_ROWS(res);
    ...

  • RES_ROW_N(res) Macro.

    Use this macro if you need to obtain the number of rows in the result.

    Example 14. RES_ROW_N Macro

    ...
    int n = RES_ROW_N(res);
    ...

1.8.2. Functions

There are several functions that implement the database API logic. All function names start with db_ prefix, except bind_dbmod. bind_dbmod is implemented in db.c file, all other functions are implemented in a standalone module. Detailed description of functions follows.

1.8.2.1. bind_dbmod

This function is special, it's only purpose is to call find_export function in the SER core and find addresses of all other functions (starting with db_ prefix). This function MUST be called FIRST !

int bind_dbmod( );  
 

The function takes no parameters.

The function returns 0 if it was able to find addresses of all other functions, otherwise value < 0 is returned.

1.8.2.2. db_init

Use this function to initialize the database API and open a new database connection. This function must be called after bind_dbmod but before any other function is called.

db_con_t* db_init( _sql_url);  
const char* _sql_url;
 

The function takes one parameter, the parameter must contain database connection URL. The URL is of the form mysql://username:password@host:port/database where:

  • username - Username to use when logging into database (optional).

  • password - Password if it was set (optional).

  • host - Hostname or IP address of the host where database server lives (mandatory).

  • port - Port number of the server if the port differs from default value (optional).

  • database - If the database server supports multiple databases, you must specify name of the database (optional).

The function returns pointer to db_con_t* representing the connection if it was successful, otherwise 0 is returned.

1.8.2.3. db_close

The function closes previously open connection and frees all previously allocated memory. The function db_close must be the very last function called.

void db_close( _h);  
db_con_t* _h;
 

The function takes one parameter, this parameter is a pointer to db_con_t structure representing database connection that should be closed.

Function doesn't return anything.

1.8.2.4. db_query

This function implements SELECT SQL directive.

int db_query( _h,  
  _k,  
  _v,  
  _c,  
  _n,  
  _nc,  
  _o,  
  _r);  
db_con_t* _h;
db_key_t* _k;
db_val_t* _v;
db_key_t* _c;
int _n;
int _nc;
db_key_t* _o;
db_res_t** _r;
 

The function takes 8 parameters:

  • _h - Database connection handle.

  • _k - Array of column names that will be compared and their values must match.

  • _v - Array of values, columns specified in _k parameter must match these values.

  • _c - Array of column names that you are interested in.

  • _n - Number of key-value pairs to match in _k and _v parameters.

  • _nc - Number of columns in _c parameter.

  • _o - Order by.

  • _r - Address of variable where pointer to the result will be stored.

If _k and _v parameters are NULL and _n is zero, you will get the whole table. If _c is NULL and _nc is zero, you will get all table columns in the result

_r will point to a dynamically allocated structure, it is necessary to call db_free_result function once you are finished with the result.

Strings in the result are not duplicated, they will be discarded if you call db_free_result, make a copy yourself if you need to keep it after db_free_result.

You must call db_free_result BEFORE you can call db_query again !

The function returns 0 if everything is OK, otherwise value < 0 is returned.

1.8.2.5. db_free_result

This function frees all memory allocated previously in db_query, it is necessary to call this function for a db_res_t structure if you don't need the structure anymore. You must call this function BEFORE you call db_query again !

int db_free_result( _h,  
  _r);  
db_con_t* _h;
db_res_t* _r;
 

The function takes 2 parameters:

  • _h - Database connection handle.

  • _r - Pointer to db_res_t structure to destroy.

The function returns 0 if everything is OK, otherwise the function returns value < 0.

1.8.2.6. db_insert

This function implements INSERT SQL directive, you can insert one or more rows in a table using this function.

int db_insert( _h,  
  _k,  
  _v,  
  _n);  
db_con_t* _h;
db_key_t* _k;
db_val_t* _v;
int _n;
 

The function takes 4 parameters:

  • _h - Database connection handle.

  • _k - Array of keys (column names).

  • _v - Array of values for keys specified in _k parameter.

  • _n - Number of keys-value pairs int _k and _v parameters.

The function returns 0 if everything is OK, otherwise the function returns value < 0.

1.8.2.7. db_delete

This function implements DELETE SQL directive, it is possible to delete one or more rows from a table.

int db_delete( _h,  
  _k,  
  _v,  
  _n);  
db_con_t* _h;
db_key_t* _k;
db_val_t* _v;
int _n;
 

The function takes 4 parameters:

  • _h - Database connection handle.

  • _k - Array of keys (column names) that will be matched.

  • _v - Array of values that the row must match to be deleted.

  • _n - Number of keys-value parameters in _k and _v parameters.

If _k is NULL and _v is NULL and _n is zero, all rows are deleted (table will be empty).

The function returns 0 if everything is OK, otherwise the function returns value < 0.

1.8.2.8. db_update

The function implements UPDATE SQL directive. It is possible to modify one or more rows in a table using this function.

int db_update( _h,  
  _k,  
  _v,  
  _uk,  
  _uv,  
  _n,  
  _un);  
db_con_t* _h;
db_key_t* _k;
db_val_t* _v;
db_key_t* _uk;
db_val_t* _uv;
int _n;
int _un;
 

The function takes 7 parameters:

  • _h - Database connection handle.

  • _k - Array of keys (column names) that will be matched.

  • _v - Array of values that the row must match to be modified.

  • _uk - Array of keys (column names) that will be modified.

  • _uv - New values for keys specified in _k parameter.

  • _n - Number of key-value pairs in _k and _v parameters.

  • _un - Number of key-value pairs in _uk and _uv parameters.

The function returns 0 if everything is OK, otherwise the function returns value < 0.

1.8.2.9. db_use_table

The function db_use_table takes a table name and stores it in db_con_t structure. All subsequent operations (insert, delete, update, query) are performed on that table.

int db_use-table( _h,  
  _t);  
db_con_t* _h;
cons char* _t;
 

The function takes 2 parameters:

  • _h - Database connection handle.

  • _t - Table name.

The function returns 0 if everything is OK, otherwise the function returns value < 0.

1.9. Locking Interface

Revision History
Revision $Revision$ $Date$

1.9.1. Why use it ?

The main reason in creating it was to have a single transparent interface to various locking methods. For example right now SER uses the following locking methods, depending on their availability on the target system.

  • FAST_LOCK

    Fast inline assembly locks, defined in fast_lock.h. They are currently available for x86, sparc64, strong-arm (amv4l) and ppc (external untested contributed code). In general if the assembly code exists for a given architecture and the compiler knows inline assembly (for example sun cc does not) FAST_LOCK is preferred. The main advantage of using FAST_LOCK is very low memory overhead and extremely fast lock/unlock operations (like 20 times faster then SYSV semaphores on linux & 40 times on solaris). The only thing that comes close to them are pthread mutexes (which are about 3-4 times slower).

  • PHTREAD_MUTEX

    Uses pthread_mutex_lock/unlock. They are quite fast but they work between processes only on some systems (they do not work on linux).

  • POSIX_SEM

    Uses posix semaphores (sem_wait/sem_post). They are slower then the previous methods but still way faster then SYSV semaphores. Unfortunately they also do not work on all the systems (e.g. linux).

  • SYSV_SEM

    This is the most portable but also the slowest locking method. Another problem is that the number of semaphores that can be allocated by a process is limited. One also has to free them before exiting.

1.9.2. How to use it ?

First of all you have to include locking.h. Then when compiling the code one or all of FAST_LOCK, USE_PTHREAD_MUTEX, USE_PTHREAD_SEM or USE_SYSV_SEM must be defined (the ser Makefile.defs takes care of this, you should need to change it only for new architectures or compilers). locking.h defines 2 new types: gen_lock_t and lock_set_t.

1.9.3. Simple Locks

The simple locks are simple mutexes. The type is gen_lock_t.

Warning

Do not make any assumptions on gen_lock_t base type, it does not have to be always an int.

1.9.3.1. Allocation And Initialization

The locks are allocated with: gen_lock_t* lock_alloc() and initialized with gen_lock_t* lock_init(gen_lock_t* lock). Both functions return 0 on failure. The locks must be initialized before use. A proper alloc/init sequence looks like:

gen_lock_t* lock;

lock=lock_alloc();
if (lock==0) goto error;
if (lock_init(lock)==0){
   lock_dealloc(lock);
   goto error; /* could not init lock*/
}
...

Lock allocation can be skipped in some cases: if the lock is already in shared memory you don't need to allocate it again, you can initialize it directly, but keep in mind that the lock MUST be in shared memory.

Example:

struct s {
    int foo;
    gen_lock_t lock;
} bar;

bar=shm_malloc(sizeof struct s); /* we allocate it in the shared memory */
if (lock_init(&bar->lock)==0){
 /* error initializing the lock */
 ...
}
1.9.3.2. Destroying And Deallocating the Locks
void lock_destroy( gen_lock_t* lock);  
gen_lock_t* lock;
 
void lock_dealloc( gen_lock_t* lock);  
gen_lock_t* lock;
 

The lock_destroy function must be called first. It removes the resources associated with the lock, but it does not also free the lock shared memory part. Think of sysv rmid. Please don't forget to call this function, or you can leave allocated resources in some cases (e.g sysv semaphores). Be careful to call it in your module destroy function if you use any global module locks.

Example:

lock_destroy(lock);
lock_dealloc(lock);

Of course you don't need to call lock_dealloc if your lock was not allocated with lock_alloc.

1.9.3.3. Locking And Unlocking
void lock_get( gen_lock_t* lock);  
gen_lock_t* lock;
 
void lock_release( gen_lock_t* lock);  
gen_lock_t* lock;
 
1.9.3.4. Lock Sets

The lock sets are kind of sysv semaphore sets equivalent. The type is lock_set_t. Use them when you need a lot of mutexes. In some cases they waste less system resources than arrays of gen_lock_t (e.g. sys v semaphores).

1.9.3.4.1. Allocating And Initializing
lock_set_t* lock_set_alloc( int no);  
int no;
 
lock_set_t* lock_set_init( lock_set_t* set);  
lock_set_t* set;
 

Both functions return 0 on failure.

Warning

Expect the allocation function to fail for large numbers. It depends on the locking method used and the system available resources (again the sysv semaphores example).

Example:

lock_set_t *lock_set;

lock_set=lock_set_alloc(100);
if (lock_set==0) goto error;
if (lock_set_init(lock_set)==0){
   lock_set_dealloc(lock_set);
   goto error;
}
1.9.3.4.2. Destroying And Deallocating
void lock_set_destroy( lock_set_t* s);  
lock_set_t* s;
 
void lock_set_dealloc( lock_set_t* s);  
lock_set_t* s;
 

Again don't forget to "destroy" the locks.

1.9.3.4.3. Locking And Unlocking
void lock_set_get( lock_set_t* sint i);  
lock_set_t* s int i ;
 
void lock_set_release( lock_set_t* sint i);  
lock_set_t* s int i ;
 

Example:

lock_set_get(lock_set, 2);
/* do something */
lock_set_release(lock_set, 2);
Revision History
Revision $Revision$ $Date$

1.10.1. Extend select framework with module related functions

If you want to extend functions which select framework can call with some dependent on module you have to follow next steps:

  • define working functions

  • create module's select parsing table

  • in module's mod_init call register_select_table

1.10.2. Define your working functions

Working functions are the piece of code, which is called as when SER needs get result of select function (defined as @name.foo[2].bar[5]). The working copy has following definition:

int select_function_name(str* res, struct select *s, struct sip_msg *msg) Pointer to the string result workspace res, don't allocate memory for the string, but use static buffer. There is no way to free the allocated memory. Pointer to the parsed select structure s. Holds all names, numbers divided by the dot and square bracket notation. Use that if any of the part used CONSUME_NEXT_STR or CONSUME_NEXT_INT to get the value. Pointer to the processed message structure msg.

FIXUP CALL: If you set FIXUP_CALL flag for the final function, the fixup call will be done immediatelly after function resolution. Such call is indicated with res==NULL && msg==NULL. Such call can convert any of the select's parameter into internal data (can hold one pointer), if you do that, set param_type to SEL_PARAM_DATA.

Result code of the function declares if the call was sucessful and if the result is valid string or empty string.

  • -1 error

  • 0 success, res contains non-empty string

  • 1 success, result of select call is empty string (res can be left unchanged)

1.10.3. Create module's select parsing table

Define static table of select_row_t type and initialize it directly.

The table is representation of tree (or oriented graph if you want), where first column represents current (up-to now) resolved function (starting with NULL), next two columns define if next parameter is integer or particullar string, next column is new resolved function which will be tested in next round and the last column is set of flags.

static select_row_t test_sel[] = {
	{ NULL, SEL_PARAM_STR, STR_STATIC_INIT("test"), select_test, 0},
	{ select_test, SEL_PARAM_STR, STR_STATIC_INIT("value"), select_test_val, 0},
	{ select_test, SEL_PARAM_STR, STR_STATIC_INIT("echo"), select_test_echo, CONSUME_NEXT_STR},
	{ NULL, SEL_PARAM_INT, STR_NULL, NULL, 0}
};

So in the previous example, the first line will accept syntax @test and set the resolved function to select_test. In the next round, all rows with the select_test in the first column will be used to match, so the next two lines are candidates to match depending on the next part of select syntax. If it matches @test.value, the function is resolved to select_test_val, if it matches @test.echo.anystring, it is resolved into select_test_echo. Flag CONSUME_NEXT_STR will accept any string at the 3rd position. As the OPTIONAL flag is not present, it won't accept just the @test.echo syntax.

The table ends with the NULL in the 1st and 4th column, other columns are not checked (the notation in the example suits well).

At the resolving time all function names must be already defined. For functions which are not leaves, you can use macro ABSTRACT_F(name) to define empty function, for working function you can use advance definition using SELECT_F(name) macro.

1.10.4. Register the created table

In the module initialization function call register_select_table(table) where table is the parsing tree/table you have defined in previous step. This call ensures, that the table will become part of the parsing logic for all select framework calls defined in the script file or called by another module's parse_select.

NOTE: The tables are inserted into the beginning of the list, so the core's table (and thus parseable names and definitions) can be overrided by module's function, if it is defined with the same name. To avoid such situation, the best practice is to start module's select with the module's name. E.g in our example code both select functions start with @test...

1.10.5. Example - module defining select extension

Example module test, which defines two select function.

  • @test.value - returns value passed as modules parameter "value"

  • @test.echo.xxx - returns xxx regardless of what you put as xxx (shows CONSUME_NEXT_STR option)

Example 15. test.c

#include <string.h>
#include "../../sr_module.h"
#include "../../str.h"
#include "../../dprint.h"
#include "../../select.h"

MODULE_VERSION

static cmd_export_t cmds[] = {
	{0, 0, 0, 0, 0}
};

static char* value=NULL;

static param_export_t params[] = {
	{"value", STR_PARAM, &value},
	{0, 0, 0}
};

int select_test_val(str* res, struct select* s, struct sip_msg* msg) {
	DBG("SELECT_TEST_VAL called test_val=%s\n", value);
	res->s=value;
	res->len=strlen(value);
	return 0;
}

int select_test_echo(str* res, struct select* s, struct sip_msg* msg) {
	DBG("SELECT_TEST_ECHO called\n");
	if (s->params[s->n-1].type==SEL_PARAM_STR) {
		*res=s->params[s->n-1].v.s;
		return 0;
	} else
		return -1;
}

ABSTRACT_F(select_test)
	
static select_row_t test_sel[] = {
	{ NULL, SEL_PARAM_STR, STR_STATIC_INIT("test"), select_test, 0},
	{ select_test, SEL_PARAM_STR, STR_STATIC_INIT("value"), select_test_val, 0},
	{ select_test, SEL_PARAM_STR, STR_STATIC_INIT("echo"), select_test_echo, CONSUME_NEXT_STR},
	{ NULL, SEL_PARAM_INT, STR_NULL, NULL, 0}
};
 
static int init(void) {
	register_select_table(test_sel);
	return 0;
};

struct module_exports exports = {
	"test", 
	cmds,           /* Exported commands */
	0,              /* RPC methods */
	params,         /* Exported parameters */
	init,           /* module initialization function */
	0,              /* response function*/
	0,              /* destroy function */
	0,              /* oncancel function */
	0               /* per-child init function */
};



[1] Module registration is a process when the core tries to find what functions and parameters are offered by the module.