Libevt Programming
Libevt Programming
21 Sep 2010 Building a modern server application requires a method of accepting hundreds, thousands, and even tens of thousands of events simultaneously, whether they are internal requests or network connections effectively handling their operation. There are many solutions available, but the libevent library and libev libraries have both revolutionized the performance and event handling capability. In this article, we will examine the basic structure and methods available for using and deploying these solutions within your UNIX applications. Both libev and libevent can be used in your high performance applications, including those deployed within the IBM Cloud or Amazon EC2 environment, where you need to support large numbers of simultaneous clients or operations.
Introduction
One of the biggest problems facing many server deployments, particularly web server deployments, is the ability to handle a large number of connections. Whether you are building cloud-based services to handle network traffic, distributing your application over IBM Amazon EC instances, or providing a high-performance component for your web site, you need to be able to handle a large number of simultaneous connections. A good example is the recent move to more dynamic web applications, especially those using AJAX techniques. If you are deploying a system that allows many thousands of clients to update information directly within a web page, such as a system providing live monitoring of an event or issue, then the speed at which you
Boost network performance with libevent and libev Copyright IBM Corporation 2010. All rights reserved. Trademarks Page 1 of 16
developerWorks
ibm.com/developerWorks
can effectively serve the information is vital. In a grid or cloud situation, you might have permanent open connections from thousands of clients simultaneously, and you need to be able to serve the requests and responses to each client. Before looking at how libevent and libev are able to handle multiple network connections, let's take a brief look at some of the traditional solutions for handling this type of connectivity.
ibm.com/developerWorks
developerWorks
interaction. The key is that the loop and network sockets need a lot of management code to ensure that you are listening, updating, and controlling the different connections and interfaces. An alternative method of handling many different connections is to make use of the multi-threading support in most modern kernels to listen and handle connections, opening a new thread for each connection. This shifts the responsibility back to the operating system directly but implies a relatively large overhead in terms of RAM and CPU, as each thread will need it's own execution space. And if each thread (ergo network connection) is busy, then the context switching to each thread can be significant. Finally, many kernels are not designed to handle such a large number of active threads.
Boost network performance with libevent and libev Copyright IBM Corporation 2010. All rights reserved.
Trademarks Page 3 of 16
developerWorks
ibm.com/developerWorks
registering a callback function each time the accept() function needs to be called to open a new connect
int main(int argc, char **argv) { ... ev_init(); /* Setup listening socket */ event_set(&ev_accept, listen_fd, EV_READ|EV_PERSIST, on_accept, NULL); event_add(&ev_accept, NULL); /* Start the event loop. */ event_dispatch(); }
The event_set() function creates the new event structure, while event_add() adds the event to the event queue mechanism. The event_dispatch() then start the event queue system and starts listening (and accepting) requests. A more complete example if provided in Listing 2, which builds a very simple echo server: Listing 2. Building a simple echo server
#include #include #include #include #include #include #include #include #include #include <event.h> <sys/types.h> <sys/socket.h> <netinet/in.h> <arpa/inet.h> <string.h> <stdlib.h> <stdio.h> <fcntl.h> <unistd.h>
#define SERVER_PORT 8080 int debug = 0; struct client { int fd; struct bufferevent *buf_ev; }; int setnonblock(int fd) { int flags; flags = fcntl(fd, F_GETFL); flags |= O_NONBLOCK; fcntl(fd, F_SETFL, flags); } void buf_read_callback(struct bufferevent *incoming, void *arg) { struct evbuffer *evreturn; char *req;
Boost network performance with libevent and libev Copyright IBM Corporation 2010. All rights reserved.
Trademarks Page 4 of 16
ibm.com/developerWorks
developerWorks
req = evbuffer_readline(incoming->input); if (req == NULL) return; evreturn = evbuffer_new(); evbuffer_add_printf(evreturn,"You said %s\n",req); bufferevent_write_buffer(incoming,evreturn); evbuffer_free(evreturn); free(req); } void buf_write_callback(struct bufferevent *bev, void *arg) { } void buf_error_callback(struct bufferevent *bev, short what, void *arg) { struct client *client = (struct client *)arg; bufferevent_free(client->buf_ev); close(client->fd); free(client); } void accept_callback(int fd, short ev, void *arg) { int client_fd; struct sockaddr_in client_addr; socklen_t client_len = sizeof(client_addr); struct client *client; client_fd = accept(fd, (struct sockaddr *)&client_addr, &client_len); if (client_fd < 0) { warn("Client: accept() failed"); return; } setnonblock(client_fd); client = calloc(1, sizeof(*client)); if (client == NULL) err(1, "malloc failed"); client->fd = client_fd; client->buf_ev = bufferevent_new(client_fd, buf_read_callback, buf_write_callback, buf_error_callback, client); bufferevent_enable(client->buf_ev, EV_READ); } int main(int argc, char **argv) { int socketlisten; struct sockaddr_in addresslisten; struct event accept_event; int reuse = 1; event_init();
Boost network performance with libevent and libev Copyright IBM Corporation 2010. All rights reserved.
Trademarks Page 5 of 16
developerWorks
ibm.com/developerWorks
socketlisten = socket(AF_INET, SOCK_STREAM, 0); if (socketlisten < 0) { fprintf(stderr,"Failed to create listen socket"); return 1; } memset(&addresslisten, 0, sizeof(addresslisten)); addresslisten.sin_family = AF_INET; addresslisten.sin_addr.s_addr = INADDR_ANY; addresslisten.sin_port = htons(SERVER_PORT); if (bind(socketlisten, (struct sockaddr *)&addresslisten, sizeof(addresslisten)) < 0) { fprintf(stderr,"Failed to bind"); return 1; } if (listen(socketlisten, 5) < 0) { fprintf(stderr,"Failed to listen to socket"); return 1; } setsockopt(socketlisten, SOL_SOCKET, SO_REUSEADDR, &reuse, sizeof(reuse)); setnonblock(socketlisten); event_set(&accept_event, socketlisten, EV_READ|EV_PERSIST, accept_callback, NULL); event_add(&accept_event, NULL); event_dispatch(); close(socketlisten); return 0; }
The different functions and their operation are discussed below: main(): The main function creates the socket to be used for listening to connections, and then creates the callback for accept() to handle each connection through the event handler. accept_callback(): The function called by the event system when a connection is accepted. The function accepts the connection to the client; adds the client socket information and a bufferevent structure; adds callbacks for read/write/error events on the client socket to the event
Boost network performance with libevent and libev Copyright IBM Corporation 2010. All rights reserved. Trademarks Page 6 of 16
ibm.com/developerWorks
developerWorks
structure; and passes the client structure (with the embedded eventbuffer and client socket) as an argument. Each time a corresponding client socket contains any read, write or error operations, the corresponding callback function is called. buf_read_callback(): Called when the client socket has data to be read. As an echo service, the function writes "you said..." back to the client. The socket remains open to accept new requests. buf_write_callback(): Called when there is data to be written. In this simple service you don't need this service, and so the definition is blank. buf_error_callback(): Called when an error condition exists. This includes when the client disconnects. In all situations the client socket is closed, and the event entry for the client socket is removed from the event list. The memory for the client structures is freed. setnonblock(): Sets the network socket to non-blocking I/O. As the number of clients connect, new events to handle the client connection are added to the event queue and removed when the client disconnects. Behind the scenes, libevent is handling the network sockets, identifying which clients need to be serviced, and calling the corresponding functions in each case. To build the application, compile the C source code adding the libevent library: $ gcc -o basic basic.c -levent. From a client perspective, the server just echoes back any text sent to it (see Listing 3 below). Listing 3. Server echoes back text sent to it
$ telnet localhost 8080 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. Hello! You said Hello!
Network applications like this can be useful with large scale distributed deployments, such as an IBM Cloud system, where you need to service multiple connections. It is difficult with such a simple solution to see the performance benefits and the large number of simultaneous connections. Using the embedded HTTP implementation can help to understand the mass scalability.
developerWorks
ibm.com/developerWorks
The plain network-based libevent interface is useful if you want to build native applications, but it is increasingly common to develop an application based around the HTTP protocol and a web page that loads, or more commonly dynamically reloads, information. If you are using any of the AJAX libraries, it expects an HTTP at the other end, even if the information you are returning is XML or JSON. The HTTP implementation within libevent is not going to replace Apache's HTTP server, but it can be a practical solution for the sort of large-scale dynamic content associated with both cloud and web environments. For example, you could deploy a libevent based interface to your IBM Cloud management or other solution. Since you can communicate using HTTP, the server can integrate with the other components. To use the libevent service, you use the same basic structure as already described for the main network event model, but instead of having to handle the network interfacing, the HTTP wrapper handles that for you. This turns the entire process into the four function calls (initialize, start HTTP server, set HTTP callback function, and enter event loop), plus the contents of the callback function that will send data back. A very simple example is provided in Listing 4: Listing 4. Simple example of using the libevent service
#include <sys/types.h> #include <stdio.h> #include <stdlib.h> #include <unistd.h> #include <event.h> #include <evhttp.h> void generic_request_handler(struct evhttp_request *req, void *arg) { struct evbuffer *returnbuffer = evbuffer_new(); evbuffer_add_printf(returnbuffer, "Thanks for the request!"); evhttp_send_reply(req, HTTP_OK, "Client", returnbuffer); evbuffer_free(returnbuffer); return; } int main(int argc, char **argv) { short http_port = 8081; char *http_addr = "192.168.0.22"; struct evhttp *http_server = NULL; event_init(); http_server = evhttp_start(http_addr, http_port); evhttp_set_gencb(http_server, generic_request_handler, NULL); fprintf(stderr, "Server started on port %d\n", http_port); event_dispatch(); return(0); }
Boost network performance with libevent and libev Copyright IBM Corporation 2010. All rights reserved.
Trademarks Page 8 of 16
ibm.com/developerWorks
developerWorks
Given the previous example, the basics of the code here should be relatively self-explanatory. The main elements are the evhttp_set_gencb() function, which sets the callback function to be used when an HTTP request is received, and the generic_request_handler() callback function itself, which populates the response buffer with a simple message to show success. The HTTP wrapper provides a wealth of different functionality. For example, there is a request parser that will extract the query arguments from a typical request (as you would use in a CGI request), and you can also set different handlers to be triggered within different requested paths. An interface to your database could be provided using the path '/db/', or an interface through to memcached as '/memc', with different callbacks and handling accordingly. One other element of the libevent toolkit is support for generic timers. These allow you to schedule events after a specific period. You can combine this with the HTTP implementation to provide a lightweight service to serve up the contents of a file, updating the data returned as the file content is modified. For example, if you were providing a live update service during a busy news event where the front-end web application kept periodically reloading the news item, you could easily serve up the content. The entire application (and web service) would be in memory making the response times very quick. This is the main purpose behind the example in Listing 5: Listing 5. Using a generic timer to provide a live update service during a busy news event
#include #include #include #include #include #include #include #include <sys/types.h> <stdio.h> <stdlib.h> <string.h> <unistd.h> <sys/stat.h> <event.h> <evhttp.h>
#define RELOAD_TIMEOUT 5 #define DEFAULT_FILE "sample.html" char *filedata; time_t lasttime = 0; char filename[80]; int counter = 0; void read_file() { int size = 0; char *data; struct stat buf; stat(filename,&buf); if (buf.st_mtime > lasttime) { if (counter++)
Boost network performance with libevent and libev Copyright IBM Corporation 2010. All rights reserved.
Trademarks Page 9 of 16
developerWorks
ibm.com/developerWorks
fprintf(stderr,"Reloading file: %s",filename); else fprintf(stderr,"Loading file: %s",filename); FILE *f = fopen(filename, "rb"); if (f == NULL) { fprintf(stderr,"Couldn't open file\n"); exit(1); } fseek(f, 0, SEEK_END); size = ftell(f); fseek(f, 0, SEEK_SET); data = (char *)malloc(size+1); fread(data, sizeof(char), size, f); filedata = (char *)malloc(size+1); strcpy(filedata,data); fclose(f); fprintf(stderr," (%d bytes)\n",size); lasttime = buf.st_mtime; } } void load_file() { struct event *loadfile_event; struct timeval tv; read_file(); tv.tv_sec = RELOAD_TIMEOUT; tv.tv_usec = 0; loadfile_event = malloc(sizeof(struct event)); evtimer_set(loadfile_event, load_file, loadfile_event); evtimer_add(loadfile_event, &tv); } void generic_request_handler(struct evhttp_request *req, void *arg) { struct evbuffer *evb = evbuffer_new(); evbuffer_add_printf(evb, "%s",filedata); evhttp_send_reply(req, HTTP_OK, "Client", evb); evbuffer_free(evb); } int main(int argc, char *argv[]) { short http_port = 8081; char *http_addr = "192.168.0.22"; struct evhttp *http_server = NULL; if (argc > 1) { strcpy(filename,argv[1]); printf("Using %s\n",filename); } else { strcpy(filename,DEFAULT_FILE);
Boost network performance with libevent and libev Copyright IBM Corporation 2010. All rights reserved.
Trademarks Page 10 of 16
ibm.com/developerWorks
developerWorks
} event_init(); load_file(); http_server = evhttp_start(http_addr, http_port); evhttp_set_gencb(http_server, generic_request_handler, NULL); fprintf(stderr, "Server started on port %d\n", http_port); event_dispatch(); }
The basic mechanics of the server are the same as the previous example. First, the script sets up an HTTP server which will just respond to requests on the basic URL host/port combination (no processing of the request URI). The first step is to load the file (read_file()). The same function is used to load the original and will be used during the callback by the timer event. The read_file() function uses the stat() function call to check the modification time of the file, only re-reading the contents of the file if it changed since the last time the file was loaded. The function loads the file data using a single call to fread(), copying the data into a separate structure, before using a strcpy() to move the data from the loaded string to the global string. The load_file() function is the one that will act as the function when the timer is triggered. It calls read_file() to load the content, and then sets the timer using the RELOAD_TIMEOUT value as the number of seconds before the file load is attempted. The libevent timer uses the timeval structure, which allows timers to be specified in both seconds and microseconds. The timer is not periodic; you set it when the timer event is triggered, and then the event is removed from the event queue. To compile, use the same format as the previous examples: $ gcc -o basichttpfile basichttpfile.c -levent Now create a static file to be used as the data; the default is sample.html, but you can specify any file as the first argument on the command-line (see Listing 6 below). Listing 6. Create a static file to be used as the data
$ ./basichttpfile Loading file: sample.html (8046 bytes) Server started on port 8081
At this point, the program is ready to accept requests, but the reload timer is also in operation. If you change the content of sample.html, the file should automatically be reloaded with a message logged. For example, the output in Listing 7 shows the initial load and two reloads:
Boost network performance with libevent and libev Copyright IBM Corporation 2010. All rights reserved.
Trademarks Page 11 of 16
developerWorks
ibm.com/developerWorks
Note that to get the full benefit, you must ensure that your environment does not have an ulimit on the number of open file descriptors. You can change this (with suitable permissions or root access) using the ulimit command. The exact setting will depend on your OS, but on Linux you can set the number of open file descriptors (and therefore network sockets) with the -n option: Listing 8. Using the -n option to set the number of open file descriptors
$ ulimit -n 1024
To increase your limit, specify a figure: $ ulimit -n 20000. To check the performance of the server, you can use a benchmarking application, such as Apache Bench 2 (ab2). You can specify the number of simultaneous queries, as well as total number of requests. For example, to run a benchmark using 100,000 requests, 1000 of them simultaneously: $ ab2 -n 100000 -c 1000 http://192.168.0.22:8081/. Running this sample system, using the 8K file shown in the server sample, I achieved almost 11,000 requests/s. Keep in mind that the libevent server is running in a single thread, and a single client is unlikely to stress the server, since it will also be limited by the method of opening requests. Even so, that rate is impressive for a single threaded application given the comparative size of the document being exchanged.
Boost network performance with libevent and libev Copyright IBM Corporation 2010. All rights reserved.
Trademarks Page 12 of 16
ibm.com/developerWorks
developerWorks
The library implementations in these languages tend to support the core of the libevent system and not always the HTTP wrapper. Using these solutions with a scripted application is therefore made more complex. There are two routes: either embed the language into your C-based libevent application or use one of the many HTTP implementations built on top of the scripted language environment. For example, Python includes the very capable HTTP server class (httplib/httplib2). Despite this functionality, it should be pointed out that there is nothing in a scripting language that cannot be re-implemented in C. However, there is a time consideration and integrating with your existing codebase may be more critical.
Boost network performance with libevent and libev Copyright IBM Corporation 2010. All rights reserved.
Trademarks Page 13 of 16
developerWorks
ibm.com/developerWorks
require 'rubygems' require 'rev' PORT = 8081 class EchoServerConnection < Rev::TCPSocket def on_read(data) write 'You said: ' + data end end server = Rev::TCPServer.new('192.168.0.22', PORT, EchoServerConnection) server.attach(Rev::Loop.default) puts "Listening on localhost:#{PORT}" Rev::Loop.default.run
The Ruby implementation is particularly nice, since wrappers have been provided for many common network solutions, including HTTP client, OpenSSL and DNS. Other scripted languages include a comprehensive Perl and Python implementation that you might want to try.
Summary
Both libevent and libev provide a flexible and powerful environment for supporting high-volume network (and other I/O) interfaces for servicing either server-side or client-side requests. The aim is to support thousands, and even tens of thousands, of connections in an efficient (low CPU/RAM) format. In this article, you have seen examples of this, including the built-in HTTP service in libevent, that can be used to support IBM Cloud, EC2 or AJAX-based web applications.
Boost network performance with libevent and libev Copyright IBM Corporation 2010. All rights reserved.
Trademarks Page 14 of 16
ibm.com/developerWorks
developerWorks
Resources
Learn The C10K problem provides an excellent overview of the issues of handling 10,000 connections. The IBM Cloud Computing web site provides information on different cloud implementations. Read System Administration Toolkit: Standardizing your UNIX command-line tools (Martin Brown, developerWorks, May 2006): Learn how to use the same command across multiple machines. For an article series that will teach you how to program in bash, see Bash by example, Part 1: Fundamental programming in the Bourne again shell (bash) (Daniel Robbins, developerWorks, March 2000), Bash by example, Part 2: More bash programming fundamentals (Daniel Robbins, developerWorks, April 2000), and Bash by example, Part 3: Exploring the ebuild system (Daniel Robbins, developerWorks, May 2000). Making UNIX and Linux work together (Martin Brown, developerWorks, April 2006): A guide to getting traditional UNIX distributions and Linux working together. Navigate the cloud computing labyrinth (Brett McLaughlin, developerWorks, March 2009): Helps you make an educated decision about the best cloud computing platform for your particular application requirements. Read Cloud Computing with Amazon Web Services (Prabhakar Chaganti, developerWorks, July 2008): A step-by-step guide to using Amazon Web Services. IBM products are available for the Amazon EC2 platform from the developerWorks Cloud Computing Resource Center. New to AIX and UNIX: Visit the New to AIX and UNIX page to learn more about AIX and UNIX. The developerWorks AIX and UNIX zone hosts hundreds of informative articles and introductory, intermediate, and advanced tutorials. Listen to interesting interviews and discussions for software developers on developerWorks podcasts. developerWorks technical events and webcasts: Stay current with developerWorks technical events and webcasts. Get products and technologies Get the libev library, including download and documentation.
Boost network performance with libevent and libev Copyright IBM Corporation 2010. All rights reserved. Trademarks Page 15 of 16
developerWorks
ibm.com/developerWorks
Get the libevent library. The ruby libev (rev) library and documentation. Memcached is an RAM cache for storing and handling data (which uses libevent at it's core, as well as being used with other libevent servers). Innovate your next open source development project with IBM trial software, available for download or on DVD. Discuss Participate in developerWorks blogs and get involved in the developerWorks community. Follow developerWorks on Twitter. Get involved in the My developerWorks community. Participate in the AIX and UNIX forums: AIX Forum AIX Forum for developers Cluster Systems Management IBM Support Assistant Forum Performance Tools Forum Virtualization Forum More AIX and UNIX Forums
Boost network performance with libevent and libev Copyright IBM Corporation 2010. All rights reserved.
Trademarks Page 16 of 16