An Introduction to OpenSSL Programming (Part I)

Eric Rescorla
RTFM, Inc.
ekr@rtfm.com
Version 1.0: October 5, 2001

1 Introduction
The quickest and easiest way to secure a TCP-based network application is with SSL. If you’re
working in C, your best choice is probably to use OpenSSL, (the web site is at
http://www.openssl.org/). OpenSSL is a free (BSD-style license) implementation of
SSL/TLS based on Eric Young’s SSLeay package. Unfortunately, the documentation and sample
code distributed with OpenSSL leaves something to be desired. Where they exist, the manual
pages are pretty good, but they often miss the big picture, as manual pages are intended as a ref-
erence, not a tutorial.
We provide an introduction to OpenSSL programming. The OpenSSL API is vast and com-
plicated so we don’t attempt to provide anything like complete coverage. Rather, the idea is to
teach you enough to work effectively from the manual pages. In this article, the first of two, we
build a simple Web client and server pair that demonstrates the basic features of OpenSSL. In
the second article we introduce a number of advanced features, such as session resumption and
client authentication.
We assume that you’re already familiar with SSL and HTTP at least at a conceptual level. If
you’re not, a good place to start is with the RFCs (see the end of this article for references).

2 Source Code
For space reasons, this article only includes excerpts from the source code. The complete source
code is available in machine-readable format from the author’s web site at
http://www.rtfm.com/openssl-examples/

3 Our Programs
Our client is a simple HTTPS (see RFC 2818) client. It initiates an SSL connection to the server
and then transmits an HTTP request over that connection. It then waits for the response from the
server and prints it to the screen. This is a vastly simplified version of the functionality found in
programs like fetch and cURL.
The server program is a simple HTTPS server. It waits for TCP connections from clients.
When it accepts one it negotiates an SSL connection. Once the connection is negotiated, it reads
the client’s HTTP request. It then transmits the HTTP response to the client. Once the response
is transmitted it closes the connection.

3.1 Context Initialization

Our first task is to set up a context object (an SSL_CTX). This context object is then used to cre-
ate a new connection object for each new SSL connection. It is these connection objects which
are used to do SSL handshakes, reads, and writes.
This approach has two advantages. First, the context object allows many structures to be ini-
tialized only once, improving performance. In most applications, every SSL connection will use
This article is Copyright © 2001 Eric Rescorla. It may be redistributed for any purpose and without fee provided
that this notice is retained. An earlier version of this article first appeared in the September 2001 issue of Linux
Journal.
the same keying material, certificate authority (CA) list, etc. Rather than reloading this material
for every connection, we simply load it into the context object at program startup. When we
wish to create a new connection, we can simply point that connection to the context object. The
second advantage of having a single context object is that it allows multiple SSL connections to
share data, such as the SSL session cache used for session resumption.
Context initialization consists of four primary tasks, all performed by the initial-
ize_ctx() function, shown in Figure 1.

42 SSL_CTX *initialize_ctx(keyfile,password)
43 char *keyfile;
44 char *password;
45 {
46 SSL_METHOD *meth;
47 SSL_CTX *ctx;
48
49 if(!bio_err){
50 /* Global system initialization*/
51 SSL_library_init();
52 SSL_load_error_strings();
53
54 /* An error write context */
55 bio_err=BIO_new_fp(stderr,BIO_NOCLOSE);
56 }
57
58 /* Set up a SIGPIPE handler */
59 signal(SIGPIPE,sigpipe_handle);
60
61 /* Create our context*/
62 meth=SSLv23_method();
63 ctx=SSL_CTX_new(meth);
64
65 /* Load our keys and certificates*/
66 if(!(SSL_CTX_use_certificate_chain_file(ctx,
67 keyfile)))
68 berr_exit("Can’t read certificate file");
69
70 pass=password;
71 SSL_CTX_set_default_passwd_cb(ctx,
72 password_cb);
73 if(!(SSL_CTX_use_PrivateKey_file(ctx,
74 keyfile,SSL_FILETYPE_PEM)))
75 berr_exit("Can’t read key file");
76
77 /* Load the CAs we trust*/
78 if(!(SSL_CTX_load_verify_locations(ctx,
79 CA_LIST,0)))
80 berr_exit("Ca’t read CA list");
81 #if (OPENSSL_VERSION_NUMBER < 0x0090600fL)
82 SSL_CTX_set_verify_depth(ctx,1);
83 #endif
84
85 return ctx;
86 }

Figure 1 initialize_ctx()
Initialize the library and create the context
Before OpenSSL can be used for anything, the library as a whole must be initialized. This is
accomplished with SSL_library_init(), which primarily loads up the algorithms that
OpenSSL will be using. If we want good reporting of errors, we also need to load the error
strings using SSL_load_error_strings(). Otherwise, we won’t be able to map
OpenSSL errors to strings.
We also create an object to be used as an error printing context. OpenSSL uses an abstrac-
tion called a BIO object for input and output. This allows the programmer to use the same func-
tions for different kinds of I/O channels (sockets, terminal, memory buffers, etc.) merely by
using different kinds of BIO objects. In this case we create a BIO object attached to stderr to be
used for printing errors.

Load our own keys

If you’re writing server--or a client which is able to perform client authentication--you’ll need to
load your own public/private key pair and the associated certificate. The certificate is stored in
the clear and is loaded together with the CA certificates forming the certificate chain using
SSL_CTX_use_certificate_chain_file(). We use SSL_CTX_use_Private-
Key_file() to load the private key. For security reasons, the private key is usually encrypted
under a password. If the key is encrypted, the password callback (set using
SSL_CTX_set_default_passwd_cb()) will be called to obtain the password.

Load root CA list

If you’re going to be authenticating the host you’re connected to OpenSSL needs to know what
certificate authorities (CAs) you trust. The SSL_CTX_load_verify_locations() call is
used to load the CAs.

Load Randomness
In order to have good security, SSL needs a good source of strong random numbers. In general,
it’s the responsibility of the application to supply some seed material for the random number
generator. However, OpenSSL automatically uses /dev/urandom to seed the RNG, if it is
available. Since /dev/urandom is standard on Linux, we don’t have to do anything for
this--which is convenient since gathering random numbers is tricky and easy to screw up. Note
that if you’re on a system other than Linux you may get an error at some point because the ran-
dom number generator is unseeded. OpenSSL’s rand(3) manual page provides more informa-
tion.

4 The Client
Once the client has initialized the SSL context, it’s ready to connect to the server. OpenSSL
requires us to create a TCP connection between client and server on our own and then use the
TCP socket to create an SSL socket. For convenience we’ve isolated the creation of the TCP
connection to the tcp_connect() function (which is not shown here but is available in the
downloadable source).
Once the TCP connection has been created, we create an SSL object to handle the connec-
tion. This object needs to be attached to the socket. Note that that we don’t directly attach the
SSL object to the socket. Rather, we create a BIO object using the socket and then attach the
SSL object to the BIO.
This abstraction layer allows you to use OpenSSL over channels other than sockets, pro-
vided you have an appropriate BIO. For instance, one of the OpenSSL test programs connects
an SSL client and server purely through memory buffers. A more practical use would be to sup-
port some protocol that can’t be accessed via sockets. For instance, you could run SSL over a
serial line.
Handshake
The first step in an SSL connection is to perform the SSL handshake. The handshake authenti-
cates the server (and optionally the client) and establishes the keying material that will be used
to protect the rest of the traffic. The SSL_connect() call is used to perform the SSL hand-
shake. Because we’re using blocking sockets, SSL_connect() will not return until the hand-
shake is completed or an error has been detected. SSL_connect() returns 1 for success and 0
or negative for an error. This call is shown in Figure 2.

114 /* Connect the TCP socket*/

115 sock=tcp_connect(host,port);
116
117 /* Connect the SSL socket */
118 ssl=SSL_new(ctx);
119 sbio=BIO_new_socket(sock,BIO_NOCLOSE);
120 SSL_set_bio(ssl,sbio,sbio);
121 if(SSL_connect(ssl)<=0)
122 berr_exit("SSL connect error");
123 if(require_server_auth)
124 check_cert(ssl,host);

Figure 2 connecting to the server

Server Authentication
When we initiate an SSL connection to a server, we need to check the server’s certificate chain.
OpenSSL does some of the checks for us but unfortunately some of the checks are application
specific and so we have to do those ourselves. The primary test that our sample application does
is to check the server identity. This check is performed by the check_cert() function, shown
in Figure 3.

31 void check_cert(ssl,host)
32 SSL *ssl;
33 char *host;
34 {
35 X509 *peer;
36 char peer_CN[256];
37
38 if(SSL_get_verify_result(ssl)!=X509_V_OK)
39 berr_exit("Certificate doesn’t verify");
40
41 /*Check the cert chain. The chain length
42 is automatically checked by OpenSSL when
43 we set the verify depth in the ctx */
44
45 /*Check the common name*/
46 peer=SSL_get_peer_certificate(ssl);
47 X509_NAME_get_text_by_NID
48 (X509_get_subject_name(peer),
49 NID_commonName, peer_CN, 256);
50 if(strcasecmp(peer_CN,host))
51 err_exit
52 ("Common name doesn’t match host name");
53 }

Figure 3 check_cert() function

Server Identity
Once you’ve established that the server’s certificate chain is valid, you need to verify that the
certificate you’re looking at matches the identity that we expect the server to have. In most
cases, this means that the server’s DNS name appears in the certificate, either in the Common
Name field of the Subject Name or in a certificate extension. Although each protocol has slightly
different rules for verifying the server’s identity, RFC 2818 contains the rules for HTTP over
SSL/TLS. Following RFC 2818 is generally a good idea unless you have some explicit reason to
do otherwise. to
Since most certificates still contain the domain name in Common Name rather than an
extension, we show only the Common Name check. We simply extract the server’s certificate
using SSL_get_peer_certificate() and then compare the common name to the host
name we’re connecting to. If they don’t match, something is wrong and we exit.

Chain Length
Before version 0.9.5, OpenSSL was subject to a certificate extension attack. To see what this is,
consider the case where you a server authenticates with a certificate signed by Bob, as shown in
Figure 4. Bob isn’t one of your CAs but his certificate is signed by a CA you trust.

Root
CA

Bob

Server

Figure 4 An extended certificate chain

If you accept this certificate you’re likely going to be in a lot of trouble. The fact that the CA
signed Bob’s certificate means that the CA believes that it has verified Bob’s identity, not that
Bob can be trusted. If you know that you want to do business with Bob, that’s fine, but it’s not
very useful if you want to do business with Alice and Bob (who you’ve never heard of) is vouch-
ing for Alice.
Originally, the only way to protect yourself against this sort of attack was to restrict the
length of certificate chains so that you knew that the certificate you’re looking at was signed by
the CA. X.509 version 3 contains a way for a CA to label certain certificates as other CAs. This
permits a CA to have a single root that then certifies a bunch of subsidiary CAs.
Modern versions of OpenSSL (0.9.5 and later) check these extensions so you’re automati-
cally protected against extension attacks whether or not you check chain length. Versions prior
to 0.9.5 do not check the extensions at all, so you have to enforce the chain length if using an
older version. 0.9.5 has some problems with checking so if you’re using 0.9.5, you should prob-
ably upgrade. The #ifdef-ed code in initialize_ctx() provides chain length checking
with older versions. We use the SSL_CTX_set_verify_depth() to force OpenSSL to
check the chain length. In summary, it’s highly advisable to upgrade to 0.9.6, particularly since
longer (but properly constructed) chains are becoming more popular.

When To Check Certicates

The code shown here allows the entire handshake to complete, whether or not the peer has a
valid certificate. It’s important to note that this is not the only way to check the server’s
certificate. Only after the handshake completes do we check the certificate and possibly termi-
nate the connection. This approach has the advantage that it’s easy to implement but the disad-
vantage that if we don’t like the server’s certificate the server just sees a generic error rather than
one indicating that the certificate was bad.
An alternative is to use the SSL_CTX_set_verify() API call to have OpenSSL require
valid certificates during the handshake—if the server presents an invalid certificate the hand-
shake will fail and the server will receive an error indicating that it had a bad certificate. You can
also use SSL_CTX_set_verify() to set a callback to be called during certificate checking,
allowing you to apply extra checks or permit certificates which OpenSSL might otherwise not
accept.

Request
We use the code shown in Figure 5 to write the HTTP request. For demonstration purposes, we
use a more-or-less hardwired HTTP request, found in the REQUEST_TEMPLATE variable.
Because the machine we’re connecting to may change we do need to fill in the Host header.
This is done using the sprintf() in line 30. We then use SSL_write() to send the data to
the server. SSL_write()as write(), except that we pass in the SSL object instead of the
file descriptor.
Experienced TCP programmers will notice that instead of looping around the write we
throw an error if the return value doesn’t equal the value we’re trying to write. In blocking
mode, SSL_write() semantics are ’all-or-nothing’: the call won’t return until all the data is
written or an error occurs, whereas write() may only write part of the data.

Note: The SSL_MODE_ENABLE_PARTIAL_WRITE flag (not used here) enables partial

writes, in which case you’d need to loop.

Response
In old-style HTTP/1.0, the server transmits its response and then closes the connection. In later
versions, persistent connections that allow multiple sequential transactions on the same connec-
tion were introduced. For convenience and simplicity we will not use persistent connections. We
omit the header that allows them, causing the server to use a connection close to signal the end
of the response. Operationally, this means that we can keep reading until we get an end of file,
which simplifies matters considerably.
OpenSSL uses the SSL_read() API call to read data, as shown in Figure 6. As with
read() we simply choose an appropriate sized buffer and pass it to SSL_read(). Note that
the buffer size isn’t really that important here. The semantics of SSL_read(), like the seman-
tics of read(), are that it returns the data available, even if it’s less than the requested amount.
On the other hand, if no data is available, then the call to read blocks.
The choice of BUFSIZZ, then, is basically a performance tradeoff. The tradeoff is quite dif-
ferent here than when we’re simply reading from normal sockets. In that case, each call to
read() requires a context switch into the kernel. Because context switches are expensive, pro-
grammers try to use large buffers to reduce them. However, when we’re using SSL the number
of calls to read()—and hence context switches—is largely determined by the number of
records the data was written in rather than the number of calls to SSL_read().
26 request_len=strlen(REQUEST_TEMPLATE)+
27 strlen(host)+6;
28 if(!(request=(char *)malloc(request_len)))
29 err_exit("Couldn’t allocate request");
30 sprintf(request,REQUEST_TEMPLATE,
31 host,port);
32
33 /* Find the exact request_len */
34 request_len=strlen(request);
35
36 r=SSL_write(ssl,request,request_len);
37 switch(SSL_get_error(ssl,r)){
38 case SSL_ERROR_NONE:
39 if(request_len!=r)
40 err_exit("Incomplete write!");
41 break;
42 default:
43 berr_exit("SSL write problem");
44 }

Figure 5 writing the HTTP request

For instance, if the client wrote a 1000-byte record and we call SSL_read() in chunks of 1
byte, then the first call to SSL_read() will result in the record being read
in and the rest of the calls will just read it out of the SSL buffer. Thus, the choice of buffer size
is less significant when we’re using SSL than with normal sockets.

Note that if the data were written in a series of small records, you might want to read
all of them at once with a single call to read(). OpenSSL provides a flag
SSL_CTRL_SET_READ_AHEAD that turns on this behavior.

48 while(1){
49 r=SSL_read(ssl,buf,BUFSIZZ);
50 switch(SSL_get_error(ssl,r)){
51 case SSL_ERROR_NONE:
52 len=r;
53 break;
54 case SSL_ERROR_ZERO_RETURN:
55 goto shutdown;
56 case SSL_ERROR_SYSCALL:
57 fprintf(stderr,
58 "SSL Error: Premature close0);
59 goto done;
60 default:
61 berr_exit("SSL read problem");
62 }
63
64 fwrite(buf,1,len,stdout);
65 }

Figure 6 reading the response

Note the use of the switch on the return value of SSL_get_error() in line 12. The con-
vention with normal sockets is that any negative number (typically −1) indicates failure and that
one then checks errno to determine what actually happened. Obviously errno won’t work
here since that only shows system errors and we’d like to be able to act on SSL errors. Also,
errno requires careful programming in order to be thread safe.
Instead of errno, OpenSSL provides the SSL_get_error() call. This call lets us
examine the return value and figure out whether an error occurred and what it was. If the return
value was positive, we’ve read some data and we simply write it to the screen. A real client
would of course parse the HTTP response and either display the data (e.g. a web page) or save it
to disk. However, none of this is interesting as far as OpenSSL is concerned so we don’t show
any of it.
If the return value was zero, this does not mean that there was no data available. In that case,
we would have blocked as discussed above. Rather, it means that the socket is closed and there
will never be any data available to read. Thus, we exit the loop.

Error Handling
If the return value was something negative then some kind of error occurred. There are two
kinds of errors we’re concerned with: ordinary errors and "premature closes". We use the
SSL_get_error() call to determine which kind of error we have. Error handling in our
client is pretty primitive so with most errors we simply call berr_exit() to print an error
message and exit. Premature closes have to be handled specially.

Closure
TCP uses a FIN segment to indicate that the sender has sent all of its data. SSL version 2 simply
allowed either side to send a TCP FIN to terminate the SSL connection. This allowed for a
"truncation attack": the attacker could make it appear that a message was shorter than it was
simply by forging a TCP FIN. Unless the victim had some other way of knowing what message
length to expect it would simply believe that it had received a shorter message.
In order to prevent this security problem, SSLv3 introduced a close_notify alert. The
close_notify is an SSL message (and therefore secured) but is not part of the data stream itself
and so is not seen by the application. No data may be transmitted after the close_notify is sent.
Thus, when SSL_read() returns 0 to indicate that the socket has been closed, this really
means that the close_notify has been received. If the client receives a FIN before receiving a
close_notify, SSL_read() will return with an error. This is called a "premature close".
A naive client might decide to report an error and exit whenever it received a premature
close. This is the behavior that is implied by the SSLv3 specification. Unfortunately, sending
premature closes is a rather common error, particularly common with clients. Thus, unless you
want to be reporting errors all the time you often have to ignore premature closes. Our code
splits the difference. It reports the premature close on stderr but doesn’t exit with an error.

Shutdown
If we read the response without any errors then we need to send our own close_notify to the
server. This is done using the SSL_shutdown() API call. We’ll cover SSL_shutdown()
more completely when we talk about the server but the general idea is simple: it returns 1 for a
complete shutdown, 0 for an incomplete shutdown, and -1 for an error. Since we’ve already
received the server’s close_notify, about the only thing that can go wrong is that we have trouble
sending our close_notify. Otherwise SSL_shutdown() will succeed (returning 1).

Cleanup
Finally, we need to destroy the various objects we’ve allocated. Since this program is about to
exit, freeing the objects automatically, this isn’t strictly necessary, but would be in a real pro-
gram.
5 Server
Our web server is mainly a mirror of the client, but with a few twists. First, we fork() in order
to let the server handle multiple clients. Second, we use OpenSSL’s BIO APIs to read the
client’s request one line at a time, as well as to do buffered writes to the client. Finally, the
server closure sequence is more complicated.

Accept and Fork

On Linux, the simplest way to write a server that can handle multiple clients is to create a new
server process for each client that connects. We do that by calling fork() after accept()
returns. Each new process executes independently and just exits when it’s finished serving the
client. Although this approach can be quite slow on busy web servers it’s perfectly acceptable
here. The main server accept loop is shown in Figure 7.

94 while(1){
95 if((s=accept(sock,0,0))<0)
96 err_exit("Problem accepting");
97
98 if((pid=fork())){
99 close(s);
100 }
101 else {
102 sbio=BIO_new_socket(s,BIO_NOCLOSE);
103 ssl=SSL_new(ctx);
104 SSL_set_bio(ssl,sbio,sbio);
105
106 if((r=SSL_accept(ssl)<=0))
107 berr_exit("SSL accept error");
108
109 http_serve(ssl,s);
110 exit(0);
111 }
112 }

Figure 7 server accept loop

Server Accept
After forking and creating the SSL object, the server calls SSL_accept() which causes
OpenSSL to perform the server side of the SSL handshake. As with SSL_connect(),
because we’re using blocking sockets, SSL_accept() will block until the entire handshake
has completed. Thus, the only situation in which SSL_accept() will return is when the hand-
shake has completed or an error has been detected. SSL_accept() returns 1 for success and 0
or negative for error.

Buffered I/O
OpenSSL’s BIO objects are to some extent stackable. Thus, we can wrap an SSL object in a
BIO (the ssl_bio object) and then wrap that BIO in a buffered BIO object, as shown in Fig-
ure 8. This allows us to perform buffered reads and writes on the SSL connection by using the
BIO_* functions on the new io object. At this point you might ask: why is this good? Primarily,
it’s a matter of programming convenience: It lets the programmer work in natural units (lines
and characters) rather than SSL records.
13 io=BIO_new(BIO_f_buffer());
14 ssl_bio=BIO_new(BIO_f_ssl());
15 BIO_set_ssl(ssl_bio,ssl,BIO_CLOSE);
16 BIO_push(io,ssl_bio);

Figure 8 creating a buffered BIO

Request
An HTTP request consists of a request line followed by a bunch of header lines and an optional
body. The end of the header lines is indicated by a blank line (i.e., a pair of CRLFs, though
sometimes broken clients will send a pair of LFs instead). The most convenient way to read the
request line and headers is to read one line at a time until you see a blank line. We can do this
using the OpenSSL BIO_gets() call, as shown in Figure 9.
The BIO_gets() call behaves analogously to the stdio fgets() call. It takes an arbi-
trary size buffer and a length and reads a line from the SSL connection into that buffer. The
result is always null terminated (but includes the terminating LF). Thus, we simply read one line
at a time until we get a line which consists of simply a LF or a CRLF.
Because we use a fixed length buffer, it is possible though unlikely that we will get an over-
long line. In that event the long line will be split over two lines. In the (extremely unlikely) event
that the split happens right before the CRLF, the next line we read will consist only of the CRLF
from the previous line. In this case we’ll be fooled into thinking that the headers have finished
prematurely. A real Web server would check for this case but it’s not worth doing here. Note that
no matter what the incoming line length there’s no chance of a buffer overrun. All that can hap-
pen is that we’ll misparse the headers.
Note that we don’t really DO anything with the HTTP request. We just read it and discard it.
A real implementation would read the request line and the headers, figure out of there was a
body and read that too. However, none of these things show anything interesting about SSL so
they don’t add anything to this demonstration.

18 while(1){
19 r=BIO_gets(io,buf,BUFSIZZ-1);
20
21 switch(SSL_get_error(ssl,r)){
22 case SSL_ERROR_NONE:
23 len=r;
24 break;
25 default:
26 berr_exit("SSL read problem");
27 }
28
29 /* Look for the blank line that signals
30 the end of the HTTP headers */
0) ||
31 if(!strcmp(buf,"
32 !strcmp(buf,"0))
33 break;
34 }

Figure 9 reading the request

Response
The next step is to write the HTTP response and close the connection. Figure 10 shows this
code. Note that we’re using BIO_puts() instead of SSL_write(). This allows us to write
the response one line at a time but have the entire response written as a single SSL record. This
is important because the cost of preparing an SSL record for transmission (computing the
integrity check and encrypting it) is quite significant. Thus, it’s a good idea to make the records
as large as possible.
It’s worth noting a couple of subtleties about using this kind of buffered write:
1. You need to flush the buffer before you close. The SSL object has no knowledge that you’ve
layered a BIO on top of it, so if you destroy the SSL connection you’ll just leave the last
chunk of data sitting in the buffer. The BIO_flush() call in line 46 takes care of this.
2. By default, OpenSSL uses a 1024-byte buffer size for buffered BIOs. Because SSL records
can be up to 16K bytes long, using a 1024-byte buffer cause excessive fragmentation (and
hence lower performance.) You can use the BIO_ctrl() API to increase the buffer size.

36 if((r=BIO_puts
37 (io,"HTTP/1.0 200 OK\r\n"))<0)
38 err_exit("Write error");
39 if((r=BIO_puts
40 (io,"Server: EKRServer\r\n\r\n"))<0)
41 err_exit("Write error");
42 if((r=BIO_puts
43 (io,"Server test page\r\n"))<0)
44 err_exit("Write error");
45
46 if((r=BIO_flush(io))<0)
47 err_exit("Error flushing BIO");

Figure 10 writing the response

Shutdown
Once we’ve finished transmitting the response we need to send our close_notify. As before, this
is done using SSL_shutdown(). Unfortunately, things get a bit trickier when the server
closes first. Our first call to SSL_shutdown() sends the close_notify but doesn’t look for it on
the other side. Thus, it returns immediately but with a value of 0, indicating that the closure
sequence isn’t finished. It’s then the application’s responsibility to call SSL_shutdown()
again.
It’s possible to have two attitudes here:
1. We’ve seen all of the HTTP request that we care about. we’re not interested in anything else.
Hence, we don’t care whether the client sends a close_notify or not.
2. We strictly obey the protocol and expect others to as well. Thus, we require a close_notify.
If we have the first attitude then life is simple: We call SSL_shutdown() to send our
close_notify and then exit right away, regardless of whether the client has sent one or not. If we
take the second attitude (which our sample server does), then life is more complicated, because
clients often don’t behave correctly.
The first problem we face is that client’s often don’t send close_notifys all. In fact, some
clients close the connection as soon as they’ve read the entire HTTP response (some versions of
IE do this). When we send our close_notify, the other side may send a TCP RST segment, in
which case the program will catch a SIGPIPE. We install a dummy SIGPIPE handler in ini-
tialize_ctx() to protect against this problem.
The second problem we face is that the client may not send a close_notify immediately in
response to our close_notify. Some versions of Netscape require you to send a TCP FIN first.
Thus, we call shutdown(s,1) before we call SSL_shutdown() the second time. When
called with a "how" argument of 1, shutdown() sends a FIN but leaves the socket open for
reading. The code to do the server shutdown is shown in Figure 11 (see next page).
6 What’s Missing
In this article, we’ve only scratched the surface of the issues involved with using OpenSSL.
Here’s a (non-exhaustive) list of additional issues.

Better Certificate Checking

A more sophisticated approach to checking server certificates against the server hostname is to
use the X.509 subjectAltName extension. In order to make this check, you would need to extract
this extension from the certificate and then check it against the hostname. Additionally, it would
be nice to be able to check host names against wildcarded names in certificates. We don’t show
how to do this either.

51 r=SSL_shutdown(ssl);
52 if(!r){
53 /* If we called SSL_shutdown() first then
54 we always get return value of ’0’. In
55 this case, try again, but first send a
56 TCP FIN to trigger the other side’s
57 close_notify*/
58 shutdown(s,1);
59 r=SSL_shutdown(ssl);
60 }
61
62 switch(r){
63 case 1:
64 break; /* Success */
65 case 0:
66 case -1:
67 default:
68 berr_exit("Shutdown failed");
69 }

Figure 11 calling SSL_shutdown()

Better Error Handling

Note that these applications handle errors simply by exiting with an error. A real application
would, of course, be able to recognize errors and signal them to the user or some audit log rather
than just exiting.

6.1 Coming Soon

In the second part, we’ll be discussing a number of advanced OpenSSL features, including...

Session Resumption
SSL handshakes are expensive but SSL provides a feature called session resumption that allows
you to bypass the full handshake for a peer you’ve communicated with before.

Multiplexed and Non-blocking I/O

We used blocking sockets for all of our reads and writes. This means that while the application
is waiting for data from the peer it’s completely stalled. This wouldn’t be acceptable in a real
GUI application like a browser. Instead, you need to use the select() call to multiplex
between SSL and other sources of I/O.
Client Authentication
Server authentication occurs as a part of almost every SSL connection. However, SSL also offers
certificate-based client authentication, at the server’s request.

6.2 External Resources

Parts of this article were adapted from my book "SSL and TLS: Designing and Building Secure
Systems", Addison-Wesley 2000. "SSL and TLS" offers a comprehensive guide to the SSL pro-
tocol and its applications. See http://www.rtfm.com/sslbook for more information.
Machine readable source code for the programs presented here can be downloaded from the
author’s web site at: http://www.rtfm.com/openssl-examples/. The source code is
available under a BSD-style license.
You can get OpenSSL from http://www.openssl.org/. The docs are online here as well.
The relevant specifications are at:
SSLv2: http://www.netscape.com/eng/security/SSL_2.html
SSLv3: http://home.netscape.com/eng/ssl3/index.html
TLS (RFC 2246): http://www.ietf.org/rfc/rfc2246.txt
HTTPS (RFC 2818): http://www.ietf.org/rfc/rfc2818.txt

6.3 Acknowledgements
Thanks to Lisa Dusseault, Steve Henson, Lutz Jaenicke, and Ben Laurie for help with OpenSSL
and review of this article.
 An Introduction to OpenSSL Programming (Part II)
Eric Rescorla
RTFM, Inc.
ekr@rtfm.com
Version 1.0: January 9, 2002

1 Introduction
The quickest and easiest way to secure a TCP-based network application is with SSL. If you’re
working in C, your best choice is probably to use OpenSSL, (the web site is at
http://www.openssl.org/). OpenSSL is a free (BSD-style license) implementation of
SSL/TLS based on Eric Young’s SSLeay package. Unfortunately, the documentation and sample
code distributed with OpenSSL leaves something to be desired. Where they exist, the manual
pages are pretty good, but they often miss the big picture, as manual pages are intended as a ref-
erence, not a tutorial.
We provide an introduction to OpenSSL programming. The OpenSSL API is vast and com-
plicated so we don’t attempt to provide complete coverage. Rather, the idea is to teach you
enough to work effectively from the manual pages. In the first part, published in the September
issue of Linux Journal, we introduced the basic features of OpenSSL. In this article we show
how to use a number of advanced features such as session resumption and client authentication.

2 Source Code
For space reasons, this article only includes excerpts from the source code. The complete source
code is available in machine-readable format from the author’s web site at
http://www.rtfm.com/openssl-examples/.

2.1 Our Programs

For most of this article we’ll be extending the simple client/server pair (wclient and wserver) we
presented in Part 1 to create two new programs: wclient2 and wserver2. Like wclient, wclient2
is a simple HTTPS (see RFC 2818) client. It initiates an SSL connection to the server and then
transmits an HTTP request over that connection. It then waits for the response from the server
and prints it to the screen. This is a vastly simplified version of the functionality found in pro-
grams like fetch and cURL.
wserver2 is a simple HTTPS server: it waits for TCP connections from clients and when it
accepts one it negotiates an SSL connection. Once the connection is negotiated, it reads the
client’s HTTP request. It then transmits the HTTP response to the client. Once the response is
transmitted it closes the connection.
Towards the end we’ll also show a more interactive client (sclient) that is usable for debug-
ging or simple remote login.

3 Session Resumption
When a client and server establish an SSL connection for the first time they need to establish a
shared key called the master_secret. The master_secret is then used to create all the bulk
This article is Copyright © 2001 Eric Rescorla. It may be redistributed for any purpose and without fee provided
that this notice is retained. An earlier version of this article first appeared in the September 2001 issue of Linux
Journal.
encryption keys used to protect the traffic. The master_secret is almost invariably established
using one of two public key algorithms: RSA or Diffie-Hellman (DH). Unfortunately, both of
these algorithms are quite slow—on my Pentium II/400 a single RSA operation takes 19 ms. DH
can be even slower.
An operation that takes 19 ms may not sound that expensive but if it has to be done for every
connection then it limits the server’s throughput to less than 50 connections/second. Without
SSL, most web servers can handle hundreds of connections a second. Thus, having to do a key
exchange for every client seriously degrades the performance of a web server. In order to
improve performance, SSL contains a "session resumption" feature that allows a client/server
pair to skip this time consuming step if they have already established a master_secret in a previ-
ous connection.

The performance of RSA is highly asymmetric. Operations performed with the private
key (such as when the server decrypts the shared key) are much slower than operations
performed with the public key. Thus, in most situations most of the computational load is
on the server.

What’s a session?
SSL makes a distinction between a connection and a session. A connection represents one spe-
cific communications channel (typically mapped to a TCP connection), along with its keys,
cipher choices, sequence number state, etc. A session is a virtual construct representing the
negotiated algorithms and the master_secret . A new session is created every time a given client
and server go through a full key exchange and establish a new master_secret.
Multiple connections can be associated with a given session. Although all connections in a
given session share the same master_secret, each has its own encryption keys. This is absolutely
necessary for security reasons because reuse of bulk keying material can be extremely danger-
ous. Resumption allows the generation of a new set of bulk keys and IVs from a common
master_secret because the keys depend on the random values which are fresh for each connec-
tion. The new random values are combined with the old master_secret to produce new keys.

How It Works
The first time a client and server interact, they create both a new connection and a new session.
If the server is prepared to resume the session, it assigns the session a session_id and transmits
the session_id to the client during the handshake. The server caches the master_secret for later
reference. When the client initiates a new connection with the server, it provides the session_id
to the server. The server can either choose to resume the session or force a full handshake. If the
server chooses to resume the session the rest of the handshake is skipped and the stored
master_secret is used to generate all the cryptographic keys.

Session Resumption on the Client

Figure 1 shows the minimal OpenSSL code required for a client to do session resumption.
OpenSSL uses a SESSION object to store the session information and SSL_get1_ses-
sion() allows us to get the SESSION for a given SSL object. Once we have obtained the
SESSION object we shut down the original connection (using SSL_shutdown()) and create
a new SSL object. We use SSL_set_session() to attach the SESSION to the new SSL
object before calling SSL_connect(). This causes OpenSSL to attempt to resume the session
when it connects to the server. This code is activated by using the ’-r’ flag to wclient2.

In OpenSSL SESSION objects are reference counted so that they can be freed whenever
the last reference is destroyed. SSL_get1_session() increments the SESSION ref-
erence count and thus allows SESSION objects to be used after the SSL is freed.
144 /* Now hang up and reconnect, if requested */
145 if(reconnect) {
146 sess=SSL_get1_session(ssl); /*Collect the session*/
147 SSL_shutdown(ssl);
148 SSL_free(ssl);
149 close(sock);
150
151 sock=tcp_connect(host,port);
152 ssl=SSL_new(ctx);
153 sbio=BIO_new_socket(sock,BIO_NOCLOSE);
154 SSL_set_bio(ssl,sbio,sbio);
155 SSL_set_session(ssl,sess); /*And resume it*/
156 if(SSL_connect(ssl)<=0)
157 berr_exit("SSL connect error (second connect)");
158 check_cert(ssl,host);
159 }

Figure 1 Reconnect to the server with resumed session

Of course, this code isn’t suitable for a production application because it will only work with a
single server. A client can only resume sessions with the same server that it created them with
and this code makes no attempt to discriminate between various servers because the server name
is constant for any program invocation. In a real application, you would want to have some sort
of lookup table that maps hostname/port pairs to SESSION objects.

Session Resumption on the Server

OpenSSL provides a mechanism for automatic session resumption on the server. Each
SSL_CTX has a session cache. Whenever a client requests resumption of a given session the
server looks in the cache. As long as two SSL objects share a cache, session resumption will
work automatically with no additional intervention from the programmer. In order for the ses-
sion cache to function correctly, the programmer must associate each SSL object or the
SSL_CTX with a session ID context. This context is simply an opaque identifier that gets
attached to each session stored in the cache. We’ll see how the session ID context works when
we discuss client authentication and rehandshake. Figure 2 shows the appropriate code to set the
session ID context.

149 SSL_CTX_set_session_id_context(ctx,
150 (void*)&s_server_session_id_context,
151 sizeof s_server_session_id_context);

Figure 2 Activating the server session cache

Unfortunately, OpenSSL’s built-in session caching is inadequate for most production applica-
tions, including the ordinary version of wserver. OpenSSL’s default session cache implementa-
tion stores the sessions in memory. However, we’re using a separate process to service each
client so the session data won’t be shared and resumption won’t work. The ’-n’ flag to wserver2
will stop it from forking a new process for each connection. This serves to demonstrate session
caching but without the fork() the server can handle only one client at a time which makes it
much less useful for real world applications.
 We also have to slightly modify the server read loop so that SSL_ERROR_ZERO_RETURN
causes the connection to be shut down rather than an error and an exit. wserver assumes that the
client will always send a request and so exits with an error when the client shuts down the first
connection. This change makes wserver2 handle the close properly where wserver does not.
There are a number of different techniques that can be used to share session data between
processes. OpenSSL does not provide any support for this sort of session caching at all, but it
does provide hooks for programmers to use their own session caching code. One approach is to
have a single session server. The session server stores all of its session data in memory. The SSL
servers access the session server via interprocess communication mechanisms, typically some
flavor of sockets. The major drawback to this approach is that it requires the creation of some
entirely different server program to do this job, as well as the design of the communications pro-
tocol used between the SSL server and the session server. It can also be difficult to ensure access
control so that only authorized server processes can obtain access.
Another approach is to use shared memory. Many operating systems provide some method
of allowing processes to share memory. Once the shared memory segment is allocated, data can
be accessed as if it were ordinary memory. Unfortunately, this technique isn’t as useful as it
sounds because there’s no good way to allocate out of the shared memory pool, so the processes
need to allocate a single large segment and then statically address inside of it. Worse yet, shared
memory access is not easily portable.
The most commonly used approach is to simply store the data on a file on the disk. Then,
each server process can open that file and read and write out of it. Standard file locking routines
such as flock() can be used to provide synchronization and concurrency control. One might
think that storing the data to disk would be dramatically slower than shared memory, but remem-
ber that most operating systems have disk caches and this data would likely be placed in such a
cache.
A variant of this approach is to use one of the simple UNIX key-value databases (DBM and
friends) rather than a flat file. This allows the programmer to simply create new session records
and delete them without worrying about placing them in the file. If such a library is used, care
must be taken to flush the library buffers after each write, because data stored in the buffers has
not been written to disk.
Because OpenSSL has no support for cross-process session caches, each OpenSSL-based
application needs to solve this problem on its own. ApacheSSL (based on OpenSSL) uses the
session server approach. mod_ssl (also based on OpenSSL) can support either the disk-based
database approach or a shared memory approach. The code for all of these approaches is too
complicated to discuss here, but see Appendix A of "SSL and TLS: Designing and Building
Secure Systems" for a walkthrough of mod_ssl’s session caching code. In any case, you proba-
bly don’t need to solve this problem yourself. Rather, you should be able to borrow the code
from mod_ssl or ApacheSSL.

4 Client Authentication
Most modes of SSL only authenticate the server (and some infrequently used anonymous modes
authenticate neither the client nor the server). However, the server can request that the client
authenticate using a certificate. OpenSSL provides the SSL_CTX_set_verify() and
SSL_set_verify() API calls, which allow you to configure OpenSSL to require client
authentication. The only difference between the calls is that SSL_CTX_set_verify() sets
the verification mode for all SSL objects derived from a given SSL_CTX —as long as they are
created after SSL_CTX_set_verify() is called—whereas SSL_set_verify() only
affects the SSL object that it is called on.
SSL_CTX_set_verify() takes three arguments: the SSL_CTX to change, the
certificate verification mode and a verification callback. The verification callback is called by
OpenSSL for each certificate that is verified. This allows fine control over the verification pro-
cess but is too complicated to discuss here. Check the OpenSSL man pages for more detail.
We’re primarily concerned with the verification mode. The mode is an integer consisting of
a series of logically or’ed flags. On the server, these flags have the following effect (on the client
the effect is somewhat different):
SSL_VERIFY_NONE—don’t do certificate-based client authentication
SSL_VERIFY_PEER—attempt to do certificate-based client but don’t require it. Note that you
must not set SSL_VERIFY_PEER and SSL_VERIFY_NONE together.
SSL_VERIFY_FAIL_IF_NO_PEER_CERT—fail if the client doesn’t provide a valid
certificate. This flag can only be used with SSL_VERIFY_PEER.
SSL_VERIFY_CLIENT_ONCE—if you renegotiate a connection where the
client has authenticated, don’t requires client authentication
(we won’t use this flag but it’s mentioned for completeness).

wserver2 offers three client authentication options which are

set in the switch statement shown in Figure 3. The ’-c’ switch
requests client auth but doesn’t require it, using SSL_VERIFY-
_PEER only. The ’-C’ switch requires client auth using SSL_VERIFY_PEER |
SSL_VERIFY_FAIL_IF_NO_PEER_CERT. The third option is more complicated. If the ’-x’
switch is specified we allow the client to connect without requesting client authentication. Once
the client has sent its request we then force renegotiation with client authentication. Thus, this
branch of the switch does nothing.

158 switch(client_auth){
159 case CLIENT_AUTH_REQUEST:
160 SSL_CTX_set_verify(ctx,SSL_VERIFY_PEER,0);
161 break;
162 case CLIENT_AUTH_REQUIRE:
163 SSL_CTX_set_verify(ctx,SSL_VERIFY_PEER |
164 SSL_VERIFY_FAIL_IF_NO_PEER_CERT,0);
165 break;
166 case CLIENT_AUTH_REHANDSHAKE:
167 /* Do nothing */
168 break;
169 }

Figure 3 Setting client authentication mode

At this point you might ask "why not just require client authentication from the beginning?". For
some applications this would be just as good but for a real web server it might not be. Imagine
that you have a web server which requires SSL for all requests but has a super-secure section for
which you want to require certificate-based authentication. Because the SSL handshake occurs
before the HTTP request is transmitted there’s no way to tell in advance which part of the web
site the client wants to access and therefore whether client authentication is required.
The workaround is to allow the client to connect without client authentication. The server
then reads the client’s request and determines whether or not client authentication is required. If
it is, it requests a new SSL handshake and requires client authentication. However, since this
code is just demonstration code we don’t bother to actually examine the client request. We just
force a rehandshake if the ’-x’ flag is specified.

Rehandshaking on the Server

The OpenSSL rehandshake API is unfortunately pretty complicated. In order to understand it
you first need to understand a little bit about how SSL rehandshake works. Ordinarily the client
initiates the SSL handshake by sending a ClientHello message to the server. The client can initi-
ate a rehandshake simply by sending a new ClientHello. If the server wishes to initiate a rehand-
shake, it sends a HelloRequest message. When it receives a HelloRequest the client may—or
may not—initiate a new handshake. Moreover, handshake messages are part of a different
stream from data so data can be flowing while the handshake is happening.
Figure 4 shows the code necessary to implement the rehandshake. The first thing we do is
use SSL_set_verify() to tell the server to require the client to perform client authentica-
tion. Unfortunately, this setting only controls what OpenSSL does on new handshakes. If a client
were to attempt a resumed handshake this setting would be ignored. To prevent this problem we
use SSL_set_session_id_context() to set the resumption context to
s_server_auth_session_id_context instead of s_server_session_id_con-
text.
It’s worth taking a minute to understand why changing the session ID context works. When-
ever an SSL object stores a session in the session cache it tags it with the current ID context.
Before resuming a session, OpenSSL checks that the current ID context matches the one stored
with the cache— but only if SSL_VERIFY_PEER is set. Therefore, changing our ID context
ensures that the client can’t simply resume the session it just created because it was created
under a different ID context. However, if the client had previously communicated with the server
and established a client authenticated session under s_server_auth_session_id_con-
text then it could resume that session.

Only checking the session ID context if SSL_VERIFY_PEER is set is somewhat coun-

terintuitive; the intent is to ensure that you don’t resume sessions in one context which
were client authenticated in another context, perhaps with different certificate checking
rules. Presumably the idea is that aside from client authentication, one session is rather
like another. This assumption is somewhat questionable; one might similarly wish to pre-
vent a session established with one set of cipher preferences from being resumed in an
SSL object with a different set of cipher preferences. The only way to accomplish this
separation with OpenSSL is to use a totally different SSL_CTX for each cipher suite pol-
icy.

Next, we call SSL_negotiate() to move the connection into renegotiate state. Note that this
does not cause the connection to be renegotiated, it merely sets the renegotiate flag in the object
so that when we call SSL_do_handshake() the server sends a HelloRequest. Note that the
server doesn’t do the entire handshake at this point. That requires a separate call to
SSL_do_handshake().

50 /* Now perform renegotiation if requested */

51 if(client_auth==CLIENT_AUTH_REHANDSHAKE){
52 SSL_set_verify(ssl,SSL_VERIFY_PEER |
53 SSL_VERIFY_FAIL_IF_NO_PEER_CERT,0);
54
55 /* Stop the client from just resuming the
56 un-authenticated session */
57 SSL_set_session_id_context(ssl,
58 (void *)&s_server_auth_session_id_context,
59 sizeof(s_server_auth_session_id_context));
60
61 if(SSL_renegotiate(ssl)<=0)
62 berr_exit("SSL renegotiation error");
63 if(SSL_do_handshake(ssl)<=0)
64 berr_exit("SSL renegotiation error");
65 ssl->state=SSL_ST_ACCEPT;
66 if(SSL_do_handshake(ssl)<=0)
67 berr_exit("SSL renegotiation error");
68 }

Figure 4 Rehandshake with client authentication

The only remaining piece to explain is explicitly setting the state to SSL_ST_ACCEPT in line
65. Recall that we said that the client isn’t required to start a rehandshake when it gets the Hel-
loRequest. Moreover, OpenSSL allows the server to keep sending data while the handshake is
happening. However, in this case the whole point of the rehandshake is to get the client to
authenticate so we want to finish the handshake before proceeding. Explicitly setting the state
forces the server to wait for the client rehandshake. If we don’t set the state, the second call to
SSL_do_handshake() will return immediately and the server will send its data before doing
the rehandshake. Having to set internal variables of the SSL object is rather ugly but it’s the
standard practice in HTTPS servers such as mod_ssl and ApacheSSL and it has the virtue that it
causes the server to send an unexpected_message alert if the client does not renegotiate as
requested.

OpenSSL does offer a SSL_set_accept_state() call but it isn’t useful here

because it clears the current crypto state. Since we’re already encrypting data, clearing
the crypto state causes MAC errors.

For simplicity’s sake we’ve decided to always force a rehandshake when wserver2 is
invoked with the -x flag. However, this can create an unnecessary performance load on the
server. What if the client has already authenticated? In that case there’s no point in authenticat-
ing it again. This situation can occur if the client is resuming a previous session in which client
authentication occurred. (It cannot happen if this is the first time the client is connecting because
client authentication only happens when the server requests it and we only request it on the
rehandshake).
An enhanced version of wserver2 could check to see if the client already provided a
certificate and skip the rehandshake if it had. However, great care must be taken when doing this
because of the way that OpenSSL resumes sessions. Consider the case of a server which has two
different sets of certificate verification rules, each associated with a separate ID context. Because
OpenSSL only checks the ID context when SSL_VERIFY_PEER is set—which it is not for our
initial handshake—the client could resume a session associated with either context. Thus, in
addition to getting the certificate we would need to check that the session being resumed came
from the right session ID context. If it didn’t, we’d still have to rehandshake to be sure that we
get the right kind of client authentication.

Rehandshaking on the Client

Automatic rehandshake support is built into SSL_read(), so the client doesn’t have to do any-
thing explicit to enable rehandshake. However, this doesn’t necessarily mean that arbitrary
clients will work. In particular, the client must be prepared to handle the
SSL_ERROR_WANT_READ return value from SSL_read(). Ordinarily, if the client is using
blocking I/O—as we are in wclient2— SSL_read() would never return this error. However,
it’s required if rehandshaking occurs.
To see why this is, imagine that your client uses select() to determine when there is data
available from the server. When select() returns you call SSL_read(). However, if the
server is requesting rehandshake the data will be a HelloRequest message. SSL_read() con-
sumes the message and performs the handshake. There’s no guarantee that there will be any
more data available on the socket so SSL_read() needs to return without any data—hence the
SSL_ERROR_WANT_READ return value.
Our original wclient program, despite functioning properly in every other respect, broke
utterly when dealing with rehandshake, thereby proving once again that if you haven’t tested
some feature it almost certainly doesn’t work. The fix in wclient2 is to change the read loop that
is shown in Figure 5.
48 /* Now read the server’s response, assuming
49 that it’s terminated by a close */
50 while(1){
51 r=SSL_read(ssl,buf,BUFSIZZ);
52 switch(SSL_get_error(ssl,r)){
53 case SSL_ERROR_NONE:
54 len=r;
55 break;
56 case SSL_ERROR_WANT_READ:
57 continue;
58 case SSL_ERROR_ZERO_RETURN:
59 goto shutdown;
60 case SSL_ERROR_SYSCALL:
61 fprintf(stderr,
62 "SSL Error: Premature close0);
63 goto done;
64 default:
65 berr_exit("SSL read problem");
66 }
67
68 fwrite(buf,1,len,stdout);
69 }

Figure 5 New client read loop with rehandshake support

Since this is an HTTP client and it’s already written the request, there’s no need to wonder if
there is data coming from the server. The next traffic on the wire will always be the server’s
response. Thus, we don’t need to select() on the socket. If we get
SSL_ERROR_WANT_READ we just go back to the top of the loop and call SSL_read()
again.

Using Client Authentication Information

When using client authentication it’s important to understand what it does and doesn’t provide.
So far, all the server knows is that the client possessed the private key corresponding to some
valid certificate. In some cases, this might be enough but in most cases the server wants to know
who the client is in order to make authorization decisions.
Checking the client’s certificate is roughly similar to the check_cert() function we used
to check the server’s identity. The server would extract the client’s name from the certificate and
check it against some access control list. If the name is on the list the client will be accorded the
appropriate privileges. Otherwise, access will be denied.

5 Controlling Cipher Suites

SSL offers a multiplicity of cryptographic algorithms. Since they are not equally strong and fast,
users often want to choose one algorithm over another. By default, OpenSSL supports a broad
variety of ciphers, however, it provides an API for restricting the cipher that it will negotiate. We
expose this functionality in wclient2 and wserver2 with the ’-a’ flag.
To use the -a flag, the user provides a colon separated list of ciphers, as in ’-a
RC4-SHA:DEC-CBC3-SHA:DES-CBC-SHA’. This string is then passed to OpenSSL via the
SSL_CTX_set_cipher_list() function, as shown in Figure 6.
You can get a list of all the algorithms that OpenSSL supports using the ’openssl’ command.
Try ’openssl ciphers’. Also, you might want to try using the ’-a’ flag with both client and server
and verify for yourself that if you use the same cipher (or have the same one on both lists) things
work and otherwise the connection fails.

126 /* Set our cipher list */

127 if(ciphers){
128 SSL_CTX_set_cipher_list(ctx,ciphers);
129 }

Figure 6 Setting the cipher list

6 Multiplexed I/O
Our wclient program is just about the most trivial client program possible because the I/O
semantics are so simple. The client always writes everything it has to write and then reads every-
thing that the server has to read. Reads and writes are never interlaced and the client just stops
and waits for data from the server. This works fine for simple applications but there are many
situations in which it’s unacceptable. One such application is a remote login client.
A remote login client such as telnet or ssh needs to process at least two sources of input: the
keyboard and the network. Input from the keyboard and the server can appear asynchronously.
That is to say that they can appear in any order. This means that the read/write type I/O disci-
pline that we had in wclient is fundamentally inadequate.
It’s easy to see this. Consider an I/O discipline analogous to the one we used in wclient, rep-
resented by the pseudo-code in Figure 7.

1 while(1){
2 read(keyboard,buffer);
3 write(server,buffer);

4 read(server,buffer);
5 write(screen,buffer);
6 }

Figure 7 A broken client I/O discipline

Consider the case in which you’re remotely logged into some machine and you request a direc-
tory listing. Your request is a single line (ls on UNIX boxes) but the response is a large number
of lines. In general, these lines will be written in more than one write. Thus, it may very well
take more than one read in order to read them from the server. However, if we use the I/O disci-
pline in Figure 7, we’ll run into a problem.
We read the command from the user and the first chunk of the server’s response, but after
that we get deadlocked. The client is waiting in line 2 for the user to type something but the user
is waiting for the rest of the directory listing. We’re deadlocked. In order to break the deadlock
we need some way to know when either the keyboard or the network is ready to read. We can
then service that source of input and keep from deadlocking. Conveniently, there is a UNIX sys-
tem call that does exactly that— select(2). select() is the standard tool for doing multi-
plexed I/O. It lets you determine whether any of a set of sockets is ready to read or write. If
you’re not familiar with it already, read the man page or consult Richard Stevens’s fine book
"Advanced Programming in the UNIX environment" (Addison-Wesley 1992).
Unfortunately, although select() is a common UNIX idiom, its use with OpenSSL is far
from clean and requires understanding of some subtleties of SSL. In order to demonstrate them,
we present a new program, sclient. sclient is a simple model of an SSLized remote access
client. It connects to the server and then transfers anything typed at the keyboard to the server
and anything sent from the server to the screen.

Read
The basic problem we’re facing is that SSL is a record-oriented protocol. Thus, even if we want
to read only one byte from the SSL connection, we still need to read the entire record containing
that byte into memory. Without the entire record in hand, OpenSSL can’t check the record MAC
and so we can’t safely deliver the data to the programmer. Unfortunately, this behavior interacts
poorly with select(), as shown in Figure 8.

Network
buffer
SSL_read(1)

SSL
buffer

Figure 8 Read interaction with SSL

The left-hand side of Figure 8 shows the situation when the machine has received a record
but it’s still waiting in the network buffers. The arrow represents the read pointer which is set at
the beginning of the buffer. The bottom row represents data decoded by OpenSSL but not yet
read by the program (the SSL buffer). This buffer is currently empty so we haven’t shown a box.
If the program calls select() at this point, it will return immediately indicating that a call to
read() will succeed. Now, imagine that the programmer calls SSL_read() requesting one
byte. This takes us to the situation at the right side of the figure.
As we said earlier, the OpenSSL has to read the entire record in order to deliver even a sin-
gle byte to the program. In general, the application does not know the size of records and so its
reads will not match the records. Thus, the box in the upper right-hand corner shows that the
read pointer has moved to the end of the record. We’ve read all the data in the network buffer.
When the implementation decrypts and verifies the record, it places the data in the SSL buffer.
Then it delivers the one byte that the program asked for in SSL_read(). We show the SSL
buffer in the lower right-hand corner. The read pointer points somewhere in the buffer, indicat-
ing that some of the data is available for reading but some has already been read.
Consider what happens if the programmer calls select() at this point. select() is
concerned solely with the contents of the network buffer, and that’s empty. Thus, as far as
select() is concerned there’s no data to read. Depending on the exact arguments it’s passed,
it will either return saying that there’s nothing to read or wait for some more network data to
become available. In either case we wouldn’t read the data in the SSL buffer. Note that if
another record arrived then select() would indicate that the socket was ready to read and
we’d have an opportunity to read more data.
Thus, select() is an unreliable guide to whether there is SSL data ready to read. We
need some way to determine the status of the SSL buffer. This can’t be provided by the operat-
ing system because it has no access to the SSL buffers. It must be provided OpenSSL. OpenSSL
provides exactly such a function. The function SSL_pending() tells us whether there is data
in the SSL buffer for a given socket. Figure 9 shows SSL_pending() in action.
The logic of this code is fairly straightforward. select() has been called earlier, setting
the variable readfds with the sockets that are ready to read. If the SSL socket is ready to read,
we go ahead and try to fill our buffer unless the variable write_blocked_on_read is set
(this variable is used when we’re rehandshaking and we’ll discuss it later). Once we’ve read
some data, we write it to the console. Then we check with SSL_pending() to see if the
record was longer than our buffer. If it was, we loop back and read some more data.

49 /* Now check if there’s data to read */

50 if((FD_ISSET(sock,&readfds) && !write_blocked_on_read) ||
51 (read_blocked_on_write && FD_ISSET(sock,&writefds))){
52 do {
53 read_blocked_on_write=0;
54 read_blocked=0;
55
56 r=SSL_read(ssl,s2c,BUFSIZZ);
57
58 switch(SSL_get_error(ssl,r)){
59 case SSL_ERROR_NONE:
60 /* Note: this call could block, which blocks the
61 entire application. It’s arguable this is the
62 right behavior since this is essentially a terminal
63 client. However, in some other applications you
64 would have to prevent this condition */
65 fwrite(s2c,1,r,stdout);
66 break;
67 case SSL_ERROR_ZERO_RETURN:
68 /* End of data */
69 if(!shutdown_wait)
70 SSL_shutdown(ssl);
71 goto end;
72 break;
73 case SSL_ERROR_WANT_READ:
74 read_blocked=1;
75 break;
76
77 /* We get a WANT_WRITE if we’re
78 trying to rehandshake and we block on
79 a write during that rehandshake.
80
81 We need to wait on the socket to be
82 writeable but reinitiate the read
83 when it is */
84 case SSL_ERROR_WANT_WRITE:
85 read_blocked_on_write=1;
86 break;
87 default:
88 berr_exit("SSL read problem");
89 }
90
91 /* We need a check for read_blocked here because
92 SSL_pending() doesn’t work properly during the
93 handshake. This check prevents a busy-wait
94 loop around SSL_read() */
95 } while (SSL_pending(ssl) && !read_blocked);
96 }

Figure 9 Reading data using SSL_pending()

Note that we’ve added a new branch to our switch statement: a check for
SSL_ERROR_WANT_READ. What’s going on here is that we’ve set the socket for nonblocking
operation. Recall that we said in Section 8.7 that if you called read() when the network
buffers were empty, it would simply block (wait) until they weren’t. Setting the socket to non-
blocking causes it to return immediately saying that it would have blocked.
To understand why we’ve done this, consider what happens if an SSL record arrives in two
pieces. When the first piece arrives, select() will signal that we’re ready to read. However,
we need to read the entire record in order to return any data so this is a false positive. Attempt-
ing to read all that data will block, leading to exactly the deadlock we were trying to avoid.
Instead, we set the socket to nonblocking and catch the error, which OpenSSL translates to
SSL_ERROR_WANT_READ.
It’s worth noting that the call to fwrite() that we use to write to the console can block.
This will cause the entire application to stall. This is reasonable behavior in a terminal client—if
the user isn’t looking at the screen we want the server to wait for him—but in other applications
we might have to make this file descriptor non-blocking and select() on it as well. This is left as
an exercise for the reader.

Write
When we’re writing to the network we have to face the same sort of inconsistency that we had
when reading. Again, the problem is the all-or-nothing nature of SSL record transmission. For
simplicity’s sake, let’s consider the case where the network buffers are mostly full and the pro-
gram attempts to perform a modest-sized write, say 1K. This is illustrated in Figure 10.

Program
buffer

SSL SSL_write(1024)
buffer

Network
buffer

Figure 10 Write interaction with SSL

Again, the left-hand side of the figure represents our initial situation. The program has 1K to
write in some buffer. The write pointer is set at the beginning of that buffer. The SSL buffers are
empty. The network buffer is half-full (the shading indicates the full region). The write pointer is
set at the beginning. We’ve deliberately obscured the distinction between the TCP buffers and
the size of the TCP window because it’s not relevant here. Suffice it to say that the program can
safely write 512 bytes without blocking.
Now, the program calls SSL_write() with a 1024-byte block. OpenSSL has no way of
knowing how much data it can write safely, so it simply formats the buffer as a single record,
thus moving the write pointer in the program buffer to the end of the buffer. We can ignore the
slight data expansion from the SSL header and MAC and simply act as if the data to be written
to the network was 1024 bytes.
Now, what happens when OpenSSL calls write()? It successfully writes 512 bytes but
gets a would block error when it attempts to write to the end of the record. As a consequence,
the write pointer in the SSL buffer is moved halfway across—indicating that half of the data has
been written to the network. The network buffer is shaded to indicate that it’s completely full.
The network write pointer hasn’t moved.
We now need to concern ourselves with two questions: first, how does the toolkit indicate
this situation to the application and, second, how does the programmer arrange that the SSL
buffer gets flushed when space is available in the network buffer? The kernel will automatically
flush the network buffer when possible so we don’t need to worry about arranging for that. We
can use select() to see when there is more space available in the network buffer and we
should therefore flush the SSL buffer.
Once OpenSSL has received a would block error from the network, it aborts and propagates
that error all the way up to the application. Note that this does not mean that it throws away the
data in the SSL buffer. This is impossible because part of the record might already have been
sent.
In order to flush this buffer, we must call SSL_write() again with the same buffer that it
called the first time (it’s permissible to extend the buffer but the start must be the same.)
OpenSSL automatically remembers where the buffer write pointer was and only writes the data
after the write pointer. Figure 11 shows this process in action.
The first thing we need to do is have some data to write. Thus, we check to see if the console
is ready to read, and if so read whatever’s there (up to BUFSIZZ bytes) into the buffer c2s,
placing the length in the variable c2sl.
If c2sl is nonzero and the network buffers are (at least partially) empty, then we have data
to write to the network. As usual, we call SSL_write() with the buffer c2s. As before, if we
manage to write some but not all of the data, we simply increment c2s_offset and decre-
ment c2sl.
The new behavior here is that we check for the error SSL_ERROR_WANT_WRITE. This
error indicates that we’ve got unflushed data in the SSL buffer. As we described above, we need
to call SSL_write() again with the same buffer, so we simply leave c2sl and c2s_off-
set unchanged. Thus, the next time SSL_write() is called it will automatically be with the
same data.

OpenSSL actually provides a flag called SSL_MODE_ACCEPT_MOVING_

WRITE_BUFFER which allows you to call SSL_write() with a different buffer after a
would block error. However, this merely allows you to allocate a new buffer with the
same contents. SSL_write() still seeks to the same write pointer before looking for
new data.

The interaction of multiplexed I/O with rehandshake

We saw earlier that rehandshaking could cause SSL_write() to return with
SSL_ERROR_WANT_READ. Similarly, SSL_read() can return with
SSL_ERROR_WANT_WRITE if rehandshake occurs and the write buffers fill up (this is rela-
tively unusual on modern platforms since the write buffers are generally large enough to accom-
modate typical SSL handshake records). In either case, the correct procedure is to select for
read/write-ability and then reissue the offending call. This leads to some confusing logic since it
means that under certain circumstances you respond to a socket being ready to read by calling
SSL_write()!
Let’s take the case of SSL_write() first. If SSL_write() returns
SSL_ERROR_WANT_READ we set the state variable write_blocked_on_read. Ordi-
narily, this would cause us to call SSL_read() but because write_blocked_on_read is
set we skip the read loop and fall through to the write loop where the call to SSL_write() is
reissued. Note that since the SSL handshake requires a number of reads and writes it’s quite pos-
sible that this call won’t complete either and we’ll have to reenter the select() loop.
98 /* Check for input on the console*/
99 if(FD_ISSET(fileno(stdin),&readfds)){
100 c2sl=read(fileno(stdin),c2s,BUFSIZZ);
101 if(c2sl==0){
102 shutdown_wait=1;
103 if(SSL_shutdown(ssl))
104 return;
105 }
106 c2s_offset=0;
107 }
108
109 /* If the socket is writeable... */
110 if((FD_ISSET(sock,&writefds) && c2sl) ||
111 (write_blocked_on_read && FD_ISSET(sock,&readfds))) {
112 write_blocked_on_read=0;
113
114 /* Try to write */
115 r=SSL_write(ssl,c2s+c2s_offset,c2sl);
116
117 switch(SSL_get_error(ssl,r)){
118 /* We wrote something*/
119 case SSL_ERROR_NONE:
120 c2sl-=r;
121 c2s_offset+=r;
122 break;
123
124 /* We would have blocked */
125 case SSL_ERROR_WANT_WRITE:
126 break;
127
128 /* We get a WANT_READ if we’re
129 trying to rehandshake and we block on
130 write during the current connection.
131
132 We need to wait on the socket to be readable
133 but reinitiate our write when it is */
134 case SSL_ERROR_WANT_READ:
135 write_blocked_on_read=1;
136 break;
137
138 /* Some other error */
139 default:
140 berr_exit("SSL write problem");
141 }
142 }

Figure 11 Client to server writes using OpenSSL

Now consider the case of SSL_read(). If SSL_read() returns SSL_ERROR_WANT_WRITE

we set the state variable read_blocked_on_write and reenter the select() loop wait-
ing for the socket to be writable. When it is we arrange to call SSL_read() again (see the sec-
ond line of the test right after the call to select(). Note that after we call SSL_read() we fall
through the the write section of the loop. However, even though the socket is writeable and so
the FD_ISSET test passes, SSL_write() will only be called if c2sl is nonzero and there’s really
data to write.
6.1 What’s still missing?
Taken together, these articles demonstrate most of the essentials of writing SSL clients and
servers with OpenSSL. However, OpenSSL offers a number of other powerful features that we
don’t cover, including:

Cryptographic Toolkit
Underlying OpenSSL’s SSL implementation is a crypto toolkit implementing all the major cryp-
tographic primitives, including RSA, DH, DES, 3DES, RC4, AES, SHA and MD5, as well as an
ASN.1 encoder. Thus, OpenSSL is useful for people writing other kinds of cryptographic appli-
cations, even if they don’t involve SSL.

Certificate Authority
OpenSSL contains the basic software required to write a certificate authority (CA). A number of
CAs have been written on top of OpenSSL, including the free OpenCA project (see the refer-
ences section).

S/MIME
OpenSSL now includes an S/MIME implementation, allowing it to be used to write secure mail
clients. This is still somewhat of a hard hat area—the S/MIME support isn’t quite complete yet
and neither is the documentation.

6.2 External Resources

Parts of this article were adapted from my book "SSL and TLS: Designing and Building Secure
Systems", Addison-Wesley 2000. "SSL and TLS" offers a comprehensive guide to the SSL pro-
tocol and its applications. See http://www.rtfm.com/sslbook for more information.
Machine readable source code for the programs presented here can be downloaded from the
author’s web site at: http://www.rtfm.com/openssl-examples/. The source code is
available under a BSD-style license.
You can get OpenSSL from http://www.openssl.org/. The docs are online here as well.
The relevant specifications are at:
SSLv2: http://www.netscape.com/eng/security/SSL_2.html
SSLv3: http://home.netscape.com/eng/ssl3/index.html
TLS (RFC 2246): http://www.ietf.org/rfc/rfc2246.txt
HTTPS (RFC 2818): http://www.ietf.org/rfc/rfc2818.txt

The mod_ssl project lives at http://www.modssl.org/.

The ApacheSSL project lives at http://www.apachessl.org/.
The OpenCA project lives at http://www.openca.org/.

6.3 Acknowledgements
Many thanks to Lutz Jaenicke and Bodo Moeller for help with OpenSSL and catching a number
of problems with the example programs. Thanks to Lisa Dusseault for review of this article.