ACE Semaphore Classes

Semaphores are a powerful mechanism used to lock and/or synchronize access to shared resources
in concurrent applications. A semaphore contains a count that indicates the status of a shared
resource. Application designers assign the meaning of the semaphore's count, as well as its initial
value. Semaphores can therefore be used to mediate access to a pool of resources.
Since releasing a semaphore increments its count regardless of the presence of waiters, they are
useful for keeping track of events that change shared program state. Threads can make decisions
based on these events, even if they have already occurred. Although some form of semaphore
mechanism is available on most operating systems, the ACE semaphore wrapper facades resolve
issues arising from subtle variations in syntax and semantics across a broad range of environments.

Class Capabilities
The ACE_Thread_ semaphore and ACE_Process_ Semaphore classes portably encapsulate
process-scoped and system-scoped semaphores, respectively, in accordance with the Wrapper
Facade pattern. The constructor is slightly different, however, since semaphore initialization is
more expressive than mutexes and readers/writer locks, allowing the semaphore's initial count to
be set. The relevant portion of the ACE_Thread_ Semaphore API is shown below:

class ACE_Thread_Semaphore
{
public:
// Initialize the semaphore, with an initial value of <count>,
// a maximum value of <max>, and unlocked by default.
ACE_Thread_ semaphore (u_int count = 1,
const char *name = 0,
void *arg = 0,
int max = 0x7FFFFFFF);
// ... same as pseudo <ACE_LOCK> signatures.
};

The ACE_Process_Semaphore has the same interface, though it synchronizes threads at the system
scope rather than at the process scope.
These two ACE classes encapsulate OS-native semaphore mechanisms whenever possible,
emulating them if the OS platform does not support semaphores natively. This allows applications
to use semaphores and still be ported to new platforms regardless of the native semaphore support,
or lack thereof.
The ACE_Null_Semaphore class implements all of its methods as "no-op" inline functions. Two
of its acquire() methods are implemented below:

class ACE_Null_Semaphore
{
public:
int acquire () { return 0; }
int acquire (ACE_Time_Value *) { errno = ETIME; return -1; }
// ...

1
};

Although semaphores can coordinate the processing of multiple threads, they do not themselves
pass any data between threads. Passing data between threads is a common concurrent
programming technique, however, so some type of lightweight intraprocess message queueing
mechanism can be quite useful. Therefore, a Message_Queue class implementation that provides
the following capabilities is shown:

 It allows messages (ACE_Message_Block objects) to be enqueued at the rear of the queue

and dequeued from the front of the queue.
 It supports tunable flow control that prevents fast message producer threads from
swamping the resources of slower message consumer threads.
 It allows timeouts to be specified on both enqueue and dequeue operations to avoid
unlimited blocking when necessary.

the key parts of the Message_Queue class implementation is shown below and the use of
ACE_Thread_Semaphore is showcased.

Definition of the Message_Queue class:

class Message_Queue
{
public:
// Default high and low water marks.
enum {
DEFAULT_LWM = 0, // 0 is the low water mark.
DEFAULT_HWM = 16 * 1024 // 16 K is the high water mark.
};
// Initialize.
Message_Queue (size_t = DEFAULT_HWM, size_t = DEFAULT_LWM);

// Destroy.
~Message_Queue ();

// Checks if queue is full/empty.

int is_full () const;
int is_empty () const;

// Interface for enqueueing and dequeueing ACE_Message_Blocks.

int enqueue_tail (ACE_Message_Block *, ACE_Time_Value * = 0);
int dequeue_head (ACE_Message_Block *&, ACE_Time_Value * = 0);

private:
// Implementations that enqueue/dequeue ACE_Message_Blocks.
int enqueue_tail_i (ACE_Message_Block *, ACE_Time_Value * = 0);
int dequeue_head_i (ACE_Message_Block *&, ACE_Time_Value * = 0);

2
 // Implement the checks for boundary conditions.
int is_empty_i () const;
int is_full_i () const;

// Lowest number before unblocking occurs.

int low_water_mark_;
// Greatest number of bytes before blocking.
int high_water_mark_;
// Current number of bytes in the queue.
int cur_bytes_;
// Current number of messages in the queue.
int cur_count_;
// Number of threads waiting to dequeue a message.
size_t dequeue_waiters_;
// Number of threads waiting to enqueue a message.
size_t enqueue_waiters_;

// C++ wrapper facades to coordinate concurrent access.

mutable ACE_Thread_Mutex lock_;
ACE_Thread_Semaphore notempty_;
ACE_Thread_Semaphore notfull_;

// Remaining details of queue implementation omitted....

};

The Message_Queue constructor shown below creates an empty message list and initializes the
ACE_Thread_Semaphores to start with a count of O (the mutex lock_ is initialized automatically
by its default constructor).

Message_Queue::Message_Queue (size_t hwm, size_t lwm)

: low_water_mark_ (lwm),
high_water_mark (hwm),
cur_bytes_(0),
cur_count_ (0),
dequeue_waiters_ (0),
enqueue_waiters_ (0),
notempty_ (0),
notfull_ (0)
{ /* Remaining constructor implementation omitted ... */ }

The following methods check if a queue is "empty," that is, contains no messages, or "full," that
is, contains more than high_water_mark_ bytes in it.

Starting with the is_empty() and is_full() interface methods:

3
int Message_Queue::is_empty () const {
ACE_GUARD_RETURN (ACE_Thread_Mutex, guard, lock_, -1);
return is_empty_i ();
}

int Message_Queue::is_full () const {

ACE_GUARD_RETURN (ACE_Thread_Mutex, guard, lock_, -1);
return is_full_i ();
}

These methods acquire the lock_ and then forward the call to one of the following implementation
methods:

int Message_Queue::is_empty_i () const {

return cur_bytes_ <= 0 && cur count_ <= 0;
}

int Message_Queue::is_full_i () const {

return cur _bytes_ >= high_water_mark_;
}

These methods assume the lock_ is held and actually perform the work.

The enqueue_tail() method inserts a new item at the end of the queue and returns a count of the
number of messages in the queue. As with the dequeue_head() method, if the timeout parameter
is 0, the caller will block until action is possible. Otherwise, the caller will block only up to the
amount of time in *timeout. A blocked call can return when a signal occurs or if the time specified
in timeout elapses, in which case errno is set to EWOULDBLOCK.

int Message_Queue::enqueue_tail (ACE_Message_Block *new_mblk,

ACE_Time_Value *timeout)
{
ACE_GUARD_RETURN (ACE_Thread_Mutex, guard, lock_, -1);
int result = 0;

// Wait until the queue is no longer full.

while (is_full_i () && result != -1) {
++enqueue_waiters_;
guard.release ();
result = notfull_.acquire (timeout);
guard.acquire ();
}

if (result == -1) {
if (enqueue_waiters_ > 0)
--enqueue_waiters_;

4
 if (errno == ETIME)
errno = EWOULDBLOCK;
return -1;
}

// Enqueue the message at the tail of the queue.

int queued_messages = enqueue_tail_i (new_mblk);

// Tell any blocked threads that the queue has a new item!
if (dequeue_waiters_ > 0) {
--dequeue_waiters_;
notempty_.release ();
}
return queued_messages; // guard's destructor releases lock_.
}

The enqueue_tail() method releases the notempty_ semaphorewhen there is at least one thread
waiting to dequeue a message. The actual enqueueing logic resides in enqueue_tail_i(), which is
omitted here since it is a low-level implementation detail. Note the potential race condition in the
time window between the call to not_full_acquire() and reacquiring the guard lock. It is possible
for another thread to call dequeue_head (), decrementing enqueue_waiters_ in that small window
of time. After the lock is reacquired, therefore, the count is checked to guard against decrementing
enqueue_waiters_ below 0.

The dequeue_head() method removes the front item from the queue, passes it back to the caller,
and returns a count of the number of items still in the queue, as follows:

int Message_Queue::dequeue_head (ACE_Message_Block *&first_item,

ACE_Time_Value *timeout)
{
ACE_GUARD_RETURN (ACE_Thread_Mutex, guard, lock_, -1);
int result = 0;

// Wait until the queue is no longer empty.

while (is_empty_i () && result != -1) {
++dequeue_waiters_;
guard.release ();
result = notempty_.acquire (timeout);
guard.acquire ();
}

if (result == -1) {
if (dequeue_waiters_ > 0)
--dequeue_waiters
if (errno == ETIME)
errno = EWOULDBLOCK;

5
 return -1;
}

// Remove the first message from the queue.

int queued_messages = dequeue_head_i (first_item);

// Only signal if we've fallen below the low water mark.

if (cur_bytes_ <= low_water_mark_ && enqueue_waiters_ > 0) {
enqueue_waiters_--;
notfull_.release ();
}
return queued messages; // <guard> destructor releases <lock_>
}

Iterative, Concurrent, and Reactive Servers

Servers can be categorized as either iterative, concurrent, or reactive. The primary trade-offs in
this dimension involve simplicity of programming versus the ability to scale to increased service
offerings and host loads.

Iterative servers

Iterative servers handle each client request in its entirety before servicing subsequent requests.
While processing a request, an iterative server therefore either queues or ignores additional
requests. Iterative servers are best suited for either

 Short-duration services, such as the standard Internet ECHO and DAYTIME services, that
have minimal execution time variation or
 Infrequently run services, such as a remote file system backup service that runs nightly
when platforms are lightly loaded

Iterative servers are relatively straightforward to develop. Iterative servers execute their service
requests internally within a single process address space, as shown by the following pseudo-code:

void iterative server()

{
initialize listener endpoint(s)

for (each new client request) {

retrieve next request from an input source
perform requested service
if (response required) send response to client
}
}

6
Due to this iterative structure, the processing of each request is serialized at a relatively coarse-
grained level, for example, at the interface between the application and an OS synchronous event
demultiplexer, such as select() or WaitForMultipleObjects(). However, this coarse-grained level
of concurrency can underutilize certain processing resources (such as multiple CPUs) and OS
features (such as support for parallel DMA transfer to/from I/O devices) that are available on a
host platform.

Iterative servers can also prevent clients from making progress while they are blocked waiting for
a server to process their requests. Excessive server-side delays complicate application and
middleware-level retransmission time-out calculations, which can trigger excessive network
traffic. Depending on the types of protocols used to exchange requests between client and server,
duplicate requests may also be received by a server.

Concurrent servers

Concurrent servers handle multiple requests from clients simultaneously. Depending on the OS
and hardware platform, a concurrent server either executes its services using multiple threads or
multiple processes. If the server is a single-service server, multiple copies of the same service can
run simultaneously. If the server is a multiservice server, multiple copies of different services may
also run simultaneously.

Concurrent servers are well-suited for I/O-bound services and/or long-duration services that
require variable amounts of time to execute. Unlike iterative servers, concurrent servers allow finer
grained synchronization techniques that serialize requests at an application-defined level.

Concurrent servers can be structured various ways, for example, with multiple processes or
threads. A common concurrent server design is thread-per-request, where a master thread spawns
a separate worker thread to perform each client request concurrently:

void master thread()

{
initialize listener endpoint(s)

for (each new client request) {

receive the request
spawn new worker thread and pass request to this thread
}
}

The master thread continues to listen for new requests, while the worker thread processes the client
request, as follows:

void worker thread()

{

7
 perform requested service
if (response required) send response to client
terminate thread
}

It's straightforward to modify this thread-per-request model to support other concurrent server
models, such as thread-per-connection:

void master thread()

{
initialize listener endpoint(s)

for (each new client connection) {

accept connection
spawn new worker thread and pass connection to this thread
}
}

In this design, the master thread continues to listen for new connections, while the worker thread
processes client requests from the connection, as follows:

void worker_thread()
{
for (each request on the connection) {
receive the request
perform requested service
if (response required) send response to client
}
}

Thread-per-connection provides good support for prioritization of client requests. For instance,
connections from high-priority clients can be associated with high-priority threads. Requests from
higher-priority clients will therefore be served ahead of requests from lower-priority clients since
the OS can preempt lower-priority threads.

Reactive servers

Reactive servers process multiple requests virtually simultaneously, although all processing is
actually done in a single thread. Before multithreading was widely available on OS platforms,
concurrent processing was often implemented via a synchronous event demultiplexing strategy
where multiple service requests were handled in round-robin order by a single-threaded process.
For instance, the standard X Windows server operates this way.

A reactive server can be implemented by explicitly time-slicing attention to each request via
synchronous event demultiplexing mechanisms, such as select() and WaitForMultipleObjects().

8
The following pseudo-code illustrates the typical style of programming used in a reactive server
based on select():

void reactive_server()
{
initialize listener endpoint(s)

// Event loop.
for (;;) {
select() on multiple endpoints for client requests
for (each active client request) {
receive the request
perform requested service
if (response is necessary) send response to client
}
}
}

Although this server can service multiple clients over a period of time, it is fundamentally iterative
from the server's perspective. Compared with taking advantage of full-fledged OS support for
multithreading, therefore, applications developed using this technique possess the following
limitations:

 Increased programming complexity. Certain types of networked applications, such as I/O-

bound servers, are hard to program with a reactive server model. For example, developers
are responsible for yielding the event loop thread explicitly and saving and restoring
context information manually. For clients to perceive that their requests are being handled
concurrently rather than iteratively, therefore, each request must execute for a relatively
short duration. Likewise, long-duration operations, such as downloading large files, must
be programmed explicitly as finite state machines that keep track of an object's processing
steps while reacting to events for other objects. This design can become unwieldy as the
number of states increases.
 Decreased dependability and performance. An entire server process can hang if a single
operation fails, for example, if a service goes into an infinite loop or hangs indefinitely in
a deadlock. Moreover, even if the entire process does not fail, its performance will degrade
if the OS blocks the whole process whenever one service calls a system function or incurs
a page fault. Conversely, if only nonblocking methods are used, it can be hard to improve
performance via advanced techniques, such as DMA, that benefit from locality of reference
in data and instruction caches. OS multithreading mechanisms can overcome these
performance limitations by automating preemptive and parallel execution of independent
services running in separate threads. One way to work around these problems without
going to a full-blown concurrent server solution is to use asynchronous I/O.