C++ Network Programming with Patterns, Frameworks, and ACE

Douglas C. Schmidt
Professor [email protected] www.cs.wustl.edu/schmidt/ Department of EECS Vanderbilt University (615) 343-8197

Sponsors
NSF, DARPA, ATD, BBN, Boeing, Cisco, Comverse, GDIS, Experian, Global MT, Hughes, Kodak, Krones, Lockheed, Lucent, Microsoft, Mitre, Motorola, NASA, Nokia, Nortel, OCI, Oresis, OTI, QNX, Raytheon, SAIC, Siemens SCR, Siemens MED, Siemens ZT, Sprint, Telcordia, USENIX

Advanced ACE Tutorial

Douglas C. Schmidt

Roadmap to Levels of Middleware

APPLICATIONS
HUD Nav
AVIONICS REPLICATION SERVICE

Observations
WTS

DOMAIN-SPECIFIC MIDDLEWARE SERVICES

COMMON MIDDLEWARE SERVICES

EVENT CHANNEL

Cons

Cons Cons

DISTRIBUTION MIDDLEWARE

HOST INFRASTRUCTURE MIDDLEWARE

OPERATING SYSTEMS & PROTOCOLS HARDWARE DEVICES

Historically, apps built atop OS Today, apps built atop middleware Middleware has multiple layers  Just like network protocol stacks

www.cs.wustl.edu/schmidt/PDF/ middleware-chapter.pdf
Vanderbilt University 1

Advanced ACE Tutorial

Douglas C. Schmidt

Motivation for Concurrency

SERVER

maxfdp1
CLIENT

Leverage hardware/software
e.g., multi-processors and OS thread support

read_fds
WORK REQUEST WORK REQUEST WORK REQUEST
CLIENT

CLIENT

WORK REQUEST
CLIENT

Increase performance
e.g., overlap computation and communication

(1) ITERATIVE SERVER

SERVER

Improve response-time
CLIENT

WORK REQUEST WORK REQUEST WORK REQUEST

CLIENT

e.g., GUIs and network servers

WORK REQUEST
CLIENT

CLIENT

Simplify program structure

e.g., sync vs. async

(2) CONCURRENT SERVER

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Motivation for Distribution

PRINTER COMPUTER FILE SYSTEM

Collaboration interworking Performance and locality

! connectivity and ! multi-processing ! !

CD ROM

(1) STAND-ALONE APPLICATION ARCHITECTURE

NAME SERVICE DISPLAY SERVICE NETWORK PRINT SERVICE CD ROM PRINTER FILE SYSTEM FI LE SERVICE TIME SERVICE CYCLE SERVICE

Reliability and availability replication Scalability and portability modularity

Extensibility dynamic conguration and reconguration Cost effectiveness open systems and resource sharing

(2) DISTRIBUTED APPLICATION ARCHITECTURE

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Challenges and Solutions

Developing efcient, robust, and extensible concurrent networking applications is hard e.g., must address complex topics that are less problematic or not relevant for non-concurrent, stand-alone applications OO techniques and OO language features help to enhance software quality factors Key OO techniques include patterns and frameworks Key OO language features include classes, inheritance, dynamic binding, and parameterized types Key software quality factors include modularity, extensibility, portability, reusability, and correctness

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Caveats
OO is not a panacea Though when used properly it helps minimize accidental complexity and improve software quality factors Its also essential to understand advanced OS features to enhance functionality and performance, e.g.,

Multi-threading Multi-processing Synchronization Shared memory Explicit dynamic linking Communication protocols and IPC mechanisms

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Tutorial Outline
Brief overview of key OO networking and concurrency concepts and OS platform mechanisms Emphasis is on practical solutions Examine a range of examples in detail

Networked Logging Service Concurrent Web Server Application-level Telecom Gateway Call Center Manager Event Server

Discuss general concurrent programming strategies Provide URLs for further reading on the topic

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Software Development Environment

The topics discussed here are largely independent of OS, network, and programming language Currently used successfully on UNIX/POSIX, Windows, and RTOS platforms, running on TCP/IP networks using C++ Examples are illustrated using freely available ADAPTIVE Communication Environment (ACE) OO framework components Although ACE is written in C++, the principles covered in this tutorial apply to other OO languages e.g., Java, Eiffel, Smalltalk, etc. In addition, other networks and backplanes can be used, as well

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Sources of Complexity
PRINTER

Inherent complexity
FILE SYSTEM

COMPUTER

CD ROM

(1) STAND-ALONE APPLICATION ARCHITECTURE

NAME SERVICE DISPLAY SERVICE NETWORK PRINT SERVICE CD ROM PRINTER FILE SYSTEM FI LE SERVICE TIME SERVICE CYCLE SERVICE

Latency Reliability Synchronization Deadlock

Accidental Complexity Low-level APIs Poor debugging tools Algorithmic decomposition Continuous re-invention
8

(2) DISTRIBUTED APPLICATION ARCHITECTURE

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Sources of Inherent Complexity

Inherent complexity results from fundamental domain challenges, e.g.:
Concurrent programming Eliminating race conditions Deadlock avoidance Fair scheduling Performance optimization and tuning Distributed programming Addressing the impact of latency Fault tolerance and high availability Load balancing and service partitioning Consistent ordering of distributed events

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Sources of Accidental Complexity

Accidental complexity results from limitations with tools and techniques used to develop concurrent applications, e.g.,
Lack of portable, reentrant, type-safe and extensible system call interfaces and component libraries Inadequate debugging support and lack of concurrent and distributed program analysis tools Widespread use of algorithmic decomposition Fine for explaining concurrent programming concepts and algorithms but inadequate for developing large-scale concurrent network applications Continuous rediscovery and reinvention of core concepts and components
Vanderbilt University 10

Advanced ACE Tutorial

Douglas C. Schmidt

OO Contributions to Concurrent and Distributed Applications

Concurrent network programming is Patterns and frameworks elevate traditionally performed using development level to focus on low-level OS mechanisms, e.g., application concerns, e.g.,

fork/exec Shared memory and semaphores Memory-mapped les Signals sockets/select Low-level thread APIs

Service functionality and policies Service conguration Concurrent event demultiplexing and event handler dispatching Service concurrency and synchronization
11

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Overview of Patterns
Patterns represent solutions to problems that arise when developing software within a particular context i.e., Patterns == problem/solution pairs within a context Patterns capture the static and dynamic structure and collaboration among key participants in software designs They are particularly useful for articulating how and why to resolve non-functional forces Patterns facilitate reuse of successful software architectures and designs

Vanderbilt University

12

Advanced ACE Tutorial

Do

Example: the Proxy Pattern

1: METHOD : BROKER : QUOTER 4: METHOD
RETURN PROXY CALL

3: CLIENT

RESPONSE

2: FORWARD

REQUEST

NETWORK

: QUOTER

SERVER

Intent: Provide a surrogate for another object that controls access to it

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Overview of Frameworks and Components

A framework is: An integrated collection of components that collaborate to produce a reusable architecture for a family of related applications Frameworks differ from conventional class libraries: 1. Frameworks are semi-complete applications 2. Frameworks address a particular application domain 3. Frameworks provide inversion of control Frameworks facilitate reuse of successful networked application software designs and implementations Applications inherit from and instantiate framework components

Vanderbilt University

14

Advanced ACE Tutorial

Douglas C. Schmidt

Class Libraries versus Frameworks

APPLICATIONSPECIFIC FUNCTIONALITY
LOCAL INVOCATIONS

MATH CLASSES

Key distinctions

ADT
CLASSES DATABASE CLASSES

Class libraries
Reusable building blocks Domain-independent Limited in scope Passive

EVENT LOOP

GLUE CODE

GUI
CLASSES

NETWORK

IPC

CLASSES

(A) CLASS

LIBRARY ARCHITECTURE
NETWORKING
EVENT LOOP

ADT
CLASSES INVOKES MATH CLASSES

Frameworks
GUI
EVENT LOOP

APPLICATIONSPECIFIC FUNCTIONALITY

CALL BACKS

DATABASE

EVENT LOOP

(B) FRAMEWORK
Vanderbilt University

ARCHITECTURE

Reusable, semi-complete applications Domain-specic Broader in scope Active

15

Advanced ACE Tutorial

NETWORKED SERVICE COMPONENTS LAYER
JAWS ADAPTIVE WEB SERVER

Douglas C. Schmidt
STANDARDS BASED MIDDLEWARE
THE ACE ORB (TAO)

The ADAPTIVE Communication Environment (ACE) TOKEN SERVER NAME SERVER GATEWAY SERVER TIME SERVER LOGGING SERVER

FRAMEWORK LAYER

SERVICE HANDLER

ACCEPTOR

CONNECTOR

CORBA HANDLER

C++
WRAPPER FACADE LAYER

PROCESS/ THREAD MANAGERS SYNCH WRAPPERS SPIPE SAP

STREAMS

LOG MSG FIFO SAP

SOCK SAP/ TLI SAP

REACTOR/ PROACTOR

SERVICE CONFIGURATOR

SHARED MALLOC MEM MAP FILE SAP

OS ADAPTATION LAYER

C APIS

PROCESSES/ THREADS

WIN32 NAMED PIPES & UNIX STREAM PIPES

SOCKETS/

TLI

UNIX FIFOS

SELECT/ IO COMP

DYNAMIC LINKING

SHARED MEMORY

FILE SYS APIS

PROCESS/THREAD SUBSYSTEM

COMMUNICATION SUBSYSTEM

VIRTUAL MEMORY & FILE SUBSYSTEM

GENERAL OPERATING SYSTEM SERVICES

www.cs.wustl.edu/schmidt/ACE.html

Vanderbilt University

16

Advanced ACE Tutorial

Douglas C. Schmidt

ACE Statistics
ACE library contains  250,000 lines of C++ Over 40 person-years of effort Ported to UNIX, Windows, MVS, and RT/embedded platforms e.g., VxWorks, LynxOS, Chorus Large user and open-source developer community schmidt/ACE-users.html Currently used by dozens of companies Bellcore, BBN, Boeing, Ericsson, Hughes, Kodak, Lockheed, Lucent, Motorola, Nokia, Nortel, Raytheon, SAIC, Siemens, etc. Supported commercially by Riverace www.riverace.com
Vanderbilt University 17

Advanced ACE Tutorial

Douglas C. Schmidt

The Key Frameworks in ACE

AcceptorConnector Service Configurator Reactor Proactor

Streams

Task

ACE contains a number of frameworks that can be used separately or together This design permits ne-grained subsetting of ACE components Subsetting helps minimize ACEs memory footprint $ACE_ROOT/doc/ACE-subsets.html
Vanderbilt University 18

Advanced ACE Tutorial

Douglas C. Schmidt

Patterns for Communication Middleware

Thread-per Request Thread-per Session Thread Pool Thread Specific Storage Active Object Half-Sync/ Half-Async Leader/ Followers Reactor Proactor Wrapper Facade Acceptor Connector Component Configurator Object Lifetime Manager Double Checked Locking Scoped Locking ThreadSafe Interface Strategized Locking

Observation

Failures rarely result from unknown scientic principles, but from failing to apply proven engineering practices and patterns
Benets of Patterns Facilitate design reuse Preserve crucial design information Guide design choices
19

Asynchronous Completion Token External Polymorphism Concurrency Patterns

Event Patterns

Initialization Synchronization Patterns Patterns

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

The ACE ORB (TAO)

in args

CLIENT

OBJ REF

operation()
out args + return value

OBJECT (SERVANT) IDL

TAO Overview ! A real-time, high-performance ORB Leverages ACE Runs on POSIX, Windows, RTOSs Related efforts ! QuO at BBN MIC/GME at Vanderbilt XOTS
20

STUBS

IDL

ORB RUN-TIME
SCHEDULER

SKELETON

REAL-TIME OBJECT ADAPTER

IOP

REAL-TIME ORB CORE

PLUGGABLE ORB & XPORT PROTOCOLS PLUGGABLE ORB & XPORT PROTOCOLS

IOP

OS KERNEL REAL-TIME I/O

SUBSYSTEM HIGH-SPEED NETWORK INTERFACE

COMPONENTS

ACE

OS KERNEL REAL-TIME I/O

SUBSYSTEM HIGH-SPEED NETWORK INTERFACE

NETWORK

www.cs.wustl.edu/schmidt/TAO. html
Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

TAO Statistics
TAO order of magnitude Core ORB 300,000 LOC IDL compiler 200,000 LOC CORBA Object Services 250,000 LOC Leverages ACE heavily Ported to UNIX, Windows, & RT/embedded platforms e.g., VxWorks, LynxOS, Chorus, WinCE
 50 person-years of effort

Currently used by many companies e.g., Boeing, BBN, Lockheed, Lucent, Motorola, Raytheon, SAIC, Siemens, etc. Supported commercially by OCI and PrismTech www.ociweb.com www.prismtechnologies.com

Vanderbilt University

21

Advanced ACE Tutorial

Douglas C. Schmidt

JAWS Adaptive Web Server

WWW
CLIENT

1: GET ~schmidt HTTP/1.0 2: index.html

WWW
SERVER

JAWS Overview A high-performance Web server  Flexible concurrency and dispatching mechanisms Leverages the ACE framework  Ported to most OS platforms Used commercially by CacheFlow  www.cacheow.com

HTML
PARSER

PROTOCOL HANDLERS

GUI
REQUESTER

DISPATCHER

GRAPHICS ADAPTER

COMMUNICATION PROTOCOL (E.G., HTTP)

OS KERNEL
OS I/O SUBSYSTEM NETWORK ADAPTERS

OS KERNEL
OS I/O SUBSYSTEM NETWORK ADAPTERS

NETWORK

www.cs.wustl.edu/jxh/ research/
Vanderbilt University

22

Advanced ACE Tutorial

Douglas C. Schmidt

Java ACE
DISTRIBUTED SERVICES AND COMPONENTS TOKEN SERVER LOGGING SERVER NAME SERVER TIME SERVER

Java ACE Overview A Java version of ACE Used for medical imaging prototype

FRAMEWORKS AND CLASS CATEGORIES

ACCEPTOR

CONNECTOR

SERVICE HANDLER

ADAPTIVE SERVICE EXECUTIVE (ASX)

JAVA

SYNCH

WRAPPERS WRAPPERS

SOCK_SAP

THREAD MANAGER

LOG MSG

TIMER QUEUE

SERVICE CONFIGURATOR

JAVA VIRTUAL MACHINE (JVM)

www.cs.wustl.edu/schmidt/JACE.html www.cs.wustl.edu/schmidt/C++2java. html www.cs.wustl.edu/schmidt/PDF/ MedJava.pdf

Vanderbilt University

23

Advanced ACE Tutorial

Do

Networked Logging Service

PRINTER

P1 P2
LOCAL

IPC

CLIENT LOGGING DAEMON

CONSOLE

P3

RE MO TE

IPC

SERVER LOGGING DAEMON

HA A OST HOST B B

HOST

CLIENT

SERVER

NETWORK
HOST

RE MO TE

IPC
STORAGE DEVICE

CLIENT P2

P1
LOCAL

IPC

CLIENT LOGGING DAEMON

P3

Intent: Server logging daemon collects, formats, and outputs logging records forwarded from client logging daemons residing throughout a network or Internet
Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Networked Logging Service Programming API

The logging API is similar to printf(), e.g.:
ACE_ERROR ((LM_ERROR, "(%t) fork failed"));

Generates on logging server host:

Oct 31 14:50:13 [email protected]@2766@LM_ERROR@client ::(4) fork failed

and
ACE_DEBUG ((LM_DEBUG, "(%t) sending to server %s", server_host));

generates on logging server host:

Oct 31 14:50:28 [email protected]@18352@LM_DEBUG@drwho ::(6) sending to server bastille
Vanderbilt University 25

Advanced ACE Tutorial

Douglas C. Schmidt

Conventional Logging Server Design

Typical algorithmic pseudo-code for The grand mistake: networked logging server: Avoid the temptation to void logging_server (void) { step-wise rene this initialize acceptor endpoint algorithmically decomposed loop forever { pseudo-code directly into wait for events the detailed design and handle data events implementation of the handle connection events logging server! } }

Vanderbilt University

26

Advanced ACE Tutorial

Do

The select()-based Logging Server Implementation

SERVER SERVER LOGGING DAEMON

NETWORK
CLIENT CONNECTION REQUEST

maxhandlep1 acceptor read_handles

LOGGING RECORDS

LOGGING RECORDS

LOGGING RECORDS

CLIENT

CLIENT

CLIENT

Serializes server processing at select() demuxing level

Vanderbilt University

Advanced ACE Tutorial

Do

Conventional Logging Server Implementation

Note the excessive amount of detail required to program at the socket level...
// Main program static const int PORT = 10000; typedef u_long COUNTER; typedef int HANDLE; // Counts the # of logging records processed static COUNTER request_count; // Acceptor-mode socket handle static HANDLE acceptor; // Highest active handle number, plus 1 static HANDLE maxhp1; // Set of currently active handles static fd_set activity_handles; // Scratch copy of activity_handles static fd_set ready_handles;

Vanderbilt University

Advanced ACE Tutorial

Do

Main Event Loop of Logging Server

int main (int argc, char *argv[]) { initialize_acceptor (argc > 1 ? atoi (argv[1]) : PORT); // Loop forever performing logging // server processing. for (;;) { // struct assignment. ready_handles = activity_handles; // Wait for client I/O events. select (maxhp1, &ready_handles, 0, 0, 0); // First receive pending logging records. handle_data (); // Then accept pending connections. handle_connections (); } }

Vanderbilt University

Advanced ACE Tutorial

Do

Initialize Acceptor Socket

static void initialize_acceptor (u_short port) { struct sockaddr_in saddr; // Create a local endpoint of communication. acceptor = socket (PF_INET, SOCK_STREAM, 0); // Set up the address info. to become server. memset ((void *) &saddr, 0, sizeof saddr); saddr.sin_family = AF_INET; saddr.sin_port = htons (port); saddr.sin_addr.s_addr = htonl (INADDR_ANY); // Associate address with endpoint bind (acceptor, (struct sockaddr *) &saddr, sizeof saddr); // Make endpoint listen for connection requests. listen (acceptor, 5); // Initialize handle sets. FD_ZERO (&ready_handles); FD_ZERO (&activity_handles); FD_SET (acceptor, &activity_handles); maxhp1 = acceptor + 1; }
Vanderbilt University

Advanced ACE Tutorial

Do

Handle Data Processing

static void handle_data (void) { // acceptor + 1 is the lowest client handle for (HANDLE h = acceptor + 1; h < maxhp1; h++) if (FD_ISSET (h, &ready_handles)) { ssize_t n = handle_log_record (h, 1); // Guaranteed not to block in this case! if (n > 0) ++request_count; // Count the # of logging records else if (n == 0) { // Handle connection shutdown. FD_CLR (h, &activity_handles); close (h); if (h + 1 == maxhp1) { // Skip past unused handles while (!FD_ISSET (--h, &activity_handles)) continue; maxhp1 = h + 1; } } } }

Vanderbilt University

Advanced ACE Tutorial

Do

Receive and Process Logging Records

static ssize_t handle_log_record (HANDLE in_h, HANDLE out_h) { ssize_t n; size_t len; Log_Record lr; // The first recv reads the length (stored as a // fixed-size integer) of adjacent logging record. n = recv (in_h, (char *) &len, sizeof len, 0); if (n <= 0) return n; len = ntohl (len); // Convert byte-ordering // The second recv then reads <len> bytes to // obtain the actual record. for (size_t nread = 0; nread < len; nread += n n = recv (in_h, ((char *) &lr) + nread, len - nread, 0); // Decode and print record. decode_log_record (&lr); if (write (out_h, lr.buf, lr.size) == -1) return -1; else return 0; }

Vanderbilt University

Advanced ACE Tutorial

Do

Handle Connection Acceptance

static void handle_connections (void) { if (FD_ISSET (acceptor, &ready_handles)) { static struct timeval poll_tv = {0, 0}; HANDLE h; // Handle all pending connection requests // (note use of selects polling feature) do { // Beware of subtle bug(s) here... h = accept (acceptor, 0, 0); FD_SET (h, &activity_handles); // Grow max. socket handle if necessary. if (h >= maxhp1) maxhp1 = h + 1; } while (select (acceptor + 1, &ready_handles, 0, 0, &poll_tv) == 1); }

Vanderbilt University

Advanced ACE Tutorial

Do

Conventional Client Logging Daemon Implementation

The main() method receives logging records from client applications and forwards them on to the logging server
int main (int argc, char *argv[]) { HANDLE stream = initialize_stream_endpoint (argc > 1 ? atoi (argv[1]) : PORT); Log_Record lr; // Loop forever performing client // logging daemon processing. for (;;) { // ... get logging records from client // application processes ... size_t size = htonl (lr.size); send (stream, &size, sizeof size); encode_log_record (&lr); send (stream, ((char *) &lr), sizeof lr); } }
Vanderbilt University

Advanced ACE Tutorial

Do

Client Connection Establishment

static HANDLE initialize_stream_endpoint (const char *host, u_short port) { struct sockaddr_in saddr; // Create a local endpoint of communication. HANDLE stream = socket (PF_INET, SOCK_STREAM, 0); // Set up the address info. to become client. memset ((void *) &saddr, 0, sizeof saddr); saddr.sin_family = AF_INET; saddr.sin_port = htons (port); hostent *hp = gethostbyname (host); memcpy ((void *) &saddr, htonl (hp->h_addr), hp->h_length); // Associate address with endpoint connect (stream, (struct sockaddr *) &saddr, sizeof saddr); return stream; }

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Limitations with Algorithmic Decomposition

Algorithmic decomposition tightly couples application-specic functionality and the following conguration-related characteristics: Application Structure The number of services per process Time when services are congured into a process Communication and Demultiplexing Mechanisms The underlying IPC mechanisms that communicate with other participating clients and servers Event demultiplexing and event handler dispatching mechanisms Concurrency and Synchronization Model The process and/or thread architecture that executes service(s) at run-time
Vanderbilt University 36

Advanced ACE Tutorial

Douglas C. Schmidt

Overcoming Limitations via OO

The algorithmic decomposition illustrated above species many low-level details Moreover, the excessive coupling impedes reusability, extensibility, and portability... In contrast, OO focuses on application-specic behavior, e.g.,
int Logging_Handler::handle_input (void) { ssize_t n = handle_log_record (peer ().get_handle (), ACE_STDOUT); if (n > 0) ++request_count; // Count the # of logging records return n <= 0 ? -1 : 0; }

Vanderbilt University

37

Advanced ACE Tutorial

Douglas C. Schmidt

OO Contributions to Software
Patterns facilitate the large-scale reuse of software architecture
Even when reuse of algorithms, detailed designs, and implementations is not feasible

Frameworks achieve large-scale design and code reuse

In contrast, traditional techniques focus on the functions and algorithms that solve particular requirements Note that patterns and frameworks are not unique to OO! However, objects and classes are useful abstraction mechanisms

Vanderbilt University

38

Advanced ACE Tutorial

Douglas C. Schmidt

Patterns in the Networked Logging Server

Active Object Component Configurator
STRATEGIC PATTERNS
TACTICAL PATTERNS

Acceptor Reactor

Adapter

Iterator

Factory Method

Template Method

Wrapper Facade

Strategic and tactical are relative to the context and abstraction level
Vanderbilt University 39

Advanced ACE Tutorial

Douglas C. Schmidt

Wrapper Facade ! Encapsulates the functions and data provided by existing non-OO APIs within more concise, robust, portable, maintainable, and cohesive OO class interfaces Reactor ! Demultiplexes and dispatches requests that are delivered concurrently to an application by one or more clients Acceptor ! Decouple the passive connection and initialization of a peer service in a distributed system from the processing performed once the peer service is connected and initialized Component Congurator ! Decouples the implementation of services from the time when they are congured Active Object ! Decouples method execution from method invocation to enhance concurrency and simplify synchronized access to an object that resides in its own thread of control
Vanderbilt University 40

Summary of Pattern Intents

Advanced ACE Tutorial

Douglas C. Schmidt

Components in the OO Logging Server

Application-specic components
Process logging records received from clients

Connection-oriented application components

ACE_Svc_Handler (service handler)  Performs I/O-related tasks with clients ACE_Acceptor factory  Passively accepts connection requests  Dynamically creates a service handler for each client and activates it

Application-independent ACE framework components

Perform IPC, explicit dynamic linking, event demultiplexing, event handler dispatching, multi-threading, etc.
Vanderbilt University 41

Advanced ACE Tutorial

Do

Class Diagram for OO Logging Server

APPLICATIONSPECIFIC COMPONENTS
Logging_Handler SOCK_Acceptor 1 SOCK_Stream Null_Synch

Logging Acceptor

n
<<activates>>

Logging Handler

CONNECTIONORIENTED COMPONENTS

SVC_HANDLER PEER_ACCEPTOR

PEER_STREAM SYNCH_STRAT

Acceptor

Svc Handler

PEER ACCEPTOR

PEER STREAM

ACE FRAMEWORK COMPONENTS

Connection IPC_SAP

Stream

Service Configurator

Concurrency

Reactor

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Addressing Robustness, Portability, and Maintainability Challenges

Problem Building distributed applications using low-level APIs is hard Forces Low-level APIs are verbose, tedious, and error-prone to program Low-level APIs are non-portable and non-maintainable Solution Apply the Wrapper Facade pattern to encapsulate low-level functions and data structures

Vanderbilt University

43

Advanced ACE Tutorial

Douglas C. Schmidt

The Wrapper Facade Pattern

Intent

Encapsulates the functions and data provided by existing lower-level, non-OO APIs within more concise, robust, portable, maintainable, and cohesive higher-level OO class interfaces

client
1: method_k()

Wrapper Facade
method_1() ... method_m()

2: function_k()

Functions
function_1() ... function_n()

POSA2 (www.cs.wustl.edu/ schmidt/POSA/)

Forces Resolved Avoid tedious, error-prone, and non-portable system APIs Create cohesive abstractions
Vanderbilt University 44

Advanced ACE Tutorial

Do

Motivating the Wrapper Facade Pattern: the Socket API

2: ACTIVE socket() bind() connect() send()/recv() close()
PROCESSING (optional) ROLE

1: PASSIVE
ROLE

3: SERVICE

socket() bind() listen() accept() send()/recv() close()

CLIENT

NETWORK

SERVER

Sockets are the most common network programming API and are available on most OS platforms

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Problem with Sockets: Lack of Type-safety

int buggy_echo_server (u_short port_num) I/O handles are { // Error checking omitted. not amenable to sockaddr_in s_addr; strong type int acceptor = checking at socket (PF_UNIX, SOCK_DGRAM, 0); compile-time s_addr.sin_family = AF_INET; s_addr.sin_port = port_num; The adjacent s_addr.sin_addr.s_addr = INADDR_ANY; bind (acceptor, (sockaddr *) &s_addr, code contains sizeof s_addr); many subtle, int handle = accept (acceptor, 0, 0); common bugs for (;;) { char buf[BUFSIZ]; ssize_t n = read (acceptor, buf, sizeof buf); if (n <= 0) break; write (handle, buf, n); } }
Vanderbilt University 46

Advanced ACE Tutorial

Douglas C. Schmidt

Problem with Sockets: Steep Learning Curve

Many socket/TLI API functions have complex semantics, e.g.: Multiple protocol families and address families e.g., TCP, UNIX domain, OSI, XNS, etc. Infrequently used features, e.g.: Broadcasting/multicasting Passing open le handles Urgent data delivery and reception Asynch I/O, non-blocking I/O, I/O-based and timer-based event multiplexing

Vanderbilt University

47

Advanced ACE Tutorial

Douglas C. Schmidt

Problem with Sockets: Portability

Having multiple standards, i.e., sockets and TLI, makes portability difcult, e.g., May require conditional compilation In addition, related functions are not included in POSIX standards  e.g., select(), WaitForMultipleObjects(), and poll() Portability between UNIX and Windows Sockets is problematic, e.g.: Header les Error numbers Handle vs. descriptor types Shutdown semantics I/O controls and socket options
48

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Problem with Sockets: Poorly Structured

Limitations Socket API is linear rather than hierarchical There is no consistency among names... Non-portable

Vanderbilt University

socket() bind() connect() listen() accept() read() write() readv() writev() recv() send() recvfrom() sendto() recvmsg() sendmsg() setsockopt() getsockopt() getpeername() getsockname() gethostbyname() getservbyname()

49

Advanced ACE Tutorial

Do

Socket Taxonomy
N/ TIO ION EC AT E NN NIC SIV S CO MU LE PA M RO ER CO XF IVE CT A
COMMUNICATION DOMAIN LOCAL LOCAL/REMOTE

TYPE OF COMMUNICATION SERVICE

DATA GRAM

socket(PF_UNIX)/bind() sendto()/recvfrom() socket(PF_UNIX)/bind()

socket(PF_INET)/bind() sendto()/recvfrom()

socket(PF_INET)/bind()

STREAM CONNECTED DATAGRAM

socket(PF_UNIX) socket(PF_INET) bind()/connect() bind()/connect() send()/recv() send()/recv() socket(PF_UNIX) socket(PF_INET) bind()/connect() bind()/connect()
bind()/listen()/accept()

socket(PF_UNIX)

bind()/listen()/accept()

socket(PF_INET)

send()/recv() socket(PF_UNIX) bind()/connect()

send()/recv() socket(PF_INET) bind()/connect()

The Socket API can be classied along three dimensions 1. Connection role 2. Communication domain 3. Type of service
Vanderbilt University

Advanced ACE Tutorial

Do

Solution: ACE Socket Wrapper Facades

N/ IO ION CT T E NE ICA N SIV N S CO MU LE PA M RO ER CO XF E TIV AC ram Dg _ CK O LS
COMMUNICATION DOMAIN LOCAL LOCAL/REMOTE

TYPE OF COMMUNICATION SERVICE

m gra _D CK SO SOCK_Dgram_Bcast SOCK_Dgram_Mcast

SOCK_Dgram SOCK_CODgram SOCK_Acceptor SOCK_Stream SOCK_Connector

CONNECTED DATAGRAM

DATA GRAM

LSOCK_Dgram LSOCK_CODgram

STREAM

LSOCK_Acceptor LSOCK_Stream LSOCK_Connector

The ACE C++ wrapper facades more explicitly model the key socket components using OO classes

Vanderbilt University

Advanced ACE Tutorial

Do

The ACE Connection-Oriented Socket Wrapper Facades

ACE_IPC_SAP ACE_Addr

ACE_SOCK_IO

ACE_SOCK

ACE_SOCK_Acceptor

ACE_INET_Addr

ACE_SOCK_Stream

ACE_SOCK_Connector

Participants Passive and active connection factories ACE_SOCK_Acceptor and ACE_SOCK_Connector Streaming classes ACE_SOCK_Stream and ACE_SOCK_IO Addressing classes ACE_Addr and ACE_INET_Addr

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

The ACE Connection-Oriented Socket Wrapper Facade Factories

class ACE_SOCK_Connector { public: // Traits typedef ACE_INET_Addr PEER_ADDR; typedef ACE_SOCK_Stream PEER_STREAM; int connect (ACE_SOCK_Stream &new_sap, const ACE_INET_Addr &raddr, ACE_Time_Value *timeout, const ACE_INET_Addr &laddr); // ... }; class ACE_SOCK_Acceptor : public ACE_SOCK { public: // Traits typedef ACE_INET_Addr PEER_ADDR; typedef ACE_SOCK_Stream PEER_STREAM;

ACE_SOCK_Acceptor (const ACE_INET_Addr &); int open (const ACE_INET_Addr &addr); int accept (ACE_SOCK_Stream &new_sap, ACE_INET_Addr *, ACE_Time_Value *); //... };

Vanderbilt University

53

Advanced ACE Tutorial

Douglas C. Schmidt

ACE Connection-Oriented Socket Wrapper Facade Streaming and Addressing Classes

class ACE_SOCK_Stream : public ACE_SOCK { public: // Trait. typedef ACE_INET_Addr PEER_ADDR; ssize_t send (const void *buf, int n); ssize_t recv (void *buf, int n); ssize_t send_n (const void *buf, int n); ssize_t sendv_n (const iovec *iov, int n); ssize_t recv_n (void *buf, int n); int close (void); // ... }; class ACE_INET_Addr : public ACE_Addr { public: ACE_INET_Addr (u_short port, const char host[]); u_short get_port_number (void); ACE_UINT_32 get_ip_addr (void); // ... };

Vanderbilt University

54

Advanced ACE Tutorial

Douglas C. Schmidt

Design Interlude: Motivating the Socket Wrapper Facade Structure

Q: Why decouple the ACE_SOCK_Acceptor and the ACE_SOCK_Connector from ACE_SOCK_Stream? A: For the same reasons that ACE_Acceptor and ACE_Connector are decoupled from ACE_Svc_Handler, e.g., An ACE_SOCK_Stream is only responsible for data transfer  Regardless of whether the connection is established passively or actively This ensures that the ACE_SOCK* components arent used incorrectly...  e.g., you cant accidentally read() or write() on ACE_SOCK_Connectors or ACE_SOCK_Acceptors, etc.
Vanderbilt University 55

Advanced ACE Tutorial

Do

An Echo Server Written using ACE C++ Socket Wrapper Facades

int echo_server (u_short port_num) { // Local server address. ACE_INET_Addr my_addr (port_num); // Initialize the acceptor mode server. ACE_SOCK_Acceptor acceptor (my_addr); // Data transfer object. ACE_SOCK_Stream new_stream; // Accept a new connection. acceptor.accept (new_stream); for (;;) { char buf[BUFSIZ]; // Error caught at compile time! ssize_t n = acceptor.recv (buf, sizeof buf); new_stream.send_n (buf, n); } }
Vanderbilt University

Advanced ACE Tutorial

Do

A Generic Version of the Echo Server

template <class ACCEPTOR> int echo_server (u_short port) { // Local server address (note traits). typename ACCEPTOR::PEER_ADDR my_addr (port); // Initialize the acceptor mode server. ACCEPTOR acceptor (my_addr); // Data transfer object (note traits). typename ACCEPTOR::PEER_STREAM stream; // Accept a new connection. acceptor.accept (stream); for (;;) { char buf[BUFSIZ]; ssize_t n = stream.recv (buf, sizeof buf); stream.send_n (buf, n); } }
Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Scope of the ACE IPC Wrapper Facades

ACE IPC SAP SOCK SAP
SOCKET API A

TLI SAP
TLI API

SPIPE SAP
STREAM PIPE API

SSL SAP
SSL API

FIFO SAP
NAMED PIPE API

MEM SAP
MMAP API

SysV IPC
SYSTEM V IPC API

C++NPv1 (www.cs.wustl.edu/schmidt/ACE/book1/)

Vanderbilt University

58

Advanced ACE Tutorial

Do

Using the Wrapper Facade Pattern for the Logging Server

Note we havent improved the overall design (yet)
// ... Same as before ... // Acceptor-mode socket handle. static ACE_SOCK_Acceptor acceptor; // Set of currently active handles static ACE_Handle_Set activity_handles; // Scratch copy of activity_handles static ACE_Handle_Set ready_handles; static void initialize_acceptor (u_short port) { // Set up address info. to become server. ACE_INET_Addr saddr (port); // Create a local endpoint of communication. acceptor.open (saddr); // Set the <SOCK_Acceptor> into non-blocking mode. acceptor.enable (ACE_NONBLOCK); activity_handles.set_bit (acceptor.get_handle ()); }
Vanderbilt University

Advanced ACE Tutorial

Do

Main Event Loop of Logging Server

int main (int argc, char *argv[]) { initialize_acceptor (argc > 1 ? atoi (argv[1]) : PORT); // Loop forever performing logging // server processing. for (;;) { // object assignment. ready_handles = activity_handles; // Wait for client I/O events. ACE::select (int (maxhp1), // calls operator fd_set *(). ready_handles); // First receive pending logging records. handle_data (); // Then accept pending connections. handle_connections (); } }

Vanderbilt University

Advanced ACE Tutorial

Do

Handling Connections and Data Processing

static void handle_connections (void) { if (ready_handles.is_set (acceptor.get_handle ())) ACE_SOCK_Stream str; // Handle all pending connection requests. while (acceptor.accept (str) != -1) activity_handles.set_bit (str.get_handle ()); } } static void handle_data (void) { ACE_HANDLE h; ACE_Handle_Set_Iterator iter (ready_handles); while ((h = iter ()) != ACE_INVALID_HANDLE) { ACE_SOCK_Stream str (h); ssize_t n = handle_log_record (str, ACE_STDOUT); if (n > 0) // Count # of logging records. ++request_count; else if (n == 0) { // Handle connection shutdown. activity_handles.clr_bit (h); s.close (); } }

Vanderbilt University

Advanced ACE Tutorial

Do

Receive and Process Logging Records

static ssize_t handle_log_record (ACE_SOCK_Stream s, ACE_HANDLE out_h) ACE_UINT_32 len; ACE_Log_Record lr; // The first recv reads the length (stored as a // fixed-size integer) of adjacent logging record. ssize_t n = s.recv_n ((char *) &len, sizeof len); if (n <= 0) return n; len = ntohl (len); // Convert byte-ordering // Perform sanity check! if (len > sizeof (lr)) return -1; // The second recv then reads <len> bytes to // obtain the actual record. s.recv_n ((char *) &lr, sizeof lr); // Decode and print record. decode_log_record (&lr); if (ACE_OS::write (out_h, lr.buf, lr.size) == -1) return -1; else return 0; }

Vanderbilt University

Advanced ACE Tutorial

Do

OO Client Logging Daemon Implementation

int main (int argc, char *argv[]) { ACE_SOCK_Stream stream; ACE_SOCK_Connector con; // Establish connection. con.connect (stream, ACE_INET_Addr (argc > 1 ? atoi (argv[1]) : PORT)); ACE_Log_Record lr; // Loop forever performing client // logging daemon processing. for (;;) { // ... get logging records from client // application processes ... ACE_UINT_32 size = lr.size; lr.size = htonl (lr.size); encode_log_record (&lr); iovec iov[2]; iov[0].iov_len = sizeof (ACE_UINT_32); iov[0].iov_base = &lr.size; iov[1].iov_len = size; iov[1].iov_base = &lr; // Uses writev(2); stream.sendv_n (iov, 2); } }

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Evaluating the Wrapper Facade Solution

Benets More concise More robust More portable More maintainable More efcient Liabilities Potentially more indirection Additional learning curve Still havent solved the overall design problem i.e., the overall design is still based on step-wise renement of functions

Vanderbilt University

64

Advanced ACE Tutorial

Douglas C. Schmidt

ACE C++ Wrapper Facade Design Refactoring Principles

Enforce typesafety at compile-time Allow controlled violations of typesafety Simplify for the common case Replace one-dimensional interfaces with hierarchical class categories Enhance portability with parameterized types Inline performance critical methods Dene auxiliary classes to hide error-prone details
Vanderbilt University 65

Advanced ACE Tutorial

Douglas C. Schmidt

Enforce Typesafety at Compile-Time

Sockets cannot detect certain errors at compile-time, e.g.,
int acceptor = socket (PF_INET, SOCK_STREAM, 0); // ... bind (acceptor, ...); // Bind address. listen (acceptor); // Make a acceptor-mode socket. HANDLE n_sd = accept (acceptor, 0, 0); // Error not detected until run-time. read (acceptor, buf, sizeof buf);

ACE enforces type-safety at compile-time via factories, e.g.:

ACE_SOCK_Acceptor acceptor (port); // Error: recv() not a method of <ACE_SOCK_Acceptor>. acceptor.recv (buf, sizeof buf);

Vanderbilt University

66

Advanced ACE Tutorial

Douglas C. Schmidt

Allow Controlled Violations of Typesafety

Make it easy to use the C++ Socket wrapper facades correctly, hard to use it incorrectly, but not impossible to use it in ways the class designers did not anticipate e.g., it may be necessary to retrieve the underlying socket handle:
ACE_SOCK_Acceptor acceptor; // ... ACE_Handle_Set ready_handles; // ... if (ready_handles.is_set (acceptor.get_handle ()) ACE::select (acceptor.get_handle () + 1, ready_handles);

Vanderbilt University

67

Advanced ACE Tutorial

Douglas C. Schmidt

Supply Default Parameters

ACE_SOCK_Connector (ACE_SOCK_Stream &new_stream, const ACE_Addr &remote_sap, ACE_Time_Value *timeout = 0, const ACE_Addr &local_sap = ACE_Addr::sap_any, int protocol_family = PF_INET, int protocol = 0);

The result is extremely concise for the common case:

ACE_SOCK_Stream stream; // Compiler supplies default values. ACE_SOCK_Connector con (stream, ACE_INET_Addr (port, host));

Vanderbilt University

68

Advanced ACE Tutorial

Do

Dene Parsimonious Interfaces

e.g., use LSOCK to pass socket handles:
ACE_LSOCK_Stream stream; ACE_LSOCK_Acceptor acceptor ("/tmp/foo"); acceptor.accept (stream); stream.send_handle (stream.get_handle ());

versus the less parsimonious BSD 4.3 socket code

ACE_LSOCK::send_handle (const ACE_HANDLE sd) const { u_char a[2]; iovec iov; msghdr send_msg; a[0] = 0xab, a[1] = 0xcd; iov.iov_base = (char *) a; iov.iov_len = sizeof a; send_msg.msg_iov = &iov; send_msg.msg_iovlen = 1; send_msg.msg_name = (char *) 0; send_msg.msg_namelen = 0; send_msg.msg_accrights = (char *) &sd; send_msg.msg_accrightslen = sizeof sd; return sendmsg (this->get_handle (), &send_msg, 0);

Note that SVR4 and BSD 4.4 APIs are different than BSD 4.3!
Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Combine Multiple Operations into One Operation

Creating a conventional acceptor-mode socket requires multiple calls:
int acceptor = socket (PF_INET, SOCK_STREAM, 0); sockaddr_in addr; memset (&addr, 0, sizeof addr); addr.sin_family = AF_INET; addr.sin_port = htons (port); addr.sin_addr.s_addr = INADDR_ANY; bind (acceptor, &addr, addr_len); listen (acceptor); // ...

ACE_SOCK_Acceptor combines this into a single operation:

ACE_SOCK_Acceptor acceptor ((ACE_INET_Addr) port);

Vanderbilt University

70

Advanced ACE Tutorial

Douglas C. Schmidt

Create Hierarchical Class Categories

ACE IPC SAP
ACE SOCK Dgram Bcast ACE SOCK Dgram Mcast
GROUP COMM

ACE SOCK

ACE SOCK Dgram ACE LSOCK Dgram

DATAGRAM COMM

ACE SOCK CODgram ACE LSOCK CODgram

ACE SOCK Stream ACE LSOCK Stream

STREAM COMM

ACE SOCK Connector ACE LSOCK Connector

ACE SOCK Acceptor ACE LSOCK Acceptor

ACE LSOCK

CONNECTION ESTABLISHMENT

Vanderbilt University

71

Advanced ACE Tutorial

Douglas C. Schmidt

Enhance Portability with Parameterized Types

DISTRIBUTED 1 APPLICATION APPLICATION1 DISTRIBUTED 2 APPLICATION APPLICATION2 DISTRIBUTED 3 APPLICATION APPLICATION3

(PARAMETERIZED TYPES)

COMMON INTERFACE

SOCK_SAP
SOCKET BSDSOCKET API API

TLI_SAP
BSDSOCKET SYSTEM V API TLI API
USER SPACE

// Conditionally select IPC mechanism. #if defined (USE_SOCKETS) typedef ACE_SOCK_Acceptor PEER_ACCEPTOR; #elif defined (USE_TLI) typedef ACE_TLI_Acceptor PEER_ACCEPTOR; #endif // USE_SOCKETS. int main (void) { // ... // Invoke with appropriate // network programming interface. echo_server<PEER_ACCEPTOR> (port); }

OS KERNEL PROTOCOL MECHANISMS (TCP/IP, OSI, ETC.)

NETWORK INTERFACE

KERNEL SPACE

Switching wholesale between sockets and TLI simply requires instantiating a different ACE C++ wrapper facade
Vanderbilt University 72

Advanced ACE Tutorial

Douglas C. Schmidt

Inline Performance Critical Methods

Inlining is time and space efcient since key methods are very short:
class ACE_SOCK_Stream : public ACE_SOCK { public: ssize_t send (const void *buf, size_t n) { return ACE_OS::send (this->get_handle (), buf, n); } ssize_t recv (void *buf, size_t n) { return ACE_OS::recv (this->get_handle (), buf, n); } };

Vanderbilt University

73

Advanced ACE Tutorial

Douglas C. Schmidt

Dene Auxiliary Classes to Hide Error-Prone Details

Standard C socket addressing is awkward and error-prone

e.g., easy to neglect to zero-out a sockaddr_in or convert port numbers to network byte-order, etc.
ACE C++ Socket wrapper facades dene classes to handle details
class ACE_INET_Addr : public ACE_Addr { public: ACE_INET_Addr (u_short port, long ip_addr = 0) { memset (&this->inet_addr_, 0, sizeof this->inet_addr_); this->inet_addr_.sin_family = AF_INET; this->inet_addr_.sin_port = htons (port); memcpy (&this->inet_addr_.sin_addr, &ip_addr, sizeof ip_addr); } // ... private: sockaddr_in inet_addr_; };
Vanderbilt University 74

Advanced ACE Tutorial

Douglas C. Schmidt

Demultiplexing and Dispatching Events

Problem The logging server must process several different types of events simultaneously from different sources of events Forces Multi-threading is not always available Multi-threading is not always efcient Multi-threading can be error-prone Tightly coupling event demuxing with server-specic logic is inexible

Solution Use the Reactor pattern to decouple event demuxing/dispatching from server-specic processing
Vanderbilt University 75

Advanced ACE Tutorial

Douglas C. Schmidt

The Reactor Pattern

Reactor handle_events() register_handler(h) remove_handler(h)
uses handlers 1 owns select (handles); foreach h in handles loop table[h].handle_event(type) end loop N

Intent

Handle
notifies

Event Handler handle_event(type) get_handle() Concrete Event Handler

Demuxes & dispatches requests that are delievered concurrency to an application by one or more clients
Forces Resolved Serially demux events synchronously & efciently Extend applications without changing demuxing code

Synchronous Event Demultiplexer select()

www.cs.wustl.edu/schmidt/ POSA/

Vanderbilt University

76

Advanced ACE Tutorial

Douglas C. Schmidt

Collaboration in the Reactor Pattern

callback : main Reactor Concrete program Event_Handler Reactor() register_handler(callback) get_handle() handle_events() select() handle_input() handle_output() handle_signal() handle_timeout() remove_handler(callback) handle_close()

INITIALIZATION MODE

Note inversion of control Also note how long-running event handlers can degrade quality of service since callbacks steal Reactors thread of control...

INITIALIZE

REGISTER HANDLER EXTRACT HANDLE START EVENT LOOP FOREACH EVENT DO

EVENT HANDLING MODE

DATA ARRIVES OK TO SEND SIGNAL ARRIVES TIMER EXPIRES REMOVE HANDLER CLEANUP

Vanderbilt University

77

Advanced ACE Tutorial

Do

Structure and Implementations of the ACE Reactor Framework

Reactor framework participants
ACE_Timer_Queue ACE_Event_Handler

0..1

ACE_Reactor ACE_Time_Value Application Event Handler

Common Reactor implementations in ACE

1

ACE_Reactor

ACE_Reactor_Impl

ACE_WFMO_Reactor

TOKEN

ACE_Select_Reactor_Impl

ACE_Select_Reactor_T
bind <ACE_Select_Reactor_Token>

ACE_TP_Reactor

ACE_Select_Reactor

Vanderbilt University

Advanced ACE Tutorial

Do

Using the ACE Reactor Framework in the Logging Server

REGISTERED 2: sh = new Logging_Handler : Logging OBJECTS 3: accept (sh->peer()) : Logging 4: sh->open() Handler Handler : Logging : Event Acceptor : Event 5: handle_input() Handler Handler 6: recv(msg) : Event 7:process(msg) Handler 1: handle_input()

FRAMEWORK LEVEL

APPLICATION LEVEL

:Timer Queue

: Handle Table

: Reactor

KERNEL LEVEL

OS EVENT DEMULTIPLEXING INTERFACE

Benets Straightforward to program Concurrency control is easy

Liabilities Callbacks are brittle Cant leverage multi-processors

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Addressing Acceptor Endpoint Connection and Initialization Challenges

Problem The communication protocol used between applications is often orthogonal to its connection establishment and service handler initialization protocols Forces Low-level connection APIs are error-prone and non-portable Separating initialization from processing increases software reuse Solution Use the Acceptor pattern to decouple passive connection establishment and connection handler initialization from the subsequent logging protocol
Vanderbilt University 80

Advanced ACE Tutorial

Do

The Acceptor-Connector Pattern (Acceptor Role)

Svc Handler
Svc Handler

peer_stream_ open()
APPLICATIONDEFINED

Acceptor
ACTIVATES

peer_acceptor_ accept()

APPLICATIONINDEPENDENT

Reactor

www.cs.wustl.edu/schmidt/POSA/ Intent of Acceptor Role Forces resolved

Decouple the passive connection and initialization of a peer service in a distributed system from the processing performed once the peer service is connected and initialized
Vanderbilt University

Reuse passive connection setup and service initialization code Ensure that acceptor-mode handles arent used to read/write data

Advanced ACE Tutorial

Do

Structure of the ACE Acceptor-Connector Framework

ACE_Event_Handler
SVC_HANDLER, PEER_CONNECTOR SYNCH_STRATEGY

ACE_Task

ACE_Connector
PEER_STREAM, SYNCH_STRATEGY SVC_HANDLER, PEER_ACCEPTOR

ACE_Svc_Handler
bind

ACE_Acceptor

Application Service

Framework characteristics Uses C++ parameterized types to strategize IPC and service aspects Uses Template Method pattern to strategize creation, connection establishment, and concurrency policies
Vanderbilt University

Advanced ACE Tutorial

Do

Using the ACE_Acceptor in the Logging Server

: Logging Handler : Svc Handler : Logging Handler : Svc Handler : Logging Handler : Svc Handler : Logging Handler : Svc Handler

: Logging Acceptor : Acceptor

PASSIVE LISTENER

ACTIVE CONNECTIONS

1: handle_input() 2: sh = make_svc_handler() 3: accept_svc_handler(sh) 4: activate_svc_handler(sh)

: Reactor

The ACE_Acceptor is a factory i.e., it creates, connects, and activates an ACE_Svc_Handler Theres often one ACE_Acceptor per-service/per-port
Vanderbilt University

Advanced ACE Tutorial

Do

ACE_Acceptor Class Public Interface

template <class SVC_HANDLER, // Service aspect class PEER_ACCEPTOR> // IPC aspect class ACE_Acceptor : public ACE_Service_Object { // Inherits indirectly from <ACE_Event_Handler> public: // Initialization. virtual int open (typename const PEER_ACCEPTOR::PEER_ADDR &, ACE_Reactor * = ACE_Reactor::instance ()); // Template Method. virtual int handle_input (ACE_HANDLE); protected: // Factory method creates a service handler. virtual SVC_HANDLER *make_svc_handler (void); // Accept a new connection. virtual int accept_svc_handler (SVC_HANDLER *); // Activate a service handler. virtual int activate_svc_handler (SVC_HANDLER *); private: // Acceptor IPC connection strategy. PEER_ACCEPTOR peer_acceptor_; };

Vanderbilt University

Advanced ACE Tutorial

Do

ACE_Acceptor Class Implementation

// Shorthand names. #define SH SVC_HANDLER #define PA PEER_ACCEPTOR // Template Method that creates, connects, // and activates service handlers. template <class SH, class PA> int ACE_Acceptor<SH, PA>::handle_input (ACE_HANDLE) { // Factory method that makes a service handler. SH *svc_handler = make_svc_handler (); // Accept the connection. accept_svc_handler (svc_handler); // Delegate control to the service handler. activate_svc_handler (svc_handler); }

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

The Template Method Pattern

Abstract Class
template_method() primitive_operation1() primitive_operation2()

Intent
... primitive_operation1() ... primitive_operation2() ...

Dene the skeleton of an algorithm in an operation, deferring some steps to subclasses

Gamma et al., Design Patterns: Elements of Reusable Object-Oriented Software AW, 94

Concrete Class
primitive_operation1() primitive_operation2()

Vanderbilt University

86

Advanced ACE Tutorial

Douglas C. Schmidt

Using the Template Method Pattern in the ACE Acceptor Implementation

Acceptor
handle_input() make_svc_handler() accept_svc_handler() activate_svc_handler() ... make_svc_handler() ... accept_svc_handler() ... activate_svc_handler()

Benets Straightforward to program via inheritance and dynamic binding Liabilities Design is brittle and can cause explosion of subclasses due to whitebox design

My Acceptor
make_svc_handler() activate_svc_handler()

Vanderbilt University

87

Advanced ACE Tutorial

Douglas C. Schmidt

The Strategy Pattern

Intent
Context
context_interface()
STRATEGY

Strategy
algorithm_interface()

Dene a family of algorithms, encapsulate each one, and make them interchangeable
Gamma et al., Design Patterns: Elements of Reusable Object-Oriented Software AW, 94

Concrete Strategy A
algorithm_interface()

Concrete Strategy C
algorithm_interface()

Concrete Strategy B
algorithm_interface()

Vanderbilt University

88

Advanced ACE Tutorial

Douglas C. Schmidt

Using the Strategy Pattern in the ACE Acceptor Implementation

Acceptor
handle_input()
<<delegates>>
2: activate_svc_handler(sh)

Concurrency Strategy
activate_svc_handler()

Benets More extensible due to blackbox design Liabilities More complex and harder to develop initially

sh = create_svc_handler () ... accept_svc_handler (sh) ... 1: activate_svc_handler(sh) ...

Reactive Strategy

Thread Strategy

Process Strategy

Vanderbilt University

89

Advanced ACE Tutorial

Do

ACE_Acceptor Template Method Hook Implementations

Template method hooks can be overridden
// Factory method for creating a service handler. template <class SH, class PA> SH * ACE_Acceptor<SH, PA>::make_svc_handler (ACE_HANDLE) return new SH; // Default behavior. } // Accept connections from clients. template <class SH, class PA> int ACE_Acceptor<SH, PA>::accept_svc_handler (SH *sh) { peer_acceptor_.accept (sh->peer ()); } // Activate the service handler. template <class SH, class PA> int ACE_Acceptor<SH, PA>::activate_svc_handler (SH *sh) { if (sh->open () == -1) sh->close (); }

Vanderbilt University

Advanced ACE Tutorial

Do

ACE_Acceptor Initialization Implementation

Note how the PEER_ACCEPTORs open() method hides all the details associated with passively initializing communication endpoints
// Initialization. template <class SH, class PA> int ACE_Acceptor<SH, PA>::open (typename const PA::PEER_ADDR &addr, ACE_Reactor *reactor) { // Forward initialization to the concrete // peer acceptor. peer_acceptor_.open (addr); // Register with Reactor. reactor->register_handler (this, ACE_Event_Handler::ACCEPT_MASK); }

Vanderbilt University

Advanced ACE Tutorial

Do

ACE_Svc_Handler Class Public Interface

Note how IPC and synchronization aspects are strategized
template <class PEER_STREAM, // IPC aspect class SYNCH_STRAT> // Synch aspect class ACE_Svc_Handler : public ACE_Task<SYNCH_STRAT> // Task is-a Service_Object, // which is-an Event_Handler { public: // Constructor. ACE_Svc_Handler (Reactor * = ACE_Reactor::instance ()); // Activate the handler (called by the . // <ACE_Acceptor> or <ACE_Connector>). virtual int open (void *); // Return underlying IPC mechanism. PEER_STREAM &peer (void); // ... private: PEER_STREAM peer_; // IPC mechanism. virtual ACE_Svc_Handler (void); };

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

ACE_Svc_Handler Implementation
#define PS PEER_STREAM #define SS SYNCH_STRAT template <class PS, class SS> ACE_Svc_Handler<PS, SS>::ACE_Svc_Handler (ACE_Reactor *r): ACE_Service_Object (r) {} template <class PS, class SS> int ACE_Svc_Handler<PS, SS>::open (void *) { // Enable non-blocking I/O. peer ().enable (ACE_NONBLOCK); // Register handler with the Reactor. reactor ()->register_handler (this, ACE_Event_Handler::READ_MASK); }

By default, a ACE_Svc_Handler object is registered with the singleton ACE_Reactor This makes the service reactive so that no other synchronization mechanisms are necessary

Vanderbilt University

93

Advanced ACE Tutorial

Do

Object Diagram for OO Logging Server

SERVER LOGGING DAEMON Logging Acceptor Logging Handler Logging Handler

Service Config Service Repository

Service Manager

Reactor

LOGGING RECORDS

CONNECTION REQUEST

SERVER

REMOTE CONTROL OPERATIONS

CLIENT CLIENT CLIENT CLIENT

Vanderbilt University

Advanced ACE Tutorial

Do

The Logging_Handler and Logging_Acceptor Classes

// Performs I/O with client logging daemons. class Logging_Handler : public ACE_Svc_Handler<ACE_SOCK_Acceptor::PEER_STREAM, // Trait! ACE_NULL_SYNCH> { public: // Recv and process remote logging records. virtual int handle_input (ACE_HANDLE); }; // Logging_Handler factory. class Logging_Acceptor : public ACE_Acceptor<Logging_Handler, ACE_SOCK_Acceptor> { public: // Dynamic linking hooks. virtual int init (int argc, char *argv[]); virtual int fini (void); };

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Design Interlude: Parameterizing IPC Mechanisms

Q: How can you switch between different IPC mechanisms? A: By parameterizing IPC Mechanisms with C++ Templates, e.g.:
#if defined (ACE_USE_SOCKETS) typedef ACE_SOCK_Acceptor PEER_ACCEPTOR; #elif defined (ACE_USE_TLI) typedef ACE_TLI_Acceptor PEER_ACCEPTOR; #endif /* ACE_USE_SOCKETS */ class Logging_Handler : public ACE_Svc_Handler<PEER_ACCEPTOR::PEER_STREAM, // Trait! ACE_NULL_SYNCH> { /* ... /* }; class Logging_Acceptor : public ACE_Acceptor <Logging_Handler, PEER_ACCEPTOR> { /* ... */ };

Vanderbilt University

96

Advanced ACE Tutorial

Douglas C. Schmidt

Logging_Handler Input Method

Callback routine that receives logging records Implementation of application-specic logging method code

int Logging_Handler::handle_input (ACE_HANDLE) { This is the main // Call existing function to recv supplied by a // logging record and print to stdout. developer! ssize_t n = handle_log_record (peer ().get_handle (), ACE_STDOUT); if (n > 0) // Count the # of logging records ++request_count; return n <= 0 ? -1 : 0; }

Vanderbilt University

97

Advanced ACE Tutorial

Do

Logging_Acceptor Initialization and Termination

// Automatically called when a Logging_Acceptor // object is linked dynamically. Logging_Acceptor::init (int argc, char *argv[]) { ACE_Get_Opt get_opt (argc, argv, "p:", 0); ACE_INET_Addr addr (DEFAULT_PORT); for (int c; (c = get_opt ()) != -1; ) switch (c) { case p: addr.set (atoi (getopt.optarg)); break; default: break; } // Initialize endpoint and register // with the <ACE_Reactor>. open (addr, ACE_Reactor::instance ()); } // Automatically called when object is unlinked. Logging_Acceptor::fini (void) { handle_close (); }
Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Putting the Pieces Together at Run-time

Problem Prematurely committing ourselves to a particular logging server conguration is inexible and inefcient Forces It is useful to build systems by scripting components Certain design decisions cant be made efciently until run-time It is a bad idea to force users to pay for components they do not use Solution Use the Component Congurator pattern to assemble the desired logging server components dynamically
Vanderbilt University 99

Advanced ACE Tutorial

Douglas C. Schmidt

The Component Congurator Pattern

APPLICATION LAYER

Concrete Component

Intent

CONFIGURATION LAYER

Component
suspend() resume() init() A fini() info()

Component Config n
1 1 1 1

Decouples the implementation of services from the time when they are congured
Forces Resolved Reduce resource utilization Support dynamic (re)conguration www.cs.wustl.edu/ schmidt/POSA/
100

Component Repository

REACTIVE LAYER

Event Handler n

Reactor

Vanderbilt University

Advanced ACE Tutorial

Do

Structure of the ACE Service Congurator Framework

ACE_Event_Handler
ACE_Service_Config

ACE_Service_Object

ACE_Service_Repository

Application Service

ACE_Service_Repository_Iterator

Framework characteristics ACE_Service_Config uses a variant of the Monostate pattern Can be accessed either via a script or programmatically

Vanderbilt University

Advanced ACE Tutorial

Do

Using the ACE Service Congurator Framework for the Logging Server
SERVICE CONFIGURATOR RUNTIME

Reactive Logger Service Object Reactor DLLS

Thread Logger Service Object Thread Pool Logger Service Object

Service Repository Service Config

svc.conf
FILE

dynamic Logger Service_Object * logger:make_Logger() "-p 2001"

The existing Logging Server service is single-threaded Other versions could be multi-threaded Note how we can script this via the svc.conf le
Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Dynamically Linking a Service

Dynamically linked factory function that allocates a new Logging_Acceptor
extern "C" ACE_Service_Object * make_Logger (void); ACE_Service_Object * make_Logger (void) { return new Logging_Acceptor; // Framework automatically // deletes memory. }

Application-specic factory function used to dynamically create a service The make_Logger() function provides a hook between an application-specic service and the application-independent ACE mechanisms ACE handles all memory allocation and deallocation

Vanderbilt University

103

Advanced ACE Tutorial

Douglas C. Schmidt

Service Conguration
The logging service is congured Generic event-loop to dynamically congure service via scripting in a svc.conf le: daemons % cat ./svc.conf
# Dynamically configure int main (int argc, char *argv[]) # the logging service { dynamic Logger // Initialize the daemon and Service_Object * // configure services logger:_make_Logger() "-p 2001" ACE_Service_Config::open (argc, # Note, .dll or .so suffix argv); # added to the logger // Run forever, performing the # automatically // configured services ACE_Reactor::instance ()-> run_reactor_event_loop (); /* NOTREACHED */ }

Vanderbilt University

104

Advanced ACE Tutorial

Douglas C. Schmidt

State Chart for the Service Congurator Framework

CONFIGURE/ Service_Config::process_directives() INITIALIZED IDLE START EVENT LOOP/ Reactor::run_event_loop() AWAITING EVENTS NETWORK EVENT/ Reactor::dispatch() PERFORM CALLBACK

SHUTDOWN/ Service_Config::close()

RECONFIGURE/ Service_Config::process_directives()

CALL HANDLER/ Event_Handler::handle_input()

Vanderbilt University 105

Advanced ACE Tutorial

Douglas C. Schmidt

Advantages of OO Logging Server

The OO architecture illustrated thus far decouples application-specic service functionality from: Time when a service is congured into a process The number of services per-process The type of IPC mechanism used The type of event demultiplexing mechanism used

We can use the techniques discussed thus far to extend applications without: Modifying, recompiling, and relinking existing code Terminating and restarting executing daemons The remainder of the Logging Server slides examine a set of techniques for decoupling functionality from concurrency mechanisms, as well
Vanderbilt University 106

Advanced ACE Tutorial

Douglas C. Schmidt

Concurrent OO Logging Server

The structure of the Logging Server can benet from concurrent execution on a multi-processor platform This section examines ACE C++ classes and patterns that extend the logging server to incorporate concurrency Note how most extensions require minimal changes to the existing OO architecture... This example also illustrates additional ACE components involving synchronization and multi-threading

Vanderbilt University

107

Advanced ACE Tutorial

Do

Concurrent OO Logging Server Architecture

SERVER LOGGING SERVER 1: SOCK Acceptor 2: accept() 4: handle_input() 5: spawn() Logging Acceptor Reactor 3: connect() CLIENT A 6: send() NETWORK

7: recv() 7: recv() 8: write() 8: write() Logging Handler Logging Handler 6: send() CLIENT B

Runs each client connection in a separate thread

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Pseudo-code for Concurrent Server

Pseudo-code for multi-threaded Logging_Handler factory Logging Server
void handler_factory (void) { initialize acceptor endpoint foreach (pending connection event) { accept connection spawn a thread to handle connection and run logging_handler() entry point } }

Pseudo-code for logging_handler() function

void logging_handler (void) { foreach (incoming logging records from client) call handle_log_record() exit thread }

Vanderbilt University

109

Advanced ACE Tutorial

Douglas C. Schmidt

Concurrency Overview
SHARED ADDRESS SPACE

A thread is a sequence of instructions executed in one or more processes One process ! stand-alone systems More than one process ! distributed systems

THREADS

PROCESS

Traditional OS processes contain a single thread of control This simplies programming since a sequence of execution steps is protected from unwanted interference by other execution sequences...
Vanderbilt University 110

Advanced ACE Tutorial

Douglas C. Schmidt

Traditional Approaches to OS Concurrency

1. Device drivers and programs with signal handlers utilize a limited form of concurrency

e.g., asynchronous I/O Note that concurrency encompasses more than multi-threading...
2. Many existing programs utilize OS processes to provide coarse-grained concurrency

e.g., Client/server database applications Standard network daemons like UNIX INETD Multiple OS processes may share memory via memory mapping or shared memory and use semaphores to coordinate execution The OS kernel scheduler dictates process behavior
Vanderbilt University 111

Advanced ACE Tutorial

Douglas C. Schmidt

Evaluating Traditional OS Process-based Concurrency

Advantages Easy to keep processes from interfering  A process combines security, protection, and robustness Disadvantages Complicated to program, e.g., Signal handling may be tricky Shared memory may be inconvenient

Inefcient
The OS kernel is involved in synchronization and process management Difcult to exert ne-grained control over scheduling and priorities
Vanderbilt University 112

Advanced ACE Tutorial

Douglas C. Schmidt

Modern OS Concurrency
Modern OS platforms typically provide a standard set of APIs that handle Process/thread creation and destruction Various types of process/thread synchronization and mutual exclusion Asynchronous facilities for interrupting long-running processes/threads to report errors and control program behavior Once the underlying concepts are mastered, its relatively easy to learn different concurrency APIs e.g., traditional UNIX process operations, Solaris threads, POSIX pthreads, WIN32 threads, Java threads, etc.

Vanderbilt University

113

Advanced ACE Tutorial

Douglas C. Schmidt

Lightweight Concurrency
Modern operating systems provide lightweight mechanisms that manage and synchronize multiple threads within a process Some systems also allow threads to synchronize across multiple processes Benets of threads 1. Relatively simple and efcient to create, control, synchronize, and collaborate Threads share many process resources by default 2. Improve performance by overlapping computation and communication Threads may also consume less resources than processes 3. Improve program structure e.g., compared with using asynchronous I/O
Vanderbilt University 114

Advanced ACE Tutorial

Do

Example: Single-threaded vs. Multi-threaded Applications

CLIENT CLIENT

SERVER

USER KERNEL

REQUEST

USER KERNEL

THREAD BLOCKED

SERVICE EXECUTES RESPONSE

SINGLETHREADED RPC
USER KERNEL

SERVER

USER KERNEL

REQUEST

SERVICE EXECUTES RESPONSE REQUEST RESPONSE

SERVER

USER KERNEL

SERVICE EXECUTES

CLIENT

MULTITHREADED RPC

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Hardware and OS Concurrency Support

USER-LEVEL

Four typical abstractions

LWP LWP LWP LWP LWP LWP LWP

1. Application threads 2. Lightweight processes 3. Kernel threads 4. Processing elements

KERNEL-LEVEL

PE

PE

PE

PE

PE

PE

PE

PE

SHARED MEMORY

PE
THREAD PROCESSING ELEMENT

LWP
LIGHTWEIGHT PROCESS

UNIX PROCESS

Vanderbilt University

116

Advanced ACE Tutorial

Douglas C. Schmidt

Application Threads
Most process resources are Each thread also contains unique equally accessible to all threads information, e.g., in a process, e.g., Identier Virtual memory Register set (e.g., PC and SP) User permissions and access Run-time stack control privileges Signal mask Open les Priority Signal handlers Thread-specic data (e.g., errno) Note, there is no MMU protection for threads in a single process
Vanderbilt University 117

Advanced ACE Tutorial

Douglas C. Schmidt

Kernel-level vs. User-level Threads

Application and system characteristics inuence the choice of user-level vs. kernel-level threading A high degree of virtual application concurrency implies user-level threads (i.e., unbound threads) e.g., desktop windowing system on a uni-processor A high degree of real application parallelism implies lightweight processes (LWPs) (i.e., bound threads) e.g., video-on-demand server or matrix multiplication on a multi-processor

Vanderbilt University

118

Advanced ACE Tutorial

Douglas C. Schmidt

Overview of OS Synchronization Mechanisms

Threads share resources in a process address space Therefore, they must use synchronization mechanisms to coordinate their access to shared data Traditional OS synchronization mechanisms are very low-level, tedious to program, error-prone, and non-portable ACE encapsulates these mechanisms with wrapper facades and higher-level patterns/components

Vanderbilt University

119

Advanced ACE Tutorial

Douglas C. Schmidt

Common OS Synchronization Mechanisms

Mutual exclusion (mutex) locks Serialize thread access to a shared resource Counting semaphores Synchronize thread execution Readers/writer (R/W) locks Serialize resources that are searched more than changed Condition variables Used to block threads until shared data changes state File locks System-wide R/W locks accessed by processes
Vanderbilt University 120

Advanced ACE Tutorial

Douglas C. Schmidt

Additional ACE Synchronization Mechanism

Events Gates and latches Barriers Allows threads to synchronize their completion Token Provides FIFO scheduling order Task Provides higher-level active object for concurrent applications Thread-specic storage Low-overhead, contention-free storage
Vanderbilt University 121

Advanced ACE Tutorial

Do

Concurrency Mechanisms in ACE

MANAGERS Thread Manager Process Manager ACTIVE OBJECTS
SYNCH

CONDITIONS Null Condition

MUTEX

Task

ADVANCED SYNCH Token Barrier

Thread
LOCK TYPE

Atomic Op

TYPE

Condition

TSS GUARDS

SYNCH WRAPPERS Mutex Null Mutex Thread Mutex Process Mutex Events RW Mutex File Lock Semaphore Thread Semaphore Process Semaphore

Guard Read Guard Write Guard

All ACE Concurrency mechanisms are ported to all OS platforms www.cs.wustl.edu/schmidt/ACE/ book1/

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Addressing Logger Server Concurrency Challenges

Problem Multi-threaded logging servers may be necessary when single-threaded reactive servers inefcient, non-scalable, or non-robust Forces Multi-threading can be very hard to program No single multi-threading model is always optimal Solution Use the Active Object pattern to allow multiple concurrent logging server operations using an OO programming style

Vanderbilt University

123

Advanced ACE Tutorial

Do

The Active Object Pattern

Proxy
Future m1() Future m2() Future m3()
VISIBLE TO CLIENTS

loop { m = act_list.dequeue() if (m.guard()) m.call() else act_list.enqueue (m); 1: enqueue(new M1) }

Scheduler
dispatch() enqueue()

3: dispatch()

enqueue() 2: enqueue(M1) dequeue() 1 n 1

Activation List

HIDDEN FROM CLIENTS

m1() m2() m3()

Servant
1

Method Request

guard() 4: m1() call()

M1 M2 M3

www.cs.wustl.edu/schmidt/POSA/ Intent Forces Resolved Allow blocking operations Permit exible concurrency strategies

Decouples method execution from method invocation to enhance concurrency and simplify synchronized access to an object that resides in its own thread of control
Vanderbilt University

Advanced ACE Tutorial

Do

ACE Support for Active Objects

t2 : Task
: Message Queue
ACTIVE

: TASK
STATE

2: enqueue (msg)

3: svc () 4: dequeue (msg) 5: do_work(msg)

1: put (msg)

6: put (msg)

t1 : Task
: Message Queue

t3 : Task
: TASK
STATE

: TASK
STATE

: Message Queue
ACTIVE

ACTIVE

The ACE Task framework can be used to implement the complete Active Object pattern or lighterweight subsets

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

The ACE Task Framework

An ACE_Task binds a separate thread of control together with an objects data and methods Multiple active objects may execute in parallel in separate lightweight or heavyweight processes ACE_Task objects communicate by passing typed messages to other ACE_Task objects Each ACE_Task maintains a queue of pending messages that it processes in priority order ACE_Task is a low-level mechanism to support active objects

Vanderbilt University

126

Advanced ACE Tutorial

Do

Structure of the ACE Task Framework

ACE_Event_Handler ACE_Service_Object

0..1

SYNCH

ACE_Thread_Manager

ACE_Task

SYNCH

ACE_Message_Block
* 1

ACE_Message_Queue

Framework characteristics 1. ACE_Tasks can register with an ACE_Reactor 2. They can be dynamically linked 3. They can queue data 4. They can run as active objects in 1 or more threads
Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

The ACE_Task Class Public Interface

template <class SYNCH_STRAT> // Synchronization aspect class ACE_Task : public ACE_Service_Object { public: // Initialization/termination hooks. virtual int open (void *args = 0) = 0; virtual int close (u_long = 0) = 0; // Transfer msg to queue for immediate processing. virtual int put (ACE_Message_Block *, ACE_Time_Value * = 0) = 0; // Run by a daemon thread for deferred processing. virtual int svc (void) = 0; // Turn task into active object. int activate (long flags, int threads = 1);

Vanderbilt University

128

Advanced ACE Tutorial

Douglas C. Schmidt

ACE_Task Class Protected Interface

Many of the following methods are used by put() and svc()
// Accessors to internal queue. ACE_Message_Queue<SYNCH_STRAT> *msg_queue (void); void msg_queue (ACE_Message_Queue<SYNCH_STRAT> *); // Accessors to thread manager. ACE_Thread_Manager *thr_mgr (void); void thr_mgr (ACE_Thread_Manager *); // Insert message into the message list. int putq (ACE_Message_Block *, ACE_Time_Value *tv = 0); // Extract the first message from the list (blocking). int getq (ACE_Message_Block *&mb, ACE_Time_Value *tv = 0); // Hook into the underlying thread library. static void *svc_run (ACE_Task<SYNCH_STRAT> *);
Vanderbilt University 129

Advanced ACE Tutorial

Douglas C. Schmidt

Design Interlude: Combining Threads & C++ Objects

Q: What is the svc_run() function and why is it a static method? A: OS thread APIs require C-style functions as entry point The ACE Task framework encapsulates the svc_run() function within the ACE_Task::activate() method:
template <class SYNCH_STRAT> int ACE_Task<SYNCH_STRAT>::activate (long flags, int n_threads) { if (thr_mgr () == NULL) thr_mgr (ACE_Thread_Manager::instance ()); thr_mgr ()->spawn_n (n_threads, &ACE_Task<SYNCH_STRAT>::svc_run, (void *) this, flags); }

4. template <SYNCH_STRATEGY> void * 1. ACE_Task::activate () ACE_Task<SYNCH_STRATEGY>::svc_run 2. ACE_Thread_Manager::spawn (ACE_Task<SYNCH_STRATEGY> *t) { (svc_run, this); // ... 3. _beginthreadex void *status = t->svc (); (0, 0, // ... svc_run, this, R UN-T I ME return status; // Thread return. 0, &thread_id); T HR EA D ST A CK }

Vanderbilt University

130

Advanced ACE Tutorial

Douglas C. Schmidt

The svc_run() Adapter Function

ACE_Task::svc_run() is static method used as the entry point to execute an instance of a service concurrently in its own thread
template <class SYNCH_STRAT> void * ACE_Task<SYNCH_STRAT>::svc_run (ACE_Task<SYNCH_STRAT> *t) { // Thread added to thr_mgr() automatically on entry. // Run service handler and record return value. void *status = (void *) t->svc (); t->close (u_long (status)); // Status becomes "return" value of thread... return status; // Thread removed from thr_mgr() automatically on return. }

Vanderbilt University

131

Advanced ACE Tutorial

Douglas C. Schmidt

Design Interlude: Motivation for the ACE_Thread_Manager

Q: How can groups of collaborating threads be managed atomically? A: Develop the ACE_Thread_Manager class that: Supports the notion of thread groups  i.e., operations on all threads in a group Implements barrier synchronization on thread exits Shields applications from incompatibilities between different OS thread libraries  e.g., detached threads and thread joins

Vanderbilt University

132

Advanced ACE Tutorial

Douglas C. Schmidt

Using ACE Task Framework for Logging Server

Process remote logging records by looping until the client terminates connection The OO implementation localizes the application-specic part of the logging service in a single point, while leveraging off int Thr_Logging_Handler::svc (void) reusable ACE components
{ while (handle_input () != -1) // Call existing function // to recv logging record // and print to stdout. continue; return 0; }

Compare with original, which borrows the Reactor thread

int Logging_Handler::handle_input (void) { handle_log_record (peer ().get_handle (), ACE_STDOUT); // ... }
133

Vanderbilt University

Advanced ACE Tutorial

Do

Class Diagram for Concurrent OO Logging Server

APPLICATIONSPECIFIC COMPONENTS
Thr_Logging_Handler SOCK_Acceptor 1

Thr Logging Acceptor

<<activates>>

n Thr Logging Handler

SOCK_Stream NULL_Synch

CONNECTIONORIENTED COMPONENTS

SVC_HANDLER PEER_ACCEPTOR

PEER_STREAM SYNCH

Acceptor

Svc Handler

PEER ACCEPTOR

PEER STREAM

ACE FRAMEWORK COMPONENTS

Connection IPC_SAP

Stream

Service Configurator
global

Concurrency

Reactor

Vanderbilt University

Advanced ACE Tutorial

Do

Thr_Logging_Acceptor and Thr_Logging_Handler Interfaces

Template classes that create, connect, and activate a new thread to handle each client
class Thr_Logging_Handler : public Logging_Handler // Inherits <handle_input> { public: // Override definition in <ACE_Svc_Handler> // class to spawn a new thread! This method // is called by the <ACE_Acceptor>. virtual int open (void *); // Process remote logging records. virtual int svc (void); }; class Thr_Logging_Acceptor : public ACE_Acceptor<Thr_Logging_Handler, ACE_SOCK_Acceptor> { // Same as <Logging_Acceptor>... };

Vanderbilt University

Advanced ACE Tutorial

Do

Thr_Logging_Handler Implementation
Override denition in the ACE_Svc_Handler class to spawn a new thread
int Thr_Logging_Handler::open (void *) { // Spawn a new thread to handle // logging records with the client. activate (THR_DETACHED); }

Process remote logging records by looping until client terminates connection

int Thr_Logging_Handler::svc (void) { while (handle_input () != -1) // Call existing function to recv // logging record and print to stdout. continue; }

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Dynamically Reconguring the Logging Server

The logging service is congured via scripting in a svc.conf le: Dynamically linked factory function that allocates a new threaded Logging_Acceptor

extern "C" % cat ./svc.conf ACE_Service_Object *make_Logger (void); # Dynamically reconfigure # the logging service ACE_Service_Object * remove Logger make_Logger (void) dynamic Logger { Service_Object * return new Thr_Logging_Acceptor; thr_logger:_make_Logger() } "-p 2002" # .dll or .so suffix added to # "thr_logger" automatically

Logging service is recongured by changing the svc.conf le and sending SIGHUP signal to server
Vanderbilt University 137

Advanced ACE Tutorial

Douglas C. Schmidt

Caveats for the Concurrent Logging Server

The concurrent Logging Server has several problems Output in the handle_log_record() function is not serialized The auto-increment of global variable request_count is also not serialized Lack of serialization leads to errors on many shared memory multi-processor platforms... Note that this problem is indicative of a large class of errors in concurrent programs... The following slides compare and contrast a series of techniques that address this problem

Vanderbilt University

138

Advanced ACE Tutorial

Douglas C. Schmidt

Explicit Synchronization Mechanisms

One approach for serialization uses OS mutual exclusion mechanisms explicitly, e.g.,
// at file scope mutex_t lock; // SunOS 5.x synchronization mechanism // ... handle_log_record (ACE_HANDLE in_h, ACE_HANDLE out_h) { // in method scope ... mutex_lock (&lock); if (ACE_OS::write (out_h, lr.buf, lr.size) == -1) return -1; mutex_unlock (&lock); // ... }

However, adding these mutex calls explicitly causes problems...

Vanderbilt University 139

Advanced ACE Tutorial

Douglas C. Schmidt

Problem: Explicit mutex_* Calls

Inelegant ! Impedance mismatch with C/C++ Obtrusive
Must nd and lock all uses of write() Can yield inheritance anomaly

Error-prone
C++ exception handling and multiple method exit points Thread mutexes wont work for separate processes Global mutexes may not be initialized correctly

Non-portable ! Hard-coded to Solaris 2.x Inefcient ! e.g., expensive for certain platforms/designs
Vanderbilt University 140

Advanced ACE Tutorial

Do

Solution: Synchronization Wrapper Facades

class ACE_Thread_Mutex { public: ACE_Thread_Mutex (void) { mutex_init (&lock_, USYNCH_THREAD, 0); } ACE_Thread_Mutex (void) { mutex_destroy (&lock_); int acquire (void) { return mutex_lock (&lock_); } int tryacquire (void) { return mutex_trylock (&lock_); } int release (void) { return mutex_unlock (&lock_); private: // SunOS 5.x serialization mechanism. mutex_t lock_; void operator= (const ACE_Thread_Mutex &); ACE_Thread_Mutex (const ACE_Thread_Mutex &); };

Note how we prevent improper copying and assignment by using C++ access control speciers

Vanderbilt University

Advanced ACE Tutorial

Do

Porting ACE_Thread_Mutex to Windows NT

class ACE_Thread_Mutex { public: ACE_Thread_Mutex (void) { lock_ = CreateMutex (0, FALSE, 0); } ACE_Thread_Mutex (void) { CloseHandle (lock_); } int acquire (void) { return WaitForSingleObject (lock_, INFINITE); } int tryacquire (void) { return WaitForSingleObject (lock_, 0); } int release (void) { return ReleaseMutex (lock_); } private: ACE_HANDLE lock_; // Windows locking mechanism. // ...

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Using C++ wrapper facades improves portability and elegance

// at file scope. ACE_Thread_Mutex lock; // Implicitly unlocked. // ... handle_log_record (ACE_HANDLE in_h, ACE_HANDLE out_h) { // in method scope ... lock.acquire (); if (ACE_OS::write (out_h, lr.buf, lr.size) == -1) return -1; lock.release (); // ...

Using the C++ Mutex Wrapper Facade

However, this doesnt really solve the tedium or error-proneness problems www.cs.wustl.edu/schmidt/PDF/ObjMan.pdf
Vanderbilt University 143

Advanced ACE Tutorial

Douglas C. Schmidt

Automated Mutex Acquisition and Release

To ensure mutexes are locked and unlocked, well dene a template class that acquires and releases a mutex automatically
template <class LOCK> class ACE_Guard { public: ACE_Guard (LOCK &m): lock_ (m) { lock_.acquire (); } ACE_Guard (void) { lock_.release (); } // ... other methods omitted ... private: LOCK &lock_; }

ACE_Guard uses the Scoped Locking idiom whereby a constructor acquires a resource and the destructor releases the resource
Vanderbilt University 144

Advanced ACE Tutorial

Douglas C. Schmidt

The ACE_GUARD Macros

ACE denes a set of macros that simplify the use of the ACE_Guard, ACE_Write_Guard, and ACE_Read_Guard classes These macros test for deadlock and detect when operations on the underlying locks fail

#define ACE_GUARD(MUTEX,OB,LOCK) \ ACE_Guard<MUTEX> OB (LOCK); if (OB.locked () == 0) return; #define ACE_GUARD_RETURN(MUTEX,OB,LOCK,RET) \ ACE_Guard<MUTEX> OB (LOCK); if (OB.locked () == 0) return RET; #define ACE_WRITE_GUARD(MUTEX,OB,LOCK) \ ACE_Write_Guard<MUTEX> OB (LOCK); if (OB.locked () == 0) return; #define ACE_WRITE_GUARD_RETURN(MUTEX,OB,LOCK,RET) \ ACE_Write_Guard<MUTEX> OB (LOCK); if (OB.locked () == 0) return RET; #define ACE_READ_GUARD(MUTEX,OB,LOCK) \ ACE_Read_Guard<MUTEX> OB (LOCK); if (OB.locked () == 0) return; #define ACE_READ_GUARD_RETURN(MUTEX,OB,LOCK,RET) \ ACE_Read_Guard<MUTEX> OB (LOCK); if (OB.locked () == 0) return RET;
Vanderbilt University 145

Advanced ACE Tutorial

Do

Thread-safe handle_log_record() Function

template <class LOCK = ACE_Thread_Mutex> ssize_t handle_log_record (ACE_HANDLE in, ACE_HANDLE out) { // beware static initialization... static LOCK lock; ACE_UINT_32 len; ACE_Log_Record lr; // The first recv reads the length (stored as a // fixed-size integer) of adjacent logging record. ssize_t n = s.recv_n ((char *) &len, sizeof len); if (n <= 0) return n; len = ntohl (len); // Convert byte-ordering // Perform sanity check! if (len > sizeof (lr)) return -1; // The second recv then reads <len> bytes to // obtain the actual record. s.recv_n ((char *) &lr, sizeof lr); // Decode and print record. decode_log_record (&lr); // Automatically acquire mutex lock. ACE_GUARD_RETURN (LOCK, guard, lock, -1); if (ACE_OS::write (out, lr.buf, lr.size) == -1) return -1; // Automatically release mutex lock. return 0; }
Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Design Interlude: Motivating the ACE_Guard Design

Q: Why is ACE_Guard parameterized by the type of LOCK? A: since many different avors of locking can benet from the Scoped Locking protocol e.g., non-recursive vs. recursive mutexes, intra-process vs. inter-process mutexes, readers/writer mutexes, POSIX and System V semaphores, le locks, and the null mutex Q: Why are templates used, as opposed to inheritance/polymorphism? A: since they are more efcient and can reside in shared memory All ACE synchronization wrapper facades use the Adapter pattern to provide identical interfaces to facilitate parameterization
Vanderbilt University 147

Advanced ACE Tutorial

Douglas C. Schmidt

The Adapter Pattern

1: request ()

Intent

client Adaptee1 Adaptee2 specific_request()

Adapter
request() 2: specific_request()

Convert the interface of a class into another interface client expects

Force resolved: Provide an interface that captures similarities between different OS mechanisms, e.g., locking or IPC
148

specific_request() Adaptee3 specific_request()

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Remaining Caveats
int Logging_Handler::handle_input (void) { ssize_t n = handle_log_record (peer ().get_handle (), ACE_STDOUT); if (n > 0) // Count # of logging records. ++request_count; // Danger, race condition!!! return n <= 0 ? -1 : 0; }

There is a race condition when incrementing the request_count variable Solving this problem using the ACE_Thread_Mutex or ACE_Guard classes is still tedious, low-level, and error-prone

A more elegant solution incorporates parameterized types, overloading, and the Strategized Locking pattern, as discussed in C++NPv1
Vanderbilt University

149

Advanced ACE Tutorial

Do

Transparently Parameterizing Synchronization Using C++

Use the Strategized Locking pattern, C++ templates, and operator overloading to dene atomic operators
template <class LOCK = ACE_Thread_Mutex, class TYPE = u_long> class ACE_Atomic_Op { public: ACE_Atomic_Op (TYPE c = 0) { count_ = c; } TYPE operator++ (void) { ACE_GUARD (LOCK, guard, lock_); return ++count_; } operator TYPE () { ACE_GUARD (LOCK, guard, lock_); return count_; } // Other arithmetic operations omitted... private: LOCK lock_; TYPE count_; };

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Final Version of Concurrent Logging Server

Using the Atomic_Op class, only one change is made
// At file scope. typedef ACE_Atomic_Op<> COUNTER; // Note default parameters... COUNTER request_count;

request_count is now serialized automatically

for (; ; ++request_count) // ACE_Atomic_Op::operator++ handle_log_record (get_handle (), ACE_STDOUT);

The original non-threaded version may be supported efciently as follows:

typedef ACE_Atomic_Op<Null_Mutex> COUNTER; //... for (; ; ++request_count) handle_log_record<Null_Mutex> (get_handle (), ACE_STDOUT);

Vanderbilt University

151

Advanced ACE Tutorial

Douglas C. Schmidt

Concurrent Web Client/Server Example

The following example illustrates a concurrent OO architecture for a high-performance Web client/server Key functional and non-functional system requirements are: Robust implementation of HTTP 1.0 protocol  i.e., resilient to incorrect or malicious Web clients/servers Extensible for use with other protocols  e.g., DICOM, HTTP 1.1, CORBA Simple Flow Protocol (SFP) Leverage multi-processor hardware and OS software  e.g., Support various concurrency patterns

Vanderbilt University

152

Advanced ACE Tutorial

Do

General Web Client/Server Interactions

WWW
CLIENT

1: GET ~schmidt HTTP/1.0 2: index.html

WWW
SERVER

HTML
PARSER

PROTOCOL HANDLERS

GUI
REQUESTER

DISPATCHER

GRAPHICS ADAPTER

COMMUNICATION PROTOCOL (E.G., HTTP)

OS KERNEL
OS I/O SUBSYSTEM NETWORK ADAPTERS

OS KERNEL
OS I/O SUBSYSTEM NETWORK ADAPTERS

NETWORK

www.cs.wustl.edu/jxh/research/

Vanderbilt University

Advanced ACE Tutorial

Do

Pseudo-code for Concurrent Web Server

Pseudo-code for master server
void master_server (void) { initialize queue and acceptor at port 80 spawn pool of worker threads foreach (pending work request from clients) { receive and queue request on queue } exit process }

Pseudo-code for thread pool workers

void worker (void) { foreach (work request on queue) dequeue and process request exit thread }

As usual, make sure to avoid the grand mistake

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Design Interlude: Motivating a Request Queue

Q: Why use a request queue to store messages, rather than directly reading from I/O handles? A: Promotes more efcient use of multiple CPUs via load balancing Enables transparent interpositioning and prioritization Makes it easier to shut down the server correctly and portably Improves robustness to denial of service attacks Moves queueing into the application process rather than OS

Drawbacks
Using a message queue may lead to greater context switching and synchronization overhead... Single point for bottlenecks
Vanderbilt University 155

Advanced ACE Tutorial

Do

Thread Entry Point

typedef ACE_Unbounded_Queue<Message> MESSAGE_QUEUE; typedef u_long COUNTER; // Track the number of requests COUNTER request_count; // At file scope. // Entry point into the Web HTTP 1.0 protocol, // which runs in each thread in the thread pool. void *worker (MESSAGE_QUEUE *msg_queue) { Message mb; // Message containing HTTP request. while (msg_queue->dequeue_head (mb)) > 0) { // Keep track of number of requests. ++request_count; // Print diagnostic cout << "got new request" << ACE_OS::thr_self () << endl; // Identify and perform Web Server // request processing here... } return 0; }

Vanderbilt University

Advanced ACE Tutorial

Do

Master Server Driver Function

// Thread function prototype. typedef void *(*THR_FUNC)(void *); int main (int argc, char *argv[]) { parse_args (argc, argv); // Queue client requests. MESSAGE_QUEUE msg_queue; // Spawn off NUM_THREADS to run in parallel. for (int i = 0; i < NUM_THREADS; i++) thr_create (0, 0, THR_FUNC (&worker), (void *) &msg_queue, THR_BOUND, 0); // Initialize network device and // recv HTTP work requests. thr_create (0, 0, THR_FUNC (&recv_requests), (void *) &msg_queue, THR_BOUND, 0); // Wait for all threads to exit (BEWARE)! while (thr_join (0, &t_id, (void **) 0) == 0) continue; // ... }

Vanderbilt University

Advanced ACE Tutorial

Do

Pseudo-code for recv_requests()

void recv_requests (MESSAGE_QUEUE *msg_queue) { initialize socket acceptor at port 80 foreach (incoming request}) { use select to wait for new connections or data if (connection) establish connections using accept() else if (data) { use sockets calls to read() HTTP requests into msg msg_queue.enqueue_tail (msg); } } }

This is the supplier thread

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Limitations with the Web Server

The algorithmic decomposition tightly couples application-specic functionality with various conguration-related characteristics, e.g., The HTTP 1.0 protocol The number of services per process The time when services are congured into a process The solution is not portable since it hard-codes SunOS 5.x threading sockets and select() There are race conditions in the code

Vanderbilt University

159

Advanced ACE Tutorial

Douglas C. Schmidt

Overcoming Limitations via OO

The algorithmic decomposition illustrated above species too many low-level details Moreover, the excessive coupling complicates reusability, extensibility, and portability... In contrast, OO focuses on decoupling application-specic behavior from reusable application-independent mechanisms The OO approach described below uses reusable framework components and commonly recurring patterns

Vanderbilt University

160

Advanced ACE Tutorial

Douglas C. Schmidt

Eliminating Race Conditions

Problem A naive implementation of MESSAGE_QUEUE will lead to race conditions  e.g., when messages in different threads are enqueued and dequeued concurrently Forces Producer/consumer concurrency is common, but requires careful attention to avoid overhead, deadlock, and proper control Solution Utilize the Monitor Object pattern and condition variables
Vanderbilt University 161

Advanced ACE Tutorial

Douglas C. Schmidt

The Monitor Object Pattern

Intent
+ synchronized_method_1() ... + synchronized_method_m() # monitor_lock_ # monotor_condition_1_ ... # monitor_condition_n_

Monitor Object

Synchronizes method execution to ensure only one method runs within an object at a time. It also allows an objects methods to cooperatively schedule their execution sequences.
Forces Resolved Synchronization corresponds to methods

schmidt/POSA/

Objects, not clients, are responsible for synchronization Cooperative method scheduling

Vanderbilt University

162

Advanced ACE Tutorial

Douglas C. Schmidt

Overview of Condition Variables

Condition variables (CVs) are used to sleep/wait until a particular condition involving shared data is signaled CVs can wait on arbitrarily complex C++ expressions Sleeping is often more efcient than busy waiting... This allows more complex scheduling decisions, compared with a mutex i.e., a mutex makes other threads wait, whereas a condition variable allows a thread to make itself wait for a particular condition involving shared data

Vanderbilt University

163

Advanced ACE Tutorial

Douglas C. Schmidt

Condition Variable Usage Patterns

// Initially unlocked. static ACE_Thread_Mutex lock; static ACE_Condition_Thread_Mutex cond (lock);

// synchronized void acquire_resources (void) { // synchronized // Automatically acquire lock. void release_resources (void) { ACE_GUARD (ACE_Thread_Mutex, g, lock); // Automatically acquire lock. ACE_GUARD (ACE_Thread_Mutex, g, lock); // Check condition in loop while (condition expression false) // Atomically modify shared // Sleep. // information... cond.wait (); cond.signal (); // Atomically modify shared // Could use cond.broadcast() here. // information. // guard automatically // Destructor releases lock. // releases lock. } }

Note how the use of the Scoped Locking idiom simplies the solution since we cant forget to release the lock!

Vanderbilt University

164

Advanced ACE Tutorial

Douglas C. Schmidt

ACE Condition Variable Interface

class ACE_Condition_Thread_Mutex public: // Initialize the CV. ACE_Condition_Thread_Mutex (const ACE_Thread_Mutex &); // Implicitly destroy the CV. ACE_Condition_Thread_Mutex (void); // Block on condition, or until // time passes. If time == 0 block. int wait (ACE_Time_Value *time = 0); // Signal one waiting thread. int signal (void); // Signal *all* waiting threads. int broadcast (void) const; private: cond_t cond_; // Solaris CV. const ACE_Thread_Mutex &mutex_; };

The ACE_Condition_ Thread_Mutex class is a wrapper for the native OS condition variable abstraction

e.g., cond_t on SunOS 5.x, pthread_cond_t for POSIX, and a custom implementation on Windows and VxWorks

Vanderbilt University

165

Advanced ACE Tutorial

Douglas C. Schmidt

Overview of ACE_Message_Queue and ACE_Message_Block

Message Queue
SYNCH STRATEGY

head_ tail_

next() prev() cont()

Message Block
Data_Block

An ACE_Message_Queue is a list of ACE_Message_Blocks Efciently handles arbitrarily-large message payloads An ACE_Message_Block is a Composite Similar to BSD mbufs or SVR4 STREAMS m_blks Design parameterizes synchronization and allocation aspects
166

next() prev() cont()

Message Block
Data_Block

next() prev() cont()

Message Block
Data_Block

next() prev() cont()

Message Block
Data_Block

Vanderbilt University

Advanced ACE Tutorial

Do

The ACE_Message_Block Class

ACE_Message_Block # # # # # # + + + + + + + + + + + + + + + + + + rd_ptr_ : size_t wr_ptr_ : size_t cont_ : ACE_Message_Block * next_ : ACE_Message_Block * prev_ : ACE_Message_Block * data_block_ : ACE_Data_Block * init (size : size_t) : int msg_type (type : ACE_Message_Type) msg_type () : ACE_Message_Type msg_priority (prio : u_long) msg_priority () : u_long clone () : ACE_Message_Block * duplicate () : ACE_Message_Block * release () : ACE_Message_Block * set_flags (flags : u_long) : u_long clr_flags (flags : u_long) : u_long copy (buf : const char *,n : size_t) : int rd_ptr (n : size_t) rd_ptr () : char * wr_ptr (n : size_t) wr_ptr () : char * length () : size_t total_length () : size_t size () : size_t

ACE_Data_Block # base_ : char * # refcnt_ : int

Class characteristics Hide messaging implementations from clients

ACE_Message _Block
cont() data_block() wr_ptr() rd_ptr()

ACE_Message _Block
cont() data_block() wr_ptr() rd_ptr()

ACE_Message _Block
cont() data_block() rd_ptr() wr_ptr()

ACE_Data _Block
PAYLOAD (1) SIMPLE MESSAGE STRUCTURE

reference_count() = 2

ACE_Data_Block

(2) COMPOSITE MESSAGE STRUCTURE

Vanderbilt University

Advanced ACE Tutorial

Do

The ACE_Message_Queue Class

SYNCH_STRATEGY

ACE_Message_Queue
# # # # head_ : ACE_Message_Block * tail_ : ACE_Message_Block * high_water_mark_ : size_t low_water_mark_ : size_t

+ ACE_Message_Queue (high_water_mark : size_t = DEFAULT_HWM, low_water_mark : size_t = DEFAULT_LWM, notify : ACE_Notification_Strategy * = 0) + open (high_water_mark : size_t = DEFAULT_HWM, low_water_mark : size_t = DEFAULT_LWM, notify : ACE_Notification_Strategy * = 0) : int + flush () : int + notification_strategy (s : ACE_Notification_Strategy *) : void + is_empty () : int + is_full () : int + enqueue_tail (item : ACE_Message_Block *, timeout : ACE_Time_Value * = 0) : int + enqueue_head (item : ACE_Message_Block *, timeout : ACE_Time_Value * = 0) : int + enqueue_prio (item : ACE_Message_Block *, timeout : ACE_Time_Value * = 0) : int + dequeue_head (item : ACE_Message_Block *&, timeout : ACE_Time_Value * = 0) : int + dequeue_tail (item : ACE_Message_Block *&, timeout : ACE_Time_Value * = 0) : int + high_water_mark (new_hwm : size_t) : void + high_water_mark (void) : size_t + low_water_mark (new_lwm : size_t) : void + low_water_mark (void) : size_t + close () : int + deactivate () : int + activate () : int + pulse () : int + state () : int

Class characteristics Note how the synchronization aspect can be strategized!

Vanderbilt University

Advanced ACE Tutorial

Do

The ACE_Message_Queue Public Interface

template <class SYNCH_STRAT = ACE_MT_SYNCH> // Synchronization aspect class ACE_Message_Queue { public: // Default high and low water marks. enum { DEFAULT_LWM = 0, DEFAULT_HWM = 4096 }; // Initialize a Message_Queue. Message_Queue (size_t hwm = DEFAULT_HWM, size_t lwm = DEFAULT_LWM); // Check if full or empty (hold locks) int is_empty (void) const; int is_full (void) const; // Enqueue and int enqueue_prio int enqueue_tail int dequeue_head int dequeue_tail dequeue Message_Block *s. (ACE_Message_Block *, ACE_Time_Value *); (ACE_Message_Block *, ACE_Time_Value *); (ACE_Message_Block *&, ACE_Time_Value *); (ACE_Message_Block *&, ACE_Time_Value *);

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Design Interlude: Parameterizing Synchronization Strategies

Q: What is ACE_MT_SYNCH and how does it work? A: ACE_MT_SYNCH provides a thread-safe synchronization strategy for a ACE_Svc_Handler e.g., it ensures that an ACE_Svc_Handlers ACE_Message_Queue is thread-safe Any ACE_Task that accesses shared state can use the ACE_MT_SYNCH traits
Vanderbilt University

Note the use of traits:

struct ACE_MT_SYNCH { typedef ACE_Thread_Mutex MUTEX; typedef ACE_Condition_Thread_Mutex COND; }; struct ACE_NULL_SYNCH { typedef ACE_Null_Mutex MUTEX; typedef ACE_Null_Condition COND; };

170

Advanced ACE Tutorial

Do

ACE_Message_Queue Class Private Interface

private: // Check boundary conditions & dont hold locks. int is_empty_i (void) const; int is_full_i (void) const; // Routines that actually do the enqueueing // and dequeueing and dont hold locks. int enqueue_prio_i (ACE_Message_Block *); int enqueue_tail_i (ACE_Message_Block *); int dequeue_head_i (ACE_Message_Block *&); int dequeue_tail_i (ACE_Message_Block *&); // ... // Parameterized types for synchronization // primitives that control concurrent access. // Note use of C++ traits typename SYNCH_STRAT::MUTEX lock_; typename SYNCH_STRAT::COND not_empty_cond_; typename SYNCH_STRAT::COND not_full_cond_; size_t size_t size_t size_t }; high_water_mark_; low_water_mark_; cur_bytes_; cur_count_;

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Design Interlude: Tips for Intra-class Locking

Q: How should locking be performed in an OO class? A: Apply the Thread-Safe Interface pattern: Interface functions should lock and do no work implementation functions should do the work and not lock  This pattern helps to avoid intra-class method deadlock This is actually a variant on a common OO pattern that public functions should check, private functions should trust  Naturally, there are exceptions to this rule... This pattern avoids the following surprises  Unnecessary overhead from recursive mutexes  Deadlock if recursive mutexes arent used www.cs.wustl.edu/schmidt/POSA/
Vanderbilt University 172

Advanced ACE Tutorial

Do

ACE_Message_Queue Class Implementation

template <class SYNCH_STRAT> ACE_Message_Queue<SYNCH_STRAT>::ACE_Message_Queue (size_t hwm, size_t lwm) : not_empty_cond_ (lock_), not_full_cond_ (lock_), ... {} template <class SYNCH_STRAT> int ACE_Message_Queue<SYNCH_STRAT>::is_empty_i (void) const { return cur_bytes_ == 0 && cur_count_ == 0; } template <class SYNCH_STRAT> int ACE_Message_Queue<SYNCH_STRAT>::is_full_i (void) const { return cur_bytes_ > high_water_mark_; } template <class SYNCH_STRAT> int ACE_Message_Queue<SYNCH_STRAT>::is_empty (void) const { ACE_GUARD_RETURN (SYNCH_STRAT::MUTEX, g, lock_, -1); return is_empty_i (); } template <class SYNCH_STRAT> int ACE_Message_Queue<SYNCH_STRAT>::is_full (void) const { ACE_GUARD_RETURN (SYNCH_STRAT::MUTEX, g, lock_, -1); return is_full_i (); }

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

ACE_Message_Queue Operations
template <class SYNCH_STRAT> int template <class SYNCH_STRAT> int ACE_Message_Queue<SYNCH_STRAT>:: ACE_Message_Queue<SYNCH_STRAT>:: enqueue_tail (ACE_Message_Block *item, dequeue_head (ACE_Message_Block *&item, ACE_Time_Value *tv) { ACE_Time_Value *tv) { ACE_GUARD_RETURN (SYNCH_STRAT::MUTEX, ACE_GUARD_RETURN (SYNCH_STRAT::MUTEX, guard, lock_, -1); guard, lock_, -1); // Wait while the queue is empty. // Wait while the queue is full. while (is_empty_i ()) { while (is_full_i ()) { // Release lock_ and wait for timeout, // Release the <lock_> and wait // signal, or a new message being // for timeout, signal, or space // placed in the list. // to become available in the list. if (not_empty_cond_.wait (tv) == -1) if (not_full_cond_.wait (tv) == -1) return -1; return -1; } } // Actually dequeue the first message. // Actually enqueue the message at dequeue_head_i (item); // the end of the list. enqueue_tail_i (item); // Tell blocked threads that list // is no longer full. // Tell blocked threads that if (cur_bytes_ <= low_water_mark_) // list has a new item! not_full_cond_.signal (); not_empty_cond_.signal (); } }

Vanderbilt University

174

Advanced ACE Tutorial

Douglas C. Schmidt

Overcoming Algorithmic Decomposition Limitations

Previous slides illustrate tactical techniques and patterns that: Reduce accidental complexity e.g.,  Automate synchronization acquisition and release (Scoped Locking idiom)  Improve synchronization mechanisms (Adapter, Wrapper Facade, Monitor Object, Thread-Safe Interface, Strategized Locking patterns) Eliminate race conditions Next, we describe strategic patterns, frameworks, and components to: Increase reuse and extensibility e.g.,  Decoupling service, IPC, and demultiplexing Improve the exibility of concurrency control
Vanderbilt University 175

Advanced ACE Tutorial

Douglas C. Schmidt

Selecting the Servers Concurrency Architecture

Problem A very strategic design decision for high-performance Web servers is selecting an efcient concurrency architecture Forces No single concurrency architecture is optimal Key factors include OS/hardware platform and workload Solution Understand key alternative concurrency patterns

Vanderbilt University

176

Advanced ACE Tutorial

Douglas C. Schmidt

Concurrency Patterns in the Web Server

The following example illustrates the patterns and framework components in an OO implementation of a concurrent Web Server There are various architectural patterns for structuring concurrency in a Web Server

Reactive Thread-per-request Thread-per-connection Synchronous Thread Pool  Leader/Followers Thread Pool  Half-Sync/Half-Async Thread Pool Asynchronous Thread Pool

Vanderbilt University

177

Advanced ACE Tutorial

Do

Reactive Web Server

2: HANDLE INPUT 3: CREATE HANDLER 4: ACCEPT CONNECTION 5: ACTIVATE HANDLER HTTP Acceptor

HTTP Handler

HTTP Handler

Reactor

6: PROCESS

HTTP REQUEST

SERVER

1: CONNECT
CLIENT CLIENT CLIENT

Vanderbilt University

Advanced ACE Tutorial

Do

Thread-per-Request Web Server

HTTP Handler HTTP Handler 2: HANDLE INPUT 3: CREATE HANDLER 4: ACCEPT CONNECTION 5: SPAWN THREAD HTTP Acceptor

HTTP Handler

Reactor

6: PROCESS

HTTP REQUEST

SERVER

1: CONNECT
CLIENT CLIENT CLIENT

Vanderbilt University

Advanced ACE Tutorial

Do

Thread-per-Connection Web Server

3: SPAWN THREAD
PER CONNECTION

2: CREATE, ACCEPT,
AND ACTIVATE HTTP_HANDLER

HTTP HTTP Handler Handler HTTP Handler

HTTP Acceptor Reactor

4: PROCESS
CLIENT

HTTP REQUEST

1: HTTP

REQUEST

SERVER

CLIENT

CLIENT

Vanderbilt University

Advanced ACE Tutorial

Do

Leader/Followers Synchronous Thread Pool Web Server

HTTP Handler HTTP Handler

Event Dispatcher 2: ACCEPT CONNECTION 3: MORPH INTO HANDLER HTTP Handler

HTTP Acceptor

HTTP Acceptor

4: PROCESS

HTTP REQUEST

1: HTTP
CLIENT CLIENT CLIENT

REQUEST

SERVER

Vanderbilt University

Advanced ACE Tutorial

Do

Half-Sync/Half-Async Synchronous Thread Pool Web Server

Message Queue 4: DEQUEUE & Active Object
PROCESS REQUEST

2: HANDLE INPUT 3: ENQUEUE REQUEST HTTP Handler HTTP Handler

Active Object

HTTP Handler HTTP Acceptor

Active Object

Active Object

Reactor

5: PROCESS

HTTP REQUEST

1: HTTP

REQUEST

SERVER CLIENT CLIENT CLIENT

Vanderbilt University

Advanced ACE Tutorial

Do

Asynchronous Thread Pool Web Server

I/O Completion Port 1: INITIATE ASYNC ACCEPT 2: RUN EVENT LOOP 4: ACCEPT COMPLETES 5: QUEUE COMPLETION Async Async Async ReadAccept Read Async

HTTP Handler

6: DEQUEUE COMPLETION & PROCESS HTTP REQUEST Handler HTTP Handler HTTP Handler

Accept

Proactor

7: PROCESS

HTTP REQUEST

3: HTTP

REQUEST

SERVER CLIENT CLIENT CLIENT

Vanderbilt University

Advanced ACE Tutorial

Do

Web Server Software Architecture

HTTP Handler HTTP Handler
Sock Stream Sock Stream

HTTP Handler
Sock Stream

HTTP Acceptor
Sock Acceptor

Event Dispatcher

Event Dispatcher Encapsulates Web server concurrency and dispatching strategies HTTP Handlers Parses HTTP headers and processes requests HTTP Acceptor Accepts connections and creates HTTP Handlers
Vanderbilt University

Advanced ACE Tutorial

Do

Patterns in the Web Server Implementation

Thread-per Request Thread Pool Active Object Half-Sync/ Half-Async Reactor/ Proactor Acceptor Connector Component Configurator Double Checked Locking

Thread-per Connection

Asynchronous Completion Token

Wrapper Facade

STRATEGIC PATTERNS TACTICAL PATTERNS

Strategy Abstract Factory
Adapter

Singleton

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Patterns in the Web Client/Server (contd)

The Web Client/Server uses same patterns as distributed logger i.e., Reactor, Component Congurator, Active Object, and Acceptor It also contains patterns with the following intents: Connector ! Decouple the active connection and initialization of a peer service in a distributed system from the processing performed once the peer service is connected and initialized Double-Checked Locking Optimization ! Allows atomic initialization, regardless of initialization order, and eliminates subsequent locking overhead Half-Sync/Half-Async ! Decouples synchronous I/O from asynchronous I/O in a system to simplify concurrent programming effort without degrading execution efciency
Vanderbilt University 186

Advanced ACE Tutorial

Do

Architecture of Our Web Server

REQUEST PROCESSING LAYER

svc_run

svc_run

svc_run

svc_run

QUEUEING LAYER

Msg HTTP Queue Processor

Options

HTTP Handler

HTTP Handler Reactor

HTTP Handler HTTP Acceptor

I/O DEMUXING
LAYER

www.cs.wustl.edu/schmidt/PDF/HPL.pdf

Vanderbilt University

Advanced ACE Tutorial

Do

An Integrated Reactive/Active Web Server

REGISTERED OBJECTS svc_run 4: getq(msg) 5:svc(msg) svc_run svc_run

HTTP Handler HTTP Handler HTTP Event Handler HTTP Handler Event Processor Handler Event Handler 2: recv_request(msg) 3: putq(msg)
1: handle_input()

APPLICATION

LEVEL

HTTP Acceptor Event Handler

FRAMEWORK

LEVEL

: Handle Table

: Reactor
OS EVENT DEMULTIPLEXING INTERFACE

KERNEL

Were focusing on the Reactive layer here

Vanderbilt University

LEVEL

Advanced ACE Tutorial

Douglas C. Schmidt

HTTP_Handler Public Interface

template <class ACCEPTOR> The HTTP_Handler is class HTTP_Handler : public the Proxy for ACE_Svc_Handler<ACCEPTOR::PEER_STREAM, communicating with ACE_NULL_SYNCH> { clients (e.g., Web public: browsers like Netscape // Entry point into <HTTP_Handler>, // called by <HTTP_Acceptor>. or i.e.,) virtual int open (void *) { It implements the // Register with <ACE_Reactor> asynchronous // to handle input. portion of Halfreactor ()->register_handler (this, ACE_Event_Handler::READ_MASK); Sync/Half-Async // Register timeout in case client pattern // doesnt send any HTTP requests. reactor ()->schedule_timer (this, 0, ACE_Time_Value (CLIENT_TIMEOUT)); }
Vanderbilt University 189

Advanced ACE Tutorial

Douglas C. Schmidt

HTTP_Handler Protected Interface

protected: These methods are // Reactor dispatches this invoked by callbacks // method when clients timeout. from ACE_Reactor virtual int handle_timeout (const ACE_Time_Value &, const void *) REGISTERED HTTP { Handler OBJECTS // Remove from the Reactor. Event 2: remove_handler(this) reactor ()->remove_handler Handler (this, 1: handle_timeout() ACE_Event_Handler::READ_MASK); } : Timer : Reactor Queue // Reactor dispatches this method Reactor // when HTTP requests arrive. virtual int handle_input (ACE_HANDLE); // Receive/frame client HTTP // requests (e.g., GET). int recv_request (ACE_Message_Block *&); };
Vanderbilt University 190

Advanced ACE Tutorial

Douglas C. Schmidt

Integrating Multi-threading
Problem Multi-threaded Web servers are needed since Reactive Web servers are often inefcient and non-robust Forces Multi-threading can be very hard to program No single multi-threading model is always optimal Solution Use the Active Object pattern to allow multiple concurrent server operations in an OO-manner

Vanderbilt University

191

Advanced ACE Tutorial

Do

Using the Active Object Pattern and ACE Task Framework in the Web Server
REGISTERED OBJECTS 4: getq(msg) 5:svc(msg) svc_run svc_run svc_run

HTTP Handler HTTP Handler HTTP Event Handler HTTP Handler Event Processor Handler Event Handler 2: recv_request(msg) 3: putq(msg)
1: handle_input()

APPLICATION

LEVEL

HTTP Acceptor Event Handler

FRAMEWORK

LEVEL

: Handle Table

: Reactor
OS EVENT DEMULTIPLEXING INTERFACE

KERNEL

Were focusing on the Active Object layer here

Vanderbilt University

LEVEL

Advanced ACE Tutorial

Douglas C. Schmidt

The HTTP_Processor Class

class HTTP_Processor : public ACE_Task<ACE_MT_SYNCH> { private: HTTP_Processor (void); public: // Singleton access point. static HTTP_Processor *instance (void); // Pass a request to the thread pool. virtual int put (ACE_Message_Block *, ACE_Time_Value *); // Entry point into a pool thread. virtual int svc (void) { ACE_Message_Block *mb = 0;

Processes HTTP requests using the Thread-Pool concurrency model This method implements the synchronous task portion of the HalfSync/Half-Async pattern

// Wait for messages to arrive. for (;;) { getq (mb); // Inherited from <ACE_Task> // Identify and perform HTTP // Server request processing...

Vanderbilt University

193

Advanced ACE Tutorial

Do

Using the Singleton Pattern

// Singleton access point. HTTP_Processor * HTTP_Processor::instance (void) { // Beware of race conditions! if (instance_ == 0) // Create the Singleton "on-demand." instance_ = new HTTP_Processor; return instance_; } // Constructor creates the thread pool. HTTP_Processor::HTTP_Processor (void) { // Inherited from class Task. activate (THR_BOUND, Options::instance ()->threads ()); }

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Subtle Concurrency Woes with the Singleton Pattern

Problem The canonical Singleton implementation has subtle bugs in multi-threaded applications Forces Too much locking makes Singleton too slow... Too little locking makes Singleton unsafe... Solution Use the Double-Checked Locking optimization pattern to minimize locking and ensure atomic initialization

Vanderbilt University

195

Advanced ACE Tutorial

Douglas C. Schmidt

The Double-Checked Locking Optimization Pattern

if (instance_ == NULL) { mutex_.acquire (); if (instance_ == NULL) instance_ = new HTTP_Processor; mutex_.release (); } return instance_;

Intent

Allows atomic initialization, regardless of initialization order, and eliminates subsequent locking overhead
Forces Resolved: Ensures atomic object initialization

HTTP Processor
static instance() static instance_

Mutex

Minimizes locking overhead Caveat! This pattern assumes atomic memory access
196

www.cs.wustl.edu/ schmidt/POSA/

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

The ACE Singleton Template

template <class TYPE, class LOCK> Features class ACE_Singleton : public ACE_Cleanup { public: Turns any class into static TYPE *instance (void) { a singleton // Memory barrier could go here... if (s_ == 0) { ACE_GUARD_RETURN (LOCK, g, Automates ACE_Object_Manager Double-Checked ::get_singleton_lock (), -1); if (s_ == 0) Locking Optimization s_ = new ACE_Singleton<TYPE>; // Memory barrier could go here. Ensures automatic ACE_Object_Manager::at_exit (s_); } cleanup when return s_->instance_; process exits } virtual void cleanup (void *param = 0); www.cs.wustl.edu/ protected: ACE_Singleton (void); schmidt/PDF/ TYPE instance_; ObjMan.pdf static ACE_Singleton<TYPE, LOCK> *s_; };
Vanderbilt University 197

Advanced ACE Tutorial

Douglas C. Schmidt

Integrating Reactive and Multi-threaded Layers

Problem Justifying the hybrid design of our Web server can be tricky Forces Engineers are never satised with the status quo ;-) Substantial amount of time is spent re-discovering the intent of complex concurrent software design Solution Use the Half-Sync/Half-Async pattern to explain and justify our Web server concurrency architecture

Vanderbilt University

198

Advanced ACE Tutorial

Douglas C. Schmidt

The Half-Sync/Half-Async Pattern

SYNCHRONOUS TASK LAYER
SYNC TASK 1 SYNC TASK 2 SYNC TASK 3

Intent

QUEUEING

LAYER

1, 4: read(data)
MESSAGE QUEUES

Decouples synchronous I/O from asynchronous I/O in a system to simplify concurrent programming effort without degrading execution efciency
Forces Resolved: Simplify programming Ensure efcient I/O www.cs.wustl.edu/ schmidt/POSA/
199

ASYNCHRONOUS

TASK LAYER

3: enqueue(data)
ASYNC TASK

2: interrupt
EXTERNAL EVENT SOURCES

Vanderbilt University

Advanced ACE Tutorial

Do

Using the Half-Sync/Half-Async Pattern in the Web Server

SYNC TASK LEVEL
svc_run 4: getq(msg) 5:svc(msg) svc_run svc_run

QUEUEING LEVEL

HTTP Processor

HTTP Handler HTTP Handler HTTP Event Handler Handler Event Handler Event Handler
ASYNC TASK LEVEL
1: handle_input() 2: recv_request(msg) 3: putq(msg) : Handle Table

HTTP Acceptor Event Handler

: Reactor

Vanderbilt University

Advanced ACE Tutorial

Do

Joining Async and Sync Tasks in the Web Server

// The following methods form the boundary // between the Async and Sync layers. template <class PA> int HTTP_Handler<PA>::handle_input (ACE_HANDLE h) { ACE_Message_Block *mb = 0; // Try to receive and frame message. if (recv_request (mb) == HTTP_REQUEST_COMPLETE) { reactor ()->remove_handler (this, ACE_Event_Handler::READ_MASK); reactor ()->cancel_timer (this); // Insert message into the Queue. HTTP_Processor<PA>::instance ()->put (mb); } } int HTTP_Processor::put (ACE_Message_Block *msg, ACE_Time_Value *timeout) { // Insert the message on the Message_Queue // (inherited from class Task). putq (msg, timeout); }
Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Optimizing Our Web Server for Asynchronous Operating Systems

Problem Synchronous multi-threaded solutions are not always the most efcient Forces Purely asynchronous I/O is quite powerful on some OS platforms  e.g., Windows NT 4.x or UNIX with aio_()* calls Good designs should be adaptable to new contexts Solution Use the Proactor pattern to maximize performance on Asynchronous OS platforms
Vanderbilt University 202

Advanced ACE Tutorial

Douglas C. Schmidt

The Proactor Pattern

Intent
1
APPLICATIONINDEPENDENT APPLICATIONDEPENDENT

Demultiplexes and dispatches service requests that are triggered by the completion of asynchronous operations
Resolves same forces as Reactor

overlapped_result = GetQueuedCompleteStatus(); overlapped_result->complete()

HTTP Handler

Async Write HTTP Acceptor

Completion Handler
handle_accept() handle_read_file() handle_write_file() handle_timeout() get_handle() A

Async Op
open() cancel() 1

Async Accept

n Proactor
handle_events()

Handles

Timer_Queue
schedule_timer(h) cancel_timer(h)

www.cs.wustl.edu/schmidt/POSA/

Vanderbilt University

203

Advanced ACE Tutorial

Do

Structure of the ACE Proactor Framework

ACE_Timer_Queue ACE_Service_Handler

ACE_Proactor

ACE_Asynch_Acceptor

ACE_Asynch_Connector

ACE_Handler

ACE_Asynch_Read_Stream

ACE_Asynch_Write_Stream

ACE_Asynch_Result

Framework characteristics Similar to the ACE Reactor framework, except behavior is inverse Portable to Windows and various UNIX platforms that support aio_*() family of methods
Vanderbilt University

Advanced ACE Tutorial

Do

Using the ACE Proactor Framework for the Web Server

Connection Setup Phase
1: accept connections Acceptor 4: connect Web Browser 6: accept complete 2: accept (Acceptor, Dispatcher)

Web Server
7: create HTTP Handler 8: read (Handler, Dispatcher)

Completion Dispatcher 3: handle events 5: accept complete

Operating System

1: GET /etc/passwd Web Browser

Data Transfer Phase

Web Server
HTTP Handler 4: parse request

6: write (File, Conn., 8: write Handler, Dispatcher) 5: read (File) 3: read complete complete 7: write complete Completion Operating File Dispatcher System System 2: read complete

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Structuring Service Initialization

Problem The communication protocol used between clients and the Web server is often orthogonal to the initialization protocol Forces Low-level connection establishment APIs are tedious, error-prone, and non-portable Separating initialization from use can increase software reuse substantially Solution Use the Acceptor and Connector patterns to decouple passive service initialization from run-time protocol
Vanderbilt University 206

Advanced ACE Tutorial

Do

Using the ACE_Acceptor in the Web Server

: HTTP Handler : Svc Handler : HTTP Handler : Svc Handler : HTTP Handler : Svc Handler : HTTP Handler : Svc Handler

: HTTP Acceptor : Acceptor

PASSIVE LISTENER

ACTIVE CONNECTIONS

1: handle_input() 2: sh = make_svc_handler() 3: accept_svc_handler(sh) 4: activate_svc_handler(sh)

: Reactor

The HTTP_Acceptor is a factory that creates, connects, and activates an HTTP_Handler

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

HTTP_Acceptor Class Interface

template <class ACCEPTOR> The HTTP_Acceptor class HTTP_Acceptor : class implements the public ACE_Acceptor<HTTP_Handler< Acceptor role ACCEPTOR::PEER_STREAM>, // Note use of a "trait". i.e., it accepts ACCEPTOR> connections/initializes { HTTP_Handlers public: // Called when <HTTP_Acceptor> is // dynamically linked. virtual int init (int argc, char *argv[]); // Called when <HTTP_Acceptor> is // dynamically unlinked. virtual int fini (void); // ... };

Vanderbilt University

208

Advanced ACE Tutorial

Do

HTTP_Acceptor Class Implementation

// Initialize service when dynamically linked. template <class PA> int HTTP_Acceptor<PA>::init (int argc, char *argv[]) { Options::instance ()->parse_args (argc, argv); // Initialize the communication endpoint and // register to accept connections. peer_acceptor ().open (typename PA::PEER_ADDR (Options::instance ()->port ()), Reactor::instance ()); } // Terminate service when dynamically unlinked. template <class PA> int HTTP_Acceptor<PA>::fini (void) { // Shutdown threads in the pool. HTTP_Processor<PA>::instance ()-> msg_queue ()->deactivate (); // Wait for all threads to exit. HTTP_Processor<PA>::instance ()-> thr_mgr ()->wait (); }

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Using the ACE Service Congurator Framework in the Web Server

SERVICE CONFIGURATOR RUNTIME

TP Web Server Service Object Reactor DLLS

Service Repository Service Config

Reactive Web Server Service Object TPR Web Server Service Object

svc.conf
FILE

dynamic Web_Server Service_Object * web_server:make_Web_Server() "-port 80"

Vanderbilt University

210

Advanced ACE Tutorial

Douglas C. Schmidt

Component Congurator Implementation in C++

The concurrent Web Server is congured and initialized via a conguration script
% cat ./svc.conf dynamic Web_Server Service_Object * web_server:_make_Web_Server() "-p 80 -t $THREADS" # .dll or .so suffix added to # "web_server" automatically

Factory function that dynamically allocates a Half-Sync/Half-Async Web Server object

extern "C" ACE_Service_Object * make_Web_Server (void); ACE_Service_Object * make_Web_Server (void) { return new HTTP_Acceptor<ACE_SOCK_Acceptor>; // ACE dynamically unlinks and // deallocates this object. }

Vanderbilt University

211

Advanced ACE Tutorial

Douglas C. Schmidt

Main Program for the Web Server

int main (int argc, char *argv[]) { // Initialize the daemon and // dynamically configure services. ACE_Service_Config::open (argc, argv); // Loop forever, running services // and handling reconfigurations. ACE_Reactor::instance ()-> run_reactor_event_loop (); /* NOTREACHED */ }

The main() function is totally generic! Dynamically congure & execute Web Server Make any application Web-enabled

Vanderbilt University

212

Advanced ACE Tutorial

Douglas C. Schmidt

Optimizing the JAWS Framework

Reactor/Proactor I/O Strategy Framework State Singleton Cached Virtual Filesystem

Asynchronous Completion Token Protocol Handler Protocol Filter Adapter

Tilde Expander

~
/home/...
Strategy

Event Dispatcher

Protocol Pipeline Framework Streams

Concurrency Strategy Framework Service Configurator

www.cs.wustl.edu/jxh/research/
Vanderbilt University

Use lightweight concurrency Minimize locking Apply le caching and memory mapping Use gather-write mechanisms Minimize logging Pre-compute HTTP responses Avoid excessive time() calls Optimize the transport interface

Active Object

Strategy

Acceptor

213

Advanced ACE Tutorial

Douglas C. Schmidt

Application-level Telecom Gateway Example

SATELLITES TRACKING STATION PEERS

STATUS INFO

This example explores the patterns and reusable framework components for an application-level Gateway The Gateway routes messages between Peers Gateway and Peers are connected via TCP/IP

WIDE AREA NETWORK

COMMANDS BULK DATA TRANSFER

GATEWAY

LOCAL AREA NETWORK

GROUND STATION PEERS

Vanderbilt University

214

Advanced ACE Tutorial

Do

OO Software Architecture of the Gateway

Consumer Handler Routing Table Supplier Handler

Reactor Consumer Handler Connector Acceptor

Supplier Handler

INCOMING MESSAGES CONNECTION REQUEST

GATEWAY
CONNECTION REQUEST

OUTGOING MESSAGES

www.cs.wustl.edu/schmidt/PDF/ TAPOS-00.pdf All components in this architecture are based on patterns from ACE
Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Gateway Behavior
Components in the Gateway behave as follows: 1. Gateway parses conguration les that specify which Peers to connect with and which routes to use 2. Proxy_Handler_Connector connects to Peers, then creates and activates Proxy_Handler subclasses (Supplier_Handler or Consumer_Handler) 3. Once connected, Peers send messages to the Gateway Messages are handled by an Supplier_Handler Supplier_Handlers work as follows:  Receive and validate messages  Consult a Routing_Table  Forward messages to the appropriate Peer(s) via Consumer_Handlers
Vanderbilt University 216

Advanced ACE Tutorial

Do

Patterns in the Gateway

Active Object AcceptorConnector Half-Sync/ Half-Async
STRATEGIC PATTERNS

Non-blocking Buffered I/O Component Configurator

Reactor

TACTICAL Iterator Template Factory Proxy Wrapper PATTERNS Method Method Facade

The Gateway components are based upon a common pattern language

Vanderbilt University

Advanced ACE Tutorial

Do

Class Diagram for Single-Threaded Gateway

APPLICATIONSPECIFIC COMPONENTS
Proxy_Handler SOCK_Connector

1 Proxy Handler Connector

SOCK_Stream Null_Synch

<<activates>>

Supplier/Consumer n Handler

CONNECTIONORIENTED COMPONENTS

SVC_HANDLER PEER_CONNECTOR

PEER_STREAM SYNCH

Connector

Svc Handler

PEER CONNECTOR

PEER STREAM

ACE FRAMEWORK COMPONENTS

Connection IPC_SAP

Stream

Service Configurator
global

Concurrency

Reactor

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

OO Gateway Architecture
Application-specic components
Proxy_Handlers route messages among Peers

Connection-oriented application components

ACE_Svc_Handler  Performs I/O-related tasks with connected clients ACE_Connector factory  Establishes new connections with clients  Dynamically creates an ACE_Svc_Handler object for each client and activates it

Application-independent ACE framework components

Perform IPC, explicit dynamic linking, event demultiplexing, event handler dispatching, multi-threading, etc.
Vanderbilt University 219

Advanced ACE Tutorial

Do

Using the ACE Reactor Framework for the Gateway

APPLICATION LEVEL
CONCRETE EVENT HANDLERS Consumer Consumer Handler 4: send(msg) Handler 2: recv(msg) Event Event 3: route(msg) Handler Handler Supplier Handler Event Handler

FRAMEWORK LEVEL

1: handle_input() Timer Queue Handle Table

: Reactor

Benets Straightforward to program Concurrency control is trivial

Vanderbilt University

KERNEL LEVEL

OS EVENT DEMULTIPLEXING INTERFACE

Liabilities Design is brittle Cant leverage multi-processors

Advanced ACE Tutorial

Douglas C. Schmidt

Addressing Active Endpoint Connection and Initialization Challenges

Problem Application communication protocols are often orthogonal to their connection establishment and service initialization protocols Forces Low-level connection APIs are error-prone and non-portable Separating initialization from processing increases software reuse Asynchronous connections are important over long-delay paths Solution Use the Acceptor-Connector pattern to decouple connection and initialization protocols from the Gateway routing protocol
Vanderbilt University 221

Advanced ACE Tutorial

Do

The Acceptor-Connector Pattern (Connector Role)

Svc Handler
Svc Handler

peer_stream_ open()
APPLICATIONDEFINED

Connector
ACTIVATES

connect(sh, addr) complete()

HANDLE ASYNC CONNECTION COMPLETION APPLICATIONINDEPENDENT

Reactor

www.cs.wustl.edu/schmidt/POSA/ Intent of Connector Role Forces Resolved:

Decouple the active connection and initialization of a peer service in a distributed system from the processing performed once the peer service is connected and initialized
Vanderbilt University

Reuse connection code Efciently setup connections with many peers or over long delay paths

Advanced ACE Tutorial

Douglas C. Schmidt

Structure of the Acceptor-Connector Pattern in ACE

ACE_Event_Handler
SVC_HANDLER, PEER_CONNECTOR SYNCH_STRATEGY

ACE_Task

Additional features of the ACE_Connector Uses C++ parameterized types to strategize IPC and service aspects Uses Template Method pattern to strategize creation, connection establishment, and concurrency policies

ACE_Connector
PEER_STREAM, SYNCH_STRATEGY SVC_HANDLER, PEER_ACCEPTOR

ACE_Svc_Handler
bind

ACE_Acceptor

Application Service

Vanderbilt University

223

Advanced ACE Tutorial

Do

Using the ACE_Connector in the Gateway

Svc Svc Handler Handler Svc Handler Svc Handler

Svc Handler

Hash_Map

ACTIVE CONNECTIONS

Connector
PENDING CONNECTIONS

Svc Handler

Reactor

The ACE_Connector is a factory i.e., it connects and activates an ACE_Svc_Handler Theres typically 1 ACE_Connector per-service

Vanderbilt University

Advanced ACE Tutorial

Do

ACE_Connector Class Public Interface

A reusable template factory class that establishes connections with clients
template <class SVC_HANDLER, // Type of service class PEER_CONNECTOR> // Connection factory class ACE_Connector : public ACE_Service_Object { public: // Initiate connection to Peer. virtual int connect (SVC_HANDLER *&svc_handler, typename const PEER_CONNECTOR::PEER_ADDR &ra, ACE_Synch_Options &synch_options); // Cancel a <svc_handler> that was // started asynchronously. virtual int cancel (SVC_HANDLER *svc_handler);

Vanderbilt University

Advanced ACE Tutorial

Do

Design Interlude: Motivation for the ACE_Synch_Options Class

Q: What is the ACE_Synch_Options class? A: This allows callers to dene the synchrony/asynchrony policies, e.g.,
class ACE_Synch_Options { // Options flags for controlling // synchronization. enum { USE_REACTOR = 1, USE_TIMEOUT = 2 }; ACE_Synch_Options (u_long options = 0, const ACE_Time_Value &timeout = ACE_Time_Value::zero, const void *act = 0); // This is the default synchronous setting. static ACE_Synch_Options synch; // This is the default asynchronous setting. static ACE_Synch_Options asynch; };

Vanderbilt University

Advanced ACE Tutorial

Do

ACE_Synch_Options and ACE_Connector Semantics

Reactor Yes Timeout 0,0 Behavior Return 1 with errno EWOULDBLOCK; service handler is closed via reactor event loop. Return 1 with errno EWOULDBLOCK; wait up to specied amount of time for completion using the reactor. Return 1 with errno EWOULDBLOCK; wait for completion indenitely using the reactor. Close service handler directly; return 1 with errno EWOULDBLOCK. Block in connect_svc_handler() up to specied amount of time for completion; if still not completed, return 1 with errno ETIME. Block in connect_svc_handler() indenitely for completion.

, ,

Yes

time

Yes

NULL

No No

0,0 time

No

NULL

Vanderbilt University

Advanced ACE Tutorial

Do

ACE_Connector Class Protected Interface

protected: // Make a new connection. virtual SVC_HANDLER *make_svc_handler (void); // Accept a new connection. virtual int connect_svc_handler (SVC_HANDLER *&sh, typename const PEER_CONNECTOR::PEER_ADDR &addr, ACE_Time_Value *timeout); // Activate a service handler. virtual int activate_svc_handler (SVC_HANDLER *); // Demultiplexing hooks. virtual int handle_output (ACE_HANDLE);// Success. virtual int handle_input (ACE_HANDLE); // Failure. virtual int handle_timeout (ACE_Time_Value &, const void *); // Table maps I/O handle to an ACE_Svc_Tuple *. Hash_Map_Manager<ACE_HANDLE, ACE_Svc_Tuple *, ACE_Null_Mutex> handler_map_; // Factory that establishes connections actively. PEER_CONNECTOR connector_; };

Vanderbilt University

Advanced ACE Tutorial

Do

ACE_Connector Class Implementation

// Initiate connection using specified // blocking semantics. template <class SH, class PC> int ACE_Connector<SH, PC>::connect (SH *&sh, const PC::PEER_ADDR &r_addr, ACE_Synch_Options &options) { ACE_Time_Value *timeout = 0; int use_reactor = options[ACE_Synch_Options::USE_REACTOR]; if (use_reactor) timeout = &ACE_Time_Value::zero; else timeout = options[ACE_Synch_Options::USE_TIMEOUT] ? (Time_Value *) &options.timeout () : 0; // Hook methods. if (sh == 0) sh = make_svc_handler (); if (connect_svc_handler (sh, raddr, timeout) != -1) activate_svc_handler (sh); }

Vanderbilt University

Advanced ACE Tutorial

Do

ACE_Connector Hook Method Implementations

template <class SH, class PC> SH * ACE_Connector<SH, PC>::make_svc_handler (void) { return new SH; } template <class SH, class PC> int ACE_Connector<SH, PC>::connect_svc_handler (SH &*sh, typename const PEER_CONNECTOR::PEER_ADDR &addr, ACE_Time_Value *timeout) { // Peer_Connector factory initiates connection. if (connector_.connect (sh, addr, timeout) == -1) // If the connection hasnt completed, then // register with the Reactor to call us back. if (use_reactor && errno == EWOULDBLOCK) // Create <ACE_Svc_Tuple> for <sh> & return -1 } else // Activate immediately if were connected. activate_svc_handler (sh); } template <class SH, class PC> int ACE_Connector<SH, PC>::activate_svc_handler (SH *sh) { if (sh->open ((void *)this) == -1) sh->close (); }

Vanderbilt University

Advanced ACE Tutorial

Do

Specializing ACE_Connector and ACE_Svc_Handler

APPLICATIONINDEPENDENT APPLICATIONSPECIFIC

Svc Handler

Proxy Handler Consumer Handler Supplier Handler

Message Queue

Producing an application that meets Gateway requirements involves specializing ACE components ACE_Connector ! ACE_Proxy_Handler_Connector ACE_Svc_Handler ! ACE_Proxy_Handler ! ACE_Supplier_Handler and ACE_Consumer_Handler
Vanderbilt University

Advanced ACE Tutorial

Do

ACE_Proxy_Handler Class Public Interface

// Determine the type of threading mechanism. #if defined (ACE_USE_MT) typedef ACE_MT_SYNCH SYNCH; #else typedef ACE_NULL_SYNCH SYNCH; #endif /* ACE_USE_MT */ // Unique connection id that denotes Proxy_Handler. typedef short CONN_ID; // This is the type of the Routing_Table. typedef ACE_Hash_Map_Manager <Peer_Addr, Routing_Entry, SYNCH::MUTEX> ROUTING_TABLE; class Proxy_Handler : public ACE_Svc_Handler<ACE_SOCK_Stream, SYNCH> { public: // Initialize the handler (called by the // <ACE_Connector> or <ACE_Acceptor>). virtual int open (void * = 0); // Bind addressing info to Router. virtual int bind (const ACE_INET_Addr &, CONN_ID);
Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Design Interlude: Parameterizing Synchronization into the ACE_Hash_Map_Manager

Q: Whats a good technique to implement a Routing Table? A: Use a ACE_Hash_Map_Manager container ACE provides a ACE_Hash_Map_Manager container that associates external ids with internal ids, e.g.,  External ids (keys) ! URI  Internal ids (values) ! pointer to memory-mapped le Hashing provides O1 performance in the average-case

Vanderbilt University

233

Advanced ACE Tutorial

Douglas C. Schmidt

Applying the Strategized Locking pattern to the ACE_Hash_Map_Manager Class

template <class EXT_ID, class INT_ID, ACE_Hash_Map_Manager class LOCK> class ACE_Hash_Map_Manager { public: uses the template-based Strategized Locking pattern bool bind (EXT_ID, INT_ID *); bool unbind (EXT_ID); bool find (EXT_ID ex, INT_ID &in) to { // Exception-safe code... Enhance reuse ACE_READ_GUARD (LOCK, g, lock_, false); // lock_.read_acquire (); Parameterize different if (find_i (ex, in)) return true; synchronization else return false; // lock_.release (); strategies, e.g.: } private: ACE_Null_Mutex, LOCK lock_; bool find_i (EXT_ID, INT_ID &); ACE_Thread_Mutex, // ... ACE_RW_Mutex, etc. };
Vanderbilt University 234

Advanced ACE Tutorial

Do

Detailed OO Architecture of the Gateway

Consumer Handler
SOCK Stream Message Queue

Routing Table
Hash Map Manager

Supplier Handler
SOCK Stream

Reactor Supplier Handler

SOCK Stream

Connector
SOCK Connector Hash Map Manager

Acceptor
SOCK Acceptor

Consumer Handler
SOCK Stream Message Queue

INCOMING MESSAGES CONNECTION REQUEST

GATEWAY
CONNECTION REQUEST

OUTGOING MESSAGES

Note the use of other ACE components, such as the socket wrapper facades and the ACE_Hash_Map_Manager

Vanderbilt University

Advanced ACE Tutorial

Do

ACE_Supplier_Handler Interface
class Supplier_Handler : public Proxy_Handler { public: Supplier_Handler (void); protected: // Receive and process Peer messages. virtual int handle_input (ACE_HANDLE); // Receive a message from a Peer. virtual int recv_peer (ACE_Message_Block *&); // Action that routes a message from a Peer. int route_message (ACE_Message_Block *); // Keep track of message fragment. ACE_Message_Block *msg_frag_; };

Vanderbilt University

Advanced ACE Tutorial

Do

ACE_Consumer_Handler Interface
class Consumer_Handler : public Proxy_Handler { public: Consumer_Handler (void); // Send a message to a Gateway // (may be queued). virtual int put (ACE_Message_Block *, ACE_Time_Value * = 0); protected: // Perform a non-blocking put(). int nonblk_put (ACE_Message_Block *mb); // Finish sending a message when // flow control abates. virtual int handle_output (ACE_HANDLE); // Send a message to a Peer. virtual int send_peer (ACE_Message_Block *); };

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

ACE_Proxy_Handler_Connector Class Interface

class Proxy_Handler_Connector : public ACE_Connector <Proxy_Handler, // Type of Svc Handler ACE_SOCK_Connector> // Connection factory { public: // Initiate (or reinitiate) // a connection on // the Proxy_Handler. int initiate_connection (Proxy_Handler *); }

ACE_Proxy_Handler_ Connector is a concrete factory class that: Establishes connections with Peers to produce ACE_Proxy_Handlers Activates ACE_Proxy_Handlers, which then route messages ACE_Proxy_Handler_ Connector also ensures reliability by restarting failed connections
238

Vanderbilt University

Advanced ACE Tutorial

Do

ACE_Proxy_Handler_Connector Implementation
// (re)initiate a connection to a Proxy_Handler int Proxy_Handler_Connector::initiate_connection (Proxy_Handler *ph) { // Use asynchronous connections... if (connect (ph, ph->addr (), ACE_Synch_Options::asynch) == -1) { if (errno == EWOULDBLOCK) // No error, were connecting asynchronously. return -1; else // This is a real error, so reschedule // ourselves to reconnect. reactor ()->schedule_timer (ph, 0, ph->timeout ()); } else // Were connected synchronously! return 0; }

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

The Non-blocking Buffered I/O Pattern

Routing Table
find() 1

Intent
Message Queue

ROUTING LAYER

Supplier Handler
handle_input()

n Consumer Handler
handle_output() put()

Decouple multiple input sources from multiple output sources to prevent blocking
Forces Resolved: Keep misbehaving connections from disrupting the QoS for well-behaved connections Different concurrency strategies for Supplier_Handlers and Consumer_Handlers
240

REACTIVE LAYER

Event Handler n

Reactor

www.cs.wustl.edu/schmidt/PDF/ TAPOS-00.pdf

Vanderbilt University

Advanced ACE Tutorial

Do

Collaboration in Single-threaded Gateway Routing

Routing Table Consumer Handler
Message Queue Consumer Handlers

5: send_peer(msg) Consumer Handler

ROUTE ID

3: find()

4: pu t( m sg )
6: put (msg)

Message Queue

Supplier Handler

1: handle_input() 2: recv_peer(msg)

7: send_peer(msg) 8: enqueue(msg) 9: schedule_wakeup() --------------10: handle_output() 11: dequeue(msg) 12: send_peer(msg)

Note the complex cooperative scheduling logic required to handle output ow control correctly

Vanderbilt University

Advanced ACE Tutorial

Do

Supplier_Handler and Consumer_Handler Implementations

int Supplier_Handler::handle_input (ACE_HANDLE) { ACE_Message_Block *route_addr = 0; int n = recv_peer (route_addr); // Try to get the next message. if (n <= 0) { if (errno == EWOULDBLOCK) return 0; else return n; } else route_message (route_addr); } // Send a message to a Peer (queue if necessary). int Consumer_Handler::put (ACE_Message_Block *mb, ACE_Time_Value *) { if (msg_queue_->is_empty ()) // Try to send the message *without* blocking! nonblk_put (mb); else // Messages are queued due to flow control. msg_queue_->enqueue_tail (mb, &ACE_Time_Value::zero); }

Vanderbilt University

Advanced ACE Tutorial

Do

Supplier_Handler Message Routing

// Route message from a Peer. int Supplier_Handler::route_messages (ACE_Message_Block *route_addr) { // Determine destination address. CONN_ID route_id = *(CONN_ID *) route_addr->rd_ptr (); const ACE_Message_Block *const data = route_addr->cont (); Routing_Entry *re = 0; // Determine route. Routing_Table::instance ()->find (route_id, re); // Initialize iterator over destination(s). Set_Iterator<Proxy_Handler *> si (re->destinations ()); // Multicast message. for (Proxy_Handler *out_ph; si.next (out_ph) != -1; si.advance ()) { ACE_Message_Block *newmsg = data->duplicate (); if (out_ph->put (newmsg) == -1) // Drop message. newmsg->release (); // Decrement ref count. } delete route_addr; }

Vanderbilt University

Advanced ACE Tutorial

Do

Peer_Message Schema
// Peer address is used to identify the // source/destination of a Peer message. class Peer_Addr { public: CONN_ID conn_id_; // Unique connection id. u_char logical_id_; // Logical ID. u_char payload_; // Payload type. }; // Fixed sized header. class Peer_Header { public: /* ... */ }; // Variable-sized message (sdu_ may be // between 0 and MAX_MSG_SIZE). class Peer_Message { public: // The maximum size of a message. enum { MAX_PAYLOAD_SIZE = 1024 }; Peer_Header header_; // Fixed-sized header. char sdu_[MAX_PAYLOAD_SIZE]; // Message payload. };

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Design Interlude: Tips on Handling Flow Control

Q: What should happen if put() fails? e.g., if a queue becomes full? A: The answer depends on whether the error handling policy is different for each router object or the same... Strategy pattern: give reasonable default, but allow substitution A related design issue deals with avoiding output blocking if a Peer connection becomes ow controlled

Vanderbilt University

245

Advanced ACE Tutorial

Do

Supplier Handler Message Reception

// Pseudo-code for recving msg via non-blocking I/O int Supplier_Handler::recv_peer (ACE_Message_Block *&route_addr) { if (msg_frag_ is empty) { msg_frag_ = new ACE_Message_Block; receive fixed-sized header into msg_frag_ if (errors occur) cleanup else determine size of variable-sized msg_frag_ } else determine how much of msg_frag_ to skip non-blocking recv of payload into msg_frag_ if (entire message is now received) { route_addr = new Message_Block (sizeof (Peer_Addr), msg_frag_) Peer_Addr addr (id (), msg_frag_->routing_id_, 0); route_addr->copy (&addr, sizeof (Peer_Addr)); return to caller and reset msg_frag_ } else if (only part of message is received) return errno = EWOULDBLOCK else if (fatal error occurs) cleanup }

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Design Interlude: Using the ACE_Reactor to Handle Flow Control

Q: How can a ow controlled Consumer_Handler know when to proceed again without polling or blocking? A: Use the ACE_Event_Handler::handle_output() notication scheme of the Reactor i.e., via the ACE_Reactors methods schedule_wakeup() and cancel_wakeup() This provides cooperative multi-tasking within a single thread of control The ACE_Reactor calls back to the handle_output() hook method when the Proxy_Handler is able to transmit again
Vanderbilt University 247

Advanced ACE Tutorial

Douglas C. Schmidt

Performing a Non-blocking put() of a Message

int Consumer_Handler::nonblk_put This method is called (ACE_Message_Block *mb) { in two situations: // Try sending message 1. When rst trying // via non-blocking I/O if (send_peer (mb) != -1 to send over a && errno == EWOULDBLOCK) { connection // Queue in *front* of the 2. When ow control // list to preserve order. msg_queue_->enqueue_head abates (mb, &ACE_Time_Value::zero); // Tell Reactor to call us // back its ok to send again. reactor ()->schedule_wakeup (this, ACE_Event_Handler::WRITE_MASK); } }
Vanderbilt University 248

Advanced ACE Tutorial

Do

Sending a Message to a Consumer

int Consumer_Handler::send_peer (ACE_Message_Block *mb) { ssize_t n; size_t len = mb->length (); // Try to send the message. n = peer ().send (mb->rd_ptr (), len); if (n <= 0) return errno == EWOULDBLOCK ? 0 : n; else if (n < len) // Skip over the part we did send. mb->rd_ptr (n); else /* if (n == length) */ { // Decrement reference count. mb->release (); errno = 0; } return n; }

Vanderbilt University

Advanced ACE Tutorial

Do

Finish Sending when Flow Control Abates

// Finish sending a message when flow control // conditions abate. This method is automatically // called by the Reactor. int Consumer_Handler::handle_output (ACE_HANDLE) { ACE_Message_Block *mb = 0; // Take the first message off the queue. msg_queue_->dequeue_head (mb, &ACE_Time_Value::zero); if (nonblk_put (mb) != -1 || errno != EWOULDBLOCK) { // If we succeed in writing msg out completely // (and as a result there are no more msgs // on the <ACE_Message_Queue>), then tell the // <ACE_Reactor> not to notify us anymore. if (msg_queue_->is_empty () reactor ()->cancel_wakeup (this, ACE_Event_Handler::WRITE_MASK); } }
Vanderbilt University

Advanced ACE Tutorial

Do

The Gateway Class

Connector Service Object Hash Map Manager
APPLICATIONINDEPENDENT APPLICATIONSPECIFIC

Proxy Handler Connector

Config Table
SUPPLIER HANDLER CONSUMER HANDLER

Routing Table

Gateway

This class integrates other application-specic and application-independent components

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Dynamically Conguring Gateway into an Application

Parameterized by proxy handler Example of the Component Congurator pattern

template <class SUPPLIER_HANDLER, int main (int argc, char *argv[]) class CONSUMER_HANDLER> { class Gateway // Initialize the daemon and : public Service_Object // dynamically configure services. { ACE_Service_Config::open (argc, public: argv); // Perform initialization. // Run forever, performing the virtual int init // configured services. (int argc, char *argv[]); ACE_Reactor::instance ()-> run_reactor_event_loop (); // Perform termination. /* NOTREACHED */ virtual int fini (void); }

Vanderbilt University

252

Advanced ACE Tutorial

Do

Using the ACE Service Congurator Framework for the Gateway

SERVICE CONFIGURATOR RUNTIME

Reactive Gateway Service Object Reactor DLLS

Service Repository Service Config

Thread-per Connection Gateway Service Object Thread Pool Gateway Service Object

svc.conf
FILE

dynamic Gateway Service_Object * gateway:make_Gateway() "-p 2001"

We can replace the single-threaded Gateway with a multi-threaded Gateway

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Dynamic Linking a Gateway Service

The Gateway service is congured via scripting in a svc.conf le:
% cat ./svc.conf static Svc_Manager "-p 5150" dynamic Gateway Service_Object * gateway:_make_Gateway() "-d -p $PORT" # .dll or .so suffix # added to "gateway" # automatically

Dynamically linked factory function that allocates a new single-threaded Gateway

extern "C" ACE_Service_Object *make_Gateway (void); ACE_Service_Object *make_Gateway (void) { return new Gateway<Supplier_Handler, Consumer_Handler>; // ACE automatically deletes memory. }

Vanderbilt University

254

Advanced ACE Tutorial

Douglas C. Schmidt

Concurrency Strategies for Patterns

The Acceptor-Connector pattern does not constrain the concurrency strategies of a ACE_Svc_Handler There are three common choices: 1. Run service in same thread of control 2. Run service in a separate thread 3. Run service in a separate process Observe how our patterns and ACE framework push this decision to the edges of the design This greatly increases reuse, exibility, and performance tuning

Vanderbilt University

255

Advanced ACE Tutorial

Do

Using the Active Object Pattern for the Gateway

Consumer Consumer Handler Handler Event Handler Event Handler Supplier Supplier Handler Supplier Handler Handler Event Event Handler 4: send(msg) Event Handler Handler 2: recv(msg) 3: route(msg) 1: handle_input() Timer Queue Handle Table

FRAMEWORK LEVEL

APPLICATION LEVEL

CONCRETE EVENT HANDLERS

Signal Handlers

Reactor
OS EVENT DEMULTIPLEXING INTERFACE

Each Consumer_Handler is implemented as an Active Object

Vanderbilt University

KERNEL LEVEL

Advanced ACE Tutorial

Do

Collaboration in Multi-threaded Gateway Routing

Routing Table Consumer Handler
Message Queue Consumer Handlers

5: send_peer(msg) Consumer Handler

ROUTE ID

3: find()

4: pu t( m sg )

Message Queue

Supplier Handler

6: put (msg) 1: handle_input() 2: recv_peer(msg)

5: send_peer(msg)

Note that this design is much simpler since the OS thread scheduler handles blocking

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Using the Half-Sync/Half-Async Pattern in the Gateway

SYNCHRONOUS TASK LAYER
Consumer Handler Consumer Handler Consumer Handler

ACE_Reactor plays the role of async layer ACE_Task active object plays the role of sync layer This particular conguration is a common variant of the Half-Sync/Half-Async pattern, as described in POSA2

1: dequeue(msg) 2: send(msg)

QUEUEING LAYER

MESSAGE QUEUES

ASYNCHRONOUS TASK LAYER

Supplier Handler Supplier Supplier Handler Handler

2: recv(msg) 3: get_route(msg) 4: enqueue(msg) 1: dispatch() Reactor

Vanderbilt University

258

Advanced ACE Tutorial

Do

Class Diagram for Multi-Threaded Gateway

APPLICATIONSPECIFIC COMPONENTS
Proxy_Handler SOCK_Connector SOCK_Stream MT_Synch

1 Proxy Handler Connector

<<activates>>

Supplier/Thr_Consumer Handler n

CONNECTIONORIENTED COMPONENTS

SVC_HANDLER PEER_CONNECTOR

PEER_STREAM SYNCH

Connector

Svc Handler

PEER CONNECTOR

PEER STREAM

ACE FRAMEWORK COMPONENTS

Connection IPC_SAP

Stream

Service Configurator
global

Concurrency

Reactor

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Thr_Consumer_Handler Class Interface

#define ACE_USE_MT #include Proxy_Handler.h class Thr_Consumer_Handler : public Consumer_Handler { public: // Initialize the object and // spawn new thread. virtual int open (void *); // Send a message to a peer. virtual int put (ACE_Message_Block *, ACE_Time_Value *); // Transmit peer messages // in separate thread. virtual int svc (void); };
Vanderbilt University

New subclass of Proxy_Handler uses the Active Object pattern for the Consumer_Handler Uses multi-threading and synchronous I/O (rather than non-blocking I/O) to transmit message to Peers Transparently improve performance on a multi-processor platform and simplify design

260

Advanced ACE Tutorial

Douglas C. Schmidt

Thr_Consumer_Handler Class Implementation

Override denition in the Consumer_Handler class
int Thr_Consumer_Handler::open (void *) { // Become an active object by // spawning a new thread to // transmit messages to Peers. activate (THR_DETACHED); }

The multi-threaded version of open() is slightly different since it spawns a new thread to become an active object! activate() is a pre-dened method on ACE_Task

Vanderbilt University

261

Advanced ACE Tutorial

Do

Thr_Consumer_Handler Class Implementation

// Queue up a message for transmission. int Thr_Consumer_Handler::put (ACE_Message_Block *mb, ACE_Time_Value *) { // Perform non-blocking enqueue. msg_queue_->enqueue_tail (mb, &ACE_Time_Value::zero); } // Transmit messages to the peer (note // simplification resulting from threads...) int Thr_Consumer_Handler::svc (void) { ACE_Message_Block *mb = 0; // Since this method runs in its own thread it // is OK to block on output. while (msg_queue_->dequeue_head (mb) != -1) send_peer (mb); }
Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Dynamic Linking a Threaded Gateway Service

% cat ./svc.conf remove Gateway dynamic Gateway Service_Object * thr_gateway:_make_Gateway() "-d" # .dll or .so suffix added # to "thr_Gateway" # automatically

Dynamically linked factory function that allocates a multi-threaded Gateway object

extern "C" ACE_Service_Object *make_Gateway (void); ACE_Service_Object *make_Gateway (void) { return new Gateway<Supplier_Handler, Thr_Consumer_Handler>; // ACE automatically deletes memory. }

Vanderbilt University

263

Advanced ACE Tutorial

Do

Call Center Manager (CCM) Event Server Example

SUPER VISOR SUPER VISOR SUPER VISOR

CCM Stream

Session Router Module Event Filter Module

MIB ACE RUN-TIME

TELECOM SWITCHES

Event Analyzer Module Switch Adapter Module

EVENT SERVER

Vanderbilt University

Advanced ACE Tutorial

Do

Patterns in the CCM Event Server

Publisher Subscriber Layers AcceptorConnector
STRATEGIC PATTERNS TACTICAL PATTERNS Iterator

Pipes & Filters Component Configurator Reactor

Factory Composite Method

Proxy Wrapper Facade

The Event Server components are based upon a common pattern language www.cs.wustl.edu/schmidt/PDF/ DSEJ-94.pdf

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Overview of the ACE Streams Framework

An ACE_Stream allows exible conguration of layered processing modules It is an implementation of the Pipes and Filters architecture pattern This pattern provides a structure for systems that process a stream of data Each processing step is encapsulated in a lter ACE_Module component Data is passed through pipes between adjacent lters, which can be re-combined The CCM Event Server was design and implemented using ACE Streams
Vanderbilt University 266

Advanced ACE Tutorial

Do

Structure of the ACE Streams Framework

SYNCH SYNCH SYNCH

ACE_Task
2

ACE_Module
2..*

ACE_Stream

Framework characteristics An ACE_Stream contains a stack of ACE_Modules Each ACE_Module contains two ACE_Tasks i.e., a read task and a write task Each ACE_Task contains an ACE_Message_Queue and a pointer to an ACE_Thread_Manager

Vanderbilt University

Advanced ACE Tutorial

Do

Implementing a Stream in ACE

APPLICATION

Stream

APPLICATION

Stream

STREAM Head

DOWNSTREAM

UPSTREAM

Multiplexor STREAM Tail

NETWORK INTERFACE OR PSEUDO-DEVICES

open()=0 close()=0 put()=0 svc()=0

MESSAGE

MODULE

WRITE TASK

READ TASK

Note similarities to System V STREAMS

Vanderbilt University

Advanced ACE Tutorial

Do

Alternative Concurrency Models for Message Processing

ACTIVE ACTIVE ACTIVE

Module A

ACTIVE

4: svc() 3: put()

Module A

2: put()
Module B

Module B

ACTIVE

2: svc() 1: put()

Module C

ACTIVE

Module C

1: put()

TASK-BASED PROCESS ARCHITECTURE

MESSAGE-BASED PROCESS ARCHITECTURE

MESSAGE OBJECT

MODULE OBJECT

WRITE TASK OBJECT

READ TASK OBJECT

PROCESS OR THREAD

Task-based models are more intuitive but less efcient than Message-based models
Vanderbilt University

Advanced ACE Tutorial

Do

Using the ACE Streams Framework for the CCM Event Server
Session Router Module Presentation Module Event Filter Module Event Analysis Module Presentation Module Switch Adapter Module
MD110 ERICSSON

SUPER VISORS SUPER VISORS SUPER VISORS

Session IO
Reactor

Switch IO
MD110 ERICSSON

MD110

ERICSSON

TELECOM SWITCHES

www.cs.wustl.edu/schmidt/PDF/ DSEJ-94.pdf

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Broader Context: External OS for Telecom Switches

CLIENT CLIENT CLIENT NETWORK CLIENT

Features Allow clients to manage various aspects of telecom switches without modifying the switch software Support reuse of existing components based on a common architectural framework

SWITCHES

DIRECTORY MANAGER EXTENSION MANAGER

CALL CENTER MANAGER

APPLICATION FRAMEWORK

SERVER

Vanderbilt University

271

Advanced ACE Tutorial

Douglas C. Schmidt

Applying ACE Streams to External OS

CLIENT CLIENT CALL CENTER MANAGER DIRECTORY MANAGER

CLIENT

NETWORK

Event Analyzer Switch Adapter

FRAMEWORK

DATABASE

SWITCHES

Session Router Reactor

EXTENSION MANAGER

ACE

SERVER

Vanderbilt University

272

Advanced ACE Tutorial

Douglas C. Schmidt

ACE Stream Example: Parallel I/O Copy

5: write()
Consumer Module active

4: svc() 3: put()

Program copies stdin to stdout via the use of a multi-threaded ACE_Stream Stream implements a bounded buffer Since the data ow is uni-directional the read ACE_Task is always ignored

Producer Module

active

2: svc()

1: read()
Vanderbilt University 273

Advanced ACE Tutorial

Do

Producer Class Interface

typedef short-hands for templates
typedef ACE_Stream<ACE_MT_SYNCH> MT_Stream; typedef ACE_Module<ACE_MT_SYNCH> MT_Module; typedef ACE_Task<ACE_MT_SYNCH> MT_Task;

Dene the Producer interface

class Producer : public MT_Task { public: // Initialize Producer. virtual int open (void *) { // activate() is inherited from class Task. activate (THR_BOUND); } // Read data from stdin and pass to consumer. virtual int svc (void); // ... };

Vanderbilt University

Advanced ACE Tutorial

Do

Producer Class Implementation

Runs as an active object in a separate thread
int Producer::svc (void) { for (;;) { ACE_Message_Block *mb; // Allocate a new message. ACE_NEW_RETURN (mb, ACE_Message_Block (BUFSIZ), -1); // Keep reading stdin, until we reach EOF. ssize_t n = ACE_OS::read (ACE_STDIN, mb->wr_ptr (), mb->size ()); if (n <= 0) { // Send shutdown message to other // thread and exit. mb->length (0); this->put_next (mb); break; } else { mb->wr_ptr (n); // Adjust write pointer. // Send the message to the other thread. this->put_next (mb); } } }
Vanderbilt University

Advanced ACE Tutorial

Do

Consumer Class Interface

Dene the Consumer interface
class Consumer : public MT_Task { public: // Initialize Consumer. virtual int open (void *) { // <activate> is inherited from class Task. activate (THR_BOUND); } // Enqueue the message on the Message_Queue // for subsequent processing in <svc>. virtual int put (ACE_Message_Block *, ACE_Time_Value * = 0) { // <putq> is inherited from class Task. return putq (mb, tv); } // Receive message from producer // and print to stdout. virtual int svc (void); };

Vanderbilt University

Advanced ACE Tutorial

Do

Consumer Class Implementation

Consumer dequeues a message from the ACE_Message_Queue, writes the message to the stderr stream, and deletes the message
int Consumer::svc (void) { ACE_Message_Block *mb = 0; // Keep looping, reading a message from the queue, // until we get a 0 length message, then quit. for (;;) { int result = getq (mb); if (result == -1) break; int length = mb->length (); if (length > 0) ACE_OS::write (ACE_STDOUT, mb->rd_ptr (), length); mb->release (); if (length == 0) break; } }

The Producer sends a 0-sized message to inform the Consumer to stop reading and exit
Vanderbilt University

Advanced ACE Tutorial

Do

Main Driver Function for the Stream

Create Producer and Consumer Modules and push them onto the Stream
int main (int argc, char *argv[]) { // Control hierarchically-related // active objects. MT_Stream stream; // All processing is performed in the // Stream after <push>s complete. stream.push (new MT_Module ("Consumer", new Consumer); stream.push (new MT_Module ("Producer", new Producer)); // Barrier synchronization: wait for // the threads, to exit, then exit // the main thread. ACE_Thread_Manager::instance ()->wait (); }

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Evaluation of the ACE Stream Framework

Structuring active objects via an ACE_Stream allows interpositioning i.e., similar to adding a lter in a UNIX pipeline New functionality may be added by pushing a new processing ACE_Module onto an ACE_Stream, e.g.:
stream.push (new MT_Module ("Consumer", new Consumer)) stream.push (new MT_Module ("Filter", new Filter)); stream.push (new MT_Module ("Producer", new Producer));

Communication between ACE_Modules is typically anonymous

Vanderbilt University

279

Advanced ACE Tutorial

Douglas C. Schmidt

Concurrency Strategies
Developing correct, efcient, and robust concurrent applications is challenging Below, we examine a number of strategies that addresses challenges related to the following:

Concurrency control Library design Thread creation Deadlock and starvation avoidance

Vanderbilt University

280

Advanced ACE Tutorial

Douglas C. Schmidt

General Threading Guidelines

A threaded program should not arbitrarily enter non-threaded (i.e., unsafe) code Threaded code may refer to unsafe code only from the main thread e.g., beware of errno problems Use reentrant OS library routines ( r) rather than non-reentrant routines Beware of thread global process operations, such as le I/O Make sure that main() terminates cleanly e.g., beware of pthread_exit(), exit(), and falling off the end
Vanderbilt University 281

Advanced ACE Tutorial

Douglas C. Schmidt

Thread Creation Strategies

Use threads for independent jobs that must maintain state for the life of the job Dont spawn new threads for very short jobs Use threads to take advantage of CPU concurrency Only use bound threads when absolutely necessary If possible, tell the threads library how many threads are expected to be active simultaneously e.g., use thr_setconcurrency()

Vanderbilt University

282

Advanced ACE Tutorial

Douglas C. Schmidt

General Locking Guidelines

Dont hold locks across long duration operations (e.g., I/O) that can impact performance Use ACE_Token instead... Beware of holding non-recursive mutexes when calling a method outside a class The method may reenter the module and deadlock Dont lock at too small of a level of granularity Make sure that threads obey the global lock hierarchy But this is easier said than done...

Vanderbilt University

283

Advanced ACE Tutorial

Douglas C. Schmidt

Locking Alternatives
Code locking
Associate locks with body of functions  Typically performed using bracketed mutex locks Often called a Monitor Object

Data locking
Associate locks with data structures and/or objects Permits a more ne-grained style of locking Data locking allows more concurrency than code locking, but may incur higher overhead

Vanderbilt University

284

Advanced ACE Tutorial

Douglas C. Schmidt

Single-lock Strategy
One way to simplify locking is use a single, application-wide mutex lock Each thread must acquire the lock before running and release it upon completion The advantage is that most legacy code doesnt require changes The disadvantage is that parallelism is eliminated Moreover, interactive response time may degrade if the lock isnt released periodically

Vanderbilt University

285

Advanced ACE Tutorial

Douglas C. Schmidt

Monitor Object Strategy

A more OO locking strategy is to use a Monitor Object www.cs.wustl.edu/schmidt/POSA/ Monitor Object synchronization mechanisms allow concurrent method invocations Either eliminate access to shared data or use synchronization objects Hide locking mechanisms behind method interfaces  Therefore, modules should not export data directly Advantage is transparency Disadvantages are increased overhead from excessive locking and lack of control over method invocation order
Vanderbilt University 286

Advanced ACE Tutorial

Douglas C. Schmidt

Active Object Strategy

Each task is modeled as an active object that maintains its own thread of control Messages sent to an object are queued up and processed asynchronously with respect to the caller i.e., the order of execution may differ from the order of invocation This approach is more suitable to message passing-based concurrency The ACE_Task class can be used to implement active objects www.cs.wustl.edu/schmidt/POSA/

Vanderbilt University

287

Advanced ACE Tutorial

Douglas C. Schmidt

Invariants
In general, an invariant is a condition that is always true For concurrent programs, an invariant is a condition that is always true when an associated lock is not held However, when the lock is held the invariant may be false When the code releases the lock, the invariant must be re-established

e.g., enqueueing and dequeueing messages in the ACE_Message_Queue class

Vanderbilt University

288

Advanced ACE Tutorial

Douglas C. Schmidt

Run-time Stack Problems

Most threads libraries contain restrictions on stack usage The initial thread gets the real process stack, whose size is only limited by the stacksize limit All other threads get a xed-size stack  Each thread stack is allocated off the heap and its size is xed at startup time Therefore, be aware of stack smashes when debugging multi-threaded code Overly small stacks lead to bizarre bugs, e.g.,  Functions that werent called appear in backtraces  Functions have strange arguments

Vanderbilt University

289

Advanced ACE Tutorial

Douglas C. Schmidt

Deadlock
Permanent blocking by a set of threads that are competing for a set of resources Caused by circular waiting, e.g., A thread trying to reacquire a lock it already holds Two threads trying to acquire resources held by the other  e.g., T1 and T2 acquire locks L1 and L2 in opposite order One solution is to establish a global ordering of lock acquisition (i.e., a lock hierarchy) May be at odds with encapsulation...

Vanderbilt University

290

Advanced ACE Tutorial

Douglas C. Schmidt

Avoiding Deadlock in OO Frameworks

Deadlock can occur due to properties of OO frameworks, e.g., Callbacks Inter-class method calls There are several solutions Release locks before performing callbacks  Every time locks are reacquired it may be necessary to reevaluate the state of the object Make private helper methods that assume locks are held when called by methods at higher levels Use an ACE_Token or ACE_Recursive_Thread_Mutex

Vanderbilt University

291

Advanced ACE Tutorial

Do

ACE_Recursive_Thread_Mutex Implementation
Here is portable implementation of recursive thread mutexes available in ACE:
class ACE_Recursive_Thread_Mutex { public: // Initialize a recursive mutex. ACE_Recursive_Thread_Mutex (void); // Implicitly release a recursive mutex. ACE_Recursive_Thread_Mutex (void); // Acquire a recursive mutex. int acquire (void); // Conditionally acquire a recursive mutex. int tryacquire (void); // Releases a recursive mutex. int release (void); private: ACE_Thread_Mutex nesting_mutex_; ACE_Condition_Thread_Mutex mutex_available_; ACE_thread_t owner_; int nesting_level_; };

Vanderbilt University

Advanced ACE Tutorial

Do

Acquiring an ACE_Recursive_Thread_Mutex
int ACE_Recursive_Thread_Mutex::acquire (void) { ACE_thread_t t_id = ACE_Thread::self (); ACE_GUARD_RETURN (ACE_Thread_Mutex, guard, nesting_mutex_, -1); // If theres no contention, grab mutex. if (nesting_level_ == 0) { owner_ = t_id; nesting_level_ = 1; } else if (t_id == owner_) // If we already own the mutex, then // increment nesting level and proceed. nesting_level_++; else { // Wait until nesting level drops // to zero, then acquire the mutex. while (nesting_level_ > 0) mutex_available_.wait (); // Note that at this point // the nesting_mutex_ is held... owner_ = t_id; nesting_level_ = 1; } return 0;

Vanderbilt University

Advanced ACE Tutorial

Do

Releasing and Initializing an ACE_Recursive_Thread_Mutex

int ACE_Recursive_Thread_Mutex::release (void) { ACE_thread_t t_id = ACE_Thread::self (); // Automatically acquire mutex. ACE_GUARD_RETURN (ACE_Thread_Mutex, guard, nesting_mutex_, -1); nesting_level_--; if (nesting_level_ == 0) { // Put the mutex into a known state. owner_ = ACE_OS::NULL_thread; // Inform waiters that the mutex is free. mutex_available_.signal (); } return 0; } ACE_Recursive_Thread_Mutex:: ACE_Recursive_Thread_Mutex (void) : nesting_level_ (0), owner_ (ACE_OS::NULL_thread), mutex_available_ (nesting_mutex_){}

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Avoiding Starvation
Starvation occurs when a thread never acquires a mutex even though another thread periodically releases it The order of scheduling is often undened This problem may be solved via: Use of voluntary pre-emption mechanisms  e.g., thr_yield() or Sleep() Using an ACE Token that strictly orders acquisition and release

Vanderbilt University

295

Advanced ACE Tutorial

Douglas C. Schmidt

Drawbacks to Multi-threading
Performance overhead
Some applications do not benet directly from threads Synchronization is not free Threads should be created for processing that lasts at least several 1,000 instructions

Correctness
Threads are not well protected against interference Concurrency control issues are often tricky Many legacy libraries are not thread-safe

Development effort
Developers often lack experience Debugging is complicated (lack of tools)
Vanderbilt University 296

Advanced ACE Tutorial

Douglas C. Schmidt

Lessons Learned using OO Patterns

Benets of patterns
Enable large-scale reuse of software architectures Improve development team communication Help transcend language-centric viewpoints

Drawbacks of patterns
Do not lead to direct code reuse Can be deceptively simple Teams may suffer from pattern overload

Vanderbilt University

297

Advanced ACE Tutorial

Douglas C. Schmidt

Lessons Learned using OO Frameworks

Benets of frameworks
Enable direct reuse of code (cf patterns) Facilitate larger amounts of reuse than stand-alone functions or individual classes

Drawbacks of frameworks
High initial learning curve  Many classes, many levels of abstraction The ow of control for reactive dispatching is non-intuitive Verication and validation of generic components is hard

Vanderbilt University

298

Advanced ACE Tutorial

Douglas C. Schmidt

Lessons Learned using C++

Benets of C++
Classes and namespaces modularize the system architecture Inheritance and dynamic binding decouple application policies from reusable mechanisms Parameterized types decouple the reliance on particular types of synchronization methods or network IPC interfaces

Drawbacks of C++
Some language features are not implemented Some development environments are primitive Language has many dark corners and sharp edges  Purify helps alleviate many problems...

Vanderbilt University

299

Advanced ACE Tutorial

Douglas C. Schmidt

Lessons Learned using OOD

Good designs can be boiled down to a few key principles: Separate interface from implementation Determine what is common and what is variable with an interface and an implementation Allow substitution of variable implementations via a common interface  i.e., the open/closed principle & Aspect-Oriented Programming (AOP) Dividing commonality from variability should be goal-oriented rather than exhaustive Design is not simply drawing a picture using a CASE tool, using graphical UML notation, or applying patterns Design is a fundamentally creative activity
Vanderbilt University 300

Advanced ACE Tutorial

Douglas C. Schmidt

Software Principles for Distributed Applications

Use patterns/frameworks to decouple policies/mechanisms Enhance reuse of common concurrent programming components Decouple service functionality from conguration Improve exibility and performance Use classes, inheritance, dynamic binding, and parameterized types Improve extensibility and modularity Enhance performance/functionality with OS features e.g., implicit and explicit dynamic linking and multi-threading Perform commonality/variability analysis Identify uniform interfaces for variable components and support pluggability of variation
Vanderbilt University 301

Advanced ACE Tutorial

Douglas C. Schmidt

Conferences and Workshops on Patterns

Pattern Language of Programs Conferences PLoP, September, 2002, Monticello, Illinois, USA OOPSLA, November, 2002, Seattle, USA hillside.net/patterns/conferences/ Distributed Objects and Applications Conference Oct/Nov, 2002, UC Irvine www.cs.wustl.edu/schmidt/activities-chair.html

Vanderbilt University

302

Advanced ACE Tutorial

Douglas C. Schmidt

Patterns, Frameworks, and ACE Literature

Books Gamma et al., Design Patterns: Elements of Reusable Object-Oriented Software AW, 94 Pattern Languages of Program Design series by AW, 95-99. Siemens & Schmidt, Pattern-Oriented Software Architecture, Wiley, volumes 96 & 00 (www.posa.uci.edu) Schmidt & Huston, C++ Network Programming: Mastering Complexity with ACE and Patterns, AW, 02 (www.cs.wustl.edu/schmidt/ACE/book1/) Schmidt & Huston, C++ Network Programming: Systematic Reuse with ACE and Frameworks, AW, 03 (www.cs.wustl.edu/schmidt/ACE/book2/)

Vanderbilt University

303

Advanced ACE Tutorial

Do

How to Obtain ACE Software and Technical Support

All source code for ACE is freely available www.cs.wustl.edu/schmidt/ACE. html Mailing lists [email protected] [email protected] [email protected] [email protected]

Newsgroup comp.soft-sys.ace Commercial support from Riverace and OCI www.riverace.com www.theaceorb.com

Vanderbilt University

Advanced ACE Tutorial

Douglas C. Schmidt

Concluding Remarks
Developers of networked application software confront recurring challenges that are largely application-independent e.g., service conguration and initialization, distribution, error handling, ow control, event demultiplexing, concurrency, synchronization, persistence, etc. Successful developers resolve these challenges by applying appropriate patterns to create communication frameworks containing components

Frameworks and components are an effective way to achieve systematic reuse of software

Vanderbilt University

305