Solving Integration Problems using Patterns

Dr. Amor Lazzez

Office location: Tarbia Campus (C/ 306)

Phone: 0549071465 (cell)
Email: [email protected]

502510-3/Systems-Integration
Week 7
Middleware - Recap
• Middleware is any type of software that facilitates
communication between two or more software systems
– Can be simple communication connection between applications
– Can be as sophisticated as information sharing and logic execution
mechanisms
• Middleware is a technology that allows us to move
information between multiple systems that may reside within
and outside an organization
• Message-Oriented Middleware (MOM) is queuing software
that uses is messages as a mechanism to move information
from point to point
– MOM uses the notion of messages to communicate between
applications,
– MOM resolves the problem of tight coupling between applications
Tight Coupling
• A great example of tight coupling is a local method invocation.
• Invoking a local method inside an application is based on a lot of
assumptions between the called and the calling routine.
– Both methods have to run in the same process (e.g. a virtual machine) and be written in
the same language (or at least use a common intermediate language or byte code).
– The calling method has to pass the exact number of expected parameters, each using
the correct type.
• The call is immediate, i.e. the called method starts processing
immediately after the calling method makes the call.
• Meanwhile, the calling method will only resume processing when the
called method completes (meaning the invocation is synchronous).
• The communication between the methods is immediate and
instantaneous, so neither the caller nor the called method have to worry
about security in the form of eavesdropping 3rd parties.
• All these assumptions make it very easy to write well structured
applications that break functionality into individual methods to be called by
other methods.
Tight Coupling Example Problem
• Assume we are building an on-line banking system that
allows customers to deposit money into their account from
another bank.
• Let’s assume that the remote function that deposits money
into a person’s account takes only the person’s name and the
Dollar amount as arguments.
• To perform this function, the front-end Web application has to
be integrated with the back-end financial system that
manages fund transfers.
• The easiest way to connect the two systems is through the
TCP/IP protocol.
– Operating system or programming library include a TCP/IP stack. TCP/IP is
the ubiquitous communications protocol that transports data between the
millions of computers connected to the Internet and local networks.
Tight Coupling Example Solution
Code to call such a function over TCP/IP:
String hostName = "www.boa.com";
int port = 80;

IPHostEntry hostInfo = Dns.GetHostByName(hostName);

IPAddress address = hostInfo.AddressList[0];

IPEndPoint endpoint = new IPEndPoint(address, port);

Socket socket = new Socket(address.AddressFamily, SocketType.Stream,

ProtocolType.Tcp);
socket.Connect(endpoint);

byte[] amount = BitConverter.GetBytes(1000); byte[] name =

Encoding.ASCII.GetBytes("Joe");

int bytesSent = socket.Send(amount);

bytesSent += socket.Send(name);

socket.Close(); • Code opens a socket connection to the address

www.boa.com and sends two data items (the amount and
the customer’s name) across the network.
• When we run this code it tells us: “7 bytes sent”.
Tight Coupling Example Assumptions
• This minimalist integration solution is fast and cheap, but it results in a
very brittle solution because the two participating parties make the
following assumptions about each other:
– Platform Technology – internal representations of numbers and objects
• 32 bit vs 64 bit internal representation
• A system using 64 bits would be inclined to read 8 bytes off the network and would end up
interpreting the whole message (including the customer name) as a single number.
– Location – hard-coded machine addresses
• Remote functions and machine location are hard-coded. Any change in anything, we would
have to change code. If we use a lot of remote functions this could become very tedious.
– Time – all components have to be available at the same time
• Establishing a TCP connection requires that both machines and the network are all available at
the same time.
– Data Format – the list of parameters and their types must match
• If we want to insert a third parameter, e.g. the name of the currency, we would have to modify
both sender and receiver to use the new data format.
• Coupling is a measure of how many assumptions parties make about each other
when they communicate. Our simple solution requires the parties to make a lot of
assumptions. Therefore, this solution is tightly coupled.
Loosely coupled solution
• Instead, we can develop loosely
coupled solution by using a
message-oriented middleware
(MOM) infrastructure
• MOM mechanisms provides services such as
– a common data format and transformers
• Removes platform and data format dependencies as the sender no longer
has to depend on the receiver's internal data format
– queuing channels
• Removes location and time dependencies as we do not have to pay
attention to computer’s identity & location or whether the other computer
is ready to accept requests or not.
• Removing these dependencies between the systems makes
the overall solution more tolerant to change, the key benefit
of loose coupling.
Things that make up Middleware
• Middleware – the things that sit between applications.
• In order to connect two systems via an integration solution, a
number of things have to happen.
• Applications want to exchange data – Messages
– Message: a snippet of data that has an agreed-upon meaning to both
applications that are to be integrated.
– This piece of data can be very small, such as the phone number of a
single customer that has changed, or very large, such as the complete
list of all customers and their associated addresses.
Message Construction
• When two applications wish to exchange a piece of data, they do so by
wrapping it in a message.
• Message intent
– Command Message: invoking a function or method on the receiver
– Document Message: enabling the sender to transmit one of its data structures
– Event Message: notifying the receiver of a change in the sender.
• Returning a response
– Request-Reply: The request is usually a Command Message, and the reply is a
Document Message containing a result value or an exception.
– Return Address: The requestor specifies what channel to use to transmit the reply.
– Correlation Identifier: Responder specifies which request this reply corresponds to.
• Huge amounts of data
– Message Sequence: break the data into more manageable chunks and send them as a
sequence of messages, so that the receiver can reconstruct the original data structure.
• Slow messages
– Message Expiration: the sender can specify an expiration date. If the messaging
system cannot deliver a message by its expiration, it should discard the message. If a
receiver gets a message after its expiration, it should discard the message.
Things that make up Middleware: Channel

• Data needs to be transported, usually across a network.

• We need a communications channel that can move
information from one application to the other. This channel
could be a series of TCP/IP connections, a shared file, or a
shared database.
– A channel is a logical address that both sender and receiver can
agree on the same channel without being aware of each other’s
identity.
• The application selects a particular channel to send the data
knowing that the receiver will be one that is looking for that
sort of data on that channel.
Messaging Channels
• Point-to-Point Channel: To send the data to a single application (1-to-1
interaction)
• Publish-Subscribe Channel: When you send a piece of data this way, the
channel effectively copies the data for each of the receivers (One to many
interaction)
• Datatype Channel: all of the data on a channel has to be of the same type (many
to one interaction)
• Invalid Message Channel: When receiver receives a message that doesn’t meet
these expectations
• Dead Letter Channel: for messages which are successfully sent but ultimately
cannot be successfully delivered.
• Guaranteed Delivery: makes channels persistent so that their messages are
stored on disk
• Channel Adapter: can be used to connect a channel (or set of channels) to the
application without having to modify the application
• Messaging Bridge: connecting two message systems, effectively connecting
them into one composite messaging system.
• Message Bus: a backbone providing access to all of the enterprise’s various and
ever-changing applications and functionality.
Things that make up Middleware:
Translation
• Middleware needs to provide some mechanism to
convert one application’s data format in the other’s.
– Internal data format of an application can often not be
changed
– For example, one data format may store the customer
name in two fields, called FIRST_NAME and
LAST_NAME, while the other system may use a single
field called Customer_Name.
Message Transformation
• Envelope Wrapper: wrap message payload data into an envelope
that is compliant with the requirements of the messaging
infrastructure.
• Content Enricher: when the target system requires data fields that
the originating system cannot supply. It has the ability to look up
missing information or compute it from the available data.
• Content Filter: removes unwanted data from a message.
• Claim Check: removes data from a message but stores it for later
retrieval.
• Normalizer: translates messages arriving in many different
formats into a common format.
• Canonical Data Model: Design an independent data model from
any specific application. Require each application to produce and
consume messages in this common format
Things that make up Middleware: Routing

• Middleware needs to take care of sending

messages to multiple systems
– As the number of systems increases it becomes very
tedious and requires the sending system to have
knowledge about all other systems.
– For example, if the customer address changes in the
customer care system we could make that system
responsible for sending the data to all other systems that
store copies of the customer address.
– We could expect each application to specify the target
system(s) for the data it is sending over the channel.
Message Routing
Things that make up Middleware: Systems
management
• Integration solutions can quickly become complex because
they deal with multiple applications, data formats, channels,
routing and transformation.
– All these elements may be spread across multiple operating platforms
and geographic locations.
• In order to have any idea what is going on inside the system
we need a systems management function.
– This subsystem monitors the flow of data, makes sure that all
applications and components are available and reports error
conditions to a central location.
System Management
• Control Bus: provides a single point of control to manage and monitor a
distributed solution
• Detour: route messages through additional steps, such as validation or
logging – with ability to switch on or off these additional steps
• Wire Tap: inspect the contents of a message without affecting the primary
message flow.
• Message History: great aid to know where a specific message has been
• Message Store: can provide a complete account of every message that
traveled through the system
• Smart Proxy: track messages sent to request-reply services
• Test Message: actively verifying that the running messaging system is
functioning properly
• Channel Purger: remove all remaining messages from a channel so that
the components under test do not receive 'leftover' messages.
Things that make up Middleware: Endpoint

• Most packaged and legacy applications and many custom

applications are not prepared to participate in an integration
solution.
– As they were designed to perform specific functionality with specific
input/output
• We need a message endpoint to connect the system
explicitly to the integration solution.
– The endpoint can be a special piece of code or a Adapter provided by
an integration software vendor.
Messaging Endpoints
• Messaging Gateway: a class than wraps messaging-specific method calls and exposes
domain-specific methods to the application.
• Messaging Mapper: creates a mapping logic between the messaging infrastructure and the
domain objects.
• Transactional Client: make the client’s session with the messaging system transactional so
that the client can specify transaction boundaries.
• Polling Consumer: explicitly makes a call when it wants to receive a message.
• Event-Driven Consumer: automatically handles messages as they’re delivered on the
channel.
• Competing Consumers: Create multiple consumers on a single channel so that multiple
messages can be processed concurrently.
• Message Dispatcher: consume messages from a channel and distribute them to
performers.
• Selective Consumer: filters messages delivered to a channel so that it only receives the
ones that match its criteria.
• Durable Subscriber: to make the messaging system save messages published while the
subscriber is disconnected.
• Idempotent Receiver: can safely receive the same message multiple times.
• Service Activator: connects the messages on the channel to the service being accessed.
Widgets & Gadgets ‘R Us (WGRUS)
• Widgets & Gadgets ‘R Us (WGRUS) an on-line retailer that
buys widgets and gadgets from manufacturers and resells
them to customers.
WGRUS IT Infrastructure
• Four different channels to interact with customers.
• Six internal information systems
• Design an integration solution to integrated these
systems
Requirements – Business Processes
• Take Orders
– Customers can place orders via Web, phone or fax
• Process Orders
– Processing an order involves multiple steps, including verifying inventory, shipping the
goods and invoicing the customer
• Check Status
– Customers can check the order status
• Change Address
– Customers can use a Web front-end to change their billing and shipping address
• New Catalog
– The suppliers update their catalog periodically. WGRUS needs to update its pricing and
availability based in the new catalogs.
• Announcements
– Customers can subscribe to selective announcements from WGRUS.
• Testing and Monitoring
– The operations staff needs to be able to monitor all individual components and the
message flow between them.
Take Orders and Process Orders
Take Orders

Identified Requirements:
• Interactions between
– 1 and 4
– 2 and 5
– 3 and 6
– 4, 5, 6 and 7

Identifying Integration Requirements: Look for change in Performers.

Performers are application that are supporting a specific activity within a process
First Requirement

Message

Channel

Router
Integration Solution – Set of Patterns for the requirements:
• Message
– Document Message or Event Message Transformation
– Document message would be best fit
• Channel
– Point-to-Point Channel
Endpoint
• Endpoint
– Messaging Gateway
System Management
Second Requirement

Message

Channel

Router
Integration Solution – Set of Patterns for the requirements:
• Message
– Document Message
Transformation
• Channel
– Point-to-Point Channel
– Channel Adapter
Endpoint

System Management
Third Requirement

Message

Channel

Router
Integration Solution – Set of Patterns for the requirements:
• Message
– Document Message Transformation
• Channel
– Point-to-Point Channel
– Channel Adapter
Endpoint

System Management
Fourth Requirement

Message

Channel

Router

Transformation

Integration Solution – Set of Patterns for the requirements:

• Message
Endpoint
– Document Message
• Channel
– Datatype Channel System Management
Develop Integration Solutions for Process Orders