Module 7
Module 7
INTEGRATION AND
ARCHITECTURE
This chapter sets the stage for application integration concepts and introduces
ways to view application integration solution patterns. The reader should focus
on the concepts rather than the details. More technical details will come later in
this lecture.
1
IT 322 | Advanced Systems Integration and Architecture
Prepared by: Atienza, John Robert | Mendoza, Arjonel
MODULE 7: APPROACHING APPLICATION INTEGRATION
Topical Overview:
1. Moving from Information-Oriented to Service-Oriented Application
Integration
2. Application Integration Approaches
3. Application Service
2
IT 322 | Advanced Systems Integration and Architecture
Prepared by:
Application integration is a strategic approach to binding many information
systems together, at both the service and information levels, supporting their
ability to exchange information and leverage processes in real time. While this
sounds like a pure technology play, the resulting information and process flow
between internal and external systems provides enterprises with a clear strategic
business advantage: the ability to do business in real time, in an event-driven
atmosphere, and with reduced latency. The business value of this is apparent
Keep in mind that the application integration concept is nothing new. We've been
dealing with mechanisms to connect applications together since we've had more
than two business systems and a network to run between them.
3
IT 322 | Advanced Systems Integration and Architecture
Prepared by:
Moving to Real-Time
Business Integration: An Example
We need to think more comprehensively about how we capture and react to events.
We need to recognize that all components of the integrated enterprise, or extended
enterprise, affect the supply chain itself. For example, when a customer walks into
our car dealership and orders a car, or when the customer orders a car via the
Internet, that action is, in and of itself, a business event that is captured. Our system
must react to this event by performing several tasks instantaneously: logging the
event, processing the rules bound to such an event, and moving information to other
interested systems or humans.
The event must be logged so that it won't be forgotten should there be a failure as
it is being processed. We need to process rules bound to the event, such as price
limits and credit requirements. The internal (e.g., inventory) systems and external
(supplier) systems must be informed of the event. Finally, the information created
by this event, in this example, customer and car configuration information, must
move forward to the appropriate systems. Typically, this should be a second, sub-
process.
What is of note here is that all relevant systems are notified of the event and are
supplied with all appropriate information, in real time, so that they can, in turn,
instantly react to the event. In our car purchase example above, the sales event
captured by our manufacturer's system generates an instant requirement for parts
to create the car. In turn, this information triggers a cascading series of additional
events within systems owned by the suppliers, events such as notifying a supplier
of the raw materials required to build the parts. A single, primary event could thus
trigger as many as several hundred other events, which, in turn, could trigger
several thousand more events. It is exactly this chain reaction of event that serve a
business need that we hope to create.
4
IT 322 | Advanced Systems Integration and Architecture
Prepared by:
Remember, this real-time application integration scenario is an instantaneous
process. Within seconds of the initial order, the suppliers are processing requests
for the raw materials, the factory floor is scheduling workers, and the logistic group
is assigning resources in order to ship a car to a particular dealer. There may be
hundreds of systems involved with the sale, creation, and movement of this car,
all exchanging event information simultaneously. Of equal relevance is that all
systems participating in the event will be notified instantly should there be any
change along the supply chain. That is, if demand changes (e.g., if car sales are
down) or if there is a parts shortage. Instantaneous notification is a two-way
street, from orders to suppliers, from suppliers to orders.
We are seeing that, as the problem grows, so do the potential benefits of the
solution. The technology continues to respond to a perceived need. In this context,
our pursuit of application integration is like chasing the tail of a growing beast. For
now, that "beast" has remained ahead of us. A great deal of work remains ahead
of us. But rest assured, a solution will be found and the once-unimaginable benefits
of application integration will become an everyday reality.
5
IT 322 | Advanced Systems Integration and Architecture
Prepared by:
Moving from
Information-Oriented to
Service-Oriented Application Integration
The uses for this type of integration are endless, including the creation of
composite applications, or applications that aggregate the processes and
information of many applications. For example, using this paradigm, application
developers simply need to create the interface and add the application services by
binding the interface to as many Internet-connected application services as are
required.
6
IT 322 | Advanced Systems Integration and Architecture
Prepared by:
The downside, at least with service-based integration, is that this makes it
necessary to change the source and target applications or, worse in a number of
instances, to create a new application (a composite application). This has the effect
of adding cost to the application integration project and is the reason many choose
to stay at the information level.
Still, the upside of this approach is that it is consistent with the "baby step"
approach most enterprises find comfortable when implementing solutions to
integration problems. Service-based solutions tend to be created in a series of
small, lower-risk steps. This type of implementation can be successful from the
department to the enterprise to the trading community, but never the other way
around from the trading community to the department.
7
IT 322 | Advanced Systems Integration and Architecture
Prepared by:
Information-Oriented
Data Replication
Data replication is simply moving data between two or more databases. These
databases can come from the same vendor, or from many vendors (see Figure
1.2). They can even be databases that employ different models. The fundamental
requirement of database replication is that it accounts for the differences between
database models and database schemas by providing the infrastructure to
exchange data. Solutions that provide for such infrastructures are plentiful and
inexpensive.
8
IT 322 | Advanced Systems Integration and Architecture
Prepared by:
Figure 1.2. Database
replication is the simple exchange of information between databases.
The advantages of database replication are simplicity and low cost. Database
replication is easy to implement, and the technology is cheap to purchase and
install. Unfortunately, these advantages are quickly lost if methods need to be
bound to the data, or if methods are shared along with the data. If these
requirements exist, service-based solutions must be considered.
Data Federation
9
IT 322 | Advanced Systems Integration and Architecture
Prepared by:
Figure 1.3. Data federation allows
many databases to appear as a
single database.
The advantage of using this software is that it can bind many different data types
into a unified model that supports information exchange.
10
IT 322 | Advanced Systems Integration and Architecture
Prepared by:
Interface Processing
The efficient integration of many different types of applications defines the primary
advantage of using application integration-oriented products. In just days, it is
possible to connect an SAP R/3 application to an Oracle application, with the
application interface processing solution accounting for differences between
schema, content, and application semantics by translating on the fly the
information moving between the systems.
The efficient integration of many different types of applications defines the primary
advantage of using application integration-oriented products. In just days, it is
possible to connect an SAP R/3 application to an Oracle application, with the
application interface processing solution accounting for differences between
schema, content, and application semantics by translating on the fly the
information moving between the systems.
11
IT 322 | Advanced Systems Integration and Architecture
Prepared by:
The downside to using application interface-oriented products is that there is little
regard for business logic and methods within the source or target systems logic
and methods that may be relevant to a particular integration effort. In such a case,
service-based solutions probably make the better choice. Ultimately, application
interface processing technology will learn to share methods as well as information,
perhaps by joining forces with service-based approaches.
12
IT 322 | Advanced Systems Integration and Architecture
Prepared by:
Business process integration (BPI) is the science and mechanism of managing the
movement of data, and the invocation of processes in the correct and proper order
to support the management and execution of common processes that exist in and
between applications. Business Process Integration-Oriented Application
Integration (BPIOAI) provides another layer of easily defined and centrally
managed processes that exist on top of an existing set of processes and data
contained within a set of applications.
13
IT 322 | Advanced Systems Integration and Architecture
Prepared by:
Thus, business process integration technology must be flexible, providing a
translation layer between the source and target systems, and the business process
integration engine.
There are many differences between more traditional application integration and
business process integration.
a. A single instance of business process integration typically spans many
instances of traditional application integration.
b. Application integration typically means the exchange of information between
two or more systems without visibility into internal processes.
c. Business process integration leads with a process model and moves
information between applications in support of that model.
d. Application integration is typically a tactical solution, motivated by the
requirement for two or more applications to communicate.
e. Business process integration is strategic, leveraging business rules to
determine how systems should interact and better leverage the business
value from each system through a common abstract business model.
While some may question the relevance of Business Process Integration Oriented
Application Integration, and even of application integration itself, I would argue
that BPIOAI is the ultimate destination of application integration (acknowledging
that we still have a long way to go to perfect the middleware). Despite current
shortcomings, many application integration vendors are aggressively promoting
BPIOAI as a vital component of their application integration technology package.
In doing so, their strategy is clear they are anxious to join the world of high-end,
BPIOAI modeling tools. They hope that their application integration-enabled
middleware, such as integration servers and application servers, will accomplish
just that.
14
IT 322 | Advanced Systems Integration and Architecture
Prepared by:
BPIOAI is best defined as applying appropriate rules, in an agreed-upon logical
sequence, in order to pass information between participating systems, as well as
visualize and share application-level processes, including the creation of a common
abstract process that spans both internal and external systems. This definition
holds true regardless of whether the business processes are automated or not. For
example, processing an insurance claim and delivering a car to a customer are
business events that can be automated with BPIOAI.
To this end, there are three main services that business process integration
provides:
• the visualization of processes contained within all trading partner systems
• interface abstraction,
• and the real-time measurement of business process performance.
15
IT 322 | Advanced Systems Integration and Architecture
Prepared by:
BPI by example:
Note: See the file that I’ve uploaded on our Facebook group.
Another way to view the process of creating a business process integration model
is defining the hierarchy of processes within the trading community. This means
that smaller subprocesses can be linked at the lower tier of integration or are
native to the source or target systems. Building up from the lower-level processes
to the higher-level processes, you may link the subprocesses into higher-level
processes within the domain of the trading community.
Moreover, business process integration enables the technology user to track and
direct each instance of a business process; for example, processing individual
orders or medical insurance claims through a life cycle that may consume seconds,
minutes, hours, days, or weeks. Finally, we need to measure and maintain
contextual information for the duration of a process
instance that spans many individual activities.
In general, business process integration logic addresses only process flow and
integration. It is not a traditional programming logic, such as processing a user
interface, updating a database, or executing a transaction. Indeed, in most BPIOAI
scenarios, the process logic is separated from the application logic. It functions
solely to coordinate, or manage, the information flow between many source and
target applications that exist within organizations.
16
IT 322 | Advanced Systems Integration and Architecture
Prepared by:
Service-Oriented
17
IT 322 | Advanced Systems Integration and Architecture
Prepared by
Attempts to share common processes have a long history, one that began more
than ten years ago with the multitiered client/server set of shared services on a
common server that provided the enterprise with the infrastructure for reuse and,
now, for integration and the distributed object movement. "Reusability" is a
valuable objective. A common set of methods among enterprise applications
invites reusability and, as a result, significantly reduces the need for redundant
methods and/or applications.
While most methods exist for single-organization use, we are learning that there
are times when it makes sense to share between organizations. In a new twist on
the longstanding practice of reusability, we are now hoping to expand this sharing
beyond intra-enterprise to trading partners, as well; for example, sharing a
common logic to process credit requests from customers or to calculate shipping
costs using a set of Web services.
Sounds great, doesn't it? The downside might give you pause, however. This
"great-sounding" application integration solution also confronts us with the most
invasive level of application integration, thus the costliest. This is no small matter
if you're considering Web services, distributed objects, or transactional
frameworks.
.
18
IT 322 | Advanced Systems Integration and Architecture
Prepared by:
While IOAI generally does not require changes to either the source or target
applications, SOAI requires that most, if not all, enterprise applications be changed
in order to take advantage of the paradigm. Clearly, this downside makes SOAI a
tough sell. However, it is applicable in many problem domains. You just need to
make sure you leverage SOAI only when you need it.
Before embracing the invasiveness and expense of SOAI, enterprises must clearly
understand both its opportunities and its risks. Only then can its value be evaluated
objectively. The opportunity to share application services that are common to
many applications and therefore making it possible to integrate those applications
represents a tremendous benefit. However, that benefit comes with the very real
risk that the expense of implementing SOAI will outpace its value.
Good question. Here's a better question: How does an application service differ
from information integration?
When using an application service, we leverage a remote method or behavior
versus simply extracting or publishing information to a remote system. Moreover,
we typically abstract this remote service into another application known as a
composite application (see Figure 1.7).
19
IT 322 | Advanced Systems Integration and Architecture
Prepared by:
A good example of an application service is a risk analysis process, which runs
within an enterprise to calculate the risk of a financial transaction. This remote
application service is of little use by itself, but when abstracted into a larger
application for example, a trading system then that remote application service has
additional value. Note that we leverage the behavior of this remote service more
than the information it produces or consumes. If you're a programmer, you can
view application services as subroutines or methods; something you invoke to
make something happen.
The basic notion of SOAI is to leverage these remote services using some
controlled infrastructure that allows applications to invoke remote application
services as if they were local to the application. The result (or goal) is a composite
application made up of many local and remote application services. Think about
the possibilities.
20
IT 322 | Advanced Systems Integration and Architecture
Prepared by:
Application integration gives us the tools and techniques to learn how to share
common application services. More than that, these tools and techniques create
the infrastructure that can make such sharing a reality. By taking advantage of
this opportunity, we can integrate applications to share information, even as we
provide the infrastructure for the reuse of business logic.
Fact: As the number of changes to source or target applications increases,
so do the costs.
Although IOAI generally does not require changes to source or target applications,
SOAI does require changes to most if not all enterprise and B2B applications in
order to take advantage of the paradigm. This downside makes the service-
oriented approach a tough sell between enterprises. Web services promise to
change this, putting everyone on the same technology standards, so to speak, but
there are still some changes that inevitably have to occur within source and target
systems in support of Web services. In other words, most systems that we desire
to leverage using SOAI are existing systems, built prior to the arrival of Web
services (or other SOAI technology, for that matter). Those systems will have to
be changed or rebuilt from scratch.
By now, some of you may be a bit confused about SOAI and its use when
establishing B2B connections between two or more organizations. Although many
businesses are looking to exchange information with enterprises, and even to
participate in shared processes, few are looking to create applications that share
application services with systems not under their control.
21
IT 322 | Advanced Systems Integration and Architecture
Prepared by:
A clear example of the potential benefit of SOAI is the simple binding of two or
more applications in order to integrate both business processes and data. Let us
assume that two applications exist within a given enterprise. One application is
C++ based and runs on a Linux machine. The other is an NT-based client/server
application written in Java on the front end, with Sybase serving as the back-end
database.
Unlike other application integration levels, at the service level, the architect has
no option but to rebuild the applications to support SOAI. The architect has only
two choices in determining how to accomplish this. One, she can move much of
the business logic to a shared server, such as an application server. Two, she can
rebuild each application using an application service-sharing mechanism, such as
distributed object technology, to create a tightly coupled application that allows
easy cross-accessing of application services.
If the architect decides the second choice is the most attractive, she will have to
"wrap" the application logic encapsulated inside both applications. To accomplish
this, she will use a distributed object technology (CORBA, COM+, or perhaps Web
services) that provides a common mechanism to share application services
remotely. This will require rewriting the applications and then testing them.
Fortunately, this task is not as daunting as it might appear. Tools exist for each
environment to wrap each application, re-creating the applications as truly
distributed systems able to share both application services and data. Even with
such tools, the process is laborious. For example, if both applications need to add
a common customer to their systems, they may invoke different application
services; for example:
Add_Cust();
on the Linux/C++ system and:
AddNewCustomer();
on the NT/Java system
22
IT 322 | Advanced Systems Integration and Architecture
Prepared by:
By using a distributed object standard or a custom programming solution, the
architect could expose each application service. As a result, she could bind the
application services, or invoke one or the other. Once the application services are
bound, the applications move to a coupled state where application services and
data are easily shared within both domains, thus solving the application integration
problem.
Portal-Oriented
23
IT 322 | Advanced Systems Integration and Architecture
Prepared by:
While the other types of application integration are focused on the real-time
exchange of information (or adherence to a common process model) between
systems and companies, POAI is concerned with externalizing information out of a
multitude of enterprise systems to a single application and interface. That's clearly
an approach that goes against the notions of the other types of application
integration, which are more real-time- and event-driven-oriented, and the
inclusion in this book of POAI was somewhat of a judgment call.
POAI by Example
An example of POAI is an automobile parts supplier that would like to begin selling
parts to retail stores (B2B) using a portal. This portal would allow the retail stores
to access catalog information, place orders, and track orders over the Web.
Currently, the parts supplier leverages SAP as its preferred inventory control
system, and a custom-built mainframe application written in COBOL/DB2 serves
as its sales order system. Information from each system is required for the B2B
portal, and the portal users need to update those backend systems as well.
In order to create a portal, the parts supplier must first design the portal
application, including the user interface and application behavior, as well as
determine which information contained within the back-end systems (SAP and the
mainframe) needs to be shared with the portal application. The portal application
requires a traditional analysis-and-design life cycle and a local database. This
portal application must be able to control user interaction, capturing and
processing errors and controlling the transaction from the user interface all the
way to the back-end systems.
24
IT 322 | Advanced Systems Integration and Architecture
Prepared by:
Although you can employ many types of enabling technologies when creating
portals, most portals are built using application servers (discussed in detail later
in this book). Application servers provide the interface development environments
(IDEs) for designing the user interface, a programming environment to define
application behavior, and back-end connectors to move information in and out of
back-end systems, including SAP and mainframe systems. Although not
integrating the application directly, the portal externalizes the information to the
trading partner in this case, the owner of a retail auto parts store and also updates
the back-end systems in this case, with orders placed by the store owner or
perhaps with the status of existing orders.
Other examples of portals include entire enterprises that are integrated with a
single portal application. As many as a dozen companies may provide real-time
information for a portal, and hundreds of companies may use that portal, B2B, to
purchase goods and services from many companies at the same time. The same
type of architecture and enabling technology applies in this case; however, the
number of systems integrated with the portal application greatly increases.
Portal Power
The use of portals to integrate enterprises has many advantages. The primary one
is that there is no need to integrate back-end systems directly between companies
or within enterprises, which eliminates the associated cost or risk. What's more,
you usually don't have to worry about circumventing firewalls or application-to-
application security, because portals typically do nothing more than Web-enable
existing systems from a single enterprise. With portals, you simply connect to each
back-end system through a point of integration (user interface, database,
application server, etc.) and externalize the information into a common user
interface (Web browser). Of course, portals themselves are applications and must
be designed, built, and tested like any other enterprise application.
POAI also provides a good facility for Web-enabling existing enterprise systems for
any purpose, including B2B and Business-to-Consumer (B2C) selling over the Web.
If you need to move information to a user interface for any reason, this is the best
approach.
25
IT 322 | Advanced Systems Integration and Architecture
Prepared by:
In many application integration problem domains, the users prefer to interact with
the back-end systems through a user interface rather than have the systems
automatically exchange information behind the scenes (as in data-oriented
application integration). Today, more B2B information flows through user
interfaces (POAI) than automatically through back-end integration. However, the
trend is moving from portals to real-time information exchange, which is the topic
of this book. We will eventually remove the end user who is the most obvious point
of latency when considering POAI from the equation.
The advantages of POAI are clear:
1. Information does not flow in real time and so requires human interaction. As
a result, systems do not automatically react to business events within an
enterprise (such as the depletion of inventory).
2. Information must be abstracted, most typically, through another application
logic layer (e.g., an application server). As a result, some portal-oriented
solutions actually add complexity to the solution.
3. Security is a significant concern when enterprise data is being extended to
users over the Web.
Web-Enabled World
26
IT 322 | Advanced Systems Integration and Architecture
Prepared by:
The notion of POAI has gone through many generations, including single system
portals, multiple-enterprise-system portals, and now, enterprise portals (also
known as digital exchanges).
Single-System Portals
Single-system portals, as you might expect, are single enterprise systems that
have their user interfaces extended to the Web (see Figure 1.9).
Multiple-Enterprise-System Portals
27
IT 322 | Advanced Systems Integration and Architecture
Prepared by:
This type of portal represents a classic application server architecture, where
information is funneled from several enterprise systems such as SAP R/3,
mainframes, PeopleSoft, and inventory systems through a single Web-enabled
application. Users are able to extract information from these systems and update
them through a single Web browser interface accessed over an extranet or over
the Web.
Trading Community
28
IT 322 | Advanced Systems Integration and Architecture
Prepared by:
Application servers are a good choice for enterprises, funneling information from
the connected back-end enterprise systems. However, because hundreds of
systems could be connected to this type of portal, it sometimes makes sense to
leverage application servers within each enterprise to manage the externalization
of information flowing out of the enterprise, then funnel that information through
a single master application server and Web server. The result of this structure is
the information found in hundreds of systems spread across an enterprise,
available to anyone who uses the portal. This is an extremely attractive proposition
29
IT 322 | Advanced Systems Integration and Architecture
Prepared by: