Tushar Jain Enterprise SOA Reference Architecture

[email protected]
http://www. architecture-soa-bpm-eai.blogspot.com

Enterprise SOA Reference Architecture

Tushar Jain
http://www. architecture-soa-bpm-eai.blogspot.com

Tushar Jain Enterprise SOA Reference Architecture

[email protected] 1
http://www. architecture-soa-bpm-eai.blogspot.com
 Tushar Jain Enterprise SOA Reference Architecture
[email protected]
http://www. architecture-soa-bpm-eai.blogspot.com

Contents

Enterprise SOA Reference Architecture........................................................................................................3

Data Persistent Layer.....................................................................................................................................3
Service Providers...........................................................................................................................................4
Services Layer................................................................................................................................................4
Enterprise Service Bus...................................................................................................................................5
Presentation Services.....................................................................................................................................5
Human .......................................................................................................................................................5
Machine .....................................................................................................................................................6
Integration with Outer World.........................................................................................................................6
Security..........................................................................................................................................................6
Governance....................................................................................................................................................7
Rule/Policy Repository & Engine..................................................................................................................9
Service Directory...........................................................................................................................................9
Event Management........................................................................................................................................9
Generic Event Handling.............................................................................................................................9
Time Based Event Handling....................................................................................................................11
Heart Beat Event Handling......................................................................................................................11
Transaction Management.............................................................................................................................12
Exception & Error Management......................................................................................................................15
Relationship.....................................................................................................................................................18

Tushar Jain Enterprise SOA Reference Architecture

[email protected] 2
http://www. architecture-soa-bpm-eai.blogspot.com
 Tushar Jain Enterprise SOA Reference Architecture
[email protected]
http://www. architecture-soa-bpm-eai.blogspot.com

Enterprise SOA Reference Architecture

Here I propose SOA Enterprise Reference Architecture. This is for enterprise and
certainly for an application it should be inspiration.

Reference Architecture is independent of platforms offered by various vendors. It is

also possible that none of the vendors in market provide complete solution to realize
this reference architecture.

Data Persistent Layer

Data Persistent Layer is primarily consists of Enterprise Data - Master, Transaction

and Meta. It may also include Datamart and Data driven reporting infrastructure.

Tushar Jain Enterprise SOA Reference Architecture

[email protected] 3
http://www. architecture-soa-bpm-eai.blogspot.com
 Tushar Jain Enterprise SOA Reference Architecture
[email protected]
http://www. architecture-soa-bpm-eai.blogspot.com
In most of the large enterprises Data Persistent Layer consists of relational (RDBMS),
hierarchical (LDAP, IMS), unstructured (Emails, chat sessions, file systems, etc) and
any other type. These variations in data characteristics warrant unique data handling
and treatment technologies.

Service Providers

This Layer consists of different business applications. These applications may be

packaged (e.g. SAP, Reteck, etc) or may be custom developed in various technologies
(e.g. Java, .Net, etc). Again application in this layer may be service enabled or legacy
once, which do not expose their functionalities as service.

At this layer EAI exist. Some may argue that SOA is one form of integration then how
EAI can exist at this layer. I will discuss this argument in one of my future posting but
not now.
Services Layer

This layer is consists of multiple sub layers which encapsulate different class of
services. It encompasses Connectivity Services, Data Services, Business Activity
Services and Business Process Services. Each layer in turn consists of other sub-types
of services.

This layer exposes different aspects of business - functional and infrastructural

aspects. Connectivity and Data services are consisting of Infrastructural services
while Business Activity and Process services expose functional Services.

These services though in layers but still breach layered architecture to expose these
services independent of layers and for better performance. These services can also
be exposed to multiple ESBs if situation warrants.

1. Connectivity services primarily cater to accessing system level functionalities, any

adapter & connector services and partner (B2B) integration services.

2. Data services comprises of services, which caters to Business, Technical and Meta
data related services.

3. Business Activity Services consist of Atomic business/functionalities and any

custom business logic developed by combing one or more Data & Connectivity
services.

4. Business Process Services primarily consist of Process integration, Machine and

Human centric services. These services primarily comprise of Business Activity
services.
Tushar Jain Enterprise SOA Reference Architecture
[email protected] 4
http://www. architecture-soa-bpm-eai.blogspot.com
 Tushar Jain Enterprise SOA Reference Architecture
[email protected]
http://www. architecture-soa-bpm-eai.blogspot.com

Enterprise Service Bus

ESB provides four functionalities:

Orchestration
Routing
Mediation
Enrichment

Some ESBs may provide one more functionality - Choreography. This functionality is
associated with BPM systems and currently none of Commercially available ESB
provides the same.

Enrichment may be of two types: Data Transformation and Channel Transformation.

Most of the analysts & architects keep Data Transformation & Channel Transformation
separate functionality.

Presentation Services
These services support various user interfaces (human or machine centric) and let
outer world (intra & inter enterprise) interact with enterprises IT assets. The
consumers of presentation services are

Human
Thin Client

Tushar Jain Enterprise SOA Reference Architecture

[email protected] 5
http://www. architecture-soa-bpm-eai.blogspot.com
 Tushar Jain Enterprise SOA Reference Architecture
[email protected]
http://www. architecture-soa-bpm-eai.blogspot.com
Browser clients: May be based on traditional HTML based or AJAX
based

Portable devices: May be based on open standards (WAP) or

propitiatory (device or carrier specific)

Thick Client

Alway Connected: These clients connect to service providers and

consume the services in real time. Traditional think client falls in this category

Offline Clients: These clients do not connect to service providers in

Always On fashion but an actor can work in offline mode and when ever connectivity
available, services consumed. Java WebStart and Adobe AIR are two most widely
used technologies in this field.

Voice Based Client

Voice based clients are gaining popularly to reduce load on contact center.
To facilitate this type of user interface, IVR plays great role.

Machine
These types of interfaces are not very obvious but very prevent at enterprise
level. Application level interaction (Software to software) and interaction with various
devices like RFID tags, Bar code readers, etc (Hardware to Software or vice versa)
are part of this type of interfaces.

Integration with Outer World

Most of the large enterprises also get integrated with external world. This integration
is both ways - information flow is in both directions. To further complicate the
scenario, enterprises increasingly using SAAS. This situation requires special attention
and tools. In case of integration with business partners B2B Engine plays major role
in conjunction with Partner services. SAAS services may be consumed directly or via
B2B engine. In case of Human interaction (CRM - Salesforce.com) B2B Engine should
be bypassed but for m/c interaction or substantial data transfer (Document
management for insurance companies) B2B engine should be engaged.

Security

Tushar Jain Enterprise SOA Reference Architecture

[email protected] 6
http://www. architecture-soa-bpm-eai.blogspot.com
 Tushar Jain Enterprise SOA Reference Architecture
[email protected]
http://www. architecture-soa-bpm-eai.blogspot.com

Security in SOA is one of the biggest concern due to distributed & heterogeneous
systems interacting with each other. The security related aspects should be handled
at each level of enterprise. In the proposed model security is considered from
physical and virtual perspective. It also takes into account security from network and
application view.

At monitoring front it take care of Technical - For support staff, Process & Managerial
- For business process owners & managers and Executive - for top rung of business
leadership requirements.

Governance

Governance is one of the burning issues in SOA environment. To discuss SOA

Governance's Technical architecture, it is necessary to understand SOA Governance
model.

Following figure depicts the SOA Governance Model.

Tushar Jain Enterprise SOA Reference Architecture

[email protected] 7
http://www. architecture-soa-bpm-eai.blogspot.com
 Tushar Jain Enterprise SOA Reference Architecture
[email protected]
http://www. architecture-soa-bpm-eai.blogspot.com

The picture below intends to depict SOA Governance Technical Architecture.

Service Life Cycle Management

Service Life Cycle Management essentially helps in managing service life cycle from
its conception to decommission. This is one of the vital components, which determine
the success of SOA initiative in any enterprise and also its long-term viability.

Tushar Jain Enterprise SOA Reference Architecture

[email protected] 8
http://www. architecture-soa-bpm-eai.blogspot.com
 Tushar Jain Enterprise SOA Reference Architecture
[email protected]
http://www. architecture-soa-bpm-eai.blogspot.com
Rule/Policy Repository & Engine
Rule/Policy Repository contains various Business Rules and Governance & Security
Policies. The engine part of the same will help in execution of Rules and Policies as
and when demanded.

Service Directory
Service directory essentially consists of UDDI, which facilitates consolidated listing of
services.

Event Management
To exploit SOA to its full extent event handling plays crucial role. In SOA environment
Events can be classified into three categories

1. Generic Event
Event fired/raised in response of event or interaction with other service/s.

2. Time Based Event

Event fired/raised at particular moment of time to perform certain task

3. Heart Beat Event

Event fired/raised by services at regular interval to announce its existence and may
be level of throughput it is working.

Generic Event Handling

These are events which fire/raise other events or results due to interaction of services
in ecosystem.

Tushar Jain Enterprise SOA Reference Architecture

[email protected] 9
http://www. architecture-soa-bpm-eai.blogspot.com
 Tushar Jain Enterprise SOA Reference Architecture
[email protected]
http://www. architecture-soa-bpm-eai.blogspot.com

1. Event Generator generates events with minimal data set.

2. These events should be formatted as per the event protocol defined in ecosystem,
so that theses event are available to entire ecosystem for further processing.

Ø Event Pre-processing: Check that mandatory attributes are present in the incoming
events
Ø Event Refinement: Introduce the other system-generated attributes like start time
etc.
Ø Situation Refinement: Check the target status on which action would be performed
Ø Impact assessment: Required result assessment after that impact applies on the
target

3. Event factory determine the specific type of action, which the given event have.

4. Event listener executes the specific action, which stated at the specific action code.
There we have some general action apply to all type of system is present in generic
action code and specific to specific action is present in the specific action listener

5. Event listener executes the action and calls the required system to complete it.

Tushar Jain Enterprise SOA Reference Architecture

[email protected] 10
http://www. architecture-soa-bpm-eai.blogspot.com
 Tushar Jain Enterprise SOA Reference Architecture
[email protected]
http://www. architecture-soa-bpm-eai.blogspot.com
6. Some event, which completed their actions, and no further support needed from
any other services are terminated

7. Some events which need further processing from some other services or may be
generate other event in response of event is send back to the event engine

8. Event in response of event is again run as a new event and again it initiates its
processing cycle in the event engine

Time Based Event Handling

Scheduled event delivered to a service at specific time in the future. Event deliveries
can be deferred for short periods of time (such as seconds or minutes) or for long
stretches of time (for example, hours later for batch processing). Until that delivery
time, the Event is essentially invisible until it is delivered, allowing you to schedule
work at a particular time in the future.

Heart Beat Event Handling

Heartbeats, a common software construct, verify the continual operation of a specific

service. With heartbeats, a targeted service continually broadcasts a signal across its
environment. System works normally when client services can detect a targeted
service's heartbeat signals

Tushar Jain Enterprise SOA Reference Architecture

[email protected] 11
http://www. architecture-soa-bpm-eai.blogspot.com
 Tushar Jain Enterprise SOA Reference Architecture
[email protected]
http://www. architecture-soa-bpm-eai.blogspot.com
There is number of interdependent services in SOA ecosystem such that, if one failed,
others will also fail.

Heartbeat generally propagated across SOA ecosystem using publish/subscribe

paradigm.

Transaction Management

Transaction Management is one of the most difficult aspects of any enterprise

architecture due to varying level of understanding among stakeholders, various
technological platforms involvement and spread of transactions – intra & inter
enterprise.

Tushar Jain Enterprise SOA Reference Architecture

[email protected] 12
http://www. architecture-soa-bpm-eai.blogspot.com
 Tushar Jain Enterprise SOA Reference Architecture
[email protected]
http://www. architecture-soa-bpm-eai.blogspot.com
In general any enterprise grade transaction framework follows ACID but due to
presence of long running business processes and involvement of asynchronous
communication traditional ACID becomes irrelevant. To accommodate long running
business processes and asynchronous communication relaxed ACID is used which
allows violation of ACID but with capability of correcting the violations in reasonable
time period and with little business impact.

Tushar Jain Enterprise SOA Reference Architecture

[email protected] 13
http://www. architecture-soa-bpm-eai.blogspot.com
 Tushar Jain Enterprise SOA Reference Architecture
[email protected]
http://www. architecture-soa-bpm-eai.blogspot.com

Tushar Jain Enterprise SOA Reference Architecture

[email protected] 14
http://www. architecture-soa-bpm-eai.blogspot.com
 Tushar Jain Enterprise SOA Reference Architecture
[email protected]
http://www. architecture-soa-bpm-eai.blogspot.com
SOA capable enterprise grade Transaction Framework should have following features:

· Multi level undo/redo

· Coalescing of transactions
· Automatic aggregation of nested transactions
· Supports batching of transactions (Aggregation)
· Rollback of aggregated transactions for error recovery
· Listener support
· Synchronous and Asynchronous Transaction
· Compensatory Service
· Transaction Propagation

Exception & Error Management

Exception & Error Management is consist of

Exception & Error Handling
Exception & Error Logging
Exception & Error Notification
In any enterprise under SOA ecosystem Exception & Error Management poses
following challenges:

Exception & Error Handling

No predefined format for passing exception information among participating services

Web services do not have capability of maintaining stack trace
Lack of common exception vocabulary across enterprise

Exception & Error Logging

Scattered logs across services/applications

Varying formats of logs across services/applications
Toggling logs at run time is service/application dependent

Exception & Error Notification

Every enterprise/LOB has its own unique notification requirement

Toggling Notification at run time is necessity
SLA and exceptions are coupled
To counter these challenges there must be a comprehensive Exception and Error
Management framework which can manage at enterprise level and form the
foundation for Remediation framework.

Tushar Jain Enterprise SOA Reference Architecture

[email protected] 15
http://www. architecture-soa-bpm-eai.blogspot.com
 Tushar Jain Enterprise SOA Reference Architecture
[email protected]
http://www. architecture-soa-bpm-eai.blogspot.com
On the high level, Exception and Error Management framework’s architecture can be
depicted as follows:

Tushar Jain Enterprise SOA Reference Architecture

[email protected] 16
http://www. architecture-soa-bpm-eai.blogspot.com
 Tushar Jain Enterprise SOA Reference Architecture
[email protected]
http://www. architecture-soa-bpm-eai.blogspot.com

Remediation Framework

Remediation framework plays very important role in any SOA ecosystem. It can be
though of extension of Error & Exception Management and Transaction Management
frameworks. Remediation framework leverages Error & Exception Management
Framework by utilizing logged error & exceptions. It also supplement Transaction
Management Framework via helping to implement relaxed ACID where human
intervention might be needed. Remediation framework also makes whole of SOA
ecosystem robust via utilizing compensatory services.

Tushar Jain Enterprise SOA Reference Architecture

[email protected] 17
http://www. architecture-soa-bpm-eai.blogspot.com
 Tushar Jain Enterprise SOA Reference Architecture
[email protected]
http://www. architecture-soa-bpm-eai.blogspot.com

Relationship

In SOA ecosystem Security, Governance, Operations, Quality are closely related. Each
face of SOA ecosystem affects other.

Tushar Jain Enterprise SOA Reference Architecture

[email protected] 18
http://www. architecture-soa-bpm-eai.blogspot.com
 Tushar Jain Enterprise SOA Reference Architecture
[email protected]
http://www. architecture-soa-bpm-eai.blogspot.com

Tushar Jain Enterprise SOA Reference Architecture

[email protected] 19
http://www. architecture-soa-bpm-eai.blogspot.com
 Tushar Jain Enterprise SOA Reference Architecture
[email protected]
http://www. architecture-soa-bpm-eai.blogspot.com

Tushar Jain Enterprise SOA Reference Architecture

[email protected] 20
http://www. architecture-soa-bpm-eai.blogspot.com