Implementing Oracle Integration Cloud Service

Table of Contents

Implementing Oracle Integration Cloud Service

Credits

Foreword

About the Authors

Acknowledgement from both the authors

About the Reviewer

www.PacktPub.com

Why subscribe?

Customer Feedback

Preface

What this book covers

How we have approached this book

What you need for this book

Introduction to apiary

Introducing Mockable

Creating an instance of Integration Cloud Service

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Downloading the color images of this book

Errata

Piracy

Questions

1. Introducing the Concepts and Terminology

Typical workflow and steps to execute

Connections define our integration points

SaaS adapters

What's the difference with native APIs?

Technology adapters

SOAP adapter

REST adapter

Resource examples

FTP adapter

On-premises adapters

What is the Connectivity Agent?

Architecture Guidelines

Architecture

Message Exchange Patterns

What is the Execution Agent?

Noticeable differences

Restrictions between connectivity and execution agent

Social and productivity adapters

Integrations

Point-to-point integration

Publish-subscribe integration

Topology

Advantages of pub-sub

Scalability

Loosely coupled

Disadvantages of pub-sub

Inflexibility of decoupling

Issues with message delivery

Content-based routing

Topology

Simple example architecture

Advantages of content-based routing

Very efficient

Sophisticated routing decisions

Disadvantages of content-based routing

Additional consumers

Sequential processing

Orchestration for complex integrations

Topology

Practical example

Advantages of Orchestration

Loosely coupled

Enables automation

Disadvantages of Orchestration

Transformation and lookups

Transformations

XQuery

XSLT

XPath expressions

XPath Examples

XLST constructs

Lookups

Summary

2. Integrating Our First Two Applications

Getting ready

Setting up apiary

Time to switch to integration cloud

Define the necessary connections

Inbound SOAP connection

Outbound REST connection

Troubleshooting

Integrate the two applications

Map message data

Completing and activating the integration

Testing the Integration

Invoke SOAP endpoint using SoapUI

Providing a valid WS-Security header

Invoking the integration for the first time

Monitoring the instance flow

Verifying the message processed by the backend

Summary

3. Distribute Messages Using the Pub-Sub Model

Getting ready

Setting up Mockable

Setting up SoapUI

Stage 1 - creating the direct connection integration

Defining the connections in ICS

Testing the integration

Stage 2 - reworking to use the pub-sub approach

Defining the publication service

Defining the subscription service

Running the test again

Stage 3 - adding a second subscriber

Setting up a second Mockable.io service

Setting up the subscriber integration

Running the test with two subscribers

Enhanced techniques

Multiple sources

Canonical data model pattern and normalizers

Summary

4. Integrations between SaaS Applications

Getting ready

Setting up Salesforce

Step 1 - get access to a Salesforce instance

Step 2 - generate the Enterprise service definition

Step 3 - obtain or reset your security token

Step 4 - create workflow rule for escalated cases

Step 5 - define outbound message and generate message WSDL

Setting up Twilio

Step 1 - get access to a Twilio account

Step 2 - create phone number for sending messages

Step 3 - add verified callers IDs

Step 4 - obtain live and/or test API credentials

Define the necessary connections

Troubleshooting

Integrate the SaaS applications

Map message data

Map Salesforce notifications to Twilio requests

Map Twilio's response to Salesforce's case comment

Map Twilio's fault to Salesforce's feed comment

Completing and activating the integration

Testing the integration

Troubleshooting

Next steps

Summary

5. Going Social with Twitter and Google

Tweet changes in flight schedules

Getting ready

Setting up Twitter

Step 1 - get access to a Twitter account

Step 2 - register a new application

Step 3 - create access token and obtain API credentials

Inbound WSDL with multiple operations

Define the necessary connections

Tweet when a flight schedule has changed

Map message data

Map FlightScheduleUpdate to Twitter's request

Map Twitter's response to ScheduleUpdateResult

Mapping Twitter's fault to BusinessFault

Completing and activating the integration

Testing the integration

Invoke cloud endpoint using SoapUI

Troubleshooting

Twitter rejects duplicate messages

No valid authentication token

No rights to write a tweet on the timeline

Send missing person report by e-mail

Getting ready

Setting up Google

Step 1 - get access to a Google account

Step 2 - register a new project and enable API

Step 3 - create access token and obtain API credentials

Define the necessary connections

E-mail the front desk to report a missing person

Map message data

Map MissingPersonReport to Google's sendMsg request

Map Google's response to MissingPersonResult

Map Twitter's fault to BusinessFault

Completing and activating the integration

Testing the integration

Invoking the Cloud endpoint using SoapUI

Troubleshooting

No valid client ID found

Use of Gmail API is disabled in Google's API Manager

Wrong scope defined for chosen operation

Summary

6. Creating Complex Transformations

Using variables to enrich messages

Preparation

Client connection

Creating the enriched mapping

Setting up SoapUI and Mockable

Mockable

SoapUI

Enrichment services

Preparation

Creating the connector

Creating the integration

Testing the enriched integration

Using lookups

Creating a lookup

Alternate lookup creation

Incorporating a lookup into a mapping

How to get function documentation

Executing the integration with a lookup

Summary

7. Routing and Filtering

Preparation

Creating connections

Creating a filtered integration

Applying a filter

Setting up Mockable

Testing the integration

Routing by Message Content

Creating the Routing Integration

Mapping the alternate route

Testing the routing

Extending the filter integration to use a REST source

Creating the trigger REST connection

Cloning the filter integration

Changing the invoke connector

Parameters

HTTP Headers and CORS

Configuring the Request

Reapplying mappings

Defining a multipart filter

Running the REST filter service

Summary

8. Publish and Subscribe with External Applications

Preparation

Overview of our Java application

Configuring ready to go

OMCS connection

Creating the integration

Configuring Trigger and Invoke

Connecting endpoints and tracking

Testing the integration

Summary

9. Managed File Transfer with Scheduling

Differences between File and FTP connectors

Scenario

Prerequisites

Setting up FTP locations

Creating the FTP connector

Creating the FTP to FTP integration

How to describe FTP file structure

Scheduling the integration

Using encryption

Common FTP use cases with Orchestration

Extra steps to define structure for file content

Calculated filenames

FTP integrations with interesting behaviors

Using FTP without a schema mapping in Orchestration

Implementing the single value mapping technique

Summary

10. Advanced Orchestration with Branching and Asynchronous Flows

Getting ready

Setting up Trello

Step 1 – Getting access to a Trello account

Step 2 – Creating a new board and tasks list

Step 3 – Obtaining a list identification

Step 4 – Obtaining API credentials

Updating the apiary Flight API

Step 1 – Log in to apiary and switch the API

Step 2 – Change the source of the API Blueprint

Defining the necessary connections

Checking if all connections are created

Building the orchestration

It uses a different UI and workflow

From simple to more advanced actions

Extracting data into simple variables

Branching the integration into multiple routes

Completing and activating the integration

Testing the orchestration

Invoke a cloud endpoint using SoapUI

Summary

11. Calling an On-Premises API

What kinds of agents exist?

When can an agent help?

Prerequisites and deploying an agent

Setting up and starting the VM

Checking everything is ready

Agent download

Creating the Agent Group

Installing the Connectivity Agent

Installing the Execution Agent

Differences between execution and connection agents

Upgrading your agent

Starting and stopping the Agent

Troubleshooting

Agent log files

The agent as a WebLogic container

Just start WebLogic

Sanity check the configuration

Building the integration

Preparation

Creating the integration

On-premises database connection

REST connection

Basic Map Data integration

Connecting endpoints and tracking

Testing the integration

Summary

12. Are My Integrations Running Fine, and What If They Are Not?

Core monitoring information

ICS - all is well

Dashboard

Inducing errors for integrations

Runtime health

Design time metrics

Looking at integrations

Agents view

Tracking view

Errors view

Advanced resolution

Things to check

Examining logs

Reporting incidents and downloading incidents information

Where to go for information about errors

System maintenance windows

Incorporating ICS monitoring into the enterprise

Administration

Certificate management

Changing log levels

E-mail reporting

Summary

13. Where Can I Go from Here?

Import and export

Import and export scenarios

Controlling change in a live environment

Configuration management

Common service deployment around the world

Pre-built integrations

Synchronizing lookup data

Complex editing and using developer tooling

Exporting and importing integrations

Individual export

Illustrating import and export

Individual import

Regeneration

Lookup export

Lookup import

Packaging

Package export and import

Alternate tools

ICS API

Getting and using cURL

Cloud adapters software development kit

Keeping up to date

Summary

Implementing Oracle Integration Cloud Service

Implementing Oracle Integration Cloud Service

Copyright © 2017 Packt

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: January 2017

Production reference: 1130117

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham 

B3 2PB, UK.

ISBN 978-1-78646-072-1

www.packtpub.com

Credits

Foreword

As the digital age embraces each of us, our businesses and the world around us, the importance of Software as a Service (SaaS), Cloud Integration, and API Management and is higher than ever before. Today, every business is not only expected to lead but also be able to pivot and flex to ride over disruptions or even disrupt age old industries themselves-be it banking, public sector, hospitality, music or any other, even the 300 year old taxi industry. Today businesses expect their IT to be agile and rapid enough to enable them to compete and lead in an environment where the next technology-led disruption is always just around the corner.

Enterprises, large, and small, are seeing rapid uptake of best of breed SaaS applications, often led not by the CIO or IT, but by the Line of Business (LOB) such as the HR manager, the sales VP or the marketing officer. SaaS bring unprecedented advantages of the cloud with rapid deployments, ease of use and huge savings in time and money. While SaaS undoubtedly brings these benefits, it doesn't take long for the LOB to realize the importance of their SaaS applications integrating with each other, with existing on-premises applications and with business processes across the enterprise – without this SaaS is nothing but silos as a service. This is where the rapid development and elevated user experience of the leading Integration Platform as a Service (iPaaS)-Oracle Integration Cloud Service (ICS) comes in.

In this book, Phil Wilkins and Robert van Mölken, take us through an exciting and insightful journey of how Oracle ICS solves exactly this crucial and urgent need that businesses face today – an iPaaS solution that complements your SaaS and on-premises application landscape allowing SaaS business users to rapidly build cloud and hybrid integrations, while allowing IT to have visibility of these integrations at design time and runtime, across the enterprise.

Phil and Robert, bring years of rich experience solving integration problems for market leading businesses across the globe. Through this book, they not only share their expertise of Oracle ICS through in-depth information on leveraging the service but also take us through an exciting journey weaving through various aspects of rapidly building cloud and on-premises integrations-simple as well as complex, multi-step orchestrations – leveraging tens of out of the box connectivity adapters and monitoring capabilities that Oracle ICS brings, for the SaaS user as well as IT. Phil and Robert, have not only shared when and how to use Oracle ICS for your business but have also articulated how to effectively solve modern business integration problems.

Your drive and expertise will always have the biggest impact on your IT and business, but I hope the knowledge you gain from this book of Oracle ICS enables you further to build the technical, architectural, and integration capabilities your business critically needs today, whether in the cloud, on-premises or both, to innovate and to future proof your business and IT.

Vikas Anand

Vice President, Product Management, Oracle Integration Platform Oracle Corporation

About the Authors

Robert van Mölken lives in Utrecht, the Netherlands, and is a Fusion Middleware specialist. He has over 9 years of experience in IT. Robert studied computer science at the University of Applied Sciences in Utrecht and received his BCS in 2007. Before his graduation, he started as a graphic designer and web developer, but soon shifted his focus to Fusion Middleware. His career started just before the release of Oracle SOA Suite 10gR3 and progressed heavily from there. Currently, Robert is one of the expertise leads on integration and cloud at AMIS. Before he started working at AMIS, he already had 4 years of experience in SOA Suite 10g and 11g. AMIS is specialized in most of the facets of the Oracle Red Stack and is an initiator of the Red Expert Alliance, a group of well-known Oracle partners. His main emphasis is on building service-oriented business processes using SOA Suite 12c, but lately his focus has shifted towards cloud and on-premise integrations, using Oracle's offerings and custom JEE solutions. Robert is a speaker at international conferences and is frequently on the AMIS Technology blog, the Oracle Technology Network, and OTN ArchBeat Podcasts. He is one of the two SOA/BPM SIG leads for the Dutch Oracle User Group (OGh) and organizes these meetups. He also works closely with the SOA Oracle Product Management team by participating in the Beta programs. In 2016, Robert was named Oracle ACE, promoted from ACE Associate, for SOA and middleware, because of these contributions. He served as a technical reviewer for the book Applied SOA Patterns on the Oracle Platform. It was published in 2014.

I would like to thank the people who have helped me over the years to shape my professional career. I'm most thankful of the people at AMIS who gave me the chances to excel in my career. Without them, I wouldn't have come this far. My utmost gratitude goes to my friends and family for their loving support in writing this book. And finally, a special thanks goes out to my coauthor, Phil Wilkins, for his efforts and hard work in making this title a reality.

Phil Wilkins has spent over 25 years in the software industry with a breadth of experience in different businesses and environments, from multinationals to software start-ups and customer organizations, including a global optical and auditory healthcare provider. He started out as a developer on real-time, mission-critical solutions and has worked his way up through technical and development leadership roles, primarily in Java-based environments. Phil now works for Capgemini, specializing in cloud integration and API technologies and more generally with Oracle technologies. Outside of his work commitments, he has contributed his technical capabilities to support others in a wide range of activities, from the development of local community websites to providing input and support to the development of technical books (particularly with Packt) and software ideas and businesses. He has also had a number of articles published in technical journals in his own right and is an active blogger. The journal contributions have been part of a wider commitment to the UK Oracle User Group (UKOUG), where Phil is also a member of the Middleware Special Interest Group Committee. He has been recognized as an Oracle ACE Associate. When not immersed in work and technology, he spends his time pursuing his passion for music and with his wife and two sons.

I would like to thank those who, over the years, have offered me opportunities, encouraged me, and supported me to reached the point where this book has become a reality. I would also like to thank my coauthor, Robert van Mölken, not only for his hard work but for also his passion and commitment that has taken an idea to a conclusion. Lastly, but most importantly to me, I would like to take this opportunity to thank my wife, Catherine, and our two sons, Christopher and Aaron, for their tolerance of the many hours I spent in front of a computer, not only on this project but also many that have preceded and those that will surely follow.

Acknowledgement from both the authors

We would like to thank the Oracle middleware team, particularly Ramkumar Menon, Yogesh Sontakke, and Jürgen Kress, who have been very supportive and engaging; they shared with us the details of the ICS Roadmap, providing access and an early glance of the product and answering the questions we posed.

We would also like to thank the Packt team, particularly David Barnes, who took the time to help us get this project up and running.

About the Reviewer

With a passion for system and application integration, Rolando Carrasco has spent most of his professional career working with customers to solve a common long-time problem: application integration. He started working with Hewlett Packard (Mexico) back in 2001, when he was in college. Even though his tenure with HP was short, as he realized very early that his professional career should be focused on Applications Integration. He started to implement integration solutions with JAVA, XML, Web Services, and EAI. He graduated with honors and was the best student of his batch (1997-2001). He studied in Mexico at Universidad Iberoamericana. The HP and Compaq fusion initiated a lot of changes in HP, so Rolando moved to Oracle, and that changed his professional career. At Oracle, he was always focused on the integration technology that Oracle had at that time; it was not as many products as today, but it was something to start with.

Then the Collaxa acquisition by Oracle happened, and that was the first step in this journey that turned Rolando into one of the most respected professionals in the Oracle SOA space for the Latin-American market. Rolando started to work with Oracle BPEL PM and had the opportunity to join the Oracle Product Management Team. He was the first PM for LAD in those days, covering Mexico to Brazil.

From 2005 to 2010, he was a Principal Product Manager for the Latin-American region and was the in charge the whole Fusion Middleware stack. Oracle acquired most of the components that are the foundation of the current Middleware offering: BEA, Thor, SUN, Oblix, Tangosol, and so on, at that time. Rolando had to be proficient in the whole stack, which was great challenge because of the extension of every product. All this kept Rolando very busy in the whole region and gave him the opportunity to work with the most important customers of the region. From Mexico to Argentina, Rolando collaborated with the different Oracle subsidiaries to promote the usage of Fusion Middleware.

Then in 2010 he joined S&P Solutions as an associate. S&P Solutions is one of the most important Oracle partners in the Latin-American region. In S&P, Rolando has had the opportunity to implement most of the Oracle Fusion Middleware stack, with the top companies in Mexico (telcos, financial institutions, retailers, manufacturing, and construction). Rolando is an Oracle ACE and is also one of the leaders of the ORAMEX Oracle Users Group in Mexico. He has a lot of articles and posts published on his blog, (http://oracleradio.blogspot.com/) as well as in the Oracle Technology Network for the Spanish speaking community.

Rolando wrote, back in 2015, the Oracle API Management 12c implementation book together with some other friends and colleagues, and this has been one of the greatest achievements in his career.

I would like to thank, first and foremost, my savior and lord Jesus Christ. Everything I do is to thank him and for him. I also thank my wife, Cristina, and my daughter, Constanza, as well as my parents, Rolando and Mercedes, and my brother, Manuel, for being my support. I would also like to thank my company, S&P Solutions, and my friends, in particular Paola, Erick, Ricardo, and Leonardo. I would also like to thank Robert van Mölken and Phil Wilkins, who are the authors of this book. I want to thank them for allowing me to be the technical reviewer. It was a very interesting and fun time to be reviewing all the chapters and sharing my thoughts with such a great Oracle professionals.

www.PacktPub.com

For support files and downloads related to your book, please visit www.PacktPub.com.

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.

https://www.packtpub.com/mapt

Get the most in-demand software skills with Mapt. Mapt gives you full access to all Packt books and video courses, as well as industry-leading tools to help you plan your personal development and advance your career.

Why subscribe?

Fully searchable across every book published by Packt

Copy and paste, print, and bookmark content

On demand and accessible via a web browser

Customer Feedback

Thank you for purchasing this Packt book. We take our commitment to improving our content and products to meet your needs seriously-that's why your feedback is so valuable. Whatever your feelings about your purchase, please consider leaving a review on this book's Amazon page. Not only will this help us, more importantly it will also help others in the community to make an informed decision about the resources that they invest in to learn. You can also review for us on a regular basis by joining our reviewers club. If you're interested in joining, or would like to learn more about the benefits we offer, please contact us: [email protected].

Preface

As you are reading this, we can assume you will have some sense of what Oracle Integration Cloud Service (ICS) is; if you don't, then do not worry, as this book will guide you through the product and the underlying concepts. We start by putting ICS into context in terms of the rapidly growing cloud marketplace, and given that we are writing for the benefit of a broad audience, let's try to get a common understanding. As you probably know, enterprise cloud solutions can be roughly split into three layers: Software as a Service (SaaS), Platform as a Service (PaaS), and Infrastructure as a Service (IaaS). ICS fits into the broad band of PaaS, which itself is divided into many forms, but we will come back to that in a moment. IaaS services such as Amazon Web Services, Microsoft Azure, and Oracle IaaS are well understood–effectively, the provision of virtualized hardware and the idea is not that new. We just have cool new names and have developed the concept. If you were involved with websites 10 years ago, you would have talked about hosting using a third-party server on which you would have installed a chosen website tool or written something from scratch. The difference between then and now is primarily the hyperscales, the speed of provisioning, and the ability to provision as many servers as you need. SaaS, at the end of the scale, is essentially a full-blooded application that users may be able to configure to meet their specific needs, but fundamentally, there is no major development involved. Illustrations include commodity blogging solutions such as WordPress and enterprise-class capabilities such as Salesforce and Office 365.

PaaS differentiates itself from the top and the bottom tiers by the fact that the platform will give you a foundation far greater than just the operating system and network, but not a complete solution that is ready to configure and use. PaaS represents the cloud-based provision of the resources you would need to build a contemporary enterprise solution such as an empty database and an application container. To return to our IaaS analogy, if in setting up a hosted website you were given all the software preinstalled to then build the website (for example, a MySQL database and perhaps a CMS solution such as Drupal) and your provider took care of the patching and so on, of the software (that is, deployment of the latest software versions with new features, bug fixes, and security fixes for the O/S, MySQL, and Drupal), then you were buying into the idea of PaaS. The challenge in understanding is the vast breadth of functionality here – from almost SaaS-like offerings such as WordPress (which can be used to build whole websites) to cloud-hosted databases, which are only a bit more than IaaS. As a result of all the flavors of PaaS we talk about, the different areas of PaaS have developed their own terms to make them distinct from other PaaS offerings; for example, integration platforms have adopted the term of iPaaS (integration Platform as a Service), and cloud databases are referred to as DBaaS, and so on.

This fragmentation, as well as the desire for a similar acronym for all layers, has spawned the adoption of the expression XaaS—anything (or X) as a Service.

iPaaS can be commonly characterized in two flavors of products—the heavily technical solutions such as Oracle SOA Suite (known as SOA CS) and IBM's WebSphere Application Server on Cloud, along with the second type with graphical user interface-driven solutions in which ICS competes, which includes products such as MuleSoft, Dell Boomi, and SnapLogic. These more visual and often less technical tools can often draw on more consumer-like experiences, and you can trace the origins of this group of iPaaS solutions to consumer predecessors such as products in the form of solutions such as IFTTT (If This Then That – https://ifttt.com/) and Zapier (https://zapier.com), where you could set up in a fairly visual manner integrations like, when a favorite RSS feed posts something, you can get it to be tweeted by on your Twitter account by exploiting the APIs provided.

The difference between solutions such as IFTTT and iPaaS at the enterprise end of the spectrum is sophistication, capability, security, and reliability; that is, you expect enterprise solutions to be 24/7 resilient and very robust, not to mention, having a focus on connecting enterprise solutions such as Salesforce, Workday, Oracle Taleo, Office 365, and SuccessFactors. That said, as more social and collaborative platforms influence more and more of the workplace, we will see this line blur.

The last thing to keep in mind with our first iPaaS group is that these more technical products are typically cloud-enabled versions of their mature on-premises solutions. As mentioned previously, SOA CS is Oracle SOA Suite when deployed on-premises (with a few additional tools added to address the need to hide the platform considerations of the deploying SOA). The newer generation products, such as ICS, Boomi, and MuleSoft, are largely focused on a cloud delivery approach and cannot be deployed as on-premises solutions in the traditional manner (such as SOA Suite), although vendors such as Oracle are leading innovative thinking in this space.

Before we look at why ICS is a significant product in the iPaaS space, it is important to remember that you do not need to have acquired any other products from Oracle to use it. In simple terms, ICS can be used to integrate with any application/service if it complies with one of a number of supported standards or has an adaptor. The ability to offer adapters that simplify integration is going to be an area of growth and market differentiation.

Oracle has several distinct advantages in the marketplace that makes ICS a very significant player. Setting aside the arguments of Oracle's deep pockets, the factors you might consider are as follows:

Mature leading integration solutions that can be exploited to create a new feature rich tool. Even while Oracle had focused on on-premises solutions, they have had to build connectors to major cloud solutions such as Salesforce and Workday, and these are being ported to the cloud.

Oracle has a very large partner base and has created an ecosystem for partners, customers, and the wider community to build, sell, or donate their adaptors to work with ICS (and potentially for SOA CS and other parts of their PaaS offering).

ICS can be easily adopted by both its middleware (SOA Suite and so on) and customer base, as well as its applications customers (E-Business Suite, Seibel, Fusion applications, and so on).

Hyper-convergence means that Oracle owns solutions from microchips and other hardware all the way to SaaS, which in turn creates economies and optimizations as the hardware is