The Definitive Guide

to API Integrations
Explore API Integrations Below the Surface
 Table of Content

03 DISCOVER YOUR API INTEGRATION STRATEGY 17 QUALITY CHECK

LOGGING AND MONITORING..................................... 18
COMMON INTEGRATION FAILURE MODES............18

05 PRE-INTEGRATION WORK
DEFINE USER STORIES.................................................... 6
KNOW YOUR USE CASE................................................. 7
SELECT INTEGRATION ENDPOINTS...........................7
20 CLOSING
SELECT INTEGRATION ENDPOINTS...........................8

21 ABOUT CLOUD ELEMENTS

09 LETS INTEGRATE
AUTHENTICATION...........................................................10
DATA DISCOVERY.............................................................12
MAP & TRANSFORM DATA............................................ 13
SEARCH, BROWSE, AND PAGE....................................15
EVENTS, TRIGGERS & ORCHESTRATION..................16
CRUD ACTIONS CREATE EVENTS...............................16

Cloud-Elements.com 2
D I S C O V E R YO U R A P I
INTEGRATION STRATEGY Define User
Stories

Authenticate The
Endpoints

Know Your
Discover Standard And Select Integration Use Case
Custom Data Objects Endpoints

Options For Search Determine Event

Map & And/Or Browse Triggers & Orchestration
Transform Data

CRUD Operations
If you follow these steps to plan, integrate, and test
your API integration, you will provide your customers
with the absolute best app integration experience.
Integration Testing
GET MORE DETAILS IN OUR EBOOK:
THE DEFINITIVE GUIDE TO API INTEGRATIONS

Try Cloud Elements for Free!

 YOUR CUSTOMERS
KEEP ASKING FOR MORE.
The demand for API integrations is accelerating. Your customers want apps that easily connect to the other apps
they love to use. The rapid fragmentation of the application software market, led by the demand for best of breed
apps and services at a departmental level is accelerating the demand for apps that just work together, instantly.
They demand that data moves freely from one technology to another in real time. And that its user friendly and
easy to set up that integration, with no developers required.

Trick is, building integrations one at a time takes too long and costs too much. Your developers are superheroes,
but that doesnt mean we have to expect them to have super powers. When it comes to API integration, there is so
much below the surface that most of us dont think about on a day-by-day case.

This ebook is designed for everyone involved in app development or app synchronization, but most specifically
product managers. It will guide you through how to build your API Integration strategy and help you choose the
best fit for your product.

Well cover:
01. What you need to do before integrating
02. Tips for diving in and integrating cloud services
03. How to quality test & check

Happy Integrating.

You build the killer app.

Mark Geene
CEO - Cloud Elements
[email protected]

Well take care of the REST.

Cloud-Elements.com 4
 BEFORE YOU START:

PRE-INTEGRATION WORK
Integration is a means to an end; a connected, cooperative experience where data
flows seamlessly between your app and the other apps your customers are using, is
the end result that you are seeking through integration.

In order to build a cooperative app experience, pre-integration work (defining user

stories, knowing your use cases, and selecting integration endpoints) is a critical.

Cloud-Elements.com 5
 Product managers are responsible for deciding on which integrations to build into their application to create a
DEFINE USER STORIES cooperative experience with the services and applications used by their customers. To guide the development
of these integrations, its helpful to write integration user stories that provide a clear picture of the cooperative
experience you envision for your end-customers. User stories can be defined from two different perspectives: a
system persona or a user persona. We recommend user personas, because they typically provide more context
and focus on the customer experience.

Lets use Mary Marketing as an administrator user example. A Marketing Administrator selects and
authenticates their marketing automation application (e.g., HubSpot, Marketo) so that leads are automatically
created from the email addresses of webinar attendees. This is a one-time setup that doesnt need to be in the
direct flow of the application for the end-users - a set it and forget it function.

SYSTEM PERSONA USER PERSONA

By developing user stories in the system persona, Product Managers can A great user persona is written from the point-of-view of an end user. That
become too focused on the technical aspects of the integration instead of is contrary to a system persona, designed to specify technical interactions
the collaborative user experience. System based integration stories dont between systems. Its important to get as much use out of these personas as
enable the Product Manager to fully visualize the workflow that the user possible to ensure that your stories relate back to an experience model from the
will experience; they guide the integration as a technical solution but not as viewpoint of your end users.
effectively as a business solution.

STORY STORY

The Webinar Management Application will automatically create leads and As Mary Marketing, I can select the Marketing Automation Service that I
associate them with accounts in the selected Marketing Automation System so would like to connect to my Webinar Application so that I can create leads from
that the two systems will stay in-sync. webinar attendees.

Cloud-Elements.com 6
 In this example, and as a theme throughout the rest of the eBook, we will be discussing
KNOW YOUR USE CASE API Integration design for SaaS (cloud-based) applications to other cloud services. This is
an external integration use case, app-to-app.

Follow these steps to develop your own integration user stories:

01. Start by developing stories as a user persona whenever possible to provide the clearest perspective
on how and why your users will benefit from the integration. Capture the user experience through
these stories.

02. Refer to the target and source systems that youre integrating to clarify how the data is moving
between your application and the service(s) youre integrating and cooperating with.

Only write user stories with a system persona when you need to elaborate on the behind the scenes
integration tasks that dont involve users.

Next, you will want to think about optimizing your app by selecting the end-points and data needed to integrate
SELECT INTEGRATION with your app. Your goal is to provide a natural experience for selecting and authenticating end-points.

ENDPOINTS Start by guiding end-users within the natural workflow:

01. Mockup a simple selection and authentication flow to determine

where it best fits within your app.

02. Determine if this a one time event.

03. Talk with UX about the best option for your application:
Will there be a pop-up window,
full-window, or redirect?

Cloud-Elements.com 7
 The best integration experience makes end users forget theyre even connecting your app with another
SELECT INTEGRATION service. End-point selection and authentication should happen within your application with your brand

ENDPOINTS requesting access to the end-point. You manage the experience for the user from end-to-end within a
cooperative application and develop cooperative use cases optimized for your application. Dont let your
users leave your app! The last piece to selecting integration endpoints is to anticipate future integration
needs. Many mature apps provide dozens, if not hundreds, of end-points as options for their users to
connect to. As you design your cooperative app user experience, take the time to get to know your
customers and learn what matters most to them.

Ask yourself these questions to anticipate future integration needs:

Which categories of services do my customers want my app to connect to?

What end-points do my customers use regularly within each of these categories?
How would you quantify the value of this connected environment?
What value do these integration points have on the user experience?

In the ReadyTalk example, the integration end-points are selected by marketing or office administrators.
When setting up the integrations, the app admins (or end users) can map their data objects from a marketing
automation system to ReadyTalk.

Cloud-Elements.com 8
 DIVE IN:

LETS INTEGRATE
SUCCESS!

Pre-integration work is complete! But wait Once you are ready to integrate, there
are initially more questions than answers. API Integration design is about building
apps with pre-built, pre-orchestrated integrations to complementary applications.
However, its more complex than point-to-point integrations, and its bigger than
simply writing directly to the API.

Below the surface of writing to an API, developers are faced with complexities
including authentication, custom data discovery, data mapping and transformation,
logging and event management.

This section will walk you through all of these areas and break down steps/tips
to help you swan-dive into integration.

Cloud-Elements.com 9
 Securely and reliably authenticating to the cloud service endpoints that connect with your app is the foundation
AUTHENTICATION for your API integration experience. Youll need to determine the type of authentication mechanism used by the
endpoint and the workflow required.

To get started, ask yourself these questions:

01. Which persona will be authenticating to the service? A system administrator or the individual end-users?

02. What are the types of authentication mechanisms supported by each service Im connecting my app with?

03. Are there specific security concerns based on the type of data that I am enabling my user to access?

04. Where should the data for logins, passwords, keys and refresh keys be securely stored?

Once a user is presented with the UI to select the endpoint they wish to connect to, your application will open
an authentication screen. Users will use their own login credentials to access the service and grant access to
connect the apps.

Start by identifying the authentication mechanism. The authentication mechanism drives the workflow for each
instance an endpoint connects to your app. There are four primary types of authentication mechanisms you can
use to connect your app to a cloud service endpoint: Basic Authentication, OAuth, OAuth 2, and SAML.

THE FOUR PRIMARY TYPES OF AUTHENTICATION USED AT OPEN API ENDPOINTS:

Basic Auth (or Basic access authentication) is a widely used protocol for simple username/
password authentication. This type of mechanism provides no confidentiality protection for the
transmitted credentials.

OAuth An Open Protocol that provides a process for end-users to authorize third-party access to
their server resources without sharing their credentials using user-agent redirections. Credential
tokens are long lived, typically a year.

OAuth 2 Delegates security to the HTTPS protocol. OAuth 1 does not require this and uses
alternative methods to remain secure. It also introduced the use of refresh tokens that allow
authentications to expire, unless refreshed on a periodic basis.

SAML An XML-based open standard data format for exchanging authentication and authorization
data between parties, in particular, between an identity provider and a service provider. SAML
is popular with large single sign on organizations, corporate applications and in many older
applications.

Cloud-Elements.com 10
 Next, determine how to manage refresh tokens and make sure they are securely stored. If you want your user to
AUTHENTICATION authenticate once and then not have to authenticate again as they interact with the endpoint, your application
will need to manage the refresh token from the endpoint (if available) in addition to the initial access token. Some
access tokens expire in an hour, while others may last as long a year or never expire. (See a related blog for details
on the duration of cloud storage refresh tokens).

Tokens are storing very private user information. As a developer for the integration between your application and
an endpoint, its your responsibility to protect this type of sensitive information. If your application stores tokens, it
is highly recommended that you encrypt with 256 bit encryption at rest within your data storage system with the
key owned by the end user. Encrypted tokens stored with 256 bit encryptions are really tough to break, protecting
your application and your customers user-names and passwords. Once you have a strategy for managing access
and refresh tokens, you will need to figure out how to manage permissions from the endpoint. This all depends on
how that endpoint is structured. As youre designing your API integration, consider whether or not your client will
be authenticating an administrator account or an end-user account. How will you handle and pass permissions
from the endpoint? An admin account at the endpoint will allow you to manage objects and data across your
organization, while protecting client data with specified users permissions.

Lastly, you will want to track and log activity for each instance for support, monitoring and security purposes.
When you embed integrations into your application, your customers will expect you to support the integration.
Your support team will need access to API call activity logs, error reports and alerts to provide the success
experience for your customers. Tagging the API calls with an identifier for each customers instance of an
integration will make it easier for your team to manage and support the specific user.

QUICK TIPS

Refresh Tokens:
OAuth 2 is the only mechanism that has refresh tokens. This allows you to keep the token alive,
so that users do not have to log in again. SAML does not use refresh tokens. Services like Xero,
Quickbooks, etc. do not yet support

OAuth 2.0. Refresh tokens have similar variations in how long they are authorized for access.
You will need to develop a strategy for managing access and refresh tokens for each endpoint

Cloud-Elements.com 11
 When integrating to application endpoints such as CRM, Marketing Automation or Finance systems, your app
DATA DISCOVERY will need to discover the standard and custom data objects and fields used by your customers instance of the
endpoint. This can be done by following a 3-step process.

STEP 1:
IDENTIFY RELEVANT DATA OBJECTS
Identify which of your applications data objects are relevant to the data youre targeting at the endpoint(s). In most
cases, you will only be integrating a subset of the data objects and fields from the endpoint. Your applications data
model will provide the basis for determining the data youre interested in integrating with. You will then identify
the standard objects at the endpoint that you will be integrating to your app.

STEP 2:
DISCOVER STANDARD & CUSTOM OBJECTS
Your integration will likely need an automated means to discover custom objects and fields at the endpoint if you
plan to support the integration and mapping of custom data. Since the majority of SaaS application endpoints make
extensive use of custom data fields and objects, you will likely need to discover the data structure at the specific
instance of the endpoint in addition to understanding the standard data objects. A standard object will persist
across multiple instances of an endpoint, but custom objects and data fields will only exist for a specific instance of
an endpoint.

Currently only a small subset of endpoints provide discovery APIs to automate the discovery of objects and their
metadata. Plan to use these services when they are available at the endpoint. When data discovery services are
not available you can discover the data by doing a GET by ID to retrieve a sample of the data structure at the
endpoint.

STEP 3:
START TO STANDARDIZE ON PATTERNS
The beauty of taking the extra effort to discover data objects your app is creating is that once youve collected
enough data and evidence, you can start to figure out patterns your customers create. Those patterns are
what you can use to automate the mapping and transformation process. From those patterns you can define
additional fields or objects that your application can potentially use to enhance the integration experience for
your customers. For example, you can create a pre-built custom object for your app thats installed when clients
integrate to a particular endpoint or category of services (e.g., Marketing Automation) that accommodates the
typical data fields you find your customers creating when they integrate to those endpoints.

Cloud-Elements.com 12
 CONSIDER WHETHER YOUR INTEGRATION USE CASE IS STATIC OR DYNAMIC.
MAP & TRANSFORM DATA
Static integrations provide a fixed mapping between data fields and do not discover custom data and do not
provide the end-user with a facility to change mappings. Static integrations are suitable for services with
fixed payloads such as cloud storage, payment and messaging services.

Dynamic integrations are designed to accommodate custom data. These integrations require services to
discover custom data, the ability to map data and custom fields for each instance of an integration and often
the ability to transform data values between endpoints. Theyre dynamic in the sense that they enable
each integration to be uniquely designed based on the data discovered at the endpoint.

If youre providing a Dynamic integration experience for your users, youll want to consider how your users will be
able to map data fields and transform data values. With a UI for data mapping, you will enhance the experience to
seamlessly synchronize applications.

HERE ARE SOME KEY CONSIDERATIONS FOR DATA TRANSFORMATIONS:

Evaluate each endpoint and determine if values are consistent with the values your application expects.
For example, help desk ticketing systems use different priority schemes (High, Medium, Low vs. 1, 2, 3)
and you will want to consider if you will transform these values into a consistent data set required by your
application.

Understand each individual payload structure in order to connect to the different endpoints. Cloud services
and technologies deal with a variety of payloads, including XML, SOAP, JSON, etc, which means that there
will be a variety of formats coming in with the different data objects.

You cant anticipate all of the transformation needs for each endpoint and each customer. Consider
providing a programmatic facility to incorporate custom logic into your transformations.

Cloud-Elements.com 13
 Weve broken down the process to setting up mapping and transformations into five easy steps

Define a Default Map

01
A default mapping is a template that will define how standard data structures from an endpoint will map into your applications data model. This
mapping establishes the association of standard fields within a standard object into the fields within your application. The primary goal here is to
save your users time by pre-mapping standard data from the endpoint reducing the effort they need to undertake when connecting to your app.

Present Discovered Data

02 Once your app Discovers the data structure at your customers endpoint you will need to present the discovered structure, custom objects and
custom fields for your users to evaluate and map based on their understanding of the data.

Map Custom Data from Each Instance

Custom data objects and custom fields within standard objects are a common feature of most SaaS applications. Invariably your customers will
03 make extensive use of these capabilities. Therefore default mapping will only be the beginning for a majority of your customers. The challenge
here is that youre going to need to support unique data maps at the Instance level. The instance level data maps will be specific to that
authenticated account of an endpoint.

Make your Defualt a Guideline - Not a Requirement

04
There is no way to always anticipate how your users may use standard fields. Occasionally, users will use standard fields for a custom purpose and
they will likely want to map this data to a field other than your default mapping. Just as you handle Custom Data on a dynamic self-service basis,
its best to enable your users to override your default settings.

Consider Embedding Self-Service Data Mapping Tools

05
As discussed previously the optimal experience for your users is to have a user interface integrated with your app to facilitate the mapping of data
from their cloud service apps into your data structure. By providing this facility, your customers can make their own mapping using their best
judgement and respond instantly to accommodating new data fields and objects.

Cloud-Elements.com 14
 Once youve mapped the data and ensured the values can be transformed, the next step is to present the user with
SEARCH, BROWSE, a way to search and/or browse the data that they wish to access.

AND PAGE A few example user stories for search APIs are:

01. As a cloud document storage user, I want to browse by cloud folders and find my documents, so that I can
load them into the app integrated to that service.

02. As a CRM user, I want to browse customer accounts by company name, so that I can easily sort and find the
right account to sync with the integrated app.

03. As a Finance user, I want to search for paying and non-paying customers, so that our app can stay in sync
with my reports.

There are a few areas to consider when planning and defining the API design to have search and browse
functionality. First, the search functionality provided by APIs from leading SaaS applications varies dramatically
and these limitations will impact the type of functionality you can offer across endpoints. It is important to
research the search capabilities that are offered first to determine if you will take a least common denominator
approach or write specialized search integrations to each endpoint. Due to this variability by endpoint, search will
be one of the most custom parts of your integrations and it may change the use case of your integration depending
on the service your client is integrating with.

Another area of variability is how to page the results returned from each endpoint. When presenting the search
results from the endpoint you will want to implement a common approach to pagination for your users. Paging is a
big factor in how users can search and find the data they are looking for.

Cloud-Elements.com 15
 It is important to determine how youll consume events from the endpoint(s) youre integrating with. Many
EVENTS, TRIGGERS & endpoints support webhooks that can be used to track create, update and delete actions as they occur to objects
notifying your application. Webhooks keep your app up-to-date with changes (events) at the endpoint. If the
ORCHESTRATION endpoint does not support webhooks, then your application will need to poll the endpoint to identify changes
as they occur. With polling, youre querying the timestamps on objects at the endpoint to see if actions (create,
update or delete) have occurred. Your app becomes responsible for reaching out on a periodic basis based on a
frequency that you set such as every minute, hourly or daily.

Events are critical to giving you the trigger to sync data between multiple endpoints. RESTful methods initiate
CRUD ACTIONS Create, Retrieve, Update and Delete actions to resources at endpoints using GET, POST, PUT, PATCH and DELETE
methods. Since the goal is to synchronize data when its changed, you will need to consume events that Create,
CREATE EVENTS Update and Delete data. Consuming events will enable your application to execute corresponding methods to
Create, Update or Delete when notified that an event has occurred at the endpoint; thereby synchronizing actions
that change data.

Cloud-Elements.com 16
 ALMOST DONE:

QUALITY CHECK
Youve made it. You most certainly deserve a pat on the back. But lets reserve that
for the very end of these final and vital steps: logging/monitoring. Did you really
think that simply deploying an integration was the end goal? Well think again. Now
that your API integration is off to production and your users are LIVE, its time you
set up API monitoring to keep track of key usage data.

Cloud-Elements.com 17
 Integrations are mission critical. Your customers are depending on your integrations to be up and running, no
LOGGING AND matter what. We hear over and over that if an apps integrations are not working, customers and end users are

MONITORING frustrated, reflecting poorly on your proprietary application.

Some of the most common errors to test for include:

01. Authentication credentials change due to changes in passwords

02. API calls are failing

03. The endpoint service may be down completely or purposely shut down for scheduled maintenance

We have compiled years of integration experience into useful advice to help you not only prepare for, but hopefully
avoid common integration failure modes.

Lack of insightful usage data: When you do not measure your integration stats, or even feed the data into your
COMMON INTEGRATION application teams performance monitoring tools (APM), you lack user experience data. By passing the usage
data to your monitoring team, you enable support to better understand usage patterns and gain the insights
FAILURE MODES required to establish proactive support of your clients integrations.

No health checks: With frequent endpoint health checks, your app will not get the blame when things go wrong
and you can proactively stay ahead of unhappy customers who often voice their concerns about the stability
of your product. We recommend automating health checks for your apps integrations to determine if the
endpoint is the issue or if your integration or your app are at fault. With a ping to an endpoint you can diagnose
one of the most frequent integration issues.

Slow response times: Response time issues are among the most frequent complaints with integrated apps. The
key for your operations team is to understand the response time from the endpoint, to isolate your applications
performance vs. the performance of the endpoint you are connecting to. Some endpoints are notoriously slow
and you will want to make sure that the proper expectations are set with your clients for performance delays
when integrating to slower endpoints.

Cloud-Elements.com 18
 If slower response times persist, you can also choose to implement a cache or a number of other mechanisms to
COMMON INTEGRATION improve performance.

FAILURE MODES No tags on usage data: Tagging your usage data enables rapid filtering of your logs to isolate and debug issues.
Tags are vital to enriching your reporting and dashboard experience. Use tags to dig into the granular data and
fully support filters, searches, and queries.

Inconsistent error codes: Your operations team will not have time to research every vendors error code.
Standardize to the error responses your application currently runs on, so that your support team can
understand and react to errors more rapidly and efficiently. We advise that you normalize the error code, but
still provide a payload back from the endpoint on how they display the errors.

Lastly, you will want to track and log activity for each instance for support, monitoring and security purposes.
When you embed integrations into your application, your customers will expect you to support the integration.
Your support team will need access to API call activity logs, error reports and alerts to provide the success
experience for your customers. Tagging the API calls with an identifier for each customers instance of an
integration will make it easier for your team to manage and support the specific user.

Heres an example of a typical usage report your API can show:

Cloud-Elements.com 19
 Closing
And there you have it. If you follow these steps to plan, integrate, and test your API
integration, you will provide your customers with the absolute best app integration
experience. For a quick cheat sheet, download our 10 Step Guide to API Integrations,
and check out our resource center for other easy to follow information. If you have
questions, or if you would like to learn more about Cloud Elements, contact us today!

Try Cloud Elements for Free!

Cloud-Elements.com 20
 About Cloud Elements
Its bigger than just a connection, integrating APIs is about moving data, powering
transactions, connecting mobile apps and thousands of other functions.

Cloud Elements is a cloud API integration platform that enables developers to pub-
lish, integrate, aggregate and manage all of their APIs through a unified platform.
Using Cloud Elements, developers can quickly connect entire categories of cloud
services (e.g., CRM, Documents, Finance) using uniform APIs or simply synchronize
data between multiple cloud services (e.g. Salesforce, Zendesk and Quickbooks)
using its innovative integration toolkit.

Founded in October 2012, Cloud Elements is purpose built for developers to help
organize their world of APIs through a one-to-many approach. A Visionary in API
Management, according to Gartner Inc., Cloud Elements is headquartered in Den-
ver, CO, but serves customers worldwide.

Learn more about us at: http://cloud-elements.com/

Cloud-Elements.com 21