Architect 1

Architect Overview Module

Version 2.1
 • Architect Overview

• Alerts & Events Framework

Agenda • Selecting a Runtime Architecture

• Runtime Installation

• Runtime Configuration

4
 • Document Flow Fundamentals
Module 1 • Error Handling
Agenda • Source Control

• Versioning

5
 Document Flow

6
What is a Document?
A Boomi Document is the main unit that powers the execution in a process flow.
Dell Boomi supports five raw document types:
• XML
• JSON
• Flat File
• Database
• EDI
XML JSON Flat File Database EDI EDI
1,Contact1,Address1 S5*1*CL~
<Account> { |^|ColumnA|^|ColumnB S5*1*CL~
"values": [ L11*0003187*BM~
|^|ColumnA|^|ColumnB
{ 2,Contact2,Address1 L11*0003187*BM~
<activeBillingAddress/> "_key": "~",
|^|ColumnA|^|ColumnB
L11*122*PO~
"connections": {"_total": 3,Contact3,Address1 |^|ColumnA|^|ColumnB L11*122*PO~
<activeResAddress/> 129}, |^|ColumnA|^|ColumnB
L11*ZOR*SO~
"firstName": "Adam“ 4,Contact4,Address1 |^|ColumnA|^|ColumnB L11*ZOR*SO~
}, |^|ColumnA|^|ColumnB
<addressComplement/> G62*10*20110809*Y*0*CD~
{
"_key": "hks0NPUMZF", 5,Contact6,Address1 |^|ColumnA|^|ColumnB G62*10*20110809*Y*0*CD~
G62*79*20110809*Y*9*CD~
|^|ColumnA|^|ColumnB
<addressTypeCode/> "connections": {"_total":
500}, 6,Contact6,Address1 |^|ColumnA|^|ColumnB G62*79*20110809*Y*9*CD~
"firstName": "Brandon“ AT8*G*L*48*41**E*2154.1~
|^|ColumnA|^|ColumnB
</Account> } 7,Contact7,Address1 AT8*G*L*48*41**E*2154.1~
|^|ColumnA|^|ColumnB
NTE*GEN*A S~
],
"_total": 2
|^|ColumnA|^|ColumnB
NTE*GEN*A S~
}

7
What is a Document?
Documents are presented in four different formats:
• Records
• Transactions
• File Instances
• Empty

8888888888888888888888888888888888888888888888
8888888888888888888888888888888888888888888888
888888888888888888888888
888888888888888888888888888888888888
88888888888888888888888888888
8888888888888888888888888
888888888888888888888888888888888888
8888888888888888888888888
888888888888888888888888888888888
888888888888888888888888888888888
8888888888888888888888
88888888888888888888
8888888888888888888888888888888888
888888888888888888888888888
8888888888888888888888888
888888888888888888888888888888888888
888888888888888888888888
8888888888888888888888888888
8888888888888888888888888888888888888
88888888888888888888888888888888
88888888888888888888888888888888

8888888888 8888888888
8888888 8888888
888888888 888888888
88888888 88888888
8888888888 8888888888
888888 888888
8888888888 8888888888

8888888888 8888888888
8888888 8888888
888888888 888888888
88888888 88888888
8888888888 8888888888
888888 888888
8888888888 8888888888

8
Document Flow Concepts

• Shape Execution
• Execution Path
• Batch Document
• Original Document ID
• Document Failures
• Altering Document Flow

9
Concepts: Shape Execution
Shape Execution refers to how documents move through the process shapes.
Behavior:
Documents move as a group from shape to shape. Each document executes on a given shape
IN SEQUENCE and generally INDEPENDENTLY before any of them move to the next shape.
A shape executes only if at least one doc reaches it. For example: if a start shape doesn't return
any docs (e.g. nothing matches query), no subsequent steps are executed.
A No Data start shape actually generates a single, empty document.

10
Concepts: Shape Execution

Behavior (continued):
Documents flow from left to right and never affect a prior execution shape, as all documents
have already passed that shape.
Processing as a group is more efficient because the shape logic is loaded into memory once for
the entire group of documents.

11
Concepts: Execution Path
Execution Path refers to the route documents take through the process.
Behavior:
• Documents move from the start shape to one or many end points.
• All documents may not have to take the same path: Decision, Route and Business Rule
steps cause documents to take different paths.
• Shapes with multiple out paths have a defined execution sequence.

12
Concepts: Execution Path
continued)
Behavior (continued):
• Paths execute sequentially, not in parallel. A path completes before the next path starts.
• Branches cannot communicate document changes (data or properties).
• Must use other techniques to pass data to next path.

13
Concepts: Batch Document

A batch document refers to a single document that contains multiple logical records
Behavior:
• Shapes configured to reference individual profile elements (e.g. Decision) will use the value
from the first record in the batch document.
• Shapes that reference the entire profile (e.g. Map) will iterate through all logical records.
Use Cases:
• Any type of data can be a batch document, most common with flat file (CSV) and database
data.
• Records within a single batch document can be aggregated in Maps.

14
Concepts: Original Document ID

Each document read into the process via the start shape is assigned an internal/unpublished
identifier, known as the original document ID.
This identifier follows the document through the process.
Why important? Primarily to understand how document failures behave and track.

Discussion points:
• Splitting a document propagates the original ID to the resulting documents.
• Combining documents consolidates the original IDs.

15
Concepts: Original Document ID
Discussion points (continued):
• Connector calls within the process create a new set of document IDs (However, there is still
traceability from a doc failure perspective).
• In deployed process logs, the “view linked documents” command can be used to associate
different sets of documents with common IDs.
• Tracking numbers are useful in associating document IDs with business IDs.

16
Concepts: Document Failure
When an error occurs, the affected document(s) stops immediately and does not continue to
subsequent shapes
Behavior:
• Errors can occur per-document or for the entire process.
• “Process-level” errors stop all documents immediately.
• “Document-level” errors stop the individual document(s) but other documents continue
processing.

17
Concepts: Document Failure

Behavior (continued):
Documents are identified by their original ID. This is especially important to remember when
splitting batch documents or combining documents:
Split: If one of the split documents encounters an error, the single original batch document is
marked as failed and stops processing. This means all the other split documents from that
original batch document will stop processing as well.
Combine: If a combined document encounters an error, all the original documents are marked
as failed and stop processing.

18
Document Error Example One document errors
in connector, the other
document continues.

Branch shape
duplicates inbound Only failed documents stop
Documents documents. processing; other documents
loaded from complete normally.
database query.

19
Document Error Example with Split 1 document errors, causing all
documents that began as part of
the original FTP file to fail.
File splits into
multiple
documents.

All documents that started

Branch creates a as part of the same original
Single FTP file is duplicate set of document fail if any single
read as 1 original documents for document fails.
document. each path.

20
Concepts: Altering Document Flow
The Try/Catch shape can alter how batched
documents flow through the process.

Configuration and behavior:

• The Try/Catch automatically resets a
document’s Original Document ID so that it
can fail individually.
• All documents are sent down the Try path.
• Only documents that fail down the Try path
will then flow down the Catch path, even if
they came from a split batched document.

21
Try/Catch Example with Document Split 1 document errors. After a
Try/Catch, each document succeeds
or fails individually even if it is part
Try/Catch “resets” the of the same original document.
File splits into document IDs so that
multiple each document can fail
documents. individually.

All failed documents

follow the Catch
All failed documents path.
Single FTP file is follow the Catch
read as 1 original path.
document.

22
Concepts: Altering Document Flow

The Flow Control shape can alter how

documents flow through the process.
Configuration and behavior:
• Execute documents individually or as
an arbitrary group size vs. as a
single group.
• Execute documents simultaneously
in multiple threads vs. a single
thread.
• Each document executes to
completion before next document
starts, one at a time, subset at a
time, and/or across multiple threads.

23
Concepts: Altering Document Flow

Implications:
Notable performance decrease as each shape’s logic is initialized multiple times vs. once per
group so only use if appropriate for scenario.
Use Cases:

• Cases where one record post impacts the next.

• Incremental inventory counts.
• Detecting duplicates across documents with the group.

24
 Participants to Complete:

Account Setup and

Document Flow Activity
Participants
to Complete

Page: 4-19

26
Document Flow Summary
• Documents flow together: all documents processed in a shape then continue on to the next
shape.
• Documents flow down one path, then the next. Documents can have different paths.
• Documents can flow as a batch of logical records or as a single logical record.
• Shapes act differently on documents and batches and can modify document groupings into
batches and split them out of batches.
• Document failure changes execution path. Batching impacts path.

27
 Source Control

28
Technique: Source Control

Differences in development with Boomi AtomSphere:

• No personal working copy.
• Always at Head Revision – working on the latest version.
• No default check-out/check-in. (Component Locking feature)
• No version merges.
• Each Component has its own independent revision history, even the process itself.

29
Technique: Source Control Management

How is Component source control managed?

• AtomSphere is the source control engine; there is no external management or import/export.
• There is a single copy of processes and components representing the last revision.
• All users work with the latest revision.
• Users do not have their own “copy” of the entire code set and contribute changes.
• There is no trunk vs. branch concept.
• There is no automated way to “merge” differences between revisions.

30
Technique: Source Control - Folder Permissions

Folder Permissions
• Configured by Administrators.
• Permissions set by roles that allow users to modify a folder and its components.
• If not assigned permission, then the components are read-only.

31
 Versioning

32 Worldwide Technical Training

Versioning - Component

How does Component versioning work?

• Every time you save a change to a component, the Revision number is incremented along
with who changed it.
• Each component has its own independent revision history.
• You can restore (roll back) to a previous revision – but be careful.
• Make sure other processes/components are not relying on the most recent version.
• Restoring will not delete any other components referenced by the newer configuration.
• Show Where Used feature lets you see all the components that reference a given component

33
Revision History

34
Viewing Revision History

35
Viewing Previous Revisions

36
Versioning - Restoring A Process

Be Aware That:
• A Process restore is independent of its components (“canvas-only”).
• This means that the restored process will refer to the most current sub-components.
• Create a deep “snapshot” when desired via deploying a process.
• Deployments are versions and can be used to recover a prior state.

37
Versioning - Retrieving a Deployment

38
Versioning - Retrieve All or Part of a Deployment

39
 Error Handling

41
Why is Error Handling design important?

• Errors are common: A Boomi Integration connects two (or more) different systems and
differences in data format or content appear at the interface. Ensure that correct data is
transferred.
• Errors are disruptive: Can stop a business process, or damage data integrity
• Handing errors properly can make everyone’s job easier, reduce integration delays, or
“friction”, minimize process interruption and speed up integrated processes.
• Quickly resolve problems
• Handling errors properly by ensuring successful recovery is a sign of a robust and mature
integration architecture.
• Communicating status and details keeps confidence in the integration high and ensures that
the right people are focused on the solution.

42
 Error Example:
Error Example: Process Reporting View
Process Reporting

Image of Process Logs Here

43
 Error Example:
Error Example: DocumentConnectivity/Login
Detail View Failure

44
Error Type: Process Errors

Definition:
• Errors halt a deployed process, and all documents, immediately.
• The scope of the error is the entire process, not a set of documents.
• No further processing occurs, unless there is an active Try/Catch step that will resume
processing of all documents on the Catch leg.

Impact:
• Process errors halt the entire process.
• No steps are executed after this type of error occurs.
• All documents fail, no additional processing (except for Try/Catch) occurs.
• No checkpoint or commit mechanisms - any steps that executed are not rolled back.

45
Errors Type: Process Errors

Causes:
• Connector connectivity errors (varies by connector).
• Connector app responses (varies by connector). Severe connectivity errors.
• Exception step – when Stop Single Document = False.
• Unhandled Data Process scripting exceptions.
• Incomplete process/component configuration, e.g NullPointer (dev-time fix).
• Connector errors on the Start step (note can’t catch because no docs yet).

46
Error Type: Document Errors

Definition:
• Errors in a deployed process that halt the errored document.
• The scope of the error is the Document, which can cover multiple logical records.
• No further processing occurs on the Document, unless there is an active Try/Catch step that
will resume processing on the Catch leg.

Impact:
• Document errors halt the processing of a specific document.
• No additional steps or processing for that document (except Try/Catch)
• No checkpoint or commit concept - any steps that executed are not rolled back for that
document.

47
Error Type: Document Errors

Document Errors
• Mapping/data errors (function failures, profile mismatch/no output, data formatting/missing required
fields)
• Script errors
• Data format errors
• Connector connectivity errors (varies by connector)
• Connector app responses (varies by connector but more common as doc level)
• Exception step – when Stop Single Document = True

48
Document Error Granularity

When a document fails:

- The entire “original document ID” fails.
What constitutes the entire original document?
- All logical records that made up the Document in the Start shape - the source
document.
- Documents that trace back to the source document and if any component of the
source document fails, they all fail.
- Batching and splits can impact the scope of document error failure - e.g. all
components of a batched document fail together.

Try/Catch shape changes impact of document errors by “resetting” the document IDs
- Only the document that failed will stop processing

49
Error Type: Atom Errors

• Example of a run-time error

- Halts the process at inception.
- Generally not recoverable and cannot be “caught” in the process.
• May be detected/handled by runtime monitoring techniques (discussed in a separate
module)
• Effectively cause a process level error
- All documents halt
- A failure event is reported when the atom comes back online.
• During a process execution, any activities at the Atom operational level which fail,
irrespective of the design of the process, would be considered an Atom error and are
generally caused by a lack of Atom resources.

50
Error Type: Atom Errors

Causes:
• Java Virtual Machine runs out of heap space.
• Atom runs out of memory.
• Atom runs out of write-able disk storage.
• Intentional or unintentional server restart.

51
Error Type: Recoverable & Unrecoverable

Recoverable:
• Errors which can be trapped and which do not prevent further processing.
• Try/Catch is a Boomi shape which allows for trapping this type of error and continuing the process.
• The vast majority of errors in Boomi processes are recoverable.

Unrecoverable:
• Errors which can not be handled through Boomi or custom code mechanisms.
• The process halts and some error condition information may be available.
• Unrecoverable errors are usually associated with Atom issues.

52
Errors: Common Characteristics

All errors…
• Are written to the process log.
• Display as “red” in Process Reporting.
• Display error messages in Process Reporting.
• Create an event in the account Event Log and may send alert emails to subscribed users.
• Cause a document or the entire process to stop normal processing.
• Do not automatically retry, alert or notify unless the developer has designed or provided for these
actions.
• May be recovered using one of several techniques which can result in further processing of the
document or process.
• Can be handled via Try-Catch, and then do not appear in process reporting as errors.

53
 Participants to Complete:

Error Handling Activity

Participants
to Complete

Page: 20-24

54
 Error Handling Techniques

55
Try/Catch

What is it?

• The Try/Catch shape captures document-level errors for one or more documents that fail during an
execution.
• Try/Catch can also trap Process errors.
• The goal of the Try/Catch step is to prevent process-level failures in the event that a document fails so
that you can act upon it later in your process flow.
• Try/Catch can be set to retry documents with errors.
• Try/Catch is an important error-handling component that will handle all types of errors.

56
Try/Catch

How does it work?

• Must be placed before the main processing shapes in your

process.
• Sends documents down the "Try" path to be processed.
• Documents that fail are caught and sent down the "Catch"
path, no matter how far they’ve travelled.
• Multiple Try/Catch steps handle failures at the nearest
Try/Catch.

57
Try/Catch

How does it work? (continued)

• Catch executes after all documents have completed the

Try path.
• Allows the design of advanced logging and processing
for failed documents.
• Allows capture of the exception logged within the
original path and includes it in your messaging via a
parameter reference to a “Base” document property.

58
Try/Catch

When/how to use:
Setting the Try/Catch Failure Trigger field to
Document Errors.
The following errors are captured:
• Connectors that log errors for specific
documents when errors occur (e.g., if an
update operation fails because the
document has an invalid ID)
• An Exception step that has the Stop Single
Document check box turned on
• A Map step, if it does not produce any
output, generates a validation error or
generates a connector error.

59
Try/Catch

When/how to use
Setting the Try/Catch Failure Trigger field to All
Errors will catch process errors as well as
document errors.
The following errors are:
- NullPointerExceptions and similar process-
level errors
- An Exception step that has the Stop Single
Document check box turned off
- Connectors that cause a full exception error
to be thrown back to the process instead of
capturing the error internally and logging it per
document.

60
Avoiding Errors: Validation Techniques

Decision
Conditionally sends documents down True
or False path based upon the comparison
of two values.
Values for Decision shape can be:
 Field values in the current document
(profile elements)
 Static values
 SQL Statement
 Document or Process Properties
 Database or application queries
 Document Cache lookup
 Stored Procedure

61
Avoiding Errors: Validation Techniques

Route
Conditionally sends documents down user
defined paths based upon the value of a
user specified field. Documents not
matching the field will be sent down the
Default path.
Route by options:
 Field values in the current document
(profile elements)
 Static values
 SQL Statement
 Document or Process Properties
 Stored Procedure

62
Avoiding Errors: Validation Techniques

Business Rules
Conditionally sends documents down
Accepted or Rejected paths based upon
advanced logic and validation rules.
Business Rules can contain multiple rules
and includes a customized Error Message.
Business Rules Inputs can be:
 Field values in the current document
(profile elements)
 Function
 Static values

63
Avoiding Errors: Validation Techniques

Cleanse
Conditionally sends documents down
Clean or Rejected paths based upon
document’s profile restrictions for
validation. Cleanse can repair the
document fields so that they pass
validation.
Cleanse Profile Tree can be:
 Field values in the current document
(profile elements)

64
Avoiding Errors: Validation Techniques

Multiple Customized Error Two Simple

Tests Message Branches Single Test Syntax Repair or Reject

Business Rule
P P P
Decision
P P P
Cleanse
P P P
Route
P
65
Technique: Exception Shape

Exception

- Able to terminate the data flow down a path

- Able to define custom error messages
- Often used when document data fails to meet certain conditions of a Route or Decision step
and should not be processed further.
Custom error messages
- Can be a mix of static and dynamic content.
- Dynamic content is populated using parameters.
- You can use multiple parameters when creating the message.

66
Technique: Exception Shape

Unchecked, stops all

documents and
functions as a
Process Error

Message is
returned to logs
and events

Parameters are
inserted into
brackets in
Message in order

67
Technique: Notify Shape

Notify
The Notify shape enables you to build custom execution logs and/or send customized
messages to your subscribed email alerts or RSS feed.

- These notifications can be either static messages or include dynamic content from
parameters.
- Creates one or more notifications at the document level for each process execution.
- Data is aggregated into a single message, then sent to the user's email or RSS feed after
process completion.
- It is used within a process flow and does not alter the document data.

68
Technique: Notify Shape

Level sets the message

level in the Log Enable Events makes the
Notify visible in the
account event log

Message is built with

static text and dynamic
parameter references
Enable User Log writes
Parameter values are Notify to local atom file
inserted into curly
brackets in Message

When checked all

notifications summarized
into a single entry

69
 Error Handling
Best Practices

71
Error Handling: Key Objectives

• Ensure correct data is transferred

• Minimize process interruption
• Quickly resolve problems
• Ensure successful recovery
• Maintain data integrity
• Communicate status
• Appropriate effort: don’t over-invest
in error handling

72
Error Handling: Common Scenarios and Techniques to Consider
Assured
Auto App Delivery Temp Try/ Targeted
Retry Logging Response Validation Techniques Files Catch Notification
Business
Critical P P
Unreliable
Connection P P
Complex
Recovery P P
Transient Data
Source P P
Unreliable Data
Quality P P
Scalability
Issues P P
Targeted Users
P P P P
73
Error Handling: Default Option

What if you implement nothing extra?

Pros:
No additional development and testing needed
Users with Boomi Alerts configured get document and process errors
Keeps processes simpler, reduces development effort

Cons:
Business users (who can fix data errors) don’t get notified – IT operations is in the middle of every error
Lack of selectivity – everyone gets alerted on every error on every process

Appropriate if data errors are not a problem and the only errors that occur are connectivity errors that the
IT Operations team must address

74
Error Handling: Strategic Questions
Different processes require different recovery techniques:

What are the different types of errors that might be encountered?

How likely and severe are the errors? Implications of no action?
What is the recovery process?
Can the process be re-run?
Is the source data easily available for re-run?
Are there partial update or corruption issues?
What will the recovery team need?
Details of the error
Copy of source documents?
“Rerun” version of the process
Documentation or recovery steps?
How will the process store or save resources needed for recovery?

75
 Participants to Complete:

Optional Extra Error Handling Activity

Participants
to Complete

Page: 25-29

76
 Congratulations!

77
Copyrights and Trademarks
This guide contains proprietary information protected by copyright and/or other legal grounds. The software described in this guide is furnished under a software license or nondisclosure
agreement. This software may be used or copied only in accordance with the terms of the applicable agreement. No part of this guide may be reproduced or transmitted in any form or by
any means, electronic or mechanical, including photocopying and recording for any purpose other than the purchaser’s personal use without the written permission of Boomi, Inc. (“Dell
Boomi”).

The information in this document is provided in connection with Dell Boomi products. No license, express or implied, by estoppel or otherwise, to any intellectual property right is granted by
this document or in connection with the sale of Dell Boomi products. EXCEPT AS SET FORTH IN THE TERMS AND CONDITIONS AS SPECIFIED IN THE LICENSE AGREEMENT FOR
THIS PRODUCT, DELL BOOMI (TOGETHER WITH DELL INC. AND ITS DIRECT AND INDIRECT SUBSIDIARIES) ASSUMES NO LIABILITY WHATSOEVER AND DISCLAIMS ANY
EXPRESS, IMPLIED OR STATUTORY WARRANTY RELATING TO ITS PRODUCTS INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTY OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. IN NO EVENT SHALL DELL BE LIABLE FOR ANY DIRECT, INDIRECT, CONSEQUENTIAL, PUNITIVE, SPECIAL
OR INCIDENTAL DAMAGES (INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF PROFITS, BUSINESS INTERRUPTION OR LOSS OF INFORMATION) ARISING OUT OF
THE USE OR INABILITY TO USE THIS DOCUMENT, EVEN IF ANY OF THEM HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. Dell Boomi makes no representations
or warranties with respect to the accuracy or completeness of the contents of this document and reserves the right to make changes to specifications and product descriptions at any time
without notice. Dell Boomi does not make any commitment to update the information contained in this document.

If you have any questions regarding your potential use of this material, contact:

Boomi, Inc.
Attn: LEGAL Dept.
[email protected]

With a copy to:

Boomi, Inc., Legal Department, 1400 Liberty Ridge Drive, Chesterbrook, PA 19087

Trademarks
Copyright © 2017 Boomi, Inc. All rights reserved. Dell, the Dell logo, Dell Boomi, Boomi, AtomSphere, Atom, and AtomSphere Integration Cloud are trademarks of Dell Inc. and/or its
subsidiaries in the United States and/or other countries. Other trademarks and trade names may be used in this document to refer to either the entities claiming the marks and names or their
products.

79