Context Management Options

Comparison

0
Table of contents
Document Properties ............................................................................................... 2
Metadata............................................................................................................................................ 2
Version History ................................................................................................................................... 2
Due Diligence ..................................................................................................................................... 2

Context Variables Comparison ................................................................................. 3

Introduction ....................................................................................................................................... 4
Context Variables within Talend Studio <Contexts> tab .................................................................... 4
Context Varaibles using file configuration <local file> ....................................................................... 4
Context variables using database configuration <database table> ................................................... 6
Context Varaibles using <Connections> ............................................................................................. 7
Context variables using <Resources> ................................................................................................. 8

About Talend ......................................................................................................... 10

1 Context Management Options Comparison

Document Properties
Please provide the following document properties so we can classify this document appropriately. (Note this page should be deleted before this
document gets published on Community or Academy)

Metadata
Property Value Notes

Author Alexandre Peng

MBO Sponsor Sunil Jagnani

MBO JIRA PSMBO-439 Enter MBO JIRA number

Document status Draft Choose one of – Draft, Review, Published, Deprecated

Choose one of the following – API Services / Automation / Governance

Product Category Integration
/ Integration

Product (s) and version(s) Talend Cloud Which product(s) does this document refer to (Talend or Third Party)

Target Medium Talend Community Document to be published to – Wiki / Community or Talend Academy

Content Type Article Knowledgebase Article – captures a piece of useful knowledge

Content Format DOCX

Version History
Date Updated By Changes made / Signed off

15/02/2021 Alexandre Peng First draft version

09/03/2021 Sunil Jagnani Reviewed by sponsor

12/03/2021 Alexandre Peng Reviewed by member of Delivery Innovation

<date> <technical enablement> Reviewed by members of Technical Enablement and good to publish

Due Diligence
Question Answer

• Talend Customers
Who is this for? Target Audience? • Talend Professional Services Consultants, Talend Support staffs
• Talend Technology Implementation Partners
Why create this document? • This information describes a real-life use-case

2 Context Management Options Comparison

3 Context Management Options Comparison
Context Variables Comparison
Introduction
A context variable is a variable which is defined by the user for a particular context. Depending on the circumstances the Job or Route is being used
in, you might want to manage it differently for various execution types, known as contexts (Prod or Test).

In the on-Premise environment, Talend provides three options for managing context variables:

1. Context variables within Talend Studio <Contexts> tab

2. Context variables Variables using file configuration <local file>
3. Context variables using database configuration <database table>
Talend cloud provides two additional options:

4. Context variables using <Connections>

5. Context variables using <Resources>

Context Variables within Talend Studio <Contexts> tab

There are two possible ways to create context.

1. Create “built-in” contexts. The context is hard coding within a job and cannot share between jobs.
• Open a job, select Contexts table.
• Add context variables, add context environment.
2. Create a “Repository Contexts”. A repository context is attached to a context group. It can be used across multiple jobs.
• Create a context group in Repository contexts.
• Add context variables, add context environment.
• Add context group to the job.
PROS CONS

Easy to create, change and manage Context is hard-coded in the Job itself

Allows you to automatically fill in the default contexts in the job Password can be visible for developers
depending on the environment when publishing to Talend Cloud

Repository context can be shared between jobs Password needs to re-enter when publishing to Talend Cloud

Value of repository context’s changes can be propagated Additional context value couldn’t propagate to the job, need to add
automatically to jobs manually to the job

Repository context can be exported alone or within the job

Developers can run the testing in the local environment(default)

Context Varaibles using file configuration <local file>

Load a context from files using tContextload which modifies dynamically the values of the active context.
The component tContextload performs two controls. It alerts when the parameters defined in the incoming flow are not defined in the context, and
the other way around, it also alerts when a context value is not initialized in the incoming flow. But this does not block the processing.

4 Context Management Options Comparison

There are two ways to create file configuration context.

1. Implicit file context load.

Setup from “Project Settings” Select dedicated project you want to apply Implicit Context Load
Settings

Or define within each job in Job tab.

2. Explicit file context load.

The component tContextLoad can be used within a job or a joblet.

• Drop a tFileInputDelimited component and a tContextLoad component from the Palette onto the design workspace.
• Link them using a Row > Main connection to form the first subJob.

PROS CONS

Context file is stored in the customer’s server and managed by the A subjob tContextLoad in the beginning of each job
customer

Context file can be protected by access rights(read only) Need a file system/servers to manage context file

Password could be encrypted in the context file and decrypted in the Context file cannot be restored if deleting mistakes
job

5 Context Management Options Comparison

 Need to manage as many context files as the number of contexts

Cannot use context files with Cloud Engine

Additional context value couldn’t propagate to the job, needs to add

both in the file and job context

Context variables using database configuration <database table>

Using the same way as the file configuration. Two methods possible creating database configuration context.

1. Implicit database context load.

Setup from “Project Settings” Select dedicated project you want to apply Implicit Context Load
Settings

2. Explicit database context load.

The component tContextLoad can be used in a job or in a joblet.

• Drop a tDBInput component and a tContextLoad component from the Palette onto the design workspace.
• Link them using a Row > Main connection to form the first subJob.

PROS CONS

Context table is stored in the customer’s database and managed by Need a subjob of tContextLoad in the beginning of each job
the customer

Context table is protected by access rights Need a database dedicated to manage context table

Password variable could be encrypted in the context table and Additional context value couldn’t propagate to the job, needs to add
decrypted in the job both in the table and job context

6 Context Management Options Comparison

 Context tables for different environments can be centralized in one Need to manage as many context tables as the contexts (If required for
database security purpose)

Context Varaibles using <Connections>

Connection parameters enable you to share configuration settings with web users by allowing them to select the connection to use when running
Tasks in Talend Cloud Management Console.
Connections can be created from the MANAGEMENT page or during task creation.

Define connection parameters under the Contexts node in the Repository tree view following the pattern
connection_<application_name>_<parameter_name>, where:

• connection_ : fixed prefix

• <application_name> : variable for the name of the system to which you want to connect
• <parameter_name> : variable for the name of the connection parameter

Create context group Create a connection

Add connection parameters

7 Context Management Options Comparison

 PROS CONS

The connection is dedicated for each environment, developers cannot Only certain roles can create connections in Talend Cloud (Talend Cloud
use QA/Prod connection to run from Studio Admin role)

Password is invisible in connection properties Can’t run from Studio with connections defined in Cloud

Useful in the case where customer has only Cloud Engine and Can’t create a custom connection with the same application name as a
wouldn’t store context values from Studio native connection,
If not all the custom parameters previously defined will lose and
resetting the existing parameter values

Additional value will auto-propagate to the tasks that include the Connections can not be used across workspace
connection

Connections can be managed by Talend Public API(GET)

Context variables using <Resources>

Talend Cloud Management Console can be used as a cloud data source to store files and directories to use within your task, just as Talend
Studio artifacts can interact with resources from your file system.

Resource parameters follow the pattern all in lower case:

resource_<parameterType>_<parameter_name> where:
• resource_ : fixed prefix
• <parameterType> : name of the external resource
• <parameter_name> : name of the resource parameter

There are 3 types of resource parameters :

• resource_file_<parameter_name> and resource_directory_<parameter_name>
Use these parameters if the Job needs a file or a directory to be executed. The rules for using external resources:
o The resource size limit per file is 100Mib with a 5% tolerance.
o Total size of resources per account is 10GiB.
o Directories are imported as .zip archives.

• resource_flow_temp_folder
Use this parameter if the web user needs to have a temporary folder during the execution of the Task in the web application.

• resource_webhook_payload
Define a webhook context parameter in a Job to allow the web user to generate unique URLs of tasks from Talend Cloud Management
Console.

Create a context file Upload to TMC

8 Context Management Options Comparison

 Create a context group Use tContextLoad

PROS CONS

The resource is dedicated for each environment, developers cannot Only certain roles can create connections in Talend Cloud (Talend
use QA/Prod resource to run from Studio Cloud Admin role)

Can use webhook context parameter Resources can not be used across workspace

Useful in the case where customer has only Cloud Engine and wants Can’t run from Studio if resources are defined in Cloud
load context from the file

Password context could be encrypted in the resource file and Additional context value cannot add directly from resource file.(delete
decrypted in the job and replace resource file)

Resources files can be versioned in Git Resource file size is limited by 100Mib

Managed by Talend, don’t use customer’s resource

Resources can be managed by Talend Public API(GET)

9 Context Management Options Comparison

About Talend
Talend, a leader in data integration and data integrity, enables every company to find clarity amidst the chaos.

Talend Data Fabric brings together in a single platform all the necessary capabilities that ensure enterprise data is complete, clean, compliant, and
readily available to everyone who needs it throughout the organization. It simplifies all aspects of working with data for analysis and use, driving
critical business outcomes.

From Domino’s to L’Oréal, over 4,250 organizations across the globe rely on Talend to deliver exceptional customer experiences, make smarter
decisions in the moment, drive innovation, and improve operations. Talend has been recognized as a leader in its field by leading analyst firms and
industry publications including Forbes, InfoWorld and SD Times.

Talend is Nasdaq listed (TLND) and based in Redwood City, California.

For more information, please visit www.talend.com and follow us on Twitter: @Talend.

10 Context Management Options Comparison

11 Context Management Options Comparison