Lab Guide - PDF - EN
Lab Guide - PDF - EN
Lab Guide - PDF - EN
TDM Essentials
Version 6.1
Copyright 2016 Talend Inc. All rights reserved.
Information in this document is subject to change without notice. The software described in this document is furnished under a license
agreement or nondisclosure agreement. The software may be used or copied only in accordance with the terms of those agree-
ments. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or any means electronic
or mechanical, including photocopying and recording for any purpose other than the purchaser's personal use without the written
permission of Talend Inc.
Talend Inc.
800 Bridge Parkway, Suite 200
Redwood City, CA 94065
United States
+1 (650) 539 3200
Welcome to Talend Training
Congratulations on choosing a Talend training module. Take a minute to review the following points to help you get the most from
your experience.
Technical Difficulty
Instructor-Led
If you are following an instructor-led training (ILT) module, there will be periods for questions at regular intervals. However, if you
need an answer in order to proceed with a particular lab, or if you encounter a situation with the software that prevents you from pro-
ceeding, don’t hesitate to ask the instructor for assistance so it can be resolved quickly.
Self-Paced
If you are following a self-paced, on-demand training (ODT) module, and you need an answer in order to proceed with a particular
lab, or you encounter a situation with the software that prevents you from proceeding with the training module, a Talend Support
Engineer can provide assistance. Double-click the Live Expert icon on your desktop and follow the instructions to be placed in a
queue. After a few minutes, a Support Engineer will contact you to determine your issue and help you on your way. Please be con-
siderate of other students and only use this assistance if you are having difficulty with the training experience, not for general ques-
tions.
Exploring
Remember that you are interacting with an actual copy of the Talend software, not a simulation. Because of this, you may be tempted
to perform tasks beyond the scope of the training module. Be aware that doing so can quickly derail your learning experience, leaving
your project in a state that is not readily usable within the tutorial, or consuming your limited lab time before you have a chance to fin-
ish. For the best experience, stick to the tutorial steps! If you want to explore, feel free to do so with any time remaining after you've fin-
ished the tutorial (but note that you cannot receive assistance from Tech Support during such exploration).
Additional Resources
After completing this module, you may want to refer to the following additional resources to further clarify your understanding and
refine and build upon the skills you have acquired:
Talend product documentation (help.talend.com)
Talend Forum (talendforge.org/)
Documentation for the underlying technologies that Talend uses (such as Apache) and third-party applications that com-
plement Talend products (such as MySQL Workbench)
This page intentionally left blank to ensure new chapters
start on right (odd number) pages.
LESSON 9 Refactoring
Overview 148
Lesson Overview 148
Objectives 148
Next Step 148
Upgrading the Output Inheritance 149
Next Step 151
Upgrading the Output Mapping 152
Next Step 154
Upgrading the Input Inheritance 155
Next Step 159
Upgrading the Input Mapping 160
Next Step 163
Wrap-Up 164
Next Step 164
Overview 10
Creating a Structure 11
Creating a Map 20
Wrap-Up 28
Overview
Lesson Overview
This lab is a simple exercise to learn the basics of Structure management and how to map elements between a source and a des-
tination.
Imagine you are requested to transform an XML file into another XML file with similar structure, removing some nodes to reduce the
output file size.
Objectives
After completing this lesson, you will be able to:
Create a new Structure from an existing local XML file
Explore a Structure and interactively display a sample of it
Create a new Map
Use existing Structures to specify the Input and Output areas of a Map
Automap the elements whose names match in both the Input and Output areas of a Map
Remove some elements from the output
Test Run the Map and save the results to an external file, or display them in the internal viewer
Next Step
The first step is to import an existing XML structure.
3. Click the Start Now button if the Welcome screen shows up. Otherwise, go to the next step.
LESSON 1 | 11
2. The Mapping perspective has two different views by default: Functions and Data Mapper. Select the Data Mapper tab.
LESSON 1 | 13
Selecting the Type of Import
1. Click XML Sample Document and click Next >.
Note: When an XSD schema is available, it should be used instead of the actual XML document. An XSD schema contains
the full definition of the XML document, while the XML document could implement only a part of the original schema.
LESSON 1 | 15
Displaying the Structure
1. Expand the Hierarchical Mapper > Structures node and double-click POXML to display the contents of the Structure:
2. Click the Browse... button and select the C:\StudentFiles folder to display its contents.
3. Select the POs.xml file and click Finish.
LESSON 1 | 17
4. Once the Sample Document has been selected, the actual data will appear in the Document tab. In addition, selecting a spe-
cific node in the Structure will highlight the matching data in the preview.
LESSON 1 | 19
Creating a Map
1. Right-click Hierarchical Mapper > Maps and select New > Map.
In this example, the POXML Structure is used for both the source and the target. In a more general use case, two different
Structures would be used for the mapping.
5. Drag PO (1:*) from the Input area to the matching group level on the Output area. Doing so will perform an automatic map-
ping on all fields whose names match. In this case, as the input and output structure are the same, all the fields will be
mapped to each other. Now let's remove some elements from the output.
LESSON 1 | 21
6. Select the Urgency node in the Output area, right-click it and select Remove Expressions (Mappings) to remove the
mappings associated with this particular element. It means the node will not be part of the output.
7. Note that the Urgency node no longer appears in bold in both the Input and Output areas, to indicate it is no longer
mapped. Also, selecting the PO (1:*) node in the Output area clearly shows one of the red arrows disappeared.
9. Save the Map by hitting Ctrl+S or clicking the floppy icon in the toolbar.
10. Expand Test Run then click Test Run to External File.
11. Browse toC:\StudentFiles then type POXML.xml in the File name: field and click Save.
LESSON 1 | 23
12. The Test Run results will be saved to the POXML.xml file specified in the previous step.
13. Open C:\StudentFiles\POXML.xml with Notepad++ (pre-installed on the Virtual Machine). It should be equal to the original
POs.xml file, minus the Urgency and Vendor children that now appear as empty nodes.
15. This time, the results are displayed in the internal XML viewer.
LESSON 1 | 25
16. Right-Click Delivery in the Output (XML) area and select Test Run.
17. Note that only the Delivery level and everything below is displayed.
LESSON 1 | 27
Wrap-Up
This first lab served as a general introduction to Talend Data Mapper.
In this lesson, you learned how to:
Create a new Structure from an existing local XML file
Explore a Structure and interactively display a sample of it
Create a new Map
Use existing Structures to specify the Input and Output areas of a Map
Automap the elements whose names match in both the Input and Output areas of a Map
Remove some elements from the output
Test Run the Map and save the results to an external file, or display them in the internal viewer
Next Step
Congratulations, you successfully completed this lesson. Click the Check your status with this unit button below in order to save
your progress. Then click Completed. Let's continue > on the next screen to jump to the next lesson.
Overvieww 30
Preparation 31
Creating a New Map 32
Testing the Map 42
Adding a Filter 47
Wrap-Up 52
Overvieww
Lesson Overview
This lab takes the previous example a step further and explains the simple looping concept. It also shows how to sort and filter the out-
put values of a Map.
Objectives
After completing this lesson, you will be able to:
Add a Simple Loop manually for elements that can appear multiple times as children of a common parent node
Sort an Output based on the value of a given Input field
Filter an Output based on the value of a given Input field
Change the representation and type of a given field
Next Step
First, let's set up and prepare the required Structures.
2. Right-click the main Hierarchical Mapper > Structures folder and select Paste.
Next Step
With the example Structures copied to the workspace, let's create a new Map.
LESSON 2 | 31
Creating a New Map
1. Right-click Hierachical Mapper > Maps and select New > Map.
LESSON 2 | 33
34 | TDM Essentials - Lab Guide
Defining the Mapping
1. Map the elements as illustrated below by drag and dropping from the Input area to the Output area.
2. By default, only the active mapping is displayed on screen. To display all mappings at the same time as illustrated above,
right-click the grid between the Input and Output areas and select Preferences.
LESSON 2 | 35
3. In the Mapping > Map Linessection, select All Elements - Show lines connecting all visible elements.
LESSON 2 | 37
These Simple Loops indicate that the Notices node can have multiple Notice children. In turn, each Notice node can have
multiple Items children. Talend Data Mapper creates Simple Loops automatically when you drag and drop a looping ele-
ment to another looping element. The Map will iterate over all available children from the Input area to create a list in the
Output area. It is also possible to create loops manually when needed. To illustrate this, let's remove one of the Simple Loop
and create it back manually.
LESSON 2 | 39
Adding Back the Simple Loop
1. Make sure the Items (0:*) is still selected.
2. Drag a SimpleLoop element from the Functions tab on the left and drop it in the Loop tab.
Next Step
The mapping is done, and Simple Loops were added automatically or manually. Now let's test the Map.
LESSON 2 | 41
Testing the Map
LESSON 2 | 43
2. Check the result looks like the following:
4. Click Test Run and check the result looks like the following:
LESSON 2 | 45
Note that Quantity is a string field, so the sort order highlighted above is actually correct. 1000 is smaller than 500 when rep-
resented as a string.
Next Step
The output is sorted by ascending quantity. Now let's add a filter in order to remove the entries whose Quantity value is less than
1000.
3. Drag the Quantity node from the Input area to the First Value field of the Lesser function.
4. Drag a Functions > General > Constant function to the Second Value field of the Lesser function. Double-click it, enter
1000 for the Value and click OK.
LESSON 2 | 47
5. Click Test Run and check the result looks like the following:
LESSON 2 | 49
7. Change the Structure Element from Read Only to Editable.
8. Set the Data Type field to Integer (32) (instead of String by default).
9. Save your changes to MandPPOFile by hitting Ctrl+S or clicking the floppy disk icon in the toolbar (on the far left). Then go
back to the Simple_Looping Map.
10. Repeat steps 6-9 for the Quantity element in the Output area.
11. Click Test Run and check the results are actually different. For example, there is now a Notice element with a Quantity
value of 500, as 500 is considered less than 1000 when represented as a number. It was not the case when both values
were stored as strings.
LESSON 2 | 51
Wrap-Up
In this lesson, you learned how to:
Add a Simple Loop (automatically and manually) for elements that can appear multiple times as children of a common par-
ent node
Sort an Output based on the value of a given Input field
Filter an Output based on the value of a given Input field
Change the representation and type of a given field
Next Step
Congratulations, you successfully completed this lesson. Click the Check your status with this unit button below in order to save
your progress. Then click Completed. Let's continue > on the next screen to jump to the next lesson.
Overview 54
Creating a New Map 55
Testing the Map 63
Wrap-Up 65
Overview
Lesson Overview
This lab goes beyond the Simple Loop concept introduced in the previous exercise. It introduces the Nested Looping concept
Objectives
After completing this lesson, you will be able to:
Set up an Nested Loop (automatically and manually) for elements that require several layers of looping to be accessed
Next Step
The first step is to create a new Map.
2. Drag Hierarchical Mapper > Structures > Simple > POStandard > POFile to the Input area.
3. Drag Hierarchical Mapper > Structures > Simple > POStandard > SimpleShipNoticeFile to the Output area.
LESSON 3 | 55
4. Map the Number element to the Num element.
5. Select the Notice (0:*) element in the Output area. Notice that the SimpleLoop element was added automatically.
LESSON 3 | 57
This Change Loop Specification pop-up window appears to warn you that the new mapping is breaking the original
Simple Loop on Notice(0:*). The original loop is no longer compatible because it needs to loop over PO and then loop again
over Delivery to get to the DeliverToName element. Clicking Yes will add a Simple Loop inside the existing Simple Loop,
resulting in what is called a Nested Loop (because two Simple Loops are nested in each other).
7. To check that this Nested Loop was indeed created automatically, select the Notice(0:*) element again. Notice the new Sim-
pleLoop element in the Contexts > NestedContext branch:
LESSON 3 | 59
Removing the Nested Loop and Adding It Back Manually
1. To remove the SimpleLoop over the Notice (0:*)element, right-click the outermost SimpleLoop and select Delete:
2. Drag a Functions > Loop > SimpleLoop function and drop it inside the Loop tab to recreate the outermost
Simple Loop.
3. Drag and drop the PO (0:*) element to the Input Map Element field of the SimpleLoop:
5. Drag a second Functions > Loop > SimpleLoop function under the NestedContext just created:
LESSON 3 | 61
6. Finally, drag Delivery (0:*) to the Input Map Element field of the nested SimpleLoop:
7. The result is identical to the automatic Nested Loop created when you clicked Yes in the pop-up dialog.
Next Step
The mapping is done, and Nested Loops were added automatically or manually. Now let's test the Map.
LESSON 3 | 63
5. Check the results look like the following:
Next Step
This lesson is almost over. Head to the Wrap-Up section for a summary of the concepts reviewed in this lesson.
Next Step
Congratulations, you successfully completed this lesson. Click the Check your status with this unit button below in order to save
your progress. Then click Completed. Let's continue > on the next screen to jump to the next lesson.
LESSON 3 | 65
This page intentionally left blank to ensure new chapters
start on right (odd number) pages.
LESSON 4
Aggregation Looping
This chapter discusses the following.
Overview 68
Aggregation Looping 69
Wrap-Up 77
Overview
Lesson Overview
This lab extends the Simple and Nested Loops exercises and presents the concept of Aggregation Looping.
Objectives
After completing this lesson, you will be able to:
Set up an Aggregation Loop to count the number of elements of a given entity in the Output
Next Step
The first step is to create a new Map.
2. Drag Hierarchical Mapper > Structures > Simple > POStandard > POFile to the Input area.
3. Drag Hierarchical Mapper > Structures > Simple > POStandard > SimpleShipNoticeFile to the Output area.
LESSON 4 | 69
4. Map the following elements, click Yes in Change Loop Specification dialogs when needed and check the loop expres-
sion for Notice (0:*) at the end. Note that a Nested Loop was created automatically for you.
2. Drag the Items(0:*) > ItemNumber element from the Output area to the AgCount function. A SimpleLoop expression
LESSON 4 | 71
will be created automatically based on the selected element:
2. Review the results. Each Purchase Order should be associated with two Items.
LESSON 4 | 73
Setting Up Aggregation for Total Notices
Let's apply the same principle to count the total number of Notices per document.
1. Set up the aggregation by selecting the TotalItems node in the Output area. Then drag a Functions > Aggregate >
AgCount function to the Value tab.
2. Drag the Notice(0:*) > Num element from the Output area to the AgCount function. A SimpleLoop expression will be cre-
Testing
1. Right-click Notices in the Output area and select Test Run.
2. Review the results. There should be 3 Notices in the XML output.
LESSON 4 | 75
Next Step
This lesson is almost over. Head to the Wrap-Up section for a summary of the concepts reviewed in this lesson.
Next Step
Congratulations, you successfully completed this lesson. Click the Check your status with this unit button below in order to save
your progress. Then click Completed. Let's continue > on the next screen to jump to the next lesson.
LESSON 4 | 77
This page intentionally left blank to ensure new chapters
start on right (odd number) pages.
LESSON 5
Using More Functions
This chapter discusses the following.
Overview 80
Using a Concatenation Function 81
Using an If/Then/Else Condition 85
Wrap-Up 91
Overview
Lesson Overview
After duplicating the Lab01_XML2XML Map created in a previous exercise, you will learn how to use different types of Functions,
such as Concatenation and If-Then-Else logic.
The Concatenation Function will be used to add the word Company to the Company input node. For example, "Fargo" will turn
into "Fargo Company".
The If-Then-Else Function will be used to make decisions based on the value of one or several existing nodes. For example, the
value of the Numbernode will be used to decide whether to add " Company" or " CO." to the Company input node.
Objectives
After completing this lesson, you will be able to:
Duplicate an existing Map
Use a Concatenation Function to alter the output of a Map, like adding a " Company" suffix to the Company input node
Use an If-Then-Else Function to make decisions based on the value of one or several existing nodes. For example, you used
the Number value to decide whether to add " Company" or " CO." to the Company input node.
Next Step
Let's start with using a Concatenation Function to add the Company word to the Company field.
LESSON 5 | 81
3. Open the new AddFunction_XML2XML Map.
4. Right-click the output element Company then click Remove Expressions ( Mapping).
5. Notice the Value tab is now empty for this particular node.
6. Drag a Functions > General > Concat Function and drop it to the Value tab:
8. Now let's add the word " Company" to the Concatenation expression. Drag a Functions > General > Constant Function
to the Value tab, below the Company element (until a black horizontal line appears) and then drop it:
9. Double-click the Constant Function in the Value tab. Enter Company in the Value field and click OK
LESSON 5 | 83
Note the space before Company. Without this extra space at the beginning, "Fargo" would turn into "FargoCompany"
instead of the desired "Fargo Company".
10. Perform a Test Run on the Company element in the Output area and examine the output.
11. Notice that each Company element ends with " Company":
Next Step
Now imagine you want to use a Company suffix for the first PO only, and a Co. suffix for every other PO. This can be achieved by
adding an If/Then/Else condition in order to change the behavior of the Concatenation Function based on the PO Number value.
Note that the Concat Function is under the Condition branch by default. Let's move it to the Then branch, as Concat is
the action to be performed rather than the test itself.
2. Select the Concat function and drag it directly on top of the Then branch:
3. Drag a Functions > Comparison and Logical > Equal Function to the Condition branch:
LESSON 5 | 85
Now let's define and apply the actual test. It can be expressed as the following pseudo code:
if Number == 1 then
concatenate with ' Company'
else
concatenate with ' Co.'
4. Drag the Number element from the Input area to the First Value field of the Equal branch.
6. Double-click the Constant Function to edit its value, enter 1 and click OK.
7. To make things easier and avoid redefining a Concat action from scratch in the Else branch, right-click the Concat function
from the Then branch and click Copy:
LESSON 5 | 87
8. Right-click the Else branch and click Paste:
9. Double-click the Constant in the Else branch, enter Co. (with a leading space) and click OK.
11. Right-click Company in the Output area and select Test Run.
12. Check that the first PO ends with Company, and the second one ends with CO.
LESSON 5 | 89
Next Step
This lesson is almost over. Head to the Wrap-Up section for a summary of the concepts reviewed in this lesson.
Next Step
Congratulations, you successfully completed this lesson. Click the Check your status with this unit button below in order to save
your progress. Then click Completed. Let's continue > on the next screen to jump to the next lesson.
LESSON 5 | 91
This page intentionally left blank to ensure new chapters
start on right (odd number) pages.
LESSON 6
Using the tHMap Component
This chapter discusses the following.
Overview 94
Using the tHMap Component 95
Wrap-Up 108
Overview
Lesson Overview
The tHMap Component can be used in Data Integration (DI) Jobs to execute transformations between different sources and des-
tinations by harnessing the capabilities of Talend Data Mapper, available in the Mapping perspective.
Objectives
After completing this lesson, you will be able to:
Use the tHMap Wizard to create a Map interactively, from Schemas and Structures already available or imported manually
Next Step
Let's learn how to use the tHMap Component in the Integration perspective.
Importing Data
1. First, let's create some data to operate on.
Click Window > Perspective > Integration to switch to the Integration perspective and be able to create a Data Integ-
ration Job.
Note that you can also click the Integration button at the top right of the Studio to achieve the same result.
2. In the Integration perspective, right-click Metadata > File delimited and select Create file delimited.
LESSON 6 | 95
4. Click Browse..., change the extension filter to *.*, select C:\StudentFiles\item.out and click Next.
LESSON 6 | 97
6. Click Next to explore and validate the schema. Note that some of the fields are imported as Integers and then click Finish.
The imported file appears in the Metadata > File delimited branch of the Repository.
LESSON 6 | 99
6. Place a tHMap component onto the canvas and acknowledge the red exclamation mark stating that the Map is not found. It
has not been specified yet.
7. Right-click tFileInputRaw and select Row > Main. Drop the link on the tHMap Component:
9. Right-click tHMap and select Row > Main. Drop the link on the tFileOutputDelimited Component.
LESSON 6 | 101
10. Click Yes when asked whether to propagate the schema from the tHMap Component to the tFileOutputDelimited Com-
ponent.
LESSON 6 | 103
5. Step 2 of the wizard is about selecting an output Structure.
Keep the default Generate hierarchical mapper structure based on the schema option and click Next.
Note that you cannot perform a Test Run when the Structure has been created by a tHMap Wizard.
9. Save the tHMap.
LESSON 6 | 105
3. Use the Data Viewer to inspect the results. Alternatively, you can open C:\StudentFiles\item.out in Notepad++ and check
the results are the same.
LESSON 6 | 107
Wrap-Up
In this lesson, you learned how to:
Use the tHMap Wizard to create a Map interactively, from Schemas and Structures already available or imported manually
Next Step
Congratulations, you successfully completed this lesson. Click the Check your status with this unit button below in order to save
your progress. Then click Completed. Let's continue > on the next screen to jump to the next lesson.
Overview 110
Creating the DI Job 111
Using Context Variables 117
Wrap-Up 123
Overview
Lesson Overview
In this exercise, you will use an existing Map that writes from a COBOL file to an XML file. This Map will be called from a Data Integ-
ration (DI) Job to demonstrate the integration between Talend Data Mapper and the Integration perspective of Talend Studio. Con-
text variables will be used to pass values to the Job at runtime, so the output can take different values based on the context (a
development or production environment, for example).
Objectives
After completing this lesson, you will be able to:
Call a Talend Data Mapper Map from a DI Job using the tHMap Component
Use Contexts and Context Variables to change the value of the output at runtime
Next Step
Let's create a DI Job in the Integration Perspective.
Note that you can also click the Integration button at the top right of the Studio to achieve the same result.
2. Right-click Job Designs in the Repository on the left, then select Create Standard Job (or Create Job, depending on
the version of the Studio). Enter COBOL2XML in the Name field, fill in the Purpose and Descriptionfields as illustrated
below and click Finish.
LESSON 7 | 111
4. Double-click the tFileInputRaw component to display its Component view.
5. Click the ... button next to the Filename field. Select the C:\StudentFiles\cobol.dat file and set the Mode field to Read the file
as a bytes array.
9. Finally, place a tHMap component on the canvas. This component is responsible for calling a Talend Data Mapper Map.
Note that it needs to be connected to an input and an output before being configured.
Also note the red exclamation point indicating the Map is not found (yet). The actual Map to use will be specified later on.
10. Right-click tFileInputRaw, select Row > Main and drop the link on the tHMap component:
LESSON 7 | 113
11. Right-click tHMap, click Row > Main and drop the link on the tFileOutputRaw component:
12. Click Yes when asked to propagate the schema from the tHMap Component to the tFileOutputRaw Component.
13. In the tHMap Component view, click the … button to the right of Map Path.
14. From the Repository Content pop-up window, select Maps/COBOL2XML and click OK:
LESSON 7 | 115
Next Step
The DI Job is almost configured. Let's add some context variables.
2. Click the green + icon at the bottom left of the table to add a Context Variable. Set the Name column to environment.
3. Click the green + icon at the top right of the table to configure the available Contexts for the current job. By default, there is
LESSON 7 | 117
only one Default Context, so every Context Variable can have only one value. Adding additional Contexts will allow the same
Variable to have multiple values. The runtime Context selected in the Run Job view will determine which value should be
used. For example, let's add a Development Context and a Production Context.
Click New to create a new Context (i.e. a group of Context Variables), enter the name dev then click OK.
4. Click New again to add a second Context. Name it prod then click OK and OK again to close the Configure Contexts win-
dow.
This way, the environment Context Variable can be used to determine the current development stage. It can be used to gen-
erate file names and paths that depend on the Context.
Mapping Perspective
1. Switch back to the Mapping perspective, open the COBOL2XML Map and select the TRAILER > FILLER element in the
Output area.
2. Remove the existing /Root/TRAILER/FILLER value from the Value tab.
3. Drag a Functions > Special > GetMapProperty Function to the Value tab.
4. Double-click the getMapProperty Function and enter context.environment in the Property Name field.
This means the FILLER field will contain the value of the environment Context Variable.
Finally, click OK to save the changes.
LESSON 7 | 119
5. Check that the Value tab looks like the following.
6. Save the changes to the COBOL2XML Map by hitting Ctrl+S or clicking File > Save.
Integration Perspective
1. Switch back to the Integration perspective, open the COBOL2XML Job and go to the Run (Job COBOL2XML) view.
2. Select one of the available Contexts from the drop-down menu on the right ( Default, dev, or prod) before running the Job.
3. Run the Job by clicking the Run button, and wait for the Job to complete. A successful run will display the following message.
LESSON 7 | 121
Next Step
This lesson is almost over. Head to the Wrap-Up section for a summary of the concepts reviewed in this lesson.
Next Step
Congratulations, you successfully completed this lesson. Click the Check your status with this unit button below in order to save
your progress. Then click Completed. Let's continue > on the next screen to jump to the next lesson.
LESSON 7 | 123
This page intentionally left blank to ensure new chapters
start on right (odd number) pages.
LESSON 8
Using the cMap Component
This chapter discusses the following.
Overview 126
Creating a Route 127
Importing the Input XML Structure 131
Creating the Output JSON Structure 134
Creating the XML to JSON Mapping 138
Configure cMap and Testing the Route 141
Wrap-Up 145
Overview
Lesson Overview
In this exercise, you will configure a Route to perform an XML to JSON transformation using a cMap Component. This Component is
only available in the Mediation Perspective. The Route will listen for changes in an XML file containing a list of accounts, and will pro-
duce a JSON file containing a list of contacts for each account.
Objectives
After completing this lesson, you will be able to:
Create a JSON Structure by adding and configuring nodes manually. The same method can be applied to XML, COBOL...
Use the cMap Component in Routes in the Mediation Perspective. It demonstrates the integration between the various
Talend products. A Map can be reused in other perspectives/products.
Next Step
Switch to the Mediation Perspective and create a Route.
LESSON 8 | 127
4. Place a cMap Component to the right of the cFile.
5. Copy cFile_1 and paste it to the right of the cMap Component to create cFile_2.
6. Right-click cFile_1, select Row > Route and drop the route on top of cMap.
3. Configure cFile2 to drop a JSON file in the same Path ("C:/StudentFiles") with a fileName equal to "account_
LESSON 8 | 129
contacts.json".
Next Step
The cMap Component will be configured later in the exercise. Let's now define the Structures and Mappings using in Talend Data
Mapper.
2. Select the Import a structure definition, e.g. XML Schema, JSON, COBOL copybook, CSV file, etc. option and
click Next:
LESSON 8 | 131
4. Check Local file, click Browse...to select C:\StudentFiles\account.xml then click Next:
5. Name the new Structure Exercise_Account, click Next and then Finish:
LESSON 8 | 133
Creating the Output JSON Structure
1. Right-click Hierarchical Mapper > Structures and select New > Structure:
2. Check the Create a new structure where you manually enter elements option and click Next.
3. Select the TDM-Essentials > Structures folder and name the Structure Exercise_Contacts_JSON before clicking
Next:
5. Select Don't select a sample document for now and click Finish:
LESSON 8 | 135
Adding Elements
1. Right-click the design area and select New Element. Name it Root and set the Data Type to None (last item in the list):
2. Right-click the Root element and select New Elementto create a child node. Name it Account_Contacts.
5. Create three additional nodes under Contacts (1:*): Name, Email, and Phone.
Next Step
With the input and output Structures created, let's create the Map between them.
LESSON 8 | 137
Creating the XML to JSON Mapping
1. Right-click Hierarchical Mapper > Maps and select New > Map.
2. Select a Standard Map and click Next >:
6. Map Account > contact to Root > Account_Contacts > Contacts. Note that the mapping is done automatically for
nodes whose names match.
LESSON 8 | 139
9. The following JSON document should appear in the output window.
Next Step
With the Map completed, let's configure the cMap Component in the Route created in the first step.
4. Select the Exercise_Account_XML_to_Contacts_JSON Map created in a previous step and click OK.
5. It's time to test the "polling" effect of the cFile_1 Component. This Component is responsible for listening to changes in the
account.xml file, so let's remove it temporarily and restore it later on from the Recycle Bin to simulate a new file appearing all
of a sudden. Imagine this file is created by another application.
Delete C:\StudentFiles\account.xml to send it to the Recycle Bin.
LESSON 8 | 141
6. Click Run to run the Route and examine the canvas.
8. Examine the canvas again and note that the Route is no longer in the Starting state. It processed account.xml and created
the output JSON file.
LESSON 8 | 143
10. Finally, open C:\StudentFiles\account_contacts.json in Notepad++ to examine its contents.
Next Step
This lesson is almost over. Head to the Wrap-Up section for a summary of the concepts reviewed in this lesson.
Next Step
Congratulations, you successfully completed this lesson. Click the Check your status with this unit button below in order to save
your progress. Then click Completed. Let's continue > on the next screen to jump to the next lesson.
LESSON 8 | 145
This page intentionally left blank to ensure new chapters
start on right (odd number) pages.
LESSON 9
Refactoring
This chapter discusses the following.
Overview 148
Upgrading the Output Inheritance 149
Upgrading the Output Mapping 152
Upgrading the Input Inheritance 155
Upgrading the Input Mapping 160
Wrap-Up 164
Overview
Lesson Overview
This lesson focuses on refactoring best practices. Suppose you need to upgrade the Structure of an element to a new version of a
standard. The new standard is very similar tot the old one, but you still need to map all the elements again. What are the best prac-
tices for this kind of refactoring? This lab provides tips and tricks on how to perform this operation for two different Maps and the asso-
ciated Structures.
Objectives
After completing this lesson, you will be able to:
Change the inheritance and architecture of an existing Structure
Move the old mapping expressions to the new elements
Next Step
First, let's focus on upgrading the Output inheritance.
2. Expand the output Structure until the contents of the Loop-ISA (0:*)element are visible.
3. Further expand Loop-GS (0:*) > Loop-ST (0:*) > Transaction-270-A1 to reveal the Loop-HL Information Source
element.
4. Right-click Transaction-270-A1 and select Go to Structure Element to jump to the element definition.
LESSON 9 | 149
5. Change Read Only to Editable to be able to modify the definition of the element (type, size...).
6. Scroll down to the Inherits From field and click the ... button next to it to modify the inheritance of this specific node. Note
that the node currently inherits its architecture from the /X12_4010_HIPAA/Structures/Transactions/270-A1 Structure.
7. Select the X12_5010_HIPAA > Structures > Transactions > 270-B1 Structure and click OK to change the inheritance of
the current node.
Next Step
Now that the output Structure changed, let's also modify the mapping so it matches the new architecture.
LESSON 9 | 151
Upgrading the Output Mapping
1. Display the Map again and spend some time examining the changes and warnings introduced by the new inheritance.
2. Also read the warnings in the Mapper Problems tab at the bottom of the Studio:
3. Finally, notice a output element called Transaction-270-B1. Expand it and examine the new elements which do not have a
mapping associated with them (yet). Let's copy the old mappings to the new elements.
6. Notice that both the old and new elements are mapped.
7. Right-click on Transaction-270-A1 and select Delete to remove the node itself and all associated mappings. Then save
the Map.
LESSON 9 | 153
8. Finally, click Test Run to check the output.
Next Step
With the new mapping completed on the output side, let's modify the inheritance of the input Structure.
1. Expand the Loop-ISA > Loop-GS > Loop-ST element in the Input area until the Transaction-271-A1 node is visible.
2. Click Show Document to display some sample data and click OK to close the dialog.
LESSON 9 | 155
3. Click Test Run to generate the associated output and click OK again to close the dialog.
LESSON 9 | 157
6. Scroll down to the Inherits From field and notice the element currently inherits from the /X12_4010_HIPAA/Struc-
tures/Transactions/271-A1-ENV Structure.
7. Click the ... button next to the Inherits From field, select X12_5010_HIPAA > Structures > Transactions > 271-B1-ENV
Structure then click OK to change the inheritance of the Interchange element.
LESSON 9 | 159
Upgrading the Input Mapping
1. Display the Map again and notice the new Transaction-271-B1 element, with no mappings associated yet.
2. Examine the warnings in the Mapper Problems tab. Let's fix these warnings by moving the mappings from Transaction-
271-A1 to Transaction-271-B1.
5. Notice the new element is mapped correctly. Also note that the Move Expression References operation was much faster
than the copy/paste/delete sequence used on the output mapping.
LESSON 9 | 161
6. Right-click Transaction-271-A1 and select Delete Invalid Map Elements to remove all elements with wrong mappings.
LESSON 9 | 163
Wrap-Up
In this lesson, you learned how to:
Change the inheritance and architecture of an existing Structure
Move the old mapping expressions to the new elements
Next Step
Congratulations, you successfully completed this lesson. Click the Check your status with this unit button below in order to save
your progress. Then click Completed. Let's continue > on the next screen to jump to the next lesson.
Overview 166
Setting Up a Test Case 167
Wrap-Up 173
Overview
Lesson Overview
You will create a test case for the existing COBOL2XML Map, based on the default output. Then you will introduce a bug intentionally,
check that the test case no longer succeeds, fix the bug and check that everything is back on track.
Objectives
After completing this lesson, you will be able to:
Use the output of a given Mapping to define a reference test case
Run test cases for a given node, to make sure no feature was broken by accident
Next Step
Let's use an existing Map and create a test case from a reference output.
2. Right-click the Root node in the Output area and select Test Run.
3. The usual output window is displayed. Now, instead of clicking OK to close it, click Create Test Case in order to use the cur-
rent output as the output of a future unit test. Future outputs will be compared to this reference to check whether the trans-
formations performed successfully or not.
LESSON 10 | 167
4. The Test Cases tab shows up and displays a list of all saved Test cases.
6. Right-click Root again and select Execute Test Cases to run all test cases registered with the Root node.
LESSON 10 | 169
7. Examine the Test Execution tab at the bottom and note the progress bar is red, indicating a failure. It is confirmed by the
Failures: 1 message at the top right. It is the expected behavior, as we removed some of the mappings from the
output. Some nodes must be missing compared to the reference output saved earlier in the exercise.
Double-click out$/Root - Test1 to get some details about the failure.
8. The Expected and Actual results are displayed so it is possible to check exactly where the error occurred. As mentioned in
the error message at the top, the failure occurred in the COBOL2XML Map, in the Root/HEADER/FILE-ID output
node. The reference output has a value associated to this node, while the current output has a null value (indicating the node
does not exist). Let's fix this error and run the test cases again.
10. Right-click Root again and select Execute Test Cases to run all test cases registered with the Root node.
LESSON 10 | 171
11. Examine the Test Execution tab at the bottom and note the progress bar is now green, indicating the absence of failure. It
is confirmed by the Failures: 0 message at the top right.
Of course, it is possible to register several test cases for a given output. Having multiple unit tests covering different use
cases buys peace of mind for the developer. It makes it possible to tinker with the application or fix bugs while maintaining the
integrity of the application. For example, it is common for a bug fix to break another feature (unintentionally). Having tests
covering every use case is very interesting from a quality engineering point of view. It makes sure a new feature or a bug fix
did not break another feature.
Next Step
This lesson is almost over. Head to the Wrap-Up section for a summary of the concepts reviewed in this lesson.
Next Step
Congratulations, you successfully completed this lesson. Click the Check your status with this unit button below in order to save
your progress. Then click Completed. Let's continue > on the next screen to jump to the next lesson.
LESSON 10 | 173
This page intentionally left blank to ensure new chapters
start on right (odd number) pages.