Introduction To Teradata Data Mover Create Your First Job
Introduction To Teradata Data Mover Create Your First Job
http://developer.teradata.com/print/6785
TDM Architecture
TDM's command-line interface allows users to define their intent for copying objects between two Teradata systems via a simple XML file. Users only
need to provide the logon credentials for the source and target systems and the objects they want to copy as input. TDM will take care of the rest by
automatically generating a job plan to execute the data movement based on the current state of the source and target systems. TDM accomplishes this
by utilizing ARC, TPTAPI, and JDBC under the covers to perform the object copy. This abstracts the complexity of the underlying methods to perform
the data movement, which means users no longer have to generate low-level utility scripts to copy objects between Teradata systems.
Click 'expand source' below to see the contents of SQL file to insert data into the table that we will be copying:
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
1 of 5
into
into
into
into
into
into
into
into
into
into
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
values
values
values
values
values
values
values
values
values
values
7/7/2013 7:38 PM
2 of 5
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
http://developer.teradata.com/print/6785
7/7/2013 7:38 PM
3 of 5
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
insert
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
into
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
jg185041.items1
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
values
http://developer.teradata.com/print/6785
COMMIT;
Please refer to The Friday Night Project #5 - TZA-Database article on how to use Ant and Eclipse to execute both SQL files so that our table is created
properly in the dmdev database.
Create XML
The next step in creating our first TDM job is to generate the XML file that specifies the intent of the job. We want to copy the jg185041.items1 table from
to dmsmp, so here is the XML to do so:
dmdev
7/7/2013 7:38 PM
http://developer.teradata.com/print/6785
<source_password>jg185041</source_password>
<target_tdpid>dmsmp</target_tdpid>
<target_user>jg185041</target_user>
<target_password>jg185041</target_password>
<overwrite_existing_objects>true</overwrite_existing_objects>
<log_level>1</log_level>
<online_archive>false</online_archive>
<database selection="unselected">
<name>jg185041</name>
<table selection="included">
<name>items1</name>
</table>
</database>
</dmCreate>
This command successfully created the job in the Daemon's repository and started its execution.
Job status
If we want to see how the job executed we can use TDM's status command to look at the results. Use the following to execute the status command:
datamove status -job_name <name_of_job> -output_level 3
TYPE ID
STATUS
CURRENT STEP
START TIME
DURATION TIME
-----------------------------------------------------------------------------------------------------------------Job: 1
COMPLETED_SUCCESSFULLY
4
10/8/10 2:03 PM
0:0:12
TYPE ID
STATUS
STEP TYPE
START TIME
DURATION TIME
---------------------------------------------------------------------------------------------------------------------------------Step: 3
COMPLETED_SUCCESSFULLY
MOVE_TABLE_DATA
10/8/10 2:03 PM
0:0:10
Step: 4
COMPLETED_SUCCESSFULLY
RESOLVE_TABLE_AFTER_LOAD
10/8/10 2:03 PM
0:0:1
TYPE ID
NAME
MOVE PHASE
STATUS
TYPE
ROWS
BYTES
AGENT
UTILITY
TIMESTAMP
-----------------------------------------------------------------------------------------------------------------------------------------------------Task: 5
items1 MOVING_DATA
ARCHIVE
table
200
9864
Agent1
ARC
10/8/10 2:04 PM
Task: 5
items1 MOVING_DATA
COPY
table
200
9864
Agent1
ARC
10/8/10 2:04 PM
Task: 5
items1 MOVING_DATA
COMPLETE table
200
9864
Agent1
ARC
10/8/10 2:04 PM
Task: 6
items1 POST_DATA_MOVE COMPLETE unknown 0
0
Agent1
SQL
10/8/10 2:04 PM
By looking at this output we can see that TDM used ARC to copy this table and that the job completed successfully in 12 seconds. This concludes our
example of executing your first Data Mover job. You should now be able to see the jg185041.items1 table on your target system.
4 of 5
7/7/2013 7:38 PM
5 of 5
http://developer.teradata.com/print/6785
The next TDM article will cover how to execute partial table copies.
DISCUSSION
24 May 2011
Is it a replace table if the object exists on the target system?
IntensityMan
2 comments
Joined 02/10
Dave
28 Jul 2011
If ARC is used then the table is replaced on the target system.
If TPTAPI/JDBC is used, rows from the target table are deleted and the source rows are inserted into the target table. This
is done by moving the source rows to a staging table and then doing an INSERT/SELECT or MERGE from the staging
table.
sjpeeris
3 comments
Joined 11/10
There is a property called "overwrite_existing_objects" that is defined at the job level. If this property is set to false, then
Data Mover will give you an error if object exists on the target.
There is another property called "compare_ddl". When set to true, DM gives an error if source and target table DDLs are
different.
Jaison
2 months ago
geethareddy
54 comments
Joined 10/11
hi Jason,
We recently opened production flood gates for TDM jobs. I can say bcz of the too many TDM jobs system is getting
hampered in terms of the performance and resources. I am thinking to apply a throttle to delay and restrict the TDM jobs.
But as this is first tiime on our box, i would like to know your thoughts on imposing the throttle rules on TDM jobs.
Please share your thoughts.
Thanks,
Geeta.
jgordon
2 comments
Joined 05/10
2 months ago
Hi Geeta,
If the source/target TD system is getting hampered by running too many TDM jobs at the same time then it is probably a
good idea to throttle the TDM jobs. How are you planning to throttle the jobs? One option is to set up TASM rules in the
DBS to restrict the number of sessions each TDM job can use. Another option is to change the
"jobExecutionCoordinator.maxConcurrentJobs" property in TDM's daemon.properties file to restrict how many concurrent
jobs Data Mover can execute at any given time.
Jason
http://developer.teradata.com/tools/articles/intro-to-tdm-create-your-first-job
7/7/2013 7:38 PM