IBM WebSphere DataStage
DataStage EE Basics
Creating a Job
DW & BI IMPACT Training 2008
Kolkata
© 2007 IBM Corporation
IBM GBS | WebSphere DataStage PX Training
Where We are
MODULE 01 Introduction to DataStage
MODULE 02 DataStage Installation on Windows Platform
MODULE 03 Features of DataStage Clients
MODULE 04 Creating a Job
MODULE 05 Accessing Sequential Data
MODULE 06 Combining Data
MODULE 07 Splitting Data
MODULE 08 Transforming Data
MODULE 09 Sorting and Aggregating Data
MODULE 10 Accessing Relational Data
MODULE 11 Job Control
MODULE 12 Architecture and Parallelism Concepts
© 2007 IBM Corporation
IBM GBS | WebSphere DataStage PX Training
Module 04
Creating a Job
© 2007 IBM Corporation
IBM GBS | WebSphere DataStage PX Training
Developing Jobs in DataStage
Study Given Technical Specification
Plan Job Design
Decide Stage Types
Import Metadata of source and target [and lookup reference] in
manager
Build job in Designer
Configure Stages
Define Job parameters
Maintain Design Standards
Compile job in Designer
Run and monitor job in Director
Module 03 © 2007 IBM Corporation
IBM GBS | WebSphere DataStage PX Training
Study Given Technical Specification
source and target column details
Source Information
Target Information
Mapping Rules – Business logic
Module 03 © 2007 IBM Corporation
IBM GBS | WebSphere DataStage PX Training
Attaching to a Project
Module 03 © 2007 IBM Corporation
IBM GBS | WebSphere DataStage PX Training
Open DataStage Designer
Module 03 © 2007 IBM Corporation
IBM GBS | WebSphere DataStage PX Training
Plan Job Design
1. Analyze the source data and what stage we can use to extract the
source data
2. Implement the business logic in DataStage
3. Load the Data into the database table
© 2007 IBM Corporation
IBM GBS | WebSphere DataStage PX Training
Job Parameters
Go to Job Properties
Use parameters defined in Project Level: User defined
parameters in environment variable. Default value can be hard
coded or retrieved at runtime [$PROJDEF]
Create Job- level parameters
Parameters can be passed from a sequence. Value defined in
higher level takes precedence
© 2007 IBM Corporation
IBM GBS | WebSphere DataStage PX Training
A sample Job in Designer
Compile Job
Annotation
Job properties
Run Job
© 2007 IBM Corporation
IBM GBS | WebSphere DataStage PX Training
Job Properties View generated script
Job parameters in OSH
Run any BASIC sub-routine
before or after Job run
Short description of
the Job
Full Job description to
track modification
history
© 2007 IBM Corporation
IBM GBS | WebSphere DataStage PX Training
Job Parameters
Parameter name
Parameter
Parameter prompt to default value
be seen at run time
Parameter type
Project level
parameters
Job level
parameters
Add new
environment variable
© 2007 IBM Corporation
IBM GBS | WebSphere DataStage PX Training
Design Standards – Naming Convention
Maintain Naming Conventions for stages and links
Use Job and stage annotations
Name the stages after the Data they access/Function they
perform
DO NOT leave default stage names like Sequential_File_0
Use 2-character prefixes to indicate stage type
Links named for the data they carry
DO NOT leave default link names like DSLink1
Prefix all link names with “lnk_”
© 2007 IBM Corporation
IBM GBS | WebSphere DataStage PX Training
Design Standards – Development Approach
Follow Iterative Job Design
• Use Copy and Peek stages as stubs
• Start small and build to final solution
• Start from source and work out
Test job in phases
• Small sections first, then increasing in complexity
• Use Peek stage/datasets to examine records
• Check data at various locations
• Check before and after processing
Solve the business problem before the performance problem
© 2007 IBM Corporation
IBM GBS | WebSphere DataStage PX Training
Compile/Validate/Run
Compiling a Job:
• creates the OSH script for the Job
• C++ code for the transformers
Validating a Job:
• performs connectivity check and authentication
• Basically does everything except process rows. It opens any required files and
connect to data sources.
• Prepares SQL and captures any return message referring to syntax errors from
database
• It is not necessary to validate a job every time you change its design. If the
change does not affect any passive stage, then re-validating it won't really
prove anything. Missing property values will be picked up when the job is
compiled.
• Validation is useful on initial deployment.
Run a Compiled Job
• DataStage Engine runs the processes on the available nodes [ details later]
© 2007 IBM Corporation
IBM GBS | WebSphere DataStage PX Training
Q&A
© 2007 IBM Corporation
