Mapping Parameters and Variables Overview
Mapping Parameters and Variables Overview
By PenchalaRaju.Yanamala
In the Designer, use mapping parameters and variables to make mappings more
flexible. Mapping parameters and variables represent values in mappings and
mapplets. If you declare mapping parameters and variables in a mapping, you
can reuse a mapping by altering the parameter and variable values of the
mapping in the session. This can reduce the overhead of creating multiple
mappings when only certain attributes of a mapping need to be changed.
When you use a mapping parameter or variable in a mapping, first you declare
the mapping parameter or variable for use in each mapplet or mapping. Then,
you define a value for the mapping parameter or variable before you run the
session.
Mapping Parameters
A mapping parameter represents a constant value that you can define before
running a session. A mapping parameter retains the same value throughout the
entire session.
When you use a mapping parameter, you declare and use the parameter in a
mapping or mapplet. Then define the value of the parameter in a parameter file.
The Integration Service evaluates all references to the parameter to that value.
For example, you want to use the same session to extract transaction records for
each of the customers individually. Instead of creating a separate mapping for
each customer account, you can create a mapping parameter to represent a
single customer account. Then use the parameter in a source filter to extract only
data for that customer account. Before running the session, you enter the value
of the parameter in the parameter file.
To reuse the same mapping to extract records for other customer accounts, you
can enter a new value for the parameter in the parameter file and run the
session. Or, you can create a parameter file for each customer account and start
the session with a different parameter file each time using pmcmd. By using a
parameter file, you reduce the overhead of creating multiple mappings and
sessions to extract transaction records for different customer accounts.
When you want to use the same value for a mapping parameter each time you run the
session, use the same parameter file for each session run. When you want to change the
value of a mapping parameter between sessions you can perform one of the following
tasks:
Mapping Variables
When you use a mapping variable, you declare the variable in the mapping or
mapplet, and then use a variable function in the mapping to change the value of
the variable. At the beginning of a session, the Integration Service evaluates
references to a variable to determine the start value. At the end of a successful
session, the Integration Service saves the final value of the variable to the
repository. The next time you run the session, the Integration Service evaluates
references to the variable to the saved value. To override the saved value, define
the start value of the variable in a parameter file or assign a value in the pre-
session variable assignment in the session properties.
You can use parameters and variables in the SQL query, user-defined join,
source filter, and pre- and post-session SQL commands of a Source Qualifier
transformation. Use any parameter or variable type that you can define in the
parameter file. You can enter a parameter or variable within the SQL statement,
or you can use a parameter or variable as the SQL query. For example, you can
use a session parameter, $ParamMyQuery, as the SQL query, and set
$ParamMyQuery to the SQL statement in a parameter file.
The Integration Service first generates an SQL query and expands each
parameter or variable. It replaces each mapping parameter, mapping variable,
and workflow variable with its start value. Then it runs the query on the source
database.
When you use a string mapping parameter or variable in the Source Qualifier
transformation, use a string identifier appropriate to the source system. Most
databases use a single quotation mark as a string identifier. For example, to use
the string parameter $$IPAddress in a source filter for a Microsoft SQL Server
database table, enclose the parameter in single quotes as follows: ‘$$IPAddress.’
When you use a datetime mapping parameter or variable, or when you use the
built-in variable $$$SessStartTime, change the date format to the format used in
the source. The Integration Service passes datetime values to source systems as
strings in the SQL query.
Tip: To ensure the format of a datetime parameter or variable matches that used
by the source, validate the SQL query.
You can create mapping parameters and variables in the Mapping Designer or
Mapplet Designer. Once created, mapping parameters and variables appear on
the Variables tab of the Expression Editor. Use them in any expression in the
mapplet or mapping. The Designer validates mapping parameters and variables
in the Expression Editor of mapplets and mappings.
When the Integration Service needs an initial value, and you did not declare an
initial value for the parameter or variable, the Integration Service uses a default
value based on the datatype of the parameter or variable.
Table 7-1 lists the default values the Integration Service uses for different types
of data:
Table 7-1. Default Values for Mapping Parameters and Variables Based on
Datatype
Data Default Value
String Empty string.
Numeric 0
Datetime 1/1/1753 A.D. or 1/1/1 when the Integration Service is configured for
compatibility with 4.0.
For example, you create a new mapping using an Integer mapping variable, $
$MiscellaneousExpenses. You do not configure an initial value for the variable or
define it in a parameter file. The first time you run a session with the mapping,
the Integration Service uses the default value for numeric datatypes, 0.
As long as you do not define the parameter value in the parameter file, the
Integration Service replaces $$MiscellaneousCosts with 0. When you want to
include miscellaneous expenses in mapping calculations, set $
$MiscellaneousCosts to that value in the parameter file.
STATE = ‘$$State’
During the session, the Integration Service replaces the parameter with a string.
If $$State is defined as MD in the parameter file, the Integration Service replaces
the parameter as follows:
STATE = ‘MD’
You can perform a similar filter in the Filter transformation using the PowerCenter
transformation language as follows:
STATE = $$State
If you enclose the parameter in single quotes in the Filter transformation, the
Integration Service reads it as the string literal “$$State” instead of replacing the
parameter with “MD.”
You can configure the Integration Service to relax code page validation when you run the
Integration Service in Unicode data movement mode. However, you might get
unexpected results in the following situations:
The mapping variable value that the Integration Service saves in the repository
is not compatible with the repository code page.
For example, the repository uses the ISO 8859-1 Latin1 code page and you
configure the Integration Service to relax code page validation. If the mapping
variable value contains Japanese character data, such as JapanEUC, the saved
mapping variable value in the repository could be incorrect. There could be data
loss converting from the JapanEUC code page to the Latin1 code page. Make
sure the saved mapping variable value is two-way compatible with the repository
code page.
To ensure the Integration Service can write all metadata to the repository, use 7-bit
ASCII characters for all repository metadata or use UTF-8 for the repository.
The parameter file contains characters that are not compatible with the
Integration Service code page.
The Integration Service interprets the data in the parameter file using the
Integration Service code page. For example, the Integration Service uses the
ISO 8859-1 Latin1 code page and you configure the Integration Service to relax
code page validation. If you create a parameter file and use Greek character
data, such as ISO 8859-7, the value the Integration Service reads from the file
could be incorrect. There could be data loss converting from the ISO 8859-7
code page to the Latin1 code page. Make sure the characters in the parameter
file are a subset of the Integration Service code page.
Mapping Parameters
Before you run a session, define the mapping parameter value in a parameter file
for the session. Use any constant value. During the session, the Integration
Service evaluates all references to the parameter to the specified value. If the
parameter is not defined in the parameter file, the Integration Service uses the
user-defined initial value for the parameter. If the initial value is not defined, the
Integration Service uses a default value based on the datatype of the mapping
parameter. For a list of default values, see Table 7-1.
You can change the value of a mapping parameter between sessions by editing
the parameter file or by changing the parameter file used by the session.
You might use a mapping parameter instead of a database lookup. For example,
you want to perform calculations using monthly gross earnings. Instead of using
a Lookup transformation to connect to a database table for that information, you
can create a gross earnings mapping parameter and update its value in the
parameter file each month to reflect current earnings.
You can create mapping parameters for any mapping or mapplet. You can
declare as many mapping parameters as you need. Once declared, use the
parameter in the mapping or mapplet.
1.In the Mapping Designer, click Mappings > Parameters and Variables.
-or-
In the Mapplet Designer, click Mapplet > Parameters and Variables.
2.Click the Add button.
3.Enter the following information and click OK:
Table 7-2. Options for Declaring Mapping Parameters
Field Description
Name Parameter name. Name parameters $$ParameterName. The syntax
for the parameter name must be $$ followed by any alphanumeric or
underscore characters.
Type Variable or parameter. Select Parameter.
Datatype Datatype of the parameter. Select a valid transformation datatype.
Use any datatype except Binary or Raw.
Precision or Precision and scale of the parameter.
Scale
Aggregation Use for variables.
IsExprVar Determines how the Integration Service expands the parameter in
an expression string. If true, the Integration Service expands the
parameter before parsing the expression. If false, the Integration
Service expands the parameter after parsing the expression. Default
is false.
Note: If you set this field to true, you must set the parameter
datatype to String, or the Integration Service fails the session.
Initial Value Initial value of the parameter. If you do not set a value for the
parameter in the parameter file, the Integration Service uses this
value for the parameter during sessions. If this value is also
undefined, the Integration Service uses a default value based on the
datatype of the mapping variable.
Use any of the following formats for initial values for Date/Time
parameters:
-MM/DD/RR
-MM/DD/RR HH24:MI:SS
-MM/DD/YYYY
-MM/DD/YYYY HH24:MI:SS.US
Description Description associated with the parameter.
You can also use mapping parameters in the Expression Editor. When using
mapping parameters in the Expression Editor, do not enclose string parameters
in string identifiers. The Integration Service handles parameters just like any
other port identifiers.
You can also use mapping parameters in transformation overrides in the session
properties in the Workflow Manager. You can override properties such as a filter
or user-defined join in a Source Qualifier transformation.
Step 3. Define a Parameter Value
Before you run a session, define values for mapping parameters in the parameter
file. When you do not define a parameter in the parameter file, the Integration
Service gets the parameter value from another place. The Integration Service
looks for the value in the following order:
Mapping Variables
In the Designer, you can create mapping variables in a mapping or mapplet. After
you create a mapping variable, it appears in the Expression Editor. You can then
use it in any expression in the mapping or mapplet. You can also use mapping
variables in a source qualifier filter, user-defined join, or extract override, and in
the Expression Editor of reusable transformations.
Unlike mapping parameters, mapping variables are values that can change
between sessions. The Integration Service saves the latest value of a mapping
variable to the repository at the end of each successful session. During the next
session run, it evaluates all references to the mapping variable to the saved
value. You can override a saved value with the parameter file. You can also clear
all saved values for the session in the Workflow Manager.
You might use a mapping variable to perform an incremental read of the source.
For example, you have a source table containing timestamped transactions and
you want to evaluate the transactions on a daily basis. Instead of manually
entering a session override to filter source data each time you run the session,
you can create a mapping variable, $$IncludeDateTime. In the source qualifier,
create a filter to read only rows whose transaction date equals $
$IncludeDateTime, such as:
TIMESTAMP = $$IncludeDateTime
In the mapping, use a variable function to set the variable value to increment one
day each time the session runs. If you set the initial value of $$IncludeDateTime
to 8/1/2004, the first time the Integration Service runs the session, it reads only
rows dated 8/1/2004. During the session, the Integration Service sets $
$IncludeDateTime to 8/2/2004. It saves 8/2/2004 to the repository at the end of
the session. The next time it runs the session, it reads only rows from August 2,
2004.
Variable Values
The Integration Service holds two different values for a mapping variable during a
session run:
The start value is the value of the variable at the start of the session. The start
value could be a value defined in the parameter file for the variable, a value
assigned in the pre-session variable assignment, a value saved in the repository
from the previous run of the session, a user defined initial value for the variable,
or the default value based on the variable datatype. The Integration Service looks
for the start value in the following order:
For example, you create a mapping variable in a mapping or mapplet and enter
an initial value, but you do not define a value for the variable in a parameter file.
The first time the Integration Service runs the session, it evaluates the start value
of the variable to the configured initial value. The next time the session runs, the
Integration Service evaluates the start value of the variable to the value saved in
the repository. If you want to override the value saved in the repository before
running a session, you need to define a value for the variable in a parameter file.
When you define a mapping variable in the parameter file, the Integration Service
uses this value instead of the value saved in the repository or the configured
initial value for the variable.
Current Value
The current value is the value of the variable as the session progresses. When a
session starts, the current value of a variable is the same as the start value. As
the session progresses, the Integration Service calculates the current value using
a variable function that you set for the variable. Unlike the start value of a
mapping variable, the current value can change as the Integration Service
evaluates the current value of a variable as each row passes through the
mapping. The final current value for a variable is saved to the repository at the
end of a successful session. When a session fails to complete, the Integration
Service does not update the value of the variable in the repository. The
Integration Service states the value saved to the repository for each mapping
variable in the session log.
When you declare a mapping variable in a mapping, you need to configure the
datatype and aggregation type for the variable.
The datatype you choose for a mapping variable allows the Integration Service to
pick an appropriate default value for the mapping variable. The default is used as
the start value of a mapping variable when there is no value defined for a
variable in the parameter file, in the repository, and there is no user defined initial
value.
The Integration Service uses the aggregate type of a mapping variable to
determine the final current value of the mapping variable. When you have a
pipeline with multiple partitions, the Integration Service combines the variable
value from each partition and saves the final current variable value into the
repository.
Count
Max
Min
You can configure a mapping variable for a Count aggregation type when it is an
Integer or Small Integer. You can configure mapping variables of any datatype
for Max or Min aggregation types.
To keep the variable value consistent throughout the session run, the Designer
limits the variable functions you use with a variable based on aggregation type.
For example, use the SetMaxVariable function for a variable with a Max
aggregation type, but not with a variable with a Min aggregation type.
Table 7-3 describes the available variable functions and the aggregation types
and datatypes you use with each function:
Variable Functions
Variable functions determine how the Integration Service calculates the current value of a
mapping variable in a pipeline. Use variable functions in an expression to set the value of
a mapping variable for the next session run. The transformation language provides the
following variable functions to use in a mapping:
Use variable functions only once for each mapping variable in a pipeline. The
Integration Service processes variable functions as it encounters them in the
mapping. The order in which the Integration Service encounters variable
functions in the mapping may not be the same for every session run. This may
cause inconsistent results when you use the same variable function multiple
times in a mapping.
The Integration Service does not save the final current value of a mapping variable to the
repository when any of the following conditions are true:
When you declare a mapping variable for a mapplet and use the mapplet multiple
times within the same mapping, the same mapping variable value is shared
across all mapplet instances.
You can create a mapping variable for any mapping or mapplet. You can create
as many variables as you need. Once created, use the variable in the mapping or
mapplet.
1.In the Mapping Designer, click Mappings > Parameters and Variables.
-or-
In the Mapplet Designer, click Mapplet > Parameters and Variables.
2.Click the Add button.
3.Specify the variable information.
Table 7-4 describes the options on the Declare Parameters and Variables dialog
box:
Table 7-4. Options for Creating Mapping Variables
Field Required/ Description
Optional
Name Required Variable name. Name variables $$VariableName. The
syntax for the variable name must be $$ followed by any
alphanumeric or underscore characters.
Type Required Variable or parameter. Select Variable.
Datatype Required Datatype of the variable. Select a valid transformation
datatype. Use any datatype except Binary. The datatype
you select can affect the Aggregation type you can
select. For example, when you create a String variable,
you cannot configure it with a Count aggregation type.
Precision or Required Precision and scale of the variable.
Scale
Aggregation Required Aggregation type of the variable. Determines the type of
calculation you can perform with the variable.
Set the aggregation to Count if you want to use the
mapping variable to count number of rows read from
-source.
Set the aggregation to Max if you want to use the
mapping variable to determine a maximum value from a
-group of values.
Set the aggregation to Min if you want to use the
mapping variable to determine a minimum value from a
-group of values.
IsExprVar Required Determines how the Integration Service expands the
variable in an expression string. If true, the Integration
Service expands the variable before parsing the
expression. If false, the Integration Service expands the
variable after parsing the expression. Default is false.
Note: If you set this field to true, you must set the
variable datatype to String, or the Integration Service fails
the session.
Initial Value Optional Initial value of the variable. The Integration Service uses
this value for the variable when the variable value is not
saved in the repository or defined in the parameter file. If
this value is also undefined, the Integration Service uses
a default value based on the datatype of the mapping
variable.
Use any of the following formats for initial values for
datetime variables:
-MM/DD/RR
-MM/DD/RR HH24:MI:SS
-MM/DD/YYYY
-MM/DD/YYYY HH24:MI:SS.US
Description Optional Description associated with the variable.
4.Click OK.
After you declare a variable, use it in any expression in the mapping or mapplet.
You can also use a mapping variable in a Source Qualifier transformation or
reusable transformation.
In a Source Qualifier transformation, mapping variables appear on the Variables tab in
the SQL Editor. When using mapping variables in a Source Qualifier transformation
follow these rules:
You can also use mapping variables in transformation overrides in the session
properties. You can override properties such as a filter or user-defined join in a
Source Qualifier transformation.
When you use a mapping variable, you have to determine how to set the value of the
mapping variable. Use a variable function to set a variable value. Use a variable function
in any of the following transformations:
Expression
Filter
Router
Update Strategy
After a session completes successfully, the Integration Service saves the final
value of each variable in the repository. When you do not want to use that value
the next time you run the session, you can override the value in the parameter
file or the pre-session variable assignment in the session properties.
When you do not want to use any of the variable values saved for a session, you
can clear all saved values. You can clear variable values for a session using the
Workflow Manager. After you clear variables values from the repository, the
Integration Service runs the session as if for the first time.
The Integration Service expands mapping parameters and variables when you
run a session. If you use a mapping parameter or variable in an expression, the
Integration Service expands the parameter or variable after it parses the
expression. You might want the Integration Service to expand a parameter or
variable before it parses the expression when you create expressions to
represent business rules that change frequently. Define the expressions in a
parameter file so you do not have to change the mappings every time the
business rules change.
For example, you create an expression that generates a color name based on an
ID string as follows:
IIF(color=‘A0587’,‘white’)
IIF(color=‘A0587’,‘white’,IIF(color=‘A0588’,‘off white’))
Instead of updating the mappings that use this expression every time the
business rule changes, you can define the expression in a parameter file and
update the file when the expression changes.
$$ExpColor
$$ExpColor=IIF(color=‘A0587’,‘white’)
When the color name expression changes, you can update the value of the
mapping parameter in the parameter file. You do not need to modify the
mapping.
When you know a logical default value for a mapping parameter or variable, use
it as the initial value when you create the parameter or variable. This allows the
Integration Service to use the default value instead of a datatype-based default
value.
Enclose string and datetime parameters and variables in quotes in the SQL
Editor.
When you use a string parameter or variable in a Source Qualifier
transformation, enclose it in string identifiers recognized by the source system,
such as single quotation marks.
If you keep all parameter files in one of the process variable directories, such as
$SourceFileDir, use the process variable in the session property sheet. If you
need to move the source and parameter files at a later date, you can update all
sessions by changing the process variable to point to the new directory.
When you plan to rotate through sets of parameters for a session, create a
parameter file for each set of parameters. Use pmcmd to start the session and
specify the name of the parameter file you want to use, or edit the session to use
a different parameter file.