IMS Tools 2010 Bootcamp

IMS High Performance Unload – Overview

• IMS High Performance Unload (HP Unload) provides high performance
database retrieval service called High Speed Sequential Retrieval (HSSR)
Engine
• HSSR Engine provides DL/I call analyzer, segment retriever, and buffer
handler that are optimized for GN calls
– It is similar to OSAM Sequential Buffering, but HSSR Engine supports VSAM
ESDS also
• HP Unload provides two unload utilities (FABHURG1 and FABHFSU)
• HP Unload provides “Unload API”, which enables user DL/I unload
applications to run on HP Unload without modification
No change is needed in many cases
JCL & output compatibility
IMS User Unload User
HD Unload Unload utility Unload
utility Application FABHURG1 Application

IMS DLIBATCH IMS HP Unload

Native DL/I Call Handler
Native Buffer Handler HSSR Engine

DB

IBM Confidential © 2010 IBM Corporation

 IMS Tools 2010 Bootcamp

HP Unload – Major features of Unload utilities

• Common to FABHURG1 and FABHFSU

– Can unload a database without decompressing compressed
segments
 This can decrease the CPU time and elapsed time
– Supports multiple pre-defined unload record formats other than
the standard HD format
– Can use HSSR Engine’s optional features to be explained later

IBM Confidential © 2010 IBM Corporation

 IMS Tools 2010 Bootcamp

HP Unload – FABHURG1 unload utility

• Is a simple alternative of IMS HD Reorganization Unload
utility
– Can run with using the same JCL statements
• Can run on DLI and DBB regions in addition to ULU region
• Supports a system exit for function extensions:
– An exit routine name is specified in SYSIN
– IBM-provided exit routines:
 FABHEXTR
→ Creating a database extract for database tuning experiments or testing
 FABHKEYX
→ Separating output data set at migration unload based on a set of high key
values specified
– Users can write their own exit routines to add functions
IBM Confidential © 2010 IBM Corporation
 IMS Tools 2010 Bootcamp

HP Unload – FABHFSU unload utility

• Is an unload utility for special purposes
– Uses JCL statements similar to IMS DLIBATCH
– Supports not only ULU region, but also DLI region with a PSB
having multiple PCBs for multiple databases
 Sensitivity can be controlled by using PCBs
• Can produce multiple outputs with different format for each
output by a single run
• Additional segment editing can be done by using user exit
routines
– Different user exit routine can be specified for each output
• Provides Parallel Scan Facility
 For the compatibility with the old product FSU II

IBM Confidential © 2010 IBM Corporation

 IMS Tools 2010 Bootcamp

HP Unload – Unload API

• HP Unload provides Through this API, the application program can

“Unload API” retrieve and process a large number of
segments efficiently
– Optimized for GN and
GNP calls with an
unqualified SSA
– Enables an IMS DL/I
batch application
program, which reads a
database sequentially,
to use HSSR Engine
without being
recompiled or relink-
edited
– Supports DL/I calls and
EXEC DLI commands

IBM Confidential © 2010 IBM Corporation

 IMS Tools 2010 Bootcamp

HP Unload – HSSR Engine’s optional features

• Can produce Database Tuning Statistics, which provides
information on
– the need for database reorganization
– the quality of HDAM or PHDAM randomizing

• Can produce information on long database records

– DB records that have the following characteristics:
 DB records whose length are greater than the specified bytes
 DB records that would require more than the specified number of I/Os for
retrieval
– Information written for each DB record:
 The length of the DB record
 The number of database I/Os required to read all database segments of
the DB record
 The key value of the root segment of the DB record
IBM Confidential © 2010 IBM Corporation
 IMS Tools 2010 Bootcamp

HP Unload – Other features of HSSR Engine

• Ability to continue processing after sequence errors
– Optional sequence-key checks for twin chains
– Three options on an error:
 ABEND
 Returns a GG status code and not unload the segment containing the
error, or its dependents
 Returns a GX status code and unloads the segment in error
• Reading a corrupted database
– Two options:
 Bypassing pointer errors
 Force the roots of HIDAM/PHIDAM DB to be accessed from the primary
index

• Detailed diagnostic information can be provided for sequence errors

and pointer errors

IBM Confidential © 2010 IBM Corporation

 IMS Tools 2010 Bootcamp

IMS High Performance Load – Overview

• IMS High Performance Load (HP Load) provides high

performance database load service
• HP Load provides a reload utility that is functionally
compatible with IMS HD Reorganization Reload utility
• HP Load provides “Load API”, which enables user DL/I
load applications to run on HP Load without modification

No change is needed in many cases

JCL & input compatibility
IMS User Reload User
HD Reload Load utility Load
utility Application Application

IMS DLIBATCH IMS HP Load

Native DL/I Call Handler
Native Buffer Handler HP Load service

DB

IBM Confidential © 2010 IBM Corporation

 IMS Tools 2010 Bootcamp

HP Load – Major features and functions

• Can process compressed segments as input
※ The unload data set must be created by an HP Unload’s unload utility
• Supports various input record formats, including
 Formats supported by HP Unload’s unload utility
 User-defined record header formats
• Provides a user exit facility for additional user processing of each
segment
• Generates statistics reports to aid in tuning the database
• Can check updates to the database between the unload and the reload
to prevent the use of an invalid unload data set
※ The function can be activated automatically if the input data set was created by HP
Unload with certain options specified
• Can initialize an empty database or partition
 HDAM/PHDAM partition: the entire root addressable area is preformatted
 HIDAM/PHIDAM partition: a root segment with a key of all X'FF' is inserted
• Can select options for whether/how to build ILDSs for HALDB partitions

IBM Confidential © 2010 IBM Corporation

 IMS Tools 2010 Bootcamp

HP Load – Major features and functions…

• WF1 record processing
– WF1 records can be split into two groups:
 Records for logical relationship maintenance
 Records for secondary index maintenance
and be written into separate outputs
– WF1 record generation can be suppressed
• Sorting before reload
– Segment records in the input data set can be sorted internally before
the reload processing starts
– With this function, the SORT step before the reload step can be
eliminated when the randomizing for HDAM/PHDAM has been
changed between unload and reload
– The function can be activated automatically if the input data set was
created by HP Unload with certain options specified

IBM Confidential © 2010 IBM Corporation

 IMS Tools 2010 Bootcamp

HP Load – User-controllable bitmap setting

IMS bitmap for HD database data sets
A bit in the bitmap block indicates whether a block
• HP Load can set the bitmap has free space, and the order of the bits in the
bitmap block corresponds to the order of blocks.
bits for HD databases to
Normally,
accommodate denser packing  Bit '1' means the corresponding block
of the database blocks contains a contiguous free space large
enough to put the largest segment in the data
– The user can specify a threshold set group*
 Bit '0' means the block has no space for it
as his/her own “largest segment
* The “largest segment in the data set group” is
length in the data set group” for determined as follows:
each data set group  For a fixed-length segment, use its length
 For a variable segment, the longest length is
– HP Load uses these thresholds used for determination.
to set bitmap bits  Therefore, the actual segment length might be
smaller than the maximum length defined in the
DBD
• Stand-alone Bitmap Resetter is  For a compression segment, the original
provided also length is used for determination.

IBM Confidential © 2010 IBM Corporation

 IMS Tools 2010 Bootcamp

HP Load – Load API

• IMS HP Load provides the Load API

– Load API is optimized for ISRT calls
– User initial load program can run on HP Load much faster than
that on IMS DL/I
 The load program must specify a PCB with PROCOPT=L or LS
 For HDAM, internal physical sequence sort will be made before
actual insert for better performance
– Calls that use AIB is supported
– EXEC DLI is supported
• Benefits of Load API
– Time and cost for the initial load process can be reduced

IBM Confidential © 2010 IBM Corporation

 IMS Tools 2010 Bootcamp

IMS High Performance Prefix Resolution – Overview

• IMS High Performance Prefix Resolution for z/OS (HP Prefix

Resolution) is an IMS tool that enables you to resolve and update
prefixes of IMS databases that are involved in logical relationships as a
single job step
• HP Prefix Resolution supports the following three modes of operation:
– Stand-Alone Prefix Resolution
 Replacement of IMS Database Prefix Resolution utility (DFSURG10)
– Stand-Alone Prefix Update
 Replacement of the IMS Database Prefix Update utility (DFSURGP0),
– Concurrent Prefix Update
 Running prefix resolution and prefix update in a single job step
 This function is not available in IMS standard utilities

IBM Confidential © 2010 IBM Corporation

 IMS Tools 2010 Bootcamp

HP Prefix Resolution – Concurrent Prefix Update

• A data transfer service called

HPPR Pipe is used
• HPPR Pipe
 Eliminates the intermediate on-memory
asynchronous
work files WF2 and WF3 data transfer
 Reduces much of the I/O and
handling of the tape and
DASDs
 With the use of HPPRPIPE and
HPPR buffer handler, each
function improves the
performance considerably over
the IMS Database Prefix
Resolution utility or the IMS
Database Prefix Update utility

IBM Confidential © 2010 IBM Corporation

 IMS Tools 2010 Bootcamp

HP Prefix Resolution – Comparison with IMS Utilities

Function IMS Prefix IMS Prefix HP Prefix Resolution
Resolution utility Update utility (resolution + update)
Prefix Resolution Yes n/a Yes
Prefix Update n/a Yes Yes
Concurrent prefix update n/a n/a Yes
Support of HISAM Yes Yes No
Allowing an LP that has no LCs (option) DFS878W n/a Yes
(in both Resolution and Update)

Summary of LPs without LCs (option) Yes n/a Yes

Suppressing updates of counter field of an LP n/a No Yes
segment that has no LC (option)
Reporting “orphan” LP or LC segments (option) No No Yes
Generate “scan list” (option) Yes n/a No
SNAP option for problem determination n/a Yes No
ABEND option for problem determination n/a Yes No
Can run under UCF Yes Yes No
Support of checkpoint and restart No Yes No
Support IMS logging n/a Yes No

IBM Confidential © 2010 IBM Corporation

 IMS Tools 2010 Bootcamp

IPR – High function Unload/Reload utilities

• IPR also provides the following reorganization utilities:
– IPR Unload utility
 Provides unload function and the application programming interface
– IPR Reload utility
 Provides reload function
– IPR DB Scan utility
 invokes IMS HP Unload and provides the DB Scan function.
• A simple JCL interface similar to IPR Driver is provided for these
utilities
• These utilities can be used as the functional replacements:
– IPR Unload utility for IMS HD Reorganization Unload utility
(DFSURGU0)
– IPR Reload utility for IMS HD Reorganization Reload utility
(DFSURGL0)
– IPR DB Scan utility for IMS Database Scan utility (DFSURGS0)
IBM Confidential © 2010 IBM Corporation
 IMS Tools 2010 Bootcamp

IPR Unload/Reload – Comparison with IMS utilities

IMS HD Reorganization Unload utility Standard JCL statements for IPR Unload utility
//HDUNLD EXEC PGM=DFSRRC00,PARM='ULU,DFSURGU0,DI32DB02' //IPRUNLD EXEC PGM=HPSCMAIN,PARM='FUNC=UL,DBD=DI32DB02'
//STEPLIB DD DSN=IMS.SDFSRESL,DISP=SHR //STEPLIB DD DSN=IMSTOOL.LOADLIB,DISP=SHR
//DFSRESLB DD DSN=IMS.SDFSRESL,DISP=SHR // DD DSN=IMS.DYNLIB,DISP=SHR * DFSMDA members
//IMS DD DSN=IMS.DBDLIB,DISP=SHR // DD DSN=IMS.SDFSRESL,DISP=SHR
//SYSPRINT DD SYSOUT=A,DCB=BLKSIZE=1210 //DFSRESLB DD DSN=IMS.SDFSRESL,DISP=SHR
//DFSUCKPT DD DSN=IMS.CHKPT,DISP=(NEW,KEEP), //IMS DD DSN=IMS.DBDLIB,DISP=SHR
// UNIT=SYSDA,VOL=SER=222222,SPACE=(TRK,(50)) //HPSOUT DD SYSOUT=A * corresponds to SYSPRINT
//DFSURGU1 DD DSN=IMS.UNLOAD1,DISP=(NEW,KEEP), //DFSURGU1 DD DSN=IMS.UNLOAD1,DISP=(NEW,KEEP),
// UNIT=TAPE,VOL=SER=TAPE11,LABEL=(,SL) // UNIT=TAPE,VOL=SER=TAPE11,LABEL=(,SL)
//HDPAYROL DD DSN=DATABASE.PAYROLL,DISP=OLD,
// UNIT=SYSDA,VOL=SER=DB0001 • Database data sets are allocated dynamically by using the DFSMDA
//HDINDEXO DD DSN=DATABASE.INDEXO,DISP=OLD, member for the database in IMS.DYNLIB
// UNIT=SYSDA,VOLUME=SER=DB0003
//DFSVSAMP DD input for VSAM and OSAM buffers and options • Buffering parameters are optional, and neither //DFSVSAMP DD nor
//DFSCTL DD * //DFSCTL DD is needed
SBPARM ACTIV=COND • If a utility option needs to be specified, //HPSIN DD statement is coded.
/*
• Checkpointing is not supported.

IMS HD Reorganization Reload utility Standard JCL statements for IPR Reload utility
//HDRELD EXEC PGM=DFSRRC00,PARM='ULU,DFSURGL0,HD32DB02', //IPRRELD EXEC PGM=HPSCMAIN,PARM='FUNC=RL,DBD=HD32DB02'
//STEPLIB DD DSN=IMS.SDFSRESL,DISP=SHR //STEPLIB DD DSN=IMSTOOL.LOADLIB,DISP=SHR
//DFSRESLB DD DSN=IMS.SDFSRESL,DISP=SHR // DD DSN=IMS.DYNLIB,DISP=SHR * DFSMDA members
//IMS DD DSN=IMS.DBDLIB,DISP=OLD // DD DSN=IMS.SDFSRESL,DISP=SHR
//SYSPRINT DD SYSOUT=A //DFSRESLB DD DSN=IMS.SDFSRESL,DISP=SHR
//DFSUINPT DD DSN=IMS.UNLOAD1,DISP=OLD, //IMS DD DSN=IMS.DBDLIB,DISP=SHR
// UNIT=TAPE,VOL=SER=TAPE11,LABEL=(,SL) //HPSOUT DD SYSOUT=A * corresponds to SYSPRINT
//DFSURWF1 DD DSN=IMS.WRKTAPE1,DISP=(NEW,KEEP), //DFSUINPT DD DSN=IMS.UNLOAD1,DISP=OLD,
// UNIT=TAPE,VOL=SER=WKTAPE,LABEL=(,SL) // UNIT=TAPE,VOL=SER=TAPE11,LABEL=(,SL)
// DCB=(BLKSIZE=1008,LRECL=900,RECFM=VB) //DFSURWF1 DD DSN=IMS.WRKTAPE1,DISP=(NEW,KEEP),
//HDPAYROL DD DSN=DATABASE.PAYROLL,DISP=OLD, // UNIT=TAPE,VOL=SER=WKTAPE,LABEL=(,SL)
// UNIT=SYSDA,VOL=SER=DB0001 // DCB=(BLKSIZE=1008,LRECL=900,RECFM=VB)
//HDINDEXO DD DSN=DATABASE.INDEXO,DISP=(NEW,KEEP), //DFSURCDS DD DSN=IMS.RLCDS,DISP=(OLD,KEEP),
// UNIT=SYSDA,VOL=SER=DB0003,SPACE=(CYL,(10,10)) // UNIT=SYSDA,VOL=SER=IMSMSC
//DFSURCDS DD DSN=IMS.RLCDS,DISP=(OLD,KEEP),
// UNIT=SYSDA,VOL=SER=IMSMSC
//DFSVSAMP DD input for VSAM and OSAM buffers and options • Same comments as above.
//DFSCTL DD *
SBPARM ACTIV=COND
/*

IBM Confidential © 2010 IBM Corporation

 IMS Tools 2010 Bootcamp

JCL Compatibility with IMS Utilities

IBM Confidential © 2010 IBM Corporation

 IMS Tools 2010 Bootcamp

Batch Application Exit (DFSISVI0)

• IMS provides a batch application exit (DFSISVI0)

– The exit is intended to be used to allow the initialization of
products that run with IMS
– The exit takes control immediately before linking to the batch
application program and prior to passing control to the
application program
– The exit is applicable to IMS DB and IMS TM batch
environments, and batch types DBB, DLI, and ULU

Reference:
Batch application exit routine (DFSISVI0) in IMS Version 11 Exit Routines (SC19-2437)
http://publib.boulder.ibm.com/infocenter/dzichelp/v2r2/index.jsp?topic=/com.ibm.ims11.doc.err/ims_batchappexit.htm

IBM Confidential © 2010 IBM Corporation

 IMS Tools 2010 Bootcamp

JCL compatibility with IMS HD Unload/Reload utility

• The following IMS Tools utility programs use the DFSISVI0 exit
to run them on IMS batch region controller DFSRRC00
IMS Tools Purpose Exit routine in the Activation of the function
Product product library (minimum requirements)
HP Unload High performance unloading FABHSVI0 Link-edit FABHSVI0 into a
STEPLIB library with the alias
Providing FABHURG1 with the JCL of ‘DFSISVI0’
compatibility with DFSURGU0
HP Load High performance execution of HPSRSVI0 Link-edit HPSRSVI0 into a
user load program STEPLIB library with the alias
of ‘DFSISVI0’
Providing DLIBATCH JCL
compatibility for user IMS load
applications

IPR High performance and high Either FABHSVI0 or After activating DFSISVI0 for
function unloading HPSRSVI0 HP Unload or HP Load (see
above), modify the installation
Running IPR Unload utility, instead default option for the IPR
of FABHURG1, by the JCL for * They are exactly
same. Unload utility to specify
DFSURGU0 JCLTYPE=DFS

IBM Confidential © 2010 IBM Corporation

 IMS Tools 2010 Bootcamp

How to identify which is running

IMS Tools Utility job When IMS Tools is used When IMS is used
Product
HP Unload High performance unloading FABH1001I DFSISVI0 EXIT The message FABH1001I is
IS CALLED AND FABHURG1 not issued
Running FABHURG1 utility by IS BEING INVOKED
using JCL for DFSURGU0
HP Load High performance execution HPSR0001I RELOAD The message HPSR0001I is
of user load program FUNCTION ENDED, not issued or the following
HIGHEST RETURN CODE IS message is issued:
Running user load applications xxx (REASON CODE: yyyy)
on HP Load by using DLIBATCH HPSR0203I NO CALLS ARE
PROCESSED BY LOAD API
PROCESSING

IPR High performance and high HPSU3003I DFSISVI0 EXIT The message HPSU3003I is
function unloading IS CALLED AND IPR not issued or the following
UNLOAD UTILITY IS BEING message is issued:
Running IPR Unload utility by INVOKED
using JCL for DFSURGU0 HPSU3004I DFSISVI0 EXIT
IS CALLED BUT IPR
UNLOAD UTILITY CANNOT
BE INVOKED

IBM Confidential © 2010 IBM Corporation

 IMS Tools 2010 Bootcamp

Comparing unload utility functions

Function IMS HD Unload FABHURG1 IPR Unload
Creating an HD unload data set Yes Yes Yes
Unload without decompressing segments No Yes Yes
Creating unload data set in various formats No Yes Yes
Creating unload data set in user-defined formats No No Yes
User exit for segment editing and output control No Yes Yes (x2)
Dynamic allocation of database data sets No No Yes
Statistics for database tuning (incl. buffering statistics) No Yes Yes
IPR’s common statistics report for database tuning No No Yes
Reports can be stored in IMS Tools common repository No No Yes
Unloading a corrupted database No Yes Yes
Sequential buffering or its equivalent (for OSAM and Yes for OSAM Yes for both Yes for both
VSAM ESDS) No for ESDS
Checkpoint and restart Yes No No
Running under UCF Yes No No
Migration Unload for migration to HALDB Yes Yes No
MIGRATX for migration to PSINDEX Yes No No
Fallback Unload for fallback from HALDB Yes Yes No
Unloading selected partitions IBM Confidential Yes Yes YesIBM Corporation
© 2010
 IMS Tools 2010 Bootcamp

Comparing reload utility functions

Function IMS HD Reload HP Load IPR Reload
Using an HD unload data set as input Yes Yes Yes
Reloading compressing segments No Yes Yes
Accepting HP Unload provided unload record formats No Yes Yes
Accepting input records in user-defined format No Yes Yes
User exit for segment editing and output control No Yes Yes
Dynamic allocation of database data sets No Yes Yes
Statistics for loaded database, data sets, and pointers No Yes Yes
Reports can be stored in IMS Tools common repository No Yes Yes
Creating empty database data sets No Yes Yes
Partition initialization No Yes Yes
Use-controlled Bitmap setting No Yes Yes
Splitting secondary index maintenance records in WF1 No Yes Yes
Load API No Yes No
IPR’s common statistics report for database tuning No No Yes
Checkpoint and restart Yes No No
Running under UCF Yes No No

IBM Confidential © 2010 IBM Corporation