®

IBM Software Group

Code Coverage
RDz 9.5 and IDz

DevOps
Jon Sayles, IBM z Products - [email protected]

© 2020 IBM Corporation

@Copyright March, 2021

IBM Trademarks and Copyrights
© Copyright IBM Corporation 2008 through 2021.

All rights reserved – including the right to use these materials for IDz instruction.

The information contained in these materials is provided for informational purposes only, and is
provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible
for any damages arising out of the use of, or otherwise related to, these materials. Nothing
contained in these materials is intended to, nor shall have the effect of, creating any warranties
or representations from IBM or its suppliers or licensors, or altering the terms and conditions of
the applicable license agreement governing the use of IBM software. References in these
materials to IBM products, programs, or services do not imply that they will be available in all
countries in which IBM operates.

This information is based on current IBM product plans and strategy, which are subject to change
by IBM without notice. Product release dates and/or capabilities referenced in these materials
may change at any time at IBM’s sole discretion based on market opportunities or other factors,
and are not intended to be a commitment to future product or feature availability in any way.

IBM, the IBM logo, the on-demand business logo, Rational, the Rational logo, and other IBM
Rational products and services are trademarks or registered trademarks of the International
Business Machines Corporation, in the United States, other countries or both. Other company,
product, or service names may be trademarks or service marks of others.

2
Code Coverage – Overview
Extension to Debugging:
Tracks lines of code that have been
executed during test
Improves application testing quality
Focuses testing resource usage
Numerous reports on tested code
and trends
Supports: Batch, CICS and IMS TM
Many options:
▪ Exclude/Include specific modules
▪ Filter and Export Reports
▪ Integrates with ADDi

Detailed/Source
Line Coverage

Module Level
Code Coverage
3
Code Coverage Architecture - Concepts
z/OS
IDz Client Remote
Debugger Your
......Batch example.......
Run-Time Application
//JOB ...
Engine Load Module
//...........
//CEEOPTS DD *
TEST + Code Coverage parms TCP/IP
passed to Remote Debugger Port#
Data
Code Coverage Source
Systems .exe

Upon GOBACK pass Code Coverage

IDz meta-data to IDz
 JCL or Online config passes ENVAR parameters to the Remote Debugger
 Which invokes Code Coverage modules on z/OS
 Which interact with your application code – using the Debugger functionality to
synchronize source statements:
▪ Standard Mode
▪ Debug Tool Compatibility Mode
 Code Coverage sends meta-data back to IDz, which formats into Code
Coverage reports
4
Code Coverage – Setup
▪ Process:
 Compile/Link with standard Debug Tool JCL
Parms (TEST…etc.)
 Submit run JCL – passing in a PARM with:
▪ Your I/P address and port# NOTES:
▪ ENVAR("EQA_STARTUP_KEY=CC") • //STEPLIB reference to Debug Tool runtime is only
required if not installed in Link Pack
 Make sure that the Debug runtime is in • You can also add the ENVAR… parameter into
STEPLIB, JOBLIB or installed into Link Pack //CEEOPTS DD card

 Run your debug session.

▪ Instead of dropping into interactive Debug, your
program will execute
▪ After the job finishes a Code Coverage report
opens in the editor
– (this takes a few seconds)
▪ Double-click a program to view the detailed lines ➔
▪ The report will provide varying levels of Code
Coverage…………………………….

5
Specifying ENVAR Parm
Batch:
▪ PARM card

▪ //CEEOPTS DD card TEST

parameter

Online:
▪ DTCN Profile

▪ DTCN Transaction
– PF9 ➔ Options
6
Code Coverage – Reports 1 of 2
▪ When the Code Coverage batch
job or transaction finishes – and
you’re running RDz interactively:
 The current Code Coverage report
opens in the editor window
 The Compiled Code Coverage Results
view is opened

▪ Code Coverage Reports present:  Program Entry

 Coverage Totals

Paragraph
coverage
▪ The lines that were executed
 A warning symbol to let you know of
any modules and/or paragraphs that
were accessed at below a specified
coverage threshold
▪ You set the threshold for coverage warnings
in Preferences
 Individual program paragraph
coverage statistics

Compiled Code Coverage Results view allows

you to see previous Code Coverage Reports.
These are stored in your Workspace

 Opens…
7
Code Coverage – Reports 2 of 2

Opens…

▪ From the report window,

you can drill-down to a
Program or Function
(COBOL paragraph, etc.)
and double-click the entry to
open a hyper-linked source
file
 Executable statement/lines are
annotated within the
PROCEDURE DIVISION (as
well as the procedural logic in
PL/I programs)
▪ Red == this line was not executed

▪ Green == this line was executed

8
Reports IDz v14.1.x and later

Web-Based UI & Additional Functionality:

• Group data by Load Modules (Files) or CSECTs (Modules)
• Open source files by clicking on the file names.
• Expand rows to view the children of the files.
• View file-related messages by clicking on the icons in the Messages column.
• Sort the report by clicking on the column headings (Name, Coverage, Lines Covered, etc.)
• Filter results according to coverage percentage by specifying a threshold in the Show below field and
clicking the Refresh button. note that you must turn On Show Below. You can also set preference
• Click export to open a dialog that will allow you to export to either CCZip, SonarQube or PDF.
• In Comparison reports you can click the swap button to swap the direction of comparison.
9
Paragraph list/Modules breakout

10
Compare Reports- RDz
▪ Can compare two Code
Coverage Reports
▪ Later (Date/Timestamped)
report is compared to
earlier selected report
– Colored deltas highlight
differences
▪ Deltas are calculated at
various levels
– Totals
– Module
– Paragraph

In this example a batch job is run twice

using two different input datasets
Reports are compared…

11
Compare Reports - IDz v14.1.x
▪ New format for UI

12
Filtering Reports - RDz - Coverage Threshold
▪ You can manipulate the combination of Threshold, Above and
Below – then click: Refresh to filter the paragraphs displayed in
the report...

13
Filtering Detailed Reports - IDz v14.1.x

▪ Click Off ➔ On toggle

▪ Set Filter
▪ Click Refresh
14
Merge Code coverage Results - RDz
• Combine multiple results

• Hover over area of code to see individual test result

15
Merge Detailed Reports - IDz v14.1.x

▪ By selecting & opening two or more reports you

automatically merge their results
16
Save Code Coverage to Local Workstation - RDz
Can Export reports to a local file system, and them back as a Result Location...

17
Code Coverage Options

Large number of options for Code Coverage. Detailed information

https://www.ibm.com/support/knowledgecenter/SSQ2R2_14.0.0/com.ibm.debug.pdt.codecoverage.zpcl.do
c/topics/tccstartup.html

18
Code Coverage Options – Filtering out Modules

▪ You can exclude modules from Code Coverage by adding a parm to the
ENVAR: EQA_STARTUP_KEY=CC,,moduleexcludelist=PGM)

Notes
• Two (2) commas before the parm
• lowercase
• You can also use moduleincludelist=
19
Code Coverage Options – Using tag=
If you wish to selectively cover only code that you’ve added you can pinpoint
specific lines as follows:
1. Add source Tags to the line(s)

2. Compile/Link
3. Add a tag=xxx specification to your JCL
4. Run Code Coverage
5. Open the tag= Report as a Filtered File Report

6. Specify the same Filtered Result as your tag

Note that you can use Autocomment

to tag the source for your Filter 20
Running Code Coverage in Batch (UNIX) Mode - 1 of 4
• Run AKGCC PROC
• Output to z/OS Unix
• Import Unix clcoverage file(s) to Code Coverage Results view

• Runstream JCL AKGCC PROC

21
Running Code Coverage in Batch (UNIX) Mode - 2 of 4
▪ Launch Job that executes AKGCC
▪ Note: Better to Submit from menu

▪ Job finishes

Code Coverage data

written to your Unix
Home directory

22
Running Code Coverage in Batch (UNIX)
Mode - 3 of 4
From Code Coverage Results: Add a Result Location
• Right-click on Name
• Select Add Result Location…

Click Remote Systems…

Select your Connection ➔

• Open the Drop Down to find your LPAR connection(s)

Click OK
23
Running Code Coverage in Batch (UNIX) Mode - 4 of 4

Name the Result Location and click OK

As before, double click the

report in Code Coverage
Results to load the statistics
and annotated source file
into the Editor

24
Running Code Coverage in Batch (Headless) Mode
▪ Small footprint non-gui code coverage collector
▪ Runs native on target system
• Requires Java 6 or higher
▪ Collects CC results and optionally generates
• Source that matches CC results
• HTML reports
• Zip of results
▪ Results can be imported into RDz for further analysis
• Comparisons
• View source lines annotated with hit information
▪ Integrate into automated builds and testing
• Run as continuous daemon that collects all data
• OR Run instance per build/test
▪ Can be invoked from JCL
▪ Supports adding “tags” for future reference
•e.g. testcase ID

25
Running Code Coverage in Batch Mode – Example JCL
//GYOUNGG JOB XXXXXXXX,CLASS=A,MSGCLASS=H,MSGLEVEL=(1,1),
// REGION=256M,NOTIFY=&SYSUID,TIME=1
//STEP1 EXEC PGM=COB01
//STEPLIB DD DISP=SHR,DSN=GYOUNG.CC5.LOADPDSE ➔ COB01 – Load Module
COB01A - Subroutine
//SYSOUT DD SYSOUT=*
//CEEOPTS DD * GYOUNG.CC5.LOADPDSE
(Load library)
TEST(ALL,*,PROMPT,TCPIP&9.76.69.100%8005:*),
ENVAR("EQA_STARTUP_KEY=CC,COB01A,
CCLEVEL=LINE,MODULEINCLUDE=COB01A")

Start daemon ➔

Code Cover COB01A ➔

Save Coverage Report ➔

Can specify DIRECT instead of TCPIP

Excellent White Paper here…

https://developer.ibm.com/mainframe/2018/03/13/capture-code-coverage-without-starting-idz/ Windows Server
26
Code Coverage with Assembler - LE Assembler
▪ You can run Code Coverage with Both LE & Non-LE Assembler ("BAL")
▪ No changes are needed to the Load Module (Assemble) process. But you will need to modify the JCL
slightly - example here of an LE-Assembler program and its Code Coverage

27
Code Coverage with Assembler - Non-LE Assembler
▪ Note changes to the TEST statement - and the addition of the LDD statements, which identify the Debug
Data. Note; Thanks to Francisco Anaya/IBM for the syntax examples

28
Debug Tool – Additional C.C. Functionality
▪ Debug Tool allows
you to specify
Code Coverage
reports for:
Specific programs
Specific areas within
specific programs
▪ Single lines
Involves setting up JCL that includes an
▪ Ranges of lines EQAOPTS DD used to specify input/options and
▪ Optional authorization output/observations files
by Group IDs

For more information check out the “Debug Tool Users Guide”
29
Debug Tool Additional Functionality: Each instruction line of the listing has a character that annotates what happened
during the test run:

Code Coverage Utility & A conditional branch instruction that has executed both ways
> A conditional branch instruction that has branched but not
fallen through
There is a V A conditional branch instruction that has fallen through but
Debug Tool not branched
Code Coverage : Non-branch instruction that has executed
Utility that ¬ Instruction that has not executed
provides a
considerable
number of code
coverage
metrics through
reports.

The reports are

available for
green-bar
printout (!) –
and as HTML.

There is a bit
more setup to do
to use the Utility.

For more
information check
out the “Debug
Tool Users Guide”

30