IBM Rational Quality

Manager:
Tips, Tricks and Best
Practices

John Nason
RQM Chief Programmer

Sachin Patel
RQM Dev Lead

2015 IBM Corporation

IBM Rational Quality Manager (RQM)

Tips, Tricks and Best Practices
Intro To RQM
Sticking Points
Best Practices: Setup
Best Practices: Planning
Best Practices: Execution
Advanced Topics: DevOps
Advanced Topics:
Command Line Test
Adapter

Intro to RQM

Rational Quality Manager: Central Hub for Quality

Management
Rational Quality Manager
Test Planning

Requirements
Management

Defect
Management

Test Construction
Test Execution

Rational Team Concert

Team Collaboration
Rational DNG

Rational ClearQuest

Reporting & Dashboard

Rational DOORS
Rational RequisitePro

Functional
Testing

Performance
Testing

Rational Functional Tester

Worksoft Certify

Integration Testing
and
Test Virtualization

Rational
Performance
Tester

Security and
Compliance

Test
Data

AppScan
Optim

Rational
Test Workbench

Rational Quality Manager: A Closer Look

Test Planning

Test Construction

Test Execution

Comprehensive test plan

Requirement driven testing

Manual test execution

Shared objectives

Test environments coverage

Use test automation tools

Scope, Timeline, Resources

Manual test authoring

Record test results

Risk assessment

Test lab management

Submit & track defects

Team Collaboration

Reporting & Dashboard

Process enactment and enforcement

Status and progress tracking

Review and approval

Customizable live dashboard

Task management

Real-time metrics and reports

Rapid team member on-boarding

Compliance and quality audit

Sticking Points

Sticking Point: Execution Records

New users, especially those coming from other test
management tools, are often confused about the Test Execution
Record (TCERs & TSERs)
Execution Records associate the following with a test:

Who (owner)

What (test case or suite)

When (iteration/milestone)

Where (Plan & Environment)

Testers run TCERs or TSERs, not test cases or suites

Sticking Point: Execution Records

Reporting generally is against the last result of a given TCER

Progress tracking views (more later) work against TCERs

Tip: from the ER view of a test plan use Duplicate this Execution
Record to (this test plan | another test plan) to quickly plan
regression testing, rerun failed tests, or those that you did not get to

Sticking Point: Execution Records

A Test Suite Execution Record tracks running a test suite
Test Suites are generally used for:

Regression buckets

Test scenarios with interdependencies

Test Suite Execution Records are composed of Test Case

Execution Records

Note: running a test standalone with the same iteration,

plan, test case, and environment counts the same as a
test that's part of a suite with the same characteristic

Last result is what's used in reporting

Tip: plan your suites at the same time as your cases for accurate
progress tracking!
9

Best Practices:
Setup

Best Practices: Setup

Customizing the Menu
You can customize the menu to only show items relevant to your
organization. Find this in Admin > Manage This Project Area > Menu
Organization.

11

Best Practices: Setup

Customizing the Menu
Menu after preceding customization
Menu choices reduced

Drop down choices reduced

12

Best Practices: Setup

Artifact Templates

13

Customize sections present

Add new sections
Add default content to sections, e.g. How To, best practices, etc.
Tip: Replace your wiki with self-documenting process in the tool!

Best Practices: Setup

Customizing Permissions
Permissions can be viewed and
configured by Role or Operation.

14

Best Practices: Setup

Custom Workflows
Draft
Ready
for review

Tailor integrated process to fit your needs

Each artifact type comes with a predefined set of state transitions,

which can be customized
Reject

Each artifact supports a set of pre-defined state groups, which can

be used to write process enforcement against

Ready
for Review

Each state will then map to each state group

Reopen

Approve

Transition between states can be customized too

Approved
Retire

Retired

15

Return
to Approve

Can assign process advisers on each state that act as

pre-conditions or follow-up actions

Best Practices: Setup

Process Advisors
- Process Advisors provide fine grained control over product behavior and
enforce organizational best practices.
- Require approvals before state transitions
- Require tests to be in a specific state before running them
- Configure (extensible) e-Signatures
- Automatically lock and unlock test artifacts

16

Best Practices: Setup

Process Templates Reuse best practices and quickly setup new projects
Ability to extract a process template from an existing QM project area
A QM process template includes:
Roles and permissions
Test artifacts workflow customization
Test artifacts templates
QM project properties and preferences such as custom attributes, categories
or quality objectives
Process template can be reused to quickly create new QM project area

17

Best Practices:
Planning & Construction

Test Planning and Construction

Test Case and Test Suite Categories
RQM now provides functionality to allow a more
comprehensive parent child relationship between categories

Sub categories
can also be
associated to
multiple parents
for re-use

19

Test Planning and Construction

Test Case and Test Suite Categories

Pre-defined filter queries based on test case categories.

20

Test Planning and Construction

Test Case and Test Suite Categories

Saved queries can

be promoted to the
menu

21

Test Planning and Construction

Test Case and Test Suite Categories

Test Case and Suite categories are visible in Execution Record tables
Saved execution record
queries can be promoted
to the dashboard.

Group by capability
available. Filtering by
Product and/or capability

22

Product quality and defects

listed per test case where
found.

Test Planning and Construction

Test Case and Test Suite Categories

Test Case
category
filters
available

Detailed
point
progress
report
23

Test Planning and Construction

Test Case and Test Suite Categories

Report suggestions:
Test Case Coverage by TCER(Live)
TCER Listing (Live) with parameters for test case
category type and name to gather data
Execution Status by Owner using TCER Count (Live)
Test Cases (Live) parameters of Test Case Type and
Name used to gather data
Test Cases by Team (Live) - parameters of Test Case
Type and Name used to gather data

24

Best Practices:
Execution

Test Execution
Topics
Tester best practices
Test lead best
practices

26

Test Execution Tester best practices

Performing test execution tasks

As a tester, what
test case execution
records do I need to
run or resume?

27

Test Execution Tester best practices

Monitoring tester execution progress of a suite
A test suite is created, approved and ready for test. It has been scheduled for an
iteration.
Application is deployed Tanuj gets started with his testing. Stops for lunch.
Restarts for the afternoon after the staff meeting.
Pauses for the day. X TCERs completed with 4 defects entered.
Lets show the best way for Tanuj to get started the next day
Lets show how Tanuj arranges his personal dashboard to track his progress
How can he decide what TCER to execute next
What is status of defects
What reports is test lead Tammy running to track progress

28

Test Execution Tester best practices

Performing test execution tasks
Enable testers to use dashboard viewlets as their execution to do lists by:
Creating shared execution record queries that leverage the Current User filter
value for the Owner column and the Last Result column. Examples:
My Not Run test case execution records
My In Progress/Paused test case execution records
Using the Execution Record(s) widget to expose each test case execution
record query to a tester tab in the project dashboard.

29

Content seen by Tester 1:

Content seen by Tester 2:

Total count returned by dashboard

viewlet when viewed by Tester 1.

Total count returned by dashboard

viewlet when viewed by Tester 2.

Test Execution Tester best practices

Resuming execution of a suite
Test can be resumed from the test
suite execution record
Resume
suite
execution
from the
execution
console or
a
dashboard
query

30

Test Execution Tester best practices

Monitoring tester execution progress of a suite

Option to create TCERs when a

TSER is created.

31

Test Execution Test lead best

practices
Monitoring execution progress
As a test lead, how
can I monitor
execution progress
and determine
where defects are
being found by my
team?

32

Test Execution
Monitoring execution progress
Tammy, our test lead, needs to create some shared tables and reports to monitor
Product application quality across the iteration as test progresses.
She want to know which high priority test cases are passing, failing or blocked and
wants to be able to see how the defects entered are progressing to resolved so they
can be verified by her team.
What is the best way she can do that?
She will gather the specific test cases she needs by using a pre-defined filter on
test case categories in the execution record table. Gets a lot of records returned!
She will then narrow that down by determining the iteration she wants can be
specific or current iteration
Then she groups by priority or perhaps Owner.
She adds a column for defects so she can see resolution status and link to them
quickly
She can drag/drop columns for quick visual editing.
What reports can she run to track progress

33

Test Execution
Monitoring Execution Progress

Use test Case Categories

to narrow down the test
case execution records

Narrowed but still

more work to filter
needed.
34

Test Execution

Column for defects to

see resolution status
and link to them quickly

Monitoring Execution Progress

Group by Priority to see
progress bars

Saved query for re-use

and sharing

35

Test Execution
Monitoring Execution Progress

For the team

Dashboard, a set of
query links.
Reports can be added
to widgets on the
dashboard also

Report suggestions:
Execution and defects by owner* - Displays the status of
execution records and their associated defects for each
owner. Data Warehouse report.

Defect list* - Lists defects associated with the selected Test

Plan(s) and iterations. Data warehouse report.

36

Advanced Topic:
DevOps

Running tests as part of your build:

a key to devOps success
The RQM Execution Tool allows for headless (unattended) execution of
automated tests
Two options:
Run a standalone java program
Embed execution directly as an Ant task
Standard pattern:
Build
Provision (uDeploy, Chef, etc)
Test (automated tests via RQM Execution Tool)
Tip: use execution variables to parameterize your test
https://jazz.net/wiki/bin/view/Main/RQMExecutionTool
38

devOps in ALM

Develop

Develop

Manual Test (daily but not

completed in a day)

Build (multiple per day)

Build

Unit
Test

ALM
Integration
Test

Performance
Test
Function
Test

System Test
Integration
Test

Rational
Collaborative
Lifecycle
Management

Rational JUnit
Jazz Build

CSPF

Production
Environment

Staging (daily)
Staging
Environment

IBM Workload
Deployer
IBM UrbanCode
Deploy
39

Production
(end of each
sprint)

IBM Workload
Deployer

IBM UrbanCode Deploy

IBM UrbanCode Deploy

Existing integrations with test automation tools

Quality Manager
Selenium Junit tests

Quick Test Professional

LoadRunner

Rational Integration Tester

(GreenHat)

Rational Functional Tester

Rational Performance Tester

Rational Service Tester for SOA

Quality

Worksoft Certify

Rational AppScan Tester Edition

Rational Test RealTime

Accelerating test sandbox creation

through service virtualization
Embedded SW Testing

Managing mobile applications testing

across a global handset test
environment

VeriStand & TestStand

40

Telecom Test Automation

Selenium Junit test execution adapter

Support developer testing of Web UI

Selenium primarily automates web applications for testing purposes

Support for executing JUnit tests that call the Selenium 2 APIs to drive web
browsers
Gather the JUnit results and present them in a test case result

41

Advanced Topic:
Command Line Adapter

Command Line Test Adapter

Offers a low barrier to entry for most of your custom, home
grown, or unsupported testing tools
Run any test program from the shell and:

Capture verdict via return code

Capture standard out and standard error as the execution log that's
uploaded to RQM

Tip: GREAT article - https://jazz.net/library/article/809

43

Command Line Test Adapter

Tip 1: you can map return codes from the command line script to
RQM verdicts using the commandline.properties file

Note: using * maps any undefined return to that verdict.

44

Command Line Test Adapter

Tip 2: there is a set of OOTB execution variables that include
most details of the test being run, and you can define custom
execution variables
Passed via the ENV block to the test being run
Use Project execution variables to define common vars and
enumerated values to minimize error and promote reuse!

45

Command Line Test Adapter

Did you know you can track progress?

Use com.ibm.rqm.commandline.executionproperties to map a

file where you write progress to, and it's visible in the RQM web
UI. Cool!
You can periodically update an integer from 0-99 to show
percentage complete
You can add a status message
You can even embed a clickable link, e.g. if your automation
harness has a management console, etc.

46

Command Line Test Adapter

Need a bigger boat?

You can write your own test automation integration using

the OSLC Automation specification for RQM

More flexibility, but higher cost of entry compared to the

command lined adapter

You can provide custom XHTML to be embedded into

results, giving more UI flexibility

https://jazz.net/wiki/bin/view/Main/RQMTestAutomationAdap
terAPI

47

Questions?

Where are we in the Solution EXPO?

T1 T2

Business Process &

Decision Management
Application Infrastructure
& Integration
Mobile Enterprise
Cloud Business
Applications

Asset Management &

Internet of Things
Cloud Infrastructure & IT
Optimization
DevOps & Continuous
Engineering
Security Intelligence

T1 IBM RTW (integration testing)

IBM RTVS (Service Virtualization)
T2 - IBM RTW (functional,
performance, & mobile testing)
IBM RQM (quality management)

49

Notices and Disclaimers

Copyright 2015 by International Business Machines Corporation (IBM). No part of this document may be reproduced or
transmitted in any form without written permission from IBM.
U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with
IBM.
Information in these presentations (including information relating to products that have not yet been announced by IBM) has been
reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM
shall have no responsibility to update this information. THIS document is distributed "AS IS" without any warranty, either express
or implied. In no event shall IBM be liable for any damage arising from the use of this information, including but not limited to, loss
of data, business interruption, loss of profit or loss of opportunity. IBM products and services are warranted according to the terms
and conditions of the agreements under which they are provided.
Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without
notice.
Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are
presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual
performance, cost, savings or other results in other operating environments may vary.
References in this document to IBM products, programs, or services does not imply that IBM intends to make such products,
programs or services available in all countries in which IBM operates or does business.
Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not
necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither
intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation.
It is the customers responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal
counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customers
business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or
represent or warrant that its services or products will ensure that the customer is in compliance with any law.

Notices and Disclaimers (cont)

Information concerning non-IBM products was obtained from the suppliers of those products, their published
announcements or other publicly available sources. IBM has not tested those products in connection with this
publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM
products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those
products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products
to interoperate with IBMs products. IBM expressly disclaims all warranties, expressed or implied, including but not
limited to, the implied warranties of merchantability and fitness for a particular purpose.
The provision of the information contained herein is not intended to, and does not, grant any right or license under any
IBM patents, copyrights, trademarks or other intellectual property right.

IBM, the IBM logo, ibm.com, Bluemix, Blueworks Live, CICS, Clearcase, DOORS, Enterprise Document
Management System, Global Business Services , Global Technology Services , Information on Demand,
ILOG, Maximo, MQIntegrator, MQSeries, Netcool, OMEGAMON, OpenPower, PureAnalytics,
PureApplication, pureCluster, PureCoverage, PureData, PureExperience, PureFlex, pureQuery,
pureScale, PureSystems, QRadar, Rational, Rhapsody, SoDA, SPSS, StoredIQ, Tivoli, Trusteer,
urban{code}, Watson, WebSphere, Worklight, X-Force and System z Z/OS, are trademarks of International
Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names
might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at
"Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.

Thank You
Your Feedback is
Important!
Access the InterConnect 2015
Conference CONNECT Attendee
Portal to complete your session
surveys from your smartphone,
laptop or conference kiosk.