2010 Hewlett-Packard Development Company, L.P.

The information contained herein is subject to change without notice

Intelligent automated testing with
HP Business Process Testing
Susan Bockhoff
Software QA Manager
McKesson Provider Technologies
Agenda
McKesson and Project Background
Project Goals and Test Automation ROI
Why BPT was Selected
Test Automation Approach and Results
Behind the scenes: the BPT architecture
Creating Business Components
Setting up Parameters
Creating automated tests using BPT
Components
2
McKesson: Who Are We?
Largest healthcare services company in the world
Fortune 15 $106 billion in revenues (FY09)
More than 32,000 employees dedicated to healthcare
Oldest U.S. healthcare company
Established 1833
Only company offering solutions at every point of care
Deep clinical, IT and process expertise
3
Horizon Clinicals Software Products
Manage information across the healthcare lifecycle
4
Patient
& Family
Pharmacy
Local
Hospital
Pediatrician
Primary Care
Physician /
Medical Home
Grandmas
Physician
Alternative
Medicine
Practitioner
Our Software Is Complex .
We develop software in multiple locations
Westminster, Colorado
Alpharetta, Georgia
Roseville, MN
Bangalore, India
Using many development tools
Java
.Net
GWT
Standard HTML
New release contains 35 different applications and over 40,000 tests
5
Why Automate Our Tests?
Revenue: Faster time to adoption will speed
license revenue recognition, market penetration
Cost: Reduced cost to execute tests
Agility: Ability to run regression tests more
quickly to speed time to market of our products.
Repeatability: Detect regression errors more
quickly without human errors or variability.
Coverage: Increase platform coverage on each
release.
Increased confidence in our application quality.
6
Status at the Start of the Project
2000 working automated tests using QTP
These took 3 years to develop
The team was using a keyword-driven
framework
Tests were scripted using VB script (no record
and playback)
Tests required a lot of maintenance and many
had to be abandoned
There were many false failures
7
The Challenge!
5000 new working automated tests in one year
Must be in production and executed successfully
Broad product coverage
Be able to execute the tests in 3 days
Tests must be re-usable over multiple releases
with a minimum of maintenance
Automate first SME Centric approach
Use automated testing as part of agile
development process
8
Business Process Testing
HPs Business Process Testing (BPT) product
was selected for this project
The QTP framework had some problems we
wanted to resolve:
Tests still required a lot of maintenance
Tests required automation engineers to write, and
they did not know the applications well enough to
know what the tests should do
When tests failed the failures could not be resolved
by the automation engineers (did not know enough
about the app to know if it was a bug)
9
Advantages of BPT
Components are created by automation
engineers who can create them without really
knowing everything the application needs to do
Tests themselves are created by subject matter
experts who know what the tests need to do and
how best to prioritize them.
Test failures are analyzed by subject matter
experts who know if it is a software defect or a
problem with the automated script itself.
10
How to Select Good Candidates for
Automated Tests
App must be in a technology that is automatable
(no apps requiring reading of bitmaps).
Frequently run tests are automated first since
they will give the greatest payback
This requires that a full risk-based analysis be
performed on your regression suite to prioritize
all regression tests
High-priority regression tests (wide and shallow)
are the ones that should be automated for
highest ROI.
11
Frequency of Automated Tests
Environmental Health Check 1 hr. Can you log
in to all apps? Executed on every build.
Smoke tests 4 hrs. Can all main pages be
accessed? Executed every two weeks.
Critical automated regression 6 hrs. can all
major transactions be performed? Executed at
start of regression testing and also last thing
before release.
High priority regression - executed several
times on a release.
12
Test Automation Architecture
Application Under Test
Business Process Testing (BPT)
Component Component Component
Test Test Test Test Test
Resources:
-Functions
-Object Rep.
-Environment
HP Quality Center (QC)
HP QuickTest Pro (QTP)
SME-centricTest
Automation Process
A
u
t
o
m
a
t
i
o
n
E
n
g
i
n
e
e
r
Automation Architecture Design
Create Function Libraries
Create Object Repositories
Create Business Components
Drag Components to create tests in QC
Configure Input/Output parameters
Debug tests in QC Test Plan module
Add tests to test set in Test Lab module
S
u
b
j
e
c
t

M
a
t
t
e
r

E
x
p
e
r
t
o
r
T
e
s
t

E
n
g
i
n
e
e
r
SME-centric Process Advantages
Responsibilities are clearly defined:
Automation developers define architecture and create
components
Subject matter experts create automated test cases
SME doesnt need to know/use QTP.
Uses very flexible Keyword/Data-Driven approach.
Reduced learning curve due to same approach across
different applications/technologies
Applicable to all areas of automated testing, like
regression, smoke, acceptance, etc.
Designed for Maintainability
Automated Testing with Agile
Automation Engineers
SME Testers
Shared between Agile teams
Create BPT components
during each sprint
Modify components as
needed
Embedded in Agile teams
Write automated tests using BPT
components during each sprint
Execute automated tests at the end
of each sprint
Use BPT tests as regression tests
during later sprints
Benefits of New Approach
Leverage product knowledge of SMEs to write
more meaningful and reliable tests
Increase test coverage -> fewer product
escapes
Ability to keep up with increased product
workload without increase in staff
Reduce time spent analyzing automation failures
Faster execution of automated tests
The Results
The team created 5500 new, working BPT tests
in one year using this approach.
With the same budget, the team increased the
number of automated tests created by 5 times
using this approach.
Automated tests now comprise 30% of our total
different tests, and 60% of the total tests
executed on a given release.
The current project is to get test execution even
faster.
18
Igor Gershovich
President
Connected Testing
19
Automated Testing Process
Quality
Center
QTP
AUT
BPT Components,
Automation Resources
Results
AUT
Testing
Test case
Data
SETTING UP THE BPT
ARCHITECTURE
Step 1
21
Test Automation Elements
IDE: Quality Center with BPT/Mercury QuickTest
Professional (QTP)
Environment Files
Application Areas
Function Libraries
Object Repositories
Business Components and Test Scripts
Environment (BPT Test) for Computer
NameTest Environment Mapping
Computer name Test Environment mapping
Environment.xls
Function Libraries
Technology libraries
Web
GWT
.Net
Win32
Common libraries
Utility
DB
CommProtocols
Project Specific libraries
CREATING REUSABLE BPT
COMPONENTS
Step 2
26
Business Components
A component is a reusable module to perform
tasks on logically independent application parts
(Page or Window).
Components use Input/Output parameters for
data input/verification and navigation within
Page or Window.
Input/Output parameters also are used to pass
data between components.
Components can use iterations.
Login Component Steps
(Example for Web Application)
Component initialization
Load environment variables like URL, login
credentials, DB connection strings, etc.
Close all browsers except QC
Invoke an application
Login
Verification that login is successful
Functional Component Steps
(Example for Web Application)
Component initialization ComponentInit() function
Load environment variables like URL, login
credentials, DB connection strings from
Environment.xls file using QC OTA
Verify that Expected page is displayed
Side or Top menu navigation
Web Page internal flow
BPT Component Script and AUT
AUT
QTP
CREATING AUTOMATED TESTS
BY CONFIGURING PARAMETERS
WITHIN COMPONENTS
Step 3
31
BPT Test Case in Quality Center
Data
Parameter Naming Conventions
Prefix Object Example
lst List lst_User
txt Edit txt_User
chk Checkbox chk_User
rad Radiobutton/RadioGroup rad_User
tab Tab tab_User
tbl Table tbl_User
cell Cell in Table cell_User
file WebFile file_User
ele WebElement ele_User
cmb Combobox cmb_User
st Static Text (Read Only) st_User
.Apply Method
Object.Apply Parameter(Parameter_Name)
.WebEdit(Name).Apply Parameter(txt_Name)
Apply method functionality:
Performs data substitution
Verifies if object exists and visible
Makes decision what to do with Name edit object
based on Parameter data
Input Constraints for Edit (Text Box)
Object
Value Action Example
No action
<CLEAR> Clear edit field <CLEAR>
text Enter text Hello
[text] Verify text [Hello]
[<ENABLED>], [<DISABLED>] Verify if object is
Enabled/Disabled
[<EXIST>],[<NOEXIST>] Verify if object exist/no-
exist
[<REGEX>Pattern] Verify that Pattern
matches text
[<REGEX>TestCase.*]
Working with Objects in Grids
row=WebTable().GetRowIndex(Parameter(search_),clmn)
WebTable().CellApply row, clmn, objType,
Parameter(cell_)
iRow = WebTable().GetRowIndex(Parameter(search_Name),2)
WebTable().CellApply iRow,3,WebElement,Parameter(cell_Age)
Parameter prefixed with the word search_ indicates
which row to work with.
Row can be selected by using row number: #n
the text contained in the specified Column
Pattern
Parameters for objects in table cells are prefixed by
cell_
Example: Working with Grids
Data Substitution
Special tags for Dates, Times, SSNs, Unique
values, DB Queries, Random, etc.
<SSN> - get substituted with unique SSN
<TODAY> - get substituted with today date (03/19/10)
<RNDDIG 3> get substituted with 3 random digits
<DateTime YYYYMMDDHHmmss>
<DB DBEnvironment::SQLStr>
Tags can be combined
[<REGEX><TODAY> at .* AM]
Summary
This method provides a way for test automation
engineers to create re-usable components without
having to know all the ways the application will be used.
Subject matter experts can use the components to easily
create tests that can do anything that the application can
do without having to be software engineers.
This method allowed us to create a great number of
working automated tests in a relatively short time.
The project has been a major success and has helped
us achieve greater confidence in our release quality.
39
Q&A
40
41 2010 Hewlett-Packard Development Company, L.P.
To learn more on this topic, and to connect with your peers after
the conference, visit the HP Software Solutions Community:
www.hp.com/go/swcommunity