Presented by IBM developerWorks ibm.

com/developerworks/

Effective Software Testing

Tools and strategies for project success

Effective Software Testing

2007 IBM Corporation

Presented by IBM developerWorks

Agenda
Quality Management Test Management Code and Unit Testing Manual Functional Testing Break Automated Functional Testing Performance Testing Wrap Up & Resources

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Agenda
Quality Management Test Management Code and Unit Testing Manual Functional Testing Break Automated Functional Testing Performance Testing Wrap Up & Resources

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

IBM Rational software

A strategic partner in business innovation and transformation

Business imperatives
Global capability Business flexibility Time to value Risk & compliance management

Governance & lifecycle management

Business innovation & transformation

Governing the business process of software & systems delivery

Decades of proven process expertise & customer success

Alignment Control Efficiency

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

IBM Rational quality management solutions

Process and products to ensure quality throughout the delivery lifecycle and deployment
Implement

Increase code quality

Define & Analyze

Enable requirements based testing

Assess business analytics

Build & Deploy

Automate & trace the quality process

Test & Production

Ensure functionality, system performance & compliance

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

IBM Rational quality management

Connecting software results to business objectives
Continuous governance across the delivery lifecycle
Early and iterative quality management Insight and control business processes Efficient compliance management from the start
Attention to QUALITY should be here

COST

NOT here

LIFECYCLE STAGES

Comprehensive integrated platform

Quality support for entire delivery lifecycle Integrated test, change and defect management

Collaborative information and data sharing across roles

Open standards-based solutions Sharing of data, processes and common user-interfaces Support for multiple IDEs, APIs and test scripting languages
6

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Agenda
Quality Management Test Management Code and Unit Testing Manual Functional Testing Break Automated Functional Testing Performance Testing Wrap Up & Resources

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Integrated quality capabilities

ANALYZE

DESIGN VALIDATION

UNIT TESTING

OPTIMIZE GOVERN TEST LIFECYCLE

BUILD TESTING

FUNCTIONAL & SYSTEM TEST DEPLOY & MANAGE PERFORMANCE TEST

2008 IBM Corporation

Effective Software Testing

Presented by IBM developerWorks

Collaboration: Requirements Modeling Test Planning

Requirements Features Use Cases

Models

ANALYZE

Architect

Test - Plans, Strategy, Cases

Manage Change/Test
9 Effective Software Testing

Eclipse Makes Collaboration Happen

2008 IBM Corporation

Presented by IBM developerWorks

What is Test Management?

Plan
Plan Objective:

Create

Execute

Analyze

Organize and maintain an inventory of items to test

Create Objective:
Capturing the steps required to complete a given test

Execute Objective:
Assembling scripts in sequence to perform a suite of tests

Analyze Objective:
Determine current status of testing project
10 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

Challenges of Test Management

Comprehensive
Keep track of test assets including requirements, test scripts, logs, results and reports Ensure testing addresses the requirements and business use cases

Plan

Distributed

Analyze

Accommodate the requirements of both dedicated, independent QA teams as well as integrated project teams and distributed teams to ensure high operational integrity.

Extensible
Lower the cost of test lab management Simplify the management of tools from multiple vendors

Create

Configurable
Ensure compliance and enforcement of project and company policies Project-specific needs and workflow automation
11 Effective Software Testing

Execute
2008 IBM Corporation

Presented by IBM developerWorks

ClearQuest: The hub for life-cycle management

Govern the test and development process

IBM Rational ClearQuest

Test Change Defects

Single project view Global test project coordination Configurable, enforceable processes Extensible test ecosystem

12

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Single project view

Single solution to manage tests, defects and project change
Benefits: Consolidated, real-time view of the entire project status Traceable, auditable relationships between development, test and project artifacts
Manages test planning, test results, quality metrics, and defects Central repository with integrated version control Comprehensive quality metrics in project dashboards and reports Centralized user management

13

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Plan: Enterprise test management & planning

Manages test planning, test results, quality metrics, and defects Link requirements to tests Automates sharing and reuse of distributed assets Extensible customization of forms and workflows Auditable test change history e-Signature support Supports enterprise-level databases (Oracle, DB2, SQL Server, etc)

14

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Create and Execute: Extensible test ecosystem

Integrated with Rational, open source and 3rd party test automation tools for Test Execution and Analysis
Benefits: Lowers the cost of test lab management Simplifies the management of tools from multiple vendors
Integrated with Rational Functional Tester, Rational Manual Tester and Rational Performance Tester Partner and customer extensibility Open source plug-ins such as jUnit
Open source solutions
15

Partner solutions

Test automation tools

Third party test automation tools

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Analyze: Comprehensive quality metrics

Generate reports to monitor test progress
Summarize test case results Measure requirements coverage

Key considerations when creating schedules

Ability to report against test motivators Ability to monitor current project status

16

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Analyze: Test Reporting and Metrics

Built-in queries, charts, and reports:
Coverage/Suspicion Impact of requirement change on Test Cases Impact of defects change on Test Cases Status Planned tests vs. implemented Passed vs. failed in this iteration Failed test cases without attached defect Requirements without any test case defined Trend over time # of Test Cases # of Passed vs. Failed Test Cases

Link test results to defects and requirements Web publishing of quality metrics Build or Customize your own!

17

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Global project coordination

Distributed and co-located testing and development support
Benefits: Ability to leverage globally distributed talent Simplifies the complexities of geographically distributed test coordination
High scalability for large enterprise projects Web-based test planning and quality metrics reporting Automates sharing and reuse of distributed test assets

18

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Configurable, enforceable process

Customize and enforce your test processes and workflows
Benefits:
Ensure compliance and enforcement of project and company policies Configurable for project-specific needs and workflow automation Data collection and process customization & enforcement Auditable test change history e-Signature support for regulatory compliance Project security

e-Signature Approval
Requirement
19

Test Case

Test Execution

Validate Results

Approve

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Process Guidance
Integration with Rational Unified Process for Systems Engineering (RUP-SE) Tool Mentors provide guidance for activities User customizable views with user defined content
RUP

Improved navigation of RUP

RUP Advisor provides context sensitive guidance

Search is integrated with Eclipse search

20 2008 IBM Corporation

Effective Software Testing

Presented by IBM developerWorks

What Youll See: Test Planning with IBM Rational ClearQuest Test plan creation and analysis Test Reporting and Analysis

21

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Agenda
Quality Management Test Management Code and Unit Testing Manual Functional Testing Break Automated Functional Testing Performance Testing Wrap Up & Resources

22

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Collaboration: Modeling Test Planning Unit Tests Design/Code Reviews

Models
Design/Code Best Practice Review

Architect

Code

Test - Plans, Strategy, Cases

Unit Tests

Manage Change/Test
23 Effective Software Testing

Eclipse Makes Collaboration Happen

2008 IBM Corporation

Presented by IBM developerWorks

Collaboration: Test Planning Build (Unit) Tests

Build Tests {Unit Tests)

Code

Build

Results Test - Plans, Strategy, Cases (Defects)

Manage Change/Test
24 Effective Software Testing

Eclipse Makes Collaboration Happen

2008 IBM Corporation

Presented by IBM developerWorks

Challenge: Reliable Software Is Critical To Success

Challenges: Customers expect delivery of reliable applications
Application quality can be a key differentiator

Building reliable applications is hard!

Difficult to validate component and code quality Too much time do you spent tracking down memory corruption problems Integration always takes longer than planned Developers spend too much time tracking down irreproducible errors?

Tracking down just one problem can take a developer days or even weeks! Late Discovery of Defects increases rework

25

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Challenge: Reduce Rework Costs

CMM Level 1 CMM Level 3

Requirements 4.5 Months

Design 6.6 Months

Code 8.8 Months

Test 9.4 Months

Rework

Requirements 4.0 Months

Design 6.0 Months

Code 6.8 Months

Test 3.75 Months

Rework

A clear testing strategy & efficient testing processes can result in spending 50% less on testing costs and 20% improvement in development productivity!

Early, iterative testing delivers 80% less defects to production, resulting in significantly less rework and 45% more overall productivity!

Note: CMM Level 1 organizations spend 30%+ of project dollars on testing. CMM Level 3 organizations spend 15%.
26 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

Services, Business Processes & Component Tests

Benefits: Reduces risk of late defect discovery Validate component and code quality
Create test scenarios from WSDL files Support data-driven testing and stubbing WS-I conformance of WSDL files Validate SOAP message Automated code review Run-time analysis Embedded systems testing Test component Test |code

Test service

27

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Develop fast, reliable code

IBM Rational PurifyPlus
Developer test solution to ensure creation of fast, reliable code Detect memory leaks and corruption even in third party code Graphical view of performance data to identify bottlenecks Identify untested and dead code Multi-platform and multilanguage support
Unix, Windows, Linux, Java, .NET, VB6, C/C++ and more
28 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

Architecture review and rules-based analysis

IBM Rational Software Architect
Automatic detection of design patterns and important Object Oriented structures Structural anti-pattern detection of tangles, hubs, butterflies Automated refactoring of tangles through QuickFixes Verification of architectural integrity
The architect defines architectural constraints for the packages, classes, interfaces or their relationships These architectural constraints are recorded as rules based on pre-defined templates The rules verify the integrity of the architecture and report all violations of the architectural constraints

29

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Test web services, Java, J2EE, and Portal applications

IBM Rational Application Developer
Code Analysis Tools
Both static code analysis and dynamic runtime analysis Sequence diagram runtime trace with performance, thread and code coverage Advanced memory leak detection Remote data collection across multiple servers User-defined run-time probes Interactive reports and metrics

30

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Challenge: Validating SOA component functionality & performance

Ensuring functionality of composite application units
Web Services are the building blocks of SOA applications. A single service is a standalone component but may be used in dozens of systems. Some of those services will be yours, some not. How do you test a service without the rest of the system? Does it work? All the time? Under load? Are you sure?

The Web Service is ready for testing, but there is no user interface to exercise it.

We dont want to pull Programmers off their projects to write complex test harnesses.

How can we leverage our Business Process analysis in our services testing efforts?

31

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

IBM Rational Tester for SOA Quality

Code free test development
Simplify testing of GUI-less service
Automated generation of web service test client No programming knowledge is necessary to create, comprehend, modify, and execute a functional or performance test

Support a wide variety of users

Automated data correlation and data driven testing through a code-free test environment

Simplify service integration testing

Automated test creation from WebSphere Business Process Execution Language resources

Create, comprehend, modify, and execute a functional or performance test

Code Free Test Editor

32 Effective Software Testing

Graphical test editor enables both high-level and deeper detail views Support Web services standards (e.g. SOAP, HTTP(S), JMS, WS-Security, UDDI)
2008 IBM Corporation

Presented by IBM developerWorks

IBM Rational Performance Tester Extension for SOA Quality

Performance testing for Web applications
Validate system scalability
Automated generation of web service performance tests Real-time reporting of server response time and throughput
Web Service Response Time

Eliminate need for costly development of test code

Automated generation of web service test client

Get real time feedback on SOA efficiency

Broad platform monitoring support for deployed web services

User action throughput

33 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

Agenda
Quality Management Test Management Code and Unit Testing Manual Functional Testing Break Automated Functional Testing Performance Testing Wrap Up & Resources

34

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Collaboration: Test Planning (Test Cases) Requirements Test Scripts - Results (defects)

Requirements

Architect

Features Use Cases

Code

Test - Plans, Strategy, Cases

Functional Tests (scripts)

Results (Defects)

Manage Change/Test
35 Effective Software Testing

Functional Test
2008 IBM Corporation

Presented by IBM developerWorks

What is Manual Functional Testing?

The manual process of ensuring fitness for use Using human labor to validate the business use cases and fitness for use of an application, system or device Includes labor to plan, design, execute and evaluate the status of the application or device under test Experience indicates over 75% of the testing done today is performed manually

36

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Challenges of Manual Functional Testing

Three things that complicate the life of a manual tester Three things that complicate the life of a manual tester Labor Intensive Labor Intensive
Too much time is spent maintaining Too much time is spent maintaining manual test documentation for manual test documentation for application changes application changes

Resistant to Change Resistant to Change

Each project has diverse processes, Each project has diverse processes, tester skill sets and types of tester skill sets and types of applications and devices under test applications and devices under test

Unreliable Unreliable
Accidental human-error with data Accidental human-error with data entry and validation often skews test entry and validation often skews test results results
37 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

Transform Manual Testing with IBM Rational Manual Tester

From ad-hoc testing to controlled and efficient testing From ad-hoc testing to controlled and efficient testing Labor Intensive
Minimizes test maintenance with linked content. Make changes in only one location to update all tests.

Resistant to Change
Provide a consistent interface that can be used to document tests for any device from can opener to computer

Unreliable
Automate data entry and data validation during test execution to minimize human error Rational Process advisor provides guidance and expertise to enhance tool experience
38 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

Consolidate Test Creation and Maintenance

Benefits: Benefits: Leverage Existing Test Documents Leverage Existing Test Documents Rich Editing Environment Rich Editing Environment
Rational Manual Tester test Microsoft Word documents

Jump-start adoption of Rational Manual Jump-start adoption of Rational Manual Tester by leveraging existing assets Tester by leveraging existing assets Bulk import manual tests documented in Bulk import manual tests documented in Word or Excel Word or Excel Provides a rich text editing environment Provides a rich text editing environment to document clear, easy to understand to document clear, easy to understand tests tests Allows embedding of images or attach Allows embedding of images or attach files to add clarity and detail files to add clarity and detail Provide a consistent Test Interface Provide a consistent Test Interface

Microsoft Excel spreadsheets

39

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Reduce test maintenance and ensure consistency

Benefit: Benefit: Enable Shared Content Across Enable Shared Content Across Multiple Tests Multiple Tests

Create commonly used test statements Create commonly used test statements once once Reuse statements across multiple tests Reuse statements across multiple tests through simple drag and drop through simple drag and drop Maintenance of shared content is Maintenance of shared content is efficient with single-point updating of efficient with single-point updating of shared content shared content Ensures consistency and reduces test Ensures consistency and reduces test maintenance maintenance

40

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Assisted Data Entry and Data Verification

Benefit: Reduce human error
Store critical data used during the test as part of the test instructions Automate data entry during test execution to speed entry and reduce errors Automate data validation during test execution to ensure accurate results

41

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

What Youll See: Manual testing with IBM Rational Manual Tester Test creation, reuse and execution

42

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Agenda
Quality Management Test Management Code and Unit Testing Manual Functional Testing Break Automated Functional Testing Performance Testing Wrap Up & Resources

43

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

What weve seen so far: Test Planning challenges, tools and demo Code and Unit testing challenges and tools Manual Functional Testing challenges, tools and demo Whats coming up next: Automated Functional Testing challenges and tools Performance Testing challenges, tools and demo

44

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Agenda
Quality Management Test Management Code and Unit Testing Manual Functional Testing Break Automated Functional Testing Performance Testing Wrap Up & Resources

45

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Collaboration: Test Planning (Test Cases) Requirements Test Scripts - Results (defects)

Requirements

Architect

Features Use Cases

Code

Test - Plans, Strategy, Cases

Functional Tests (scripts)

Results (Defects)

Manage Change/Test
46 Effective Software Testing

Functional Test
2008 IBM Corporation

Presented by IBM developerWorks

What is Automated Functional Testing?

The automated process of ensuring fitness for use Using automation tools to capture actions taken against an application in script format.

Record
Three Step Process
Record actions into a script format either VB.NET or Java Optionally enhance scripts with custom coding Execute scripts ideally overnight to increase test productivity to 24/7

Enhance Execute

47

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Challenge of Manual Functional Testing with Short Test Cycles

...the new tests... Rerun the first tests and...

Manual testing cant keep pace!

...take more time Time

Tests Build 1
48

Build 2

Build 3, 4, 5

Manual testing alone cant keep pace with the rapid development of applications..
Quality will be sacrificed to keep on schedule
2008 IBM Corporation

Effective Software Testing

Presented by IBM developerWorks

Challenges of Automated Functional Testing

Three things that complicate the life of an automated functional tester Three things that complicate the life of an automated functional tester Control cost of test automation Control cost of test automation
Inexperienced testers face steep Inexperienced testers face steep learning curve for tools learning curve for tools Test maintenance grows with project Test maintenance grows with project

Testing Tools Lack Flexibility Testing Tools Lack Flexibility

Power users hands are tied by tool Power users hands are tied by tool limitations limitations

Test Team Integration Test Team Integration

Managing script changes with parallel Managing script changes with parallel development development Sharing test assets with the Sharing test assets with the larger software team larger software team
49 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

Functional Testing with IBM Rational Functional Tester Plus

Test automation for the novice and the professional Control cost of test automation
ScriptAssure technology ensures test script playback even with changes in application geography Wizard assistance for common tasks such as data driven tests

Testing Tools Lack Flexibility

Full flexibility of Java or VB.net to address all application testing challenges Firefox browser support v1.5 and v2.0

Test Team Integration

ClearCase LT included to incorporate support for parallel development and geographically distributed teams Rational Process advisor provides guidance and expertise to enhance tool experience
50 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

Control of Test Automation

Recording Scripts
Benefits: Lowers cost of application testing Improves test productivity and accuracy

Creates a Test Script that captures all system-user interaction Test scripts are recorded on the fly, as user navigates application Verification points are inserted to validate system response Formalize end-to-end tests that exercise business processes Automate functional testing of composite applications Broad support for modern and legacy application environments Minimize test maintenance with tests resilient to frequent changes

51

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Flexibility: Recording Scripts

Environment Support
IBM Rational Functional Tester Plus supports the following environments:
Web, Java, J2EE Any VS.NET application running under the .NET Framework Terminal Emulation
Support for 3270/5250/VT100/VT220 terminal-based applications

Legacy Applications:
Visual Basic C/C++ PowerBuilder Delphi Win32

Enterprise Resource Planning Applications

Support for Siebel and SAP Implementations Peoplesoft supported as a Web Application

52

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Flexibility: Enhancing Scripts

Flexible Coding Language
IBM Rational Functional Tester uses Java or VB.net for scripting
Standard language syntax Not a custom version of Java or VB.net Augments language commands with test specific functions Click, Verify, Select, etc

Flexible power to enhance scripts

Programmatic access to all GUI objects Datapool facility enables data driven tests Leverage existing code and resources from a variety of sources Books, Internet, developerWorks, etc

53

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Control Cost of Test Automation

Reliable Playback with Script Assure

Version 1.0
Tester Sees

Version 2.0

No User Intervention Required With Tool TM Sees ScriptAssure

Determines Match
54 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

Test Team Integration

Version Control of Scripts
Functional Tester Functional Tester includes ClearCase LT includes ClearCase LT for test script version for test script version control control ClearCase LT enables ClearCase LT enables parallel script parallel script development development
Work simultaneously on Work simultaneously on multiple releases multiple releases Coordinated workflows Coordinated workflows across the team across the team Secure, versioned Secure, versioned software test assets software test assets

55

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

What Youll See: Functional Testing with IBM Rational Functional Tester Test creation, modification and execution

56

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Summarizing Key Features in IBM Rational Functional Tester Plus

Control Cost of Test Automation Control Cost of Test Automation
Create data driven tests without coding Create data driven tests without coding Static data and properties verification Static data and properties verification Dynamic data validation without coding Dynamic data validation without coding Reliable playback with ScriptAssure Reliable playback with ScriptAssure Remote and local playback on various platforms Remote and local playback on various platforms

Flexibility Flexibility
Flexible coding language Flexible coding language Broad Environment Support: Broad Environment Support: Web, Java. .net, IE and Firefox Web, Java. .net, IE and Firefox Powerful, professional debugger Powerful, professional debugger Object map editing flexibility Object map editing flexibility

Test Team Integration Test Team Integration

Version control of scripts Version control of scripts
57 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

Agenda
Quality Management Test Management Code and Unit Testing Manual Functional Testing Break Automated Functional Testing Performance Testing Wrap Up & Resources

58

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Collaboration: Performance Testing routine and issue related

Requirements

Architect

Features Use Cases

Code

Test - Plans, Strategy, Cases

Performance Tests

Results (Problems)

Manage Change/Test
59 Effective Software Testing

Performance Test
2008 IBM Corporation

Presented by IBM developerWorks

What Is Performance Testing?

The process of exercising an application by emulating actual users with a load generation tool for the purpose of finding system bottlenecks

Controller

System Under Test

60 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

Why do Performance Testing?

Because a break at any point in your system means your customers are not getting the service you think they are

System Under Test

61 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

Challenges of Performance Testing

Three things that complicate the life of a performance tester
No performance test experience
Tool complexity and lack of experience intimidates many first time users

Complexity of system under test

Complexity of system under test prohibits simple record and playback

Testing tools lack insight

Tool can find the problem, but not diagnose the root cause

62

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Performance Testing with IBM Rational Performance Tester

Problem discovery and diagnosis with Day 1 Productivity
No performance test experience
Performance Tester masks complexity of scripts and automates task of ensuring data consistency

Problem Discovery

Complexity of system under test

Automatic data referencing and data pooling eliminates need for time consuming script modifications

Problem Diagnosis

Testing tools lack insight

Pinpoint cause of application slow downs for J2EE applications Interface with Tivoli products for complete environment analysis
63 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

Creating a Performance Test

Creating a performance test is a three step process

Build Scripts

Schedule Workload

Execute & Analyze

Script Creation Considerations

Visual test editor, varying input data & correlating server responses

Scheduling Considerations
Accurately representing a true user workload

Execute and Analyze Considerations

Validating responses & finding the bottleneck
64 2008 IBM Corporation

Effective Software Testing

Presented by IBM developerWorks

Reduce Complexity with Performance Tester:

Create Performance Tests without Coding
Create simple, tree-based tests Remove the need for programming Tests are represented in a tree view as a sequential flow of web pages Flexible modeling and emulation of diverse user populations User interface enables:
Data pooling
Easy to Read: Performance Tests are represented as tree view of web pages. No code involved.

Automated identification and variation of dynamic data

Looping Conditional events

Data Referencing Data value used during playback will be dynamically linked to previous server response Data Pooling Data value used during playback will be unique value for each user read from datapool

65

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Testing Flexibility with Performance Tester

Performance Tester provides support for the following Environments: Web Siebel SAP SAP GUI and mySAP Citrix Presentation Server Partner BSD offering for Oracle eBusiness Provides cross-tier transaction breakdown and resource monitoring to pinpoint bottlenecks Optional insertion of Java code for advanced customization Use Windows, Linux, and UNIX servers or z/OS hardware to generate load

66

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Performance Problem Identification During Test

Page Performance Report Shows average response time per page. Highest bar = Performance Problem

Performance Testing finds bottlenecks

Next logical question is Why? Root Cause Analysis provides to tools to answer this question
67 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

Performance & Resource Statistic Report Overlay

Identifying hardware related performance problems

Data from resource monitoring can be displayed on same graph as response time data Single view to visually correlate system resource and system response data for faster problem solving
68 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

Response Time Breakdown

Identifying software related performance problems

Response data broken down by tier and by transaction component (JDBC, JSP, Servlet, etc)
Breakdown page response times into composite element response time
Breakdown response time usage by physical application tier On each tier, break down response time by coding function Ability to drilldown to source code of poorly performing code
69 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

Deep Diagnostic Data

Providing the details you need to solve the performance problem mystery

Execution Statistics Shows response time for every method

UML Sequence diagram showing method calls between classes with timing information. Find out what happened just before the slowdown.
70 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

What Youll See: Performance Testing with IBM Rational Performance Tester Test creation, modification and execution Response and Resource Utilization Analysis What You Wont See: Programming or Code
71 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

Summarizing Key Features in Performance Tester

Built for Day 1 Productivity
Create code free tests Mask complexity to get the job done

Advanced Data Access & Manipulation

Create code free schedules Automated data variation and synchronization

Find and Diagnose Performance Issues

Quickly locate performance problems Diagnose the root cause of performance problems

72

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Agenda
Quality Management Test Management Code and Unit Testing Manual Functional Testing Break Automated Functional Testing Performance Testing Wrap Up & Resources

73

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Get additional resources from IBM developerWorks

Technical resources How-to information Sample code Roadmaps Trial downloads Tutorials Technical articles Webcasts Demos Academic portals Sample projects Web communities Technology zones Brand zones alphaWorks Discussion forums Blogs Wikis RSS Feeds Podcasts Localized sites in key non-US markets
Effective Software Testing
#1 Rated Vendor Developer Program (IDC) 2nd largest developer community in industry 32 industry awards

Support in the community developerWorks Live! Briefings User Group support Academic & University events
2008 IBM Corporation

74

Presented by IBM developerWorks

Test-drive IBM Rational software

developerWorks provides easy access to trial versions of the latest IBM Software Download free trial versions of the Rational products youve seen today:
IBM Rational Functional Tester IBM Rational Manual Tester IBM Rational Performance Tester IBM Rational ClearQuest IBM Rational RequisitePro IBM Rational PurifyPlus

Dont want to download? Request your copy of the Software Evaluation Kit (SEK) DVD More than 14 GB of downloadable trial code on two DVDs
Software Evaluation kit DVD

Available for Linux or Windows For more trial downloads and a collection of supporting materials, including complimentary tech support, discussion forums, installation guides, demos and tutorials visit:

ibm.com/developerWorks/downloads
75 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

New! Test-drive IBM Rational software online

In addition to the trial downloads, you may now explore the capabilities of some Rational products online.
Register to receive an entrance ID & password and explore the product for up to three hours per session Use your own data and files during your online trials to try out real-world scenarios Products available:
Evaluate without installing or configuring your system!

New! IBM Rational Build Forge

ibm.com/developerWorks/onlinetrials
76 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

New! IBM Rational Tester eKit

Its never been more important to achieve predictability and quality in businesscritical applications. The IBM Rational tester kit provides a collection of tools and best practices across the life-cycle to make your job easier. The kit contains:
Rational product demos Case studies Webcasts Articles Tutorials designed to help you optimize manual testing Datasheets on IBM Rational products

Register for the complimentary IBM Rational tester kit and make quality an integral part of all aspects of the application life-cycle:

ibm.com/developerworks/rational/kits/tester/
77 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

Complimentary events from developerWorks

IBM Rational Software Delivery Platform webcast series: View technical webcasts covering IBM Rational. See whats coming up in our 2007 schedule or browse the library of more than 80 recorded titles. Topics include:
Scaling the Business of Global Architecture and Design Why good architecture matters for SOA and how to achieve it Flexibility of choice: IBM Rational solutions for open source software

Register for complimentary live, worldwide events covering PPM, the IBM Rational Software Delivery Platform, SOA, Linux, WebSphere, Information Management, Integration & Infrastructure, IBM Workplace, and more!

ibm.com/developerworks/offers/techbriefings/events.html
78 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

2008 Planned Briefings

IBM Middleware on Linux Information on Demand Live!: Building the Next Generation of Database Applications Architecture, Design & Construction using the IBM Rational Software Delivery Platform Effective Software Testing Open Community Tools: An Open Stack Development Platform Open Source Development: Tools and Open standards Eclipse: Empowering the Universal Platform Achieving Enterprise Application Security The developer and the IBM Environment The Developer and the IBM on demand Environment Building SOA Solutions and Managing the SOA Lifecycle SOA Governance: Implementing the IBM method Rational vs Mercury Rational vs Microsoft Collaboration with Lotus Get Started on SOA with WebSpheres Proven, Flexible Entry Points Experience the Power of Information Management Solutions Change and Release Management for Software Development 79 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

2008 Briefings (page 1 of 3)

Achieving Enterprise Application Security: This briefing covers application security end-toend. It covers the goals of application security with a review of basic security concepts and shows how the basics are applied to building of security stacks, to add additional layers of defense, using Java / J2EE as the programming language and environment to illustrate technologies and techniques. day

Architecture, Design, and Construction using the IBM Rational Software Delivery Platform: This briefing will demonstrate the latest version of the IBM Rational Software
Development tools, their broad range of functionality and their use throughout the entire software development process, focusing on application modeling, design, development, coding, and testing. day

Eclipse: Empowering the Universal Platform: This briefing takes a deep dive into some of
the most important and feature rich projects that the Eclipse community is developing. From multilanguage support to plug-in development, it shows how Eclipse has evolved into a universal platform complete software development. day

Effective Software Testing: Focus on variety and thoroughness of IBM Rationals Software
Quality solutions and best practices and demonstrates the framework and tools needed in software testing as a strategic business advantage. day

ibm.com/developerworks/offers/techbriefings
80 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

2008 Briefings (page 2 of 3)

IBM Middleware On Linux: This briefing includes an overview of Linux as the premier open
computing platform and the value it brings to any business. It introduces key IBM middleware products that run on Linux with a focus of products in the development space. day

Information on Demand Live! - Building the Next Generation of Database Applications: A demonstration of IBM Information Management tools & services available to
support rapid development of integrated solutions for managing expanding volumes of data by efficiently handling XML and relational data with ease to build agile applications faster. day

Open Community Tools: An Open Stack Development Platform: This briefing explores
the innovation of Open Source solutions available from IBM and illustrate the power and flexibility of IBMs Open Community development tools. day

Open Source Development: Tools and Open Standards: Focus on IBMs role in "Open"
source tools and Open standards day briefing or workshop

ibm.com/developerworks/offers/techbriefings
81 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

2008 Briefings (page 3 of 3)

The Developer and the IBM on demand Environment: This briefing discusses
why IBM Software? Open, Scalable, Secure, and Industry-focused; IBM software products and industry solutions help you innovate and become more flexible while making the most of current resources and controlling costs. This briefing is for anyone seeking more knowledge on what the IBM software is capable of today in todays on demand environment and provides an overview of the 5 main Software brands from IBM 1 day

Building SOA Solutions and Managing the SOA Lifecycle: This briefing
provides an in-depth look at the Service Lifecycle, from service planning and definition, analysis and modeling, development, deployment, and versioning through service retirement. It is intended for software developers, solution architects, and development project team members day

Change and Release Management for Software Development: This briefing

focuses on the challenges and solutions of Change and Release Management in a Distributed Development Environment. Issues such as Communication, Collaboration and Automation will be addressed. Case studies, based on the Game Development industry, will be referenced to show how improved speed, quality and automation of software production has been achieved with IBM Rational Build Forge integrated with ClearCase and ClearQuest day

ibm.com/developerworks/offers/techbriefings
82 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

developerWorks podcasts
Tune in to hear the latest MP3 recordings on a variety of technical subjects by industry experts and luminaries. This week on developerWorks podcast features developerWorks content highlights for the week and interviews authors, editors, and other developerWorks contributors in an entertaining 5-8 minute podcast. Other podcasts include IBM Fellow Grady Booch as he discusses innovation and evolution in IT, being an IBM Fellow, and keeping an eye on the horizon.

Subscribe to the RSS feeds and the new developerWorks podcasts will come to you!

ibm.com/developerworks/podcast
83 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

developerWorks community resources

User Groups:

Join independent, user-run member-ship organizations to collaborate and learned from shared knowledge and experiences

Blogs:

Read the latest blogs from IBM & other industry experts
Technical forums:

Chat with your peers and other technical experts about IBM products and technologies

ibm.com/developerWorks/forums
84 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

Fast track your skills with On demand demos

Visit developerWorks or our partner portal on DevX for easy access to: On demand demos featuring the latest IBM products and technology Real world development scenarios Technology-focused how to training demos ibm.com/developerWorks/offers/lp/demos Short video clips Beginner to advanced skill levels Additional product and technical resources to accompany each demo

devx.com/OnDemandDemos
85 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

alphaWorks emerging technologies

alphaWorks.ibm.com
86 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

developerWorks Software Testing Briefing Space

IBM Podcast Collection

Blogs

Tutorials and Articles

ibm.com/developerworks/briefings/testing
87 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

User Group Communities

ibm.com/developerworks/usergroups
88 Effective Software Testing 2008 IBM Corporation

Presented by IBM developerWorks

IBM Software User Group Community

Tivoli
Tivoli software user communities: ibm.com/software/tivoli/tivoli_user_groups/community.html Global Software User Group community: tivoli-ug.org

WebSphere
Global WebSphere User Group community: websphere.org

Information Management
Enterprise Content Management/UserNet: filenet.com/usergroup U2 User Groups: u2ug.org

Rational
Global Rational User Group community: rational-ug.org

89

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

IBM Academic Initiative

ibm.com/university/scholars/academicinitiative
90 Effective Software Testing 2008 IBM Corporation

Presented Group developerWorks IBM Softwareby IBM | Rational Software

IBM Rational Software Development Conference

June 1-5, 2008

Walt Disney World Swan and Dolphin Orlando, Florida

Attend the 2008 Executive Summit Over 300 Sessions 14 Tracks 3 & 5 Hour Technical Workshops Keynotes with Industry Leading Experts Exhibit Hall Showcasing Complimentary Product and Services Unlimited Networking Opportunities Access to the Executive Summit BOA Workgroup Sessions IBM Solution Center Interactive Birds-of-a-Feather Sessions Evening Receptions Over 3,000 Customers and Partners Access to IBM Engineers & IBM Research Register with this discount code (HDDE) and receive $100 off your registration fee!

91

Visit www.ibm.com/rational/rsdc for more information

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

Download a PDF of this technical presentation from

ibm.com/developerworks/offers/techbriefings/details/testing.html

Michael Dundek [email protected]

92

Effective Software Testing

2008 IBM Corporation

Presented by IBM developerWorks

93

Effective Software Testing

2008 IBM Corporation