By Sajid Manzoor
AGENDA OF THE DISCUSSION
What is Performance Testing Goals of Performance Testing Importance of Performance Testing Types of Performance Testing Pre-requisites of Performance Testing Performance Management Performance Testing Cycle Performance Testing Activities Performance Symptoms and Issus Typical Order of Fixes Challenges with Performance Testing Performance Testing Best Practices Difference Between Performance Testing and Performance Engineering
WHAT IS PERFORMANCE TESTING??
Non-Functional testing to determine the system responsiveness, stability, reliability and scalability
PERFORMANCE TESTING GOALS
Access production readiness Compare two platform to compare which performs better Compare performance characteristics of system configurations Evaluate system against performance criteria Discover what parts of the application perform poorly and under what conditions Finding the source of performance problems Support system tuning
IMPORTANCE OF PERFORMANCE TESTING
Most users click away after 8 seconds of delay $4.4 billion business revenue loss due to poor web applications performance Failure of US Based Airlines computerized system to maintain flight plans in Nov 2009 Aberdeen found that inadequate performance could impact revenue by up to 9% Business performance begins to suffer at 5.1 seconds of delay in response times of web applications and 3.9 for critical applications
IMPORTANCE OF PERFORMANCE TESTING
An additional second of waiting on a website significantly impact customer satisfaction and visitor conversions Page viewers, conversions rate and customer satisfaction drops 11%, 7% and 16% respectively
IMPORTANCE OF PERFORMANCE TESTING
TYPES OF
PERFORMANCE TESTING
Load Testing Stress Testing Soak/Endurance Testing Volume/Spike Testing Scalability Testing Configuration Testing
TYPES OF PERFORMANCE TESTING (COND..)
PRE-REQUISITES OF PERFORMANCE TESTING
Stable build free from all major functional defects Performance testing environment similar to production environment No other testing should be performed while Performance testing Conduct Performance testing before going live Complete understanding and knowledge of the application
CALL CENTER APPLICATION INFRASTRUCTURE EXAMPLE
TYPES OF PERFORMANCE MANAGEMENT
PERFORMANCE MANAGEMENT
Proactive Approach
Reactive Approach
REACTIVE APPROACH
Performance Testing after the System Testing Advantages
Cost effective
Disadvantages
Difficult to resolve the performance bottlenecks after the complete development Defect removal cost will exponentially increase Whole system can be useless
PERFORMANCE TESTING CYCLE
CORE PERFORMANCE TESTING ACTIVITIES
Identify Test Environment Identify Performance Acceptance Criteria Plan and Design Tests Configure Test Environment Implement Test Design Execute Tests Analyze, Report and Retest
PERFORMANCE TESTING ACTIVITIES
Identify Test Environment: Performance test environment should be exact
replica of production environment
Identify Performance Acceptance Criteria: Identification of performance
testing matrices and acceptable value of each matrix Plan and Design Test: Identification of user test scenarios based on their importance, frequency and performance impact Configure Test Environment: Test environment setup as identified at first step Implement Test Design: Scripting of user scenarios with the help of a performance testing tool Execute Tests: Execution of scripted user scenarios against different user sets with different configurations Analyze, Report and Retest: Analyze the executed tests results with identified matrices, report results to stake holders and Retest if tests were not passed in previous iteration
PERFORMANCE SYMPTOMS AND ISSUES
Symptoms of Web Application Performance Problems Database Problems
Web Server Problems
Application Server problems
Network Problems
WEB APPLICATIONS PERFORMANCE PROBLEMS
Long user response time Memory leaks High CPU usage Too many open connections Length queues for requests Too many table scans of database Database deadlocks Erroneous data returned HTTP errors Pages not available
DATABASE PERFORMANCE PROBLEMS
Insufficient Indexing Fragmented Database Out-of-date Statistics Faulty Application Design
WEB SERVER PERFORMANCE PROBLEMS
High CPU Usage Poor Server Design Memory Problems
APPLICATION SERVER PERFORMANCE PROBLEMS
Poor Database Tuning Poor Cache Management Poor Session Management Poor Security Design
NETWORK PERFORMANCE PROBLEMS
TYPICAL ORDER OF FIXES
Improve current application design (Algorithms, caching, DB calls, memory use)
Upgrade hardware (RAM, CPU, network bandwidth)
Upgrade software infrastructure (OS, web server, database)
Upgrade system architecture (Client-server to basic n-tier, software and hardware changes, Use hardware load balancing /SSL
CHALLENGES WITH PERFORMANCE TESTING
Infrastructure Setup Collection and Analysis of Huge Data Identifying the Problem Root cause Team effort required (Product vendors, Architects, Developers, Testers, Database administrators, Network administrators) Accuracy of results Client involvement Testing inside the firewall Multidimensional applications (web, SAAS, mobile etc.) Testing on Live Server Expensive
PERFORMANCE TESTING BEST PRACTICES
Use Ramp up approach Repeat the tests 2-3 times to confirm the results Run tests at different times Run individual tests before combining them in a single test
[email protected] +92-42-35782346 www.Twitter.com/SwaamTech www.FaceBook.com/SwaamTech
