Software Testing in

Today’s world – What,

Why, How?

What’s your Perception?

Agenda

 Software Development/Test Life Cycle

 Testing Defined
 Test classification
 Requirements - translated into test cases across
various tests phases – walk thru
 Software Test Industries expectations
 Q&A
 Challenges to a Tester
 Taking testing further
 Wrap up
Software Development and Test Life
Cycle

Requirements Architecture/H Code Review System

gather/analysis LD review Testing

Unit/Module
Feasibility Make by re- Testing Acceptance/Be
Analysis use analysis ta Testing

Integration
Project/Product Design Plan Testing Packaging
Committed Freeze tests

Low Level Feature

Scoping of Testing Release
Requirements Design

Load Testing Maintenance

Coding
High Level tests
Design/Archite
cture Code Performance
Inspection Testing
What’s Testing? – This is my
Definition

 The art of performing actions to

measure, validate, verify and certify a
system’s or product’s capability, capacity
etc based on what it is designed or
intended to deliver
Testing Arena classified

 Unit/Module test (White box level)

 Integration test (Partial White box)
 Feature test/Feature Integration test (Black box test)
 Conformance test
 Usability test
 Scalability test
 Maintenance test
 Future proof test
 Third party integration/Interoperability test
 Load/Volume test
 Performance, Stress test (partial white box)
 Acceptance/Beta test
 Torture test
 Complimentary test
 Smoke test
Lets walk thru testing for a product

 Mobile
 Washing Machine
 ATM
 Elevator
 Banking software application
 Soft switch
 Vacuum Cleaner
Mobile – Requirement analysis

 Mobile (Lets take a feature – lock/unlock)

•Requirement description – The Lock/unlock feature should be available to
the user of Handset

• Functional description – The Lock/unlock feature to block and unblock the

handset should be available to the user of Handset. This should be possible
in 2 modes
• Automatic
• Manual
• Functional description continued –
• The parameter’s available will be – phone settings > display settings >
Auto lock (available values off, 10 sec, 15 sec, 30 sec)
• Manually the user can lock the system by pressing and holding the “*”
key continuously
Mobile – Test Phases (1)

• Unit/Module/Dev Integration testing –

• Check on Function points to work,
• module to integrate into main module.
• Input values * to trigger lock and “unlock” * to trigger unlock.
• Set timer, check on lock function call after timer fired.
• Feature testing –
• Deploy the Binary/build on Win CE OS
• Insert the std SIM – check on status of SIM detection/ main menu
• Test for auto lock
• Test for manual lock
• Tests for unlock (normal, at time of call, sms, alarm, fm, transfer….)
• Conformance testing –
• Per standards - * should be used for lock and unlock mechanism
• Use other keys to check on Locking system, Use other keys in
combination with * check on locking
• Use multiple */ multiple # for lock mechanism
Mobile – Test Phases (2)

• Usability testing –
• How easy is it to understand lock/unlock system
• How is it documented in Manuals.
• Does it work the way it is intended (response time?)
• Is it accessible and usable by everyone?
• Scalability testing –
• Does not apply?
• Maintenance testing –
• Apply abrupt call to check on unlock automatic, prolonged use
• Works with screensavers ON and does not create any latency
• Doesn’t get into problem at the time when both modes can be done?
• Future Proof testing –
• Can I upgrade firmware and still feature is not impacted?
• Are these standards laid out by standard body – not for change?
Mobile – Test Phases (3)

• Third party Integration/Interop testing –

• Third party system drivers dependencies exist? Works fine?
• Across different models/different flavors of OS?
• Works with third party with default/custom settings?
• Load/Volume test –
• Lock and unlock using simulation for 2 hours alternatively
• Lock/Unlock instructions per minute and execution
• Measure system memory, performance, latency, responsiveness, other
side impacts
• Performance/Stress testing –
• Ideal scenario – Always works – How much is it practically?
• Over a period of time with varied durations of testing * and unlock *
• Bombard the system with Lock and unlock - simulation
• Acceptance/Beta testing –
• Production setup/ Actual firmware – Deployment from step 1 to final
• Done in Customer premises – so you can deploy S/w on handsets
Mobile – Test Phases (4)

• Torture test –
• Give many *** to lock
• Pressing many keys altogether while locked
• Pressing all keys at a time
• Complimentary test –
• What has been tested in earlier phase which I need not duplicate
• Basic lock/unlock can be skipped – test further what’s left

• Smoke test– ( What’s it? Why required? How?)

• With all the tests written above chose tests at random and test
Testing Industries expectations

 Nice to have product for everyone

 Cover more bugs per test case
 Identify bugs in early phases (No last minute major issues)
 Device mechanisms to have minimum impact due to changes in code
 Achieve more than 90% code coverage
 Failure rate should be 0.01%
 Minimum resources for testing - No compromise on quality
 Build test cases that can be run by anyone
 Testing should not leave any major/critical issue in customer’s hand
 Maximum testing to be done with minimum cycle time.
 Nothing goes untested in spite of design modifications in between
 Whatever content promised has to be delivered 100% without any deviation
 Tests will be cumulative but time is not directly proportional to number of test
cases
Questions !!!

Who dares first?

Challenges to a Tester (1)

 How can I make sure I have tested enough?

 How can I decide on Configurations I need to test?
 How do I get enough information for my tests?
 When can I start testing?
 When should I stop testing?
 How can I measure performance?
 How could I substantiate that the test cases I have
framed are enough to justify quality?
 Is it possible to always produce quality output irrespective
of design?
Challenges to a Tester (2)

 Incase of Frequent code changes, how can I make sure

that I sustain quality ?
 Quality and number of tests - Are they directly
proportional !!!
 Can I test all possible combinations ?
 Is there any concrete tool to measure test efficiency ?
 Do I test functionality only as per requirement/design ?
 What are the test areas I am responsible for ?
 How much am I accountable ?
 Why do we need to prioritize test cases ?
 How do I prioritize test cases ?
 How is the decision to release a product made ?
Alas I ended up in testing
your patience !!! 

Did your Perception Change?