Instructor: Dr.

Syed Ali Raza

Department of Computer Science
GC University Lahore

Software
Software Testing Strategies
Software Testing
• Generic characteristics of software testing strategies
• Testing begins at module level and works outward towards the integration
of entire software system.

• Different testing techniques are required at different points of time.

• Testing and debugging are different activities, but debugging must be
accommodated in any testing strategy.
What Testing Shows
errors

requirements conformance

performance

an indication of quality
Verification and Validation
• Verification refers to the set of tasks that ensure that
software correctly implements a specific function.
• Validation refers to a different set of tasks that ensure
that the software that has been built is traceable to
customer requirements.

• Verification
• Does the product meet its specifications?
• Validation
• Does the product perform as desired?
Who Tests the Software?
• Developer
• Understands the system
• will test "gently“
• is driven by "delivery"
• Independent tester
• Must learn about the system
• will attempt to break it
• is driven by quality
Testing Strategy

System engineering
Analysis modeling
Design modeling
Code generation Unit test

Integration test
Validation test
System test
Testing Strategy
• We begin by ‘testing-in-the-small’ and move toward ‘testing-in-the-
large’
• For conventional software
• The module (component) is our initial focus
• Integration of modules follows
• For OO software
• our focus when “testing in the small” changes from an individual module
(the conventional view) to an OO class that encompasses attributes and
operations and implies communication and collaboration
Strategic Issues
• Specify product requirements in a quantifiable manner long before testing
commences.
• State testing objectives explicitly.
• Understand the users of the software and develop a profile for each user
category.
• Develop a testing plan that emphasizes “rapid cycle testing.”
• Build “robust” software that is designed to test itself
• Use effective technical reviews as a filter prior to testing
• Conduct technical reviews to assess the test strategy and test cases
themselves.
• Develop a continuous improvement approach for the testing process.
Unit Testing
• Unit testing -- focuses on the smallest element of software design
viz. the module.
• Corresponds to class testing in the OO context.
• Makes heavy use of white-box testing.

module
software to be
engineer tested results

test cases
Unit Testing

module
to be
tested
interface
local data structures

boundary conditions
independent paths
error handling paths
test cases
Unit Testing
Unit Test Generation Considerations:
Review Design information - develop unit test cases.
Stub: A stub is a simple implementation of a called
method or component that mimics the behavior of the interface
real method but usually with simplified, pre-determined local data structures
boundary conditions
responses. For example if a method in your class calls an driver independent paths
external service or database, you can create a stub for that
external service to return a fixed response, allowing you to error handling paths
focus on testing the logic of your method without actually
invoking the real external service. Module to
Driver: A driver is the opposite of a stub. It is a piece of
code that simulates the behavior of the component or
be tested
system that invokes the unit of code being tested. A driver
is used when the unit being tested is a lower-level module Test
and needs to interact with higher-level modules that are stub stub
not yet implemented. For example, if you are testing a
small component (like a function) that is called by a larger cases
system or main program. RESULTS
Unit Test Generation
• Interface considerations
• # of input parameters = # arguments?
• Parameter and argument attributes match?
• Parameter and argument units match?
• Correct order (if important)?
• Number and order of arguments for built-ins?
• Attempt to modify input-only arguments?
• Global variable definitions consistent?
• Constraints passed as arguments?
Unit Test Generation
• External I/O considerations
• Files attributes correct?
• OPEN/CLOSE correct?
Format specification matches I/O statement?
• Buffer size matches record size?
• Files opened before use?
• EOF handled correctly?
• I/O errors handled?
• Textual errors in output?
Unit Test Generation
• Data structure considerations
• Improper or inconsistent typing?
• Erroneous initialization or default values?
• Incorrect variable names?
• Inconsistent data types?
• Underflow, overflow and addressing exceptions?
Unit Test Generation
• Test cases must cover all execution paths
• Common computational errors to be checked:
• incorrect arithmetic
• mixed mode operations
• incorrect initialization
• precision inaccuracy
• incorrect symbolic representation of expression
• Other tests needed
• incompatible data types in comparisons
• incorrect logical operators or precedence
• comparison problems (e.g., == on floats)
• loop problems
Unit Test Generation
• Error handling tests
• Exception-handling is incorrect?
• Error description is unintelligible, insufficient or incorrect?
• Error condition causes system interrupt before error handling completed?
Benefits of Unit Testing
• Unit testing increases confidence in changing/ maintaining code.
• If good unit tests are written and if they are run every time any code is changed, we will be able
to promptly catch any defects introduced due to the change.
• Codes are more reusable.
• In order to make unit testing possible, codes need to be modular which increases reuability
• Development is faster.
• If you do not have unit testing in place, you write your code and perform that fuzzy ‘developer test’
(You set some breakpoints, fire up the GUI, provide a few inputs that hopefully hit your code and
hope that you are all set.)
• But, if you have unit testing in place, you write the test, write the code and run the test. Writing
tests takes time but the time is compensated by the less amount of time it takes to run the tests.
• The cost of fixing a defect detected during unit testing is lesser in comparison to that of
defects detected at higher levels. Compare the cost (time, effort, destruction,
humiliation) of a defect detected during acceptance testing or when the software is live.
Integration Testing Strategies
• A systematic approach for constructing program structure while
conducting tests to uncover errors associated with interfacing.
• Options:
• Tendency for Non-Incremental integration.. Big Bang approach …. Chaos !!
( usually ).
• Incremental integration - program is constructed and tested in small
segments.
• Top-Down Integration testing
• Bottom-Up Integration testing
• Sandwich Testing
• Regression Testing
• Smoke Testing
Big-Bang Approach
Unit Test
A
try this with extreme caution!
Unit Test
B
Unit Test
C
Integration Test
Unit Test
D
Unit Test
E
Unit Test
F
Top-Down Integration
• Begin construction and testing with main module.
• Stubs are substituted for all subordinate modules.
• Stubs are the modules that act as temporary replacement for a called module and
give the same output as that of the actual product
• Stubs may also record information about calls, such as an email gateway stub that
remembers the messages it 'sent', or maybe only how many messages it 'sent'.
• Subordinate stubs are replaced one at a time by actual modules.
• Tests are conducted as each module is integrated.
• On completion of each set of tests, another stub is replaced with
the real module.
• Regression testing may be conducted to ensure that new errors
have not been introduced.
 Top Down Integration
A
top module is tested with
stubs

B F G
stubs are replaced one at
a time, "depth first"

C
as new modules are integrated,
some subset of tests is re-run

D E
Top-Down Integration
• Advantages:
• Verifies major control or decision points early in the test process.
• With the use of depth-first integration testing, a complete function of the
software can be demonstrated. -- Confidence builder for
developer/customer.
• Disadvantages:
• Since stubs replace lower level modules, no significant data can flow
upwards to the main module.
• Stubs must allow all possible conditions to be tested.
• Possibly a very large number of stubs may be required, especially if the
lowest level of the system contains many methods.
Bottom-Up Integration
• This approach begins construction and testing with modules at the
lowest levels in the program structure.
• Low-level modules are combined into clusters.
• A driver is written to coordinate test case input and output.
• Drivers are dummy code, which is used when the sub modules are ready but the main
module is still not ready.
• The cluster is tested.
• Drivers are removed and clusters are combined moving upward in the
program hierarchy.
Bottom-Up Integration
A

B F G

drivers are replaced one at a

time, "depth first"
C

worker modules are grouped into

builds and integrated
D E
cluster
Bottom-Up Integration
Bottom Up Approach
• Advantages:
• Easier test case design and lack of stubs.
• Disadvantages:
• The program as an entity does not exist until the last module is added.
Sandwich Testing
• It combines the advantages of both Bottom-up testing and Top-
down testing at a time. A
Top modules are
• Combined approach tested with stubs
• Top down strategy for upper levels
• Bottom up strategy for subordinate levels. B F G

Worker modules are grouped into

builds and integrated
D E

cluster
Regression Testing
• Re-execution of some subset of tests already conducted to ensure that the
new changes do not have unintended side effects.
• Whenever software is corrected, some aspect of the software configuration
(the program, its documentation, or the data that support it) is changed.
• Regression testing helps to ensure that changes (due to testing or for other
reasons) do not introduce unintended behavior or additional errors.
• Regression testing may be conducted manually, by re-executing a subset of
all test cases or using automated capture/playback tools.
• The Regression test suite should contain three different classes of test cases :
• A representative sample of tests that will exercise all software functions
• Additional tests that focus on functions that are likely to be affected by the change.
• Tests that focus on software components that have changed.
Smoke Testing
• A common approach for creating “daily builds” for product software
• Smoke testing steps:
• Software components that have been translated into code are integrated into a “build.”
• A build includes all data files, libraries, reusable modules, and engineered components that are required to
implement one or more product functions.
• A series of tests is designed to expose errors that will keep the build from properly
performing its function.
• The intent should be to uncover “show stopper” errors that have the highest likelihood of throwing the
software project behind schedule.
• The build is integrated with other builds and the entire product (in its current form) is
smoke tested daily.
• The integration approach may be top down or bottom up.
Integration Testing
Integration Test Documentation
1 2 3 4 5
Scope of Test Actual Test Ref. &
testing Test plan
Procedure n Results Appendix

Test Schedule Environment Unit Test Expected

phases / Resources test case Results
and Overhead data for build
Test n
builds software
Order of environment
Integration
High Order Testing
• Validation testing
• Focus is on software requirements

• System testing
• Focus is on system integration

• Alpha/Beta testing
• Focus is on customer usage

• Recovery testing
• forces the software to fail in a variety of ways and verifies that recovery is properly performed

• Security testing
• verifies that protection mechanisms built into a system will, in fact, protect it from improper penetration

• Stress testing
• executes a system in a manner that demands resources in abnormal quantity, frequency, or volume

• Performance Testing
• test the run-time performance of software within the context of an integrated system
Final Thoughts
• Think -- before you act to correct
• Use tools to gain additional insight
• If you’re at an impasse, get help from someone else
• Once you correct the bug, use regression testing to uncover any
side effects