SOFTWARE QUALITY

ASSURANCE
(BISE 3033)
CHAPTER 6

 Software Testing
Integration Testing
 Integration testing is an intermediate phase
between unit testing and system testing
 Involves testing the modules in executable
combination (exe file)
 Is an iterative process involving the testing of sub-

assemblies which are repeatedly integrated into

larger sub-assemblies culminating in the testing of
the system as a whole
 Two aspects to consider:
 The testing of interfaces between modules
 The testing of the interface between the integrated
assembly and its environment
Integration Testing
 Each of the following modules, shown below in the
application's control flow chart, has finally passed unit
testing standards. How do you plan to conduct integration
testing?

B C D E

F G H I J K

L M
User Interface Testing
 User Interface Validation
 paper prototypes made early are great

 User Interface Evaluation

 Expert / Heuristic Evaluation
 Label Testing
 is a "rose" still a rose by any other name?
User Interface Testing
 User Interface Testing
 Cognitive Walkthroughs
 Usability Testing
 hopefully the SRS specifically defines usability criteria

 Random Testing
 automated random key/mouse presses
What's the problem?

http://homepage.mac.com/bradster/iarchitect/color.htm
Dialog Boxes
Cognitive Walkthroughs
 Participants
 real end user - sits at keyboard and performs tasks
 evaluator - takes notes and asks questions, mostly quiet
 developer - probably hidden or watches video
 Results
 effectiveness
 Will the user associate the next action with the appropriate
interface control?
 Will the user notice an action is available?
 If the correct action is performed, will the user see that
progress is being made toward solution of the task?
Heuristics
 Simple and natural dialogue
 Aesthetic and minimalist design
 Speak the user's language
 Minimize user memory load
 Recognition rather than recall
 Consistency
 Feedback
 Visibility of system status
 User control and freedom
 Clearly marked exits
 Shortcuts and Flexibility
 Good error messages
 Help users recognize, diagnose, and recover from errors
modified from http://en.wikipedia.org/wiki/Heuristic_evaluation
Validation Testing
 Alpha Testing

 Beta Testing
 Alpha Test Beta Test

What they do Improve the quality of

the product, integrate
Improve the quality of
customer input on the
the product and
complete product, and
ensure beta readiness.
ensure release
readiness.
When they happen Toward the end of a
Just prior to launch,
development process
sometimes ending
when the product is in
within weeks or even
a near fully-usable
days of final release.
state.
How long they last Usually very long and Usually only a few
see many iterations. weeks (sometimes up
It’s not uncommon for to a couple of months)
alpha to last 3-5x the with few major
length of beta. iterations.
 Alpha Test Beta Test

Who cares about it Usually involves

product marketing,
Almost exclusively
support, docs, quality
quality/engineering
and engineering
(bugs, bugs, bugs).
(basically the entire
product team).
Who participates Normally performed by
(tests) test engineers, Tested in the “real
employees, and world” with “real
sometimes “friends customers” and the
and family”. Focuses feedback can cover
on testing that would every element of the
emulate ~80% of the product.
customers.
 Alpha Test Beta Test

What testers should Plenty of bugs, crashes, Some bugs, fewer

expect missing docs and crashes, most docs,
features. feature complete.
How they’re addressed Much of the feedback
Most known critical collected is considered
issues are fixed, some for and/or implemented
features may change or in future versions of the
be added as a result of product. Only
early feedback. important/critical
changes are made.
What they achieve About methodology, About chaos, reality,
efficiency and regiment. and imagination. Beta
A good alpha test sets tests explore the limits
well-defined of a product by allowing
benchmarks and customers to explore
measures a product every element of the
against those product in their native
benchmarks. environments.
System Testing
 A complex activity that aims to test the system as a
whole
 Initially system testing is concerned with proving that
the system works correctly as a whole:
 it is merely a higher level of integration testing
 once this has been demonstrated to developers to their own
satisfaction, the process broadens in a number of
dimensions
 First
 the performance of the system rather than correctness
 Second
 test on the business system
 Third
 demonstrate technical correctness in the eyes of users
System Testing (cont..)

 System testing involve a broad range of users i.e.

true users, their managers, system support
staff, computer operations staff and auditors
 Many of the problems of system testing are not

technical problem, but problems of communication

and human nature.
 Those problems must have been dealt with

continuously throughout design and development.

 If not, system testing is likely to be not the finale of

development.
 Regression Testing Retesting

Regression testing is a type of software Retesting is done to make sure that the
testing that intends to ensure that tests cases which failed in last
changes like defect fixes or execution are passing after the defects
enhancements to the module or against those failures are fixed.
application have not affecting
unchanged part.
Regression testing is not carried out on Retesting is carried out based on the
specific defect fixes. It is planned as defect fixes.
specific area or full regression testing.
In Regression testing, you can include In Retesting, you can include the test
the test cases which passed earlier. We cases which failed earlier. We can say
can say that check the functionality that check the functionality which was
which was working earlier. failed in earlier build.
Regression test cases we use are Test cases for Retesting cannot be
derived from the functional specification, prepared before start testing. In
the user manuals, user tutorials, and Retesting only re-execute the test cases
defect reports in relation to corrected failed in the prior execution.
problems.
 Regression Testing Retesting

Automation is the key for regression You cannot automate the test cases for
testing. Retesting.
Manual regression testing tends to get
more expensive with each new release.
Regression testing is right time to start
automating test cases.
Defect verification is not comes under Defect verification is comes under
Regression testing. Retesting.

Based on the availability of resources Priority of Retesting over Regression

the Regression testing can be carried testing is higher, so it is carried out
out parallel with Retesting. before regression testing.
And the glue that holds it all
together…

 The Test Plan

 who

 what
 when
 where
 how
Test Plan Considerations
 What are the critical or most complex modules?
 make sure they get integration tested first
 probably deserve white-box attention

 Where have you had problems in the past?

 What training is required?
 conducting formal reviews
 use of testing tools
 defect report logging
IEEE 829 - Standard for
Software Test Documentation
Recommends 8 types of testing documents:
1. Test Plan
 next slide
2. Test Design Specification
 expected results, pass criteria, ...
3. Test Case Specification
 test data for use in running the tests
4. Test Procedure Specification
 how to run each test
5. Test Item Transmittal Report
 reporting on when components have progressed from one stage of
testing to the next
6. Test Log
7. Test Incident Report
 for any test that failed, the actual versus expected result
8. Test Summary Report
 management report
Test Plan Contents (IEEE 829 format)

1. Test Plan Identifier 11. Test Deliverables

2. References 12. Remaining Test Tasks
3. Introduction 13. Environmental Needs
4. Test Items 14. Staffing and Training Needs
see next slide
15. Responsibilities
5. Software Risk Issues
16. Schedule
6. Features to be Tested
17. Planning Risks and
7. Features not to be Tested Contingencies
8. Approach 18. Approvals
9. Item Pass/Fail Criteria 19. Glossary
10. Suspension Criteria and
Resumption Requirements

http://en.wikipedia.org/wiki/Test_Plan
Test Items
 Requirements Specification
 Design
 Modules
 User/Operator Material
 the user interface
 User Guide
 Operations Guide
 Features
 response time, data accuracy, security, etc
 System Validation
 alpha and beta testing
The V-Model
 The V-Model of software development illustrates
when testing activities should take place
 The test at each level exercise the corresponding
development activity-the same principles apply
no matter what software life cycle model used
 The most important factor for successful
application of the V-Model is the issue of when
the test cases are designed
 The test design activity always find defects in
whatever the tests are designed against

26
The V-Model (cont..)
 For example:
 Designing acceptance test cases will find defects
in the requirement
 Designing system test cases will find defects in the
functional specification
 Designing integration test cases will find defects in
the design
 Designing unit test cases will find defects in the
code
 If test design is left until the last possible
moment, these defects will only be found
immediately before those tests would be run,
when it is more expensive to fix them

27
The V-Model (cont..)

Requirements test Acceptance

s test
Functions test
System test
s
Design test Integration test
s
Write test Code Unit Run
test
s test tests

28
Tool Support for Life-Cycle
Testing
 Tool support is available for testing in every
stage of the software development life cycle
 It helps to derive test inputs or test data

29
Progress….
 Pre-Project Activities
 how will we define quality
 contract review
 writing a plan for quality assurance
 Development Activities
 Formal Reviews, Peer Reviews, unit testing, integration testing
 Test plan You are here
 Infrastructure Needs and Management Tasks
 Procedures and Work Instructions
 Training
 Configuration Control
 Tracking Progress
 Preventative Actions
 CMM
Thank You