Software Quality

Engineering
Software Quality Testing

SQE-SE3613 1
Software Testing Techniques
and Strategies
Contents

Static vs dynamic testing

Static analysis, Inspection, reviews and their types
Black-box, white-box testing and their types
 Software testing levels (unit, integration, system, acceptance and regression)

SQE-SE3613 Ibrar Arshad 2

Software Testing Technique
Software testing techniques are methods used to design and execute
tests to evaluate software applications.
Software Testing Techniques
• Static testing is a software testing method that examines a program
along with any associated documents but does not require the
program to be executed.
• Instead of executing the code, static testing is a process of checking
the code and designing documents and requirements before it's run
to find errors. The main goal is to find flaws in the early stages of
development because it is normally easier to find the sources of
possible failures this way.
SQE-SE3613 Ibrar Arshad 3
Cont..
• Dynamic testing, the other main category of software
testing, requires testers to interact with the program while it
runs.
• Dynamic testing assesses the feasibility of a software
program by giving input and examining output.
• Dynamic testing will try to find active bugs.
• From a time and cost perspective, dynamic testing is often
more expensive than static testing.

SQE-SE3613 Ibrar Arshad 4

1. Static Analysis
Static analysis involves a set of methods used to analyze software source code or object
code determine how the software functions and establish criteria to check its correctness.
Static analysis studies the source code without executing it and reveals a wide variety of
information such as the structure of the model used, data and control flow, syntax
accuracy, and more.
There are several types of static analysis methods-
• Control Analysis
• Data Analysis
• Fault/ Failure Analysis
• Interface Analysis

SQE-SE3613 Ibrar Arshad 5

2. Review and its Types
Software review is a systematic inspection of software by one or more individuals who work
together to find and resolve errors and defects in the software during the early stages of the
Software Development Life Cycle (SDLC). A software review is an essential part of the Software
Development Life Cycle (SDLC) that helps software engineers in validating the quality,
functionality, and other vital features and components of the software. It is a whole process that
includes testing the software product and it makes sure that it meets the requirements stated by
the client.
Objectives of Software Review
• Detecting Problems Early
• Enhancing Quality
• Team Collaboration
• Following Standards
SQE-SE3613 Ibrar Arshad 6
Types of Reviews
• Code review : is systematic examination (often as peer review) of computer
source code.
• Pair programming: is a type of code review where two persons develop code
together at the same workstation.
• Inspection: is a very formal type of peer review where the reviewers are
following a well-defined process to find defects.
• Walkthrough: is a form of peer review where the author leads members of the
development team and other interested parties through a software product and
the participants ask questions and make comments about defects.
• Technical review: is a form of peer review in which a team of qualified personnel
examines the suitability of the software product for its intended use and
identifies discrepancies from specifications and standards.
SQE-SE3613 Ibrar Arshad 7
Inspection
• Inspections are a formal type of review that involves
checking the documents thoroughly before a meeting and is
carried out mostly by moderators. A meeting is then held to
review the code and the design.
• Inspection meetings can be held both physically and virtually.
The purpose of these meetings is to review the code and the
design with everyone and to report any bugs found.

SQE-SE3613 Ibrar Arshad 8

Cont..
• There are several ways that software inspection can improve software quality:
• Identifying and resolving defects early: By identifying and resolving defects early in the development
process, inspection can prevent costly rework and delays later in the project.
• Enhancing code readability: Inspection can help improve the readability of the code, making it easier for
other developers to understand and maintain.
• Improving team collaboration: Inspection can help improve team collaboration by encouraging developers
to share their ideas and knowledge.
• Enhancing code maintainability: Inspection can help identify areas where code is difficult to maintain and
suggest improvements to make it more maintainable.
• Improving code efficiency: Inspection can help identify areas where code is inefficient and suggest
improvements to make it more efficient.
• Enhancing security: Inspection can help identify security vulnerabilities in the code and suggest
improvements to make the software more secure.
• Improving the overall quality of the software: By identifying and resolving defects early in the
development process, inspection can help improve the overall quality of the software and ensure that it
meets the needs of its users.
SQE-SE3613 Ibrar Arshad 9
Functional Vs. Structural Testing: What to
Test?
• Main difference: Perspective/view & focus
• Functional testing (External behavior)
• Views the object to be tested as black-box
• Focus is on input-output relationship
• Without involving internal knowledge in making the test cases
• Structural testing (Internal Implementation)
• Views the object to be tested as white-box
• Focus is on internal elements
• Develops test cases on the basis of knowledge of internal
implementations i.e. code

SE3613 Software Quality Engineering 10

Functional or Black-box Testing
(BBT)
• Verifies the correct handling of the external functions provided by the
software
• How?
• Through observation of the program external behavior during execution
• Simplest form: Ad hoc testing
• Start running the software and notice un-expected behavior such as crash etc.
• Another common form of BBT: Specification Checklist
• List the external functions that are supposed to be present
• Along with expected behavior or input-output pairing

SE3613 Software Quality Engineering 11

Functional or Black-box Testing
(BBT)
• BBT can follow the generic testing process
• In test planning, the focus is on
• Identifying the external functions to test
• External functions are usually associated with some user expectations,
from which both the input and the expected output can be derived to
form the test cases
• Deriving input-output pairing
The emphasis is on reducing the chances of encountering functional
problems by target customers

SE3613 Software Quality Engineering 12

Functional or Black-box Testing
(BBT)
• Example: Compiler
• Identified external functions:
• Compiler executes & terminates without any error
• Compiler executes but terminates abnormally
• Error message displayed on abnormal termination
• Input-output pairing
• Input: actual source code, illegal programs
• Expected Output
• Normal termination resulting into object or executable code within a
limited amount of time
• Abnormal termination resulting into absence of object or executable
code - Error message displaying the reason for abnormal termination
SE3613 Software Quality Engineering 13
 Structural or White-box Testing
(WBT)
• Verifies the correct implementation of internal units (statements, data
structures, blocks etc.) and relations among them
• Connection between execution behavior and internal units needs to be
made in WBT
• Various software tools are used for this purpose
• The simplest form of WBT is Statement Coverage Testing using various
debugging tools, or debuggers, which help us in tracing through program
executions
• Necessitates intimate knowledge of code
• So, who is best to do WBT?

14
Structural or White-box Testing
(WBT)
• Statement Coverage Testing

If A = 3, B = 9

SE3613 Software Quality Engineering 15

Structural or White-box Testing
(WBT)
• Statement Coverage Testing

If A = -3, B = -9

SE3613 Software Quality Engineering 16

 Structural or White-box Testing
(WBT)
• WBT can also follow the generic testing process but
• Planning plays a much less important role in WBT than in BBT
• Reasons:
• Due to extensive implementation details, WBT is typically limited to a
small scale
• Defect fixing is made easy through the dual role played by the
programmers as testers

17
 Comparing BBT & WBT
• Perspective: A Key difference
• BBT
• Views the objects of testing as a black-box
• Focus: on input-output relations or external functional behavior
• WBT
• Views the objects as a glass-box
• Focus: internal implementation details are visible and tested
• Objects:
• WBT tests small objects e.g. small software products or small parts of large
products
• BBT tests large objects i.e. software systems as a whole
18
 Comparing BBT & WBT
• Timeline:
• WBT in early sub-phases of testing such as unit and component testing
• BBT in late sub-phases such as system and acceptance testing
• Defect focus:
• BBT detects failures related to external functions
• WBT focuses on implementation related faults

19
 Comparing BBT & WBT
• Defect detection and fixing:
• BBT is effective for problems of interfaces and interactions
• WBT for problems localized within a small unit
• Tester:
• BBT by dedicated professional testers, or 3rd party IV&V (independent
verification and validation) bodies
• WBT by developers themselves

20
 When to Stop Testing?
• The issue may be broken down into two sub-issues:
• Local/small scale:
• When to stop a specific test activity?
• Global scale
• When to stop all the major test activities?
• May yield different answers, leading us to different
testing techniques
• Decision to stop testing can usually be made taking two
approaches
• Informal
• Formal 21
When to Stop Testing? - Informal
Approaches
• Two types:
• Resource-based criteria: Employed only when cost & schedule is
dominant attribute
• Stop when you run out of time
• Stop when you run out of money
• Activity-based criteria:
• Stop when you complete planned test activities
• Irresponsible → quality/other problems
• Irresponsible criteria to stop testing based on time & money as far as
product quality is concerned
• Completion of planned activities does not mean quality is achieved
22
 When to Stop Testing? - Formal
Approaches
• Two levels: Global level & Localized Level
• Global level: Exit from testing is associated with product release
• Stop when quality goals reached
• Obvious way to make such product release decisions is the use of various
reliability assessments (Direct quality measure)
• Various formal reliability assessment models exist
• Conclusion: reliability assessments should be close to what actual users would
expect
• This requires that the testing right before product release resembles actual usages by target users
• This requirement resulted in the so-called usage-based testing

23
 When to Stop Testing? - Formal
Approaches
• Localized level:
• Reliability definition based on customer usage scenarios
may not hold good in this situation
• Reason: many of the internal components are never
directly used by actual users
• Result: reliability criterion may not be meaningful
• Alternative exit criteria are needed e.g. “Products should
not be released unless every component has been tested”
known as coverage criteria

24
 Usage-based Statistical Testing
(UBST)
• Actual use of product by the users
• Users report the problems faced, post-product-release defect fixing
activities
• Expensive & harmful for reputation
• Vendors name it Beta testing probably to save reputation!!
• If actual usage can be captured and used in testing → product
reliability directly assured
• Problem: massive number of customers & diverse usage
patterns cannot be captured in an exhaustive set of test cases
• Statistical sampling is needed
25
 Usage-based Statistical Testing
(UBST)
• For practical implementation of such a testing strategy:
• Actual usage information needs to be captured in various models,
commonly referred to as “Operational Profiles” or Ops
• Applicable to the final stage of testing
• Exit criteria: reliability goals

26
 Coverage-based Testing (CBT)
• Both BBT/WBT employ test coverage (with different scope)
• Simple Checklists:
• BBT: Checklist of major functions
• WBT: Checklist of all the product components (e.g., all the statements)
• Other Approaches (More Formal methods)
• Formally defined partitions are similar to checklists but ensure:
• Mutual exclusion of checklist items to avoid unnecessary repetition,
• Complete coverage defined accordingly
• Specialized type of partitions:
• Input domain partitions into sub-domains
• Finite state machines
27
 Coverage-based Testing (CBT)
• Applicability:
• Particularly unit and component testing
• If used in later phases, then works at high abstraction levels
• Termination criteria: coverage goals

28
 Comparing CBT with UBST
• Perspective:
• UBST: user’s perspective
• CBT: developer’s perspective
• Stopping Criteria:
• UBST: reliability criteria
• CBT: coverage criteria
• Objects:
• UBST: large software systems
• CBT: small units

29
 Comparing CBT with UBST
• Timeline:
• UBST: late sub-phases (system/acceptance)
• CBT: early sub-phases (unit/component)
• Testing environment:
• UBST: similar to customer installation
• CBT: specific test environment
• Customer and user roles:
• UBST: active
• CBT: not active
• Tester:
• UBST: dedicated, professional testers
• CBT: developers usually perform the testing 30
 References
Software testing
glossary:
http://thiyagarajan.w
ordpress.com/glossa
ry/

31
Software Testing Levels
• Unit Testing
• Integration Testing
• System Testing
• Acceptance Testing
• Performance Testing
• Regression Testing

SQE-SE3613 Ibrar Arshad 32

Cont..
• Unit Testing
Unit testing is a type of software test approach where individual components or units of the
software are tested. The purpose is to validate that each unit of the software performs as
expected. The developers usually carry out unit testing in the development process.
• Integration Testing
Integration testing is where individual units are combined and tested as a group. The purpose
is to validate the interfaces between various units. Integration testing is usually carried out in
the later stages of the development process.
• System Testing
System testing is a software test approach where the entire software system is tested. The
purpose of system testing is to validate that the software system meets the functional and
non-functional requirements. The difference between system testing and integration testing is
that, in system testing, the entire software system is tested, while, in integration testing, only
the interfaces between various units are tested.

SQE-SE3613 Ibrar Arshad 33

Cont..
• Acceptance Testing
Acceptance software testing is where the software is tested to see if it
meets the customer’s or client’s requirements. The customer or client
usually carries out acceptance testing. The purpose of acceptance
testing is to ensure that the software meets the customer’s or client’s
expectations
• Performance Testing
Performance testing is a software test approach where the software is
tested to see if it meets the performance requirements. This includes
tests such as load testing, stress testing, and scalability testing.

SQE-SE3613 Ibrar Arshad 34

Cont..
• Regression Testing
This is a type of software testing where the software is tested
to see if there are any regressions. The purpose of this type of
testing is to ensure that the software still works as expected
after changes have been made to it. Regression testing is
usually carried out after the software has undergone changes
such as bug fixes, new features, or configuration changes.

SQE-SE3613 Ibrar Arshad 35