Quality Assurance and Testing

Introduction to Software Testing

Methods

Software Testing, Birzeit University – Samer Zein (Ph.D.) 1

My Favorite Quote on Software Testing

2
 Why?

● Software development involves: Developers often overlook

○ ambiguity
fundamental ambiguities
○
○
assumptions
Loading…
and flawed human communication in requirements in order
● Each change made to a piece of to complete the project;
software, each new piece of
functionality, each attempt to fix a
or they fail to recognize
defect, introduces the possibility of them when they see
error them.
● Software testing reduces this risk

3
 Introduction

● The procedures that will be Software testing is a

presented in this course can help
process, or a series of
developers and managers better
understand the value of processes, designed to
comprehensive software testing. make sure computer code
● Provide guidelines to achieve does what it was designed
testing goals to do and, conversely,
● Software should be consistent and
that it does not do
present no surprises to users.
anything unintended

4
 Introduction..2

● a well-known rule of thumb that in a typical programming project

approximately 27 - 50%of the elapsed time and more than 24 -
Loading…
50% of the total cost were expended in testing the program or
system being developed.

● Even though new programming languages, tools, IDEs have been

developed, but

“Software testing remains among the dark arts of

software development”, (Mayers, Sandler 2011)

5
 Developers vs Testers

● Designers and developers approach software with an optimism

based on the assumption that the changes they make are the
correct solution to a particular problem.
● The tester takes nothing at face value.
● The tester always asks the question “why?”
● They seek to drive out certainty where there is none.
● They seek to illuminate the darker part of the projects with the
light of inquiry.
6
 Test Cases and Test Scenarios

● Test Case: “A test case is a set of conditions or variables under which a

tester will determine whether an application, software system or one of
its features is working as it was originally established for it to do.”
● Test Scenario: “The exhaustive testing is not possible due to large
number of data combinations and large number of possible paths in the
software. Scenario testing is to make sure that end to end functionality
of application under test is working as expected. Also check if the all
business flows are working as expected.”
7
Test Case Elements

8
 Test Case Elements..2

● In addition to these basic elements, agile test cases may also include additional
information such as:
○ Priority: The priority of the test case, which can be used to determine which test
cases to execute first.
○ Severity: The severity of the defect that the test case is intended to detect.
○ Owner: The person responsible for executing the test case.
○ Notes: Any additional information about the test case, such as known issues or
workarounds.

9
Example Test Case

Test case ID: TC001

Test case description: Verify that a user can log in to the system.
Pre-conditions: * The user has a valid user account.
* The system is up and running.
Test steps: 1. Go to the login page.
2. Enter the user's username and password.
3. Click the "Login" button.
Test data: * Username: testuser
* Password: password123
Expected result: * The user is logged in to the system and redirected
to the home page.
Actual result: * The user is logged in to the system and redirected
to the home page.
Status Pass

10
Test Cases..cont

Test Case

Loading…
Assigned to team
member

Part of Test Set/Suit Has Preconditions Has Specified Steps Part of Test Plan

11
Let’s start by self assessment exercise

Suppose that we have a program that we want to write test cases for:

Scalene: no two sides are equal

Isosceles: has two equal sides
Equilateral: all three sides are equal

13
Suggested Test Cases

14
Suggested Test Cases..2

15
 Exercise Discussion

● The point of this exercise is to illustrate that the testing of even a trivial
program is not an easy task.
● Given this is true, think of testing of 100K statement program of air
traffic control system!
● A typical GYM desktop program has 15K statements
● A Payroll system can have much more than that

16
 Psychology and Economics of SW Testing

● It is true that SW testing is technical task, but also involves important

considerations of:
○ Economics and
○ Human psychology
● It is true that we want to test every single permutation of a program.
● But in most cases, this is simply impossible

17
Tester’s Psychology

● According to research:
○ The tester needs a proper attitude/vision to successfully test a software product.
○ Tester’s attitude is more important than the process/tools applied.

● One of the primary causes of poor application testing is the

fact that most programmers begin with a false definition of
the term:
○ ‘‘Testing is the process of demonstrating that errors are not
present.’’
○ ‘‘The purpose of testing is to show that a program performs its
intended functions correctly.’’
○ ‘‘Testing is the process of establishing confidence that a
program does what it is supposed to do.’’

18
Tester’s Psychology..2

● These definitions are upside down, because

○ When you test a program, you want to add some value to it.
○ Adding value through testing means raising the quality or reliability of the
program.
○ Raising the reliability of the program means finding and removing errors.
○ Therefore, don’t test a program to show that it works; rather, start with the
assumption that the program contains errors (a valid assumption for almost
any program) and then test the program to find as many of the errors as
possible

19
Tester’s Psychology..3

Testing is the process of executing a program with

the intent of finding errors.

All text quoted from Myers, Sandler 2011

20
Tester’s Psychology..4

● We also need to clarify the term successful and unsuccessful:

● Most project managers refer to a test case that did not find an error a
‘‘successful test run,’’ whereas a test that discovers a new error is usually called
‘‘unsuccessful.’’

21
 Economics of Testing

● After having agreed on definition for SW testing, we need to be clear that

it is not possible to find all errors in the program.
● There are two prevalent testing strategies:
○ Black-Box Testing: also known as input driven and input/output testing, no knowledge
about internal program structure.
○ White-Box Testing: testing is based on complete knowledge about internal code and its
structure and conditions

22
Verification and Validation

● Verification is the process of evaluating whether the software conforms to

specifications and requirements during the development phase. It asks, "Are we
building the product right?“
○ Purpose: Ensures that the product is being developed according to the
design specifications and standards.
○ Focus: Internal processes such as code reviews, inspections,
walkthroughs, and unit testing.
○ Timing: It happens during the development cycle, typically at every stage,
like when you complete a module or a design phase.
○ Method: Static testing (e.g., code reviews, design reviews) where you do
not execute the code.
○ Output: Documentation, reports, and artifacts to ensure that each stage of
the process produces the correct outcome.
23
Example of Verification

Example of Verification: Suppose you're working on

a mobile app. During the design and coding stages, a
code review checks whether the code adheres to
design documents, coding standards, and
requirements like proper input validation. Even if this
part is perfect, the app might still not function as
intended (e.g., users' needs might not be met).

24
Verification and Validation

● Validation is the process of evaluating whether the final software product

meets the business needs and expectations of the user. It answers the question,
"Are we building the right product?"
○ Purpose: Ensures that the product actually meets the user's requirements
and performs as expected in real-world conditions.
○ Focus: The external behavior and the functionality of the system.
○ Timing: Usually performed after the development process is complete or
during the final testing phases.
○ Method: Dynamic testing (e.g., functional testing, integration testing, user
acceptance testing) where the code is executed.
○ Output: A validated software product that users can accept and that
functions correctly in their environment.

25
Example of Validation

Example of Validation: For the same mobile app,

after the development is done, you might run User
Acceptance Testing (UAT) with actual users. During
UAT, you verify if the app's features align with what
the users need. If users find that the app lacks an
essential feature they expected, it has failed
validation, even though the code is flawless.

26
V- Model In Software Development

27
 Black-Box Testing
Validation Technique
● Finding all errors using this strategy means running the program against all
possible correct and incorrect input.

● This is totally impossible.

● But, we can maximize the errors found using a finite number of test cases

28
 White-Box Testing
advanced , better for finding bugs expensive and takes
time (good for
more , critical
systems)
● Derives test data by examining the internal program code and structure (i.e,

●
program logic)
Loading…
For instance, one tester may assume that causing every statement to execute
in a program may be the correct answer. (i.e, exhastive path testing of a
program)
● But this is totally untrue as well:
○ The number of unique logic paths in a program can be astronomically
large!

29
White-Box Testing..2

Number of
unique paths =
100 trillions

Exhaustive path
testing is not
possible

30
Software Testing Principles

1 - A necessary part of a
2- A programmer should
test case is a definition
avoid attempting to test
of the expected output
his or her own program.
or result.

4- Any testing process

3- A programming should include a thorough
organization should not inspection of the
test its own programs.
results of each test.
d

real usersshouldon 5- Test cases must be written for input

-

conditions that are invalid and QA for

unexpected, as well as valid and & test cases
expected themselves
31
Software Testing Principals..2
bad data not cause exception

roobust ei
6- Examining a program to
see if it does not do what it Most errors
rors reside
reside on
on

is supposed to do is only the other sideside

The other

half the battle; the other

half is seeing whether the
program does what it is not
supposed to do.

7-Avoid throwaway test

Do not make cases unless the
Do not make
tests
the
on the fly
fly
program is truly a
test on

throwaway program 32
Software Testing Principals..3

8- Do not plan a testing effort under the tacit

assumption that no errors will be found.

bugs tend to group together

9- The probability of the existence of more errors in a

section of a program is proportional to the number of errors
already found in that section.

10-Testing is an extremely creative and

intellectually challenging task.
33
Probability of Number of errors

34
Software Testing Principals..4

10 -Test Early, Test Often

“a bug found at design time costs ten times less

to fix than one in coding and a
hundred times less than one found after launch”

Ratio of costs in different phases 1:6:10:1000

Retesting vs Regression
unit tests) to ensure
automatedmain
tests (mainly

• Regression testing onworking

still

●
are
features
Retesting: You must retest the
fixes to ensure that issues have
other hand is the act of
been resolved before
development can progress. repeating other tests in
● So, retesting is the act of 'parallel' areas to ensure
repeating a test to verify that a
found defect has been correctly that the applied fix or a
fixed. change of code has not
introduced other errors or
unexpected behavior.
36
 Chapter 1 Summary
-

why do we need testing ? Software developments ambiguity

involves
② assumptions & flawed human communication and every new feature added
or code edited introduces the
possibility of bugs

&
e
&
! &
Software Testing and the
Development Process
Dr. Samer Zein
 Software Development Team

client

gives us requirements

Software Testing & QA, CS Dept., Birzeit University Dr. Samer Zein
 Who is responsible for testing?
• This is a favorite interview question!
• Answer:

Software Testing & QA, CS Dept., Birzeit University Dr. Samer Zein
 Cost of fixing bugs early

Software Testing & QA, CS Dept., Birzeit University Dr. Samer Zein
 The role of software testing

Software Testing & QA, CS Dept., Birzeit University Dr. Samer Zein
 Software Development lifecycle
PdBTDM

Software Testing & QA, CS Dept., Birzeit University Dr. Samer Zein
 Iterative software Development

Software Testers are involved in all stages!

testing also exists in every stage

Software Testing & QA, CS Dept., Birzeit University Dr. Samer Zein
 Responsibilities during the planning phase:

Software Testing & QA, CS Dept., Birzeit University Dr. Samer Zein
 The analysis/defining phase

each test case

should have
acceptance
criteria

by test

engineers

this will ensure

nothing is ambigious

Iwillbuild
or not-measurable /subjective

based (requirements are testable)

& Critiria
on this user
· as a
librarymember I
to test it bussiness story
how ? able toerch
system shall display "no books found" analyst and why should be
who what,
if books are found the ,

& search
,
no
product book using
found , the list of book should be ordered by year
-

a
book
if more than
- T oew
book's info
- one
- - -
owner

Software Testing & QA, CS Dept., Birzeit University Dr. Samer Zein
 Building/Development Phase

Software Testing & QA, CS Dept., Birzeit University Dr. Samer Zein
 Testing Phase

testing has a phase

when tests pass
only
we consider the task
done and move or

Software Testing & QA, CS Dept., Birzeit University Dr. Samer Zein
 Deployment and maintenance phase
• Deployment Phase: in this phase, the product is shared with
customers and potential users:
• Acceptance testing
• System monitoring and bug fixing
• Real users and real data can reveal new errors.
• Maintenance Phase:
• Very long phase.
• System monitoring and big fixing
• New features  re-testing

Software Testing & QA, CS Dept., Birzeit University Dr. Samer Zein
 Overall Testing Activities

Software Testing & QA, CS Dept., Birzeit University Dr. Samer Zein
 Testing Activities

Can use
test tools like "Sira"
tests
to manage

Software Testing & QA, CS Dept., Birzeit University Dr. Samer Zein
 Testing Activities

② -

Software Testing & QA, CS Dept., Birzeit University Dr. Samer Zein
 Testing Activities

Compareexpectesult

00
Software Testing & QA, CS Dept., Birzeit University Dr. Samer Zein
 Testing Activities

Software Testing & QA, CS Dept., Birzeit University Dr. Samer Zein
-
Test Cases
and Bugs Lifecycle
Dr. Samer Zein
 Test Cases Overview
· test senario is for a main feature and contains
many test cases

:corefeaturroduce
high when
major bugs
it's critical (maybe related to
security)

(operating System , ...

Software Testing 7 QA, CS Dept., Birzeit University Dr. Samer Zein

 Example Test Case
impact ,
how much this is a core
functionality
>
-

after execting the test case I add actual result and success status
it
can assign default priority first then chang

Software Testing 7 QA, CS Dept., Birzeit University Dr. Samer Zein

 Test Run :

collection of
cases I
test
run
Choose to

them and generate

a report

ta
based
on

Software Testing 7 QA, CS Dept., Birzeit University Dr. Samer Zein

 What is a test scenario?
• A test scenario is a high-level concept that describes what needs to
be tested. It’s a broader outline of the functionality or feature under
test without going into detailed steps. Test scenarios are generally
used to cover multiple conditions that a feature must meet and can
lead to multiple test cases.
• Key Characteristics of a Test Scenario:
• High-level, focused on "what to test"
• Describes functionality without specific steps
• Can map to multiple test cases

Software Testing 7 QA, CS Dept., Birzeit University Dr. Samer Zein

 here

Software Testing 7 QA, CS Dept., Birzeit University Dr. Samer Zein

 Checklists: Less documentation

- Less formal
- Less documentation
- Only recommended
at certain conditions.

Software Testing 7 QA, CS Dept., Birzeit University Dr. Samer Zein

 Test cases vs Checklists

Software Testing 7 QA, CS Dept., Birzeit University Dr. Samer Zein

 actual not in place
- logo
expected affecting
Software Bugs G severity- not main
low
features

high priority
- customer
needs it

6 when should I fix it?

asap

not &

iact

G
Software Testing & QA, CS Dept., Birzeit University Dr. Samer Zein
 Visual Bugs

Can be low severity, but

high priority!
-

Software Testing & QA, CS Dept., Birzeit University Dr. Samer Zein
 Functional & Performance Bugs
"
ecore
functionality - &

Can be high severity and

high priority

Software Testing & QA, CS Dept., Birzeit University Dr. Samer Zein
 Bug Reporting

I
Sometimes
even save what
are
ever values
when
in memory
bug happened
additional
~
things busa
screenshot
↑

~ Can
-

Software Testing & QA, CS Dept., Birzeit University Dr. Samer Zein
 Example

Software Testing & QA, CS Dept., Birzeit University Dr. Samer Zein
 Example 2

Software Testing & QA, CS Dept., Birzeit University Dr. Samer Zein
 Bug Lifecycle

Software Testing & QA, CS Dept., Birzeit University Dr. Samer Zein
 Bug Lifecycle States

higs)
e

pending
- Ready to

To

a developer
and
should be
discovered
to minimize
asap
assigned mistakes
time and possible - problemmeest
bug)
case not
a

Software Testing & QA, CS Dept., Birzeit University Dr. Samer Zein
 Example in Jira

assigned
y eprogrammer -
tester

Software Testing & QA, CS Dept., Birzeit University Dr. Samer Zein
Software Quality Metrics

Kloc = 1000 code lines

Function points not recommended because it's not consistant

of to Severity well
weighted number errorsa related as
Software Quality
Metrics

BY S A M E R Z E I N , P H D

BIRZEIT UNI., SOFTWARE TESTING, DR. SAMER ZEIN 1

 Introduction
“You cannot control what you cannot measure”, Tom
Demarco (1982)
Software Metrics Definitions:
◦ “A quantitative measure of the degree to which an item
possesses a given quality attribute.”
◦ “A function whose inputs are software data and whose output
is a single numerical value that can be interpreted as the degree
to which the software possesses a given quality attribute.”

BIRZEIT UNI., SOFTWARE TESTING, DR. SAMER ZEIN 2

 Introduction..2
Software metrics should be included in software
development, in order to :
◦ Control of SW development & maintenance
◦ Support of decision making
◦ Initiation of corrective actions
Analysis of SW metrics could show the results of certain
interventions such as :
◦ New development tools,
◦ Changed procedures
◦ etc
BIRZEIT UNI., SOFTWARE TESTING, DR. SAMER ZEIN 3
 Introduction..3
However Metrics as a quality assurance tool has not, unfortunately, been
applied at an adequate level in the software industry.
Nor have they provided benefits at the anticipated levels.
Only a small portion of software development organizations apply software
quality metrics systematically, and few of these report successful use of the
results of their efforts.

BIRZEIT UNI., SOFTWARE TESTING, DR. SAMER ZEIN 4

Objectives of SW Metrics

BIRZEIT UNI., SOFTWARE TESTING, DR. SAMER ZEIN 5

It is all about Comparison
Comparison provides the practical basis for management’s
application of metrics and for SQA improvement in general.
The metrics are used for the comparison of performance data
with indicators, and quantitative values such as:
◦ Defined SW quality standards
◦ Quality targets set for organizations
◦ Previous year quality achievements
◦ Previous Project quality achievements
◦ Average quality levels achieved by other similar teams

BIRZEIT UNI., SOFTWARE TESTING, DR. SAMER ZEIN 6

BIRZEIT UNI., SOFTWARE TESTING, DR. SAMER ZEIN 7
Measures of System Size
KLOC – this classic metric measures the size of
software by thousands of code lines. As the number
of code lines required for programming a given task
differs substantially with each programming tool, this
measure is specific to the programming language or
development tool used. Application of metrics that
include KLOC is limited to software systems
developed using the same programming language or
development tool

BIRZEIT UNI., SOFTWARE TESTING, DR. SAMER ZEIN

8
Measures of System Size…2

Function points – a measure of the development

resources (human resources) required to
develop a program, based on the functionality
specified for the software system

BIRZEIT UNI., SOFTWARE TESTING, DR. SAMER ZEIN 9

SW Process Quality Metrics:
Error Density Metrics
Calculation of error density metrics involves
two measures:
◦ (1) software volume, : use of number of lines of
code
◦ and (2) errors counted:
◦ Number of errors
◦ Weighted number of errors
More accurate in
many situations

BIRZEIT UNI., SOFTWARE TESTING, DR. SAMER ZEIN 10

Example:

BIRZEIT UNI., SOFTWARE TESTING, DR. SAMER ZEIN Source: Galin, 2004
11
X

best
M

BIRZEIT UNI., SOFTWARE TESTING, DR. SAMER ZEIN Source: Galin, 2004
12
 ·

BIRZEIT UNI., SOFTWARE TESTING, DR. SAMER ZEIN Source: Galin, 2004
13
Error Severity Metrics
-high severity /I
errors is

BIRZEIT UNI., SOFTWARE TESTING, DR. SAMER ZEIN Source: Galin, 2004
14
Software Process Timetable Metrics

Software process timetable metrics may be based on accounts of success

(completion of milestones per schedule) in addition to failure events (non-
completion per schedule).

An alternative approach calculates the average delay in completion of

milestones.

BIRZEIT UNI., SOFTWARE TESTING, DR. SAMER ZEIN 15

 3%
maybe release date #35
>
-
.

0
-918

usually working days -1 99.15 / .

BIRZEIT UNI., SOFTWARE TESTING, DR. SAMER ZEIN 16

Error Removal Effectiveness
Metrics
Software developers can measure the effectiveness of error removal by
the software quality assurance system after a period of regular
operation (usually 6 or 12 months) of the system.
The metrics combine the error records of the development stage with
the failures records compiled during the first year (or any defined
period) of regular operation.

BIRZEIT UNI., SOFTWARE TESTING, DR. SAMER ZEIN 17

Error Removal Effectiveness

6 Ni
months

&
-

1 sild Is
before production after production
acceptances after acceptance
& , testing
testing

BIRZEIT UNI., SOFTWARE TESTING, DR. SAMER ZEIN Source: Galin, 2004
18
Software Process Productivity
Metrics
deal with a project’s human resources productivity as well as “indirect”
metrics that focus on the extent of software reuse.

Software reuse substantially affects productivity and effectiveness.

BIRZEIT UNI., SOFTWARE TESTING, DR. SAMER ZEIN 19

ebuilding libraries maybe

BIRZEIT UNI., SOFTWARE TESTING, DR. SAMER ZEIN 20

 Limitations of Software Quality
Metrics

O
O
O

BIRZEIT UNI., SOFTWARE TESTING, DR. SAMER ZEIN 21

 and the
to make central
try-catch save
exception
I need
a

usually , avoid duplicated

I embed it in the actual code
enough sometimes
it's not always ,

should cancel
happens I
But ,
database if an error
to
like:
① I save 9

should roll-back put it back

then not
everything ,
I
currancy
I can't take money
change money try catch
,

like I use

transaction object , and

a
So , I make

try [
Comm1

Commecommit ; Ging
el E
&

(SQLException -1
3 Catch -

if I put the try-catch

objTrans roll Backi) ; it's
>
- .

it to when not necessary ,

throw
throw e; centralized that
im
a performance hit
documents
else
the catch or

catch I need
to put smth in

if I use try "throw"

buried I should put a

it's literally , the mosthod

throws an exception
when
comments I should write
in the ,
 this Code
Review
a
O does① yeais at last
of code
did I else
start from G there's no

one Bepyr
?

]
called Fel

- should be
constants
55 S

= numbers
-
notactualnumber
of constants
an array
magime
make

deci d ina
ed it's

his shibe
⑤ private
a public
it in
and just call
 is the
number
magic should make
problem here , Clockwise
3-degrees
constant called
-

a
~
?
⑧
⑧?
 is best ?
which
X
X

-
X
 they didn't
bad comments ,
value
add any
write a
comment
should
thiso
I

explainingwhatis
-
 -

-
V

X
manual functional testing :- testcases ;i

I need to re-test them all at

every code change
s
test cases & iges, 83. 61 inputs/1 in -> equivilance participitation
much better than &
randomly
generated data for inputs

time 7 : 00 10 jo)
Mon ,
one ,
,

(sat Member 21 : 00
,
17 yu
, ,

Member 20
: 00
,
70 yo)
(Friday
,
,

18 yo)
(Thursday M, 6 : 00
,
,

19)
(Thursday ,
M , 5 :05 ,

1900a
invaliFitsinpu Thursday Ot ,
,
equivilant classes for days >
- weekday ,
weekend
,
status >
-

hours -
Ot ,

6-14
M ,
invalid e if free text
19 %- 24 boundaries (3 2 6)
1 - -
-
-

((6)
early
*
> .
=

, ,
120 boundaries
visitor age > 0 +6
-
,
16 1-60 .

,
60 1
.

- , so
---- ,

-- 1)
L ~

I
-----xx)
 of them
just
some

(b) invalid

13) invalid

13) below
(4) negative
(5)abovea

(4) negative
15) below

(6) exactly
Confidential Customized for Lorem Ipsum LLC Version 1.0

Black-Box Testing
Methods
Software QA, Birzeit University - Dr. Samer Zein
 Black-Box Testing: Intro
•Think of its techniques as a complement to White-Box
testing techniques, not a substitute.

Software QA, Birzeit University - Dr. Samer Zein

 Black-Box Testing:
Equivalence Classes Technique
•The focus here is to generate a set of test cases.
•The set has to be chosen carefully.
•Equivalence classes techniques aims at:
▫A) Increasing the efficiency of testing
▫B) Minimizing the number of test cases.
•That is, improved choice of test cases by efficient use of
equivalence class partitioning

Software QA, Birzeit University - Dr. Samer Zein

 Black-Box Testing:
Equivalence Classes Technique..2
•What is an Equivalent Class (EC)?
•It is a set of input variable values that are
processed identically to produce the same output.
•We also have:
▫Valid EC
▫Invalid EC
•Always remember that your test suit should contain both.

Software QA, Birzeit University - Dr. Samer Zein

 Black-Box Testing:
Equivalence Classes Technique..3
•The basic rule is that there should be at least one test
case for each valid and invalid EC.
•Thus, minimizing the number of test cases.
•It is your job to define the EC and their boundaries.
•EC technique is based on requirements’ specs, not
code.
•EC technique is much efficient from automated and random
techniques, Why?
Software QA, Birzeit University - Dr. Samer Zein
 Example:

Software QA, Birzeit University - Dr. Samer Zein

 Example 2:

Software QA, Birzeit University - Dr. Samer Zein

 The General Process of Identifying EC

•A) If an input condition specifies a range of values

(e.g., ‘‘the item count can be from 1 to 999’’) then:

▫identify one valid equivalence class (1<item

count<999) and
▫ two invalid equivalence classes (item count<1 and
item count>999).

Software QA, Birzeit University - Dr. Samer Zein

 The General Process of Identifying
EC ..2
•B) If an input condition specifies the number of
values
(e.g., ‘‘one through six owners can be listed for the
automobile’’),
▫identify one valid equivalence class
▫and two invalid equivalence classes (no owners
and more than six owners).
Software QA, Birzeit University - Dr. Samer Zein
 The General Process of Identifying
EC ..3
•C) If an input condition specifies a set of input values, and
there is reason to believe that the program handles
each differently
eg.(‘‘type of vehicle must be BUS, TRUCK, TAXICAB,
PASSENGER, or MOTORCYCLE’’),
▫ identify a valid equivalence class for each
▫and one invalid equivalence class (‘‘TRAILER,’’ for
example).
Software QA, Birzeit University - Dr. Samer Zein
 The General Process of Identifying
EC..4
•D) If an input condition specifies a ‘‘must-be’’
situation,
such as ‘‘first character of the identifier must be a
letter,’’
▫identify one valid equivalence class (it is a letter)
▫and one invalid equivalence class (it is not a
letter).
Software QA, Birzeit University - Dr. Samer Zein
 Important Note Regarding Invalid EC

•if the specification states ‘‘enter book type (HARDCOVER,

SOFTCOVER, or LOOSE) and amount (1–999),’’
•
the test case, (XYZ, 0), expressing two error conditions (invalid
book type and amount)

•will probably not exercise the check for the amount, since the
program may say ‘‘XYZ IS UNKNOWN BOOK TYPE’’ and not bother
to examine the remainder of the input.

Software QA, Birzeit University - Dr. Samer Zein

 EC’s Boundary Values

•It is additional criterion that you should be aware of

during assigning test cases for each EC.
•That is, pay great attention to defining test cases
to test the boundaries of each EC
•More specifically, we need three test cases:
▫Middle range
▫Lower bound
▫Upper Bound
Software QA, Birzeit University - Dr. Samer Zein
Software QA, Birzeit University - Dr. Samer Zein
 Example:

Software QA, Birzeit University - Dr. Samer Zein

Software QA, Birzeit University - Dr. Samer Zein
 Negative Testing

Software QA, Birzeit University - Dr. Samer Zein

 Example: Registration Form

Software QA, Birzeit University - Dr. Samer Zein

 Another Example Based on requirements

Software QA, Birzeit University - Dr. Samer Zein

In Class Activity: EC Testing technique
 "ends
)
(mon , the ..... invalid
weekdays
59) sinvalid
(we
,

ways invalid
-

member 01 b :

invalid (00
-

, :
one time ,
status -

-6-19 19024 ,
hours ,
19 01 , 6 24
19
(negative)
. ,

, limits -
> ,

10 0 16 00) (16 01
.
60 .
00) (60 .
01 - 120 .
0) invalid (CO)
agen
- -
.

, ,
.

( > 120)
16 60 120 (
limits (0 ,
,
,

(Mon ,
OT
,
7 : 00 ,
15 y10) >
- 5:00

(Sun ,
ot
,
7:00 ,
15 ylo) >
- 7 5 .

("M" ot 700 15 y10) - invalid input for day

, , ,

2 3
(Mon M 7 : 00 15 y() -
.

,
, ,

-invalid input for status

(Mon ,
F ,
7: 00
,
15 ylo)

(Mon 00 15 y(0) -> 6

, or 20: ,
,

(Mon , ot 6 : 00 S
15 y(0) >
- 5
,
 5
(Mon , or 19 : 00
,
13 y(0) -

6
(Mon , ot
15 y(0) -

,
19 : 01

(Mon ot
15 y(0) >
- 6
,
, 24 : 00
,

y(0) invalid hour

(Mon , ot 5 : 30 ,
15 -

(Mon , or
, 24 : 00
,
20 y(0) - 10

8
(Mon , ot
, 24 : 00
,
70y10) -

5
(Mon , ot
24 : 00
,
01
+

3
(Mon , ot
24 : 00
, 16)
+

10
(Mon , ot
24 : 00
, 16. %)
>
-

10
(Mon , ot
24 : 00
,
60)
+

,
 8
(Mon , ot
24 : 00
,
60 %).
-

8
(Mon , ot
24 : 00
,
120) -

121) invalid age

(Mon , or
24 : 00
,
-

invalid age
(Mon , or
24 : 00
, -1)
-

,
 Should
& be

done
like
this

Software QA, Birzeit University - Dr. Samer Zein

 expected

pif
⑤
↑ 319
j6

Software QA, Birzeit University - Dr. Samer Zein

 Decision Table Testing *

Software QA, Birzeit University - Dr. Samer Zein

 Example: user wants to transfer

Software QA, Birzeit University - Dr. Samer Zein

Software QA, Birzeit University - Dr. Samer Zein
Software QA, Birzeit University - Dr. Samer Zein
Software QA, Birzeit University - Dr. Samer Zein
 Example 2 4 times then system
Canchter password
it
locks

-2

around a

test cases

Software QA, Birzeit University - Dr. Samer Zein

 & Verification ?
Software QA, Birzeit University - Dr. Samer Zein
White Box Testing :

>
-
should be used for Critical Systems
of developers
-
should be done with the help
end at 9
Should
example for path :
a

1 + 2-338 -
19

1- p : empty file
109
4 =5278
+
2
1 +
+

ze 2+9
2e4e6 + 7
+
1 +
 .. d

not all paths are fiesable some

,

Paths I can see here but not

in practice
 ①I !

are fiesable
Q : which ones
/
⑧ power
& bothaa
J not complete
na
I
-

--
-
a pm to G am
I have so many paths
3-4 paths
I execute
sure each
So I make

line of cod was executed

at least once ,
so I

get full line coverage

>
- if I do all paths ,
it's
called full path coverage
 three
are

enough
For full line

Coverage
this is only to
of the path
69 19 005

.
.
 6
Complexity =

= 5- Testable
O too
<= 10e not
· difficult
dis
> 20
- high complexity

750-untestable
 White-Box
Software Testing
Techniques

By Samer Zein, PhD

Software QA & Testing, CS Dept., Birzeit University - Samer

Zein (Ph.D)
RECAP

• “At early times, Software Testing was confined to the final stage of
development (Big Bang)”

• “But later on, as the importance of early detection of software defects

penetrated quality assurance concepts, SQA professionals were
encouraged to extend testing to the partial in-process products of
coding, which led to software module (unit) testing and integration
testing.”
• Hereafter, we will concentrate on test that are performed by running the
code.

Software QA & Testing, CS Dept., Birzeit University - Samer

Zein (Ph.D)
Revisiting Definitions

Daniel, 2004
Software QA & Testing, CS Dept., Birzeit University - Samer
Zein (Ph.D)
Software Testing is:

Formal

SW
Specialized Running
Team Testin
Code
g

Approved test
procedures and test
cases

Software QA & Testing, CS Dept., Birzeit University - Samer Zein (Ph.D)

Imp note: Therefore, we prefer the phrase “acceptable, level of
quality”, meaning that a certain percentage of bugs, tolerable to
the users, will remain unidentified upon installation of the
software
Software QA & Testing, CS Dept., Birzeit University - Samer Daniel, 2004
Zein (Ph.D)
SW Testing: Two Strategies

Software QA & Testing, CS Dept., Birzeit University - Samer

Zein (Ph.D)
White-Box Testing

• Realization of the white box testing concept requires verification of

every program statement and comment.

• In order to perform data processing and calculation correctness tests,

every computational operation in the sequence of operations created
by each test case (“path”) must be examined

Software QA & Testing, CS Dept., Birzeit University - Samer

Zein (Ph.D)
White-Box Testing: Path Coverage

• Different paths in a software module are created due to IF–

THEN–ELSE or DO WHILE or DO UNTIL.
• Path testing: testing all its possible paths.

• Hence, the “path coverage” metrics: the percentage of the

program paths executed during the test

Software QA & Testing, CS Dept., Birzeit University - Samer

Zein (Ph.D)
Path Coverage is Impractical!

• For example, one simple module can contain about 50 lines

of code, but also contain lots of conditional statements, can
reach 1024 different paths!!

• Another example of a software package containing 100

modules can reach tens of thousands of test cases!!

• But remember that this is accepted at Critical Applications

Testing

Software QA & Testing, CS Dept., Birzeit University - Samer

Zein (Ph.D)
White-Box Testing:
Line Coverage (Basic Path Testing)
• The line coverage concept requires that, for full line
coverage, every line of code be executed at least once
during the process of testing.
• It is more affordable.
• But it is weaker: lots of paths will be left untested!
• All modern testing tools will show you a percentage of lines
covered by your test cases.
• They even show you which lines exactly that are not yet
executed.

Software QA & Testing, CS Dept., Birzeit University - Samer

Zein (Ph.D)
But First: CFG / Flow Graph

• A CFG captures the flow of control within a

program.
• Assists testers in the analysis of a program to
understand its behavior in terms of the flow of
control.
• A CFG can be constructed manually without much
difficulty for relatively small programs, say
containing less than about 50 statements.
• However, as the size of the program grows, so does
the difficulty of constructing its CFG and hence
arises the need for tools.
Software QA & Testing, CS Dept., Birzeit University - Samer
Zein (Ph.D)
Software QA & Testing, CS Dept., Birzeit University - Samer Zein (Ph.D)
 Basic Notation

Sequence IF While Case

Software QA & Testing, CS Dept., Birzeit University - Samer

Zein (Ph.D)
Basic Example

Software QA & Testing, CS Dept., Birzeit University - Samer

Zein (Ph.D)
Another Example

Software QA & Testing, CS Dept., Birzeit University - Samer Zein (Ph.D)

A Path:

• A path through a flow graph is

considered Complete if the first node along the
path is Start and the terminating node is End.
• A path p through a flow graph for program P is
considered feasible if there exists at least one test
case which when input to P causes p to be
traversed.
• If no such test case exists, then p is
considered infeasible.

Software QA & Testing, CS Dept., Birzeit University - Samer

Zein (Ph.D)
Exercise
P1, P2 are complete,
feasible

P3, P4 are
incomplete,

P5, P6 are complete,

But infeasible
Software QA & Testing, CS Dept., Birzeit University - Samer
Zein (Ph.D)
In-Class Example Workshop

Software QA & Testing, CS Dept., Birzeit University - Samer

Zein (Ph.D)
 Flow Chart

Software QA & Testing, CS Dept., Birzeit University - Samer

Zein (Ph.D)
 Flow Graph:

Software QA & Testing, CS Dept., Birzeit University - Samer

Zein (Ph.D)
Software QA & Testing, CS Dept., Birzeit University - Samer
Zein (Ph.D)
Software QA & Testing, CS Dept., Birzeit University - Samer
Zein (Ph.D)
Line Coverage vs Full Path

• The proportion of test cases required to test the system by full line
coverage of three test cases (by basic path testing) versus full path
coverage of 24 test cases is 1:8!
• This ratio grows rapidly with program complexity.

Software QA & Testing, CS Dept., Birzeit University - Samer

Zein (Ph.D)
McCabe’s Cyclomatic complexity metrics
• Support for Basic Path Testing is provided by McCabe’s Cyclomatic
Complexity Metrics
• Measures the complexity of a program.
• Measures the maximum number of paths needed to achieve full line
coverage

McCabe’s # of Independent
Flow Cyclomatic Paths
Graph Complexity

Software QA & Testing, CS Dept., Birzeit University - Samer Zein (Ph.D)

McCabe’s Cyclomatic Complexity Example:

Software QA & Testing, CS Dept., Birzeit University - Samer

Zein (Ph.D)
McCabe’s Cyclomatic Complexity Calculation

• V(G) = R, where R #of Regions

• V(G) = E – N + 2 where E  # of Edges,

N  # of Nodes

• V(G) = P + 1 where P  number of decisions

contained in the graph

Software QA & Testing, CS Dept., Birzeit University - Samer

Zein (Ph.D)
McCabe’s Cyclomatic Complexity Calculation
Example

Software QA & Testing, CS Dept., Birzeit University - Samer

Zein (Ph.D)
Software QA & Testing, CS Dept., Birzeit University - Samer
Zein (Ph.D)
McCabe’s Cyclomatic Complexity and
Testability

•<= 5  Testable

•<= 10  Not too difficult

•>20  High complexity

•> 50  Untestable

Software QA & Testing, CS Dept., Birzeit University - Samer Zein (Ph.D)

White-Box Testing: Branch/ Decision Testing

• Branch coverage is a measure of the percentage of

the decision points (Boolean expressions) of the
program have been evaluated as both true and false
in test cases.

• In this technique, we evaluate an Entire Boolean

Expression as one true and false expression.
• Even if it contains multiple logical operators

Software QA & Testing, CS Dept., Birzeit University - Samer

Zein (Ph.D)
Example:

Software QA & Testing, CS Dept., Birzeit University - Samer

Zein (Ph.D)
Integration Testing: Bottom-Up

Software QA & Testing, CS Dept., Birzeit University - Samer

Zein (Ph.D)
Integration Testing: Top-Down

Software QA & Testing, CS Dept., Birzeit University - Samer

Zein (Ph.D)
White-Box Testing: Condition Coverage

• Condition coverage is a measure of percentage of Boolean

sub-expressions of the program that have been evaluated as
both true or false outcome [applies to compound predicate]
in test cases.
• Notice that in line 7 there are three sub-Boolean expressions
to the larger statement (a==b), (c==d), and bug(a).
• Condition coverage measures the outcome of each of these
sub-expressions
• independently of each other.

Software QA & Testing, CS Dept., Birzeit University - Samer

Zein (Ph.D)
White-Box Testing Verdict

• The characteristics of white box testing limit its use to software

modules of :

•very high risk and very high cost of failure,

•where it is highly important to identify and
fully correct as many of the software errors
as possible.

Software QA & Testing, CS Dept., Birzeit University - Samer

Zein (Ph.D)
White-Box Testing: Verdict

Software QA & Testing, CS Dept., Birzeit University - Samer

Zein (Ph.D)
Program 1

Inspections and

Software QA & Testing -CS Dept - BZU, Dr. Samer Zein

Walkthroughs

BY SAMER ZEIN, PHD

 2
Introduction

► Software testing SHOULD NOT be

confined to testing the running
software.

► Scientific research has proven the great

value of reading code as part of a
comprehensive testing and debugging
regimen.

Software QA & Testing -CS Dept - BZU, Dr. Samer Zein

Img source:
radisson.com
 3
Introduction..2

► Human testing techniques are quite

effective in finding errors—so much so
that every programming project should
use one or more of these techniques.
► Research has proven that
► The earlier the errors are found 🡪
► The lower costs of correcting them;
► The higher probability of correcting them
correctly

Software QA & Testing -CS Dept - BZU, Dr. Samer Zein

 Factors that Influence the use of Code 4
Inspections

► Size and complexity of application

► Size of development team
► Experience of development team
► Timeline of project
► The culture and background of team
(are they professional, or will they take it
it personally?)

Software QA & Testing -CS Dept - BZU, Dr. Samer Zein

 5
Inspections and Walkthroughs

► Inspections and walkthroughs involve a team of people

► reading
► or visually inspecting a program.

► The objective here is find errors.

► In a walkthrough, a group of developers—with three or four being an

optimal number—performs the review.

► Only one of the participants is the author of the program.

► Therefore, the majority of program testing is conducted by people

other than the author

Software QA & Testing -CS Dept - BZU, Dr. Samer Zein

 6
Inspections and Walkthroughs

► Another advantage of walkthroughs, resulting in lower debugging

(error-correction) costs, is the fact that when an error is found, it
usually located precisely in the code, as opposed to black box
testing, where you only receive an unexpected result.

► These human testing methods generally are effective in finding from

30 to 70 percent of the logic-design and coding errors in typical
programs

► Inspections & Walkthroughs + Computer-based testing 🡪

Complementary
► Highly applied at Google and Mozilla!

Software QA & Testing -CS Dept - BZU, Dr. Samer Zein

 7
Inspection Team: The
Moderator

► He is a skilled programmer
► responsibilities are to:

Software QA & Testing -CS Dept - BZU, Dr. Samer Zein

 8
Inspection Team

► Programmer: the one who wrote the code

► Team leader: senior programmer/team lead

► Test Specialist:
► The specialist should be well versed in software
testing
► and familiar with the most common
programming errors

Software QA & Testing -CS Dept - BZU, Dr. Samer Zein

 9
Inspection Process

1 4 At the end, the programmer

Distribute code listing several
days before session is given a set of errors

2 During session, the 5 If errors require substantial

programmer narrates code corrections, plan for another
statement by statement session after correction

3 The program is analyzed 6 The list of errors are analyzed,

against a pre-defined set of categorized and saved for
common programming errors future inspections

Software QA & Testing -CS Dept - BZU, Dr. Samer Zein

 10
Common Programming Errors

► Does a referenced variable have a value that is unset

or uninitialized?
► For all array references, is each subscript value within
the defined bounds of the corresponding dimension?
► When indexing into a string, are the limits of the string
off by one in indexing operations or in subscript
references to arrays?
► Each variable is correctly initialized/proper value?
► Are there any mixed-mode operations?
► Is it possible for the divisor to be ZERO?

Software QA & Testing -CS Dept - BZU, Dr. Samer Zein

 11
Common Programming
errors..2

Software QA & Testing -CS Dept - BZU, Dr. Samer Zein

 12
Common Programming
Errors..3

► Will every loop eventually terminate?

► Will every loop execute in the first place?
► Loop boundaries:

Software QA & Testing -CS Dept - BZU, Dr. Samer Zein

 13
Common Programming
Errors..4

► Writing too much code inside methods.

► Incorrect use of inheritance in Object-Oriented
programming.
► Not checking methods parameters’ data before
processing them
► Incorrect exception handling:
► Catching for no reason
► Hiding/ burying exceptions.
► Propagating exceptions
► Forgetting to close/release system resources such as DB
connections and files

Software QA & Testing -CS Dept - BZU, Dr. Samer Zein

 14
Walkthroughs

► The initial procedure is identical to that of the inspection process:

► The participants are given the materials several days in advance to allow
them time to bone up on the program
► However, the difference is that rather than reading the code line by
line,
►

► Participants play computer and execute special test cases

► The state of program is monitored on whiteboard
► Test cases serve as vehicle to ask the programmer 🡪 questioning
► More errors can be revealed by asking

Software QA & Testing -CS Dept - BZU, Dr. Samer Zein

 15
Desk Checking

► A desk check can be viewed as a

one-person inspection or walkthrough.
► A person reads a program, checks it with
respect to an error list,
► and/or walks test data through it.
► Better performed by a person other than
a programmer.
► Not very effective, WHY?
Software QA & Testing -CS Dept - BZU, Dr. Samer Zein
 16
Peer Rating

► Peer rating is a technique of evaluating

anonymous programs in terms of their overall
► quality,
► maintainability,
► extensibility,
► Usability,
► and clarity.
► The purpose of the technique is to provide
programmer self evaluation.

Software QA & Testing -CS Dept - BZU, Dr. Samer Zein

 Peer Rating Process 17
• The admin selects 6-20 participants of similar backgrounds
• Each participant is asked to provide two programs: one high
quality and another with lower quality
1 • Session = 30 minutes

• Each participant is randomly assigned 4 programs: 2

high and 2 low quality
2 • Note that the process is totally anonymous
• Each participant is asked to give feedback (scale 1-10),
about:
• Clarity, readability, design, maintainability, any
3 comments?

• Feedback is given to programmer along with their

4 resulting ranking

Software QA & Testing -CS Dept - BZU, Dr. Samer Zein

 OBJECTIVES SUMMARY 18

Software QA & Testing -CS Dept - BZU, Dr. Samer Zein

 19
Error Handling Techniques

► Research has proven that a large percentage of

software errors are caused by bad error handling
► This is critical when there are novice programmers in
your team.
► Usually, error handling is considered to be an
advanced task to program.
► Keep a good eye on error handling techniques
applied, and you will be surprised at what you can
find there!

Software QA & Testing -CS Dept - BZU, Dr. Samer Zein

 20
Error Handling Techniques..2

1. Return a neutral value:

► Sometimes the best response to bad data is to
continue operating and simply return a value
that’s known to be harmless

► 2. Substitute the next piece of valid data

► Example reading from a database or reading
from a thermometer sensor in a mobile app.

Software QA & Testing -CS Dept - BZU, Dr. Samer Zein

 21
Error Handling techniques..3

► 3. Return the same answer as the previous time:

► Example: calculating velocity and reading from sensors.
► 4. Substitute the closest legal value.
► Example: temperature should only be between 0-100 C.
► 5. Log an error message.
► 6. Return an error code:
► Example: reading a record that does not exist from a
database.
► You can throw a specific exception if you want.
► Shut down:
► Applied in critical systems.

Software QA & Testing -CS Dept - BZU, Dr. Samer Zein

 22
Exceptions Best Practices

► If a method encounters an error that does not know

how to handle it, it can throw an exception
► It is like saying, “I don’t know what to do about this—I
sure hope somebody else knows how to handle it!”
► Use exceptions to notify other parts of the program
about errors that should not be ignored
► Throw an exception only for conditions that are truly
exceptional.

Software QA & Testing -CS Dept - BZU, Dr. Samer Zein

 23
Exceptions Best Practices..2

► Throw exceptions at the right level of abstraction

► Include in the exception message all information that led to the

exception.

► Avoid empty catch blocks

► Should have centralized exception handling techniques.

Software QA & Testing -CS Dept - BZU, Dr. Samer Zein

 24

try{
…..
…..
catch(SomeException ex){
throw ex;
} Software QA & Testing -CS Dept - BZU, Dr. Samer Zein
 25
Tutorial 6: General Code
Inspection Guidance

► Safe from bugs:

► Correct today and correct in the unknown future.
► Easy to understand:
► Communicating clearly with future programmers,
including you
► Ready for change:
► Designed to accommodate change without rewriting.

Software QA & Testing -CS Dept - BZU, Dr. Samer Zein

 26
Tutorial 6 Solution

► Do Not Repeat Yourself (DRY)

► Duplicating bugs.
► Nightmare for maintenance team
► How can you DRY it out?
► Comments.
► How good or bad is it commented?
► Different ways of calling the method problem.
► There are other programmers than you!
► The problem of Magic numbers
► Never hard-code constants that you calculated by hand
► Do not reuse parameters!!
Software QA & Testing -CS Dept - BZU, Dr. Samer Zein
 Software Testing and Quality Assurance

COMP438

Tutorial # 3: Code Inspection

Objective: Practice general principles of good coding, which you can look for in every code
review, regardless of programming language or program purpose. Carefully inspect the following
piece of code and try answering the following questions

PART 1
PART 1 DISCUSSION

● Inspect the above code carefully can reveal several types of errors from different

angles/perspectives. Can you identify these perspectives in general?

● What is wrong with the above code? Would you approve it?

● Is there a better way to design it?

PART 2:

Suppose you’re reading some code that uses a turtle graphics library that you don’t
know well, and you see the code:

DISCUSSION
PART 3:

For each of the following re-writes, which one you would choose and WHY?

A) B)

C) D)

PART 4:

What do you think about the following comments? What should be really written when we write
down comments for our code?
 PART 5

Which comments are useful additions to the code? Consider each comment independently, as if
the other comments weren’t there.

test-driven
development
TDD &

return of investment
ROI e