AN INTRODUCTION TO TESTING TECHNIQUES

0/
EXHAUSTIVELY TESTING IS NOT POSSIBLE

è Software are getting more and more complex:

§ Systems interact with other systems
ú Ex: a merchant website is connected to visa payment system and use social network api
§ System must be working on different environments
ú Ex: a mobile application on android

è Infinite possibilities, combination of inputs, events …

è How to choose?

è Testing techniques help to find the best test case to decrease the failure
risk.

è Depending on criticality of your component you may use one or several

test techniques.

1/
OVER QUALITY – UNDER QUALITY

We should adjust the testing in order to find the right balance (test effort).

è Test effort will be different depending on sector of activity:

§ Aeronautics
§ Automotive
§ Health

Ø Test Effort will depend on criticality of function

• Test technique used, you can also combine them
• Volume of test data
• Prioritize test execution

2/
TEST TYPE

è Static testing: Review, static analysis

è Structural test (white box testing)

è Functional test (black box testing)

è Non functional or technical test

§ Performance testing
§ Maintainability testing
§ Usability testing
§ Portability testing
§ Etc.

è Non functional test ßàCharacteristic measurement

3/
TESTING TYPE DURING PROJECT

Integration System Acceptatio

Dev part Unit test
test test n test
Review - Inspection

Static Analysis
Structural coverage

Functional test

Non functional test

4/
TESTING TECHNIQUE (FUNCTIONAL)

è Objective :
§ Depending on system, number of test cases are infinite.

Example: A software to manage a printer:

§ scenarios: configuration, Page Setup, print preview, print.
§ OS: 2000, XP, VISTA, Linux,HP
§ Parallel port, USB 1.1, USB 2
§ Test case number: 4*5*3 = 60 test case

è Findaccurate testing methods to get useful test cases among the infinity of Test
Cases
§ System complexity
§ System testability
§ Test level

What does mean system testability?

5/
DEFINITION

è Requirements-based testing: An approach to testing in which test cases are

designed based on test objectives and test conditions derived from
requirements, e.g. tests that exercise specific functions or probe non-functional
attributes such as reliability or usability. [ISTQB]

è test basis: All documents from which the requirements of a component or

system can be inferred. The documentation on which the test cases are based.
If a document can be amended only by way of formal amendment procedure,
then the test basis is called a frozen test basis. [ISTQB After TMap]

è Functional and non functional test (technical).

6/
TEST TECHNIQUE

è Based on data
§ Equivalence testing.
§ Boundary values testing.
§ Decision table
§ Pair Wise

è Based on behavior
§ State diagram
§ Use Case

7/
EQUIVALENCE CLASSES

è Equivalence classes : A portion of an input or output domain for which the

behavior of a component or system is assumed to be the same, based on
the specification.

è Rule: Minimum one value per classes.

Set 1 values for

x F behavior 1 Outputs 1

Set 2 values for

x F behavior 2 Outputs 2

8/
EQUIVALENCE CLASSES

è Valid domain : ranges of a data for which behaviour is specified in

requirement as a nominal behaviour.

è Invalid domain: ranges of a data for which behaviour is specified in

requirement as a degraded behaviour or specific behaviour in case of
invalid data.

9/
SAMPLE

è Under 4 year ticket price is free.

è Above 120 value is rejected

0 4 120
P1 P2 P3

§ P1 à ticket free (valid partition)

§ P2 à not free (valid partition)

§ P3 à rejected (invalid partition)

10 /
RULES

è This technique is applied on each parameter

è You don’t work on combination

è This technique can be applied on any test level (unit test to acceptance
testing)

è It is commonly and widely use

è For DO 178B (aeronautic standard) use of this technique is mandatory.

11 /
MORE COMPLEX SAMPLE

è Bank loan

è Duration of the loan between 5 and 30 years, other values are rejected.

è Age of borrower between, 18 and 80 years, other values are rejected.

è Between 5 and 10 loan rate: 3,5%

è Between 11 and 20 loan rate: 3,8%

è Between 21 and 30 loan rate: 4%

è Above 45 years the insurance is automatically increased by 0, 3%

12 /
DOMAIN

è Age
18 80

è Duration

5 30

13 /
CLASSES FOR AGES

è Valid classes for ages

§ C1 [18-45] : allows to check that insurance has not been increased.
§ C2 [46-80] : allows to check that insurance has been increased by 0,3%

è Invalid classes for ages

§ C3 [ ? – 18 [ : allows to check value rejected (robustness)
§ C4 ]80 - ?] : allows to check value rejected (robustness)

è Valid classes for duration

§ C5: [5-10] : allows to check loan rate = 3,5%
§ C6 :[11-20] : allows to check loan rate = 3,8%
§ C7: [21-30]: allows to check loan rate = 4%

è Invalid classes for duration

§ C8: [? – 5[ : allows to check value rejected (robustness)
§ C9: ]30 - ?] : allows to check value rejected (robustness)

14 /
EXAMPLE OF VALUE COMBINATION

Valeurs TC1 TC2 TC3 TC4 TC5 TC6 TC7 TC8

Duration 5- 10 x x
Duration 11-20 x x
Duration 21-30 x
Duration < 5 x
Duration >30 x
Age 18-45 x x
Age 46-80 x x x
Age < 18 x
Age > 80 x

15 /
COMBINATION, HOW FAR?

è Exhaustive: 5 (duration) *4 (age) = 20

è Here
§ All valid classes have been exercised.
§ Invalid classes are always combined with valid classes in order to check « rejection
behavior » is really linked to one invalid classes.

è Other combinations can be added in order to check influence of parameter

on output.

è But other consideration could be taken in account:

§ Combination probability
§ Risk
§ Business

16 /
BOUNDARY VALUES

§ boundary value: An input value or output value which is on the edge of an

equivalence partition or at the smallest incremental distance on either side of an
edge, for example the minimum or maximum value of a range.

Boundary
Invalid Values
values

Valid Values

17 /
EQUIVALENCE CLASSE/ BOUNDARY VALUE

è Both techniques are used to find test case.

è Example: Valid values should be between V1 and V2.

3 equivalence
Classe 1 Classe 2 Classe 3
classes

2 boundaries
and 4 values to
test them.

V1 V2
18 /
BANK LOAN EXAMPLE

è Bank loan

è Duration of the loan between 5 and 30 years, other values are rejected.
§ Between 5 and 10 loan rate: 3,5%
§ Between 11 and 20 loan rate: 3,8%
§ Between 21 and 30 loan rate: 4%

C8 5 C5 10 C6 20 C7 30 C9

Invalid 3,5% 3,8% 4,0% Invalid

è Limits : 4,5, 10, 11, 20,21,30, 31

è For Class C8 lower limit depends on the data type.

§ Eg: 0 if signed integer
è For Class C9 upper limit depends on the data type.
19 /
EXAMPLE WITH INT

è REQ_0001
If myIntVar is greater than 64 then mysoftware shall set myOutput to
myIntVar, otherwise mysoftware shall set myOutput to minus myIntVar

è REQ_0002
If myIntVar is invalid, mysoftware shall set myOutput to -1

è Data dictionnary:
§ myIntVar (integer) Range: [-1024;-64] ∪ [64;1024]
§ myOutput (integer)

-Min INT -1024 -64 64 1024 -Max INT

20 /
TEST CASES

è EC1 (invalid) = [MIN_INT ; -1024[, à eg: -2000

è EC2 (valid) = [-1024 ; -64] à -800

è EC3 (invalid) = ]-64 ; 64[ à0

è EC4 (valid) = [64 ; 1024] à +900

è EC5 (invalid) = ]1024 ; MAX_INT] à + 3000

è Boundaries values: MIN_INT -1025, -1024, -64, -63, 63,64,1025, MAX_INT

21 /
EXAMPLE WITH REAL

è REQ_0001:mysoftware shall set myOutput to myFltVar + cos (myFltVar)

è REQ_0002
§ If myFltVar invalid and negative, mysoftware shall set myOutput to -86401.0
§ If myFltVar invalid and positive, mysoftware shall set myOutput to 86401.0

è Data dictionnary:
§ myFltVar Float64 in [-86400.0;86400.0]
§ myOutput Float64 out [-86401.0;86401.0]

-Min DOUBLE -84400 84400 -Max DOUBLE

22 /
TEST CASES

è EC1 (invalid) = [MIN_DOUBLE ; -86400.0[, à -100000

è EC2 (valid) = [-86400.0 ; 86400.0], à 0

è EC3 (invalid) = ]86400.0 ; MAX_DOUBLE] à +120000

è Boundaries
values: MIN_DOUBLE, -86400.0 , -86400.0 –e, 86400.0, 86400.0+e,
MAX_DOUBLE

23 /
LIMIT VALUES : CASE OF CRITICAL FUNCTION

è If x > 2 x integer Limit=2

P1 P2

2 3
è By using 2 and 3 as input for checking limit value we can show the change
of behavior between partition 1 and partition 2 so in most cases only 2
values are needed. P1 P2

1 2 3
è Incase of critical function you may add a third one, the value just before
the limit.
24 /
CONDITION TABLE

è A table showing combinations of inputs and/or stimuli (causes) with their

associated outputs and/or actions (effects), which can be used to design test
cases.

è Test number if n condition takes only 2 values: 2n

Condition 1 Behavior 1
Condition 2 Behavior 2
. Item under test .
. .
Condition n Behavior m
Combinate

25 /
CONDITION TABLE : HOW TO

è Identifyconditions influencing decision. ‘-’ means condition

has no influence.

è Identify all actions as result of a condition : ‘X’ means action

is effective

è Construct the whole table with all column

è Verify table with the process expert (oracle)

è Simplify table :
§ Suppress all impossible combination
§ Suppress all combination where condition do not not have influence
on decision

26 /
CONDITION TABLE

TC1 TC2 TC3 TC4 TC5 TC6 TC7 TC8

C1 Y Y Y Y N N N N
C2 Y Y N N Y Y N N
C3 Y N Y N Y N Y N

A1 X X X X X
A2 X X X
TC1-TC2

Y
Y
-
X
27 /
EXAMPLE : CASH DISPENSER

Cash Dispenser use case.

Conditions
Invalid card Yes No No No No
Bad code pin - Yes No No No
overdrawn account - - Yes No No
Not enough money - - - Yes No
Behavior
Reject card X X
No transaction X
Transaction rejected X
Transaction accepted X

28 /
OTHER EXAMPLE

è If input is higher or equal to VAL2 then Algo1 is used to compute output.

è Ifinput is comprised between VAL1 and VAL2 then Algo2 is used to

compute output.

è If input is lower or equal to VAL1 then output is equal to input.

Conditions
Input >= VAL2 Yes No No
VAL1 < Input < VAL2 No Yes No
Input <= VAL1 No No Yes
Behavior
Algo 1 Yes No No
Algo 2 No Yes No
No Algorithm No No Yes

29 /
TABLE WITH NON BOOLEAN CONDITION

è Example

è3 parameters:

Clothe type : Man, Woman

Clothe size : 36,38,40,42

Clothe color : red, black, white

How many combination?

30 /
 TABLE WITH NON BOOLEAN CONDITION

Here the decision table:

ty M M M M M M M M M M M M F F F F F F F F F F F F
c R R R R B B B B W W W W R R R R B B B B W W W W
o
si 36 3 4 4 3 3 4 4 3 3 4 4 3 3 4 4 3 3 4 4 3 3 4 4
8 0 2 6 8 0 2 6 8 0 2 6 8 0 2 6 8 0 2 6 8 0 2

pr 10 11 1 1 1 11 1 1 11 1 1 1 11 1 1 1 11 1 1 1 1 1 1 1
2 3 0 2 3 2 3 4 2 3 4 2 3 4 2 3 4 5

31 /