Continental Test Lab

September 2014

Interior Infotaiment and Connectivity R&D Romania

Table of Contents

1 Black Box Techniques

2 White Box Techniques

3 Experienced Based

4 Test Plan

5 Test Report

Interior Infotaiment and Connectivity R&D Romania

2 June 2014 Silvia Tanase, © Continental Automotive Romania SRL
Test Techniques

In this chapter we will

Look at the different test design techniques

Look at the common features of the different techniques

3/
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Categories of Test Techniques

The purpose of a test design techniques is to identify test conditions and test
cases

Some techniques fall clearly into a single category, others have elements of more
then one category

In the next slides we will refer to :

Specification-based approaches as black-box techniques

Structure-based approaches as white-box techniques

Experiences-based

4/
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Specification-based or Black Box Techniques
Equivalence Partitioning are also known as equivalence classes
Partitions exercised: Valid partitions and invalid partitions

Equivalence Partitioning is intended for reducing the total number of tests

by grouping together data or test cases which test the same thing, and
then making do with testing one value out of each group.
The idea is that, if you expect the same result from two tests, it is
sufficient to run one of them.

Equivalence partitioning is applicable at all levels of testing

5/
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Specification-based or Black Box Techniques

Equivalence Partitioning

Exercise:
Postal rates for 'light letters' are 25p up to l0g, 35p up to 50g plus an extra
l0p for each additional 25g up to l00g. Which test inputs (in grams) would
be selected using equivalence partitioning?
a. 8,42,82,102
b. 4,15, 65, 92,159
c. 10,50,75,100
d. 5, 20, 40, 60, 80

6/
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Specification-based or Black Box Techniques

25p – 35p – 45p – 55p –

… …
10g 50g 75g 100 g

(0,9) ………. (11,49) …....... (51,74) ………. (76,99) …… (101, …)

a. 8,42,82,102
b. 4,15, 65, 92,159
c. 10,50,75,100
d. 5, 20, 40, 60, 80

7/
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Specification-based or Black Box Techniques
Boundary value analysis
Boundaries to be exercised: just below, on, just above

Note that we have both valid boundaries (in the valid partitions)
and invalid boundaries (in the invalid partitions).

Technically, because every boundary is in some partition, if you did only

boundary value analysis you would also have tested every equivalence
partition

8/
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Specification-based or Black Box Techniques
Boundary value analysis

Exercise:
In a system designed to work out the tax to be paid:
An employee has £4000 of salary tax free.
The next £1500 is taxed at 10%.
The next £28000 after that is taxed at 22%.
Any further amount is taxed at 40%.

To the nearest whole pound, which of these is a valid Boundary Value Analysis test
case?
A £28000.
B £33501.
C £32001.
D £1500.

9/
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Specification-based or Black Box Techniques

28000 ….
• 4000 • 33500
• Tax • 5500 • 22% • >33500
free • 10% • 40%
+1500 ….

To the nearest whole pound, which of these is a valid Boundary Value Analysis
test case?
A £28000.
B £33501.
C £32001.
D £1500.

10 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Specification-based or Black Box Techniques

 State Transition

• Identify possible transitions

• Identify initial state and data/events that will trigger the transition

• Identify the expected results

11 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Specification-based or Black Box Techniques
State Transition
Exercise:
Given the state diagram, which test case is the minimum series of valid transitions to cover
every state?
a. SS-S1-S2-S4-S1-S3-ES
b. SS-S1-S2-S3-S4-ES
c. SS-S1-S2-S4-S1-S3-S4-S1-S3-ES
d. SS-S1-S4-S2-S1-S3-ES

12 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Specification-based or Black Box Techniques
First try:
SS -> S1 -> S2 -> S4 -> S1 -> S2 -> S1 -> ….
Second try:
SS->S1->S2->S4->S1->S3->ES => 7 states
Third try:
SS->S1->S3->S4->S1->S2->S4->S1->
->S3->ES => 9 states
4th try:
SS->S1->S2->S4->S1->S3->S4->S1->
->S3->ES => 9 states

13 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Specification-based or Black Box Techniques

Decision table (Cause Effect Graphing)

It creates combination of conditions that might not otherwise have
been exercised during testing
It serve as a guide to creating test cases that otherwise might not be
exercised

The first task is to identify a suitable function or subsystem that has a

behavior which reacts according to a combination of inputs or events

14 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Specification-based or Black Box Techniques

Decision table (Cause Effect Graphing)

Exercise:
If you are a new customer opening a credit card account, you will get a
15% discount on all your purchases today. If you are an existing customer
and you hold a loyalty card, you get a 10% discount. If you have a
coupon, you can get 20% off today (but it can't be used with the
'newcustomer' discount). Discount amounts are added, if applicable.

Create the decision table.

15 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Specification-based or Black Box Techniques

Solution:

We may choose to test each rule/combination and if there are only a few this will usually be
the case. However, if the number of rules/combinations is large we are more likely to sample
them by selecting a rich subset for testing.

16 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Specification-based or Black Box Techniques
Use Case testing

Tests can be specified from use cases or business scenarios

Use case testing is a technique that helps us to identify test cases that exercise
the whole system on a transaction by transaction basis from start to finish

A use case is a description of a particular use of the system by an actor (a user

of the system). Each use case describes the interactions the actor has with the
system in order to achieve a specific task (or, at least, produce something of
value to the user). Actors are generally people but they may also be other
systems. Use cases are a sequence of steps that describe the interactions
between the actor and the system.

17 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Specification-based or Black Box Techniques
Use Case testing

18 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Specification-based or Black Box Techniques

Random testing

Means of testing functionality with a randomly selected set of values

Results of the output are compared against software specifications to

verify that the test output is pass or fail

19 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Specification-based or Black Box Techniques

Summary

Black-box testing focuses on functionality

20 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Structured-based or White Box Techniques

White box test techniques

Statement testing

Decision testing

White box testing can be done immediately after code is written

A combination of all techniques are required for a successful test

21 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Structured-based or White Box Techniques
Statement testing
Statement coverage = (Number of statements exercised/Total number of
statements)x100%
Exercise:
1 READ A

2 READ B

3 C =A + 2*B

4 IF C> 50 THEN

5 PRINT large C

6 ENDIF

We have two read statements, one assignment statement, and then one IF statement on
three lines, but the IF statement contains another statement (print) as part of it ? 6
statements
One test is sufficient for 100% statement coverage

22 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Structured-based or White Box Techniques
Decision testing
Decision coverage = (Number of decision outcomes exercised/Total number
of decision outcomes)x100%
Exercise:
1 READ A
2 READ B
3 C =A + 2*B
4 IF C> 50 THEN
5 PRINT large C
6 ENDIF

Two tests are needed for 100% decision coverage. Note that 100% decision coverage
guarantees 100% statement coverage, but not the other way around!

23 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Structured-based or White Box Techniques
Exercises:
1. Given the following code, which option is true about the minimum
number of test cases for full statement and decision coverage:
Read M

Read N
If M > N then print “Message 21”
ENDIF
If N >= M then
a) 1 test for statement coverage, 2 tests
Print “Message 31” for decision coverage
If N > 10 then
b) 1 test for statement coverage, 3 tests
for decision coverage
Print “Message 41” c) 2 tests for statement coverage, 2 tests
ENDIF for decision coverage
d) 2 tests for statement coverage, 3 tests
ENDIF for decision coverage
Print “END”

24 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Structured-based or White Box Techniques
Test 1: Read M

M = 12 Read N

N= 11

Message 21
Test 2:
M = 15
N = 15
Message 31

Test 3:
M=2 Message 41

N=9

END

25 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Structured-based or White Box Techniques
Exercises:
1. Given the following code, which option is true about the minimum
number of test cases for full statement and decision coverage:
Read M

Read N
If M > N then print “Message 21”
ENDIF
If N >= M then
a) 1 test for statement coverage, 2 tests
Print “Message 31” for decision coverage
If N > 10 then
b) 1 test for statement coverage, 3 tests
for decision coverage
Print “Message 41” c) 2 tests for statement coverage, 2 tests
ENDIF for decision coverage
d) 2 tests for statement coverage, 3
ENDIF tests for decision coverage
Print “END”

26 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Structured-based or White Box Techniques

Exercises:
2. If you are flying with an economy ticket, there is a possibility that you
may get upgraded to business class, especially if you hold a gold card in
the airline's frequent flier program. If you don't hold a gold card, there is a
possibility that you will get 'bumped' off the flight if it is full and you check
in late.

Three tests have been run:

Test 1: Gold card holder who gets upgraded to business class
Test 2: Non-gold card holder who stays in economy
Test 3: A person who is bumped from the flight

What is the statement coverage of these three tests?

27 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Structured-based or White Box Techniques

What is the statement coverage of these three tests?

a.60%
b.70%
c.80%
d.90%

28 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Structured-based or White Box Techniques

What is the statement coverage of these three tests?

a.60%
b.70%
c.80%
d.90%

29 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Structured-based or White Box Techniques

Summary

White-box texting focuses on code

A systematic approach is needed for both (black-box and white-box)

Tests need to be planned, prepared, executed and verified
Expected results need to be defined and understood
Tools can help increase productivity and quality

30 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Experience-based

In the Experience-based testing the tests are derived from the tester’s
skills, intuition and experiences with similar applications and technologies

These techniques can be useful in identifying special tests not easily

captured by formal techniques

Experience-based techniques go together with specification-based and

structure-based techniques, and are also used when there is no
specification, or if the specification is inadequate or out of date.

31 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Experience-based
What is Error guessing?
“A test technique where the experience of the tester is used to anticipate what defects
might be present in the component or system under test as a result of errors made, and
to design tests specifically them.”

A structured approach to the Error Guessing is to enumerate a list of possible errors and to
design tests that attack these errors
This systematic approach is called Fault Attack

Error Guessing – should still be planned

Is part of the test process
Ad-Hoc testing can be encouraged if the tester maintains a log of those tests performed
& is able to reasonably recollect what actions were performed

32 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Experience-based
Exploratory testing
Is a test execution, based on a test charter containing test objectives, and
carried out with time-boxes

Is an approach that is most useful where there are few or inadequate

specifications and severe time pressure, or in order to augment or complement
other, more formal testing

It can serve as a check on the test process, to help ensure that the most
serious defects are found

Sometimes is much more productive than testing using scripts

33 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Experience-based

Summary

Error guessing – a technique where is used the tester’s experience

Exploratory testing – An informal test design technique where the

tester actively controls the design of the tests

34 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Test Plan
 Introduction
 Scope of testing / Purpose
 Relationship to the PLC (V-cycle)
 Acceptance criteria
 System overview

 Test approach
 Tailoring of Test Strategy and Test Documentation
 Definitions of Test Categories
 Test suite types
 Communication between testing and integration team

35 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Test Plan
 Test environment
 Test configuration set-up

 Test execution strategy

 How to capture and interpret the results
 Test report

 Scheduling and milestones

 Test effort estimation

 References

36 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Test Report

37 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti
Test Report

38 /
Autor
/Interior Infotaiment and Connectivity R&D Romania
Datu
m ©
Conti