Integration testing

Ernestas Kardzys, IFME-0/2

[email protected]
www.ernestas.info
http://www.ernestas.info/ktu/pt/Integration_Te
sting.pptx

Agenda
What

Is Unit Testing?
What Is Integration Testing?
Terminology
Different Types of Integration Testing:

Big Bang Testing

Top-down Testing
Bottom-up Testing
Performance vs. Load vs. Stress Testing
Sandwich Testing

Questions

For The Audience

Discussion
Literature
2

What Is Unit Testing? (1)

In

computer programming, unit testing

is a method by which individual units of
source code are tested to determine if
they are fit for use.
A unit is the smallest testable part of an
application. In procedural programming
a unit may be an individual function or
procedure. In object-oriented
programming a unit is usually a method.
Unit tests are created by programmers
or occasionally by white box testers.
3

Example of Unit Testing

Definition of Integration Testing

Integration

testing ("I&T") is the phase in

software testing in which individual software
modules are combined and tested as a
group.
It occurs after unit testing and before
system testing.
Integration testing takes as its input
modules that have been unit tested, groups
them in larger aggregates, applies tests
defined in an integration test plan to those
aggregates, and delivers as its output the
integrated system ready for system testing.
5

Why Do You Need Integration

Testing?
To

make sure
that your
components
satisfy the
fallowing
requirements:
Functional.
Performance.
Reliability.

Terminology (1)
Stub

the dummy modules

that simulates the low level
modules.
Driver the dummy modules
that simulate the high level
modules.
Top-down approach (also
known as step-wise design) the breaking down of a
system to gain insight into its
compositional sub-systems.
A bottom-up approach is the
piecing together of systems
to give rise to grander
systems, thus making the
original systems sub-systems
of the emergent system.
7

Terminology (2)
Bottleneck

a
phenomenon where the
performance or capacity of
an entire system is limited
by a single or limited
number of components or
resources.
Regression testing - any
type of software testing that
seeks to uncover new errors,
or regressions, in existing
functionality after changes
have been made to the
software, such as functional
enhancements, patches or
configuration changes.

What Is Big Bang Testing?

In

Big Bang Integration

testing, individual modules
of the programs are not
integrated until every
thing is ready. It is called
'Run it and see' approach.
In this approach, the
program is integrated
without any formal
integration testing, and
then run to ensures that all
the components are working
properly.
9

Disadvantages of Big
Bang
Defects

present at the interfaces of

components are identified at very
late stage.
It is very difficult to isolate the
defects found, as it is very difficult
to tell whether defect is in
component or interface.
There is high probability of
missing some critical defects
which might surfaced in production.
It is very difficult to make sure that
all the cases for integration testing
are covered.
10

Big Bang Testing:

Conclusion
This

is not the way

you should
integrate and test
software!

But

it might be good
with this
assumption applied:
for small systems,
but not for
enterprise level
applications.
11

How To Integrate?
If

Big Bang integration is bad, then

how to integrate?
The answer: incremental
integration.

12

Bottom-up Integration Testing (1)

In

bottom up
integration testing,
module at the lowest
level are developed first
and other modules
which go towards the
'main' program are
integrated and tested
one at a time.
Bottom up integration
also uses test drivers to
drive and pass
appropriate data to the
lower level modules.
13

Bottom-up Integration Testing (2)

As

and when code for

other module gets ready,
these drivers are replaced
with the actual module.
In this approach, lower
level modules are tested
extensively thus make
sure that highest used
module is tested properly.
14

Bottom-up Testing Graphical

Represenation

15

Comments on Graphical
Representation
Modules

E and F are tested. Then

modules B, E, F are tested.
Module F is tested. Then
modules F and G are tested.
Module C is tested.
Finally modules A, B, C, D, E, F,
G are tested.

16

Advantages of Bottom-up Testing

Behavior

of the interaction points

are crystal clear, as components
are added in the controlled
manner and tested repetitively.
Appropriate for applications
where bottom up design
methodology is used.

17

Disadvantages of Bottom-up
Testing
Writing

and maintaining test

drivers is more difficult than
writing stubs.
This approach is not suitable for
the software development using
top-down approach.

18

Top-down Testing
Top

down integration testing is an

incremental integration testing
technique which begins by testing the
top level module and and progressively
adds in lower level module one by one.
Lower level modules are normally
simulated by stubs which mimic
functionality of lower level modules.
As you add lower level code, you will
replace stubs with the actual
components.

19

Advantages of Top-down Testing

Driver

do not have
to be written
when top down
testing is used.
It provides early
working module of
the program and
so design defects
can be found and
corrected early.
20

Disadvantages of Top-down
Testing
Stubs

have to be written with

utmost care as they will simulate
setting of output parameters.
It is difficult to have other people
or third parties to perform this
testing, mostly developers will
have to spend time on this.

21

Top-down And Bottom-up

Testing: Conclusion
Youll

probably

use a
combination of
these two
techniques.
Its

called
Sandwich testing
strategy.

22

Sandwich Testing Strategy

Combines

top-down strategy with bottom-up

strategy
The system is view as having three layers
A target layer in the middle
A layer above the target
A layer below the target
Testing converges at the target layer
How do you select the target layer if there
are more than 3 layers?
Heuristic: Try to minimize the number of
stubs and drivers
23

Graphical Representation of
Sandwich Testing Strategy
A

Bottom
Level
Tests

TestE

LevelI

LevelII
H

LevelIII

TestB,E,F
TestF
TestG
TestD,G,H

Test
A,B,C,D,
E,F,G,H

TestH
TestA,B,C,D
Top
Level
Tests

TestA
24

Comments on Graphical
Representation
Modules

E and F are tested. Then

B, E, F are tested.
Modules G and H are tested.
Then D, G, H are tested.
Module A is tested. Then A, B, C,
D are tested.
Finally modules A, B, C, D, F, G,
H are tested.
25

Sandwich Testing Strategy:

Conclusions
Top

and Bottom
Layer Tests can be
done in parallel.
Does not test the
individual
subsystems
thoroughly before
integration.

26

Performance vs. Load vs. Stress

testing
There

are three
main types of
speed-related
testing:
performance
testing, load
testing and
stress testing.

27

Performance testing
The

goal of
performance
testing is not to
find bugs, but to
eliminate
bottlenecks and
establish a
baseline for
future
regression
testing.
28

Load Testing
Load

testing is
usually defined as
the process of
exercising the system
under test by feeding
it the largest tasks it
can operate with.
Load testing is
sometimes called
volume testing, or
longevity/enduranc
e testing.

29

Examples of Load Testing

Testing

a word processor by
editing a very large document
Testing a printer by sending it a
very large job
Testing a mail server with
thousands of users mailboxes
A specific case of volume testing
is zero-volume testing, where
the system is fed empty tasks
30

Stress Testing (1)

Testing

conducted to
evaluate a system or
component at or
beyond the limits of its
specified requirements
to determine the load
under which it fails and
how.
A graceful degradation
under load leading to
non-catastrophic
failure is the desired
result.

31

Stress Testing (2)

Stress

testing tries to
break the system under
test by overwhelming its
resources or by taking
resources away from it
(in which case it is
sometimes called
negative testing).
The main purpose behind
this madness is to make
sure that the system fails
and recovers gracefully -this quality is known as
recoverability.

32

Apache Benchmarking
(ab)
ab

is a tool for
benchmarking Apache
server. It is designed to give
you an impression of how
Apache installation
performs.
Installation: #apt-get
install apache2-utils
#ab -kc 10 -t 30
http://localhost/
This will open 10
connections, using KeepAlive on them and
hammering localhost for 30
seconds through those
connections.
Boing AH64 Apache
33

Recommendations for
Integration Testing
OK,

I want to test the software. Any

recommendations?

34

Steps of Integration
Testing (1)
Select

component to test
and unit test the classes
of the component.
Put selected component
into system. Do any
preliminary fix-up
necessary to make the
integration test
operational (drivers,
stubs).
Define test cases that
exercise all uses cases
with the selected
component
35

Steps of Integration
Testing (2)
Define

test cases that

exercise the selected
component
Execute performance tests
and once again with
another component
The

primary goal of
integration testing is to
identify errors in the
(current) component
configuration.
36

Main Point: Incremental

Integration
All

the units of a
system must be
integrated
consequtively
and integrated
in step by step
process by
incrementing the
levels of testing
at one end to
other end.
37

Questions For The

Audience
What

is the main difference

between unit testing and
integration testing?
What is Big Bang testing?
What is the difference between
bottom-up and top-down testing?
What is stress testing?

38

Literature (1)
http://en.wikipedia.org/wiki/Unit_

testing
http://en.wikipedia.org/wiki/Syst
em_integration_testing
http://en.wikipedia.org/wiki/Integ
ration_testing
http://msdn.microsoft.com/en-u
s/library/aa292128%28v=vs.71%2
9.aspx
http://www.testinggeek.com/
http://swen.uwaterloo.ca/~kostas
39

Literature (2)
http://www.debianhelp.co.uk/apa

cheab.htm
http://agiletesting.blogspot.com/
2005/02/performance-vs-load-vs-st
ress-testing.html
http://en.wikipedia.org/wiki/Regre
ssion_testing
http://en.wikipedia.org/wiki/Bottl
eneck
40

Youtube Links
http://www.youtube.com/watch?v=

J22IjXlHI4Y&feature=feedu
[Lecture Software Integration
Testing, 1:50].
http://www.youtube.com/watch?
v=bgIAnWfNLOw&playnext=1&list
=PLF7935AFA888C74E2
[Integration Testing In Software
Testing Projects, 5:46]
http://www.youtube.com/watch?v
=czyoKfWp6z4

41

Topics for Discussion

Good

test is the test which

reveals errors. Is it?
Incremental integration is too
expensive? Try to integrate 200300 components
What is the best method for
testing?

42

Thank You For Attention!

Ernestas Kardzys, IFME-0/2
[email protected],
www.ernestas.info
http://www.ernestas.info/ktu/pt/Integration_Testing.p
ptx

43