Object-Oriented Software Engineering Integration and

System Testing
Using UML, Patterns, and Java

Overview

• Integration testing
• Big bang
• Bottom up
• Top down
• Sandwich
• Continuous
• System testing
• Functional
• Performance
• Acceptance testing
• Examples
• Summary

Page 1
 Integration Testing

• The entire system is viewed as a collection of

subsystems (sets of classes) determined during
the system and object design

• Goal: Test all interfaces between subsystems

and the interaction of subsystems

• The Integration testing strategy determines the

order in which the subsystems are selected for
testing and integration.

Why do we do integration testing?

• Unit tests only test the unit in isolation

• Many failures result from faults in the interaction of

subsystems

• Often many Off-the-shelf components are used that

cannot be unit tested

• Without integration testing the system test will be very

time consuming

• Failures that are not discovered in integration testing will

be discovered after the system is deployed and can be
very expensive.

Page 2
 Stubs and drivers

• Driver:
Driver
• A component, that calls the TestedUnit
• Controls the test cases

Tested
• Stub: Unit
• A component, the TestedUnit
depends on
• Partial implementation
Stub
• Returns fake values.

Example: A 3-Layer-Design (Spreadsheet)

A
Spread
A
SheetView Layer I

B C D
Entity
Data Currency
B Calculator
C D Layer II
Model Converter

E F G
BinaryFile XMLFile Currency Layer III
E F G
Storage Storage DataBase

Page 3
 Big-Bang Approach A

B C D
Test A

Test B
E F G
Test C
Test
Test D A, B, C, D,
E, F, G
Test E

Test F

Test G

1. Test xem
Big bang Example trang chu
2. Test xem
san pham
3. Test Tim
kiem
4. Test xem chi
tiet SP
5. ……..
6. Test xem
theo chung
loai
7. Test xem
theo nha SX
8. Test mua
hang

Page 4
 Bottom-up Testing Strategy
• The subsystems in the lowest layer of the call
hierarchy are tested individually
• Then the next subsystems are tested that call the
previously tested subsystems
• This is repeated until all subsystems are included
• Drivers are needed.

Bottom-up Integration A

B C D

Test E
E F G

Test B, E, F

Test F

Test C Test
A, B, C, D,
E, F, G

Test G Test D,G

10

Page 5
 1. Test xem
Bottom-up Example trang chu
2. Test xem
theo nha SX
3. Test xem
theo chung
loai
4. Test xem
san pham
5. Test mua
hang
6. Test Quan
ly gio hang
7. Test xem chi
tiet SP
8. ……..

11

Pros and Cons of Bottom-Up Integration

Testing
• Con:
• Tests the most important subsystem (user interface)
last
• Drivers needed
• Pro
• No stubs needed
• Useful for integration testing of the following systems
• Object-oriented systems
• Real-time systems
• Systems with strict performance requirements.

12

Page 6
 Top-down Testing Strategy

• Test the top layer or the controlling subsystem

first
• Then combine all the subsystems that are called
by the tested subsystems and test the resulting
collection of subsystems
• Do this until all subsystems are incorporated
into the test
• Stubs are needed to do the testing.

13

Top-down Integration A

B C D

E F G

Test
Test A Test A, B, C, D A, B, C, D,
E, F, G

Layer I Layer I + II All Layers

14

Page 7
 1. Test xem
Top-down Example trang chu
2. Test xem
san pham
3. Test xem
theo nha SX
4. Test xem
theo chung
loai
5. Test Quan
ly gio hang
6. Test mua
hang
7. Test xem chi
tiet SP
8. ……..

15

Pros and Cons of Top-down Integration

Testing
Pro
• Test cases can be defined in terms of the
functionality of the system (functional
requirements)
• No drivers needed

Cons
• Writing stubs is difficult: Stubs must allow all
possible conditions to be tested.
• Large number of stubs may be required,
especially if the lowest level of the system
contains many methods.
• Some interfaces are not tested separately.

16

Page 8
 Sandwich Testing Strategy

• Combines top-down strategy with bottom-up

strategy
• The system is viewed as having three layers
• A target layer in the middle
• A layer above the target
• A layer below the target
• Testing converges at the target layer.

17

Sandwich Testing Strategy A

B C D

Test A

E F G
Test A,B,C, D
Test E

Test
Test B, E, F A, B, C, D,
Test F E, F, G

Test D,G
Test G

18

Page 9
 Pros and Cons of Sandwich Testing

• Top and Bottom Layer Tests can be done in

parallel

• Problem: Does not test the individual

subsystems and their interfaces thoroughly
before integration

• Solution: Modified sandwich testing strategy

19

1. Test xem trang

Sandwich Example chu
2. Test xem theo
nha SX
3. Test xem theo
chung loai
4. Test xem san
pham
5. Test mua hang
6. Test Quan ly gio
hang
7. Test xem theo
nha SX
8. Test xem theo
chung loai
9. Test Quan Ly
gio hang
10. Test Mua hang
11. Test xem chi tiet
SP
12. ……..

20

Page 10
 Modified Sandwich Testing Strategy

• Test in parallel:
• Middle layer with drivers and stubs
• Top layer with stubs
• Bottom layer with drivers
• Test in parallel:
• Top layer accessing middle layer (top layer
replaces drivers)
• Bottom accessed by middle layer (bottom
layer replaces stubs).

21

Modified Sandwich Testing A

B C D
Test A
Test A,C
Test C E F G

Test B
Test
Test E Test B, E, F A, B, C, D,
E, F, G
Test F

Test D
Test D,G
Test G

22

Page 11
 Continuous Testing

• Continuous build:
• Build from day one
• Test from day one
• Integrate from day one
 System is always runnable

• Requires integrated tool support:

• Continuous build server
• Automated tests with high coverage
• Tool supported refactoring
• Software configuration management
• Issue tracking.

23

Continuous Testing Strategy

A
Spread
SheetView Layer I

B C D
Data Currency
Calculator Layer II
Model Converter

E F G
BinaryFile XMLFile Currency Layer III
Storage Storage DataBase

+ Cells
Sheet View + File Storage
+ Addition

24

Page 12
 Steps in Integration Testing

1. Based on the integration 4. Test subsystem

strategy, select a decomposition: Define test
component to be tested. cases that exercise all
Unit test all the classes in dependencies
. the component. 5. Test non-functional
requirements: Execute
2. Put selected component performance tests
together; do any 6. Keep records of the test
preliminary fix-up cases and testing activities.
necessary to make the
integration test operational 7. Repeat steps 1 to 7 until
the full system is tested.
(drivers, stubs)
3. Test functional
requirements: Define test The primary goal of integration
testing is to identify failures
cases that exercise all uses with the (current)
cases with the selected component configuration.
component

25

System Testing

• Functional Testing
• Validates functional requirements
• Performance Testing
• Validates non-functional requirements
• Acceptance Testing
• Validates clients expectations

26

Page 13
 Functional Testing

Goal: Test functionality of system

• Test cases are designed from the requirements
analysis document (better: user manual) and
centered around requirements and key functions
(use cases)
• The system is treated as black box
• Unit test cases can be reused, but new test
cases have
. to be developed as well.

27

Performance Testing

Goal: Try to violate non-functional requirements

• Test how the system behaves when overloaded.
• Can bottlenecks be identified? (First candidates for
redesign in the next iteration)
• Try unusual orders of execution
• Call a receive() before send()
• Check the system’s response to large volumes
of data
• If the system is supposed to handle 1000 items, try it
with 1001 items.
• What is the amount of time spent in different
use cases?
• Are typical cases executed in a timely fashion?

28

Page 14
 Types of Performance Testing
• Stress Testing • Security testing
• Stress limits of system • Try to violate security
requirements
• Volume testing
• Test what happens if large
• Environmental test
amounts of data are handled • Test tolerances for heat,
humidity, motion
• Configuration testing
• Test the various software and
• Quality testing
hardware configurations • Test reliability, maintain-
ability & availability
• Compatibility test
• Test backward compatibility
• Recovery testing
with existing systems • Test system’s response to
presence of errors or loss
• Timing testing of data
• Evaluate response times and
time to perform a function
• Human factors testing
• Test with end users.

29

Acceptance Testing

• Goal: Demonstrate system is • Alpha test:

ready for operational use • Client uses the software
• Choice of tests is made by at the developer’s
client environment.
• Many tests can be taken • Software used in a
from integration testing controlled setting, with
• Acceptance test is the developer always
performed by the client, not ready to fix bugs.
by the developer. • Beta test:
• Conducted at client’s
environment (developer is
not present)
• Software gets a realistic
workout in target environ-
ment

30

Page 15
 Testing has many activities

Establish the test objectives

Design the test cases
Write the test cases
Test the test cases
Execute the tests
Evaluate the test results
Change the system
Do regression testing

31

Test Team
Professional
Tester too familiar
Programmer with code
Analyst

Test System
User Team Designer

Configuration
Management
Specialist

32

Page 16
 The 4 Testing Steps
1. Select what has to be tested 3. Develop test cases
• Analysis: Completeness of • A test case is a set of test
requirements data or situations that will
• Design: Cohesion be used to exercise the unit
(class, subsystem, system)
• Implementation: Source being tested or about the
code attribute being measured
2. Decide how the testing is 4. Create the test oracle
done • An oracle contains the
• Review or code inspection predicted results for a set of
• Proofs (Design by Contract) test cases
• Black-box, white box, • The test oracle has to be
written down before the
• Select integration testing actual testing takes place.
strategy (big bang, bottom
up, top down, sandwich)

33

Guidance for Test Case Selection

• Use analysis knowledge • Use implementation
about functional knowledge about
requirements (black-box algorithms and
testing):
datastructures:
• Use cases
• Force a division by zero
• Expected input data
• Invalid input data • If the upper bound of an
array is 10, then use 11 as
• Use design knowledge index.
about system structure,
algorithms, data structures
(white-box testing):
• Control structures
• Test branches, loops,
...
• Data structures
• Test records fields,
arrays, ...

34

Page 17
 Examples

• Web site with constrains

• Over 100 user login to web site
• Response time < 1ms
• Connect SQL database
• Authenticate user: user name/password
• Access home page after login
• High security

35

Functional testing examples

Expected
Test Case ID Test Scenario Test Steps Test Data Actual Results Pass/Fail
Results

1.Go to site
http://demo.guru9
Check Customer Userid = guru99
9.com User should Login
F01 Login with valid Password = As Expected Pass
2.Enter UserId into home page
Data pass99
3.Enter Password
4.Click Submit

1.Go to site
http://demo.guru9
Check Customer Userid = guru99 User should not
9.com
F02 Login with invalid Password = Login into home As Expected Pass
2.Enter UserId
Data glass99 page
3.Enter Password
4.Click Submit

36

Page 18
 Performance test case examples
Expected
Test Case ID Test Scenario Test Steps Test Data Actual Results Pass/Fail
Results

1.Open web site

http://demo.guru9
9.com with 51th Userid = guru99
User should Login
P01 Check user limited times Password = Non As Expected Fail
into home page
2.Enter UserId pass99
3.Enter Password
4.Click Submit

1.Go to site
http://demo.guru9
9.com with 51th
Userid = guru99
Check time times
P02 Password = T<1ms Not As Expected Fail
response 2.Enter UserId
glass99
3.Enter Password
4.Click Submit

1.Go to site Userid =

http://demo.guru9 <JavaScript
9.com with 51th alert=‘Okie’>
Check SQL
times Password = SQL injecttion 
P03 Injection No user like this Fail
2.Enter UserId <JavaScript data fail
(security)
3.Enter Password alert=‘Okie’>
4.Click Submit

1.Go to site
Check web page
P04 http://demo.guru9 No data No access Access home page Fail
request (security)
9.com/home.aspx

37

Acceptant test case example

Expected
Test Case ID Test Scenario Test Steps Test Data Actual Results Pass/Fail
Results

1.Open Firefox
2.Go to site
http://demo.guru9
9.com
3.Enter UserId
4.Enter Password
5.Click Submit Userid = guru99
1. Firefox: ok Firefox: Ok
A01 Check browser 6.Open IE Password = Fail
2. IE: ok IE: not ok
7.Go to site pass99
http://demo.guru9
9.com
8.Enter UserId
9.Enter Password
10.Click Submit

1.Go to site
http://demo.guru9
Userid = user 1
9.com User can not Login
A02 Check invalid user Password = As Expected Pass
2.Enter UserId into web page
glass99
3.Enter Password
4.Click Submit

38

Page 19
 Test case guide

• Should
• Test Cases need to be simple and transparent
• Create Test Case with End User in Mind
• Avoid test case repetition.
• Do not Assume
• Using test case tools
• Ensure 100% Coverage
• Repeatable and self-standing
• Peer Review.

39

Scenario 1

• System testing
• 1 system includes:
• 2 web server
• 2 SQL database server
• 4 switch
• 3 LAN
• Constraints
• Redundancy (dự phòng)
• Cluster (gom cụm)
• Security (bảo mật)
• Available (sẳn sàng)

40

Page 20
 Scenario 2

• System testing
• 1 system includes:
• 2 mail server
• 2 Domain controller
• 2 DHCP server
• 4 switch
• 5 LAN
• 2 Access point
• Constraints
• Redundancy (dự phòng)
• Cluster (gom cụm)
• Security (bảo mật)
• Available (sẳn sàng)

41

Summary

• Testing is still a black art, but many rules and

heuristics are available
• Testing consists of
• Unit testing
• Integration testing
• System testing
• Acceptance testing
• Design patterns can be used for integration
testing
• Testing has its own lifecycle

42

Page 21