SCS 3209: Software Engineering

S/W Integration, Testing and

Documentation
Orwa , Dec 2024
 Recap on Software coding
⚫ Coding
⚫ Coding standards and guidelines
⚫ Coding typical issues
⚫ Code reviews
⚫ Code testing and debugging
⚫ Black and
⚫ white box testing
⚫ Control flow based testing
 Learning Outcomes:
Testing integration and Documentation
⚫ By the end of this lecture, learners should be able
to:

⚫ Differentiate between
⚫ integration,
⚫ testing and
⚫ integration testing.
⚫ Integrate and test software using various integration strategies.
⚫ Document software using an appropriate documentation
structure and best practices
 Integration of S/W Components
⚫ Software system integration refers to the practice
of combining individually tested software
components into an integrated whole.

⚫ Two levels of integration

⚫ Integration of units into subsystems
⚫ Integration of subsystems into the complete
system
 Integration of Components
=> Software is integrated when components are
combined into subsystems or when subsystems
are combined into products/complete
representation whole system
S/W Integration
 Integration of Components
⚫ Integration and testing of integrated components
occur concurrently.

=> units are integrated and immediately tested.

Hence the two activities tend to be combined (
integration and testing)
⚫ Two types of integration testing
 Integration of Components
⚫ Subsystem integration testing: The integration of
basic units, even those that have been adequately
unit tested, may produce failures resulting from the
interaction of the units.

⚫ System integration testing: When some critical

mass of subsystems has been fully developed and
tested, the focus shifts to representative tests of the
completed application as a whole to determine
whether a product does what it is supposed to do.
 Integration Testing
Why Integration testing?

• Unit tests only test the unit in isolation

• Many failures result from faults in the interaction of subsystems

• Often many Off-the-shelf components are used that cannot be unit tested

• Without integration testing the system test will be very time consuming

• Failures that are not discovered in integration testing will be discovered

after the system is deployed and can be very expensive.
 Integration Testing
⚫ Focus of Integration testing is on
⚫ Interfaces between modules (or components)
⚫ Integrated functional features
⚫ Interacting protocols and messages

⚫ Objective:
⚫ To technically verify proper interfacing between modules,
and within sub-systems
⚫ Methods: White and Black Box testing
⚫ Is done by the developers
 Integration Testing
⚫ What do you need?:

- Integration strategy
- Integration test environment and test suite
- Module (or component) specifications
- Interface and design documents
 Integration Testing: Integration strategy
⚫ Integration Strategy usually refers to an approach
indicating the sequence or order to integrate different
units together

⚫ Typical traditional strategies can be split into

incremental and non-incremental
 Integration Testing
⚫ Incremental Strategies to s/w integration

⚫ Top- down software integration

⚫ Bottom-up software integration
⚫ Sandwich integration
 Top-down Integration
⚫ Modules are integrated by moving downward
through the control hierarchy, beginning with the
main module

⚫ Subordinate modules are incorporated in either a

depth-first or breadth-first fashion

⚫ DF: All modules on a major control path are

integrated

⚫ BF: All modules directly subordinate at each level

are integrated

14
Top-Down Integration Testing
Top module is tested
with stubs

Stubs are replaced one at a time,

either breadth-first or depth-first
Tests are run each time new modules
are integrated
Stubs must be developed, but drivers
are not needed
Top-down Integration Testing
⚫ Test the top layer or the controlling subsystem
first
⚫ Then combine all the subsystems that are called
by the tested subsystems and test the resulting
collection of subsystems
⚫ Do this until all subsystems are incorporated into
the test
⚫ Stubs are needed to do the testing.
 Top-down Integration
Advantages

• This approach verifies major control or

decision points early in the test process

⚫ Test cases can be defined in terms of the

functionality of the system (functional requirements)
⚫ No drivers needed

17
 Top-down Integration Testing
Disadvantages
• Stubs need to be created to substitute for
modules that have not been built or tested
yet; this code is later discarded
• Because stubs are used to replace lower
level modules, no significant data flow can
occur until much later in the
integration/testing process
⚫ Large number of stubs may be required, especially if
the lowest level of the system contains many methods.
⚫ Some interfaces are not tested separately.
Bottom-Up Integration Testing

Modules are integrated in a

bottom-up fashion until the entire
system has been assembled

Tests are run each time new modules

are integrated

Drivers must be developed, but stubs

are not needed
 Bottom-up Integration Testing
Advantages
• This approach verifies low-level data
processing early in the testing process

• Need for stubs is eliminated

20
 Bottom-up Integration Testing
Disadvantages
• Driver modules need to be built to test the
lower-level modules; this code is later
discarded or expanded into a full-featured
version

• Drivers inherently do not contain the complete

algorithms that will eventually use the services
of the lower-level modules; consequently,
testing may be incomplete or more testing may
be needed later when the upper level modules
are available

21
Stubs and drivers
⚫ Are designed for testing a program’s unit.
⚫ Stubs are for top-down testing,
⚫ It refers to the section of the code that performs the
imitation of the called function
⚫ Drivers for bottom-up testing
⚫ It refers to the section of the code that performs the
imitation of the calling function.

⚫ They are short-term alternatives to the modules

being tested and are yet to be developed.
 Stubs and drivers
Parameters Stubs Drivers
It refers to the section of the It refers to the section of the
code that performs the code that performs the
Basics
imitation of the called imitation of the calling
function. function.

These are developed during an These are developed during an

Development incremental integration incremental integration
testing’s top-down approach. testing’s bottom-up approach.

It is used for the significance It is used for the significance

Significance
of the low-level modules. of the high-level modules.
The drivers perform the
Stubs perform the replication
authorisation of the test cases
of the activities of missing or
Function to a different system. It refers
not developed modules or
to the modules that are under
various components.
testing
 Sandwich Integration Testing
⚫ Consists of a combination of both top-down and
bottom-up integration
⚫ Occurs both at the highest level modules and also at the
lowest level modules
⚫ Proceeds using functional groups of modules, with each
group completed before the next

⚫ High and low-level modules are grouped based on the

control and data processing they provide for a specific
program feature

⚫ Integration within the group progresses in alternating steps

between the high and low level modules of the group

24
 Sandwich Integration Testing
⚫ When integration for a certain functional
group is complete, integration and
testing moves onto the next group

⚫ Reaps the advantages of both types of integration

while minimizing the need for drivers and stubs

⚫ Requires a disciplined approach so that integration

doesn’t tend towards the “big bang” scenario

25
 A
Sandwich Integration Testing
B C D
Tes
tA
E F G
Test A,B,C,
Tes D
tE
Test
A, B, C,
Test B, E, F
Tes D,
tF E, F, G

Test D,G
Tes
tG
 Sandwich Integration Testing
⚫ Advantage: Top and Bottom Layer Tests can be done
in parallel

⚫ Disadvantage: Does not test the individual

subsystems and their interfaces thoroughly before
integration

⚫ Solution: Modified sandwich integration testing

strategy
 A
Modified Sandwich Testing
B C D
Tes
tA
Test A,C
Tes
E F G
tC
Test
B
Test
Test Test B, A, B, C,
E E, F D,
Test E, F, G
F
Tes
tD
Test D,G
Tes
tG
 Continuous Integration Testing
⚫ Continuous build:
⚫ Build from day one
⚫ Test from day one
⚫ Integrate from day one
⇒ System is always runnable

⚫ Requires integrated tool support:

⚫ Continuous build server
⚫ Automated tests with high coverage
⚫ Tool supported refactoring (restructure or rewrite source code to
improve internal consistency, readability, etc, without changing its
function)
⚫ Software configuration management
⚫ Issue tracking.
Continuous Integration Testing Strategy
A

Spread
SheetView Layer I

B C D

Data Currency Layer II

Calculator
Model Converter

E F G

BinaryFile XMLFile Currency Layer III

Storage Storage DataBase

+ Cells
Sheet View + File Storage
+ Addition
Non Incremental Integration Testing
⚫ Big Bang - combine (or integrate) all parts at
once.
⚫ Advantages: simple
⚫ Disadvantages:
⚫ hard to debugging,
⚫ not easy to isolate errors
⚫ not easy to validate test results
⚫ impossible to form an integrated system
 Integration Testing
Typical Steps
1. Based on the integration strategy, select a
component to be tested.
- Unit test all the classes in the component.
2. Put selected component together; do any
preliminary fix-up necessary to make the integration
test operational (drivers, stubs)
3. Test functional requirements: Define test cases that
exercise all uses cases with the selected component
 Integration Testing
4 . Test subsystem decomposition:
Define test cases that exercise all dependencies
5. Test non-functional requirements: Execute
performance tests

6. Keep records of the test cases and testing activities.

7. Repeat steps 1 to 7 until the full system is tested.

The primary goal of integration testing is to identify

failures with the (current) component configuration
 Black-box Testing
⚫ Complements white-box testing by uncovering
different classes of errors

⚫ Focuses on the functional requirements and the

information domain of the software

⚫ Used during the later stages of testing after white box

testing has been performed

⚫ The tester identifies a set of input conditions that will

fully exercise all functional requirements for a program

34
 Black-box Testing
⚫ The test cases satisfy the following:

⚫ Reduce, by a count greater than one, the number of

additional test cases that must be designed to achieve
reasonable testing

⚫ Tell us something about the presence or absence of

classes of errors, rather than an error associated only
with the specific task at hand

35
 Black-box Testing Categories
⚫ Incorrect or missing functions
⚫ Interface errors
⚫ Errors in data structures or external data base
access
⚫ Behavior or performance errors
⚫ Initialization and termination errors

36
 Black-box Testing
Questions answered by black-box testing
⚫ How is functional validity tested?
⚫ How are system behavior and performance
tested?
⚫ What classes of input will make good test
cases?
⚫ Is the system particularly sensitive to certain
input values?
⚫ How are the boundary values of a data class
isolated?
⚫ What data rates and data volume can the
system tolerate?
⚫ What effect will specific combinations of data
37
have on system operation?
 Equivalence Partitioning
⚫ A black-box testing method that divides the input domain of
a program into classes of data from which test cases are
derived
⚫ An ideal test case single-handedly uncovers a complete
class of errors, thereby reducing the total number of test
cases that must be developed
⚫ Test case design is based on an evaluation of equivalence
classes for an input condition
⚫ An equivalence class represents a set of valid or invalid
states for input conditions
⚫ From each equivalence class, test cases are selected so
that the largest number of attributes of an equivalence
class are exercised at once

38
 Guidelines for Defining Equivalence
Classes
⚫ If an input condition specifies a range, one valid and two invalid
equivalence classes are defined
⚫ Input range: 1 – 10 Eq classes: {1..10}, {x < 1}, {x > 10}
⚫ If an input condition requires a specific value, one valid and two invalid
equivalence classes are defined
⚫ Input value: 250 Eq classes: {250}, {x < 250}, {x > 250}
⚫ If an input condition specifies a member of a set, one valid and one invalid
equivalence class are defined
⚫ Input set: {-2.5, 7.3, 8.4} Eq classes: {-2.5, 7.3, 8.4}, {any other x}
⚫ If an input condition is a Boolean value, one valid and one invalid class are
define
⚫ Input: {true condition} Eq classes: {true condition}, {false condition}

39
 Boundary Value Analysis
⚫ A greater number of errors occur at the boundaries of the input
domain rather than in the "center"
⚫ Boundary value analysis is a test case design method that
complements equivalence partitioning
⚫ It selects test cases at the edges of a class
⚫ It derives test cases from both the input domain and output domain

40
 Guidelines for Boundary Value Analysis (class reading)
⚫ 1. If an input condition specifies a range bounded by values a
and b, test cases should be designed with values a and b as well
as values just above and just below a and b
⚫ 2. If an input condition specifies a number of values, test case
should be developed that exercise the minimum and maximum
numbers. Values just above and just below the minimum and
maximum are also tested
⚫ Apply guidelines 1 and 2 to output conditions; produce output that
reflects the minimum and the maximum values expected; also test
the values just below and just above
⚫ If internal program data structures have prescribed boundaries
(e.g., an array), design a test case to exercise the data structure
at its minimum and maximum boundaries

41
Black Box Testing
⚫ Black-box test case design techniques are the most
prevalent during integration, although a limited
amount of white-box testing may be used to ensure
coverage of major control paths.
⚫ After the software has been integrated (constructed),
a set of high-order tests are conducted.
⚫ Validation criteria (established during requirements
analysis) must be tested.
⚫ Validation testing provides final assurance that software
meets all functional, behavioral, and performance
requirements. Black-box testing techniques are used
exclusively during validation.
 System Integration Testing
System Testing is done after integration

• Objective
• To verify that the system components perform
control functions
• To perform inter-system test
• To demonstrate that the system performs both
functionally and operationally as specified
• To perform appropriate types of tests relating to
Transaction Flow, Installation, Reliability,
Regression etc.

• Done by the development team and users

System Integration Testing
⚫ Functional Testing
⚫ Validates functional requirements
⚫ Performance Testing
⚫ Validates non-functional requirements
⚫ Acceptance Testing
⚫ Validates clients expectations
Functional Testing
Goal: Test functionality of system
⚫ Test cases are designed from the requirements
analysis document (better: user manual) and
centered around requirements and key functions
(use cases)
⚫ The system is treated as black box
.
⚫ Unit test cases can be reused, but new test cases
have to be developed as well.
Performance Testing
Goal: Try to violate non-functional requirements
⚫ Test how the system behaves when overloaded.
⚫ Can bottlenecks be identified? (First candidates for
redesign in the next iteration)
⚫ Try unusual orders of execution
⚫ Call a receive() before send()
⚫ Check the system’s response to large volumes of
data
⚫ If the system is supposed to handle 1000 items, try
it with 1001 items.
⚫ What is the amount of time spent in different use
cases?
⚫ Are typical cases executed in a timely fashion?
 Types of Performance Testing
⚫ Security testing
⚫ Stress Testing
⚫ Stress limits of system ⚫ Try to violate security requirements

⚫ Volume testing ⚫ Environmental test

⚫ Test what happens if large amounts of ⚫ Test tolerances for heat, humidity,
data are handled motion
⚫ Configuration testing ⚫ Quality testing
⚫ Test the various software and hardware ⚫ Test reliability, maintain- ability &
configurations availability
⚫ Compatibility test ⚫ Recovery testing
⚫ Test backward compatibility with existing ⚫ Test system’s response to presence
systems of errors or loss of data
⚫ Timing testing ⚫ Human factors testing
⚫ Evaluate response times and time to ⚫ Test with end users.
perform a function
 Acceptance Testing
⚫ Goal: Demonstrate system is ready ⚫ Alpha test:
for operational use ⚫ Client uses the software at the
⚫ Choice of tests is made by client developer’s environment.
⚫ Many tests can be taken from ⚫ Software used in a controlled
integration testing setting, with the developer always
⚫ Acceptance test is performed by the ready to fix bugs.
client, not by the developer. ⚫ Beta test:
⚫ Conducted at client’s environment
(developer is not present)
⚫ Software gets a realistic workout
in target environ- ment
Integration Testing- Tips on Test Cases
⚫ Use analysis knowledge about functional
requirements (black-box testing):
⚫ Use cases
⚫ Expected input data
⚫ Invalid input data
⚫ Use design knowledge about system structure,
algorithms, data structures (white-box testing):
⚫ Control structures
⚫ Test branches, loops, ...
⚫ Data structures e.g. Test records fields,
arrays, …
Integration Testing- Tips on Test Cases
⚫ Use implementation knowledge about algorithms
and data structures:
⚫ Force a division by zero
⚫ If the upper bound of an array is 10,
then use 11 as index.
 Recap
⚫ Integration testing
⚫ Integration
⚫ Testing
⚫ Why you need?
⚫ What you need for integration testing
⚫ Strategies and approaches to integration testing
⚫ Three types of integration testing?
⚫ Functional
⚫ Performance
⚫ Acceptance
 Software Documentation
⚫ Software documentation is written text or illustration
that accompanies computer software. It either
explains how it operates or how to use it, and may
mean different things to people in different roles.

⚫ Documentation is an important part of software

engineering.
(wikipedia)
 Software Documentation
⚫ External documentation is provided through
various types of supporting documents such as
⚫ users’ manual,
⚫ software requirements specification document,
⚫ design document,
⚫ test documents, etc.

⚫ A systematic software development style ensures

that all these documents are produced in an
orderly fashion.
Documentation Requirements
⚫ General requirements of all software
documentation
⚫ Should provide for communication among team
members

⚫ Should act as an information repository to be used

by maintenance engineers

⚫ Should provide enough information to management

to allow them to perform all program management
related activities

⚫ Should describe to users how to operate and

administer the system
Documentation Requirements
⚫ In all software projects some amount of
documentation should be created prior to any
code being written
⚫ Design docs, etc.
⚫ Documentation should continue after the code
has been completed
⚫ User’s manuals, etc.

⚫ The two main types of documentation created

are Process and Product documents
Process Documentation
⚫ Used to record and track the development
process
⚫ Planning documentation
⚫ Cost, Schedule, Funding tracking
⚫ Schedules
⚫ Standards
⚫ Etc.
⚫ This documentation is created to allow for
successful management of a software product
Process Documentation
⚫ Has a relatively short lifespan

⚫ Only important to internal development process

⚫ Except in cases where the customer requires a view
into this data

⚫ Some items, such as papers that describe design

decisions should be extracted and moved into the
product documentation category when they
become implemented
Product Documentation
⚫ Describes the delivered product

⚫ Must evolve with the development of the software

product

⚫ Two main categories:

⚫ System Documentation

⚫ User Documentation
Product Documentation
⚫ System Documentation
⚫ Describes how the system works, but not how to
operate it
⚫ Examples:
⚫ Requirements Specification
⚫ Architectural Design
⚫ Detailed Design
⚫ Commented Source Code
⚫ Including output such as JavaDoc
⚫ Test Plans
⚫ Including test cases
⚫ Verification & Validation plan and results
⚫ List of Known Bugs
Product Documentation
⚫ User Documentation has two main types
⚫ End User
⚫ System Administrator

⚫ In some cases these are the same people

⚫ The target audience must be well understood!
Product Documentation
⚫ There are five important areas that should be
documented for a formal release of a software
application
⚫ These do not necessarily each have to have their own
document, but the topics should be covered
thoroughly
1. Functional Description of the Software
2. Installation Instructions
3. Introductory Manual
4. Reference Manual
5. System Administrator’s Guide
Document Quality
⚫ Providing thorough and professional
documentation is important for any size product
development team

⚫ The problem is that many software professionals

lack the writing skills to create professional level
documents
Document Structure
⚫ All documents for a given product should have a
similar structure
⚫ A good reason for product standards
⚫ The IEEE Standard for User Documentation lists
such a structure
⚫ It is a superset of what most documents need
⚫ The authors “best practices” are:
1. Put a cover page on all documents
2. Divide documents into chapters with sections and
subsections
3. Add an index if there is lots of reference information
4. Add a glossary to define ambiguous terms
Standards
⚫ Standards play an important role in the
development, maintenance and usefulness of
documentation
⚫ Standards can act as a basis for quality
documentation
⚫ But are not good enough on their own
⚫ Usually define high level content and organization

⚫ There are three types of documentation standards

->
1.Process Standards
⚫ Define the approach that is to be used when
creating the documentation
⚫ Don’t actually define any of the content of the
documents

Draft

Peer Reviews
Revise Check
2. Product Standards
⚫ Goal is to have all documents created for a
specific product attain a consistent structure
and appearance
⚫ Can be based on organizational or contractually
required standards

⚫ Four main types:

1. Documentation Identification Standards
2. Document Structure Standards
3. Document Presentation Standards
4. Document Update Standards
2. Product Standards
⚫ One caveat:

⚫ Documentation that will be viewed by end users

should be created in a way that is best
consumed and is most attractive to them

⚫ Internal development documentation generally

does not meet this need
 3. Interchange Standards
⚫ Deals with the creation of documents in a format
that allows others to effectively use
⚫ PDF may be good for end users who don’t need to edit
⚫ Word may be good for text editing
⚫ Specialized CASE tools need to be considered

⚫ This is usually not a problem within a single

organization, but when sharing data between
organizations it can occur
⚫ This same problem is faced all the time during software
integration
Other Standards
⚫ IEEE
⚫ Has a published standard for user documentation
⚫ Provides a structure and superset of content areas
⚫ Many organizations probably won’t create
documents that completely match the standard
⚫ Writing Style
⚫ Ten “best practices” when writing are provided
⚫ Author proposes that group edits of important
documents should occur in a similar fashion to
software walkthroughs
Online Documentation
⚫ Either internal to the application or Web based
⚫ Requires rethinking the presentation style since
normal “paper” documentation approaches do not
carry over well
⚫ Should act as a supplement to paper
documentation
⚫ Biggest benefit of Web docs are that they are
always current
 Document Preparation
⚫ Covers the entire process of creating and formatting
a document for publication
⚫ Use specialized (and separate) tools for creating and
preparing documents
⚫ This is only important for user documentation

⚫ It is often important to have a professional writer or

document publisher evaluate documents before
publication to ensure they look good and will carry
over to paper well
Document Storage
⚫ Use
⚫ File System to store the actual documents
⚫ Database to store references to the files with
metadata to allow searching and referencing

⚫ Today it is probably better to use a content

management systems
⚫ CVS or Subversion
⚫ Free and Open Source
⚫ Easy to setup and maintain
Conclusion
⚫ Good overview of documentation
⚫ Though most documentation “requirements” are
based on contract requirements
⚫ Hard to cover things like that in a paper like this

⚫ Most of the content seemed to be referring to user

documentation
⚫ Design and other similar docs (often times more
graphical than textual) were overlooked