FM-AA-CIA-15 Rev.

0 10-July-2020

Study Guide in CC 106 – Application Development and Emerging Technologies Module No. 6

STUDY GUIDE FOR MODULE NO. 6

CHAPTER 6: SOFTWARE TESTING

MODULE OVERVIEW

Software testing is an important part of software development. If software testing is not performed properly,
applications can have errors which may lead to rework, costly failure or worse, loss of life. This module will give
you deeper knowledge about software verification and validation including the stages of software testing,
different techniques and types of testing as well as its importance as part of the whole process in software
development.

MODULE LEARNING OBJECTIVES

At the end of this chapter, the student should be able to:

1. Understand the stages of testing from testing during development to acceptance testing by system
customers;
2. have been introduced to techniques that help you choose test cases that are geared to discovering program
defects;
3. Know about three distinct types of testing—component testing, system testing, and release testing;
4. Understand the distinctions between development testing, release testing, and user testing.

LEARNING CONTENTS (TOPIC 1: SOFTWARE TESTING)

SOFTWARE TESTING

Testing is intended to show that a program does what it is intended to do and to discover program defects
before is put into use. When you test software, you execute a program using artificial data. You check the results
of the test run for errors, anomalies, or information about the program’s non-functional attributes.

When you test software, you are trying to do two (2) things / types of testing:

1. Demonstrate to the developer and the customer that the software meets its requirements, or known as
the validation testing where you expect the system to perform correctly using a set of test cases that
reflect the system’s expected use.

2. Find inputs or inputs sequences where the behaviour of the software is incorrect, undesirable, or does not
conform to its specification, or known as defect testing, where the tests cases are designed to expose
defects such as system crashes, unwanted interactions with other systems, incorrect computations, and
data corruption.

Testing cannot demonstrate that the software is free of defects or that it will behave as specified in every
circumstance. It is always possible that a test you have overlooked could discover further problems with the
system. As Edsger Dijkstra eloquently stated (Dijkstra 1972):

“Testing can only show the presence of errors, not their absence.”

PANGASINAN STATE UNIVERSITY Prepared by: REYJOHN R. FRIAS, MIT 1

 FM-AA-CIA-15 Rev. 0 10-July-2020

Study Guide in CC 106 – Application Development and Emerging Technologies Module No. 6

VERIFICATION vs. VALIDATION

Testing is part of a broader process of software verification and validation (V&V). Verification and validation
processes are concerned with checking that software being developed meets its specification and delivers the
functionality expected by the people paying for the software. These checking processes start as soon as
requirements become available and continue through all stages of the development process.

Verification and validation are not the same thing, although they are often confused. Barry Boehm succinctly
expressed the difference between them (Boehm 1979):

VERIFICATION: "Are we building the product right?‖

- The software should conform to its specification.

VALIDATION: "Are we building the right product?‖

- The software should do what the user really requires.

Figure 6.1 shows an illustration of how validation and verification differ to each other.

Figure 6.1 Software Verification and Validation

- Software verification is the process of checking that the software meets its stated functional and
non-functional requirements.

- Validation is a more general process. The aim of software validation is to ensure that the software meets the
customer’s expectations.

TWO (2) PRINCIPAL OBJECTIVES OF V&V

1. The discovery of defects in a system;

2. The assessment of whether or not the system is useful and useable in an operational situation.

V&V GOAL

The goal of verification and validation processes is to establish CONFIDENCE that the software system is ―fit
for purpose.‖ This means that the system must be good enough for its intended use.

V&V CONFIDENCE LEVEL

The level of required confidence depends on the system’s purpose, user expectations and the current
marketing environment of the system:

1. Software purpose

PANGASINAN STATE UNIVERSITY Prepared by: REYJOHN R. FRIAS, MIT 2

 FM-AA-CIA-15 Rev. 0 10-July-2020

Study Guide in CC 106 – Application Development and Emerging Technologies Module No. 6

- The level of confidence depends on how critical the software is to an organisation. The more critical
the software, the more important it is that is reliable.
Example: The level of confidence required for a software used to control a safety-critical system is
much higher than that required for a demonstrator system that prototypes new product ideas.

2. User expectations
- Users may have low expectations of software quality because of their previous experiences with
buggy and unreliable software.

3. Marketing environment
- Getting a product to market early may be more important than finding defects in the program.

STATIC vs. DYNAMIC VERIFICATION

SOFTWARE INSPECTIONS (Static Verification)

- Concerned with analysis of the static system representation to discover problems.
- Inspections analyse and check the system requirements, design models, the program source code,
and even the proposed system tests. This is static V&V techniques in which YOU DON’T NEED TO
EXECUTE THE SOFTWARE to verify it.

SOFTWARE TESTING (Dynamic Verification)

- Concerned with exercising and observing product behaviour.
- The SYSTEM IS EXECUTED WITH TEST DATA and its operational behaviour is observed.

Figure 6.2 Software Inspections and Testing

The figure above shows that software inspection and testing support V&V at different stages in the software
process. The arrows indicate the stages in the process where the techniques may be used.

TESTING STAGES

Typically, a commercial software system has to go through three stages of testing:

1. Development Testing - where the system is tested during development to discover bugs and defects.
System designers and programmers are likely to be involved in the testing process.

There are three (3) stages of development testing:

1.1 Unit testing, where individual program units or object classes are tested. Unit testing should focus on
testing the functionality of objects or methods.

PANGASINAN STATE UNIVERSITY Prepared by: REYJOHN R. FRIAS, MIT 3

 FM-AA-CIA-15 Rev. 0 10-July-2020

Study Guide in CC 106 – Application Development and Emerging Technologies Module No. 6

1.2 Component testing, where several individual units are integrated to create composite components.
Component testing should focus on testing the component interfaces that provide access to the
component functions.

1.3 System testing, where some or all of the components in a system are integrated and the system is
tested as a whole. System testing should focus on testing component interactions.

Development testing is primarily a defect testing process, where the aim of testing is to discover bugs in the
software. It is therefore usually interleaved with debugging—the process of locating problems with the code and
changing the program to fix these problems.

2. Release Testing – where a separate testing team tests a complete version of the system before it is released
to users. Release testing, therefore, has to show that the system delivers its specified functionality,
performance, and dependability, and that it does not fail during normal use.

3. User Testing – where users or potential users of a system test the system in their own environment.
Acceptance testing is one type of user testing where the customer formally tests a system to decide is it should
be accepted from the system supplier or if further development is required.

There are three different types of user testing:

3.1. Alpha testing, where a selected group of software users work closely with the development team to
test early releases of the software.

3.2. Beta testing, where a release of the software is made available to a larger group of users to allow
them to experiment and to raise problems that they discover with the system developers.

3.3. Acceptance testing, where customers test a system to decide whether or not it is ready to be
accepted from the system developers and deployed in the customer environment.

TYPES OF SOFTWARE TESTING

We have various types of testing available in the market, which are used to test the application or the software.
With the help of below image, we can easily understand the type of software testing:

Figure 6.3 Types of Software Testing

PANGASINAN STATE UNIVERSITY Prepared by: REYJOHN R. FRIAS, MIT 4

 FM-AA-CIA-15 Rev. 0 10-July-2020

Study Guide in CC 106 – Application Development and Emerging Technologies Module No. 6

1. MANUAL TESTING is a software testing process in which test cases are executed manually without using
any automated tool. All test cases executed by the tester manually according to the end user's perspective. It
ensures whether the application is working, as mentioned in the requirement document or not. Test cases are
planned and implemented to complete almost 100 percent of the software application. Test case reports are
also generated manually.

- It is one of the most fundamental testing processes as it can find both visible and hidden defects of the
software.

- Manual testing is mandatory for every newly developed software before automated testing. This testing
requires great efforts and time, but it gives the surety of bug-free software.

TYPES OF MANUAL TESTING

Here are various methods used for manual testing. Each technique is used according to its testing criteria.
Types of manual testing are given below:

A. White Box Testing

B. Black Box Testing
C. Gray Box Testing

White-box
Testing Black-box
Testing
Grey-box
Testing

Figure 6.4 Types of Manual Testing

A. White-box Testing

The term 'white box' is used because of the internal perspective of the system. The clear box or white box or
transparent box name denote the ability to see through the software's outer shell into its inner workings. It tests
internal coding and infrastructure of a software focus on checking of predefined inputs against expected and
desired outputs. It is based on inner workings of an application and revolves around internal structure testing. In
this type of testing programming skills are required to design test cases.

Developers do white box testing. In this, the developer will test every line of the code of the program. The
developers perform the White-box testing and then send the application or the software to the testing team,
where they will perform the black box testing and verify the application along with the requirements and identify
the bugs and sends it to the developer.

The developer fixes the bugs and does one round of white box testing and sends it to the testing team. Here,
fixing the bugs implies that the bug is deleted, and the particular feature is working fine on the application.

Here, the test engineers will not include in fixing the defects for the following reasons:

PANGASINAN STATE UNIVERSITY Prepared by: REYJOHN R. FRIAS, MIT 5

 FM-AA-CIA-15 Rev. 0 10-July-2020

Study Guide in CC 106 – Application Development and Emerging Technologies Module No. 6

o Fixing the bug might interrupt the other features. Therefore, the test engineer should always find the
bugs, and developers should still be doing the bug fixes.
o If the test engineers spend most of the time fixing the defects, then they may be unable to find the other
bugs in the application.

B. Black-box Testing

The black-box testing is done by the Test Engineer, where they can check the functionality of an application or
the software according to the customer /client's needs. In this, the code is not visible while performing the
testing; that's why it is known as black-box testing.

Figure 6.5 Black-box Testing

In this method, tester selects a function and gives input value to examine its functionality, and checks whether
the function is giving expected output or not. If the function produces correct output, then it is passed in testing,
otherwise failed. The test team reports the result to the development team and then tests the next function. After
completing testing of all functions if there are severe problems, then it is given back to the development team for
correction.

C. Grey-Box Testing

Grey-box testing is a combination of white-box and black-box testing. It can be performed by a person who knew
both coding and testing. And if the single person performs white box, as well as black-box testing for the
application, is known as Grey-box testing.

Figure 6.3 Grey-box Testing

Grey-box testing commonly identifies context-specific errors that belong to web systems. For example; while
testing, if tester encounters any defect then he makes changes in code to resolve the defect and then test it
again in real time. It concentrates on all the layers of any complex software system to increase testing coverage.
It gives the ability to test both presentation layer as well as internal coding structure.

PANGASINAN STATE UNIVERSITY Prepared by: REYJOHN R. FRIAS, MIT 6

 FM-AA-CIA-15 Rev. 0 10-July-2020

Study Guide in CC 106 – Application Development and Emerging Technologies Module No. 6

2. AUTOMATION TESTING

- When the testing case suites are performed by using automated testing tools is known as Automation Testing.
The testing process is done by using special automation tools to control the execution of test cases and
compare the actual result with the expected result. Automation testing requires a pretty huge investment of
resources and money.

- Generally, repetitive actions are tested in automated testing such as regression tests. The testing tools used in
automation testing are used not only for regression testing but also for automated GUI interaction, data set up
generation, defect logging, and product installation.

- The goal of automation testing is to reduce manual test cases but not to eliminate any of them. Test suits can
be recorded by using the automation tools, and tester can play these suits again as per the requirement.
Automated testing suites do not require any human intervention.

6.6 Manual and Automation Testing

LEARNING ACTIVITY 6.1

Activity 6.1 (20 pts. Essay)

1. Give at least 5 reasons why Software Testing is important.

2. Some people argue that developers should not be involved in testing their own code but that all testing should
be the responsibility of a separate team. Give arguments for and against testing by the developers themselves.

SUMMARY / KEY POINTS

■ Testing can only show the presence of errors in a program. It cannot show that there are no remaining faults.
■ Development testing is the responsibility of the software development team. A separate team should be
responsible for testing a system before it is released to customers. In the user testing process, customers or
system users provide test data and check that tests are successful.
■ Development testing includes unit testing in which you test individual objects and methods; component testing
in which you test related groups of objects; and system testing in which you test partial or complete systems.
■ When testing software, you should try to ―break‖ the software by using experience and guidelines to choose

PANGASINAN STATE UNIVERSITY Prepared by: REYJOHN R. FRIAS, MIT 7

 FM-AA-CIA-15 Rev. 0 10-July-2020

Study Guide in CC 106 – Application Development and Emerging Technologies Module No. 6

types of test cases that have been effective in discovering defects in other systems.
■ Wherever possible, you should write automated tests. The tests are embedded in a program that can be run
every time a change is made to a system.
■ Acceptance testing is a user testing process in which the aim is to decide if the software is good enough to be
deployed and used in its planned operational environment.

REFERENCES

 Sommerville, Ian. Software Engineering, 10th Edition. 2016

 Dijkstra, E. W. 1972. “The Humble Programmer.” Comm. ACM 15 (10): 859-66.
doi:10.1145//355604.361591
 https://www.javatpoint.com

PANGASINAN STATE UNIVERSITY Prepared by: REYJOHN R. FRIAS, MIT 8