1

KIET GROUP OF INSTITUTIONS

DEPARTMENT OF COMPUTER APPLICATIONS
SESSION: 2016-17, V SEMESTER
SOFTWARE TESTING, NMCA E43
UNIT 1

WHAT IS SOFTWARE TESTING

Software testing is a process of executing a program or application with the intent of finding
the software bugs.
It can also be stated as the process of validating and verifying that a software program or
application or product:

Meets the usi ess a d te h i al e ui e e ts that guided it s desig a d de elop e t

Works as expected
Can be implemented with the same characteristic.

WHY SOFTWARE TESTING

Software testing is very important because of the following reasons:
1. Software testing is really required to point out the defects and errors that were made
during the development phases.
2. It s esse tial si e it akes su e of the Custo e s elia ilit a d thei satisfa tio i the
application.
3. It is very important to ensure the Quality of the product. Quality product delivered to
the customers helps in gaining their confidence.
4. Testing is necessary in order to provide the facilities to the customers like the delivery of
high quality product or software application which requires lower maintenance cost and
hence results into more accurate, consistent and reliable results.
5. Testing is required for an effective performance of software application or product.
6. It s i po ta t to e su e that the appli atio should ot esult i to a failu es e ause
it can be very expensive in the future or in the later stages of the development.
7. It s e ui ed to sta i the usi ess.

Prepared By: Neelam Rawat

Neelam Rawat

SOFTWARE TESTING OBJECTIVES AND PURPOSE

Software Testing has different goals and objectives. The major objectives of Software testing
are as follows:

Finding defects which may get created by the programmer while developing the
software.
Gaining confidence in and providing information about the level of quality.
To prevent defects.
To make sure that the end result meets the business and user requirements.
To ensure that it satisfies the BRS that is Business Requirement Specification and SRS
that is System Requirement Specifications.
To gain the confidence of the customers by providing them a quality product.

Software testing helps in finalizing the software application or product against business and
user requirements. It is very important to have good test coverage in order to test the software
appli atio o pletel a d ake it su e that it s pe fo i g ell a d as pe the spe ifi atio s.
While determining the test coverage the test cases should be designed well with maximum
possibilities of finding the errors or bugs. The test cases should be very effective. This objective
can be measured by the number of defects reported per test cases. Higher the number of the
defects reported the more effective are the test cases.
Once the delivery is made to the end users or the customers they should be able to operate it
without any complaints. In order to make this happen the tester should know as how the
customers are going to use this product and accordingly they should write down the test
scenarios and desig the test ases. This ill help a lot i fulfilli g all the usto e s
requirements.
Software testing makes sure that the testing is being done properly and hence the system is
ready for use. Good coverage means that the testing has been done to cover the various areas
like functionality of the application, compatibility of the application with the OS, hardware and
different types of browsers, performance testing to test the performance of the application and
load testing to make sure that the system is reliable and should not crash or there should not
be any blocking issues. It also determines that the application can be deployed easily to the
machine and without any resistance. Hence the application is easy to install, learn and use.

Prepared By: Neelam Rawat

Neelam Rawat

FAULTS, ERRORS AND FAILURES

A defect is an error or a bug, in the application which is created. A programmer while designing
and building the software can make mistakes or error. These mistakes or errors mean that
there are flaws in the software. These are called defects.
When actual result deviates from the expected result while testing a software application or
product then it results into a defect. Hence, any deviation from the specification mentioned in
the product functional specification document is a defect. In diffe e t o ga izatio s it s alled
differently like bug, issue, incidents or problem.
When the result of the software application or product does not meet with the end user
expectations or the software requirements then it results into a Bug or Defect. These defects or
bugs occur because of an error in logic or in coding which results into the failure or unpredicted
or unanticipated results.
Additional Information about Defects / Bugs:
While testing software application or product if large number of defects is fou d the it s alled
Buggy.
Whe a teste fi ds a ug o defe t it s e ui ed to o e the sa e to the de elopers. Thus
they report bugs with the detail steps and are called as Bug Reports, issue report, problem
report, etc.
FAILURE
If under certain environment and situation defects in the application or product get executed
then the system will produce the wrong results causing a failure. Not all defects result in
failures, some may stay inactive in the code and we may never notice them. Example: Defects
in dead code will never result in failures. It is not just defects that give rise to failure. Failures
can also be caused because of the other reasons also like:

Because of the environmental conditions as well like a radiation burst, a strong

magnetic field, electronic field or pollution could cause faults in hardware or firmware.
Those faults might prevent or change the execution of software.
Failures may also arise because of human error in interacting with the software, perhaps
a wrong input value being entered or an output being misinterpreted.
Finally failures may also be caused by someone deliberately trying to cause a failure in
the system.

Prepared By: Neelam Rawat

Neelam Rawat

DIFFERENCE BETWEEN ERROR, FAULT, BUG, FAILURE AND DEFECT

What is an error?

Error is deviation from actual and expected value.

It represents mistake made by people.

What is a fault?

Fault is incorrect step, process or data definition in a computer program which causes
the program to behave in an unintended or unanticipated manner.
It is the result of the error.

What is a bug?

Bug is a fault in the program which causes the program to behave in an unintended or
unanticipated manner.
It is an evidence of fault in the program.

What is a failure?

Failure is the inability of a system or a component to perform its required functions

within specified performance requirements.
Failure occurs when fault executes.

What is a defect?

A defect is an error in coding or logic that causes a program to malfunction or to

produce incorrect/unexpected results.
A defect is said to be detected when a failure is observed.

Prepared By: Neelam Rawat

Neelam Rawat

FEW POINTS THAT IS IMPORTANT TO KNOW:

When tester is executing a test he/she may observe some difference in the behavior of the
feature or functionality, but this not because of the failure. This may happen because of the
wrong test data entered, tester may not be aware of the feature or functionality or because of
the bad environment. Because of these reasons incidents are reported. They are known as
incident report. The condition or situation which requires further analysis or clarification is
known as incident. To deal with the incidents the programmer need to to the analysis that
whether this incident has occurred because of the failure or not.
It s ot e essa that defe ts o ugs i t odu ed i the p odu t a e o l
the soft a e. To
u de sta d it fu the let s takes an example. A bug or defect can also be introduced by a
business analyst. Defects present in the specifications like requirements specification and
design specifications can be detected during the reviews. When the defect or bug is caught
during the review cannot result into failure because the software has not yet been executed.
These defects or bugs are reported not to blame the developers or any people but to judge the
quality of the product. The quality of product is of utmost importance. To gain the confidence
of the usto e s it s e i po ta t to deli e the ualit p odu t o ti e.
So following is a C program: Add two numbers program
Example 1:
As mentioned this program is required to add two numbers.
1
#include<stdio.h>
2
3
int main ()
4
{
5
int value1, value2, ans;
6
7
value1 = 5;
8
value2 = 3;
9
10 ans = value1 - value2;
11
12 printf("The addition of 5 + 3 = %d.", ans);
13
14 return 0;
15 }
When you compile and run this program you see the printed statement as below:

Prepared By: Neelam Rawat

Neelam Rawat

The addition of 5 + 3 = 2.
So after compiling and running this program we realize the program has failed to do what it was
supposed to do.
The program was supposed to add two numbers but it certainly did not add 5 and 3. 5 + 3
should be 8, but the result is 2. There could be various reasons as to why the program displays
the answer 2 instead of 8. For now we have detected a failure.
As the failure has been detected a defect can be raised.
No let s go a k to the p og a
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

a d a al ze hat as the fault i the p og a .

#include<stdio.h>
int main ()
{
int value1, value2, ans;
value1 = 5;
value2 = 3;
ans = value1 - value2; // ----> Bug
printf("The addition of 5 + 3 = %d.", ans);
return 0;
}

We oti e at li e u e
, the e is a - sig p ese t i stead of + sig . o the fault i the
p og a is the - sig . Li e
has the fault hi h aused the p og a to de iate f o the
functionality.
E o is the istake I ade
t pi g - i stead of + sig . We ha e o se ed failu e i
execution of the program. And in this case we can also say we have found the bug.

o e t

A tester does not necessarily have access to the code and may be just testing the functionality
of the program. In that case the tester will realize the output is faulty and will raise a defect.
Due to the observed wrong result it is known of the fact that the program has an error which
resulted in the fault in the program and due to which the program failed to give the correct
result. But the tester may not know exactly what is causing the error.
IEEE Definitions

Prepared By: Neelam Rawat

Neelam Rawat

1. Failure: External behavior is incorrect

2. Fault: Discrepancy in code that causes a failure.
3. Error: Human mistake that caused fault
Note:

Error is terminology of Developer.

Bug is terminology of Tester

BASICS OF SOFTWARE TESTING

Basics
Software
Quality
Dimensions
of Quality
Software
Quality
Assurance
Software
Quality
Control
SQA and SQC
Differences

Summary
Learn how software quality is defined and what it means. Software quality is the
degree of conformance to explicit or implicit requirements and expectations.
Learn the dimensions of quality. Software quality has dimensions such as
A essi ilit , Co pati ilit , Co u e , Effi ie

Learn what it means and what its relationship is with Software Quality Control.
Software Quality Assurance is a set of activities for ensuring quality in software
engineering processes.
Learn what it means and what its relationship is with Software Quality
Assurance. Software Quality Control is a set of activities for ensuring quality in
software products.
Learn the differences between Software Quality Assurance and Software
Quality Control. SQA is process-focused and prevention-oriented but SQC is
product-focused and detection-oriented.
Software
Learn what SDLC means and what activities a typical SDLC model comprises of.
Development Software Development Life Cycle defines the steps/stages/phases in the
Life Cycle
building of software.
Software
Learn what STLC means and what activities a typical STLC model comprises of.
Testing Life Software Testing Life Cycle (STLC) defines the steps/ stages/ phases in testing of
Cycle
software.
Definition of Lea the a ious defi itio s of the te
test . Me ia We ste defines Test
Test
as a iti al e a i atio , o se atio , o e aluatio .
Software
Testing
Myths

Just as every field has its myths, so does the field of Software Testing. We
explain some of the myths along with their related facts.

Prepared By: Neelam Rawat

Neelam Rawat

PRINCIPLES OF TESTING
There are seven principles of testing. They are as follows:
1) Testing shows presence of defects: Testing can show the defects are present, but cannot
prove that there are no defects. Even after testing the application or product thoroughly we
cannot say that the product is 100% defect free. Testing always reduces the number of
undiscovered defects remaining in the software but even if no defects are found, it is not a
proof of correctness.
2) Exhaustive testing is impossible: Testing everything including all combinations of inputs and
preconditions is not possible. So, instead of doing the exhaustive testing we can use risks and
priorities to focus testing efforts. For example: In an application in one screen there are 15
input fields, each having 5 possible values, then to test all the valid combinations you would
need 30 517 578 125 (515) tests. This is very unlikely that the project timescales would allow
for this number of tests. So, accessing and managing risk is one of the most important activities
and reason for testing in any project.
3) Early testing: In the software development life cycle testing activities should start as early as
possible and should be focused on defined objectives.
4) Defect clustering: A small number of modules contains most of the defects discovered during
pre-release testing or shows the most operational failures.
5) Pesticide paradox: If the same kinds of tests are repeated again and again, eventually the
same set of test cases will no longer be able to find any new bugs. To o e o e this Pesti ide
Pa ado , it is eall e i po ta t to e ie the test ases egula l a d e a d diffe e t
tests need to be written to exercise different parts of the software or system to potentially find
more defects.
6) Testing is context depending: Testing is basically context dependent. Different kinds of sites
are tested differently. For example, safety critical software is tested differently from an ecommerce site.
7) Absence of errors fallacy: If the system built is unusable a d does ot fulfill the use s
needs and expectations then finding and fixing defects does not help.
STLC: SOFTWARE TESTING LIFE CYCLE
Software Testing Life Cycle (STLC) defines the steps/ stages/ phases in testing of software.
However, there is no fixed standard STLC in the world and it basically varies as per the
following:

Prepared By: Neelam Rawat

Neelam Rawat

Software Development Life Cycle

Whims of the Management

Nevertheless, Software Testing Life Cycle, in general, comprises of the following phases:

Phase

Activity

Deliverables

Requirements/
Design Review

You review the software e ie

requirements/ design (Well, Reports
if they exist.)

Test Planning

Once you have gathered a Test Plan

general idea of what needs
to be tested, you pla fo
the tests.
Test Estimation

Necessity
Defe t

Curiosity

Farsightedness

Test Schedule
Test Designing

You design/ detail your tests Test

Cases / Test Creativity
on the basis of detailed Scripts /Test Data
requirements/design of the
software (sometimes, on the
basis of your imagination).
Requirements
Traceability Matrix

Prepared By: Neelam Rawat

Neelam Rawat

10

10

Test
Environment
Setup

Test Execution

Test Reporting

You
setup
the
test
environment (server/ client/
network, etc) with the goal
of replicating the end-use s
environment.
You execute your Test
Cases/ Scripts in the Test
Environment to see whether
they pass.

Test Environment

Test
(Incremental)

Rich company

Results Patience

Defect Reports

You prepare various reports Test Results (Final)

for various stakeholders.
Test/ Defect Metrics

Diplomacy

Test Closure Report

Who Worked Late & on
Weekends
(WWLW)
Report
[Depending
on how
fussy your
Management is]

REQUIREMENTS, BEHAVIOR & CORRECTNESS

Requirements & Behavior

Prepared By: Neelam Rawat

Neelam Rawat

11

11

Functional vs. Non-functional Testing

Correctness
Software Correctness: A program P is considered with respect to a specification S, if and only if,

For each valid input, the output of P is in accordance with the specification S

Correctness is:

Software should match its specifications

Software should meet its functional requirements

Note: If the software behaves incorrectly, it might get validated and verified. We cannot
guarantee correctness in the software development and testing process
Correctness from software engineering perspective can be defined as the adherence to the
specifications that determine how users can interact with the software and how the software
should behave when it is used correctly.
If the software behaves incorrectly, it might take considerable amount of time to achieve the
task or sometimes it is impossible to achieve it.
Below are some of the important rules for effective programming which are consequences of
the program correctness theory.

Prepared By: Neelam Rawat

Neelam Rawat

12

12

Defining the problem completely.

Develop the algorithm and then the program logic.
Reuse the proved models as much as possible.
Prove the correctness of algorithms during the design phase.
Developers should pay attention to the clarity and simplicity of your program.
Verifying each part of a program as soon as it is developed.

TESTING & DEBUGGING

TESTING
Testing activity is carried down by a team of testers, in order to find the defect in the software.
Test engineers run their tests on the piece of software and if they encounter any defect (i.e.
actual results don't match expected results), they report it to the development team. Along
with the nature of defect, testers also have to report at what point the defect occurred and
what happened due the occurrence of that defect. All this information will be used by
development team to DEBUG the defect.
DEBUGGING
Debugging is the activity which is carried out by the development team (or developer), after
getting the test report from the testing team about defect(s) (you may note defects can also be
reports by the client). The developer then tries to find the cause of the defect, in this quest he
may need to go through lines of code and find which part of code in causing that defect. After
finding out the bug, he tries to modify that portion of code and then he rechecks if the defect
has been finally removed. After fixing the bug, developers send the software back to testers.
TESTING VS. DEBUGGING
TESTING
1. Testing always starts with known
conditions, uses predefined methods,
and has predictable outcomes too.
2. Testing can and should definitely be
planned, designed, and scheduled.
3. It proves a programmers failure.
4. It is a demonstration of error or
apparent correctness.
5. Testing as executed should strive to
be predictable, dull, constrained, rigid,
and inhuman.

DEBUGGING
1. Debugging starts from possibly unknown initial conditions and its end
cannot be predicted, apart from
statistically.
2. The procedures for, and period of,
debugging cannot be so constrained.
3. It is the p og a
e s i di atio .
4. It is always treated as a deductive
process.
5. Debugging demands intuitive leaps,
conjectures, experimentation, and some
freedom also.

Prepared By: Neelam Rawat

Neelam Rawat

13

13

6. Much of the testing can be done

without design knowledge.
7. It can often be done by an outsider.
8. Much of test execution and design
can be automated.
9. Testing purpose is to find bug.

6. Debugging is impossible without

detailed design knowledge.
7. It must be done by an insider.
8. Automated debugging is still a dream
for programmers.
9. Debugging purpose is to find cause of
bug.

TEST METRICS AND MEASUREMENT

We a t o t ol thi gs hi h e a t

easu e .

SOFTWARE METRICS

A Metric is a quantitative measure of the degree to which a system, system component,

or process possesses a given attribute.
Met i s a e defi ed as TANDAD OF MEAUEMENT .
Software Metrics are used to measure the quality of the project. Simply, Metric is a unit
used for describing an attribute. Metric is a scale for measurement.

SOFTWARE TEST MEASUREMENT

Measurement is the quantitative indication of extent, amount, dimension, capacity, or

size of some attribute of a product or process.
Test measurement example: Total number of defects.

WHY TEST METRICS?

Generation of Software Test Metrics is the most important responsibility of the Software Test
Lead/Manager.
Test Metrics are used to,

Take the decision for next phase of activities such as, estimate the cost & schedule of
future projects.
Understand the kind of improvement required to success the project
Take decision on process or technology to be modified etc.

IMPORTANCE OF SOFTWARE TESTING METRICS:

Test Metrics are the most important to measure the quality of the software.
Now, how can we measure the quality of the software by using Metrics?

Prepared By: Neelam Rawat

Neelam Rawat

14

14

Suppose, if a project does not have any metrics, then how the quality of the work done by a
Test analyst will be measured?
For Example: A Test Analyst has to,

Design the test cases for 5 requirements

Execute the designed test cases
Log the defects & need to fail the related test cases
After the defect is resolved, need to re-test the defect & re-execute the corresponding
failed test case.

In above scenario, if metrics are not followed, then the work completed by the test analyst will
be subjective i.e. the test report will not have the proper information to know the status of his
work/project.
If Metrics are involved in the project, then the exact status of his/her work with proper
numbers/data can be published, i.e., in the Test report, we can publish:
1.
2.
3.
4.
5.
6.
7.

How many test cases have been designed per requirement?

How many test cases are yet to design?
How many test cases are executed?
How many test cases are passed/failed/blocked?
How many test cases are not yet executed?
How many defects are identified & what is the severity of those defects?
How many test cases are failed due to one particular defect? etc.

Based on the project needs we can have more metrics than above mentioned list, to know the
status of the project in detail.
Based on the above metrics, test lead/manager will get the understanding of the below
mentioned key points.
a.
b.
c.
d.

%ge of work completed

%ge of work yet to be completed
Time to complete the remaining work
Whether the project is going as per the schedule or lagging? etc.

Based on the metrics, if the project is not going to complete as per the schedule, then the
manager will raise the alarm to the client and other stake holders by providing the reasons for
lagging to avoid the last minute surprises.

Prepared By: Neelam Rawat

Neelam Rawat

15

15

METRICS LIFE CYCLE:

TYPES OF MANUAL TEST METRICS:

Testing Metrics are mainly divided into 2 categories.
a) Base Metrics
b) Calculated Metrics
BASE METRICS:
Base Metrics are the Metrics which are derived from the data gathered by the Test Analyst
during the test case development and execution.
This data will be tracked throughout the Test Life cycle. I.e. collecting the data like, Total no. of
test cases developed for a project (or) no. of test cases need to be executed (or) no. of test
cases passed/failed/blocked etc.
CALCULATED METRICS:
Calculated Metrics are derived from the data gathered in Base Metrics. These Metrics are
generally tracked by the test lead/manager for Test Reporting purpose.
VERIFICATION & VALIDATION
VERIFICATION

It makes sure that the product is designed to deliver all functionality to the customer.
Verification is done at the starting of the development process. It includes reviews and
meetings, walkthroughs, inspection, etc. to evaluate documents, plans, code,
requirements and specifications.

Prepared By: Neelam Rawat

Neelam Rawat

16

16

Suppose you are building a table. Here the verification is about checking all the parts of
the table, whether all the four legs are of correct size or not. If one leg of table is not of
the right size it will imbalance the end product. Similar behavior is also noticed in case of
the software product or application. If any feature of software product or application is
not up to the mark or if any defect is found then it will result into the failure of the end
product. Hence, verification is very important. It takes place at the starting of the
development process.

It answers the questions like: Am I building the product right?

Am I accessing the data right (in the right place; in the right way).

It is a Low level activity

Performed during development on key artifacts, like walkthroughs, reviews and

inspections, mentor feedback, training, checklists and standards.

Demonstration of consistency, completeness, and correctness of the software at each

stage and between each stage of the development life cycle.

According to the Capability Maturity Model(CMMI-SW v1.1) we can also define verification as
the process of evaluating software to determine whether the products of a given development
phase satisfy the conditions imposed at the start of that phase. [IEEE-STD-610].
ADVANTAGES OF SOFTWARE VERIFICATION:
1.

Verification helps in lowering down the count of the defect in the later stages of
development.

2.

Verifying the product at the starting phase of the development will help in
understanding the product in a better way.

3.

It reduces the chances of failures in the software application or product.

4.

It helps in building the product as per the customer specifications and needs.

Prepared By: Neelam Rawat

Neelam Rawat

17

17

VALIDATION

Determining if the system complies with the requirements and performs functions for
hi h it is i te ded a d eets the o ga izatio s goals a d use eeds.
Validation is done at the end of the development process and takes place after
verifications are completed.
It answers the question like: Am I building the right product?
Am I accessing the right data (in terms of the data required to satisfy the requirement).
It is a High level activity.
Performed after a work product is produced against established criteria ensuring that the
product integrates correctly into the environment.
Determination of correctness of the final software product by a development project
with respect to the user needs and requirements.

According to the Capability Maturity Model(CMMI-SW v1.1) we can also define validation as
The process of evaluating software during or at the end of the development process to
determine whether it satisfies specified requirements. [IEEE-STD-610].
A product can pass while verification, as it is done on the paper and no running or functional
application is required. But, when same points which were verified on the paper is actually
developed then the running application or product can fail while validation. This may happen
because when a product or application is build as per the specification but these specifications
are not up to the mark hence they fail to address the user requirements.
ADVANTAGES OF VALIDATION:
1.

During verification if some defects are missed then during validation process it can
be caught as failures.

Prepared By: Neelam Rawat

Neelam Rawat

18

18

2.

3.
4.

If during verification some specification is misunderstood and development had

happened then during validation process while executing that functionality the
difference between the actual result and expected result can be understood.
Validation is done during testing like feature testing, integration testing, system
testing, load testing, compatibility testing, stress testing, etc.
Validation helps in building the right product as per the custo e s e ui e e t a d
helps in satisfying their needs.

Validation is basically done by the testers during the testing. While validating the product if
some deviation is found in the actual result from the expected result then a bug is reported or
an incident is raised. Not all incidents are bugs. But all bugs are incidents. Incidents can also be
of t pe Questio
he e the fu tio alit is ot lea to the teste .
Hence, validation helps in unfolding the exact functionality of the features and helps the testers
to understand the product in much better way. It helps in making the product more user
friendly.
TYPES OF TESTING
BASICS OF SOFTWARE TESTING
There are two basics of software testing: blackbox testing and whitebox testing.
BLACKBOX TESTING
Black box testing is a testing technique that ignores the internal mechanism of the system and
focuses on the output generated against any input and execution of the system. It is also called
functional testing.
WHITEBOX TESTING
White box testing is a testing technique that takes into account the internal mechanism of a
system. It is also called structural testing and glass box testing.
Black box testing is often used for validation and white box testing is often used for verification.
TYPES OF TESTING
There are many types of testing like
1. Unit Testing
2. Integration Testing
3. Functional Testing

Prepared By: Neelam Rawat

Neelam Rawat

19

19

4. System Testing
5. Stress Testing
6. Performance Testing
7. Usability Testing
8. Acceptance Testing
9. Regression Testing
10. Beta Testing

UNIT TESTING
Unit testing is the testing of an individual unit or group of related units. It falls under the class
of white box testing. It is often done by the programmer to test that the unit he/she has
implemented is producing expected output against given input.
INTEGRATION TESTING
Integration testing is testing in which a group of components are combined to produce output.
Also, the interaction between software and hardware is tested in integration testing if software
and hardware components have any relation. It may fall under both white box testing and black
box testing.
FUNCTIONAL TESTING
Functional testing is the testing to ensure that the specified functionality required in the system
requirements works. It falls under the class of black box testing.

Prepared By: Neelam Rawat

Neelam Rawat

20

20

SYSTEM TESTING
System testing is the testing to ensure that by putting the software in different environments
(e.g., Operating Systems) it still works. System testing is done with full system implementation
and environment. It falls under the class of black box testing.
STRESS TESTING
Stress testing is the testing to evaluate how system behaves under unfavorable conditions.
Testing is conducted at beyond limits of the specifications. It falls under the class of black box
testing.
PERFORMANCE TESTING
Performance testing is the testing to assess the speed and effectiveness of the system and to
make sure it is generating results within a specified time as in performance requirements. It
falls under the class of black box testing.
USABILITY TESTING
Usability testing is performed to the perspective of the client, to evaluate how the GUI is userfriendly? How easily can the client learn? After learning how to use, how proficiently can the
client perform? How pleasing is it to use its design? This falls under the class of black box
testing.
ACCEPTANCE TESTING
Acceptance testing is often done by the customer to ensure that the delivered product meets
the requirements and works as the customer expected. It falls under the class of black box
testing.
REGRESSION TESTING
Regression testing is the testing after modification of a system, component, or a group of
related units to ensure that the modification is working correctly and is not damaging or
imposing other modules to produce unexpected results. It falls under the class of black box
testing.
BETA TESTING
Beta testing is the testing which is done by end users, a team outside development, or publicly
releasing full pre-version of the product which is known as beta version. The aim of beta testing
is to cover unexpected errors. It falls under the class of black box testing.

Prepared By: Neelam Rawat

Neelam Rawat

21

21

SOFTWARE QUALITY & RELIABILITY

SOFTWARE QUALITY
Quality software is reasonably bug or defect free, delivered on time and within budget, meets
requirements and/or expectations, and is maintainable.
ISO 8402-1986 sta da d defi es ualit as the totalit of featu es a d ha a te isti s of a
p odu t o se i e that ea s its a ilit to satisf stated o i plied eeds.
Some of the factors that go into determining software quality are:

Understandability - Do variables have descriptive names? Are methods commented so

that their purpose is clear? Are any deviations in logic flow adequately commented?
Conciseness - Is all code reachable? Is any code redundant? How many statements
within loops could be placed outside the loop, thus reducing computation time? Are
branch decisions too complex?
Portability - Does the program depend upon system or library routines unique to a
particular installation? Have machine-dependent statements been flagged and
commented?
Maintainability - Have design patterns that lend themselves to ease of maintainability
been employed, (i.e. Code to Interface)?

SOFTWARE RELIABLE acceptable level of breakdowns or failure:

After we have tested for all the features and their functionalities it also very important that the
application or product should be reliable. For example: There is an application of saving the
students records. This application should save all the students records and should not fail after
entering 100 records. This is called reliability.
Reliability defi ed
ANI as the probability of failure-free operation of a computer program
in a specified environment for a specified time. It is a gua le that D . P ess a s defi itio a
be applied to reliability as well, and this would be the quality of conformance half of the
equation. This would mean that reliability is simply another factor of quality. However, I tend to
separate the two, for one very important reason. Because reliability can be measured with
quantitative metrics, two benefits can be realized:
1) The amount of effort needed to achieve a certain threshold of reliability can be
estimated in the planning phase.
2) The reliability of a released application can be objectively measured.

Prepared By: Neelam Rawat

Neelam Rawat

22

22

CONCLUSION: Both terms, reliable and quality, can be used to describe a software application
that has a low degree of error. But there is one fundamental difference between the two.
Software Reliability is objective, measurable, and can be estimated, whereas Software Quality is
based on primarily subjective criteria.
In the context of software engineering, software quality measures how well software is
designed (quality of design), and how well the software conforms to that design (quality of
conformance).
SOFTWARE DEFECT TRACKING
Defect logging, a process of finding defects in the application under test or product by testing or
recording feedback from customers and making new versions of the product that fix the defects
or the clients feedback.

Defect tracking is an important process in software engineering as Complex and business

critical systems have hundreds of defects. One of the challenging factors is managing,
evaluating and prioritizing these defects. The number of defects gets multiplied over a period of
time and to effectively manage them, defect tracking system is used to make the job easier.
Examples - Hp Quality Center, IBM Rational Quality Manager
LIFE CYCLE OF A SOFTWARE DEFECT/BUG:
Defect Life Cycle (Bug Life cycle) is the journey of a defect from its identification to its closure.
The Life Cycle varies from organization to organization and is governed by the software testing
process the organization or project follows and/or the Defect tracking tool being used.

Prepared By: Neelam Rawat

Neelam Rawat

23

23

BUG OR DEFECT LIFE CYCLE INCLUDES FOLLOWING STEPS OR STATUS:

Ne : Whe a defe t is logged a d posted fo the fi st ti e. It s state is gi e as e .

Assigned: After the tester has posted the bug, the lead of the tester approves that the
bug is genuine and he assigns the bug to corresponding developer and the developer
team. Its state given as assigned.
Open: At this state the developer has started analyzing and working on the defect fix.
Fixed: When developer makes necessary code changes and verifies the changes then
he/she a
ake ug status as Fi ed a d the ug is passed to testi g tea .
Pending retest: After fixing the defect the developer has given that particular code for
retesting to the tester. Here the testing is pending on the testers end. Hence its status is
pending retest.
Retest: At this stage the tester do the retesting of the changed code which developer
has given to him to check whether the defect got fixed or not.
Verified: The tester tests the bug again after it got fixed by the developer. If the bug is
not present in the software, he approves that the bug is fixed and changes the status to
e ified .
Reopen: If the bug still exists even after the bug is fixed by the developer, the tester
ha ges the status to eope ed . The ug goes th ough the life cycle once again.
Closed: Once the bug is fixed, it is tested by the tester. If the tester feels that the bug no
lo ge e ists i the soft a e, he ha ges the status of the ug to losed . This state
means that the bug is fixed, tested and approved.

Prepared By: Neelam Rawat

Neelam Rawat

24

24

Duplicate: If the bug is repeated twice or the two bugs mention the same concept of the
ug, the o e ug status is ha ged to dupli ate .
Rejected: If the developer feels that the bug is not genuine, he rejects the bug. Then the
state of the bug is cha ged to eje ted .
Deferred: The bug, changed to deferred state means the bug is expected to be fixed in
next releases. The reasons for changing the bug to this state have many factors. Some of
them are priority of the bug may be low, lack of time for the release or the bug may not
have major effect on the software.
Not a ug: The state gi e as Not a ug if the e is o ha ge i the fu tio alit of the
application. For an example: If customer asks for some change in the look and field of
the application like change of color of some text then it is not a bug but just some
change in the looks of the application.

END OF UNIT-1

KIET GROUP OF INSTITUTIONS

DEPARTMENT OF COMPUTER APPLICATIONS
SESSION: 2016-17, V SEMESTER
SOFTWARE TESTING, NMCA E43
UNIT 2

CONTENTS
1.
2.
3.
4.
5.
6.
7.
8.

White box testing, Static testing

Static analysis tools, Structural testing: Unit/Code functional testing
Difference between Structural & Functional Testing
Code coverage testing, Code complexity testing
Black Box testing, Requirements based testing
Boundary value analysis, Equivalence partitioning
State/graph based testing, Model based testing and model checking
Differences between white box and Black box testing

WHITE BOX TESTING

White Box Testing (also known as Clear Box Testing, Open Box Testing, Glass Box Testing,
Transparent Box Testing, Code-Based Testing or Structural Testing) is a software testing method
in which the internal structure/ design/ implementation of the item being tested is known to
the tester. The tester chooses inputs to exercise paths through the code and determines the

Prepared By: Neelam Rawat

Neelam Rawat

25

25

appropriate outputs. Programming know-how and the implementation knowledge is essential.

White box testing is testing beyond the user interface and into the nitty-gritty of a system.
This method is named so because the software program, in the eyes of the tester, is like a
white/ transparent box; inside which one clearly sees.
EXAMPLE
A tester, usually a developer as well, studies the implementation code of a certain field on a
webpage, determines all legal (valid and invalid) AND illegal inputs and verifies the outputs
against the expected outcomes, which is also determined by studying the implementation code.
White Box Testing is like the work of a mechanic who examines the engine to see why the car is
not moving.

WHITE BOX TESTING ADVANTAGES

Testing can be commenced at an earlier stage. One need not wait for the GUI to be
available.
Testing is more thorough, with the possibility of covering most paths.

WHITE BOX TESTING DISADVANTAGES

Since tests can be very complex, highly skilled resources are required, with thorough
knowledge of programming and implementation.
Test script maintenance can be a burden if the implementation changes too frequently.
Since this method of testing it closely tied with the application being testing, tools to
cater to every kind of implementation/platform may not be readily available.

CLASSIFICATION OF WHITE BOX TESTING

As shown below, white box testing is classified into "static" and "structural" testing.

Prepared By: Neelam Rawat

Neelam Rawat

26

26

STATIC TESTING
Static testing is a type of testing which requires only the source code of the product, not the
binaries or executables. Static testing does not involve executing the programs on computers
but involves select people going through the code to find out whether
The code works according to the functional requirement;
The code has been written in accordance with the design developed earlier in the
project life cycle;
The code for any functionality has been missed out;
The code handles errors properly.

Static testing is the testing of the software work products manually, or with a set of
tools, but they are not executed.
It starts early in the Life cycle and so it is done during the verification process.
It does not need computer as the testing of program is done without executing the
program. For example: reviewing, walk through, inspection, etc.

NOTE: Static testing can be done by humans or with the help of specialized tools.
METHODS TO ACHIEVE STATIC TESTING

Prepared By: Neelam Rawat

Neelam Rawat

27

27

DESK CHECKING
Normally done manually by the author of the code, desk checking is a method to verify the
portions of the code for correctness. Such verification is done by comparing the code with the
design or specifications to make sure that the code does what it is supposed to do and
effectively. This is the desk checking that most programmers do before compiling and executing
the code. Whenever errors are found, the author applies the corrections for errors on the spot.
This method of catching and correcting errors is characterized by:
No structured method or formalism to ensure completeness and
No maintaining of a log or checklist.
In effect, this method relies completely on the author's thoroughness, diligence, and skills.
There is no process or structure that guarantees or verifies the effectiveness of desk checking.
This method is effective for correcting" obvious" coding errors but will not be effective in
detecting errors that arise due to incorrect understanding of requirements or incomplete
requirements.
WALKTHROUGH:

It is not a formal process

It is led by the authors
Author guide the participants through the document according to his or her thought
process to achieve a common understanding and to gather feedback.
Useful for the people if they are not from the software discipline, who are not used to or
cannot easily understand software development process.
Is especially useful for higher level documents like requirement specification, etc.

The goals of a walkthrough:

To present the documents both within and outside the software discipline in order to
gather the information regarding the topic under documentation.
To explain or do the knowledge transfer and evaluate the contents of the document
To achieve a common understanding and to gather feedback.
To examine and discuss the validity of the proposed solutions

TECHNICAL REVIEW:

It is less formal review

It is led by the trained moderator but can also be led by a technical expert
It is often performed as a peer review without management participation
Defects are found by the experts (such as architects, designers, key users) who focus on
the content of the document.

Prepared By: Neelam Rawat

Neelam Rawat

28

28

In practice, technical reviews vary from quite informal to very formal

The goals of the technical review are:

To ensure that an early stage the technical concepts are used correctly
To access the value of technical concepts and alternatives in the product
To have consistency in the use and representation of technical concepts
To inform participants about the technical content of the document

INSPECTION:

It is the most formal review type

It is led by the trained moderators
During inspection the documents are prepared and checked thoroughly by the
reviewers before the meeting
It involves peers to examine the product
A separate preparation is carried out during which the product is examined and the
defects are found
The defects found are documented in a logging list or issue log
A formal follow-up is carried out by the moderator applying exit criteria

The goals of inspection are:

It helps the author to improve the quality of the document under inspection
It removes defects efficiently and as early as possible
It improve product quality
It create common understanding by exchanging information
It learn from defects found and prevent the occurrence of similar defects

STATIC ANALYSIS TOOLS

Static analysis is analysis of computer code that is performed without actually executing
programs. And a static code analysis tool automatically checks the source code for compliance
with a predefined set of rules or best practices set by the organization. Static analysis tools are
a fast and efficient at finding code defects and inconsistencies, especially in large code bases,
including older legacy code and newly created code.
A static code analysis tool can help with your code review process by
1. Detecting areas in the code that need to be refactored and simplified,
2. Finding areas of the code that may need more testing or deeper review,
3. Identifying design issues such as Cyclomatic Complexity and helping reduce the code
complexity improve maintainability,

Prepared By: Neelam Rawat

Neelam Rawat

29

29

4. Identifying potential software quality issues before the code moves to production.
STRUCTURAL TESTING

The structural testing is the testing of the structure of the system or component.
t u tu al testi g is ofte efe ed to as hite o o glass o o lea - o testi g
e ause i st u tu al testi g e a e i te ested i
hat is happe i g i side the
s ste /appli atio .
In structural testing the testers are required to have the knowledge of the internal
implementations of the code. Here the testers require knowledge of how the software is
implemented, how it works.
During structural testing the tester is concentrating on how the software does it. For
example, a structural technique wants to know how loops in the software are working.
Different test cases may be derived to exercise the loop once, twice, and many times.
This may be done regardless of the functionality of the software.
Structural testing can be used at all levels of testing. Developers use structural testing in
component testing and component integration testing, especially where there is good
tool support for code coverage. Structural testing is also used in system and acceptance
testing, but the structures are different. For example, the coverage of menu options or
major business transactions could be the structural element in system or acceptance
testing.

ADVANTAGES OF STRUCTURAL TESTING:

Forces test developer to reason carefully about implementation

Reveals errors in "hidden" code
Spots the Dead Code or other issues with respect to best programming practices.

DISADVANTAGES OF STRUCTURAL BOX TESTING:

Expensive as one has to spend both time and money to perform white box testing.
Every possibility that few lines of code is missed accidentally.
In-depth knowledge about the programming language is necessary to perform white
box testing.

STRUCTURAL TESTING can be further classified into:

1. Unit/ Code Functional Testing
2. Code Coverage
3. Code Complexity Testing
UNIT TESTING

Prepared By: Neelam Rawat

Neelam Rawat

30

30

Unit testing is a level of software testing where individual units/ components of

software are tested. The purpose is to validate that each unit of the software performs
as designed.
A unit is the smallest testable part of an application like functions, classes, procedures,
interfaces. Unit testing is a method by which individual units of source code are tested
to determine if they are fit for use.
Unit tests are basically written and executed by software developers to make sure that
code meets its design and requirements and behaves as expected.
The goal of unit testing is to segregate each part of the program and test that the
individual parts are working correctly.
This means that for any function or procedure when a set of inputs are given then it
should return the proper values. It should handle the failures gracefully during the
course of execution when any invalid input is given.
A unit test provides a written contract that the piece of code must assure. Hence it has
several benefits.
White Box Testing method is used for executing the unit test

ADVANTAGES OF UNIT TESTING:

Issues are found at early stage. Since unit testing are carried out by developers where they
test their individual code before the integration. Hence the issues can be found very early
and can be resolved then and there without impacting the other piece of codes.

Unit testing helps in maintaining and changing the code. This is possible by making the
codes less interdependent so that unit testing can be executed. Hence chances of impact
of changes to any other code get reduced.

Since the bugs are found early in unit testing hence it also helps in reducing the cost of
bug fixes. Just imagine the cost of bug found during the later stages of development like
during system testing or during acceptance testing.

Unit testing helps in simplifying the debugging process. If suppose a test fails then only
latest changes made in code needs to be debugged.

Functional (Black Box) Testing and Structural (White Box) Testing

Functional Part is a software testing approach in which:
1. This is done by Testers.
2. Functional Testing is also known as Black Box Testing or behavioral testing.
3. The tester will have a user perspective in mind.

Prepared By: Neelam Rawat

Neelam Rawat

31

31

4. It focuses on the functional part of the application.

5. He doesnt know how the program works in back end.
6. He will only concentrate in input and output of the application or product.
7. The tester acts as if he/she is the final user of the program.
On other hand Structural Part, is a software testing approach in which:
1. Structural Testing is also known as white-box testing or Glass Box testing.
2. The tester will have a developer perspective in mind or this is done by Developer.
3. Sometime tester acts as a developer of the program who knows the internal structure of
the program very well.
4. He or she will know how the program works behind the scene, with knowing the Internal
Knowledge of the product /Application.
5. Its main focus is on the structural part means in coding/programming part.
6. Structural is mainly focused on internal code whereas functionality is verified with
respect to SRS.
Code Coverage Testing
Code coverage is a white box testing methodology that is it requires knowledge of and access to
the code itself rather than simply using the interface provided. Code coverage is probably most
useful during the module testing phase, though it also has benefit during integration testing and
probably at other times, depending on how and what you are testing.

Prepared By: Neelam Rawat

Neelam Rawat