Software Testing

Unit No.1
In general, testing is finding out how well something works. In terms of human beings, testing
tells what level of knowledge or skill has been acquired. In computer hardware and software
development, testing is used at key checkpoints in the overall process to determine whether
objectives are being met.
Software Testing:
is the act of examining the artifacts and the behavior of the software under test by validation and
verification? Software testing can also provide an objective, independent view of the software to
allow the business to appreciate and understand the risks of software implementation.
Software Testing is a method to check whether the actual software product matches expected
requirements and to ensure that software product is Defect free.
It involves execution of software/system components using manual or automated tools to
evaluate one or more properties of interest. The purpose of software testing is to identify errors,
gaps or missing requirements in contrast to actual requirements.
Software Errors:
An error is a deviation from accuracy or correctness” and “A software bug is an error, flaw,
failure, or fault in a computer program or system that causes it to produce an incorrect or
unexpected result, or to behave in unintended ways “.
• Reasons:
Error is a variance of the actual result from the expected result.
Errors are a category of software bugs.
Errors can be introduced as result of incomplete or inaccurate requirements or due to human data
entry problems.
Software Errors:
1) Functionality Errors:
Functionality is a way the software is intended to behave. Software has a functionality error if
something that you expect it to do is hard, awkward, confusing, or impossible.
E.x.
Expected Functionality for Cancel button is that the ‘Create new project’ window should close
and none of the changes should be saved (i.e. no new project must be created). If the Cancel
button is not clickable then it is a functionality error
2) Communication Errors:
These errors occur in communication from software to end-user. Anything that the end user
needs to know in order to use the software should be made available on screen.
Few examples of communication errors are – No Help instructions/menu provided, features that
are part of the release but are not documented in the help menu, a button named ‘Save’ should
not erase a file etc.
3) Missing command errors:
This happens to occur when an expected command is missing. See this screenshot:
E.X.:-Cancel’ option/button is not provided to the user, this is a missing command error.
4) Syntactic Error:
Syntactic errors are misspelled words or grammatically incorrect sentences and are very evident
while testing software GUI. Please note that we are NOT referring to syntax errors in code. The
compiler will warn the developer about any syntax errors that occur in the code
5) Error handling errors:
Any errors that occur while the user is interacting with the software needs to be handled in a
clear and meaningful manner. If not, it is called as an Error Handling Error.
Take a look at this image. The error message gives no indication of what the error actually is. Is
it missing mandatory field, saving error, page loading error or is it a system error? Hence, this is
an ‘Error Handing Error’.

6) Calculation Errors:
These errors occur due to any of the following reasons:
 • Bad logic
• Incorrect formulae
• Data type mismatch
• Coding errors
• Function call issues , etc.
In 1999, NASA lost its Mars climate orbiter because one of the subcontractors NASA employed
had used English units instead of the intended metric system, which caused the orbiter’s thrusters
to work incorrectly. Due to this bug, the orbiter crashed almost immediately when it arrived at
Mars.
7) Control flow errors:
The control flow of a software describes what it will do next and on what condition.
For example, consider a system where user has to fill in a form and the options available to user
are: Save, Save and Close, and Cancel. If a user clicks on ‘Save and Close’ button, the user
information in the form should be saved and the form should close. If clicking on the button does
not close the form, then it is a control flow error.
Testing Objectives:
• To find any defects or bugs that may have been created when the software was being
developed
• To increase confidence in the quality of the software
• To prevent defects in the final product
• To ensure the end product meets customer requirements as well as the company
specifications
• To provide customers with a quality product and increase their confidence in the
company
• Testing Principles:
• Optimal testing – it's not possible to test everything so it's important to determine the
optimal amount. The decision is made using a risk assessment. This assessment will
uncover the area that is most likely to fail and this is where testing should take place.
• Pareto Principle – this principle states that approximately 80% of problems will be found
in 20% of tests. However, there is a flaw in this principle in that repeating the same tests
over and over again will mean no new bugs will be found.
• Review and Revise – repeating the same tests will mean that the methods will eventually
become useless for uncovering new defects. To prevent this from happens only requires
the tests to be reviewed and revised on a regular basis. Adding new tests will help to find
more defects.
• Defects that are present – testing reduces the probability of the being a defect in the final
product but does not guarantee that a defect won't be there. And even if you manage to
make a product that's 99% bug free, the testing won't have shown whether the software
meets the needs of clients.
• Meeting customer needs – testing a product for the wrong requirements is foolhardy.
Even if it is bug free it may still fail to meet customer requirements.
• Test early – it's imperative that testing starts as soon as possible in the development of a
product.
• Test in context – test a product in accordance with how it will be used Software is not
identical and will be developed to meet a certain need rather than a general one. Different
techniques, methodologies, approach and type of testing can be used depending on the
applications planned use.
Software Testing Fundamentals (STF):
STF: testing: The practice of investigating a software / system under test so as to ensure that it is
of the highest quality.
Why Software Testing?
Software that does not work correctly can lead to many problems such as:
Delay / Loss of time
Futility / Loss of effort
Wastage / Loss of money
Shame / Loss of business reputation
Injury or death
The three main goals of Software Testing are:
Defect Detection: Find defects / bugs in the software during all stages of its development (earlier,
the better).
Defect Prevention: As a consequence of defect detection, help anticipate and prevent defects
from occurring at later stages of development or from recurring in the future.
User Satisfaction: Ensure customers / users are satisfied that their requirements (explicit or
implicit) are met.
The three main goals of Software Testing are:
Defect Detection: Find defects / bugs in the software during all stages of its development (earlier,
the better).
Defect Prevention: As a consequence of defect detection, help anticipate and prevent defects
from occurring at later stages of development or from recurring in the future.
User Satisfaction: Ensure customers / users are satisfied that their requirements (explicit or
implicit) are met.

THE PATH THE SAMPLE

“SOFTWARE
Get started by going through Software Testing Basics like
QUALITY is the
Quality Assurance, Quality Control, Software Development
degree of
 conformance to
explicit or implicit
Life Cycle and Software Testing Life Cycle.
requirements and
expectations.”

“UNIT TESTING is a
level of software
And, level up by understanding Software Testing
testing where
Levels like Unit Testing, Integration Testing, Syste
individual units/
m Testing and Acceptance Testing.
components of a
software are tested.”

“BLACK BOX
TESTING is a
software testing
Then, go through the various Software Testing
method in which the
Methods like Black Box Testing, White Box Testing, Gray
internal structure of
Box Testing, Agile Testing and Ad Hoc Testing.
the item being tested
is not known to the
tester.”

“SMOKE
TESTING is a type of
Further, check out the various Software Testing software testing that
Types like Smoke Testing, Functional Testing, comprises of a non-
Usability Testing, Security Testing, Performance exhaustive set of tests
Testing, Regression Testing and Compliance Testing. that aim at ensuring
that the most
important functions
work.”
 “A TEST PLAN is a
Also, learn about Software Testing document describing
Artifacts, Defects and Software Testing Metrics. software testing scope
and activities.”

Software review:
• A software review is an effective way of filtering errors in a software product.
• Reviews conducted at each of these phases i.e., analysis, design, coding, and testing
reveal areas of improvement in the product.
• Reviews also indicate those areas that do not need any improvement.
• We can use software reviews to achieve consistency and uniformity across products.
• Reviews also make the task of product creation more manageable.
• Software review:
• Some of the most common software review techniques are:
i. Inspection
ii. Walkthrough
iii. Code review
iv. Formal Technical Reviews (FTR)
v. Pair programming
Formal Technical Review:
Software review:
A software review is an effective way of filtering errors in a software product. Reviews
conducted at each of these phases i.e., analysis, design, coding, and testing reveal areas of
improvement in the product. Reviews also indicate those areas that do not need any
improvement. We can use software reviews to achieve consistency and uniformity across
products. Reviews also make the task of product creation more manageable. Some of the most
common software review techniques are:
i. Inspection
ii. Walkthrough
iii. Code review
iv. Formal Technical Reviews (FTR)
v. Pair programming
Formal technical review (FTR):
1. A formal technical review is a software quality assurance activity performed by
software engineers.
2. In addition, the FTR serves as a training ground, enabling junior engineers to observe
the different approaches to software analysis, design, and implementation.
3. The objectives of FTR are
i. To uncover errors in function, logic, or implementation for representation of
software.
ii. To verify that software under review meets its requirements.
iii. To ensure that the software has been represented according to predefined
standards.
iv. To achieve software that is developed in a uniform manner. v. To make projects
more manageable.
Steps required to conduct a successful FTR:
1. The review meeting
 Every review meeting should be conducted by considering the following constraints:
a. Short duration Advance preparation
b. Involvement of people
c. walkthrough
 Rather than attempting to review the entire design; is conducted for modules or for a
small group of modules.
 The focus of the FTR is on the work product (a software component to be review).
The review meeting is attend by the review leader., all reviewers, and the producer.
 One of the reviewers becomes a recorder who records all the important issues raised
during the review. When errors are discovere, the recorder notes each error.
 At the end of the review, the attendees decide whether to accept the product or not,
with or without modification.
2. Review reporting and record-keeping
 During the FTR, the reviewer actively records all the issues that have been raised.
  At the end of the meeting, these all raised issues are consolidating and a review issues
list is prepare.
Finally, a formal technical review summary report is produce.
3. Review guidelines
 Guidelines for the conducting of formal technical review must be establish in advance.
 These guidelines must be distribute to all reviewers, agree upon, and then followed.
Software Testing Life Cycle:
Software Testing Life Cycle (STLC) is a sequence of specific activities conducted during the
testing process to ensure software quality goals are met. STLC involves both verification and
validation activities. Contrary to popular belief, Software Testing is not just a single/isolate
activity, i.e. testing. It consists of a series of activities carried out methodologically to help
certify your software product. STLC stands for Software Testing Life Cycle.

Requirement Phase Testing

Requirement Phase Testing also known as Requirement Analysis in which test team studies the
requirements from a testing point of view to identify testable requirements and the QA team may
interact with various stakeholders to understand requirements in detail. Requirements could be
either functional or non-functional. Automation feasibility for the testing project is also done in
this stage.
Activities in Requirement Phase Testing
 Identify types of tests to be performed.
 Gather details about testing priorities and focus.
  Prepare Requirement Traceability Matrix (RTM).
 Identify test environment details where testing is supposed to be carried out.
 Automation feasibility analysis (if required).
Deliverables of Requirement Phase Testing
 RTM
 Automation feasibility report. (if applicable)
Test Planning in STLC
Test Planning in STLC is a phase in which a Senior QA manager determines the test plan
strategy along with efforts and cost estimates for the project. Moreover, the resources, test
environment, test limitations and the testing schedule are also determined. The Test Plan gets
prepared and finalized in the same phase.
Test Planning Activities
 Preparation of test plan/strategy document for various types of testing
 Test tool selection
 Test effort estimation
 Resource planning and determining roles and responsibilities.
 Training requirement
Deliverables of Test Planning
 Test plan /strategy document.
 Effort estimation document.
Test Case Development Phase
The Test Case Development Phase involves the creation, verification and rework of test cases
& test scripts after the test plan is ready. Initially, the Test data is identified then created and
reviewed and then reworked based on the preconditions. Then the QA team starts the
development process of test cases for individual units.
Test Case Development Activities
 Create test cases, automation scripts (if applicable)
 Review and baseline test cases and scripts
 Create test data (If Test Environment is available)
Deliverables of Test Case Development
 Test cases/scripts
  Test data
Test Environment Setup
Test Environment Setup decides the software and hardware conditions under which a work
product is tested. It is one of the critical aspects of the testing process and can be done in parallel
with the Test Case Development Phase. Test team may not be involved in this activity if the
development team provides the test environment. The test team is required to do a readiness
check (smoke testing) of the given environment.
Test Environment Setup Activities
 Understand the required architecture, environment set-up and prepare hardware and
software requirement list for the Test Environment.
 Setup test Environment and test data
 Perform smoke test on the build
Deliverables of Test Environment Setup
 Environment ready with test data set up
 Smoke Test Results.
Test Execution Phase
Test Execution Phase is carried out by the testers in which testing of the software build is done
based on test plans and test cases prepared. The process consists of test script execution, test
script maintenance and bug reporting. If bugs are reported then it is reverted back to
development team for correction and retesting will be performed.
Test Execution Activities
 Execute tests as per plan
 Document test results, and log defects for failed cases
 Map defects to test cases in RTM
 Retest the Defect fixes
 Track the defects to closure
Deliverables of Test Execution
 Completed RTM with the execution status
 Test cases updated with results
 Defect reports
Test Cycle Closure
Test Cycle Closure phase is completion of test execution which involves several activities like
test completion reporting, collection of test completion matrices and test results. Testing team
members meet, discuss and analyze testing artifacts to identify strategies that have to be
implemented in future, taking lessons from current test cycle. The idea is to remove process
bottlenecks for future test cycles.
Test Cycle Closure Activities
 Evaluate cycle completion criteria based on Time, Test coverage, Cost,Software, Critical
Business Objectives, Quality
 Prepare test metrics based on the above parameters.
 Document the learning out of the project
 Prepare Test closure report
 Qualitative and quantitative reporting of quality of the work product to the customer.
 Test result analysis to find out the defect distribution by type and severity.
Deliverables of Test Cycle Closure
 Test Closure report
 Test metrics