Software Testing and Automation

Unit 1
Software Testing
Software testing can be stated as the process of verifying and validating whether a
software or application is bug-free, meets the technical requirements as guided by its
design and development, and meets the user requirements effectively and efficiently by
handling all the exceptional and boundary cases.

The process of software testing aims not only at finding faults in the existing software
but also at finding measures to improve the software in terms of efficiency, accuracy,
and usability. It mainly aims at measuring the specification, functionality, and
performance of a software program or application.

Software testing can be divided into two steps:

1. Verification: it refers to the set of tasks that ensure that the software correctly
implements a specific function.
2. Validation: it refers to a different set of tasks that ensure that the software that has
been built is traceable to customer requirements.
Verification: “Are we building the product right?”
Validation: “Are we building the right product?”
What are different types of software testing?
Software Testing can be broadly classified into two types:

1. Manual Testing: Manual testing includes testing software manually, i.e., without
using any automation tool or any script. In this type, the tester takes over the role of an
end-user and tests the software to identify any unexpected behavior or bug. There are
different stages for manual testing such as unit testing, integration testing, system
testing, and user acceptance testing.
Testers use test plans, test cases, or test scenarios to test software to ensure the
completeness of testing. Manual testing also includes exploratory testing, as testers
explore the software to identify errors in it.

2. Automation Testing: Automation testing, which is also known as Test

Automation, is when the tester writes scripts and uses another software to test the
product. This process involves the automation of a manual process. Automation Testing
is used to re-run the test scenarios quickly and repeatedly, that were performed
manually in manual testing.

Apart from regression testing, automation testing is also used to test the application
from a load, performance, and stress point of view. It increases the test coverage,
improves accuracy, and saves time and money when compared to manual testing.

What are the different types of Software Testing Techniques ?

Software testing techniques can be majorly classified into two categories:

1. Black Box Testing: The technique of testing in which the tester doesn’t have access
to the source code of the software and is conducted at the software interface without
any concern with the internal logical structure of the software is known as black-box
testing.
2. White-Box Testing: The technique of testing in which the tester is aware of the
internal workings of the product, has access to its source code, and is conducted by
making sure that all internal operations are performed according to the specifications is
known as white box testing.

Black Box Testing White Box Testing

Internal workings of an application are

Knowledge of the internal workings is a must.
not required.

Also known as closed box/data-driven Also known as clear box/structural testing.

testing.

End users, testers, and developers. Normally done by testers and developers.

This can only be done by a trial and error Data domains and internal boundaries can be
method. better tested.

What are different levels of software testing?

Software level testing can be majorly classified into 4 levels:

1. Unit Testing: A level of the software testing process where individual

units/components of a software/system are tested. The purpose is to validate that each
unit of the software performs as designed.
2. Integration Testing: A level of the software testing process where individual units
are combined and tested as a group. The purpose of this level of testing is to expose
faults in the interaction between integrated units.
3. System Testing: A level of the software testing process where a complete, integrated
system/software is tested. The purpose of this test is to evaluate the system’s
compliance with the specified requirements.
4. Acceptance Testing: A level of the software testing process where a system is tested
for acceptability. The purpose of this test is to evaluate the system’s compliance with
the business requirements and assess whether it is acceptable for delivery.
Software Testing Life Cycle (STLC)
The Software Testing Life Cycle (STLC) is a systematic approach to testing a software
application to ensure that it meets the requirements and is free of defects. It is a process
that follows a series of steps or phases, and each phase has specific objectives and
deliverables. The STLC is used to ensure that the software is of high quality, reliable,
and meets the needs of the end-users.

The main goal of the STLC is to identify and document any defects or issues in the
software application as early as possible in the development process. This allows for
issues to be addressed and resolved before the software is released to the public.
The stages of the STLC include Test Planning, Test Analysis, Test Design, Test
Environment Setup, Test Execution, Test Closure, and Defect Retesting. Each of these
stages includes specific activities and deliverables that help to ensure that the software
is thoroughly tested and meets the requirements of the end users.

Overall, the STLC is an important process that helps to ensure the quality of software
applications and provides a systematic approach to testing. It allows organizations to
release high-quality software that meets the needs of their customers, ultimately leading
to customer satisfaction and business success.

Characteristics of STLC
 STLC is a fundamental part of the Software Development Life Cycle (SDLC) but
STLC consists of only the testing phases.
 STLC starts as soon as requirements are defined or software requirement document
is shared by stakeholders.
 STLC yields a step-by-step process to ensure quality software.
In the initial stages of STLC, while the software product or the application is being
developed, the testing team analyzes and defines the scope of testing, entry and exit
criteria, and also test cases. It helps to reduce the test cycle time and also enhances
product quality. As soon as the development phase is over, the testing team is ready
with test cases and starts the execution. This helps in finding bugs in the early phase.

Phases of STLC
1. Requirement Analysis: Requirement Analysis is the first step of the Software
Testing Life Cycle (STLC). In this phase quality assurance team understands the
requirements like what is to be tested. If anything is missing or not understandable then
the quality assurance team meets with the stakeholders to better understand the detailed
knowledge of requirements.
The activities that take place during the Requirement Analysis stage include:
 Reviewing the software requirements document (SRD) and other related documents
 Interviewing stakeholders to gather additional information
 Identifying any ambiguities or inconsistencies in the requirements
 Identifying any missing or incomplete requirements
 Identifying any potential risks or issues that may impact the testing process
Creating a requirement traceability matrix (RTM) to map requirements to test cases
At the end of this stage, the testing team should have a clear understanding of the
software requirements and should have identified any potential issues that may impact
the testing process. This will help to ensure that the testing process is focused on the
most important areas of the software and that the testing team is able to deliver high-
quality results.

2. Test Planning: Test Planning is the most efficient phase of the software testing life
cycle where all testing plans are defined. In this phase manager of the testing, team
calculates the estimated effort and cost for the testing work. This phase gets started
once the requirement-gathering phase is completed.
The activities that take place during the Test Planning stage include:
 Identifying the testing objectives and scope
 Developing a test strategy: selecting the testing methods and techniques that will be
used
 Identifying the testing environment and resources needed
 Identifying the test cases that will be executed and the test data that will be used
 Estimating the time and cost required for testing
 Identifying the test deliverables and milestones
 Assigning roles and responsibilities to the testing team
 Reviewing and approving the test plan
At the end of this stage, the testing team should have a detailed plan for the testing
activities that will be performed, and a clear understanding of the testing objectives,
scope, and deliverables. This will help to ensure that the testing process is well-
organized and that the testing team is able to deliver high-quality results.
3. Test Case Development: The test case development phase gets started once the test
planning phase is completed. In this phase testing team notes down the detailed test
cases. The testing team also prepares the required test data for the testing. When the test
cases are prepared then they are reviewed by the quality assurance team.
The activities that take place during the Test Case Development stage include:
 Identifying the test cases that will be developed
 Writing test cases that are clear, concise, and easy to understand
 Creating test data and test scenarios that will be used in the test cases
 Identifying the expected results for each test case
 Reviewing and validating the test cases
 Updating the requirement traceability matrix (RTM) to map requirements to test
cases
At the end of this stage, the testing team should have a set of comprehensive and
accurate test cases that provide adequate coverage of the software or application. This
will help to ensure that the testing process is thorough and that any potential issues are
identified and addressed before the software is released.

4. Test Environment Setup: Test environment setup is a vital part of the STLC.
Basically, the test environment decides the conditions on which software is tested. This
is independent activity and can be started along with test case development. In this
process, the testing team is not involved. either the developer or the customer creates
the testing environment.
5. Test Execution: After the test case development and test environment setup test
execution phase gets started. In this phase testing team starts executing test cases based
on prepared test cases in the earlier step.
The activities that take place during the test execution stage of the Software
Testing Life Cycle (STLC) include:
 Test execution: The test cases and scripts created in the test design stage are run
against the software application to identify any defects or issues.
 Defect logging: Any defects or issues that are found during test execution are
logged in a defect tracking system, along with details such as the severity, priority,
and description of the issue.
 Test data preparation: Test data is prepared and loaded into the system for test
execution
 Test environment setup: The necessary hardware, software, and network
configurations are set up for test execution
 Test execution: The test cases and scripts are run, and the results are collected and
analyzed.
 Test result analysis: The results of the test execution are analyzed to determine the
software’s performance and identify any defects or issues.
 Defect retesting: Any defects that are identified during test execution are retested to
ensure that they have been fixed correctly.
 Test Reporting: Test results are documented and reported to the relevant
stakeholders.
 It is important to note that test execution is an iterative process and may need to
be repeated multiple times until all identified defects are fixed and the software
is deemed fit for release.
 6. Test Closure: Test closure is the final stage of the Software Testing Life
Cycle (STLC) where all testing-related activities are completed and documented.
The main objective of the test closure stage is to ensure that all testing-related
activities have been completed and that the software is ready for release.
 At the end of the test closure stage, the testing team should have a clear
understanding of the software’s quality and reliability, and any defects or issues
that were identified during testing should have been resolved. The test closure
stage also includes documenting the testing process and any lessons learned so
that they can be used to improve future testing processes

 Test closure is the final stage of the Software Testing Life Cycle (STLC) where
all testing-related activities are completed and documented. The main activities
that take place during the test closure stage include:

 Test summary report: A report is created that summarizes the overall testing
process, including the number of test cases executed, the number of defects
found, and the overall pass/fail rate.
 Defect tracking: All defects that were identified during testing are tracked and
managed until they are resolved.
 Test environment clean-up: The test environment is cleaned up, and all test
data and test artifacts are archived.
 Test closure report: A report is created that documents all the testing-related
activities that took place, including the testing objectives, scope, schedule, and
resources used.
 Knowledge transfer: Knowledge about the software and testing process is
shared with the rest of the team and any stakeholders who may need to maintain
or support the software in the future.
 Feedback and improvements: Feedback from the testing process is collected
and used to improve future testing processes
It is important to note that test closure is not just about documenting the testing process,
but also about ensuring that all relevant information is shared and any lessons learned
are captured for future reference. The goal of test closure is to ensure that the software
is ready for release and that the testing process has been conducted in an organized and
efficient manner.

V-Model
The V-model is a type of SDLC model where process executes in a sequential manner
in V-shape. It is also known as Verification and Validation model. It is based on the
association of a testing phase for each corresponding development stage. Development
of each step directly associated with the testing phase. The next phase starts only after
completion of the previous phase i.e. for each development activity, there is a testing
activity corresponding to it.

The V-Model is a software development life cycle (SDLC) model that provides a
systematic and visual representation of the software development process. It is based on
the idea of a “V” shape, with the two legs of the “V” representing the progression of the
software development process from requirements gathering and analysis to design,
implementation, testing, and maintenance.

The V-Model is a linear and sequential model that consists of the following
phases:

1. Requirements Gathering and Analysis: The first phase of the V-Model is the
requirements gathering and analysis phase, where the customer’s requirements for
the software are gathered and analyzed to determine the scope of the project.
2. Design: In the design phase, the software architecture and design are developed,
including the high-level design and detailed design.
3. Implementation: In the implementation phase, the software is actually built based on
the design.
4. Testing: In the testing phase, the software is tested to ensure that it meets the
customer’s requirements and is of high quality.
5. Deployment: In the deployment phase, the software is deployed and put into use.
6. Maintenance: In the maintenance phase, the software is maintained to ensure that it
continues to meet the customer’s needs and expectations.
7. The V-Model is often used in safety-critical systems, such as aerospace and defense
systems, because of its emphasis on thorough testing and its ability to clearly define
the steps involved in the software development process.

Verification: It involves static analysis technique (review) done without executing

code. It is the process of evaluation of the product development phase to find whether
specified requirements meet.
Validation: It involves dynamic analysis technique (functional, non-functional),
testing done by executing code. Validation is the process to evaluate the software after
the completion of the development phase to determine whether software meets the
customer expectations and requirements.
So V-Model contains Verification phases on one side of the Validation phases on the
other side. Verification and Validation phases are joined by coding phase in V-shape.
Thus it is called V-Model.

Design Phase:

 Requirement Analysis: This phase contains detailed communication with the

customer to understand their requirements and expectations. This stage is known as
Requirement Gathering.
 System Design: This phase contains the system design and the complete hardware
and communication setup for developing product.
 Architectural Design: System design is broken down further into modules taking
up different functionalities. The data transfer and communication between the
internal modules and with the outside world (other systems) is clearly understood.
 Module Design: In this phase the system breaks down into small modules. The
detailed design of modules is specified, also known as Low-Level Design (LLD).

Testing Phases

 Unit Testing: Unit Test Plans are developed during module design phase. These
Unit Test Plans are executed to eliminate bugs at code or unit level.
 Integration testing: After completion of unit testing Integration testing is
performed. In integration testing, the modules are integrated and the system is
 tested. Integration testing is performed on the Architecture design phase. This test
verifies the communication of modules among themselves.
 System Testing: System testing test the complete application with its functionality,
inter dependency, and communication.It tests the functional and non-functional
requirements of the developed application.
 User Acceptance Testing (UAT): UAT is performed in a user environment that
resembles the production environment. UAT verifies that the delivered system meets
user’s requirement and system is ready for use in real world.
Industrial Challenge: As the industry has evolved, the technologies have become more
complex, increasingly faster, and forever changing, however, there remains a set of
basic principles and concepts that are as applicable today as when IT was in its infancy.

 Accurately define and refine user requirements.

 Design and build an application according to the authorized user requirements.
 Validate that the application they had built adhered to the authorized business
requirements.

Principles of V-Model:

 Large to Small: In V-Model, testing is done in a hierarchical perspective, For

example, requirements identified by the project team, create High-Level Design, and
Detailed Design phases of the project. As each of these phases is completed the
requirements, they are defining become more and more refined and detailed.
 Data/Process Integrity: This principle states that the successful design of any
project requires the incorporation and cohesion of both data and processes. Process
elements must be identified at each and every requirements.
 Scalability: This principle states that the V-Model concept has the flexibility to
accommodate any IT project irrespective of its size, complexity or duration.
 Cross Referencing: Direct correlation between requirements and corresponding
testing activity is known as cross-referencing.

SOFTWARE TESTING PRINCIPLES:

1. Testing shows the presence of defects

2. Exhaustive testing is not possible
3. Early testing
4. Defect clustering
5. Pesticide paradox
6. Testing is context-dependent
7. Absence of errors fallacy
 Testing shows the presence of defects: The goal of software testing is to make the
software fail. Software testing reduces the presence of defects. Software testing talks
about the presence of defects and doesn’t talk about the absence of defects. Software
testing can ensure that defects are present but it can not prove that software is
defect-free. Even multiple testing can never ensure that software is 100% bug-free.
Testing can reduce the number of defects but not remove all defects.
 Exhaustive testing is not possible: It is the process of testing the functionality
of the software in all possible inputs (valid or invalid) and pre-conditions is
known as exhaustive testing. Exhaustive testing is impossible means the software
can never test at every test case. It can test only some test cases and assume that
the software is correct and it will produce the correct output in every test case. If
the software will test every test case then it will take more cost, effort, etc.,
which is impractical.
 Early Testing: To find the defect in the software, early test activity shall be
started. The defect detected in the early phases of SDLC will be very less
expensive. For better performance of software, software testing will start at the
initial phase i.e. testing will perform at the requirement analysis phase.
  Defect clustering: In a project, a small number of modules can contain most of
the defects. Pareto Principle to software testing state that 80% of software defect
comes from 20% of modules.
 Pesticide paradox: Repeating the same test cases, again and again, will not find
new bugs. So it is necessary to review the test cases and add or update test cases
to find new bugs.
 Testing is context-dependent: The testing approach depends on the context of
the software developed. Different types of software need to perform different
types of testing. For example, The testing of the e-commerce site is different
from the testing of the Android application.
 Absence of errors fallacy: If a built software is 99% bug-free but it does not
follow the user requirement then it is unusable. It is not only necessary that
software is 99% bug-free but it is also mandatory to fulfill all the customer
requirements.
STAGES OF SOFTWARE TESTING

1. Unit Testing

2. Integration Testing

3. Regression Testing

4. Smoke Testing

5. System Testing

6. Alpha Testing

7. Beta Testing

8. Performance Testing

1. Unit Testing
Unit tests are typically written by developers as they write the code for a given unit.
They are usually written in the same programming language as the software and use a
testing framework or library that provides the necessary tools for creating and running
the tests. These frameworks often include assertion libraries, which allow developers to
write test cases that check the output of a given unit against expected results. The tests
are usually run automatically and continuously as part of the software build process,
and the results are typically displayed in a test runner or a continuous integration tool.

Unit testing has several benefits, including:

Early detection and isolation of defects, which can save time and money by allowing
developers to fix errors before they become more costly to fix.
Improved software quality and maintainability, as unit tests help to ensure that code
changes do not break existing functionality.
Increased developer confidence, as developers can make changes to the code with the
knowledge that any errors will be caught by the unit tests.
Facilitation of test-driven development, a software development methodology in which
tests are written before code is written, ensuring that code is written to meet the
requirements.
Overall, Unit testing is an essential part of software development that helps to ensure
the quality and reliability of the software, by identifying errors early on in the
development process.
2. Integration Testing

Integration testing is a software testing method in which individual units or components

of a software application are combined and tested as a group. The goal of integration
testing is to validate that the interactions between the units or components of the
software work as expected and that the software as a whole functions correctly.

Integration testing is typically performed after unit testing and before system testing. It
is usually done by developers and test engineers, and it is usually carried out at the
module level. Integration tests are typically automated and run frequently, as part of the
software build process, to ensure that the software remains stable and free of defects
over time.

Integration testing is done to verify that different components or modules of the

software work together as expected, and to identify and fix any issues that might arise
due to interactions between the modules. These tests can include testing different
combinations of inputs, testing how the software handles different types of data, and
testing how the software handles different error conditions.

Integration testing has several benefits, including:

Detection of defects that may not be discovered during unit testing, as it examines
the interactions between components
Improved system design, as integration testing can help identify design weaknesses
Improved software quality and reliability, as integration testing helps to ensure that
the software as a whole functions correctly.
Facilitation of continuous integration and delivery, as integration testing helps to
ensure that changes to the software do not break existing functionality
Overall, integration testing is an essential part of software development that helps to
ensure the quality and reliability of the software by identifying defects in the
interactions between the units and components of the software early on in the
development process.

3. Regression Testing

Regression testing is a software testing method in which previously developed and

tested software is retested after it has been modified or changed. The goal of regression
testing is to ensure that any changes to the software have not introduced new bugs or
broken existing functionality. It is typically done to verify that changes such as bug
fixes, new features, or updates to existing features have not affected the overall
functionality of the software.

Regression testing is typically performed after unit testing and integration testing. It is
usually done by developers and test engineers and it is usually carried out by re-running
a suite of previously passed test cases. The test cases are chosen to cover the areas of
the software that were affected by the changes, and to ensure that the most critical
functionality of the software is still working correctly. Regression testing is typically
automated and run frequently, as part of the software build process, to ensure that the
software remains stable and free of defects over time.

Regression testing has several benefits, including:

Early detection and isolation of defects, which can save time and money by allowing
developers to fix errors before they become more costly to fix.
Improved software quality and maintainability, as regression testing helps to ensure that
code changes do not break existing functionality
Increased developer and user confidence, as regression testing helps to ensure that the
software is still working correctly after changes have been made
Facilitation of continuous integration and delivery, as regression testing helps to ensure
that changes to the software can be safely released.
Overall, regression testing is an essential part of software development that helps to
ensure

4. Smoke Testing

Smoke testing, also known as “Build Verification Testing” or “Build Acceptance

Testing”, is a software testing method in which a minimal set of tests are run on a new
build of a software application to determine if it is stable enough to proceed with further
testing. The goal of smoke testing is to quickly identify and isolate major issues with
the software build, so that development can be halted if the build is found to be too
unstable or unreliable.

Smoke testing is typically performed early in the software testing process, after the
software has been built and before more extensive testing is done. It is usually done by
developers and test engineers and it is usually carried out by running a small set of
critical test cases that exercise the most important functionality of the software. Smoke
tests are usually automated and can be run as part of the software build process.

Smoke testing has several benefits, including:

Early identification of major issues, which can save time and money by allowing
developers to fix errors before they become more costly to fix.
Improved software quality and reliability, as smoke testing helps to ensure that the
software is stable enough to proceed with further testing
Facilitation of continuous integration and delivery, as smoke testing helps to ensure that
new builds of the software are stable and reliable before they are released.
Overall, smoke testing is an important part of software development that helps to ensure
the quality and reliability of the software by identifying major issues early on in the
development process. It helps to quickly determine if a new build of the software is
stable enough to proceed with further testing, providing increased confidence in the
software to the development team and end-users.

5. System Testing

System testing is a software testing method in which an entire software system is tested
as a whole, in order to ensure that it meets the requirements and specifications that it
was designed for. The goal of system testing is to validate that the software system
behaves as expected when it is used in its intended environment, and that it meets all
the requirements for functionality, performance, security, and usability.
System testing is typically performed after unit testing, integration testing, and
regression testing. It is usually done by test engineers and it is usually carried out by
running a set of test cases that cover all the functionality of the software. The test cases
are chosen to cover the requirements and specifications of the software, and to ensure
that the software behaves correctly under different conditions and scenarios. System
testing is typically automated and run frequently, as part of the software build process,
to ensure that the software remains stable and free of defects over time.

System testing has several benefits, including:

Early detection and isolation of defects, which can save time and money by allowing
developers to fix errors before they become more costly to fix.
Improved software quality and reliability, as system testing helps to ensure that the
software meets all the requirements and specifications that it was designed for.
Increased user confidence, as system testing helps to ensure that the software behaves
correctly when it is used in its intended environment.
Facilitation of acceptance testing, as system testing helps to ensure that the software is
ready for release.
Overall, system testing is an essential part of software development that helps to ensure
the quality and reliability of the software by identifying defects early on in the
development process. It helps to ensure that the software meets all the requirements and
specifications that it was designed for, providing increased confidence in the software
to the development team and end-users.