Software Testing (KCS076)

Unit - 4

Software Testing Activities: Levels of Testing, Debugging, Testing techniques and their
applicability, Exploratory Testing Automated Test Data Generation: Test Data, Approaches to test
data generation, test data generation using genetic algorithm, Test Data Generation Tools, Software
Testing Tools, and Software test Plan.

Test Process activities are:

1. Planning and Control

2. Analysis and Design
3. Implementation and Execution
4. Evaluating test exit criteria and Reporting
5. Test Closure activities
Planning :
Planning includes :

● Defining testing objectives

● Test activities specified to meet the objectives and goals
● Planning determines scope, risks, test approach, test strategy.
Control :

●Continuous ongoing activity to compare actual progress with plan

●Measure , analyse and report the status, including deviations from plan
●Initiate necessary actions to meet objectives
●You can’t control testing without monitoring.Planning takes into account
feedback from monitoring and control
Analysis and Design:
The requirements are transformed to test conditions and test cases.

Analysis and Design phase of testing includes:

●
●
● Review the test basis and understand the software specifications (such as
requirements, architecture, design and interface)
● Evaluate test-ability of requirements to specify expected results and make sure
requirements can be verified.
● Identify and prioritize Test conditions
● Design high level test cases and prioritize
● Identify test data
● Design test environment and identify any tool or infrastructure required for
testing the software.
● Create bi-directional traceability between test basis and test cases
Test Implementation and Execution:
All high level designing till this phase is now built in test cases and test procedures.

Test Implementation and Execution phase of testing includes:

●
●
● Develop and prioritize tests cases
● Develop test procedures , test suites( logical collection of test cases)
● Create test data
● Set up test environment and verify that it is ready for test execution
● Verify and update traceability
● Execute test procedures
● Log the outcome of each execution and compare actual with expected results
● Report discrepancies and provide additional information to establish if likely
cause is defect in code, test data, test document
● Retest after the defect is fixed (confirmation testing) and perform regression
testing
Evaluating test exit criteria and Reporting:
Based on the risk assessment for each test level, exit criteria is defined beforehand defining
when to stop testing.Evaluation of exit criteria comes when maximum test cases are executed
with specific pass percent or defect ratio below a certain level.

Evaluation exit criteria and Reporting phase of testing includes:

● Check test logs against exit criteria in test planning and identify what is
remaining to test or fix
● Assess if more tests are needed or if the initial exit criteria needs to be
modified
● Write a test summary report for stakeholders
Test Closure Activities:
Ending testing includes various activities to collect data from completed test activities and
consolidate experience, test ware, facts and numbers.

Levels of Testing

1. Unit Testing
2. Integration Testing
3. System Testing
4. Acceptance Testing

Debugging
Debugging is the process of identifying and resolving errors, or bugs, in a software system. It
is an important aspect of software engineering because bugs can cause a software system to
malfunction, and can lead to poor performance or incorrect results. Debugging can be a
time-consuming and complex task, but it is essential for ensuring that a software system is
functioning correctly.
There are several common methods and techniques used in debugging, including:
1. Code Inspection: This involves manually reviewing the source code of a software
system to identify potential bugs or errors.
2. Debugging Tools: There are various tools available for debugging such as debuggers,
trace tools, and profilers that can be used to identify and resolve bugs.
3. Unit Testing: This involves testing individual units or components of a software
system to identify bugs or errors.
4. Integration Testing: This involves testing the interactions between different
components of a software system to identify bugs or errors.
5. System Testing: This involves testing the entire software system to identify bugs or
errors.
6. Monitoring: This involves monitoring a software system for unusual behavior or
performance issues that can indicate the presence of bugs or errors.
7. Logging: This involves recording events and messages related to the software system,
which can be used to identify bugs or errors.

Debugging Process: Steps involved in debugging are:

● Problem identification and report preparation.
● Assigning the report to the software engineer defect to verify that it is genuine.
● Defect Analysis using modeling, documentation, finding and testing candidate flaws,
etc.
● Defect Resolution by making required changes to the system.
● Validation of corrections.
The debugging process will always have one of two outcomes:
1. The cause will be found and corrected.
2. The cause will not be found.

Steps involved in Debugging

Following are the different steps that are involved in debugging:
1. Identify the Error: Identifying an error in a wrong may result in the wastage of time.
It is very obvious that the production errors reported by users are hard to interpret, and
sometimes the information we receive is misleading. Thus, it is mandatory to identify
the actual error.
2. Find the Error Location: Once the error is correctly discovered, you will be required
to thoroughly review the code repeatedly to locate the position of the error. In general,
this step focuses on finding the error rather than perceiving it.
3. Analyze the Error: The third step comprises error analysis, a bottom-up approach
that starts from the location of the error followed by analyzing the code. This step
makes it easier to comprehend the errors. Mainly error analysis has two significant
goals, i.e., evaluation of errors all over again to find existing bugs and postulating the
uncertainty of incoming collateral damage in a fix.
4. Prove the Analysis: After analyzing the primary bugs, it is necessary to look for
some extra errors that may show up on the application. By incorporating the test
framework, the fourth step is used to write automated tests for such areas.
5. Cover Lateral Damage: The fifth phase is about accumulating all of the unit tests for
the code that requires modification. As when you run these unit tests, they must pass.
6. Fix & Validate: The last stage is the fix and validation that emphasizes fixing the
bugs followed by running all the test scripts to check whether they pass.
 Testing techniques and their applicability
Software testing techniques are methods used to design and execute tests to evaluate software
applications. The following are common testing techniques:
1. Manual testing – Involves manual inspection and testing of the software by a human
tester.
2. Automated testing – Involves using software tools to automate the testing process.
3. Functional testing – Tests the functional requirements of the software to ensure they are
met.
4. Non-functional testing – Tests non-functional requirements such as performance,
security, and usability.
5. Unit testing – Tests individual units or components of the software to ensure they are
functioning as intended.
6. Integration testing – Tests the integration of different components of the software to
ensure they work together as a system.
7. System testing – Tests the complete software system to ensure it meets the specified
requirements.
8. Acceptance testing – Tests the software to ensure it meets the customer’s or end-user’s
expectations.
9. Regression testing – Tests the software after changes or modifications have been made to
ensure the changes have not introduced new defects.
10. Performance testing – Tests the software to determine its performance characteristics
such as speed, scalability, and stability.
11. Security testing – Tests the software to identify vulnerabilities and ensure it meets
security requirements.
12. Exploratory testing – A type of testing where the tester actively explores the software to
find defects, without following a specific test plan.
13. Boundary value testing – Tests the software at the boundaries of input values to identify
any defects.
14. Usability testing – Tests the software to evaluate its user-friendliness and ease of use.
15. User acceptance testing (UAT) – Tests the software to determine if it meets the
end-user’s needs and expectations.

Types Of Software Testing Techniques

There are two main categories of software testing techniques:
Static Testing Techniques
As explained earlier, Static Testing techniques are testing techniques that do not require the
execution of a code base. Static Testing Techniques are divided into two major categories:
1. Reviews: They can range from purely informal peer reviews between two
developers/testers on the artifacts (code/test cases/test data) to
formal Inspections which are led by moderators who can be internal/external to the
organization.
1. Peer Reviews: Informal reviews are generally conducted without any formal
setup. It is between peers. For Example- Two developers/Testers review each
other’s artifacts like code/test cases.
2. Walkthroughs: Walkthrough is a category where the author of work (code or
test case or document under review) walks through what he/she has done and
the logic behind it to the stakeholders to achieve a common understanding or
for the intent of feedback.
3. Technical review: It is a review meeting that focuses solely on the technical
aspects of the document under review to achieve a consensus. It has less or no
focus on the identification of defects based on reference documentation.
Technical experts like architects/chief designers are required to do the review.
It can vary from Informal to fully formal.
4. Inspection: Inspection is the most formal category of reviews. Before the
inspection, the document under review is thoroughly prepared before going for
an inspection. Defects that are identified in the Inspection meeting are logged
in the defect management tool and followed up until closure. The discussion
 on defects is avoided and a separate discussion phase is used for discussions,
which makes Inspections a very effective form of review.
2. Static Analysis: Static Analysis is an examination of requirement/code or design to
identify defects that may or may not cause failures. For Example- Review the code for
the following standards. Not following a standard is a defect that may or may not
cause a failure. Many tools for Static Analysis are mainly used by developers before
or during Component or Integration Testing. Even Compiler is a Static Analysis tool
as it points out incorrect usage of syntax, and it does not execute the code per se.
There are several aspects to the code structure – Namely Data flow, Control flow, and
Data Structure.
1. Data Flow: It means how the data trail is followed in a given program – How
data gets accessed and modified as per the instructions in the program. By
Data flow analysis, you can identify defects like a variable definition that
never got used.
2. Control flow: It is the structure of how program instructions get executed i.e.
conditions, iterations, or loops. Control flow analysis helps to identify defects
such as Dead code i.e. a code that never gets used under any condition.
3. Data Structure: It refers to the organization of data irrespective of code. The
complexity of data structures adds to the complexity of code. Thus, it provides
information on how to test the control flow and data flow in a given code.
Dynamic Testing Techniques
Dynamic techniques are subdivided into three categories:
1. Structure-based Testing:
These are also called White box techniques. Structure-based testing techniques are focused
on how the code structure works and test accordingly. To understand Structure-based
techniques, we first need to understand the concept of code coverage.
Code Coverage is normally done in Component and Integration Testing. It establishes what
code is covered by structural testing techniques out of the total code written. One drawback
of code coverage is that- it does not talk about code that has not been written at all (Missed
requirement), There are tools in the market that can help measure code coverage.
There are multiple ways to test code coverage:
1. Statement coverage: Number of Statements of code exercised/Total number of
statements. For Example, if a code segment has 10 lines and the test designed by you covers
only 5 of them then we can say that statement coverage given by the test is 50%.
2. Decision coverage: Number of decision outcomes exercised/Total number of Decisions.
For Example, If a code segment has 4 decisions (If conditions) and your test executes just 1,
then decision coverage is 25%
3. Conditional/Multiple condition coverage: It has the aim to identify that each outcome of
every logical condition in a program has been exercised.
2. Experience-Based Techniques:
These are techniques for executing testing activities with the help of experience gained over
the years. Domain skill and background are major contributors to this type of testing. These
techniques are used majorly for UAT/business user testing. These work on top of structured
techniques like Specification-based and Structure-based, and they complement them. Here
are the types of experience-based techniques:
1. Error guessing: It is used by a tester who has either very good experience in testing or
with the application under test and hence they may know where a system might have a
weakness. It cannot be an effective technique when used stand-alone but is helpful when used
along with structured techniques.
2. Exploratory testing: It is hands-on testing where the aim is to have maximum execution
coverage with minimal planning. The test design and execution are carried out in parallel
without documenting the test design steps. The key aspect of this type of testing is the tester’s
learning about the strengths and weaknesses of an application under test. Similar to error
guessing, it is used along with other formal techniques to be useful.
3. Specification-based Techniques:
This includes both functional and non-functional techniques (i.e. quality characteristics). It
means creating and executing tests based on functional or non-functional specifications from
the business. Its focus is on identifying defects corresponding to given specifications. Here
are the types of specification-based techniques:
1. Equivalence partitioning: It is generally used together and can be applied to any level of
testing. The idea is to partition the input range of data into valid and non-valid sections such
that one partition is considered “equivalent”. Once we have the partitions identified, it only
requires us to test with any value in a given partition assuming that all values in the partition
will behave the same. For example, if the input field takes the value between 1-999, then
values between 1-999 will yield similar results, and we need NOT test with each value to call
the testing complete.
2. Boundary Value Analysis (BVA): This analysis tests the boundaries of the range- both
valid and invalid. In the example above, 0,1,999, and 1000 are boundaries that can be tested.
The reasoning behind this kind of testing is that more often than not, boundaries are not
handled gracefully in the code.
3. Decision Tables: These are a good way to test the combination of inputs. It is also called
a Cause-Effect table. In layman’s language, one can structure the conditions applicable for the
application segment under test as a table and identify the outcomes against each one of them
to reach an effective test.
1. It should be taken into consideration that there are not too many combinations so the
table becomes too big to be effective.
2. Take an example of a Credit Card that is issued if both credit score and salary limit
are met. This can be illustrated in below decision table below:
Decision Table
4. Use case-based Testing: This technique helps us to identify test cases that execute the
system as a whole- like an actual user (Actor), transaction by transaction. Use cases are a
sequence of steps that describe the interaction between the Actor and the system. They are
always defined in the language of the Actor, not the system. This testing is most effective in
identifying integration defects. Use case also defines any preconditions and postconditions of
the process flow. ATM example can be tested via use case:

Use case-based Testing

5. State Transition Testing: It is used where an application under test or a part of it can be
treated as FSM or finite state machine. Continuing the simplified ATM example above, We
can say that ATM flow has finite states and hence can be tested with the State transition
technique. There are 4 basic things to consider –
1. States a system can achieve
2. Events that cause the change of state
3. The transition from one state to another
4. Outcomes of change of state
A state event pair table can be created to derive test conditions – both positive and negative.

Exploratory Testing Automated Test Data Generation

Test Data –
Test data is an integral part of the testing process for a tester. It provides information to the
tester to facilitate finding defects and corrective actions. Test engineers load the application
with data or stress it with huge amounts of invalid data to check breakpoints and other aspects
of the application's performance. While executing test cases, test engineers need to input
some data into the application to get the expected output.
Test data is a production-like set of data used by test cases to determine whether an
application is working correctly. Test data is usually collected into a document called a test
data document that helps organize it so testers can easily access it when they run their tests.
Testing the product with test data is essential when designing a new application. This can
help determine if a product needs additional development or if it's ready to move on to further
testing. Developers can identify coding errors by testing preliminary data before completing
productivity and efficiency tests.
Test data have the following types:
 ● Valid Test Data: Valid test data is the positive data used for system testing. This
includes all possible inputs and conditions so that the application will run smoothly in
different situations.
● Invalid Test Data: Invalid Test Data is the negative data used to verify that a program
will handle negative conditions and exceptions correctly. Some examples are:
● Null values can be used in the case of mandatory fields.
● Out of Range Values.
● Special characters are not allowed.
● Invalid data format, such as a mobile phone number with alphabetical
characters in place of numbers, causes errors.
● No Data: No data helps ensure the system will work properly if a user leaves a field
blank.
● Boundary Data: It is a pair of test data values that indicate whether an observed
value falls within or outside the range of expectations.
Good test data combines valid and invalid data, including all positive and negative test
scenarios.
Approaches to test data generation
Managing large volumes of data generated while testing all significant test cases can take
time for testers.
A test data generator is a software tool that helps in software testing by generating mock data.
The test data generation process involves collecting, managing, and maintaining a large
quantity of data from various sources to implement test cases that ensure the functional
soundness of a system. The generated data can be random or specific to the desired results.
A test data generator can be used to create structured or unstructured data. Structured data is
generally more helpful for databases because these systems often save data in specific tables,
columns, and types of information. In contrast, random data is not suitable for this purpose.
There are different test data generators:
● Random Test Data Generator: Random test data generator is the most
straightforward data generator. It can be used to test many programs by having it
output random bit streams and have those bit streams represent the required data
types.
● Goal-Oriented Generator: This generator generates input for any path specified
instead of just the usual way of generating information from entry to exit. It generates
all possible combinations of good moves and has little chance of generating infeasible
paths.
● Pathwise Test Data Generator: Pathwise test data generator follows a fixed path
instead of giving it a choice among many paths. This leads to greater path knowledge,
which provides better coverage prediction. It is similar to the goal-oriented generator.
 ● Intelligent Test Data Generator: Intelligent test data generators rely on sophisticated
analysis of the code to be tested to guide their search for test data. This approach
requires significant insight into the various situations that may arise so that it can
generate appropriate test data quickly.
There are some majorly used techniques that are commonly used to generate the
datasets:
1) Manual Test data generation:
In this technique, all the datasets are generated manually by the tester with respect to
all the required test case through experience and anticipations.
Pros:
1. Easy to implement, no additional tools are needed to be deployed.
2. Increase the confidence of the tester.
Cons:
1. Accuracy of data sets generated by this scheme mostly doubtful.
2. Time-consuming process.

2) Automated Test Data Generation:

The major feature of this testing that makes it more efficient than the above technique
is the speed, automated data generation technique produces data as in an expedited
manner through analyzing large volume of data in a small-time interval. In this
scheme, we use automated tools, there are many available in the market.
Pros:
1. The data sets generated by this scheme are highly accurate.
2. Data generation speed is very fast.
Cons:
1. The one demerit of this method is that it is a costlier method to implement.
2. The second one is that these tools take time to understand the system.

3) Back end data injection Approach:

This method is done with the help of using SQL queries. Here a tester writes the
relevant query and injects it into the database in order to populate the required data
sets with respect to the test cases. This is also an easier method which generates a
large amount of data in just a few minutes. We can update the database in this scheme
if some new datasets are found through other resources like sample XML documents
etc could be updated for future use if required.
Pros:
1. It is less time-consuming technique.
2. Less expertise required as compared to the above technique as you only need to write
a correct query to populate data required.
Cons:
1. If you write any invalid query or incorrect it may populate illogical dataset or may
cause the failure of your database system so keep attention while injecting any query
into database.
4) Third-party tool:

A number of tools are available in the market that is processed or provided by the out
premises tools. These tools first understand the scenarios of your system under testing
and then generates dataset as per the requirement. These tools are customizable as per
your need of the business. These tools provide wide coverage and accuracy in
generating datasets.
Pros:
1. These tools are accurate because they first understand the entire system and then
generated the datasets accordingly.
Cons:
1. Costlier technique to implement because the price of such a tool is high as compared
to other technique.
2. Less coverage in case of heterogeneous testing environment because these tools aren’t
generic in nature.

Software Testing Tools

Software Testing tools are the tools that are used for the testing of software.
Software testing tools are often used to assure firmness, thoroughness, and
performance in testing software products. Unit testing and subsequent
integration testing can be performed by software testing tools. These tools are
used to fulfill all the requirements of planned testing activities. These tools also
work as commercial software testing tools. The quality of the software is
evaluated by software testers with the help of various testing tools.
Types of Testing Tools
Software testing is of two types, static testing, and dynamic testing. Also, the
tools used during these testing are named accordingly on these testings. Testing
tools can be categorized into two types which are as follows:
1. Static Test Tools: Static test tools are used to work on the static testing
processes. In the testing through these tools, the typical approach is taken. These
tools do not test the real execution of the software. Certain input and output are
not required in these tools. Static test tools consist of the following:
● Flow analyzers: Flow analyzers provides flexibility in the data flow from
input to output.
● Path Tests: It finds the not used code and code with inconsistency in the
software.
● Coverage Analyzers: All rationale paths in the software are assured by
the coverage analyzers.
● Interface Analyzers: They check out the consequences of passing
variables and data in the modules.
2. Dynamic Test Tools: Dynamic testing process is performed by the dynamic
test tools. These tools test the software with existing or current data. Dynamic
test tools comprise the following:
● Test driver: The test driver provides the input data to a
module-under-test (MUT).
● Test Beds: It displays source code along with the program under
execution at the same time.
● Emulators: Emulators provide the response facilities which are used to
imitate parts of the system not yet developed.
● Mutation Analyzers: They are used for testing the fault tolerance of the
system by knowingly providing the errors in the code of the software.
Top 10 Software Testing Tools
1. TestComplete: TestComplete developed by SmartBear Software is a
functional automated testing tool that ensures the quality of the application
without sacrificing quality or agility.
Features:
● TestComplete has built-in keyword-driven test editor that consists of
keyword operations that correspond to automated testing actions.
● It records the key actions that are necessary to replay test and discard all
unneeded actions.
● It can run several automated tests across separate virtual machines.
● It has built-in code editor that helps testers write scripts manually.
 ● It automatically captures screenshots during test recording and playback.
2. LambdaTest: LambdaTest is a cross-browser testing tool that helps to
evaluate how web application responds when accessed through a variety of
different browsers.
Features:
● It has Selenium scripts on 3000+ browsers and operating system
environments, giving higher test coverage.
● It can perform automated cross-browser testing of locally hosted web
pages using LambdaTest tunnel.
● It can also help to run a single test across multiple browser/ OS
configurations simultaneously.
3. TestRail: TestRail is a test management tool that helps to streamline software
testing processes, get visibility into QA. This tool is used by testers, developers,
and team leads to manage, track, and organize software testing efforts.
Features:
● It helps to manage test cases, plans, and runs.
● It helps to increase test coverage.
● It helps to get real-time insights into your QA progress.
● It helps to document test plans and track real-time progress.
4. Xray: Xray is a test management app for Jira that helps to plan, execute, and
track quality assurance with requirements traceability.
Features:
● It promotes Native Quality Management, where all tools, tests used by
QA are built natively into development environment like Jira.
● It integrates with leading automation frameworks like Cucumber,
Selenium, and JUnit to automate testing.
● It allows easy integration with CI tools like Jenkins, Bamboo, and
GitLab.
● It helps to easily map stories using BDD.
5. Zephyr Scale: Zephyr Scale is a test management provides a smarter and
more structured way to plan, manage, and measure tests inside Jira.
Features:
● It offers cross-project integration, traceability, and a structured designed
useful in large environments.
● It helps to scale tests in Jira.
● It helps to improve visibility, data analysis, and collaboration.
● It provides detailed changed history, test case versioning, and end-to-end
traceability with Jira issues and challenges.
6. Selenium: Selenium provides a playback tool for authoring tests across most
web browsers without the need to learn a test scripting language.
Features:
● It provides multi-browser support.
● It makes it easy to identify web elements on the web apps with the help of
its several locators.
● It is able to execute test cases quicker than the other tools.
7. Ranorex: Ranorex Studio is a GUI test automation framework used for
testing web-based, desktop, and mobile applications. It does not have its own
scripting language to automate application.
Features:
● It helps to automate tests on Windows desktop, then execute locally or
remotely on real or virtual machines.
● It runs tests in parallel to accelerate cross-browser testing for Chrome,
Firefox, Safari, etc.
● It tests on real iOS or Android devices, simulators, emulators, etc.
8. TestProject: TestProject is a test automation tool that allows users to create
automated tests for mobile and web applications. It is built on top of popular
frameworks like Selenium and Appium.
Features:
● It is a free end-to-end test automation platform for web, mobile, and API
testing.
● Tests are saved as local files directly on your machine with no
cloud-footprint to get a complete offline experience.
 ● It helps to create reliable codeless tests powered by self-healing, adaptive
wait, and community add-ons.
● It provides insights about release quality, step-by-step detailed report with
screenshots and logs.
9. Katalon Platform: Katalon Platform is a comprehensive quality
management platform that enables team to easily and efficiently test, launch,
and optimize the best digital experiences.
Features:
● It is designed to create and reuse automated test scripts for UI without
coding.
● It allows running automated tests of UI elements including pop-ups,
iFrames, and wait-time.
● It eases deployment and allows wider set of integrations compared to
Selenium.
10. UFT/QTP: Micro Focus UFT is a software that provides functional and
regression tests automation for software applications and environments.
Features:
● It helps to accelerate end-to-end testing.
● It boasts AI-based machine learning and advanced OCR for advanced
object recognition.
● It helps to test both front-end functionality and back-end service parts.