Unit-4

STATIC ANALYSIS:
What is Static Analysis in Software Testing?
 Static analysis involve software under test to detect possible defects before running the
program.
 During testing software is not in execution mode.
 Static analysis is done after coding.
 Static analysis can be performed by a person or a machine.
 In Static analysis a “walk through” is made through source code to detect:
 Noncomplying rules. (For example compiler which find lexical, syntactic, semantic
mistakes).
 To ensure proper coding standards are used to construct program.
Static analysis forced developers not to used risky or buggy parts of the programming
language.
What developers looks in Static Analysis in Software Testing?
 Lines of Code
 Proper nesting.
 Comment frequency
 Number of function calls
 Cyclomatic complexity
 Check of unit tests
Which quality attributes are focus in Static analysis?
 Reliability
 Maintainability
 Testability
 Re-usability
 Portability
What are the Advantages of Static analysis ?
 It can find defects in coding at exact location in source code.
 It is easy to understand source code.
 It allows fast defects fixing.
 Defects can be found in starting of the development of software which reduces overall costing.
 After Static analysis testing future test lest gives less defects compare to present test.
 Some defects are easy to detect in Static analysis only, like:
 Uncalled functions
 Undeclared, unused variables
  Unreachable code
What are the Disadvantages of Static analysis ?
 Time consuming if conducted manually (By Humans not machines).
 Static analysis automated tools are good for code scan only.
 Static analysis automated tools may produce false positive and negative.
 Static analysis not work when software is in execution.
DYNAMIC ANALYSIS:
What is Dynamic Analysis in Software testing?
 Dynamic analysis is performed when software /coding is in execution.
 Dynamic analysis often uses tools to perform testing.
 Mostly unit testing is performed in Dynamic Testing.
What are the advantages of Dynamic analysis ?
 It can be performed for any application.
 It finds defects in runtime environment.
 It allows application analysis without accessing its source code.
 Use of it validates Static analysis findings.
 It finds defects which were not included in Static Analysis.
What are the disadvantages of Dynamic Analysis?
 Cannot guarantee that full source code is covered under testing.
 Automated tools not ensure about security in testing.
 Dynamic analysis automated tools may produce false positive and negative.
 Dynamic analysis automated tools are good for code scan only.
 Time consuming to fix the problem.
 Difficult to find the exact location of defects.

What Is Code Inspection in Software Engineering?

Software engineers program code to empower a technology product to perform certain tasks.
Code inspection is a verification technique that evaluates the precision of the code within the
product. Comprehending the importance and steps of code inspection can allow you to build a
quality software product that satisfies the client and engages consumers. In this article, we
define code inspection in software engineering, explain how it works and discuss its
advantages and disadvantages.
What is code inspection in software engineering?
 Code inspection in software engineering is the process of reviewing the code in an application
to check for defects. Its purpose is to correct the issues in the programming language so the
software performs at its highest potential. After the engineers build the product and write the
code, they practice code inspection to find ways to minimize the time for the code to exercise
 commands. Resolving the issues that the inspection finds allows engineers to refine the
internal structure of the software, as well as its security features.
 Why do software engineers use code inspection?
 Software engineers use code inspection to ensure the application performs efficiently. If
there's an issue with the program's functioning, for example, and the code exercises a different
command than intended, the engineers can discern where the language faults and how to fix it.
Building a software product can be a time-consuming process, and the client may have certain
expectations of the final product's performance. A successful code inspection can allow peer
reviews of the engineers' work, and they can gain confidence that they vetted their project
thoroughly and confirmed that it meets the standards of the industry.
 Code verification methods
 Code verification illuminates mistakes in the programming language before the engineers
submit the final product to the client. There are two types of methods, which include:
 Dynamic testing: Dynamic testing executes the code to validate if the software is high-
functioning. The execution illustrates how the application performs as a whole and the
efficiency of its memory system.
 Static testing: Static testing reviews the code manually without executing it. It occurs at the
beginning of the Software Development Life Cycle (SDLC) to correct issues so that they don't
interfere with the quality of the product as it continues to evolve.
 How does code inspection work?
 The process of code inspection entails five steps. They include:
 1. Appoint a team to inspect the code
 An inspection team is a group of professionals who participate in the evaluation of the code.
Their combined efforts ensure that the results of the inspection are impartial and the engineers
implement the feedback correctly. There are four members of the inspection team:
 Author: The author is the original writer of the code. They also helped build the software
application where the code originates.
 Moderator: The moderator spearheads the code inspection, a different individual from the
author.
 Reader: The reader reads the code aloud, which allows the other team members to identify
errors.
 Recorder: The recorder, or scribe, compiles the steps of the code inspection and the defects
the team indicates in a report.
 2. Explain the purpose of the software application
 The second step of code inspection is to prepare for the code analysis. It's essential that the
inspector team understands what the engineers meant to accomplish when preparing the
software program. The author of the code explains the objectives of the project, offering
insight into how they intended the code to look and function. The other inspectors review
documents that include metrics of a correct code, which can enable them to compare the
language to its ideal formation and catch discrepancies.
 3. Complete every item on the code inspection checklist
 The third step entails the actual code inspection, where the team adheres to the steps that the
moderator created. The reader reviews the code in detail for the inspectors to hear, and the
team acknowledges mistakes. Next, the recorder scribes the errors the team finds in a
document for further reference.
 The report includes the strengths and weaknesses of the code, as well as how the engineers
can correct it. The thoroughness of the inspection requires professionals to inspect each part of
the code separately and signify errors within each part. Next, the team uses the checklist to
track their progress and provide comprehensive results.
 4. Apply the feedback from the findings of the inspection
 Once the author learns of the defects, they can change the language to boost the software's
performance. The team may establish deadlines for the engineers to incorporate the edits to
the code, which allows the project to finish promptly and meet the client's deadlines. The
author references the report to determine where the errors occurred and how to fix the
language, following instructions that the rest of the team proposed.
 5. Host a follow-up meeting to check the corrections
 The final step of the code inspection is to monitor the success of the evaluation. The
moderator orchestrates a meeting to analyze the corrections the author made to the code. They
compare the author's work to the directions that the inspection report mandated. It's also the
moderator's responsibility to design exit protocols, which dictate how to approve the edited
code. Once the code fulfills the moderator's expectations, the inspection team declares the
engineers completed the verification properly.

 Benefits of code inspection
 The advantages of performing code inspection on a software program include:
 Engineers can enhance the quality of the product. Implementing the corrections to the
software can boost how the application works. The better the quality, the more the product
may resonate with consumers.
 Reviewing the code easily identifies issues. Analyzing every part of the language aloud can
help the inspectors find errors effectively. The engineers can also resolve the issues during the
development phase, saving them the time and resources of providing updates after the product
launch.
 The process can prevent future errors. Engineers can examine how their original
configurations lead to defects, which allows them to use the corrected code for future software
development projects. The meeting with the inspection team may give them the opportunity to
ask questions about their languages and how to improve their approach.


What is Software Testing

Software testing is a process of identifying the correctness of software by considering its all
attributes (Reliability, Scalability, Portability, Re-usability, Usability) and evaluating the
execution of software components to find the software bugs or errors or defects. Backward Skip
10sPlay 10s
Software testing provides an independent view and objective of the software and gives surety of
fitness of the software. It involves testing of all components under the required services to
confirm that whether it is satisfying the specified requirements or not. The process is also
providing the client with information about the quality of the software.

Testing is mandatory because it will be a dangerous situation if the software fails any of time due
to lack of testing. So, without testing software cannot be deployed to the end user.

What is Testing

Testing is a group of techniques to determine the correctness of the application under the
predefined script but, testing cannot find all the defect of application. The main intent of testing
is to detect failures of the application so that failures can be discovered and corrected. It does not
demonstrate that a product functions properly under all conditions but only that it is not working
in some specific conditions.

Testing furnishes comparison that compares the behavior and state of software against
mechanisms because the problem can be recognized by the mechanism. The mechanism may
include past versions of the same specified product, comparable products, and interfaces of
expected purpose, relevant standards, or other criteria but not limited up to these.

Testing includes an examination of code and also the execution of code in various environments,
conditions as well as all the examining aspects of the code. In the current scenario of software
development, a testing team may be separate from the development team so that Information
derived from testing can be used to correct the process of software development.
The success of software depends upon acceptance of its targeted audience, easy graphical user
interface, strong functionality load test, etc. For example, the audience of banking is totally
different from the audience of a video game. Therefore, when an organization develops a
software product, it can assess whether the software product will be beneficial to its purchasers
and other audience.

Type of Software testing

We have various types of testing available in the market, which are used to test the application or
the software.

With the help of below image, we can easily understand the type of software testing:

Manual testing

The process of checking the functionality of an application as per the customer needs without
taking any help of automation tools is known as manual testing. While performing the manual
testing on any application, we do not need any specific knowledge of any testing tool, rather than
have a proper understanding of the product so we can easily prepare the test document.

Manual testing can be further divided into three types of testing, which are as follows:

o White box testing

 o Black box testing

o Gray box testing

Automation testing

Automation testing is a process of converting any manual test cases into the test scripts with the
help of automation tools, or any programming language is known as automation testing. With the
help of automation testing, we can enhance the speed of our test execution because here, we do
not require any human efforts. We need to write a test script and execute those scripts.

Software Testing Life Cycle (STLC)

The procedure of software testing is also known as STLC (Software Testing Life Cycle) which
includes phases of the testing process. The testing process is executed in a well-planned and
systematic manner. All activities are done to improve the quality of the software product.

Let's see, the different steps of STLC.

Software testing life cycle contains the following steps:

1. Requirement Analysis
2. Test Plan Creation
3. Environment setup
4. Test case Execution
5. Defect Logging
6. Test Cycle Closure
Requirement Analysis:

The first step of the manual testing procedure is requirement analysis. In this phase, tester
analyses requirement document of SDLC (Software Development Life Cycle) to examine
requirements stated by the client. After examining the requirements, the tester makes a test plan
to check whether the software is meeting the requirements or not.

Entry Criteria Activities Deliverable

For the planning of test plan Prepare the list of all requirements and List of all the
requirement specification, queries, and get resolved from Technical necessary
application architecture document Manager/Lead, System Architecture, tests for the
and well-defined acceptance criteria Business Analyst and Client. testable
should be available. Make a list of all types of tests (Performance, requirements
Functional and security) to be performed. andTest
Make a list of test environment details, which environment
should contain all the necessary tools to details
execute test cases.

Test Plan Creation:

Test plan creation is the crucial phase of STLC where all the testing strategies are defined. Tester
determines the estimated effort and cost of the entire project. This phase takes place after the
successful completion of the Requirement Analysis Phase. Testing strategy and effort
estimation documents provided by this phase. Test case execution can be started after the
successful completion of Test Plan Creation.

Entry Criteria Activities Deliverable

Requirement Define Objective as well as the scope of the Test strategy document.
Document software. Testing Effort estimation
List down methods involved in testing. documents are the deliverables of
Overview of the testing process. this phase.
Settlement of testing environment.
Preparation of the test schedules and control
procedures.
Determination of roles and responsibilities.
List down testing deliverables, define risk if
any.

Environment setup:
Setup of the test environment is an independent activity and can be started along with Test Case
Development. This is an essential part of the manual testing procedure as without environment
testing is not possible. Environment setup requires a group of essential software and hardware to
create a test environment. The testing team is not involved in setting up the testing environment,
its senior developers who create it.

Entry Criteria Activities Deliverab

le

Test strategy and Prepare the list of software Execution

test plan and hardware by analyzing report.
document. requirement specification. Defect
Test case After the setup of the test report.
document. environment, execute the
Testing data. smoke test cases to check
the readiness of the test
environment.

Test case Execution:

Test case Execution takes place after the successful completion of test planning. In this phase,
the testing team starts case development and execution activity. The testing team writes down the
detailed test cases, also prepares the test data if required. The prepared test cases are reviewed by
peer members of the team or Quality Assurance leader.

RTM (Requirement Traceability Matrix) is also prepared in this phase. Requirement Traceability
Matrix is industry level format, used for tracking requirements. Each test case is mapped with the
requirement specification. Backward & forward traceability can be done via RTM.

Entry Criteria Activities Deliverable

Requirement Creation of test cases. Test execution result.

Document Execution of test cases. List of functions with the
Mapping of test cases according to detailed explanation of defects.
requirements.

Defect Logging:

Testers and developers evaluate the completion criteria of the software based on test coverage,
quality, time consumption, cost, and critical business objectives. This phase determines the
characteristics and drawbacks of the software. Test cases and bug reports are analyzed in depth
to detect the type of defect and its severity.
Defect logging analysis mainly works to find out defect distribution depending upon severity and
types.If any defect is detected, then the software is returned to the development team to fix the
defect, then the software is re-tested on all aspects of the testing.

Once the test cycle is fully completed then test closure report, and test metrics are prepared.

Entry Criteria Activities Deliverable

Test case execution It evaluates the completion criteria of the software Closure
report. based on test coverage, quality, time consumption, cost, report
Defect report and critical business objectives. Test metrics
Defect logging analysis finds out defect distribution by
categorizing in types and severity.

Test Cycle Closure:

The test cycle closure report includes all the documentation related to software design,
development, testing results, and defect reports.

This phase evaluates the strategy of development, testing procedure, possible defects in order to
use these practices in the future if there is a software with the same specification.

Entry Criteria Activities Deliverable

All document and reports Evaluates the strategy of development, testing Test closure
related to software. procedure, possible defects to use these practices in report
the future if there is a software with the same
specification

Types of Software Testing

In this section, we are going to understand the various types of software
testing, which can be used at the time of the Software Development Life Cycle.

As we know, software testing is a process of analyzing an application's

functionality as per the customer prerequisite.

If we want to ensure that our software is bug-free or stable, we must perform

the various types of software testing because testing is the only method that
makes our application bug-free.
The different types of Software Testing
The categorization of software testing is a part of diverse testing activities,
such as test strategy, test deliverables, a defined test objective, etc. And
software testing is the execution of the software to find defects.

The purpose of having a testing type is to confirm the AUT (Application Under
Test).
To start testing, we should have a requirement, application-ready,
necessary resources available. To maintain accountability, we should assign a
respective module to different test engineers.

The software testing mainly divided into two parts, which are as follows:

o Manual Testing
o Automation Testing

What is Manual Testing?

Testing any software or an application according to the client's needs without
using any automation tool is known as manual testing.

In other words, we can say that it is a procedure of verification and

validation. Manual testing is used to verify the behavior of an application or
software in contradiction of requirements specification.

We do not require any precise knowledge of any testing tool to execute the
manual test cases. We can easily prepare the test document while performing
manual testing on any application.

Classification of Manual Testing

In software testing, manual testing can be further classified into three
different types of testing, which are as follows:

o White Box Testing

o Black Box Testing
o Grey Box Testing

For our better understanding let's see them one by one:

White Box Testing

In white-box testing, the developer will inspect every line of code before
handing it over to the testing team or the concerned test engineers.

Subsequently, the code is noticeable for developers throughout testing; that's

why this process is known as WBT (White Box Testing).

In other words, we can say that the developer will execute the complete
white-box testing for the particular software and send the specific application
to the testing team.

The purpose of implementing the white box testing is to emphasize the flow
of inputs and outputs over the software and enhance the security of an
application.
White box testing is also known as open box testing, glass box testing,
structural testing, clear box testing, and transparent box testing.

Black Box Testing

Another type of manual testing is black-box testing. In this testing, the test
engineer will analyze the software against requirements, identify the defects or
bug, and sends it back to the development team.

Then, the developers will fix those defects, do one round of White box testing,
and send it to the testing team.

Here, fixing the bugs means the defect is resolved, and the particular feature is
working according to the given requirement.

The main objective of implementing the black box testing is to specify the
business needs or the customer's requirements.

In other words, we can say that black box testing is a process of checking the
functionality of an application as per the customer requirement. The source
code is not visible in this testing; that's why it is known as black-box testing.
.

Types of Black Box Testing

Black box testing further categorizes into two parts, which are as discussed
below:

o Functional Testing
o Non-function Testing

Functional Testing
The test engineer will check all the components systematically against
requirement specifications is known as functional testing. Functional testing
is also known as Component testing.

In functional testing, all the components are tested by giving the value,
defining the output, and validating the actual output with the expected value.

Functional testing is a part of black-box testing as its emphases on application

requirement rather than actual code. The test engineer has to test only the
program instead of the system.

Types of Functional Testing

Just like another type of testing is divided into several parts, functional testing
is also classified into various categories.

The diverse types of Functional Testing contain the following:

o Unit Testing
o Integration Testing
o System Testing
Now, Let's understand them one by one:

1. Unit Testing
Unit testing is the first level of functional testing in order to test any software.
In this, the test engineer will test the module of an application independently
or test all the module functionality is called unit testing.

The primary objective of executing the unit testing is to confirm the unit
components with their performance. Here, a unit is defined as a single testable
function of software or an application. And it is verified throughout the
specified application development phase.

2. Integration Testing
Once we are successfully implementing the unit testing, we will go integration
testing. It is the second level of functional testing, where we test the data flow
between dependent modules or interface between two features is
called integration testing.
The purpose of executing the integration testing is to test the statement's
accuracy between each module.

Types of Integration Testing

Integration testing is also further divided into the following parts:

o Incremental Testing
o Non-Incremental Testing

Incremental Integration Testing

Whenever there is a clear relationship between modules, we go for
incremental integration testing. Suppose, we take two modules and analysis
the data flow between them if they are working fine or not.

If these modules are working fine, then we can add one more module and test
again. And we can continue with the same process to get better results.

In other words, we can say that incrementally adding up the modules and test
the data flow between the modules is known as Incremental integration
testing.
Types of Incremental Integration Testing

Incremental integration testing can further classify into two parts, which are as
follows:

1. Top-down Incremental Integration Testing

2. Bottom-up Incremental Integration Testing

Let's see a brief introduction of these types of integration testing:

1. Top-down Incremental Integration Testing

In this approach, we will add the modules step by step or incrementally and
test the data flow between them. We have to ensure that the modules we are
adding are the child of the earlier ones.

2. Bottom-up Incremental Integration Testing

In the bottom-up approach, we will add the modules incrementally and check
the data flow between modules. And also, ensure that the module we are
adding is the parent of the earlier ones.

Non-Incremental Integration Testing/ Big Bang

Method
Whenever the data flow is complex and very difficult to classify a parent and a
child, we will go for the non-incremental integration approach. The non-
incremental method is also known as the Big Bang method.

3. System Testing
Whenever we are done with the unit and integration testing, we can proceed
with the system testing.

In system testing, the test environment is parallel to the production

environment. It is also known as end-to-end testing.

In this type of testing, we will undergo each attribute of the software and test
if the end feature works according to the business requirement. And analysis
the software product as a complete system.

Non-function Testing
The next part of black-box testing is non-functional testing. It provides
detailed information on software product performance and used technologies.

Non-functional testing will help us minimize the risk of production and related
costs of the software.

Non-functional testing is a combination of performance, load, stress,

usability and, compatibility testing.

Types of Non-functional Testing

Non-functional testing categorized into different parts of testing, which we are
going to discuss further:

o Performance Testing
o Usability Testing
o Compatibility Testing
1. Performance Testing
In performance testing, the test engineer will test the working of an
application by applying some load.

In this type of non-functional testing, the test engineer will only focus on
several aspects, such as Response time, Load, scalability, and Stability of
the software or an application.

Classification of Performance Testing

Performance testing includes the various types of testing, which are as follows:

o Load Testing
o Stress Testing
o Scalability Testing
o Stability Testing
 o Load Testing

While executing the performance testing, we will apply some load on the
particular application to check the application's performance, known as load
testing. Here, the load could be less than or equal to the desired load.

It will help us to detect the highest operating volume of the software and
bottlenecks.

o Stress Testing

It is used to analyze the user-friendliness and robustness of the software

beyond the common functional limits.

Primarily, stress testing is used for critical software, but it can also be used for
all types of software applications..

o Scalability Testing

To analysis, the application's performance by enhancing or reducing the load

in particular balances is known as scalability testing.

In scalability testing, we can also check the system, processes, or database's

ability to meet an upward need. And in this, the Test Cases are designed and
implemented efficiently.

o Stability Testing
Stability testing is a procedure where we evaluate the application's
performance by applying the load for a precise time.

It mainly checks the constancy problems of the application and the efficiency
of a developed product. In this type of testing, we can rapidly find the
system's defect even in a stressful situation.

2. Usability Testing
Another type of non-functional testing is usability testing. In usability
testing, we will analyze the user-friendliness of an application and detect the
bugs in the software's end-user interface.

Here, the term user-friendliness defines the following aspects of an

application:

o The application should be easy to understand, which means that all the
features must be visible to end-users.
o The application's look and feel should be good that means the
application should be pleasant looking and make a feel to the end-user
to use it.

3. Compatibility Testing
In compatibility testing, we will check the functionality of an application in
specific hardware and software environments. Once the application is
functionally stable then only, we go for compatibility testing.

Here, software means we can test the application on the different operating
systems and other browsers, and hardware means we can test the application
on different sizes.

Grey Box Testing

Another part of manual testing is Grey box testing. It is a collaboration of
black box and white box testing.
Since, the grey box testing includes access to internal coding for designing
test cases. Grey box testing is performed by a person who knows coding as
well as testing.

In other words, we can say that if a single-person team done both white box
and black-box testing, it is considered grey box testing.

What is Testing Framework?

A testing framework is a group of guidelines that support testers in building test
cases and performing software testing effectively. It also provides robust tools and
methods to simplify the testing process. By using a testing framework, you can
increase the accuracy and performance of software testing to high levels.

What is Automated Testing Framework?

In its basic form, an automated testing framework is a set of guidelines with which
you can perform software testing efficiently. These guidelines include many
elements, such as coding standards, reusable modules, and libraries. It also
includes managing test data, storing test results, etc.

Now, the question is whether you need to follow the guidelines for every software
test or not.

It's simple! There is no such rule that you need to follow all the guidelines of
a testing framework altogether while performing software testing. However, if you
follow the guidelines, it will help to boost the performance of a testing process and
ensure the delivery of quality products.

With a testing framework, you can add, modify, and delete test scripts as well as
functions in a standard way. In other words, it provides a defined structure that
supports building and performing automated testing.

What are the various types of Testing Frameworks?

Know that there are six automated testing frameworks that you can use to test
codes. The frameworks differ in their architecture and functionalities. Testers must
choose the correct framework based on their needs by considering the factors such
as reusability, cost, maintenance, etc.

The different types of automated testing frameworks can be seen in the below image

1.Linear Automation Framework

This framework is also named the ‘Record and Playback’ framework. It is one of the
simplest automated testing frameworks. You can test small applications using this
framework. For example, you can test web applications’ user interfaces. Mainly, this
framework allows recording every step, such as navigation, checkpoints, user
inputs, etc., and playback them in the following tests. That’s why this framework is
named ‘Record and Playback’. Besides, testing is performed in sequential order.

2. Modular-based Testing Framework

This framework allows breaking the entire source code base into pieces. The pieces
of code are known as modules. Essentially, these modules are nothing but test
cases. You can create test scripts for every module application and store them in the
function library. After that, you can test the modules separately. Also, it is simple to
combine the codes together to conduct more extensive tests based on the
hierarchical method. In short, you can decompose the code base into modules, and
at the same time, you can recombine them again for performing more extensive

tests.

It is essential to note that this framework creates an abstraction layer that isolates
the modules. Simply put, abstraction is the underlying concept of this framework.

Library Architecture Framework

This framework is known as the ‘Structured Scripting’ or 'Functional
Decomposition' framework. This framework is similar to modular testing, but
only similar tasks are grouped together. In other words, similar tasks are
grouped by functions and stored in a shared library. Test scripts can call the
functions from the library whenever they require it.
The wonderful thing about this framework is that it comes with high reusability. This
is because functions are stored in a shared library. So they can be used by multiple
applications at any time.

Data-driven Framework

This framework is introduced to overcome the drawback of the modular framework.

It means that test data is not hard-coded in this approach. In this framework, we can
isolate test data and test scripts separately. In other words, we can separate
ScriptLogic from the test data. No single data is embedded with test cases. This
framework allows testing codes with multiple datasets. It eliminates the need to
modify test scripts. We can store the test data in excel sheets, CSV files, text files,
SQL tables, databases with JDBC drivers, ODBC sources, and so on. In this
framework, test scripts access data externally consistently.

. Keyword-Driven Framework

In a way, this framework is introduced to overcome the setbacks in a data-driven

framework. It is also known as the ‘Table-driven testing’ framework. Mainly, it
provides predefined actions and methods in the form of keywords. Keywords are
usually stored in keyword tables. Examples of keywords are InputText,
VerifyPropety, VerifyValue, and so on.
Moreover, keywords can be labeled as login, verifying, click, clicking, etc. Know that
you can store keywords with their associated objects with the help of shared object
repositories. Not only that, you can use these keywords in multiple scripts. You can
use this framework for testing small applications.

Software Testing Metrics

Overview

Software test metrics are used to quantify the process of software testing. The
quality, growth, or improvement needed at a stage can be measured using the
metrics. So that whatever is lagging this time can be improved for the next cycle.
Metrics provide a comparative measure of a process. Metrics also help to determine
software's quality and measures required to create a defect-free quality product. The
average time taken to fix a bug is a better parameter than the time allocated for the
same.

What are Software Testing Metrics?

A software testing metric indicates the degree to which a process, component, or

tool is efficient. Here we have a quantitative measure of the effectiveness of an
approach. Let's say we automated certain test cases and we have 50% test
coverage using tool A. Next time the tool was changed and manual testing was also
used so test coverage was 66%. Now we have solid data to justify why manual
testing in combination with automation would be better to test the product.

Why Test Metrics are Important?

1. Software testing metrics are used to increase the overall productivity of the
development process.

2. It helps to make more informed choices about the tools and technologies
being used.

3. It helps to identify unique ways and techniques that are beneficial for their
system, hence increasing performance.

4. Software testing metrics determine the health of a process, tool, and

approach used.

Types of Software Testing Metrics

There are three types of software testing metrics:

 1. Process Metrics: The quantitative measures that define the efficiency of a
process based on parameters like speed, time, utilization of resources, etc.
Managers can look at these numbers and set new guidelines for the phase.

2. Product Metrics: Measures to determine the quality, size, performance, and

efficiency of the product come under product metrics.

3. Project Metrics: Quality and productivity of the project, utilization of

resources, cost, and time come under the project metrics.

Manual Test Metrics

Since manual testing is a step-by-step testing process carried out by a quality

analyst, finding quantitative measures in the process are little different. These
metrics are manual test metrics. There are two types of manual test metrics:

1. Base Metrics: The essential data taken out via the carrying testing process
comes under base metrics. It comprises test cases and test cases completed.

2. Calculated Metrics: The base metrics data is further taken out to carry
differential results that provide more information about the process or product.
It is more useful for tracking project progress.

Some Other Metrics

 Defect metrics: The measures telling about defect ratio, speed taken to fix a
defect, and complexity of a defect come under defect metrics.

 Schedule Adherence: Schedule Adherence tells the expected time given for
a task vs the time taken to complete it.

 Defect Severity: Defect severity talks about how much the impact that
defect/bug has on the product.

 Test case efficiency: Test case efficiency covers how well a test case can
determine the impact of a test case.

 Defects finding rate: It tells what is the pattern of flaws over some time.

 Defect Fixing Time: The time difference between when the defect was
assigned vs when it got fixed.

 Test Coverage: How much of the requirements and functionalities are

covered in the testing process comes under test coverage.
  Defect cause: The modules or components causing the defect.

Metrics Life Cycle

Let's look at various stages of the test metrics lifecycle:

1. Analysis: The QA team identifies the metrics like time, effort, efficiency, etc.

2. Communicate: Communication among the testing team as to how to capture

the metrics, the process of it, and what all data needs to be extracted.

3. Evaluation: All the calculation of the data happens here. Preparation of these
metrics is done.

4. Report: These metrics are reported. Data is compared, and loopholes and
analysis of the report are done. Measures to improve are discussed so that
next time the process would be more effective.

Test Plan

A test plan is a detailed document which describes software testing areas and
activities. It outlines the test strategy, objectives, test schedule, required
resources (human resources, software, and hardware), test estimation and
test deliverables.
 The test plan is a base of every software's testing. It is the most crucial
activity which ensures availability of all the lists of planned activities in an
appropriate sequence.

The test plan is a template for conducting software testing activities as a

defined process that is fully monitored and controlled by the testing manager.
The test plan is prepared by the Test Lead (60%), Test Manager(20%), and
by the test engineer(20%).

Types of Test Plan

There are three types of the test plan

PauseNext

Unmute

Current Time 0:09

Duration 18:10

Loaded: 4.40%

Fullscreen

o Master Test Plan

o Phase Test Plan
o Testing Type Specific Test Plans

Master Test Plan

Master Test Plan is a type of test plan that has multiple levels of testing. It
includes a complete test strategy.

Phase Test Plan

A phase test plan is a type of test plan that addresses any one phase of the
testing strategy. For example, a list of tools, a list of test cases, etc.

Specific Test Plans

 Specific test plan designed for major types of testing like security testing, load
testing, performance testing, etc. In other words, a specific test plan designed
for non-functional testing.

How to write a Test Plan

Making a test plan is the most crucial task of the test management process.
According to IEEE 829, follow the following seven steps to prepare a test
plan.

o First, analyze product structure and architecture.

o Now design the test strategy.
o Define all the test objectives.
o Define the testing area.
o Define all the useable resources.
o Schedule all activities in an appropriate manner.
o Determine all the Test Deliverables.

Test plan components or attributes

The test plan consists of various parts, which help us to derive the entire
testing activity.

Objectives: It consists of information about modules, features, test data etc.,

which indicate the aim of the application means the application behavior, goal,
etc.
 Scope: It contains information that needs to be tested with respective of an
application. The Scope can be further divided into two parts:

o In scope
o Out scope

In scope: These are the modules that need to be tested rigorously (in-detail).

Out scope: These are the modules, which need not be tested rigorously.

For example, Suppose we have a Gmail application to test, where features

to be tested such as Compose mail, Sent Items, Inbox, Drafts and
the features which not be tested such as Help, and so on which means that
in the planning stage, we will decide that which functionality has to be
checked or not based on the time limit given in the product.

Software Testing Tools

Software testing tools are required for the betterment of the application or
software.

That's why we have so many tools available in the market where some are
open-source and paid tools.

The significant difference between open-source and the paid tool is that the
open-source tools have limited features, whereas paid tool or commercial
tools have no limitation for the features. The selection of tools depends on the
user's requirements, whether it is paid or free.

The software testing tools can be categorized, depending on the licensing

(paid or commercial, open-source), technology usage, type of testing, and so
on.

With the help of testing tools, we can improve our software performance,
deliver a high-quality product, and reduce the duration of testing, which is
spent on manual efforts.

The software testing tools can be divided into the following:

 o Test management tool
o Bug tracking tool
o Automated testing tool
o Performance testing tool
o Cross-browser testing tool
o Integration testing tool
o Unit testing tool
o Mobile/android testing tool
o GUI testing tool
o Security testing tool

Test management tool

Test management tools are used to keep track of all the testing activity, fast
data analysis, manage manual and automation test cases, various
environments, and plan and maintain manual testing as well.

Bug tracking tool

The defect tracking tool is used to keep track of the bug fixes and ensure the
delivery of a quality product. This tool can help us to find the bugs in the
testing stage so that we can get the defect-free data in the production server.
With the help of these tools, the end-users can allow reporting the bugs and
issues directly on their applications.

Automation testing tool

This type of tool is used to enhance the productivity of the product and
improve the accuracy. We can reduce the time and cost of the application by
writing some test scripts in any programming language.

Performance testing tool

Performance or Load testing tools are used to check the load, stability, and
scalability of the application. When n-number of the users using the
application at the same time, and if the application gets crashed because of
the immense load, to get through this type of issue, we need load testing
tools.

Cross-browser testing tool

This type of tool is used when we need to compare a web application in the
various web browser platforms. It is an important part when we are
developing a project. With the help of these tools, we will ensure the
consistent behavior of the application in multiple devices, browsers, and
platforms.

Integration testing tool

This type of tool is used to test the interface between modules and find the
critical bugs that are happened because of the different modules and ensuring
that all the modules are working as per the client requirements.

Unit testing tool

This testing tool is used to help the programmers to improve their code
quality, and with the help of these tools, they can reduce the time of code and
the overall cost of the software.

Mobile/android testing tool

We can use this type of tool when we are testing any mobile application.
Some of the tools are open-source, and some of the tools are licensed. Each
tool has its functionality and features.

GUI testing tool

GUI testing tool is used to test the User interface of the application because a
proper GUI (graphical user interface) is always useful to grab the user's
attention. These type of tools will help to find the loopholes in the
application's design and makes its better.

Security testing tool

The security testing tool is used to ensure the security of the software and
check for the security leakage. If any security loophole is there, it could be
fixed at the early stage of the product. We need this type of the tool when the
software has encoded the security code which is not accessible by the
unauthorized users.

Test Oracle is a mechanism, different from the program itself, that can be used to test the
accuracy of a program’s output for test cases. Conceptually, we can consider testing a
process in which test cases are given for testing and the program under test. The output of
the two then compared to determine whether the program behaves correctly for test cases.
✓ Testing oracles are required for testing.

✓ Ideally, we want an automated oracle, which always gives the correct answer.
However, often oracles are human beings, who mostly calculate by hand what the output
of the program should be.

✓ The human oracles typically use the program’s specifications to decide what the
correct behaviour of the program should be.

✓ A complete oracle would have three capabilities and would carry them out perfectly:
✓ A generator, to provide predicted or expected results for each test.

✓ A comparator, to compare predicted and obtained results.

✓ An evaluator, to determine whether the comparison results are sufficiently close to be

a pass.

Test Oracle
Validate the observed output against the expected output

Is the observed output the same as the expected output? Oracle

Oracle: Example

A tester often assumes the role of an oracle and thus serves as human oracle.

 How to verify the output of a matrix multiplication?

 Hand calculation: the tester might input two matrices and check if the output

 of the program matches the results of hand calculation. Oracles can also be programs.
For example, one might use a matrix

 multiplication to check if a matrix inversion program has produced the correct result: A
× A-1 = I Test case Information to include in a Formal Test case Identification and
classification:

 Each test case should have a number, title (optional). Indicate system, subsystem or
module being tested

 Test case importance – indicate this.

 Instructions:

 Tell the tester exactly what to do.

 Tester should not normally have to refer to any other documentation in

 order to execute the instructions. Test case describes an input description and an
expected output description. The set of test cases is called a test suite. Hence any
combination of test cases may generate a test suite.