Unit 05-Testing tools and measurements

Q.1] State the need of automated testing tools.

1. Automated Software Testing Saves Time and Money
2. Software tests must be repeated often during development cycles to ensure
quality.
3. Every time source code is modified software tests should be repeated.
4. They can even be run on multiple computers with different configurations.
5. Automated software testing can look inside an application and see memory
contents, data tables, file contents, and internal program states to determine
if the product is behaving as expected.
6. Once created, automated tests can be run repeatedly at no additional cost,
and they are much faster than manual tests.
Q.2] Enlist any four software testing tools.
1. Junit
2. JMeter
3. Selenium
4. QTP
5. TestComplete
Q.3] State the limitations of manual testing:
1. Manual testing is slow and costly.
2. It is very labor intensive; it takes a long time to complete tests.
3. Manual tests don’t scale well. As the complexity of the software increases the
complexity of the testing problem grows exponentially. This leads to an
increase in total time devoted to testing as well as total cost of testing.
4. Manual testing is not consistent or repeatable. One tester may approach and
perform a certain test differently from another, resulting in different results on
the same test, because the tests are not being performed identically.
5. Lack of training is the common problem.
6. GUI objects size difference and color combinations are not easy to find in
manual testing.
7. Not suitable for large scale projects and time bound projects.
8. Batch testing is not possible, for each and every test execution. Human user
interaction is mandatory.
9. Comparing large amount of data is impractical.
10. Processing change requests during software maintenance takes more time.
 Q.4] Describe any four limitations of manual testing
1. Slower Execution: Manual testing is time-consuming because each test
case must be performed manually by a tester.
2. Human Error: Since manual testing relies on human involvement,
mistakes can occur, leading to missed defects or inaccurate results.
3. Limited Coverage: Manual testing can only cover a small number of test
cases within the available time, leaving many scenarios untested.
4. Repetitive Work: Repeating the same tests for every software release is
inefficient and can be tedious for testers.
Q.5] Enlist the factors considered for selecting a testing tool for test
automation. OR
Describe any four factors for selecting a testing tool.
1. Assessment of the organization’s maturity (e.g. readiness for change);
2. Identification of the areas within the organization where tool support will
help to improve testing processes;
3. Evaluation of tools against clear requirements and objective criteria;
4. Proof-of-concept to see whether the product works as desired and meets
the requirements and objectives defined for it;
5. Evaluation of the vendor (training, support and other commercial aspects)
or open-source network of support;
6. Identifying and planning internal implementation (including coaching and
mentoring for those new to the use of the tool).

Q.6] Elaborate the term metrics and measurement and write the need
of software measurement.
Metrics and measurement :
▪ A Metric is a measurement of the degree that any attribute belongs to a
system, product or process.
▪ For example, the number of errors per person hours would be a metric.
Thus, software measurement gives rise to software metrics.
▪ A measurement is an indication of the size, quantity, amount or dimension
of a particular attribute of a product or process.
▪ For example, the number of errors in a system is a measurement.
▪ A Metric is a quantitative measure of the degree to which a system, system
component, or process possesses a given attribute. Metrics can be defined
as “STANDARDS OF MEASUREMENT”. Software Metrics are used to
measure the quality of the project.
▪ Simply, Metric is a unit used for describing an attribute. Metric is a scale
for measurement.
Need of Software measurement:

1. Establish the quality of the current product or process.

2. To predict future qualities of the product or process.
3. To improve the quality of a product or process.
4. To determine the state of the project in relation to budget and schedule.

Q.7] Elaborate the concept of Software Metrics? Describe Base

Metrics and calculated matrices with suitable example.
Software Metrics:
A metric is a measurement of the level at which any impute belongs to a system
product or process.
▪ The purpose of software testing metrics is to increase the efficiency and
effectiveness of the software testing process while also assisting in making
better decisions for future testing by providing accurate data about the
testing process.
▪ A metric expresses the degree to which a system, system component, or
process possesses a certain attribute in numerical terms.
Types of Metrics:
1. Process quality
2. Product quality
3. Objective Metrics
4. Subjective Metrics
1] Base Metrics: Analysts collect data throughout the development and
execution of test cases to provide base metrics. By generating a project status
report, these metrics are sent to test leads and project managers. It is quantified
using calculated metrics.
• The total number of test cases
• The total number of test cases completed.
2. Calculated Metrics: Data from base metrics are used to create calculated
metrics. The test lead collects this information and transforms it into more useful
information for tracking project progress at the module, tester, and other levels.
It’s an important aspect of the SDLC since it allows developers to make critical
software changes.
Example of Software Test Metrics Calculation:
Testing Metric Data retrieved during test case
development
No. of requirements 5
The average number of test cases 40
written per requirement
Total no. of Test cases written for all 200
requirements
Total no. of Test cases executed 164
No. of Test cases failed 60
No. of Test cases blocked 4
No. of Test cases unexecuted 36
Total no. of defects identified 20

1. Percentage test cases executed = (No of test cases executed / Total no of test
cases written) x 100
= (164 / 200) x 100 = 82
2. Test Case Effectiveness = (Number of defects detected / Number of test
cases run) x 100
= (20 / 164) x 100 = 12.2
3. Failed Test Cases Percentage = (Total number of failed test cases / Total
number of tests executed) x 100
= (60 / 164) * 100 = 36.59
4. Blocked Test Cases Percentage = (Total number of blocked tests / Total
number of tests executed) x 100
= (4 / 164) * 100
= 2.44
Q.8] State the advantages and disadvantages of using tools. OR
State & explain any four benefits of automation in testing
1. Save Time /Speed:
Due to advanced computing facilities, automation test
tools prevail in speed of processing the tests. Automation saves time as
software can execute test cases faster than humans.
2. Reduces the tester’s involvement in executing tests:
It relieves the testers to do some other work.
3. Test case design:
Automated tools can be used to design test cases also
through automation, better coverage can be guaranteed than if done
manually.
4. Reusable:
The automated tests can be reused on different versions of the
software, even if the interface changes.
5. Avoids human mistakes:
Manually executing the test cases may
incorporate errors. But this can be avoided in automation testing.
6. Cost Reduction:
If testing time increases, the cost of the software also
increases. Due to testing tools time and therefore cost is reduced.

Disadvantages of using tools:

1. It's more expensive to automate. Initial investments are bigger than
manual testing
2. High package development investment costs.
3. High manpower requirements for test preparation.
4. Considerable testing areas left uncovered.
5. You cannot automate everything; some tests still have to be done
manually.
6. You cannot rely on testing tools always.
Q.9] Describe object-oriented metrics in testing.
Object oriented metrics in testing:
As object-oriented approach emerged to support major applications, the
effectiveness of applying traditional software metrics to object-oriented systems
was challenged.
The object-oriented design approach gives opportunity to classify metrics
naturally. The classification captures object-oriented software features and
properties hierarchically. It begins with the high-level characteristics of an object-
oriented system and moves down to the low-level characteristics.

Source code size metrics:

Traditional metrics which are applied to object-oriented software give
insight into an overall system size and allow comparing systems and evaluating
productivity. They can also be used as a refactoring effectiveness indicator.

Lines of Code (LOC) metric

is most common software project measure. The metric becomes a baseline
to measure the degree of work performed on a project and it is used to create time
and cost estimates.

Effective Lines of Code Metric (eLOC)

is a measure of all lines that are not comments, blanks or standalone braces
or parenthesis. This metric more closely represents the quantity of work
performed.

Comment Line and Comment Percent (or Comment to Code Ratio):

is a degree of commenting within the source code. It measures the care
taken by programmers to make the source code and algorithms understandable.
Poorly commented code makes the maintenance activities an extremely
expensive. Recommended minimum is 20%.
Blank Line and White Space Percent Metric
is the number of blank lines within source code. It indicates the readability
of product. And File Count Metric counts the files processed and generates
metrics based on the file extension. It provides the distribution of the source code
types, source code types and distribution of the specifications to the
implementations.
Class metrics:
Class metrics describe structure of a class and relationship between classes.
The volume of a class is a basic size measure connected with the amount of
information inside it. The class volume can be measured by Number of Variables
and by Number of Methods. Also, Average LOC per Class and per Method
metrics can provide insight into the average module size in the system.
Q.10] Describe need for Automated Testing tools.
1] Save Time /Speed:
Due to advanced computing facilities, automation test
tools prevail in speed of processing the tests. Automation saves time as
software can execute test cases faster than humans.
2] Reusable:
The automated tests can be reused on different versions of the
software, even if the interface changes.
3] Cost Reduction:
If testing time increases, the cost of the software also
increases. Due to testing tools time and therefore cost is reduced.
4] Efficiency:
Automation is necessary to handle repetitive and time-consuming
test cases, reducing the burden on testers.

Q.11] State any two differences between manual and automated

testing.

Parameter Manual Testing Automated Testing

1. Speed Slower, human-dependent Faster, machine-executed
2. Cost Lower initial cost, higher Higher initial cost, lower long-
long-term cost term cost
3. Maintenance Low High
4. Feedback Slower feedback Instant feedback
Time
5. Test Coverage Limited coverage Can cover more tests and
environments
6. Flexibility High, allows exploration Fixed, predefined tests
7. Scalability Limited scalability Easily scalable for large
projects
8. Suitability Best for small projects, Best for large projects,
exploratory tests regression tests
 Q.12] Describe the criteria’s to select testing tools. OR Enlist
factors considered for selecting a testing tool for test automation.
1. Meeting requirements:
There are plenty of tools available in the market but
rarely do they meet all the requirements of a given product or a given
organization. Evaluating different tools for different requirements involve
significant effort, money, and time. Given of the plethora of choice available,
huge delay is involved in selecting and implementing test tools.
2. Technology expectations:
Test tools in general may not allow test developers to
extends/modify the functionality of the framework. So, extending the
functionality requires going back to the tool vendor and involves additional
cost and effort. A good number of test tools require their libraries to be linked
with product binaries
3. Training/skills:
While test tools require plenty of training, very few vendors
provide the training to the required level. Organization level training is needed
to deploy the test tools, as the user of the test suite are not only the test team
but also the development team and other areas like configuration management.
4. Management aspects:
A test tool increases the system requirement and requires the
hardware and software to be upgraded. This increases the cost of the already-
expensive test tool. When selecting the test tool, it is important to note the
system requirements and the cost involved in upgrading the software and
hardware needs to be included with the cost of the tool.

Q.13] Define metrics and measurements. Describe three types of

Metrics.
Metrics:
A Metric is a measurement of the degree that any attribute belongs to a
system, product or process.

Measurement:
A measurement is an indication of the size, quantity, amount or
dimension of a particular attribute of a product or process.

Types of Metrics:
1. Process quality
2. Product quality
3. Objective Metrics
4. Subjective Metrics
1. Process quality:
▪ Activities related to the production of software, tasks or milestones.
▪ Process metrics are collected across all projects and over long periods of
time
▪ They are used for making strategic decisions.
▪ The intent is to provide a set of process indicators that lead to long-term
software process improvement.
The only way to know how/where to improve any process is to:
• Measure specific attributes of the process.
• Develop a set of meaningful metrics based on these attributes.
• Use the metrics to provide indicators that will lead to a strategy for
improvement.
2. Product quality:
▪ Explicit result of the software development activity, deliverables, products.
▪ Product metrics help software engineers to better understand the attributes
of models and assess the quality of the software.
▪ They help software engineers to gain insight into the design and
construction of the software.
▪ Focus on specific attributes of software engineering work products
resulting from analysis, design, coding, and testing.
▪ Provide a systematic way to assess quality based on a set of clearly defined
rules.
3. Objective Metrics:
1. They are non-negotiable – that is the way they are defined doesn’t
change with respect the type of endeavour they are being applied to.
2. Actual cost or AC is always the total cost actually incurred in
accomplishing a certain activity or a sequence of activities.
4. Subjective Metrics:
These metrics are a relatively new precept and are more flexible than the
rigid framework of the objective metrics. Subjective metrics do deal with
performance but the approach is more tailored.