Software Metrics

Metrics

• Quantifiable measures that could be used to measure characteristics

of a software system or the software development process
• Required in all phases
• Required for effective management
• Managers need quantifiable information, and not subjective
information
• Subjective information goes against the fundamental goal of engineering
 Measurement

• Measurement is fundamental to any engineering discipline

• Software Metrics - Broad range of measurements for computer
software
• Software Process - Measurement can be applied to improve it on a
continuous basis
• Software Project - Measurement can be applied in estimation, quality
control, productivity assessment & project control
• Measurement can be used by software engineers in decision making.
Why Measure Software?

• Determine the quality of the current product or process

• Predict qualities of a product/process

• Improve quality of a product/process

Definitions
• Measure - Quantitative indication of the extent, amount, dimension, capacity
or size of some attribute of a product or process.
• E.g., Number of errors

• Measurement - The act of determining a measure

• Metric - A quantitative measure of the degree to which a system,

component, or process possesses a given attribute
• E.g., Number of errors found per person hours expended
Definitions

• Indicator – An indicator is a metric or combination of metrics that

provide insight into the software process, a software project or the
product itself.
• Direct Metrics: Immediately measurable attributes (e.g. line of code,
execution speed, defects reported)
• Indirect Metrics: Aspects that are not immediately quantifiable (e.g.
functionality, quantity, reliability)
• Faults:
- Errors: Faults found by the practitioners during software development
- Defects: Faults found by the customers after release
Why Do We Measure?

• To indicate the quality of the product.

• To assess the productivity of the people who produce the product
• To assess the benefits derived from new software engineering
methods and tools
• To form a baseline for estimation
• To help justify requests for new tools or additional training
• Estimate the cost & schedule of future projects
• Forecast future staffing needs
• Anticipate and reduce future maintenance needs
Example Metrics

• Defect rates
• Error rates

• Measured by:
• individual
• module
• during development

• Errors should be categorized by origin, type, cost

 A Good Manager
Measures
process
process metrics
project metrics
measurement
product metrics
product
What do we
“Not everything that can be use as a
counted counts, and not everything basis?
that counts can be counted.” - • size?
Einstein • function?
Process Metrics

• Process metrics are measures of the software development process,

such as
• Overall development time
• Type of methodology used
• Process metrics are collected across all projects and over long periods
of time.
• Their intent is to provide indicators that lead to long-term software
process improvement.
Process Metrics & Software Process
Improvement
• To improve any process, the rational way is:
• Measure Specific attributes of the process
• Derive meaningful metrics from these attributes.
• Use these metrics to provide indicators.
• The indicators lead to a strategy for improvement.
Process Metrics

• Focus on quality achieved as a consequence of a repeatable or

managed process. Strategic and Long Term.
• Statistical Software Process Improvement (SSPI). Error
Categorization and Analysis:
 All errors and defects are categorized by origin
 The cost to correct each error and defect is recorded
 The number of errors and defects in each category is computed
 Data is analyzed to find categories that result in the highest cost to the
organization
 Plans are developed to modify the process
• Defect Removal Efficiency (DRE). Relationship between errors
(E) and defects (D). The ideal is a DRE of 1:
DRE E /( E  D)
Factors Affecting Software Quality
How to Measure Effectiveness of a
Software Process
• We measure the effectiveness of a software process indirectly
• We derive a set of metrics based on the outcomes that can be
derived from the process.
• Outcomes include
• Errors uncovered before release of the software
• Defects delivered to and reported by end-users
• Work products delivered (productivity)
• Human effort
• Calendar time etc.
• Conformance to schedule
Project Metrics

• Project Metrics are the measures of Software Project and are used to
monitor and control the project. They enable a software project
manager to:

 Minimize the development time by making the adjustments

necessary to avoid delays and potential problems and risks.

 Assess product quality on an ongoing basis & modify the

technical approach to improve quality.
Project Metrics

• Used in estimation techniques & other technical work.

• Metrics collected from past projects are used as a basis from which
effort and time estimates are made for current software project.
• As a project proceeds, actual values of human effort & calendar time
expended are compared to the original estimates.
• This data is used by the project manager to monitor & control the
project.
Project Metrics

• Used by a project manager and software team to adapt project

work flow and technical activities.

• Metrics:
- Effort or time per SE task
- Errors uncovered per review hour
- Scheduled vs. actual milestone dates
- Number of changes and their characteristics
- Distribution of effort on SE tasks
Product metrics

• Product metrics are measures of the software product at any stage of

its development, from requirements to installed system. Product
metrics may measure:
• the complexity of the software design
• the size of the final program
• the number of pages of documentation produced
Types of Software Measurements

• Direct measures
• Easy to collect
• E.g. Cost, Effort, Lines of codes (LOC), Execution Speed,
Memory size, Defects etc.
• Indirect measures
• More difficult to assess & can be measured indirectly
only.
• Quality, Functionality, Complexity, Reliability, Efficiency,
Maintainability etc.
An example

• 2 different project teams are

working to record errors in a
• Which team do you
software process think is more
• Team A – Finds 342 errors effective in finding
during software process before
release errors?
• Team B- Finds 184 errors
Normalization of Metrics

• To answer this we need to know the size & complexity of the

projects.
• But if we normalize the measures, it is possible to compare the two
• Normalization: compensate for complexity aspects particular to a
product
• For normalization we have 2 ways-
• Size-Oriented Metrics
• Function Oriented Metrics
Metrics Guidelines

• Use common sense and organizational sensitivity when interpreting

metrics data
• Provide regular feedback to the individuals and teams who have
worked to collect measures and metrics.
• Don’t use metrics to appraise individuals
• Work with practitioners and teams to set clear goals and metrics
that will be used to achieve them
• Never use metrics to threaten individuals or teams
• Metrics data that indicate a problem area should not be considered
“negative.” These data are merely an indicator for process
improvement
• Don’t obsess on a single metric to the exclusion of other important
metrics
Typical Normalized Metrics
Project LOC FP Effort R(000) Pp. doc Errors Defects People
(P/M)
alpha 12100 189 24 168 365 134 29 3

beta 27200 388 62 440 1224 321 86 5

gamma 20200 631 43 314 1050 256 64 6

• Size-Oriented:
- errors per KLOC (thousand lines of code), defects per KLOC, R per LOC, page of
documentation per KLOC, errors / person-month, LOC per person-month, R / page
of documentation
• Function-Oriented:
- errors per FP, defects per FP, R per FP, pages of documentation per FP, FP per
person-month
Size-Oriented Metrics

• Based on the “size” of the software produced

• LOC - Lines Of Code
• KLOC - 1000 Lines Of Code
• SLOC – Statement Lines of Code (ignore
whitespace)
• Typical Measures:
• Errors/KLOC, Defects/KLOC, Cost/LOC, Documentation
Pages/KLOC
 Size-Oriented Metrics
Project Effort Cost LOC kLOC Doc. Errors Peopl
(person- ($) (pgs) e
month)
A 24 168,000 12100 12.1 365 29 3

B 62 440,000 27200 27.2 1224 86 5

From the above data, simple size oriented metrics
can be developed for each Project

• Errors per KLOC

• $ per KLOC
• Pages of documentation per KLOC
• Errors per person-month
• LOC per person-month
• Advantages of Size Oriented Metrics
• LOC can be easily counted
• Many software estimation models use LOC or KLOC as input.
• Disadvantages of Size Oriented Metrics
• LOC measures are language dependent, programmer dependent
• Their use in estimation requires a lot of detail which can be difficult to
achieve.
• Useful for projects with similar environment
Halstead’s Metrics

• Amenable to experimental verification [1970s]

• Program length: N = N1 + N2
• Program vocabulary: n = n1 + n2
N̂
• Estimated length: = n1 log2 n1 + n2 log2 n2
• Close estimate of length for well structured programs
N̂

• Purity ratio: PR = /N
Program Complexity

• Volume: V = N log2 n
• Number of bits to provide a unique designator for each of the n items
in the program vocabulary.

• Difficulty

• Program effort: E=D*V

• This is a good measure of program understandability
Qualities of a good metric

• simple, precisely definable—so that it is

• clear how the metric can be evaluated;
• objective, to the greatest extent possible;
• easily obtainable (i.e., at reasonable cost);
• valid—the metric should measure what it
• is intended to measure; and
• robust—relatively insensitive to (intuitive-
• ly) insignificant changes in the process or
• product.