1

Software Quality
Assurance
Lecture 3
Software Measurement and Metrics
• Measures help us understand attributes of software products or
systems and to assess their quality
NUMBER: a value (5 is a number)
DIMENSION: a descriptor (defects is a dimension)
MEASURE: A number with a dimension (5 defects)
METRIC: two related measures (6 defects/KLOC)

2
Basics
Definition of Measurement
• Measurement is the process of empirical objective assignment of numbers to
entities, in order to characterize an attribute
What Does That Mean?
• An entity is an object or event, such as a source program
• An attribute is a feature or property of an entity, such as the size of the program
• Objective means measurement must be based on a well-defined rule whose
results are repeatable, such as counting the number of source lines in the
program
• Each entity is given a number, which tells you about its attribute
• Example: Each source program has a source line count, which tells you about its
size

3
4
Common Mistakes in Software Measurement
• It’s easy to make mistakes in choosing what or how to measure
software characteristics
• To avoid mistakes, stick to the definition of measurement
• (1) You must specify both an entity and an attribute, not just one
or the other
• Example: Don’t just say you are measuring a program, say what property
of the program you are measuring
• Example: Don’t just say you are measuring the size of the software, say
what artifact of the software you are measuring the size of (e.g., source
programs)
• (2) You must define the entity precisely
• Example: Don’t just say program, say program source code

5
Why measurement?
Measurements provide us a mechanism for:
• Objective evaluation – how large?
• How maintainable?
• Will this approach be more cost effective?
• How long will it take?
• Gaining an insight into the efficacy of the software process and projects
• Assisting in estimation, quality control, productivity assessment, and
project control
• Improving software process on a continuous basis
• Pinpointing the problem areas so that remedies can be taken
6
Kinds of Metrics
• Three Basic Kinds
• There are three kinds of software quality metrics: … product metrics,
process metrics and project metrics
• Product Metrics
• Product metrics are those that describe the internal and external
characteristics of the product itself
• Examples: size, complexity, features, performance, reliability, quality level
• Most common software metrics are of this kind

7
• Process Metrics
• Process metrics measure the process of software development and
maintenance, in order to improve it
• Examples: effectiveness of defect removal during development, pattern of
defect arrival during testing, response time for fix Project Metrics
• Project metrics are those that describe the project characteristics
• Examples: number of developers, development cost, schedule, productivity

8
How to measure?
• Measurements can be categorized in two ways
• Direct Measures – Cost and Effort Applied e.g: LOC, execution speed,
memory size etc
• Indirect Measures – Abilities e.g: functionality, quality, complexity,
efficiency etc
Indirect Measures are more difficult to collect
• S/w Sizing problem:
• Direct measure LOC
• Indirect measure F

9
External Product Metrics
• Measures of the Software in its Environment
• External metrics are those we can apply only by observing the
software product in its environment (e.g., by running it)
• Includes many measures, but particularly:
• Failure rate (number of failures / unit of time)
• Availability rate (% of time system is up)
• Defect rate (Number of defects / size of code)

10
External vs internal
External quality mean all the properties of the software as a product that users
can experience and enjoy :conformity to their expectations
• reliability
• accuracy
• ease of use and comfort (including response delay)
• robustness (or adaptability to some unforeseen condition of use)
• openness (or adaptability to future extensions or evolutions)
Internal quality mean all the properties of the software as seen by the
developers that are desirable in order to facilitate the process of creating a
good product :concision : code do not suffer from duplication
• cohesion : each [module|class|routine] does one thing and does it well
• low coupling : minimal interdependencies and interrelation between objects

11
Direct measurement
• Some Direct Software Measures
• Direct measures are numbers that can be derived directly from the
entity without other information
• Examples:
• Length of source code (measured by number of lines)
• Duration of testing process (measured in elapsed hours)
• Number of defects discovered during the testing process (measured by
counting defects)
• Effort of a programmer on a project (measured by person-months worked)

12
Indirect measurement

A “person month” is the metric for expressing the effort (amount of time) principal investigators (PIs),
and other senior person devote to a specific project.

Conversion of percentage of effort to person months is straight-forward. To calculate person

months, multiply the percentage of your effort associated with the project times the
number of months of your appointment. For example:
25% of a 9 month academic year appointment equals 2.25 (AY) person months (9 x 0.25=
2.25)
13
Predictive Measurement
Prediction Systems
• Measurement for prediction requires a prediction system
A prediction system consists of:
• A mathematical model Example: E = a 𝑆 𝑏 , where E is the effort (to be
predicted), S is the estimated size in lines of code, and a and b are constant
parameters
• A procedure for determining the model parameters
• Example: Analyze past project data to determine a and b
• A procedure for interpreting the results
• Example: Use Bayesian probability analysis to determine the likelihood
that our prediction is accurate within 10%

14
Framework for Software measurement
• Products, Processes and Resources

Process
• A software-related activity or event (e.g., designing, coding, testing,...)
Product
• An item that results from a process (e.g., test plans, source code, design
and specification documents, inspection reports, ...)
Resource
• An item that is input to a process (e.g., people, hardware, software, ...)
15
Applying framework

16
Failure
• Definition of Failure
• Formal view: Any deviation from specified behaviour
• Engineering view: Any deviation from required, specified or expected
behaviour

17
Errors, Faults and Failures
• Definitions
• A (human) error is a mistake or oversight on the part of a designer or programmer
• A fault, which is a mistake in the software
• Failure is a consequence of a Defect. It is the observable incorrect behavior of the
system. Failure occurs when the software fails to perform in the real environment.
• A defect is usually defined as a fault or a failure:
• An Error that the tester finds is known as Defect.
• A Defect in a software product reflects its inability or inefficiency to comply with
the specified requirements and criteria and, subsequently, prevent the software
application from performing the desired and expected work. The defect is also
known as Fault.
• Defects = Faults + Failures (or sometimes just Faults or just Failures)

18
Defect Density Metric
Defect Density
• Defect density is a standard reliability metric:
Defect Density = Number of defect found / System Size
• Size is normally measured in KLOC (1000’s of lines of code), so units of
defect density are defects found per 1000 lines
• Widely used as an indicator of software quality
• According to best practices, one defect per 1000 lines (LOC) is
considered good.

19
Internal Product Metric
• Measures of the Product Itself
• Code Metrics
• Software Size
• The simplest and most enduring product metric is the size of the
product, measured using a count of the number of lines of source code
(LOC), most often quoted in 1000’s (KLOC)
• It is used in a number of other indirect measures, such as
• productivity ( LOC / effort )
• effort / cost estimation ( Effort = f(LOC) )
• quality assessment / estimation ( defects / LOC )

20
SLOC
• Source lines of code (SLOC or LOC) is a software metric used to measure the size of a software
program by counting the number of lines in the text of the program's source code.
• SLOC is typically used to predict the amount of effort that will be required to develop a program,
as well as to estimate programming productivity or effort once the software is produced
• physical SLOC (LOC) and logical SLOC (LLOC)
• physical SLOC is a count of lines in the text of the program's source code including comment lines.
• Logical LOC attempts to measure the number of "statements", but their specific definitions are tied to
specific computer languages (one simple logical LOC measure for C-like programming languages is the
number of statement-terminating semicolons).
• It is much easier to create tools that measure physical SLOC, and physical SLOC definitions are
easier to explain. However, physical SLOC measures are sensitive to logically irrelevant formatting
and style conventions, while logical LOC is less sensitive to formatting and style conventions.

21
Consider this snippet of C code as an example of the ambiguity encountered when determining SLOC:

for (i = 0; i < 100; i += 1) printf("hello"); /* How many lines of code is this? */

In this example we have:

1 Physical Lines of Code (LOC)
2 Logical Line of Code (LLOC) (for statement and printf statement)
1 comment line

Depending on the programmer and/or coding standards, the above "line of code" could be written on many
separate lines:

for (i = 0; i < 100; i += 1)

{
printf("hello");
} /* Now how many lines of code is this? */

In this example we have:

4 Physical Lines of Code (LOC)
2 Logical Line of Code (LLOC)
1 comment line 22
Problems with LOC Measures
1. LOC really measures length of program (a physical characteristic),
not size (a logical characteristic)
2. Does not take into account redundancy, reuse (e.g., XXXX Bank -
500 MLOC, only about 100 MLOC unique)
3. Cannot be compared across different programming languages
4. Can only be measured at end of development cycle

23
24
25
Better Size Measures
• Complexity - how complex is this software?
• Problem complexity - measures the complexity of the underlying
problem
• Algorithmic complexity - measures the complexity / efficiency of
the solution implemented by the software
• Structural complexity - measures the structure of the program
used to implement the algorithm (includes control structure,
modular structure, data flow structure and architectural structure)
• Cognitive complexity - measures the effort to understand the
software

26
Code complexity metrics
• Better Measures of Source Code
• Early explorations measured characteristics such as:
• density of decision (if) statements
• depth of blocks / loops
• length of methods / classes
• and many more…
• Best known and accepted source code complexity measures are
• “Software Science” metrics
• “Cyclomatic Complexity” and “Data Complexity” metrics

27
“Software Science” Metric
• According to Halstead's "A computer program is an implementation
of an algorithm considered to be a collection of tokens which can be
classified as either operators or operand."
• n1 = number of unique (different) operators
• n2 = number of unique (different) operands
• N1 = total number of operator uses
• N2 = total number of operand uses
• Comments are never included in the Halstead metrics
• Length of program N = N1 + N1
• Vocabulary: n1 + n2
int x = x + 1;
x occurs twice so if we take int, x and 1 as operands and =, + and ; as operators
we have 4 operands (3 unique) and 3 operators (3 unique). 28
Measurement tools for complexity
• Code Complexity Measurement Tools
• Testwell CMT++ for C, C++ (and C#)
• Testwell CMTJava for Java

29
More tools
• Radon is a Python tool that computes various metrics from the
source code. Radon can compute:
• McCabe’s complexity, i.e. cyclomatic complexity
• raw metrics (these include SLOC, comment lines, blank lines, &c.)
• Halstead metrics (all of them)
• Maintainability Index (the one used in Visual Studio)

30
• Pylint (code quality)
• Pyreverse analyses Python code and extracts UML class
diagrams and package depenndencies.

31
Next agenda
• Software metrics (complexity metrics)

32