Software Metrics

Definitions
Measure - quantitative indication of extent,
amount, dimension, capacity, or size of some
attribute of a product or process.
E.g., Number of errors

Metric - quantitative measure of degree to which a

system, component or process possesses a given
attribute. “A handle or guess about a given
attribute.”
E.g., Number of errors found per person hours expended
 Measurement
 It is a process by which numbers or symbols are assigned to attributes
in the real world.

 Entity :- Is an object or an event in the real world.

 Attribute :- Is a feature or property of an entity.

 Example Attribute can be time required for testing phase.

 Use software Measure to Derive
 A basic for estimate

 To track the project progress

 To determine the complexity

 To analyze defects

 To experimentally validate best practices

 To determine productivity
 Why Measure Software?
 Determine the quality of the current product or process

 Predict qualities of a product/process

 Improve quality of a product/process

 Why Measure Software?
Understanding :-
 Includes measures that help to understand what is happening during
development and maintenance.
Control :-
 Includes measures that help to control what is happening on our projects.
 For example :- we can monitor the complexity of code modules by reviewing
only those modules that exceed acceptable bounds.
Performance(Improvement) :-
 Includes measures that help to improve our processes and products .
 For example :- compare two design methods to see which on eyields higher
quality code.
 Measurement Process for Managers
 What does each process cost ?
(Measure time and effort )
 How productive is the staff ?
(Time taken by staff)
 How good is the code being developed ?
(Software quality by recording)
 Will the customer be satisfied with the product ?
(Measures functionality by determining)
 How can we improve ?
(Time taken to perform major development activity)
Measurement Process for Engineers
 Are the requirements testable ?

 Have we found all faults ?

 Have we meet our product or process goals ?

 Motivation for Metrics
 Estimate the cost & schedule of future projects

 Evaluate the productivity impacts of new tools and techniques

 Establish productivity trends over time

 Improve software quality

 Forecast future staffing needs

 Anticipate and reduce future maintenance needs

 Example Metrics
Defect rates
Error rates

Measured by:
individual
module
during development

Errors should be categorized by origin, type, cost

 Metric Classification
Products
Explicit results of software development activities
Deliverables, documentation, by products

Processes
Activities related to production of software

Resources
Inputs into the software development activities
hardware, knowledge, people
 Product vs. Process
Process Metrics
Insights of process paradigm, software engineering tasks, work
product, or milestones
Lead to long term process improvement

Product Metrics
Assesses the state of the project
Track potential risks
Uncover problem areas
Adjust workflow or tasks
Evaluate teams ability to control quality
Types of Measures
Direct Measures (internal attributes)
Cost, effort, LOC, speed, memory

Indirect Measures (external attributes)

Functionality, quality, complexity, efficiency, reliability,
maintainability
Size-Oriented Metrics
Size of the software produced
LOC - Lines Of Code
KLOC - 1000 Lines Of Code
SLOC – Statement Lines of Code (ignore
whitespace)
Typical Measures:
Errors/KLOC, Defects/KLOC, Cost/LOC,
Documentation Pages/KLOC
LOC Metrics
Easy to use

Easy to compute

Language & programmer dependent

Complexity Metrics
LOC - a function of complexity
Language and programmer dependent
Halstead’s Software Science (entropy measures)
n1 - number of distinct operators
n2 - number of distinct operands
N1 - total number of operators
N2 - total number of operands
Example
if (k < 2)
{
if (k > 3)
x = x*k;
}

 Distinct operators: if ( ) { } > < = * ;

 Distinct operands: k 2 3 x
 n1 = 10
 n2 = 4
 N1 = 13
 N2 = 7
 Scope of Software Metrics
 Cost and Effort Estimation
 Productivity Measures
 Data Collection
 Quality Models and Measures
 Performance Evaluation
 Structural and Complexity Metrics
 Management by Metrics
 Capability-Maturity Assessment
 Cost and Efforts Estimations
Numerous models for software costs and effort estimation have
been proposed, in order to predict project costs during early phases
of the software life cycle.
These models often share a common approach: Effort is expressed
as a function of one or more variables, such as size of the product,
capability of the developers and level of reuse.
Size is expressed in lines of codes or number of function points.
Example predictions(Estimations) needed for software development
 Coding – Size/Schedule/ Quality Predictions
 Testing – Testing efforts predictions
 End of testing predictions
 Reliability / Quality predictions
 Productivity Model and Measures
Fig. illustrates an example of the possible components that
contribute to overall productivity.
It shows productivity as function of value and cost; each is then
decomposed into other aspects, which are expressed in
measurable.
 Data Collection
The quality of any measurement program is clearly dependent
ion careful data collection.
Data collection is required to ensure that collection is
consistent, complete and data integrity is not risk.
Fig. Contains several examples of data collection, which helps
the manager to study the progress as well as problems of
development.
 Quality Models and Measures
These models are usually constructed in a tree-like fashion as in
fig.
The upper branches hold important high-level quality factors of
software products, such as reliability and usability.
Each quality factor is composed of lower-level criteria such as
structuredness and traceability.
The tree describes the relationship between the factors and their
dependent criteria.
Quality Models and Measures
 Performance Evaluation and models
Performance is another important aspects of quality.
Performance evaluation means observing the system
performance characteristics such as times, load handling
capacity and completion rates.
Performance investigation involves efficiency of algorithms
and algorithmic complexity.
Complexity can be Interpreted in the following ways :-
 Problem Complexity
 Algorithmic Complexity
 Structural Complexity
 Management by Metrics
Measurement is an important part of software project management.
Customers and developers rely on measurement-based charts and
graphs to help them to decide if the project is on track

Capability-Maturity assessment
Software Engg. Institute (SEI) proposed a Capability Maturity Model
(CMM) to measure an organization ability to develop quality software
 The CMM describes an evolutionary important path from an adhoc,
immature process(dependent on individuals) to a mature, disciplined
process that could be optimized based on continuous feedback.

Classifying software measures
Three types of software entities to measure :-

 Processes – collections of software related activities

 Design process, Coding process, Testing process

 Products – Are deliverables or documents that result from

process
activity
 Design document, Source code, test cases

 Resources – entities required by a process activity

 Programmers, Testers, designers, etc.

26
 Classifying software measures
 Within each class, we have two attributes :-
 Internal attributes –
 Internal attribute of a product, process or resource are those that can be measured
purely in terms of the entity itself.
 These attributes are measured without executing the code.
 Internal Attributes can be measured by examining the product, process or resource on
its own.
External attributes –
 External attribute of a product, process or resource are those that can be measured with
respect to how entity relates to its environment.
 Behavior of the entity is important
 Managers want to be able to measure and predict external attributes
 However, external attributes are more difficult to measure than internal ones, and are
measured late in the development process
 Desire is to predict external attributes in terms of more easily-measured internal
attributes
 Processes :-
We can measure…
 How long it takes for a process to complete ?
 How much it costs ?
 Comparison with other processes.
 Properties of sub-processes
 Internal process Attributes :-
 Duration of process or one of its activities
 Effort associated with process
 Number of requirement changes
 Number of coding faults
 External Process Attribute :-
 Quality
 Cost
 Stability
 Effectiveness
 Products :-
We can Measure…
 Size of product by measuring the number of pages or number of words it contains.
 Assessment of specification in terms of their length, reuse, redundancy and syntactic
 Reliability
 understandability of document
 Size of sub-products and average module size
 Internal Product Attributes :-
 Size, reuse, redundancy, functionality, algorithmic, complexity, control-flow etc.
 External Product Attributes:-
 Quality, reliability, usability, maintainability, integrity, reusability, portability. Etc.

NOTE :- Measures of internal software product attributes help in measurement of

software performance and reliability.
 Resources :-
We can measure…
 Magnitude of project : number of Staff working on the project.
 Cost : How much we are paying for testing tools, software, and staff.
 Productivity (P) : P = Amount of output/Effort Input
Where effort input – process measure and output – Product Measure
 Internal Resource Attributes :-
 Personnel – Age, Price
 Teams – Size, Communication level, Technical expertise etc.
 Software – Size, Price
 Hardware – Price, Speed, Memory, etc.
 External Resource Attributes :-
 Quality, Productivity, Experience, intelligence, etc.
 Complexity
 Complexity of a problem is defined as the amount of resources
required for an optimal solution to the problem.
 The complexity of a solution is the resources needed to implement
a particular solution.
 Solution complexity can be viewed with two aspects :-

 Time Complexity : Where the resources is computer time.

 Space Complexity : Where the resources is computer memory.
 A. Measuring Algorithmic Efficiency
 To measure complexity, it is necessary to measure algorithmic efficiency.
 It can measured by determining how much time or memory is required for
implementation.
 We can measure time efficiency of a program by running a program through
a particular compiler on a particular machine with a particular input,
measuring the actual processing time required. Efficiency is an external
product measure.
 It is measured indirectly in terms of a particular process.
 This approach is rather unsatisfactory, because its dependence on external
factors leads to false measurement.
 To measure the product directly, program can be modeled I term of the
algorithm.
 Small number of types of primitive arithmetic operations relevant to the
algorithm is identified.
 This information can be used to measure efficiency in terms of the number of
operations required for a given input.
 Big-O Notation
 “Big-O” notation O() is a measure of running time analysis.
 The argument tells you what the running time is proportional to.
 O(n): running time is proportional to n, the number of items being
processed.
 Example: linked list lookup. If you have a list with 2x as many
links, it’ll take 2x as long to look through the list.
 O(log n): running time is proportional to the log of n (usually base
2).
 Example: binary search tree lookup. 2x as many nodes ..only 1
more step
 Big-O Notation
 O(1): running time is proportional to 1 (a constant).
 Example: getting the n’thindex of an array. It takes no more time to
read array[10000] than it does to read array[23].
 Hashing is concerned entirely with this last scenario because it’s so
fast!
 Based on the Big-O notation, the algorithm can be characterized :-
 Constant time (O(1)) algorithms.
 Logarithmic time (O(log n)) algorithms.
 Linear time (O(n)) algorithms.
 Polynomial time (O(nk), for k>1) algorithms.
 Exponential time (O(kn), for k>1) algorithms.
Many algorithms are (O(nlog n))
Example
 Θ(n3): n3
5n3+ 4n
105n3+ 4n2 + 6n

 Θ(n2): n2
5n2+ 4n + 6
n2 + 5
 Θ(log n): log n
log n2
log (n + n3)
SOFTWARE METRICS
Measurement and Scaling
Chapter Outline
1) Overview
2) Measurement and Scaling
3) Primary Scales of Measurement
i. Nominal Scale
ii. Ordinal Scale
iii. Interval Scale
iv. Ratio Scale
4) A Comparison of Scaling Techniques
Chapter Outline
5) Comparative Scaling Techniques
i. Paired Comparison
ii. Rank Order Scaling
iii. Constant Sum Scaling
iv. Q-Sort and Other Procedures
6) Verbal Protocols
7) International Marketing Research
8) Ethics in Marketing Research
Chapter Outline
9) Internet and Computer Applications
10) Focus on Burke
11) Summary
12) Key Terms and Concepts
Measurement and Scaling
Measurement means assigning numbers or other symbols
to characteristics of objects according to certain
prespecified rules.
One-to-one correspondence between the numbers and
the characteristics being measured.
The rules for assigning numbers should be standardized
and applied uniformly.
Rules must not change over objects or time.
 
Measurement and Scaling
Scaling involves creating a continuum upon which
measured objects are located.

Consider an attitude scale from 1 to 100. Each respondent

is assigned a number from 1 to 100, with 1 = Extremely
Unfavorable, and 100 = Extremely Favorable.
Measurement is the actual assignment of a number from 1
to 100 to each respondent. Scaling is the process of placing
the respondents on a continuum with respect to their
attitude toward department stores.
 Primary Scales of Measurement
ScaleFigure 8.1
Nominal Numbers Finish
Assigned
7 8 3
to Runners

Ordinal Rank Order Finish

of Winners
Third Second First
place place place

Interval Performance
Rating on a 8.2 9.1 9.6

0 to 10 Scale
15.2 14.1 13.4
Ratio Time to
Finish, in
Primary Scales of Measurement
Nominal Scale
 The numbers serve only as labels or tags for identifying
and classifying objects.
 When used for identification, there is a strict one-to-one
correspondence between the numbers and the objects.
 The numbers do not reflect the amount of the characteristic
possessed by the objects.
 The only permissible operation on the numbers in a
nominal scale is counting.
 Only a limited number of statistics, all of which are based
on frequency counts, are permissible, e.g., percentages, and
mode.
 Measurement Basics
 Nominal scale
 Most primitive form of measurement – define classes or categories, and
place each category in a particular class or category
 Two major characteristics
 Empirical relation consists only of different classes – no notion of
ordering
 Any distinct number or symbolic representation is an acceptable
measure – no notion of magnitude associated with numbers or symbols.
 Any two mappings, M and M’, will be related to each other in that M’ can
be obtained from M by a one-to-one mapping
 Example – software faults can belong to one of the following classes,
according to where they were first introduced during development:
 Specification
 Design
 Code

44
Primary Scales of Measurement
Ordinal Scale
 A ranking scale in which numbers are assigned to objects
to indicate the relative extent to which the objects possess
some characteristic.
 Can determine whether an object has more or less of a
characteristic than some other object, but not how much
more or less.
 Any series of numbers can be assigned that preserves the
ordered relationships between the objects.
 In addition to the counting operation allowable for nominal
scale data, ordinal scales permit the use of statistics based
on centiles, e.g., percentile, quartile, median.
 Measurement Basics
 Measurement types and scale
Ordinal scale
Augments nominal scale with ordering information.
Three major characteristics
Empirical relation system consists of classes that are ordered with
respect to the attribute
Any mapping preserving the ordering (i.e., a monotonic function)
is acceptable
Numbers represent ranking only, so arithmetic operations have no
meaning
Set of admissible transformations is set of all monotonic mappings
Example – software “complexity” – two valid measures
Value Meaning Value Meaning
1 Trivial 2 Trivial
2 Simple 4 Simple
3 Moderate 6 Moderate
4 Complex 9 Complex
46 5 Incomprehensible 12 Incomprehensible
Primary Scales of Measurement
Interval Scale
 Numerically equal distances on the scale represent equal
values in the characteristic being measured.
 It permits comparison of the differences between objects.
 The location of the zero point is not fixed. Both the zero
point and the units of measurement are arbitrary.
 Any positive linear transformation of the form y = a + bx
will preserve the properties of the scale.
 It is meaningful to take ratios of scale values.
 Statistical techniques that may be used include all of those
that can be applied to nominal and ordinal data, and in
addition the arithmetic mean, standard deviation, and other
statistics commonly used in marketing research.
 Measurement Basics
 Measurement type and scale
 Interval scale
 Captures information about size of intervals that separate classes.
 Three characteristics
 Preserves order
 Preserves differences, but not ratios
 Addition and subtraction are acceptable, but not multiplication and
division
 Class of admissible transformations is the set of affine transformations:
M’=aM+b, where a>0.
 Example – software complexity – suppose the difference in complexity
between a trivial and a simple system is the same as that between a simple
and a moderate system. Where this equal step applies to each class, we have
an attribute measurable on an interval scale.
Value Meaning Value Meaning Value Meaning
1 Trivial 0 Trivial 1.1 Trivial
2 Simple 2 Simple 2.2 Simple
3 Moderate 4 Moderate 3.3 Moderate
4 Complex 6 Complex 4.4 Complex
5 Incomprehensible 8 Incomprehensible 5.5 Incomprehensible
48
Primary Scales of Measurement
Ratio Scale
 Possesses all the properties of the nominal, ordinal, and
interval scales.
 It has an absolute zero point.
 It is meaningful to compute ratios of scale values.
 Only proportionate transformations of the form y = bx,
where b is a positive constant, are allowed.
 All statistical techniques can be applied to ratio data.
 Measurement Basics
 Measurement type and scale
Ratio scale
Most useful scale, common in physical sciences – captures
information about ratios.
4 characteristics
Preserves ordering, size of intervals between entities, and ratios
between entities
There is a zero element, representing total lack of the attribute
Measurement mapping must start at 0 and increase at equal
intervals (units)
All arithmetic can be meaningfully applied to classes in the range
of the mapping.
Acceptable transformations are ratio transformations – M’=aM,
where a is a scalar.
Example – program length can be measured by lines of code,
number of characters, etc. Number of characters may be obtained
by multiplying the number of lines by the average number of
50
characters per line.
 Primary Scales of Measurement
Table 8.1
Scale Basic Common Marketing Permissible Statistics
Characteristics Examples Examples Descriptive Inferential
Nominal Numbers identify Social Security Brand nos., store Percentages, Chi-square,
& classify objects nos., numbering types mode binomial test
of football players
Ordinal Nos. indicate the Quality rankings, Preference Percentile, Rank-order
relative positions rankings of teams rankings, market median correlation,
of objects but not in a tournament position, social Friedman
the magnitude of class ANOVA
differences
between them
Interval Differences Temperature Attitudes, Range, mean, Product-
between objects (Fahrenheit) opinions, index standard moment
Ratio Zero point is fixed, Length, weight Age, sales, Geometric Coefficient of
ratios of scale income, costs mean, harmonic variation
values can be mean
compared
 A Classification of Scaling
Techniques
Figure 8.2

Scaling Techniques

Comparative Noncomparative
Scales Scales

Continuous ItemizedRating
Paired Rank Constant Q- Sort &
Rating Scales Scales
Comparison Orde Sum Other
r Procedures

Semantic Stapel
Likert Differential
A Comparison of Scaling
Techniques
 Comparative scales involve the direct comparison of
stimulus objects. Comparative scale data must be
interpreted in relative terms and have only ordinal or rank
order properties.
 
 In noncomparative scales, each object is scaled
independently of the others in the stimulus set. The
resulting data are generally assumed to be interval or ratio
scaled.
Relative Advantages of Comparative Scales
 Small differences between stimulus objects can be
detected.
 Same known reference points for all respondents.
 Easily understood and can be applied.
 Involve fewer theoretical assumptions.
 Tend to reduce halo or carryover effects from one
judgment to another.
Relative Disadvantages of Comparative Scales
 Ordinal nature of the data
 Inability to generalize beyond the stimulus objects scaled.
Comparative Scaling Techniques
Paired Comparison Scaling
 A respondent is presented with two objects and asked to
select one according to some criterion.
 The data obtained are ordinal in nature.
 Paired comparison scaling is the most widely used
comparative scaling technique.
 With n brands, [n(n - 1) /2] paired comparisons are
required
 Under the assumption of transitivity, it is possible to
convert paired comparison data to a rank order.
 Obtaining Shampoo Preferences Using Paired
Comparisons
Figure 8.3
Instructions: We are going to present you with ten pairs of
shampoo brands. For each pair, please indicate which one of the
two brands of shampoo you would prefer for personal use.
Jhirmack Finesse Vidal Head & Pert
Sassoon Shoulders
Recording
Jhirmack
Form: 0 0 1 0
Finesse 1a 0 1 0
Vidal Sassoon 1 1 1 1
Head & Shoulders 0 0 0 0
Pert 1 1 0 1
Number of Times 3 2 0 4 1
Preferredb
a
A 1 in a particular box means that the brand in that column was preferred
over the brand in the corresponding row. A 0 means that the row brand
was preferred over the column brand. bThe number of times a brand was
preferred is obtained by summing the 1s in each column.
 Paired Comparison Selling
The most common method of taste testing is paired comparison. The
consumer is asked to sample two different products and select the
one with the most appealing taste. The test is done in private and a
minimum of 1,000 responses is considered an adequate sample. A
blind taste test for a soft drink, where imagery, self-perception and
brand reputation are very important factors in the consumer’s
purchasing decision, may not be a good indicator of performance in
the marketplace. The introduction of New Coke illustrates this point.
New Coke was heavily favored in blind paired comparison taste tests,
but its introduction was less than successful, because image plays a
major role in the purchase of Coke.

A paired comparison
taste test
Comparative Scaling Techniques
Rank Order Scaling
 Respondents are presented with several objects
simultaneously and asked to order or rank them according
to some criterion.
 It is possible that the respondent may dislike the brand
ranked 1 in an absolute sense.
 Furthermore, rank order scaling also results in ordinal data.

 Only (n - 1) scaling decisions need be made in rank order

scaling.
Preference for Toothpaste Brands Using Rank Order Scaling
Figure 8.4

Instructions: Rank the various brands of toothpaste in order of

preference. Begin by picking out the one brand that you like
most and assign it a number 1. Then find the second most
preferred brand and assign it a number 2. Continue this
procedure until you have ranked all the brands of
toothpaste in order of preference. The least preferred brand
should be assigned a rank of 10.
No two brands should receive the same rank number.
The criterion of preference is entirely up to you. There is no
right or wrong answer. Just try to be consistent.
Preference for Toothpaste Brands Using Rank Order Scaling
Figure 8.4 cont.

Form
Brand Rank Order
1. Crest _________
2. Colgate _________
3. Aim _________
4. Gleem _________
5. Macleans _________

6. Ultra Brite _________

7. Close Up _________
8. Pepsodent _________
9. Plus White _________
10. Stripe _________
Comparative Scaling Techniques
Constant Sum Scaling
 Respondents allocate a constant sum of units, such as 100
points to attributes of a product to reflect their importance.
 If an attribute is unimportant, the respondent assigns it zero
points.
 If an attribute is twice as important as some other attribute,
it receives twice as many points.
 The sum of all the points is 100. Hence, the name of the
scale.
Importance of Bathing Soap Attributes Using a Constant
Sum Scale
Figure 8.5

Instructions
On the next slide, there are eight attributes of
bathing soaps. Please allocate 100 points among
the attributes so that your allocation reflects the
relative importance you attach to each attribute.
The more points an attribute receives, the more
important the attribute is. If an attribute is not
at all important, assign it zero points. If an
attribute is twice as important as some other
attribute, it should receive twice as many points.
Importance of Bathing Soap Attributes Using a Constant Sum
Scale
Figure 8.5 cont.

Form
Average Responses of Three Segments

Attribute Segment
8 I Segment
2 II Segment
4
III 2 4 17
1. Mildness 3 9 7
2. Lather 53 17 9
3. Shrinkage 9 0 19
4. Price 7 5 9
5. Fragrance 5 3 20
6. Packaging 13 60 15
Sum 100 100 100
7. Moisturizing
8. Cleaning Power
 Measurement Basics
Measurement type and scale - summary

Scale type Admissible Examples

transformations

Nominal 1-1 mapping Labeling, classifying

entities

Ordinal Monotonic increasing Preference, hardness, air

function quality, intelligence tests
(raw scores)
Interval M’=aM+b, a >0 Relative time,
temperature (Fahrenheit,
Celsius), intelligence tests
(standardized scores)
Ratio M’=aM, a> 0 Time interval, length,
temperature (Kelvin)

Absolute M’=M Counting entities

65
 Measurement Basics
 Meaningfulness in measurement
 After making measurements, key question is “can we deduce
meaningful statements about entities being measured?”
 Harder to answer than it first appears – consider these
statements:
1. The number of errors discovered during the integration testing of a
program X was at least 100
2. The cost of fixing each error in program X is at least 100
3. A semantic error takes twice as long to fix as a syntactic error
4. A semantic error is twice as complex as a syntactic error

66
 Measurement Basics
Meaningfulness in measurement (cont’d)
First statement seems to make sense
Second statement doesn’t make sense – number of errors may
be specified without reference to a particular scale, but cost to
fix them must be
Statement 3 seems sensible – the ratio of time taken is the
same, whether time is measured in second, hours, or fortnights
Statement 4 does not appear to be meaningful and requires
clarification:
 If complexity means time to understand the error, than it makes sense
 Other definitions of complexity may not admit measurement on a ratio
scale (e.g. examples in previous slides) in which case statement 4 is
meaningless.

67
 Measurement Basics
Meaningfulness in measurement
Definition: a statement involving measurement is
meaningful if its truth value is invariant of
transformations of allowable scales.

68
 Measurement Basics
Meaningfulness in measurement – examples
John is twice as tall as Fred
 Implies measures are at least on the ratio scale. It’s meaningful because no
matter what transformation we use (and all we have is ratio transformations),
the truth or falsity of the statement remains constant.
Temperature in Tokyo today is twice that in London
 Implies a ratio scale, but is not meaningful. We measure in ° F and ° C. If
Tokyo is 40° C and London is 20° C, then the statement is true, but if Tokyo
is 104° F and London is 68° F, the statement is no longer true.
Failure x is twice as critical as failure y
 Not meaningful if we only have an ordinal scale for criticality (common scale
for software failures is catastrophic, significant, moderate, minor, and
insignificant).

69
 Measurement Basics
Meaningfulness in measurement
Note that our notion of meaningfulness says nothing
about
 Usefulness
 Practicality
 Worthwhile
 Ease of measurement

70
 Measurement Basics
 Statistical operations on measures
 Analyses don’t have to be sophisticated, but we want to know
something about how a set of data is distributed.
 What types of statistical analysis are relevant to a given
measurement scale?
Scale type Defining relations Examples of appropriate statistics

Nominal Equivalence Mode, Frequency

Ordinal Equivalence, Median, Percentile, Spearman r, Kendall r,

Greater than Kendall W
Interval Equivalence, Mean, Standard deviation, Pearson
Greater than, product-moment correlation, Multiple
Known ratio of any intervals product-moment correlation

Ratio Equivalence, Geometric mean,

Greater than, Coefficient of variation
Known ratio of any intervals,
Known ratio of any two scale values

71
 Measurement Basics
Indirect measurement and meaningfulness
Done when measuring a complex attribute in terms of simpler
sub-attributes
Scale type for an indirect measure M is generally no stronger
than the weakest of the scale types of the sub-attributes
 Example – testing efficiency=defects/effort
 Defects is on the absolute scale, while effort is on the ratio scale. Therefore
effort is on the ratio scale.
 What is E=2.7v+121w+26x+12y+22z-497, where
o v is the number of program instructions
o x and y are the number of internal and external documents
o z is the program size in words
o w is a subjective measure of complexity

72
Halstead’s Metrics
 Amenable to experimental verification [1970s]

 Program length: N = N1 + N2
 Program vocabulary: n = n1 + n2

 Estimated length:
N̂ = n1 log2 n1 + n2 log2 n2
 Close estimate of length for well structured programs

 Purity ratio: PR =
N̂ /N
Program Complexity
 Volume: V = N log2 n
 Number of bits to provide a unique designator for each of the n
items in the program vocabulary.

 Difficulty

 Program effort: E=D*V

 This is a good measure of program understandability
McCabe’s Complexity Measures
McCabe’s metrics are based on a control flow
representation of the program.
A program graph is used to depict control flow.
Nodes represent processing tasks (one or more code
statements)
Edges represent control flow between nodes
Flow Graph Notation
While
Sequence

If-then-else Until
Cyclomatic Complexity
Set of independent paths through the graph (basis set)

V(G) = E – N + 2
E is the number of flow graph edges
N is the number of nodes

V(G) = P + 1
P is the number of predicate nodes
Example
i = 0;
while (i<n-1) do
j = i + 1;
while (j<n) do
if A[i]<A[j] then
swap(A[i], A[j]);
end do;
i=i+1;
end do;
Flow Graph
1

7 4 5

6
Computing V(G)
V(G) = 9 – 7 + 2 = 4
V(G) = 3 + 1 = 4
Basis Set
1, 7
1, 2, 6, 1, 7
1, 2, 3, 4, 5, 2, 6, 1, 7
1, 2, 3, 5, 2, 6, 1, 7
Another Example
1

2
4
3

5 6

8
What is V(G)?
Meaning
 V(G) is the number of (enclosed) regions/areas of the
planar graph

 Number of regions increases with the number of decision

paths and loops

 A quantitative measure of testing difficulty and an

indication of ultimate reliability

 Experimental data shows value of V(G) should be no more

then 10 - testing is very difficulty above this value
McClure’s Complexity Metric
Complexity = C + V
C is the number of comparisons in a module
V is the number of control variables referenced in the
module
decisional complexity

Similar to McCabe’s but with regard to control

variables
Metrics and Software Quality
FURPS

Functionality - features of system

Usability – aesthesis, documentation
Reliability – frequency of failure, security
Performance – speed, throughput
Supportability – maintainability
 Measures of Software Quality
 Correctness – degree to which a program operates according to
specification
 Defects/KLOC
 Defect is a verified lack of conformance to requirements
 Failures/hours of operation

 Maintainability – degree to which a program is open to change

 Mean time to change
 Change request to new version (Analyze, design etc)
 Cost to correct

 Integrity - degree to which a program is resistant to outside attack

 Fault tolerance, security & threats

 Usability – easiness to use

 Training time, skill level necessary to use, Increase in productivity, subjective questionnaire
or controlled experiment
 McCall’s Triangle of Quality
Maintainability Portability
Flexibility Reusability
Testability Interoperability

PRODUCT REVISION PRODUCT TRANSITION

PRODUCT OPERATION
Correctness Usability Efficiency
Reliability Integrity
 A Comment
McCall’s quality factors were proposed in the
early 1970s. They are as valid today as they were
in that time. It’s likely that software built to conform
to these factors will exhibit high quality well into
the 21st century, even if there are dramatic changes
in technology.
Quality Model

product

operation revision transition

reliability efficiency usability maintainability testability portability reusability

Metrics
High level Design Metrics
Structural Complexity
Data Complexity
System Complexity
Card & Glass ’80

Structural Complexity S(i) of a module i.

S(i) = fout2(i)
Fan out is the number of modules immediately
subordinate (directly invoked).
Design Metrics
Data Complexity D(i)
D(i) = v(i)/[fout(i)+1]
v(i) is the number of inputs and outputs passed to and
from i

System Complexity C(i)

C(i) = S(i) + D(i)
As each increases the overall complexity of the
architecture increases
System Complexity Metric
Another metric:
length(i) * [fin(i) + fout(i)]2
Length is LOC
Fan in is the number of modules that invoke i

Graph based:
Nodes + edges
Modules + lines of control
Depth of tree, arc to node ratio
Coupling
 Data and control flow
 di – input data parameters
 ci input control parameters
 do output data parameters
 co output control parameters

 Global
 gd global variables for data
 gc global variables for control

 Environmental
 w fan in
 r fan out
Metrics for Coupling
Mc = k/m, k=1

m = di + aci + do + bco + gd + cgc + w + r

a, b, c, k can be adjusted based on actual data
Component Level Metrics
 Cohesion (internal interaction) - a function of data objects

 Coupling (external interaction) - a function of input and

output parameters, global variables, and modules called

 Complexity of program flow - hundreds have been

proposed (e.g., cyclomatic complexity)

 Cohesion – difficult to measure

 Bieman ’94, TSE 20(8)
Using Metrics
The Process
Select appropriate metrics for problem
Utilized metrics on problem
Assessment and feedback

Formulate
Collect
Analysis
Interpretation
Feedback
Metrics for the Object Oriented
Chidamber & Kemerer ’94 TSE 20(6)

Metrics specifically designed to address object

oriented software

Class oriented metrics

Direct measures
Weighted Methods per Class
n

WMC = 
ci
i 1

ci is the complexity (e.g., volume, cyclomatic

complexity, etc.) of each method

Viewpoints: (of Chidamber and Kemerer)

-The number of methods and complexity of methods is an indicator of how

much time and effort is required to develop and maintain the object
-The larger the number of methods in an object, the greater the potential
impact on the children
-Objects with large number of methods are likely to be more application
specific, limiting the possible reuse
Depth of Inheritance Tree
DIT is the maximum length from a node to the root
(base class)

Viewpoints:
 Lower level subclasses inherit a number of methods
making behavior harder to predict
 Deeper trees indicate greater design complexity
 Number of Children
NOC is the number of subclasses immediately
subordinate to a class

Viewpoints:
 As NOC grows, reuse increases - but the abstraction may be diluted

 Depth is generally better than breadth in class hierarchy, since it promotes

reuse of methods through inheritance

 Classes higher up in the hierarchy should have more sub-classes then those
lower down

 NOC gives an idea of the potential influence a class has on the design:
classes with large number of children may require more testing
 Coupling between Classes
CBO is the number of collaborations between two classes
(fan-out of a class C)
the number of other classes that are referenced in the class C
(a reference to another class, A, is an reference to a method
or a data member of class A)
Viewpoints:
 As collaboration increases reuse decreases
 High fan-outs represent class coupling to other classes/objects and thus are
undesirable
 High fan-ins represent good object designs and high level of reuse
 Not possible to maintain high fan-in and low fan outs across the entire
system
Response for a Class
RFC is the number of methods that could be called
in response to a message to a class (local + remote)

Viewpoints:
As RFC increases
 testing effort increases
 greater the complexity of the object
 harder it is to understand
Lack of Cohesion in Methods
LCOM – poorly described in Pressman

Class Ck with n methods M1,…Mn

Ij is the set of instance variables used by Mj

LCOM
There are n such sets I1 ,…, In
P = {(Ii, Ij) | (Ii  Ij ) = }
Q = {(Ii, Ij) | (Ii  Ij )  }
If all n sets Ii are  then P = 

LCOM = |P| - |Q|, if |P| > |Q|

LCOM = 0 otherwise
Example LCOM
Take class C with M1, M2, M3
I1 = {a, b, c, d, e}
I2 = {a, b, e}
I3 = {x, y, z}
P = {(I1, I3), (I2, I3)}
Q = {(I1, I2)}

Thus LCOM = 1
Explanation
LCOM is the number of empty intersections minus
the number of non-empty intersections

This is a notion of degree of similarity of methods

If two methods use common instance variables

then they are similar

LCOM of zero is not maximally cohesive

|P| = |Q| or |P| < |Q|
Some other cohesion metrics
Class Size
CS
Total number of operations (inherited, private, public)
Number of attributes (inherited, private, public)

May be an indication of too much responsibility for a

class
Number of Operations Overridden
NOO

A large number for NOO indicates possible problems

with the design
Poor abstraction in inheritance hierarchy
Number of Operations Added
NOA

The number of operations added by a subclass

As operations are added it is farther away from super
class
As depth increases NOA should decrease
Method Inheritance Factor
n

MIF 
= M i (Ci ) .
i 1
n

 M (C )
M (C ) is the number of methods inherited and not
i 1
a i
i i
overridden in Ci
Ma(Ci) is the number of methods that can be invoked
with Ci
Md(Ci) is the number of methods declared in Ci
MIF
Ma(Ci) = Md(Ci) + Mi(Ci)
All that can be invoked = new or overloaded + things
inherited

MIF is [0,1]
MIF near 1 means little specialization
MIF near 0 means large change
Coupling Factor

CF= 
i j
is _ client (Ci , C j ) .
(TC 2  TC )
is_client(x,y) = 1 iff a relationship exists between
the client class and the server class. 0 otherwise

(TC2-TC) is the total number of relationships

possible

CF is [0,1] with 1 meaning high coupling

Polymorphism Factor
PF =
Mi o
(C i
) .

i M n (Ci )  DC (Ci )

 Mn() is the number of new methods

 Mo() is the number of overriding methods

 DC() number of descendent classes of a base class

 The number of methods that redefines inherited methods, divided by

maximum number of possible distinct polymorphic situations
Operational Oriented Metrics
Average operation size (LOC, volume)

Number of messages sent by an operator

Operation complexity – cyclomatic

Average number of parameters/operation

Larger the number the more complex the collaboration
Encapsulation
Lack of cohesion

Percent public and protected

Public access to data members

Inheritance
Number of root classes

Fan in – multiple inheritance

NOC, DIT, etc.

Metric tools
 McCabe & Associates ( founded by Tom McCabe, Sr.)

 The Visual Quality ToolSet

 The Visual Testing ToolSet
 The Visual Reengineering ToolSet

 Metrics calculated

 McCabe Cyclomatic Complexity

 McCabe Essential Complexity
 Module Design Complexity
 Integration Complexity
 Lines of Code
 Halstead
CCCC
 A metric analyser C, C++, Java, Ada-83, and Ada-95 (by
Tim Littlefair of Edith Cowan University, Australia)

 Metrics calculated
Lines Of Code (LOC)
McCabe’s cyclomatic complexity
C&K suite (WMC, NOC, DIT, CBO)

 Generates HTML and XML reports

 freely available

 http://cccc.sourceforge.net/
Jmetric
 OO metric calculation tool for Java code (by Cain and
Vasa for a project at COTAR, Australia)

 Requires Java 1.2 (or JDK 1.1.6 with special extensions)

 Metrics
Lines Of Code per class (LOC)
Cyclomatic complexity
LCOM (by Henderson-Seller)

 Availability
is distributed under GPL

 http://www.it.swin.edu.au/projects/jmetric/products/jmetric/
JMetric tool result
GEN++
(University of California,  Davis and  Bell Laboratories)

 GEN++ is an application-generator for creating code analyzers for C++

programs

 simplifies the task of creating analysis tools for the C++

 several tools have been created with GEN++, and come with the
package
 these can both be used directly, and as a springboard for other
applications
 
 Freely available
 http://www.cs.ucdavis.edu/~devanbu/genp/down-red.html
More tools on Internet
 A Source of Information for Mission Critical Software
Systems, Management Processes, and Strategies
http://www.niwotridge.com/Resources/PM-SWEResources/SWTools.htm
 Defense Software Collaborators (by DACS)
http://www.thedacs.com/databases/url/key.hts?keycode=3
http://www.qucis.queensu.ca/Software-Engineering/toolcat.html#label208
 Object-oriented metrics
http://me.in-berlin.de/~socrates/oo_metrics.html
 Software Metrics Sites on the Web (Thomas Fetcke)
 Metrics tools for C/C++ (Christofer Lott)
http://www.chris-lott.org/resources/cmetrics/