Software Reliability
Definition
Reliability is the probability of failure-
free operation of a system over a
specified time within a specified
environment for a specified purpose.
Introduction
It gives the measure of :
How much system matches with stated
specifications?
How well the system provides the
required services?
Introduction
It is difficult to define Reliability Term Objectively ,
also to measure the user’s expectations or
environmental factors
While considering reliability , Important factor to
discuss is about the System’s Failure.
All failures are not equal ; some of them gives very
serious consequences or some failures are reasonably
Failures and Faults
Failure : Unexpected runtime
behaviour observed by a user of the
software.
Fault : This is Static software
characteristic which causes a failure
to occur.
Failures and Faults
Every Fault does not lead to the failure
Examples :
Code which mostly correct
Dead or infrequently used code
If user can not notice the wrong behaviour , its
not a Fault.
If a user expects Wrong Behaviour , it’s not a
Fault
Improving Reliability
Fixing N% of the faults does not mean that
lead to an N% reliability improvement.
One of the study showed that removing 60%
of software faults led to a 3% reliability
improvement.
Types of Failures
Hardware Failure
Software Failure
Types of Failures
Hardware Failures : Mostly always physical failures.
Software Failures : Due to design faults.
Hardware reliability metrics are not much
appropriate to measure software reliability but that is
how they have evolved.
Reliability metrics
Reliability metrics are used to quantitatively expressed
the reliability of the software product.
The parameter to be used to measure the reliability
depends upon the type of system & the requirements of
the application domain.
It is difficult to find a suitable method to measure
software reliability.
The reliability cannot be measured directly
It is not like something gets measured and that gets
reflected as reliability.
Reliability metrics
Metrics
Product metrics : Are those which are used to build the
artefacts.
Artefacts such as requirement specification documents, system
design documents, etc.
Project Management Metrics : Project characteristics and
execution.
If there is proper management of the project by the
programmer, then this helps us to achieve better
products.
A relationship exists between the development process and
the ability to complete projects on time and within the
desired quality objectives.
Metrics
Process Metrics : Process metrics quantify useful
attributes of the software development process & its
environment.
They tell if the process is functioning optimally as
they report on characteristics like cycle time &
rework time.
The goal of process metric is to do the right job on
the first time through the process.
Metrics
Fault and Failure Metrics :
A fault is a defect in a program which
appears when the programmer makes an
error and causes failure when executed
under particular conditions.
These metrics are used to determine the
failure-free execution software.
Product Matrices
Here measurements are taken from the actual
body of the source code
Software size : considered to be complexity,
development effort, and reliability
Lines of Code (LOC), or LOC in
thousands (KLOC), is an initial approach to
measure the software size.
The basis of LOC is that program length can be used as a
predictor of program characteristics such as effort &ease
of maintenance.
It is a measure of the functional complexity of the
program and is independent of the programming
language.
Product Matrices
Function point metric is a technique to measure the
functionality of proposed software development based
on the count of inputs, outputs, master files, inquires,
and interfaces.
Test coverage metric size fault and reliability by
performing tests on software products, assuming that
software reliability is a function of the portion of software
that is successfully verified or tested.
Product Matrices
Complexity is directly linked to software reliability, so
representing complexity is essential. Complexity-oriented
metrics is a way of determining the complexity of a
program's control structure by simplifying the code into a
graphical representation. The representative metric is
McCabe's Complexity Metric.
Quality metrics measure the quality at various steps of
software product development. An vital quality metric
is Defect Removal Efficiency (DRE). DRE provides a
measure of quality because of different quality assurance
and control activities applied throughout the
development process.
Project Management Metrics
A relationship exists between the development process
and the ability to complete projects on time and within
the desired quality objectives.
Cost increase when developers use inadequate methods.
Higher reliability can be achieved by using a better
development process, risk management process,
configuration management process.
Project Management Metrics
These metrics are:
Number of software developers
Staffing pattern over the life-cycle of the
software
Cost and schedule
Productivity
Process Metrics
Process metrics describe the effectiveness and quality of
the processes that produce the software product.
The effort required in the process
Time to produce the product
Effectiveness of defect removal during development
Number of defects found during testing
Maturity of the process
Fault and Failure Metrics
These metrics are used to determine the failure-free
execution software.
To achieve this objective, a number of faults found during
testing and the failures or other problems which are
reported by the user after delivery are collected,
summarized, and analyzed.
Failure metrics are based upon customer information
regarding faults found after release of the software.
The failure data collected is therefore used to calculate
failure density, Mean Time between Failures (MTBF), or
other parameters to measure or predict software
reliability.
Example
Product :
1. Design features are good or not ?
2. Outlook is good or not ?
3. Is it giving comfort to the user or not ?
4. Is it attractive enough to use by the customer ?
Performance gets checked
This gets checked in this type of metrics
Example
Process
1. How much effective is the implemented process is ?
2. Is it capable for completing the task or not ?
3. Quality of the process gets measured
4. Is it following deadlines?
5. Is it matured model ? Checks appropriate outputs are
coming or not ?
Example
Project :
1. Team size
2. Staff coordination
3. Productivity of the staff
Example
Fault & Failure
1. Programming Error
2. a number of faults found during testing
3. The failures or other problems which are reported
by the user
Process Metrics
Process : Focuses on each step to be followed during
software product development
It is long term
Main Goal is to get a good quality of Product
References
Material drawn from [Somerville, Mancoridis],
https://www.cs.drexel.edu/~spiros/teaching/CS576/slides
/9.reliability.pdf
https://www.javatpoint.com/software-engineering-
software-reliability-measurement-techniques
