Advanced Software
Engineering
University of Colombo
Prabha Kularathna
Quality
IEEE Glossary:
Degree to which a system, component, or
process meets (1) specified requirements, and
(2) customer or user needs or expectations
ISO:
The totality of features and characteristics of a
product or service that bear on its ability to
satisfy specified or implied needs
Software Quality
Quality of design
Refers to characteristics designers specify for the end
product to be constructed
Quality of conformance
Degree to which design specifications are followed in
manufacturing the product
Why Software Quality Matters?
London Ambulance Disaster
1992 October 26
New automated ambulance dispatch system for London
Ambulance Services
2000-2500 calls per day
200+ ambulances
Replaced human operated system with a new software
Goal: reduce average emergency response time down to 3
minutes
CAD
Released without load-testing
81 known issues
No fallback process
No user consultation during requirements gathering phase
Had memory leaks & threading issues
System cant handle incomplete data
46 deaths
Why Software Quality Matters?
60* to 100*
1.5* to 6*
Cost
1*
Definition
Development
Cost to fix faults
Post Release
Challenges
Tension between customer quality
requirements (efficiency, reliability,
etc.) and developer quality
requirements (maintainability,
reusability, etc.)
Some quality requirements are
difficult to specify in an unambiguous
way
Software specifications are usually
incomplete and often inconsistent
Quality Costs
Prevention costs
Quality planning, test tools, training
Appraisal costs
Reviews, refactoring, testing, etc. during
development phase.
Failure costs
Diagnosing & bug fixing
External failure costs
Bug reporting, handling, communications, delivery &
installation of fixes.
Software Quality Characteristics
1. Correctness
Degree of compliance to the requirement specification
Leads to functional defects
Challenging when building with large software
systems
Conflicting requirements get discovered during
development
Software Quality Characteristics
2. Efficiency
Efficiency in performing the function/functions that the
software was built for.
1. User Experience factor
How fast user can use the software
Data creation efficiency
Data consumption efficiency
2. Resource utilization
Temporal: How fast intended functionality takes to
execute
Spatial: How much memory is consumed to
execute the intended functionality
3. Efficiency in data flow in overall system
Efficiency in data flow automation
Software Quality Characteristics
3. Flexibility
The ease of changing / adapting during its lifecycle
Changing requirements during development and
maintenance
Flexibility leads to generic designs and
architectures
Usually increases complexity
Tradeoff: depends on the problem domain
Aircraft ILS
Android Launcher
Software Quality Characteristics
4. Maintainability
The ease of maintaining the software across changes
during maintenance phase.
Needs well designed architecture
Critical for large systems
Exceptions:
Throw-away systems with limited lifespan
Ex:
Small scale web applications
Small scale mobile apps
Software Quality Characteristics
5. Reliability
The ability of a software system to operate in its target
environment without experiencing failure (system-down
scenarios)
Mission Critical Systems
Systems whose failure can lead to catastrophic
circumstances
Ex: death / damage to property / financial losses
Irrespective of criticality, software failures damage reputation /
user confidence.
Software Quality Characteristics
6. Interoperability
Describes the ease of a software system to
interoperate with other systems
Software integration
Software architecture
Loosely coupled components
Software Quality Characteristics
7. Reusability
The ability to re-use systems components / assets
across multiple systems
Modularity
Separation of functional and infrastructural layers
Software Quality Characteristics
8. Portability
Ease of transferring existing functionality from
One technology to another / multiple technologies
One platform to another / multiple platforms
Separation of technology-dependent logic (usually UI &
IO) from technology-independent logic (usually data
model / business logic)
Use of cross-platform technologies
Java
Xamarin
HTML5 / JavaScript
Software Quality Characteristics
9. Code Coverage
The degree to which the code base of a software can
be tested by automated-testing
Software testing time/cost across development and
maintenance
Enforces Correctness
Increases development and maintenance time
May increase code complexity
Software Quality Characteristics
10. Integrity
Degree of conformity to QA testing.
A higher Code Coverage is a requirement to measure code
integrity
Code coverage is not a must to enforce code integrity
Focus on maintaining higher software quality throughput
development and maintenance
Software Quality Characteristics
11. Testability
The degree to which the system can be tested at a given test
context.
Higher testability leads to assuring integrity at a lower cost
Repeatability (automated testing)
Reproducible deterministic states input / output
Ease of writing test cases
Maintainability of test cases
Implementati
on
Requirement
s
Test cases
Software Quality Measurement
Quality of
Requirements
1. Ambiguity
Number of Weak phrases
can, could, may
Weak: The results of the Initialization checks may be reported
in a special file.
Non-weak: The results of the Initialization checks shall be
reported in a special file
Number of Optional phrases
Phrases which contain an option / options
Optional: The system shall be such that the mission can be
pursued, possibly without performance degradation.
Non-optional: The system shall be such that the mission can
Software Quality Measurement
Quality of
Requirements
2. Completeness
Number of To-be-determined (TBD)
Number of To-be-added (TBA)
Software Quality Measurement
Quality of
Requirements
3. Understandability
Automated Readability Index
Software Quality Measurement
Quality of
Requirements
4. Volatility
Software Quality Measurement
Quality of
Requirements
5. Traceability
Number of requirements not traced to system
requirements.
Number of requirements not traced to code and
tests.
Software Quality Measurement
Quality of Code
1. Total defect density
Measured as defects divided by KLOC
(number of defects removed in a given
period, per thousand LOC)
n Number of defects confirmed and
agreed upon (dropped defects are not
considered)
Period ex: a given month, quarter,
year, or a phase in software lifecycle
Software Quality Measurement
Quality of Code
2. Defect Removal Efficiency (DRE)
E : number of errors found before delivery of the software
to the user
D: number of defects found after delivery
Software Quality Measurement
Quality of Code
3. Code Integrity
E : number of errors found before delivery of the software
to the user
D: number of defects found after delivery
Software Quality Measurement
Quality of Code
4. Internal Documentation
Comment percentage.
Software Quality Measurement
Quality of Code
5. External Documentation
Automated Readability Index.
Software Quality Measurement
Implementation
Efficiency
1. Resource Usage Efficiency
Staff hours spent on lifecycle activities
Software Quality Measurement
Implementation
Efficiency
2. Completion Rates
Task completions
Planned task completions
Software Quality Measurement
Test Efficiency
1. Correctness
Errors and criticality.
Time of finding of errors.
Time of error fixes.
Code Location of fault.
Software Quality Measurement
Operational Quality
1. Mean Time to Failure (a.k.a. Mean Time Before
Failure)
Software Quality Measurement
Operational Quality
2. Mean Time to Repair
Software Quality Measurement
Operational Quality
3. Availability
Software Quality Measurement
Operational Quality
3. Reliability
Software Quality Assurance (SQA)
Set of systematic activities providing evidence of the
ability of the software process to produce a software
product that is fit to use
G. Schulmeyer and J. McManus, Software Quality Handbook, Prentice Hall, 1998.
Conformance to software requirements is the foundation from which
software quality is measured.
Software must conform to implicit requirements (ease of use,
maintainability, reliability, etc.) as well as its explicit requirements.
Software Quality Assurance (SQA)
Monitoring processes and products throughout the software
development lifecycle to ensure the quality of the delivered
product(s)
Monitoring the processes
Provides management with objective feedback regarding process
compliance to:
Approved plans
Procedures
Standards
Analyses
Monitoring the products
Focuses on the quality of product within each phase of the SDLC
Requirements
Architecture
Test plans
Software Quality Assurance (SQA)
Process Assessment:
Use of standards and process models has a positive impact
on the quality of the software product
ISO 9001
CMM
CMU SEI, 5 levels
SPICE (Software Process Improvement and
Capability Determination)
a.k.a. ISO/IEC 15504
ISO joint committee, Europe, Australia
IEEE 1074, IEEE 12207,
Software Quality Assurance (SQA)
Product Assessment:
Architectural Reviews,
Code Reviews
Inspections
Measurement via metrics
Testing
Simulation
Prototyping
Formal verification
Model checking, theorem proving
