Code Analysis For Quality
Code Analysis For Quality
Code Analysis
for
Quality
in
High Integrity Systems
Currie Colket
The MITRE Corporation
Phone: (703) 883-7381
Email: [email protected] | [email protected]
3
High Integrity Systems - 1
Adherence to Guidelines or Standards has to be demonstrated
Safety – IEC 610508 International Generic Standard (Part 3
concerned with software)
Security – ISO 15408 – Multinational Generic Assessment Guide
Sector Specific Guidelines
6
Introduction
Code Analysis
10 Analyses frequently called out by High-Integrity Standards
Control Flow Analysis
Data Flow Analysis
Information Flow Analysis
Symbolic Flow Analysis
Formal Code Verification Many more, but these
Range Checking are the common ones
Stack Usage Analysis
Timing Analysis
Other Memory Usage Analysis
Object Code Analysis
7
Code Analysis –
1. Control Flow Analysis
Conducted to:
Analyses:
- Pre-condition/Post-condition analysis
- Demonstrate a particular safety/security property
- Termination of all loops
- Termination of any recursion (not normally permitted)
- Proof of absence of run time errors
Analyses:
Analyses:
Claims
Supports
Arguments
Evidence
Argument
Static Analysis
Other Has Identified
No Errors
Is Evidence For
Evidence
No errors
Detected on
Set Use Case
… Evidence
No errors
Detected on
Coding Violation
Analysis Analysis
18
Overview
Introduction
High Integrity
Code Analysis
Quality
Problems with Code Analysis
Automation of Code Analysis
Experiences with Code Analysis
Conclusion
19
How Not To Do
Systems
Engineering
For Quality
And The
Sinking Of
The Largest
Offshore
Oil Platform
March 2001
Disclaimer:
Slides
Received
From
Unknown
Author
20
For those of you who may
be involved in the
engineering of systems
21
Please read this quote from
a Petrobras executive, 22
extolling the benefits of
cutting quality assurance
and inspection costs,
23
on the project that
sunk into the Atlantic
Ocean off the coast of
Brazil in March 2001.
24
"Petrobras has established new global benchmarks for
the generation of exceptional shareholder wealth 25
through an aggressive and innovative programme
of cost cutting on its P36 production facility. 26
Conventional constraints have been successfully challenged
27
and replaced with new paradigms appropriate to
the globalised corporate market place.
28
Through an integrated network
of facilitated workshops,
29
the project successfully rejected the established constricting
and negative influences of prescriptive engineering, 30
onerous quality requirements, and
outdated concepts of inspection
and client control. 31
Elimination of these unnecessary straitjackets has
empowered the project's suppliers and contractors
32
to propose highly economical solutions,
with the win-win bonus of enhanced
profitability margins for themselves. 33
The P36 platform shows the shape of things to come
http://www.sustainability.com/developing-value/details.asp?bcid=128&sfid=3&bsid=3 34
in unregulated global market economy of the 21st Century.”
35
And now you have seen the final result of
this proud achievement by Petrobras.
36
Ada Engineered Product
Boeing 777 Commercial Aviation
Boeing 737
Boeing 747
Boeing 757
Boeing 767
Boeing 777
37
Overview
Introduction
High Integrity
Code Analysis
Quality
Problems with Code Analysis
Automation of Code Analysis
Experiences with Code Analysis
Conclusion
38
Problems With Code
Analysis
Compilation Environment
A
P
Source
P S
Code Compile
Compile Link L Y
I S
Syntactic Data C T
for Code
Analysis Syntactic &
A E
1. Semantic Data not Semantic Data for T M
Code Analysis
true to Compiler I
2. Many Semantic 1. Costly to implement
2. Change in compiler O
analyses too
complex breaks tool N
3. Tools are costly 3. No consistent set of
Code Analysis Tool queries
39
ASIS – An Interface to Support Code Analysis
The Ada Semantic Interface Specification (ASIS) exception
handling mechanism is a good example demonstrating the types of
things useful to do in a design.
40
ASIS became an ISO Standard in 1999
See http://www.acm.org/sigada/WG/asiswg
What is ASIS?
Ada Environment
A
Ada P
Source P S
Code Compile Link L Y
I S
C T
A E
ASIS Interface T M
proprietary standard
I
interface interface O
N
CASE Tool or Application
Provides Syntactic and Semantic Information from Ada Environment using a standard interface
41
Syntactic Information
Ada syntax is summarized in Ada 95 RM, Annex P as variant of Backus-Naur Form
For example:
object_declaration ::==
defining_identifier_list : [aliased] [constant] subtype_indication [:= expression]; | ...
Syntactic Element
Object Object Declaration
Tree Representation =>
A B : Latitude := 0.0
ASIS can extract desired syntactic information for every syntactic category
Of the 367 ASIS Queries, most support syntactic tree analysis
42
Semantic Information
Type
Ada semantics are provided via Digits Type Definition
mechanisms such as Corresponding_: Full Type Real Type Definition
Corresponding_Type_Declaration, Declaration
Corresponding_Name_Definition,
Floating Point Definition
Corresponding_Called_Function,
Corresponding_Called_Entity,
Corresponding_Type, Latitude 8
Corresponding_Entry, Range
etc.
Defining Static Expression
Identifier Real Range
Specification
Corresponding_Expression_Type
-90.0 +90.0
Corresponding_ Name_Declaration Static Simple Expressions
Corresponding_Name_Declaration
A B : Latitude := 0.0
A := A + B;
Element
Text Span
Kind Text Image
Component Elements
12 November 2000 ASIS 44
Ove
• Introduction rvie
• High Integrity w
• Code Analysis
• Quality
• Problems with Code Analysis
• Automation of Code Analysis
• Experiences with Code Analysis
• Conclusion
46
Analysis Tools
Design Assessment Workbench (ASIS-based)
48
Views and Measures for Data Extraction
VIEW MEASURE AFFECTED PRINCIPLE
Program Units Unit Distribution Abstraction, Information Hiding
Package Abstraction, Explicit Interfaces,
Distribution Information Hiding, Weak Coupling
Name Space Visible Abstraction, Information Hiding,
Declarations in a Modularity, Weak Coupling
Library Package
Subprograms Subprogram Explicit Interfaces, Modularity, Weak
Parameter Count Coupling
Cyclomatic Modularity
Complexity
Compilation Unit Recompilation Minimal Interfaces, Modularity
Dependency Effect
Structure Chart Fan Out Modularity
49
Package Distribution
Not Bundled
Observation: TY PES_ ONLY
Significant
SUBPROGRAM S_ ONLY
CONSTANTS_ SUBPROGRAM S
packages that
TY PES_ SUBPROGRAM S
subprograms Bundled
TY PES_ CONS_ VARIABLES
TY PES_ VARIABLES
VARIABLES_ ONLY
VARIABLES_ SUBPROGRAM S
EXCEPTIONS_ ONLY
OTHER
50
Subprogram Parameter Distribution
Observations:
80%
Nine percent of Function
Procedure
the packages 70%
Task_Entry
export global data 60%
50%
Significant
percentage of 40%
procedures / 30%
Are these 0%
abstract state 0 1 2 3 4
Number of Parameters
5 6 >7
machines, or is
this excessive
use of global
data?
51
Compilation Unit Context Coupling
Distribution
be candidates for 40 29
23
20%
further 20 10%
decomposition 0 0%
0 1 2 3-5 6 - 10 11 - 20 21 - 30 31 - 40 >40
52
Cyclomatic Complexity Distribution
Observations: 100%
90% Function_Body
Procedure_Body
80%
Most
70%
subprograms 60%
than 10 30%
10%
values in some 0%
from further
decomposition
53
Ada System Dependency Analyzer Metrics 3
Subunits 150
Parent Program Units 10
Subunits Specified but not Found 0
Subunits Found but not Specified 0
Missing Parent Units 0
Exceptions 677
Exceptions Declared More Than Once 67
Exceptions Declared but not Raised 145
Missing Exception Declarations 0
Raise Statements 2,944
Raises Not Identified At All 0
Raises Identified More Than Once 0
Machine Code Statements 0
54
Ada Analyzer – Classification of Anomalies
Category Description Recommendation
Serious This represents a possible error, which could cause the These should be evaluated by the
program to generate erroneous results or crash. development contractor and fixed, if valid
??? This represents a possible error, which could cause the These should be evaluated by the
program to generate erroneous results or crash. development contractor and fixed, if valid
However, this is most likely caused by differences in
the development environment and the analysis
environment.
Nit This represents a possible error, which should not These should be evaluated by the
impact the mission (e.g., logging operation). development contractor and fixed, if valid
Performance This represents a situation where throughput might be These should be evaluated and fixed, if a
improved if violation addressed. significant performance improvement is
possible and desirable.
Maintenance 1 Changes in these areas might provide significant short- Changes should be implemented in the
term benefits to the software short-term.
Maintenance 2 Changes in these areas might provide long-term Changes should be implemented as units
benefits to the software are modified
Maintenance 3 Changes in these areas might provide significant Changes should be implemented as units
benefits to the system for eventual porting to new are modified
hardware or compilation environment
55
Ada Analyzer Analyses
Analysis Reports
Annotations Analysis 0
Coding Violations Analysis 55
Collected Metrics 6
Compatibility Analysis 7
Exception Analysis 1
Miscellaneous Programming Error Analysis 11
Misspelling Analysis 1
Name Anomalies Analysis 0
Representation Specification Consistency Analysis 11
Set Use Problems Analysis 3
Static Constraint Violation Analysis 7
Subprogram Execution Analysis 2
Tasking Analysis 1
Unused With Clause Analysis 7
56
Ada Analyzer – Coding Violations Analysis
Set Use Analysis Summary # # # # # # # #
Violations Serious Nit
??? Perf Main1 Main2 Main3
Address Of Dynamic Objects 149 0 149 0 0 0 0 0
Static Constraint Violations 51 37 14 0 0 0 0 0
Missing Storage Error Handler For Allocators 143 0 143 0 0 0 0 0
Blocking Operations in Protected Types 7 7 0 0 0 0 0 0
Non Locals in Guards 3 3 0 0 0 0 0 0
UnInitialized Variable Declarations 592 0 0 0 0 0 592 0
Recursive Subprograms 1 1 0 0 0 0 0 0
Functions Returning Large Types 8 0 0 0 8 0 0 0
Functions Returning Unconstrained Types 160 0 0 0 160 0 0 0
Redeclaration Of Standard Names 1 0 0 0 0 1 0 0
Task Stack Size Not Specified 8 0 0 0 0 0 0 8
Anonymous Array Type Usage 69 0 0 0 0 69 0 0
Raised Exceptions Non Propagating 7 0 7 0 0 0 0 0
Raise Predefined Exceptions 8 0 0 0 0 8 0 0
Functions With Side Effects 56 0 0 0 0 56 0 0
Exit in While and For Loops 84 0 0 0 0 84 0 0
Total 8,288 58 1,465 0 1,879 426 1,958 376
57 Sample of 55 different Coding Violations Analysis Reports
Set Use Analysis
These are violations of a parameter with a mode of "in out" which either is not used as an
"in" parameter in all paths, or is not set as an out parameter in all paths.
Most of these violations are OK. Some of these violations could be quite serious.
58
Constraint Analysis
Static Constraint Analysis Summary # # # # # # # #
Violations Serious ??? Nit Perf Main Main Main
1 2 3
60
Potentially Serious Violations - Infinite
Recursion
Infinite Recursion results a task continuously expanding its
stack until it runs out of resources and raises Storage_Error
and possibly Tasking_Error.
return Transaction_Index is
begin
61
Potentially Serious Violations
- Functions Missing Return or Raise
Function paths with no return of raise statement (9
instances)
Functions must either return a value or propagate an
exception. Otherwise the exception Program_Error is raised.
Example:
if Fd = Error then Evaluates global & raises
exception for all but 1 case;
Error_Detected := True; Also global “Fd” subject to
Convert_Exception; race condition
else
return Fd;
62
Potentially Serious Violations
- Non-Locals in Guards
References to non-locals in guards should not be
made since references will not be synchronized.
Task Task body Critical_Resource is
Seize Claimed: Boolean := False; -- local
begin -- declaration
Release
loop -- Simple Example with Local Guard
select
when not Claimed =>
task Critical Resource is accept Seize do
entry Seize; Claimed := True;
entry Release; end;
end Critical_Resource; or
when Claimed =>
accept Release do
If Claimed Non-local (e.g., Claimed := False;
declared in specification or end;
another package, another ...
task could change Claimed end select;
to seize resource; also end loop;
subject to race conditions end Critical_Resource;
63 3 Instances Flagged
Unintended Violations - Operator Renames
Operator Renames: (4 instances )
64
Maintenance Violations - Anonymous Types
Anonymous Arrays have limited usefulness and complicate
program modifications. For example, they cannot be formal
parameters in subprograms as anonymous types have no
type.
65
Exception Gotchas
Missing Storage Error Handler For Allocators
Any allocator can exhaust the available space for the collection,
the use of allocators should be limited and the "out of memory"
case handled locally. An exception handler for Storage Error
should be provided in the local scope for each allocator.
66
Exception Gotchas
Exception handler with "when others => null "
Use of a "when others" whose statement body is "null" may be
inappropriate in that they catch all exceptions but provide no
further processing of conditions that led to the exception.
Exceptions should be used to trap expected problems and revert
to some known safe state. These are normally classified as
serious errors since should an exception be raised, processing
continues which is likely degraded.
67
Exception Gotchas
Use of When Others in an Exception Handler
Such handlers are a catchall and may be inappropriate in some
cases. It prevents the opportunity to return the system to a
known safe state based on the exception named. Typical action
is to log the exception and to allow the system to perform in a
degraded state. It is better practice to handle all potential
exceptions explicitly. It should be noted that it is quite valuable to
have such an exception handler within a looped block for tasks
and main programs, as the absence there can result in a system
crash. 659 Violations in recent analysis
68
Exception Gotchas
Raised Exceptions Non Propagating
Explicit raising of exceptions that are caught in the local scope is
similar in nature to a Goto statement. Use of exceptions in this
manner represents another form of an unstructured program
jump. It makes programs harder to understand, test, and modify.
If the problem can be resolved at the local level, perhaps the use
of exceptions is the wrong abstraction.
69
Exception Gotchas
Raise Predefined Exceptions
Raising predefined exceptions adds confusion as to the source of
the exception. The declaration of application exceptions keeps
system run-time errors and application errors separate. This is
considered to be a poor programming practice.
70
Exception Gotchas
Non-Visible Exception Declarations
This is the declaration of an exception in a non-visible part of the
program. Non-visible declarations can be very dangerous as they
can be only handled within the scope of the declaration (except
with a when others). Unintended propagation outside this scope
may impact remote sections of the code and be a difficult error to
find.
11 Violations in recent analysis
package QUEUE is
procedure DEQUEUE (Object : out Object_type);
procedure ENQUEUE (Object : in Object_type);
function Is_Empty return Boolean;
function Is_Full return Boolean;
end QUEUE; Declarations of
exceptions in body
and in body: is not useful for
desired abstractions
OVERFLOW, UNDERFLOW : exception;
71
Exception Gotchas
Exceptions that Propagate Out of Visible Scope of Subprogram
72
Conclusion
Code Analysis is an important part of providing evidence that
software satisfies requirements for high-integrity systems
Can provide Evidence to support Arguments to support
Claims for High Integrity Assurance
Code Analysis supports:
73