EmbeddedSystems Module 14 Software
EmbeddedSystems Module 14 Software
Module 14
Software Development and V&V
EE4673/5673
by
Dr. Maarten Uijt de Haag
SW V&V.1
Embedded Systems
Input
Sensor Actuator Actuator Actuator
Device
E E
E E E
E E
Wire
Actuator Actuator
bundle
Function, Failure,
System Design
and safety Information
Implementation
Hardware Life-Cycle Software Life-Cycle
Process Functions & Functions & Process
Implementation
Requirements Requirements
Hardware Software
Development Development
SW V&V.3
Software Standards
• There are many software-related standards
• Some standards that will be referred to in
this section of the course:
– IEC 61508-3,
– IEEE/EIA 12207,
– ISO/IEC 12207,
– ED-12B / DO-178B,
– IEEE 1012,
– IEC-62138,
– IEC-61226,
– IEC-60880.
SW V&V.4
Software Standards
SW V&V.5
Development vs. Assurance
SW V&V.6
IEC 61508 Software Life-cycle
SW V&V.7
Software Development Processes
SW V&V.8
Software V&V Processes
SW V&V.9
Software Safety Requirements
System Requirements
High-Level Requirements
Low-Level Requirements +
Software Architecture
Source Code
SW V&V.10
Software Architecture
The software architecture defines
the major components and
subsystems of the software, how
they are interconnected, and how
the required attributes, particularly
safety integrity, will be achieved.
Examples of major software
components include operating
systems, databases, plant
input/output subsystems,
communication subsystems,
application program(s),
programming and diagnostic tools
etc.
Software architecture (in DO178B/C):
The structure of the software selected to implement the software requirements
(e.g. control and data flow, inter SW component communication) SW V&V.11
Software System Design
System Requirements
High-Level Requirements
Low-Level Requirements +
Software Architecture
Source Code
SW V&V.12
Low-Level Requirements
• Software requirements from which the Source
Code can be directly implemented without
further information.
– Pseudocode? Matlab code?
Low-level requirements:
Module thrustReverseEnable
IF( wow AND (air_speed < rollout_upper_limit) ) THEN
reverseThrust = REV_THRUST_ENABLE
ELSE
reverseThrust = REV_THRUST_DISABLE
ENDIF
EndThrustReverseEnable
Coding option:
High-level H1 H2 … … … … … … Hm
Derived Low-level requirements
Low-level L1 L2 … … … … … … … … … … … … Lj
Derived Requirements
Additional requirements resulting from the software development processes, which
may not be directly traceable to higher level requirements
ARP4754
“At each phase of the development activity, decisions are made as to how particular requirements or
groups of requirements are to be met. The consequences of these design choices become
requirements for the next phase of the development. Since these requirements result from the design
process itself, they may not be uniquely related to a higher-level requirement and are referred to as
derived requirements.”
SW V&V.14
Standards
SW Requirements Standard
Define structured methods, rules and tools that will be used
to develop the high-level requirements.
System High-level
Requirements Requirements
SW Requirements
Standards
SW V&V.15
Standards
SW Design Standard
Define structured methods, rules and tools that will be used
to develop the low-level requirements and software architecture.
High-level Low-level
Requirements Requirements
SW Design
Standards
SW V&V.16
Standards
SW Code Standard
Define the programming languages, methods, rules and tools that
will be used to code the software.
SW V&V.19
SW Code Standard – MISRA C
• Rule 1.1 (required):
– All code shall conform to ISO 9899:1990 “Programming languages –
C, amended and corrected by ISO/IEC 9899/COR1:1995, ISO/IEC
9899/AMD1:1995, ISO/IEC 9899/COR2:1996.
• Rule 13.3 (required):
– Floating-point expressions shall not be tested for equality or
inequality.
Why?
If(f32aux == 0) {
/* segment is parallel to y axis */
projected->x = one.x;
projected->y = gps.y;
}
float x; float x;
if (x == 1000) { if (FloatEqual(x,1000,0.001)) {
printf("equals\n"); printf("equals\n");
} else { } else {
printf("not equals (x = printf("not equals (x = %f)\n",x);
%f)\n",x); }
} return 0;
return 0; }
}
> not > equals
Output: Output:
not equals (x = 1000.000061) equals
SW V&V.21
SW Code Standard – MISRA C
• Rule 20.3 (Required):
– The validity of values passed to library functions shall be checked.
• man sqrt: sqrt(x) returns a NaN and generates a domain error for x < 0.
SW V&V.22
Why Planning, Verification, Validation?
Relative cost of error solving:
Time detected
Time
introduced Requiremen Architecture/
Coding Test Port-release
ts Design
Requirements 1 3 5-10 10 10-100
Architecture/
Design - 1 10 15 25-100
Coding - - 1 10 10-25
SW V&V.23
Verification versus Validation (V&V)
Validation:
The process of determining that the requirements are the
correct (they satisfy the customer’s needs, wants, and desires) requirements
and that they are complete.
Verification:
The evaluation of the results of a process to ensure correctness and consistency
with respect to the inputs and standards provided to that process.
Verification ≠ Testing
but
SW V&V.25
Software Verification Activities
Reviews Analysis Testing
Requirements reviews
Timing and stack analysis
Design reviews
Data flow
Code reviews
Control flow
Test procedure reviews
Note an analysis tool can be used
Checklists can be used to make
but more on that topic later …
the reviews effective and consistent:
• Objective evidence of the review activity
• Focus on the areas most prone to error
• Provide mechanism for using “lesson-learned”
• Provide a simple means for ensuring corrective action
is taken
SW V&V.26
SW Verification – Review and Analysis
Compliance
Traceability
Consistency
Accuracy
Verifiability
Conformance
Compatibility
SW V&V.27
SW Verification – Review and Analysis
Verifiable
Can the requirement or code be verified without change?
Example:
1.2 A red indicator light shall come on if the electrical power falls below
3000W.
SW V&V.28
SW Verification – Review and Analysis
Accuracy
The requirement or code is accurate, unambiguous, and sufficiently
detailed.
1.2 The hybrid engine controller shall turn off the engine when car stands still.
Ambiguous requirement:
Need to define how long you stand still otherwise it should turn off after
after an arbitrary short time
SW V&V.29
SW Verification – Review and Analysis
Consistency
No internal conflicts within a collection of items*.
Compatibility
No conflicts between two or more specified items* or collections.
Example:
SW V&V.32
SW Verification – Review and Analysis
Traceability
All components at one stage of the lifecycle have been addressed
at the next (higher or lower) level of the life cycle.
Question yourself: Did I loose any functionality going to a lower level of requirements?
Traceability:
-Accounting function (bookkeeping)
-Downward (higher to lower level): No childless parents
-Upward (lower to higher level): No children without parents (orphans)
unless assigned to derived requirements
Compliance
Establish that there is technical agreement between products
from different stages of a life-cycle.
Compliance:
-Technical agreement between parent and child
Note that traceability is often seen as overhead imposed by the certification authorities
and as a result neglected especially during changes made to the product: DO NOT MAKE THIS MISTAKE
SW V&V.33
Traceability – Compliance Example
High-level requirement:
3.1 Test statistic = Absolute Disparity (previous time epoch) – Absolute Disparity (51 time epochs ago)
+ Absolute Disparity (current time epoch)
Traceability:
High-level requirement 3.1 and low-level requirement 5.1.3 can be traced to one another.
There are no orphans (upward) nor childless parents (downward).
Compliance:
The problem is that there is no technical agreement between 3.1 and 5.1.3.
SW V&V.34
SW Verification – Testing
Two purposes:
-Demonstrate that the software product performs its intended function completely
and correctly
-Demonstrate, with a high degree of confidence, that the SW is free of errors that
could lead to unacceptable failure conditions
SW Testing
Verify the implementation Verify the interrelationships between SW Verify the correct operation
of the low-level requirements requirements and components & of the SW in the target embedded
Verify implementation of the SW computer environment
requirements and architecture
SW V&V.35
Software Engineering
Inputs Outputs
SW V&V.37
Requirements-based Testing
Module thrustReverseEnable
IF( wow AND (air_speed < rollout_upper_limit) ) THEN
reverseThrust = REV_THRUST_ENABLE
ELSE
reverseThrust = REV_THRUST_DISABLE
ENDIF
EndThrustReverseEnable
wow: weight on wheels is “1” when the aircraft is on the ground and all the
aircraft’s weight is on the wheels.
airspeed: speed of the aircraft wit respect to the air.
rollout_upper_limit: speed above which a reverse thrust would be disastrous.
SW V&V.38
Requirements-based Testing
Module thrustReverseEnable
IF( wow AND (air_speed < rollout_upper_limit) ) THEN
reverseThrust = REV_THRUST_ENABLE
ELSE
reverseThrust = REV_THRUST_DISABLE
ENDIF
EndThrustReverseEnable
SW V&V.39
Example Tests
Test 1:
Connect the weight-on-wheels input to a low-level voltage (= 0V) and the airspeed to 220 knots.
Expected result: the reverse thrust will be disabled.
wow = 0 (false)
(airspeed < rollout_upper_limit) = (220 < 95) = false = 0
Assume: rollout_upper_limit = 95
0V => logic “0”
1V => logic “1” SW V&V.40
Example Tests
Test 1:
Connect the weight-on-wheels input to a low-level voltage (= 0V) and the airspeed to 220 knots.
Expected result: the reverse thrust will be disabled.
Test 2:
Connect the weight-on-wheels input to a high-level voltage (= 5V) and choose the airspeed to be
equal to 10 knots.
Expected result: the reverse thrust will be enabled
Test 3:
Connect the weight-on-wheels input to a high-level voltage (= 5V) and choose the airspeed to be
equal to 220 knots.
Expected result: the reverse thrust will be disabled
Assume: rollout_upper_limit = 95
0V => logic “0”
1V => logic “1”
SW V&V.41
Perform Tests on Code
Coding option in “C”:
return( reverseThrust);
}
SW V&V.42
Requirements-based Test Case Selection
• Normal range test cases • Robustness (abnormal range)
– Exercise real & integer value range test cases
variables – Exercise real & integer value
– Equivalence classes variables
– Boundary values
– Invalid Equivalence classes
– Time dependencies – Invalid Boundary values
– Include the time domain in the
equivalence classes – Abnormal system initialization
– Filters, integrators, etc. – Failure modes of external data
– State transitions – Loop robustness
– Examine every transitions – If the loop counter is a
– Logic equations computed value try out-of-
range loop values.
– Frame times State transitions
– Examine every transitions
– Arithmetic over- and
underflow
– Disallowed state transitions
SW V&V.43
SW Testing - Equivalence classes
Equivalence class:
is a set of input, state, or output values for which an object is supposed to behave
similarly. Or DO178B: The partition of the input domain of a program such that a test of
a representative value of the class is equivalent to a test of other values of the class.
0 1
Normal range tests would include:
0 1, 1 2, 2 , etc.
SW V&V.45
Errors Revealed in SW Testing
Low-level Testing
Failure of an algorithm to satisfy a SW requirement.
Incorrect loop operations
Incorrect logic decisions
Failure to process correctly legitimate combinations of input conditions
Incorrect responses to missing or corrupted data
Incorrect handling of exception (arithmetic faults, array limit violations)
Incorrect computation sequence
Inadequate algorithm precision, accuracy, and performance
SW V&V.46
Errors Revealed in SW Testing
SW Integration Testing
Incorrect initialization of variables and constants
Parameter passing errors
Data corruption (global data especially)
Inadequate end-to-end numerical resolution
Incorrect sequencing of events and operations
SW V&V.47
Errors Revealed in SW Testing
HW/SW Integration Testing
Incorrect interrupt handling
Failure to satisfy execution time requirements
Incorrect SW response to HW transients and failures
Resource contention problems (like data bus)
Inability of built-in test to detect failures
Errors in HW/SW interfaces
Incorrect behavior of feedback loops
Incorrect control of memory management HW or other HW devices under SW
control
Stack overflow
Field-loadable SW problems
Violations of SW partitioning
SW V&V.48
Verification of the Verification
Coverage Analysis
Coverage
May uncover code that was not exercised during the tests.
Inadequacies in SW
requirements Short-coming/problems Dead code Deactivated code
with the requirements-based
test cases or procedures
SW V&V.50
Software Testing Summary
Software
Requirements-based
Test Generation
Hardware/SW
Low-level SW Integration
Integration
Tests Tests
Tests
Software Requirements
Coverage Analysis
Software Structure
Coverage Analysis
Direct Path Additional
Conditional Path End of Testing Verification
SW V&V.51
Designing Tests
A
Z For an n –input AND gate:
(1) All inputs are set true (1 test),
B (2) Each and every input is set exclusively false (n tests)
Example:
Test Case # A B Z
3-input AND gate, FTT, TFT, TTF, TTT
1 F F F
2 F T F
3 T F F
4 T T T
Module thrustReverseEnable
IF( wow AND (air_speed < rollout_upper_limit) ) THEN
reverseThrust = REV_THRUST_ENABLE
ELSE
reverseThrust = REV_THRUST_DISABLE
ENDIF
EndThrustReverseEnable
SW V&V.53
Designing Tests - Example
Test 1:
Connect the weight-on-wheels input to a low-level voltage (= 0V) and the airspeed to 10 knots.
Expected result: the reverse thrust will be disabled.
FT
Test 2:
Connect the weight-on-wheels input to a high-level voltage (= 5V) and choose the airspeed to be
equal to 10 knots.
Expected result: the reverse thrust will be enabled TT
Test 3:
Connect the weight-on-wheels input to a high-level voltage (= 5V) and choose the airspeed to be
equal to 220 knots.
Expected result: the reverse thrust will be disabled TF
SW V&V.54
Designing Tests
Z
A
For an n –input OR gate:
B (1) All inputs are set false (1 test),
(2) Each and every input is set exclusively true (n tests)
2 F T T
3 T F T
4 T T T
1 F F F B
2 F T T
3 T F T
Multiple test sets possible. See the truth table:
4 T T F
Test sets: (1,2,3), (1,2,4), (1,2,4), (2,3,4)
A Z
A Z
F T
For an NOT gate:
T F (1) Input set false,
(2) Input set true.
SW V&V.56
Designing Tests
x A
• Criteria:
x x > comparison point
– Less than
– Greater than
– Less than or equal to Or
– Greater than or equal to x
– Equal to A
– No equal to x
x>y
y y
Two test cases will confirm MC/DC, one test case with a TRUE outcome and one with a FALSE outcome.
SW V&V.57
Designing Tests
Engine RPMs(x)
A
x x > 2500
3 2,500 F
5 2,501 T
SW V&V.58
Designing Tests - Example
Low-level requirements:
Module thrustReverseEnable
IF( wow AND (air_speed < rollout_upper_limit) ) THEN
reverseThrust = REV_THRUST_ENABLE
ELSE
reverseThrust = REV_THRUST_DISABLE
ENDIF
EndThrustReverseEnable
SW V&V.59
Designing Tests
Example: if C then z := x else z:=y;
Schematic representations:
C
x
z
z := x z := y
y
SW V&V.60