20028

Joint Software Systems Safety Engineering

Handbook Implementation Guide

Robert E. Smith, CSP

Booz Allen Hamilton
20th Annual NDIA Systems Engineering Conference
Springfield, VA

25 October 2017

1
 BLUF

• System Safety, to include Software Safety, is required for

acquisition programs IAW DoDI 5000.02 and MIL-STD-882E
• Detailed guidance for software safety is provided in the Joint
Software Systems Safety Engineering Handbook (JSSSEH) Version
1.0 published 27 August 2010 as referenced in MIL-STD-882E
– Comprehensive handbook, although lengthy at 344 pgs
– Acquisition Programs unfamiliar with software safety find it difficult to extract software
safety techniques and processes in order to satisfy MIL-STD-882E Software Level of
Rigor (LOR) requirements
– Programs typically re-state the LOR table from MIL-STD-882E, Table V in their
Safety Plans and do not identify and specify the artifacts and Objective Quality
Evidence (OQE) to be produced for all LOR tasks
– Could result in not performing a comprehensive software safety program and
therefore not fully characterizing software’s contribution to system risk
• Joint Boards recognized this concern and developed a JSSSEH
Implementation Guide on 1 April 2016 to further assist programs,
and was endorsed by the Joint Services Weapon Safety Review
(JSWSR) Boards on 29 June 2016
• Revised Implementation Guide (Rev A) issued 17 October 2017
2
 Software Safety Requirements

• Software Safety is required for acquisition programs

– DoDI 5000.02, Enclosure 3, Para 11 – SOFTWARE “…The SEP should address
the following: software unique risks; inclusion of software in technical reviews;
identification, tracking, and reporting of metrics for software technical
performance, process, progress, and quality; software safety and security
considerations; and software development resources.”
– DoDI 5000.02, Enclosure 3, Para 16 - ENVIRONMENT, SAFETY, AND
OCCUPATIONAL HEALTH (ESOH) “The Program Manager will integrate ESOH
risk management into the overall systems engineering process for all engineering
activities throughout the system’s life cycle. As part of risk reduction, the Program
Manager will eliminate ESOH hazards where possible, and manage ESOH risks
where hazards cannot be eliminated. The Program Manager will use the
methodology in MIL-STD-882E...”
– MIL-STD-882E, Section 4.4 Software contribution to system risk. “The
assessment of risk for software, and consequently software-controlled or software-
intensive systems, cannot rely solely on the risk severity and probability…..
Therefore, another approach shall be used for the assessment of software’s
contributions to system risk that considers the potential risk severity and the
degree of control that software exercises over the hardware.”

3
Common Approaches to Software Safety

• MIL-STD-882E references the JSSSEH and Section 4.4.2 includes

a note to “Consult the Joint Software Systems Safety
Engineering Handbook and AOP 52 for additional guidance on
how to conduct required software analyses.”
• The JSSSEH is a lengthy document making it difficult for
programs not familiar with software safety activities to extract
detailed LOR tasks and tailor for particular program needs
• Programs often default to only referencing or reusing the LOR
table from MIL-STD-882E (i.e., Table V) as their software safety
approach in their System Safety Management Plans (SSMPs)
and/or System Safety Program Plans (SSPPs)
• May result in not performing the specific LOR tasks that
comprise a comprehensive software safety program, resulting in
failure to assess software’s contribution to system risk(s)

4
 MIL-STD-882E, Table V,
Software Safety Criticality Matrix

High Level,
overarching LOR
tasks

5
 MIL-STD-882E, Table V,
Level of Rigor Tasks

• Note that the LOR tasks table contains no details on the specific
tasks, artifacts and Objective Quality Evidence (OQE) to be
produced for LOR (e.g., requirements analysis, architecture
analysis, design analysis, safety-specific testing, and code
analysis)
• The JSSSEH includes these details, but not in a specific location
• Challenge is getting Acquirers (Customer) and Developers
(software developers) to specify how they will turn the objectives
of MIL-STD-882E and the JSSSEH “guidance” into actual
Software System Safety Engineering (SSSE) Requirements 6
 Implementation Guide Overview

• Developed by the Joint Services – Software Safety Authorities

(JS-SSA) Sub-Working Group in support of the JSWSR Boards on
1 April 2016 - endorsed by the JSWSR Boards on 29 June 2016
• Titled “Software System Safety Implementation Process and
Tasks Supporting MIL-STD-882E With Joint Software System
Safety Engineering Handbook References”
– Short name – “Implementation Guide”
• Provides implementation guidance for Software System Safety
program requirements specified in MIL-STD-882E and guidance
detailed in the JSSSEH
• Updated in 2017 to address identified errors, Service comments
and create more direct alignment with the Tasks in MIL-STD-882E
• Released as “Revision A” on 17 October 2017

7
 Implementation Guide Outline and
Methodology
• The implementable process task requirements are presented as
a decomposition of parent and children activities, similar to a
Work Breakdown Structure (WBS)
• Parent tasks are graphically represented depicting inputs to the
tasks and the products that the task would typically produce
• Tasks identified as MIL-STD-882 requirements are coded in the
graphics using an extreme bold border of the task box
• Task decomposition is to the level necessary for a basic
understanding of the process, the tasks that implement the
process, and the products the tasks would likely produce
• The requirements derived that apply to each task are specified
and cross referenced to both the applicable MIL-STD-882E
requirements and JSSSEH sections and paragraphs that provide
guidance on meeting the requirements

8
 Process Tasks (2016 Guide)

• 14 Process Tasks identified in the Implementation Guide

– Process Task 1.0: Prepare the System Safety Management Plan (SSMP)
– Process Task 2.0: Prepare System Safety Program Plan (SSPP)
– Process Task 3.0: Preliminary Hazard Analysis
– Process Task 4.0: Functional Hazard Analysis (FHA)
– Process Task 5.0: LOR Allocations to Safety-Significant Functions
– Process Task 6.0: Preliminary Safety Requirements Analysis (SRA)
– Process Task 7.0: Perform In-Depth Hazard Analysis
– Process Task 8.0: Perform Detailed Safety Requirements Analysis
– Process Task 9.0: Perform Safety Requirements Traceability
– Process Task 10.0: Perform Code-Level Safety Analysis
– Process Task 11.0: Perform Software Test Planning
– Process Task 12.0: Monitor Safety-Significant Software Testing
– Process Task 13.0: Perform Residual Safety Risk Assessment
– Process Task 14.0: Participate in Life-Cycle Management and Support
• Each Process Task has Process Subtasks to amplify details
and/or additional steps associated with each Task

9
 Process Tasks (2017 Guide)

• 13* Process Tasks identified in the Implementation Guide

– Process Task 1.0: Prepare the System Safety Management Plan (SSMP)
– Process Task 2.0: Prepare System Safety Program Plan (SSPP)
– Process Task 3.0: Preliminary Hazard Analysis
– Process Task 4.0: Functional Hazard Analysis (FHA)*
– Process Task 5.0: Initiate Safety Requirements Hazard Analysis (SRHA)*
– Process Task 6.0: Perform System and Subsystem Hazard Analyses*
– Process Task 7.0: Finalize SRHA*
– Process Task 8.0: Perform Final Safety Requirements Traceability*
– Process Task 9.0: Perform Code-Level Safety Analysis
– Process Task 10.0: Perform Software Test Planning
– Process Task 11.0: Monitor Safety-Significant Software Testing
– Process Task 12.0: Perform Safety Risk Assessment*
– Process Task 13.0: Participate in Life-Cycle Management and Support
• Each Process Task has Process Subtasks to amplify details
and/or additional steps associated with each Task

* Changes in 2017: Titles of tasks revised and previous Task 5.0 combined into Task
4.0, and SRA is now System Requirements Hazard Analysis (SRHA)
10
Process Tasks 4.0 – FHA
[Partial Example]

References* to specific sections

in JSSSEH and MIL-STD-882E

Process flow diagram* provided

Process Task / Subtask

referenced for each step
* NOTE - References still the same in 2017 Guide.
Flow diagrams altered as appropriate. 11
Process Tasks 4.0 – FHA
[Partial Example]

Process Task / Subtask

described in detail in
subsequent paragraphs

12
Appendix A – LOR Task Table [Partial]

Legend:
• PR: Prerequisite Requirement – Required
regardless of LOR or required in order to
assess and determine LOR
• R: Required for assigned LOR
• AD: As directed by Customer/Contract
• IV&V: Independent Verification and
Validation
• N/A: Not Applicable for this program or LOR

13
 LOR 1 Example [Partial]

• Table indicates required (“R”) LOR

activities for LOR 1, 2, 3, and 4
• E.g., Design Practice (DP)-11:
Analyze all safety functional
threads…
– Required only for LOR 1
– One of many LOR 1 activities
required (“R”) for LOR 1
– Appendix A specifies the LOR
activity, primary and support
activities, applicable LOR, and
artifact(s) to be produced

14
 Change Management

• JS-SSA meets twice annually

• Approved path for changes:
– Any user can submit comments
– Comments collected from 4th QTR FY until end of 2nd QTR FY
(comments, corrections, additions, deletions, etc.)
– Submit comments to JS-SSA Chair
– Proposed changes adjudicated between the Service JS-SSA
Implementation Guide (IG) IPT
– Changes approved by the JS-SSA Sub-group will then be
integrated into the Implementation Guide and a new revision
released in time for the Fall meeting (or end of year)
• 100+ proposed changes submitted during the FY2017 review
period
• Proposed changes were adjudicated via email and in a face-to-
face meeting April 2017
• Draft JS-SSA IG update distributed to Working Group and
approved in August 2017
• Release of 2017 Guide Update (Rev A) on 17 October 2017 15
 2017 Summary of Changes

• Numerous changes between 2016 Guide and 2017 Guide

• Two “Critical” changes to the Implementation Guide
– Less emphasis and more controls on tailoring of LOR table by
contractors (Section 2.0)
 Changed from: “The LOR table should be tailored for any given
program as agreed to by the Acquirer and Developer.”
 To: “The LOR table should be assessed for tailored implementation
for any given program, and tailoring is permitted as long as the
tailored LOR tasks are approved by both the Acquirer and
Developer.”
– Allows risks to be carried over, if appropriate, from one
contractual activity to another following a reassessment (Section
3.2.4.2)
 Changed from: “Risk accepted in one contractual activity should
never be carried over as the baseline for the next contractual activity.”
 To: “Risk acceptance performed in one contractual activity should be
reassessed for the next contractual activity.”

16
 2017 Summary of Changes (cont.)

• Four “Significant” changes to the Implementation Guide

– SSMP tasks added to the LOR table in Appendix A as “Acquirer”
activities
– Removed requirement that Contractors must comply with future versions
of DODI 5000.02 and MIL-STD-882, just the versions under contract
– Clarified purpose of document as defining the processes and tasks
needed to implement a MIL-STD-882E compliant SSSE program
– Made the current Process Task 5.0 “LOR Allocations to Safety-
Significant Functions” a subtask of draft Process Task 4.0 “FHA”
• Majority of remaining changes are relatively minor and designed
to resolve known inconsistencies and improve alignment with
MIL-STD-882E
– Primarily changes to the process flow figures and associated paragraphs
detailing the subtasks for the analyses/reports (PHA, SRA, etc.) to better
define tasks and processes
– Many editorial and administrative corrections
• Changed “Hazard Risk Index” to “Risk Assessment Code”
• Changes to the LOR table in Appendix A
17
2017 Summary of Changes – Appendix A

• Seven new Baseline LOR SSE-related activities detailing

Acquirer (i.e., “ACQ-#”) responsibilities
• Some activity descriptions updated and enhanced, but overall,
no other new activities added

Level-Of-Rigor Activity / Task Type 2016 IG 2017 IG Change

Acquirer (ACQ-#.#) 0 7 +7
System Safety Engineering (SSE-#.#) 22 22 -
Requirements Phase (RP-#) 11 11 -
Design Phase (DP-#) 13 13 -
Implementation (Coding) Phase (IP-#) 15 15 -
Test Phase (DP-#) 23 23 -
Life Cycle Support Phase (LC-#) 12 12 -
TOTAL ACTIVITIES / TASKS 96 103 +7

18
2017 Summary of Changes – Appendix A
(cont.)
• Several activities now required to be performed at lower LOR to
align with MIL-STD-882E Table V LOR requirements

Level-Of-Rigor 2016 IG 2017 IG Change

Baseline 42 49 +7
1 54 54 -
2 47 49 +2
3 35 38 +3
4 20 27 +7
TOTAL
96 103 +7
(LOR 1 + Baseline)

19
 Conclusion

• Software Safety is required for acquisition programs IAW DoDI

5000.02 and MIL-STD-882E
• Additional guidance for software safety is provided in the
JSSSEH Version 1.0 published 27 August 2010 as referenced in
MIL-STD-882E
• Joint Boards developed a JSSSEH Implementation Guide on 1
April 2016 to further assist program perform software safety, and
was endorsed by the JSWSR Boards on 29 June 2016
• 2017 Implementation Guide Update (Rev A) release on 17
October 2017
• Implementation Guide will be updated annually, as required

Implementation Guide assists in performing a comprehensive

software safety program to fully characterize software’s
contribution to system risk
20
 Resources (location of documents)

• DAU Acquisition Community Connection Site, ESOH Community

– https://acc.dau.mil/ESOH
---or---
– https://www.dau.mil/cop/esoh/Pages/Default.aspx
– look under the “Resources” section

• DoD Joint Software System Safety Engineering Handbook, 2010

– https://www.dau.mil/cop/esoh/DAU Sponsored
Documents/SOFTWARE SYSTEM SAFETY HDBK 2010.pdf

• Software System Safety Implementation Process and Tasks

Supporting (a.k.a. “Implementation Guide”)
– https://www.dau.mil/cop/esoh/DAU%20Sponsored%20Document
s/JSWRBs%20Endorsement%20JS%20SSA%20Software%20Sy
stem%20Safety%20Implementation%20Guide%2029JUN2016.pd
f

21
 Questions?

Robert E. Smith
Lead Associate

Booz Allen Hamilton

1550 Crystal Dr, Suite 1100
Arlington, VA 22202
Tel (703) 412-7661
[email protected]

22