Software Quality Operational Readiness Review (ORR) Checklist

Date(s) of Assessment: ____________ Project: ______________________________________

Assessor(s): _____________________ Review Examined: _____________________________
_______________________________________________________________________________

Y, N, F, O Comments
NA
REVIEW PREPARATION
1 Have standards been identified to
1 clearly define the review process?
2 Were guidelines used to prepare for the
2 review?
Has the project submitted any request
3
for deviations or waivers to the defined
3
process?
4 Have entrance and exit criteria been
4 established for the review?
5 Was an agenda prepared and distributed
5 in advance of the review?
6 Was the review package provided with
6 ample time to review?
7 Were the appropriate stakeholders in
7 attendance?
REVIEW CONTENT
8 Were the goals of the review and any
8 review prerequisites provided?
9 Was the review process addressed,
9 including the method for capturing
Requests for Action (RFAs), risks, or
issues?
10 Was status given on action items from
1 the Acceptance Review?

11 Was status included on action items

from all previous software reviews?
12 Are the pertinent stakeholders involved
1 in approval of the review material?

Revision: 2.0 Page 1 of 5

Y=Yes, N=No, NA=Not Applicable, F=Finding, O=Observation

For more information, please visit the NASA GSFC Software Assurance Website, at http://sw-assurance.gsfc.nasa.gov.
 Software Quality Operational Readiness Review (ORR) Checklist
Y, N, F, O Comments
NA
13 Was an overview of the software
1 project/system provided (e.g., mission
goals, key functionality, operational
characteristics)?
14 Was a review/status of top-level
1 requirements and derived requirements
provided?
15 Was the relationship of requirements to
support provided (i.e., typical, critical,
special or contingency)?
16 Were organizational interfaces and
responsibilities defined?
17 Were operational support scenarios
identified?
18 Were the operational facilities (e.g.,
hardware environment, network) and
their availability identified?
19 Was status provided for all software
components?
20 Were test summary reports and overall
1 performance results presented?

21 Have all requirements been verified?

22 Were any workarounds and non-

1 functioning software components
identified and documented?
23 Was the operational problem escalation
2 process presented?

24 Was the operational emergency

2 notification process identified including
telephone contact numbers?
25 Is there evidence that all software
2 operations personnel are appropriately
trained?
26 Were security issues presented?

27 Were safety issues presented?

2

Revision: 2.0 Page 2 of 5

Y=Yes, N=No, NA=Not Applicable, F=Finding, O=Observation

For more information, please visit the NASA GSFC Software Assurance Website, at http://sw-assurance.gsfc.nasa.gov.
 Software Quality Operational Readiness Review (ORR) Checklist
Y, N, F, O Comments
NA
28 Are there any Configuration
2 Management (CM) issues identified
concerning the software operations?
29 Is there evidence that the software is
2 under CM control and change
procedures are being implemented as
specified in the CM Plan?
30 Is there evidence that all software
procedures and associated
documentation is under CM control and
change procedures are being
implemented as specified in the CM
Plan?
31 Is there evidence that software quality
2 procedures are being implemented as
specified in the SQ Plan?
32 Has SQ completed all audits and
2 reviews applicable to the software and
the affected system?
33 Are technical risks, mitigation plans,
2 and issues documented with plans for
tracking and closure?
SOFTWARE DOCUMENTATION STATUS
34 Does the review package address the
2 status of software documentation as
specified in the project plan?
35 Are operational documents and
3 contingency plans addressed in the
review package?
36 Was status provided for all system
documentation (e.g., OPS and
Maintenance Plans)?
37 Was status provided for all interface
documents with external organizations?
POST REVIEW ACTIVITIES
38 At the conclusion of the review is an
3 understanding reached on the validity
and degree of completeness of the
Operational Readiness Review?
39 Did all designated parties concur in the
3 acceptability of the Operational
Readiness Review?
Revision: 2.0 Page 3 of 5

Y=Yes, N=No, NA=Not Applicable, F=Finding, O=Observation

For more information, please visit the NASA GSFC Software Assurance Website, at http://sw-assurance.gsfc.nasa.gov.
 Software Quality Operational Readiness Review (ORR) Checklist
Y, N, F, O Comments
NA
40 Are there any risks, issues, or request
3 for actions (RFAs) that require follow-
up?
41 Is there a process in place for reviewing
3 and tracking the closure of risks, issues,
or RFAs?
42 Have all artifacts been placed under
3 formal configuration control (e.g.,
review packages)?
43 Were Lessons Learned addressed and
3 captured?

REFERENCE ITEMS/DOCUMENTS
IEEE Standard for Software Reviews, IEEE STD 1028-1997
Managers’ Handbook for SW Development, SEL-84-101

Review Definition – The ORR examines the actual system characteristics and the procedures used
in its operation and ensures that all flight and ground hardware, software, personnel, procedures,
and user documentation accurately reflect the deployed state of the system.

Revision: 2.0 Page 4 of 5

Y=Yes, N=No, NA=Not Applicable, F=Finding, O=Observation

For more information, please visit the NASA GSFC Software Assurance Website, at http://sw-assurance.gsfc.nasa.gov.
 Software Quality Operational Readiness Review (ORR) Checklist

Date(s) of Assessment: ______________ Project: _______________________________________

Assessor(s): ______________________ Review Examined: ______________________________
__________________________________________________________________________________

COMMENTS PAGE _____ of _____

Comments from assessment

Revision: 2.0 Page 5 of 5

Y=Yes, N=No, NA=Not Applicable, F=Finding, O=Observation

For more information, please visit the NASA GSFC Software Assurance Website, at http://sw-assurance.gsfc.nasa.gov.