PEMP ESE2516

FT12

Session 9
Engineering Design Review and Checklist

Session Speaker:
Dr Govind R
Dr. R. Kadambi

© M. S. Ramaiah School of Advanced Studies, Bengaluru 1

 PEMP ESE2516
FT12

Session Objectives

To learn and understand :

• Preliminary design review process and primary

information exchange partners
• Responsibilities of government, contractors and
input/output criterion
• Critical Design review process and underlying
responsibilities of the staffs
• Creating
C i andd managing i theh engineering
i i review
i checklists
h kli

© M. S. Ramaiah School of Advanced Studies, Bengaluru

2
 PEMP ESE2516
FT12

Session Topics

• Preliminary Design Review

• G
Government R
Responsibilities
ibili i
• Contractor Responsibilities
• Critical Design Review
• Staff Responsibilities
• Engineering Review Checklists (ERCs)
• Responsibility of Engineering Technical Support Staff
• Creating and Maintaining ERCs

© M. S. Ramaiah School of Advanced Studies, Bengaluru

3
 PEMP ESE2516
FT12
Preliminary Design Review

• Preliminary Design Review (PDR) - a formal inspection of the high-

level architectural design of an automated system and its software

• Conducted to achieve confidence that the design satisfies the

functional
u c o andd nonfunctional
o u c o requirements
equ e e s

• PDR is the second of three critical checkpoints in System Life Cycle

F
Frameworkk

• System
y development
p and major
j application
pp enhancement pprojects
j
must conduct a PDR
• PDR is performed during the Design and Engineering Phase when
the System Developer has initially baselined the high-level
high level system
architectural design
© M. S. Ramaiah School of Advanced Studies, Bengaluru
4
 PEMP ESE2516
FT12

Primary Information Exchange Partners

• Primary stakeholders who participate or have an interest in

the
h PDR:
PDR

– System Developer
– System Owner/Manager
– Project
j Owner/Manager g
– Component Lead
– Infrastructure Implementation Agent or Contractor
– Chief Technology Officer (CTO)
– Executive Steering Committee (ESC)

© M. S. Ramaiah School of Advanced Studies, Bengaluru

5
 PEMP ESE2516
FT12

Government Responsibilities
• Project Owner/Manager works with their designated
Component Lead
• Ensures completion of the entrance requirements for the
Preliminary Design Review (PDR)

• Fi
Firstt step-
t C
Completion
l ti off the
th PDR Questionnaire
Q ti i
• Determines readiness to proceed to the PDR and availability
of the input documents

• Component Lead collaborates with OIS/ISMG in determining

readiness for the PDR
• Assists the Project Owner/Manager in preparing the agenda,
scheduling, and facilitating/managing the PDR

© M. S. Ramaiah School of Advanced Studies, Bengaluru

6
 PEMP ESE2516
FT12

Government Responsibilities
• If a system is being developed in-house without outside
contractor resources, then the government developers are
responsible
ibl for
f the
th contractor
t t responsibilities
ibiliti

• Representatives
p from the keyy stakeholder ggroups p within the
Office of Information Services (OIS) are responsible for
reviewing the input documents prior to the PDR and being
pprepared
p to ppresent anyy critical issues duringg the PDR

© M. S. Ramaiah School of Advanced Studies, Bengaluru

7
 PEMP ESE2516
FT12

Government Responsibilities

• Within one week following the PDR, the participants and

the
h stakeholders
k h ld are to sendd their
h i comments to the h
Component Lead

• Component Lead consolidates, collates, and sorts the

comments by priority, page, and paragraph.

• Upon completion of the PDR session(s), the Component

Lead is responsible for ensuring completion of the PDR

© M. S. Ramaiah School of Advanced Studies, Bengaluru

8
 PEMP ESE2516
FT12

Government Responsibilities

• Exit Form by all of the key OIS stakeholder groups and for
tracking
ki all
ll critical
i i l issues
i to closure
l

• Project Owner/Manager is responsible for tracking and

resolution of all other actions resulting from the PDR.

• If appropriate, the Component Lead should offer to

schedule and lead a PDR follow-up discussion between the
Project Owner/Manager

• PDR stakeholders to discuss Priority 1 and 2 comments

and corrective actions
© M. S. Ramaiah School of Advanced Studies, Bengaluru
9
 PEMP ESE2516
FT12

Contractor Responsibilities
• Following are the responsibilities that the System Developer
has with regard to the PDR
• Would pprovide a high-level
g ppresentation to stakeholders that
addresses the following:

– Project Overview
– Design Drivers & Alternatives Considered
– System Architectural Design
– Software Architectural Design
– Requirements Traceability
– Data Overview
– Enterprise Architecture Compliance
– Resource Sizing Estimates
– Risks & Mitigation
g Strategies
g
– Issues
© M. S. Ramaiah School of Advanced Studies, Bengaluru
10
 PEMP ESE2516
FT12

Entrance Criteria/Inputs
• Mandatory artifacts that are prerequisites to the PDR:
– Business Risk Assessment
– Requirements
R i t Document
D t
– System Design Document
– Information Security Risk Assessment

• Other artifacts that may also be prerequisites to the PDR, depending

on the specific project being addressed, include:
– Logical Data Model
– Interface Control Document (ICD)
– Database Design Document
– Release Plan
– System Security Plan (SSP)

© M. S. Ramaiah School of Advanced Studies, Bengaluru

11
 PEMP ESE2516
FT12

Exit Criteria/Outputs
• Following are the results/outputs that are to be documented
from the PDR:
– Assumptions
– Constraints
– Priorities
– Issue Resolutions
– Unresolved Issues
– Risk Mitigation Strategies
– Recommendations
– Action Items
– Next Steps

© M. S. Ramaiah School of Advanced Studies, Bengaluru

12
 PEMP ESE2516
FT12
Review Process
• P
Project
j t Owner/Manager/System
O /M /S t Developer
D l h to
has t complete
l t the
th
PDR Questionnaire

• Submission of the completed questionnaire to the designated

Component Lead along with the appropriate entrance artifacts

• Component Lead will coordinate with OIS to schedule the

PDR, if one is warranted, and will notify the OIS Group
Di t accordingly
Directors di l

• PDR will be conducted as one or more iterative review

sessions attended by all of the key OIS stakeholder groups

© M. S. Ramaiah School of Advanced Studies, Bengaluru

13
 PEMP ESE2516
FT12
Review Process
• C
Component Lead L d in
i conjunction
j i with
i h the
h P j
Project
Owner/Manager and System Developer document the
results/outputs
p of the PDR

• Follow-up to ensure that all actions and any unresolved issues

identified during the PDR are satisfactorily addressed

• Component Lead ensures completion of the PDR Exit Form by

all of the key OIS stakeholder groups and submits the
completed form to the Chief Technology Officer (CTO)

© M. S. Ramaiah School of Advanced Studies, Bengaluru

14
 PEMP ESE2516
FT12

Review Process

• If ESC exists for the project, then the Component Lead,

P j Owner/Manager
Project O /M andd CTO will
ill provide
id a summary
of the results from the PDR along with an overall OIS
recommendation to the ESC in determiningg a ggo/no ggo
decision for the project to continue moving forward

• "Critical" issues must be resolved before the project can

proceed with detailed design and development activities.

© M. S. Ramaiah School of Advanced Studies, Bengaluru

15
 PEMP ESE2516
FT12

Critical Design Review

• Critical Design Review (CDR) is a multi-disciplined product
and process assessment to ensure that the system under review
can proceed into system fabrication,
fabrication demonstration,
demonstration and test

• Can meet the stated performance requirements within cost

(program budget),
budget) schedule (program schedule),
schedule) risk,
risk and
other system constraints

• Review assesses the system final design as captured in

product specifications for each configuration item in the
system (product baseline)

• Ensures that each product in the product baseline has been

captured in the detailed design documentation

© M. S. Ramaiah School of Advanced Studies, Bengaluru

16
 PEMP ESE2516
FT12

Critical Design Review

• Product specifications for hardware enable the fabrication of
configuration items

• Product specifications for software (e.g. Software Design

Documents) enable coding of a Computer Software
Configuration Item (CSCI)

• Configuration items may consist of both hardware and

software elements

© M. S. Ramaiah School of Advanced Studies, Bengaluru

17
 PEMP ESE2516
FT12
Critical Design Review
• For complex systems,
systems a CDR may be conducted for each
subsystem or configuration item

• Incremental
I t l reviews
i would
ld lead
l d up to
t an overall
ll system
t CDR

• Emphasis of the overall system CDR should be on

configuration item, functional and physical interface detail
design, as well as overall system detail design requirements

• CDR determines whether the hardware, human and software

final detail designs are complete, and the project team is
prepared to start the Execution and Build Phase

• Chief Technology Office is the Technology Services (TS)

component that has primary responsibility for the Critical
Design
i Review
i
© M. S. Ramaiah School of Advanced Studies, Bengaluru
18
 PEMP ESE2516
FT12

Critical Design Review

• At this review the project team shall also review the results of
peer reviews on requirements and final detail design
d
documentation
t ti

• Ensure that latest estimates of cost ((development,

p , pproduction,,
and support) are consistent with the detail design

• CDR should occur at the point in the Planning and Design

Phase where the “build-to” baseline has been achieved,
allowing production, and coding of software deliverables to
proceed

© M. S. Ramaiah School of Advanced Studies, Bengaluru

19
 PEMP ESE2516
FT12

Critical Design Review

• Critical Design Review is the second of two critical
checkpoints in the System Life Cycle Framework

• All system development and major application enhancement

projects
j t mustt conduct
d t a CDR

• Actual conduct of the CDR will be somewhat dependent on

the specific circumstances of the project

© M. S. Ramaiah School of Advanced Studies, Bengaluru

20
 PEMP ESE2516
FT12

Primary Stakeholders in the CDR

• System Developer
• System Owner
• Project Owner
• DPI Project
P j t Manager
M
• ITS Infrastructure hosting office
• ITS Statewide Architecture office
• DPI Enterprise Architect
• DPI Chief Technologygy Officer ((CTO))

© M. S. Ramaiah School of Advanced Studies, Bengaluru

21
 PEMP ESE2516
FT12 Staff Responsibilities
• DPI Project
j Managerg is responsible
p for:
– Ensuring completion of the entrance requirements for the
CDR
– Collaboration with the CTO in determining readiness for the
CDR
– Preparing the agenda, scheduling, and facilitating / managing
the CDR

• If a system is being developed in-house without outside

contractor resources, then the DPI developers are responsible for
the contractor responsibilities as well

• TS stakeholders are responsible for:

– Reviewing the input documents prior to the CDR and being
prepared to present any critical issues during the CDR
– Ensuring that assumptions, constraints, priorities, issues and
risks based on their individual areas of subject j matter
expertise are identified and addressed during the CDR as
appropriate
© M. S. Ramaiah School of Advanced Studies, Bengaluru
22
 PEMP ESE2516
FT12
Staff Responsibilities
• U
Upon completion
l ti off the
th CDR sessions,
i th project
the j t manager isi
responsible for:
– Tracking and resolution of all other actions resulting from the
CDR
– Providing final written responses to the comments within one
week of that meeting, to be shared with all of the CDR
participants and stakeholders
– Providing the completed CDR Summary Report and
consolidated comments to the Chief Technology Officer, who
will represent TS at the Executive Steering Committee (ESC)

• Project Manager should also attend the ESC meeting to discuss the
results
l off the
h CDR

• ESC will make a go / no go decision based on the

recommendations provided by the CTO and the Project Manager
© M. S. Ramaiah School of Advanced Studies, Bengaluru
23
 PEMP ESE2516
FT12

Contractor Responsibilities
• System Developer who has knowledge of the proposed system
architecture and the input documents shall:
– Complete the CDR Entry Criteria Checklist prior to
requesting a CDR and provide a high-level presentation to
DPI stakeholders that addresses the following:
• Project Overview
• Design Drivers & Alternatives Considered
• System Architectural Design
• Software
S f A hi
Architecturall Design
D i
• Requirements Traceability
• Data Overview
• Enterprise Architecture Compliance
• Resource Sizing Estimates
• Risks & Mitigation Strategies

© M. S. Ramaiah School of Advanced Studies, Bengaluru

24
 PEMP ESE2516
FT12

Entrance Criteria/Inputs
• Mandatory prerequisites to the CDR
– Business Risk Assessment.
– Preliminaryy Design
g Review Request
q for Action ((RFA)) forms
– Requirements Document(s)
– System Design Description (SDD)
– Software Requirements Specification
– Software Design Description
– Interface Requirements Specification
– Interface Design Description
– U d t d version
Updated i off Information
I f ti Security
S it (IS) Risk
Ri k Assessment
A t
(RA)
– Logical Data Model
– Interface
e ace Co
Control
o Document
ocu e ((ICD)C )
– Database Design Description
– Test Plans
– Software Installation Plan
– System Security Plan (SSP)

© M. S. Ramaiah School of Advanced Studies, Bengaluru

25
 PEMP ESE2516
FT12 Exit Criteria/Outputs
• Assumptions
• Constraints
• Priorities
• Issue Resolutions (if obtained, though this is not to be
the focus during the CDR session; emphasis should be
on identifying
id tif i issues,
i nott resolving
l i them
th during
d i the th CDR
session)
• Unresolved Issues (prioritized as Critical (show-
stoppers) or Non-Critical [opportunities for
improvement])
• Risk Mitigation
g Strategies
g
• Recommendations
• Action Items
• Next Steps
© M. S. Ramaiah School of Advanced Studies, Bengaluru
26
 PEMP ESE2516
FT12

Exit Criteria/ Outputs

• Comments received from the CDR participants and

stakeholders
k h ld are to be
b recorded
d d in
i the
h Request
R f Action
for A i
form and sorted by priority

• Another output of the CDR is the CDR Summary Report

that documents any critical issues that were identified that
must be resolved before the project can move forward with
detailed design and development activities

© M. S. Ramaiah School of Advanced Studies, Bengaluru

27
 PEMP ESE2516
FT12
Review Process
• Project Manager gets the System Developer complete the CDR
Entry Criteria Checklist and then submits the completed
checklist to the CTO along with the appropriate entrance artifacts

• CTO will then approve the scheduling of the CDR

• CDR will be conducted as one or more iterative review sessions
attended
d d by
b allll off the
h key
k TSS stakeholder
k h ld groups, theh System
S
Developer, the Project Manager, and the System Owner

• P
Project
j t Manager
M andd System
S t D l
Developer d
document
t the
th results
lt /
outputs of the CDR and follow-up to ensure that all actions and
any unresolved issues identified during the CDR are
satisfactorilyy addressed

• Project Manager ensures agreement of the CDR Exit Criteria

Checklist by all of the key TS stakeholder groups and submits
the completed checklist to the Chief Technology Officer (CTO)
© M. S. Ramaiah School of Advanced Studies, Bengaluru
28
 PEMP ESE2516
FT12

CDR Design Review Summary Report Format

• Introduction
• P i i
Participants
• Entry Criteria Status
• Design Review Results – Subparagraphs summarize the
status of the design approach and discuss any issues or
concerns.
• Exit Criteria Status
• Requirements to complete the CDR
• Conclusion

© M. S. Ramaiah School of Advanced Studies, Bengaluru

29
 PEMP ESE2516
FT12
Engineering Review Checklists (ERCs)
• ERCs are discipline specific spreadsheets that guide reviewers
throughout the review process to ensure a consistent
minimum level of effort at each design phase

• Each ERC lists the criteria topics that, at a minimum, are to be

reviewed at each design phase

• Lists desired and minimum levels for each criteria

• Provides a quick way to track the levels met at each review

• ERC provides uniformity in the design review process

• Provides a qquick and easyy ppaper p trail that will allow different
reviewers to continue a quality review process in the absence
of the prime reviewer
© M. S. Ramaiah School of Advanced Studies, Bengaluru
30
 PEMP ESE2516
FT12

Responsibility of ETS Staff

• ETS staff responsibilities include:

– Each design discipline shall create and maintain

individual ERCs

– Review submittals using the ERCs in accordance with

these procedures

– Engineering team shall work with the Quality

Management Office staff to ensure that the ERCs
support the QMO database efforts

© M. S. Ramaiah School of Advanced Studies, Bengaluru

31
 PEMP ESE2516
FT12

Responsibility of Quality Management Staff

• QMO staff responsibilities include:

– Maintains and manages QMO program and database

– QMO staff will incorporate the new and updated

information pprovided on the ERCs into the database
system

– Monitor and enforce the Quality Plans of outside

consultant design teams and verify that proper QA and
QC procedures have been followed

© M. S. Ramaiah School of Advanced Studies, Bengaluru

32
 PEMP ESE2516
FT12

Responsibilities of Other Staff

• Document control staff are responsible for maintenance

andd updating
d i off library
lib copy andd distribution
di ib i off procedure
d
updates to other staff

• Contracted consultant design teams have responsibility to

submit design packages for review by ETS staff

© M. S. Ramaiah School of Advanced Studies, Bengaluru

33
 PEMP ESE2516
FT12

Creating and Maintaining ERCs

• ERC template spreadsheet developed by the ETS staff provides
consistent tracking across all the disciplines

• Changes to ERCs are made by

b each respective
respecti e discipline

• Shall be approved by Engineering Technical Service Group

(
(ETSG)Manager
) g

• Where applicable, the following data is provided on the ERCs:

– Criteria ID #
– Review Priority (High, Medium, Low)
– Design Criteria Topics– a list of all requirement and criteria issues
– Reference(s)
R f ( ) – where
h the h requirement
i or criteria
i i may beb found
f d
– Desired Criteria (D) – indicating the preferred design criteria limits
– Minimum Criteria (M) – indicating the borderline limit for each
required criteria

© M. S. Ramaiah School of Advanced Studies, Bengaluru

34
 PEMP ESE2516
FT12

Creating and Maintaining ERCs

– Absolute Minimum Criteria (AM) – requires written

RTD approval

– Check Off / Level of Criteria Met – shaded boxes

indicate which criteria should be checked off at each
submittal phase (50%, 65%, 90%, 100%)

– Notes – providing
pro iding any
an special instructions
instr ctions pertaining to
each given criteria or to the review status for that
submittal

© M. S. Ramaiah School of Advanced Studies, Bengaluru

35
 PEMP ESE2516
FT12
Creating and Maintaining ERCs
• Di
Distributed
ib d to theh other
h engineering
i i disciplines
di i li so they
h may
adjust coordination efforts if needed

• An updated copy shall also be given to the QMO team and to

Document Control

• List of ‘Design Criteria TOPICS’ can be used by in-house

designers and by outside design consultant to guide design
efforts

• Designers should design to their complete best at all times.

© M. S. Ramaiah School of Advanced Studies, Bengaluru

36
 PEMP ESE2516
FT12

Using the Engineering Review Checklist

• ERC is meant to guide the reviewer through the review process
by:
– Highlighting which criteria must be checked (at a
minimum) during each design phase review
– Offers a quick paper reference to track design status by
discipline in order to quantify and support QMO database
entries.

• When performing a review:

– Fill in the date the design was submitted for review.
– Beginning with the High (H) priority criteria items,
followed by the Medium (M) priority items, and finally the
Low (L) priority items: Indicate which criteria have been
met with a check mark and indicate those that have not been
met with an ‘X’
– Indicate, where applicable, to what level (D, M, AM) the
criteria has been met.
© M. S. Ramaiah School of Advanced Studies, Bengaluru
37
 PEMP ESE2516
FT12

Using the Engineering Review Checklist

• Shaded boxes indicate which submittal phase (50%, 65%,

90% 100%) each criteria must be reviewed and marked
90%,
off

• If extra time allows,

allows review the non-shaded
non shaded criteria
– Fill in the date you complete the review
– Fill in your initials

© M. S. Ramaiah School of Advanced Studies, Bengaluru

38
 PEMP ESE2516
FT12

Session Summary

• System development and major application enhancement

projects
j must conduct
d a PDR
• First step is to complete the PDR Questionnaire to
determine readiness to proceed to the PDR and availability
of the input documents

• Component Lead is responsible for ensuring completion of

the PDR Exit Form by all of the key OIS stakeholder
groups and for tracking all critical issues to closure

© M. S. Ramaiah School of Advanced Studies, Bengaluru

39
 PEMP ESE2516
FT12
Session Summary
• CDR is a multi-disciplined product and process assessment to ensure
that the system under review can proceed into system fabrication,
demonstration, and test
• Checks if design can meet the stated performance requirements within
cost, schedule, risk and other system constraints

• CDR assesses the system final design as captured in product

specifications for each configuration item in the system

• Project Manager should attend the ESC meeting to discuss the results of
the CDR

• ESC will make a go / no go decision based on the recommendations

provided by the CTO and the Project Manager

© M. S. Ramaiah School of Advanced Studies, Bengaluru

40