Spreadsheet

Validation
By Praxis Life Sciences

1925 West Field Court, Suite 125, Lake Forest, IL 60045 Validation Center™
praxislifesciences.com | +1(847) 295-7160 validationcenter.com
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Spreadsheet
Validation
© Copyright 2019 by Praxis Life Sciences. All rights reserved. No
part of these materials may be reproduced or transmitted in any
form without the written permission of Praxis.

v.19.07

Your Praxis Facilitator

• Debra Bartel, MBA, CQA, PMP

• VP, Life Sciences

• 30+ years experience specializing in software quality

assurance, validation and regulatory compliance, Information
Systems project management, and process design.

• Prior to joining Praxis, held management positions in the

pharmaceutical industry in both Quality Assurance and
Information Systems organizations

• Active member of American Society for Quality (ASQ),

Northeastern Illinois Section, Software Division

Validation Center™ © 2019 Praxis Life Sciences 2

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 1
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Intro to Praxis Life Sciences

Follow us!
Validation Center™ © 2019 Praxis Life Sciences 3

Target Audience
• Pharmaceutical & Biologics

Industries •
•
Medical Device
Clinical Studies
• Blood Products

Regions • Operating in the US

• Selling to the US Market

• IT Personnel and Managers

Personnel • Software Quality Personnel and
Managers

Validation Center™
www.praxismi.com © 2019 Praxis Life Sciences 4

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 2
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Have you ever heard…

Spreadsheets can’t be used when FDA regulated

Spreadsheets can’t be validated

Spreadsheets can’t be compliant

It takes longer to validate the spreadsheet

than it did to build it

Validation Center™
www.praxismi.com © 2019 Praxis Life Sciences 5

Webinar Outline

1 • FDA Guidance

2 • Validation and Part 11 Compliance

3 • Risk Assessment & Mitigation Approach

4 • Spreadsheet Validation Approach

5 • Spreadsheet Validation Example

6 • FDA Warning Letter Examples

Validation Center™
www.praxismi.com
© 2019 Praxis Life Sciences 6

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 3
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

FDA Requirements and

Guidance
Part 1

Part 1: FDA Requirements and Guidance

Section Overview
• FDA Regulations
• FDA Guidance
• FDA Laboratory Reference Manual

Validation Center™ © 2019 Praxis Life Sciences 8

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 4
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Framework for Regulations, Guidelines

Guidance FDA
PIC/S
Guidance & ICH
Documents Guidance etc.
Reference Guidelines
from Regulatory Documents
Documents
Agencies

Regulations (Laws)
FDA 21 CFR ….
Eudralex Volume …

Company policies & procedures

Best Practices
GAMP
from Industry IEEE ISO
Guides etc.
Groups and (ISPE)
Guides Standards
Standards Bodies

Validation Center™ © 2019 Praxis Life Sciences 9

FDA Regulations for Validation

21 CFR 11 21 CFR 211 21 CFR 820

Persons who use closed
systems to create, modify, When computers or
Input to and output from the
maintain, or transmit electronic automated data processing
records shall employ
computer or related system
systems are used as part of
procedures and controls to or formulas …. shall be
the production or quality
designed to ensure the checked for accuracy.
system, the manufacturer
authenticity, integrity, and … shall validate computer
the confidentiality of electronic The degree and frequency
software for its intended
records, and to ensure that the of input/output verification
use according to an
signer cannot readily repudiate shall be based on the
established protocol.
the signed record ... complexity and reliability
of the computer or related
Procedures and controls shall All software changes shall
system.
include validation of systems be validated before
to ensure accuracy, approval and issuance.
A written record of the
reliability, consistent
program shall be maintained
intended performance, and These validation activities
along with validation data.
the ability to discern invalid or shall be documented.
altered records.

Validation Center™ © 2019 Praxis Life Sciences 10

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 5
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Framework for Regulations, Guidelines

Guidance FDA
PIC/S
Guidance & ICH
Documents Guidance etc.
Reference Guidelines
from Regulatory Documents
Documents
Agencies

Regulations (Laws)
FDA 21 CFR ….
Eudralex Volume …

Company policies & procedures

Best Practices
GAMP
from Industry IEEE ISO
Guides etc.
Groups and (ISPE)
Guides Standards
Standards Bodies

Validation Center™ © 2019 Praxis Life Sciences 11

FDA Software Validation Guidance

Guidance General Principles of Software Validation

Quote:

Many other commercial software applications, such as word processors,

spreadsheets, databases, and flowcharting software are used to
implement the quality system.

All of these applications are subject to the

requirement for software validation,
but the validation approach used for each application
can vary widely.

Validation Center™ © 2019 Praxis Life Sciences 12

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 6
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

FDA Software Validation Guidance

Guidance General Principles of Software Validation

Quote:
Numerous commercial software applications may be used as part of the
quality system (e.g., a spreadsheet or statistical package used for
quality system calculations, a graphics package used for trend analysis,
or a commercial database used for recording device history records or for
complaint management).
The extent of validation evidence needed for such software depends on the
manufacturer’s documented intended use of that software.
For example, a manufacturer who chooses not to use all the vendor-supplied
capabilities of the software only needs to validate those functions that will be
used and for which the manufacturer is dependent upon the software results
as part of production or the quality system.

Validation Center™
www.praxismi.com © 2019 Praxis Life Sciences 13

FDA Software Validation Guidance

Guidance General Principles of Software Validation

Quote:

When software is upgraded or any changes are made to the software,

the manufacturer should consider how those changes may
impact the “used portions” of the software
and must reconfirm the validation of those portions of the software that are used.

Validation Center™ © 2019 Praxis Life Sciences 14

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 7
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Framework for Regulations, Guidelines

Guidance FDA
PIC/S
Guidance & ICH
Documents Guidance etc.
Reference Guidelines
from Regulatory Documents
Documents
Agencies

Regulations (Laws)
FDA 21 CFR ….
Eudralex Volume …

Company policies & procedures

Best Practices
GAMP
from Industry IEEE ISO
Guides etc.
Groups and (ISPE)
Guides Standards
Standards Bodies

Validation Center™ © 2019 Praxis Life Sciences 15

FDA Spreadsheet Reference

Reference FDA Office of Regulatory Affairs Laboratory Manual

Spreadsheet Development Quote:

Although individual spreadsheet functions can be considered as reliable,

it is important to make sure that data is presented to the spreadsheet
with the proper syntax.
Also, when spreadsheets are used for multiple numerical calculations in the
form of in-house developed templates, it is important to
1. protect the spreadsheet from inadvertent changes,
2. verify the reliability of the spreadsheet by comparison with known results
from known data, and to
3. ensure that the spreadsheet can handle unforeseen data input needs.

Validation Center™ © 2019 Praxis Life Sciences 16

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 8
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

FDA Spreadsheet Reference

Reference FDA Office of Regulatory Affairs Laboratory Manual

Spreadsheet Design and Validation Quotes:

General guidance for design and validation of in-house spreadsheets and other
numerical calculation programs includes the following considerations:
1. Lock all cells of a spreadsheet, except those needed by the user to input
data.
2. Make spreadsheets read-only, with password protection, so that only
authorized users can alter the spreadsheet.

3. Design the spreadsheet so that data outside acceptable conditions is

rejected (for example, reject non-numerical inputs).

Validation Center™ © 2019 Praxis Life Sciences 17

FDA Spreadsheet Reference

Reference FDA Office of Regulatory Affairs Laboratory Manual

Spreadsheet Design and Validation Quotes:

4. Manually verify spreadsheet calculations by entering data at extreme values,

as well as at expected values, to assess the ruggedness of the spreadsheet.

5. Test the spreadsheet by entering nonsensical data (for example alphabetical

inputs, <CTRL> sequences, etc.).

6. Keep a permanent record of all cell formulas when the spreadsheet has
been developed. Document all changes made to the spreadsheet and
control using a system of version numbers with documentation.

Validation Center™ © 2019 Praxis Life Sciences 18

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 9
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

www.ValidationCenter.com

Validation Center™ © 2019 Praxis Life Sciences 19

Validation and
Part 11 Compliance
Part 2

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 10
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Part 2: Validation and Part 11 Compliance

Section Overview
– Does my spreadsheet need to be validated?
– Does my spreadsheet need to be 21 CFR Part
11 compliant?
– What is Part 11 compliance?

Validation Center™ © 2019 Praxis Life Sciences 21

What Software Needs Validation?

FDA CSV Principles on

what software needs validation:
1.Software used as a component, part,
or accessory of a medical device

2.Software that is itself a medical device

3.Software used in the production of the

product

4.Software used in implementation of

the quality system

Validation Center™ © 2019 Praxis Life Sciences 22

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 11
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Does My Spreadsheet Need Validation?

Questions to Ask

Could the spreadsheet impact:

• Patient Health? Public Safety?
• Product Quality?
• The integrity of the data and records associated with
either of the above?

Validation Center™ © 2019 Praxis Life Sciences 23

Which Spreadsheet Needs Validation?

A. Per FDA CSV D.

The Principles, which An accounting
spreadsheet require validation? spreadsheet that
that computes also tracks the
the potency of quality status of
the raw material each lot
C.
The
spreadsheet E.
that tracks The
B. which donors spreadsheet
The product are (and are that calculates
complaint not) eligible how long to dry
tracking to donate a batch of
spreadsheet plasma active
ingredient

Validation Center™ © 2019 Praxis Life Sciences 24

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 12
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

What Records Need Part 11 Compliance?

FDA Electronic Records & Signatures Regulation on
what records need to comply with Part 11:
Part 11 applies to:
1.Records in electronic form that are
created, modified, maintained, archived,
retrieved, or transmitted, under any
records requirements set forth in agency
[FDA] regulations.
2.Electronic records submitted to the
agency under requirements of the
Federal Food, Drug, and Cosmetic Act
and the Public Health Service Act, even if
such records are not specifically
identified in agency regulations.

Validation Center™ © 2019 Praxis Life Sciences 25

Does My Spreadsheet Need Part 11 Compliance?

Questions to Ask

1. Could the spreadsheet impact:

• Patient Health? Public Safety?
• Product Quality?
• The integrity of the data and records associated with either of the
above?
AND
2. Will the spreadsheet be used as an electronic record

Validation Center™ © 2019 Praxis Life Sciences 26

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 13
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Which Spreadsheet Needs Part 11 Compliance?

Per FDA 21
CFR Part 11,
A. which spreadsheets
D.
Spreadsheet would be
Spreadsheets of
of training on in scope?
manufacturing
QA SOPs
schedules

C.
Spreadsheets
B. submitted to
the FDA for a E.
Spreadsheet Spreadsheets
of lab test New Drug
Application used to
results calculate lab
(NDA)
results

Validation Center™ © 2019 Praxis Life Sciences 27

Easy to Meet Part 11 Requirements

Part 11 E-Record Requirement:
“ Controls shall include…

• Validation
11.10(a) Validation of systems to ensure accuracy, reliability,
consistent intended performance, and the ability to discern
invalid or altered records.

• Documentation 11.10(k ) Use of appropriate controls over systems documentation

including:

(1) Distribution of, access to, and use of documentation for system
operation and maintenance.
(2) Change control procedures to maintain an audit trail that
documents time-sequenced development and modification

• Record 11.10(c) Protection of records to enable their accurate and ready

Protection retrieval throughout the records retention period.

Validation Center™ © 2019 Praxis Life Sciences 28

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 14
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Easy to Meet Part 11 Requirements

Part 11 E-Record Requirement:
“ Controls shall include…

• Sequencing
Functionality 11.10(f) Use of operational system checks to enforce
permitted sequencing of steps and events.

• Training 11.10(i) Determination that persons who develop, maintain, or use

electronic record/electronic signature systems have the education,
training, and experience to perform their assigned tasks.

• Copies 11.10(b) The ability to generate accurate and complete copies of

records in both human readable and electronic form suitable for
inspection, review, and copying by the [FDA] agency.

• Accountability 11.10(j) The establishment of, and adherence to, written policies that
hold individuals accountable and responsible for actions initiated
under their electronic signatures, in order to deter record and
signature falsification.

Validation Center™ © 2019 Praxis Life Sciences 29

More Challenging Part 11 Requirements

Part 11 E-Record Requirement: Spreadsheet Compliance Tactics

“ Controls shall include…
• Limit access by keeping
11.10(d) Limiting system access spreadsheet in secured folder,
to authorized individuals. doc management system, etc.
• Use the capabilities of the
11.10(g) Use of authority checks network, document
to ensure that only authorized management system, etc. to
individuals can use the system, check authorization to input or
electronically sign a record, alter a record
access the operation or
computer system input or output • Can also password protect
device, alter a record, or perform spreadsheets and cells
the operation at hand.

Validation Center™ © 2019 Praxis Life Sciences 30

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 15
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Difficult Part 11 Requirements

Difficult due to lack of security features
Part 11 E-Record Requirement:
“ Controls shall include…
11.200(a) Electronic signatures 11.100(a) Each electronic signature
not based upon biometrics shall: shall be unique to one individual and
shall not be reused by, or reassigned
(1) Employ at least two distinct to, anyone else.
identification components such
11.300(a) Maintain the uniqueness of
as an identification code and
each combined identification code and
password. password, such that no two individuals
(2) Be used only by their genuine have the same combination.
owners;
(3) Be administered and executed 11.70 Electronic signatures shall be
to ensure that attempted use of linked to their respective electronic
an individual’s electronic records to ensure that the signatures
signature by anyone other than cannot be excised, copied, or
its genuine owner requires transferred to falsify an electronic
collaboration. record by ordinary means

Validation Center™ © 2019 Praxis Life Sciences 31

Difficult Part 11 Requirements

Difficult due to lack of security features
Part 11 E-Record Requirement:
“ Controls shall include…
11.300(b) Ensuring that 11.50(1) Signed electronic records
identification code and password shall contain information
issuances are periodically associated with the signing that
checked, recalled, or revised. clearly indicates all of the following:
(1) The printed name of the signer;
11.300(d) Use of transaction (2) The signature date and time
safeguards to prevent unauthorized (3) The meaning of the signature
use of passwords and/or
identification codes, and to detect 11.50(b) The items, above, shall be
and report in an immediate and subject to the same controls as for
urgent manner any attempts at electronic records and shall be
their unauthorized use to the included as part of any human
system security unit, and, as readable form of the electronic
appropriate, to organizational record (such as electronic display
management. or printout).

Validation Center™ © 2019 Praxis Life Sciences 32

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 16
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Difficult Part 11 Requirements

Difficult due to lack of audit trail features
Part 11 E-Record Requirement: Spreadsheet Compliance Tactics
“ Controls shall include…
• Very difficult requirement for
11.10(e) Use of secure, computer-
generated, time-stamped audit
spreadsheets – no audit
trails to independently record the trails
date and time of operator entries
• For very simple (single
and actions that create, modify, or
delete electronic records.
record) spreadsheets, could
use a document
Record changes shall not obscure management system with
previously recorded information. versioning
Audit trail documentation shall be
retained for a period at least as
long as that required for the subject
electronic records and shall be
available for agency review and copying.

Validation Center™ © 2019 Praxis Life Sciences 33

Compliance Tools
Example spreadsheet tool is available at: www.ofnisystems.com

Validation Center™ © 2019 Praxis Life Sciences 34

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 17
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Spreadsheet Risk
Assessment & Mitigation
Part 3

Part 3: Risk Assessment & Mitigation

Section Overview
– Risk Terminology
– Risk-Based Spreadsheet Validation

Validation Center™ © 2019 Praxis Life Sciences 36

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 18
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Basic Risk Terminology

Hazard
A potential source of harm

Validation Center™ © 2019 Praxis Life Sciences 37

Basic Risk Terminology

Hazard A potential source of harm

Risk
The combination of the probability of occurrence
of a hazard and the severity of the harm

Validation Center™ © 2019 Praxis Life Sciences 38

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 19
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Basic Risk Terminology

Hazard A potential source of harm

Risk Hazard probability and severity

Risk Assessment
A comprehensive evaluation of risks and
associated impacts

Validation Center™ © 2019 Praxis Life Sciences 39

Basic Risk Terminology

Hazard A potential source of harm

Risk Hazard probability and severity

Risk Assessment Evaluation of risk and impact

Risk Mitigation
Actions taken to reduce the impacts of risks

Validation Center™ © 2019 Praxis Life Sciences 40

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 20
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Risk Management Process

Risk Assessment Activities

Identification Evaluation Classification Mitigation

Determine and Determine the Categorize the Perform activities

document the severity of the risks according to that reduce the
hazards identified hazards. severity. severity of the risk
associated with or the likelihood of
use of the system. the risk

Validation Center™ © 2019 Praxis Life Sciences 41

Hazard Identification

Identification

Key Questions
– What could go wrong with this spreadsheet?
– How would this impact business operations?
– Would this negatively affect:
» Patient safety?
» Product quality?
» The Integrity of associated data?

Validation Center™ © 2019 Praxis Life Sciences 42

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 21
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Risk Classification

Evaluation Classification

Risk = The combination of the probability of occurrence

of a hazard and the severity of the hazard

“Criticality” “Complexity”
measures is used to predict
The safety severity “probability of occurrence”
of the hazard for software hazards

Validation Center™ © 2019 Praxis Life Sciences 43

Criticality Classification
Common Classification Method for Criticality
Direct impact on patient safety, product quality,
High Criticality
or the integrity of associated data
Indirect impact on patient safety, product
Medium Criticality
quality, or the integrity of associated data
No impact on patient safety, product quality, or
Low Criticality
the integrity of associated data

Validation Center™ © 2019 Praxis Life Sciences 44

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 22
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Criticality Classification, examples

High Medium Low
Criticality Criticality Criticality
Quality/
Safety Direct Indirect None
Impact
• Health product software • Calibration tracking • Manufacturing cost
• Manufacturing controls • Validation tracking reports
• Automated product • Document management • Turnaround time
inspection • Training tracking reports
• Label management & • Corrective/Preventive action • Out of office
automation tracking calendars
• Distribution tracking to • System access tracking
Examples
enable recalls • Electronic submissions to
• Laboratory test calculations regulatory agencies
• Adverse event tracking • Product work order
• Clinical trial results management
• Patient medical records • Deviation tracking
• Product quality status • Audit tracking
management

Validation Center™ © 2019 Praxis Life Sciences 45

Complexity Classification
Classification Method for Complexity (likelihood of failure)
High Medium Low
Complexity Complexity Complexity
Likelihood
of Defect
High Medium Low

• Custom Macros • Single Level • Standard

• Sophisticated Look-up Logical Functions functions, such
Functions • Advanced as calculations,
Examples • Nested Logic Functions Statistical basic statistics
• Look-up Functions Functions (e.g., averages)
• Linked Spreadsheets

Validation Center™ © 2019 Praxis Life Sciences 46

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 23
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Spreadsheet Risk Assessment Examples

Spreadsheet Function Criticality Complexity Rationale

• Direct impact on
Simple arithmetic
product quality
A calculation for content High Low • Basic spreadsheet
uniformity
functionality
• Indirect impact on
Record of training quality and/or safety
B
attendance
Medium Low • Basic spreadsheet
functionality
Statistical analysis of • Direct impact on patient
C clinical study data with High High safety
VB macros • Custom Macros
• Direct impact on
Statistical analysis of
product quality
D manufacturing data for High Medium • Complex logic and
process control
look-up functions

Validation Center™ © 2019 Praxis Life Sciences 47

Risk Mitigation via Testing

Criticality High Med Low High Med Low High Med Low
Complexity High High High Med Med Med Low Low Low
All paths, All paths, All paths, All paths, All paths,
Path testing multiple single Sampling single single Sampling single Sampling Sampling
scenarios scenario scenario scenario scenario
Boundary testing Required Required Optional Required Optional Optional Required Optional Optional

Test case degree of Specific, Medium Specific, Medium Specific, Medium

General General General
detail and specificity detailed detail detailed detail detailed detail

Test data similar to

Required Required Optional Required Optional Optional Required Optional Optional
production data
Screen Screen Screen
Screen Screen Screen
prints of prints of prints of
prints of prints of prints of
inputs and Signed inputs and Signed inputs and Signed
Testing evidence outputs, outputs, outputs,
outputs, protocol outputs, protocol outputs, protocol
signed signed signed
signed signed signed
protocol protocol protocol
protocol protocol protocol
User (vs. IT)
Required Required Optional Required Optional Optional Required Optional Optional
execution of tests

Validation Center™ © 2019 Praxis Life Sciences 48

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 24
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Risk Based Approach to QA Activities

Criticality High Med Low High Med Low High Med Low
Complexity High High High Med Med Med Low Low Low

Risk Assessment Required Required Required Required Required Required Required Required Required

Change Request Required Required Optional Required Required Optional Required Required Optional
Validation
Required Required Optional Required Required Optional Required Required Optional
Documentation
Design Review Required Required Optional Required Optional Optional Optional Optional Optional
Part 11
Compliance for Required Required Optional Required Required Optional Required Required Optional
any records
Verification of
Required Required Optional Required Required Optional Required Required Optional
SOPs **

** Spreadsheet Use and Support SOPs include: Back-up, Recovery, Security and Access, Training
Requirements, Incident Handling, Change Management, Technical Operation and Routine
Maintenance, User Operation

Validation Center™ © 2019 Praxis Life Sciences 49

Spreadsheet Risk Based Validation Example

Spreadsheet Function Criticality Complexity Rationale

• Indirect impact on
Record of training quality and/or safety
1 Medium Low
attendance • Basic spreadsheet
functionality

Documentation Testing Requirements Other Requirements

Requirements • Sampling of logic paths • SOPs Verification
• Change Request • Screen prints of outputs • Part 11 compliance
• Risk Assessment
• Validation Documentation Testing Options Optional
• Boundaries • Design Review
• Realistic Test Data
• User execution of tests

Validation Center™ © 2019 Praxis Life Sciences 50

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 25
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Spreadsheet Risk Based Validation Example

Spreadsheet Function Criticality Complexity Rationale

Statistical analysis of • Direct impact on

2 clinical study data with High High patient safety
VB macros • Custom Macros

Documentation Testing Requirements Other Requirements

Requirements • All logic paths using • SOPs Verification
• Change Request multiple scenarios • Part 11 compliance
• Risk Assessment • Boundaries • Design Review
• Validation Documentation • Realistic test data
• Screen prints of inputs
and outputs
• User execution of tests

Validation Center™ © 2019 Praxis Life Sciences 51

Spreadsheet Validation
Approach
Part 4

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 26
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Part 3: Spreadsheet Validation Approach

Section Overview
– Validation Definition
– Typical Validation Process
– Spreadsheet Validation Process

Validation Center™ © 2019 Praxis Life Sciences 53

FDA Definitions
The FDA defines software validation as… Examine the
software to
confirm that it
Document all functions as
validation Confirmation by examination and defined in
activities and requirements
test results provision of objective evidence that and it will be
software specifications conform to suitable for
the intended
Define what
the user user needs and intended uses, and use
needs to do
with the that the particular requirements Define how
software
and how
implemented through software can be the software
needs to work
they will use consistently fulfilled to enable the
the software intended use

The examination needs to confirm that the

software will work in all anticipated
situations

Source: General Principles of Software Validation: Final Guidance for Industry and FDA Staff

Validation Center™ © 2019 Praxis Life Sciences 54

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 27
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Validation Life Cycle

Planning Reporting

Specification Verification

Purchase, Coding,
and/or Configuration

GENERAL APPROACH

Validation Center™ © 2019 Praxis Life Sciences 55

Validation Life Cycle

Planning Reporting

User Requirement Requirement

Specification Testing

Functional Functional
Specification Testing
Specification Verification
Configuration Design Configuration
Specification Testing

Configuration

CONFIGURED PRODUCT APPROACH

Validation Center™ © 2019 Praxis Life Sciences 56

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 28
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Standard Validation Project

Change
Request IT
Build, Unit Write
Design
Config. Test IQs

Test
Test Test
Test
Project RunPlans RunPlans Test Val.
URS FS Plans Plans Run IQ
Plan OQs PQs Summary Summary

Test
Test WriteTest
Test
Val. Test
Plans Plans Trace
Plans OQs &
Plans
Val Team, Plan Plans Matrix Val. Val.
PQs
Users Incident Incident
Report Report
TIME

Validation Center™ © 2019 Praxis Life Sciences 57

Validation Life Cycle – Standard Project

Planning Reporting
User Requirement Requirement
Specification Testing
Functional Functional
Specification Testing
Configuration Design Configuration
Specification Testing
Configuration

Change Standard Validation Project

Request
IT
Build, Unit Write
Design
Config. Test IQs

Test
Test Test
Test
Project RunPlans RunPlans Test Val.
URS FS Plans Plans Run IQ
Plan OQs PQs Summary Summary

Test WriteTest
Val. Test
Test Test Trace
Plans
Plans OQs Plans
&
Plans
Val Team, Plan Plans Matrix Val. Val.
PQs
Users Incident Incident
Report Report
TIME

Validation Center™ © 2019 Praxis Life Sciences 58

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 29
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Validation Life Cycle – Standard Project

Planning Reporting
User Requirement Requirement
Specification Testing
Functional Functional
Specification Testing
Configuration Design Configuration
Specification Testing
Configuration

Change Standard Validation Project

Request
IT
Build, Unit Write
Design
Config. Test IQs

Test
Test Test
Test
Project RunPlans RunPlans Test Val.
URS FS Plans Plans Run IQ
Plan OQs PQs Summary Summary

Test
Test WriteTest
Test
Val. Test
Plans Plans Trace
Plans OQs &
Plans
Val Team, Plan Plans Matrix Val. Val.
PQs
Users Incident Incident
Report Report
TIME

Validation Center™ © 2019 Praxis Life Sciences 59

Validation Life Cycle – Standard Project

Planning Reporting
User Requirement Requirement
Specification Testing
Functional Functional
Specification Testing
Configuration Design Configuration
Specification Testing
Configuration

Change Standard Validation Project

Request
IT
Build, Unit Write
Design
Config. Test IQs

Test
Test Test
Test
Project RunPlans RunPlans Test Val.
URS FS Plans Plans Run IQ
Plan OQs PQs Summary Summary

Test WriteTest
Val. Test
Test Test Trace
Plans
Plans OQs Plans
&
Plans
Val Team, Plan Plans Matrix Val. Val.
PQs
Users Incident Incident
Report Report
TIME

Validation Center™ © 2019 Praxis Life Sciences 60

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 30
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Validation Life Cycle – Standard Project

Planning Reporting
User Requirement Requirement
Specification Testing
Functional Functional
Specification Testing
Configuration Design Configuration
Specification Testing
Configuration

Change Standard Validation Project

Request
IT
Build, Unit Write
Design
Config. Test IQs

Test
Test Test
Test
Project RunPlans RunPlans Test Val.
URS FS Plans Plans Run IQ
Plan OQs PQs Summary Summary

Test
Test WriteTest
Test
Val. Test
Plans Plans Trace
Plans OQs &
Plans
Val Team, Plan Plans Matrix Val. Val.
PQs
Users Incident Incident
Report Report
TIME

Validation Center™ © 2019 Praxis Life Sciences 61

Validation Life Cycle – Standard Project

Planning Reporting
User Requirement Requirement
Specification Testing
Functional Functional
Specification Testing
Configuration Design Configuration
Specification Testing
Configuration

Change Standard Validation Project

Request
IT
Build, Unit Write
Design
Config. Test IQs

Test
Test Test
Test
Project RunPlans RunPlans Test Val.
URS FS Plans Plans Run IQ
Plan OQs PQs Summary Summary

Test WriteTest
Val. Test
Test Test Trace
Plans
Plans OQs Plans
&
Plans
Val Team, Plan Plans Matrix Val. Val.
PQs
Users Incident Incident
Report Report
TIME

Validation Center™ © 2019 Praxis Life Sciences 62

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 31
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Validation Life Cycle – Standard Project

Planning Reporting
User Requirement Requirement
Specification Testing
Functional Functional
Specification Testing
Configuration Design Configuration
Specification Testing
Configuration

Change Standard Validation Project

Request
IT
Build, Unit Write
Design
Config. Test IQs

Test
Test Test
Test
Project RunPlans RunPlans Test Val.
URS FS Plans Plans Run IQ
Plan OQs PQs Summary Summary

Test
Test WriteTest
Test
Val. Test
Plans Plans Trace
Plans OQs &
Plans
Val Team, Plan Plans Matrix Val. Val.
PQs
Users Incident Incident
Report Report
TIME

Validation Center™ © 2019 Praxis Life Sciences 63

Validation Life Cycle – Standard Project

Planning Reporting
User Requirement Requirement
Specification Testing
Functional Functional
Specification Testing
Configuration Design Configuration
Specification Testing
Configuration

Change Standard Validation Project

Request
IT
Build, Unit Write
Design
Config. Test IQs

Test
Test Test
Test
Project RunPlans RunPlans Test Val.
URS FS Plans Plans Run IQ
Plan OQs PQs Summary Summary

Test WriteTest
Val. Test
Test Test Trace
Plans
Plans OQs Plans
&
Plans
Val Team, Plan Plans Matrix Val. Val.
PQs
Users Incident Incident
Report Report
TIME

Validation Center™ © 2019 Praxis Life Sciences 64

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 32
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Validation Life Cycle – Standard Project

Planning Reporting
User Requirement Requirement
Specification Testing
Functional Functional
Specification Testing
Configuration Design Configuration
Specification Testing
Configuration

Change Standard Validation Project

Request
IT
Build, Unit Write
Design
Config. Test IQs

Test
Test Test
Test
Project RunPlans RunPlans Test Val.
URS FS Plans Plans Run IQ
Plan OQs PQs Summary Summary

Test
Test WriteTest
Test
Val. Test
Plans Plans Trace
Plans OQs &
Plans
Val Team, Plan Plans Matrix Val. Val.
PQs
Users Incident Incident
Report Report
TIME

Validation Center™ © 2019 Praxis Life Sciences 65

Validation Life Cycle – Standard Project

Planning Reporting
User Requirement Requirement
Specification Testing
Functional Functional
Specification Testing
Configuration Design Configuration
Specification Testing
Configuration

Change Standard Validation Project

Request
IT
Build, Unit Write
Design
Config. Test IQs

Test
Test Test
Test
Project RunPlans RunPlans Test Val.
URS FS Plans Plans Run IQ
Plan OQs PQs Summary Summary

Test WriteTest
Val. Test
Test Test Trace
Plans
Plans OQs Plans
&
Plans
Val Team, Plan Plans Matrix Val. Val.
PQs
Users Incident Incident
Report Report
TIME

Validation Center™ © 2019 Praxis Life Sciences 66

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 33
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Approval Points – Standard Project

Change
Request
IT
Build, Unit Write
Design
Config. Test IQs

Test
Test Test
Test
Project Run
Plans RunPlans Test Val.
URS FS Plans Plans Run IQ
Plan OQs PQs Summary Summary

Test WriteTest
Val. Test
Test Test Trace
Plans
Plans OQs Plans
&
Plans
Val Team, Plan Plans Matrix Val. Val.
PQs
Users Incident Incident
Report Report
TIME

Validation Center™ © 2019 Praxis Life Sciences 67

Spreadsheets Are Different

1. Much smaller scope

2. Much shorter timeline
3. Usually less complex
• Single technology
• Single function
• Little or no programming
• Single programmer/developer
• Well understood technology
vs.

Validation Center™ © 2019 Praxis Life Sciences 68

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 34
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Spreadsheet Adaptation
How Can the Standard Validation Project Deliverables
be adapted for Spreadsheet Validation?

Change
Request
Build, Unit Write
Design
Config. Test IQs

Test
Test Test
Test
Project Run
Plans RunPlans Test Val.
URS FS Plans Plans Run IQ
Plan OQs PQs Summary Summary

Test WriteTest
Val. Test
Test Test Trace
Plans
Plans OQs Plans
&
Plans
Plan Plans Matrix Val. Val.
PQs
Incident Incident
Report Report
TIME

1. Eliminate the Project Plan

Validation Center™ © 2019 Praxis Life Sciences 69

Spreadsheet Adaptation
How Can the Standard Validation Project Deliverables
be adapted for Spreadsheet Validation?

Change
Request
Build, Unit Write
Design
Config. Test IQs

Test Test
Test
Run Test
Run
Project Plans Plans Test Val.
URS FS Plans Plans Run IQ
Plan OQs PQs Summary Summary

Test
Test WriteTest
Test
Val. Test
Plans Plans Trace
Plans OQs &
Plans
Plan Plans Matrix Val. Val.
PQs
Incident Incident
Report Report
TIME

2. Eliminate the Unit Test

Validation Center™ © 2019 Praxis Life Sciences 70

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 35
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Spreadsheet Adaptation
How Can the Standard Validation Project Deliverables
be adapted for Spreadsheet Validation?

Change
Request
Build, Unit Write
Design
Config. Test IQs

Test
Test Test
Test
Project Run
Plans RunPlans Test Val.
URS FS Plans Plans Run IQ
Plan OQs PQs Summary Summary

Test WriteTest
Val. Test
Test Test Trace
Plans
Plans OQs Plans
&
Plans
Plan Plans Matrix Val. Val.
PQs
Incident Incident
Report Report
TIME

3. Combine the User Requirements and Functional Specifications

Validation Center™ © 2019 Praxis Life Sciences 71

Spreadsheet Adaptation
How Can the Standard Validation Project Deliverables
be adapted for Spreadsheet Validation?

Change
Request
Build, Unit Write
Design
Config. Test IQs

Test Test
URS & Test
Run Test
Run
Project Plans Plans Test Val.
URS Plans Plans Run IQ
Plan FS OQs PQs Summary Summary

Test
Test WriteTest
Test
Val. Test
Plans Plans Trace
Plans OQs &
Plans
Plan Plans Matrix Val. Val.
PQs
Incident Incident
Report Report
TIME

4. Combine the Operational and Performance Qualification

Include the Trace Matrix

Validation Center™ © 2019 Praxis Life Sciences 72

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 36
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Spreadsheet Adaptation
How Can the Standard Validation Project Deliverables
be adapted for Spreadsheet Validation?

Change
Request
Build, Unit Write
Design
Config. Test IQs

Test
Test Test
Test
Project URS & Run
Plans RunPlans Test Val.
URS Plans Plans Run IQ
Plan FS OQ/PQ PQs Summary Summary

Test Test
Write
Val. Test
Test Test Trace
Plans
Plans Plans
OQ/PQ
Plans
Plan Plans Matrix Val. Val.
w.Trace
Incident Incident
Report Report
TIME

5. Combine the Validation and Test Plan

Validation Center™ © 2019 Praxis Life Sciences 73

Spreadsheet Adaptation
How Can the Standard Validation Project Deliverables
be adapted for Spreadsheet Validation?

Change
Request
Build, Unit Write
Design
Config. Test IQs

Test Test
URS & Test
Run Test
Run
Project Plans Plans Test Val.
URS Plans Plans Run IQ
Plan FS OQ/PQ PQs Summary Summary

Val. & Test

Test Test
Write
Test
Test
Plans Plans Trace
Test Plans OQ/PQ
Plans
Plans Matrix Val. Val.
Plan w.Trace
Incident Incident
Report Report
TIME

6. Combine the Validation and Test Summaries

Validation Center™ © 2019 Praxis Life Sciences 74

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 37
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Spreadsheet Adaptation
How Can the Standard Validation Project Approval Points
be adapted for Spreadsheet Validation?

Write
Change
Val. & Val &
Build, OQ/PQ Write Run
Request Test URS/ FS Design Test Run IQ
Config. w/Trace IQs OQ /PQ
Plan Summary
Matrix

Val.
Incident
Report

TIME

7. Reduce the Approval Points

Validation Center™ © 2019 Praxis Life Sciences 75

Validation Life Cycle – Spreadsheets

Planning Reporting
User Requirement Requirement
Specification Testing
Functional Functional
Specification Testing
Configuration Design Configuration
Specification Testing
Configuration

Write
Change
Val. & Val &
Build, OQ/PQ Write Run
Request Test URS/ FS Design Test Run IQ
Config. w/Trace IQs OQ /PQ
Plan Summary
Matrix

Spreadsheet Validation Project

TIME

Validation Center™ © 2019 Praxis Life Sciences 76

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 38
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Spreadsheet Validation
Example
Part 5

Part 5: Spreadsheet Validation Example

Section Overview
– Walkthrough example of validation for a small
spreadsheet application using a condensed
validation template

Validation Center™ © 2019 Praxis Life Sciences 78

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 39
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

The Spreadsheet

Purpose: Calculate the expiration date of a batch of active

pharmaceutical ingredients based on the potency of 3 samples
Validation Center™ © 2019 Praxis Life Sciences 79

Validation Table of Contents

Validation Center™ © 2019 Praxis Life Sciences 80

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 40
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Change Authorization
• Can be incorporated into the validation template or
used as a stand-alone document
• Include description and justification for change
• Approved by
• Quality Assurance
• Business Owner (function that will use the spreadsheet)
• Technology Owner (function that will build the spreadsheet)

Write
Change
Val. & Val &
Build, OQ/PQ Write Run
Request Test URS/ FS Design Test Run IQ
Config. w/Trace IQs OQ /PQ
Plan Summary
Matrix

Validation Center™ © 2019 Praxis Life Sciences 81

Change Request

Validation Center™ © 2019 Praxis Life Sciences 82

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 41
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Validation Plan
• Validation Scope
• Risk Assessment and Mitigation
• Validation Strategy
• Test Plan
• Acceptance Criteria

Write
Change
Val. & Val &
Build, OQ/PQ Write Run
Request Test URS/ FS Design Test Run IQ
Config. w/Trace IQs OQ /PQ
Plan Summary
Matrix

Validation Center™ © 2019 Praxis Life Sciences 83

Validation Plan

Validation Center™ © 2019 Praxis Life Sciences 84

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 42
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Validation Plan

Validation Center™ © 2019 Praxis Life Sciences 85

Validation Plan

Validation Center™ © 2019 Praxis Life Sciences 86

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 43
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

User and Functional Requirements

• Combine into single set of requirements
o High Level = user requirements
o Details = functional requirements
• Document the features needed for
o Intended business use
o FDA compliance
• Key areas:
o Calculations, statistical functions
o Logic
o Security

Write
Change
Val. & Val &
Build, OQ/PQ Write Run
Request Test URS/ FS Design Test Run IQ
Config. w/Trace IQs OQ /PQ
Plan Summary
Matrix

Validation Center™ © 2019 Praxis Life Sciences 87

User and Functional Requirements

Validation Center™
www.praxismi.com
© 2019 Praxis Life Sciences 88

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 44
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

User and Functional Requirements

Validation Center™
www.praxismi.com © 2019 Praxis Life Sciences 89

Build and Document Design

• Document the design of the spreadsheet

For Example …
o Reference any design standards followed
o Insert or attach an image of the spreadsheet
o Insert or attach a view of the spreadsheet showing the
formulas in each cell
o Insert or attach images of configurations, macros, security
settings, and any other design elements

Write
Change
Val. & Val &
Build, OQ/PQ Write Run
Request Test URS/ FS Design Test Run IQ
Config. w/Trace IQs OQ /PQ
Plan Summary
Matrix

Validation Center™
www.praxismi.com
© 2019 Praxis Life Sciences 90

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 45
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Design Standards, Example

• Password protection on non-data entry cells
• Formatting standards, such as inclusion of
spreadsheet ID, version number, and validation
reference (e.g. validation ID and/or date) on
spreadsheet
• Standard number of decimal points to use
• Standard for rounding rules
• Standard date formats
• Standard colors for data entry cells and calculated
cells
• Standard file naming conventions
• Standard error message formats

Validation Center™ © 2019 Praxis Life Sciences 91

Formula Printing

Validation Center™ © 2019 Praxis Life Sciences 92

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 46
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Operational and Performance Qualification

• Combine into single set of tests
• Follow approved test plan
o Scope of requirements to test
o Documentation requirements
• Document Test Design (scenarios & data) for high
criticality / high complexity features
• Cross-reference (trace) each step to the requirement
tested
• Leave space for
o Entry of actual results
o Name, signature, and date of tester

Write
Change
Val. & Val &
Build, OQ/PQ Write Run
Request Test URS/ FS Design Test Run IQ
Config. w/Trace IQs OQ /PQ
Plan Summary
Matrix

Validation Center™
www.praxismi.com © 2019 Praxis Life Sciences 93

OQ/PQ Test Design

Validation Center™ © 2019 Praxis Life Sciences 94

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 47
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

OQ/PQ Test Design

Validation Center™ © 2019 Praxis Life Sciences 95

OQ/PQ Test Protocol

Validation Center™ © 2019 Praxis Life Sciences 96

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 48
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

OQ/PQ Test Protocol

Validation Center™ © 2019 Praxis Life Sciences 97

Installation Qualification
• Describe the steps to install the spreadsheet into
production
o Location
o File security
o Password protection

• Include space for

o Entry of actual results
o Name, signature, and date of tester

Write
Change
Val. & Val &
Build, OQ/PQ Write Run
Request Test URS/ FS Design Test Run IQ
Config. w/Trace IQs OQ /PQ
Plan Summary
Matrix

Validation Center™
www.praxismi.com
© 2019 Praxis Life Sciences 98

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 49
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Installation Qualification

Validation Center™ © 2019 Praxis Life Sciences 99

Preliminary Approval
• Obtain approval of all deliverables up to this point

Write
Change
Val. & Val &
Build, OQ/PQ Write Run
Request Test URS/ FS Design Test Run IQ
Config. w/Trace IQs OQ /PQ
Plan Summary
Matrix

Validation Center™ © 2019 Praxis Life Sciences 100

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 50
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Preliminary Approval

Validation Center™ © 2019 Praxis Life Sciences 101

Run OQ and PQ
• Follow the approved test protocol
o Execute steps
o Record actual results
o Capture screen prints
o Document steps that pass (or fail)
• Create a Verification Issue Report for any failing tests
o Describe failure
o Determine error type (e.g., test error, tester error, functionality error)
o Describe resolution
o Describe re-testing and regression testing required

Write
Change
Val. & Val &
Build, OQ/PQ Write Run
Request Test URS/ FS Design Test Run IQ
Config. w/Trace IQs OQ /PQ
Plan Summary
Matrix

Validation Center™ © 2019 Praxis Life Sciences 102

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 51
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Issue Report

Validation Center™ © 2019 Praxis Life Sciences 103

Summary Reports
• Testing Summary
o Summarize the results of the verification (testing) activities
• Validation Summary
o Compare current state vs. acceptance criteria identified in the
approved Validation Plan

Write
Change
Val. & Val &
Build, OQ/PQ Write Run
Request Test URS/ FS Design Test Run IQ
Config. w/Trace IQs OQ /PQ
Plan Summary
Matrix

Validation Center™ © 2019 Praxis Life Sciences 104

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 52
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Summary Reports

Validation Center™ © 2019 Praxis Life Sciences 105

Validation Approval
• Approve the validation deliverables and authorize the
production use of the spreadsheet

Write
Change
Val. & Val &
Build, OQ/PQ Write Run
Request Test URS/ FS Design Test Run IQ
Config. w/Trace IQs OQ /PQ
Plan Summary
Matrix

Validation Center™ © 2019 Praxis Life Sciences 106

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 53
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Validation Approval

Validation Center™ © 2019 Praxis Life Sciences 107

Installation, Closure
Installation
• Execute the pre-approved Installation Qualification
o Execute steps
o Record actual results
o Capture screen prints
o Document steps that pass (or fail)

Closure
• Obtain signature to close validation project

Write
Change
Val. & Val &
Build, OQ/PQ Write Run
Request Test URS/ FS Design Test Run IQ
Config. w/Trace IQs OQ /PQ
Plan Summary
Matrix

Validation Center™ © 2019 Praxis Life Sciences 108

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 54
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Validation Closure

Validation Center™ © 2019 Praxis Life Sciences 109

FDA Warning Letter

Examples
Part 6

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 55
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Part 6: FDA Warning Letter Examples

Section Overview
• Warning Letter Data
• Warning Letter Examples

Data Source
• FDA Warning Letters related to Software and
Computer Systems
• 9 Year Range: 2010-2018

Validation Center™ © 2019 Praxis Life Sciences 111

FDA Warning Letter Example

Audit
Finding

Regulation

Observations

Validation Center™ © 2019 Praxis Life Sciences 112

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 56
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Software & Computer Warning Letters

9 Year Summary
by System Type

1%
2% 10% Device/product Software
2% Laboratory Systems
3%
27% Manufacturing Control Software
4% Complaint Systems
5% SPREADSHEETS
Inventory Control Systems
6% Blood Management Systems
40% Clinical Study Systems
Document Management
Others (< 2% each)

Validation Center™ © 2019 Praxis Life Sciences 113

Spreadsheet Warning Letters

9 Year Summary
by Usage
5%
4%
4% Laboratory Calculations
4% Complaint Tracking
Annual Product Review Data

14% 55% CAPA Tracking

Non-Conformance Tracking
14% Manufacturing Calculations
Qualification Tracking

Validation Center™ © 2019 Praxis Life Sciences 114

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 57
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Spreadsheet Warning Letters

9 Year Summary
by Topic
4%
4%
4% Lack of Security

Failure to Validate
9%
31%
Not Suitable for Intended Use

Lack of Audit Trails

22% Data Integrity

CAPA for Spreadsheet Issue

26% Change Control

Validation Center™ © 2019 Praxis Life Sciences 115

FDA Warning Letter Examples

Regulatory Reference 21 CFR 606 (Blood Products)
FDA Office Los Angeles District
Spreadsheet Usage Quality Metrics Tracking & Trending
Warning Topic Validation

There are no data to demonstrate that the quality control/quality assurance

spreadsheets used for tracking and trending various quality metrics have
been properly validated (installation qualification, operational qualification,
and performance qualification) and are performing as intended.

The percent Average and percent Standard Deviation results did not change
from month to month on the Aberrant Controls, Failed Run or Failed
Runs/Aberrant Controls spreadsheets. The error was due to failure to use the
current rolling 12-month formula in the spreadsheet calculations.
When a recalculation was performed, an alert level for HIV was identified.

Validation Center™ © 2019 Praxis Life Sciences 116

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 58
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

FDA Warning Letter Examples

Regulatory Reference 21 CFR 820 (Medical Devices)
FDA Office Detroit District
Spreadsheet Usage Complaints Tracking
Warning Topic Suitability for use

Firm tracks complaint data on a spreadsheet that contains free form text
fields that are not standardized, resulting in an
inability to adequately trend the data.
For example,
when using complaint data for part "6090," 14 complaints are shown.
However, the spreadsheet contains several different descriptions of the
same part failure that when totaled resulted in a total count of
40 complaints related to part #6090.

Validation Center™ © 2019 Praxis Life Sciences 117

FDA Warning Letter Examples

Regulatory Reference 21 CFR 211 (Pharmaceuticals)
FDA Office Baltimore District
Spreadsheet Usage Laboratory Calculation
Warning Topic Security

Your firm failed to exercise appropriate controls over computer or related

systems to ensure that changes are instituted only by authorized personnel.

Specifically

Your firm’s laboratory analysts have the ability to access and modify the
formulas in the Excel spreadsheets used to calculate assay results for drug
products. Due to the unrestrictive access, there is no assurance that the
formulas in the Excel spreadsheets are accurate and valid.

Validation Center™ © 2019 Praxis Life Sciences 118

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 59
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

FDA Warning Letter Examples

Regulatory Reference 21 CFR 820 (Medical Devices)
FDA Office CDRH
Spreadsheet Usage CAPA and Non-conformance
Management
Warning Topic Suitability for use

"Task Force Meeting" spreadsheets are used to review

all non-conformances.
However, these spreadsheet records do not document:
• Investigation of the cause of nonconformities
• Implementation of action plans needed to correct and prevent
identified quality problems,
• Whether corrective actions were verified or validated as effective,
or that the actions do not adversely affect the finished device

Validation Center™ © 2019 Praxis Life Sciences 119

FDA Warning Letter Examples

Regulatory Reference 21 CFR 211 (Pharmaceuticals)
FDA Office CDER
Spreadsheet Usage Laboratory Calculation
Warning Topics Security, Validation, Change Control

At the drug facility, the investigator noticed that the use of the Excel
spreadsheets in analytical calculations are neither controlled nor protected
from modifications or deletion.

The investigator noticed that the calculation for residual solvent uses an
Excel spreadsheet that has not been qualified. We are concerned about the
data generated by your QC laboratory from non-qualified and uncontrolled
Excel spreadsheets.

Validation Center™ © 2019 Praxis Life Sciences 120

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 60
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Conclusion

Now You Know …

Spreadsheets can be used when FDA regulated

Spreadsheets can be validated

Spreadsheets can be compliant

It doesn’t need to take longer to

validate the spreadsheet than to build it

Validation Center™ © 2019 Praxis Life Sciences 122

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 61
 Reproduction or re‐transmission in any form or by any means, electronic or mechanical
without prior written permission from Praxis is prohibited.
© 2019 Praxis Life Sciences. All rights reserved.

Need Help?

ValidationCenter.com Library of SOPs and CSV templates

Online and Classroom CSV Training

Software QA and Validation Program Implementation

Validation Services

Audit Readiness Assessments

Validation Center™ © 2019 Praxis Life Sciences 123

Thank You!
Thanks for your interest in Spreadsheet Validation

Any questions about what we have discussed today?

Please, feel free to contact me:

Debra Bartel

+1 (847) 295-7160
[email protected]

validationcenter.com | praxislifesciences.com

Follow us!

Validation Center™ © 2019 Praxis Life Sciences 124

Praxis Life Sciences | praxislifesciences.com Validation Center™

1925 West Field Court, Suite 125, Lake Forest, IL 60045 | +1 (847) 295‐7160 validationcenter.com 62