Data Integrity Issues & Concerns

PDA Meeting
St. Louis, MO February 06, 2017

CAPT Sharon K. Pederson (Thoma), PharmD

National Expert of Pharmaceutical Inspections
Food and Drug Administration
Medical Products and Tobacco Program Operations Branch
ORA/OO/OMPTO/DMPTPO

1
 Objectives
• What is Data Integrity?
• Regulations
• Why is Data Integrity Important?
• Significant Issues
• Warning Letter (W/L) / Untitled
Letter (U/L) charges and some
FDA 483 examples. 2
 What is Data Integrity?
• Complete, consistent, and accurate data to
assure patient safety and product quality.
• Complete, consistent, and accurate data should be
attributable, legible, contemporaneously recorded,
original or a ‘true copy’, and accurate (ALCOA).
• Good Documentation Practices for Static and
Dynamic Records.
• Data integrity should be maintained throughout the
data life cycle, including, but not limited to data
creation, processing, archiving and disposition after
3
record’s retention period.
 Use of “static” and “dynamic” in
relation to record format
• Static: fixed data document (e.g., paper record
or an electronic image)

• Dynamic: record format allows interaction

between the user and the record content (e.g., a
chromatogram where the integration parameters
can be modified)
4
 Trustworthy Record Characteristics

• Reliabile: complete & accurate

• Authentic: proven to be what it purports to
be
• Integrity: Complete & unaltered
• Usable: can be located, retrieved,
presented & interpreted

5
 ALCOA
• Attributable – Traceable to a unique individual
• Legible – Data must be recorded permanently and be
readable
• Contemporaneously – Activities must be recorded at the
time they occur
• Original or a true copy – first capture of data (not transcribed
data), must review the original record, must retain the
original or certified copy* of the original record.
• Accurate – records must be accurate, which is achieved thru
the Quality Management System

*Certified Copy is verified by a 2nd person who compares the copy to the
original, confirms that the copy is accurate & complete and preserves 6
Content & meaning (i.e., all data and metadata; documents the verification)
 What is Metadata?
• Contextual information required to understand
data (i.e., data about the data)
• Structured information that describes, explains,
or otherwise makes it easier to retrieve, use or
manage data
• Examples: date/time stamp, user ID, instrument
ID, audit trails, etc.
• Relationships between data and their metadata
should be preserved in a secure and traceable
manner 7
 What is an Audit Trail?
• Secure, computer-generated, time-stamped
electronic record that allows for reconstruction of
events relating to the creation, modification, or
deletion of an electronic record
• Chronology: who, what, when, and why of a
record
• Track actions at the record or system level
• CGMP-compliant record-keeping practices
prevent data from being lost or obscured
8
 Audit Trail captures…
• Overwriting
• Runs that have been aborted
• Testing into compliance
• Deleting data
• Backdating
• Altering data

(not an all-inclusive list)

9
 How often should audit trails be
reviewed?
• FDA recommends that audit trails capturing changes to
critical data be reviewed with each record and before
final approval of the record.
• Audit trails subject to regular review should include, for
example, changes to: finished product test results,
sample run sequences, sample identification, critical
process parameters.
• FDA recommends routine scheduled audit trail review
based on the complexity of the system and its intended
use.
10
 What are the Regulations?
Data in accordance with CGMP requirements for
drugs (i.e., as required by 21 CFR parts 210, 211,
and 212).

• Part 210 – Current Good Manufacturing Practice in Manufacturing,

Processing, Packing, or Holding of Drugs; General.
• Part 211 – Current Good Manufacturing Practice for Finished
Pharmaceuticals.
• Part 212 – Current Good Manufacturing Practice for Positron
Emission Tomography Drugs.
• Q7A – Active Pharmaceutical Ingredients.
11
 Regulations – Data Integrity
Requirements with respect to data integrity in parts 211 and 212
continued:
• 211.188, 211.194, and 212.60(g) (requires “complete information,”
“complete data derived from all tests,” “complete record of all data,”
“original records have been reviewed for accuracy, completeness,
and compliance with established standards,” and “complete records
of all tests performed”).
• 211.192 (requires production and control records be “reviewed”)
• 211.101(c) and (d), 211.103, 211.182, 211.186(a), 211.188(b)(11),
and 211.194(a)(8) require records be “reviewed” by a second
person.

12
 Regulations – Data Integrity
Requirements with respect to data integrity in parts 211 and 212
include:
• 211.68 (requires “backup data are exact and complete,” and “secure
from alteration, inadvertent erasures, or loss;” and that “output from
the computer” “be checked for accuracy”;
• 212.110(b) (requires data be “stored to prevent deterioration or
loss”);
• 211.100 and 211.160 (requires that certain activities be
“documented at the time of performance” and that laboratory
controls be “scientifically sound”);
• 211.180 (requires records be retained as “original records,” “true
copies,” or other “accurate reproductions of the original records”);
13
 Regulations – Data Integrity
• Electronic signature and record-keeping requirements in
21 CFR part 11 apply to certain records subject to record
requirements set forth in the regulations (i.e., 210, 211,
and 212).

• Guidance for Industry Part 11, Electronic Records;

Electronic Signatures — Scope and Application.

14
How does FDA use the term “backup” in
211.68(b)?
• True copy of the original data that is
maintained securely throughout the records
retention period and reviewed by QCU.
• Electronic CGMP data should include relevant
metadata.
• To exclude data from the release criteria
decision-making process, there must be a
valid, documented, scientific justification for its
exclusion. 15
How should access to CGMP
computer systems be restricted?
• Appropriate controls to assure only authorized
personnel change computerized: For examples,
MPCRs, Input of laboratory data into records,
etc.
• Recommend restricting the ability to alter:
– Specifications
– Process parameters
– Manufacturing or testing methods
16
How should access to CGMP
computer systems be restricted cont.
• Recommend system administrator role,
including any rights to alter files and settings, be
assigned to personnel independent from those
responsible for the record content
• Recommend maintaining a list of authorized
individuals and their access privileges for each
CGMP computer system in use
• May not be practical for small operations with
few employees
17
Why is FDA concerned with the use of
shared login accounts for computer
systems?

A firm must
• Exercise appropriate controls to assure that only
authorized personnel make changes to
computerized records.
• Ensure actions are attributable to a specific
individual.

18
Paper Record Comparison
• If actions are not attributable to a specific
individual, a BPCR would look like all the values
were entered but the people who performed and
reviewed every step would be empty.
• Firm would not know if the unidentified
individuals are authorized to perform the activity.

19
How should Blank Forms be controlled?

• Blank forms (e.g., worksheets, laboratory

notebooks, and MPCRs) should be controlled by
the quality unit or by another document control
method
• Numbered sets of blank forms may be issued and
should be reconciled upon completion of the activity
• Incomplete or erroneous forms should be kept as
part of the permanent record along with written
justification for their replacement
20
Paper Record Comparison

• Bound paginated notebooks, stamped for official

use by a document control group, allow
detection of unofficial notebooks as well as of
any gaps in notebook pages.
• An electronic document management system
could have the capability to reconcile and
document the number of copies printed.

21
When does electronic data
become a CGMP record?
• When it is generated to satisfy a CGMP
requirement.
• You must document, or save, the data at the
time of performance.
• Not acceptable to store data in temporary
memory; this allows for manipulation, before
creating a permanent record.

22
When does electronic data become a
CGMP record continued…

• You may employ a combination of technical and

procedural controls to meet CGMP
documentation practices

• Computer systems, such as LIMS or EBR

systems, can be designed to save after separate
entries.

23
Paper Record Comparison

• This is similar to recording each entry

contemporaneously on a paper batch record

24
Is it acceptable to only save the final
results from reprocessed laboratory
chromatography?

• No
• Analytical methods should be capable and
stable
• If reprocessed, written procedures must be
established and followed
• FDA requires laboratory records include
complete data derived from all tests
25
 Why is Data Integrity Important?
• FDA CGMP inspection(s) have uncovered violations with
data integrity issues.
• Data integrity is an important component of industry’s
responsibility to ensure the safety, efficacy, and quality of
drugs, and of FDA’s ability to protect the public health.
• Data integrity-related cGMP violations may lead to regulatory
actions, including warning letters, import alerts, and consent
decrees.
• The underlying premise in 210.1 and 212.2 is that CGMPs
sets forth minimum requirements to assure drugs meet
standards of the Federal Food, Drug, and Cosmetic Act
(FD&C Act) regarding safety, identity, strength, quality, and
26
purity.
 Why is Data Integrity Important?
• FDA’s authority for CGMP comes from FD&C Act section
501(a)(2)(B).
• 501(a)(2)(B) states: a drug shall be deemed adulterated
if “the methods used in, or the facilities or controls used
for, its manufacture, processing, packing, or holding do
not conform to or are not operated or administered in
conformity with current good manufacturing practice to
assure that such drug meets the requirement of the act
as to safety and has the identity and strength, and meets
the quality and purity characteristics, which it purports or
is represented to possess.”
27
 Why is Data Integrity Important?

• FDA expects data to be reliable and accurate.

CGMP regulations and guidance allow for
flexible and risk-based strategies to prevent and
detect data integrity issues.
• Firms should implement meaningful and
effective strategies to manage their data integrity
risks based upon their process understanding
and knowledge management of technologies.

28
 Why is Data Integrity Important?

• Reliability on the information used to

ensure the quality of the drugs that
consumers will take
• Data integrity problems break trust
• FDA rely on firm’s to do the right thing
when FDA is not present.

29
 Examples of significant issues

• No raw data to support records

• Creating inaccurate and incomplete records
• Test results for one batch used to release other
batches
• Backdating
• Fabricating data
• Discarding data
30
 Examples of significant issues
• Repeated tests, trial runs, sample runs (testing into
compliance)
• Changing integration parameters of chromatographic
data to obtain passing results
• Deletion/manipulation of electronic records
• Turning off audit trail
• Sharing password
• Inadequate controls for access privileges
• Inadequate/incomplete computer validation 31
 Examples of significant issues
• Inadequate investigations
• Inaccurate reporting of microbial, sterility, or
endotoxin data results
• Loss of data during changes to the system
• Activities not recorded contemporaneously
• Employees that sign that they completed
manufacturing steps when the employees were not
on premises at the time the steps were completed.
 Things to consider…
• Is data integrity a problem at your facility? What
measures are in place to prevent data integrity
problems?
• Are internal audit procedures adequate? What
measures are in place and will they detect data
integrity issues?
• Does senior management cultivate adequate and
accurate reporting of events when things go wrong
during manufacturing…during testing?
• Train personnel to detect and prevent data integrity
33
as part of routine CGMP training.
 Data Audit Considerations
• Is the data being accurately interpreted?
• Is the data set accurately calculated and
accurately reported?
• Does source data support the reported data
and conclusions?
• Was there relevant data generated that
should have been included but was not?

34
 Where does the Agency find Data
Integrity Issues?
• Domestic and international facilities
• Small and large pharmaceuticals
companies
• Manufacturing operations
• Quality units, including quality control
laboratories (chemistry and microbiology)
• Clinical Trials
35
 WL/ULs charges cited
• Failure to prepare batch production and
control records for each batch of drug
product that include documentation of the
accomplishment of each significant step in
the manufacture, processing, packing, or
holding of the batch (21 CPR 211.188(b)).

36
 WL/ULs charges cited
• Failure to ensure that laboratory records
included complete data derived from all tests
necessary to assure compliance with
established specifications and standards (21
CFR 211.194(a)).
• Failure to exercise appropriate controls over
computer or related systems to assure that only
authorized personnel institute changes in master
production and control records, or other records
(21 CFR 211.68(b)). 37
 FDA 483 Example 1
Reference 21 CFR 211.194(a)(3)
Laboratory records do not include a statement of the weight or measure of sample
used for each test, where appropriate.

Specifically, the firm's Supplemental Method Information Sheet (SMIS) for

Pharmaceutical Methods *** Pouch Testing approval date 05/07/07 indicates for
viscosity the amount on sample cup is critical try 0.12 mL using 3 cc syringe. If
there is too little sample, result will be low. If there is too much sample results
will be high.

For *** Pouch Testing (Viscosity Testing) the firm did not document the measure of
sample used in their laboratory note book pages for the following samples:
• Sample NS-05055119
• Sample NS-05055120
• Sample NS-05055121
• Sample NS-05055122
• Sample NS-05055123 38
 FDA 483 Example 2
Reference 21 CFR 211.68 – Validated for intended use

Appropriate controls are not exercised over computers or

related systems...Specifically, the *** software system has
not been validated for its intended use. Per the validation
protocol, this software is used for drug material and inventory
control, drug production scheduling, and control of finished
product during distribution. The validation does not address
user access controls or password protection, specifically
relating to the status of products in inventory. A summary
report, dated ***, in the validation material states the ***
software has, "been validated in order to see if it is
manageable and user friendly." 39
 WL/ULs charges cited
• Failure to thoroughly investigate any
unexplained discrepancy or failure of a
batch or any of its components to meet
any of its specifications, whether or not the
batch has already been distributed (21
CFR 211.192).
• Failure to follow and document at the time
of performance required laboratory control
mechanisms (21 CFR 211.160(a)). 40
 FDA 483 Example 3
Reference 21 CFR 211.192
There is a failure to thoroughly review any unexplained discrepancy for the failure of
a batch or any of its components to meet any of its specifications whether or not the
batch has been already distributed. Specifically,

The investigations for *** tablets *** mg resulting in OOS and partial batch rejections
were incomplete, for the following:

For Investigation PR ID #XXX, dated ***/14 OOS was observed for the thickness ***
tablets *** mg, Lot 123. The investigation did not include an assessment of the
manufacturing operations and the partially released finished lot 123 was not
placed on stability.

For Investigation PR ID #YYY, dated ***/14 OOS was observed for the thickness
***tablets *** mg, Lot 456. The investigation did not include an assessment of
the manufacturing operations and the partially released finished lot 456 was
not placed on stability.
41
 FDA 483 Example 4
Reference 21 CFR 211.160(a)
The establishment of test procedures including any changes thereto, are not reviewed and
approved by the quality control unit.

Specifically, the firm's Supplemental Method Information Sheet (SMIS) is used to document
information which is not present in a client specific procedure. A SMIS becomes part of the firm's
official procedure used to perform testing. The following SMIS reports were not reviewed or
approved by the firm's quality assurance group:

a. The SMIS Version Number 10-02 for method ***Bulk Release Testing; Method Number ATM
0105.000. This SMIS indicates that the sample should be added to the viscometer in small
portions distributed around the cone and plate, rather than in one single portion in the center. In
addition, the SMIS also indicates that the amount on sample cup is critical (try 0.12 mL using 3 cc
Syringe).

b. The SMIS Version Number 10-04 for method ***Pouch Testing; Method Number ATM
011.003.This SMIS indicates that the sample should be added to the viscometer in small portions
distributed around the cone and plate, rather than in one single portion in the center. In addition,
42
the SMIS also indicates that the amount on sample cup is critical (try 0.12 mL using 3 cc Syringe).
 WL/ULs charges cited
• Failure to follow written procedures for
production and process control designed
to assure that the drug products you
manufacture have the identity, strength,
quality, and purity they purport or are
represented to possess, and to document
same at the time of performance (21 CFR
211.100(b)).
43
 WL/ULs charges cited
• Your firm does not have, for each batch of
drug product, appropriate laboratory
determination of satisfactory conformance
to final specifications for the drug product,
including the identity and strength of each
active ingredient, prior to release (21 CFR
211.165(a)).

44
 FDA 483 Example 5
Reference 21 CFR 211.165(a)
Testing and release of drug product for distribution do not include appropriate laboratory
determination of satisfactory conformance to the final specifications prior to release.
Specifically,
a.) Your sampling plan allows for potency results to be reported as the average of three
samples tested for each batch. You have not established acceptance criteria for the results
of each individual test or for the standard deviation of the test results. Therefore, your current
practice allows for the release of sterile drug products despite individual potency test
results being sub-potent or super-potent compared to the potency release specification
of the finished drug product.

For example, potency sample #2 of ***, lot 567, 16.0 mg/mL was found to have an active
ingredient concentration of 17.0 mg/mL. Potency sample #3 of the same batch was tested and
found to have a concentration of 14.9 mg/mL. Your release specification for potency of is 15.2 -
16.8mg/mL. Within this batch you received potency test results that were both above and
below the release specification range. Lot 567 was approved for release and distributed.

b.) You have not conducted testing of the preservative content or determined the effectiveness
of the preservatives in your products. For examples, *** 100 mg/mL vials which contain 45
preservative ***.
 WL/UL for APIs
• Failure to ensure that laboratory records
included complete data derived from all
tests necessary to ensure compliance with
established specifications and standards.
• Failure to protect computerized data from
unauthorized access or changes.
• Failure to follow and document quality-
related activities at the time they are
performed. 46
 WL/UL for APIs
• Failure to maintain and make available for
inspectional review production and control
reworks for currently marketed APIs.
• Failure to perform laboratory testing of
APls to ensure conformance to
specifications and to accurately report
results on Certificates of Analysis (CoA).
• Failure to investigate and document out-
of-specification results. 47
 WL/UL for APIs
• Failure to maintain complete records for
APIs.
• Failure to include adequate documentation
during complaint investigation.

48
 Things to consider…
• Existing systems should be able to ensure
data integrity, traceability and reliability

• Firms who outsource operations should

have robust systems in place to verify and
compare data generated by the contractor

49
 Conclusion
• Once data integrity issues are found
during an inspection, a change to a written
procedure or firing an employee is not
enough
• Quality Risk Management approaches to
prevent, detect and control potential risk
are essential

50
FDA recommends that data integrity
problems identified during inspections
be addressed?

The firm should demonstrate effective

remediation. For example, By:
• Hire a third party auditor
• Determine the extent/scope of the problem
• Implement a global corrective action plan
• Removing individuals responsible for
problems from CGMP positions 51
 Acknowledgements

Karen Takahashi – CDER Senior Policy Advisor

52
Contact Information

CAPT Sharon K. Pederson (Thoma), PharmD

National Expert, Pharmaceutical Inspections

Food and Drug Administration (FDA)

Office of Regulatory Affairs (ORA)
Office of Regional Operations (ORO)
Office of Medical Products & Tobacco Operations (OMPTO)
Office Address: FDA, Minneapolis District
250 Marquette Avenue, Suite 600
Minneapolis, MN 55401
Office Telephone: 612-758-7159
Email Address: [email protected]
53
Thank you

54