Home About Us Advisory Board Advertise Editorial Submissions Contact Us


 
 
 
 


 Search for:

Digital Health MedTech Economics Manufacturing Execution Product Design & Development Quality/Regulatory Resource Centers  Events & Webinars Library

November 29, 2010 Upcoming Events & Webinars

Devine Guidance  April 23, 2024 – 10:00 am – 11:00 am

Statistical Techniques High Risk AI under the AI European Act: Must Know
Strategies for Medical Device Companies to Achieve
Regulatory Excellence
By Dr. Christopher Joseph Devine

This week’s Devine Guidance on 21 CFR, Part 820 – Subpart O (Statistical  April 24, 2024 – 10:00 am – 11:00 am
Mastering Compliance: Essential Insights into
Techniques) is the proverbial “Final Act” in regards to the QSR. There is a plethora
PMS Documentation
of data, standards, and websites that can provide useful information needed to
create robust procedure(s) for establishing effective statistical control and
sampling plans. Establishing robust statistical procedures will mitigate the  June 3, 2024 – 8:00 am – June 5, 2024 – 4:00 pm
potential receipt of a Form 483. MTI Regulatory Intelligence and Networking Summit

Share this:  Facebook  LinkedIn  Twitter

On Demand Events & Webinars

As the holiday season rapidly approaches, I nd myself sitting on an airplane pondering about the voluminous
 March 4, 2024
amount of prose, Dr. D has entered into his trusty laptop, in an effort to enlighten and guide readers through
Launch Medical Devices to Market in the EU vs the US
the often-ambiguous world of medical device regulations. In an adventure that began in early spring, the
doctor has been traversing through the Quality System Regulation (QSR), providing banausic (look-it up)
insights and salient points for compliance, while injecting humor and some sarcasm from the perspective of a
 December 5, 2023
long-time quality professional. Dr. D’s writing is motivated by the need to ensure all device manufacturers Post-EU MDR Compliance: Key Insights and Activities
enjoy the success that can be eventuated from sustained compliance to the QSR. to Maintain Compliance after Device Certi cation

That said, 21 CFR, Part 820 – Subpart O (Statistical Techniques) is the proverbial “Final Act” in regards to the
 November 30, 2023
QSR. This edition of DG will bring to a close a series of articles that I hope the readers have thoroughly enjoyed
Real-World Evidence: Not Just for PMCF
while gleaning some practical information.

Warning letter violations  September 28, 2023

Sometimes the doctor just feels like screaming – AAAARRRRGGGGHHHH! For this week’s edition of DG, the EU MDR & IVDR Lessons for Regulatory Strategy
three lucky recipients of FDA’s “You Suck” award (a.k.a. warning letter) each received an observation that
commences with the infamous and often used introduction of “Failure to Establish and Maintain Procedures.”
 September 26, 2023
Week after week, FDA routinely inspects the effectiveness of device manufacturers’ quality systems; and week Optimizing CER and PMS Document Writing with
Technology and AI
after week, the agency routinely issues Form 483s that often translate into warning letters. Dr D sincerely
hopes that the readers will learn from this edition and all of the previous editions of DG that procedures are
everything in the medical device industry. The expectation of FDA is that the procedures device
 September 21, 2023
manufacturers create encompass Wethe
are entire QSR and
using cookies not you
to give justthe
thebest
ones the device
experience manufacturers
on our website. believe they Crossing the eSTAR Chasm: Transitioning to FDA’s
Accept
You can
need to comply. People, I have been therendand
out want
more about which cookies
to reinforce we are
the belief: using or Letters
“Warning switch them off Fun!”
are no in settings. Mandatory 510k eSTAR Submission Process by
October 2023

For device manufacturer’s wishing to exacerbate their predicament with the agency, go ahead and provide a
response to an observation without suf cient evidence supporting the issues associated with an observation
 September 12, 2023
have been resolved. If there are doubts about what the agency is looking for in regards to corrective action
Bridging the GAP Agile/Compliance
and closure, pick up the phone and ask the agency for clari cation, just do not shoot from the hip.

Two of FDA’s “You Suck” Award recipients provided FDA with responses that were deemed insuf cient. Do  July 12, 2023
you know what happens to device manufacturers that provide poorly positioned responses to Form 483s? The Accelerating Corrective Actions for Rapid Introduction
Form 483s morph into warning letters. Do you know what happens to device manufacturers that provide of New Products: Strategies for Success

poorly positioned responses to warning letters or fail to respond to a warning letter within 15-days of receipt?
They enter FDA’s house of pain and take the rst giant leap toward Consent Decree!
 June 8, 2023
Not All Medical Products Are Created Equal.
Warning letter One (June 2010): Observation 8 of 14 – Failure to establish and maintain adequate procedures Harnessing the Power PLM!
to ensure that sampling methods are adequate for their intended use and to ensure that when changes occur
the sampling plans are reviewed as required by 21 CFR 820.250(b). For example, your rm’s procedure (b)(4)
Sampling of Finished Products, Revision (b)(4) describes the requirements for collecting samples of the  June 6, 2023
nished product. However, the procedure does not discuss sampling for additional testing to overcome a Mastering EU MDR: Essential Strategies for E ective
Risk and Document Management
failure of the nished product, other than to say that an investigation shall be conducted. As a result, the (b)(4)
which is integrity testing, was used to release lots of pre lled syringes when defects affecting the container
closure integrity were found after or during manufacturing. Sampling for (b)(4) consisted of (b)(4) regardless of
the size of the pre lled syringe lot. In some cases this size was as low as (b)(4) of the lot. Your rm released the
following lots of I.V.

Warning letter Two (May 2010): Observation 6 of 6 – Failure to establish and maintain procedures for
identifying valid statistical techniques required for establishing, controlling, and verifying the acceptability of
process capabilities and product characteristics, as required by 21 CFR 820.250(a). For example, there was no
recognized sampling plan methodology incorporating a valid statistical technique to verify the acceptability of
the process and take appropriate action when nonconforming components or products are identi ed that do
not meet the acceptable quality limit (AQL).

FDA Response to Observation 6 of 6 – We have reviewed your response dated December 17, 2009, and have
concluded that it is inadequate because it does not include documentation demonstrating that the sampling
plan methodology issues have been appropriately addressed.

Warning letter Three (March 2010): Observation 5 of 5 – Failure to establish and maintain procedures for
identifying valid statistical techniques required for establishing, controlling and verifying the acceptability of
process capability and product characteristics, as required by 21 CFR § 820.250(a). For example, your rm does
not have adequate statistical rationale to support the techniques used in trending of customer complaints per
your lm’s procedure SP-14122, “Complaint Trending and Escalation Process,” revision F, issued October 02,
2008

FDA Response to Observation 5 of 5 – Your rm’s response dated October 16, 2009, is not adequate because
your rm has not provided adequate statistical rationale for your lm’s Complaint Trending and Escalation
Process procedure. Please contact the Center for Devices and Radiological Health’s (CDRH) Of ce of
Compliance (OC) with further responses.

Quality System Regulation – 21 CFR, Part 820

QSR – Subpart O Section 820.250 Statistical Techniques

(a) Where appropriate, each manufacturer shall establish and maintain procedures foridentifying valid
statistical techniques required for establishing, controlling, and verifying the acceptability of process capability
and product characteristics.

(b) Sampling plans, when used, shall be written and based on a valid statistical rationale. Each manufacturer
shall establish and maintain procedures to ensure that sampling methods are adequate for their intended use
and to ensure that when changes occur the sampling plans are reviewed. These activities shall be
documented.

Statistical techniques
Before I dive into statistical techniques, Dr. D would like to refresh the readers with a de nition of statistics.
According to Merriam-Webster’s On-Line Dictionary, “statistics” is a branch of mathematics dealing with the
collection, analysis, interpretation, and presentation of masses of numerical data. Device manufacturers need
to understand the purpose and value of applied statistics in supporting decisions associated with initial design,
process development, and ongoing compliance to a published and approved product speci cation. When I
visit suppliers, and I ask about applied statistical methodologies, and get the answer, “we 100 percent inspect
all characteristics,” I must inform the suppliers this is not an acceptable statistical technique. In fact, it is not
even an effective approach to inspection. My belief is that process capability studies, with the results
interpreted in CpK and PpK, is the only true path to establishing effective statistical techniques and the
rationale required by the regulation.

When discussing the employment of statistical methodologies, Dr. D always like to point engineers in the
direction of Juran’s Quality Handbook, which some of us in the industry refer to as the Quality Engineer’s Bible.
Dr. D also recommends visiting Dr. Wayne Taylor’s website and reading his work on effective sampling plans.
Finally, with so many statistical tools available, such as Minitab™ (no – Dr. D is not a paid spokesperson for
Minitab) there should never be an excuse for device manufacturers not having a robust and documented
approach to statistical techniques..

Procedure(s)
It is now time for another Dr D broken-record time. As the doctor has opined on multiple occasions; DG Rule
# 6 is not an option; “All procedures, work instructions, drawings, speci cations, etc. must be written, well-
documented, and controlled within a de ned document control system.” Nestled into every single
requirement of the QSR is the phrase, “Shall establish and maintain procedures.” It seems pretty clear to Dr. D,
so I struggle to understand why device manufacturers continue to misinterpret the need for procedures.

In regards to statistical techniques, the agency is really looking for device manufacturers to collect data and
then interpret the results of the data through the application of statistics. For example, if Acme Device
Corporation ( ctional) has a critical requirement for device length, the agency expects Acme to prove that the
device meets this critical requirement on continuous basis. Can you say process capability studies? In this
example, Acme should be collecting dimensional data and using statistics, e.g., to ascertain if ongoing
processes are capable and remain in statistical process control – English translation “the device length is
within speci cation and we have the data to prove it – done!” If Acme validated the process as being Six Sigma
capable and the data is now trending below a PpK of 1.0, there now appears to be a problem.

How a device manufacturer identi es the problem, though the employment of statistics, and how the problem
is corrected is one of the fundamental foundations of this requirement. If device manufacturers are collecting
data just for the sake of having the data, well – what is the point?

In summarizing 820.250(a), device manufacturers shall have a procedure that delineates robust statistical
methodologies for driving process control; and supporting ongoing inspection activities in determining that
measured characteristics are acceptable and within their speci cation limits.

Sampling plans
As I stated earlier, Dr. D strongly suggest reading Dr. Taylor’s work on statistics and sampling plans as a whole.
Additionally, the American Society for Quality maintains two of the most recognizable standards for
determining appropriate sample sizes and creating effective sample plans, ANSI/ASQ Z1.4-2003 for the
inspection of attributes and ANSI/ASQ Z1.9-2008 for the inspection of variables. Adherence to these
mainstays of acceptable approaches to sampling methods will result in a solid foundation for device
manufacturers to support compliance to 820.250.

Over the years, I cannot count the times that Dr. D has been asked to evaluate the statistical signi cance when
the sample size employed was a N=3 or an N=5. Do you have any idea how dif cult it is to defend that type of
sample rationale to any regulatory body, especially FDA? It is the doctor’s opinion; there is no way to defend
the approach, if it is documented by procedure. The approach is irresponsible, reprehensible, lackadaisical,
nonsensical, impractical, laughable, and indefensible, when sitting across from the agency during one of their
friendly visits.

To summarize, the doctor strongly suggests that the documented approach to sampling be premised on
recognized standards. Not only does the sampling methodology need to be delineated with the procedure, so
does the ability to adjust sampling plans premised on the results. The doctor strongly suggests that sampling
plans are routinely reviewed and the results of the review documented. Additionally, the need to review
sampling plans needs to be depicted in the actual procedure.

Sampling plans need to be linked back to risk and risk indices. When determining an appropriate level of
sampling for a component or device, it is imperative that the PFMEA and DFMEA be evaluated, because the
failure mode effects analysis will lead you to applying the appropriate sample size premised on the actual risk
of failure. Finally, ensure your supplier base is capable of understanding and employing industry recognized
statistical concepts. Suppliers that routinely employ effective statistical approaches to process control will be
in position to deliver product that meet speci cation.

Takeaways
There is a plethora of data, standards, and websites that can provide useful information needed to create
robust procedure(s) for establishing effective statistical control. Additionally, this same information is available
for establishing effective sampling plans. As captured in the warning letter extractions, FDA will evaluate a
device manufacturer’s approach to statistics and sampling during one of their friendly visits. Establishing
robust procedures, in advance, will mitigate the potential receipt of a Form 483.

In closing, thank you again for joining Dr. D and I hope you nd value in the guidance provided. Until the next
installment of DG, when Dr. D will provide some guidance for managing a visit from the FDA – cheers from
Dr. D. and best wishes for continued professional success.

References:

1. Aczel, D., & Sounderpandian, J. (2006). Complete business statistics (6th ed.). Boston: McGraw-Hill Irwin.
2. Code of Federal Regulation. (2010, April). Title 21 Part 820: Quality system regulation. Washington, D.C.: U.
S. Government Printing Of ce.
3. Devine. C. (2009, July). Exploring the effectiveness of defensive-receiving inspection for medical device
manufacturers: a mixed method study. Published doctoral dissertation Northcentral University. Prescott
Valley, AZ.
4. FDA – U.S. Food and Drug Administration Website. (2010). Warning letters. Retrieved November 12, 2010,
from http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/
5. Juran, J. & Godfrey, A. (1998). Juran’s quality handbook (5th ed.). New York, New York: McGraw-Hill.
6. Merriam-Webster’s On-Line Dictionary. (2010). Statistics de nition. Retrieved November 25, 2010, from
http://www.merriam-webster.com/dictionary/statistics
7. Sampling procedures and tables for inspection by attributes. (2003). American Society for Quality
ANSI/ASQ Z1.4-2003. Milwaukee, WI.
8. Sampling procedures and tables for inspection by variables. (2008). American Society for Quality
ANSI/ASQ Z1.9-2008. Milwaukee, WI.
9. Taylor, W. (1993, November). Classifying defects and selecting AQLs. FDC Control, Food Drug & Cosmetic
Division ASQC, 103. Retrieved March 23, 2007, from http://www.variation.com/techlib/as-1.html
10. Taylor, W. (1996). Selecting statistically valid sampling plans. Quality Engineering 10(2). Retrieved March 5,
2007, from http://www.variation.com/techlib/as-7.html

Share this:  Facebook  LinkedIn  Twitter

Related Articles

Ethical Considerations for FDA Updates Breakthrough

Children in Clinical Devices Guidance Focused on

The guidance seeks to balance the need to obtain The draft guidance proposes updates to clarify
data on the safety and effectiveness of drugs, how the Breakthrough Devices Program may be
biological products and medical devices in the applicable to certain medical devices that

FDA Updates Guidance on FDA Issues Draft Guidance on

De ciencies in Accordance Computer Software

The updated guidance document clari es what The new guidance is intended to establish
constitutes a statement of the basis for the con dence in automation used for production or
de ciency and includes examples of well- quality assurance systems and describe various

About The Author

Dr. Christopher Joseph Devine
President
Devine Guidance International

Dr. Christopher Joseph Devine is the President of Devine Guidance International,

a consulting rm specializing in providing solutions for regulatory compliance,
quality, supplier management, and supply-chain issues facing the device industry.
Dr. Devine has 32 years of experience in quality assurance, regulatory compliance
and program management. He is a senior member of the American Society for
Quality (ASQ), a member of the Regulatory Affairs Professionals Society (RAPS),
and a member of the Project Management Institute, and resides on several
technical advisory boards. Dr. Devine received his doctorate from Northcentral
University, with his doctoral dissertation titled, “Exploring the Effectiveness of
Defensive-Receiving Inspection for Medical Device Manufacturers: A Mixed-
Method Study.” Dr. Devine holds a graduate degree in organizational management
(MAOM) and an undergraduate degree in business management (BSBM).

Topic Archives Contact Us Innovative Publishing Company, Inc.

Digital Health Advertising Opportunities FoodSafetyTech

MedTech Economics Editorial Submissions Food Safety Consortium Conference & Expo
Manufacturing Execution General Inquiries
Product Design & Development Privacy Policy MedTech Intelligence
Quality/Regulatory Cookie Policy
Sign Up for Newsletters Cannabis Industry Journal

© Copyright 2015 - 2024 Innovative Publishing Co., Inc., All Rights Reserved

Other Innovative Publishing Co., Inc. Sites: Food Safety Tech | Cannabis Industry Journal