Quality Assurance

Good Practice, Regulation, and Law

ISSN: 1052-9411 (Print) 1521-0677 (Online) Journal homepage: https://www.tandfonline.com/loi/uqua20

PROMOTING GMP IMPLEMENTATION: DEVELOPING

TRAINING MATERIALS FOR THE INTERNATIONAL
AUDIENCE

Kazushige Morimoto , Judith Curry , Sabine Kopp , Lembit Rägo , Andre van
Zyl , Eshetu Wondemagegnehu , Jonathan Quick & Yasuhiro Suzuki

To cite this article: Kazushige Morimoto , Judith Curry , Sabine Kopp , Lembit Rägo , Andre
van Zyl , Eshetu Wondemagegnehu , Jonathan Quick & Yasuhiro Suzuki (2003) PROMOTING
GMP IMPLEMENTATION: DEVELOPING TRAINING MATERIALS FOR THE INTERNATIONAL
AUDIENCE, Quality Assurance, 10:1, 11-27, DOI: 10.1080/10529410390198864

To link to this article: https://doi.org/10.1080/10529410390198864

Published online: 02 Feb 2011.

Submit your article to this journal

Article views: 1

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=uqua20
Quality Assurance, 10:11–27, 2003
Copyright # 2003 Taylor & Francis
1052-9411/03 $12.00 + .00
DOI: 10.1080/10529410390198864

PROMOTING GMP IMPLEMENTATION:

DEVELOPING TRAINING MATERIALS FOR THE
INTERNATIONAL AUDIENCE

Kazushige Morimoto, Judith Curry, Sabine Kopp,

Lembit R€
ago, Andre van Zyl, Eshetu Wondemagegnehu,
Jonathan Quick, and Yasuhiro Suzuki
Strengthening of Pharmaceutical Manufacturing Inspection (SPMI) Project,
World Health Organization, Health Technology and Pharmaceuticals,
Department of Essential Drugs and Medicines Policy, Quality Assurance and Safety: Medicines,
Geneva, Switzerland

This paper outlines the development of a CD-ROM training package entitled:

The WHO Basic Training Modules on GMP, intended to support the creation
of training courses aimed particularly at government compliance officials
who inspect pharmaceutical manufacturing facilities. The material was cre-
ated over a three-year period in collaboration with a team of external experts,
WHO regional and local offices, and Drug Regulatory Authorities of participat-
ing countries. The nine training workshops and courses that contributed to the
development and evaluation processes were attended by approximately 240
participants from 47 countries. To date over 5,800 copies of the CD-ROM have
been distributed.

Received 5 November 2002; accepted 9 January 2003.

The financial support of the Government of Japan to the GMP Implementation and SPMI
Projects is gratefully acknowledged. WHO also gratefully acknowledges the active participation
and constructive comments received from the previous members of the Working Group of the Pro-
ject for the Promotion of Implementation of GMP: Dr. K. Kimura; Ms. K. Bremer; Dr. J. Id€ anp€
aa€n-
Heikkil€ a; External commentators and contributors: Dr. Tor Boye (Switzerland); Mr. David Buckley
(Australia); Dr. J
ozsef Lipt

ak (Hungary); Mr. Michael How (United Kingdom); Mr. Alain Kupfer-
man (France); Dr. Kustantinah (Indonesia); Mr. Rudy Mantik (Indonesia); Ms. Kate McCormick
(United Kingdom); Professor Janos Pog
any (Hungary); Ms. Sheila Poole (United Kingdom); Dr.
Christian Robert (Switzerland); Mr. John Startup (United Kingdom); Mr. Masahiro Suzuki
(Japan); WHO staff: Mr. L. Belgharbi, Mr. A. Crump, Ms. C. Klekr; Ms. M. Martin, Ms. C. Mullen,
Ms. M. Renevier, Dr. M. Scholtz; Member States providing special project support: Australia,
Bosnia and Herzegovina, China, Egypt, India, Jamaica, Japan, Myanmar, Nepal, South Africa,
Switzerland, United States of America, Vietnam, Zimbabwe; and all those who completed the
questionnaire.
The views expressed in this paper are those of the authors, not WHO.
Address correspondence to Dr. Kazushige Morimoto, SPMI Project, WHO=HTP=EDM=QSM,
20 Avenue Appia, CH-1211 Geneva 27, Switzerland. E-mail: [email protected]

11
12 K. Morimoto et al.

INTRODUCTION
The WHO Good Manufacturing Practices (GMP) Texts
The quality of pharmaceuticals has been a concern of the World Health
Organization (WHO) since its inception (WHO, 1997), and the re-
quirement for WHO to set global standards is embodied within the
Organization’s Constitution. One of the major objectives of the WHO
Medicines Strategy, therefore, is to address the issue of quality and
safety of medicines.
Essential drugs and medicines are one of the vital tools needed to
improve and maintain health. While global standards for drug quality
are becoming increasingly rigorous, the quality of drugs on the market
in a number of developing countries still remains a major public health
concern. In recent years, there have been many alarming reports
highlighting the problems of poor quality drugs, such as those con-
taminated by toxic substances (O’Brien et al., 1998) and counterfeit and
substandard drugs (Reidenberg & Conner, 2001; Newton, White,
Rozendaal, & Green, 2002), often involving life-saving drugs, such as
antimalarial (Taylor et al., 2001) and anti-tuberculosis (Laserson,
Kenyon, Kenyon, Layloff, & Binkin, 2001) drugs. In many cases, there
may not have been adequate regulation enforcement and strict im-
plementation of current Good Manufacturing Practices (GMP), which
are an essential part of a comprehensive quality assurance system, and
of paramount importance in insuring that pharmaceutical products are
consistently produced and controlled according to quality standards.
Closely related to the overall drug quality assurance system is the
issue of inspection of manufacturers. Without a competent inspectorate
operating to high professional standards, neither GMP compliance nor
licensing provisions can effectively be enforced. In addition, inspection
of manufacturing facilities is pivotal to the operation of the WHO
Certification Scheme on the Quality of Pharmaceutical Products Mov-
ing in International Commerce.1
Much has been achieved in the 50 years since WHO began estab-
lishing international pharmaceutical standards and guidelines. Its
current texts: GMP for pharmaceutical products and Provisional
guidelines on the inspection of pharmaceutical manufacturers have
been published under the title: Quality assurance of pharmaceuticals:
A compendium of guidelines and related materials, Volume 2 (WHO,
1999). The guidelines, advisory in nature, may require some adaptation

1
The WHO Certification Scheme on the quality of pharmaceutical products moving in inter-
national commerce provides for the issuance of an attestation certifying that a given product is
being manufactured under GMP conditions, as established by periodic inspections.
 Promoting GMP Implementation by Training 13

to address specific conditions in individual countries. Supplementary

texts are currently being added (WHO, 2002), and some parts have been
recently revised.2
THE WHO=EDM PROJECTS
In 1998 WHO Department of Essential Drugs and Medicines (EDM)
Policy received assistance from the Government of Japan to fund the
three-year Project on the Promotion of the Implementation of GMP. The
aims of the project were to identify the obstacles to implementing GMP,
which in turn hinder the production of locally manufactured pharma-
ceutical products of assured quality, and to develop tools and methods
to help countries improve their GMP implementation.
At the outset of the Project, a WHO Working Group was set up to
plan, coordinate, and evaluate project activities and to oversee the
development of training modules, eventually to be produced as a CD-
ROM to support the creation of training courses on GMP and Inspec-
tion (WHO, 2001). The material would be oriented towards government
compliance officials who inspect manufacturing facilities that produce
medicines or pharmaceutical starting materials. The modules would be
complemented by a GMP implementation video.
Four countries were initially selected to participate in the activities
of the project: China, Myanmar, Nepal, and Zimbabwe. The criteria for
selection were that the countries should have operational collaboration
with WHO; should be in the category of a developing country; should
have operative drug regulatory authorities; should be in a position to be
able to serve as possible regional centres for GMP training in the fu-
ture; and should have significant levels of pharmaceutical production.
When the project concluded in December 2000, it was followed in
January 2001 by the Strengthening of Pharmaceutical Manufacturing
Inspection (SPMI) Project. This project aimed to consolidate the
achievements of its predecessor and focus on strengthening pharma-
ceutical manufacturing inspectorates by promoting the global use of
the completed training materials. The SPMI Project would conclude in
March 2003.

DEVELOPMENT PROCESS
The GMP Basic Training Modules
Early Preparatory Work
The initial draft texts were prepared during 1998 with input from a
WHO working group in collaboration with a multi-disciplinary team of
2
These texts are now available through the WHO web site: www.who.int=medicines.
14 K. Morimoto et al.

TABLE 1 Content of the WHO Basic Training Modules on GMP

Module no. Title Duration (hr)

Basic principles of GMP 1 Introduction 1–2

2 Quality management 4
3 Sanitation and hygiene 3
4 Validation 3
5 Complaints and recalls 4
6 Contract production and analysis 3
7 Self-inspection 3
8 Personnel 5
9 Premises 4
10 Equipment 5
11 Materials 3
12 Documentation 5
13 Sterile production 7
14 Active pharmaceutical ingredients 3
The GMP inspection process 15 Introduction 1
16 The role of the inspector 2
17 Preparation for the inspection 2
18 Types of GMP inspection 3
19 The inspection 2
20 Trainer’s notes

external experts. The material comprised 20 modules, with slide pre-

sentations, tutorial notes, group session exercises, handouts, and (for
modules 2–14) comprehension test and answer sheets. The compre-
hension test papers took the form of multiple choice questions. Modules
1–14 covered Basic Principles of GMP, modules 15–19 related to The
GMP Inspection Process, with the final module, module 20, providing
trainers’ notes for the setting up and running of the course (Table 1).
Second and third drafts followed during 1998–1999, and a fourth and
final draft was completed in November 1999 for testing at the first Pilot
Training Workshop to be held the following month.
First and Second WHO Pilot Training Workshops
The first WHO pilot training workshop was held in Beijing in De-
cember 1999, organized in collaboration with the State Drug Admin-
istration (SDA), Beijing, People’s Republic of China, and attended by
English-speaking government inspectors and administrative officers
with a good knowledge of GMP inspection, nominated from eight
countries from the WHO Western Pacific Region (WPRO) and WHO
South-East Asia Region (SEARO) (Table 2). During the two-week pro-
gramme, two trainers presented modules 1–19. Group sessions and
comprehension tests followed each of the modules 2–14. Following the
workshop, modifications to the module texts were made, based on
trainer and trainee feedback.
 Promoting GMP Implementation by Training 15

TABLE 2 Training Workshops Held to Test WHO Basic Training Modules on GMP

Number of
Number of countries
Title of workshop Dates Location participants represented

1st WHO Pilot Training 16–30 Nov 1999 Beijing, China 17 8a

Workshop on GMP
2nd WHO Pilot Training 3–14 July 2000 Pretoria, South Africa 20 9b
Workshop on GMP
1-day training course on 19 Oct 2000 Yangon, Myanmar 37 1
the Promotion of
Implementation of GMP
1-day training course on 21 Dec 2000 Kathmandu, Nepal 31 1
the Promotion of
Implementation of GMP
a
China, Lao People’s Democratic Republic, Indonesia, Malaysia, Nepal, Philippines, Thailand,
Myanmar.
b
Ethiopia, Ghana, Malawi, Nigeria, South Africa, Tanzania, Uganda, Zambia, Zimbabwe.

The resulting first revised draft was tested at a second WHO

Pilot Training Workshop held in Pretoria in July 2000, along with
the first viewing of the GMP implementation video. Participants
comprised mainly government pharmaceutical manufacturing in-
spectors and administrative officers, again with a good knowledge of
GMP inspection, and nominated from English-speaking countries
within the WHO African Region (AFRO). During the workshop two
of the modules (11 and 18), chosen for the simplicity and familiarity
of content, were presented by participants, to help ascertain the
user-friendliness of the materials. Subsequently module texts were
again modified in the light of participant feedback to produce final
versions.

One-Day Training Courses in Myanmar and Nepal, October and

December 2000
Two one-day courses were held at the end of 2000 in Myanmar and
Nepal (Table 2), using finalized versions of the Validation and Premises
modules, and the GMP implementation video. The participants,
nominated by their respective country, came from a broader range of
backgrounds, that is, from government institutions, national and pri-
vate pharmaceutical factories and (in the case of Myanmar) uni-
versities. Comprehension tests were undertaken on the Validation
module. Following feedback from these workshops, some minor mod-
ifications were made to the material prior to the preparation of the CD-
ROM.
16 K. Morimoto et al.

The GMP Implementation Video

The 18-minute training video, developed during the first half of 2000,
was intended to be included in the GMP modular package, as well as
being able to serve as a stand-alone training tool. Film sequences and
narrative illustrated examples of GMP-compliant and non-compliant
situations in pharmaceutical manufacturing facilities, and included
scenes of manufacturing processes, premises, and quality control pro-
cedures.
For the GMP-compliant footage, a facility in a developing country
where a full range of pharmaceutical products was being produced was
selected, the chosen location being Viet Nam. The illustrations of
non-GMP compliance were taken from black and white film footage
originally prepared under the direction of the Food and Drug Admin-
istration (FDA), United States of America (USA) during the 1970s,
entitled: No Margin for Error. The persons and company featured in
the scenes were fictional; however, the situations shown were based on
true occurrences, and demonstrated the consequences of human error
for product quality and the end-user’s well-being. ‘‘Compliant’’ ex-
amples contained some ‘‘non-compliant’’ aspects of pharmaceutical
manufacturing practices, for identification during group viewing. The
video was completed in June 2000.

The GMP CD-ROM

Work on the production of the CD-ROM began in March 2000 and was
completed in February 2001. The contents include all the modular
training texts, the WHO text: Quality assurance of pharmaceuticals,
Vol. 2: Good manufacturing practices and inspection (WHO, 1999), plus
facilities for viewing the GMP implementation video, background pro-
ject information, and other supporting material. Also included are
utilities for viewing and printing, and a software tool to help trainers
modify the course to fit local conditions and regulations.
Multi-copies (7,700) were made available for distribution in April
2001. Initial distributions included the global list of drug regulatory
authorities, WHO country and regional offices, and participants of the
first and second Pilot Training Workshops.

EVALUATION PROCESS
WHO=HQ Evaluation Meeting
At the conclusion of the three-year Project on the Promotion of the
Implementation of GMP, an Evaluation Meeting was held in Geneva in
 Promoting GMP Implementation by Training 17

January 2001 to review the project’s activities and outcomes, attended

by WHO staff, external GMP experts, and a consultant trainer. Parti-
cipants reviewed a selection of the training modules and the GMP
implementation video and discussed strategies for achieving the aims
of the new SPMI project.

GMP Training Workshops in Five Regions

Following completion of the modules, five training workshops were held
during 2001, involving WHO regional and local offices, utilizing the
modules and GMP implementation video (Table 3). A common objective
of the workshops was to verify the effectiveness of the training package
through the analysis of information supplied by participants on course
evaluation feedback forms and through the assessment of comprehen-
sion test results. A total of 134 participants drawn from 31 coun-
tries within the WHO regions undertook the training. At two of the
workshops (Chong Qing and New Delhi), participation included

TABLE 3 Subsequent Training Workshops Using WHO Basic Training Modules on GMP

Number of
Number of countries
Dates Location participants represented
a
WHO=WPRO Training 15–16 March 2001 Chong Qing, China 51 1
workshop on GMP
WHO=AMROb Training 17–27 April 2001 Kingston, Jamaica 18 12g
workshop on GMP
WHO=SEAROc Training 26–29 June 2001 New Delhi, India 28 1
workshop on GMP
WHO=EUROd Training 22–27 Oct 2001 Sarajevo, Bosnia 18 7h
workshop on GMP and and Herzegovina
Inspection
WHO=EMROe Training 10–14 Nov 2001 Cairo, Egypt 19 10i
workshop on GMP and
Inspection (in
collaboration
with NODCAR)f
a
Regional Office for the Western Pacific.
b
Regional Office for the Americas=Pan American Health Organization.
c
Regional Office for South-East Asia.
d
Regional Office for Europe.
e
Regional Office for the Eastern Mediterranean.
f
National Organization for Drug Control and Research.
g
Barbados, Belize, Brazil, Columbia, Costa Rica, Dominica, Grenada, Guyana, Jamaica, St. Lucia,
St. Vincent and the Grenadines, Trinidad and Tobago.
h
Bosnia and Herzegovina, Bulgaria, Macedonia, Romania, Slovenia, Turkey, Yugoslavia.
i
Egypt, Jordan, Lebanon, Libyan Arab Jamahiriya, Oman, Pakistan, Saudi Arabia, Sudan,
Syrian Arab Republic, Yemen.
18 K. Morimoto et al.

representation from inspectors and manufacturers of anti-tuberculosis

products.
At each venue a selection of the modules was presented by GMP
experts. In some cases trainees were invited to participate in the pre-
sentation of some of the modules (modules 3, 11, 17, and 18) (Table 1).
Presentations were followed by group sessions and comprehension
tests. Normally the time required to present all the modules is two
weeks. However, due to constraints of time and budget, the workshops
varied in length from one day to two weeks. Those of longer duration
included more modules and group sessions. Where possible, a visit to a
pharmaceutical factory site or laboratory was included in the pro-
gramme (Kingston and Cairo), to enable participants to be exposed to
practical aspects of GMP training. To facilitate greater understanding
at the Chong Qing workshop, translation into Chinese of the slides and
tests was provided by SDA. Following availability of the CD-ROM in
April 2001, copies with questionnaires were distributed to participants
at the end of the training programmes.

CD-ROM Questionnaire
The initial distribution of the CD-ROM in April 2001 was followed by a
questionnaire in June 2001 dispatched to national drug regulatory
authorities, participants of the first and second pilot training work-
shops (Table 2), and to participants of the recently held Jamaica
workshop (Table 3). Later questionnaires were distributed to partici-
pants at the close of the New Delhi, Cairo, and Sarajevo workshops.
The questionnaire consisted of four parts: three open-ended and one
multiple choice section. It aimed to elicit feedback on the five compo-
nents of the training package—slides, tutorial notes, group sessions,
handouts, and the GMP implementation video—in terms of technical
content and the user’s opinion on the usefulness of the materials for
developing skills in GMP inspection and compliance. A total of 702
questionnaires were distributed between June and November 2001.

RESULTS
First and Second WHO Pilot Training Workshops
The Beijing and Pretoria pilot training workshops played an important
role in the development process of the Basic Training Modules. It was
noted during both the workshops that although the modules had been
intended to be oriented towards the training of government inspectors,
in some cases, trainees perceived them to be more suited to the training
 Promoting GMP Implementation by Training 19

of industrial personnel. In addition, they were considered by some to be

of too basic a level for government inspectors, particularly for people
who had already attended other GMP courses. This probably reflected
the fact that there were considerable variations in GMP knowledge at
the participating country level.
Other observations from trainers and trainees included the need for
better correspondence between slide and tutorial notes, and for mod-
ifications to ensure that slides and notes accurately reflected basic
GMP principles, according to the official WHO GMP text.
Further suggestions related to modifications to presentation style,
the need for additional handouts, such as an example of standard op-
erating procedures (SOPs) and a self-inspection schedule and checklist
for the manufacture of sterile medical products. In addition it was
suggested that longer times be allowed on some modules for presenta-
tions and group exercises. Feedback also indicated that some of the
questions and answers in the comprehension tests may not have been
understood by trainees, or contained ambiguities. As a result, revisions
to improve clarity were recommended.
At the first Pilot Training Workshop in Beijing, although most
trainees demonstrated a good understanding of basic GMP, there was
less knowledge shown on certain advanced principles relating to the topics
of Sterile production, and Validation. However, scoring was generally
high on all comprehension tests on modules 2–14, the highest marks
being achieved on Equipment and Complaints and recalls (Table 4).
Comprehension test results from the second Pilot Training Work-
shop in Pretoria on modules 2–14 again indicated a good understanding
of the topics by the participants. Scoring was generally higher than
that of the first Pilot Training Workshop including the scores for the
modules covering Validation, Sterile Production, Equipment, and
Documentation (Table 4). On the topics of Sanitation and Hygiene and
Self-Inspection scoring was lower than that of the first Workshop. Some
ambiguities were still found to exist in a number of questions and
answers across the modules that required modification.

One-Day Training Courses in Myanmar and Nepal

Comprehension test results on Validation indicated a fair to moderate
level of understanding of this topic (Table 5). In both the courses, scores
were generally lower than those attained on this topic in the first and
second Pilot workshops, which may have been due to the fact that
participants came from a broader range of backgrounds. There were no
difficulties experienced with ambiguities in the question and answer
section; however, feedback from these courses resulted in some final
20 K. Morimoto et al.

TABLE 4 Results of Comprehension Test Papers—Pilot Training Workshops and Jamaica

Workshop

1st Pilot workshop 2nd Pilot workshop GMP workshop

Beijing Pretoria Jamaica
(2 weeks) (2 weeks) (10 days)

M  SD M  SD M  SD
Module number and title Range Range Range

2. Quality management 80.3  13.1 (16) 86.0  7.1 (20) 84.2  7.3 (18)
55.0–95.0 72.5–100 70.0–95.0
3. Sanitation and hygiene 85.6  7.6 (17) 74.7  13.8 (20) Group testa
77.8–100 44.4–94.4
4. Validation 71.1  17.5 (17) 76.5  14.7 (19) 74.7  12.6 (17)
42.0–100 54.2–100 44.7–91.7
5. Complaints and recalls 90.3  10.4 (16) 88.1  8.6 (20) 88.2  6.7 (18)
65.0–100 67.5–97.5 75.0–100
6. Contract production 85.4  11.5 (15) 87.9  5.7 (18) 86.5  8.1 (18)
and analysis 60.0–93.0 76.7–98.3 71.4–100
7. Self-inspection 89.1  12.0 (16) 74.3  10.2 (20) 83.6  10.0 (18)
63.0–100 55.0–95.0 70.0–100
8. Personnel 85.9  8.5 (17) 92.3  6.6 (19) 84.8  7.7 (17)
60.0–95.0 75.0–100 72.2–97.2
9. Premises 85.6  9.8 (17) 88.6  6.6 (20) Group test
55.0–95.0 80.0–100
10. Equipment 93.5  12.7 (17) 99.4  2.4 (18) Homeworkb
50.0–100 90.0–100
11. Materials 88.2  11.7 (17) 96.0  5.0 (20) Homework
60.0–100 86.7–100
12. Documentation 77.9  12.4 (17) 83.1  6.2 (20) Group test
55.0–95.0 71.3–92.5
13. Sterile production 72.9  9.9 (17) 83.4  5.5 (18) 84.2  7.0 (15)
55.0–90.0 73.3–93.3 67.9–92.9
14. Active pharmaceutical 88.2  8.1 (17) 93.9  7.2 (18) 95.3  5.1 (17)
ingredients 70.0–100 80.0–100 90.0–100
a
Trainees were divided into two or more groups and collaborated on the test answers.
b
Test paper distributed to trainees to undertake in their own time.

modifications to the trainer’s notes and to some of the photographic

content of the Premises module, to improve the clarity of the material.
The CD-ROM training materials were eagerly awaited by participants.

WHO=HQ Evaluation Meeting

An observation made in the case of the video was that since some of
the GMP ‘‘compliant’’ examples contained ‘‘non-compliant’’ aspects of
pharmaceutical manufacturing practices intended for identification
by trainees following a group viewing, it would be helpful to both
 TABLE 5 Results of Comprehension Test Papers—1 Day Courses and Training Workshops

GMP course GMP course GMP workshop GMP workshop GMP workshop GMP workshop
Yangon Kathmandu Chongqing New Delhi Sarajevo Cairo
(1 day) (1 day) (2 days) (4 days) (6 days) (5 days)
Module number
and title M
SD Range M
SD Range M
SD Range M
SD Range M
SD Range M
SD Range

2. Quality — — 83.3
8.5 (51) 86.1
6.8 (29) 89.7
7.6 (18) —
management 65.0–100 70.0–95.0 77.5–100
3. Sanitation — — — — 83.5
10.2 (17) —
and hygiene 66.7–100
4. Validation 56.5
11.7 (37) 65.2
14.0 (28) — 87.4
9.6 (25) 83.5
10.2 (17) 64.1
13.6 (18)
21.0–75.0 37.5–87.5 66.7–100 66.7–100 41.7–87.5
5. Complaints — — — — 88.6
7.6 (18) 89.1
6.2 (19)
and recalls 75.0–97.5 80.0–100
8. Personnel — — — 92.4
4.9 (28) — 89.6
7.3 (19)
77.7–100 75.0–100
9. Premises test not given test not given test not given 87.3
5.9 (28) 87.6
8.9 (18) 87.0
7.5 (19)
70.0–95 74.0–100 72.5–96
11. Materials — — — — 96.5
9.8 (17) —
60.0–100
12. Documentation — — — — 81.6
11.0 (18) 81.4
8.4 (19)
66.0–98.0 66.0–95.0

21
22 K. Morimoto et al.

trainer and trainee to include an additional slide in the training

package giving this information. This was subsequently added to the
material.
It was recommended that the CD-ROM training material be
globally distributed as soon as possible, as well as being made available
on the WHO Web site. A further recommendation was made that under
the remit of the new SPMI Project more advanced modules be devel-
oped on such topics as Validation, Quality of Water and Heating, Ven-
tilation, and Air Conditioning Systems (HVAC).
The general consensus was that both the modules and video had
been developed to a professional standard.

GMP Training Workshops in Five Regions

Training materials were well received at the five workshops held dur-
ing 2001 in Chong Qing, Kingston, New Delhi, Sarajevo, and Cairo
(Table 3). At four of the workshops (Kingston, New Delhi, Cairo, and
Sarajevo), several trainees successfully presented all or part of various
modules, and generally found them to be clear, comprehensive, and
easy to use.
Results of comprehension tests indicated the questions and answers
were clear and one could assume a good level of understanding of the
materials (Tables 4 and 5), despite the limited experience of some
trainees in the area of inspection (Kingston) and some language diffi-
culties (Chong Qing, Sarajevo, and Cairo). Results of comprehension
tests undertaken in groups and as homework (Kingston) were not in-
cluded in the data. The group comprehension tests proved very suc-
cessful in allowing for strong argument and debate amongst the
participants, especially with regard to the appropriate answers selected
(Table 4, Jamaica). Following the site visits (Kingston and Cairo),
participants were able to identify, in follow-up group discussions, a
number of areas where GMP improvements could be made. Pro-
grammes were modified at Sarajevo and Cairo to incorporate a half-day
question and answer session, in order to clarify the many questions
that participants were prompted to raise during the workshop on such
subjects as validation, water purification, ventilation, penicillin-con-
taining product production, environmental monitoring, and inspection
reports.
The GMP Video was also well received at all the venues. However
feedback from the Chong Qing participants suggested that its value
was limited without accurate translation into Chinese languages.
 Promoting GMP Implementation by Training 23

CD-ROM Questionnaire
Of the 702 questionnaires sent out 47 (6.5%) responses were returned
(Table 6). The majority of the responses came from the National Drug
Regulatory Authorities distribution. The results did not account for
negative comment that some users may have felt about the material,
but which they chose not to feed back to us.
All responders had used the material either wholly or in part, or
had plans to incorporate it into forthcoming training programmes.
Replies suggested that the Quality Management, Validation, and Doc-
umentation modules had been found to be of particular use to users.
Feedback indicated group sessions had been found to be beneficial,
though several responders indicated that more time than was allocated
should be given to these activities. The latter, however, is intended to be
adjusted by the trainer according to the composition of the group. A
third of the responders indicated plans for translating the CD-ROM
into local languages. Responses to the multiple choice section (Part 2) of
the questionnaire are given in Table 7.
In the general comment sections, some observations received on the
slides indicated improvements could be made to the clarity of the
material. Comment on the trainer’s notes and comprehension tests was
generally very favourable. However, several responders mentioned the
need for more examples to be included in the trainer’s notes, and that
difficulty had been experienced in some of the comprehension tests,
with issues of clarity of the questions and time allocated to complete
the tests cited as particular problems. Feedback on the handout ma-
terials suggested general satisfaction with this part of the package. A
suggestion was made that such materials could be handed out to
trainees well in advance of the course to assist trainees to prepare for
courses.
The GMP video was considered to be a helpful and important part of
the package, though several considered the focus of the scenarios to be
limited and that more footage covering additional GMP topics should
have been included, such as water and air systems, validation, and
sterile production. Several responders made the comment that the
‘‘non-compliant’’ sections using the black and white film would be en-
hanced by using more recent, colour film footage. On the issue of lan-
guage, the suggestion was made that subtitles in English would assist
understanding for those viewers who were not of English mother-ton-
gue. A responder from China indicated that a similar video in Chinese
was being planned.
24 K. Morimoto et al.

TABLE 6 CD-ROM Questionnaire Responses

Country Organization

AFRO
Ethiopia Drug Administration and Control Authority
Ghana (2)a Food and Drugs Board
Mozambique Pharmaceutical Department, Ministry of Health
Tanzania Pharmacy Board, Ministry of Health
Uganda National Drug Authority, Ministry of Health
Zimbabwe (3)a Medicines Control Authority and two private companies
AMRO
Barbados Barbados Drug Service, Ministry of Health
Belize Ministry of Health
Brazil (2)a Drug Inspection Management, Ministry of Health, and
Faculdade de Ciencias Farmaceuticas, Uni. de Sao Paulo
Colombia (2)a Uni Nacional de Colombia and Technimicro Laboratino,
Uni Antioquia
Costa Rica (2)a Facultad de Farmacia, Uni de Costa Rico
Jamaica Pharmaceutical and Regulatory Affairs Department,
Ministry of Health
Suriname Drug Regulatory Authority, Ministry of Health
Venezuela Instituto Nacional de Higiene R. Rangel, Uni Central
EMRO
Libya The National Pharmaceutical and Medical
Equipment Company
Oman Directorate General of Pharmaceutical Affairs and
Drug Control, Ministry of Health
Syria Directorate of Drug Quality Control, Ministry of Health
Yemen Supreme Board of Drugs and Medical Appliances,
Ministry of Health
EURO
Austria Bundesministerium für soziale Sicherheit and Generationen
Bosnia-Herzegovina Private company
Bulgaria Bulgarian Drug Agency
Germany Federal Institute for Drugs and Medical Devices
Hungary National Institute of Pharmacy
Latvia State Pharmaceutical Inspection, Ministry of Welfare
Armenia Drug Administration and Medical Technology Agency,
Ministry of Health
Lithuania State Medicines Control Agency, Ministry of Health
Uzbekistan Department of Drug and Medical Equipment Quality
Control, Public Health Ministry
Slovak Republic State Institute for the Control of Drugs
Turkey Ministry of Health
United Kingdom European Agency for the Evaluation of Medicinal Products
Yugoslavia Secretariat for Health, Labour and Health Care
SEARO
Myanmar (2)a Food and Drug Administration, Ministry of Health
Thailand Food and Drug Administration, Ministry of Public Health
WPRO
Cambodia Department of Drugs and Food, Ministry of Health
China (4)a Centre for Certification of Drug, and Drug Safety and
Inspection Department, State Drug Administration
Hong Kong, SAR Pharmaceutical Service, Department of Health
Solomon Islands Pharmacy Service Division, Ministry of Health
a
number of responses received in parentheses.
 Promoting GMP Implementation by Training 25

TABLE 7 Feedback on Part 2 of the CD-ROM Questionnaire (44 Responses Received)

Strongly Could be
Statement agree Agree improved Disagree

1. The CD-ROM was easy to use, and 25 16 3 0

files were easily accessed.
2. The material is clear and 26 17 1 0
presented in a professional manner.
3. The material covers 24 20 0 0
relevant topics.
4. The topics covered are ordered in 20 21 1 1
a logical fashiona.
5. The material contains enough 6 18 19 0
examplesa.
6. The duration of the modules is 3 32 5 2
appropriate to the respective
topic coveredb.
7. The trainer’s notes are clear and 19 22 0 1
easy to followb.
8. The trainer’s notes are consistent 21 20 1 0
with the content of the slidesb.
9. The material is useful for developing 24 14 5 0
skills in GMP inspectiona.
10. The material is useful for developing 18 23 2 0
skills in GMP compliancea.
a
Number of responses received ¼ 43.
b
Number of responses received ¼ 42.

DISCUSSION
In evaluating the outcomes achieved since 1998, it is considered that
the aims of the two WHO=EDM projects have been met. Given the
objectives it was necessary for activities to strongly focus on addressing
the huge need for training.
Throughout the various stages of development, the training mate-
rials were enthusiastically received by trainees. The pilot and sub-
sequent one-day courses in Myanmar and Nepal (Table 2) were
successful in contributing to the testing and evolution of the material.
At the same time, trainees enhanced their knowledge in GMP and
benefited from the sharing of experiences. A common theme in group
discussions at Myanmar was the isolation felt by industry, academia,
and government employees due to the lack of exchange of information
on GMP implementation with other countries. At both courses groups
raised the issue of the general lack of resources and GMP expertise in
industry, academia, and among government staff.
Results of comprehension tests conducted at all the workshops and
courses (pre- and post-completion of the modules) (Tables 4 and 5) were
useful for assessing the efficacy of the questions and answers for
26 K. Morimoto et al.

determining participant comprehension of the material, for obtaining

feedback on the perceived overall quality of content of the training
package, and for identifying common areas of difficulty across the
workshops, for example, Validation. Given that there was no baseline
data to indicate comprehension levels of the audiences before training, it
was encouraging to note that at all the locations, participants scored over
50% on every topic, despite the many differing factors that would have
influenced the results, such as language ability, the diversity in cultural
background, subject knowledge, education and experience of the trai-
nees, the varying stages of development of the materials, and so forth.
Course evaluation and feedback from participants indicated a
strong desire for further training for inspectors within regions and
countries, and for longer courses to be set up encompassing all the
modules. At the Chong Qing and New Delhi workshops (Table 3), there
was a particular focus on promoting GMP implementation in the
manufacture of anti-tuberculosis drugs, and participants expressed a
strong wish for some future training to be specifically aimed at in-
spectors and manufacturers of anti-tuberculosis products.
It is hoped that the global distributions of the CD-ROM, together with
the availability of the materials through the WHO Internet site, will
greatly enhance accessibility to the modules. For some countries, how-
ever, the viability of future training will be dependent upon the avail-
ability of computer equipment and software and upon the translation of
the materials. Following the Jamaica workshop (Table 3), the modules
were translated into Spanish and produced as a CD-ROM by WHO Pan
American Sanitary Bureau (PAHO). Feedback to date has indicated that
further translations into other languages are planned, including Chi-
nese, German, Japanese, Portuguese, Russian, and Turkish.
For the future, requests have included the setting up of courses in-
volving more practical hands-on activities, and for additional modules
on such topics as cleaning, process and retrospective validation, water
and air systems, monitoring of testing procedures for microbiological
dust and particulate contamination, and on calibration and system
suitability of analytical instruments. In response, WHO=EDM devel-
oped a supplementary training package, covering the topics of Valida-
tion, Water for Pharmaceutical Use, and Air Handling Systems. This
was finalized in March 2003. Multicopies of the CD-ROM are currently
being produced. A further module, Inspecting the Quality Control
Laboratory, will become available later in the year.
As the SPMI project draws to a conclusion, the future of GMP
training will devolve to regions and individual countries. It is hoped
that the training modules will assist countries in their efforts to
strengthen GMP to ensure medicines are of good quality. There are
 Promoting GMP Implementation by Training 27

positive signs that this process has begun. A Basic GMP course held in
South Africa at the Center for Quality Assurance of Medicine, Uni-
versity of Potchefstroom, South Africa, from 26 November to 7 De-
cember 2001 (organized by WHO=AFRO) using the modules was
attended by trainees from 13 African countries. We have recently re-
ceived information from PAHO that once the modules were translated
into Spanish, the Jamaica workshop was replicated 18 times in 18
countries of the Americas with professors of the Latin American schools
of pharmacy as trainers, and 571 professionals participating as trai-
nees. A database currently being established by WHO=EDM is in-
tended to eventually form the basis of a global training network of GMP
inspectors who would be able to contribute to future domestic training,
enabling more countries to set up their own GMP courses.
Requests for the CD-ROM from individuals across the globe con-
tinue to be received and distributions during 2002 have included a
dispatch to around 850 schools of pharmacy. To date, over 5,800 copies
of the CD-ROM have been distributed.

REFERENCES
Laserson, K. F., Kenyon, A. S., Kenyon, T. A., Layloff, T., and Binkin, N. J. 2001. Substandard tu-
berculosis drugs on the global market and their simple detection. International Journal of
Tuberculosis and Lung Disease, 5, 448–454.
Newton, P. N., White, N. J., Rozendaal, J.A., and Green, M.D. 2002. Murder by fake drugs. British
Medical Journal, 324, 800–801.
O’Brien, K. L., Selanikio, J. D., Hecdivert, C., Placide, M.-F., Louis, M., et al. 1998. Epidemic of
pediatric deaths from acute renal failure caused by diethylene glycol poisoning. Journal of
the American Medical Association, 279, 1175–1180.
Reidenberg, M. M. and B. A. Conner. 2001. Counterfeit and substandard drugs. Clinical Pharma-
cology and Therapeutics, 69, 189–193.
Taylor, R. B., Shakoor, O., Behrens, R. H., Everard, M., Low, A. S., et al. 2001. Pharmacopoeial
quality of drugs supplied by Nigerian pharmacies. The Lancet, 357, 1933–1936.
World Health Organization. 1997. Quality assurance of pharmaceuticals: A compendium of guide-
lines and related materials. Vol. 1, Geneva.
World Health Organization. 1999. Quality assurance of pharmaceuticals: A compendium of guide-
lines and related materials. Vol. 2: Good manufacturing practices and inspection. Geneva.
World Health Organization. 2001. WHO launches GMP training modules. Essential Drugs Moni-
tor, 30, 5.
World Health Organization. 2002. WHO expert committee on specifications for pharmaceutical
preparations. WHO Technical Report Series, 902. Geneva.