MEDICINE

REVIEW ARTICLE

Randomized Controlled Trials

Part 17 of a Series on Evaluation of Scientific Publications

Maria Kabisch, Christian Ruckes, Monika Seibert-Grafe, Maria Blettner

linical research lays the groundwork for progress

SUMMARY
Background: In clinical research, randomized controlled
C in medicine and is an indispensable prerequisite
for evidence-based medicine. Randomized controlled
trials (RCTs) are the best way to study the safety and effi- clinical trials (RCTs) are the gold standard for ascer-
cacy of new treatments. RCTs are used to answer patient- taining the efficacy and safety of a treatment. RCTs can
related questions and are required by governmental
demonstrate the superiority of a new treatment over an
regulatory bodies as the basis for approval decisions.
existing standard treatment or a placebo. In clinical re-
Methods: To help readers understand and evaluate RCTs, search RCTs are used to answer patient-related ques-
we discuss the methods and qualitative requirements of tions, and in the development of new drugs they form
RCTs with reference to the literature and an illustrative the basis for regulatory authorities decisions on
case study. The discussion here corresponds to exposi- approval. Alongside meta-analyses, high-quality RCTs
tions of the subject that can be found in many textbooks with a low risk of systematic error (bias) provide the
but also reflects the authors personal experience in plan- highest level of evidence (1, 2).
ning, conducting and analyzing RCTs. The aim of this article is to provide an introduction
Results: The quality of an RCT depends on an appropriate into the methods and quality requirements of RCTs in
study question and study design, the prevention of sys- order to help the reader understand and evaluate publi-
tematic errors, and the use of proper analytical tech- cations that present the results of such studies. Since
niques. All of these aspects must be attended to in the RCTs are by definition interventional, often investigat-
planning, conductance, analysis, and reporting of RCTs. ing drugs or medical devices, ethical and legal aspects
RCTs must also meet ethical and legal requirements. will also be discussed.
Conclusion: RCTs cannot yield reliable data unless they The discussion here corresponds to expositions of
are planned, conducted, analyzed, and reported in ways the subject in numerous textbooks (35) but also
that are methodologically sound and appropriate to the reflects the authors own experience of planning,
question being asked. The quality of any RCT must be conducting and analyzing RCTs. To aid understanding,
critically evaluated before its relevance to patient care can some methodological issues are illustrated by reference
be considered. to a published trial, the ALIFE study (Anticoagulants
for LIving FEtuses). The fundamental principles of
Cite this as: methodology and statistical analysis for all studies,
Kabisch M, Ruckes C, Seibert-Grafe M, Blettner M: including RCTs, have been expounded in earlier ar-
Randomized controlled trials: part 17 of a series on
ticles in this journals series on evaluation of scientific
evaluation of scientific publications. Dtsch Arztebl Int
publications (611).
2011; 108(39): 6638. DOI: 10.3238/arztebl.2011.0663
The results of the ALIFE study were published in the
New England Journal of Medicine in April 2010 (12)
and presented in the Studies in Focus series of the
German-language edition of Deutsches rzteblatt in
July 2010 (13). In this study, women who had had two
or more miscarriages were assigned randomly to one of
three treatment groups: aspirin plus heparin, aspirin
alone, or placebo. The primary objective of the study
was to investigate the efficacy of the different treat-
ments as shown by the rate of live births.

Interdisziplinres Zentrum Klinische Studien (IZKS), Universittsmedizin der Objectives

Johannes-Gutenberg-Universitt Mainz: Dipl.-Biomath. Kabisch, Dipl.-Math. The basis of every RCT is the study protocol that
Ruckes, Dr. med. Seibert-Grafe
describes the medical/scientific background, the
Institut fr Medizinische Biometrie, Epidemiologie und Informatik (IMBEI),
Universittsmedizin der Johannes-Gutenberg-Universitt Mainz: Prof. Dr. rer. risk:benefit assessment, the study design, the study
nat. Blettner methods, and the overall planning, conduct and

Deutsches rzteblatt International | Dtsch Arztebl Int 2011; 108(39): 6638 663
MEDICINE

analysis (14). The primary study question, i.e., the indispensable for methodological reasons, it can be
primary objective, results from the medical/scientific justified as long as patients will not be harmed (18).
rationale for the study. That is the case, for example, if the study is of only
To answer the primary study question, a primary short duration or if the severity of disease permits post-
endpoint is required. This is a parameter measured or ponement or interruption of treatment.
observed that is recorded at a defined time and can be As in any study of human subjects, the study popu-
assumed to reflect the effect of a treatment. The end- lation of an RCT must be clearly defined. Precise inclu-
point may be clinical, e.g., the live birth rate in the sion and exclusion criteria are elaborated to ensure that
ALIFE study. only eligible patients are recruited. The study partici-
In a confirmatory study hypotheses are formulated a pants must be homogeneous with regard to
priori according to the primary study question. If the demographic characteristics, disease state, and possibly
primary objective of the trial is to demonstrate the su- even comorbidity and comedication.
periority of a new treatment over an existing treatment To ensure fair comparison between the treatments,
or placebo, then the initial assumption (null hypothesis) the different study groups must be truly comparable.
is that the two treatments do not differ in efficacy. This can be achieved by standardization of, for
Based on statistical analysis the null hypothesis can be example, the time(s) of intake of the study medication
retained or must be rejected in favor of the alternative and the methods used to measure clinical parameters,
hypothesis. The alternative hypothesis is assumed but most important for comparability is randomization
when a statistically significant difference is ascertained of the participants.
between the two treatments. (A detailed description of
methods for statistical evaluation is given in an earlier Randomization
article in this series [15].) In RCTs the patients are randomly assigned to the dif-
The primary study question is accompanied by one ferent study groups. This is intended to ensure that all
or more ancillary study questions, i.e., secondary objec- potential confounding factors are divided equally
tives. The secondary endpoints investigate other effects among the groups that will later be compared (struc-
of the treatment, e.g., the occurrence of adverse events tural equivalence). These factors are characteristics that
or the influence on biomarkers. In the ALIFE study, the may affect the patients response to treatment, e.g.,
secondary endpoints included the rate of miscarriage, weight, age, and sex. Only if the groups are structurally
the premature birth rate, and the rate of maternal throm- equivalent can any differences in the results be at-
bopenia. tributed to a treatment effect rather than the influence
From the statistical viewpoint it is vital to distin- of confounders. If the confounders are known, struc-
guish between the primary and secondary study ques- tural equivalence of the patient groups can be attained
tions, because the number of study subjects depends by stratified randomization (Box).
solely on the primary endpoint (16). Study planning in- In the ALIFE study the patients were assigned to the
cludes calculation of the number of subjects necessary three treatment groups with a randomization ratio of
for detection by statistical analysis of a minimally rel- 1:1:1. They were randomized taking account of the
evant difference in efficacy, from the clinical view- prognostic factors of age (<36 years or 36 years) and
point, between the treatments. The number of patients number of miscarriages (2 or 3), and because the
is therefore crucial for the statistical power of a study. study was multicentric they were stratified by study
(Sample size calculation is described in detail in a center. If patients were allocated to treatment groups by
previous article in this series [17].) conscious or unconscious selection for prognosis-
In the ALIFE study a difference of 15% in live birth related characteristics, rather than randomly, this could
rate was assumed between the combination of aspirin lead to biased treatment comparison and distorted
plus heparin and aspirin alone or placebo. In order to results (selection bias).
demonstrate the postulated positive effect of the combi- The assignment to study groups must not be in any
nation therapy, 364 women were enrolled in the trial. way predictable. Predictability of group allocation is
avoided by ensuring the study staff are unaware to
Study design which treatment the next patient will be allotted. Alter-
In trials with randomized and controlled design (e.g., a nating assignment to the different treatments is not
two-armed study with parallel groups), the effects of truly random.
the study treatment (intervention) are compared with
those of a control treatment and the patients are ran- Blinding
domly assigned to the two groups. The patients in the Bias is avoided not only by randomization but also by
control group receive either another treatment or a blinding. A study may be double blind, single blind, or
placebo. The ALIFE trial is a three-armed parallel open.
group study to establish whether the combination treat- In a double-blind study neither patient nor study
ment or the monotherapy improve the live birth rate physician knows to which treatment the patient has
compared with placebo. The use of placebos in clinical been assigned. Double-blind studies are advantageous
trials is ethically justified provided that no standard if knowledge of the treatment might influence the
treatment is available. If comparison with placebo is course and therefore the results of the study. Thus it is

664 Deutsches rzteblatt International | Dtsch Arztebl Int 2011; 108(39): 6638
 MEDICINE

particularly important that the study physician is combined nor aspirin alone were demonstrated to have
blinded to treatment if the endpoints are subjective. a greater effect than placebo on the live birth rate.
Blinding of patients to their treatment is important, for
example, if their attitude could potentially affect their Quality standards and legal requirements
reliability in taking the test medication (compliance) or in Germany
even their response to treatment. Clinical trials have to be performed according to
If only one party, either patient or study physician, is national and international regulations. The Declaration
blinded to the treatment, the study is called single of Helsinki, first formulated by the World Medical
blind; a study with no blinding is described as open. Association in 1964 and revised several times in the in-
The highest possible degree of blinding should be tervening years (20), lays down fundamental ethical
chosen to minimize bias. principles for research on human beings. Trials investi-
gating drugs and medical devices have to comply with
Analysis population the relevant German laws for drugsthe German
The data subjected to statistical analysis in RCTs are Medicines Act (AMG; for German text see Bundesge-
those gathered from patient populations defined in the setzblatt I p. 2262)and the GCP regulation (GCP-
study protocol. The primary population for analysis is Verordnung [21]), and for devices the Medical Devices
the so-called intention-to-treat (ITT) population, Act (MPG; for German text see Bundesgesetzblatt I p.
comprising all randomized patients. In analysis accord- 983), revised in March 2010. The GCP regulation,
ing to the ITT principle, patients are allocated to the which came into force in 2004, made adherence to good
group to which they were randomized, thus retaining clinical practice (GCP) a legal requirement in Germany
the advantages of randomization such as structural (21). GCP Guideline ICH-E6 of 1997 forms the basis
equivalence. Because the ITT population includes all for European Directives 2001/20/EG and 2005/28/EG,
patients who were randomized, the data for analysis in- on which in turn the GCP regulation is based (14). The
clude some patients whose treatment was interrupted, aim of GCP is to protect study participants and ensure
prematurely discontinued, or did not take place at all. high quality of study data.
The analysis strategy for ITT data is therefore conser- In 2004 the International Committee of Medical
vative, i.e., the treatment effect tends to be underesti- Journal Editors made registration of a clinical trial in a
mated (19), regardless of whether the primary endpoint public registry a precondition for its publication (22).
represents an improvement or a deterioration. Many The professional code of conduct for physicians in
studies define a modified ITT (mITT) population, Germany demands that every study in human subjects
which may for example comprise the patients who be submitted to the responsible ethics committee for
received at least a defined amount of study treatment. approval. Drug trials and most studies of medical
An alternative strategy is to restrict analysis to the
data from the per-protocol (PP) population. Patients in
whom study conduct deviated from the protocol are
excluded from analysis. These so-called protocol BOX
violations include, for example, failure concerning the
application of inclusion or exclusion criteria and incor- Stratified randomization
rect administration of the study treatment. In analysis
according to the PP principle, patients are allocated to If the stratification factors sex (male, female) and age (<18
the treatment groups depending on the treatment they years, 18 years) are to be considered and 150 patients
actually received. Because the PP population includes are to be randomized in a ratio of 1:1 into the active treat-
only those patients who completed the study according ment and placebo groups (275 patients), then randomi-
to the protocol, the results may be distorted in favor of zation has to be performed for each separate subgroup
the investigational intervention (19). (stratum). Two stratification factors, each with two values,
To assess the robustness of the study findings, PP yield four strata (male and <18 years, male and 18 years,
evaluation is carried out as a sensitivity analysis if the female and <18 years, female and 18 years).
ITT population is the patient population for the primary
efficacy analysis (16). If the results of PP and ITT
evaluation of the primary endpoint are very similar, Active Placebo
they can be regarded as reliable. Should this not be the treat-
ment
case, the possible reasons for the discrepancy between
the results of the ITT and PP analyses must be Male and <18 years 10 10
discussed in the results section of the publication. Male and 18 years 16 17
The data of the ALIFE study, particularly the pri- Female and <18 years 24 23
mary endpoint, were statistically evaluated on the basis
Female and 18 years 25 25
of the ITT population. The rates of live births in the
three treatment groups did not differ significantly Total 75 75
(Table 1). Analysis according to the PP principle
confirmed this finding. Neither aspirin and heparin

Deutsches rzteblatt International | Dtsch Arztebl Int 2011; 108(39): 6638 665
MEDICINE

TABLE1 devices require not only approval from the local ethics
committee but also from regulatory bodies at the
Results of the ALIFE study (adapted from [12]) federal level (Federal Institute for Drugs and Medical
Aspirin plus Aspirin alone Placebo p-value Devices [BfArM] or Federal Institute for Vaccines and
Heparin Biomedicines, Paul-Ehrlich-Institut [PEI]). The appli-
Intention-to-treat 123 120 121 cations have to be accompanied by the study protocol,
population n the information to be supplied to the patients, the
Live births 67 (54.5) 61 (50.8) 69 (57.0) 0.63 consent form for participation, and confirmation that
n (%) adequate insurance has been arranged.
Relative risk 0.96 (0,761,19) 0.89 (0.711.13) 1.00 Trials of drugs and medical devices also have to be
(95% CI) registered with state authorities. There are legally
Absolute -2.6 (-15.09.9) -6.2 (-18.86.4) defined obligations to report suspected unexpected
difference in serious adverse reactions or early termination of a
live birth rates study, and the final study report must also be submitted.
(95% CI) %
The Federal Data Protection Act (BDSG; for German
text see Bundesgesetzblatt I p. 2814) and the AMG ob-
Relative risk and absolute difference were calculated for the comparisons between aspirin plus heparin and
placebo and between aspirin alone and placebo. The p-value applies to all treatment group comparisons. ligate researchers to pseudonymize all person-related
95% CI, 95% confidence interval data that are gathered, documented, stored, and
analyzed in the course of a clinical trial. In other words,
information revealing the identity of a patient (name or
initials) must be replaced by a code. Only patients who
have agreed in advance to the recording, storage,
processing and dissemination of their data may partici-
pate in a clinical study.

Discussion
Any publication of an RCT must lucidly describe the
planning, conduct, and analysis of the study. The CON-
SORT statement provides a minimum set of recommen-
TABLE 2 dations for reporting RCTs (23). The most important
aspects that have to be described in the publication are
Minimal requirements for a publication reporting a randomized controlled trial listed in Table 2. The progress of patients through an
(adapted from [23])
RCT and the numbers of patients whose data were
Study design Description of study design (e.g., parallel group analyzed can be depicted in a flow diagram (Figure).
comparison) The study results and their interpretation must be
Study population Specification of inclusion and exclusion criteria for discussed in detail in the study report and any subse-
patients quent publication, and the limitations of the methods
Treatments Detailed account of treatments and their application in used should be described, all with reference to the
each intervention group and control group study design, the recent literature, and the current state
Objectives Precise formulation of primary and secondary of knowledge. Critical discussion plays a decisive part
objectives/study questions in clinical evaluation of the results. In the publication
Endpoints Clear definition of primary and secondary endpoints of the ALIFE study, the findings were compared with
Sample size Description of how the required number of study those of other RCTs investigating the effects of heparin
participants was determined on reduction of miscarriages and inconsistencies were
Randomization Description of type of randomization of patients to discussed. Ultimately, the available study data did not
treatment groups (e.g., stratified randomization) justify the recommendation of anticoagulants for
Blinding Specification of degree of blinding (e.g., double blind) women with recurring miscarriages.
Although RCTs are the gold standard with regard to
Analysis population Number of patients analyzed in each treatment group
and definition of population for analysis (e.g., ITT) level of evidence, their generalizability, i.e., the extent
to which their results can be extrapolated to the wider
Results Presentation of the results for all primary and
secondary endpoints for each treatment group patient population (external validity) is often
questioned, because standardized and controlled study
Adverse events Details of all major adverse events for each treatment
group
conditions do not adequately reflect clinical reality.
Moreover, the patients selected for a study are not
Interpretation Interpretation of the results, taking into account the
study question, possible causes of bias, the current
necessarily representative, in that those seen in routine
state of knowledge, and other researchers' publications daily practice will often have numerous comorbidities
on the same topic and comedications. After marketing approval of a new
Generalizability Discussion of the applicability of the study results to treatment, phase-IV studies are carried out to establish
general patient care its efficacy and safety in a larger and more heterogen-
eous population; as a rule these studies are RCTs.

666 Deutsches rzteblatt International | Dtsch Arztebl Int 2011; 108(39): 6638
 MEDICINE

Epidemiological studies, e.g., cohort studies, are FIGURE Patient flow in

particularly suitable for detection of infrequent adverse a randomized
effects. controlled trial
Assessed for eligibility (n=) (adapted from [23])
Conclusion
RCTs are the best type of study for determining Randomized to the treatment groups (n=)
whether there is a causal relationship between interven-
tion and effect (24). Recent discussions in the scientific
Randomized to Randomized
community and the new Law on the Reorganization of active treatment (n=) to placebo (n=)
the Pharmaceutical Market (AMNOG; for German text
see Bundesgesetzblatt I p. 2262), which regulates the
use of drugs and medical devices, clearly show that Treatment Treatment
received (n=) received (n=)
RCTs are still the standard for demonstrating efficacy
and safety so that a new treatment can be approved for Treatment Treatment
use in patients. However, it seems clear that not received (n=) not received (n=)
post-marketing studies comparing new and established
treatments are still required. Treatment Treatment
completed (n=) completed (n=)
The IZKS Mainz is supported by the grant Clinical Trial Centers [Klinische Stu-
dienzentren], no. FK 01KN1103, IZKS Mainz from the Federal Ministry of Edu- Treatment Treatment
cation and Research. discontinued (n=) discontinued (n=)
Acknowledgment
The authors are grateful to Daniel Wachtlin of the Interdisciplinary Center for
Clinical Trials (IZKS) Mainz for helpful discussions. Analyzed (n=) Analyzed (n=)
Conflict of interest statement Excluded from Excluded from
The authors declare that no conflict of interest exists. analysis (n=) analysis (n=)
Manuscript received on 23 February 2011, revised version accepted on 28
June 2011.

Translated from the original German by David Roseveare.

REFERENCES 12. Kaandorp SP, et al.: Aspirin plus heparin or aspirin alone in women
1. Harbour R, Miller J: A new system for grading recommendations in with recurrent miscarriage. NEJM 2010; 362: 158696.
evidence based guidelines. BMJ 2001; 323: 3346. 13. Heinzl S: Knnen ASS und Heparin wiederholte Fehlgeburten ver-
2. Phillips B, Ball C, Sackett D, Badenoch D, Straus S, Haynes B, hindern? Dtsch Arztebl 2010; 107(27): A-1355.
Dawes M (2001): Oxford Centre for Evidence-based Medicine 14. ICH E6: Guideline for good clinical practice. London UK: Inter-
Levels of Evidence. www.cebm.net/levels_of_evidence.asp national Conference on Harmonization 1996; CPMP/ICH/135/95.
3. Meinert CL: Clinical Trials: Design, conduct, and analysis. Oxford 15. du Prel J, Rhrig B, Hommel G, Blettner M: Choosing statistical
University Press: New York 1986. tests: part 12 of a series on evaluation of scientific publications.
4. Shein-Chung C, Jen-Pei L: Design and analysis of clinical trials: Dtsch Arztebl Int 2010; 107(19): 3438.
concepts and methodologies. John Wiley & Sons: New Jersey 16. ICH E9: Statistical Principles for Clinical Trials. London UK: Inter-
2004. national Conference on Harmonization 1998; CPMP/ICH/363/96.
5. Schumacher M, Schulgen G: Methodik klinischer Studien: Metho- 17. Rhrig B, du Prel JB, Wachtlin D, Kwiecien R, Blettner M: Sample
dische Grundlagen der Planung, Durchfhrung und Auswertung. size calculation in clinical trials: part 13 of a series on evaluation
Berlin, Heidelberg: Springer-Verlag 2008. of scientific publications. Dtsch Arztebl Int 2010; 107(3132):
6. Rhrig B, du Prel, Blettner M: Study Design in medical research: 5526.
Part 2 of a series on evaluation of scientific publications. Dtsch 18. ICH E10: Choice of Control Group and Related Issues in Clinical
Arztebl Int 2009; 106(11): 1849 Trials. London UK: International Conference on Harmonization 2000;
7. Rhrig B, du Prel du J, Wachtlin D, Blettner M: Types of study in CPMP/ICH/364/96.
medical research: part 3 of a series on evaluation of scientific publi- 19. Ellenberg JH: Intention-to Treat Analysis. In: Redmond C, Colton T
cations. Dtsch Arztebl Int 2009; 106(15): 2628.
(eds): Biostatistics in Clinical Trials. New Jersey: John Wiley & Sons:
8. Spriestersbach A, Rhrig B, du Prel J, Gerhold-Ay A, Blettner M: De- 2001.
scriptive statistics: the specification of statistical measures and their
presentation in tables and graphs: part 7 of a series on evaluation 20. World Medical Association Declaration of Helsinki: Ethical principles
of scientific publications. Dtsch Arztebl Int 2009; 106(36): 57883. for medical research involving human subjects. 2008. www.wma.
net/en/30publications/10policies/b3/17c.pdf.
9. Sauerbrei W, Blettner M: Interpreting results in 22 tables: part 9 of
a series on evaluation of scientific publications. Dtsch Arztebl Int 21. Verordnung ber die Anwendung der Guten Klinischen Praxis bei
2009; 106(48): 795800. der Durchfhrung von klinischen Prfungen mit Arzneimitteln zur
10. Victor A, Elser A, Hommel G, Blettner M: Judging a plethora of Anwendung am Menschen (GCP-Verordnung GCP-V). GCP-
p-values: how to contend with the problem of multiple testing: part Verordnung vom 9. August 2004 (BGBl. I S. 2081), die zuletzt durch
10 of a series on evaluation of scientific publications. Dtsch Arztebl Artikel 4 der Verordnung vom 3. November 2006 (BGBl. I S. 2523)
Int 2010; 107(4): 506. gendert worden ist.
11. Zwiener, I, Blettner M, Hommel G: Survival analysis: part 15 of a 22. Clinical trial registration: a statement from the International Com-
series on evaluation of scientific publications. Dtsch Arztebl Int mittee of Medical Journal Editors (ICMJE). Lancet 2004: 364:
2011; 108(10): 1639. 9112.

Deutsches rzteblatt International | Dtsch Arztebl Int 2011; 108(39): 6638 667
MEDICINE

23. Schulz KF, Altman DG, Moher D: CONSORT 2010 Statement: up-
KEY MESSAGES
dated guidelines for reporting parallel group randomised trials. BMJ
2010; 340: c332.
In clinical research, randomized controlled trials are the
24. Windeler J: Bedeutung randomisierter klinischer Studien mit gold standard for demonstrating the efficacy and safety
relevanten Endpunkten fr die Nutzenbewertung. In: Diskussions-
forum zur Nutzenbewertung im Gesundheitswesen: Begriffsdefini- of a new treatment.
tionen und Einfhrung. 2007. www.gesundheitsforschung-bmbf. Randomized controlled trials cannot yield robust data
de/_media/DLR_Nutzenbewert_071122_Druckversion.pdf. unless they are planned, conducted, and analyzed in
Corresponding author
ways that are methodologically sound and appropriate
Prof. Dr. rer. nat. Maria Blettner to the question being asked.
Institut fr Medizinische Biometrie, Epidemiologie und Informatik (IMBEI)
Universittsmedizin der Johannes-Gutenberg-Universitt Mainz Methods to avoid bias, such as randomization and
Obere Zahlbacher Str. 69 blinding, can help to prevent distortion of the study
55131 Mainz, Germany
[email protected] results.
The robustness of the results is tested by statistical
analysis of the data from patient populations defined a
priori.
The quality of a randomized controlled trial depends
crucially not only on adherence to methodological stan-
dards but also on strict compliance with the protocol re-
garding the clinical conduct of the study.

668 Deutsches rzteblatt International | Dtsch Arztebl Int 2011; 108(39): 6638