Critical Review of Epidemiologic Studies
Ephrem M(Assistant professor of
Epidemiology, PhD fellow)
Outline for critiquing epidemiologic studies
A. Collection of data
– What was the context of the study?
– What were the objectives of the study?
– What was the primary exposure of interest?
Was this accurately measured?
– What was the primary outcome of interest?
Was this accurately measured
– What type of study was conducted?
Outline cont..
– Describe the sources of the study population, process of
subject selection, sample size, and ratio of proposition to
comparison subjects.
– Could there have been bias in the selection of the study
subjects?
How likely was this bias?
– Could there have been bias in the collection of information?
– How likely was this bias?
– What provisions were made to minimize the influence of
confounding factors prior to the analysis of the study?
Were these provisions sufficient?
Outline cont..
B. Analysis of data
– What methods were used to control confounding
during data analysis?
Were these methods sufficient?
– What measures of association were reported in
this study?
– What measures of statistical stability were
reported in this study?
Outline cont..
C. Interpretation of data
– What were the major results of this study?
– How is the interpretation of these results affected by
selection bias, information bias, and confounding?
Discuss both the direction and magnitude of any bias
–
What were the authors’ main conclusions? Were they
justified by the findings?
– To what larger population can the results of this study be
generalized?
A. COLLECTION OF DATA
1. What was the context of the study?
• This qes deals with the investigators’ motivation of
the study.
• E.g observations that stimulated the study may have
come from
– laboratory research
– clinical reports
– other epidemiological studies.
What was the context of the study?
Cont…
• The answer to this question gives the reader a sense
of the particular contribution made by the study at
hand.
• For example:
– the study may be the first of its kind
– it may be replicating results of an earlier study in
a different population, with larger numbers, or by
using more refined methods.
2. What were the objectives of the study?
• deals with the purpose of the study/ hypothesis being
tested.
• Examples of hypotheses
-unprotective receptive anal intercourse increases the
risk of HIV-1 transmission among homosexual men
3. What was the primary exposure of interest?
Was this accurately measured?
The exposure of interest is the factor that either
causes, prevents, or treats the outcome
4. What was the primary outcomes of interest?
Was this accurately measured?
• consider whether the outcome was accurately defined
and measured
• look for measures taken to improve the accuracy of
the outcome data
• E.g use of a combination of signs and symptoms,
tests to define cases of disease etc.
5. What type of study was conducted?
• Which design was used?
• Important to know which design was used because
certain designs are more prone to bias than others
6. Describe the source of the study population, process
of subject selection, sample size,
• This allows the reader to evaluate many important
topics like:
– the potential for selection bias and confounding
– the statistical power of the study
– generalizability of the results.
• The source of the study population is the underlying
population from which the subjects were drawn.
.
7. Could there have been bias in the selection of the
study subjects? How likely was this bias?
• Selection bias arises from systematic differences in
selecting the study groups, or in following the selected
groups.
• These differences lead to an effect estimate among
subjects in the study that is different from the estimate
obtainable from the entire underlying population.
• Losses greater than 20%, especially if they occur among
different types of subjects than those who remain, call the
validity of the study into question.
9. What provisions were made to minimize the influence of
confounding factors prior to the data analysis?
Were these provisions sufficient?
• When evaluating a study, the reader should review
the information on confounder that was collected.
• The methods to control for confounding prior to data
analysis include:
– randomization,
– restriction,
– matching, and
– use of the same source population for the
compared groups.
B. ANALYSIS OF DATA
10. What methods were used to control confounding
during the data analysis?
Were these methods sufficient?
• During analysis, the main ways to control
confounding are standardization, stratification,
matched analysis, and multivariate analysis.
11. What measures of association were
reported in this study?
What measures of statistical stability were reported in
this study?
• Hypothesis testing and interval estimation are used to
assess the precision of study findings and to evaluate
the role of chance in epidemiologic studies.
• P values and confidence intervals are the two main
ways to assess the role of chance in epidemiologic
research.
13. How is the interpretation of these results affected
by information bias, selection bias, and confounding?
Discuss both the direction and magnitude of any bias
The reader should gather all of the relevant information
from the other sections of the critique to determine the
net effect of any information bias, selection bias, and
confounding that may be present in the study.
• A small amount of bias or confounding will usually not
have a major impact on the study results, but
• a large amount of bias or residual confounding could
completely alter the findings.
How is the interpretation of these results
affected cont…
•The impact of bias and confounding will also depend
on the true size of the association.
•They can account entirely for a weak association, but
they are unlikely to account entirely for a strong one.
It may be useful to conduct a “sensitivity analysis” in
order to determine the magnitude of the bias.
This analysis determines the robustness of the results by
examining the extent to which they are affected by
assumption about the amount of bias.
15. Did the discussion section adequately
address the limitations of the study?
A self-critical examination of the study methods and
findings should be included in the discussion section
of the article
16. What were the authors’ main conclusions? Were
they justified by the findings?
• The conclusion are a brief summary of the findings
and their implications
• In addition the authors sometimes include
recommendations for future research and public
health policy
• All conclusions should be supported by the study
findings
To what larger population can the results
of this study be generalized?
• The evaluation of generalizability or “external
validity” requires review of:
– the study methods –
Eg was restriction used to control for confounding?
– the composition of the study population
Eg- were minorities included?
– subject matter knowledge such as the biological
basis of the association
Eg. would the same results be expected among men
and women?
