STUDY DESIGNS

IN BIOMEDICAL RESEARCH

NUTS & BOLTS OF BASIC DESIGNS

Cross-Sectional Designs
Factor Factor
Present Absent
Disease

No Disease

Take One
Sample
In a cross sectional study, investigators draw a
sample, randomly, from the population, then make
all measurements for all variables on a single
occasion- or within a very short period of time –
without a follow up. They study distributions the
variables within that sample; sometimes
designating predictor and outcome variables based
on “biological plausibility”, then correlating one to
the other.
Example:
In the National Health and Nutrition Examination Survey
(NHANES), a sample designed to represent the U.S.
population is interviewed and examined. These surveys
have been carried out periodically and all data are
available for public use. They make up a major source of
information about the health and the habits of the U.S.
population; one could obtain estimates such as
prevalence of smoking or a disease.
 In addition to studying distributions and obtaining
parameter estimates, cross-sectional studies can
also be used for examining associations. For
examples, a cross-sectional finding in NHANES III is
an association between childhood obesity and
hours watching television. The choice of which
variables to label as predictors and which as
outcomes depends on the cause-and-effect
hypotheses of the investigator rather than on the
study design.
Serial Surveys:
These form a special case. A series of cross-
sectional studies of a single population
observed at several points in time – the case
of those NHANES – is sometimes used to draw
(informal) inferences about changing patterns
of population characteristics over time.
Strengths:
A major strength of cross-sectional studies is that
there is no waiting time for the outcome to occur,
and no loss to follow-up. This makes them fast and
inexpensive. And the obvious strength of their sizes.
A cross-sectional study, because of its low cost,
could be included as the first step in a cohort study
or an experiment.
Weaknesses:
The major weakness of cross-sectional studies is
the difficulty of establishing causal relationships from
“observational” data collected in a cross-sectional
time frame.
Cross-sectional studies are also “impractical” for
the study of rare diseases if the sample was collected
from the general population; we might need 10,000
subjects or more to find just one case of a rare
disease. What would happen to statistical power?
CASE-CONTROL STUDIES
Case-Control Design
Factor Factor
Present Absent
Disease Sample 1:
Cases
No Disease Sample 2:
Controls

These are “retrospective”; obtaining past

data from cases and from controls (people
without the disease). The research focus is
the disease.
Case-control studies “began” as epidemiologic studies to
try to identify risk factors for diseases. Therefore, the
term “cases” refer to those with the disease under
investigation. However, the term has become more
generic; case-control design can also be used to look at
other outcomes, such “disability” among those who
already have a disease. In that case, “controls” are those
with the disease but not disability. The “case” in a case-
control study maybe a patient who has had a good but
rare outcome; say, recovery from a usually fatal disease.
Some investigators and scientists refer to case-
control studies as “confirmatory observational
studies” and to cross-sectional studies as
“exploratory observational studies”. Both are
observational; without interventions. Case-
control studies are “retrospective”; obtaining
“past” data from cases and from controls. The
research focus is the disease (‘status” under
investigation).
STEPS IN THE CASE-CONTROL DESIGN

 Select the sample of cases

 Select the sample of controls
 Measure predictor variable & potential
confounders and effect modifiers
Case-control studies provide information on
the characteristics of the cases and an
estimate of the strength of the association
between each predictor variable and the
presence or absence of the disease. These
estimates are in the form of the odds ratio
which approximates the Relative Risk if the
disease is relatively rare.
Strength: Efficiency for Rare Outcomes

A major strength of case-control studies is their

rapid, high yield of information from relatively few
subjects. For rare diseases or outcomes, cross-
sectional design is impractical; it requires a study
size that no investigator could afford. Cohort
design, which is normally larger, is also impractical
because it requires a follow-up time longer than
most investigators could afford.
Strength: Generating Hypotheses

The retrospective approach of case-control studies,

and their ability to investigate a large numbers of
possible predictors make them useful for
generating hypotheses about, say, the causes of
new outbreak of disease so that a more thorough
investigation or investigations could follow.
Weaknesses:
Case-control studies have great strengths but they
also have major limitations; among them:
(1) Accuracy of exposure histories.
(2) Appropriateness of controls.
(3) Unlike cohort designs, we can only study one
disease/outcome at a time.
Sampling Bias:
The data collection in a case-control study begins with
the cases. But how do we know if these cases are
representative of all patients who developed the disease;
those who are undiagnosed, or died, are not included. And
some included might be misdiagnosed (easier problem
here). The more difficult decisions faced by investigators
of a case-control study, however, relates to the more open-
ended task of selecting controls. The ones included might
be inappropriate and confound study conclusion.
Hospital- or Clinic-based Controls

One strategy to compensate for the possible

selection bias caused by obtaining cases from a
hospital or clinic is to select controls from the same
facility. However, the risk factor of interest might be
related to causes for which those control seek care;
if so, prevalence of the risk factor in the control
group would be falsely high, biasing the study
results toward “the null”. That’s why, some studies
use two control groups.
Matching & Multiple-matching:

Matching, and multiple matching, is a relatively

simple method of ensuring that cases and controls
are comparable with respect to major factors that are
related to the disease but are not interest to the
investigators. Examples are gender and Age group.
Matching does have its adverse consequences,
especially when “modifiable factors”, such as
income and cholesterol levels, are matched.
Population-based Cases:
Population-base case-control studies are now
possible for many diseases, like cancers, because of
a rapid increase in the creation of maintenance of
“disease registries”. Cases obtained from disease
registries are generally more representative of the
population of patients. When information on the
cases and controls can come from the same sources,
the design has the potential for eliminating sampling
bias. Later, we will cover such a form, the “nested
case-control design”.
Two or more Control Groups:

Selection of controls can be very tricky, especially

might not be representative of patients; for example,
hospital-based cases. In those studies, it is advisable
to use more than one control groups; for example a
hospital-based and a population-based control
groups. The former might be biased toward the Null.
On the other hand, be prepare to deal with “multiple
decision problem”.
Measurement Bias: The Need for Blinding
Besides selection bias, case-control studies might be
bias due to measurement error caused by their
retrospective approach: the “recall bias” especially
when it occurs to a different extent in cases and in
controls. A necessary solution is the need for blinding:
both observers (interviewers, for example) and study
subjects could be both blinded to the case-control
status of each subject and to the risk factor being
studied (not an easy task at all!).
Case-crossover Studies:
A variant of the case-control design, useful for the short-
term effects of varying or intermittent exposures, is the
“case-crossover design”. As with the “regular” case-control
studies, case-crossover studies are retrospective studies
that begin with a group of cases. However, in regular case-
control studies, the exposures of the cases are compared
with exposures of a group of controls. Each case in case-
crossover studies serves as his/her own control. Exposures
of the cases at or right before outcome time are compared
with exposures of those same cases at one/more other
points in time.
There are a few things we can do in the “Design
Stage” to enhance results of causal inference.
Suppose that a study reveals an “association” between
coffee drinking and myocardial infarction (MI). There are 5
possibilities: (1) Coffee drinking and MI are not related;
what revealed was a chance finding (random error); (2)
Coffee drinking and MI are not related; what revealed was
caused by some bias, systematic error; (3) MI is a cause of
coffee drinking, a so-called “Effect-Cause” phenomenon;
(4) Coffee drinking is associated with a third extrinsic
factor, called confounder, and the confounder is a cause
for MI; and (5) Coffee drinking is a cause for MI; this is the
real Cause-Effect phenomenon (bingo!) – the ideal
possibility .
Before reaching the ideal conclusion –
something the investigators wanted to prove,
the other four rival possibilities have to be
considered and ruled out. How? What can or
should we do in the design stage?
Strategies for addressing random errors (Possibility #1)
are available in both design and analysis stages. In data
analysis, you focus on “statistical significance” (p-value).
The design strategies include: (1) Increasing the
“precision” of measurements, and more important, (2)
Increasing sample size. So, sample size estimation is
needed not just for budget justification!
Ruling out spurious associations due to bias
(Possibility #2) is trickier, more difficult. Here are 3 basic
questions to ponder: (1) Do the samples really represent
the target populations?; (2) Do the measurements of the
predictor variables really represent the predictors of
interest (the issue of randomization included here)?;
and (3) Do the measurements of the outcome variables
really represent the outcomes of interest (the use of
surrogate markers included here).
 Strategies for coping with confounders (Possibility #3)
require that investigators be aware of, be able to measure,
and use them. The most common way to “use” them is
matching – especially for factors which are not easy for
quantify for use in data analysis (e.g. geographical
factor). However, be cautious, you might overdo it!
Overmatching can reduce statistical power and making it
more difficult to generalize the findings.
About the only way to rule out “Effect-Cause”
possibility (Possibility $4) from an
observational study, the possibility that “the
cart has come before the horse”, is to follow
up with a cohort, longitudinal study – as
outlined in an earlier section of this lecture,
“Natural History of Research”.
COHORT STUDIES
“Cohort” was the Roman term for a group of soldiers
that march together. In clinical research, a cohort is a
group of subjects followed over time. In itself, the term
“cohort” does not yet mean “prospective”. In the
design terminology, we have “prospective cohorts” but
we also have retrospective cohorts which may appear
under “nested case-control” or “case-cohort” options.
PROSPECTIVE COHORTS
In a prospective cohort study, the investigator:
Selects a sample from a target population;
Measures (baseline) values of predictor variables;
Measures the outcomes during follow-up
In the most simple case, one binary predictor: presence
or absence of a risk factor and for the outcome,
whether a disease occurs. This type of design is
prospective & longitudinal.
An Example:
(1) In 1976, investigators obtained lists of registered nurses
aged 25 to 42 in the most populous states and mailed them
an invitation to participate in the study; those who agreed
became the cohort;
(2) They mailed a questionnaire about weight, exercise, and
other potential risk factors; they obtained 121,700
completed questionnaires, that’s the size of the cohort;
(3) They send periodic questionnaires about the occurrence
of a variety of disease outcomes, heart diseases and
cancers included.
Some Results:
The investigators succeeded in following 95% of
the nurses and 1,517 cases of breast cancer were
confirmed during the next 12 years. They found
that, for example, women who gained more weights
have a higher risk of breast cancer after
menopause; those who gained more than 20 kg
since age 18 had a twofold increased risk of
developing breast cancer.
Strengths:
(1) Suitable for assessing “disease incidence” (new cases);
helpful in investigating potential “causes” because cohort
members were free of the disease under investigation to start
with;
(2) Measurements of predictors are not influenced by knowledge
of the outcome;
(3) Prospective approach allows investigators to measure
variables more completely and more accurately, to update the
status of risk factors – especially important for “time-
dependent” covariates; the large size of the cohort and long
period of follow-up provide substantial “Statistical Power”.
Weaknesses:
(1) Cohort studies, even prospective cohort studies,
are basically “observational”; causal inference
could be challenged and interpretation often
muddied by potential influences of confounders
and effect modifiers;
(2) Time and cost consuming. It could be more
feasible if outcomes are more common and
immediate; for example, a prospective study of risk
factors for progression (or relapse) after treatment
of patients with breast cancer.
RETROSPECTIVE COHORTS
A retrospective cohort differs from a prospective
cohort in that the assembly of the cohort,
baseline measurements, and follow-up all
happened in the past. It was assembled for other
purposes; however, important data about risk
factors are still possible to obtained for the new
purpose – for example, from banked blood
samples.
Design of Retrospective Cohorts:
The Investigator:
(1) Identifies a cohort that has been assembled;
(2) Collects data on predictors (“measured” in the past);
(3) Collects data on the Outcome (measured in/at the
present.
Example #1:
To study thoracic aortic aneurysm, investigators:
(1) Search the database of Olmsted County, Minnesota –
which is considered a cohort because of thorough
medical records of its residents – an found 133 cases of
aneurysm;
(2) They reviewed patients’ records to collect data on age,
size of aneurysm, and other factors of cardiovascular
diseases at the time of diagnosis;
(3) For the outcomes, they collected data from the medical
records of these 133 patients to determine whether the
aneurysm ruptured or was surgically repaired
Example #2:
The Singapore Cohort was drawn from residents in
government-built housing estates (roughly 86% of the
population resided in such facilities); enrollment period
was 1993 - 1998. Men and women between the ages of 45
and 74 years (35,298 were women), representing 85% of
eligible subjects, were enrolled. At the time of
recruitment, each cohort subject was interviewed in-
person using a structured questionnaire that focused on
current diet. Blood samples were requested and a total
28,346 blood samples were archived and banked. To date,
only <0.05% of subjects are lost to follow-up.
To date, there are 304 incident breast cancer
cases with a stored blood sample, and a study
was proposed to investigate the roles of some
genetic factors and diets as possible risk
factors/protectors for breast cancer.
Strengths and Weaknesses:
Retrospective cohorts have many of the same
strengths as prospective cohorts and they have the
advantage of being less costly and less time
consuming.
The main disadvantages are the limited control
investigators could have over the nature and the
quality of data; existing data on predictors could be
incomplete (too late now!) and not ideal for
answering the research question.
NESTED CASE-CONTROL
A nested case-control design is a case-control study
“nested” within a cohort study.
Investigators begin with a suitable cohort having
enough cases (to assure adequate statistical power) to
answer the research question. Then, they select a random
sample of the subjects who have not developed the
outcome/disease under investigation (the controls); they
could increase the power by selecting two or three
controls matched to a case
Example:
Back to the Singapore cohort assembled in 1993-
1998. By the end of 2011, there are 304 incident
breast cancer cases with a stored blood sample,
and a study is proposed to investigate the roles
of some genetic factors and diets as possible risk
factors/protectors for breast cancer. The
proposed design was a 2-to-1 matched case-
control study of roughly 900 women with the
following Specific Aims (there are more aims):
Specific Aims:
(1) Investigating the T-reg, T-cell and NK cell levels
as a risk factor for breast cancer occurrence;
(2) Correlating T-reg, T-cell and NK cell levels with
diet factors (from baseline interview) focusing on
soybean products and green tea.
NESTED CASE-COHORT
The nested case-cohort option is almost the same
design as the nested case-control except that the
“controls” are a random sample of all the members of
the cohort “regardless of outcomes”. This means
there might be some cases among those sampled for
the comparison group; these cases appear in both
groups. This approach has the advantage that the
“controls” (even some of them are cases) represent
the cohort in general, and therefore provide a basis for
estimating incidence and prevalence in the population
from which it was drawn.
Strengths and Weaknesses:
Nested case-control and nested case-cohort designs
are especially useful for costly measurements on
serum, electronic images (MRI and mammograms),
and hospital charts, etc… that have been archived at
the assembly time of the cohort and preserved for
later analysis.
When data are available, or can be obtained easily,
for the entire cohort, nothing is gained by studying
only a sample; the whole cohort should be used.
MULTIPLE-COHORT STUDIES
Multiple-cohort studies begin with two or more
groups of subjects; typically, one group with no
exposure to a potential risk factor and one or more
other groups with different levels of exposure. This is
different from case-control design because in a case-
control study the two groups are chosen based on the
presence or absence of the outcome.
Multiple-cohort design is particularly useful and
popular for studying rare exposures such as
occupational and environmental hazards.
ISSUES WITH COHORTS
The hallmark of a cohort study is the identification of a
group of subjects at the beginning of a period of
follow-up:
 Subjectsshould be appropriate to the research
question;
 Subjects should be available for follow-up;
 Subjects
should be resemble to the population to
which the results will be generalized;
 Number of subjects should provide adequate
(statistical) power.
The quality of the study (and future studies)
will depend on the precision and accuracy of
the measurements of predictor (s) and outcome
variable.
The ability to draw inferences about cause and
effects will also depend on the degree to
investigators have identified and measured all
potential confounders and effect modifiers.
Predictors may change during the follow-up;
whether and how frequently measurements should
be repeated depends on how they are likely to
change and, of course, depends on the cost and the
importance to the research question of observing
these changes.

Outcomes should be observed/assessed using

standardized criteria and, ideally, blindly without
knowing the values of the predictor variable.
Follow-up of the entire cohort is important; investigators
should take a number of steps to achieve this goals; for
example:

 Exclude those likely to be lost, or collect adequate

information (physician or friends or relatives) that can be
used if they move or die;

 Prepare for periodic contacts (by mail, by phone, etc…);

 Show respect and appreciation!

Suggested Readings
Retrospective cohorts were assembled by
someone else and for other purposes; using
these databases may involve complicated issues.
Search and learn about the issue of authorship
(sources for this complicated topic are hard to
find, so work hard to collect materials.