PharmacoEconomics (2020) 38:1153–1164

https://doi.org/10.1007/s40273-020-00937-z

PRACTICAL APPLICATION

Estimating Transition Probabilities from Published Evidence: A Tutorial

for Decision Modelers
Risha Gidwani1,2,3 · Louise B. Russell4,5

Published online: 14 August 2020

© This is a U.S government work and its text is not subject to copyright protection in the United States; however, its text may be subject to foreign
copyright protection 2020, corrected publication 2020

Abstract
This tutorial presents practical guidance on transforming various typesof information published in journals, or available online
from government and other sources, into transition probabilities for use in state-transition models, including cost-effectiveness
models. Much, but not all, of the guidance has been previously published in peer-reviewed journals. Our purpose is to collect
it in one location to serve as a stand-alone resource for decision modelers who draw most or all of their information from
the published literature. Our focus is on the technical aspects of manipulating data to derive transition probabilities. We
explain how to derive model transition probabilities from the following types of statistics: relative risks, odds, odds ratios,
and rates. We then review the well-known approach for converting probabilities to match the model’s cycle length when
there are two health-state transitions and how to handle the case of three or more health-state transitions, for which the two-
state approach is not appropriate. Other topics discussed include transition probabilities for population subgroups, issues to
keep in mind when using data from different sources in the derivation process, and sensitivity analyses, including the use
of sensitivity analysis to allocate analyst effort in refining transition probabilities and ways to handle sources of uncertainty
that are not routinely formalized in models. The paper concludes with recommendations to help modelers make the best use
of the published literature.

1 Introduction
Metastatic disease to Death. Transition probabilities would
describe the probabilities of moving from Cancer-Free to
A set of health states, or events, and the probabilities of tran-
Local Cancer, from Local to Regional, from Regional to
sitioning from onestate to others during a specified period of
Metastatic, and from any of those states to Death, over, say,
time (“transition probabilities”) are the fundamental building
1 year. Different probabilities would be needed to describe
blocks of decision models. A state-transition model to evalu-
the natural (untreated) course of the disease versus its course
ate cancer interventions, for example, might start with the
with treatment. The yearly time period, called the cycle
Cancer-Free state and proceed through Local, Regional, and
length, would repeat until an appropriate stopping point was
reached. The total number of years (cycles) represents the
* Risha Gidwani model’s time horizon. The usual recommendation is that the
[email protected] time horizon should be long enough to capture all significant
health outcomes, which often requires modeling the remain-
1
Department of Health Management and Policy, UCLA ing lifetimes of patients [1, 2].
School of Public Health, Los Angeles, CA, USA
There are two common challenges a modeler faces
2
Health Economics Resource Center, VA Palo Alto Health when deriving transition probabilities for use in a decision
Care System, Menlo Park, CA, USA
model. One challenge is that the data from the published
3
Center for Innovation To Implementation, VA Palo Alto and publicly available literature, such as data published by
Health Care System, Menlo Park, CA, USA
the United States (US) Census Bureau or the Centers for
4
Department of Medical Ethics and Health Policy, Disease Control and Prevention, are often not reported as
Perelman School of Medicine, University of Pennsylvania,
Philadelphia, USA probabilities. Rather, evidence that is relevant to the deci-
5 sion model may be in the form of counts, rates, relative risks
Center for Health Incentives and Behavioral Economics
and Leonard Davis Institute of Health Economics, University (RRs), or odds ratios (ORs) that need to be converted into
of Pennsylvania, Philadelphia, USA probabilities. A second challenge is that when the evidence

Vol.:(0123456789)
1154 R. Gidwani, L. B. Russell

derivation process. We then discuss how to derive transi-

Key Points tion probabilities from probabilities whose time frames do
not match the model’s cycle length. We explain the well-
A set of health states, or events, and the probabilities of known approach for deriving transition probabilities when
transitioning from one state to others during a specified the model, or model node, has only two state transitions, and
period of time (“transition probabilities”) are the fun- discuss how to handle three or more possible transitions, for
damental building blocks of decision models. These are which the two-state formulas are not appropriate. Lastly, we
often not available in the published literature in a format discuss how to use sensitivity analyses to allocate analyst
directly suitable for use in decision models. effort to refine probabilities and ways to handle sources of
Procedures for estimating transition probabilities from uncertainty that are not routinely formalized in models. The
published evidence, including deriving probabilities paper concludes with recommendations to help the modeler
from other types of summary statistics and modifying make the best use of the published literature.
the time frame to which a probability applies, have been
discussed in disparate places in the literature.
2 The Published Evidence
This tutorial article aggregates this information in one
location, to serve as a stand-alone resource for the deci- The International Society for Pharmacoeconomics and Out-
sion modeler. The information is meant to assist decision comes Research (ISPOR)—Society for Medical Decision
modelers in the practical tasks of building high-quality Making (SMDM) Modeling Task Force recommends that
decision models. “[t]ransition probabilities and intervention effects should be
derived from the most representative data sources for the
decision problem” [3]. Although modelers may find all the
is expressed as probabilities, the published probabilities will required data in a single published report, more commonly
often not match the cycle length of the decision model. For the data come from multiple studies. Some of it may involve
example, annual probability data may be published, while samples that are small, unrepresentative, or both. Some stud-
the decision model has a 3-month cycle length. ies may be 20 or 30 years old. The task force recommends
This tutorial grew out of a popular VA Health Econom- conducting a systematic literature review when multiple
ics Resource Center (HERC) cybercourse, which generated studies are available to inform the same parameter(s). The
many requests for the slides and suggestions that they be systematic review can be used to select the study that best
written up. As an example, during the first week the World fits the model, or it could be used as the first step in a meta-
Health Organization defined the COVID-19 crisis as a pan- analysis producing a quantitative pooled estimate of the indi-
demic, modelers from a national US agency read the slides vidual study-level outcomes [4].
and reached out to us for guidance in transforming model The task force suggests that transition probabilities for
probability inputs. Responding to those requests, this tuto- the natural history of a condition, sometimes called the “Do
rial presents practical guidance for transforming published Nothing” model arm, are best drawn from population-based
estimates into appropriate transition probabilities. Much of epidemiological studies. Studies with longer follow-up times
the guidance is already available in peer-reviewed journals. are preferred since they allow realistic modeling of the dis-
Our purpose here is to collect it in one location to serve as a ease for a larger portion of the model’s time horizon. In cir-
stand-alone resource for the decision modeler. Our intended cumstances where the model contains interventions from all
audience is decision modelers who find most or all of their arms of a single randomized controlled trial (RCT), the trial’s
information in the published literature. The principles pre- control arm can be used to represent natural history, but that
sented to transform summary statistics into probabilities is less desirable because trial participants are often selected
apply more widely, but for simplicity, we talk about state- using criteria that make them unrepresentative of the popu-
transition models, often called Markov models. We focus lation to which an intervention will be applied in practice.
on the technical aspects of manipulating published data to For the model’s intervention arms, RCTs represent the
derive transition probabilities and touch only lightly on how highest-quality evidence of efficacy, since properly conducted
to select the most appropriate evidence. randomization balances measured and unmeasured confound-
The paper begins by outlining the types of evidence ers across treatment and control groups. The generalizability
available in the published literature. We first discuss how to problem, however, arises here as well. RCTs may not repre-
derive transition probabilities from common types of sum- sent effectiveness in real-world practice for at least two, pos-
mary statistics, such as RRs, odds, and ORs, and issues to sibly offsetting, reasons: (1) trialists work hard to maintain
keep in mind when using data from different sources in the the quality and consistency of an intervention and to keep
patient adherence high, while compliance in actual practice
Guidance for Model Transition Probabilities 1155

Table 1 Common forms of Statistic Evaluates Range

published data and their
definitions Probability/risk #of events that occurred in a time period 0–1
#of people followed for that time period
Rate #of events that occurred in a time period 0 to ∞
Total time period experienced by all subjects followed
Relative risk Probability of outcome in exposed 0 to ∞
Probability of outcome in unexposed
Odds Probability of outcome 0 to ∞
1 − Probability of outcome
Odds ratio Odds of outcome in exposed 0 to ∞
Odds of outcome in unexposed

cycle length. Or it may be published in another form that can

be used to derive transition probabilities—RRs, ORs, and the
like—in which case the modeler needs to know best practices
may be lower, reducing the intervention’s effectiveness; and for deriving probabilities and be aware of pitfalls and limitations.
(2) control groups may benefit from the placebo effect of Note that some valuable information that is available when all
participating in a trial, raising the possibility that the inter- the data come from a single study and individual-level data are
vention will be more effective in practice than in the trial. available—such as correlations between probabilities—is not
In a model that compares interventions with each other, available when the data come from different studies.
but has no “Do Nothing” or placebo arm, model data should
maintain randomization within RCTs. For example, suppose
the model compares interventions A, B, and C and there are 3 Model Time Horizon and Cycle Length
separate RCTs providing treatment efficacy data for each
intervention. If each RCT compares active treatment to pla- The modeler must choose an appropriate cycle length for
cebo, using the relative treatment effect of A versus placebo the model. Shorter cycles yield more accurate estimates of
from the first RCT when populating efficacy for A, the relative life expectancy and costs, but at higher computational cost.
treatment effect of B versus placebo from the second RCT When the required cycle length changes over the disease/
when populating efficacy for B, and so on, maintains rand- intervention pathway, say because events can occur quickly
omization within trials. This approach is still subject to some in the first month after diagnosis or surgery but are much
bias, as patients have not been randomized across studies. less frequent over the longer term, the model may begin with
The best approach to maintain randomization within a decision tree, to represent events that can occur within days
RCTs is to conduct a network meta-analysis (also sometimes or weeks, and shift to Markov nodes based on a cycle length
referred to as an indirect treatment comparisons analysis) to of, for example, 1 year to represent longer-term events [2].
derive appropriate transition probabilities. A network meta- The discount rate recommended by the Second Panel
analysis can generate the relative treatment effects of two on Cost-Effectiveness in Health and Medicine is 3% per
or more interventions, such as A, B and C, when the inter- year [10]. When the model does not have an annual cycle
ventions have not been directly compared in a single RCT: length, the discount rate must be modified using the formula
A versus B, A versus C, and B versus C. If conducted in a (1 + annual rate)1/t, where t is the number of model cycles in
Bayesian framework, it provides probabilities for direct use a year, so that it remains 3% across a 1-year period [11]. If
in a model. Network meta-analytic estimates are less sub- the model has a 1-month cycle length, for example, and a 3%
ject to selection bias than estimates from observational data annual discount rate, the formula yields a monthly discount
because relative treatment effects are calculated within RCTs rate of 0.247% [derived from (1 + 0.03)1/12].
prior to the comparison across treatments; again, within-trial Often, the model’s time horizon exceeds the follow-
randomization is maintained. For more information about up time of published studies. For example, a clinical trial
network meta-analyses, see the following excellent introduc- evaluating drug efficacy may have a follow-up period of
tory articles: Sutton et al. [5], Jansen et al. [6], Jansen et al. only 2 years, while the model has a 30-year time horizon.
[7], Hoaglin et al. [8], and Welton et al. [9]. There is a large amount of literature on the many methods
Table 1 shows the most common ways of reporting data from available for extrapolating beyond the available data and a
single studies and their definitions. The evidence may be pub- small amount of literature on the success of such extrapola-
lished as probabilities, but need to be converted to the model’s tions [12–16]. For modelers working with publicly available
1156 R. Gidwani, L. B. Russell

data, it may be possible to extrapolate using life tables, or the resulting modeler-derived estimate of p1 to be valid and
mortality rates by cause, available from national vital statis- applicable to the population being modeled.
tics systems. An important issue for such extrapolations, to Of note, when the RR reported in the study has been
which we return in the section on uncertainty analysis, is the adjusted for covariates and the probability of the event in
need to increase the variance around the model’s estimates the unexposed group has not, the denominator of the RR
to include extrapolation-based error. In addition, in models does not cancel out:
with long time horizons, transition probabilities, including ( )
p1_adjusted
probabilities of adverse events, which are too often omitted, p1 ≈ × p0_unadjusted . (3)
usually need to be modified as the cohort ages. p0_adjusted

The modeler may, for lack of other data, use Eq. 3 any-
way, recognizing that there is an unknown degree of error in
4 Relative Risks and Odds Ratios
the resulting estimate of p1. A plausible range of values for
p1 should be tested in sensitivity analyses to determine the
As noted, published information about disease burden and
likely importance of this error to the analysis.
treatment efficacy is often summarized in the form of RRs or
ORs. Here we discuss how to derive transition probabilities
from these statistics.
4.2 Using Relative Risks to Derive Subgroup
Transition Probabilities
4.1 Using Relative Risks to Derive Transition
Probabilities are often available for a population, but not
Probabilities for the Treated Group
for subgroups that are important for the model. RRs can
help in this situation because a population probability is a
Investigators often compare the probability of an event in
weighted average of subgroup probabilities and RRs pro-
people exposed to an intervention or condition to the proba-
vide the weights. To illustrate, an analysis of the cost-effec-
bility in those not exposed—the probability of lung cancer in
tiveness of maternal immunization to prevent pertussis in
smokers versus nonsmokers, for example, or the probability
infants, which compared maternal immunization plus rou-
of heart attacks in those who take statins versus those who
tine infant vaccination with routine infant vaccination alone,
do not. A relative risk (RR) (also called a risk ratio) is the
required probabilities of pertussis death in infants by vac-
ratio formed by the probability of the event in the exposed
cination status and age group [17]. Probabilities of pertussis
group divided by the probability of that same event in the
death for infants aged 0–1, 2–3, 4–5, 6–8, and 9–11 months
unexposed group:
were available from Brazilian mortality and hospitalization
p1 data systems. Probabilities that infants in each of those age
RR = , (1)
p0 groups had received no, one, or two to three doses of vac-
cine were modeled from survey data [18]. But probabilities
where p1 is the probability of the event in exposed persons, of pertussis death by vaccination status, needed to estimate
and p0 is the probability of the event in unexposed persons. the impact of vaccination on pertussis mortality in infants,
When the RR is multiplied by the probability of the event were not available.
in unexposed persons, p0, the denominator of the RR cancels To derive probabilities by vaccination status, the over-
out, leaving the probability of the event in the exposed, p1: all probability of pertussis death in each age group was
( )
p1 expressed as a weighted average of the probabilities of death
p1 = RR × p0 = × p0 . (2) by vaccination status:
p0
( ) ( ) ( )
pmtotal = p0 × pm0 + p1 × pm1 + p23 × pm23 , (4)
Use of Eq. 2 necessitates knowing p0. Often, the probabil-
ity of the event in the unexposed is reported in the same arti- pmtotal is the known probability of dying of pertussis for the
cle that reports the RR. In other cases, this information will age group as a whole; pm0, pm1, and pm23 are the unknown
come from a different source. For example, the probability, probabilities of dying of pertussis for infants who received
p0, that an untreated diabetic person develops diabetic retin- no dose, one dose, or two to three doses of vaccine. p0, p1,
opathy may come from one source (such as the control arm and p23 are the known proportions of children of that age
of an RCT) and the RR of diabetic retinopathy for treatment who had received no dose, one dose, or two to three doses
versus no treatment from another, such as an epidemiologic of vaccine.
study. The modeler needs to decide whether p0 and the RR Multiplying the right-hand side by pm0/pm0 allows the
come from sufficiently similar populations (or whether there equation to be restated in terms of the RRs of death by vac-
is reason to believe the RR is similar in all populations) for cination status:
Guidance for Model Transition Probabilities 1157

( ( ( )) ( ( )))
pm1 pm23 Assume the
pmtotal = pm0 × p0 + p1 × + p23 × , Outcome is yes OR
pm0 pm0 <10% or OR is
approximates
close to 1.0
a RR
or,
( ( ) ( )) no
pmtotal = pm0 × p0 + p1 × RR1 + p23 × RR23 ,
Use Equa on Obtain p0 from
RRs were available from Juretzko et al. [19] (albeit for acel- 7 to convert data sources as
the OR to a RR Mul ply RR by
lular pertussis vaccine, not the whole-cell vaccine used in similar as possible
p0 to obtain p1
to the ones used
Brazil), and p0, p1, and p23 were obtained from Clark using to derive the RR
the methods described in Clark et al. [18], so the equation
could be solved for pm0. Once pm0 is known, the RR equa-
Fig. 1 Deriving a transition probability from a reported OR. OR odds
tions can be used to solve for pm1 and pm23, ratio, p0 probability of the event in unexposed persons, p1 probability
of the event in exposed persons, RR relative risk
RR1 = 0.32 and RR23 = 0.05.

Then, employing Eq. 2:

the coefficients of logistic regressions are logged ORs, which
pm1 = 0.32 × pm0 , can be exponentiated to get ORs. The OR is the odds of the
event in one group, A, divided by the odds of the same event
and, in another group, B:
pm23 = 0.05 × pm0 . oddsA
Odds ratio = . (6)
Another example is given in Black et al. [20], where the oddsB
method is used to derive probabilities of survival for non- ORs can be converted into probabilities using one of two
smokers, former smokers, and current smokers from the methods (Fig. 1). If one of the outcomes is rare (< 10%) and/
2009 mortality tables published by the US National Center or the OR is close to 1.0, the OR is a reasonable approxima-
for Health Statistics. tion to the RR [23] and can be inserted directly into Eq. 2, in
Combining evidence from studies that investigated dif- place of the RR, to derive the probability. To see how well
ferent populations, at different times, and/or under different an OR approximates an RR, readers are referred to Zhang
conditions may produce unrealistic results, a problem that and Yu [23] or Grant [24].
is not always easy to detect. As an example, for an analysis If the OR cannot be used to approximate the RR, the RR
of smoking cessation, survival probabilities for smokers, by can be derived from the OR using the following equation
sex, ethnicity, and age, were derived from US life tables. [23, 24]:
Survival probabilities for quitters were initially estimated by
applying quitter/smoker survival ratios by age from another OR
RR = ( ( )) , (7)
study [21] to the probabilities for smokers, but the resulting 1 − p0 + p0 × OR
estimates exceeded 1.0 at some ages in some sex–ethnicity
groups. where p0 is the probability of the event in the unexposed
group.
4.3 Deriving Relative Risks from Odds Ratios As the equation shows, the same OR produces different
RRs depending on p0, the probability of the event in the
The odds of an event, defined as the probability of the event, unexposed group. For example, an OR of 1.5 yields an RR
p, divided by 1 minus the probability of that event, can also of 1.429 when p0 is 0.1, 1.250 when p0 is 0.4, 1.154 when
be used to derive a probability: p0 is 0.6, and 1.071 when p0 is 0.8 [24]. Thus, as the prob-
ability of the event in the unexposed group increases, the OR
p
Odds = . (5) becomes a poorer approximation of the RR.
(1 − p) When the OR comes from a multivariable logistic regres-
For example, the odds that a mother reported post-partum sion, as it often does, there is no single baseline risk. Within
depression after a live birth in the USA were 0.13 in 2012. a single regression, the baseline risk depends on the values
Thus, the probability of reporting post-partum depression of the covariates, and there are as many baseline risks as
was 0.115 [22]. there are combinations of covariate values; for example,
Odds are not commonly reported, but are the basis for a one baseline risk for a smoker with low blood pressure and
frequently encountered summary statistic, the OR, because another baseline risk for a smoker with high blood pressure.
Regressions based on the same dataset but with different
1158 R. Gidwani, L. B. Russell

covariates will yield different baseline risks. And, of course, 5.1 The Probability‑Rate Equations when there are
regressions that use different datasets will produce different Two State Transitions
baseline risks [25]. In principle, it is possible to calculate
an average baseline risk for a specific regression by insert- Equations 8 and 9 show the relation between a probability
ing the means of the covariates into the published logis- (p), rate (r), and time (t) [11, 26–28].
tic regression [24], or to calculate a baseline risk that best
represents the model population by specifying appropriate −ln(1 − p)
r= , (8)
values for the covariates. In practice, this is usually not pos- t
sible because even when authors publish the coefficients for
all covariates, they rarely include the value of the intercept, p = 1 − exp(−rt). (9)
which is also needed.
To convert a probability from one time frame to another,
If the modeler does not have access to the complete
the modeler can use Eqs. 8 and 9, which are the ones most
original regression, Grant recommends establishing a range
frequently found in published articles, or the equivalent
of baseline risks and calculating the corresponding range
Eq. 10 [11].
of RRs, then conducting one-way sensitivity analysis to
determine the influence of that range on the model’s results p = 1 − (1 − p)1∕n . (10)
[24]. A plausible range can be based on previous published
research or on expert opinion. The Appendix demonstrates both approaches for a hypo-
thetical example.
As a real-world example, consider the 12-month prob-
5 Converting Probabilities to the Model’s ability, 10.8%, that a child under age 6 living in Milwaukee,
Cycle Length Wisconsin is newly diagnosed with elevated blood lead lev-
els (defined as ≥ 5 mcg/dL of blood) in 2016 [29]. If the
Once the evidence is in the form of probabilities, it may need model has a 3-month cycle length, a 3-month probability is
to be converted to the model’s cycle length. For example, a needed. Using Eq. 8, we convert the 12-month probability
trial may report outcomes at 2 years’ follow-up, while the to a 12-month rate. Since the time period does not change,
model has an annual cycle length. For a model node with the denominator is 1,
only two branches, that is, two possible state transitions, the −ln(1 − 0.108)
relationship between probabilities and rates provides a sim- 12 month rate = = 0.114289.
1
ple way to derive probabilities that match the model’s cycle
length. Recall that a probability is the number of events in a Next, using Eq. 9, we convert this 12-month rate to a
time period divided by the total number of people followed 3-month probability,
for that time period, and ranges from 0 to 1.0. A rate is the
1
( )
number of events divided by the total time at risk experi- 3 month probability = 1 − exp −0.114289 ×
4
enced by all people followed, and ranges from 0 to infinity.
= 1 − 0.97183 = 0.0282.
Thus, probabilities and rates for the same event are differen-
tiated by their denominators: the calculation of a rate takes The 3-month probability is thus 0.0282 (alternatively, we
into account the time spent at risk, while the calculation could have converted the 12-month probability to a 3-month
of a probability does not [26]. See Appendix for a detailed rate, and then the 3-month rate to a 3-month probability).
example and the assumptions involved in the formula. Using Eq. 10 yields the same 3-month probability:

Table 2 Markov model of A B C D

elevated lead levels
Children tested Elevated lead Cumulative elevated Sum of persons in all
levels lead levels health states by cycle
(A + C)

Baseline 10,000.00 0 0 10,000.00

End of cycle 1 9718.32 281.68 281.68 10,000.00
End of cycle 2 9444.58 273.75 555.42 10,000.00
End of cycle 3 9178.54 266.03 821.46 10,000.00
End of cycle 4 8920.00 258.54 1080.00 10,000.00
Guidance for Model Transition Probabilities 1159

Table 3 Example of the three-state problem

Health states for 124 patients with severe congestive heart failure who were good candi-
dates for heart transplant
Good candidate (gc)b Transplant (t) Dead (d)

Panel A: The transition probability matrixa

Good candidate (gc) 1 − pgct − pgcd pgct pgcd
Transplant (t) 0 1 − ptd ptd
Dead (d) 0 0 1
Panel B: Study data and (incorrect) transition probabilities derived by the two-state formula for the first row of the transition matrix (panel A),
for the study cohort of 124 people
Study outcomes at 3 years 9 92 23
3-year probability from study 0.0726 0.7419 0.1855
Annual probability, derived using the two-state c 0.3633 0.0661
formula
Projection end of year Good candi- Transplants (B) Cumulative trans- Dead (D) Cumulative Sum of
dates (A) plants (C) dead (E) health states
(A + C + E)

Panel C: Results of a Markov model using the annual two-state probabilities in panel B to project 3 years of outcomes for a cohort of 124
personsd
1 70.7494 45.0543 45.0543 8.1963 8.1963 124
2 40.3667 25.7061 70.7604 4.6765 12.8728 124
3 23.0316 14.6669 85.4273 2.6682 15.5411 124
Correct numbers 9 92 23 124

Source: Anguita 1993 [31], Fig. 1

a
The rows represent the state a person can transition “from”; the columns represent the state a person can transition “to”
b
pgct proportion of good candidates who receive a transplant during cycle t, pgcd proportion of good candidates who die from all-cause mortality
during cycle t, ptd proportion of transplanted patients who die during cycle t
c
This probability is not derived from the formula. It is 1 minus the probabilities of transitioning to transplant or dead. See panel A, first row
d
Applying annual transition probabilities derived by the two-state formula results in incorrect numbers experiencing health-state transitions, as
shown by the differences between the last two rows: 23 projected good candidates vs. the correct 9, 85.4 transplants vs. 92, and 15.5 deaths vs.
23 (see the values formatted in bold)

3 month probability = 1 − (1 − 12 month p)1∕n transplant and followed for 3 years, there are three possible
1∕4 transitions: remain a good candidate, receive a transplant, or
= 1 − (1 − 0.108) = 1 − 0.8921∕4
die. The two-state formula will give incorrect annual transi-
= 1 − 0.97193 = 0.0282. tion probabilities for this row.
The 3-month rate can be verified by using it to calculate Panel B shows the study data applicable to the first row of
the probability that a child will be diagnosed over a year the transition matrix; study outcomes, 3-year probabilities
(Table 2, especially column C, end of cycle 4). calculated from the study outcomes, and (incorrect) annual
probabilities derived from the 3-year probabilities using the
5.2 Changing Cycle Length When There are Three two-state method. Panel C shows the results of a Markov
or More State Transitions model that used the incorrect annual probabilities to pro-
ject health outcomes for a baseline cohort of 124 patients
The conversion procedure for two state transitions (124 chosen to match the source study and facilitate com-
(Eqs. 8–10) does not yield correct probabilities when three parisons). The correct numbers, calculated using the 3-year
or more state transitions can occur in a cycle, a common probabilities from the study, are shown in bold in the last
situation [11, 26, 28, 30]. The problem is illustrated in row of panel C. The annual probabilities derived using the
Table 3, which is based on a study of patients with severe two-state method substantially overestimate the number of
congestive heart failure evaluated for a heart transplant [31]. good candidates remaining at 3 years and underestimate the
Panel A depicts the transition probability matrix of a Markov other two health states.
model. Among those considered good candidates for heart There are three possible solutions to the problem of
deriving model transition probabilities when more than two
1160 R. Gidwani, L. B. Russell

Survive resulting probabilities are not a major driver of the model

results, modelers may wish to consider using this approach.
Transplant

Heart Transplant Die

Indicated 6 Sensitivity Analyses
Survive

No Transplant The purpose of sensitivity analyses is to understand how

uncertainty about parameter values, including transition
Die
probabilities, affects a model’s results. Each parameter has
some error, which should be represented and evaluated in the
Fig. 2 Conditional nodes for decision models decision model [1]. The error is represented by a plausible
range around the base-case value, based on 95% confidence
intervals or expert opinion if formal estimates of uncertainty
transitions can occur within a cycle. The first is to revise the are lacking. Arbitrary ranges should be avoided. If the base-
model structure so that each node has only two branches case estimate of a parameter required conversion from the
(two transitions). This would be easy to do in the heart trans- study time period to the model’s cycle length, the same pro-
plant case if the diagnostic pathway led from “good candi- cedure can be used to transform the bounds of its range to
date/heart transplant indicated” to transplant/no transplant, the appropriate cycle length. For example, the upper and
and then each of those branches led to survive/die (Fig. 2). lower bounds of the reported 95% confidence interval for
If the model cannot easily be reduced to a series of two- a probability can also be transformed to the model’s cycle
state branches, because of the nature of the states or because length using Eqs. 8–10.
it becomes too “bushy,” eigendecomposition methods offer The two main ways to evaluate the effects of uncertainty
a second possible solution. Eigendecomposition consists on model results are deterministic sensitivity analyses and
of decomposing the transition probability matrix into a probabilistic sensitivity analyses (PSAs). Deterministic sen-
set of eigenvectors and eigenvalues [11, 28, 30, 32]. To sitivity analyses entail varying the value of one parameter
employ eigendecomposition, a modeler must have a single (one-way sensitivity analysis) or a few parameters (multi-
data source to inform the transitions from an initial health way sensitivity analysis) while holding all other parame-
state, or have data from multiple studies that each have the ters at their base-case values. A series of one-way sensi-
same follow-up time (Chhatwal, personal communication). tivity analyses plotted in a tornado diagram shows which
These conditions are rarely met. For example, a rheumatoid parameters have the most influence on the model’s results.
arthritis model in which treatment efficacy and side effects If the ranges in the tornado diagram reasonably represent
are obtained from a single RCT may still require all-cause the parameters’ uncertainty, the diagram points to the most
mortality data from another source due to the short follow- influential parameters. These are the parameters that require
up time of the trial. If this second source has a different the most attention and effort to reduce uncertainty and the
follow-up period, the eigendecomposition approach cannot possibility of bias in the model’s results. Thus, sensitivity
be applied. For this reason, eigendecomposition may also be analyses conducted early in model development can be a
inapplicable for models in which scenario analyses are con- useful guide for allocating effort for further refinement of
ducted on cohort subgroups, with multiple sources providing parameters.
probability data. In addition, technical problems having to PSAs entail replacing the base-case values for all model
do with the nature of the data may make eigendecomposition parameters with probability distributions. Each distribution
impossible or produce values that do not meet the conditions represents the range of values the parameter can take and the
required of transition probabilities. For example, eigende- likelihood of each value. The model is run multiple times,
composition may produce negative numbers or complex say 1000 times, and each iteration plucks a new set of param-
numbers (combinations of real and imaginary numbers). eter values from the distributions. The results for the 1000
A full exposition of these complex methods is beyond the iterations show the uncertainty in the output, such as the
scope of this paper. Interested readers can consult the arti- cost-effectiveness ratio, due to uncertainty in all parameters.
cles cited earlier in this paragraph [11, 28, 30, 32]. Since PSAs vary all parameters simultaneously, some transi-
Lastly, in cases where (1) there are only three health state tion probabilities may need to be linked to ensure that the
transitions possible, (2) two of the published probabilities values selected in a single iteration are congruent. Model-
are very small, and (3) the model cycle length is shorter than ers who are running their own regressions to obtain model
the published probability, the error in using the two-state for- input parameters can use the variance–covariance matrix to
mula to convert probabilities to the appropriate cycle length specify the correlation between two or more model param-
will be small. If all three of these conditions are met, and the eters [33]. When the modeler does not have access to the
Guidance for Model Transition Probabilities 1161

individual-level data, this linkage can be done through other models for extrapolation [34]. Modelers who are limited to
mechanisms. For example, in a model evaluating multiple the published data will not be able to use these approaches,
lines of treatment for cancer, it may be necessary to link the but should consider widening the range around extrapolated
probability of response to second-line treatment to the prob- probabilities to reflect the additional uncertainty associated
ability of response to first-line treatment, perhaps by defin- with extrapolation. The importance of the problem is illus-
ing the probability for second-line treatment as a fraction of trated by an analysis of artificial hips that found extrapola-
the probability for first-line treatment. If the two values are tions based on the 8 years of follow-up data available at the
left independent, the PSA can produce implausible model time of the original analysis turned out, once 16 years of
iterations. follow-up became available, to have identified the wrong
Another source of uncertainty derives from the time artificial hip as the most successful and cost-effective [15].
frame of the original statistic. Consider again the probability
of new mothers reporting post-partum depression symptoms,
11.5%. This outcome was reported for a mean follow-up 7 Discussion
time of 125 days, approximately 4 months, after delivery
[22]. To derive a transition probability, the modeler may There are many complex issues to be addressed in the pro-
treat the value of 11.5% as the probability for 4 months and cess of developing a decision model. Here, we summarize
use Eqs. 8 and 9, or Eq. 10, to transform it to the cycle length some best practices for using data from the published litera-
of the model. But because not everyone was followed for at ture that may mitigate downstream challenges.
least 4 months (the mean follow-up was 4 months), this is First, as Miller and Homan warned more than 2 decades
not correct, and the probability has not only the usual sam- ago, statistics are not always correctly described in the origi-
pling error, but also an additional error associated with the nal source [27]. Modelers should carefully review whether
time frame. While this remains an area for future research, the reported statistic is actually a probability, an RR, a rate,
modelers should test the impact of this measurement error or something else; sometimes statistics reported as rates are
by conducting a series of one-way deterministic sensitiv- actually probabilities. As noted earlier, one clue to the dif-
ity analyses using values associated with varying the time ference is that a rate has time at risk explicitly stated in the
frame—for example, transition probabilities derived using denominator, (e.g., ten events per 100 person-years) while
the mean, median, minimum, and maximum follow-up times a probability does not (e.g., ten events per 100 persons).
reported for the statistic. If the model results are sensitive Another clue is that for a probability, persons must be fol-
to the difference, modelers may wish to contact the study lowed for the entire time period, whereas for a rate, persons
investigators for data stratified by follow-up time or explore are followed only until the event occurs [26].
alternative sources of data. Once the published data relevant to the decision model
An important issue, for which there is as yet no good are correctly identified, the methods described in this paper
solution, is how to represent the uncertainty involved in can be used to derive transition probabilities appropriate to
extrapolating transition probabilities beyond the time hori- the model. Our purpose here has been to collect the meth-
zon of the available data. When the modeler has access to ods available in the literature in a single place to make the
the original data, the standard approach is to fit a variety process of derivation easier for modelers. We have described
of parametric models to the data, and, using a goodness- how RRs can be used to derive transition probabilities for
of-fit statistic such as the Akaike and/or Bayesian informa- disease or for treatment efficacy, or can serve as weights
tion criterion, choose the best-fitting model to extrapolate for deriving transition probabilities for population sub-
beyond the original data, even though goodness-of-fit to groups. We described how to derive probabilities from the
the observed data is not an appropriate test of the fitted frequently-reported OR, including in situations where the
model’s ability to extrapolate accurately. Negrin et al. [34] event is not rare. Probabilities derived from summary sta-
and Latimer [16] suggest conducting sensitivity analyses by tistics such as RRs or ORs will be affected by the accuracy
comparing, say, the cost-effectiveness results for the best- and suitability of additional elements required by the deriva-
fitting model with results based on those that fit less well. tion, such as the probability of an event in the unexposed,
This approach shows whether an intervention deemed cost- and we have discussed how modelers can incorporate the
effective (or not) using the best-fitting model remains so uncertainty introduced by these elements.
when other models are used and focuses attention on the We have discussed several types of statistics that are of
effects of the extrapolation method on decision uncertainty. direct use for estimating transition probabilities for decision
Another approach is to fit parametric models to different models. There are other statistics that, while not directly
subperiods within the observed data to explore the stability useful, are excellent leads to sources of the statistics needed
of the estimated parameters and, rather than using the single for models. The population attributable fraction (PAF) is
best-fitting model, use Bayesian model averaging to combine one such example [35, 36]. Since it shows the maximum
1162 R. Gidwani, L. B. Russell

amount of disease or mortality that can be attributed to a 8 Conclusion

condition, such as obesity [37], it is not directly useful for
cost-effectiveness analysis models, which evaluate specific Decision modelers populating their models with transition
interventions and need measures of the effectiveness of those probabilities based on published data face numerous chal-
interventions. However, the calculation of PAF is based on lenges, ranging from finding only comparative statistics
prevalence and risk ratios, from which transition probabili- (such as RRs or ORs) from which to derive probabilities to
ties can be derived, so articles about PAFs can lead to good needing to convert published data to match the model’s cycle
sources for those statistics. They also often provide helpful length. We present here guidance, based on current thinking
discussions of the appropriateness and consistency of the and literature, to help modelers populate their models with
statistics for particular populations, so can help the modeler high-quality transition probabilities.
decide which statistics to use in the model.
Modelers will often need to modify a published prob- Author contributions This paper was conceptualized by RG, who
ability to match the cycle length of the model. When there wrote the first draft. Both authors critically revised the manuscript,
often focusing on different parts. RG and LBR contributed equal intel-
are only two possible transitions within a model cycle, the lectual effort towards this article.
conversion is straightforward, as described in Eqs. 8 and 9
or Eq. 10. When three or more transitions can occur within a Disclosures
single model cycle, modelers can avoid the need to use more
Funding No financial support was provided for this study.
complex methods for deriving appropriate probabilities by
creating two-branch, conditional nodes, as suggested for the
Conflict of Interest The authors, Risha Gidwani and Louise B. Russell,
heart transplant example. In other cases, such conditional declare that there is no conflict of interest.
nodes may not be appropriate, or may result in a model with
a confusingly large number of branches, and modelers can Ethics approval Not applicable.
instead consider eigendecomposition to obtain model tran-
Consent to participate Not applicable.
sition probabilities for nodes with three or more branches.
Regardless of the approach used, the resulting probability Consent for publication Not applicable.
values are estimates only and therefore contain uncertainty
additional to that which is present in reported probabili- Availability of data and material Not applicable.
ties, which should be considered in the analysis. For fur- Code availability Not applicable.
ther reading on the appropriateness of creating conditional
two-branch nodes, see Sendi and Clemen, who point out
that two-branch nodes can sometimes complicate sensitivity
analyses [38]. Appendix: Probabilities and rates
Aspects of the model’s structure can be chosen to accom-
modate the available data or to simplify the process of using Suppose a study followed 100 people with congestive heart
it. Modelers may choose, for example, to match the model’s failure for 4 years. At the end of the 4 years, 40 had died.
cycle length to the follow-up time for the data considered The probability of death over 4 years is 40/100 = 0.40.
most important to model results. Whatever the cycle length, A rate takes into account the time each person was at
the discount rate needs to be adapted to match it. risk. The 60 people who survived were at risk the entire 4
Sensitivity analyses are a standard part of reporting model years and contributed 60 × 4 = 240 years at risk. Once a per-
results. They can also be useful early in model development son dies, he/she is no longer at risk. When a study does not
to help allocate effort to the refinement of parameters. Deter- report time at risk, the conventional assumption is that the
ministic sensitivity analyses and, specifically, the tornado events were spread evenly over the time period. Using this
diagram produced from a series of one-way deterministic assumption, the average time at risk for the 40 who died was
sensitivity analyses are an excellent mechanism for deter- 2 years, adding 40 × 2 = 80 years at risk. Total time at risk
mining the importance of model parameters to results. An for the cohort of 100 people is 320 person-years, and thus,
iterative process may be useful in which plausible place- the rate is 40/320 = 0.125 deaths per person-year at risk.
holder values are entered, a tornado diagram is run, and the Suppose the decision model requires an annual probabil-
results are used to allocate further effort in proportion to ity of death. Starting with the published 4-year probability,
each parameter’s effects on the model. Attempting to derive use Equation 8 to convert the probability to an annual rate on
“perfect” probabilities for every branch in the model delays the assumption that the deaths occurred evenly throughout
the model’s completion while adding little to its ultimate the study period. (Note that the answer is close to the manu-
quality and usefulness. ally calculated rate above, but not quite the same.)
Guidance for Model Transition Probabilities 1163

−ln(1 − p) −ln(1 − 0.40) 9. Welton NJ, Caldwell DM, Adamopoulos E, et al. Mixed treat-
r= = = 0.127706 deaths per person-year. ment comparison meta-analysis of complex interventions: psycho-
t 4
logical interventions in coronary heart disease. Am J Epidemiol.
Then use Eq. 9 to convert this rate to a probability. 2009;169(9):1158–65.
10. Basu A, Ganiats TG. Discounting in cost-effectiveness analysis.
p = 1 − exp (rt) = 1 − exp (−0.127706 × 1) In: Neumann PJ, Sanders GD, Russell LB, Siegel JE, Ganiats TG,
editors. Cost-effectiveness in health and medicine. 2nd ed. New
= 0.119888, annual probability. York: Oxford University Press; 2016. p. 277–288.
11. Chhatwal J, Jayasuriya S, Elbasha EH. Changing cycle lengths
Since Eq. 8 already divided by t = 4 to adjust to 1 year, in state-transition models: challenges and solutions. Med Decis
t = 1 in Eq. 9. (Alternatively, the 4-year probability could Making. 2016;36(8):952–64.
first be converted to a 4-year rate and then the 4-year rate to 12. Guyot P, Ades AE, Beasley M, et al. Extrapolation of survival
curves from cancer trials using external information. Med Decis
an annual probability. In this case, t = 1 in Eq. 8 and t = ¼
Making. 2017;37(4):353–66.
in Eq. 9. The results are the same.) 13. Jackson C, Stevens J, Ren S, et al. Extrapolating survival from
Instead of using Eqs. 8 and 9, Eq. 10 yields the same randomized trials using external data: a review of methods. Med
annual probability in a single step, Decis Making. 2017;37(4):377–90.
14. Hawkins N, Grieve R. Extrapolation of survival data in cost-effec-
p = 1 − (1 − p)1∕n = 1 − (1 − 0.40)1∕4 tiveness analyses: the need for causal clarity. Med Decis Making.
2017;37(4):337–9.
= 0.119888, annual probability. 15. Davies C, Briggs A, Lorgelly P, et al. The "hazards" of extrapolat-
ing survival curves. Med Decis Making. 2013;33(3):369–80.
That this is the correct annual probability can be verified 16. Latimer NR. Survival analysis for economic evaluations along-
by using it to calculate the number of survivors at 4 years. side clinical trials–extrapolation with patient-level data: incon-
sistencies, limitations, and a practical guide. Med Decis Making.
Year 1 ∶ 100 × (1 − 0.119888) = 88.011174 2013;33(6):743–54.
17. Russell LB, Pentakota SR, Toscano CM, et al. What pertussis
Year 2 ∶ 88.011174 × (1 − 0.119888) = 77.459667 mortality rates make maternal acellular pertussis immunization
Year 3 ∶ 77.459667 × (1 − 0.119888) = 68.173162 cost-effective in low- and middle-income countries? A decision
analysis. Clin Infect Dis. 2016;63(suppl 4):S227–S23535.
Year 4 ∶ 68.173162 × (1 − 0.119888) = 60. 18. Clark A, Sanderson C. Timing of children’s vaccinations in 45
low-income and middle-income countries: an analysis of survey
data. Lancet. 2009;373(9674):1543–9.
19. Juretzko P, von Kries R, Hermann M, et al. Effectiveness of acel-
References lular pertussis vaccine assessed by hospital-based active surveil-
lance in Germany. Clin Infect Dis. 2002;35(2):162–7.
20. Black WC, Gareen IF, Soneji SS, et al. Cost-effectiveness of CT
1. Neumann PJ, Sanders GD, Russell LB, et al. Cost-effectiveness in screening in the National Lung Screening Trial. N Engl J Med.
health and medicine. 2nd ed. New York: Oxford University Press; 2014;371(19):1793–802.
2016. 21. Jha P, Ramasundarahettige C, Landsman V, et al. 21st-century
2. Siebert U, Alagoz O, Bayoumi AM, et al. State-transition mod- hazards of smoking and benefits of cessation in the United States.
eling: a report of the ISPOR-SMDM Modeling Good Research N Engl J Med. 2013;368(4):341–50.
Practices Task Force–3. Value Health. 2012;15(6):812–20. 22. Ko JY, Rockhill KM, Tong VT, et al. Trends in postpartum depres-
3. Caro JJ, Briggs AH, Siebert U, et al. Modeling good research sive symptoms—27 states, 2004, 2008, and 2012. MMWR Morb
practices–overview: a report of the ISPOR-SMDM Mod- Mortal Wkly Rep. 2017;66(6):153–8.
eling Good Research Practices Task Force–1. Value Health. 23. Zhang J, Yu KF. What’s the relative risk? A method of correcting
2012;15(6):796–803. the odds ratio in cohort studies of common outcomes. JAMA.
4. Borenstein M, Hedges LV, Higgins JPT, et al. Introduction to 1998;280(19):1690–1.
meta-analysis. Chichester, West Sussex, United Kingdom: Wiley; 24. Grant RL. Converting an odds ratio to a range of plausible rela-
2009. tive risks for better communication of research findings. BMJ.
5. Sutton A, Ades AE, Cooper N, et al. Use of indirect and mixed 2014;348:f7450.
treatment comparisons for technology assessment. Pharmacoeco- 25. Norton EC, Dowd BE. Log odds and the interpretation of logit
nomics. 2008;26(9):753–67. models. Health Serv Res. 2018;53(2):859–78.
6. Jansen JP, Crawford B, Bergman G, et al. Bayesian meta-analysis 26. Fleurence RL, Hollenbeak CS. Rates and probabilities in eco-
of multiple treatment comparisons: an introduction to mixed treat- nomic modelling: transformation, translation and appropriate
ment comparisons. Value Health. 2008;11(5):956–64. application. Pharmacoeconomics. 2007;25(1):3–6.
7. Jansen JP, Fleurence R, Devine B, et al. Interpreting indirect treat- 27. Miller DK, Homan SM. Determining transition probabilities: con-
ment comparisons and network meta-analysis for health-care deci- fusion and suggestions. Med Decis Making. 1994;14(1):52–8.
sion making: report of the ISPOR Task Force on Indirect Treat- 28. Jones E, Epstein D, Garcia-Mochon L. A procedure for deriving
ment Comparisons Good Research Practices: part 1. Value Health. formulas to convert transition rates to probabilities for multistate
2011;14(4):417–28. Markov models. Med Decis Making. 2017;37(7):779–89.
8. Hoaglin DC, Hawkins N, Jansen JP, et al. Conducting indi- 29. Christensen K, Coons M, Walsh R. 2016 report on childhood lead
rect-treatment-comparison and network-meta-analysis stud- poisoning in Wisconsin. Wisconsin Department of Health Ser-
ies: report of the ISPOR Task Force on Indirect Treatment vices, Division of Public Health, Bureau of Environmental and
Comparisons Good Research Practices: part 2. Value Health. Occupational Health, P-01202–16. 2017. https://www.dhs.wisco
2011;14(4):429–37. nsin.gov/publications/p01202-16.pdf. Accessed 11 Apr 2019.
1164 R. Gidwani, L. B. Russell

30. Craig BA, Sendi PP. Estimation of the transition matrix of a dis- 35. World Health Organization. Metrics: population attributable frac-
crete-time Markov chain. Health Econ. 2002;11(1):33–42. tion (PAF), https://www.who.int/healthinfo/global_burden_disea
31. Anguita M, Arizon JM, Valles F, et al. Influence of heart trans- se/metrics_paf/en/. Accessed 20 May 2020.
plantation on the natural history of patients with severe congestive 36. Greenland S. Concepts and pitfalls in measuring and interpreting
heart failure. J Heart Lung Transpl. 1993;12(6 Pt 1):974–82. attributable fractions, prevented fractions, and causation prob-
32. Welton NJ, Ades AE. Estimation of Markov chain transition prob- abilities. Annals of Epidemiology 2015;25:xs155–161.
abilities and rates from fully and partially observed data: uncer- 37. Flegal KM, Panagiotou OA, Graubard BI. Estimating population
tainty propagation, evidence synthesis, and model calibration. attributable fractions to quantify the health burden of obesity. Ann
Med Decis Making. 2005;25(6):633–45. Epidemiol. 2015;25:201–7.
33. Briggs A, Sculpher M, Claxton K. Decision modelling for health 38. Sendi PP, Clemen RT. Sensitivity analysis on a chance node with
economic evaluation. Oxford: Oxford University Press; 2006. more than two branches. Med Decis Making. 1999;19(4):499–502.
34. Negrin MA, Nam J, Briggs AH. Bayesian solutions for han-
dling uncertainty in survival extrapolation. Med Decis Making.
2017;37(4):367–76.