Health Technology Assessments

and Systematic Reviews

Marcia Reinhart, DPhil CMPP
Principal, Tantalus Medical Communications
Health Technology Assessments
Many definitions

Technology assessment in health care is a multidisciplinary field of policy

analysis. It studies the medical, social, ethical, and economic implications of
development, diffusion, and use of health technology.

International Association of HTA (INAHTA)

A form of policy research that examines short- and long-term

consequences of technology. . . safety, efficacy, patient-reported outcomes,
real-world effectiveness, cost, and cost-effectiveness as well as social,
legal, the application of a health-care ethical, and political impacts.

International Society for Pharmacoeconomics and Outcomes Research

(ISPOR)
Regardless of the technology assessed,
HTAs include similar elements

Clinical
efficacy

HTA Safety

Cost-
effectiveness
HTAs are used to support many health
care decisions

Clinicians and Public and

Hospitals
patients private payers
Prescribing Drug plan Technology
decisions formularies acquisition
Practice Level of Hospital
guidelines coverage formularies
What is 'Market Access'?
Governments/private insurers determine the price
and availability of drugs via formulary designation

Figure: Eichler GH et al. Nat Rev Drug Discov 2010;9:277-91.

Market Access around the world
Just as health care systems vary worldwide, the payer
assessment process differs from country to country
Who conducts and uses HTAs?
National/regional government-based agencies

Independent academic or consulting groups

How do HTAs fit into the decision-
making process?
HTAs as part of the market access continuum

Manufacturer submission HTA agency report Payer

Conducts literature Assesses Makes final

review, economic manufacturer decision on
analysis, etc. submission reimbursement and
Submits HTA-like May conduct availability of drug
document based independent review or device
on agency template or rely on HTAs
from 3rd parties
Submits
recommendation to
payer


Payer often agrees with
HTA agency, although

may be influenced by
additional factors

Systematic review an essential step of
the HTA process
Cochrane Collaboration definition:
attempts to identify, appraise and synthesize
all the empirical evidence that meets pre-
specified eligibility criteria to answer a given
research question. Researchers conducting
systematic reviews use explicit methods aimed
at minimizing bias, in order to produce more
reliable findings that can be used to inform
decision making.
Systematic review: process
1. Define research question
2. Develop study inclusion/exclusion criteria
Patients
Intervention(s)
Comparator(s) PICOS
Outcomes
Study type

3. Develop literature search strategy

Typically conducted across multiple databases
Search strategies should be broad enough to capture all relevant
publications, but narrow enough to avoid excessive irrelevant
information
4. Screen studies for inclusion and collect relevant data
Example acupuncture for the
treatment of fibromyalgia
1. Define research question
Example acupuncture for the
treatment of fibromyalgia
1. Define research question
2. Develop study inclusion/exclusion criteria
Patients
Diagnostic
criteria?
Age? Disease
severity?
Comorbidities?
Ethnicity?
Example acupuncture for the
treatment of fibromyalgia
1. Define research question
2. Develop study inclusion/exclusion criteria
Patients
Intervention(s)
Frequency?

Definition of
acupuncture? Adjunct
therapies?
Example acupuncture for the
treatment of fibromyalgia
1. Define research question
2. Develop study inclusion/exclusion criteria
Patients
Intervention
Comparator(s)
Placebo?

No treatment?
Drugs?

Other TCM
Physical
techniques?
therapy?
Example acupuncture for the
treatment of fibromyalgia
1. Define research question
2. Develop study inclusion/exclusion criteria
Patients
Intervention(s)
Pain reduction?
Comparator(s)
Outcomes
Physical
Quality of life? function?

Days off work?

Adverse events?
Fatigue?
Example acupuncture for the
treatment of fibromyalgia
1. Define research question
2. Develop study inclusion/exclusion criteria
Patients
Intervention(s) RCTs?
Comparator(s)
Outcomes
Study type Non-RCTs? Real-world
evidence?

Economic
evaluations? Quasi-
randomized?
Small group exercise
develop PICOS criteria
Example acupuncture for the
treatment of fibromyalgia
3. Develop literature search strategy
Use both indexing terms (e.g. MeSH) and plain text
terms
Draft searches to identify studies in categories (e.g.
disease terms, intervention terms), combined at end
Disease
terms
 Example acupuncture for the
treatment of fibromyalgia
3. Develop literature search strategy
Combined

Therapy
terms
Example acupuncture for the
treatment of fibromyalgia
3. Develop literature search strategy
Example acupuncture for the
treatment of fibromyalgia
4. Screen studies for inclusion
Data extraction and synthesis: The
meta-analysis
Multiple studies are combined statistically to
reveal the overall effect of an intervention
A summary (pooled) effect estimate is calculated as a
weighted average of the effects estimated in the
individual studies

Individual studies
Box represents estimated effect
Lines represent 95% CI
Size of box represents weighting

Summary intervention effect

Example acupuncture for the
treatment of fibromyalgia
Meta-analysis for mean difference in pain
Risk of bias
A bias is a systematic error, or deviation from the
truth, in results or inferences
Selection, performance, detection, attrition, reporting
Several tools have been developed to assess bias
Role of medical writers in HTAs
NICE single technology appraisal (STA)
Section A Decision problem
1. Description of technology under assessment
2. Context
3. Equality
4. Innovation
5. Statement of decision problem
May include >1 research question
Role of medical writers in HTAs
Systematic review
Meta-analysis
NICE single technology appraisal (STA)
Non-RCT evidence
Section B Clinical and cost effectiveness Safety data

6. Clinical evidence
7. Cost effectiveness
Section C Implementation
8. Assessment of factors relevant to Systematic review for
NHS/other parties published analyses
Report and interpret
9. References results of own economic
10. Appendices model(s)

11. Related procedures for evidence

submission
Example new class of drug to treat
rheumatoid arthritis
Section B Clinical and cost effectiveness
6. Clinical evidence
Example new class of drug to treat
rheumatoid arthritis
Section B Clinical and cost effectiveness
6. Clinical evidence

Results from the systematic review:

Phase III clinical trial of new drug (B) vs placebo (A)
Phase II clinical trial of new drug (B) vs placebo (A)
3 RCTs of competitor (C) vs placebo (A)
2 RCTs of competitor (C) vs other competitor (D)
3 single-arm non-RCTs

How do we use this information to compare the new

drug (B) to competitors C and D??
Network meta-analyses/indirect
treatment comparison
Network meta-analysis can be used to infer the
comparative effectiveness of treatments of interest
that have not been directly compared in trials
Example new class of drug to treat
rheumatoid arthritis
Section B Clinical and cost effectiveness
6. Clinical evidence

Safety data
AEs from RCTs and non-RCTs
Post-marketing surveillance data (if available)
Other sources of real-world evidence
Example new class of drug to treat
rheumatoid arthritis
Section B Clinical and cost effectiveness
6. Clinical evidence
7. Cost effectiveness
The cost-effectiveness plane
A new treatment can be compared with an existing
therapy based on its effect on cost and effect
differences
Cost difference (+)

Dominated Trade-off
Effect Effect
SC
difference difference
() (+)
Trade-off Dominant

Cost difference ()
SC = standard care
Key concept: QALYs

In cost-utility analyses, effectiveness is

measured in quality-adjusted life years (QALYs)
QALYs incorporate both quality and quantity of life
gained from an intervention
QoL

Improvement of Improvement of
quality of life with quantity of life with
new treatment new treatment
QALYs without new
treatment
Estimated using
(standard care)
utility values

Time

Figure: adapted from McCabe C. Hayward Group Ltd. 2009

Key concept: ICER

Incremental cost-effectiveness ratio

costs per outcome (cost-effectiveness analysis) or
costs per QALY (cost-utility analysis)

CostsTreatment CostsStandard care

QALYsTreatment QALYsStandard care = ICER
Key concept: ICER

CostsTreatment CostsStandard care

QALYsTreatment QALYsStandard care = ICER
Total costs Life-years Utility for LYG QALYs
gained (LYG)
Treatment $20,000 7 0.5 3.5
Standard Care $10,000 5 0.6 3.0

Incremental Incremental Incremental Incremental ICER

cost, Treatment vs life-years cost/LYG QALYs cost/QALY
Std. Care
$10,000 2 $5000/year 0.5 $20,000/
QALY
 Optional group exercise
calculate the ICER of our new
arthritis drug
Economic analysis: willingness to
pay threshold
Some countries/health care systems use an
official or unofficial threshold of acceptable ICERs
for new technology assessments
Cost difference (+)
WTP threshold
Reject
Effect Effect
SC
difference difference
() (+)
Accept

Cost difference ()
Example new class of drug to treat
rheumatoid arthritis
An economic model determines that the ICER for the new
drug is $40,000/QALY
If the WTP was $50,000/QALY, this drug would fall into the
range of acceptable cost/QALY
Cost difference (+)
WTP threshold
X
Reject
Effect Effect
SC
difference difference
() (+)
Accept

Cost difference ()
Cost-utility: sensitivity analysis
Calculating costs per QALY requires estimation of
several parameters
utility values
cost of treatment
duration of treatment/patient lifespan (time horizon)

Base case = best estimate of all parameters

Sensitivity analyses test alternative parameter
estimates to assess the range of possible results
One-way sensitivity analysis

Each parameter varied one-at-a-time over

plausible range
base case

Tornado
diagram

ICER ($thousand/QALY)
Figure: Reynolds MR et al. Circ-Arrhythmia Elec 2009;2:36269.
Example new class of drug to treat
rheumatoid arthritis
Sensitivity analysis shows that the new drug may
rise above the WTP threshold in certain
circumstances
Cost difference (+)
X
WTP threshold
X
X XX
Reject X X
Effect Effect
SC
difference difference
() (+)
Accept

Cost difference ()
Other sensitivity analyses used in HTA

Scenario analysis
Similar to a one-way sensitivity analysis, however,
assessments are made by varying multiple parameters at
the same time
Example = best-case and worst-case analyses
Probabilistic sensitivity analysis
Drug X has a 86% probability of falling within the WTP
threshold of $50,000/QALY
More statistically complex; assesses distributions of data
for multiple inputs over the course of multiple simulations
Cost-utility is not the only economic
analysis used in HTAs
Cost-consequences Estimates cost and value of interventions, but leaves it to
analysis the reader to draw conclusions

Cost-minimization Compares input costs, but assumes outcomes are

analysis equivalent (e.g., bioequivalent drug comparisons)

Cost-effectiveness Measures costs in dollars and reports outcomes in natural

analysis health units (e.g., mmHg reduction) or ratios (differences in
cost/difference in outcomes)
Cost-utility analysis Measures outcomes based on years of life and quality of life
obtained with treatment
Cost-benefit analysis Enumerates and compares costs and benefits achieved in
monetary terms
Budget impact Estimates effect of intervention on overall cost to
analysis organization or health plan
Desirable qualities in HTA medical
writers
Well-versed in literature searching
Experience with multiple databases and multiple
interfaces (e.g. PubMed vs Ovid)
Attention to detail screening and data extraction
Understanding of evidence grading systems
Knowledge of statistical methods (e.g. meta-
analysis, NMA, sensitivity analyses)
Understanding of health economic models
Excellent writing skills
Thank you