Clark Et Al (2017) - Transparency About The Outcomes of Mental Health Services (IAPT Approach) An Analysis of Public Data

Download as pdf or txt
Download as pdf or txt
You are on page 1of 8

Articles

Transparency about the outcomes of mental health services


(IAPT approach): an analysis of public data
David M Clark, Lauren Canvin, John Green, Richard Layard, Stephen Pilling, Magdalena Janecka

Summary
Background Internationally, the clinical outcomes of routine mental health services are rarely recorded or reported; Lancet 2018; 391: 679–86
however, an exception is the English Improving Access to Psychological Therapies (IAPT) service, which delivers Published Online
psychological therapies recommended by the National Institute for Health and Care Excellence for depression and December 7, 2017
http://dx.doi.org/10.1016/
anxiety disorders to more than 537 000 patients in the UK each year. A session-by-session outcome monitoring system
S0140-6736(17)32133-5
ensures that IAPT obtains symptom scores before and after treatment for 98% of patients. Service outcomes can then
See Comment page 636
be reported, along with contextual information, on public websites.
Department of Experimental
Psychology, University of
Methods We used publicly available data to identify predictors of variability in clinical performance. Using β regression Oxford, Oxford, UK
models, we analysed the outcome data released by National Health Service Digital and Public Health England for the (Prof D M Clark DPhil,
2014–15 financial year (April 1, 2014, to March 31, 2015) and developed a predictive model of reliable improvement L Canvin BA Hons); The Oxford
Academic Health Sciences
and reliable recovery. We then tested whether these predictors were also associated with changes in service outcome Network, Oxford, UK
between 2014–15 and 2015–16. (Prof D M Clark, L Canvin);
Department of Clinical Health
Findings Five service organisation features predicted clinical outcomes in 2014–15. Percentage of cases with a problem Psychology, Central and North
West London National Health
descriptor, number of treatment sessions, and percentage of referrals treated were positively associated with outcome. Service Trust, London, UK
The time waited to start treatment and percentage of appointments missed were negatively associated with outcome. (J Green PhD); Centre for
Additive odd ratios suggest that moving from the lowest to highest level on an organisational factor could improve Economic Performance,
service outcomes by 11–42%, dependent on the factor. Consistent with a causal model, most organisational factors London School of Economics,
London, UK (R Layard MSc);
also predicted between-year changes in outcome, together accounting for 33% of variance in reliable improvement Department of Clinical,
and 22% for reliable recovery. Social deprivation was negatively associated with some outcomes, but the effect was Educational and Health
partly mitigated by the organisational factors. Psychology, University College
London, London, UK
(Prof S Pilling PhD); and Seaver
Interpretation Traditionally, efforts to improve mental health outcomes have largely focused on the development of Autism Center, Department of
new and more effective treatments. Our analyses show that the way psychological therapy services are implemented Psychiatry, Icahn School of
could be similarly important. Mental health services elsewhere in the UK and in other countries might benefit from Medicine at Mount Sinai,
New York, NY, USA
adopting IAPT’s approach to recording and publicly reporting clinical outcomes.
(M Janecka PhD)
Correspondence to:
Funding Wellcome Trust. Prof David M Clark, University of
Oxford, Oxford, UK
Copyright © The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY 4.0 license. [email protected]

Introduction known as a clinical commissioning group; CCG) now has


In most countries, if you have a mental health problem an IAPT service, which provides psychological treatments
patients cannot obtain information about the clinical in line with the stepped-care clinical guidelines2,3 issued by
outcomes achieved by the psychological therapy service the UK’s National Institute for Health and Care Excellence For more on NICE see
that they might be considering for treatment. Additionally, (NICE). For all anxiety disorders, cognitive behavioural https://www.nice.org.uk

when a service does hold such data, they will not usually therapy (CBT) is recommended. For depression, a wider
be made public. This absence of transparency is a range of treatments (CBT, counselling, couples therapy,
disservice to patients. It is also an impediment to the interpersonal therapy, and brief psychodynamic therapy)
development of more effective health care, because it are recommended. The latest data2 show that around
makes it difficult to study, and learn from, the variation in 950 000 people a year have an initial assessment and
service outcomes. advice from the IAPT service, with more than 537 000 going
The Improving Access to Psychological Therapies on to have a course of therapy (defined as two or more
(IAPT) programme1 is a rare exception to the general sessions), with the predominant method being CBT.4,5
absence of transparency about outcomes for mental health A distinctive feature of IAPT is the use of an outcome
services. Starting in 2008, the UK Government developed monitoring system6 that ensures 98% of patients have
a plan to expand access to evidence-based psychological scores recorded on well-validated self-report measures of
therapies for depression and anxiety disorders by training depression and anxiety at the beginning and end of
a new workforce and deploying it in specialised services treatment, with CCG level summaries of such data
throughout England. Every local health area (otherwise publicly available. This system is a great improvement on

www.thelancet.com Vol 391 February 17, 2018 679


Articles

Research in context
Evidence before this study used the national public data to predict outcomes. None had
Many randomised controlled trials have shown that investigated service organisation.
psychological therapies are effective interventions for a wide Added value of this study
range of mental health conditions. On the basis of these trials, We describe the IAPT outcome monitoring system, discuss why
the National Institute of Health and Care Excellence (NICE) now it is so effective, and illustrate the value of the publicly available
recommends certain psychological therapies as first-line data by construction of statistical models to predict local
interventions for common mental health conditions such as variation in outcomes in 2014–15. We also tested the
depression and the anxiety disorders. robustness of the models by using them to predict changes in
However, a large gap exists between recommendation and outcomes from one year to the next. We identified five aspects
implementation. In particular, in most countries routine of the organisation of a service that are associated with
psychological therapy services do not record and publish their improved clinical outcomes.
clinical outcomes, which makes it difficult to know if outcomes
Implications of all the available evidence
are in line with expectation and hinders attempts to study, and
Traditionally, clinicians have been sceptical about the
learn from, between-service variation in clinical outcomes.
possibility of obtaining outcome data for most people who
The English Improving Access to Psychological Therapies (IAPT) are treated in routine psychotherapy services. We show that
programme is an exception. IAPT services treat more than getting such data is possible and describe a system that could
537 000 patients with depression or anxiety disorders each year be applied in other countries. To date, efforts to improve the
using NICE-recommended psychological therapies. A unique outcomes achieved with psychotherapies have mainly focused
outcome monitoring system allows IAPT to gather outcome on the development of new and more effective treatments.
data for 98% of treated patients. Since May 2012, the clinical Our analyses suggest that the way in which psychological
performance of each service has been reported on public therapy services are implemented could be similarly
websites. We searched MEDLINE with search terms “IAPT” and important. The findings open up new possibilities for the
“outcome” for articles published from May, 2012, to November, improvement of mental health care.
2017. Only one article, which focused on social deprivation, had

the national situation before the start of IAPT. At that In this Article, we aimed to illustrate the value of
time, a survey7 found that only 38% of patients had an information from IAPT’s outcome monitoring system by
assessment of their symptoms at the beginning and end using public data from the websites to identify
of treatment. organisational and other characteristics of services with
To monitor IAPT outcomes, patients complete brief better and worse outcomes.
measures of depression and anxiety every session so that a
symptom score for after treatment is available even if Methods
patients complete therapy earlier than expected. Therapists IAPT outcome monitoring system
make use of the measures in treatment planning and IAPT reports clinical outcomes for all patients who have
supervision. IAPT services have specialised information had at least two sessions of treatment and have been
technology systems that record patient data and make it discharged. Around 81% of people who are believed suitable
See Online for appendix available to therapists, supervisors, and managers. This for treatment have two or more sessions4 (appendix). We
session-by-session approach to outcome monitoring was used publicly available data to identify the characteristics
successfully piloted in a community therapy service for of IAPT services that achieve better and worse clinical
victims of the Omagh bomb8 (County Tyrone, Northern outcomes. We analysed the outcome data released by NHS
Ireland) and was enhanced for use in IAPT.9 Digital and Public Health England for the 2014–15 financial
IAPT gathers detailed information about patients, their year (April 1, 2014, to March 31, 2015) and developed a
course of treatment, and clinical outcomes.9 Once a predictive model. We then waited until the IAPT data for
For more on NHS Digital see month these data are sent to National Health Service 2015–16 (April 1, 2015, to March 31, 2016) were released and
www.digital.nhs.uk (NHS) Digital, which issues regular reports for the tested whether the identified predictors replicated in the
number of people accessing services and their outcomes, new dataset. We also tested whether change in predictors
along with a range of process variables (eg, average identified in the 2014–15 model was associated with change
For more on Public Health number of sessions). The most complete dataset in outcomes between 2014–15 and 2015–16. Similarity of
England’s Common Mental appears in the annual reports. Most data provided by findings in analyses of between-service variation at a
Health Disorders Profiling Tool NHS Digital are also available in Public Health England’s particular time and within service change over time would
see https://fingertips.phe.org.uk/
profile-group/mental-health/
Common Mental Health Disorders Profiling Tool, along strengthen the argument that the identified predictors have
profile/common-mental- with other contextual information about CCGs (eg, a causal role because spurious third variables are unlikely
disorders social deprivation score). to be similar in the two types of analysis.

680 www.thelancet.com Vol 391 February 17, 2018


Articles

Measures of clinical outcome were treated, because this variable captures the extent
The Patient Health Questionnaire 9-item (PHQ-9) score10 to which services focus on treatment, as opposed to
(clinical cutoff >9) is used to measure symptoms of providing only assessment, advice, and signposting. The
depression. The Generalised Anxiety Disorder 7-item third was the percentage of missed appointments (patient
(GAD-7) score11 (cutoff >7) is the default measure of did not attend and failed to give advance warning). The
anxiety but services can also use more specific measures fourth was the number of days patients waited between
for particular anxiety disorders.9 We modelled two of the referral and starting treatment, the fifth was the number
standard outcome indices included in NHS Digital’s of treatment sessions, and the sixth was the Index of
reports: the proportion of patients in a CCG who have Multiple Deprivation (IMD),14 for the area. The IMD
reliably improved and the proportion who have reliably covers seven domains, including income, employment,
recovered. Patients are reliably improved if their scores barriers to services, and crime. We did not use a seventh
on depression or anxiety, or both, have reduced by a possible predictor (number of NICE-recommended
reliable amount9,12 (ie, more than the measurement error depression treatments that the service could deliver) in
of the scale) and neither measure has shown a reliable the main analyses because preliminary analysis showed
increase. Patients are reliably recovered if they reliably the NICE predictor did not relate to outcome. For the
improve and their scores on both depression and anxiety third and fifth predictors, in 2014–15, data were available
are below the clinical cutoff scores9 at the end of only for the last 3 months of the study. For other
treatment. We assumed that patients (2%) without scores predictors, data covering the full periods were used. The
after treatment had not improved or recovered. The total IMD values are from September, 2015, and represent the
treated cohort is the denominator for calculation of latest available figures.
reliable improvement. Only cases above the clinical
cutoffs9 for depression, anxiety, or both before treatment Statistical analysis
are included in the computation of reliable recovery. The Analyses of predictors of IAPT performance per CCG
scores used to calculate reliable improvement and during single years were done in R software (version 3.2.2)
recovery are the last available after baseline scores, with β regression15 (betareg package),16 as recommended
usually from the final therapy session but occasionally for models with continuous but bounded outcomes (eg,
from an earlier session. NHS Digital does not report how proportions, with all data points falling between 0 and 1).16
often earlier session scores are used but data from an The method exploits the flexibility of β distribution,
NHS Trust that provides multiple IAPT services suggest which accommodates data heteroscedasticity (unequal
that the use of such scores is rare (2·5%; Green J, CNWL variance) and absence of symmetry around the mean,
NHS Trust, personal communication). both common features of proportions data that are less
well dealt with in commonly used logistic and linear
Possible predictors of outcome regression models. A logit link function in β regression
Six possible predictors were investigated. The first renders the coefficients interpretable in terms of odds
predictor was the percentage of treated patients for whom ratios (ORs). Analyses were initially run on the 2014–15
a problem descriptor (ICD-10 code)13 was recorded. This data and then repeated on the 2015–16 data.
variable was regarded as important because the type of To explore the associations between predictor and
treatment a patient should receive is based on ICD-10 outcome variables, we first fitted simple regressions.
codes.13 The second was the percentage of referrals who Then we fitted a multiple regression model to each of the

2014–15 2015–16
CCG (n) Mean (SD) Median (range) CCG (n) Mean (SD) Median (range)
Reliable improvement (%) 211 60·60% (7·80) 61·50% (24·80–76·80) 209 62·47% (6·63) 62·50% (35·40–80·10)
Reliable recovery (%) 211 42·87% (7·50) 43·00% (17·60–64·60) 209 44·44% (6·34) 44·60% (20·40–58·70)
Patients finishing treatment (n) 211 2217 (1424) 1830 (335–10 470) 209 2567 (1579) 2220 (510–10 430)
Patients finishing treatment classified as a clinical case at pretreatment (n) 211 1978 (1289) 1615 (270–9650) 209 2330 (1436) 2020 (465–9635)
Patients with problem descriptor completeness (%) 211 67·75% (29·87) 75·80% (0–100) 209 76·17% (25·66) 85·07% (0–100)
Patients who enter treatment and receive a course of treatment (%) 211 58·28% (12·94) 58·32% (19·44–90·84) 209 57·74% (12·87) 57·35% (26·54–88·29)
Missed appointments (%) 208 11·30% (4·10) 10·36% (4·48–27·27) 209 11·87% (4·17) 10·87% (3·65–25·62)
Treatment appointments (n) 208 6·35 (0·98) 6·33 (4·23–8·83) 209 6·41 (0·91) 6·34 (3·99–8·62)
Days before entering treatment 211 33·73 (20·56) 28·10 (6·70–124·10) 209 30·98 (23·26) 23·60 (5·90–139·30)
Index of Multiple Deprivation 208 21·96 (8·71) 20·65 (5·80–47·40) 208 21·96 (8·71) 20·65 (5·80–47·40)

CCG=clinical commissioning group.

Table 1: Descriptive statistics for the CCG sample in 2014–15 and 2015–16

www.thelancet.com Vol 391 February 17, 2018 681


Articles

outcomes, entering all predictors that were independently if this CCG were to match the highest values of this
significantly associated with the outcome, which allowed predictor recorded in the sample. To facilitate
us to assess the joint contribution of the predictors and interpretation for all predictors that were negatively
account for possible correlations between them. We associated with outcome, we present the reciprocal of
examined goodness of fit for multiple regression models the original ORs (ie, original OR of 0·75 is presented
using half normal plots of deviance residuals (plots of as OR 1·33), which is in line with published
absolute values of standardised residuals). To detect any recommendations17 to report ORs standardised to less
outliers that could bias our results, we identified CCGs than 1. For those variables, the possible gains in outcome
with an especially large effect over model fit using Cook’s indicated by the additive ORs would result from a
distance statistics. We inspected the characteristics of decrease in the predictor value.
these CCGs, and residual plots before and after the We also investigated whether changes in predictors
exclusions, to decide whether to retain these CCGs in the between 2014–15 and 2015–16 were related to changes in
final models (appendix). Up to two CCGs (<1% of the clinical outcomes between the 2 years. For each CCG, we
total sample) were excluded from a model when justified computed Δ scores, which relate to differences between a
by these indices. We then plotted the results, and curves variable’s value in 2014–15 and in 2015–16. We first
were fitted with the LOESS (locally weighted smoothing) compared the mean Δ scores of our predictors in the
function. Finally, for each predictor, we estimated the 10% of CCGs that improved most, the 10% that improved
increase in the odds of improvement or recovery if the least or deteriorated, and the overall mean. To formally
worst-performing CCG matched the best-performing test these findings, we ran linear regressions using the
CCG on that predictor. Δ scores of predictors and outcomes, because outcomes
We used ranges of predictor values recorded across the were no longer proportion scores and were normally
CCGs, and the ORs derived from the β regressions. distributed. To identify the strongest predictors and
Because ORs are additive on the log scale, the formula obtain standardised indices of the variance explained
for these calculations was exp (log[OR]  × range). For by correlated predictors, we computed semi-partial
example, if the difference between worst-performing and correlations in full models (package ppcor), which
best-performing CCG on a predictor was 100 units, and indicate correlations between the outcome and any given
a unit of that predictor was associated with the predictor, while controlling for the effects of all other
odds of recovery of 1·001, the formula would be exp predictors. Although IMD did not change, we included
(log[1·001] × 100), producing an additive OR of 1·105. absolute IMD score in all models to account for the
This OR value suggests that patients in the worst- possibility that social deprivation might affect the degree
performing CCG would be 10·5% more likely to recover of change in clinical performance.

Reliable recovery Reliable improvement


Single regression Multiple regression Single regression Multiple regression
OR p value Additive OR p value Additive OR p value Additive OR p value Additive
(95% CI) OR (95% CI) OR (95% CI) OR (95% CI) OR
Patients with problem descriptor 1·002 0·0015 1·221 1·002 0·0081 1·221 1·003 0·0003 1·349 1·001 0·269 1·105
completeness (%) (1·001– (1·105– (1·000– (1·000– (1·001– (1·105– (0·999– (0·905–
1·004) 1·490) 1·003) 1·349) 1·004) 1·491) 1·002) 1·221)
Patients who enter treatment and 1·009 <0·0001 1·896 1·004 0·016 1·330 1·014 <0·0001 2·698 1·009 <0·0001 1·896
receive a course of treatment (%) (1·006– (1·532– (1·001– (1·074– (1·012– (2·344– (1·006– (1·533–
1·012) 2·344) 1·007) 1·646) 1·017) 3·332) 1·012) 2·344)
Missed appointments (%)* 1·025 <0·0001 1·755 1·014 0·0028 1·373 1·031 <0·0001 2·005 1·015 0·0006 1·404
(1·015– (1·404– (1·005– (1·120– (1·022– (1·642– (1·006– (1·146–
1·035) 2·190) 1·022 1·642) 1·043) 2·610) 1·024) 1·717)
Mean number of treatment 1·077 0·0005 1·407 1·041 0·032 1·203 1·106 <0·0001 1·590 1·027 0·162 1·130
appointments (1·033– (1·161– (1·003– (1·014– (1·059– (1·302– (0·990– (0·955–
1·122) 1·698) 1·080) 1·425) 1·154) 1·933) 1·065) 1·336)
Mean number of days before 1·004 0·0005 1·598 1·002 0·015 1·264 1·006 <0·0001 2·018 1·003 0·0001 1·421
entering treatment* (1·002– (1·264– (1·001– (1·125– (1·004– (1·598– (1·002– (1·264–
1·005) 1·796) 1·004) 1·598) 1·008) 2·548) 1·005) 1·796)
Index of Multiple Deprivation* 1·014 <0·0001 1·783 1·009 <0·0001 1·452 1·010 0·0008 1·513 1·002 0·296 1·087
(1·010– (1·513– (1·005– (1·231– (1·005– (1·231– (0·998– (0·920–
1·019) 2·188) 1·013) 1·711) 1·015) 1·858) 1·006) 1·283)

ORs were derived by exponentiation of the β regression coefficients, and represent change in the probability of recovery or improvement with a unit increase in the predictor. Additive ORs represent ORs of
recovering in the CCGs with the highest value compared with the lowest value of the given parameter. All results were taken from the model where the two CCGs identified by high Cook’s Distance statistics were
excluded from the analysis. OR=odds ratio. CCG=clinical commissioning group. *Where the reciprocal of the OR originally derived from the model is given.

Table 2: Predictors of CCG performance in 2014–15

682 www.thelancet.com Vol 391 February 17, 2018


Articles

Validation procedures
A B
To mitigate against any bias due to excessive statistical CCG
power or other data artifacts, we repeated all analyses in 70 Regression line
permuted datasets—ie, when the associations between
outcomes and predictors were disrupted by random

Reliable improvement (%)


65
sampling. To further investigate whether associations
identified with our preferred method of β regression were 60
robust, we repeated the analyses using standard logistic
regression. Both validation analyses suggest our results 55
arise from true associations between outcomes and
predictors, rather than any analytical artifacts (appendix). 50

Role of the funding source 0


0 25 50 75 100 125 0 10 20
The funder of the study had no role in study design, data
collection, data analysis, data interpretation, or writing of Mean number of days people waited Missed appointments (%)
to enter treatment
the report. The corresponding author had full access to
all the data in the study and had final responsibility for Figure: Reliable improvement of patients in relation to time waited to start treatment in a CCG (A) and
the decision to submit for publication. missed appointments (B)
Each blue dot represents predicted percentage of patients who reliably improve in a particular CCG, based on the
predictor values recorded in that CCG and the effect sizes derived from the β regression. Therefore, although each
Results graph models expected values in reference to only one of the predictors, the plotted results are a function of all
In 2014–15, 211 CCG-based IAPT services (table 1) treated six of them, accounting for the non-linear trends. Red lines were smoothed with the LOESS method, and the dark
468 881 patients.5 In 2015–16, mergers of some grey areas around them represent the SE around these line estimates. CCG=clinical commissioning group.
neighbouring CCGs resulted in 209 CCG-based IAPT
services (table 1), which together treated 537 131 patients.4 model, CCGs with the largest decreases in missed
In 2014–15, all potential predictors were significantly appointments and waiting times, and with the largest
associated with reliable improvement and recovery when increases in problem descriptor completeness and
considered on their own (table 2), and for reliable percentage of referred patients who had a course of
recovery, all predictors were significant in multiple therapy, were also the CCGs that showed the largest
regression (table 2). Percentage of cases with a problem increases in reliable improvement rates (table 3). Similar
descriptor, the mean number of treatment sessions, and associations were noted for changes in reliable recovery,
the percentage of referred patients who received a course although fewer predictors were significant (table 3).
of treatment were all positively associated with reliable Although social deprivation scores did not change
recovery rates (table 2). Mean waiting time to enter between 2014–15 and 2015–16, CCGs with low amounts of
treatment, the percentage of appointments that were social deprivation showed larger improvements in
missed, and the social deprivation of a CCG were reliable recovery rates between 2014–15 and 2015–16 than
negatively associated with reliable recovery rates (table 2). CCGs with high amounts of social deprivation (table 3).
Mean waiting time to enter treatment, percentage of Overall, the predictors we investigated explained a
appointments missed, and percentage of referred variance of 33% for change in reliable improvement
patients who received a course of treatment were also and 22% for recovery rates.
significant predictors of reliable improvement in the As in 2014–15, all six predictors in 2015–16 were
multiple regression model (figure; other plots are shown significantly associated with both reliable improvement
in the appendix). and reliable recovery in single regression models.
Table 2 shows additive ORs that estimate the changes β regressions with multiple predictors showed that most
in reliable improvement and recovery that might be predictors of CCGs IAPT outcomes in 2014–15 continued
achievable if the CCG with the lowest score for a to be significant predictors in 2015–16 (table 4).
particular predictor were to match the highest-scoring The 2015–16 dataset includes some potential predictors
CCG. The analysis reveals considerable potential for that were not available in 2014–15. Stepped care is a key
improvement in the outcomes of some IAPT services. organising feature of IAPT. Simple regressions showed
For example, if the IAPT service with the lowest that CCGs in which a larger proportion of patients
percentage of patients being offered a course of treatment had low-intensity treatment only had low reliable
were to increase the proportion of people having improvement and recovery rates, whereas those CCGs
treatment to the level of the most treatment-oriented with high proportions of patients who had both low-
service, recovery rates would increase by 33% and reliable intensity and high-intensity interventions (stepped care)
improvement rates would increase by 90% (table 2). in their course of treatment had the highest reliable
All six predictors were individually associated with improvement and recovery rates (appendix). The
change in reliable improvement and recovery between proportion of patients with low-intensity and high-
2014–15 and 2015–16 (appendix). In a multiple regression intensity treatment was also a significant additional

www.thelancet.com Vol 391 February 17, 2018 683


Articles

Reliable recovery Reliable improvement


Single regressions Multiple regression Single regressions Multiple regression
β (SE) p value R² β (SE) p value Partial β (SE) p value R² β (SE) p value Partial
correlation correlation
Patients with problem descriptor 0·051 0·0027 0·039 0·041 0·011 0·137 0·055 0·0023 0·045 0·036 0·016 0·142
completeness (%) (0·017) (0·016) (p=0·056) (0·018) (0·015) (p=0·047)
Patients who enter treatment and receive 0·134 0·0002 0·069 0·046 0·231 0·033 0·272 <0·0001 0·249 0·187 <0·0001 0·248
a course of treatment (%) (0·034) (0·039) (p=0·64) (0·033) (0·037) (p=0·0004)
Missed appointments (%)* –0·609 <0·0001 0·123 –0·503 0·0002 –0·268 –0·761 <0·0001 0·171 –0·420 0·0003 –0·274
(0·115) (0·116) (p=0·0001) (0·113) (p=0·0001)
Mean number of treatment appointments 1·261 0·015 0·029 0·447 0·361 0·066 1·935 0·0004 0·057 0·553 0·238 0·068
(0·514) (0·488) (p=0·36) (0·536) (0·467) (p=0·343)
Mean number of days before entering –0·053 0·012 0·030 –0·036 0·080 –0·106 –0·096 0·0002 0·088 –0·046 0·022 –0·147
treatment* (0·021) (0·021) (p=0·138) (0·022) (0·020) (p=0·040)
Index of Multiple Deprivation 0·258 0·0009 0·054 0·196 0·0051 0·204 0·109 0·190 0·009 <–0·001 0·996 0·002
(0·077) (0·069) (p=0·0040) (0·083) (0·066) (p=0·974)

All results were taken from the model where the one CCG identified by high Cook’s Distance statistics was excluded from analyses. CCG=clinical commissioning group. *Where the reciprocal of the OR originally
derived from the model is given.

Table 3: Predictors of change in CCG performance between 2014–15 and 2015–16

Reliable recovery Reliable improvement


Single regressions Multiple regression Single regressions Multiple regression
OR p value Additive OR OR p value Additive OR OR p value Additive OR p value Additive OR
(95% CI) (95% CI) (95% CI) OR (95% CI)
Patients with problem 1·003 <0·0001 1·221 1·001 0·083 1·105 1·003 0·0003 1·349 1·001 0·110 1·105
descriptor completeness (%) (1·002– (1·349– (1·000– (1·000– (1·002– (1·221– (1·000– (1·000–
1·004) 1·491) 1·002) 1·349) 1·004) 1·491) 1·002) 1·221)
Patients entering treatment 1·006 0·0004 1·447 1·001 0·478 1·064 1·010 <0·0001 1·849 1·006 <0·0001 1·447
who receive a course of (1·003– (1·203– (0·999– (0·940– (1·007– (1·538– (1·003– (1·203–
treatment (%) 1·008) 1·636) 1·002) 1·131) 1·013) 2·220) 1·008) 1·636)
Missed appointments (%)* 1·027 <0·0001 1·844 1·017 <0·0001 1·473 1·032 <0·0001 2·062 1·021 <0·0001 1·612
(1·018– (1·506– (1·011– (1·286– (1·024– (1·724– (1·014– (1·376–
1·034) 2·155) 1·025) 1·763) 1·040) 2·462) 1·027) 1·844)
Mean number of treatment 1·075 0·0001 1·398 1·017 0·276 1·081 1·085 0·0008 1·459 1·011 0·508 1·052
appointments (1·036– (1·178– (0·987– (0·941– (1·042– (1·210– (0·978– (0·902–
1·116) 1·662) 1·048) 1·242) 1·129) 1·754) 1·046) 1·231)
Mean number of days before 1·004 <0·0001 1·703 1·002 0·0006 1·305 1·004 <0·0001 1·703 1·002 0·0002 1·305
entering treatment* (1·003– (1·491– (1·001– (1·143– (1·003– (1·491– (1·001– (1·143–
1·005) 1·945) 1·004) 1·703) 1·006) 1·703) 1·004) 1·703)
Index of Multiple Deprivation* 1·013 <0·0001 1·711 1·010 <0·0001 1·513 1·009 0·0002 1·452 1·004 0·018 1·181
(1·010– (1·513– (1·007– (1·337– (1·005– (1·231– (1·001– (1·042–
1·017) 2·016) 1·013) 1·711) 1·013) 1·711) 1·007) 1·337)

ORs were derived by exponentiation of the β regression coefficients, and represent change in the probability of recovery or improvement with a unit increase in the predictor. Additive ORs represent ORs of
recovering in the CCGs with the highest value compared with the lowest value of the given parameter. All estimates were taken from the model with the CCGs identified by Cook’s Distance statistics excluded.
OR=odds ratio. CCG=clinical commissioning group. *Where the reciprocal of the OR originally derived from the model is given.

Table 4: Predictors of CCG performance in 2015–16

predictor in the multiple regression model (appendix). Discussion


To investigate whether differences between CCGs in the In this Article, our analyses suggest that looking at the
initial severity of patients’ symptoms might have affected way in which therapy services are implemented might
our findings, we ran additional multiple regressions that also be important for the improvement of clinical
included mean pretreatment depression (PHQ-9) and outcomes. Traditionally, researchers interested in the
anxiety (GAD-7) scores as additional predictors. The improvement of mental health outcomes have largely
predictors that were significant in table 4 were significant focused on trying to develop new and more effective
for the additional multiple regressions analysis, and therapies, and this work has led to major advances18,19 in
pretreatment symptom severity did not emerge as an psychological therapies. However, the 209 IAPT services
additional predictor (appendix). included in the national dataset all aim to implement

684 www.thelancet.com Vol 391 February 17, 2018


Articles

NICE-recommended psychological therapies2,3 for halved in the multiple regressions compared with the
depression and anxiety disorders using a stepped-care single regressions.
model with therapists who have been through a national IAPT is a rare example of consistent outcome
training programme.1 Although the overall outcomes are monitoring and reporting in routine mental health
broadly in line with expectation from clinical trials,19 the services. Initially, many practitioners were sceptical about
performance of individual services varies greatly. the feasibility of obtaining outcome data from almost all
Key aspects of the way a service is organised predict a patients treated by IAPT services. This scepticism was
substantial amount of this variability, both within year and understandable because previous attempts that were
when modelling changes between years. The organisational based on data collection at only the start and end of
factors identified make clinical sense. Services that are treatment were associated with high rates of missing data
better able to identify the problems they are treating for after treatment.7 Furthermore, evidence that people
(problem descriptor completeness) are presumably more who failed to provide data after treatment tended to have a
likely to give the right NICE-recommended treatment. poorer clinical outcome was worrying.23 IAPT manages to
Short waits between referral and treatment might ensure obtain an end of treatment symptom measure for almost
patients remain enthusiastic about engaging with everyone (98%) who has a course of treatment by the
treatment. A high average dose of treatment (number of simple manoeuvre of asking patients to complete
sessions) is likely to improve outcomes, as is consistency symptom measures every time they are seen.
of attendance (low percentage of missed appointments), Now that the possibility to obtain more or less complete
and a service that is predominantly focused on treatment, outcome data is clear, and that these data can be used to
rather than only assessment (high percentage of patients identify potential ways to improve patients’ outcomes, we
entering treatment). Focusing on treatment is reminiscent hope mental health services elsewhere in the UK and
of the positive association in surgery between postoperative other countries will consider adopting a similar approach
outcome and the volume of operations undertaken by to data collection and reporting. In England the IAPT
hospitals and individual surgeons.20 dataset has greatly improved public transparency about
The range of IAPT data that is placed in the public the mental health outcomes associated with routinely
domain on NHS Digital and Public Health England’s delivered psychological therapies. Patients can see what
websites is constantly expanding. Over time it will enable their local service offers and the outcomes it achieves.
interested parties to explore the importance of new Commissioners and clinicians working in the services can
variables, and study how the effect of previously studied now benchmark their service against others and consider
variables changes as services work to improve the way the development of collaborative networks in which
they are organised. For the first time, NHS Digital’s latest services come together to discuss common problems and
annual report4 (October, 2016) includes CCG-level data learn from each other’s solutions.
on how stepped care is implemented. When we included Our study has some limitations. Because patient-level
this information in our model we found that services in data were not available, we could not simultaneously
which a high proportion of patients had some high- estimate the effects on outcome of both patient-level
intensity therapy in their course of treatment had variables (such as initial symptom severity, sex, age, and
significantly better outcomes than those with a low anxiety measures) and service organisation factors.
proportion of patients. This outcome suggests services However, we were able to show that the severity of clinical
should make full use of stepped care, with patients who problems that services treat varied very little and that
do not recover after low-intensity intervention (such as this omission cannot explain our findings. Outcome
guided self-help) being given the opportunity to be assessment was restricted to patient self-report. Although
stepped up to high-intensity intervention (traditional our analyses illustrate potential gains achievable in the
face-to-face therapy) rather than simply be discharged. A worst-performing CCGs by bringing their organisational
similar finding emerged from analysis of data from the characteristics to the level shown by the best-performing
first year of the IAPT programme, when only 32 services ones, we acknowledge that these are predictions that will
were available.21 Future IAPT reports are likely to include need to be further tested.
funding information, which could be included in Before IAPT, little was known about the outcomes
statistical models. achieved in routine mental health services. The session-
In line with previous research,22 we found that social by-session outcome monitoring system used in IAPT
deprivation is a significant predictor of outcome. enables the services to collect outcome data from almost
Although acting on social deprivation is a matter for everyone. Publication of this data is improving public
local and national policy and economic development, transparency and also allows analyses to be done that
some of its effects on outcome could possibly be help us to understand, and hopefully reduce, local
mitigated by ensuring that IAPT services in socially variability in mental health outcomes.
deprived areas are of high quality and adequately funded. Contributors
Consistent with this, we found that the effect sizes for DMC contributed to the study design, data acquisition, analysis plan,
social deprivation as a predictor of outcome were about literature search, drafting of the manuscript and appendix, and final

www.thelancet.com Vol 391 February 17, 2018 685


Articles

approval. MJ contributed to the study design and analysis plan, did the 8 Gillespie K, Duffy M, Hackmann A, Clark DM. Community based
analyses, produced the appendix, and was involved in drafting the cognitive therapy in the treatment of posttraumatic stress disorder
manuscript and final approval. LC contributed to the study design, data following the Omagh bomb. Behav Res Ther 2002; 40: 345–57.
acquisition, literature search, drafting of the manuscript, and final 9 Department of Health. The IAPT data handbook version 2.0.1. 2011.
approval. RL, SP, and JG contributed to the study design, analysis plan, http://webarchive.nationalarchives.gov.uk/20160302160058/http://
literature search, drafting of the manuscript, and final approval. www.iapt.nhs.uk/silo/files/iapt-data-handbook-v2.pdf (accessed
March 30, 2017).
Declaration of interests 10 Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief
DMC and RL were two of the originators of the Improving Access to depression severity measure. J Gen Intern Med 2001; 16: 606–13.
Psychological Therapies (IAPT) programme. DMC is also a National 11 Spitzer Rl, Kroenke K, Williams JW, Löwe B. A brief measure for
Health Service (NHS) England Clinical Advisor for the programme. assessing generalized anxiety disorder: The gad-7. Arch Intern Med
The views expressed in this Article are personal and not necessarily the 2006; 166: 1092–97.
same as those of NHS England. SP is the Director of the National 12 Jacobson NS, Truax P. Clinical significance: a statistical approach to
Collaborating Centre for Mental Health, which is involved in producing defining meaningful change in psychotherapy research.
National Institute for Health and Care Excellence guidelines. JG is the J Consult Clin Psychol 1991; 59: 12–19.
lead psychologist for the Central and North West London NHS Trust, 13 World Health Organization. The ICD-10 classification of mental
which manages five IAPT services. All other authors have no and behavioural disorders: Clinical descriptions and diagnostic
competing interests. guidelines. Geneva: WHO; 1992.
14 Department of Communities and Local Government. English indices
Acknowledgments of deprivation 2015. https://www.gov.uk/government/collections/
The Wellcome Trust funded this study. DMC is a Wellcome Trust and english-indices-of-deprivation (accessed March 30, 2017).
National Institute for Health Research Senior Investigator. The authors 15 Ferrari SLP, Cribari-Neto F. Beta regression for modelling rates and
would like to thank IAPT clients and therapists for their remarkable proportions. J Appl Stat 2004; 31: 799–815.
consistency in completing and reporting clinical outcome measures. 16 Cribari-Neto F, Zeileis A. Beta Regression in R. J Stat Softw 2010;
References 34: 1–24.
1 Clark DM. Implementing NICE guidelines for the psychological 17 Osborne JW. Bringing balance and technical accuracy to reporting
treatment of depression and anxiety disorders: the IAPT experience. odds ratios and the results of logistic regression analyses.
Int Rev Psychiatry 2011; 23: 375–84. Pract Assess Res Eval 2006; 11: 1–6.
2 NICE. Common mental health disorders: identification and 18 Roth A, Fonagy P. What works for whom? A critical review of
pathways to care. Clinical Guideline 123. www.nice.org.uk/ psychotherapy research, 2nd edn. New York, NY: Guilford Press,
guidance/cg123 (accessed March 30, 2017). 2005.
3 NICE. Social anxiety disorder: recognition, assessment and 19 Layard R, Clark DM. Thrive: the power of psychological therapies.
treatment. Clinical Guideline 159. www.nice.org.uk/guidance/cg159 London: Penguin, 2014.
(accessed March 30, 2017). 20 Hu JC, Gold KF, Pashos CL, Mehta SS, Litwin MS. Role of surgeon
4 NHS Digital. Psychological therapies: annual report on the use of volume in radical prostatectomy outcomes. J Clin Oncology 2003;
IAPT services in England (2015–16). Leeds: NHS Digital, Oct 18, 2016. 21: 401–05.
5 HSCIC. Psychological therapies: annual report on the use of IAPT 21 Gyani A, Shafran R, Layard R, Clark DM. Enhancing recovery rates:
services in England (2014–15). Leeds: Health and Social Care lessons from year one of IAPT. Behav Res Ther 2013; 51: 597–606.
Information Centre, Nov 24, 2015. 22 Delgadillo J, Asaria M, Ali S, Gilbody S. On poverty, politics and
6 Public Health England. Common mental health disorders profiling psychology: the socioeconomic gradient of mental healthcare
tool. 2017. https://fingertips.phe.org.uk/profile-group/mental-health/ utilisation and outcomes. Br J Psychiatry 2015; 209: 429–30.
profile/common-mental-disorders (accessed March 30, 2017). 23 Clark DM, Layard R, Smithies R, Richards DA, Suckling R,
7 Clark DM, Fairburn CG, Wessely S. Psycholgical treatment outcomes Wright B. Improvng access to psychological therapy: initial
in routine NHS services: a commentary on Stiles et al (2007). evaluation of two UK demonstration sites. Behav Res Ther 2009;
Psychol Med 2008; 38: 629–34. 47: 910–20.

686 www.thelancet.com Vol 391 February 17, 2018

You might also like