Administration and Policy in Mental Health and Mental Health Services Research

https://doi.org/10.1007/s10488-021-01113-x

ORIGINAL ARTICLE

Recidivism Treatment for Justice‑Involved Veterans: Evaluating

Adoption and Sustainment of Moral Reconation Therapy in the US
Veterans Health Administration
Daniel M. Blonigen1,2 · Paige M. Shaffer3,4 · Jennifer S. Smith1 · Michael A. Cucciare5,6,7 · Christine Timko1,2 ·
David Smelson3,4 · Jessica Blue‑Howells8 · Sean Clark8 · Joel Rosenthal8

Accepted: 17 January 2021

© This is a U.S. government work and not under copyright protection in the U.S.; foreign copyright protection may apply 2021

Abstract
Moral Reconation Therapy (MRT), an evidence-based intervention to reduce risk for criminal recidivism among justice-
involved adults, was developed and primarily tested in correctional settings. Therefore, a better understanding of the imple-
mentation potential of MRT within non-correctional settings is needed. To address this gap in the literature, we evaluated the
adoption and sustainment of MRT in the US Veterans Health Administration (VHA) following a national training initiative in
fiscal years 2016 and 2017. In February 2019, surveys with 66 of the 78 VHA facilities that participated in the training were
used to estimate the prevalence of MRT adoption and sustainment, and qualitative interviews with key informants from 20
facilities were used to identify factors associated with sustainment of MRT groups. Of the 66 facilities surveyed, the majority
reported adopting (n = 52; 79%) and sustaining their MRT group until the time of the survey (n = 38; 58%). MRT sustain-
ment was facilitated by strong intra-facility (e.g., between veterans justice and behavioral health services) and inter-agency
collaborations (e.g., between VHA and criminal justice system stakeholders), which provided a reliable referral source to
MRT groups, external incentives for patient engagement, and sufficient staffing to maintain groups. Additional facilitators of
MRT sustainment were adaptations to the content and delivery of MRT for patients and screening of referrals to the groups.
The findings provide guidance to clinics and healthcare systems that are seeking to implement MRT with justice-involved
patient populations, and inform development of implementation strategies to be formally tested in future trials.

Keywords Criminal recidivism · Moral reconation therapy · Justice-involved veterans · Veterans health administration ·
Adoption · Sustainment

5
* Daniel M. Blonigen HSR&D Center for Mental Healthcare and Outcomes
[email protected] Research, Central Arkansas Veterans Affairs Healthcare
System, North Little Rock, AR, USA
1
HSR&D Center for Innovation to Implementation, 6
Department of Psychiatry, University of Arkansas
Department of Veterans Affairs, Palo Alto Health Care
for Medical Sciences, Little Rock, AR, USA
System, 795 Willow Road (152), Menlo Park, CA 94025,
7
USA Veterans Affairs South Central Mental Illness Research,
2 Education, and Clinical Center, Central Arkansas Veterans
Department of Psychiatry and Behavioral Sciences, Stanford
Healthcare System, North Little Rock, AR 72205, USA
University School of Medicine, Palo Alto, CA, USA
8
3 Veterans Justice Programs, Veterans Health Administration,
HSR&D Center for Health Care Organization
Washington, DC, USA
and Implementation Research, Bedford VA Medical Center,
Bedford, MA, USA
4
University of Massachusetts Medical School, Worcester, MA,
USA

13
Vol.:(0123456789)
 Administration and Policy in Mental Health and Mental Health Services Research

Introduction Milkman and Wanberg 2007). Moral Reconation Therapy

(MRT) is a cognitive-behavioral intervention that aims
Among formerly incarcerated adults in the United States to modify criminogenic thinking among justice-involved
(US), criminal recidivism is the norm. Data from the US adults to reduce their likelihood of criminal recidivism
Bureau of Justice Statistics on incarcerated adults who (Little and Robinson 1988). The intervention protocol is
were released from prison in 2005 found that 68 and 77% standardized using both a participant and provider man-
were rearrested within 3 and 5 years, respectively (Durose ual to guide treatment. These manuals help participants
et al. 2014). Recidivism is also common among veterans advance through 12 steps of moral development. Partici-
who have been detained by, or are under the supervision pants attend groups with open enrollment (i.e., group ses-
of, the criminal justice system (“justice-involved veter- sions can incorporate new members at any time). Partici-
ans”). Among all veterans in the US, the rate of incar- pants complete homework assignments and exercises in a
ceration is 0.86%. However, among veterans in behavioral workbook between group sessions and then present their
health treatment, up to 75% have been incarcerated at some work to group members at the next session. The efficacy of
point in their lifetime (Blonigen et al. 2020). Further, data MRT is supported by multiple reviews, including a meta-
from the 2011–2012 US National Inmate Survey shows analysis of 33 studies which found that the rate of recidi-
that 62% of veterans in jails report four or more prior vism among MRT participants is reduced by one-third
arrests and 68% of veterans in prison report at least one relative to control samples (Ferguson and Wormith 2013).
prior episode of incarceration (Bronson et al. 2015). Data
from the Veterans Health Administration (VHA) also indi- MRT Implementation in Non‑correctional Settings
cate that those served by VHA’s Veterans Justice Programs
(VJP) have an average of eight arrests over their lifetime MRT was originally developed for use within correctional
(Department of Veterans Affairs 2012). Collectively, these settings in which there are strong external incentives and/or
data suggest that many justice-involved veterans have a mandates to participate in the intervention. Consequently,
chronic history of criminal justice involvement and have knowledge about the implementation potential of MRT in
recidivated at some point in their lifetime. non-correctional settings, such as VHA, is limited. Inter-
A chronic cycle of contact with the criminal justice sys- views with VJP specialists regarding their perceptions of
tem can limit an individual’s access to healthcare services. the implementation potential of treatments for recidivism
For example, per federal regulations, VHA cannot provide in VHA provide some insights by highlighting cross-ser-
healthcare services to veterans while they are incarcer- vice (e.g., VJP and behavioral health) and cross-system
ated (Department of Veterans Affairs 2011). The adverse (e.g., healthcare and criminal justice system) partnerships
impact of not receiving, or having intermittent, healthcare as potential facilitators to MRT implementation in this non-
may be especially pronounced for the behavioral health correctional setting (Blonigen et al. 2018a). The broader lit-
of justice-involved adults, given the high rates of men- erature on implementation of evidence-based practices for
tal health and substance use problems in this population justice-involved populations also point to facility character-
(Blonigen et al. 2019). Accordingly, the VHA has placed a istics such as appropriate staffing levels (Prendergast et al.
high priority on implementing best practices for reducing 2017), collaborations across stakeholders (Abdel-Salam
criminal recidivism among their justice-involved patient et al. 2015; Green et al. 2015; Lamberti 2016), communi-
population (Blue-Howells et al. 2013; Hartley and Bald- ties of practice among practitioners (Pearson et al. 2015),
win 2019). and partnerships with treatment courts (Abdel-Salam et al.
2015) as positively impacting adoption and sustainment of
these practices.
Best Practices for Recidivism Reduction: Moral The gap in knowledge regarding implementation of MRT
Reconation Therapy in non-correctional settings is significant, given that the
majority of justice-involved adults in the US are on parole
In the offender rehabilitation literature, cognitive-behav- or probation and reside in the community (Kaeble and Glaze
ioral treatments are regarded as best practices for reduc- 2016). In addition, there has been a policy shift in the US
ing criminal recidivism among justice-involved adults criminal justice system in recent years away from incarcera-
(Andrews and Bonta 2010). Interventions that use a treat- tion and towards diversion of justice-involved adults (Scott
ment manual to standardize their protocol and target crimi- et al. 2013). Treatment courts in which justice-involved
nogenic thinking (i.e., antisocial attitudes, cognitions, and adults can participate in mental health and/or substance use
behaviors) have the strongest evidence for reducing recidi- treatment as an alternative to formal charges and/or incar-
vism risk (Aos et al. 2006; Landenberger and Lipsey 2005; ceration are one example (Tsai et al. 2018). For example,
the number of Veterans Treatment Courts have expanded

13
Administration and Policy in Mental Health and Mental Health Services Research

rapidly over the past decade, with 128 such courts operat- framework) was defined as having started an MRT group
ing in 2011 and 575 operational by 2020 (Clark and Flatley after the initial training, and the adoption phase as the time
2019; Department of Veterans Affairs 2020). Healthcare period in which there was active support for the implemen-
systems that partner with treatment courts are often tasked tation of MRT (see “Training Program Overview” section).
with providing behavioral health services to justice-involved Sustainment was defined as maintenance of MRT after the
adults (Hartley and Baldwin 2019). In VHA, the number of active support ended (Moullin et al. 2020). Accordingly, we
veterans served by the program that coordinates with Vet- (1) estimated the prevalence of MRT adoption and sustain-
erans Treatment Courts increased from 18,303 in 2013 to ment across VHA facilities, (2) identified facility character-
49,816 in 2019 (Blue-Howells et al. 2013; Department of istics associated with MRT adoption and sustainment, and
Veterans Affairs 2020). Consequently, the responsibility of (3) used qualitative approaches to understand the facilitators
rehabilitation and recidivism reduction for justice-involved and barriers to sustainment of MRT across facilities.
veterans is falling increasingly on the VHA.

MRT Implementation in VHA Methods

VHA has been a national leader in the care for justice- Training Program Overview
involved adults in non-correctional settings through the
work of the VJP (Clark and Flatley 2019), which provides In July 2016, the VJP disseminated a solicitation for interest
outreach to justice-involved veterans across all points of the in MRT training to each of the 21 regional VHA Home-
criminal justice system. VJP specialists link justice-involved less Coordinators and Mental Health Liaisons. These indi-
veterans to VHA and non-VHA services to address their viduals were asked to identify up to one VJP specialist and
healthcare and psychosocial needs (Blue-Howells et al. one behavioral health provider for training from each VHA
2013; Clark and Flatley 2019), and serve as the healthcare medical facility in their region with the expectation that
partner in Veterans Treatment Courts by working with VHA these individuals would jointly initiate and co-facilitate an
behavioral health and homeless services to coordinate care MRT group, post training. A total of 155 individuals (77
(McCall and Pomerance 2019; Tsai et al. 2018). In 2013, a VJP specialists; 60 behavioral health providers, 10 homeless
structured evidence review of the treatment needs of justice- program providers, 8 unknown) employed across 78 VHA
involved veterans that was sponsored by the VJP concluded medical facilities were supported to attend an MRT training
that, among interventions that target criminogenic thinking, in either FY16 (n = 139) or FY17 (n = 16). The mean num-
MRT had the strongest evidence for reducing risk for crimi- ber of individuals trained per facility was 1.99 (SD = 0.61;
nal recidivism (Blodgett et al. 2013). Subsequently, VJP Min = 1, Max = 4). MRT trainings were provided by Correc-
worked to increase access to MRT for justice-involved vet- tional Counseling Inc., the copyright holder of MRT. Train-
erans by consulting in the development of a veteran-specific ings were held in-person over four days (32 h total) at sites
version of the MRT manuals (Little and Robinson 2013). In across the US. After this initial training, there was a period
addition, in fiscal years (FY) 2016 and 2017, the VJP sup- of active support by VJP for MRT implementation, which
ported training in MRT by providing funding for tuition and lasted approximately one year and consisted of regional
travel costs for 155 VHA providers across 78 VHA medical calls to provide guidance on procedures for establishing the
centers. The goal of this national training initiative was to MRT groups, a community-of-practice email listserv, and a
increase the adoption of MRT in VHA homeless programs monthly call series focused on MRT group implementation.
and behavioral health services system-wide. At the time of the present evaluation, this active support had
been terminated for approximately 16 months.
The Current Study
Design and Procedures
The present study evaluated the adoption and sustainment of
MRT at VHA medical centers following the national training The study used a mixed-methods approach to evaluate
initiative in FY16-17. To guide this evaluation, we used the MRT adoption and sustainment. Of the 155 individu-
Exploration, Preparation, Implementation, and Sustainment als who attended an MRT training, 148 who were still
(EPIS) framework, which describes distinct phases in the employed by the VHA were contacted by email in Feb-
implementation of an evidence-based practice into a system ruary 2019 and asked to complete a brief survey inquir-
(Aarons et al. 2011; Moullin et al. 2019). Consistent with ing whether their site had started an MRT group after the
this framework, we conceptualized adoption and sustain- training (i.e., adopted MRT). Respondents who had were
ment as distinct phases in this evaluation. Specifically, adop- also queried on how long the MRT group was active, if
tion (corresponding to the Implementation phase of the EPIS the group was currently active, and in which settings the

13
 Administration and Policy in Mental Health and Mental Health Services Research

group was implemented: behavioral health (e.g., mental Data Sources

health and/or substance use inpatient, residential, or outpa-
tient treatment program), homeless programs (e.g., HUD- Facility Characteristics
VASH), or a Veterans Treatment Court.
Out of 78 facilities that were contacted, 66 (84.6%) had Facilities were categorized as urban or rural (USDA Eco-
at least one individual who completed the survey (n = 98 nomic Research Service 2020). Records provided by VJP
individuals). The survey results were used to identify which were used to determine (a) the number of VJP specialists
VHA facilities after the initial MRT training by Correc- employed at the facility and (b) whether a Veterans Treat-
tional Counseling Inc. (a) never started an MRT group— ment Court was affiliated with the facility’s VJP service
Non-adopters (n = 16 sites; 24.2% of facilities responding at the time of the MRT training, as well as (c) whether a
to the survey), (b) started an MRT group but discontinued provider from that facility had joined the MRT community-
the group after one year or less—Adopters (n = 12; 18.2%), of-practice listserv following the training. Facilities that
or (c) adopted and sustained an MRT group until the time had started an MRT group were categorized according to
of the survey, with the group duration extending beyond the whether the group was implemented into a behavioral health
period of active support for implementation (i.e., more than program (yes/no), a homeless program (yes/no), and/or a
one year)—Sustainers (n = 38, 57.6%). Following the survey, Veterans Treatment Court (yes/no); this information was
a subset of respondents from each site type (Non-adopters, obtained from the survey.
Adopters, and Sustainers) participated in a one-time, semi-
structured phone interview (approximately 45–60 min) to Qualitative Interviews
elicit perceptions of MRT and experiences with its imple-
mentation. As VHA employees, they were not able to receive In considering an implementation science framework to
financial compensation. Interviews were conducted by three guide the qualitative interviews, we sought one focused
trained research assistants from March 2019 to July 2019. on contextual factors given the study’s aim to understand
All procedures were reviewed by the local Institutional MRT implementation at the facility level. Accordingly,
Review Board, which determined that the study did not meet we selected the Consolidated Framework for Implementa-
criteria for human subjects research and was exempt from tion Research (CFIR) since it focuses on contextual factors
further review. that can affect the implementation of new clinical practices
(Damschroder et al. 2009). CFIR comprises a menu of con-
structs that can impact the successful implementation of
Sampling Strategy an intervention and is organized into five broad domains:
Characteristics of Individuals, Inner Setting, Outer Setting,
A stratified purposeful sampling strategy was used to recruit Intervention Characteristics, and Implementation Process.
survey respondents from each site type (Non-adopters, We adapted CFIR and used it to inform the content of
Adopters, and Sustainers) to complete the qualitative inter- the interview guide. Specifically, prior to data collection
view. Within each site type, facility selection was based the authors reviewed the interview guide tool on the CFIR
on obtaining representation across five broad geographic website (https​://cfirg​uide.org) and used a consensus process
regions of the US: North Atlantic, Southeast, Midwest, to identify the domains and constructs most relevant to the
Continental, and Pacific (Department of Veterans Affairs intervention (i.e., MRT), patient population (i.e., justice-
2016). For Adopters and Sustainers, facility selection was involved veterans), and clinical context (e.g., VHA medi-
also based on an effort to obtain representation across the cal centers; behavioral health and homeless programs, and
different settings in which an MRT group was implemented Veterans Treatment Courts and other criminal justice set-
at the facility. Survey respondents from selected facilities tings). Based on prior and ongoing research on the imple-
were contacted by email with a letter of invitation to partici- mentation potential of MRT in VHA (Blonigen et al. 2018a,
pate. Recruitment continued until at least 25% of facilities b), participants were questioned on CFIR constructs in the
in each site type (Non-adopters, Adopters, and Sustainers) domains of Characteristics of Individuals (Knowledge &
had been enrolled and thematic saturation was reached (Hen- Beliefs about the Intervention, e.g., “What did others at
nink et al. 2017). At the end of the interview, participants your site think of your decision to attend the training?”),
were asked to provide the name and contact information for Inner Setting (Implementation Climate [e.g., “To what extent
someone in leadership at their facility who could comment were people receptive to implementing MRT at your site?”],
further on why MRT was (or was not) adopted by the facil- Compatibility [e.g., “How did MRT fit with existing services
ity. These leadership referrals were then contacted via email and treatments for justice-involved veterans at your site?”],
and invited to participate. Available Resources [e.g., “Did you have all the necessary
infrastructure and resources to implement MRT such as staff,

13
Administration and Policy in Mental Health and Mental Health Services Research

space, time?”]), Outer Setting (Patient Needs & Resources, cases and columns being each question of the interview
e.g., “Tell me about how veterans were able to access MRT guide, grouped by CFIR domain. This first step was a deduc-
groups at your site.”), Intervention Characteristics (Adapt- tive approach such that the CFIR domains represented an ini-
ability, e.g., “Please describe any changes you made to tial set of pre-defined codes from which broader, cross-cut-
MRT?”), and Implementation Process (Planning, Engag- ting themes could be identified using inductive approaches.
ing, e.g., “Who was involved with implementation of MRT Three analysts used this matrix to conduct thematic content
at your site?” “Who were the key partners to get on board analysis of the textual data from the qualitative interviews.
with implementing MRT?”). Interviews with participants Each analyst was initially assigned a site type (i.e., Non-
were used to better understand the implementation process adopter, Adopter, Sustainer) and reviewed the detailed notes
at that facility. All interviews were audio-recorded (with per- from each interview and entered the data into the relevant
mission) and transcribed verbatim. After each interview, the cell in the spreadsheet. Analysts also used the transcription
interviewer also took detailed notes by CFIR domain, which of the audio-recording to fill in any gaps or missing informa-
were used to summarize responses to each question. tion in the spreadsheet based on the interviewer’s notes. We
then conducted thematic analysis using an inductive (emer-
Analysis gent) approach by having each analyst independently review
the data from one of the site types and identify an initial
To examine facility characteristics associated with adoption list of barrier and facilitator themes relevant to sustainment
(i.e., starting an MRT group after the initial training), we of MRT groups. The analysis was conducted across par-
conducted 2-group univariate chi-square tests to examine if ticipants’ comments, not by the specific interview questions
Adopter and Sustainer sites differed from Non-adopter sites or corresponding CFIR domain. The analysts then met to
on the following: urban vs. rural, number of VJP specialists review their independently-derived lists of themes, mapped
employed, Veterans Treatment Court affiliation (yes/no), and out commonalities and differences between their emergent
provider joined community-of-practice listserv (yes/no). To themes, and engaged in a consensus process to rectify disa-
examine facility characteristics associated with sustainment, greements and refined the themes to identify similarities and
we conducted 2-group univariate chi-square tests to examine differences between site types. This step was facilitated by
if Sustainer sites differed from Adopter and Non-adopter Venn diagrams to group similarities and differences by site
sites on these variables. We also examined if Sustainer sites type. Systematic comparisons were made between the dif-
differed from Adopter sites on settings of MRT group imple- ferent site types to identify the most robust themes that were
mentation (behavioral health [yes/no], homeless program barriers or facilitators to sustainment of MRT groups. After
[yes/no], Veterans Treatment Court [yes/no], and whether this initial review, a consensus list of barrier and facilita-
MRT was implemented in more than one setting). tor themes to sustainment was developed and each analyst
Textual data from the qualitative interviews was analyzed then applied those themes to a second review of the matrix,
using the framework method, a form of thematic analysis which included interviews from all site types. The analysts
that can be applied to deductive and/or inductive approaches met again to discuss these themes and revise them into their
(Gale et al. 2013). The defining feature of this method is final structure.
use of a matrix or spreadsheet to organize the textual data
in which rows refer to cases, columns refer to codes, and
cells represent summaries of the data (e.g., excerpts from Results
interview transcripts, detailed summaries of participants’
responses from interviewer notes and/or audio-recordings; Prevalence of MRT Adoption and Sustainment
Neal et al. 2015). This approach allows researchers to sys-
tematically reduce large amounts of textual data to facili- Of the 66 facilities that responded to the survey, 52 (78.8%)
tate analysis. The analytic process begins with each analyst reported starting an MRT group after the training (adop-
independently coding and analyzing a subset of interviews tion), and 38 (57.6%) reported having an active MRT group
and then meeting to compare the initial codes and themes at the time of the survey (sustainment) (see Table 1). Out of
identified. During these initial meetings, visual aids or dia- the 52 facilities that started a group, the majority (82.7%)
grams are used to map out commonalities and differences reported starting only one group. On length of time groups
between emergent themes from each analyst. This process were active, the majority (65.4%) were active for more than
continues iteratively until consensus is reached among the one year and 44.2% were active for at least 2 years. Of the
analysts and the final set of themes and codes is applied to 52 facilities that started a group, the majority of facilities
the rest of the interviews. reported that the group was implemented in a behavio-
In applying the framework method, the research team ral health program (71.2%), and 42.3% of facilities each
developed a matrix in Microsoft Excel, with rows being reported the group was implemented in a homeless program

13
 Administration and Policy in Mental Health and Mental Health Services Research

Table 1  Survey results regarding adoption of MRT post-training Qualitative Interview Sample
(n = 66 facilities)
N (%) Individuals from 32 facilities (11 Non-adopters, 10 Adop-
ters, and 11 Sustainers) were contacted and invited to par-
Facilities that started an MRT g­ roupa 52 (78.8%)
ticipate. Out of the facilities contacted, 20 (62.5%) agreed
Facilities with an active group 38 (57.6%)
to participate (4 Non-adopters, 6 Adopters, and 10 Sus-
Number of MRT groups started (out of 52)
tainer sites). Interview participants were 13 VJP special-
1 group 43 (82.7%)
ists, 4 behavioral health providers, and 3 homeless service
2+ groups 7 (13.5%)
providers who had attended one of the MRT trainings. Out
Unknown 2 (3.8%)
of 11 leadership referrals that were provided by these par-
Length of time group(s) were active (out of 52)
ticipants and contacted by the research team, five (45.5%)
< 6 months 5 (9.6%)
agreed to participate and included Directors of Substance
6 months–1 year 6 (11.5%)
Use Treatment Services, Social Work, or Homeless Services
1–2 years 11 (21.2%)
at their respective facility. In total, 25 key informants were
2+ years 23 (44.2%)
interviewed (see Fig. 1 for a flowchart of study participa-
Unknown 7 (13.5%)
tion). Participants were mostly female (n = 13; 52.0%) and
Settings in which an MRT group was implemented (out of 52)b
non-Hispanic Caucasian (n = 20; 80.0%), with a mean age
Behavioral health program 37 (71.2%)
of 47.44 years (SD = 10.44). Participants reported being in
Homeless program 22 (42.3%)
their current role for 4.73 years (SD = 2.82). Across the site
Veterans treatment court 22 (42.3%)
types, participants did not differ on any of these sociodemo-
Unknown 2 (3.8%)
graphic variables.
a
Two facilities did not indicate whether their MRT groups were
active, therefore we were unable to categorize these facilities as either
an Adopter or Sustainer site Facilitators of MRT Sustainment
b
Percentages are over 100% as MRT groups could have been imple-
mented into more than one setting at a given site Five themes characterized the sustainment of MRT groups
across facilities: (1) Buy-in among VHA colleagues and
leadership; (2) Multiple co-facilitators; (3) Partnerships with
or a Veterans Treatment Court (percentages are over 100%
the criminal justice system; (4) Screening of referrals; and
as MRT groups could have been implemented into more
(5) Adaptations to intervention content and delivery. These
than one setting at a given site). A majority of the same 52
themes are described below. Illustrative quotes are also pro-
facilities reported only one setting in which an MRT group
vided below and in Table 2.
was implemented (58%).

Buy‑in Among VHA Colleagues and Leadership

Facility Characteristics Associated with Adoption
and Sustainment of MRT Participants at Sustainer sites described the importance of
buy-in and support for MRT among colleagues and leader-
Of the 66 facilities that responded to the survey, a majority ship at their facility. In particular, participants at all of the
were from urban areas (n = 55; 83.3%) and had a Veterans Sustainer sites reported that after they attended the MRT
Treatment Court that was affiliated with the local VJP ser- training they gave a presentation on MRT to their colleagues
vice at the time of the training (n = 61, 92.4%). A minority of at their VHA facility to clarify the purpose of MRT as well
facilities had a provider that joined the MRT community-of- as which patients would be appropriate referrals to these
practice listserv following the training (n = 28; 42.4%). On groups. Further, VJP specialists also specified the impor-
average, 1.82 VJP specialists (SD = 0.96) were employed by tance of establishing buy-in with colleagues and leadership
these facilities. In terms of facility characteristics associated from behavioral health services at their facilities. Partici-
with MRT adoption, Adopter and Sustainer sites were more pants described these outreach efforts as an ongoing process
likely than Non-adopter sites to have a provider that joined that occurred through regular meetings with colleagues and
the MRT community-of-practice listserv (50 vs. 19%), also highlighted the value of these efforts for generating
χ2 = 4.85, p = 0.041. In terms of facility characteristics asso- referrals to the groups.
ciated with MRT sustainment, Sustainers were more likely
We sent out emails initially…and I made a lot of effort,
than Adopters to report that a Veterans Treatment Court was
so I tried to initiate staff, made myself available for
involved in the implementation of an MRT group at the facil-
Veterans if they were interested. I’ve done presenta-
ity (52.6 vs. 16.7%, χ2 = 4.79, p = 0.029).

13
Administration and Policy in Mental Health and Mental Health Services Research

Providers trained in MRT

in FY16 and FY17
(n = 155)

Providers who were sent the survey

Providers who no longer (n = 148)
work for the VA (n = 7) _______________________________
Facili­es surveyed (n = 78)

Providers who completed the survey

(n = 98; 66.2%)
____________________________________
Facili­es with at least 1 provider who
completed the survey (n = 66; 84.6%)

Non-Adopter sites Adopter sites Sustainer sites

(n = 16, 24.2%) (n = 12, 18.2%) (n = 38, 57.6%)

Non-Adopter Sites Interviewed Adopter Sites Interviewed Sustainer Sites Interviewed

(n = 4, 25.0%) (n = 6, 50.0%) (n = 10, 26.3%)
_____________________________ ____________________________ __________________________
Interviewees Interviewees Interviewees
(n = 5; 4 providers, 1 leadership) (n = 6; 6 providers) (n = 14; 10 providers, 4 leadership)

Fig. 1  Flowchart of study participation

tions too, of the social workers at our medical center. We added a second group that was run by myself and
I did a lunchtime presentation about our program as a co-facilitator that went to the same training I did. He
well. [Site ID: 324–Sustainer]. joined and started a new group and co-facilitated. I
think it had us communicating to make sure we had a
Multiple Co‑facilitators consistent message. [Site ID: 321–Sustainer].

Participants at Sustainer sites noted the importance of hav- Partnerships with the Criminal Justice System
ing multiple group facilitators at their facility who could
deliver MRT. For example, more than half of Sustainer sites Participants at Sustainer sites reported strong partner-
reported having at least one other individual at their facil- ships with the criminal justice system, particularly Veter-
ity who was also trained in MRT and had the bandwidth to ans Treatment Courts that were staffed by VJP specialists.
help co-facilitate groups or provide coverage when the other Participants noted the importance of such partnerships for
facilitator was not available. To provide sufficient coverage, establishing a reliable stream of referrals into their facility’s
some Sustainer sites used interns or trainees. Participants at MRT groups. They also highlighted the value of these courts
these sites also highlighted the value of multiple facilitators in terms of either incentivizing veterans’ participation in
in terms of maintaining fidelity to the protocol and a consist- MRT (e.g., offering early graduation to court participants
ent message to group participants. who completed their MRT program) or mandating veterans’

13
 Administration and Policy in Mental Health and Mental Health Services Research

Table 2  Facilitators of MRT sustainment

Themes Sample quotations

Buy-in among VHA colleagues and leadership I’d go to a monthly behavioral health staff meeting and we have a monthly social work staff
meeting. I went to both of those meetings and presented MRT when we decided to open up
and accept referrals…just to educate staff on what it was and who would be an appropriate
referral. [Site ID: 309–Sustainer]
It was supported by my immediate leaders. And then also the person who went with me from
our site was from my substance abuse clinic, so her supervisor and chain was supportive
as well. We’ll go back every now and then just to say, hey, we still have this program, don’t
forget about us. [Site ID: 303–Sustainer]
We sent out emails initially…and I made a lot of effort, so I tried to initiate staff, made myself
available for Veterans if they were interested. I’ve done presentations too, of the social work-
ers at our medical center. I did a lunchtime presentation about our program as well. [Site ID:
324–Sustainer]
Multiple co-facilitators [The VJOs] we co-facilitate one group together so that we can try to maintain that fidelity with
each other and at least, just to keep that cohesion. [Site ID: 309–Sustainer]
We added a second group that was run by myself and a co-facilitator that went to the same
training I did. He joined and started a new group and co-facilitated. I think it had us com-
municating to make sure we had a consistent message. [Site ID: 321–Sustainer]
Partnerships with the criminal justice system They’re making it a requirement in their court that everyone participate in MRT. [Site ID:
309–Sustainer]
Two of the courts actually require the Veteran to complete MRT prior to graduation. [Site ID:
327–Sustainer]
I made it mandatory that any Veterans who are in the drug court have to do MRT during phase
2 because that’s when [they] have few obligations. And this keeps them on track for me to
help them through the drug court. [Site ID: 304–Sustainer]
Screening referrals The way we do that in Veterans Court…it’s in phases. When they’re in Phase 1 of Veteran’s
Court, they do not participate in MRT until they’re at least in Phase 2 because they need to
get used to Veterans court first and then have some time of sobriety. We want them to have
90 days of recovery before starting MRT. [Site ID: 309–Sustainer]
We realized that some of those Veterans were not ready for MRT and needed a level of moti-
vational interviewing at first to identify just even a behavior or a goal that might be able to
be targeted or addressed within MRT. So we changed resident referrals…a consult process to
get a sense of what supports the referrals. [Site ID: 327–Sustainer]
Adaptations to intervention content and delivery We have a fishbowl, and in the fishbowl we have a mixture of inspirational quotes, as well as
gift cards. Every time they complete a step they draw from the fishbowl…We also started an
evening group to accommodate people who had to work. [Site ID: 309–Sustainer]
We do a group that is ‘MRT informed.’ [MRT] is more shame-based than strengths-based.
[MRT workbook]…I really like the activities, but some of the wording in the chapters is
[shame-based]. I like to focus more on the positive…Some patients after they graduate call
in and do it over the phone. [Site ID: 337–Sustainer]
Not every veteran was going to be able to be face-to-face in person. Our medical center made
the agreement to offer video health available at every one of our outpatient clinics to allow
Veterans to appear closest to their residence and do it telephonically or through webcam to
our site. [Site ID: 327–Sustainer]
My Veterans court is a ways from the medical center and some of the Veterans live in outer
parts of the county, so it was next to impossible for them to attend face to face meetings. We
addressed that by starting a video on demand group. [Site ID: 324–Sustainer]

attendance at these groups as a condition for remaining in from Sustainer sites described a process of screening the
the treatment court. referrals they received in order to maximize the veter-
ans’ outcomes. For example, screening was used to assess
They’re making it a requirement in their court that eve-
whether the veteran would be an appropriate referral based
ryone participate in MRT. [Site ID: 309–Sustainer].
on their progress on their treatment plan as well as their
recovery from substance use (e.g., the veteran was mak-
Screening Referrals ing progress on their requirements for the treatment court;
the veteran had achieved sobriety for at least 90 days). In
In addition to having reliable referral sources from VHA other cases, the screening process included psychoeduca-
services and/or criminal justice system partners, participants tion to veterans on the purpose of the group and probing

13
Administration and Policy in Mental Health and Mental Health Services Research

their internal motivations for attending these groups (e.g., providing referrals to MRT group facilitators had multiple
conducting motivational interviewing to determine veterans’ responsibilities and did not provide referrals on a regular
readiness for change). basis. Other sites highlighted a lack of stronger partnerships
with other VHA treatment services and/or the criminal jus-
We realized that some of those Veterans were not ready
tice system as limiting opportunities for receiving referrals
for MRT and needed a level of motivational interview-
to the groups.
ing at first to identify just even a behavior or a goal that
might be able to be targeted or addressed within MRT. I felt supportive in going to the training, but when I
So we changed resident referrals…a consult process came back and I needed to get clients into the group
to get a sense of what supports the referrals. [Site ID: I was not being supported. I didn’t get one referral. I
327–Sustainer]. could implement it if I had support from the residen-
tial unit; then there would be enough people. [Site ID:
Adaptations to Intervention Content and Delivery 202–Adopter].

Participants at Sustainer sites described various adaptations Conflicts with In‑Person Attendance
that were made to the content and/or delivery of MRT at
their facility. In terms of content, adaptations were typically Another barrier to MRT sustainment pertained to patients’
described as efforts to increase patient engagement in the ability to attend MRT groups in person. A lack of trans-
groups. For example, one site described adding contingency portation for patients living in the community, particularly
management (e.g., fishbowl prize drawings) to promote those living in rural settings, was highlighted by partici-
attendance. Another site described modifying the curricu- pants as a barrier to MRT attendance. In addition, some sites
lum to make it more strengths-based because of a perception highlighted scheduling conflicts due to competing treatment
among facility staff that the content focused too much on demands for patients (e.g., conflicts with other outpatient
shame, and another site noted the addition of office hours groups; work attendance as part of compensated work ther-
to allow patients to receive assistance with their homework apy that precluded attending groups during the day) as a
outside of regularly scheduled groups. In terms of adapta- barrier to MRT sustainment.
tions to delivery of MRT, these were typically changes to
overcome barriers to patients’ access to groups, particularly I had opened it up to outpatient. The scheduling was
those who worked during the day (e.g., evening groups) or difficult. There’s so many groups going on here, so
those living in rural settings (e.g., offering video telehealth trying to find a group time. Sometimes we’d find veter-
options for attendance). ans that were appropriate, but had conflicting appoint-
ments. [Site ID: 208–Adopter].
My veterans court is a ways from the medical center
and some of the Veterans live in outer parts of the Low Patient Engagement
county, so it was next to impossible for them to attend
face to face meetings. We addressed that by starting Sites that struggled to sustain MRT highlighted low patient
a video on demand group.” [Site ID: 324–Sustainer]. engagement as a significant factor. Some sites highlighted
a lack of internal motivation on the part of the patient to
Barriers to MRT Sustainment engage with the time-intensive curriculum of MRT (e.g.,
homework assignments). Similarly, sites highlighted a lack
Four themes emerged with respect to barriers to sustaining of external pressures for patients to attend, such as a mandate
MRT groups at a facility: (1) lack of referrals, (2) conflicts from a treatment court or parole/probation services.
with in-person attendance, (3) low patient engagement, and
(4) insufficient staffing. These themes are described in detail I didn’t have anybody being mandated to be there….
below. Illustrative quotes for each theme are provided below When you have that type of external motivation for
and in Table 3. the person to be there it seems to work better. [Site
ID: 209–Adopter].
Lack of Referrals
Insufficient Staffing
Sites that did not sustain MRT highlighted an inability to
establish a consistent referral stream as a key barrier. For Sites that struggled to sustain MRT discussed a lack of suf-
example, Non-adopter sites reported being unable to get ficient staff at their site who were trained and/or had the time
enough referrals to start a group. In some cases, sites indi- to facilitate MRT groups. For example, several sites indicated
cated that VJP specialists at a facility who were tasked with that sustainment was not feasible because only one person was

13
 Administration and Policy in Mental Health and Mental Health Services Research

Table 3  Barriers to MRT sustainment

Theme Sample quotation

Lack of referrals [VJO] wanted to collaborate with us in the substance abuse treatment program to offer this service to
Veterans. But the VJOs are rarely at the hospital to co-facilitate a group. And so the plan that we had to
partner with them fell through. So that was how we had hoped to get referrals to the group…We needed
folks who were court involved or had recent history of being involved with the courts. And we didn’t get
those referrals. [Site ID: 101–Non-adopter]
VJO was heavily encouraged by MRT, but not something that was mandated. We didn’t get too many refer-
rals from them, so I would have to say that the receptiveness was there, but the referrals weren’t. [site ID:
206–adopter]
I felt supportive in going to the training, but when I came back and I needed to get clients into the group I
was not being supported. I didn’t get one referral. I could implement it if I had support from the residen-
tial unit; then there would be enough people. [Site ID: 202–Adopter]
Conflicts with in-person attendance Some of our Veterans don’t drive, so transportation may have been an issue. [Site ID: 206–Adopter]
I think transportation is [a barrier]. I am in a very rural area. I can’t get enough people to do a group. I will
be able to get one person in the group, but then they’re going to have to drive two hours to the VA to
attend a two hour group and drive home two hours. They’re rural, and they’re poor, and they don’t have
vehicles, so it gets really hard. [Site ID: 202–Adopter]
I had opened it up to outpatient. The scheduling was difficult. There’s so many groups going on here, so
trying to find a group time. Sometimes we’d find Veterans that were appropriate, but had conflicting
appointments. [Site ID: 208–Adopter]
Low patient engagement Those few that we got to get the group going were already kind of motivated individuals. We were trained
as if [MRT] was held in jail [and] they had a captive audience that they knew would be there and would
have to participate. So trying to change it for an outpatient setting with volunteers was kind of tough….
[Site ID: 208–Adopter]
I didn’t have anybody being mandated to be there…When you have that type of external motivation for the
person to be there it seems to work better. [A veteran who was referred] never showed back up because
there was nothing other than just his own motivation driving him because there was nothing external
whatsoever other than just being recommended. [Site ID: 209–Adopter]
Insufficient staffing It probably would have been better to train somebody who’s actually on the main campus, who has access
to the substance abuse clinic, or the clinics where they have readily available Veterans there…the girl that
was doing it got moved to a different position. So it just dissipated right there. [Site ID: 201–Adopter]
I tried to get our SUD coordinator involved a little bit and that didn’t really work well…there’s really not
a lot of help. I realized that it was going to be up to me to do this. Everybody’s stretched pretty thin
already. I was trying to get somebody else trained right here, so we could have two of us, and it never
happened. [Site ID: 209–Adopter]
Our VJO had very limited staff. Two social workers from our VJO were at the training also…they were
overwhelmed with work already. They couldn’t really add this on to their plate…they did not have time to
co-facilitate the group so the model that we had planned could not be implemented. [Site ID: 101–Non-
adopter]

trained in MRT. In some cases, the lone staff member who was Discussion
trained changed positions and the groups ended thereafter. In
other cases, sites noted the lack of sufficient staffing at key We evaluated the adoption and sustainment of MRT at VHA
services at their facility (e.g., behavioral health) to either help facilities following a national training initiative in FY16-17.
with facilitation of the MRT groups or to provide referrals to In one respect, the training initiative was successful, given
the groups. that the majority of VHA facilities that sent providers to be
It probably would have been better to train somebody trained in MRT reported adoption of MRT at some point
who’s actually on the main campus, who has access to thereafter. Perhaps more notably, the majority of facilities
the substance abuse clinic, or the clinics where they have also reported that their MRT group(s) had been sustained
readily available veterans there…the girl that was doing up until the time of the survey, nearly half of whom had
it got moved to a different position. So it just dissipated sustained their group(s) for at least 2 years. Consistent with
right there. [Site ID: 201–Adopter]. the stated expectation of the VJP that support for the training

13
Administration and Policy in Mental Health and Mental Health Services Research

include a collaboration across VJP and behavioral health argued that the risk of perceived coercion can be mitigated
services at each facility, the majority of VHA facilities that through participatory decision-making and attending to
adopted MRT reported that the group was implemented into clients’ internal motivations (Hachtel et al. 2019). The
a behavioral health program. However, nearly half of the qualitative data from the current evaluation provide some
facilities that adopted MRT also reported that the group was support for this argument as Sustainer sites tended to have
implemented in the facility’s homeless programs and/or a a process of screening referrals to their MRT groups to
local Veterans Treatment Court. The brief survey limited provide psychoeducation to referred veterans and assess
our ability to discern the specific ways in which these ser- for their internal motivations, which may have helped to
vices and settings were involved in implementation of MRT, minimize any perceived coercion.
although the qualitative findings provide some insights. In addition to collaborations between VHA and the
Nonetheless, the survey results indicate that cross-service criminal justice system (Becan et al. 2018), sustainment
and cross-system collaborations were common following the of MRT groups were also marked by collaborations within
training, which may have contributed to the high prevalence VHA facilities. Indeed, participants from Sustainer sites
of MRT post-training. highlighted the importance of buy-in from other VHA ser-
In terms of facility characteristics, MRT adoption was vices, particularly behavioral health services. Establishing
associated with having a provider that joined the MRT com- these collaborations was perceived as impacting the abil-
munity-of-practice listserv. This listserv was organized and ity of a facility to have sufficient staffing to sustain groups
managed by VJP leadership to disseminate information to and to have a reliable referral stream for the MRT groups.
individuals involved in the implementation of MRT in VHA Whether inter- or intra-agency collaborations, successful
and to provide a forum for these individuals to seek guid- partnerships between justice program and behavioral health
ance regarding the planning, implementation, and sustain- services likely requires discussion of differences in the goals
ment of MRT groups. For example, VJP leadership would and practices of these services and how to integrate them
post information such as how to purchase patient workbooks (Lamberti 2016). For example, behavioral health providers
for MRT groups and dates of consultation calls hosted by may consider a treatment for recidivism such as MRT as out-
VJP leadership and representatives from Correctional Coun- side their scope of practice. Through didactics, in-services,
seling Inc., and listserv members would post lessons learned and other forms of outreach, justice program providers can
from starting up groups at their local facility. Communities clarify the potential impact of MRT on outcomes that are
of practice have been promoted in healthcare as a means more typically the focus of behavioral health providers (e.g.,
of enhancing knowledge, promoting standardization of reduced substance use; better interpersonal relationships),
practices, and facilitating innovation and quality of care for thereby generating buy-in and building a coalition of imple-
patients (Ranmuthugala et al. 2011). Although causality can- mentation partners in the facility (Powell et al. 2015).
not be determined in the present evaluation, the association As is often required with interventions that are imple-
between engagement in this listserv and adoption of MRT mented in contexts outside of where they were their origi-
at a facility suggests that this approach may be beneficial to nally designed, sustaining MRT in a healthcare system will
include in an implementation strategy to support the uptake likely require adaptations that are carefully tailored to the
of MRT in a healthcare system. non-correctional context (Stirman et al. 2019). Given that
Among facilities that did adopt MRT post training, MRT was designed for correctional settings with a captive
those that sustained their group(s) were more likely to audience that has few barriers to attendance, it is perhaps
report involvement of a Veterans Treatment Court. This not surprising that sites that were able to sustain MRT
finding was echoed in the qualitative data, which sug- tended to make adaptations to both the content and/or deliv-
gested that these partnerships facilitated sustainment of ery of MRT to increase access and engagement. As MRT
MRT groups at a facility by establishing a reliable refer- becomes implemented more widely in non-correctional
ral stream and providing a strong external incentive (i.e., settings, understanding what adaptations are beneficial and
mandate) to engage in the intervention (Lamberti 2016). can guide healthcare providers on when and how to modify
This notwithstanding, concerns have been raised in the MRT is a critical area for future research. Such efforts will
criminal justice literature that mandated treatment can help establish when adaptations are fidelity-inconsistent and
be perceived as coercive and lead to poorer outcomes for can adversely impact the core elements of the intervention.
offenders compared to voluntary treatment (Parhar et al. For example, video telehealth can greatly increase access to
2008). Conversely, other research with justice-involved MRT to justice-involved adults who are not able to attend
adults has suggested that rates of criminal recidivism in-person groups—an issue that is all the more salient in
(Young et al. 2004) and substance use outcomes (Kelly the era of COVID-19 (Heyworth et al. 2020). However, it
et al. 2005) are comparable between those who are and is not clear if this format for delivery of the intervention
are not mandated to treatment. Further, a recent review adversely impacts the social processes that are inherent in

13
 Administration and Policy in Mental Health and Mental Health Services Research

many group-based interventions (Gerhart et al. 2015) and Funding This work was supported by supplemental funding from the
whether this would limit the potential effectiveness of MRT. VA Quality Enhancement Research Initiative (Grant No. QUE 15-284).
Christine Timko was supported by a Senior Research Career Scientist
Award from VA HSR&D (Grant No. RCS-00-001). Joel Rosenthal
Limitations and Future Directions is now retired from the Veterans Health Administration. The views
expressed are those of the authors and do not necessarily reflect those
Several limitations should be noted. First, most participants of the Veterans Health Administration.
were trained in MRT in 2016 or 2017 but were not surveyed
or interviewed until 2019; therefore the accuracy of the data Compliance with Ethical Standards
may be limited due to retrospective recall biases. Second,
Conflict of Interest The authors declare that they have no conflict of
no data were systematically collected to assess the impact interest.
of adoption and sustainment of MRT on either criminal
recidivism or other health-related outcomes at the patient Ethical Approval All procedures were reviewed by the local Institu-
or facility level. This limitation is underscored by the lack of tional Review Board, which determined that the study did not meet cri-
teria for human subjects research and was exempt from further review.
randomized controlled trials of MRT in general; thus, addi-
tional outcome data are sorely needed. In particular, the evi-
dence of MRT’s effectiveness among individuals with men-
tal illness is lacking, which is a notable gap given the high
prevalence of mental illness among justice-involved veterans References
(Blodgett et al. 2015). The pending results from a multisite,
Hybrid I trial of MRT in VHA will help to address this gap Aarons, G. A., Hurlburt, M., & Horowitz, S. M. (2011). Advancing a
conceptual model of evidence-based practice implemenation in
in the literature (Blonigen et al. 2018b). Third, despite the public service sectors. Administration and Policy in Mental Health
high response rate to the email survey, the data collected and Mental Health Services Research, 38, 4–23.
were limited in scope. Further, the numbers of qualitative Abdel-Salam, S., Kilmer, A., & Visher, C. A. (2015). Building bridges
interviews for participants at Non-adopter and Adopter sites in New Jersey: Strengthening interagency collaboration for offend-
ers receiving drug treatment. International Journal of Offender
were modest. Future evaluations should endeavor to collect Therapy and Comparative Criminology, 61(2), 210–228. https​://
more detailed descriptions of MRT groups and interview doi.org/10.1177/03066​24x15​59895​9.
a wider range of participants and sites to increase the gen- Andrews, D. A., & Bonta, J. L. (2010). Rehabilitating criminal jus-
eralizability of the findings. Fourth, this initiative did not tice policy and practice. Psychology, Public Policy, and Law, 16,
39–55. https​://doi.org/10.1037/a0018​362.
involve a formal implementation platform such as Getting Aos, S., Miller, M. G., & Drake, E. (2006). Evidence-based adult cor-
to Outcomes (Chinman et al. 2017) or Facilitation (Ritchie rections programs: What works and what does not. Olympia, WA:
et al. 2017), which are designed to systematically implement Washington State Institute for Public Policy.
an evidence-based practice into large healthcare systems. Becan, J. E., Bartkowski, J. P., Knight, D. K., Wiley, T., DiClemente,
R., Ducharme, L., et al. (2018). A model for rigorously applying
Finally, although it was a national evaluation, the findings the exploration, preparation, implementation, sustainment (EPIS)
may not generalize beyond the VHA, which has a well- framework in the design and measurement of a large scale col-
established outreach and linkage program (i.e., VJP) that laborative multi-site study. Health and Justice, 6(1), 9. https:​ //doi.
other healthcare systems may not have. org/10.1186/s4035​2-018-0068-3.
Blodgett, J. C., Fuh, I. L., Maisel, N. C., & Midboe, A. M. (2013). A
structured evidence review to identify treatments needs of justice-
Summary and Conclusions involved veterans and associated psychological interventions.
Menlo Park, CA: Center for Health Care Evaluation, VA Palo
MRT sustainment was facilitated by intra- and inter-agency Alto Health Care System.
Blodgett, J. C., Avoundjian, T., Finlay, A. K., Rosenthal, J., Asch, S.
collaborations; these collaborations likely facilitated both M., Maisel, N. C., & Midboe, A. M. (2015). Prevalence of mental
the referral and engagement of patients into MRT groups as health disorders among justice-involved veterans. Epidemiologic
well as adequate staffing to initiate and maintain the groups. Reviews, 37(1), 163–176.
Adaptations to the content and delivery of MRT are likely Blonigen, D. M., Cucciare, M. A., Timko, C., Smith, J. S., Harnish, A.,
Kemp, L., et al. (2018a). Study protocol: A hybrid effectivenes
critical to facilitate the uptake and maintenance of MRT implementation trial of moral reconation therapy in the US Veter-
in non-correctional settings, as are efforts to screen refer- ans Health Administration. BMC Health Services Research, 18(1),
rals to minimize the perception of coercion among referred 164. https​://doi.org/10.1186/s1291​3-018-2967-3.
patients. Collectively, the findings provide guidance to Blonigen, D. M., King, C. M., & Timko, C. (2019). Criminal justice
involvement among veterans. In J. Tsai & E. Seamone (Eds.),
healthcare systems that are seeking to implement MRT as Intersections between mental health and law among veterans (pp.
well as help develop an implementation strategy, grounded 13–43). Cham: Springer.
in the building of coalitions and promoting adaptability Blonigen, D. M., Macia, K. S., Smelson, D., & Timko, C. (2020).
(Powell et al. 2015), that can be formally tested in future Criminal recidivism among justice-involved veterans following
implementation trials (Landes et al. 2019).

13
Administration and Policy in Mental Health and Mental Health Services Research

substance use disorder residential treatment. Addictive Behaviors. Hachtel, H., Vogel, T., & Huber, C. G. (2019). Mandated treatment and
https​://doi.org/10.1016/j.addbe​h.2020.10635​7. its impact on therapeutic process and outcome factors. Frontiers
Blonigen, D. M., Rodriguez, A. L., Manfredi, L., Nevedal, A., in Psychiatry, 10, 219. https​://doi.org/10.3389/fpsyt​.2019.00219​.
Rosenthal, J., McGuire, J. F., et al. (2018b). Cognitive-behavio- Hartley, R. D., & Baldwin, J. M. (2019). Waging war on recidivism
ral treatments for criminogenic thinking: Barriers and facilitators among justice-involved veterans: An impact evaluation of a large
to implementation within the Veterans Health Administration. urban veterans treatment court. Criminal Justice Policy Review,
Psychological Services, 15, 87–97. https​://doi.org/10.1037/ser00​ 30, 52–78.
00128​. Hennink, M. M., Kaiser, B. N., & Marconi, V. C. (2017). Code satura-
Blue-Howells, J. H., Clark, S. C., van den Berk-Clark, C., & McGuire, tion versus meaning saturation: How many interviews are enough?
J. F. (2013). The U.S. Department of Veterans Affairs veterans jus- Qualitative Health Research, 27, 591–608.
tice programs and the sequential intercept model: Case examples Heyworth, S., Kirsh, S., Zulman, D., Ferguson, J. M., & Kizer, K.
in national dissemination of intervention for justice-involved vet- W. (2020). Expanding access through virtual care: The VA’s
erans. Psychological Services, 10, 48–53. https:​ //doi.org/10.1037/ early experience with Covid-19. NEJM Catalyst. https​://doi.
a0029​652. org/10.1056/cat.20.0327.
Bronson, J., Carson, A. C., Noonan, M., & Berzofsky, M. (2015). Vet- Kaeble, D., & Glaze, L. (2016). Correctional populations in the United
erans in prison and jail, 2011–12. Washington, DC: U.S. Depart- States, 2015. Washington, DC: U.S. Department of Justice, Office
ment of Justice, Office of Justice Programs, Bureau of Justice of Justice Programs, Bureau of Justice Statistics.
Statistics. Kelly, J. F., Finney, J. W., & Moos, R. (2005). Substance use disor-
Chinman, M., McCarthy, S., Hannah, G., Byrne, T. H., & Smelson, der patients who are mandated to treatment: Characteristics,
D. A. (2017). Using getting to outcomes to facilitate the use of treatment process, and 1- and 5-year outcomes. Journal of Sub-
an evidence-based practice in VA homeless programs: A cluster- stance Abuse Treatment, 28, 213–223. https​://doi.org/10.1016/j.
randomized trial of an implementation support strategy. Imple- jsat.2004.10.014.
mentation Science, 12(1), 34. https​://doi.org/10.1186/s1301​ Lamberti, J. S. (2016). Preventing criminal recidivism through mental
2-017-0565-0. health and criminal justice collaboration. Psychiatric Services,
Clark, S., & Flatley, B. (2019). VA programs for justice-involved veter- 67(11), 1206–1212. https​://doi.org/10.1176/appi.ps.20150​0384.
ans. In J. Tsai & E. Seamone (Eds.), Intersections between mental Landenberger, N. A., & Lipsey, M. W. (2005). The positive effects
health and law among veterans (pp. 45–58). Cham: Springer. of cognitive behavioral programs for offenders: A meta-analysis
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, of factors associated with effective treatment. Journal of Experi-
J. A., & Lowery, J. C. (2009). Fostering implementation of health mental Criminology, 1, 451–476. https​://doi.org/10.1007/s1129​
services research findings into practice: A consolidated framework 2-005-3541-7.
for advancing implementation science. Implementation Science, Landes, S. J., McBain, S. A., & Curran, G. M. (2019). An intro-
4, 50. https​://doi.org/10.1186/1748-5908-4-50. duction to effectiveness-implementation hybrid designs. Psy-
Department of Veterans Affairs. (2011). Medical benefits package (38 chiatry Research, 280, 112513. https​://doi.org/10.1016/j.psych​
CFR § 17.38). DC: Veterans Justice Program Washington. res.2019.11251​3.
Department of Veterans Affairs. (2012). HOMES: Veterans justice Little, G. L., & Robinson, K. D. (1988). Moral reconation therapy:
outreach and health care for re-entry veterans summary data. A systematic step-by-step treatment system for treatment resist-
Washington, DC: Veterans Justice Program. ant clients. Psychological Reports, 62, 135–151. https​://doi.
Department of Veterans Affairs (2016). Retrieved from https​://www. org/10.2466/pr0.1988.62.1.135.
va.gov/myva/. Accessed 1 Feb 2020. Little, G. L., & Robinson, K. D. (2013). Winning the invisible war: An
Department of Veterans Affairs (2020). Veterans treatment courts and MRT workbook for veterans. Memphis, TN: Eagle Wing Books.
other Veteran-focused courts served by VA Veterans Justice Out- McCall, J. D., & Pomerance, B. (2019). Veterans treatment courts. In
reach Specialists (fact sheet, July 2020). Retrieved from http:// J. Tsai & E. Seamone (Eds.), Intersections between mental health
www.va.gov/homel​ess/vjo.asp. Accessed on 1 Dec 2020 and law among veterans (pp. 89–110). Cham: Springer.
Durose, M. R., Cooper, A. D., & Snyder, H. N. (2014). Recidivism Milkman, H., & Wanberg, K. (2007). Cognitive behavioral treatment:
of prisoners released in 30 states in 2005: Patterns from 2005 A review and discussion for corrections professionals. Retrieved
to 2010. Washington, DC: U.S Department of Justice, Office of from https​://s3.amazo​naws.com/stati​c.nicic​.gov/Libra​r y/02165​
Justice Program, Bureau of Justice Statistics. 7.pdf. Accessed 1 Feb 2020.
Ferguson, L. M., & Wormith, J. S. (2013). A meta-analysis of moral Moullin, J. C., Dickson, K. S., Stadnick, N. A., Rabin, B., & Aarons,
reconation therapy. International Journal of Offender Therapy G. A. (2019). Systematic review of the exploration, preparation,
and Comparative Criminology, 57, 1076–1106. https​: //doi. implementaiton, sustainment (EPIS) framework. Implementation
org/10.1177/03066​24X12​44777​1. Science, 14, 1.
Gale, N. K., Heath, G., Cameron, E., Rashid, S., & Redwood, S. Moullin, J. C., Sklar, M., Green, A., Dickson, K. S., Stadnick, N. A.,
(2013). Using the framework method for the analysis of qualitative Reeder, K., & Aarons, G. A. (2020). Advancing the pragmatic
data in multi-disciplinary health research. BMC Medical Research measurement of sustainment: A narrative review of measures.
Methodology, 13, 117. https:​ //doi.org/10.1186/1471-2288-13-117. Implementation Science Communications, 1, 76.
Gerhart, J., Holman, K., Seymour, B., Dinges, B., & Ronan, G. F. Neal, J. W., Neal, Z. P., VanDyke, E., & Kornbluh, M. (2015). Expe-
(2015). Group process as a mechanism of change in the group dting the analysis of qualitaive data in evaluation: A procedure for
treatment of anger and aggression. International Journal the rapid identification of themes from audio recordings (RITA).
Group Psychotherapy, 65(2), 180–208. https​://doi.org/10.1521/ American Journal of Evaluation, 36(1), 118–132.
ijgp.2015.65.2.180. Palinkas, L. A., Aarons, G. A., Horwitz, S. M., Chamberlain, P., Hurl-
Green, A. E., Trott, E., Willging, C. E., Finn, N. K., Ehrhart, M. G., burt, M., & Landsverk, J. (2011). Mixed method designs in imple-
& Aarons, G. A. (2015). The role of collaborations in sustain- mentation research. Administration and Policy in Mental Health
ing an evidence-based intervention to reduce child neglect. Child and Mental Health Services Research, 38, 44–53.
Abuse and Neglect, 53, 4–16. https​://doi.org/10.1016/j.chiab​ Parhar, K. K., Wormith, J. S., Derkzen, D. M., & Beauregard, A.
u.2015.11.013. M. (2008). Offender coercion in treatment: A meta-analysis of

13
 Administration and Policy in Mental Health and Mental Health Services Research

effectiveness. Criminal Justice and Behavior, 35(9), 1109–1135. Scott, D. A., McGilloway, S., Dempster, M., Browne, F., & Donnelly,
https​://doi.org/10.1177/00938​54808​32016​9. M. (2013). Effectiveness of criminal justice liaison and diversion
Pearson, M., Brand, S. L., Quinn, C., Shaw, J., Maguire, M., Michie, S., services for offenders with mental disorders: A review. Psychiatric
et al. (2015). Using realist review to inform intervention develop- Services, 64(9), 843–849. https​://doi.org/10.1176/appi.ps.20120​
ment: Methodological illustration and conceptual platform for col- 0144.
laborative care in offender mental health. Implementation Science, Stirman, S., Baumann, A. A., & Miller, C. J. (2019). The FRAME: An
10, 134. https​://doi.org/10.1186/s1301​2-015-0321-2. expanded framework for reporting adaptations and modifications
Powell, B. J., Waltz, T. J., Chinman, M. J., Damschroeder, L. J., Smith, to evidence-based interventions. Implementation Science, 14(1),
J. L., Matthieu, M. M., et al. (2015). A refined compilation of 58. https​://doi.org/10.1186/s1301​2-019-0898-y.
implementation strategies: Results from the expert recommenda- Tsai, J., Finlay, A., Flatley, B., Kasprow, W. J., & Clark, S. (2018).
tions for implementing change (ERIC) project. Implementation A national study of veterans treatment court participants: Who
Science, 10, 21. https​://doi.org/10.1186/s1301​2-015-0209-1. benefits and who recidivates. Administration and Policy in Mental
Prendergast, M., Welsh, W. N., Stein, L., Lehman, W., Melnick, G., Health and Mental Health Services Research, 45, 236–244. https​
Warda, U., et al. (2017). Influence of organizational character- ://doi.org/10.1007/S1048​8-017-0816-z.
istics on success in implementing process improvement goals USDA Economic Research Service (2020). Rural-urban commuting
in correctional treatment settings. Journal of Behavioral Health area codes. Ag Data Commons. Retrieved from https​://data.nal.
Services and Research, 44(4), 625–646. https​://doi.org/10.1007/ usda.gov/datas​et/rural​-urban​-commu​ting-area-codes​
s1141​4-016-9531-x. Young, D., Fluellen, R., & Belenko, S. (2004). Criminal recidivism
Ranmuthugala, G., Plumb, J. J., Cunningham, F. C., Georgiou, A., in three models of mandatory drug treatment. Journal of Sub-
Westbrook, J. I., & Braithwaite, J. (2011). How and why are com- stance Abuse Treatment, 27, 313–323. https​://doi.org/10.1016/j.
munities of practice established in the healthcare sector? A sys- jsat.2004.08.007.
tematic review of the literature. BMC Health Services Research,
11, 273. https​://doi.org/10.1186/1472-6963-11-273. Publisher’s Note Springer Nature remains neutral with regard to
Ritchie, M. J., Parker, L. E., Edlund, C. N., & Kirchner, J. E. (2017). jurisdictional claims in published maps and institutional affiliations.
Using implementation facilitation to foster clinical practice quality
and adherence to evidence in challenged settings: A qualitative
study. BMC Health Services Research, 17(1), 294. https​://doi.
org/10.1186/s1291​3-017-2217-0.

13