ORIGINAL ARTICLE

Results of a Multisite Randomized Trial of Supported

Employment Interventions for Individuals
With Severe Mental Illness
Judith A. Cook, PhD; H. Stephen Leff, PhD; Crystal R. Blyler, PhD; Paul B. Gold, PhD; Richard W. Goldberg, PhD;
Kim T. Mueser, PhD; Marcia G. Toprac, PhD; William R. McFarlane, MD; Michael S. Shafer, PhD;
Laura E. Blankertz, PhD; Ken Dudek, MSW; Lisa A. Razzano, PhD; Dennis D. Grey, BA; Jane Burke-Miller, MS

Context: National probability surveys indicate that most tal group programs (330/648 [51%]) were more likely
individuals with schizophrenia and other severe mental than those in comparison programs (245/625 [39%]) to
illnesses are not employed. This multisite study tested work 40 or more hours in a given month (␹2 = 17.66;
the effectiveness of supported employment (SE) models P⬍.001). Finally, participants in experimental group pro-
combining clinical and vocational rehabilitation ser- grams had significantly higher monthly earnings than
vices to establish competitive employment. those in the comparison programs (mean, $122/mo
[n=639] vs $99/mo [n=622]); t1259 = −2.04; P⬍.05). In
Methods: We randomly assigned 1273 outpatients with the multivariate longitudinal analysis, experimental con-
severe mental illness from 7 states in the United States dition subjects were more likely than comparison group
to an experimental SE program or to a comparison or a
subjects to be competitively employed, work 40 or more
services-as-usual condition, with follow-up for 24 months.
hours in a given month, and have higher earnings, de-
Participants were interviewed semiannually, paid em-
ployment was tracked weekly, and vocational and clini- spite controlling for demographic, clinical, work his-
cal services were measured monthly. Mixed-effects ran- tory, disability beneficiary status, and study site con-
dom regression analysis was used to predict the likelihood founders. Moreover, the advantage of experimental over
of competitive employment, working 40 or more hours comparison group participants increased during the 24-
in a given month, and monthly earnings. month study period.

Results: Cumulative results during 24 months show that Conclusion: The SE models tailored by integrating clini-
experimental group participants (359/648 [55%]) were cal and vocational services were more effective than ser-
more likely than those in the comparison programs (210/ vices as usual or unenhanced services.
625 [34%]) to achieve competitive employment
(␹2 = 61.17; P⬍.001). Similarly, patients in experimen- Arch Gen Psychiatry. 2005;62:505-512

I
N THE PAST SEVERAL DECADES, RE- abilities to enter and remain in the labor
search from a variety of fields has force.11 Strong patient and family advo-
presented powerful evidence of cacy movements have asserted the right to
the importance of employment to equal employment opportunities for
people with schizophrenia and people with mental illnesses.12 Federal leg-
other severe mental illnesses.1-4 People with islation has mandated fair hiring and rea-
severe and persistent mental illnesses not sonable accommodation practices for
only want to work2 but can successfully par- people with disabilities,13 although pro-
ticipate in the labor market in a variety of tections have been curtailed by recent Su-
competitive jobs.3,4 Researchers also have preme Court decisions.14 Advancements
explored the benefits of work to individu- in mental health services, including the for-
als with mental illness in terms of allevia- mulation and use of new psychopharma-
tion of poverty,5 therapeutic gain,3,6,7 skill cological agents, have provided patients
acquisition,8 and improvement in quality with more treatment options.15 Despite
of life.3 Society also benefits through re- these developments, most people with psy-
ductions in the use of disability entitle- chiatric disabilities in the United States re-
ments5,9 and the overall costs of care.10 main outside the labor force.16
At the same time, social and scientific Large-scale and nationally representa-
Author Affiliations are listed at developments have provided greater op- tive probability surveys such as the Health
the end of this article. portunities for people with psychiatric dis- Care for Communities Study and the Na-

(REPRINTED) ARCH GEN PSYCHIATRY/ VOL 62, MAY 2005 WWW.ARCHGENPSYCHIATRY.COM

505

©2005 American Medical Association. All rights reserved.

tional Health Interview Survey Disability Supplement in- through consensus if possible and by a democratic vote when
dicate that the unemployment rate among people with opinions were not unanimous.
psychiatric disorders is 3 to 5 times higher than among
those with no disorders.17,18 In these studies, propor- PARTICIPANTS
tions of people with severe mental illness who are out of
the labor force (defined as not working and not looking The sample includes subjects with severe and persistent men-
for work) are substantially higher than these propor- tal illness receiving outpatient psychiatric services. All sub-
tions among people with no mental disorder. For ex- jects met criteria for severe and persistent mental illness based
ample, the 1994-1995 National Health Interview Sur- on diagnosis, duration, and level of disability as established by
the federal Center for Mental Health Services25 along with the
vey Disability Supplement found that 61% of working age following inclusion criteria: 18 years or older at the time of study
adults with mental health disabilities are out of the la- enrollment; willing and able to provide informed consent; and
bor force, compared with only 20% of working-age adults unemployed at the time of entry into the study. All sites re-
in the general population.19 cruited subjects from existing clinical populations via clini-
The early 1980s witnessed the development of a vo- cian referral, self-referral, and word of mouth. The Massachu-
cational rehabilitation model called supported employ- setts site also used newspaper advertisements. Sites adhered to
ment (SE),20 used primarily for individuals with pro- human subjects protection and confidentiality safeguards de-
found mental retardation and severe physical disabilities. termined by their organizations’ institutional review boards.
This new model differed from previous approaches by Across all sites, the eligible pool of subjects was estimated at
emphasizing rapid job placement in socially integrated 10 653; of this initial group, 2883 were approached for partici-
pation (these numbers exclude the Massachusetts site, which
work settings followed by training and ongoing support was unable to provide this information). Across all sites (in-
with no time limits. In the intervening decades, SE has cluding Massachusetts), a total of 1750 individuals consented
been refined for use with patients with severe mental ill- to participate, 1655 completed baseline interviews, and 1648
ness. Reviews and meta-analyses of randomized con- were randomized. Reasons for consent but study nonpartici-
trolled trials of these refined SE approaches have estab- pation included ineligibility, the patient’s decision to with-
lished them as an evidence-based practice in mental health draw, and the researcher’s inability to locate the patient.
treatment.21-23 However, SE services for adults with psy- Of the 1648 participants, 1273 were included in this analy-
chiatric disabilities can be organized under a wide vari- sis. We excluded data from the Pennsylvania site (n=182), which
ety of service delivery models and can incorporate ele- tested an intervention for already employed patients. As a re-
ments of a number of different program and agency types, sult, Pennsylvania subjects did not meet the study inclusion
criterion of unemployment, and the distribution of their out-
calling for rigorous studies of implementation in this field. come data was inappropriate for pooling with that of the re-
This study tested the following 3 hypotheses: (1) ex- maining study sites. The remaining 193 excluded participants
perimental group participants receiving an SE interven- consisted of 65 at the Connecticut site, which used a second
tion would show a greater likelihood than comparison comparison condition (the only site that included 2 control con-
group subjects of achieving competitive employment, 40 ditions); 100 with no vocational outcome data; and 28 who were
or more hours of work per month, and higher monthly employed at the first study interview (ie, although all subjects
earnings; (2) the difference between experimental and reported no employment at the time of study recruitment and
control group outcomes would increase over time; and informed consent, some were later determined to have been
(3) the experimental group would achieve superior out- employed at the time of the baseline interview). No EIDP par-
comes despite the effects of participant characteristics in- ticipant was excluded from the analysis for any other reason,
given its intent-to-treat design. Subjects were interviewed from
cluding demographic features, clinical indicators, prior February 22, 1996, through May 19, 2000, and all were mon-
work history, comorbid physical/developmental disabili- etarily compensated, with amounts determined by each site’s
ties, and receipt of disability income support. local economy (range, $10-$20 per interview).

METHODS INTERVENTIONS

MULTISITE STUDY BACKGROUND Experimental Conditions

The Employment Intervention Demonstration Program (EIDP) All experimental conditions provided SE interventions consist-
included 8 study sites located in Maryland, Connecticut, South ing of (1) integrated services delivered by a multidisciplinary team
Carolina, Pennsylvania, Arizona, Massachusetts, Maine, and that met 3 or more times per week to plan and coordinate em-
Texas. Via the Cooperative Agreement funding mechanism, re- ployment interventions with case management and psychiatric
searchers, federal personnel, and patient representatives de- treatment; (2) placement into competitive employment, de-
veloped and implemented a common protocol and uniform data fined as jobs paying at least minimum wage, in regular, socially
collection methods.24 This effort was led by a coordinating cen- integrated community settings, not reserved for individuals with
ter (CC) based at the Department of Psychiatry, University of disabilities, and held by patients rather than provider agencies;
Illinois at Chicago, in partnership with the Human Services Re- (3) development of jobs tailored to patients’ career preferences;
search Institute in Cambridge, Mass. The original EIDP fund- (4) use of a job search process beginning immediately after pro-
ing announcement (Request for Applications SM 94-09, Cata- gram entry and moving as quickly as a patient desired; and (5)
log of Federal Domestic Assistance 93.125) specified that the provision of ongoing vocational supports freely available through-
program’s governance body would consist of a steering com- out the entire study period rather than gradual withdrawal of sup-
mittee composed of the principal investigator of the CC, the port following successful employment.
individual site principal investigators, federal staff, and con- Several of the study sites implemented preexisting SE mod-
sumer representatives. Decisions were made collaboratively, els tailored for psychiatric populations such as Individual Place-

(REPRINTED) ARCH GEN PSYCHIATRY/ VOL 62, MAY 2005 WWW.ARCHGENPSYCHIATRY.COM

506

©2005 American Medical Association. All rights reserved.

ment and Support26 and the Program of Assertive Community not unusual in research on mental health care,35,36 given that
Treatment.27 The Maryland and Connecticut sites tested the In- randomized designs implemented at multiple sites in real-
dividual Placement and Support model, which emphasizes team- world environments, along with subjects’ ability to seek simi-
based delivery of integrated vocational and mental health ser- lar services outside study parameters, introduce confounds that
vices that are responsive to recipient preferences and offer rapid can only be controlled statistically.
job search for competitive employment followed by ongoing
supports.28,29 The Massachusetts site tested the Program of As-
sertive Community Treatment vocational model, which offers MEASURES
a team-based approach integrating vocational and mental health
services, delivered in clients’ natural environments using a shared The multisite common protocol was a structured set of assess-
caseload, with a team vocational specialist assessing, placing, ments administered at intake and every 6 months for the sub-
training, and supporting service recipients in community- sequent 24-month period. This included the Positive and Nega-
based jobs.30 The South Carolina site blended 2 established mod- tive Syndrome Scale (PANSS),37 which was used to measure the
els, that is, the clinical approach of the Program of Assertive severity of psychiatric symptoms across the following 3 cat-
Community Treatment model and the employment services ap- egories: (1) positive symptoms or productive symptoms, such
proach of the Individual Placement and Support model. as hallucinations and delusions; (2) negative symptoms or defi-
Other experimental conditions consisted of SE interven- cit features, such as blunted affect and emotional withdrawal;
tions defined by the 5 features described above along with en- and (3) general symptoms common across categories of psy-
hancements developed specifically for this study. The Texas site chopathology, such as disorientation, unusual thought con-
added services that were designed to enrich clients’ natural sup- tent, and depression. At study enrollment, the Structured Clini-
port networks by strengthening existing social relationships, cal Interview for DSM-IV38 was administered at 2 sites, whereas
developing new and diverse relationships, and improving reci- case record diagnoses were extracted from clinical files at the
procity in social networks. The Maine site combined SE with remaining sites. Other data collected biannually included pub-
psychoeducational multifamily groups and a Mental Health Em- lic disability income beneficiary status, self-rated level of func-
ployer Consortium. This consortium formed partnerships among tioning, marital status, and educational attainment. Labor force
businesses and mental health and vocational service providers participation data regarding the nature of all paid work were
to share knowledge and resources and develop and support a collected weekly and included earnings, number of hours
diverse workforce including people with mental illnesses.31 The worked, job duties, eligibility for health care benefits and sick
Arizona site delivered comprehensive case management and SE leave, and level of workplace integration. Sites also collected
services through integrated service teams of psychiatrists, case data regarding the types and amounts (in hours) of all voca-
managers, rehabilitation counselors, employment specialists, tional and clinical services received on a monthly basis. Fur-
job developers, and benefits specialists, all located within a single ther information about the EIDP common protocol and data
administrative organization. Further information about the mod- collection methods is available on the EIDP Web site.39
els tested at each site is available at the EIDP Web site (http: Three vocational outcomes were selected for analysis, and
//www.psych.uic.edu/eidp). each was considered to be a fundamentally different concep-
tualization of employment. The first, competitive employ-
ment, is defined as a job that pays minimum wage or higher; is
located in a mainstream, socially integrated setting; is not set
Control Conditions
aside for people with disabilities; and is held independently (ie,
not controlled by a service agency). The second variable, work-
Under the cooperative agreement mechanism,24 sites were given ing for 40 or more hours in a single month, is an outcome used
leeway in choosing their experimental models and compari- by the Centers for Medicare and Medicaid Services in their dem-
son conditions. Owing to ethical considerations, none of the onstration program, Demonstration to Maintain Indepen-
sites used a no-treatment control condition. Because subjects dence and Employment, issued June 7, 2000 (Catalog of Fed-
entered the study with an interest in working, comparison con- eral Domestic Assistance 93.779). This outcome evaluates the
dition subjects were not prevented from receiving vocational intensity of employment in terms of a minimum number of hours
services outside of the study. Thus, as with many multisite stud- worked during a 1-month period. The third outcome, monthly
ies, the comparison conditions varied somewhat from site to earnings from paid employment, is a standard labor force mea-
site. In 4 sites (Arizona, Connecticut, South Carolina, and Mary- sure that captures the economic return to the individual from
land), the comparison condition consisted of services as usual, his or her labor force participation.
defined by standard vocational rehabilitation services avail-
able in the local area. In 2 sites (Maine and Texas), a weaker
version of the experimental condition served as the compari- DATA QUALITY ASSURANCE
son (ie, no social network enhancements in Texas and no em-
ployer consortium in Maine). The Massachusetts site com- Given its critical importance in a multisite study, the CC con-
pared 2 preexisting vocational models, each hypothesized to ducted routine checks on data quality throughout the study. These
be superior to the other for different vocational outcomes. For included programmed logic checks of data at the time of sub-
the outcomes analyzed herein, the Massachusetts site’s com- mission to the CC to identify outlier or out-of-range values, con-
parison condition was the International Center for Clubhouse flicting subject reports within interviews, and disparities be-
Development program, which offers facility-based services tween patient self-reports and provider reports. At biannual
through membership in a mutually supportive community that meetings of the EIDP Steering Committee, the CC presented de-
operates according to a work-ordered day, with clients and staff tailed reports on more than 150 data elements (excluding out-
working together in Clubhouse-owned jobs (transitional em- come variables to which sites remained blinded throughout the
ployment placements) or independent competitive jobs.32,33 study), thus allowing the group to monitor and correct poten-
Within the nomenclature of randomized designs, this study is tial data problems as the study progressed. In addition, telecon-
an “implementation effectiveness trial,”34 a randomized experi- ference recalibration trainings for all EIDP PANSS interviewers
ment designed to test the effectiveness of an intervention with were conducted from November 1, 1996, through January 31,
established efficacy (SE) via implementation under real-world 2000. These included monthly distribution of videotaped inter-
conditions of availability and patient acceptance. Such trials are views that were rated by every interviewer and then submitted

(REPRINTED) ARCH GEN PSYCHIATRY/ VOL 62, MAY 2005 WWW.ARCHGENPSYCHIATRY.COM

507

©2005 American Medical Association. All rights reserved.

 Table 1. Sample Characteristics by Study Condition*

Study Condition

Experimental Comparison
Characteristic (n = 648) (n = 625) Statistic P Value
Education, high school 426 (66) 413 (66) ␹2
1 = 0.02 .91
Job in past 5 y 385 (64) 378 (60) ␹2
1 = 0.74 .81
White 329 (51) 307 (49) ␹2
1 = 3.47 .58
Male 345 (53) 333 (53) ␹2
1 = 0.00 .99
Married/cohabiting 86 (13) 67 (11) ␹2
1 = 2.01 .16
Coresident children 124 (20) 114 (19) ␹2
1 = 0.13 .72
DSM-IV Axis I diagnosis† 333 (51) 313 (50) ␹2
1 = 0.22 .64
Level of functioning, fair/poor 356 (55) 342 (55) ␹2
1 = 0.00 .98
Health comorbidity 258 (40) 251 (40) ␹2
1 = 0.16 .90
SSI beneficiary 210 (32) 227 (36) ␹2
1 = 2.16 .14
SSDI beneficiary 170 (26) 137 (22) ␹2
1 = 3.24 .07
SSI and SSDI beneficiary 68 (10) 85 (14) ␹2
1 = 2.90 .09
No. of symptoms at baseline, mean (SD)‡
Positive 14.4 (5.2) 14.4 (5.1) t1266 = 0.01§ .99
Negative 16.1 (5.5) 16.3 (5.4) t1266 = 0.56§ .57
General 34.1 (9.0) 34.5 (8.6) t1266 = 0.81§ .42
Age, mean (SD), y 38.5 (9.1) 38.4 (9.8) t1270 = −0.24§ .81

Abbreviations: SSDI, Social Security Disability Insurance; SSI, Supplemental Security Income.
*Unless otherwise indicated, data are given as number (percentage) of participants. Numbers vary because of missing data.
†Indicates DSM-IV code 295.xx (schizophrenia spectrum disorders).
‡Symptoms were assessed using the Positive and Negative Syndrome Scale.37
§Degrees of freedom vary because of missing data.

to the CC for computation of within- and cross-site interrater RESULTS

reliability measures. In addition, gold standard ratings of each
tape were performed by a PANSS expert (PANSS coauthor Lewis
A. Opler, MD), and these were distributed to all sites and dis- SUBJECT CHARACTERISTICS
cussed on mandatory conference calls led by the expert. An in-
dependent evaluation of the psychometric properties of the EIDP
common protocol, commissioned by the CC, documented high
Demographics, employment histories, clinical status, and
levels of interrater and test-retest reliability.40 public disability income beneficiary status of study sub-
jects are shown in Table 1. The success of randomiza-
tion was confirmed by the absence of statistically signifi-
DATA ANALYSIS cant differences at study baseline between experimental
and comparison group subjects on education, work his-
Outcome data were analyzed in multiple stages. First, the cu- tory, race/ethnicity, sex, marital status, coresident chil-
mulative effect of the study condition on each of the depen-
dent variables was tested in unadjusted bivariate analyses. Sec-
dren, diagnosis of schizophrenia spectrum disorder, level
ond, the longitudinal relationships between study condition and of functioning, health comorbidity, disability income ben-
observed outcomes on each dependent variable were graphed eficiary status, age, and severity of positive, negative, and
across all 24 months of study participation. Third, a multivar- general symptoms.
iate, longitudinal random-effects logistic regression analysis was
conducted to test for differences between experimental and com- INTERVENTION IMPLEMENTATION
parison conditions over time. A 2-level random intercepts model,
controlling for study site as a fixed effect along with other co-
variates, was fit to the data. This approach was chosen be- Subjects were offered but were not required to accept ser-
cause of the superiority of random regression models41 in ad- vices in this implementation effectiveness trial. More-
dressing issues commonly found in longitudinal multisite data, over, because the experimental condition consisted of dif-
including state dependency or serial correlations among re- ferent SE models, no studywide measure of service fidelity
peated observations within individual participants; individual was available. Thus, as a manipulation check of pro-
heterogeneity or varying propensities toward the outcomes of gram fidelity, we compared vocational and clinical ser-
interest due to subjects’ predispositions and other unobserved vices data for subjects in the 2 study conditions. Al-
influences; missing observations due to the fact that not all sub- though 96% of the experimental group received vocational
jects completed all assessments; and inclusion of fixed covar-
iates (sex, age, race/ethnicity, study site, prior work history,
services (including vocational assessment, treatment plan-
marital status, presence of coresident children younger than ning and counseling, job placement, employment sup-
18 years, level of education, baseline diagnosis, and baseline port, and vocational skills training), only 67% of those
health comorbidity), along with time-varying covariates (posi- in the control conditions received these services. Con-
tive, negative, and general symptoms, level of functioning, and versely, clinical services (including psychiatric evalua-
disability income beneficiary status). tion and diagnosis, medication evaluation and manage-

(REPRINTED) ARCH GEN PSYCHIATRY/ VOL 62, MAY 2005 WWW.ARCHGENPSYCHIATRY.COM

508

©2005 American Medical Association. All rights reserved.

 Table 2. Manipulation Check of Fidelity to Supported Employment Model: Differences in Vocational and Clinical Services Received
by Study Condition*

Study Condition

Experimental Comparison
Service Type (n = 648) (n = 625) Statistic P Value
Vocational services
Vocational assessment 505 (78) 223 (36) ␹12 = 231.98 ⬍.001
Job development 544 (84) 261 (42) ␹12 = 243.60 ⬍.001
Employer collaboration 296 (46) 154 (25) ␹12 = 61.62 ⬍.001
Job support groups 283 (44) 107 (17) ␹12 = 105.55 ⬍.001
Family/friend collaboration 367 (57) 161 (26) ␹12 = 124.95 ⬍.001
Vocational treatment planning 497 (77) 201 (32) ␹12 = 254.81 ⬍.001
Job skills training 439 (68) 239 (38) ␹12 = 111.27 ⬍.001
Vocational counseling 501 (77) 180 (29) ␹12 = 301.00 ⬍.001
On-site job support 279 (43) 131 (21) ␹12 = 71.14 ⬍.001
Transportation 293 (45) 103 (16) ␹12 = 122.59 ⬍.001
Any vocational services 625 (96) 417 (67) ␹12 = 189.33 ⬍.001
Any clinical services† 640 (99) 616 (99) ␹12 = 0.10 .81
Vocational services, mean (median), h 74 (43) 43 (5) t1271 = −5.78 ⬍.001
Clinical services, mean (median), h 125 (69) 124 (47) t1271 = −0.08 .93

*Unless otherwise indicated, data are given as number (percentage) of participants.

†Includes case management, individual counseling, group counseling, family/couples counseling, emergency services, evaluation/diagnosis, medication
evaluation/maintenance, and partial hospitalization/psychosocial rehabilitation.

ment, individual counseling, family or couples counseling,

30
group counseling, case management, psychosocial reha- Experimental Group Comparison Group
Employed Competitively, Mean %

bilitation/partial hospitalization, and emergency ser-

vices) were received by identical proportions (99%) of 20
experimental and comparison subjects (Table 2). Simi-
larly, experimental group subjects received an average
of 74 hours (median, 43 hours) of vocational services per 10
person, whereas the comparison group received an av-
erage of 43 hours (median, 5 hours) of vocational ser-
vices. The experimental and comparison groups re- 0

ceived almost identical average amounts of clinical 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Month in Study
services, ie, 125 hours (median, 69 hours) and 124 hours
(median, 47 hours), respectively.
Figure 1. Mean proportion of participants employed competitively per month
of study participation by treatment group. Groups are described in the
FOLLOW-UP RATES AND ATTRITION “Interventions” subsection of the “Methods” section.

There were no statistically significant differences in fol- erage of 1 year older than noncompleters (39 vs 38 years;
low-up rates between the experimental and control con- t1270 =−2.24; P =.03).
ditions. Interviews were completed by 578 (89%) of the
experimental condition and 546 (87%) of comparison con- EMPLOYMENT OUTCOMES
dition subjects at the 6-month follow-up; by 547 (84%)
of the experimental condition and 504 (81%) of com- Viewed longitudinally, higher proportions of experimen-
parison condition subjects at the 12-month follow-up; tal group subjects secured competitive employment and
by 521 (80%) of the experimental condition and 481 worked 40 or more hours in a given month during most
(77%) of comparison condition subjects at the 18- of the 24-month study period; they also had significantly
month follow-up; and by 515 (79%) of the experimen- higher earnings (Figures 1, 2, and 3, respectively).
tal condition and 478 (76%) of comparison condition sub- Viewed cumulatively, experimental group participants
jects at the 24-month follow-up. Of 1273 participants, (359/648 [55%]) were more likely than those in the com-
824 (65%) completed 5 interviews, 173 (14%) com- parison programs (210/625 [34%]) to achieve competi-
pleted 4, 122 (10%) completed 3, 111 (9%) completed tive employment (␹2 =61.17; P⬍.001); experimental group
2, and the remaining 43 (3%) completed 1. Those com- subjects (330/648 [51%]) were more likely than those in
pleting 5 interviews were compared with all others re- the comparison group (245/625 [39%]) to work 40 or more
garding study condition and model covariates. The only hours in a given month (␹2 =17.66 P⬍.001); and experi-
significant differences were in sex and age; ie, 420 (51%) mental group participants had significantly higher monthly
of completers were men compared with 258 (57%) of non- earnings than the comparison groups (mean, $122/mo vs
completers (␹12=4.92; P=.03); and completers were an av- $99/mo); t1259 =−2.04; P=.04).

(REPRINTED) ARCH GEN PSYCHIATRY/ VOL 62, MAY 2005 WWW.ARCHGENPSYCHIATRY.COM

509

©2005 American Medical Association. All rights reserved.

 that included transitional employment placements did
30
Experimental Group Comparison Group not substantially alter the results reported herein. The
Employed ≥ 40 Hours, Mean %
final report for the Massachusetts site of the EIDP43 dem-
20 onstrates further details of the site-specific analysis us-
ing this alternative definition.

10
COMMENT

0 A major finding of this study was that, compared with

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
services as usual or unenhanced comparison programs,
Month in Study
the experimental programs had a significant and posi-
tive effect on each of the 3 employment outcomes, even
Figure 2. Mean proportion of participants employed 40 hours or more per
month of study participation by treatment group. Groups are described in the when controlling for the effects of time, time⫻condition
“Interventions” subsection of the “Methods” section. interaction, participant characteristics, and study site. In
addition, the advantage of the experimental group par-
ticipants increased over time relative to the comparison
No. of Dollars Earned per Month, Mean

190 group. A number of programs were tested in the experi-

170 mental condition and, as a group, they demonstrated con-
150
sistently better results than their comparisons. More-
130
over, the experimental program models shared common
110
characteristics, including a focus on integration of clini-
90
cal and employment services delivered by providers work-
70
ing on the same treatment team, availability of ongoing
50
Experimental Group Comparison Group job support vs gradual withdraw of services after suc-
30
cessful employment, development of jobs consonant with
10
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 patients’ career preferences, and an emphasis on rapid
Month in Study job placement into permanent, community-based, so-
cially integrated, competitive employment.
Figure 3. Mean number of dollars earned per month of study participation by The first major caveat to our results concerns the fact
treatment group. Groups are described in the “Interventions” subsection of
the “Methods” section. that the study’s subjects were not drawn from a national
probability sample of individuals with severe and per-
sistent mental illness, which limits the generalizability
Multivariate analyses of longitudinal patterns of all 3 of the findings. Instead, the subjects consist of 7 cohorts
employment outcomes (Table 3) showed significant of individuals seeking vocational rehabilitation services
main effects for treatment group, with experimental group at 6 urban sites and 1 rural site in 7 states, 5 of which
participants achieving significantly better outcomes. In were on the East Coast, 1 in the South, and 1 in the south-
this analysis, monthly earnings were grouped into 3 or- west region of the United States. A second major caveat
dinal categories because of their nonnormal distribu- concerns the nature of the study as an implementation
tion: zero earnings; $1 to $199; and $200 or greater per effectiveness trial. Because the experimental conditions
month. The effects of time were similar for the experi- tested were not identical, it was impossible to conduct a
mental and comparison groups across all 3 outcomes, that study-wide assessment of program fidelity. Manipula-
is, the probability of competitive employment, working tion checks included an analysis of vocational and clini-
40 or more hours per month, and higher monthly earn- cal services delivered to subjects in the 2 conditions, and
ings increased significantly over time. In addition, the ef- fidelity assessments were completed independently by in-
fects of time were nonlinear (as shown by the signifi- vestigators regarding degree of implementation of the 5
cant estimate for time2) for all 3 outcomes, with the essential components of SE, with results of both checks
probability of competitive employment, working 40 or indicating high levels of adherence to the SE model. How-
more hours per month, and higher earnings increasing ever, uniformity in experimental models would have al-
dramatically during the first 6 to 9 months of program lowed us to monitor fidelity more closely and to make
participation and then leveling off for the remainder of more definitive statements about the quality of program
the study. Thus, after adjusting for patient characteris- implementation. Third, the comparison conditions did
tics and study site, all 3 employment outcomes were more not represent a no-treatment control condition; thus, some
likely among the experimental group than comparison of the subjects in these conditions received substantial
group subjects, and this difference increased over time. amounts of vocational services. As such, study findings
A final analysis issue concerned the definition of com- must be viewed with caution, since it is possible that the
petitive employment used in this study. Under this defi- cross-site study underestimated the effects of SE rela-
nition, transitional employment placements, such as those tive to conditions in which far fewer employment ser-
provided through the International Center for Club- vices and supports were available, as is typical for most
house Development program, were not coded as com- psychiatric outpatients attempting to return to work. On
petitive employment. Multisite analysis (not shown) us- the other hand, it is also possible that, because of the large
ing an alternative definition of competitive employment sample size, some differences achieved statistical signifi-

(REPRINTED) ARCH GEN PSYCHIATRY/ VOL 62, MAY 2005 WWW.ARCHGENPSYCHIATRY.COM

510

©2005 American Medical Association. All rights reserved.

 Table 3. Effects of Study Condition (Experimental vs Comparison) on 3 Employment Outcomes*

Competitive Employment Worked ⱖ40 h/mo Monthly Earnings (Ordinal)

Variable Est (SE) z Score P Value Est (SE) z Score P Value Est (SE) z Score P Value
Intercept −5.08 (0.22) −23.61 ⬍.001 −5.31 (0.22) −23.62 ⬍.001 −5.17 (0.19) −27.70 ⬍.001
Treatment (experimental)† 0.67 (0.10) 6.75 ⬍.001 0.33 (0.11) 3.06 .002 0.38 (0.08) 4.60 ⬍.001
Time (month) 0.15 (0.01) 10.98 ⬍.001 0.20 (0.01) 14.82 ⬍.001 0.19 (0.01) 20.18 ⬍.001
Treatment ⫻ time 0.07 (0.02) 4.02 ⬍.001 0.10 (0.02) 5.66 ⬍.001 0.03 (0.01) 2.68 .007
Time2 ⬍−0.01 (0.00) −6.80 ⬍.001 ⬍−0.01 (0.00) −11.88 ⬍.001 ⬍−0.01 (0.00) −15.26 ⬍.001
Treatment ⫻ (time)2 ⬍−0.01 (0.00) −4.60 ⬍.001 ⬍−0.01 (0.00) −4.82 ⬍.001 ⬍−0.01 (0.00) −2.50 .01
Education (high school) 0.30 (0.06) 5.23 ⬍.001 0.52 (0.06) 8.06 ⬍.001 0.50 (0.05) 9.51 ⬍.001
Work history (job in past 5 y) 1.16 (0.06) 18.33 ⬍.001 0.73 (0.06) 11.34 ⬍.001 1.19 (0.05) 22.43 ⬍.001
White −0.11 (0.06) −1.98 .047 0.46 (0.06) −7.65 ⬍.001 −0.31 (0.05) −5.95 ⬍.001
Age (10-y intervals) 0.10 (0.02) 3.98 ⬍.001 −0.14 (0.03) −4.77 ⬍.001 −0.18 (0.02) −7.56 ⬍.001
Male −0.14 (0.05) −2.71 .007 0.67 (0.06) 11.52 ⬍.001 −0.32 (0.05) −6.81 ⬍.001
Married/cohabiting −0.15 (0.06) −2.51 .012 0.25 (0.06) 4.41 ⬍.001 0.04 (0.05) 0.76 .450
Coresident children 0.09 (0.05) 1.63 .10 −0.26 (0.05) −4.80 ⬍.001 −0.18 (0.04) −4.04 ⬍.001
DSM-IV Axis I diagnosis‡ −0.42 (0.05) −7.77 ⬍.001 −0.29 (0.06) −4.66 ⬍.001 ⬍−0.01 (0.05) −0.01 .99
Symptoms at baseline§
General −0.02 (0.00) −6.35 ⬍.001 −0.04 (0.00) 12.85 ⬍.001 ⬍−0.01 (0.00) −3.28 .001
Positive ⬍−0.01 (0.00) −1.10 .27 ⬍−0.01 (0.00) 0.26 .80 ⬍0.01 (0.00) 0.22 .82
Negative ⬍0.01 (0.00) 1.51 .13 −0.01 (0.00) −3.35 .001 −0.02 (0.00) −5.56 ⬍.001
Level of functioning (fair/poor) 0.05 (0.02) 2.70 .007 0.24 (0.02) 12.01 ⬍.001 0.14 (0.02) 9.56 ⬍.001
Health comorbidity −0.24 (0.05) −4.47 ⬍.001 −0.40 (0.06) −6.89 ⬍.001 0.08 (0.05) 1.70 .09
SSI beneficiary (vs nonbeneficiary) −0.38 (0.06) −6.83 ⬍.001 −0.19 (0.05) −3.44 .001 −0.41 (0.04) −9.34 ⬍.001
SSDI beneficiary (vs nonbeneficiary) 0.05 (0.05) 1.05 .30 0.27 (0.06) 4.90 ⬍.001 −0.26 (0.04) −5.76 ⬍.001
SSI and SSDI beneficiary −0.51 (0.07) −7.61 ⬍.001 −0.04 (0.07) −0.54 .59 −0.53 (0.06) −9.32 ⬍.001
(vs nonbeneficiary)

Abbreviations: Est, estimate; SSDI, Social Security Disability Insurance; SSI, Supplemental Security Income.
*Maximum marginal likelihood estimates from mixed-effects random regression models controlling for study site. Analyses were conducted with random-
effects ordinal regression analysis (MIXOR 2.042). The overall sample size is 1273 (648 in the experimental, 625 in the comparison condition).
†The variable is treatment group, where 1 equals the experimental group and 0, the comparison group.
‡Indicates DSM-IV code 295.xx (schizophrenia spectrum disorders).
§Symptoms were assessed using the Positive and Negative Syndrome Scale.37

cance no matter how inconsequential they were clini- other approaches such as standard vocational rehabili-
cally or vocationally. Finally, it must also be acknowl- tation services as usual. Given research cited earlier con-
edged that a longer period of data collection (which was cerning the benefits of employment for individuals with
achieved by some sites that continued to collect data for psychiatric disabilities, this knowledge can be used to cre-
36 months or longer) might have revealed different find- ate the next generation of models that contribute to re-
ings than those attained at the end of the 24 months covery and increased community integration of those with
tracked in this study. All of these limitations suggest that severe psychiatric disorders.
caution should be applied to interpretations from study
results. However, to date and to our knowledge, this is Submitted for Publication: May 13, 2004; final revision
the most comprehensive, rigorously monitored, and thor- received August 5, 2004; accepted October 7, 2004.
oughly analyzed set of data collected using a uniform pro- Author Affiliations: Department of Psychiatry, Univer-
tocol that exhibits noteworthy validity and reliability sity of Illinois at Chicago (Drs Cook and Razzano, Mr Grey,
across the largest number of sites ever examined in a study and Ms Burke-Miller); Human Services Research Insti-
of SE interventions. tute, Cambridge, Mass (Dr Leff); Center for Mental Health
Study results build on prior evidence concerning best Services, Rockville, Md (Dr Blyler); Department of Psy-
practices in vocational rehabilitation, but go further in chiatry & Behavioral Sciences, Medical University of South
demonstrating the effectiveness of SE on a larger scale, Carolina, Charleston (Dr Gold); Department of Psychia-
with more racially and ethnically diverse populations, at try, University of Maryland, Baltimore (Dr Goldberg); New
multiple geographic sites, and using different SE mod- Hampshire–Dartmouth Psychiatric Research Center, Dart-
els. Given the diversity of sites and models tested, the mouth University, Concord, NH (Dr Mueser); Texas De-
results suggest that these programs have the potential to partment of Mental Health, Austin (Dr Toprac); Depart-
work anywhere, for a wide variety of service consumers. ment of Psychiatry, Maine Medical Center, Portland (Dr
At the same time, it must be acknowledged that most of McFarlane); Community Rehabiliation Division, Univer-
the study participants did not achieve success on each sity of Arizona, Tucson (Dr Shafer); Connections, CSP In-
of the outcome measures, suggesting a need for contin- corporated, Wilmington, Del (Dr Blankertz); and Foun-
ued enhancement of these models. tain House, New York, NY (Mr Dudek).
The findings of this study confirm the superiority of Correspondence: Judith Cook, PhD, Department of Psy-
SE tailored for individuals with psychiatric disability over chiatry, University of Illinois at Chicago, 104 S Michi-

(REPRINTED) ARCH GEN PSYCHIATRY/ VOL 62, MAY 2005 WWW.ARCHGENPSYCHIATRY.COM

511

©2005 American Medical Association. All rights reserved.

gan Ave, Suite 900, Chicago, IL 60603 (cook@ripco and Mental Health Services Administration, US Dept of Health and Human Ser-
vices; 1998:113-122.
.com). 19. Kaye HS. Employment and Social Participation Among People with Mental Health
Funding/Support: This study is part of the Employ- Disabilities. San Francisco, Calif: National Disability Statistics and Policy Forum; 2002.
ment Intervention Demonstration Program (EIDP), a mul- 20. Wehman P, Kregel J. A supported work approach to competitive employment
tisite collaboration among 8 research demonstration sites, for individuals with moderate to severe handicaps. J Assoc Pers Sev Handicaps.
a Coordinating Center, and the Center for Mental Health 1985;10:3-11.
21. Lehman AF. Vocational rehabilitation in schizophrenia. Schizophr Bull. 1995;21:
Services/Substance Abuse and Mental Health Services Ad- 645-656.
ministration (CMHS/SAMHSA), Rockville, Md. This re- 22. Tsang HW, Pearson V. A work-related social skills training for people with schizo-
search was supported by cooperative agreement SM51820 phrenia in Hong Kong. Schizophr Bull. 2001;27:139-148.
from CMHS/SAMHSA. 23. Twamley EW, Jeste DV, Lehman AF. Vocational rehabilitation in schizophrenia
and other psychotic disorders: a literature review and meta-analysis of random-
Disclaimer: The views expressed herein are those of the ized controlled trials. J Nerv Ment Dis. 2003;191:515-523.
authors and do not necessarily reflect the policy or po- 24. Cook JA, Carey MA, Razzano LA, Burke JK, Blyler CR. The Pioneer: the Employ-
sition of any federal agency. ment Intervention Demonstration Program. N Dir Eval. 2002;94:31-44.
25. Manderscheid RW, Sonnenschein MA. Mental Health, United States, 1992. Wash-
ington, DC: US Government Printing Office; 1992.
REFERENCES 26. Drake RE, McHugo GJ, Bebout RR, Becker DR, Harris M, Bond GR, Quimby E.
A randomized clinical trial of supported employment for inner-city patients with
1. Cook JA, Razzano L. Vocational rehabilitation for persons with schizophrenia: severe mental disorders. Arch Gen Psychiatry. 1999;56:627-633.
recent research and implications for practice. Schizophr Bull. 2000;26:87- 27. Russert MG, Frey JL. The PACT vocational model: a step into the future. Psy-
103. chosocial Rehabil J. 1991;14:7-18.
2. Rogers ES, Anthony WA, Toole J, Brown MA. Vocational outcomes following psy- 28. Becker DR, Drake RE. A Working Life: The Individual Placement and Support (IPS)
chosocial rehabilitation: a longitudinal study of three programs. J Vocational Program. Concord, NH: Dartmouth Psychiatric Research Center; 1993.
Rehabil. 1991;1:21-29. 29. Drake RE, Becker DR. The Individual Placement and Support model of sup-
3. Bond GR, Becker DR, Drake RE, Rapp CA, Meisler N, Lehman AF, Bell MD, Blyler ported employment. Psychiatr Serv. 1996;47:473-475.
CR. Implementing supported employment as an evidence-based practice. Psy- 30. Frey JL, Godfrey M. A comprehensive clinical vocational assessment: the PACT
chiatr Serv. 2001;52:313-322. approach. J Appl Rehabil Counseling. 1991;22:25-28.
4. Crowther RE, Marshall M, Bond GR, Huxley P. Helping people with severe men- 31. Balser R, Hornby H, Fraser K, McKenzie C. Business Partnerships, Employment
tal illness to obtain work: systematic review. BMJ. 2001;322:204-208. Outcomes: The Mental Health Employer Consortium. Portland, Me: Maine Medi-
5. Polak P, Warner R. The economic life of seriously mentally ill people in the cal Center; 2001.
community. Psychiatr Serv. 1996;47:270-274. 32. Wolff N. Using randomized controlled trials to evaluation socially complex ser-
6. Bell MD, Lysaker PH, Milstein RM. Clinical benefits of paid work activity in vices: problems, challenges, and recommendations. J Ment Health Policy Econ.
schizophrenia. Schizophr Bull. 1996;22:51-67. 2000;3:97-109.
7. Rogers ES, Sciarappa K, MacDonald-Wilson K. A benefit-cost analysis of a sup- 33. Beard JH. The rehabilitation services of Fountain House. In: Stein L, Test MA,
ported employment model for persons with psychiatric disabilities. Eval Pro- eds. Alternatives to Mental Hospital Treatment: Proceedings of a Conference on
gram Plann. 1995;18:105-115. Alternatives to Mental Hospital Treatment Held in Madison, Wisconsin. New York,
8. Kern RS, Liberman RP, Kopelowicz A, Mintz J, Green MF. Applications of error- NY: Plenum Press; 1987:201-208.
less learning for improving work performance in persons with schizophrenia. 34. Gordis L. Epidemiology. 2nd ed. Philadelphia, Pa: WB Saunders Co; 2000.
Am J Psychiatry. 2002;159:1921-1926. 35. Anderson SB. We Are Not Alone: Fountain House and the Development of Club-
9. Kouzis AC, Eaton WW. Psychopathology and the initiation of disability payments. house Culture. New York, NY: Fountain House Inc; 1998.
Psychiatr Serv. 2000;51:908-913. 36. Essock SM, Drake RE, Frank RG, McGuire TG. Randomized clinical trials in evidence-
10. Drake RE, Mchugo GJ, Becker DR, Anthony WA, Clark RE. The New Hampshire based mental health care: getting the right answer to the right questions. Schizophr
study of supported employment for people with severe mental illness: voca- Bull. 2003;29:115-123.
tional outcomes. J Consult Clin Psychol. 1996;64:391-399. 37. Kay SR, Fiszbein A, Opler LA. The Positive and Negative Syndrome Scale (PANSS)
11. Cook JA, Burke JK. Public policy and employment of people with disabilities: ex- for schizophrenia. Schizophr Bull. 1987;13:261-276.
ploring new paradigms. Behav Sci Law. 2002;20:541-557. 38. First MB, Spitzer RL, Gibbon M, Williams JBW. Structured Clinical Interview for
12. Cook JA, Wright ER. Medical sociology and the study of severe mental illness: DSM-IV: Patient Edition (SCID-P), Version 2. New York: New York State Psy-
reflections on past accomplishments and directions for future research. J Health chiatric Institute, Division of Biometrics Research; 1995.
Soc Behav. 1995;36:95-114. 39. Employment Intervention Demonstration Program: common protocol and
13. Americans With Disabilities Act of 1990, Public Law 101-336, 42 USC §12101 documentation. Available at http://www.psych.uic.edu/eidp/. Accessed March 11,
(1990). 2005.
14. Petrila J, Brink T. Mental illness and changing definitions of disability under the 40. Salyers MP, McHugo GJ, Cook JA, Razzano LA, Drake RE, Mueser KT. Reliabil-
Americans with Disabilities Act. Psychiatr Serv. 2001;52:626-630. ity of instruments in a cooperative, multi-site study: Employment Intervention
15. Meyer PS, Bond GR, Tunis SL, McCoy ML. Comparison between the effects of Demonstration Program. Ment Health Serv Res. 2001;3:129-140.
atypical and traditional antipsychotics on work status for clients in a psychiatric 41. Gibbons RD, Hedeker D, Elkin I, Waternaux C, Kraemer HC, Greenhouse JB, Shea
rehabilitation program. J Clin Psychiatry. 2002;63:108-116. MT, Imber SD, Sotsky SM, Watkins JT. Some conceptual and statistical issues
16. New Freedom Commission on Mental Health. Achieving the Promise: Trans- in analysis of longitudinal psychiatric data: application of NIMH Treatment of De-
forming Mental Health Care in America: Final Report. Rockville, Md: US Dept of pression Collaborative Research Program dataset. Arch Gen Psychiatry. 1993;
Health and Human Services; 2003. DHHS publication SMA-03-3832. 50:739-750.
17. Sturm R, Gresenz CR, Pacula RL, Wells KB. Labor force participation by per- 42. Gibbons RD, Hedeker D. MIXOR: a computer program for mixed-effects ordinal
sons with mental illness. Psychiatr Serv. 1999;50:1407. regression analysis. Comput Methods Programs Biomed. 1996;49:157-176.
18. Willis AG, Willis GG, Male A, Henderson M, Manderscheid RW. Mental illness 43. Macias C. An experimental comparison of PACT and clubhouse Fountain House
and disability in the US adult household population. In: Manderscheid RW, Hender- Web site. 2001. Available at: http://www.fountainhouse.org/pdfs/samhsa_final
son MJ, eds. Mental Health, United States, 1998. Rockville, Md: Substance Abuse .pdf. Accessed March 11, 2005.

(REPRINTED) ARCH GEN PSYCHIATRY/ VOL 62, MAY 2005 WWW.ARCHGENPSYCHIATRY.COM

512

©2005 American Medical Association. All rights reserved.