Canadian Psychology / Psychologie canadienne

ISSN: 0708-5591 2019, Vol. 60, No. 4, 290 –301

© 2019 Canadian Psychological Association http://dx.doi.org/10.1037/cap0000188

Recommendations to Enhance Psychotherapy Supervision in Psychology

Edward A. Johnson
University of Manitoba

Two primary goals of clinical supervision are to promote and evaluate trainees’ acquisition of profes-
sional competencies and to protect and enhance client welfare. To help supervisors achieve these goals,
this article reviews the current usage and benefits of 5 important supervisory practices: direct observation
(live or recorded); progress monitoring; experiential learning methods (modelling, role-playing, and
providing constructive feedback based on observation); less biased summative evaluation procedures;
and diversity-focused supervision. The article concludes with recommendations for research, changes in
supervisory practice, and changes in the regulation of supervision and supervisor training within
accreditation standards and provincial requirements for licensure.

Public Significance Statement

Professional psychology training relies extensively on supervision to develop, maintain, and evaluate
the professional competencies of psychologists-in-training and to oversee the welfare of the clients
they serve. This literature review identified gaps between recommended and typical supervision of
professional psychology supervisees. Recommendations to enhance the practice of supervision
during the course of professional training are to incorporate direct observation, client data, experi-
ential learning methods, unbiased evaluation methods, and a greater focus on diversity.

Keywords: supervision, clinical supervision, supervisory recommendations, effective supervision,

supervision competencies

I am delighted to have the opportunity to write on the profes- and the restorative obliges supervisors to help trainees cope with
sional training summit theme of supervision. In reflecting on the emotional demands of clinical practice (Proctor, 1988). This
which facets of this immensely broad and rich theme would be article examines how practices relating to the first two supervisory
most helpful to discuss, I chose to focus on five psychotherapy functions can be enhanced.
supervision practices that offer considerable promise for enhanc- The importance of increasing the efficacy of supervision arises
ing the effectiveness and impact of supervision on supervisee from the centrality of its role in professional training. Professional
learning or client outcomes yet are greatly underutilized in typical psychology training relies extensively on supervision to develop,
practice or are underdeveloped in the research literature. In each of maintain, and evaluate professional competencies. For example, in
the five areas, I review evidence documenting a gap between best clinical psychology, supervisees obtain an average of 600 hr of
practice and typical practice and discuss how these shortfalls may clinical supervision during their graduate training and prelicensure
undermine important outcomes. Although this review casts a crit- period (Association of Psychology Postdoctoral and Internship
ical light on current supervisory practice, it does so in the hope of Centers, 2012). This significant investment of time underscores the
enhancing supervision practice to further the training summit goal importance to the profession of both developing supervision skills
of benefitting the development of professional psychologists. as a core competency, and of ensuring that supervisors employ
Clinical supervision (hereafter supervision) is a complex role effective supervision practices so that supervision achieves its
that serves three distinct functions: the normative entrusts super- purposes and is consistently delivered competently and safely.
visors with responsibility to see that trainees serve clients in a safe, Supervision’s formative influence is central to trainees’ profes-
ethical, and effective manner; the formative tasks supervisors with sional development at the practicum, internship, and postdoctoral
responsibility for developing and evaluating trainees’ competency; levels and, in some countries and Quebec, continues after licensure
(Bernard & Goodyear, 2019). Supervision enhances supervisee
professional self-awareness and self-efficacy, while providing
theoretical guidance and interpersonal support (Johnson, 2017;
Thanks to Lesley Graff, Leslie Johnson, Ruth Johnson, David Martin,
Wheeler & Richards, 2007; Wilson, Davies, & Weatherhead,
and Donald Stewart for their comments on a previous version or portions
of this article. 2016). Novice psychotherapists identify supervision as the most
Correspondence concerning this article should be addressed to X Edward positive influence on their professional development (Orlinsky &
A. Johnson, Department of Psychology, University of Manitoba, 190 Dysart Ronnestad, 2005), although some supervisees report harm (Ellis,
Road, Winnipeg, MB R3T 2N2, Canada. E-mail: [email protected] 2017). Supervisors accept ethical and legal responsibility for en-

290
 ENHANCING PSYCHOTHERAPY SUPERVISION 291

suring clients receive safe care and, through their gatekeeping doctoral training clinics are poor, with “no reliable improvement”
function, for protecting the public from incompetent or unethical being the modal outcome (Callahan et al., 2014). Taken together,
practitioners (J. T. Thomas, 2014). Theories of supervisor devel- these findings reveal that the apparent increases in learning and
opment agree that supervisors initially model their supervision on competency that students achieve as a result of supervision some-
the supervision they have received, suggesting that supervision’s how do not translate into improved outcomes for clients. Consid-
impact can be multigenerational (Watkins, 2012). ering the enormous investment in supervision noted above, as well
as the implications of these findings for the life span effectiveness
of psychologists, these findings are concerning and demand that
Competency-Based Supervision
we consider whether our supervisory practices can be improved.
Competency-based supervision seeks to identify, develop, and The apparent disconnect between therapist competency and
evaluate the particular knowledge, skills, and attitudes needed to client outcome has also been observed in the therapy literature at
perform specific functions (Falender & Shafranske, 2004; large. A meta-analysis of studies that have examined the relation-
Shafranske & Falender, 2016; J. C. Thomas & Hersen, 2009). ship between competence, in the form of expert ratings of adher-
According to the cube model, foundational competencies, such as ence and competence in the delivery of psychotherapy, have
scientific knowledge, inform the application of functional compe- shown neither has an appreciable impact on treatment outcome
tencies, such as intervention, which deepen with training across the (Webb, DeRubeis, & Barber, 2010). Although there are a variety
stages of professional development (Rodolfa et al., 2005). Within of possible explanations for this discrepancy, in reference to the
this framework, supervision is conceived as a distinct functional findings on the limited impact of training, one plausible explana-
competency that consists of several subcompetencies (Olds & tion is that supervision does not sufficiently utilize methods that
Hawkins, 2014). Consistent with this perspective, the present help supervisees learn competencies in ways that translate into
review addresses the supervisory competencies required to pro- efficacy with clients. In the remainder of the article I consider
mote supervisee learning and development required to become the utility of enhancements to the use of five supervision
effective psychotherapists, and to conduct valid supervisee evalu- practices: (a) direct observation of treatment (live or electronic
ations. These particular competencies were selected for review and recording), (b) progress monitoring (PM), (c) experiential
recommendations because the present state of the literature sug- learning methods, (d) methods of summative evaluation, and (e)
gests that typical practice in these areas requires improvement and diversity-focused supervision.
that there are sufficient grounds to offer specific recommendations
for practice and additional research. Direct Observation Provides High Quality Idiographic
Competency-based supervision clearly delineates the necessary
Data Needed for Effective Supervision
components required for competence, thereby helping ensure train-
ees can demonstrate knowledge and skills that are safe and reflect One potential explanation for poor outcomes in training clinics
theoretically important concepts. Within a competency-based per- and plateauing in supervisee effectiveness is supervisors’ failure to
spective it is assumed that training and supervising students to use evidence-based supervision methods (Milne, 2009, 2014). A
deliver services such as psychotherapy competently will result in premise of evidence-based practice is that relevant evidence is
positive outcomes for both supervisees and clients. Accordingly, available and is used to guide practice. Relevant evidence for
examining the outcomes of supervision is an important basis for psychotherapy supervision is available in two forms: nomothetic
evaluating the state of supervision at large. and idiographic. Examples of nomothetic evidence relevant to
Unfortunately, research on supervision outcomes with clients is supervision include research on evidence-based treatments and the
remarkably sparse; more is known about how it affects students. influence of various therapist qualities, such as warmth, empathy,
As a result, supervision does not reflect evidence-based practice and responsiveness in promoting positive client outcomes
(Callahan & Watkins, 2018a). Although many supervision theories (Lambert, 2013; Norcross & Wampold, 2011). Knowledge and
have been proposed, the validity and efficacy of these theories skill development related to this nomothetic evidence is initially
have not been adequately researched (Watkins, 2011). In particu- acquired in coursework and further refined and developed under
lar, supervisors fail to evaluate their theories and methods against supervision.
the “acid test” of client outcome (Ellis & Ladany, 1997). In the Once a treatment approach is selected and a treatment plan
only well-controlled study of supervision’s effects, clients treated developed, attention in supervision shifts to monitoring and guid-
by supervised therapists improved more, dropped out less, and ing the supervisee’s delivery of the treatment and tracking the
evaluated treatment more highly than clients of unsupervised ther- client’s response to it. This is where idiographic evidence about the
apists (Bambling, King, Raue, Schweitzer, & Lambert, 2006). particulars of clients and their response to treatment is paramount
While these findings in support of supervision’s efficacy are for the quality and efficacy of supervision. Much of the idiographic
reassuring, the study does not address the efficacy of supervision data of interest in supervision consists in the nature and quality of
for trainees. the moment-by-moment interactions between therapist and client
Research examining the effects of supervised training on doc- as treatment unfolds. An important task of supervision is therefore
toral students reveals that supervision typically leads to substantial to identify and discuss the most important and relevant of these
learning (Bernard & Goodyear, 2019; Hill & Knox, 2013), yet episodes to inform the goals of competency development and
results in little (Owen, Wampold, Kopta, Rousmaniere, & Miller, guidance of service delivery.
2016) or no (Erekson, Janis, Bailey, Cattani, & Pedersen, 2017; How best to bring the raw, idiographic data of psychotherapy
Hill et al., 2015) improvement in the client outcomes students into supervision? The main choice is between having the super-
achieve over time. Relatedly, client outcomes obtained in many visee provide a written or verbal report, or for the supervisor to
292 JOHNSON

directly observe the session. While supervisee reports save super- viewing audio/video recordings (99%), with about a third also
visor time, these benefits are more apparent than real once the engaging in live observation (37%).
disadvantages of relying exclusively on supervisee reports are In sum, the findings indicate that with some exceptions, routine
considered. The disadvantages arise from the fact that a filter has supervision frequently includes little to no direct observation of
been placed between the data and the supervisor. Supervisee supervisees delivering therapy. As such, it puts supervisors at risk
filtering takes a variety of forms including inevitable distortions of increased professional and legal liability for client welfare.
and biases in recall, loss of critical details in “gist” reports, These grounds alone justify the inclusion of direct observation in
preferential attention to the verbal channel relative to the nonver- best practice guidelines and accreditation standards.
bal channel, and supervisee selectivity and withholding (Haggerty The empirical evidence concerning the effects of delayed video
& Hilsenroth, 2011; Knox, 2015). In contrast, direct observation feedback on supervisee learning, although not as specific to learn-
(either live or recorded) provides supervisors with full access to ing within supervision of psychology supervisees as might be
the nuanced data required to provide corrective feedback and desired, nonetheless provides additional support for the recom-
promote skill development that is tailored for efficacy with the mendation for including direct observation in supervision. Begin-
particular client, generate valid summative evaluations, and meet ning in the 1960s, the emergence, and widespread availability, of
their clinical oversight obligations. These pedagogical and oversight videorecording technology revolutionized the teaching and learn-
benefits have led to direct observation being included as a best ing of interpersonal communication skills within health care and
practice in supervision guidelines (e.g., American Psychological As- education, including professional psychology. Video feedback of-
sociation [APA], 2015; Association of State and Provincial Psychol- fers an enduring objective representation of supervisees’ perfor-
ogy Boards [ASPPB], 2015; Pilling & Roth, 2014). Conversely, the mance that affords many enriching opportunities for learning,
failure to directly observe supervisees’ work with clients has been feedback, and reflection. Perhaps as a result, psychology trainees
deemed inadequate supervision (Ellis et al., 2014). value supervision that includes video feedback (Hage, Ayala, &
Research on the use of direct observation in supervision is Schwartz, 2019). These advantages have led video feedback to be
limited and variable in quality but overall suggests that its usage is considered the “gold standard of communication teaching” (Kurtz,
far from optimal. In an early study, training directors of APA- Silverman, & Draper, 2005, p. 83).
accredited psychology programs reported that videorecording Fukkink, Trienekens, and Kramer (2011) conducted a meta-
playback in supervision was common in their programs, however, analysis to synthesize research examining the effect of video
the indirect nature of these reports leaves the actual frequency of feedback on interaction skills of “interpersonal professionals”
usage unclear (Romans, Boswell, Carlozzi, & Ferguson, 1995). In from a variety of professions (including students and professionals
contrast, first-hand reports from psychology supervisees suggest from psychology, social work, medicine, nursing, and education).
direct observation is not common. Amerikaner and Rose (2012) Thirty-three intervention studies were selected, resulting in 217
obtained reports on supervisory practices from psychology practi- experimental comparisons. To be included, instructional interven-
cum and internship students and postdegree licensure candidates tions had to use videos of the participants themselves and must
during a two-week “snapshot” period. Few supervisees indicated have employed an external evaluation of behaviour using an ob-
that supervision included review of therapy video (11%) or audio servation instrument (vs. self-evaluations or client perceptions).
(3%), or live observation of therapy through a one-way mirror The outcome measures included communication behaviours and
(3%). Instead, most supervisees reported presenting oral case re- skills ranging from microskills (e.g., frequency counts of questions
ports without supporting video or audio recordings of clients the asked or making eye contact) to molar skills (e.g., rating scale
supervisor had never observed or worked with directly (73%). measures of the participant’s degree of empathy or responsive-
Over the entire supervisory relationship, 49% of supervisees indi- ness). Overall, the aggregate effect size favouring the use of video
cated that their supervisor had never directly observed their work, feedback was 0.4 standard deviations, indicating a moderate effect.
and only 24% indicated that their supervisor directly observed Moderator analyses of the Fukkink et al. (2011) meta-analysis
their work on a regular basis. Finally, this survey found the use of revealed that video feedback effects were larger in studies that
supervision modalities generally did not differ across the levels of used a structured observation form of the relevant interaction skills
professional development of the supervisees. A larger, more re- that were the training focus, suggesting that supervisees will likely
cent, survey of professional psychology students showed that benefit more from reviewing video feedback when supervisors
limited direct observation of both practicum and internship stu- draw their attention to relevant aspects of their performance and
dents persists and that the problem is greater for internship offer feedback on it. Such guidance is important given the over-
students (Rodriguez-Menendez, Dempsey, Albizu, Power, & whelming amount of information contained within even a brief
Campbell Wilkerson, 2017). Such findings of rare direct observa- recorded interaction. Indeed, by identifying clinically significant
tion are not confined to the training of psychology students but also episodes within the session and interpreting them through the lens
have been documented in surveys of members of the British Associ- of relevant professional skills (e.g., empathy or responsiveness, cf.
ation of Behavioural and Cognitive Psychotherapists (BABCP; Friedlander, 2012), supervisors can help supervisees acquire and
Townend, Iannetta, & Freeston, 2002) as well as of psychotherapy internalize the capacity for parsing the flow of therapist-client
trainees in Germany (Weck, Kaufmann, & Witthöft, 2017). interactions, with the aid of theory, into discrete, meaningful
In contrast to these findings, Reiser and Milne (2016) found episodes, an ability known as “professional vision” (Goodwin,
encouraging evidence that training in supervision is associated 1994). Consequently, the use of video recordings helps supervisees
with greater use of direct observation. Among a sample of accred- bring their self-assessment more closely in line with the supervi-
ited BABCP supervisors selected for their supervisory expertise sor’s perception (Gelso, 1974; Poling, 1968; Walz & Johnston,
and training, almost all reported “sometimes or frequently” re- 1963). Importantly, the level of professional experience or devel-
 ENHANCING PSYCHOTHERAPY SUPERVISION 293

opment of the learner did not moderate the effect of video feed- & Kleinstäuber, 2018). The efficacy of PM appears to result
back observed by Fukkink et al. This suggests that video feedback primarily from alerting clinicians to client deterioration, which
likely benefits not only beginning practicum students, but also otherwise often goes undetected (Shimokawa, Lambert, & Smart,
supervisees at more advanced professional levels (cf. Huhra, 2010). In light of its beneficial effects on client outcome, PM has
Yamokoski-Maynhart, & Prieto, 2008). been identified as an essential evidence-based practice by the CPA
As for client outcomes, relatively few studies exist, however, a (2018) and as a recommendation in APA’s supervision guidelines
meta-analysis of studies examining the use of delayed audio/video (APA, 2015).
review in psychodynamic psychotherapy supervision found no Despite its documented value, few psychologists appear to be
clear benefit to clients for including recordings in supervision aware of, or employ, PM. A large national survey of psychologists
(Town et al., 2012). Additional research, ideally using experimen- in Canada showed that two thirds of the sample were unfamiliar
tal methodologies across a variety of therapeutic approaches, is with progress monitoring measures and only 12% use them (Ionita
needed before conclusions about effects of the use of video feed- & Fitzpatrick, 2014). Those engaged in supervision were more
back in supervision on client outcome can be drawn. likely than nonsupervisors to be aware of such measures. Interest-
Live supervision links live observation of therapy (e.g., through ingly, however, among those aware of PM, supervisors were less
a one-way mirror or electronically) with some means of commu- likely than nonsupervisors to use PM in their own clinical practice.
nicating with the supervisee during therapy to provide guidance Similarly, a survey of North American program directors about
(e.g., sending brief text messages via bug-in-the-eye technology; PM use at their predoctoral internship sites indicated that most
Bernard & Goodyear, 2019). As such, it offers the opportunity for sites (73.5%) are not currently using PM and that over a third of
supervisors to hone supervisees’ skills in the moment, thereby the respondents had not heard of PM measures (Overington,
allowing trainees to immediately observe the impact of these Fitzpatrick, Drapeau, & Hunsley, 2016). These findings contrast
modifications to their practice on their client. Two studies have with a smaller and narrower survey of directors of training clinics
compared delayed video review with live supervision. An early with membership in the Association of Psychology Training Clin-
quasi-experimental study found that relative to delayed video ics, of whom 67% reported using PM measures (Peterson & Fagan,
review, live supervision of individual therapy (involving the su- 2017). Overall, it appears that awareness and uptake of PM by
pervisor observing through a one-way mirror and entering the psychologists and training sites in North America are patchy at
therapy room to comment on the process and make suggestions) best.
resulted in therapists using more relationship and support inten- In light of the poor client outcomes observed in doctoral training
tions and their clients reporting stronger working alliances clinics previously noted, there is a clear client welfare rationale for
(Kivlighan, Angelone, & Swafford, 1991). More recent research incorporating PM into the operation of training clinics and super-
using bug-in-the-eye live supervision suggests that when initial vision more generally (Worthen & Lambert, 2007). However, as
differences in therapist competence and the therapeutic alliance are doing so will likely increase awareness and scrutiny within super-
controlled, the impact of live versus delayed video feedback su- vision of clients that are not benefitting from therapy, it is impor-
pervision on client measures of alliance and outcome are generally tant to consider whether employing PM might have negative
similar (Probst et al., 2018; Weck et al., 2016). effects on supervision or on trainees’ professional confidence. Two
In sum, the empirical findings suggest that the use of video studies have examined these questions.
feedback benefits student learning and skill development. Re- In a study by Reese et al. (2009), individual therapy clients
search is needed to establish whether and how such learning differs completed PM prior to each therapy session. Counselling trainees
relative to the experience of live supervision. Moreover, the impact assigned to the continuous-feedback condition reviewed progress
of both approaches on client outcome is largely unknown. The measures with clients and in supervision, whereas those in the
poor supervision practice of not including direct observation has no-feedback condition had no access to PM data and thus did not
long elicited calls for supervision guidelines and professional discuss it with clients or in supervision. Over one year, trainees in
standards to be clarified to address such problems (O’Connor, both conditions demonstrated improved client outcomes, although
2001). From this perspective it is concerning that the Canadian those in the feedback condition improved more. No between-
Psychological Association [CPA] Accreditation Standards (CPA, condition differences were observed in supervisory alliance, sat-
2011), unlike the APA, Commission on Accreditation (2015) isfaction with supervision, or counsellor self-efficacy. However,
Standards of Accreditation, do not require supervisors to directly the relationship between counsellor self-efficacy and client out-
observe supervisees’ work with clients. It is therefore recom- come was stronger in the feedback condition than the no-feedback
mended that some amount of direct observation be made a require- condition.
ment for all supervision within CPA’s accreditation standards and Grossl, Reese, Norsworthy, and Hopkins (2014) also examined
in all provincial and territorial regulatory standards for supervision the impact of using PM in supervision. Unlike Reese et al. (2009),
of professional psychology candidates for licensure. however, all trainees used PM with their clients and were aware of
the results, but were assigned to either a PM-included supervision
condition or one in which PM data were not included in supervi-
Incorporating Progress Monitoring in Supervision
sion. This design allowed the study to more precisely examine the
A second form of idiographic client data relevant to supervision differential effects of using PM data in supervision versus merely
is the client’s session-to-session progress as measured using a providing trainees with PM data. Clients in both conditions
standardized PM measure. A large and growing body of evidence showed large improvements, and these changes were not signifi-
supports the efficacy of using PM to track session-by-session cantly different across supervision conditions. These findings also
client progress for enhancing client outcomes (Lambert, Whipple, support the value of using PM in training contexts insofar as the
294 JOHNSON

results demonstrate much higher levels of client benefit than is (2009) proposes that providing corrective feedback of observed
often seen in training clinics. The study also found that trainees performance, along with modelling and role-playing preferred
who used PM data in supervision reported higher levels of satis- responses, constitute effective experiential learning methods for
faction with supervision than trainees in the no-feedback supervi- promoting competency development in supervision. These meth-
sion condition. The mean level of supervisory alliance was also ods utilize and align with the four main learning modes (italicized)
higher in the feedback condition relative to no-feedback condition in Kolb’s (1984) experiential learning cycle. Applied to supervi-
(medium effect size), although the difference did not reach statis- sion, Kolb’s cycle might look like this: concrete experience (e.g.,
tical significance. a therapeutic impasse) stimulates reflection that may elicit a re-
Taken together, these two studies support the use of PM in quest for observation-based feedback resulting in a novel concep-
training clinics for improving client outcomes. The benefit to client tualisation of the problem and better responses to it, which,
outcome appears to come primarily from supplying student ther- through supervisor modelling, scaffolds active experimentation to
apists with PM data rather from any additional value added from consolidate learning through role-play in supervision and use in
their discussing it in supervision, although this conclusion remains therapy.
tentative pending further research. The studies also indicate that Extending back over many years, empirical research and theory
attending to PM data in supervision, with its likely heightened has supported the potent learning value of observation-based feed-
focus on client difficulties, does not harm the supervision alliance back (Berkhof, van Rijssen, Schellart, Anema, & van der Beek,
or supervisees’ professional confidence. In fact, it may increase 2011; Dawes, 1994; Heckman-Stone, 2004; Westberg & Jason,
supervisees’ satisfaction with supervision and the accuracy of their 1993) and modelling (e.g., Bandura, 1971; Hill & Lent, 2006;
professional self-efficacy. Regarding the latter point, supervision Rosenthal & Bandura, 1978). Evidence suggests that role-play is
discussions that link fluctuations in client PM data with changes to highly effective for teaching communication skills to physicians
fundamental therapeutic processes may help clarify supervisees’ (Berkhof et al., 2011) and may be as effective as, and more feasible
understanding of when and how their therapeutic efforts have an than, using simulated patients for learning (Ladouse, 1987; Lane &
impact. Rollnick, 2007). Conversely, empirical research consistently dem-
There are encouraging signs that the field may be ready to onstrates that didactic teaching methods (e.g., abstract discussions,
embrace the use of PM in training. First, students support making readings) of psychotherapy theory or case conceptualisation that
the inclusion of PM data in clinical training mandatory (Grossl et are not tied to specific experiential learning of recommended
al., 2014). Second, many training directors express interest in practice elements have little impact on competency development
learning more about and incorporating PM in the future and (Herschell, Kolko, Baumann, & Davis, 2010).
suggest that the reasons they have not implemented PM are more A small, but growing, number of studies have empirically ex-
practical than philosophical, such as a lack of resources and amined the content of supervision sessions at the microskill level.
supervisor barriers (Overington et al., 2016; Peterson & Fagan, Studies using trainee or supervisor reports suggest that experiential
2017). Fortunately, resources providing guidance for the use of learning methods are rarely used. For instance, most psychology
PM in supervision are increasingly available (e.g., Holt et al., supervisees reported that their supervisor had not modelled a
2015; Swift et al., 2015), as are the number of PM systems therapy strategy or technique nor engaged them in role-play of one.
available to choose from. Instead, psychotherapy supervision consists mostly of case-
In light of the above findings and consistent with CPA’s (2018) conceptualisation and discussion of therapy interventions based on
Task Force on Outcomes and Progress Monitoring in Psychother- supervisee verbal reports about clients (Amerikaner & Rose,
apy, it is recommended that the CPA accreditation standards make 2012). Furthermore, the modal experience of psychology practi-
training in the use of PM mandatory for students in their capacity cum and internship students is never to have observed a supervi-
as both practitioners and future supervisors, and that it be manda- sor conduct an intake, therapy, or an assessment (Rodriguez-
tory for supervisors within accredited programs to employ PM Menendez et al., 2017).
within supervision where it is applicable. Furthermore, it is rec- These reports have been supported and extended through observa-
ommended that Canadian provincial and territorial regulatory bod- tional research. Bailin, Bearman, and Sale (2018) recorded su-
ies require the use of PM during the process of prelicensure pervision sessions of experienced supervisors working with
supervision of candidates for licensure, where applicable. supervisees. Frequently used microskills included (percentage
of session time allotted noted in parentheses): administrative
tasks (22.3%), praise (27.0%), supervisor self-disclosure
Experiential Supervision Methods
(19.1%), case-conceptualisation (14%), and discussion of a
Supervision practices can be divided into macro and micro recommended practice element (26.1%). Infrequently used mi-
elements. Macro elements organise and guide supervision. They croskills included use of client data (1.3%), corrective feedback
include supervision theory, the supervision contract, supervision (0.9%), and role-play (0.3%). Finally, the average competency
format, and evaluation instruments and procedures. Supervision with which microskills were delivered was rated as below
best practice guidelines (e.g., APA, 2015) and CPA’s ethical adequate.
guidelines (CPA, 2009) focus primarily on these macro aspects of The above findings collectively suggest that, across a variety of
supervision. Micro elements are the moment-to-moment activities contexts, evidence-based experiential learning methods of model-
in supervision that make up the content of the supervision hour. ling, role-play, and corrective feedback are effective means of
Supervisory microskills that use experiential learning methods enhancing learning, yet are rarely used in supervision and may be
have been identified as essential for advancing trainee learning and poorly implemented when they are used. Accordingly, it is rec-
development (Milne, Aylott, Fitzpatrick, & Ellis, 2008). Milne ommended that current supervisors learn how to effectively em-
 ENHANCING PSYCHOTHERAPY SUPERVISION 295

ploy these methods, that they be taught as a mandatory element of Taken together, leniency bias and halo effects markedly inter-
supervisor training in accredited programs, and that their collective fere with the valid and objective evaluation of competencies.
impact on supervisee learning and client outcomes be further Moreover, trainees may come to expect uniformly high compe-
examined empirically. tency ratings on practicum and internship as a matter of course,
creating a mindset that interferes with a developmental focus.
Interestingly, most supervisors are aware of their own bias toward
Summative Evaluation of Competencies leniency (Gonsalvez et al., 2015). While some may justify this
To promote supervisee competency development and enable practice by pointing to the need to maintain a positive alliance with
goal setting, planning, and gatekeeping supervisors provide ongo- the supervisee or to enhance the supervisee’s confidence, the
ing formative feedback and a summative evaluation. Theoretically, collective impact of these practices is nonetheless concerning.
supervisors are well placed to render informed, accurate, and Leniency and halo biases have also been commonly observed in
nuanced judgments about supervisees’ performance. However, supervisor evaluations carried out in other professions, including
research on supervisor ratings of trainee competencies suggest that medical education (Gray, 1996), business school field placements
(Wolf, 2015), and education (Welsh, 2011).
they are subject to leniency and halo (glowing and global) biases
Two lines of research offer promising directions for attenuating
(Gonsalvez & Crowe, 2014).
these problems. One study showed that when individual differ-
In a leniency bias, ratings are consistently overly positive. One
ences in supervisor leniency style are statistically controlled, halo
manifestation is a reluctance to give low ratings. In a multisite
effects are also diminished. The study also yielded a leniency-
study of supervisory evaluations of Australian clinical psychology
adjusted rank ordering of difficulty levels for functional and foun-
trainees’ practicum, only 1.6% of the competency ratings fell in
dational competencies that training programs may find useful for
the lower half of the scale (Gonsalvez et al., 2015). A similar
comparison purposes (Price et al., 2017). Further research is
pattern was observed in ratings of pre-internship psychology train-
needed to translate these statistical-control methods into practical
ees in the US (Price, Callahan, & Cox, 2017).
methods for controlling leniency and halo biases at the level of
Ceiling effects also signal leniency bias and produce range
individual supervisor ratings.
restriction, making it difficult to reliably discriminate competency
A second line of research proceeds from the idea that the source
levels within and between trainees. Ceiling effects have been
of these biases may lie more in the Likert-scale rating method than
found in ratings of relatively junior practicum trainees for whom
in the raters. This assumption is consistent with decades of re-
high ratings are presumably not yet warranted (Gonsalvez & search on performance appraisal showing that modification of
Freestone, 2007). For instance, in the Gonsalvez et al. (2015) study rating scales to reduce rater errors and biases have little to no
described earlier, the observed mean competency rating of students impact (DeNisi & Murphy, 2017). A promising direction is to
in their second, third, and fourth practicum was above 9 on the supplement the use of competency rating scales, which suffer from
10-point scale. The ceiling effect occurred even though the authors ill-defined criteria for what constitute different levels of compe-
anchored the highest possible rating of 10 by “competent,” defined tence, with training that clearly specifies and illustrates different
as the level of competent professional practice expected of a levels of competence (e.g., novice, beginner, proficient) for a range
clinical psychologist working in his or her first job postgraduation. of different competencies. Terry, Gonsalvez, and Deane (2017)
Furthermore, the means of trainees at these three levels were not used written vignettes that succinctly illustrated varying levels of
significantly different from one another. These findings illustrate competence in a domain with a brief training procedure to reduce
how ceiling effects can obscure whether trainees are demonstrating leniency bias in supervisor evaluations. Relative to supervisors
progress as a result of additional training, undermining meaningful who did not receive the training, trained supervisors subsequently
competency development planning. rated actual trainees’ competency level lower than did nontrained
A halo effect exists when an evaluation on one performance supervisors, suggesting a reduction in leniency bias. Moreover,
dimension influences judgments on other dimensions. Halo effects trained supervisors’ ratings showed higher agreement with expert
tend to homogenize ratings undermining the discriminative valid- ratings than did untrained supervisors’ ratings, possibly indicating
ity of domain-specific competency evaluations and reducing inter- a reduction in halo effects. Supervisors reported finding the online
rater agreement. Halo findings suggest that supervisors may rely vignette procedure acceptable and efficient to use. Additional
on a simple global impression of supervisee competency when research on this approach is needed to evaluate its applicability to
rating different specific competencies. Gonsalvez and Freestone training in other countries and contexts.
(2007) found evidence of a halo bias in supervisor ratings of In light of the strong, supportive relationships that supervisors
clinical psychology trainees’ performance in field placements. often have with their supervisees, supervisors from a number of
Trainee performance dimensions were highly correlated when professions (including psychology, social work, and medicine) cite
rated by the same supervisor, but poorly correlated when rated by concerns about the inherent dual-role conflict between that sup-
different supervisors. Similarly, low agreement has been observed portive role and the role of gatekeeper (Vinton & Wilke, 2011).
between supervisors and independent judges’ evaluations of cog- Indeed, more than a third of supervisors strongly endorsed feelings
nitive therapy competencies, with supervisor ratings being more of guilt or fear at the possibility of lengthening or terminating their
positive (Dennhag, Gibbons, Barber, Gallop, & Crits-Christoph, supervisees’ professional education and training as a source of bias
2012). Also, supervisor ratings of different competencies are well in their ratings (Gonsalvez, Wahnon, & Deane, 2017). Accord-
modelled by a single factor of “competence” (Price et al., 2017). ingly, another approach that has been suggested is to incorporate
However, a single competence factor did not predict posttraining competency assessments from independent assessors who have not
outcomes, casting doubt on its validity (Strom et al., 2016). been involved in the supervision of the trainee and thus may not be
296 JOHNSON

as affected by such considerations. A pilot study of therapist engage in self-reflection to develop the self-awareness and cultural
competency ratings showed that independent raters exhibited less humility needed to discuss diversity issues sensitively in therapy
leniency than did supervisor, therapist, or patient ratings (Rozek et and supervision so as to avoid unintentional acts of prejudice such
al., 2018). Research into the use of independent assessor ratings as microaggressions (Constantine & Sue, 2007; Hernández &
indicates that while this can provide useful data, it can also be McDowell, 2010; Hook et al., 2016; Jisrawi & Arnold, 2018).
labour intensive. Because levels of interrater reliability are often Beyond merely avoiding offending particular groups of people,
relatively weak, numerous assessors are required to achieve ac- however, supervisors are expected to use diversity relevant skills
ceptable levels of reliability (Keen & Freeston, 2008). Also, each and techniques proactively in supervision to articulate and exam-
assessor must evaluate a sufficient number and amount of work ine diversity issues in a culturally safe and supportive manner.
samples necessary to be able to evaluate a sufficient range of Diversity and multiculturalism are umbrella terms for a broad
competencies (Muse & McManus, 2013). array of identity dimensions often summarised with the ADRESS-
Finally, the ASPPB is developing a novel computerized com- ING acronym: Age, (Dis)ability, Religion, Ethnicity, Social Status,
petency assessment system, the Examination for Professional Prac- Sexual Orientation, Indigeneity, Nationality, and Gender (Hays,
tice in Psychology-II (EPPP-II), that uses standardized vignettes to 1996). The diversity of diversity inheres not only in these nine
assess a defined list of competencies. If shown to be reliable and distinct dimensions, but also across their many intersections, in-
valid, it promises an efficient, automated means of standardiz- dicating the fluidity of sociocultural identity (APA, 2017).
ing the assessment of professional skills. Moreover, because it Diversity-competent supervisors are capable of integrating multi-
is an independent, standardized assessment that is free of the ple identity aspects within their conceptualisation of supervision
relationship-based concerns noted above, it would likely not be and adapt their practice of supervision in a culturally sensitive
prone to leniency and halo biases. However, for several reasons, manner to meet the needs of supervisees and clients (Fouad et al.,
the EPPP-II will not offer a complete solution to the problems of 2009). In developing an adequate conceptualisation of supervision,
professional competency evaluation. First, this system is intended supervisors are advised to consider diversity from a variety of
for licensure purposes and would not address competency assess- perspectives including intrapersonal identity and interpersonal as-
ment at prelicensure stages. More substantively, it is unclear sumptions and biases, and how these may interact with profes-
whether computerized simulations using avatars can portray the sional practices, organisational context, cultural context, and the
types of rich, contextualized interactions that challenge the inter- political context (Bernard & Goodyear, 2019; Tsui, O’Donoghue,
personal, diversity, and affective skills of trainees with compe- & Ng, 2014). Clearly, the broad scope of this domain suggests that
tency problems (Callahan & Watkins, 2018b). In sum, while the the acquisition of diversity competence requires ongoing attention
EPPP-II may prove less prone to leniency and halo biases, it may both within and beyond supervision.
have complementary limitations in the form of oversimplification Unfortunately, the limited evidence available suggests that
that may limit its usefulness. within supervision, diversity-related discussions occur infre-
Further research into enhancing competency evaluation remains quently (Allison, Crawford, Echemendia, Robinson, & Knepp,
a priority. Conceivably, greater use of direct observation, progress 1994; Gatmon et al., 2001) and receive relatively little time
monitoring, and experiential learning methods in supervision will (Novoa-Gómez et al., 2019). This is unfortunate, as research
collectively permit greater attention to the specific and discrete suggests that novice clinicians prefer supervision as a means of
aspects of supervisees’ skills and clients’ corresponding response. developing multicultural competence (Allison et al., 1994). In-
Research could establish whether one or more of these elements deed, when supervisors are culturally responsive and competent,
alone, or in combination, suffice to reduce halo and leniency they are more likely to create a strong supervisory working alli-
biases. The most promising options for eliminating supervisor ance and greater satisfaction with supervision within supervisees
biases from summative evaluations are to include independent (Gatmon et al., 2001; Inman, 2006; Schroeder, Andrews, &
assessor ratings with instruments that demonstrate strong interrater Hindes, 2009). Although evidence of the efficacy of specific
reliability and to train assessors to better calibrate evaluations multicultural models for supervisory and client outcomes is often
using vignettes. Observation-based tools that evaluate competence limited or altogether lacking, meta-analytic evidence drawn from
for particular therapeutic orientations such as CBT are available 18 studies indicates that client perceptions of therapist multicul-
(e.g., Muse, McManus, Rakovshik, & Thwaites, 2017), but require tural competency is associated with moderate-sized client symp-
further validation to ensure they meet the rigorous requirements of tom reductions and large effects on psychotherapy process mea-
competence assessment for summative evaluations in training and sures such as the therapeutic alliance (Tao, Owen, Pace, & Imel,
for licensure (Barber, Sharpless, Klostermann, & McCarthy, 2015). These findings support the importance of developing su-
2007). pervisee multicultural competence for enhancing both the process
and outcome of psychotherapy.
The findings that diversity discussions are uncommon in super-
Diversity Competence
vision suggest that for supervision to contribute to advancing
The broadening diversity of Canadian society means that diver- diversity competence within Canadian psychology supervisors
sity competence is more important than ever. It is an essential must achieve greater diversity competence in order to initiate
supervision competency that has an important influence on the culturally sensitive discussions of diversity. To that end, it is
quality of supervision and supervised psychological services. Su- recommended that CPA’s (2011) Accreditation Standards be re-
pervisors are ethically responsible for preventing discrimination vised to require that training in supervision include diversity
against clients and supervisees on the basis of diversity character- competence. Practicing supervisors can meet their ethical obliga-
istics (CPA, 2009). This requires both supervisor and supervisee to tions to maintain diversity competence through a variety of means,
 ENHANCING PSYCHOTHERAPY SUPERVISION 297

for instance, by reviewing written resources (e.g., APA, 2017; supervision in Canada, changes are recommended to CPA’s (2011)
CPA, 1996/2017; CPA and The Psychology Foundation of Accreditation Standards, which govern how accredited programs
Canada, 2018; Falender, Shafranske, & Falicov, 2014; Hays, 2009; deliver supervision. Similarly, the professional regulations govern-
Ober, Granello, & Henfield, 2009; Sue, Sue, Neville, & Smith, ing the supervision of psychologists for licensure within each
2019), participating in workshops and courses, and consulting with province must be changed. Specifically, these standards and reg-
knowledgeable colleagues. ulations should be revised so that supervision of psychological
Finally, the lack of Canadian research addressing diversity in services delivered in either accredited training programs or toward
supervision encountered in the present review was noteworthy the licensure of professional psychologists includes (a) some
given the distinct nature of Canadian professional psychology, amount of direct observation (as a means of guiding service
Canadian culture, history, and diversity issues, including reconcil- delivery and grounding feedback and evaluation), (b) progress
iation with the indigenous people of Canada (CPA and The Psy- monitoring measures (where applicable), (c) experiential learning
chology Foundation of Canada, 2018). From this vantage, there is methods, and (d) attention to diversity. Relatedly, the accreditation
a dire need for more “made in Canada” research to support the standards governing the education and training in supervisory
development of supervisors’ diversity competence that reflects the competencies of doctoral students should include knowledge of the
Canadian context. This work could profitably examine the nature five methods outlined in this review while those standards gov-
and sufficiency of diversity competency training offered in Cana- erning internships should include the opportunity to engage in
dian psychology programs by updating and extending earlier sur- supervised supervision that uses direct observation, PM, experien-
veys of directors of doctoral (Hertzsprung & Dobson, 2000) and tial learning methods, and attention to diversity.
internship programs (Brooks, Mintz, & Dobson, 2004) to consider In addition, further research is needed to further understand and
the degree to which students are satisfied with such training, and fully explore the efficacy of all five recommendations. This is
the extent to which it enhances client outcomes. A recent survey of especially true regarding the need to improve the reliability, va-
psychologists in the United States suggests that diversity training lidity, and objectivity of methods used for the summative evalua-
that recognizes the idiographic nature of culture and the impor- tion of competencies among practicum and internship students and
tance of intersectionality is highly valued and that clinical super- candidates for licensure. Researchers and training programs are
vision, perhaps because of its idiographic nature, is regarded as the encouraged to explore ways of incorporating independent asses-
most important means for training culturally competent psychol- sors into the summative evaluation process as one means toward
ogists (Benuto, Singer, Newlands, & Casas, 2019). Such research this goal, along with efforts to develop other, novel means of
will be important for fashioning future recommendations on train- strengthening summative enhancements.
ing and supervision of diversity competence. This review and the accompanying recommendations are in-
tended as a stimulus to a national conversation related to enhanc-
Conclusion ing supervision. Happily, this discussion is already underway, as
many of the suggested recommendations were endorsed at CPA’s
The importance of clinical supervision to professional psychol- professional training summit. Fittingly, the training summit also
ogy stems from its ability to help trainees transmute their abstract recommended that CPA create a supervision working group, con-
foundational knowledge into ethical and effective functional skills stituted with careful attention to the diversity of its members, to
that help clients achieve their goals. The present review suggests devise a national set of best practice guidelines for supervision of
that by improving common supervisory practice in five areas, research and practice in Canada. Such a group would develop a
supervisors can potentially better achieve these aims and thereby comprehensive set of supervision guidelines that will benefit Ca-
serve the profession more effectively and efficiently. The five nadian psychology supervisors and supervisees for years to come.
recommended enhancements to supervisor practices are to increase
the use of (a) direct observation of therapy, (b) progress monitor-
ing, (c) experiential learning methods (providing corrective feed- Résumé
back, modelling, and role-playing), (d) less biased summative
evaluation methods, and (e) culturally sensitive discussions of Deux principaux objectifs de la supervision clinique consistent à
diversity. Although these enhancements have been discussed sep- promouvoir et à évaluer l’acquisition de compétences profession-
arately, it is likely that their collective use will amplify their nelles chez les stagiaires et de protéger et améliorer le bien-être des
respective benefits. For instance, direct observation of therapy clients. Afin d’aider les superviseurs à réaliser ces objectifs, cet
along with progress monitoring will likely enhance the validity and article examine l’utilisation et les bienfaits actuels de cinq impor-
utility of both formative and summative feedback. Similarly, direct tantes pratiques de supervision : l’observation directe (en direct
observation of service delivery is essential for addressing uncon- ou enregistrée); le suivi des progrès réalisés; les méthodes
scious biases that affect supervisees’ diversity competence and d’apprentissage expérientiel (modélisation, jeux de rôle, et rétro-
will provide a concrete basis for discussion in supervision. Also, action constructive basée sur l’observation); les procédures
the changes to client functioning that these supervisory changes d’évaluation sommative moins biaisées; et la supervision axée sur
bring about may be usefully tracked through confidential client la diversité. L’article se termine par des recommandations pour la
progress monitoring at the level of individual supervisees and recherche, des changements dans les pratiques de supervision et
supervisors, and programs, and through collaborative research des changements dans la réglementation de la formation des su-
endeavours. perviseurs et de la supervision dans le respect des normes
In order to ensure that these five supervisory practices become d’agrément et des exigences provinciales en matière d’autorisation
consistently embedded in the practice of professional psychology d’exercer.
298 JOHNSON

Mots-clés : supervision, supervision clinique, recommandations de internship training. Canadian Psychology/Psychologie canadienne, 45,
supervision, supervision efficace, compétences de supervision. 308 –312. http://dx.doi.org/10.1037/h0088237
Callahan, J. L., Gustafson, S., Misner, J., Paprocki, C. M., Sauer, E.,
Saules, K. K., . . . Wise, E. H. (2014). Introducing the Association of
References Psychology Training Clinics’ Collaborative Research Network: A study
on client expectancies. Training and Education in Professional Psychol-
Allison, K. W., Crawford, I., Echemendia, R., Robinson, L. V., & Knepp, ogy, 8, 95–104. http://dx.doi.org/10.1037/tep0000047
D. (1994). Human diversity and professional competence: Training in Callahan, J. L., & Watkins, C. E., Jr. (2018a). Evidence-based training: The
clinical and counseling psychology revisited. American Psychologist, time has come. Training and Education in Professional Psychology, 12,
49, 792–796. 211–218. http://dx.doi.org/10.1037/tep0000204
American Psychological Association. (2015). Guidelines for clinical su- Callahan, J. L., & Watkins, C. E., Jr. (2018b). The science of training III:
pervision in health service psychology. American Psychologist, 70, Supervision, competency, and internship training. Training and Educa-
33– 46. http://dx.doi.org/10.1037/a0038112 tion in Professional Psychology, 12, 245–261. http://dx.doi.org/10.1037/
American Psychological Association. (2017). Multicultural guidelines: An tep0000208
Ecological approach to context, identity, and intersectionality, 2017. Canadian Psychological Association. (2009). Ethical guidelines for super-
Washington, DC: Author. Retrieved from apa.org/about/policy/ vision in psychology: Teaching, research, practice, and administration.
multicultural-guidelines.pdf Ottawa, Ontario, Canada: Author.
American Psychological Association, Commission on Accreditation. Canadian Psychological Association. (2011). Accreditation standards and
(2015). Standards of accreditation for health service psychology. procedures for doctoral programmes and internships in professional
Washington, DC: Author. Retrieved from http://www.apa.org/ed/ psychology (5th revision). Ottawa, Ontario, Canada: Author.
accreditation/about/policies/standards-of-accreditation.pdf Canadian Psychological Association. (2017). Guidelines for non-
Amerikaner, M., & Rose, T. (2012). Direct observation of psychology discriminatory practice (Updated ed.). Ottawa, Ontario, Canada: Author.
supervisees’ clinical work: A snapshot of current practice. The Clinical (Original work published 1996)
Supervisor, 31, 61– 80. http://dx.doi.org/10.1080/07325223.2012 Canadian Psychological Association. (2018). Outcome and progress mon-
.671721
itoring in psychotherapy. Ottawa, Ontario, Canada: Author.
Association of Psychology Postdoctoral and Internship Centers. (2012).
Canadian Psychological Association and the Psychology Foundation of
2012 APPIC match: Survey of Internship Applicants Part 1: Summary of
Canada. (2018). Psychology’s response to the Truth and Reconciliation
survey results. Retrieved from https://www.appic.org/Internships/Match/
Commission of Canada’s report. Ottawa, Ontario, Canada: Author.
Match-Statistics/Applicant-Survey-2012-Part-1
Constantine, M. G., & Sue, D. W. (2007). Perceptions of racial microag-
Association of State and Provincial Psychology Boards. (2015). Supervi-
gressions among Black supervisees in cross-racial dyads. Journal of
sion guidelines for education and training leading to licensure as a
Counseling Psychology, 54, 142–153. http://dx.doi.org/10.1037/0022-
health service provider. Retrieved from https://www.asppb.net/page/
0167.54.2.142
Guidelines
Dawes, R. M. (1994). House of cards: Psychology and psychotherapy built
Bailin, A., Bearman, S. K., & Sale, R. (2018). Clinical supervision of
on myth. New York, NY: The Free Press.
mental health professionals serving youth: Format and microskills. Ad-
DeNisi, A. S., & Murphy, K. R. (2017). Performance appraisal and per-
ministration and Policy in Mental Health and Mental Health Services
formance management: 100 years of progress? Journal of Applied Psy-
Research, 45, 800 – 812. http://dx.doi.org/10.1007/s10488-018-0865-y
chology, 102, 421– 433. http://dx.doi.org/10.1037/apl0000085
Bambling, M., King, R., Raue, P., Schweitzer, R., & Lambert, W. (2006).
Clinical supervision: Its influence on client-rated working alliance and Dennhag, I., Gibbons, M. B. C., Barber, J. P., Gallop, R., & Crits-
client symptom reduction in the brief treatment of major depression. Christoph, P. (2012). Do supervisors and independent judges agree on
Psychotherapy Research, 16, 317–331. http://dx.doi.org/10.1080/ evaluations of therapist adherence and competence in the treatment of
10503300500268524 cocaine dependence? Psychotherapy Research, 22, 720 –730. http://dx
Bandura, A. (1971). Psychotherapy based upon modeling principles. In .doi.org/10.1080/10503307.2012.716528
A. E. Bergin & S. L. Garfield (Eds.), Handbook of psychotherapy and Ellis, M. V. (2017). Narratives of harmful clinical supervision. The Clinical
behavior change: An empirical analysis (pp. 653–708). New York, NY: Supervisor, 36, 20 – 87. http://dx.doi.org/10.1080/07325223.2017
Wiley. .1297752
Barber, J. P., Sharpless, B. A., Klostermann, S., & McCarthy, K. S. (2007). Ellis, M., Berger, L., Hanus, A., Ayala, E., Swords, B., & Siembor, M. (2014).
Assessing intervention competence and its relation to therapy outcome: Inadequate and harmful clinical supervision. The Counseling Psychologist,
A selected review derived from the outcome literature. Professional 42, 434 – 472. http://dx.doi.org/10.1177/0011000013508656
Psychology: Research and Practice, 38, 493–500. http://dx.doi.org/10 Ellis, M. V., & Ladany, N. (1997). Inferences concerning supervisees and
.1037/0735-7028.38.5.493 clients in clinical supervision: An integrative review. In C. E. Watkins,
Benuto, L. T., Singer, J., Newlands, R. T., & Casas, J. B. (2019). Training Jr. (Ed.), Handbook of psychotherapy supervision (pp. 467–507). New
culturally competent psychologists: Where are we and where do we need York, NY: Wiley.
to go? Training and Education in Professional Psychology, 13, 56 – 63. Erekson, D. M., Janis, R., Bailey, R. J., Cattani, K., & Pedersen, T. R.
http://dx.doi.org/10.1037/tep0000214 (2017). A longitudinal investigation of the impact of psychotherapist
Berkhof, M., van Rijssen, H. J., Schellart, A. J., Anema, J. R., & van der training: Does training improve client outcomes? Journal of Counseling
Beek, A. J. (2011). Effective training strategies for teaching communi- Psychology, 64, 514 –524. http://dx.doi.org/10.1037/cou0000252
cation skills to physicians: An overview of systematic reviews. Patient Falender, C. A., & Shafranske, E. P. (2004). Clinical supervision: A
Education and Counseling, 84, 152–162. http://dx.doi.org/10.1016/j.pec competency-based approach. Washington, DC: American Psychologival
.2010.06.010 Association. http://dx.doi.org/10.1037/10806-000
Bernard, J. M., & Goodyear, R. K. (2019). Fundamentals of clinical Falender, C. A., Shafranske, E. P., & Falicov, C. (Eds.). (2014). Multicul-
supervision (6th ed.). New York, NY: Pearson. turalism and diversity in in clinical supervision: A competency-based
Brooks, B. L., Mintz, A. R., & Dobson, K. S. (2004). Diversity training in approach. Washington, DC: American Psychological Association.
Canadian predoctoral psychology internships: A survey of directors of http://dx.doi.org/10.1037/14370-000
 ENHANCING PSYCHOTHERAPY SUPERVISION 299

Fouad, N. A., Grus, C. L., Hatcher, R. L., Kaslow, N. J., Hutching, P. S., ing and Education in Professional Psychology, 4, 29 –35. http://dx.doi
Madson, M. B., . . .Crossman, R. E. (2009). Competency benchmarks: A .org/10.1037/a0017064
model for understanding and measuring competence in professional Herschell, A. D., Kolko, D. J., Baumann, B. L., & Davis, A. C. (2010). The
psychology across training levels. Training and Education in Profes- role of therapist training in the implementation of psychosocial treat-
sional Psychology, 3, S5–S26. http://dx.doi.org/10.1037/a0015832 ments: A review and critique with recommendations. Clinical Psychol-
Friedlander, M. L. (2012). Therapist responsiveness: Mirrored in supervi- ogy Review, 30, 448 – 466. http://dx.doi.org/10.1016/j.cpr.2010.02.005
sor responsiveness. The Clinical Supervisor, 31, 103–119. http://dx.doi Hertzsprung, M., & Dobson, K. S. (2000). Diversity training: Conceptual
.org/10.1080/07325223.2012.675199 issues and practices in Canadian clinical psychology programs. Cana-
Fukkink, R., Trienekens, N., & Kramer, L. (2011). Video feedback in dian Psychology, 41, 184 –191. http://dx.doi.org/10.1037/h0086868
education and training: Putting learning in the picture. Educational Hill, C. E., Baumann, E., Shafran, N., Gupta, S., Morrison, A., Rojas,
Psychology Review, 23, 45– 63. http://dx.doi.org/10.1007/s10648-010- A. E., . . . Gelso, C. J. (2015). Is training effective? A study of
9144-5 counseling psychology doctoral trainees in a psychodynamic/
Gatmon, D., Jackson, D., Koshkarian, L., Martos-Perry, N., Molina, A., interpersonal training clinic. Journal of Counseling Psychology, 62,
Patel, N., & Rodolfa, E. (2001). Exploring ethnic, gender, and sexual 184 –201. http://dx.doi.org/10.1037/cou0000053
orientation variables in supervision: Do they really matter? Journal of Hill, C. E., & Knox, S. (2013). Training and supervision in psychotherapy.
Multicultural Counseling and Development, 29, 102–113. http://dx.doi In M. J. Lambert (Ed.), Bergin and Garfield’s handbook of psychother-
.org/10.1002/j.2161-1912.2001.tb00508.x apy and behavior change (6th ed., pp. 775– 812). Hoboken, NJ: Wiley.
Gelso, C. J. (1974). Effects of recording on counselor and clients. Coun- Hill, C. E., & Lent, R. W. (2006). A narrative and meta-analytic review of
selor Education and Supervision, 14, 5–12. http://dx.doi.org/10.1002/j helping skills training: Time to revive a dormant area of inquiry. Psy-
.1556-6978.1974.tb01987.x chotherapy, 43, 154 –172. http://dx.doi.org/10.1037/0033-3204.43.2.154
Gonsalvez, C. J., & Crowe, T. P. (2014). Evaluation of psychology prac- Holt, H., Beutler, L. E., Kimpara, S., Macias, S., Haug, N. A., Shiloff, N.,
titioner competence in clinical supervision. American Journal of Psy- . . . Stein, M. (2015). Evidence-based supervision: Tracking outcome
chotherapy, 68, 177–193. http://dx.doi.org/10.1176/appi.psychotherapy and teaching principles of change in clinical supervision to bring science
.2014.68.2.177 to integrative practice. Psychotherapy, 52, 185–189. http://dx.doi.org/10
Gonsalvez, C. J., Deane, F. P., Blackman, R., Matthias, M., Knight, R., .1037/a0038732
Nasstasia, Y., . . . Bliokas, V. (2015). The hierarchical clustering of Hook, J. N., Farrell, J. E., Davis, D. E., DeBlaere, C., Van Tongeren, D. R.,
clinical psychology practicum competencies: A multisite study of su- & Utsey, S. O. (2016). Cultural humility and racial microaggressions in
pervisor ratings. Clinical Psychology: Science and Practice, 22, 390 – counseling. Journal of Counseling Psychology, 63, 269 –277. http://dx
403. http://dx.doi.org/10.1111/cpsp.12123 .doi.org/10.1037/cou0000114
Gonsalvez, C. J., & Freestone, J. (2007). Field supervisors’ assessments of Huhra, R. L., Yamokoski-Maynhart, C. A., & Prieto, L. R. (2008). Re-
trainee performance: Are they reliable and valid? Australian Psycholo- viewing videotape in supervision: A developmental approach. Journal of
gist, 42, 23–32. http://dx.doi.org/10.1080/00050060600827615 Counseling & Development, 86, 412– 418. http://dx.doi.org/10.1002/j
Gonsalvez, C., Wahnon, T., & Deane, F. P. (2017). Goal-setting, feedback, .1556-6678.2008.tb00529.x
and evaluation practices reported by clinical supervisors. Australian Inman, A. G. (2006). Supervisor multicultural competence and its relation
Psychologist, 52, 21–30. to supervisory process and outcome. Journal of Marital and Family
Goodwin, C. (1994). Professional vision. American Anthropologist, 96, Therapy, 32, 73– 85. http://dx.doi.org/10.1111/j.1752-0606.2006
606 – 633. http://dx.doi.org/10.1525/aa.1994.96.3.02a00100 .tb01589.x
Gray, J. D. (1996). Global rating scales in residency education. Academic Ionita, G., & Fitzpatrick, M. (2014). Bringing science to clinical practice:
Medicine, 71, S55–S63. http://dx.doi.org/10.1097/00001888- A Canadian survey of psychological practice and usage of progress
199601000-00043 monitoring measures. Canadian Psychology/Psychologie canadienne,
Grossl, A. B., Reese, R. J., Norsworthy, L. A., & Hopkins, N. B. (2014). 55, 187–196. http://dx.doi.org/10.1037/a0037355
Client feedback data in supervision: Effects on supervision and outcome. Jisrawi, A. N., & Arnold, C. (2018). Cultural humility and mental health
Training and Education in Professional Psychology, 8, 182–188. http:// care in Canadian Muslim communities. Canadian Journal of Counsel-
dx.doi.org/10.1037/tep0000025 ling and Psychotherapy, 52, 43– 64.
Hage, S. M., Ayala, E. E., & Schwartz, J. (2019). The experience of Johnson, E. A. (2017). Working together in clinical supervision: A guide
doctoral students at program-affiliated practicum training clinics. Train- for supervisors and supervisees. New York, NY: Momentum Press.
ing and Education in Professional Psychology. Advance online publi- Keen, A. J., & Freeston, M. H. (2008). Assessing competence in cognitive-
cation. http://dx.doi.org/10.1037/tep0000264 behavioural therapy. The British Journal of Psychiatry, 193, 60 – 64.
Haggerty, G., & Hilsenroth, M. J. (2011). The use of video in psychother- http://dx.doi.org/10.1192/bjp.bp.107.038588
apy supervision. British Journal of Psychotherapy, 27, 193–210. http:// Kivlighan, D. M., Angelone, E. O., & Swafford, K. G. (1991). Live
dx.doi.org/10.1111/j.1752-0118.2011.01232.x supervision in individual psychotherapy: Effects on therapist’s intention
Hays, P. A. (1996). Addressing the complexities of culture and gender in use and client’s evaluation of session effect and working alliance.
counseling. Journal of Counseling & Development, 74, 332–338. http:// Professional Psychology: Research and Practice, 22, 489 – 495. http://
dx.doi.org/10.1002/j.1556-6676.1996.tb01876.x dx.doi.org/10.1037/0735-7028.22.6.489
Hays, P. A. (2009). Integrating evidence-based practice, cognitive– Knox, S. (2015). Disclosure—and lack thereof—in individual supervision.
behavior therapy, and multicultural therapy: Ten steps for culturally The Clinical Supervisor, 34, 151–163. http://dx.doi.org/10.1080/
competent practice. Professional Psychology: Research and Practice, 07325223.2015.1086462
40, 354 –360. http://dx.doi.org/10.1037/a0016250 Kolb, D. A. (1984). Experiential learning: Experience as the source of
Heckman-Stone, C. (2004). Trainee preferences for feedback and evalua- learning and development. Englewood Cliffs, NJ: Prentice Hall.
tion in clinical supervision. The Clinical Supervisor, 22, 21–33. http:// Kurtz, S., Silverman, J., & Draper, J. (2005). Teaching and learning
dx.doi.org/10.1300/J001v22n01_03 communication skills in medicine. Oxford, United Kingdom: Radcliffe.
Hernández, P., & McDowell, T. (2010). Intersectionality, power, and Ladouse, G. P. (1987). Role play. New York, NY: Oxford University Press.
relational safety in context: Key concepts in clinical supervision. Train- Lambert, M. J. (2013). The efficacy and effectiveness of psychotherapy. In
300 JOHNSON

M. J. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy C. E. Watkins, Jr., & D. L. Milne (Eds.), The Wiley international
and behavior change (6th ed., pp. 169 –218). Hoboken, NJ: Wiley. handbook of clinical supervision (pp. 20 –37). Chichester, United King-
Lambert, M. J., Whipple, J. L., & Kleinstäuber, M. (2018). Collecting and dom: Wiley. http://dx.doi.org/10.1002/9781118846360.ch2
delivering progress feedback: A meta-analysis of routine outcome mon- Poling, E. G. (1968). Video tape recordings in counseling practicum: II.
itoring. Psychotherapy, 55, 520 –537. http://dx.doi.org/10.1037/ Critique considerations. Counselor Education and Supervision, 8, 33–
pst0000167 38. http://dx.doi.org/10.1002/j.1556-6978.1968.tb00962.x
Lane, C., & Rollnick, S. (2007). The use of simulated patients and role- Price, S. D., Callahan, J. L., & Cox, R. J. (2017). Psychometric investiga-
play in communication skills training: A review of the literature to tion of competency benchmarks. Training and Education in Profes-
August 2005. Patient Education and Counseling, 67, 13–20. http://dx sional Psychology, 11, 128 –139. http://dx.doi.org/10.1037/tep0000133
.doi.org/10.1016/j.pec.2007.02.011 Probst, T., Jakob, M., Kaufmann, Y. M., Müller-Neng, J. M. B., Bohus, M.,
Milne, D. (2009). Evidence-based clinical supervision: Principles and & Weck, F. (2018). Patients’ and therapists’ experiences of general
practice. Chichester, United Kingdom: Wiley. change mechanisms during bug-in-the-eye and delayed video-based
Milne, D. (2014). Towards an evidence-based approach to clinical super- supervised cognitive-behavioral therapy. A randomized controlled trial.
vision. In C. E. Watkins, Jr., & D. L. Milne (Eds.), The Wiley interna- Journal of Clinical Psychology, 74, 509 –522. http://dx.doi.org/10.1002/
tional handbook of clinical supervision (pp. 38 – 60). Chichester, United jclp.22519
Kingdom: Wiley. http://dx.doi.org/10.1002/9781118846360.ch3 Proctor, B. (1988). A cooperative exercise in accountability. In M. Marken
Milne, D., Aylott, H., Fitzpatrick, H., & Ellis, M. (2008). How does clinical & M. Payne (Eds.), Enabling and ensuring (pp. 21–34). Leicester,
supervision work? Using a “best evidence synthesis” approach to con- United Kingdom: Leicester National Youth Bureau and Council for
struct a basic model of supervision. The Clinical Supervisor, 27, 170 – Education and Training in Youth and Community Work.
190. http://dx.doi.org/10.1080/07325220802487915 Reese, R. J., Usher, E. L., Bowman, D. C., Norsworthy, L. A., Halstead,
Muse, K., & McManus, F. (2013). A systematic review of methods for J. L., Rowlands, S. R., & Chisholm, R. R. (2009). Using client feedback
assessing competence in cognitive-behavioural therapy. Clinical Psy- in psychotherapy training: An analysis of its influence on supervision
chology Review, 33, 484 – 499. http://dx.doi.org/10.1016/j.cpr.2013.01 and counselor self-efficacy. Training and Education in Professional
.010 Psychology, 3, 157–168. http://dx.doi.org/10.1037/a0015673
Muse, K., McManus, F., Rakovshik, S., & Thwaites, R. (2017). Develop- Reiser, R. P., & Milne, D. L. (2016). A survey of CBT supervision in the
ment and psychometric evaluation of the Assessment of Core CBT Skills U. K.: Methods, satisfaction and training, as viewed by a selected sample
(ACCS): An observation-based tool for assessing cognitive behavioral of CBT supervision leaders. The Cognitive Behaviour Therapist, 9, e20.
therapy competence. Psychological Assessment, 29, 542–555. http://dx http://dx.doi.org/10.1017/S1754470X15000689
.doi.org/10.1037/pas0000372 Rodolfa, E., Bent, R., Eisman, E., Nelson, R., Rehm, L., & Ritchie, P.
Norcross, J. C., & Wampold, B. E. (2011). Evidence-based therapy rela- (2005). A cube model for competency development: Implications for
tionships: Research conclusions and clinical practices. Psychotherapy, psychology educators and regulators. Professional Psychology: Re-
48, 98 –102. http://dx.doi.org/10.1037/a0022161 search and Practice, 36, 347–354. http://dx.doi.org/10.1037/0735-7028
Novoa-Gómez, M., Córdoba-Salgado, O., Rojas, N., Sosa, L., Cifuentes, .36.4.347
D., & Robayo, S. (2019). A descriptive analysis of the interactions Rodriguez-Menendez, G., Dempsey, J. P., Albizu, T., Power, S., & Camp-
during clinical supervision. Frontiers in Psychology, 10, 669. http://dx bell Wilkerson, M. (2017). Faculty and student perceptions of clinical
.doi.org/10.3389/fpsyg.2019.00669 training experiences in professional psychology. Training and Educa-
Ober, A. M., Granello, D. H., & Henfield, M. S. (2009). a synergistic tion in Professional Psychology, 11, 1–9. http://dx.doi.org/10.1037/
model to enhance multicultural competence in supervision. Counselor tep0000137
Education and Supervision, 48, 204 –221. http://dx.doi.org/10.1002/j Romans, J. S., Boswell, D. L., Carlozzi, A. F., & Ferguson, D. B. (1995).
.1556-6978.2009.tb00075.x training and supervision practices in clinical, counseling, and school
O’Connor, B. P. (2001). Reasons for less than ideal psychotherapy super- psychology programs. Professional Psychology: Research and Practice,
vision. The Clinical Supervisor, 19, 173–183. http://dx.doi.org/10.1300/ 26, 407– 412. http://dx.doi.org/10.1037/0735-7028.26.4.407
J001v19n02_10 Rosenthal, T. L., & Bandura, A. (1978). Psychological modeling: Theory
Olds, K., & Hawkins, R. (2014). Precursors to measuring outcomes in and practice. In S. L. Garfield & A. E. Bergin (Eds.), Handbook of
clinical supervision: A thematic analysis. Training and Education in psychotherapy and behavior change: An empirical analysis (2nd ed., pp.
Professional Psychology, 8, 158 –164. http://dx.doi.org/10.1037/ 621– 658). New York, NY: Wiley.
tep0000034 Rozek, D. C., Serrano, J. L., Marriott, B. R., Scott, K. S., Hickman, L. B.,
Orlinsky, D. E., & Ronnestad, M. H. (2005). How psychotherapists de- Brothers, B. M., . . . Simons, A. D. (2018). Cognitive behavioural
velop: A study of therapeutic work and professional growth. Washing- therapy competency: Pilot data from a comparison of multiple perspec-
ton, DC: American Psychological Association. http://dx.doi.org/10 tives. Behavioural and Cognitive Psychotherapy, 46, 244 –250. http://
.1037/11157-000 dx.doi.org/10.1017/S1352465817000662
Overington, L., Fitzpatrick, M., Drapeau, M., & Hunsley, J. (2016). Per- Schroeder, M., Andrews, J. J. W., & Hindes, Y. L. (2009). Cross-racial
spectives of internship training directors on the use of progress moni- supervision: Critical issues in the supervisory relationship. Canadian
toring measures. Canadian Psychology/Psychologie canadienne, 57, Journal of Counselling, 43, 295–310.
120 –129. http://dx.doi.org/10.1037/cap0000051 Shafranske, E. P., & Falender, C. A. (2016). Clinical supervision. In J. C.
Owen, J., Wampold, B. E., Kopta, M., Rousmaniere, T., & Miller, S. D. Norcross, G. R. VandenBos, & D. K. Freedheim (Eds.), APA handbook
(2016). As good as it gets? Therapy outcomes of trainees over time. of clinical psychology: Vol. 5. Education and profession (pp. 175–196).
Journal of Counseling Psychology, 63, 12–19. http://dx.doi.org/10.1037/ Washington, DC: American Psychological Association. http://dx.doi
cou0000112 .org/10.1037/14774-012
Peterson, A. P., & Fagan, C. (2017). Training the next generation in routine Shimokawa, K., Lambert, M. J., & Smart, D. W. (2010). Enhancing
outcome monitoring: Current practices in psychology training clinics. treatment outcome of patients at risk of treatment failure: Meta-analytic
Training and Education in Professional Psychology, 11, 182–189. http:// and mega-analytic review of a psychotherapy quality assurance system.
dx.doi.org/10.1037/tep0000148 Journal of Consulting and Clinical Psychology, 78, 298 –311. http://dx
Pilling, S., & Roth, A. D. (2014). The competent clinical supervisor. In .doi.org/10.1037/a0019247
 ENHANCING PSYCHOTHERAPY SUPERVISION 301

Strom, T., Arbisi, P., Thuras, P., Leskela, J., Finn, J., Anderson, C., & Supervisor, 30, 235–256. http://dx.doi.org/10.1080/07325223.2011
Siegel, W. (2016). Summative intern performance evaluations: A wealth .619417
of information or a gathering of “two cents”? Training and Education Watkins, C. E., Jr. (2012). Development of the psychotherapy supervisor:
in Professional Psychology, 10, 78 – 83. http://dx.doi.org/10.1037/ Review of and reflections on 30 years of theory and research. American
tep0000114 Journal of Psychotherapy, 66, 45– 83. http://dx.doi.org/10.1176/appi
Sue, D. W., Sue, D., Neville, H. A., & Smith, L. (2019). Counseling the .psychotherapy.2012.66.1.45
culturally diverse: Theory and practice (8th ed.). Hoboken, NJ: Wiley. Webb, C. A., Derubeis, R. J., & Barber, J. P. (2010). Therapist adherence/
Swift, J. K., Callahan, J. L., Rousmaniere, T. G., Whipple, J. L., Dexter, K., competence and treatment outcome: A meta-analytic review. Journal of
& Wrape, E. R. (2015). Using client outcome monitoring as a tool for Consulting and Clinical Psychology, 78, 200 –211. http://dx.doi.org/10
supervision. Psychotherapy, 52, 180 –184. http://dx.doi.org/10.1037/ .1037/a0018912
a0037659 Weck, F., Jakob, M., Neng, J. M., Höfling, V., Grikscheit, F., & Bohus, M.
Tao, K. W., Owen, J., Pace, B. T., & Imel, Z. E. (2015). A meta-analysis (2016). The effects of bug-in-the-eye supervision on therapeutic alliance
of multicultural competencies and psychotherapy process and outcome. and therapist competence in cognitive-behavioural therapy: A random-
Journal of Counseling Psychology, 62, 337–350. http://dx.doi.org/10 ized controlled trial. Clinical Psychology & Psychotherapy, 23, 386 –
.1037/cou0000086 396. http://dx.doi.org/10.1002/cpp.1968
Terry, J., Gonsalvez, C., & Deane, F. P. (2017). Brief online training with Weck, F., Kaufmann, Y., & Witthöft, M. (2017). Topics and techniques in
standardised vignettes reduces inflated supervisor ratings of trainee clinical supervision in psychotherapy training. The Cognitive Behaviour
practitioner competencies. Australian Psychologist, 52, 130 –139. http:// Therapist, 10, e3. http://dx.doi.org/10.1017/S1754470X17000046
dx.doi.org/10.1111/ap.12250
Welsh, M. (2011). Measuring teacher effectiveness in gifted education:
Thomas, J. T. (2014). International ethics for psychotherapy supervisors:
Some challenges and suggestions. Journal of Advanced Academics, 22,
Principles, practices, and future directions. In C. E. Watkins, Jr., & D. L.
750 –770. http://dx.doi.org/10.1177/1932202X11424882
Milne (Eds.), The Wiley international handbook of clinical supervision
Westberg, J., & Jason, H. (1993). Providing constructive feedback. In J.
(pp. 129 –154). Chichester, United Kingdom: Wiley. http://dx.doi.org/
Westberg & H. Jason (Eds.), Collaborative clinical education: The
10.1002/9781118846360.ch6
foundation of effective health care (pp. 297–318). New York, NY:
Thomas, J. C., & Hersen, M. (Eds.). (2009). Handbook of clinical psychol-
Springer.
ogy competencies. New York, NY: Springer.
Wheeler, S., & Richards, K. (2007). The impact of clinical supervision on
Town, J. M., Diener, M. J., Abbass, A., Leichsenring, F., Driessen, E., &
counselors and therapists, their practice and their clients: A systematic
Rabung, S. (2012). A meta-analysis of psychodynamic psychotherapy
outcomes: Evaluating the effects of research-specific procedures. Psy- review of the literature. Counselling & Psychotherapy Research, 7,
chotherapy, 49, 276 –290. http://dx.doi.org/10.1037/a0029564 54 – 65. http://dx.doi.org/10.1080/14733140601185274
Townend, M., Iannetta, L., & Freeston, M. (2002). Clinical supervision in Wilson, H. M., Davies, J. S., & Weatherhead, S. (2016). Trainee therapists’
practice: A survey of U. K. cognitive behavioural psychotherapists experiences of supervision during training: A meta-synthesis. Clinical
accredited by the BABCP. Behavioural and Cognitive Psychotherapy, Psychology & Psychotherapy, 23, 340 –351. http://dx.doi.org/10.1002/
30, 485–500. http://dx.doi.org/10.1017/S1352465802004095 cpp.1957
Tsui, M., O’Donoghue, K., & Ng, A. N. T. (2014). Culturally competent Wolf, K. (2015). Leniency and halo bias in industry-based assessments of
and diversity-sensitive clinical supervision: An international perspec- student competencies: A critical, sector-based analysis. Higher Educa-
tive. In C. E. Watkins, Jr., & D. L. Milne (Eds.), The Wiley international tion Research & Development, 34, 1045–1059. http://dx.doi.org/10
handbook of clinical supervision (pp. 238 –254). Chichester, United .1080/07294360.2015.1011096
Kingdom: Wiley. http://dx.doi.org/10.1002/9781118846360.ch10 Worthen, V., & Lambert, M. (2007). Outcome oriented supervision: Ad-
Vinton, L., & Wilke, D. J. (2011). Leniency bias in evaluating clinical vantages of adding systematic client tracking to supportive consulta-
social work student interns. Clinical Social Work Journal, 39, 288 –295. tions. Counselling & Psychotherapy Research, 7, 48 –53. http://dx.doi
http://dx.doi.org/10.1007/s10615-009-0221-5 .org/10.1080/14733140601140873
Walz, G. R., & Johnston, J. A. (1963). Counselors look at themselves on
video tape. Journal of Counseling Psychology, 10, 232–236. http://dx
.doi.org/10.1037/h0047698 Received December 18, 2018
Watkins, C. E., Jr. (2011). Does psychotherapy supervision contribute to Revision received July 31, 2019
patient outcomes? Considering thirty years of research. The Clinical Accepted August 2, 2019 䡲