See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/51976929

Factors confounding the assessment of reflection: A critical review

Article in BMC Medical Education · December 2011

DOI: 10.1186/1472-6920-11-104 · Source: PubMed

CITATIONS READS

137 4,092

7 authors, including:

Sebastiaan Koole Tim Dornan

Vrije Universiteit Brussel Maastricht University
36 PUBLICATIONS 800 CITATIONS 309 PUBLICATIONS 28,497 CITATIONS

SEE PROFILE SEE PROFILE

Martin Valcke Janke Cohen-Schotanus

Ghent University University of Groningen
558 PUBLICATIONS 17,627 CITATIONS 129 PUBLICATIONS 3,587 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Martin Valcke on 23 May 2014.

The user has requested enhancement of the downloaded file.

Koole et al. BMC Medical Education 2011, 11:104
http://www.biomedcentral.com/1472-6920/11/104

REVIEW Open Access

Factors confounding the assessment of reflection:

a critical review
Sebastiaan Koole1*, Tim Dornan2, Leen Aper1, Albert Scherpbier3, Martin Valcke4, Janke Cohen-Schotanus5 and
Anselme Derese1

Abstract
Background: Reflection on experience is an increasingly critical part of professional development and lifelong
learning. There is, however, continuing uncertainty about how best to put principle into practice, particularly as
regards assessment. This article explores those uncertainties in order to find practical ways of assessing reflection.
Discussion: We critically review four problems: 1. Inconsistent definitions of reflection; 2. Lack of standards to
determine (in)adequate reflection; 3. Factors that complicate assessment; 4. Internal and external contextual factors
affecting the assessment of reflection.
Summary: To address the problem of inconsistency, we identified processes that were common to a number of
widely quoted theories and synthesised a model, which yielded six indicators that could be used in assessment
instruments. We arrived at the conclusion that, until further progress has been made in defining standards,
assessment must depend on developing and communicating local consensus between stakeholders (students,
practitioners, teachers, supervisors, curriculum developers) about what is expected in exercises and formal tests.
Major factors that complicate assessment are the subjective nature of reflection’s content and the dependency on
descriptions by persons being assessed about their reflection process, without any objective means of verification.
To counter these validity threats, we suggest that assessment should focus on generic process skills rather than the
subjective content of reflection and where possible to consider objective information about the triggering situation
to verify described reflections. Finally, internal and external contextual factors such as motivation, instruction,
character of assessment (formative or summative) and the ability of individual learning environments to stimulate
reflection should be considered.

Background treatments, defining new personal learning objectives,

Physicians and other healthcare workers act in challen- and planning future actions in pursuit of excellence
ging professional environments. There is an exponential [2-5]. Hence, the ability to reflect is an important out-
growth in knowledge and treatment options, patients come parameter for health care professionals [6-9]. As a
are becoming more articulate and demanding, and result, many educational institutions incorporate the
inter-professional collaboration is the rule rather than ability to reflect as an objective of their vocational pro-
the exception. Lifelong learning is, consequently, crucial grams, premised on a belief that reflective thinking is
to the provision of up-to-date healthcare services [1]. something that can be developed rather than a stable
Rather than just attending conferences, lifelong learning personality trait [4,10,11].
today is seen as a continuous process, embedded in There is, however, uncertainty about how best to help
everyday professional practice. At its core lies practi- people develop their ability to reflect [11]. Lack of an
tioners’ ability to reflect upon their own actions, con- agreed way of assessing reflection is a particular obstacle
tinuously reviewing the processes and outcomes of to progress because assessment is needed for the identi-
fication of effectiveness of educational strategies and for
* Correspondence: [email protected] research purposes [3]. Assessment has also a motiva-
1
Centre for Educational Development, Faculty of Medicine and Health tional influence as a source for feedback (formative
Sciences, Ghent University, Ghent, Belgium
Full list of author information is available at the end of the article
assessment) and when to judge whether requisite levels

© 2011 Koole et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons
Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in
any medium, provided the original work is properly cited.
Koole et al. BMC Medical Education 2011, 11:104 Page 2 of 9
http://www.biomedcentral.com/1472-6920/11/104

of competence have been attained (summative assess- as the founder of the concept of reflection in an educa-
ment) [3,4,12]. The persisting lack of clarity about how tional context. He described reflective thought as “active,
to operationalise reflective learning is symptomatic of an persistent, and careful consideration of any belief or
even deeper problem. Different, widely accepted theories supposed form of knowledge in the light of the grounds
define reflection in different ways, consider different that support it, and the further conclusions to which it
outcomes as important, define different dimensions tends” [15]. He saw reflective thinking in the education
along which reflection could be assessed and point of individuals as a lever for the development of a wider
towards different standards [11]. Consequently, research democratic society.
findings are hard to compare. This unsatisfactory state In line with his work, Boud et al emphasised reflection
of affairs leaves curriculum leaders without practical as a tool to learn from experience in experiential learn-
guidelines, ways of identifying and supporting students ing [16]. They identified reflection as a process that
who are weak reflectors, and ways of judging whether looks back on experience to obtain new perspectives
interventions are improving learners’ ability to reflect. and inform future behaviour. A special feature of their
The purpose of this article is to review four factors, description of reflection in three stages - 1. Returning to
which confound the assessment of reflection: an experience; 2. attending to feelings; and 3. re-evaluat-
1. Non-uniformity in defining reflection and linking ing the experience - was the emphasis it placed on the
theory with practice. role of emotions.
2. A lack of agreed standards to interpret the results Moon described reflection as an input-outcome pro-
of assessments. cess [17]. She identified reflection as a mental function
3. Threats to the validity of current methods of asses- transforming factual or theoretical, verbal or non-verbal
sing reflection. knowledge, and emotional components generated in the
4. The influence of internal and external contextual past or present into the output of reflection (e.g. learn-
factors on the assessment of reflection. ing, critical review or self-development).
Our approach was to identify all widely quoted the- Schön’s concept of the reflective practitioner identi-
ories, read them in depth, and triangulate them against fied reflection as a tool to deal with complex profes-
one another to find what they (dis)agreed on and gaps sional situations [18,19]. Reflection in a situation
between them. The result of this exercise was an inter- (reflection-in-action) is linked to practitioners’ immedi-
pretive framework, which we used to structure the ‘Dis- ate behaviour. Reflection after the event (reflection-on-
cussion’ section. A test of the framework is beyond the action) provides insights that can improve future prac-
scope of this article, whose aim is to make the frame- tice. Those two types of reflection together form a con-
work and guidelines available to other people interested tinuum for practice improvement.
in implementing and/or assessing reflection in The term ’reflective learning’ describes reflection in
education. the context of experiential learning. Kolb’s widely
accepted experiential learning cycle describes four stages
Discussion of learning: 1. having an experience (concrete experi-
1. Defining reflection ence), 2. reflective observation (reflecting on this experi-
Studies about reflection in professional practice and ence), 3. abstract conceptualisation (learning from the
education are widespread in the literature; however, experience) and 4. active experimentation (trying out
their results are hard to generalise or compare because what you have learned) [20]. These four stages are con-
they conceptualise reflection in such different ways. Boe- ceptualised as a spiral, each of whose turns is a step for-
nink et al [10] described reflection in terms of the num- ward in a person’s experiential learning.
ber of different perspectives a person used to analyse a Lifelong learning is considered today as essential for
situation. Reflection ranged from a single perspective to maintaining a high standard of professional practice.
a balanced approach considering multiple relevant per- Mezirow’s transformative learning theory described life-
spectives. Aukes et al [13] emphasised emotional and long learning in terms of learners’ transforming frames
communication components when they conceptualised of reference, in which reflection is the driving force [21].
personal reflection as a combination of self-reflection, Towards an ‘eclectic model’ of common elements
empathic reflection, and reflective communication. Although contemporary reflection models build on
Sobral’s [14] emphasis on reflection-in-learning those theories, the diversity between them is a cause of
approached reflection from a learning perspective. continuing uncertainty. In response, we have assembled
If those three perspectives exemplify inconsistency in a simple comprehensive model from their common
the field, the work of Dewey, Boud, Schön, Kolb, Moon, parts (table 1). Atkins and Murphy [22] identified reflec-
and Mezirow exemplifies shared ground between reflec- tion as: 1. ‘awareness of uncomfortable feelings and
tion theories and used terms. Dewey is usually regarded thoughts’, resulting in 2. an ‘analysis of feelings and
Koole et al. BMC Medical Education 2011, 11:104 Page 3 of 9
http://www.biomedcentral.com/1472-6920/11/104

Table 1 Overview of theories/models/findings integrated into the model of common elements

Author Theory/Model/Finding Summary
Atkins & Reflective processes (model) Three key stages of the reflective process: 1. Awareness of uncomfortable feelings, 2. Critical
Murphy analysis of feelings and knowledge, 3. New perspectives; associated skills: self-awareness,
description, critical analysis, synthesis and evaluation.
Boud Promoting reflection in learning Reflection process consists of 3 interrelated stages: 1. returning to the experience, 2.
(model) attending to feelings, 3. re-evaluating experience, triggered by experiences and leading to
outcome.
Bourner Interrogating experiences with searching questions distinguishes reflective from unreflective
thinking.
Korthagen ALACT model Reflection is a cyclic process of: Action, Looking back on action, Awareness of essential
aspects, Creating alternative methods of action, and Trial.
Mamede & The structure of reflective practice in Reflective practice consists of a 5 factor model: deliberate induction; deliberate deduction;
Schmidt medicine (finding) testing and synthesising; openness for reflection, and meta-reasoning.
Mezirow Transformative learning theory Reflection leads to changed assumptions and frames of references which ground
transformative learning.
Moon Input-outcome model Reflection is a mental process that is based on input (theories, constructed knowledge or
feelings) that has an outcome/purpose (self-development, learning, decisions, resolutions of
uncertainty, ...).
Schön The reflective practitioner (theory) Reflection is a key factor for professionals to deal with complex situations and for
professional development. He identified reflection-in-action and reflection-on-action.
Stockhausen Clinical learning spiral (model) Reflective practice is related to professional growth; Clinical learning consists of preparative
phase, constructive phase, reflective practice and reconstructive phase.

knowledge’, finally leading to 3. ‘new perspectives’. They reference’ that, according to Mezirow, directs the analy-
described self-awareness, critical analysis, synthesis, and sis and represents “the structure of assumptions and
evaluation as requisite skills for this process. Since those expectations through which we filter sense impressions”
three phases are common to the work of previous [21]. This personal perspective, made up of our percep-
authors, they provided a logical starting point for our tions, cognitions, feelings and dispositions (intentions,
model. We complemented Atkins and Murphy’s phases expectations and purposes), creates a context in which
with insights from other models. Korthagen’s ALACT we give meaning to our sensory experiences. If the first
model (’Action, Looking back on action, Awareness of phase of reflection, then, is identified as the description
essential aspects, Creating alternative methods of action, of an experience and the awareness of feelings, thoughts,
and Trial’) [23] describes the first phase of ‘becoming and other essential aspects, our second phase of reflec-
aware’ in two steps: a general retrospective action and a tion is analysing experiences by reflective inquiry, which
more interpretive action. Integrating those two theories, triggers a process of analysis within a person’s unique
resulted in a first phase (’reviewing an experience’) with frame of reference.
two subcomponents: 1. generally describing what hap- Moon’s input-outcome model emphasises that reflec-
pened and 2. identifying essential aspects by considering tion is purposeful [17]. This purpose is identified by
both thoughts, feelings and contextual factors. Atkins and Murphy in the third phase of reflection as
Just reviewing an experience, however, does not neces- the ‘identification of new perspectives’ [22]. Both
sarily lead to effective reflection. For Bourner [24], using Korthagen and Boud, however, included an additional
searching questions to interrogate an experience was the stage - the conversion of those new perspectives into
key difference between reflecting and thinking and he actions that are the starting point for new reflective
saw ‘reflective inquiry’ as a crucial component of reflec- cycles [16,23]. The ‘reconstruction phase’ of Stockhau-
tion. This aspect of reflection was also represented in sen’s clinical learning spiral model of reflective practice
Mamede and Schmidt’s proposed structure of reflective among undergraduate nursing students in clinical prac-
practice as ‘openness to reflection’ [25]. Bourner only tice settings had the same function [26]. During this
emphasised posing searching questions, however, not phase, reflective insights were transformed into plans for
answering them. Korthagen’s approach supplements future actions. Since those actions could lead to further
Bourner’s by contributing ‘creating alternative methods reflections, reflecting on experiences was identified as a
of action’ as a process of answering questions. This cyclic process that transformed significant experiences
addition is compatible with Boud’s characterization of into deliberate, well informed practical actions. We
analysis as a combination of association, integration, incorporated those insights into the eclectic model by
validation and appropriation. The internal dialogue that defining the outcome of a reflection process as the iden-
results is conducted within a ‘personal frame of tification of new perspectives, which leads to future
Koole et al. BMC Medical Education 2011, 11:104 Page 4 of 9
http://www.biomedcentral.com/1472-6920/11/104

actions informed by reflection. Stockhausen also into behaviour that has been informed by reflection’.
described a preparatory phase to establish objectives for This behaviour generates new experiences and so a new
a new clinical experience. This phase, which other reflection cycle begins.
authors have labelled as reflection-before-action [27,28], From model building to developing indicators for
is incorporated into the eclectic model by representing assessment of reflection
reflection as a cyclical process. It allows reflection to be The aim of identifying common elements was to ground
informed by learning goals arising from past reflections the assessment of reflection in existing, widely used the-
and stresses the importance of reflection as a develop- ories. It is practically useful because each of the six
mental process. Both Korthagen and Stockhausen have items in the three phase model can be translated into
highlighted this process with the term reflection spiral an indicator of the adequacy of reflection processes
with each winding leading to a higher order of under- (table 2). Together, they provide a comprehensive over-
standing, practice or learning [23,26]. view of a person’s ability to go through the process and
Figure 1 shows the complete eclectic model, which are in line with the reflective skills identified by Duke
describes reflection in three phases: 1. ‘Reviewing the and Appleton [29]. Taken individually, the indicators
experience’, 2. ‘Critical analysis’, and 3. ‘Reflective out- can provide specific feedback about components of
come’. Reflection, according to the model, is a cyclical reflection, which makes it possible to give structured,
process, which originates from experience and informs focused feedback, and direct training towards aspects of
future behaviour. Each phase has two items, described reflection that the indicators have defined as insufficient.
in practical terms to make it possible to put the model Such training could, for example, provide exercises on
into practice. Reviewing the experience has two compo- describing personal thoughts and feelings or identifying
nents: ‘description of the experience as a whole’, and learning goals. So, in summary, the modular nature of
‘awareness of essential aspects based on the considera- the model and its indicators makes it possible to tailor
tion of personal thoughts, feelings, and important con- education to individual needs. But, for that, criteria to
textual factors’. Critical analysis starts with ‘reflective judge someone as competent in reflection are needed.
inquiry’ - posing searching questions about an experi-
ence - and progresses to ‘searching for answers’ while 2. Standards to interpret reflection assessment
remaining aware of the ‘frame of reference’ within Here, again, there is a lack of consensus in published lit-
which the inquiry is being conducted. Reflective out- erature. A few researchers have attempted to rank
come comprises the ‘new perspectives’ resulting from reflections. Wong et al [30] evaluated reflection in writ-
phase two, and the ‘translation of those perspectives ten papers by identifying reflective activities using two
coding schemes. One, based on Boud’s theory, had six
items: attending to feelings, association, integration, vali-
dation, appropriation and outcome of reflection. The
other, based on the work of Mezirow, labelled students
as: non-reflectors (no evidence of reflective thinking),
reflectors (evidence of relating experience to learning
opportunities) and critical reflectors (evidence of inte-
grating reflective outcomes in professional behaviour).
The researchers found Boud’s categories hard to apply
to written materials, resulting in less reliable coding
than using Mezirow’s scheme. With only three cate-
gories, however, this latter scheme had a limited capa-
city to discriminate between people. Kember et al [31]
addressed this issue by using a finer-tuned coding
scheme based on the work of Mezirow. Their seven
categories ranged from unreflective thinking (habitual
action, introspection and thoughtful action) to reflective
thinking (content reflection, process reflection, content
and process reflection and premise reflection). They
dealt with the complexity of the coding scheme by pro-
viding guidelines for assessors, which resulted in an
acceptable interrater reliability (Cronbach alpha 0.74).
Figure 1 Model of common elements describing the reflection Boenink et al [10] used an alternative approach, which
process.
ranked reflections from 1-10. Their scale was based on
Koole et al. BMC Medical Education 2011, 11:104 Page 5 of 9
http://www.biomedcentral.com/1472-6920/11/104

Table 2 Operational indicators of the reflection process

Aspect of the reflection process Indicators
Reviewing the experience 1. The ability to describe an event/situation adequately.
2. The ability to identify essential elements and to describe own thoughts and feelings
Critical analysis 3. The ability to ask searching questions.
4. The ability to answer searching questions and being aware of the frames of references in use.
Reflective outcome 5. The ability to draw conclusions.
6. The ability to describe concrete learning goals and plans for future action.

the number of perspectives students described in short positive and negative experiences [18,34]. That situation
written reflective reactions to a case vignette describing need not be left unchallenged because there is research
a challenging situation. The advantage of this approach showing reflection can be influenced positively by train-
was the limited need to make interpretations when iden- ing [14,32,35], but the minimum level of reflection
tifying the perspectives. The scale was limited, however, needed to have a positive effect on practice remains to
by measuring only one aspect of reflection (being aware be defined.
of the frame of reference used). Duke and Appleton [29] Until standards have been formulated that can identify
developed a broader marking grid to score reflective practitioners whose level of reflection is adequate, it
reports. It assessed eight skills that support reflection, seems reasonable to clarify to stakeholders (curriculum
identified by a literature review, on five-level scales, developers, students, practitioners, assessors) what
‘anchored’ and linked to a grade (A, B+, B, C and F). By reflection skills are expected and urge learners to
providing grades, these authors were the first to set develop them as far as possible. We offer the presented
standards for reflective skills. Despite having based the model of common elements as a way of doing that. In
reflection skills that were included in the scale on a lit- promoting reflective learning, however, a balance has to
erature review, however, the authors did not disclose be struck between developing an ability to reflect and
how they linked the levels to grades. increasing the frequency of reflection. It has been
Coding schemes have also been used to evaluate argued that critically analysing personal practice after
reflection in interviews. Boyd [32] assessed reflective every experience can cause a disabling level of uncer-
judgement using a coding scheme based on seven stages tainty [36,37]. Future standards will therefore have to
of intellectual development described by King and consider the balance between the quality of reflection
Kitchener: Pre-reflective thinking (stages 1-3); quasi- and its efficient and systematic application in practice,
reflective thinking (stages 4 and 5); and reflective think- but not to the stage of being counterproductive.
ing (stages 6 and 7). Measurements made with the scale
had an interrater reliability of 0.76 (Cronbach alpha). 3. Factors that complicate assessment
Based on the approach coding schemes can be divided The metacognitive nature of reflection is an important
into two groups. A first approach ranks reflections complicating factor of reflection assessment [4]. It
according to levels, ranging from descriptive and/or implies a thinking process only accessible to the reflect-
unreflective to reflective or critical reflective based on ing person and hence only observable by assessors
the used theory [30-32]. The other approach is the iden- through that person’s interpretative descriptions. Sub-
tification of phases in the reflection process considering jects are most often asked to ‘translate’ their reflections
items of reviewing an experience, analysis and reflective into written words, which are assessed against coding
outcome based on the used model of reflection [29,30]. schemes or scoring grids [29-31,38-40]. Other suggested
This discrepancy is a complicating factor for interpret- methods to ‘visualise’ reflections include the verbalisa-
ing results as levels and phases are incompatible. tion in interviews [32,41,42], written responses to vign-
Notwithstanding limited ability to compare the find- ettes [10], or reflective writings in portfolios [34,43].
ings in the reported studies, because of the variety in Assessors’ dependency on a person’s interpretative
used the scales and models of reflection, their results description is a serious threat to the validity of assess-
share a common feature. Within their own scale, all stu- ments of reflection because they have to judge selective
dies demonstrate learners to have very limited mastery descriptions without being able to verify their adequacy.
of reflection, indicating an apparent room for improve- Accordingly this approach fails to detect bias caused by
ment. Inadequate reflection has a negative effect on a lack of (un)intentional hindsight and introspection
practice [3,18,33], presumably because learners with a ability [44,45], reflections being determined by the
limited ability to reflect let ‘tunnel vision’ stop them requirements of the assessment and selectivity and/or
questioning their behaviour in response to significant incompleteness of aspects they portray [44]. Interviews
Koole et al. BMC Medical Education 2011, 11:104 Page 6 of 9
http://www.biomedcentral.com/1472-6920/11/104

have the advantage that they can pose clarifying ques- Furthermore, reflections are intimately linked to their
tions and monitor a reflecting person’s reactions, but triggering situations [16,18,19,53] so information about
they still leave assessors to ground their judgements in this initial event can provide an objective frame of refer-
potentially subjective and selective narrative accounts of ence to verify elements of the reflection. For example,
reflective activity. There are two related problems in when someone describes his communication as good,
that. Although the semantic skill of describing reflec- the real-time presence of an assessor or video-recording
tions is considered integral to effective reflection [46], of the event could give supporting information [54].
skills other than pure reflective skills are needed to turn Finding a feasible way of obtaining a rich picture of
reflection into writing and/or speech, which has a self- events that precede the reflection that has to be assessed
evident effect on reflective narratives [44]. The other is an important topic for future development.
problem lies in a decrease of motivation caused by the
non-alignment between the written approach to assess- 4. Internal and external contextual factors affecting
ment and a learners preferred learning style [12]. Find- reflection assessment
ings of Sandars and Homer [47] suggest the discrepancy The results of assessments of reflection are influenced
between ‘net generation’ students learning preference of by contextual factors as well as people’s ability to reflect.
group-based and technological multimedia activities Our argument now turns to those modulating factors.
(blogs, social networks, digital storytelling) and the text Motivation is considered to be an important mediator of
based approaches to reflective learning. Moreover, sup- learning and achievement in medical education [55,56].
porting learners to reflect with the creative use of multi- The expectancy-value model proposed by Wigfield and
media, will likely increase their commitment to reflect Eccles identifies the subjective value of a task to a per-
and stimulate even more efficient reflection [48]. son and their expectation of performing it successfully
Self-assessment questionnaires have the advantage of as main predictors of task performance [57]. Applied to
circumventing indirect observation [13,14,49,50], but reflection, it predicts that the perceived importance of
their requirement to introspect accurately introduces reflection for (professional) practice will determine the
another validity threat [22,51], because it is then unclear time and effort a person is willing to invest in it; those
if it is reflection or the ability to introspect that is being who do not expect a positive return are unlikely to
tested. Eva and Regehr [45] concluded that it is best not reflect profoundly and critically [4]. This motivational
to build solely on self-assessment approaches as they model also explains how personal factors like prior
tend to be inaccurate and they recommended triangulat- experience of reflective learning and a person’s under-
ing introspection with other forms of feedback. Asses- standing of the reflection process will influence motiva-
sor-based methods could meet this requirement, tion and consequently reflective behaviour. Hence
providing assessors could be relied upon to provide introductory sessions are important to frame the value
valid feedback. and intended outcomes of reflection [4]. Furthermore
Since there are such serious validity threats, the ques- the expectancy-value model also stresses external vari-
tion remains whether it is possible to assess reflection at ables, which might include aspects of teaching and/or
all. Two elements appear to be important. In search for assessment. It is reciprocal in nature. If involvement in
a valid approach, Bourner [24] suggested the content reflective activities results in perceived better perfor-
and the process of reflection should be viewed as two mance (internal) and/or external appraisal, rewards, or
separate entities. While the content is a barrier to reinforcement, a feedback loop starts to operate.
assessment because of its subjective nature, the process Whereas reflection was traditionally conceived of as a
has a more general character. He transferred this strictly individual process, ideas are shifting towards
approach from the assessment of critical thinking where conceptualising it as a process facilitated by social inter-
the use of questions to analyse ideas, evidence and action [4,45]. A stimulating environment in which
assertions demonstrates a person’s capacity for critical supervisors and peers give learners regular feedback and
thinking [24]. Similarly Bourner proposed that observa- ask thought-provoking questions can, from that point of
ble items, like the ability to formulate learning goals, view, be expected to improve reflection. With non-jud-
should be used to demonstrate a person’s capacity for gemental questions, facilitators can encourage to fully
reflecting. This approach demonstrates some parallels explore the situation, to consider alternative perspectives
with the content specificity of clinical reasoning [52]. and solutions, and to uncover taken-for-granted
However, opposed to elements in reflections such as assumptions [3]. Furthermore, situations and reflection
learning goals or plans for future actions which meaning upon can provoke strong emotions and negative
for the learner is subjective, content specific knowledge thoughts which could potentially form a barrier
has a more objective character. obstructing efficient reflection. A facilitator can help to
Koole et al. BMC Medical Education 2011, 11:104 Page 7 of 9
http://www.biomedcentral.com/1472-6920/11/104

assimilate these strong emotions and refocus on the assessment questionnaires shares both that validity pro-
reflection process [12,16]. To fully explore reflective blem and the inherent limitations of self-assessment. To
thoughts, feelings and possible emotions, it is crucial to counter these validity threats, it has been proposed that
create a safe environment established between the assessment should focus on the process rather than the
reflecting person and the facilitator(s) [3]. Next to sup- subjectively coloured content of reflection. In addition,
porting others, being a facilitator is also reported as as reflections are intimately entangled with their trigger-
even more effective for a person’s own reflections [58]. ing situational context, we suggest where possible to
Schön, however, warned that an unbalanced relationship consider objective information about this triggering
between learner and coach and an undue influence of situation allowing assessors to verify described reflec-
contextual factors could hinder reflective practice, as it tions. The reflection process is influenced by internal
could lead to defensiveness [18]. In line with this (eg. motivation, expectancy and prior experiences with
emphasis on contextual factors, Schaub et al developed reflection) and external factors (formative or summative
a scale to assess teachers’ competence in encouraging character of assessment, presence of facilitators and
reflective learning [59]. It asks learners to identify introduction to the assessment). Awareness of these fac-
whether teachers support self-insights, create a safe tors are important to develop effective educational stra-
environment, and encourage self-regulation. tegies, interpreting assessment results and finally the
Because of their influence on reflections contextual increase in understanding about the reflection process.
factors should be accounted for in educational and Based on the preceding discussion, we offer the follow-
assessment approaches. In education it will help to ing practical guidelines for educating and assessing
develop effective educational strategies and predict their reflection.
results to match the intended outcome. In assessment 1. Clearly define the concept of reflection and verify
considering contextual factors will contribute to the that all stakeholders (curriculum developers, students,
interpretation of results and in the understanding of the assessors and supervisors) adopt the same definition and
reflection process. Hence we suggest to consider internal intended outcomes.
and external contextual factors in education and 2. Be specific about what level of reflection skills is
assessment. expected, identifying good and inadequate reflection and
communicate this to all stakeholders.
Summary 3. Be aware of possible bias in self-assessment meth-
Whilst it is generally accepted that the ability to reflect ods, caused by inadequate ability to introspect.
is an important attribute for healthcare professionals, 4. Provide assessors with a perspective on the situation
there is considerable uncertainty about how best to fos- triggering the reflection to create the ability to verify the
ter it in educational practice. Lack of an agreed way of described reflections in an objective frame of additional
assessing reflection is a very important factor contribut- information.
ing to this uncertainty. There is, however, clearly dis- 5. Consider and report contextual factors when asses-
cernible common ground between reflection theories. By sing reflection and/or when engaging in reflective educa-
defining that common ground, we have been able to tion in support to interpret the outcomes.
assemble an eclectic model, which sees reflection as
comprised of: 1. reviewing experience; 2. critical analysis;
Acknowledgements
and 3. reflective outcomes. A way of reliably measuring The authors would like to thank Professor Karen Mann for critically reviewing
reflection is needed so summative judgements can be the manuscript and her supportive comments to improve the paper.
made and learners can receive effective feedback but
Author details
one has not, yet, been developed. Standards defining an 1
Centre for Educational Development, Faculty of Medicine and Health
essential minimum level of reflective ability are also Sciences, Ghent University, Ghent, Belgium. 2Department of Educational
needed. Until they are we urge to develop and commu- Development and Research, Faculty of Health, Medicine and Life Sciences,
Maastricht University, Maastricht, the Netherlands. 3Institute for Medical
nicate a local consensus between stakeholders (students, Education, Faculty of Health, Medicine and Life Sciences, Maastricht
practitioners, teachers, supervisors, curriculum develo- University, the Netherlands. 4Department of Educational Studies, Faculty of
pers) about what is expected in exercises and formal Psychology and Educational Sciences, Ghent University, Belgium. 5Centre for
Research and Innovation in Medical Education, Faculty of Medical Sciences,
tests. University of Groningen and University Medical Centre Groningen,
Because reflection is a metacognitive process, it can Groningen, the Netherlands.
only be assessed indirectly; through written reflections
Authors’ contributions
in vignettes or portfolios, or spoken expressions in inter- SK conceptualized the idea and SK, TD were involved in writing the initial
views. These methods do not allow assessors to verify drafts. All authors were involved in the revised drafts and made essential
information related to the reflections reported, which is contributions to this paper and critically reviewed and approved the final
manuscript.
a serious limitation. The widespread use of self-
Koole et al. BMC Medical Education 2011, 11:104 Page 8 of 9
http://www.biomedcentral.com/1472-6920/11/104

Competing interests 26. Stockhausen L: The Clinical Learning Spiral - A Model to Develop
The authors declare that they have no competing interests. Reflective Practitioners. Nurse Education Today 1994, 14(5):363-371.
27. Greenwood J: Reflective practice: a critique of the work of Argyris and
Received: 15 June 2011 Accepted: 28 December 2011 Schön. Journal of Advanced Nursing 1993, 18(8):1183-1187.
Published: 28 December 2011 28. Cheetham G, Chivers G: A New Look at Competent Professional Practice.
Journal of European Industrial Training 2000, 24(7):374-383.
References 29. Duke S, Appleton J: The use of reflection in a palliative care programme:
1. Collins J: Education Techniques for Lifelong Learning Lifelong Learning a quantitative study of the development of reflective skills over an
in the 21st Century and Beyond. Radiographics 2009, 29(2):613-622. academic year. Journal of Advanced Nursing 2000, 32(6):1557-1568.
2. Andersen RS, Hansen RP, Sondergaard J, Bro F: Learning based on patient 30. Wong FKY, Kember D, Chung LYF, Yan L: Assessing the Level of Student
case reviews: an interview study. BMC Medical Education 2008, 8:43. Reflection from Reflective Journals. Journal of Advanced Nursing 1995,
3. Plack MM, Greenberg L: The Reflective Practitioner: Reaching for 22(1):48-57.
Excellence in Practice. Pediatrics 2005, 116(6):1546-1552. 31. Kember D, Jones A, Loke A: Determining the level of reflective thinking
4. Sandars J: The use of reflection in medical education: AMEE Guide No. from students’ written journals using a coding scheme based on the
44. Medical Teacher 2009, 31(8):685-695. work of Mezirow. International Journal of Lifelong Education 1999,
5. Li STT, Paterniti DA, Co JPT, West DC: Successful Self-Directed Lifelong 18(1):18-30.
Learning in Medicine: A Conceptual Model Derived From Qualitative 32. Boyd LD: Development of reflective judgement in the pre-doctoral
Analysis of a National Survey of Pediatric Residents. Academic Medicine dental clinical curriculum. European Journal of Dental Education 2008,
2010, 85(7):1229-1236. 12(3):149-158.
6. General Medical Council: Tomorrow’s Doctors. London: GMC; 2009 [http:// 33. Evans AW, McKenna C, Oliver M: Self-assessment in medical practice.
www.gmc-uk.org/TomorrowsDoctors_2009.pdf_39260971.pdf], [accessed Journal of the Royal Society of Medicine 2002, 95(10):511-513.
October 28th 2010]. 34. Pinsky LE, Monson D, Irby DM: How excellent teachers are made:
7. Scottish Deans’ Medical Curriculum Group: Learning Outcomes for the reflecting on success to improve teaching. Advances in Health Sciences
Medical Undergraduate in Scotland: A Foundation for Competent and Education 1998, 3:207-215.
Reflective Practitioners 3rd Edition. 2007 [http://www.scottishdoctor.org/ 35. Driessen EW, van Tartwijk J, Overeem K, Vermunt JD, van der Vleuten CPM:
resources/scottishdoctor3.doc], [accessed January 17th 2011]. Conditions for successful reflective use of portfolios in undergraduate
8. Nederlandse Federatie van Universitair Medische Centra: Raamplan artsen medical education. Medical Education 2005, 39(12):1230-1235.
opleiding. 2009 [http://www.nfu.nl/fileadmin/documents/ 36. Burnard P: Reflections on reflection. Nurse Education Today 2005,
Raamplan_Artsopleiding_2009.pdf], [accessed January 17th 2011]. 25(2):85-86.
9. Cowpe J, Plasschaert A, Harzer W, Vinkka-Puhakka H, Walmsley AD: Profile 37. Aukes LC: Personal reflection in medical education. PhD thesis University
and competences for the European dentist - update 2009.[http://www. of Groningen; 2008.
adee.org/cms/uploads/adee/ 38. Carr S, Carmody D: Experiential learning in women’s health: medical
ProfileCompetencesGraduatingEuropeanDentist1.pdf], [accessed March 3rd student reflections. Medical Education 2006, 40(8):768-774.
2011]. 39. Brady DW, Corbie-Smith G, Branch WT: “What’s Important to You?": The
10. Boenink AD, Oderwald AK, de Jonge P, van Tilburg W, Smal JA: Assessing Use of Narratives To Promote Self-Reflection and To Understand the
student reflection in medical practice. The development of an observer- Experiences of Medical Residents. Annals of Intern Medicine 2002,
rated instrument: reliability, validity and initial experiences. Medical 137(3):220-223.
Education 2004, 38(4):368-377. 40. Howe A, Barrett A, Leinster S: How medical students demonstrate their
11. Mann K, Gordon J, MacLeod A: Reflection and reflective practice in health professionalism when reflecting on experience. Medical Education 2009,
professions education: a systematic review. Advances in Health Sciences 43(10):942-951.
Education 2009, 14(4):595-621. 41. Hatton N, Smith D: Reflection in Teacher-Education - Towards Definition
12. Grant A, Kinnersley P, Metcalf E, Pill R, Houston H: Students’ views of and Implementation. Teaching and Teacher Education 1995, 11(1):33-49.
reflective learning techniques: an efficacy study at a UK medical school. 42. Peden-McAlpine C, Tomlinson PS, Forneris SG, Genck G, Meiers SJ:
Medical Education 2006, 40(4):379-588. Evaluation of a reflective practice intervention to enhance family care.
13. Aukes LC, Geertsma J, Cohen-Schotanus J, Zwierstra RP, Slaets JPJ: The Journal of Advanced Nursing 2005, 49(5):494-501.
development of a scale to measure personal reflection in medical 43. Tigelaar DEH, Dolmans DHJM, de Grave WS, Wolfhagen IHAP, Van
practice and education. Medical Teacher 2007, 29:177-182. derVleuten CPM: Portfolio as a tool to stimulate teachers’ reflections.
14. Sobral DT: An appraisal of medical students’ reflection-in-learning. Medical Teacher 2006, 28(3):277-282.
Medical Education 2000, 34(3):182-187. 44. Hargreaves J: So how do you feel about that? Assessing reflective
15. Dewey J: How we think Boston: DC Heath; 1910. practice. Nurse Education Today 2004, 24(3):196-201.
16. Boud D, Keogh R, Walker D: Reflection: Turning experience into learning 45. Eva KW, Regehr G: Self-assessment in the health professions: A
London: Kogan Page; 1985. reformulation and research agenda. Academic Medicine 2005, 80(10):
17. Moon JA: Reflection in learning and professional development: theory and S46-S54.
practice London: Kogan Page; 1999. 46. Pee B, Woodman T, Fry H, Davenport ES: Appraising and assessing
18. Schön DA: The reflective practitioner: How professionals think in action New reflection in students’ writing on a structured worksheet. Medical
York: Basic Books; 1983. Education 2002, 36(6):575-585.
19. Schön DA: Educating the reflective practitioner San Francisco: Jossey-Bass Inc. 47. Sandars J, Homer M: Reflective learning and the Net Generation. Medical
Publishers; 1987. Teacher 2008, 30(9-10):877-879.
20. Kolb DA: Experiential learning: Experience as a source for learning and 48. Sandars J, Murray C, Pellow A: Twelve tips for using digital storytelling to
development New Jersey: Prentice Hall; 1984. promote reflective learning by medical students. Medical Teacher 2008,
21. Mezirow J and Associates: Learning as Transformation: Critical Perspectives on 30(8):774-777.
a Theory in Progress San Francisco: Jossey-Bass; 2000. 49. Mamede S, Schmidt H: Correlates of reflective practice in medicine.
22. Atkins S, Murphy K: Reflection - A Review of the Literature. Journal of Advances in Health Sciences Education 2005, 10(4):327-337.
Advanced Nursing 1993, 18(8):1188-1192. 50. Kember D, Leung DYP, Jones A: Development of a Questionnaire to
23. Korthagen F, Vasalos A: Levels in reflection: core reflection as a means to Measure the Level of Reflective Thinking. Assessment and evaluation in
enhance professional growth. Teachers and Teaching: theory and practice higher education 2000, 25(4):381-395.
2005, 11(1):47-71. 51. Kruger J, Dunning D: Unskilled and Unaware of It: How Difficulties in
24. Bourner T: Assessing reflective learning. Education + Training 2003, Recognizing One’s Own Incompetence Lead to Inflated Self-
45(5):267-272. Assessments. Journal of Personality and Social Psychology 1999,
25. Mamede S, Schmidt HG: The structure of reflective practice in medicine. 77(6):1121-1134.
Medical Education 2004, 38(12):1302-1308.
 Koole et al. BMC Medical Education 2011, 11:104 Page 9 of 9
http://www.biomedcentral.com/1472-6920/11/104

52. Wimmers PF, Splinter TAW, Hancock GR, Schmidt HG: Clinical competence:
General ability or case-specific? Advances in Health Sciences Education
2007, 12(3):299-314.
53. Korthagen F, Lagerwerf B: Reframing the relationship between teacher
thinking and teacher behaviour: levels in learning about teaching.
Teachers and Teaching: theory and practice 1996, 2(2):161-190.
54. Liimatainen L, Poskiparta M, Karhila P, Sjogren A: The development of
reflective learning in the context of health counselling and health
promotion during nurse education. Journal of Advanced Nursing 2001,
34(5):648-658.
55. Artino AR, La Rochelle JS, Durning SJ: Second-year medical students’
motivational beliefs, emotions, and achievement. Medical Education 2010,
44(12):1203-1212.
56. Mann KV: Motivation in medical education: How theory can inform our
practice. Academic Medicine 1999, 74(3):237-239.
57. Wigfield A, Eccles JS: Expectancy-Value Theory of Achievement
Motivation. Contemporary Educational Psychology 2000, 25(1):68-81.
58. Dahlgren LO, Eriksson BE, Gyllenhammar H, Korkeila M, Saaf-Rothoff A,
Wernerson A, Seeberger A: To be and to have a critical friend in medical
teaching. Medical Education 2006, 40(1):72-78.
59. Schaub-de Jong MA, Schonrock-Adema J, Dekker H, Verkerk M, Cohen-
Schotanus J: Development of a student rating scale to evaluate teachers’
competencies for facilitating reflective learning. Medical Education 2011,
45(2):155-165.

Pre-publication history
The pre-publication history for this paper can be accessed here:
http://www.biomedcentral.com/1472-6920/11/104/prepub

doi:10.1186/1472-6920-11-104
Cite this article as: Koole et al.: Factors confounding the assessment of
reflection: a critical review. BMC Medical Education 2011 11:104.

Submit your next manuscript to BioMed Central

and take full advantage of:

• Convenient online submission

• Thorough peer review
• No space constraints or color figure charges
• Immediate publication on acceptance
• Inclusion in PubMed, CAS, Scopus and Google Scholar
• Research which is freely available for redistribution

Submit your manuscript at

www.biomedcentral.com/submit

View publication stats