Learning Outcomes

Brussels/Thessaloniki, 7 October 2016
Defining
NOTE and writing learning outcomes for vocational
education and training - A European handbook
DRAFT for discussion at Cedefop 2nd Policy Learning Forum 13-14

October 2016
Europe 123, 570 01 Thessaloniki (Pylea), GREECE | Postal address: PO Box 22427, 551 02 Thessaloniki, GREECE
Τel. +30 2310490111 | Fax +30 2310490049 | E-mail: [email protected] | www.cedefop.europa.eu
Direct line: +30 2310490.......... | Fax +30 2310490.......... | [email protected]
DRAFT European Handbook learning outcomes October 2016.docx page 1 of 52

CONTENTS
Contents.............................................................................................................. 2
1. THE PURPOSE OF THE HANDBOOK ........................................................ 4
1.1. The context of the handbook ..................................................................... 5
1.2. The structure of the handbook ................................................................... 6
PART I. PERSPECTIVES AND RULES OF THUMB INFORMING THE

WRITING AND DEFINITION OF LEARNING OUTCOMES ......................... 8
2. BASIC PERSPECTIVES AND RULES OF THUMB FOR THE DEFINITION
AND WRITING OF LEARNING OUTCOMES .............................................. 9
2.1. Basic perspectives ..................................................................................... 9
2.2. The writing of learning outcomes statements ........................................... 11
2.3. Using learning outcomes statements ....................................................... 15
PART II. LEARNING OUTCOMES – OPPORTUNITIES AND DILEMMAS .... 18

3. THE DEFINITION OF LEARNING OUTCOMES ........................................ 19
4. THE MAIN PURPOSES AND THE PERCEIVED ADDED VALUE OF
LEARNING UTCOMES ............................................................................. 22
4.1. Learning outcomes for different purposes ................................................ 22
4.2. The perceived added value of learning outcomes .................................... 23
5. QUESTIONING THE ADDED VALUE OF LEARNING OUTCOMES ......... 26
5.1. Learning outcomes as ‘dumbing down’ of education and training ............ 26
5.2. Addressing the imperfections of learning outcomes ................................. 27
5.3. Learning outcomes and governance ........................................................ 28
6. WRITING LEARNING OUTCOMES – HOW TO CAPTURE PROGRESSION
IN AND COMPLEXITY OF LEARNING?.................................................... 29
6.1. Learning outcomes and learning progression – The dominant position of
Bloom’s taxonomy ................................................................................... 29
6.2. Alternative taxonomies and their potential impact on the definition and
writing of learni8ng outcomes .................................................................. 31
6.3. Capturing the vertical dimension of learning outcomes – key questions... 33
7. A COMMON EUROPEAN FORMAT FOR PRESENTING LEARNING
OUTCOMES BASED QUALIFICATIONS .................................................. 35
7.1. Principles to be applied............................................................................ 35
7.2. Technical and conceptual requirements to the format .............................. 36
PART III. RESOURCES AND SOURCES SUPPORTING THE

DEFINITION, WRITING AND USE OF LEARNING OUTCOMES ......... 38
Overview of the guidance material for writing LOs ............................................. 39

Overview over research material supporting the writing, definition and use of
learning outcomes ..................................................................................... 47

1. THE PURPOSE OF THE HANDBOOK
Learning outcomes state what a learner is expected to know, be able to do

and understand at the end of a learning process or sequence. The way
these learning outcomes are defined and written is of key importance to
the orientation of education and training and – eventually – to the
relevance of the resulting qualifications and programmes to individual
learners, the labour market and society in general.
This handbook is written for individuals and institutions actively involved in

the definition and writing of learning outcomes in education and training in
general and in vocational training on particular. The handbook is launched
in a situation where there is broad political agreement among European
policy makers and stakeholders on the need for and usefulness of the
learning outcomes approaches. This strong political commitment,
however, does not guarantee that learning outcomes are written and
applied in ways which benefit end-users, be these learners, teachers,
parents or employers. More than before is it important to identify not only
the added value of learning outcomes but also point to limitations and
possible negative implications.
Cedefop recognises that a significant amount of guidance material have

already been developed in this area (1), offering advice on how to write
and use learning outcomes for different purposes. We also observe that a
considerable amount of research, over a long period of time and in a wide
range of disciplines, has been carried out. An important purpose of this
handbook is to therefore present this material in a more systematic
manner. The handbook will serve two main purposes:
 It will act as an ‘entry point’ to existing guidance and research

material on the definition and writing of learning outcomes.
 It will serve as a tool for better understanding the possibilities as
well as the dilemmas and challenges faced when defining, writing
and using learning outcomes.
The handbook should help to identify the choices faced by practitioners in

this area, subsequently pointing to existing alternatives. For Cedefop it is
of particular importance to clarify what learning outcomes means for
vocational education and training. Much of the guidance material and
research focus on the role of learning outcomes in higher, academically
oriented education. One purpose of this handbook is to identify the
1
See part III of this handbook.

particular challenges and opportunities facing the definition, writing and
implementation of learning outcomes in vocational education and training.
1.1. The context of the handbook
Recent Cedefop studies (2009 and 2016) document that learning

outcomes based approaches are becoming increasingly influential in
European education and training policies and practices (most visibly
expressed by the rapid introduction of learning outcomes based
qualifications frameworks since 2005). While explicitly introduced in
European policy documents as late as 2003, national learning outcomes
based initiatives date further back, exemplified by reforms in the UK and
Finland in the 1980s and 1990s. Most European countries now use
learning outcomes to express what they expect a student or pupil to know
and be able to do and understand at the end of a programme or learning
sequence. Learning outcomes are commonly used to set qualification
standards, to describe programmes and courses, to orient curricula and
set assessment standards. Learning outcomes are also – to some extent -
influencing teaching methods, learning environments and assessment
practises. At European level, both in the Bologna and Copenhagen
processes, learning outcomes are viewed as the ‘glue’ binding diverse
policy initiatives and instruments together. This growing influence of
learning outcomes in a majority of European countries and in (almost) all
education and training sectors reflects a strong political consensus on the
conceived usefulness of this approach. Far from developing in a vacuum,
European developments are directly influenced by broader international
developments, exemplified by the ‘outcome based education’ movement in
the US from the 1960s and onwards. Lines can also be drawn back to a
wide diversity of educational research and reform movements, some of
them dating back to the early 20 th century, promoting outcome based
approaches under (diverse) headings like ‘scientific curricula’,
‘instructional objectives’, ‘criterion-based assessment’ and ‘learner-centred
education’. As will become clear throughout the handbook, the writing and
articulation of learning outcomes must be followed by implementation –
through teaching, learning and assessment. As Biggs and Tang (2007)
underlines, an alignment between statements of learning outcomes, the
teaching/learning activity and assessment is critically important and
decides whether learning outcomes add value or not. Our ability to move
from statements of intended learning outcomes to actually achieved
outcomes depends on this alignment and high quality teaching and
learning.

1.2. The structure of the handbook
The first part of the handbook outlines a limited number of basic steps,
‘rules of thumb’, to be taken into consideration when defining and writing
learning outcomes. This part is practically oriented and summarises
concrete experiences in this field. The following issues and questions are
addressed:
 Which basic perspectives underpin the writing of learning outcomes

and which are the choices and challenges arising for these?
 Which are the rules of thumb supporting the practical writing of
learning outcomes?
 Which are the rules of thumb supporting the use of learning
outcomes, notably for assessment?
The second part of the handbook provides a more in-depth discussion on

the issues addressed above and seeks to identify the dilemmas and
choices faced by practitioners in this area. This part seeks to integrate
findings from research and is divided into a number of sub-chapters,
addressing the following issues:
 How is the learning outcome concept defined and what is the

relation to concepts like competence, learning aims and learning
objectives (draft included)?
 Which are the main purposes of learning outcomes and what added
value is perceived (draft included)?
 Which are the perceived negative implications of using learning
outcomes and why is it by some considered it to be a threat (draft
included)?
 Which are the key terminological issues to be addressed when
writing learning outcomes; how to use action verbs how capture the
context and increasing complexity of the learning process (draft
included).
 How can learning outcomes be written so as to best align with
teaching, learning and assessment practices to learning outcomes
(draft yet to be developed).
 How can qualifications frameworks support the definition, writing
and reviewing of learning outcomes. This is linked to the
collaborative character of writing learning outcomes and how the
dialogue between education and training and labour market is
critical to the overall quality and relevance of the learning outcomes
(draft yet to be developed).

 Could a common European format for presenting learning
outcomes provide a practical step towards increased transparency
of qualifications and increased cooperation between countries (draft
included)?
The third part of the handbook contains an extensive overview and list of
existing guidance material and research in this area. As far as possible
direct web-links are provided, making it possible to access this material
when needed.
All parts will be further developed in close dialogue with practitioners and
researchers in the coming period. As far as possible the handbook should
become a living document subject to continuous updates and
improvements.

PART I. PERSPECTIVES AND RULES OF THUMB INFORMING
THE WRITING AND DEFINITION OF LEARNING OUTCOMES
The following ‘rules of thumb’ have been extracted from guidance material
currently in use across Europe, at national level, in institutions and as part
of projects. Covering various aspects of the learning outcomes ‘process,
from the actual writing to their use, these rules of thumb provide an
important reference point for stakeholders and practitioners in this area.

2. BASIC PERSPECTIVES AND RULES OF THUMB FOR THE
DEFINITION AND WRITING OF LEARNING OUTCOMES
2.1. Basic perspectives
 Using learning outcomes represents a perspective and a mode of

thinking (Kennedy et.al. 2006). Learning outcomes statements as
found in programme and qualification documents only represents the
visible surface of this perspective and thinking; it is wrong to assume
that all aspects and facets of learning can be capture through learning
outcomes. Learning outcomes cannot stand alone; their potential can
only be released when interacting with teaching, learning and
assessment.
 We distinguish between intentional and actually achieved learning

outcomes. The former are statements and expressions of intentions,
the latter can only be identified following the learning process, through
(criterion referenced) assessments and demonstration of achieved
learning in real life, for example at work.
 Consistent application of learning outcomes requires continuous

dialogue between intended and actual outcomes, seeking to improve
stated expectations (intended learning outcomes) based on the
actually achieved outcomes.
 Writing learning outcomes is not a purely technical and neutral activity

but requires a reflection on the political and practical implications of the
activity. Learning outcomes (and competence) based approaches have
different origins and have been influenced by different schools of
thought.
 It is possible to observe a main tension between what can be

described as behaviouristic and constructivist schools of thought. While
the behaviouristic tradition will emphasise learning outcomes as result
oriented, full-ended, clearly observable and (objectively) measurable,
the constructivist approach will emphasise the need for learning
outcomes to be process-oriented and open-ended, somewhat limiting
measurability. This tension is illustrated by the following figure
developed by Prøytz (2014)

 Learning outcomes are written for different purposes. The learning
outcomes descriptors used by a national qualifications framework
differs significantly from those used in a qualification profile/standard, a
programme description, a curriculum or in an assessment standard.
Learning outcomes have to be ‘fit for purpose’ and the level of
detail/granularity and generality/specificity will have to reflect this.
 Not all learning can be defined in learning outcomes. The learning

process can rarely be fully predicted and described; it has intended as
well as unintended, desirable as well as undesirable outcomes
(Hussey and Smith, 2003). Researchers ask how to escape ‘what we
know now’ and how to remain open to what has yet to be experienced
and articulated.
 Learning outcomes need to defined and written in a way which allows

for local adaptation and interpretation. Too detailed and prescriptive
statements can undermine and lead to a ‘dumbing down’ of the
learning and assessment process. There is a need to balance
autonomy and regulation.
 We need to be careful about treating ‘knowledge’ as information that

can be divided into bits and selected and combined at will. This can
ignore the extent to which knowledge is organised in bodies of
hierarchical conceptual relationships and neglect the conditions in
which such knowledge is acquired (Allais, 2012, 2014).
 When writing learning outcomes, learners and students are put at the
centre. A key question is whether learners and students see outcome-
descriptions as meaningful and can relate to them. The role of the
student and learner, however, is limited in the sense that ‘only those

who have already obtained a specific qualification can fully understand
this qualification‘. (Young 2011, p.97).
 When writing learning outcomes for vocational education and training it

should be kept in mind that most VET profiles/standards will balance
and combine general knowledge subjects (languages, math, history
etc.) with generic and transversal skills and occupation specific
knowledge, skills and competences. This balance and combination is
of critical importance to the profile and relevance of a programme or
qualification to individuals, the labour market and society in general.
 Defining and writing learning outcomes in VET and the balancing of

general, transversal and occupational specific aspects, will in most
cases be the outcome of a collaboration process and involvement of
different stakeholders – state, employers, trade unions, teachers. This
is reflected in the balance between occupation specific, transversal and
broader competences.
 Defining and writing learning outcomes is a continuous process

requiring systematic updating and reviews. In vocational education and
training it is of particular importance to ensure systematic feed-back
between stakeholders in education and training on the one hand and in
the labour market on the other hand.
 Learning outcomes statements should not be written in isolation, but be

articulated (calibrated) with a reference to the national qualifications
frameworks and to ‘related or neighbouring’ qualifications and
programmes.
 Learning outcomes never operate in isolation but have to be defined

and written within a broader context where learning inputs are taken
into consideration. The balance between learning outcomes and other
aspects depends on the context and purpose of a qualification.
2.2. The writing of learning outcomes statements
 Simplicity is important when writing learning outcomes; avoid too much

detail and seek a balance between specificity and generality.
 In general there should not be more than one action verb for each
learning outcome.

 When writing learning outcomes, focus on the learner and start with an
action verb, continue with the object of the verb and complete with an
indication of the context (which can be related to learning and/or work).
The tables below illustrate this
Table 1. The basic structure of learning outcomes statements
The basic structure of learning outcomes statements

It should It should use an It should indicate It should clarify the
address the action verb to the object and occupational and/or
learner signal the level scope of social context in
of learning learning, notably which the
expected by identifying qualification is
relevant learning relevant
domains
Table 2. Examples of applying the basic structure of learning

outcomes statements
 Learning outcomes statement - combining action

verb/object/context - need to be articulated along a vertical and a
horizontal dimension.
 Introducing the vertical dimension of learning outcomes statements

is about indicating the level and complexity of learning. This will
normally require referring to a (implicit or explicit) hierarchy of
intended learning outcomes and achievements. Action verbs will
play a key role in defining and articulating this level but need to be
supported in this by specifying the occupational and/or social
context in which the learning takes place and where the outcomes
are to be used.
 The introduction of learning outcomes based qualifications

frameworks support this levelling and can be used as a ‘yard-stick’
to ensure consistency across institutions and programmes.
 Action or activity verbs play a particularly important role in writing

learning outcomes statements. Most guidelines on writing learning
outcomes makes an explicit reference to the taxonomies developed

by Bloom and colleagues from 1956 and onwards. In most cases
reference is made to action verbs associated with the cognitive
dimension of learning, only exceptionally to the psycho-motor and
affective domains, also forming part of this approach. From the
perspective of vocational education and training, all three domains
are of relevance. An important question is whether one taxonomy
can be used to describe learning outcomes in all three domains or
different taxonomies can better capture progression in knowledge,
skills and competences. Figure 1 provides an overview over these
three domains:
Figure 1. Bloom’s taxonomy: cognitive, psycho-motor and affective

domains
Internalises
Naturalisation
values
Articulation Organisation
Precision Valuing
Responds to
Manipulation
phenomena
Receiving
Imitation
phenomena
Affective domain,
Psycho-motor domain as
Krathwohl, Bloom, Masia
developed by Dave (1970)
(1973)
Sources: Bloom et al. (1956); Dave (1970); Anderson and Krathwohl

(2001).
 As far as possible avoid ambiguous verbs. The following verbs are

commonly presented as ambiguous and precise:
Ambiguous Unambiguous
Know Distinguish between
Understand Assemble
Determine Adjust
Appreciate Identify
Grasp Solve
Become familiar with Write
Enjoy Recite
Believe Construct
Compare

 What counts as an ambiguous verb differs according to school of
thought. A ‘constructivist’ approach to the definition and writing of
learning outcomes (see for example Biggs and Tang 2007) will
emphasise the need for learning outcomes to be process-oriented and
open-ended as opposed to being objectively measurable and
observable.
 A good way to distinguish between the different categories of verbs is

to reflect on the difference between declarative knowledge (knowing
WHAT) and procedural knowledge (knowing HOW). Declarative
knowledge is about recalling and representing theories and facts.
Procedural knowledge requires that the facts and theories referred to
above is turned into use in increasingly complex occupational and
social settings. Verbs must be able to indicate the relational character
of knowledge and skills, pointing to the growing complexity of the
context in which the learner have to operate.
Declarative verbs Procedural/

relational verbs
Repeat Reflect
Describe Hypothesize
Identify Solve unseen
Memorise problems
Recall Generate new
alternatives
 Introducing the horizontal dimension of learning outcomes statements

is about clarifying the object and the scope of the intended learning,
notably by specifying the learning domains being addressed. Are we
for example focussing mainly on theoretical knowledge or are we
addressing practical or analytical skills? Action verbs play a role
when describing the horizontal dimension but needs to be supported
by a clarification of the object of learning, notably the learning
domains to be addressed.
 The object and the scope of the learning outcomes statement can
be described in different ways. Some countries and institutions
have chosen to use pre-defined domains when writing learning
outcomes (for example knowledge, skills and competence, as
commonly used by qualifications frameworks), others avoid pre-
defined domains (Cedefop 2016, forthcoming) and insist that the

description should be ‘holistic’ and evolve from the task at hand and
not be steered by pre-defined categories.
 When writing learning outcomes for a unit, programme or qualification,

take into account the time available and effort required. Learning
outcomes statements can easily be overloaded and lose their
relevance as tools supporting the learning process.
 Given that learning outcomes provide a reference point for the

recognition and validation of non-formal and informal learning, focus
should be on the efforts required, not on a particular time required.
The term ‘notional time’ as used for the ECTS, can be helpful here.
2.3. Using learning outcomes statements
 The definitions and descriptions of learning outcomes as used in

qualifications frameworks, qualification standards and curricula are
statements and expressions of intentions. Achieved learning outcomes
can only be identified following the learning process, through
assessments and demonstration of achieved learning in real life, for
example at work.
 Consistent application of learning outcomes requires continuous

dialogue between intended and actual outcomes, seeking to improve
stated expectations (intended learning outcomes) based on the
actually achieved outcomes.
 Involve all relevant stakeholders in the development and review of

learning outcomes, teachers and trainers as well as external
stakeholders, notably social partners. Learning outcomes needs to be
a ‘living thing’ and continuously reviewed and improved.
 The application of learning outcomes is a question of aligning learning

outcomes statements with teaching and learning. Learning outcomes
statements should assist teachers in identifying and combining
teaching methods.
 The implementation of learning outcomes depends on a clear link

being established between the learning outcomes statements and the
learning and teaching process. This requires that learning outcomes
statements for different purposes (qualifications standards, programme

profiles and curricula) are related to each other and not operates as
isolated and seprated elements.
 The application of learning outcomes is a question of aligning learning

outcomes statements with assessment As Ramsden (1992) puts it
‘…the assessment is the curriculum…, as far as the students are
concerned. They will learn what they think they will be assessed on,
not necessarily what the learning outcomes in the programme or
curriculum states. The trick, according to Biggs (2003), is to make sure
the assessment tasks mirror the learning outcomes.
 Learning outcomes must promote reflection on assessment and

development of assessment criteria and more effective and varied
assessment. Much depends on how they are constructed and whether
they introduce rigid and closed criteria or whether they open up for a
wider interpretation of the diverse learning experiences of individuals.
 The link between learning outcomes statements and assessment

points to a tension between reliability and validity. While reliability,
assuring that the same assessment outcomes are achieved
independently of the time and location of the assessment, validity
requires that the assessment is able to capture the essence of the
learning experiences of the individual and compare these to the
assessment criteria. Too rigid and narrow assessment criteria can
‘dumb down’ the assessment process in a way which do not respect
the experiences of the learner.
 It is commonly stated that learning outcomes needs to be written as

threshold statements - as minimum requirements to be met by the
learner.
 It should be made clear that learning outcomes, written as threshold

statements, do not prevent learners to go beyond these thresholds.
Learning outcomes orient a learning process; they should not contain
or restrict it. Some of the criticism towards learning outcomes seems to
be linked to this point, implying that the process of stating outcomes
dumbs down the learning process and prevents learners to go beyond
these minimum expectations.
 The link between learning outcomes and quality assurance is of key

importance; the following should be noted:

o Do the learning outcomes reflect and balance the interests and
requirements of both the internal staff (pedagogy) and external
stakeholders (labour market and society requirements)
o Is there a systematic feed-back between education and training and
labour market/society stakeholders organised; is this dialogue
organised regularly; what kind of information is exchanged; can we
observe ‘breaks’ in this loop.
o Are learning outcomes reviewed from a perspective of ‘control’ or a
perspective of development?

PART II. LEARNING OUTCOMES – OPPORTUNITIES AND
DILEMMAS
This part of the handbook provides a more in-depth discussion of

particular aspect related to the definition, writing and use of learning
outcomes. An important purpose of this part is to identify existing
opportunities and dilemmas, building on existing research in this area.

3. THE DEFINITION OF LEARNING OUTCOMES
Throughout Europe, the term ‘learning outcomes’ is increasingly

embedded in the vocabulary of education and training policies (Prøitz,
2014). Cedefop (Cedefop, 2014) provides two interrelated definitions of
this concept:
 learning outcomes are defined as statements of what a learner knows,

understands and is able to do on completion of a learning process,
which are defined in terms of knowledge, skills and competence;
 learning outcomes are defined as ‘sets of knowledge, skills and/or

competences an individual has acquired and/or is able to demonstrate
after completion of a learning process, either formal, non-formal or
informal.
These two definitions point to the relationship between intended and

actually achieved learning outcomes. As we will see in later chapters, this
relation can be expressed as a loop where the interaction between what is
intended and what has actually been achieved feeds into a continuous
improvement process.
Figure 2. Relationship between intended and achieved learning

outcomes
Intended Achieved
learning learning
outcomes outcomes
Source: Cedefop.
The definitions and descriptions of learning outcomes as used in

qualifications frameworks, qualification standards and curricula are
statements and expressions of intentions. They are not outcomes of
learning, but desired targets. Achieved learning outcomes can only be
identified following the learning process, through assessments and

demonstration of achieved learning in real life, for example at work.
Consistent application of learning outcomes requires continuous dialogue
between intended and actual outcomes, seeking to improve stated
expectations (intended learning outcomes) based on the actually achieved
outcomes. Dialogue between the world of education and work is crucial to
successful implementation – and continuous review and renewal - of the
learning outcomes approach. The focus on actually achieved learning
outcomes brings us to the concept of competence, which is defined by
Cedefop (2014) as follows:
 Competence is defined as the ability to apply learning outcomes

adequately in a defined context (education, training, work or
professional development).
Competence can thus be understood as actually achieved learning

outcomes, validated through the ability of the learner autonomously to
apply knowledge and skills in practice, in society and at work. Learning
outcomes are validated by their relationship to competences (Cedefop,
2012, p. 35). While the term competence is widely used throughout
Europe, and in several countries substitutes the term learning outcomes,
many different definitions and interpretations exist, creating some
confusion when operating at international level. The definition provided by
the 2008 Recommendation on the EQF can be seen as a compromise
pointing towards a shared approach:
 Competence means the proven ability to use knowledge, skills and

personal, social and/or methodological abilities, in work or study
situations and in professional and personal development.
Related to the above concepts are terms like learning aims and learning
objectives2. These are sometimes used interchangeably, potentially
(2) Allen (1997) offers a detailed insight into the evolution of these concepts. The term learning
th
objective date back to the first half of the 20 century (Bobbit 1918, Tyler 1949) and is clearly
oriented towards clarification of teachers’ intentions. The term learning outcomes is introduced from
the 1970s and onwards, signalling a more learner centred approach. The distinction between
objectives and outcomes can also be captured through the distinction between ‘product’ and
‘process’ models for curriculum development. Tyler (1949, op.cit) presents one of the first rational
curriculum design models, also known as ’means-end’ or ‘product’ model. To some extent
influenced by behaviourism (see also chapter 4), the focus is very much on the definition of precise
and observable results of teaching. Stenhouse (1975) questions whether curriculum and pedagogy
could be oriented by logic other than the means-end model. He saw the model as not beginning

creating confusion. Kennedy et.al. (2006, p.5) understand learning aims
and learning objectives in the following way:
 ‘(A learning aim) is a broad statement of teaching intention, i.e. it

indicates what the teacher intends to cover in a block of learning.
Aims are usually written from the teachers’ point of view to indicate
the general content and direction of a programme…’
 ‘(A learning objective) is a usually a specific statement of teaching
intention, i.e. it indicates one of the specific areas that the teacher
intends to cover in a block of learning’.
with behavioural objectives but by focussing on the learner, the learning process and the conditions
of instruction and learning to be created.

4. THE MAIN PURPOSES AND THE PERCEIVED ADDED VALUE OF
LEARNING UTCOMES
4.1. Learning outcomes for different purposes
Learning outcomes are used for a wide range of purposes and at many
levels. There is thus not one single way of defining and writing learning
outcomes; the approach has to reflect the particular purpose and context
in question. As documented by Cedefop (2016) the following purposes are
common across Europe:
 Qualifications frameworks play an increasingly important role at

international, national and sector level. Their learning outcomes based
levels provide a an important reference point for qualifications,
enhancing transparency, allowing for comparison and provide a
reference point for review and reform of qualifications. The learning
outcomes descriptions of qualifications frameworks are described
according to the horizontal and vertical dimensions described above,
but normally at a very general and level. This latter is necessary in
order to be relevant to the big diversity of individual qualifications
forming part of national or international qualifications systems.
 Qualification profiles or standards may define the expected outcomes

of the learning process, leading to the award of a full qualification In
VET a qualification standard or profile normally answers the question
‘What does the student need to learn to be effective in employment?
These standards or profiles will normally be articulated at national
level, seeking to address the different purposes of initial education and
training, notably by balancing between general knowledge,
occupational specific and transversal skills and competences. In some
countries these qualification profiles and standards will be divided into
modules or units with separate and specific learning outcomes
statements. In many cases this will mean that the level of detail and
specificity of the learning outcoms will increase,
 Occupational profiles or standards are normally set outside the

education and training system but can have a significant impact on the
way learning outcomes statements are defined and written.
Occupational profiles or standards specify ‘the main jobs that people
do’, describing the professional tasks and activities as well as the
competences typical of an occupation. Occupational standards answer

the question ‘What does the student need to be able to do in
employment?’
 Curricula set the framework for planning learning experiences.

Depending on the country, the type of education and training, and the
institution, learning outcomes statements form an important part;
normally combined a specification of teaching methods
 Assessment criteria and standards may specify the general orientation

of the assessment, assessment methods and – notably – the criteria
for the assessment itself, normally formulated as concrete threshold
levels which have to be met by the candidate. Assessment standards
answer the question: How will we know what the student has learned
and is able to do in employment? Assessment criteria and standards
will be more detailed than qualifications profiles/standards and
curricula in the sense that they have to precisely describe the
requirements to the learner. Assessment standards play a critical role
in deciding the orientation of the learning outcomes approach; a too
detailed and rigid approach can limit the learning process.
A main difference between these different purposes is the level of detail

applied in the learning outcomes description. While qualifications
frameworks provide a general reference for comparing qualifications and
defining levels, qualifications standards, curricula and – not least –
assessment standards have to be directly relevant to the learner and the
concrete learning process he or she is going through.
4.2. The perceived added value of learning outcomes
Learning outcomes are perceived as adding value for these different

purposes and for a number of reasons. While these benefits cannot be
taken for granted, they eventually depend on the way the approach is
interpreted and adopted, many see the added value of the learning
outcomes approach as significant:
 For the learner: Learning outcomes statements clarify what a learner is

expected to know and be able to do and understand when having
complete learning sequence, a module, a programme or a qualification.
They support his or her initial choice of education, training and/or
learning paths; they help to orient the learning process itself; and they
clarify what to expect during assessment. For learning outcomes
statements to make any difference to learners, they must be present

and visible not only in (written) qualification standards and programme
descriptions. Their visibility in practise, throughout the teaching and
learning process as well in assessment arrangements is of critical
importance and decides whether they will make a difference to the
learner.
 For the teacher/instructor: The learning outcomes approach helps to

orient teaching, to select methods and to facilitate the learning process.
Learning outcomes, through their focus on levels of and requirements
to learning, are crucial for promoting a more systematic reflection on
assessment criteria and assessment methods and how these interact
with and support the learning process.
 For the education and training institution: Learning outcomes provides

an important instrument for planning, for internal as well as external
dialogue. The perspective helps to determine the purpose and
orientation of a course, a programme or qualification and to clarify how
it relates to and/or overlaps with other courses/programs and
qualifications. Learning outcomes can provide an important reference
point for quality assurance, the relationship between intended and
actual learning outcomes (as expressed through assessments)
provides important input to the continuous review and development
which is expected from education and training institutions.
 For society and labour market: Learning outcomes offers a meeting

place for different stakeholders and education and training users. If
used systematically, this allows for a systematic review of the quality
and relevance of education and training, focussing on the relationship
between intended and actually achieved learning outcomes. The
definition of learning outcomes requires a systematic reflection on the
use of labour market intelligence and how this will be balanced with the
needs of the education and training system and teachers to facilitate
and support education, training and learning.
In recent years learning outcomes are increasingly seen as a way to

increase overall transparency of qualifications and as a way to improve
comparability of qualifications across national borders. The development
of learning outcomes based qualifications frameworks play a key role in
this respect, seeing learning outcomes as the common language allowing
for transparency and comparison. This ‘governance perspective’ is by
some also seen as a way to strengthen the accountability of education and

training institutions; making it possible for external stakeholders to judge
what is delivered by these institutions. This illustrates that learning
outcomes are not politically neutral, but can indeed be used for different
purposes.

5. QUESTIONING THE ADDED VALUE OF LEARNING OUTCOMES
Not everybody agrees in the added value of learning outcomes. A number

of researchers have criticised the conceptual basis of the approach and
questioned its practical and political implications. An important purpose of
the handbook is to present the main elements of this criticism and
consider whether and how it can influence future practises. We can
distinguish two main lines of criticism towards learning outcomes; a
conceptual/ideological and a technical/practical. While the first line of
criticism tends to argue against the approach as a whole, the second is
more pragmatic and points to weaknesses in its current understanding and
application. While not pretending to give a full overview over research in
this area, the chapter seeks to identify the most important issues currently
addressed and debated.
5.1. Learning outcomes as ‘dumbing down’ of education and training
Allais (2016) stands out as a vocal representative of the

conceptual/ideological criticism3. Based mainly on experiences from South
Africa and the UK, she questions the added value of learning outcomes,
arguing that it can potentially undermine the development of high quality
education and learning. Focussing mainly on the (perceived failed) role of
qualifications frameworks in these countries, her contributions connect to a
research tradition arguing that the shift to learning outcomes can inhibit
and restrict the learning process and ‘dumb down’ teaching and
assessment. The learning process, which is always context-bound, can be
harmed by introducing too concrete and specific outcome statements. The
focus on observable and assessable outcomes, it is argued, links back to
a behaviouristic tradition seeking to reduce complex (personal and social)
learning processes into measurable and delimited objects. This tradition
according to Allais, assume the learner to be passive and (exclusively)
responding to environmental stimuli (Schuman and Ritchie, 1996).
Focusing on the ‘conditioning’ and ‘reinforcement’ of individuals, attention
is given to the external change in behaviour. Critics point out that this
perspective has profound implications as it requires outcomes to be
3
O’Brien and Brancaleone (20??, p. 8) discusses the epistemological and pedagogical
validity of learning outcomes approach – pointing to the gap between conceptual origins
and intended action.

described in specified, unambiguous, quantifiable, full-ended and
measurable terms. This approach is seen as fundamentally clashing with
liberal traditions, notably in higher education, emphasising the open
character of learning (as a condition for research and innovation). The
criticism of learning outcomes has grown stronger as the approach has
gained more political support across Europe. By some seen as the
embodiment of a neo-liberalist, market-driven philosophy, the shift to
learning outcome, as exemplified by the rapid growth in national
qualifications frameworks, is seen as ‘policy hype’ and as a threat to high
quality education and training.
Not questioning the overall relevance and usefulness of learning

outcomes, a number of researchers (Allen 1997, Biggs 1999) have raised
questions regarding particular aspects of the approach. While addressing
some of the same issues as listed above, these contributions seek to
identify ways in which to improve existing practises:
5.2. Addressing the imperfections of learning outcomes
It is acknowledged that learning outcomes, for example when indicating

precise threshold level, actually can inhibit and restrict the learning
process. Too much specificity and detail, it is argued, makes it difficult to
give room for innovation and exploit the unexpected present in any
situation. Researchers influenced by constructivism (see for example
Hoskins and Deakin Crick 2010, p. 122) have made an effort to establish
an alternative based on an understanding of learning as deeply
contextualised and not to be separated from social identity, values and
relationships. It puts the learner at the centre of the learning process, as
an active constructor of knowledge and not just a passive receiver, who
not only ‘assimilates’ but also ‘accommodates’ knowledge, skills and
competences based on previous experiences, mental structures and
beliefs. According to this school of thought, knowledge, skills and
competences cannot be treated as isolated or de-contextualised entities
and/or subjects, but need to be addressed in the context where they are
situated (Lave and Wenger, 1991). The implication of this for defining and
applying the learning outcomes approach is that learning outcomes
statements are descriptive (not prescriptive), holistic and defined from a
perspective of an individual and his/her abilities (Cedefop, 2010; Anthony,
1996). They are process and context-oriented and need to avoid a too
rigid definition of outcomes. This open-ended approach respects individual
diversity and the inherent richness of learning processes, but risks
reduced measurability (Prøitz, 2014).

An important strand of research (Biggs 1999 op.cit., Biggs and Tang,
2007) stresses the importance of aligning learning outcomes statements to
teaching and learning practises as well as to assessment tasks. The
potential impact of the learning outcomes approach depends on this
alignment, or as Biggs and Tang expresses it (Biggs and Tang 2007,
p.52); ‘The alignment in constructive alignment reflects the fact that the
learning activity in the intended outcomes, expressed as a verb, needs to
be activated in the teaching/learning process if the outcomes is to be
achieved and in the assessment task to verify that the outcome has been
achieved’. This approach requires that learning outcomes are treated as
open-ended: ‘Unlike some outcomes based education, constructively
aligned teaching is not a closed loop, focussing only on what is pre-
determined. As explained later, we use outcome statements and open-
ended assessments tasks that allow for unintended but desirable
outcomes’ (Biggs and Tang, op.cit. p.53).
5.3. Learning outcomes and governance
Reflecting the rapid expansion of learning outcomes based qualifications

frameworks in Europe and beyond, a number of authors have questioned
the relationship between learning outcomes and the governance of
education and training (Bohlinger 2012, Lassnigg 2012, Hussey and
Smith, 2003). While acknowledging that learning outcomes can have
important pedagogical results, their impact on policy-making and
governance is seen as over-stated. Lassnigg (op.cit., p. 303) argues that
the majority of research has focused on pedagogy and the minor
proportion of the literature focuses on a policy and governance level; and
that in European policy, the emphasis on policies linked to the
implementation of the EQF - has driven attention on learning outcomes
towards the policy level.
Not pretending to fully reflect the criticism towards learning outcomes as

expressed by research, the above offer an insight into challenges faced by
those responsible for defining, writing and applying learning outcomes.
Much research is currently focussing on the critical relationship between
intended and actually achieved learning outcomes: The key question is
how the articulation of learning intentions influence the teaching, learning
and assessment processes and whether this influence is positive or
negative?

6. WRITING LEARNING OUTCOMES – HOW TO CAPTURE
PROGRESSION IN AND COMPLEXITY OF LEARNING?
Learning outcomes are best understood as a as an approach that can be

adapted to and applied in different policy, teaching and learning settings. It
follows that there is no single correct or apt way of approaching them. The
term can have a range of connotations and denotations, precisely because
it is used in different contexts (Cedefop, 2009). However, the conceptual
basis for the definition of learning outcomes can directly influence the
character and quality of the learning process as experienced by the
individual learner.
6.1. Learning outcomes and learning progression – The dominant position

of Bloom’s taxonomy
Bloom’s taxonomy is one of the most important theoretical influences on

thinking about learning outcomes and progression. The earliest iteration of
the taxonomy (Bloom et al., 1956) sets out a hierarchical categorisation of
cognitive learning, moving from basic (knowledge and comprehension) to
increasingly complex skills (application, analysis, synthesis and evaluation
of concepts, processes, procedures, and principles). Anderson and
Krathwohl (2001) revised the cognitive domain of the taxonomy by
changing the nouns used in the original version to verb form (knowledge
was changed to remembering; comprehension to understanding) and
placing synthesis (creating) above evaluation (evaluating) in the highest
order of complexity. A second publication (Bloom et al., 1964) set out a
hierarchy of learning for the affective domain, starting with the basic
(receiving, responding) and moving to more complex levels (valuing,
organisation, characterisation by a value or value complex). A further
development introduced a hierarchy describing the psycho-motor domain
(skills), starting with imitation and moving via manipulation precision to
articulation and naturalisation. The three hierarchies are shown in Figure
3.

Figure 3. Bloom’s taxonomy: cognitive, psycho-motor and affective
domains
Internalises
Naturalisation
values
Articulation Organisation
Precision Valuing
Responds to
Manipulation
phenomena
Receiving
Imitation
phenomena
Affective domain,
Psycho-motor domain as
Krathwohl, Bloom, Masia
developed by Dave (1970)
(1973)
Sources: Bloom et al. (1956); Dave (1970); Anderson and Krathwohl

(2001).
Additionally, Anderson and Krathwohl (2001) have created a matrix

combining the elements of the hierarchy with levels of knowledge (Figure
4).
Figure 4. Bloom’s revised taxonomy: the cognitive process

dimension and dimensions of knowledge
The
knowledge Rememb Understa Appl Analys Evalua Creat
er nd y e te e
dimension
Factual
Conceptual
Procedural
Metacogniti
ve
Source: Anderson and Krathwohl (2001).
This approach has been subject to a variety of criticisms. Beretier and

Scardamalia (2005) argued that ‘… we need ways to think about
knowledge that allow us to be reasonably clear and definite about what we
are trying to achieve yet do not require reducing knowledge to itemisable
objects in the mind …’ (4). Depth and coherence of knowledge in the
development of expertise, they argue, requires ‘… getting beneath the
surface, making contact with the underlying patterns and principles that
give meaning and support intelligent action' (5). This mirrors the criticism of
(4 ) Ibid. pp. 12-13.

(5 ) Ibid. p. 10.

behaviourism discussed above and warns against a reductionist approach
narrowing down the scope of expected (and assessed) learning. The
inclusion of increasingly complex verbs in the three hierarchies argues
against this, and points to more open-ended statements allowing for
unintended outcomes.
6.2. Alternative taxonomies and their potential impact on the definition

and writing of learni8ng outcomes
Two alternative taxonomies, with deeper roots in constructivist theories,

have emerged in recent years. The first, the Dreyfus taxonomy, describes
learner progression from ‘novice to expert’ (Dreyfus, 1981; Dreyfus and
Dreyfus, 1986):
 novice learners have incomplete understanding and approach tasks

mechanistically. Novice learners need supervision;
 advanced beginners have a working understanding of concepts. They
tend to see actions as a series of steps. Advanced beginners can
complete simple tasks without supervision;
 competent learners are able to understand context. They may
complete work independently to an acceptable standard;
 proficient learners have deeper understanding and are able to see
actions holistically. They are consistently able to achieve a high
standard;
 expert learners have an authoritative, deep and holistic understanding.
They are able to deal with routine matters ‘intuitively’, to go beyond
existing interpretations. They consistently achieve excellence.
The SOLO taxonomy (Structure of Observed Learning Outcomes) (Biggs

and Collis, 1982; Biggs, 1999; 2014) similarly describes progressively
complex levels of understanding. Within the SOLO taxonomy,
understanding is described as an increase in the number and complexity
of connections learners make as they progress from low to high levels of
competence. Learning is shaped by prior knowledge, misconceptions,
learning intentions and strategies. The focus is on the depth and quality of
understanding, rather than the quantity of information.

Table 1. The structure of observed learning outcomes (SOLO)
Levels of understanding Phase of Indicative verbs
displayed learning
Extended abstract: conceptualises Qualitative
Theorise,
at level extending beyond what has phase
generalise,
been dealt with in the actual
hypothesise,
teaching and learning process. Can
reflect, generate
generalise to new areas.
Relational: indicates orchestration
Compare,
between facts and theory, action
contrast, explain
and purpose. Understanding of
causes,
several components which are all
integrate,
integrated conceptually. Can apply
analyse, relate,
the concept to familiar problems
apply
and work situations
Multi-structural: indicates
understanding of boundaries but Enumerate,
not of systems. Understanding of classify,
several but discreet components. describe, list,
Disorganised collection of ideas or combine, do
concepts around an issue. Not algorithms
relating items in list.
Uni-structural: concrete,
Identify,
minimalistic understanding of an
memorise, do
area, focuses on one conceptual
simple procedure
issue in a complex case
Pre-structural; no understanding Quantitative
Misses the point
demonstrated phase
Source: Adapted from Biggs (1999)
There is tendency, see for example Campbell (2014), to argue against

and oppose the shift to learning outcomes due to what is seen as a
(negatively perceived) behaviouristic bias. According to this criticism, the
learning outcomes approach risks reducing the richness of learning by
imposing a simplistic stimulus-response paradigm of learning where only
observable and measurable outcomes counts. This, according to the
critics, assumes a linear and overly simplistic learning process where
complex activity verbs (for example ‘understand’) should be avoided and
replaced by narrower, terms with clear borderlines. Allais (2012; 2014),
repeats this criticism with reference to the way knowledge is treated ‘…. as
information that can be divided into little bits that can be selected and
combined at will’ (Allais, 2014, p. 39). This, according to her ‘ignores the
extent to which knowledge is organised in bodies of hierarchical
conceptual relationships, the value of such knowledge’ (idem) does not

respect the conditions in which knowledge is acquired. Others (Dobbins,
2014), however, argue against the assumption that the shift to learning
outcomes by default implies reductionism. Learning outcomes can, to the
contrary, focus on a wide range of knowledge, skills and competences and
while some of these may be behavioural in character (for example, using a
particular tool for a particular purpose), others imply more complex and
ambiguous processes (for example, linked to the critical evaluation of
arguments supporting a policy decision) (Dobbins, 2014, p. 2). Biggs
(1999; 2014) pursues this point and states that in the design of learning
outcomes and assessment tasks teachers are free to use open-ended
verbs such as ‘design’, ‘create’, ‘hypothesise’, ‘reflect’ and so on and that
this is a way to avoid predetermined or rigid design of teaching and
assessment. A key question is how to define and apply learning outcomes
in ways which avoid the reductionism attributed to behaviourism. Some
guidelines on writing learning outcomes warn against broad terms like
‘understand’ and ‘appreciate’ and recommend to replace them with terms
like ‘describe’, ‘formulate’, and ‘identify’. Biggs argues against this advice,
stating that at an advanced level appropriate verbs for learning outcomes
would include ‘hypothesise’, ‘reflect’, and apply to unseen domains or
problems. These higher order learning outcomes require open-ended
tasks, allowing for emergent and unintended outcomes (Hussey and Smith
2008). Following this, it can be argued that complex verbs like
‘understanding’ will be at the core of most skills and activities; it forms part
of the definitions of learning outcomes cited above. Learning outcomes
can help learners to articulate what they will be doing about their
understanding, and how this reflects different levels of understanding.
6.3. Capturing the vertical dimension of learning outcomes – key

questions
When defining, writing and applying learning outcomes it is essential to be

aware of these dilemmas. Two key questions stand out:
 is it possible to write learning outcomes reflecting the constructivist

ambition of holistic and active learning outlined above?
 will we, by striving for clearly defined and observable outcomes of

learning, risk focus on isolated and entities of learning poorly reflecting
the diversity and richness of learning strived for by policy-makers?
The practical definition and writing of learning outcomes, and how we

respond to the dilemma outlined above, is influenced by a limited number

of taxonomies seeking to clarify how individuals progress in their learning.
Unpacking the way these taxonomies have been constructed and evolved
can help us to understand better the conditions for defining and writing
learning outcomes fit-for purpose.

7. A COMMON EUROPEAN FORMAT FOR PRESENTING LEARNING
OUTCOMES BASED QUALIFICATIONS
The EQF, and the related national qualifications frameworks, seek to

strengthen transparency of qualifications by introducing commonly
understood, learning outcomes based levels. For many, the information on
levels is important but insufficient. As countries present their qualifications
in very different ways, comparing the content and profile of qualifications is
difficult. The 2016 proposal by the European Commission on the revision
of the EQF therefore states the following:
‘(The Commission) in cooperation with Member States and stakeholders

(should develop) a standard format for the description of learning
outcomes to be used for comparison purposes.’
7.1. Principles to be applied
A common format for presenting learning outcomes can be seen as

deepening the approach to increased transparency of qualifications
started by the EQF in 2008, and is particularly relevant for future
developments of qualifications supplements and national databases on
qualifications. A common format could be developed according to the
following principles:
 It should not replace existing learning outcomes descriptions as used

at national or institutional level
 It should be used as a voluntary supplement
 It should allow national authorities to present the content and profile of
qualifications in their national databases in a concise yet comparable
way
 It should allow national authorities to present the content and profile of
qualifications in qualifications and certificate supplements
 It should allow private and international providers to present their
certificates and qualifications in concise and comparable way
Overall, the format can
 Increase transparency and comparability

 Further promote the shift to learning outcomes

7.2. Technical and conceptual requirements to the format
Initial discussions (in the EQF AG and the ESCO Board) points to the
following requirements:
 It must be short (+/- 1000 characters)

 It must use a pre-defined structure and syntax
 It must refer to agreed but flexible learning domains
 It must be supported by guidelines
 It must be supported by standardised lists of action verbs
The following basic structure could be used as a starting point for

developments:
Figure xx. A common format for presenting learning outcomes - The basic
structure
The learning outcomes description should be limited to +/- 1000

characters and be written talking the following elements into
consideration:
{learner}>{action verb}>{object and scope with reference to

learning domains}> {context} 6
6
1.1. The following example, from Ireland, illustrate how the basic format can be used to
present the profile of a national VET qualification in a transparent way:
1.2. The learner
• will be able to demonstrate understanding of a broad range of the main features of both a
CNC Milling Machine to include the main console controls, axis configurations, methods
of tool changing programming techniques, thread cutting and tool mounting;
• will be able to demonstrate a broad range of skills and tools in the use of a CNC Milling
Machine and CNC Lathe to include the use of part programming functions, the correct
starting and termination functions, using a tool library, producing parts, mounting and
positioning work pieces and applying programs.
• Will be able to demonstrate good practice in using the main consoles to input information,
correctly starting and terminating formats, employing risk reduction measures for the use
of both machines and setting up, monitoring and maintaining typical types of both
machines.
1.3. To access programmes leading to this award the learner should have reached the
standards of knowledge, skill and competence associated with the preceding level of the
National Framework of Qualifications.

PART III. RESOURCES AND SOURCES
SUPPORTING THE DEFINITION, WRITING
AND USE OF LEARNING OUTCOMES
This part of the handbook contains links to existing material on learning

outcomes, in the form of guidance material presented at national or
institutional level and in the form of research. This part of the handbook is
meant to develop and grow over time and provide a resource to be used
by policy makers, practitioners and researchers alike.

OVERVIEW OF THE GUIDANCE MATERIAL FOR WRITING LOS
Country No Institution Guidance Material-Titles Hyperlinks
Federal Educational standards in

Ministry of Vocational Education http://www.bildungsstandards.berufsbilden
Education, deschulen.at/fileadmin/content/bbs/Handb
Arts and uch_BIST_25.03.2013.pdf
Culture
Federal How to formulate Learning
Ministry for Outcomes. http://wissenschaft.bmwfw.gv.at/bmwfw/st
Science, udium/studieren-im-europaeischen-
Research and hochschulraum/bologna-prozess/bologna-
Economy worum-
gehts/curriculumsentwicklung/learning-
outcomes-und-der-bologna-prozess/wie-
formuliert-man-learning-outcomes/
Federal Booklet Educational

Ministry of Standards Social and http://bildungsstandards.berufsbildendesc
Education, Personal Competences 9.- hulen.at/fileadmin/content/bbs/AGBroschu
AT
Arts and 13th grade. Educational eren/SozialePersonaleKompetenzen_Bros
Culture Standards in Vocational chuere_Oktober2011.pdf
Education - Competence
model, descriptors and
selected methodological and
lesson samples
University of Guideline for Curricula http://www.tuwien.ac.at/fileadmin/t/rechtsa
Technology Development. bt/downloads/Leitfaden_zur_Curricula_Ers
Vienna tellung.pdf
Development Foundations and Materials for http://www.virtuelle-

group, 5 the Development of Curricula. ph.at/pluginfile.php/43796/coursecat/descr
pedagogical iption/Grundlagen_und_Materialien_zur_C
universities urriculumentwicklung_Druckversion.pdf
University of Compendium - Support for a http://senat.univie.ac.at/fileadmin/user_upl

Vienna modular curriculum oad/senat/Arbeitsbehelfe/Kompendium_20
13_v9_Jaenner.pdf

University of Assistance for the formulating http://www.boku.ac.at/fileadmin/data/H010
Natural of learning outcomes 00/H10220/studien-
Resources zukunft/Richtliniengruppen/Qualit%C3%A4
and Life tssicherung/Ergebnisse/Handreichung_Ler
Sciences nergebnisse.pdf
Ibw Austria - Guidelines for the description http://ibw4.m-

Research & of Learning Outcomes services.at/zoom/pdf/wp2/Leitfaden_DE_fi
Development nal_2.pdf
in VET
Federal Learning Outcome orientation http://erwachsenenbildung.at/downloads/s
Ministry of in adult education - guidelines ervice/Lernergebnisorientierung_Leitfaden
Education, for learning outcome oriented _web.pdf
Arts and curriculum development
Culture
Flemish inter- Manual Writing Domain http://www.nvao.net/page/downloads/Lear
university Specific learning Outcomes ning_outcomes_handleiding_VLIR-
Council Frameworks VLHORA.pdf
(VLIR) and
Flemisch UAS
Council
(VLHORA
Katholic Learning at the KULeuven: https://www.kuleuven.be/onderwijs/werken
University of vision and implementation _opl/leerresultaten/leerresultaten-aan-de-
Leuven ku-leuven
BE/FL
Dutch Flemish Domain-specific learning http://www.ond.vlaanderen.be/kwalificaties
Accreditation outcomes tructuur/kwalificatiedatabank/onderwijskwa
Organisation With respect to the Master of lificaties/master-in-de-ergotherapeutische-
occupational therapy studies wetenschap.pdf
(master) of Ghent University
Dutch Flemish Domain Specific Learning http://www.ond.vlaanderen.be/kwalificaties

Accreditation Outcomes Framework tructuur/kwalificatiedatabank/onderwijskwa
Organisation Education: Bachelor of lificaties/bachelor-in-de-verpleegkunde.pdf
Nursing (professional
Bachelor)
National Methodological Guidelines for
Agency for Development of Learning
Vocational Outcomes based VET Metodicheskii_ukaza
nia_DOI_28.04.2015.doc
Education and Standards
BG Training
http://www.navet.government.bg/bg/media/
Metodicheskii_ukazania_DOI_07_04_2015.pd
f

Agency for Methodology for developing http://www.asoo.hr/UserDocsImages/projekti
VET and Standards and curricula /kvalifikacije/eu%20knjige/3%20Metodologija.
Adult
pdf
Education
Agency for Learning outcomes example http://e-

VET and of a standard kvalifikacije.asoo.hr/pages/search/index.xhtm
CR
Adult
l
Education
Ministry of Guidelines for developing http://www.kvalifikacije.hr/br-

Science, standards bvrednovanje-skupova-ishoda-ucenja-
Education and radionica-za
Sport
National Support Guide for drafting,
Agency for implementing and evaluating http://www.aneca.es/eng/Press-
Quality learning outcomes [official service/News/2013/ANECA-launches-the-
Assessment translation]. Support-Guide-for-drafting-implementing-
and and-evaluating-learning-outcomes
Accreditation
Ministry of Basis for the development of https://sede.educacion.gob.es/publiventa/d
Education, the national catalogue of escarga.action?f_codigo_agc=16748&req
Culture and professional qualifications. uest_locale=en
ES Sport.
National
Institute of
Qualifications
Directorate- Learning outcomes No hyperlink (to be further checked)
General of
VET. Ministry
of Education,
Social Policy
and Sport.
University of Learning outcomes –
Applied terminology, relations, http://opus.bibliothek.fh-
Science application and identification aachen.de/files/195/schermutzki_bologna_
Aachen of success. Learning 6_a5_sw.pdf
outcomes and imparting of
competence as key elements
of the Bologna Process.
FI The Finnish Implementation of the
National vocational requirements as
Board of initial vocational education http://www.oph.fi/download/142550_Amm
Education. and training and competence- atillisten_perustutkintojen_perusteiden_toi
based qualifications meenpano.pdf.

Ministry of Writing guide of professional
agriculture diplomas standards http://www.chlorofil.fr/diplomes-et-
FR referentiels/formations-et-
diplomes/btsa.html
EOPPEP Methodology for the design

Dželalija Mile and development of Learning
GR Methodology for
Outcomes
learning Outcomes
NCP Hungary Guidelines prepared (by the http://www.oktatas.hu/pub_bin/dload/LLL/e

NCP) for VET kkr/Tanulasieredmenyek_VET.pdf
HU
NCP Hungary Guidelines prepared (by the http://www.oktatas.hu/pub_bin/dload/LLL/e
NCP) for HE kkr/Tanulasieredmenyek_HE.pdf
Dublin Guide to Writing Learning modularisation.dit.ie/docs/Learning%20Ou

Institute of Outcomes tcomes%20Guide.doc
Technology
University of Writing Learning

Limerick Outcomes - A guide to http://www.nfqnetwork.ie/_fileupload/Imag
academics e/Writing%20Learning%20Outcomes%20a
IE t%20Programme%20and%20Module%20
Levels.pdf
Kennedy D. et Writing and Using Learning http://www.tcd.ie/teaching-

al (2012) Outcomes: a Practical guide learning/academic-
development/assets/pdf/Kennedy_Writing
_and_Using_Learning_Outcomes.pdf
Copyright: Methodological guidelines for http://visc.gov.lv/profizglitiba/dokumenti/met

State the development of modular mat/metiet_modul_progr_izstr_2015.pdf
Education vocational education
Development programmes
Agency
LV
Responsible
for content:
National
Centre for
Education

National Methodological materials and http://visc.gov.lv/profizglitiba/programmas.sh
Centre for sample curricula (including tml
Education learning outcomes)
(responsible
for education
content)
subordinate to
the Ministry
Ministry of Amendments (2016/25) to http://visc.gov.lv/profizglitiba/dokumenti/pro

Education and ‘Internal regulation for grammas/izm_noteikumi_20160526_25.pdf
Science developing vocational
education programmes’
(2010)
MT
© Nuffic / A Tuning Guide to http://core-
TUNING Formulating Degree project.eu/documents/Tuning%20G%20Fo
Association Programme Profiles Including rmulating%20Degree%20PR4.pdf
Programme Competences
and Programme Learning
Outcomes
Foundation Instructions in the http://www.kwalificatiesmbo.nl/files/bestan
for vocational development of vocational den/Herziening/Instructies%20bij%20ontwi
education and qualification files, including kkeling%20kwalificatiedossiers%20mbo_v
labour market optional modules and ersie%201%20november%202013.pdf
justification.
Dee Fink - A Self-Directed
Guide to Designing Courses http://www.deefinkandassociates.com/Gui
for Significant Learning detoCourseDesignAug05.pdf
NL
Manual professional bachelor To be further checked
level: Guide to testing,
realization and justification of
professional bachelor
education level.
Leer- en Assessing competences http://lic.avans.nl/service.lic/
Innovatiecentr
um (LIC)
University of Competence 1: learning https://www.utwente.nl/ces/celt/what-we-
Twente objectives do/teacher-development/
(Centre of
Expertise in
learning and
teaching)

University of What characterizes a good https://vimeo.com/album/1730051/video/2
Utrecht learning objective? About 17 9314808
(Faculty of keys
Humanities)
Centre for
teaching and
learning
University of Guide for Testing and https://score.hva.nl/Bronnen/Leidraad%20
applied Assessment Toetsen%20en%20Beoordelen.pdf
science of
Amsterdam
EIndniveau Qualifying level Associate
Associate Degree http://www.leido.nl/publicaties/documents/
Degree BeschrijvingvanAd-Eindniveau-LN-Ad-
juli2013-versie1.doc
Ministry of National qualification

Education and framework for lifelong learning http://www.regjeringen.no/nb/dep/kd/tema/
Research livslang-laring/nasjonalt-
kvalifikasjonsrammeverk.html?id=601327
NO
The Guidelines for working with
Norwegian curricula at local level http://www.udir.no/Lareplaner/Veiledninger
Directorate for -til-lareplaner/Veiledning-i-lokalt-arbeid-
Education and med-lareplaner/
Training
Ministry of The autonomy of the http://www.nauka.gov.pl/g2/oryginal/2013_
Science and university curriculum. 05/a46de52fb98a0fcc7377f73724f36672.p
Higher Qualifications framework for df
Education higher education.
Minister of Regulation of the Minister of
Science and Science and Higher Education http://www.nauka.gov.pl/g2/oryginal/2013_
Higher of 2 November 2011 on the 05/478e9241dffed3a0bcd4fb28792392a8.
Education National Qualifications pdf
Framework for Higher
PL
Education (Journal of Laws
2011, No. 253, item., 1520).
National Vocational and continuing
Centre for education. Vadamecum. http://new.koweziu.edu.pl/publikacje/item/5
Supporting 7-kształcenie-zawodowe-i-ustawiczne
Vocational
and
Continuing
Education

National Methodological handbook on
Agency for learning outcomes
Qualification Methodological Guide Guia metodologico
LO_Portugal RA
and
PT
Vocational
Education and
Training
(ANQEP, I.P.)
National Methodological aspects for http://www.scriptmedia.ro/cript-
Centre for the the design of training newsletter/docs/aspecte_metodologice.pdf
Development standards in vocational and
of TVET technical education – part of
(Centrul the Reference framework for
National de TVET curriculum.
Dezvoltare a
Invatamantulu
i Profesional
si Tehnic
RO (CNDIPT) –
subordinate to
the Ministry of
Education.
Info sent by Information on LO in the Law http://lege5.ro/Gratuit/geztsobvgi/legea-

the National of National Education educatiei-nationale-nr-1-2011
Qualifications
Authority
National Methodological guide for http://www.cpi.si/files/cpi/userfiles/Publikac

Education drafting educational standards ije/kurikul.pdf
Institute of the and VET curricula.
SL
Republic of
Slovenia for
VET
Scottish Writing National and Higher
Qualification National Units - Scottish http://www.sqa.org.uk/sqa/files_ccc/16Gui
Authority Qualification Authority (2011) deUnitWritingAugust2012.doc
UK Guide to developing National

Commission Occupational Standards https://www.gov.uk/government/uploads/s
for ystem/uploads/attachment_data/file/30423
UK Sco
Employment 9/nos-guide-for-_developers-2011.pdf
and Skills
UK Guide to writing National
Commission Occupational Standards http://www.lantra.co.uk/Downloads/Standa
for rds-qualifications/NOS/Guide-for-Writing-
Employment NOS-Standards-(Feb-2010).aspx
and Skills

University of Assessment Notes http://web2.uconn.edu/assessment/docs/r
Connecticut esources/Eric_Soulsby_Assessment_Note
USA
(Eric Soulsby) s.pdf

OVERVIEW OVER RESEARCH MATERIAL SUPPORTING THE WRITING,
DEFINITION AND USE OF LEARNING OUTCOMES
Teaching for quality education at university

By Bigg, J and C.Tang, 2007, McGrawHill, New York
http://www.umweltbildung-noe.at/upload/files/OEKOLOG%202014/2_49657968-
Teaching-for-Quality-Learning-at-University.pd
Curriculum design in higher education using a learning outcome-led model: Its

influence on how students perceive learning
By Allan, J., 1997, University of Wolverhampton
http://wlv.openrepository.com/wlv/handle/2436/30415
Effects of experimental learning - outcomes of an empirical study in the vocational

field of structural engineering
by Bünning, Frank
International Journal of Training Research, 04/2013, Volume 11, Issue 1
Assessing the quality of the learning outcome in vocational education: the Expero
model
by Cervai, Sara Cian, Luca
Journal of Workplace Learning, 03/2013, Volume 25, Issue 3
Learning Outcomes as a Key Concept in Policy Documents throughout Policy Changes

by Prøitz, Tine Sophie
Scandinavian Journal of Educational Research, 05/2015, Volume 59, Issue 3
Evaluating learning outcomes: in search of lost knowledge

by O'Brien, Stephen Brancaleone, David
Irish Educational Studies, 03/2011, Volume 30, Issue 1
The missing link to connect education and employment: recognition of non-formal and
informal learning outcomes
by Werquin, Patrick
Journal of Education and Work, 07/2012, Volume 25, Issue 3
Learning outcomes of a work-based training programme: The significance of

managerial support
by Ellström, Eva Ellström, Per-Erik
European Journal of Training and Development, 2014, Volume 38, Issue 3
Examining the mediating role of learning engagement, learning process and learning
experience on the learning outcomes through localized real case studies
by Nkhoma, Mathews Sriratanaviriyakul, Narumon
Education + Training, 05/2014, Volume 56, Issue 4
Learning Outcomes — From Policy Discourse to Practice

by Stanley, Julian

European Journal of Education, 12/2015, Volume 50, Issue 4
New study aims to compare outcomes-based learning approaches across Europe

by Elizabeth Eyre
Training Journal, 06/2011
Governance for Learning Outcomes in European Policy-Making: Qualification

Frameworks Pushed through the Open Method of Coordination
by Odd Bjørn Ure
International Journal for Research in Vocational Education and Training, 12/2015,
Volume 2, Issue 4
Assessment of Learning Outcomes in Finnish Vocational Education and Training

by Räisänen, Anu Räkköläinen, Mari
Assessment in Education: Principles, Policy & Practice, 2014, Volume 21, Issue 1
Learning outcomes of a work-based training programme

by Ellström, Eva Ellström, Per-Erik
European Journal of Training and Development, 04/2014, Volume 38, Issue 3
Qualifications frameworks and learning outcomes: challenges for Europe's lifelong

learning area
by Bohlinger, Sandra
A critical perspective on learning outcomes and the effectiveness of experiential

approaches in entrepreneurship education
by Scott, Jonathan M Penaluna, Andy
Education + Training, 01/2016, Volume 58, Issue 1
Curriculum on the European policy agenda: Global transitions and learning outcomes
from transnational and national points of view
by Sivesind, Kirsten Wahlström, Ninni
European Educational Research Journal, 05/2016, Volume 15, Issue 3
Qualifications frameworks and learning outcomes: challenges for Europe’s lifelong

learning area
by Bohlinger, Sandra
Learning outcomes: good, irrelevant, bad or none of the above?

by Souto-Otero, Manuel
Re-Contextualizing Vocational Didactics in Norwegian Vocational Education and

Training
by Hæge Nore
International Journal for Research in Vocational Education and Training, 12/2015,
Volume 2, Issue 3

Anrechnungspraxis und –potenziale von Lernergebnissen aus humanberuflichen
höheren Schulen im hochschulischen Sektor
by Norbert Lachmayr et al.
Lachmayr Norbert ; Mayerl Martin ;
Wien : ÖIBF, 2014
Education and training governance through learning outcomes: possibilities and

constraints in Italy
by Frontini, Sara; Psifidou, Irene;
Madrid : Bordon, 2014
http://recyt.fecyt.es/index.php/BORDON/article/view/Bordon.2015.67110/17187
Peer learning activity: the writing of learning outcomes for assessment and validation:
19-20 November 2014, Cork, Ireland: summary report
European Commission. Directorate General for Employment, Social Affairs and Inclusion;
Brussels: European Commission, 2
http://extendedcampus.cit.ie/contentfiles/PLA%20Cork%2019_20%2011%202014%20R
eport%20final.pdf
Summary report on the Peer learning activity: the writing of learning outcomes for
assessment and validation: 19-20 November 2014
European Commission
Luxembourg: Publication Office, 2014
http://extendedcampus.cit.ie/contentfiles/PLA%20Cork%2019_20%2011%202014%20R
eport%20final.pdf
Claims vs. practicalities: lessons about using learning outcomes

by Allais, Stephanie
Abingdon, Oxon, UK: Taylor and Francis, 2012
http://www.tandfonline.com/doi/pdf/10.1080/13639080.2012.687570
Curriculum reform in Europe: the impact of learning outcomes

Cedefop
Luxembourg : Publications Office, 2012
http://www.cedefop.europa.eu/EN/Files/5529_en.pdf
Learning outcomes: good, irrelevant, bad or none of the above?

by Souto Otero, Manuel
Abingdon: Routledge, 2012
http://dx.doi.org/10.1080/13639080.2012.689648
'Tuning' education for the market in 'Europe'? qualifications, competences and

learning outcomes: reform and action on the shop floor
by Antunes, Fátima
Oxford : Symposium Journals, 2012
http://www.swetswise.com/eAccess/viewAbstract.do?articleID=167110915

A Tunning-AHELO conceptual framework of expected/desired learning outcomes in
engineering
Tuning Association; Assessment of Higher Education Learning Outcomes; Assessment
of Higher Education Learning Outcomes
Paris : OECD, 2011
http://dx.doi.org/10.1787/5kghtchn8mbn-en
Entität und Relation von Lernergebnissen als Herausforderung bei der Konstruktion
von Qualifikationsrahmen
by Schlögl, Peter
Linz: Trauner Verlag, 2011
Gestaltungsfeld Anrechnung : Bildung im Wandel

by Freitag, Walburga Katherina
Münster: Waxmann, 2011
http://www.his.de/pdf/22/gestaltungsfeld_anrechnung.pdf
Mācīšanās rezultātu nozīme ESF projektā Nozaru kvalifikācijas sistēmas izveide un

profesionālās izglītības efektivitātes un kvalitātes paaugstināšana
by Valaine, Sarmite
Riga: Academic Information Centre - National Coordination Point, 2011
http://www.nki-latvija.lv/wp-content/uploads/2011/10/Valaine.pdf
Peer Learning Activity on how the learning outcomes approach, NQFs and quality
assurance can best support lifelong learning: summary report
European Commission
http://www.kslll.net/Documents/Report%20on%20PLA%20Istanbul_KSLLL.pdf
Peer learning activity on the role of quality assurance in defining, describing and
assessing learning outcomes
European Commission
http://kslll.net/Documents/Helsinki%20PLA%20Report.pdf
Rozporządzenie Ministra Nauki i Szkolnictwa Wyższego z dnia 4 listopada 2011 r. w

sprawie wzorcowych efektów kształcenia
Warsaw: MEN, 2011
http://www.dziennikustaw.gov.pl/DU/2011/s/253/1521
Stärkung der Lernergebnisorientierung im Hochschulbereich : Hintergründe, Beispiele

und Empfehlungen im Kontext interner und externer Qualitätssicherung. Projekt
"Stärkung der Learning-Outcome-Orientierung bei der Curriculumsentwicklung und
Akkreditierung im österreichischen FH-Sektor"
by Luomi-Messerer, Karin; Brandstetter, Genoveva
Vienna: facultas.wuv, 2011
Empowering teachers to focus on the learner: the role of learning outcomes in

curricula

by Psifidou, Irene
[unpublished], 2010
http://libserver.cedefop.europa.eu/vetelib/2010/75176.pdf
Learning outcomes approaches in VET curricula: a comparative analysis of nine

European countries
Cedefop
Luxembourg: Publications Office, 2010
http://www.cedefop.europa.eu/EN/Files/5506_en.pdf
Leitfaden zur Gestaltung von kompetenzbasierten und lernergebnisorientierten

Lehrplänen für Berufsbildende Höhere Schulen (BHS) und Bildungsanstalten (BA).
Bundeministerium für Unterricht, Kunst und Kultur. BS. Berufsbildende Schulen; Kreiml,
Peter et al. (eds.)
[Vienna]: BM UKK, 2010
http://libserver.cedefop.europa.eu/vetelib/2010/73499.pdf


Learning Outcomes

Uploaded by

Copyright:

Available Formats

Learning Outcomes

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Learning Outcomes

Uploaded by

Copyright:

Available Formats

Brussels/Thessaloniki, 7 October 2016

DRAFT for discussion at Cedefop 2nd Policy Learning Forum 13-14

Direct line: +30 2310490.......... | Fax +30 2310490.......... | [email protected]

DRAFT European Handbook learning outcomes October 2016.docx page 1 of 52

PART I. PERSPECTIVES AND RULES OF THUMB INFORMING THE

PART II. LEARNING OUTCOMES – OPPORTUNITIES AND DILEMMAS .... 18

PART III. RESOURCES AND SOURCES SUPPORTING THE

Overview of the guidance material for writing LOs ............................................. 39

DRAFT European Handbook learning outcomes October 2016.docx page 2 of 52

Learning outcomes state what a learner is expected to know, be able to do

This handbook is written for individuals and institutions actively involved in

Cedefop recognises that a significant amount of guidance material have

 It will act as an ‘entry point’ to existing guidance and research

The handbook should help to identify the choices faced by practitioners in

DRAFT European Handbook learning outcomes October 2016.docx page 4 of 52

1.1. The context of the handbook

Recent Cedefop studies (2009 and 2016) document that learning

DRAFT European Handbook learning outcomes October 2016.docx page 5 of 52

 Which basic perspectives underpin the writing of learning outcomes

The second part of the handbook provides a more in-depth discussion on

 How is the learning outcome concept defined and what is the

DRAFT European Handbook learning outcomes October 2016.docx page 6 of 52

DRAFT European Handbook learning outcomes October 2016.docx page 7 of 52

DRAFT European Handbook learning outcomes October 2016.docx page 8 of 52

2.1. Basic perspectives

 Using learning outcomes represents a perspective and a mode of

 We distinguish between intentional and actually achieved learning

 Consistent application of learning outcomes requires continuous

 Writing learning outcomes is not a purely technical and neutral activity

 It is possible to observe a main tension between what can be

DRAFT European Handbook learning outcomes October 2016.docx page 9 of 52

 Not all learning can be defined in learning outcomes. The learning

 Learning outcomes need to defined and written in a way which allows

 We need to be careful about treating ‘knowledge’ as information that

DRAFT European Handbook learning outcomes October 2016.docx page 10 of 52

 When writing learning outcomes for vocational education and training it

 Defining and writing learning outcomes in VET and the balancing of

 Defining and writing learning outcomes is a continuous process

 Learning outcomes statements should not be written in isolation, but be

 Learning outcomes never operate in isolation but have to be defined

2.2. The writing of learning outcomes statements

 Simplicity is important when writing learning outcomes; avoid too much

DRAFT European Handbook learning outcomes October 2016.docx page 11 of 52

Table 1. The basic structure of learning outcomes statements

The basic structure of learning outcomes statements

Table 2. Examples of applying the basic structure of learning

 Learning outcomes statement - combining action

 Introducing the vertical dimension of learning outcomes statements

 The introduction of learning outcomes based qualifications

 Action or activity verbs play a particularly important role in writing

DRAFT European Handbook learning outcomes October 2016.docx page 12 of 52

Figure 1. Bloom’s taxonomy: cognitive, psycho-motor and affective

Sources: Bloom et al. (1956); Dave (1970); Anderson and Krathwohl

 As far as possible avoid ambiguous verbs. The following verbs are

DRAFT European Handbook learning outcomes October 2016.docx page 13 of 52

 A good way to distinguish between the different categories of verbs is

Declarative verbs Procedural/

 Introducing the horizontal dimension of learning outcomes statements

DRAFT European Handbook learning outcomes October 2016.docx page 14 of 52

 When writing learning outcomes for a unit, programme or qualification,