Training Programme Evaluation: Training and Learning Evaluation, Feedback Forms, Action Plans and Follow-Up
Training Programme Evaluation: Training and Learning Evaluation, Feedback Forms, Action Plans and Follow-Up
Training Programme Evaluation: Training and Learning Evaluation, Feedback Forms, Action Plans and Follow-Up
introduction
A vital aspect of any sort of evaluation is its effect on the person being evaluated.
Feedback is essential for people to know how they are progressing, and also, evaluation is
crucial to the learner's confidence too.
And since people's commitment to learning relies so heavily on confidence and a belief that the
learning is achievable, the way that tests and assessments are designed and managed, and
results presented back to the learners, is a very important part of the learning and development
process.
People can be switched off the whole idea of learning and development very quickly if
they receive only negative critical test results and feedback. Always look for positives in
negative results. Encourage and support - don't criticize without adding some positives, and
certainly never focus on failure, or that's just what you'll produce.
This is a much overlooked factor in all sorts of evaluation and testing, and since this element is
not typically included within evaluation and assessment tools the point is emphasised point loud
and clear here.
So always remember - evaluation is not just for the trainer or teacher or organisation or
policy-makers - evaluation is absolutely vital for the learner too, which is perhaps the most
important reason of all for evaluating people properly, fairly, and with as much encouragement as
the situation allows.
Most of the specific content and tools below for workplace training evaluation is based on the
work of Leslie Rae, an expert and author on the evaluation of learning and training programmes,
and this contribution is greatly appreciated. W Leslie Rae has written over 30 books on training
and the evaluation of learning - he is an expert in his field. His guide to the effective evaluation of
training and learning, training courses and learning programmes, is a useful set of rules and
techniques for all trainers and HR professionals.
This training evaluation guide is augmented by an excellent set of free learning evaluation and
follow-up tools, created by Leslie Rae.
There are other training evaluation working files on the free resources page.
It is recommended that you read this article before using the free evaluation and training followup tools.
Particularly see the notes on this page about using self-assessment in measuring abilities before
and after training (i.e., skills improvement and training effectiveness) which specifically relate to
the 3-Test tool (explained and provided below).
See also the section on Donald Kirkpatrick's training evaluation model, which represents
fundamental theory and principles for evaluating learning and training.
Also see Bloom's Taxonomy of learning domains, which establishes fundamental principles for
training design and evaluation of learning, and thereby, training effectiveness.
Erik Erikson's Psychosocial (Life Stages) Theory is very helpful in understanding how people's
training and development needs change according to age and stage of life. These generational
aspects are increasingly important in meeting people's needs (now firmly a legal requirement
within age discrimination law) and also in making the most of what different age groups can offer
work and organisations. Erikson's theory is helpful particularly when considering broader
personal development needs and possibilities outside of the obvious job-related skills and
knowledge.
Multiple Intelligence theory (section includes free self-tests) is extremely relevant to training and
learning. This model helps address natural abilities and individual potential which can be hidden
or suppressed in many people (often by employers).
Learning Styles theory is extremely relevant to training and teaching, and features in Kolb's
model, and in the VAK learning styles model (also including a free self-test tool). Learning Styles
theory also relates to methods of assessment and evaluation, in which inappropriate testing can
severely skew results. Testing, as well as delivery, must take account of people's learning styles,
for example some people find it very difficult to prove their competence in a written test, but can
show remarkable competence when asked to give a physical demonstration. Text-based
evaluation tools are not the best way to assess everybody.
The Conscious Competence learning stages theory is also a helpful perspective for learners and
teachers. The model helps explain the process of learning to trainers and to learners, and is also
helps to refine judgements about competence, since competence is rarely a simple question of
'can or cannot'. The Conscious Competence model particularly provides encouragement to
teachers and learners when feelings of frustration arise due to apparent lack of progress.
Progress is not always easy to see, but can often be happening nevertheless.
organisational priorities of the day, then you will seriously hamper your chances of fostering a
happy productive society within your workforce, assuming you want to, which I guess is another
subject altogether.
Assuming you do want to develop a happy and productive workforce, it's useful to consider and
learn from the mistakes that have been made in children's education:
the range of learning is far too narrowly defined and ignores individual potential, which is
then devalued or blocked
the range of learning focuses on arbitrary criteria set from the policy-makers' own
perspectives (classic arrogant X-Theory management - it's stifling and suppressive)
testing and assessment of learners and teachers is measuring the wrong things, too
narrowly, in the wrong way - like measuring the weather with a thermometer
testing (the wrong sort, although none would be appropriate for this) is used to assess
and pronounce people's fundamental worth - which quite obviously directly affects selfesteem, confidence, ambition, dreams, life purpose, etc (nothing too serious then..)
wider individual development needs - especially life needs - are ignored (many
organisations and educational policy-makers seem to think that people are robots and that
their work and personal lives are not connected; and that work is unaffected by feelings of
well-being or depression, etc)
individual learning styles are ignored (learning is delivered mainly through reading and
writing when many people are far better at learning through experience, observation, etc again see Kolb and VAK)
testing and assessment focuses on proof of knowledge in a distinctly unfair situation only
helpful to certain types of people, rather than assessing people's application, interpretation
and development of capabilities, which is what real life requires (see Kirkpatrick's model and consider the significance of assessing what people do with their improved capability,
beyond simply assessing whether they've retained the theory, which means relatively very
little)
children's education has traditionally ignored the fact that developing confident happy
productive people is much easier if primarily you help people to discover what they are good
at - whatever it is - and then building on that.
Teaching, training and learning must be aligned with individual potential, individual
learning styles, and wider life development needs, and this wide flexible individual approach
to human development is vital for the workplace, just as it is for schools.
Returning to consider workplace training itself, and the work of Leslie Rae:
To what extent were the identified training needs objectives achieved by the programme?
What commitment have the learners made about the learning they are going to
implement on their return to work?
To what extent has the action listed above achieved a Return on Investment (ROI) for the
organization, either in terms of identified objectives satisfaction or, where possible, a
monetary assessment.
The HR department and trainers, do not have sufficient time to do so, and/or
The HR department does not have sufficient resources - people and money - to do so.
Obviously the evaluation cloth must be cut according to available resources (and the culture
atmosphere), which tend to vary substantially from one organization to another. The fact remains
that good methodical evaluation produces a good reliable data; conversely, where little evaluation
is performed, little is ever known about the effectiveness of the training.
evaluation of training
There are the two principal factors which need to be resolved:
senior management
the trainer
line management
the trainee
The necessity of involving the Training Manager (or equivalent) in senior management
meetings where decisions are made about future changes when training will be essential.
Identification at the start of the programme of the knowledge and skills level of the
trainees/learners.
Provision of training and learning resources to enable the learners to learn within the
objectives of the programme and the learners' own objectives.
At the end of the programme, assessment of and receipt of reports from the learners of
the learning levels achieved.
Ensuring the production by the learners of an action plan to reinforce, practise and
implement learning.
Support of pre-event preparation and holding briefing meetings with the learner.
Holding a debriefing meeting with the learner on their return to work to discuss, agree or
help to modify and agree action for their action plan.
Final review of implementation success and assessment, where possible, of the ROI.
Management of the training department and agreeing the training needs and the
programme application
Liaison with the learners' line managers and arrangement of learning implementation
responsibility learning programmes for the managers
Liaison with line managers, where necessary, in the assessment of the training ROI.
Involvement in the planning and design of the training programme where possible
Involvement in the planning and design of the evaluation process where possible
Obviously, to take interest and an active part in the training programme or activity.
To complete a personal action plan during and at the end of the training for
implementation on return to work, and to put this into practice, with support from the line
manager.
Take interest and support the evaluation processes.
N.B. Although the principal role of the trainee in the programme is to learn, the learner must be
involved in the evaluation process. This is essential, since without their comments much of the
evaluation could not occur. Neither would the new knowledge and skills be implemented. For
trainees to neglect either responsibility the business wastes its investment in training. Trainees
will assist more readily if the process avoids the look and feel of a paper-chase or numbercrunching exercise. Instead, make sure trainees understand the importance of their input exactly what and why they are being asked to do.
1 - do nothing
Doing nothing to measure the effectiveness and result of any business activity is never a good
option, but it is perhaps justifiable in the training area under the following circumstances:
If the organization, even when prompted, displays no interest in the evaluation and
validation of the training and learning - from the line manager up to to the board of directors.
If you, as the trainer, have a solid process for planning training to meet organizational and
people-development needs.
If you have a reasonable level of assurance or evidence that the training being delivered
is fit for purpose, gets results, and that the organization (notably the line managers and the
board, the potential source of criticism and complaint) is happy with the training provision.
You have far better things to do than carry out training evaluation, particularly if evaluation
is difficult and cooperation is sparse.
However, even in these circumstances, there may come a time when having kept a basic system
of evaluation will prove to be helpful, for example:
You receive have a sudden unexpected demand for a justification of a part or all of the
training activity. (These demands can spring up, for example with a change in management,
or policy, or a new initiative).
You see the opportunity or need to produce your own justification (for example to
increase training resource, staffing or budgets, new premises or equipment).
You seek to change job and need evidence of the effectiveness of your past training
activities.
Doing nothing is always the least desirable option. At any time somebody more senior to you
might be moved to ask "Can you prove what you are saying about how successful you are?"
Without evaluation records you are likely to be at a loss for words of proof...
2 - minimal action
The absolutely basic action for a start of some form of evaluation is as follows:
At the end of every training programme, give the learners sufficient time and support in the form
of programme information, and have the learners complete an action plan based on what they
have learned on the programme and what they intend to implement on their return to work. This
action plan should not only include a description of the action intended but comments on how
they intend to implement it, a timescale for starting and completing it, and any resources
required, etc. A fully detailed action plan always helps the learners to consolidate their thoughts.
The action plan will have a secondary use in demonstrating to the trainers, and anyone else
interested, the types and levels of learning that have been achieved. The learners should also be
encouraged to show and discuss their action plans with their line managers on return to work,
whether or not this type of follow-up has been initiated by the manager.
This sort of reactionnaire is described in the book ('Assessing the Value of Your Training', Leslie
Rae, Gower, 2002). This evaluation seeks a score for each question against a 6-point range of
Good to Bad, and also the learners' own reasons for the scores, which is especially important if
the score is low.
Reactionnaires should not be automatic events on every course or programme. This sort of
evaluation can be reserved for new programmes (for example, the first three events) or when
there are indications that something is going wrong with the programme.
Sample reactionnaires are available in the set of free training evaluation tools.
The next evaluation instrument, like the action plan, should be used at the end of every course if
possible. This is the Learning Questionnaire (LQ), which can be a relatively simple instrument
asking the learners what they have learned on the programme, what they have been usefully
reminded of, and what was not included that they expected to be included, or would have liked to
have been included. Scoring ranges can be included, but these are minimal and are subordinate
to the text comments made by the learners. There is an alternative to the LQ called the Key
Objectives LQ (KOLQ) which seeks the amount of learning achieved by posing the relevant
questions against the list of Key Objectives produced for the programme. When a reactionnaire
and LQ/KOLQ are used, they must not be filed away and forgotten at the end of the programme,
as is the common tendency, but used to produce a training evaluation and validation summary. A
factually-based evaluation summary is necessary to support claims that a programme is
good/effective/satisfies the objectives set'. Evaluation summaries can also be helpful for publicity
for the training programme, etc.
Example Learning Questionnaires and Key Objectives Learning Questionnaires are included in
the set of free evaluation tools.
Planning, design and preparation of the training programmes against the objectives
Pre-course identification of people with needs and completion of the preparation required
by the training programme
file - Arabic translation 'three-test' version and the same tool as a doc file - Arabic
translation 'three-test' version.)
Assessment of ROI
Whatever you do, do something. The processes described above allow considerable latitude
depending on resources and culture environment, so there is always the opportunity to do
something - obviously the more tools used and the wider the approach, the more valuable and
effective the evaluation will be. However be pragmatic. Large expensive critical programmes will
always justify more evaluation and scrutiny than small, one-off, non-critical training activities.
Where there's a heavy investment and expectation, so the evaluation should be sufficiently
detailed and complete. Training managers particularly should clarify measurement and evaluation
expectations with senior management prior to embarking on substantial new training activities, so
that appropriate evaluation processes can be established when the programme itself is designed.
Where large and potentially critical programmes are planned, training managers should err on
the side of caution - ensure adequate evaluation processes are in place. As with any investment,
a senior executive is always likely to ask, "What did we get for our investment?", and when he
asks, the training manager needs to be able to provide a fully detailed response.
The '3-Test' before-and-after training example (see manual version (pdf) and manual version (xls)
and working file version) is a useful tool and helpful illustration of the challenge in measuring
improvement in ability after training, using self-assessment.
A vital element within the tool is the assessment called 'revised pre-trained ability', which is
carried out after training.
The 'revised pre-trained ability' is a reassessment to be carried out after training of the
ability level that existed before training.
This will commonly be significantly different to the ability assessment made before training,
because by implication, we do not fully understand competence and ability in a skill/area before
we are trained in it.
People commonly over-estimate their ability before training. After training many people realise
that they actually had lower competence than they first believed (i.e., before receiving the
training).
It is important to allow for this when attempting to measure real improvement using selfassessment. This is the reason for revising (after training) the pre-trained assessment of ability.
Additionally, in many situations after training, people's ideas of competence in a particular
skill/area can expand hugely. They realise how big and complex the subject is and they become
more conscious of their real ability and opportunities to improve. Because of this it is possible for
a person before training to imagine (in ignorance) that they have a competence level of say 7 out
of 10. After training their ability typically improves, but also so does their awareness of the
true nature of competency, and so they may then judge themselves - after training - only to be
say 8 or 7 or even 'lower' at 6 out of 10.
This looks like a regression. It's not of course, which is why a reassessment of the pre-trained
ability is important. Extending the example, a person's revised assessment of their pre-trained
ability could be say 3 or 4 out of 10 (revised downwards from 7/10), because now the person can
make an informed (revised) assessment of their actual competence before training.
A useful reference model in understanding this is the Conscious Competence learning model.
Before we are trained we tend to be unconsciously incompetent (unaware of our true ability and
what competence actually is). After training we become more consciously aware of our true level
of competence, as well as hopefully becoming more competent too. When we use selfassessment tools it is important to allow for this, hence the design of the '3-Test' before-and-after
training tool - see also manual version (pdf) and manual version (xls).
In other words: In measuring improvement, using self-assessment, between before and after
training it is useful first to revise our pre-trained assessment, because before training usually our
assessment of ability is over-optimistic, which can suggest (falsely) an apparent small
improvement or even regression (because we thought we were more skilled than we actually now
realise that we were).
Note that this self-assessment aspect of learning evaluation is only part of the overall evaluation
which can be addressed. See Kirkpatrick's learning evaluation model for a wider appreciation of
the issues.
I am grateful to F Tarek for sharing this pdf file - Arabic translation 'three-test' version and the
same tool as a doc file - Arabic translation 'three-test' version.