PROCEDURE FOR TRAINING PROGRAMS ASSESSMENT

August 2011

1. Applicability:

2. Scope: Training evaluation/assessment is a critical component of analysing, designing, developing, and implementing an effective training program; the more care has been taken in the assessment of needs and the more precise the objectives, the greater will be the possibility of effective evaluation. The focus is on measuring four kinds of outcomes that should result from a highly effective training program: To validate training as a business tool; To justify the costs incurred in training; To help improve the design of training; To assist in selecting training methods;

3. Responsibilities: Senior management Ensure that procedures on training evaluation include assessing the effectiveness of training are available and used properly; Systematically review and evaluate training effectiveness information and data to assess the effectiveness of personnel training. This should include training provided by outside/external training organizations and in-house training courses; Allocate resources necessary for comprehensive evaluation of training

effectiveness. Make policy and strategic decisions based on results and ROI data. Monitor overall human resource performance against established goals and objectives such as quality, production, and cost. Evaluate whether problems and opportunities indicate the need to initiate additional training or to upgrade that currently in place.

The department manager Systematically review and evaluate training effectiveness information and data to assess training effectiveness for programs within own department; Personally monitor and coach personnel; Identify and document problems and opportunities to identify new training needs. Ensure self-assessments of personal, functional, and process performance and ;

The HR manager/learning and development responsible Introduce and maintain training evaluation systems; produce regular reports for senior management Monitor the conducting of training, feedback from trainees and instructors, and employees performance to assess training effectiveness. Use this information to institute additional training or upgrades to address noted deficiencies; Identify, collect (or ensure the collection) of data and information useful in the evaluation of the effectiveness of companys personnel training; Evaluate the effectiveness of training provided early so that improvements can be initiated in time to promote efficient implementation; Evaluate methods to deliver training more effectively and efficiently. Establish a system to control and track the implementation of training development recommendations; In-house trainer

Analyse and evaluate training effectiveness information and data to identify areas where training to be improved; Interact frequently with companys personnel and trainees to evaluate the effectiveness of training. Conduct follow-ups with students to obtain candid feedback and suggestions to improve the content and delivery of training,

Conduct post-training evaluation, generic trends, strengths and areas for improvement and write periodic reports based on available internal data and information, Provide feedback to line and training management and to training review committees on the effectiveness of training and suggestions for improvements

The trainee or learner Assess and provide a feedback on the value of training received in providing useful knowledge and skills by completing Training feedback questionnaire (Annex A); Routinely meet to formulate recommendations needed to improve the effectiveness of training; Complete a personal action plan during and at the end of the training for implementation on return to work, and to put this into practice, with support from the line manager.

4. Procedure The training effectiveness matrix is based on the work of Donald L. Kirkpatrick, who introduced a four-level approach to training evaluation in 1959 Level 1 Reaction This level measures how the learners react to the training and if a certain training program fails to satisfy learners needs, a determination is made as to whether it is a fault of the programme design or delivery. The evaluation performed at this level is extremely important as the interest, attention and motivation of the participants are often critical to the success of the training process. What is more, Learning (Level 2 outcomes) and transfer of learning (Level 3 outcomes) are unlikely to occur unless participants have positive attitudes towards the training program. Level 1 evaluation is performed by means of Training feedback questionnaire (Annex A), which is given to employees who have just undergone training. The data gathered from this survey allows trainers to know the weak areas that need improvement, while upholding the strong areas. Taking into account trainees anonymous responses to this questionnaire the training manager or the in-house trainer produces a report on the training programme effectiveness. Level 2 Knowledge The goal is to determine what the training program participants learned during the training event, whether the learning objectives have been met and learning outcomes can be clearly identified. Learning outcomes can include changes in knowledge, skills or attitudes. 3

Level 2-evaluation is performed by comparing results from pre-training and post-training tests to differentiate between what they already knew prior to training and what they actually learned during the training program. The tests can be: written, oral or computerbased and they should be aimed at evaluating theoretical issues and their practical application. Level 3 Behaviour The goal is to find out the extent to which the trainees apply the learning and change their on-the-job behaviour as a result of having attended and participated in the training program. Level three-evaluation specifically involves measuring the transfer of knowledge, skills and attitudes from the training context to the workplace. 360 degree feedback can be used in order to evaluate changes in behaviour as a result of a coaching/mentoring/training program; Annex B presents a template for assessing communication skills; the template is to be customized according to the core skills/competencies/attitudes aimed to be developed by the respective development program, to be filled in before and after the delivery of the program. In the first column the trainees should assess themselves and the next two columns are for a line manager and a trainees peer/colleague. Level 4 Results This level of evaluation provides the basis for assessing the benefits of the training against its costs. The objective is to determine the added value of learning and development programs how they contribute to raising organizational performance significantly above its previous level. The evaluation has to be based on before and after measures and has to determine the extent to which the fundamental objectives of the training have been achieved in areas such as increasing sales, raising productivity, reducing accidents or increasing customer satisfaction. Perhaps the most powerful method of demonstrating that learning programs pay is to measure the return on investment (ROI) The formula for calculating ROI is:

Attempting financial ROI assessment of training is a controversial issue. It's a difficult task to do in absolute terms due to the many aspects to be taken into account, some of which are very difficult to quantify at all, let alone to define in precise financial terms. Investment - the cost - in training may be easier to identify, but the benefits - the return are notoriously tricky to pin down. 5. Annexes: Annex A - Training feedback questionnaire (1) Annex B - Training feedback questionnaire (2) Annex C - 360 degree communication skills assessment tool

Annex A

Training feedback questionnaire (1)

A. General - Tick the column you feel most closely represents your views on the rate applicable for each criteria 6 1. To what extent have the objectives of the programme been achieved? 2. To what extent have your personal objectives for attending the programme been achieved? 3. To what extent has your understanding of the subject improved or increased as a result of the programme/ To what extent has the programme helped to enhance your appreciation and understanding of your job as a whole? What is your overall rating of this programme? 5 4 3 2 1

If you have scored 3, 2, or 1, please comment why you have given this rating. B. Trainer evaluation Please rate the trainer with the relevant score for each aspect from (a) to (f) taking into account that 4-very effective; 3-good; 2-not very effective; 1-not effective; (a) (b) (c) Knowledge of subject Organization of sessions Obvious preparation 6

(d) (e) (f)

Style and delivery Responsiveness to group Producing a good learning climate

Any other comments: C. Balance of programme 1. How do you rate the balance between input sessions, activities, discussions, and videos? Good balance 6 5 4 3 2 1 Poor balance

Why do you give this rating? 2. How did you feel about the length of the programme? Too short Just right Too long

3. To what extent was the programme logically sequenced? Well sequenced In what way? 4. How did you feel about the pacing of the programme? Too short Just right Too long 6 5 4 3 2 1 Poorly sequenced

5. How effective were the practical activities? Very effective 6 5 4 3 2 1 Ineffective

Why do you give this rating? 6. What was the level of time given for (a) the activities and (b) the follow-up discussion? a) Sufficient 6 5 4 3 2 1 Insufficient

b) Sufficient

Insufficient

What level of time would you like to have seen? D. Programme content 1. What did you like best about the programme?

2. What did you like least about the programme?

3. What did you learn from the programme?

4. What did you not learn from the programme that you were expecting to learn?

5. What do you think should be added to the programme?

6.

What do you think should be dropped from the programme?

7. To what extent did the programme duplicate what you had learned previously

8. What are your views on the hand-outs issued? Excellent quality Too many Very relevant 6 5 Just right Satisfactory 4 3 2 Too few Not at all relevant 1 Poor quality

Any comments on the above ratings?

9. What are your views on the visual aids used?

OHP/PowerPoint slides: Excellent quality 6 5 4 3 2 1 Poor quality

Too many Well used 6 5

Just right 4 3 2 1

Too few Badly used

Flipchart: Excellent quality 6 5 4 3 2 Just right 1 Poor quality Too few

Too many Well used 6 5 4

Badly used

E. Post-training 1. Has a post-training debriefing meeting been arranged with your line manager? Yes No

2. If 'No', will you have a post-training debriefing with your line manager? Yes Don't know No

3. To what extent will you be helped to implement your Action Plans: (a) by your line manager (b) by your colleagues? A lot 6 5 4 3 2 1 Not at all

A lot

Not at all

4. If your line manager does not perform post-programme reviews and follow-up of your action plans, do you agree that the Training Department can contact you for this purpose? 9

Yes

No

Don't know

5. Are there any other comments about the training programme that have not been covered and that you would like to make?

10

Annex B

Training feedback questionnaire (2)

1. TRAINING SEMINAR EVALUATION 1. To what extent has this training course taken into account your needs? Rate this on a scale from 1 to 5, where 1 means to a small extent and 5 to a high extent. 1 2 3 4 5

2. How relevant do you consider this training course for your professional activity? Rate the relevance of training materials used on a scale from 1 to 5 where 1 stands for irrelevant and 5 very relevant . 1 2 3 4 5

3. Rate the relevance of training materials used on a scale from 1 to 5 where 1 stands for irrelevant and 5 very relevant . In point of quantity In point of quality 1 1 2 2 3 3 4 4 5 5

3. In your opinion, which are the strong points of this training program? 4. What about weak points? 5. Do you consider that there are other related training needs that this course failed to cover?

6. What suggestions can you provide so as to improve this training course? 11

 . 2. TRAINER EVALUATION

Rate the trainers behavior and attitude on a scale from 1 to 5, where 1 stands for ineffective and 5 very effective 1. Professional attitude (knowledge of the subject, preparation of materials and training modules) 1 2 3 4 5

2. Communication and motivating skills 1 2 3 4 5

3. Training Management skills 1 2 3 4 5

4. Active listening and integration of learners opinions 1 2 3 4 5

5. Facilitation skills (style and delivery) 1 2 3 4 5

6. Successful blending of theory and practice 1 2 3 4 5

7. Flexibility and availability to learn new things 1 2 3 4 5

8. Continuous feedback throughout the training seminar 1 2 3 4 5

12

Annex C

360 degree Communication skills assessment tool

Self

1st view

2nd view

bridge

listening

techniques

for

improved

Assertive skills

Exhibit confident body language and assertively managing real-world pressures and demands Apply assertiveness techniques and adopting effective assertiveness strategies Give and receiving structured feedback Effectively handle conflict using a powerful conflict resolution method Anticipate the causes of conflict and respond proactively Manage strong emotions in a conflict situation; Defuse an emotional situation while maintaining your composure Embrace constructive conflict as an opportunity for team and organizational growth Create a personal speaking profile Construct informative, persuasive and audience-centered talks and presentations Emphasizing your points with effective body language, gestures and use of space Create informative and interesting content that your readers will understand and use Write clear, effective technical and business documents and e-mails to accomplish business objectives Fine-tune language to improve persuasiveness and impact Apply influence strategies to gain commitment from others and foster collaboration Dynamically adjust your approach to others to gain buy-in Deal effectively with challenging behaviours to overcome resistance and inertia in others 13

Conflict management

Public speaking

Writing skills

Interpersonal skills