Chapter 7

EVALUATING HRD PROGRAMS

HRD evaluation is defined as – “The methodical collection of
descriptive and judgmental information necessary to make effective
training decisions related to the selection, implementation, value,
and modification of various developmental, activities.”
The important points highlighted in this definition are:

1. Evaluation involves the methodical collection of information

according to a predetermined plan to ensure that the
information is appropriate and useful.

2. While conducting an HRD evaluation, both descriptive and

judgmental information should be collected.

3. Evaluation is conducted to help managers, employees and

HRD professionals make informed decisions about particular
programs and methods.

Evaluation of HRD Program can serve a number of purposes within

the organization.

According to Phillips, evaluation can help to do the

following:

1. Establish whether a program is accomplishing its objectives as

proposed.

2. Identify the strengths and weaknesses of HRD programs.

3. Conclude the cost-benefit ratio of an HRD program.

4. Decide the number and nature of participants who benefited

the most or least from the program.
 5. Emphasize major points to be made to the participants.

6. Gather data to assist in designing future programs.

7. Determine the appropriateness of the program.

8. Better and more informed decision making in future relating

to design of HRD programs.

There are other reasons for conducting HRD evaluation as well.

Evaluation can build credibility with top managers and others in the
organization. If HRD staff cannot substantiate its contribution to
the organization, it’s funding and program may be revised during
the budgeting process. Thus, evaluation is a critical step in the HRD
process. It is the only way one can know whether an HRD program
has fulfilled its objectives.

HRD evaluation outlines the criteria for and focuses on the

evaluation effort.

What are the steps in the process of HRD evaluation?

1. Data Collection for HRD Evaluation:

The first step of an evaluation effort requires the collection of data

to providing the decision makers with facts and judgments upon
which they can base their decisions is vital. Three important aspects
of providing information for HRD Evaluation include data
collection methods, types of data, and the use of report.

2. Research Design:

Research design is significant to HRD evaluation as it specifies the

expected results of the evaluation, the methods of data collection,
and the method of data analysis.

3. Ethical Issues Concerning Evaluation:

Many of the decisions supervisors and HRD professionals make
when conducting HRD evaluations have ethical dimensions. Actions
such as – assigning participants to training and Control groups,
reporting results, and the actual conduct of the evaluation study
itself all raise ethical questions like that of confidentiality and
biasness. Some evaluation research project involves asking
participants questions about their or others’ job performance.

The results of these inquiries may be embarrassing or lead to

adverse treatment by others if they are made public. Also
supervisors dislike being rated by their subordinates on
performance aspects and rater’s bias can creep in. Halo effect,
recency effect are other potential biases that can creep in.

Evaluation studies should be monitored by a review board to ensure

that participants are aware that they are participating in a study and
know its purpose, what they will be expected to do, and the
potential risks and benefits of participating. In some cases, an
investigator may feel that the study will yield better results if
employees don’t realize they are in an evaluation study, or if they
are given some false or misleading information during the study.

HRD professionals and their managers may feel pressurized to

make sure that the results of their evaluation reveal that the
program was effective. This may be one reason why meticulous
evaluation of HRD programs is not done more often. The HRD
people are the ones who design and develop, implement, and
evaluate the program, if the evaluation shows the program was
ineffective the HRD department may lose financial support and
have their activities curtailed.

Although the possibility exists for “deception” in the form of

doctoring results, reporting partial results, or setting up biased
studies, it is unclear how often this occurs in HRD evaluation. But it
is imperative on the part of Management to be cautious while
evaluating the HRD Programs.
One of the most tenuous and unsatisfactory aspects of training
programs is the evaluation of their effectiveness. Evaluation is any
attempt to obtain feedback on the effects of a training program and
to assess the value of the training in the light of information thus
collected. Evaluation leads to decision making and control which
means deciding whether or not the tr training
aining was worth the effort
and what improvements are required to make it even more effective.

What are the models and frameworks of HRD evaluation?

A model of evaluation outlines the criteria for and focuses of an

evaluation. Many different frameworks of HRD evaluation
valuation have
been suggested.

The most widely used evaluation approach has been the framework
laid out by Donald Kirkpatrick.

KIRKPATRICK’S EVALUATION FRAMEWORK

The most popular and influential framework for training evaluation

is articulated by Kirkpatrick. Kirkpatrick argues that training efforts
can be evaluated according to four criteria: reaction, learning,
behavior, and results.
1. Reaction (Level 1) Did the trainees like the program and feel it
was valuable? At this level, the focus is on the trainees’ perceptions
about a program and its effectiveness. This is useful information.
Positive reactions to a training program may make it easier to
encourage employees to attend future programs. But if trainees did
not like the program or think they didn’t learn anything (even if
they did), they may discourage others from attending and be
reluctant to use the skills or knowledge obtained in the program.
The main limitation of evaluating HRD programs at the reaction
level is that this information cannot indicate whether the program
met its objectives beyond ensuring participant satisfaction.

2. Learning (Level 2) Did the trainees learn what the HRD

objectives said they should learn? This is an important criterion
that an effective HRD program should satisfy. Measuring whether
someone has learned something in training may involve a quiz or
test—clearly a different method from assessing the participants’
reaction to the program.

3. Job Behavior (Level 3) Does the trainee use what was learned
in training back on the job? This is also a critical measure of
training success. We all know coworkers who have learned how to
do something but choose not to. If learning does not transfer to the
job, the training effort cannot have an impact on employee or
organizational effectiveness. Measuring whether training has
transferred to the job requires observation of the trainee’s on-the-
job behavior or viewing organizational records (e.g., reduced
customer complaints, a reduction in scrap rate).

4. Results (Level 4) Has the training or HRD effort improved the

organization’s effectiveness? Is the organization more efficient,
more profitable, or better able to serve its clients or customers as a
result of the training program? Meeting this criterion is considered
the bottom line as far as most managers are concerned. It is also the
most challenging level to assess, given that many things beyond
employee performance can affect organizational performance.
Typically at this level, economic and operating data (such as sales or
waste) are collected and analyzed.
Another Model for evaluation of training is the Brinkerhoff’s
Model which extends the training evaluation model to six stages
questioning the rationale, process and benefits of having an HRD
Program –

1. Goal Setting – What is the need for HRD Program?

2. Program Design – What will work to meet the need?
3. Program Implementation – Is the HRD Program working, with
the focus on the implementation of the program?
4. Immediate Outcomes – What did the participants learn?
5. Intermediate Outcomes – Are the participants using what they
learned?
6. Impacts and Worth – Did it make a worthwhile difference to
the organizations performance?

Bushnell suggests a model based on a systems view of the

HRD function – input -throughput-output containing four
stages:

1. Input – What goes into the training effort? This consists of

performance indications such as – trainee qualification and
trainer ability.
2. Process – The planning, design, development, and
implementation of the HRD program.
3. Output – Trainee reactions, knowledge or skills gained, and
improved job behavior.
4. Outcome – Effects on the organization, including profits,
productivity and customer satisfaction.

The evaluation measurement should be done to ensure that the

program is well designed and meets its objectives. In its simplest
form, evaluation should address the question of whether the
training program achieved its objectives. Basing training objectives
on needs assessment information, and then evaluating those
objectives, is the most economical way of summarizing what
training evaluation efforts can focus on.