EDUCATION & TRAINING

LEVEL 5 DIPLOMA

Student Name Mohan Lal Arya

Student ID NAIS/L5/23163

Module Name Module 4 Evaluating learning environment

Assignment No 4

Date of Submission

Signature (your Mohan Lal Arya

name)
Chapter 1: Understanding the Principles and Methods of Evaluating Learning Programs.

1.1 Analyze the principles of evaluating learning.

1.2 Explain how the principles of evaluating learning can be applied to evaluation of learning programs.

1.3 Analyze methods used for evaluating the effectiveness of learning programs.

1.4 Analyze methods of data collection and analysis used to evaluate learning programs.

As an introduction to your self-assessment report provide a summary of the different principles of evaluation and
explain the one you have selected and why? (1.1 and 1.2). Use the table to help to present your information Example:
Introduction: To evaluate the following principles of evaluation were considered

Evaluation of learning program is the process of gathering and analyzing data about the program and evaluating its
effectiveness. It’s not about measuring one students’ performance but of the program as a whole

Principal of evaluation How can this be effective

The student’s experience academic success This can be effective if we measure the
and learning experience is at the center of all teacher’s performance against proven
activities, reviews, and continual research-based performance targets
improvement. associated with improvements in the
student’s performance

Giving fair and equal chance of success to all This help in better evaluation of learning
the students also making and reasonable program as the data collected from different
adjustments for equity. student will be more accurate. Plus chances
of success of learning program are more if its
fair at the same time inclusive

Setting clear and measurable objectives. That The very firth thing that need to be done
is defining clearly what need to be evaluated. while evaluating the learning program is
setting clear objectives of evaluation. Why
the evaluation need to be done what
information is exactly seeked.

Implanting minimum core skills along with It will help in evaluation as minimum core
the subject in question. That is including skill will help in better assessment. It provides
literacy, numeracy, and technology in the a benchmark to see improvement in students
subject being taught. learning.

Including the students in evaluation process. Evaluation of learning program is done so the
Their feedback and voice is heard. learning for student, thus including students
in evaluation process and taking feedback
from them will give more accurate data that
is less likely to be influenced by other
variables.
Explain the method of evaluation you have selected for your evaluation process. Provide additional notes for the
external verifier on other methods of evaluation you have considered and why you have decided to use the specific
chosen method (1.3)

The various methods of data evaluation are:-

1. Kirkpatrick’s model of evaluation

2. CIPP model
3. Kaufman’s five level model
4. Phillips ROI Mode
5. Anderson’s Model of learning evaluation

The method of evaluation I have chosen is Kirkpatrick’s model of evaluation. This model was developed by Donald
Kirkpatrick in 1959. Although it was originally developed to evaluate the training programs in the organizations but it can
be effectively applied to evaluate the learning programs at schools as well. It has four levels.

Le
ve Reaction
l1
Le
ve Learning
l2
Le
ve Behaviour
l3
Le
ve Results
l4

Level 1: Reaction
This level evaluate how the students felt. Their reaction to the learning program. Did students enjoy the learning
program? Do they find the content of the program relevant? What was the level of participation of the students? Etc.

Level 2: Learning
This level evaluates the how much students have learned. Is there any change in their knowledge level, behavior,
attitude and skills due to the program? Have the students achieved the learning outcome of the program?

Level 3: Behavior
This level measures the impact on students’ behavior. In simple words “Are the students applying what they have
through the program in real life?”

Level 4: Results
The final level is to measure the results. That is the outcomes because of the learning program. For example if the
student’s performance have improved because of the program or not.
Advantages

 Simple and easy to understand.

 It can give about specific aspect of the learning program.
 Give students an opportunity to demonstrate that learning has taken place.
 Help in identifying the gap between the learning outcome and actual learning that has taken place.
 The information from this can be used to improve the effectiveness of learning program in future.

Disadvantage

 It can be time consuming and costly thus an immediate feedback on the program can’t be provided.
 Evaluation in Kirkpatrick’s model is mostly learner centric.

Conclusion

Overall the advantages surpass the disadvantages. This model provides an organized structure for evaluation of learning
program that is easy to understand and follow. It provides valuable information to improve the effectiveness of the
learning program. It is learners centric thus it provides information about what works for them and what doesn’t. Thus
pointing at the areas where improvement can be made.

Explain the methods you have used to collect data you require for the evaluation process and how these provide an
accurate overview of the success of the learning program. (1.4)

Data collection methods How this data will be How this provides an
used collected and used? accurate measure to help in
evaluating learning program

Surveys Help in getting the Through survey we can

information directly from collect standardized
group of students in order to responses which are easy to
get their perception about analyze and interpret. Plus
the program. large data can be obtain
quickly thus quicker means to
evaluate the learning
program.

Students Portfolios This will include students’ A rubric can be used to

notebook, any project done evaluate students’ work.
by them, minute papers, Thus data can be converted
collection of their work etc. to quantitative data for easy
Data observed by looking at interpretation. A group of
students work will give faculty members may review
information how well portfolios in order to identify
students are able to where improvements in the
understand and apply what program are needed.
they have learned.

Pre and post test Pre and post test are Comparing the results of pre
conducted before and at the and post test will give a
end of the program. Same set better picture about
 of questions should be given attainment of learning
in order to evaluate the objectives of the program by
extent of learning taking the students. Whether the
place. students have acquired the
intended knowledge or not.

Performance in formative We will use data obtain from This can be an accurate tool
and summative assessment measuring performance of to evaluate the learning
the students in formative and program students are
summative assessments to provided fair and equal
see if the student has able to platform when performing
improve in his/her these assessments. The
performance during and at summative assessments are
the end of the program. available with standardized
Whether he or she was able scores which can be used to
to achieve targets set? find out where the program
needs improvement. While
the formative assessment
which take place during the
classroom (like quizzes,
games, group discussions)
can be conducted multiple
times during the program and
data from this can be used to
make changes in teaching
and learning strategies

Teachers observation Data though teachers Observations are effective as

observation is qualitative in allows us to view the
nature it provide important program in real time while
data to measure the the teaching and learning is
strengths and weaknesses of taking place. Can be done
a particular learning program covertly also which provide
and provide valuable unanticipated information
feedback
However it is time consuming
and observer’s biasness can
provide incorrect data.

2.1 Develop a framework for evaluation of learning program.

2.2 Devise objectives in order to achieve evaluation aims.
2.3 Select method for evaluating evaluation program.
2.4 Select method for collecting data to evaluate the effectiveness of the learning program

A good evaluation framework should take into account following stokehold-:

1. Program and assessment should comply with legislative requirement. Quality of teaching learning meeting the
benchmarks set by government body.
2. Learners should have as a safe and engaging learning environment so feel motivate to learn and succeed.
3. Teachers must be able to work in a safe working environment and have access to training and recognition
programs.
4. Parents’ involvement, so their children will have a better chance at learning. Parents should be available for
communication with the teachers and school in order to strengthen the flow of feedback.
5. Management should set clear policies and guidelines which align with the aims and mission of the organization.

Evaluation framework
Here is the sample framework I have mad to evaluate my Physical Education program. It is a six step framework
starting from defining the purpose and scope of evaluation, why the program need to be evaluated? What is need?
Than next step is to select the evaluation method, than planning about various tools of data collection and how the
data will be collected. Once the tools for data collection are designed than next step is collecting the actual data.
After that analyzing the data and making interpretation from the same. The last step is to give feedback for
improvement of the program on the basis of analysis of data.

Evalution
Define Develop
Select report for
purpose data Collect Analyse
evalution program
Of collection data the data
method imorove
evalution plan
ment

The evaluation shall be based on Kirkpatrick’s four level model of evaluation

Evaluation Framework

2.3 Information required

Information required will be results of pre and post- test, students’ initial and diagnostic
assessment results students’ performance in formative and summative assessment and
factors that affect the learning program.

2.4 How will this data be collected?

1. Data will be collected through survey, to know the reaction of students toward
the program. Do they like the program? Did they find the program relevant? How
was the resources used? Etc.
2. Data will also be collected through the individual learning plan; results will be
compiled together with other performance assessment. Average mean will be
calculated. The average than can be compared to see if students result are
consistent with class average or are above or below the class average to discover
find the anomalies.
3. Data will be collected by observing students sample work will be observed to see
there learning and its application.
4. Students performance in summative and formative assessment will be recorded
5. Data will be also be collected by comparing pre and post-test results and set of
questions in pre and post-test will be kept same to see the level of learning taking
place.

2.4 How will this data be recorded

1. Data will be uploaded in software provided by school however if data is not

extremely large simple software like excel can also be used to record the data.
2. Individual results from summative assessments will be recorded in Individual
learning plans and in student records.
3. Data from survey will be recorder and mean average would be calculated to
present it in from of pie chart
4. Data from pre and post-test will be shown in form of bar graph for better
comparison.
5. Data from group work in formative assessment will be recorded at the end of
each unit- This table will be converted into a graph for a quick summary.

What will it show

1. This data will show whether the program objectives have been achieved or not.
2. Whether learning is taking place or not. If yes than to what extent, it will show
the change in knowledge, skills, behavior and attitude of the students.
3. It will show the effectiveness of teaching, learning resources and overall the
effectiveness of the learning program.
4. It will show students’ reaction toward the learning program.
5. It will show students’ performance in the assessments which will help identify
gaps in knowledge. Thus if any changes are required it can be made in the
learning program.
2.2 How will it be used? (Objectives of evaluation)
 This data will be used to identify the strengths and weakness of the program, identify the
areas of improvement in the program and making changes accordingly in future for
making the learning of students better and making the learning program more effective.

3.1 Apply selected methods to evaluate the effectiveness of the learning program.
3.2 Apply the selected method to collect data to evaluate the effectiveness of the learning program.
3.3 Analyze the data to evaluate the effectiveness of the learning program.
3.4 Apply relevant guidelines and legislation relevant to data collection and analysis.
3.5 present the analysis of evaluation result.
3.6 Explain how the result can be used to improve the effectiveness of a learning program.

Physical Education program evaluation report

This report is physical education program evaluation report of grade 7B for the year of 2019. It will show the
effectiveness of the program.

Purpose of evaluation

1. Determine the effectiveness of the PE program for future improvement.

2. To evaluate whether the learning outcome of the program were achieved or not
3. To assess the impact and result of physical education training program
4. To identify the strength and weakness of the PE learning program.

Background of the program and learners

The program I am evaluating is Physical education program for grade 7 students. This program helps students to develop
competence to take part in range of physical activities. The main aim of this program is to implant passion for physical
activity and take accountability of their physical health in future as well. Through this program students get the
opportunity to participate in wide range of activities and games and develop their sensory and motor skills..

The learners of this program are children of grade 7B of age between 11 to 13 years. They belong to different cultural
background and ethnicity. I have taken their VARK test at the beginning of the year, all the students have more than one
preferred way of learning, I have chosen this class especially because it has highest number of kinesthetic learners. Most
students are from middle class background.

Introduction and rationale

Physical education program is important tool for developing healthy lifestyle habits in children at young age. It help
them to develop knowledge and skills that they can implement in other aspects of life as well, such as discipline, self-
control, team work, sportsmanship, emotional intelligence and self-efficiency.

Research method

I have used Kirkpatrick’s four level model for evaluating the PE program. It consists of four stages:

Stage 1: reaction
Knowing students’ reaction about the PE program. Did they enjoy the PE program?. Did they find it helpful? Were the
activities engaging and motivating? A point scale survey was conducted to know the answer to such question.
Participants were all the students who have chosen the PE course.
Stage 2: Learning
This stage measure if the learning is taking place? Change in students’ level of knowledge, skills, behavior and attitude
has taken place or not? This was measured through the pre and post test

Stage 3: Behavior
It is rare for student to apply the knowledge acquired immediately but that’s not the case with physical education
program. Here the knowledge acquired through theory is almost immediately applied in the practical field work. Thus
change is behavior is easily evident through observation.

Stage 4: Result
It measure the results achieved because of the learning program, this rely on performance indicators thus students’
performance in standard test are the best tool to measure it.

How Data was collected (3.2)

Level 1 Level 2 Level 3 Level 4

Reaction Learning Behavior Result
 Survey  Pre- post test  Teacher observation  Performance
records

Relevant guidelines and legislation regarding data collection that needs to be followed while collecting the data (3.4)

 Data shouldn’t be shared with a third party without permission from the student or their parents. For example
sharing students’ sample work for illustration purpose can be done but only after taking written permission from
the student or parent.
 Once the data has been used and it became obsolete, of no value, it should be destroyed so it is not misused.
 Keep same set of questions for pre and post-test so that amount of learning student has acquired can be
measured
 To maintain honesty in the survey the replies can be kept anonymous, as such also the name of the student is
not of importance but their reaction and perception toward the program is.
 Survey can be done in person or by sending through email, for the purpose of this evaluation I have chosen to do
it through email and give enough time and reminders to students to finish it.
 Students should be explained in advance why this data is being collected and how it will be used.

Analysis and presentation of data (3.3, 3.5)

The information will used to formulate findings by using mean, percentage and average. Data itself is just facts and
figures, through data analysis and interpretation meaning is assign to the collected information which in turn helps in
making informed decision. The main purpose of data analysis here was to find the effectiveness and means to improve
physical education program in future. Once data is analyzed it is presented in form of bar charts, graphs, pie chart for
easy understanding.

This chart shows overall distribution of grades in the classroom

Here is the chart showing pre-test and post-test results

Pie chart to show teachers observation regarding the change in behavior and skills of the students. The qualitative data
was changed to quantitative data using the rubric and checklist.

Students’ reaction toward the PE program

1. Do you like the physical education training session in the school?

2. Five point scale survey result

 Students’ performance in the formative and summative was mostly between 71-80% or 41-50% this show there
is subsequent gap in between high end achievers and average achievers.
 In pre-test mostly student scored between 20-40% while in the post-test students mostly scored from 80-100%
this show there is great improvement in their performance and learning.
 Teachers’ observation regarding students’ behavior change suggested that 64% of the students were able to
apply what they have learned in the program very well in practice.
 90% of the students love the physical education program only 10% of students were not satisfied.
 The rating survey showed that students are highly satisfied with teacher’s style and knowledge while there was
dissatisfaction regarding class contents, program material.

Strength of the program

 1. Students were highly satisfied with the teacher’s knowledge and teaching style that show that teacher’s passion
is well translated into his teaching.
2. Students are highly engaged during the class and are well taught to perform well in the assessment.
3. Students are applying what they are learning in theory into practical practice.
4. Students’ satisfaction level is high which suggested that students found PE program very effective.
5. Sound learning is taking place and is evident from pre and post test results.
6. Inclusive program, the teachers ability to support individual needs of the learner

Weakness of the program

1. The class content is not very good.

2. The parents involvement is limited
3. Practical training practices are too difficult for some students.
4. Location of the practical training need to be improved.

Feedback (3.6)

1. Try and involve parents in the learning as well, for example arrange child parent football match, parent coaching day
etc.
2. Content of the course need to be less theory and more practical. The theory part of the sessions needs to be reduced.
3. Continue with the same instruction method and helping individual students with their learning needs.
4. Finding alternate training method for the students who find training too difficult.

Thankyou

Mohan Lal Arya

Reference

Wiggins, G.P. (1993). Assessing Student Performance: Exploring the Purpose and Limits of Assessment. Jossey-Bass
education series, Vol. 1. San Francisco, CA: Jossey-Bass.

The Four Levels: An Overview” (pp. 21-26), Evaluating Training Programs: The Four Levels.

“5. Evaluation Models, Approaches, and Designs” Building Evaluation Capacity

https://www.cdc.gov/healthyyouth/evaluation/