+

Evaluation of Learning

Lesson 4

The Concept of Evaluation

 Improvement of the educational program Achievement of educational goals Motivation and guidance of learning of the individual student Motivation of the teacher to evaluate critically her teaching practices Motivation of the faculty to work together for the improvement of the curriculum

Why is evaluation important?

To determine what the student has learned To inform the instructor to what extent and how materials and courses have been presented and whether students have achieved the established objectives

Why is evaluation important?

Helps identify the strength and weakness in educational objectives and in the curriculum Serves to motivate teachers and students alike

Purpose of Evaluation
Identification of strengths and weaknesses in the objectives of an educational program. Identification of areas to be improved in the curriculum and in the course content. Identification of weaknesses and strengths in students performance.

Purpose of Evaluation

Identification of weakness that may exist in the quality, the process, and the product of teaching Establishment of a basis for motivating both teachers and students improvement.

Major Purposes of Evaluation

1. Method of acquiring and processing the evidence needed to improve students learning and teaching. 2. Great variety of evidence beyond the usual final paper and pencil examination. 3. Aid in clarifying the significant goals and objectives of education

Major Purposes of Evaluation

4. System of quality control

5. Tool in educational practice

EVALUATION Measurement Aspect Quantitative Uses tests and other measuring devices Valuation Aspect Qualitative Utilizes standards i.e. criteria or norm

For obtaining data on learning outcomes or gains

For judging whether such gains are on acceptable level or not

Complete evaluation program includes:

Evaluation of educational objectives Evaluation of teaching of learning procedures Evaluation of students progress Evaluation of outcome

 A. Quantitative gathering of data on student learning in terms of scores in a test.

Measurement an appraisal in terms of some fixed and absolute standard.

 B. Qualitative judgement as to the acceptability or non-acceptability of the learning level based on present standard. C. Summative refers to the extent to which the learner has realized all of specified behavioural objectives.

 Matching Principle it should assess the behaviour to be performed and the degree of performance against the behaviour and degree specified in the objective

 Evaluation is important for the assessment of the growth of learners; of the teacher; of the teaching-learning process; and growth of the program and institution offering the program.

 1. Sampling of objectives all of the objectives in each course must be tested. 2. Sampling course content since it is not possible for the teacher to test student in the entire course content, the teacher must resort to sampling.

 3. Validity the validity of the test is the degree to which it measures what is intended to measure. 3 requirements of a valid test are: relevance to the test proper representation of the elements of the thing knowledge/skill being tested and reliable 4. Reliability accuracy of measurement

 5. Test practically refers to the development of evaluation devices capable of being administered and scored with reasonable ease and within the limits of time and of resources imposed by circumstances 6. Test usefulness tests may be used for pretesting, for diagnosis of students abilities, for measuring speed, power, and comprehensiveness and for many other purposes.

 The presentation will dwell on the three stages of the process:

BEFORE (preparation of test instrument) DURING (administration of the instrument) AFTER (utilization of the results)

 A. Making a table of specification

the design or blueprint that serves as a guide to the test.

Advantages:
It ensures an adequate coverage of instructional objectives and content taken up within a certain prescribed time frame, say, one academic quarter. It enables the test constructor to determine which of the determine content covered within the period need to be given emphasis and which could be glossed over. It helps in attaining an equitable distribution of the skills measurement.

 Remembering the ability to simply recall previously learned information, facts, ideas and principles.
Comprehending the ability to restate or interpret previously learned information. Thinking a more or less complex intellectual activity involving analysis, synthesis and judgement.

 1. Non-Essay or Objective Type

It generally requires a succinct answer and lend itself to the lower cognitive level objectives.

 A. It allows for a broader, albeit superficial, coverage of subject matter.

B. It is appropriate for checking knowledge of a factual or informational nature. C. It is less prone to subjective interpretation of the response. D. It is easier to correct although it may be time-intensive in its preparation

 2. Essay Type
calls for more extended response and may be referred in certain instances due to the following advantages:
a. It allows a deeper treatment of subject matter b. It is useful for testing the higher levels of thinking and reasoning c. It can be used to check the students ability to organize and express his ideas. d. It is easier to contrast although the checking of responses maybe tedious and time consuming

Objective Type Essay Type Supply type Unrestricted type *Short answers *Completion or Fill in the blanks Selection Type Restricted type *Alternative response *Matching Test *Multiple Choice

 Short answer type

This type uses a direct question to which a response in one or two sentences is expected. Ex. What are the characteristics of a malignant cancer cell?

 Completion type/Filling the blanks

In this type, there is a statement that has to be completed with an appropriate word phrase. Ex. Philippine Nursing Association was founded in the year ______.

 Alternative answer type

This type limits the answer to only two options in a forced-choice situation.

Ex. ___ Cardiovascular disease is the leading cause of mortality in the Philippines. True or false

 Matching test type

This is useful for checking the ability of the student to associate factual information.

Multiple choice type

It has an advantage over other selection type tests since it gives the students more options to choose from.

 Unrestricted or extended
The student has almost complete freedom in responding. Ex. What is the importance of sterile technique in the hospital setting?

Restricted
Some limitations are put forth on the nature and length of the students response. Ex. What are the two advantages and disadvantages of sterile technique in the hospital setting? Explain each briefly.

 Summative test
Given at the end of a course of instruction within the course such as a semester or quarter.

 The administration of a summative test will take the ff. account: 1. Test schedule - the administrative decisions should be in consultation with the faculty. Students and parents should be appraised of the schedule ahead of time. 2. Test condition - students should be properly conditioned for the tests.

 1. Checking test results

2. Translating test scores into grades

3. Reporting student's achievement 4. Making administrative decisions

Steps in Evaluation:
1. Stating objectives 2. Defining changes in behavior expected as educational outcomes 3. Listing and briefly describing situations that give the opportunity for the expression of the behavior desire 4. Developing appropriate and systemic means of eliciting kinds of behavior implied objectives to be evaluated

Steps in Evaluation:
5. Deciding on ways of recording and summarizing (scoring, rating or describing) behavior and the basis of evidence collected 6. Checking validity, reliability and difficulty of the measures used. 7. Establishing conditions that permit the student to give her best performance 8. Assigning scores on the basis of the above steps 9. Developing methods of interpretation

 Methods of Evaluation:
A. Classroom B. Paper and pencil tests

The following are the principles of construction: 1. The directions for answering the questions must be exact
2. The truth or falsity should be expressed between subject and verb, never by a phrase tacked on at the end or by inserting an incidental phrase in the question

3. The statement itself should be confined to one idea

4. Specific determinate such as always, never, usually, generally, should be avoided.

 5. There should be approximately the same number of false and true items
6. A double space between items provides for easier reading 7. Modifications can help to eliminate guessing. Suggestions include:
A. Explanation of false statement B. statement of reason for choice C. Correction and/or revision of incorrect statement

Example: Behavioral Objectives

Directions: For each of the following statements circle the T if the statement is true and F if the statement is false. T.F. Consultation and patient interview are two primary sources of data collection relative to patients needs. T.F. A decrease in breath sounds occurs in a condition, such as atelectasis.
T.F. Endocarditis is an inflammation of the heart muscle.

a. The question consists of a statement (stem) and a list of responses (alternatives or distracters). b. When the stem is an incomplete sentence, all alternatives should correctly complete the sentence grammatically. c. The articles a or b should not be used to introduce alternatives

d. Alternatives should appear at the statement.

e. In order to obtain a reasonable degree of item difficulty, at least four alternatives should be given. However it is better to use fewer than include a poor one. f. Irrelevant answers or one that gives clues should be avoided.

 Blank used at the end of a sentence simplify scoring.

Too many blanks in a statement should be avoided as the question could become a puzzle. Key words are the one to be omitted.

The use of a or an immediately preceding a blank should be avoided.

Only one answer should be possible for each blank.

 A. Terms used should be stated clearly. B. More choices should be listed in the right hand column than things to be identified in the left hand column. C. The terminology on the left-hand column should not give clues to the expected response on the right hand column. D. If each item on the right-hand column can be used more than once, this information should be included in the instructions.

 1. The question should be constructed carefully so that the nature of the task is clearly stated. 2. It should be constructed carefully so the exact limits of areas are clearly defined. 3. The desired outcomes of the area tested should be set up as a basis for grading.

 4. The students must know the basis of evaluation, especially where grading is to follow the evaluation. The basis might include:
A. Evidence of meeting intended behaviors

B. Evidence of the ability to select, organize and present material in a coherent manner.

 1. Knowledge - list, in order, the stages of the grief process. 2. Comprehension - predict the effect of smoke on health of a person with emphysema.
3. Application - relate Erikson's crisis theory to the determination of the type of nursing interview you would use in meeting the needs of parents with an acutely ill child.

 4. Analysis - analyze one of the proposals for national health insurance and identify the assumptions underlying the major points in the plan. 5. Synthesis - write an essay in which you present your reaction to the statement "Health is a human right. 6. Evaluation - select a report taken from the "Nursing Research Journal". Compare this report with another report in the same area that is recognized as valid and reliable study.

General Principle of Construction:

A. The problem solving situation should be geared to the learners knowledge and experience B. Information about the incident or situation should be sufficient to assure clarity in the presentation.

 C. The situation must be reasonable in length so that the learner is not required to devote a long period of time reading and re-reading the description. D. The critical incidence needs to be sharply presented with no extraneous material included. E. Questions posed must be related directly to the data incorporated in the descriptions.

Situation:
Ms. J, a 20 year-old mother of 3 children, ranging in age from 6 months to 3 years, lives with her mother and younger brother in a 3-room tenement in the squatter area of the city. Never married, Ms. J . has been receiving assistance from the government since the birth of her first child. Last evening, Ms. J. Was brought to the emergency ward of the hospital in acute distress from heroin overdose. She is now a patient in the hospital ward and you have been assigned to provide her with nursing care.

 What are your feeling about Ms. J. As you prepare to plan her care? What stressors do you identify in her lifes situation as described here? Explain the dynamics of this patient family situation on the basis of the stress adaptation theory. What is your opinion about the way Ms. J. Is handling her stressors?

 Viewed as the way or the medium through which a professional practitioner minister to his client.

 1. To determine the background ability of each student brings to the situation. 2. To determine the progress by each student at frequent intervals during the assignment. 3. To determine the ability of students to use nursing theory as a whole while progressing in the development of clinical skills.

 4. To discover learning difficulties of individual students and the group in order to adjust the teaching. 5. To foster the development of self-evaluation. 6. To determine the effectiveness of teaching techniques. 7. To obtain data for conducting research studies relating to students achievement.

 I. Observation
an integral part of the daily teaching-learning process.

A. Anecdotal Notes consist of informal description of observation of behaviour and activities of the learner during a particular performance at a specific short duration.
Five Essential Elements of anecdotal notes includes: Identification of student The date and time of observation The narrative report Signature The role of observer

B. Critical Incident Technique is used to assess the students clinical performance and not to lend itself to a definite grade assignment.
Example: Behavioral Objective
Shares own assessment of patients needs with colleagues in the nursing team.

C. Rating Scale represents a standardized method of recoding qualitative and quantitative judgement of observed performance. D. Checklist developed by listing all the steps necessary to perform the skills adequately, then the student performs the skills, the nurseteacher check off each step on when it occurs.

 Forced choice rating

Self-rating scale
Ranking

 Best use a basis for testing students abilities to apply theoretical knowledge to lifelike situations.
Sources: Nurses notes an integral part of the total care of the patient. Not some routine activity to be accomplished after each shift. Problem oriented method centered around patients health problem

 Nursing care studies students undertake the comprehensive assessment for patients problem leading to planning, implementing and evaluation of nursing care measures.

Process recording serial reproduction of verbal and non-verbal communication between two individuals to assess interpersonal relationships.

 Conveyance of ideas and thinking via spoken words.

Sources: Nursing Patients care conference Nursing team conference

Group Project
Point to remember 1. Evaluation of group projects should provide opportunity for assessing the group as a whole as well as individual performance within the group. 2. When group presentation is given before a class the evaluation form is completed immediately after the presentation while the perception of the process is clear in the mind of evaluators.

 It should be a cooperative venture between student and the teacher who continually seek opportunity for professional growth and development based on mutual respect and understanding. There are five significant reasons for providing diversities in evaluation strategies:

 1. Complexity of human behavior 2. Individual differences in response to learning 3. Suitability of specific evaluation approaches to specific types of learning behaviors. 4. Motivational factor of evaluation 5. Creative dimension to the evaluation process.

 It initiates context evaluation and therefore the evaluation of the entire educational process.

 Formative Program evaluation consists of the periodic feedback arrangements.

Summative Evaluation comes at the end of the whole process and deals directly with the assessment of the total curriculum plan.

 To determine the worth of the curriculum as well as the program in general.

To make decision as to whether to continue, terminate, modify, refocus a program and other authors suggest that the function is to determine whether to improve, maintain, or terminate a program - Stufflebeam

 Context represents the environment, both internal and external to the school. Input refers to curriculum content and learning objectives used in the program. Process - consists of the wide range of learning strategies and resources used in the program or the mechanism of implementation. Product - refers to the learning gains of the students are revealed by the tests and evaluated the school standards.

 Stage I
compliance to change, goal of change agents is to create within the population (faculty) the motivation to change. Then faculty allows itself to the influence.

 Stage II
identification with the change.

Stage III
internalization of change or refreezing.

 Evaluation is continuous and forever. Change is never ending. Change is dynamic. It is a continuous process.

The pattern of courses and the selected learning activities must: 1. Derive its theory from and test its theory in the practice of nursing in a real community of health need. 2. Cope with the knowledge explosion and the short half-life of scientific knowledge. 3. Use the logical, precise, effective and efficient educational technology of students.

4. Respond to the health needs of society and the immediate concern of students. 5. Use teaching personnel in the most economical and efficient way (time, energy, money). 6. Be consistent with the theoretical framework and implement the theoretical framework commitments.

7. Enable simulated utilization of nursing practice implication of cognitive material. 8. Provide for student testing of learning behaviours in reality situation. 9. Produce a graduate capable of creative nursing care for the next 15 to 29 years. 10. Spend a reasonable length of time accomplishing the goals of the curriculum.

Thank you for listening!