ASSESSING STUDENT ("4 LEARNING OUTCOMES ¥ LEARNING OUTCOMES > : ; Clarify the principles in assessing learning outcomes. » Explain the phases of outcomes assessment > Determine alignment of learning outcomes and assessment tasks > Discuss various assessment methods, tools and tasks including portfolios > Construct a scoring rubric » Give sample assessment task for each MI Explain at least 3 key features of assessment in the K to 12 Program INTRODUCTION Outcome assessment is the process of gathering information on whether the instruction, services and activities that the program provides are producing the desired student learning outcomes. 4.1. Principles of Good Practice in Assessing Learning Outcomes 1, The assessment of student learning starts with the institution’s vision, mission and core values. There should be a clear statement on the kinds of learning that the institution values most for its students. 2. Assessment works best when the program has clear statement of objectives aligned with the institutional vision, mission and core values. Such alignment ensures clear, shared and implementable objectives. 3. Outcome - based assessment focuses on the student activities that will still be relevant after formal schooling concludes. The approach is to design assessment activities which are observable and less abstract such as “to determine the student’s ability to write a paragraph” which is more observable than “to determine the student’s verbal ability.” 43Assessment oF LearNne Outcomes (Assessment 1) Assessment requires attention not only to outcomes but also and equally to the activities and experiences that lead to the attainment of learning outcomes. These are supporting student activities. ‘Assessment works best when it is continuous, ongoing and not episodic. Assessment should be cumulative because improvement is best achieved through a linked series of activities done over time in an instructional cycle. Begin assessment by specifying clearly and exactly what you want to assess. What you want to assess is/are stated in your learning outcomes/lesson objectives. The intended learning outcome/lesson objective NOT CONTENT is the basis of the assessment task. You use content in the development of the assessment tool and task but it is the attainment of your learning outcome NOT content that you want to assess. This is Outcome-based Teaching and Learning. Set your criterion of success or acceptable standard of success. It is against this established standard that you will interpret your assessment results. Example: Is a score of 7 out of 10 (the highest possible score) acceptable or considered success? Make use of varied tools for assessment data-gathering and multiple sources of assessment data. It is not pedagogically sound to rely on just one source of data gathered by only one assessment tool. Consider multiple intelligences and learning styles. DepED Order No. 73, s. 2012 cites the use of multiple measures as one assessment guideline. Learners must be given feedback about their performance. Feedback must be specific. “Good work!” is positive feedback and is welcome but actually is not a very good feedback since it is not specific. A more specific better feedback is “You observed rules on subject-verb agreement and variety of sentences. Three of your commas were misplaced.” _ Assessment should be on real-world application and not on out- of-context drills. . Emphasize on the assessment of higher-order thinking. . Provide opportunities for self-assessment.Chapter 4 — Assessing Student Learning Outcomes ik 4.2. Samples of Supporting Student Activities Student Learning Outcome #1: Students can organize information from secondary sources as basis of a research topic. Supporting Student Activities practise differentiating source material and one’s opinion reading articles and formulating an original paragraph from quotes, paraphrases and summaries 1.3. writing of essays to develop the topic 1.4, integrating bibliographic entries in appropriate format Student Learning Outcome #2: Students apply principles of logical thinking and persuasive argument in writing. Supporting Student Activities 2.1, forming opinion about the topic 2.2. researching and writing about a variety of perspectives 2.3. adapting style to the identified audience 2.4. employing clear argument in writing Student Learning Outcome #3: Students write multiple-page essays complying with standard format and style s a oa 3.1. analyzing and evaluating texts 3.2. writing about a variety of perspectives on single topic 3.3. adapting tone and style to address one’s audience 3.4. reviewing grammar and essay format in readings 3.5. holding group discussion about various topics ad‘Agsesswenr oF Learnne Ourcowes (Assessvent 1) ae Inst ional sa, B ‘Summative c Assessment of| ~ Desired Student Outcomes Learning Outcomes Program Subject Goals | objectives ? a a Mastery Learning| ‘Diagnostic Assessment Review/Reteach \ Deciding on Lesson Focus Assessment of Outcomes SS Sole a Student Activities Figure 6. Outcome Assessment in the Instructional Cycle Formative 4.3. Phases of Outcome Assessment in the Instructional Cycle Study the phases of outcome assessment in the instructional cycle as shown in the Figure above then as a group answer the following questions: On which is assessment on the institutional cycle based after the vision mission of an institution, program goals and subject objectives? 2. What is the purpose of diagnostic assessment? 3. Based on the institutional cycle, how can you ensure mastery learning? 4. How does formative assessment of outcomes differ from the summative assessment of outcomes? 5. Why does summative assessment of outcomes point to program goals and subject objectives?Chapter 4 ~ Assessing Student Leaming Outcomes ] 4.4, Constructive Alignment tae : Below is another diagram that illustrates the principle of constructive alignment in the assessment process. Study it well. What is the main message of Figure 7. Learning Outcome Teaching- Learning Activities Assessment Task Figure 7. Constructive Alignment Figure 7 illustrates the principle of constructive alignment. The principle of constructive alignment simply means that the teaching- learning activity or activities and assessment tasks are aligned with the intended learning outcome. The intended learning outcome is “to drive a car.” The teaching-leaming activity is driving a car not giving lectures on car driving. The assessment task is to let the student drive @ car not to describe how to drive a car. You have been victims of teachers who taught you one thing but assessed you on another. The result? Much confusion and disappointment? If you have been victims of lack of constructive alignment, then break the cycle by not victimizing your students, too. Observe the principle of constructive alignment. Make sure your assessment tasks are aligned with your learning outcomes. Why the term “constructive?” Constructive alignment is based on the constructivist theory (Biggs, 2007) that learners use their own activity to construct their knowledge or other outcome/s.‘Assesswenr oF Leanwne Outcoues (Assessvenr 1) A Mental Exercise In this mental exercise, determine whether or not the assessment task is aligned with the learning outcome. If the assessment task is not aligned with the learning outcome, improve it to align it with the learning outcome. Assessment Task Trace the history of tango What is your favorite line in the poem? Why is it your favorite line? Interpret a given poem Demonstrate how to do a PowerPoint presentation Present a report with PowerPoint Derive the meaning of at least | Match the meaning of the words 5 words by means of context In Column 1 with those in clues Column 2 Determine what are given and Solve a given problem what is asked Encircle the words with short a sound Pronounce short a sound correctly Trace the historical development | Trace the historical development of the Philippine basic education | of the Philippine basic education curriculum curriculum with the use of an appropriate graphic organizer 4.5, Variety of Assessment Methods, Tools and Tasks Assessment methods ciin be classified as traditional’and duthentic. ‘Traditional assessment method refers to the usual paper-and-pencil test while authentic assessment refers to non-paper-and-pencil test, Authentic assessment is also called alternative assessment, it being an alternative to the traditional. ‘The paper-and-pencil test (traditional assessment) assesses learning in the cognitive domain (Bloom) or declarative knowledge (Kendall and Marzano, 2012). ; ‘The paper-and-pencil test, however, is inadequate to measure all forms of learning, Psychomotor learning (Kendall and Marzano, 2012) or procedural knowledge (Kendall and Marzano, 2012) and learning‘Chapter 4 ~ Assessing Student Leaming Outcomes proven by a product and by a performance cannot be measured by a paper-and-pencil test. Assessment tools for the cognitive domain (declarative knowledge) are the different paper-and-pencil tests. Basic examples of paper-and pencil tests are shown in Figure 8. Bye tora ceo) eN etl yee ied -response Completion Essay- restricted or non-restricted Figure 8. Two groups of Written Tests and Specific Examples Examples of selected response type of tests are alternate response (True or False, Yes or No, 4 or 6); matching type and the multiple choice type. Examples of constructed type of tests are the completion type (Fill-in-the-blanks), short answer, the essay test and problem solving. These will be discussed in greater detail in Chapter 5.‘Assesswenr oF Leanne Outcomes (AsseSswent 1) Examples of authentic assessment tools ae the demonstrations of what have been learned by either a product or a performance. (Refer to Figure 9). Product es tasks sie | Product Output sar Construct a Table of Specifications > Construct paper-and-pencil tests in accordance with the guidelines in test construction INTRODUCTIOI Tn this Chapter we are concerned with developing paper-and- paper tests for assessing the attainment of educational objectives based on Bloom’s taxonomy. Paper-and-pencil test can either be selected response or constructed-response types. Selected response type includes (a) true-false items, (b) multiple-choice type items and (c) matching type. Constructed-response type of test includes (a) enumeration, (b) completion and (c) essays. The construction of valid test items begins with a Table of Specifications. 5.1 Planning a Test and Construction of Table of Specifications (TOS) The important steps in planning for a test are: Identifying test objectives/lesson outcomes Deciding on the type of objective test to be prepared Preparing a Table of Specifications (TOS) Constructing the draft test items ‘Try-out and validation 69‘Assesswenr oF Leanne Outcomes (AssesswenT 1) Identifying Test Objectives. An objective test, if it is to be comprehensive, must cover the various levels of Bloom’s taxonomy. Each objective consists ofa statement of what is to be achieved preferably by the students. Example. We want to construct a test on the topic: “Subject-Verb Agreement in English” for a Grade V class. The following ate typical objectives: Knowledge/Remembering. The students must be able to identify the subject and the verb in a given sentence. Comprehension/Understanding. The students must be able to determine the appropriate form of a verb to be used given the subject of a sentence. ‘Application/Applying. The students must be able to write sentences observing rules on subject-verb agreement. ‘Analysis/Analyzing. The students must be able to break down a given sentence into its subject and predicate. Evaluation/Evaluating. The students must be able to evaluate whether or not a sentence observes rules on subject-verb agreement. Synthesis/Creating. The students must be able to formulate mules to be followed regarding subject-verb agreement. Deciding on the type of objective test. The test objectives guide the kind of objective tests that will be designed and constructed’by the teacher. This means aligning the test with the lesson objective/outcome. For instance, for the first four (4) levels, we may want to construct a multiple-choice type of test while for application and judgment, we may ‘opt to give an essay test or a modified essay test. At all times, the test to be formulated must be aligned with the learning outcome. This is the principle of constructive alignment. Preparing a table of specifications (TOS). & Table of Specifications or TOS is a test map that guides the teacher in constructing a test. The TOS ensures that there is balance between items that test lower level thinking skills and those which test higher order thinking skills (or alternatively, a balance between easy and difficult items) in the test: The siinplest TOS consists of four (4) columns: (a) level of objective to be tested, (b) statement of objective, (c) item numbers where such an objective is being tested and (d) number of items and percentage out of the total for that particular objective. A prototype table is shown below:Chapter 5 Distinguishing and Constructing Various Paper-and-Pencl Test n ’ Table of Specifications Prototype ITEM cence [ula =| = | Identify subject-verb 2. Understanding | Form appropriate Write ‘sentences observing rules on subject-verb agreement Determine subject and predicate Evaluate whether or not a sentence 5. Evaluating observes rules ‘on subject-verb agreement Formulate rules (on subject-verb | | ao | 51% agreement [rom [a5 00% | In the Table of Specifications we see that there are five items that deal with knowledge and these items are items 1,3,5,7,9. Similarly, from the same table we see that five items represent analysis, namely: 11, 15, 18, 21, 23. The first five levels of Bloom’s taxonomy are equally represented in the test while synthesis (tested through essay) is weighted equivalent to ten (10) points or double the weight given to any of the first four levels. The Table of Specifications guides the teacher in formulating the test. As we can see, the TOS also ensures that each of the objectives in the hierarchy of educational objectives is well represented in the test. As such, the resulting test that will be constructed by the teacher will be more or less comprehensive. Without the Table of Specifications, the tendency for the test maker is to focus too much on facts and concepts at the Recall level. Constructing test items based on a TOS ensures alignment of learning outcomes and assessment tasks. Constructing the test items. The actual construction of the test items follows the TOS. As a general rule, it is advised that the‘Assessuenr oF Learwne Outcones (Assessuenr 1) actual number of items to be constructed in the draft should be double the desired number of items, For instance, if there are five (5) recall level items to be included in the final test form, then at least ten (10) recall level items should be included in the draft. The subsequent test try-out and item analysis will most likely eliminate many of the constructed items in the draft (either they are too difficult, too easy or non-discriminatory), hence it will be necessary to construct more items than will actually be included in the final test form. Most often, however, the try-out is not done due to lack of time. Item analysis and try-out, The test draft is tried out to a group of pupils or students. The purpose of this try-out is to determine the: (a) item characteristics through item analysis, and (b) characteristics of the test itself-validity, reliability and practicality. Mental Exercise Construct a ToS for the test grading period. You may choose any subject and base your ToS on the test quarter of a subject of your choice from the K to 12 Curriculum Guide. 5.2 Types of Paper-and-Pencil Test Development of paper-and-pencil tests requires careful planning and expertise in terms of actual test construction. The more seasoned teachers can produce true-false items that can test even higher order thinking skills and not just rote memory learning. Essays are easier to construct than the other types of objective test, but the difficulty in scoring essay examinations teachers from using this particular form of examination in actual practice. 5.3. Constructing Selected-Response Type 5.3.1 True-False Test Binomial-choice or alternate response tests are tests that have only two (2) options such as true or false, right or wrong, yes or no good or better, check (4) or cross out (6) and so on. A student who knows nothing of the content of the examination would have 50% chance of getting the correct answer by sheer guess work. Although correction-for-guessing formulas exist, it is best that the teacher ensures that a true-false item is able toChapter 5 - Distinguishing and Constructing Various Paper-and-Pencil Test discriminate properly between those who know and those who are just guessing. A modified true-false test can offset the effect of guessing by requiring students to explain their answer and to disregard a correct answer if the explanation is incorrect. Here are some rules of thumb in constructing true-false items. Guidelines for Constructing Alternate-Response Test Rule 1. Do not give a hint (inadvertently) in the body of the question, Example. The Philippines gained its independence in 1898 and therefore celebrated its centennial year in 2000 Obviously, the answer is FALSE because 100 years from 1898 is not 2000 but 1998, Rule 2. Avoid using the words “always,” “never,” “often” and other words that tend to be either always true or always false. Example: Christmas always falls on a Sunday because it is a Sabbath day. Statements that use the word “always” are almost always false. A test-wise student can easily guess his way through a test like these and get high scores even if he does not know anything about the test. Rule 3. Avoid long sentences as these tend to be “true.” Keep sentences short. Example: Tests need to be valid, reliable and useful, although, it would require a great amount of time and effort to ensure that tests possess these test characteristics. Notice that the statement is true. However, we are also not sure which part of the sentence is deemed true by the student. It is just fortunate that in this case, all parts of the sentence are true and, hence, the entire sentence is true. The following example illustrates what can go wrong in long sentences: Example: Tests need to be valid, reliable and useful since it takes very little amount of time, money and effort to construct tests with these characteristics. The first part of the sentence is true but the second part is debatable and may, in fact, be false, Thus, a' /Assesewent oF Learwne Ourcowes (Assesswenr 1) “true” response is correct and, also, a “false” response is correct. ‘ Rule 4. Avoid trick statements with some minor misleading word or spelling anomaly, misplaced phrases, etc. A wise student who does not know the subject matter may detect this strategy and thus get the answer correctly. The Raven was written by Edgar Allen Poe. Allen is misspelled and the answer would be false! This is an example of a tricky but utterly useless item. Rule 5. Avoid quoting verbatim from reference materials or textbooks. This practice sends the wrong signal to the students that it is necessary to memorize the textbook word for word and, thus, acquisition of higher level thinking skills is not given due importance. Rule 6. Avoid specific. determiners or give-away qualifiers. “Students quickly learn that strongly worded statements are more likely to be false than true, for example, statements with “never” “no” “all” or “always.” Moderately worded statements are more likely to be true than false. Statements that are moderately worded use “many” “ofien” “sometimes” “generally” “frequently” or “some” usually should be avoided. e.g. Executives usually suffer from hyperacidity. The statement tends to be correct. The word “usually” leads to the answer. Rule 7. With true or false questions, avoid a grossly disproportionate number of either true or false statements or even patterns in the occurrence of true and false statements. Tle 6.F 457. 6.F get iF ad fot. 3.F 8.F o 37 8.F 4E 9.F 4. F 9T 5..F 10. F 5.T 10. F For ease of correction, teachers sometime create a pattern of True or False answers. Students will sense it and may arrive at a correct answer not because he/she really knows the answer but because he/she senses the pattern.(Chapter 5 — Distinguishing and Constructing Various Paper-and-Pencil Test Rule 8. Avoid double negatives. This makes test item unclear and definitely will confuse the student. e.g. The changes that take place in early childhood are NOT Unchangeable. The test item simply means “The changes in early childhood are changeable”. 5.3.2 Multiple Choice Tests The multiple choice type of test offers the student with more than two (2) options per item to choose from. Each item in a multiple choice test consists of two parts: (a) the stem and (b) the: options. In the set of options, there is a “correct” or “best” option while all the others are considered “distracters.” The distracters are chosen in such a way that they are attractive to those who do not know the answer or who are guessing but at the same time, have no appeal to those who actually know the answer. It is this feature of multiple choice type tests that allows the teacher to test higher order thinking skills.even if the options are clearly stated. As in true-false items, there are certain rules of thumb to be followed in constructing multiple choice tests. Guidelines for Constructing Multiple Choice Items 1) Do not use unfamiliar words, terms and phrases. The ability of the item to discriminate or its level of difficulty should stem from the subject matter rather than from the wording of the question. Example: What would be the system reliability of a computer system whose slave and peripherals are connected in parallel circuits and each one has a known time to failure probability of 0.05? A student completely unfamiliar with the terms “slave” and “peripherals”may not be able to answer correctly even if he knew the subject matter of reliability. 2) Do not use modifiers that are vague and whose meanings can differ from one person to the next such as: much, often, usually, ete. Example: Much of the process of photosynthesis takes place in the: a. bark b. leaf c. stem‘Assessvenr oF Leawwne Ourcones (Assessuent 1) The qualifier “much” is vague and could have been replaced by more specific qualifiers like: “90% of the photosynthetic process” or some similar phrase that would be more precise. Be quantitative. 3) Avoid complex or awkward word arrangements. Also, avoid use of negatives in the stem as this may add unnecessary comprehension difficulties. Example: (Poor) As President of the Republic of the Philippines, Corazon Cojuangco Aquino would stand next to which President of the Philippine Republic subsequent to the 1986 EDSA Revolution? (Better) Who was the President of the Philippines after Corazon C. Aquino? 4) Do not use negatives or double negatives as such statements tend to be confusing. It is best to use simpler sentences rather than sentences that would require expertise in grammatical construction. Example: (Poor) Which of the following will not cause inflation in the Philippine economy? (Better) Which of the following will cause inflation in the Philippine economy? Poor: What does the statement “Development patterns acquired during the formative years are NOT Unchangeable” imply? A. B. Cc. D. Better: What does the statement “Development patterns acquired during the formative years are changeable” imply? a A. B. Cc. D.Chapter 5 — Distinguishing and Constructing Various Paper-and-Pencil Test 5) Each item stem should be as short as possible; otherwise you risk testing more for reading and comprehension skills. 6) Distracters should be equally plausible and attractive. Example: The short story: May Day’s Eve, was written by which Filipino author? a. Jose Garcia Villa b. Nick Joaquin c. Genoveva Edrosa Matute d, Robert Frost e. Edgar Allan Poe Tf distracters had all been Filipino authors, the value of the item would be greatly increased. In this particular instance, only the first three carry the burden of the entire item since the last two can be essentially disregarded by the students. 7) All multiple choice options should be grammatically consistent with the stem. Example: As compared to the autos of the 1960s autos in the 1980s A. traveling slower —_C. to use less fuel B. bigger interiors D. contain more safety measures Option A, B and C are obviously wrong for the language smart because when added to the stem the sentence is grammatically wrong. D is the only option which when connected to the stem retains the grammatical accuracy of the sentence, thus obviously is the correct answer. fe 8) The length, explicitness, or degree of technicality of alternatives should not be the determinants of the correctness of the answer. The following is an example of this rule: eT Example: If the three angles of two triangles are congruent, then the triangles are: a. congruent whenever one of the sides of the triangles are congruent‘Assessuent oF Learns Ourcoues (Assessuenr 1) 78 b. similar c. equiangular and/therefore, must also be congruent d. equilateral if they are equiangular The correct choice, “b,” may be obvious from its length and explicitness alone. The other choices are long and tend to explain why. they must be the correct choices forcing the students to think that they are, in fact, not the correct answers! 9) Avoid stems that reveal the answer to another item. Example: a, Who will most strongly disagree with the progressivist who claims that the child should be taught only that which interests him and if he is not interested, wait till the child gets interested? A, Essentialist C. Progressivist B, Empiricist D, Rationalist b. Which group will most strongly focus its teaching on the interest of the child? A. Progressivist C. Perrenialist B. Essentialist D, Reconstructionist One may arrive at a correct answer (letter b) by looking at item “a,” that gives the answer to “b.” 10) Avoid alternatives that are synonymous with others or those that include or overlap others. Example: What causes ice to transform from solid state to liquid state? a. Change in temperature b. Changes in pressure c. Change in the chemical composition d. Change in heat levels The options “a” and “d” are essentially the same. Thus, a student who spots these identical choices would right away narrow down the field of choices to a, b,‘Chapter 5 ~ Distinguishing and Constructing Various Paper-and-Pencl Test and c. The last distracter would play no significant role in increasing the value of the item. + “If this happens then the item has two answers, which is not acceptable. 11) Avoid presenting sequenced items in the same order as in the text. 12) Avoid use of assumed qualifiers that many examinees may not be aware of: 13) Avoid use of unnecessary words or phrases which are not relevant to the problem at hand (unless such discriminating ability is the primary intent of the evaluation). The item's value is particularly damaged if the unnecessary material is designed to distract or mislead. Such items test the student's reading comprehension rather than knowledge of the subject matter. Example: The side opposite the thirty degree angle in a right triangle is equal to half the length of the hypotenuse. If the sine of a 30-degree is 0.5 and its hypotenuse is 5, what is the length of the side opposite the 30-degree angle? a 2:5 bas. Cup Se d 15 The sine of a 30-degree angle is really quite unnecessary since the first sentence already gives the method for finding the length of the side opposite the thirty-degree angle. This is a case of a teacher who wants to make sure that no student in his class gets the wrong answer! 14) Avoid use of non-relevant sources of difficulty such as requiring a complex calculation when only knowledge of a principle is being tested. Note in the previous example, knowledge of the sine of the 30-degree angle would have led some students to use the sine formula for calculation even if a simpler approach would have sufficed. Mi Bat aici as: ay agAssessvenr oF Leann Ourcones (ASSESSvENT 1) 15) Pack the question in the stem. Here is an example of a question which has no question. Avoid it by all means. Example: The Roman Empire a. had no central government _ b. had no definite territory c. had no heroes d. had no common religion ~ 16) Use the “None of the above” option only when the keyed answer is totally correct. When choice of the “best” response is intended, “none of the above” is not appropriate, since the implication has already been made that the correct response may be partially inaccurate. 17) Note that use of “all of the above” may allow credit for partial knowledge. In a multiple option item, (allowing only one option choice) if a student only knew that two (2) options were correct, he could then deduce the correctness of “all of the above.” This assumes you are allowed only one correct choice. 18) Better still use “none of the above” and “all of the above” sparingly but best not to use them at all. 19) Having compound response choices may purposefully increase difficulty of an item. The difficulty in a multiple choice item may be controlled by varying the homogeneity or degree of similarity of responses. The more homogeneous, the more difficult the item because they all look like the correct answer. Example: (Less Homogeneous) Thailand is located in: Southeast Asia . Eastern Europe South America = East Africa Central America sposeChapter 5 - Distinguishing and Constructing Various Paper-and-Pencll Test (More Homogeneous) __Thailand is located next to: a, Laos and Kampuchea b. India and China c. China and Malaya d. Laos and China e. India and Malaya Mental Exercise Write - Pair - Share - Critique 1. Construct a multiple - choice test item. Pair with a classmate. Partner A shares multiple - choice test item. Partner B critiques test item. Partner B shares test item. Partner A critiques. NOTE: If " you can’t agree consult teacher, aAkeD 5.3.3 Matching Type The matching type items may be considered modified multiple choice type items where the choices progressively reduce as one successfully matches the items on the left with the items on the right. Guidelines for Constructing Matching Type of Test Here are some guidelines to observe in the formulation of good matching type of test. 1. Match homogeneous not heterogeneous items. The items to match must be homogeneous. If you want your students to match authors with their literary works, in one column will be authors and in the second column must be literary works. Don’t insert nationality for instance with names of authors. That will not be a good item since it is obviously wrong. Example of homogeneous items. The items are all about the Filipino heroes, nothing more. Match the items in Column A with the items in Column B,‘Assessuent oF Lexus Outcomes (Assessuenr 1) Perfect Matching Type Example: Match the items in column A with the items in column B, A B __1. First President of the Republic a. Magellan __2. National Hero b. Mabini __3. Discovered the Philippines c. Rizal __4, Brain of Katipunan: d. Lapu-Lapu __5. The great painter e. Aguinaldo ___6. Defended Limasawa island f, Juan Luna g. Antonio Luna bad The stem (longer in construction than the options) must be in the first column while the options (usually shorter) must be in the second column. The options must be more in number than the stems to prevent the student from arriving at the answer by mere process of elimination. wo To help the examinee find the answer easier, arrange the options alphabetically or chronologically, whichever is applicable. ‘ > v Like any other test, the direction of the test must be given. The examinees must know exactly what to do. Mental Exercise ‘Analyze the matching type of test below. Is this perfect (an answer may not be repeated) matching type of test written in accordance with the guidelines given? Exercise’ Matching Type of Test Column A Column B 1, Poly A. Sides 2. Triangle B. Eight-sided polygon 3. Pentagon ©. Ten-sided polygon 4, Square D. Close plane figure 5. Decagon E. Irving : 6. Hexagon F. James 7. Isosceles triangle G. Melville * 8. Octagon H. Mark Twains (Clemens) 9. Gons |. Wharton 10. Circle J. Many(Chapter 5 ~ Distinguishing and Constructing Various Paper-and-Pencll Test Matching type-items, unfortunately, often test lower order thinking skills (knowledge level) and are unable to test higher order:thinking skills such as application and judgement skills. Another type of a matching type of test is the imperfect type. Below is an example of an imperfect matching type of test, Imperfect because an answer may be repeated and so like an unfaithful husband or wife can pair with one other than his/her lawful wife, husband. In Column 1 are works and writings in American literature and in Column 2 are their authors. Write the letter of the author which corresponds to his work on the blank provided before each author. In some cases, an answer may be repeated. Imperfect Matching Type of Test Column A Column B 1. The Alhambra A. Cooper 2. The Pioneers B. Dana 3.The Guardian Angel C. Emerson 4. Two Years Before the Mast D. Holmes 5. Moby Dick f E. Irving 6. The World in a Man of War F. James 7. The Last of the Mohicans G. Melville 8. The American Scholar H. Mark Twains (Clemens) 9. The Autocrat of the Breakfast Table | |. Wharton 10. Tom Sawyer If you intend to make use of this imperfect type of matching test, make sure you indicate so in the “Direction” to caution the students who usually think that an answer may not be repeated. Mental Exercises Get a partner. Construct a 15-item perfect matching type of test and a 10-item imperfect matching type of test. Present your work to the class for critiquing.‘Assessvent oF Leanne Outcoues (Assessvent 1) 5.4 Constructing Supply Type or Constructed- Response Type Another useful device for testing lower order thinking skills is the supply type of tests. Like the multiple choice test, the items in this kind of test consist of a stem and a blank where the students would write the correct answer, Example: The study of life and living organisms is called Supply type tests depend heavily on the way the stems are constructed. These tests allow for one and only one answer and, hence, often test only the students’ recall of knowledge. 5.4.1 Completion Type of Test It is, however, possible to construct supply type of tests that will test higher order thinking as the following example shows: Example: Write an appropriate synonym for each of the following. Each blank corresponds to a letter: Metamorphose: Flourish: _ The appropriate synonym for the first is CHANGE with six (6 letters while the appropriate synonym for the second is GROW with four (4) letters. Notice that these questions require not only mere recall of words but also understanding of these words. Another example of a completion type of text that measures higher order ~ thingking skill is given below: Example: Write G if the item on the left is greater than the item on the right; L if the item on the left is less than the item on the right; E if the item on the left equals the item on the right and D if the relationship cannot be determined. A -B 1. Square root of 9 _- a.-3 2. Square of 25 b. 615" 2 3. 36 inches c. 3.meters 4.4 feet d. 48 inches 5. 1 kilogram e. 1 poundChapter 5 — Distinguishing and Constructing Various Paper-and-Pencl Test Guidelines for the Formulation of a Completion Type of Test The following guidelines can help you formulate a completion type of test, the fill-in-the blank type. 1. Avoid overmutilated sentences like this test item. Give enough clue to the student. The produced by the is used by the green to change the and into . This process is called Avoid open-ended item. There should be only one acceptable answer. This item is open-ended, hence no good test item. Ernest Hemingway wrote The blank should be at the end or near the end of the sentence. The question must first be asked before an answer is expected. Like the matching type of test, the stem (where the question is packed) must be in the first column: Ask question on more significant item not on trivial matter. Jose Rizal was born on June __, 1861. There are other more significant items to ask other than specific birthdates. The length of the blanks must not suggest the answer. So better to make the blanks uniform in size. A part of speech that names persons, places or things A word used to connect clauses or sentences or to coordinate words in the same clause is called 5.4.2 Essays Essays, classified as non-objective tests, allow for the assessment of higher order thinking skills. Such tests require students to organize their thoughts on a subject matter in coherent sentences in order to inform an audience. In essay tests, students are required to write one or more paragraphs on a specific topic. Essay questions can be used to measure attainment of a variety of objectives. is di dediahtdlnclih‘Assesswent oF Leanne Outcoes (Assesswent 1) v w ly o a, cy co Comparing - Describe the similarities and differences between... - Compare the following methods for ... . Relating cause-and- effect - What are the major causes of - What would be the most likely effects of ... Justifying - Which of the following alternatives would you favor and why? - Explain why you agree or disagree with the ff. statement. }. Summarizing - State the points included in ... - Briefly summarize the contents of ... . Generalizing - Formulate several valid generalizations from the following data, - State a set of principles that can explain the following events. Inferring = In the light of the facts presented, what is most likely to happen when... - How would Senator X be most likely to react to the bomb explosion after the bar examination last September? Classifying : - Group the following items according to ... - What do the following items have in common? ~ Applying - Using the principles of __ as guide, describe how you would solve the following problem situation. - Describe a situation that illustrates the principle ofChapter 5 - Distinguishing and Constructing Various Paper-and-Penol Test + 9. Analyzing . , = Describe the reasoning errors in the following : . 7. paragraphs. - List and describe the main characteristics of ... 10. Evaluating - Describe the strengths and weaknesses of the following ... - Using the criteria developed in class, write an evaluation of ... 11. Creating - Make up a story describing what would happen if ... - Design a plan to prove that ... - Write a well-organized report that shows ... 5.4.2.1 Types of Essay Restricted Essay It is also referred to as short focused response. Examples are asking students to “write an example,” “list three reasons” or “compare and contrast two techniques.” Sample Short Response Question (10" Grade Reading) How are the scrub jay and the mockingbird different? ‘Support your answer with details and information from the article. Non-restricted /Extended Essay | Extended responses can be much longer and complex than short responses, but students are encouraged to remain focused and organized.‘Assesswenr oF Learnine Ourcowes (ASSESSMENT 1) Sample Extended Response Question (5t" Grade Science) Robert is designing a demonstration to display at his school's science fair. He will show how changing the position of a fulcrum on a lever changes the amount of force needed to lift an object. To do this, Robert will use a piece of wood for a lever and a block of wood to act as a fulcrum. He plans to move the fulcrum to different places on the lever to see how its placement affects the force needed to lift an object. Part - A Identify at least two other actions that would make Robert's demonstration better. Part - B Explain why each action would improve the demonstration. (Source: https:/cit.usf edulassessment/constructed/constructb.html) Note that all these involve the higher-level skills mentioned in Bloom’s Taxonomy. Guidelines for the Formulation and Scoring of Essay Tests Rule 1: Phrase the direction in such a way that students are guided on the key concepts to be included. Specify how the students should respond. Example Using details and information from the article (Hundred Islands), summarize the main points of the article. For a complete and correct response, consider these points: * its history (10 pts) * its interesting features (10 pts) * why it is a landmark (5 pts) Non-example i Using details and information from the article (Hundred Islands) summarize the main points of the article. Source: https:/fcit.usf.edu/assessment/constructed/constructb. him] - Rule 2: Inform the students on the criteria to be used for grading their essays. This rule allows the students to focus on relevant and substantive materials ratherChapter 5 ~ Distinguishing and Constructing Various Paper-anc-Pencil Test than on peripheral and unnecessary facts and bits of information. Example: Write an essay on the topic: “Plant Photosynthesis” % using the keywords indicated. You will be graded according to the following criteria: (a) coherence, (b) accuracy of statements, (c) use of keywords, (d) clarity and (e) extra points for innovative presentation of ideas. Rule 3: Put a time limit on the essay test. Rule 4: Decide on your essay grading system prior to getting the essays of your students. Rule 5: Evaluate all of the students’ answers to one question before proceeding to the next question. Scoring or grading essay tests question by question, rather than student by student, makes it possible to maintain a more uniform standard for judging the answers to each question. This procedure also helps offset the halo effect in grading. When all of the answers on one paper are read together, the grader’s impression of the paper as a whole is apt to influence the grades he assigns to the individual answers. Grading question by question, of course, prevents the formation of this overall impression of a student’s paper. Each answer is more apt to be judged on its own merits when it is read and compared with other answers to the same question, than when it is read and compared with other answers by the same student. Rule 6: Evaluate answers to essay questions without knowing the identity of the writer. This is another attempt to control personal bias during scoring. Answers to essay questions should be evaluated in terms of what is written, not in terms of what is known about the writers from other contacts with them. The best way to prevent our prior knowledge from influencing our judgment is to evaluate each answer without knowing the identity of the writer. This can be done by having the students write their names on the back of the paper or by using code numbers in place of names.ASSESSMENT OF LEARNING OuTCOMES (ASSESSMENT 1) Rule 7: Whenever possible, have two or more persons grade each answer. The best way to check on the reliability of the scoring of essay answers is to obtain two or more independent judgments. Although this may not be a feasible practice for routine classroom testing, it might be done periodically with a fellow teacher (one who is equally competent in the area). Obtaining two or more independent ratings becomes especially vital where the results are to be used for important and irreversible decisions, such as in the selection of students for further training or for special awards. Here the pooled ratings of several competent persons may be needed to attain level of reliability that is commensurate with the significance of the decision being made. Some teachers use the cumulative criteria i.e. adding the weights given to each criterion, as basis for grading while others use the reverse. In the latter method, each student begins with a score of : 100. Points are then deducted every time a teacher encounters a mistake or when a criterion is missed by the student in his essay. Rule 8: Do not provide optional questions. It is difficult to construct questions of equal difficulty and so teacher cannot have valid comparison of students’ achievement. Rule 9: Provide information about the value/weight of the question and how it will be scored. : Rule 10: Emphasize higher level thinking skills. Example: Scientists have found that oceans can influence the temperature of nearby landmasses. Coastal landmasses tend to have more moderate temperatures in summer and winter than inland landmasses of the same latitude. * Non Example: Considering the influence of ocean temperatures, explain why inland temperatures vary in summer and winter to a greater degree than coastal temperatures. List three coastal land masses.Chapter 5 Distinguishing and Constructing Various Paper-and-Penci Test Mental Exercise + Formulate one restricted essay and one extended essay. Present to the class for critiquing. 5.5. Exercises Let’s have some mental exercises to test your understanding. EXERCISE | ‘A. Give non-examples of each of the following rules of thumb in the construction of a true-false test. Improve on the non- examples for them to become good examples of tests, 1. Avoid giving hints in the body of the question. 2. Avoid using the words “always,” “never” and other such adverbs which tend to be always true or always false. 3. Avoid long sentences which tend to be true. Keep sentences short. 4, Avoid a systematic pattern for true and false statements. 5. Avoid ambiguous sentences which can be interpreted as true and at the same time false. B. Give non examples of each of the following rules of thumb in the construction of multiple choice test. Improve on the non-examples for them to become good examples of tests. 1. Phrase the stem to allow for only one correct or best answer. 2. Avoid giving away the answer in the stem. 3. Choose distracters appropriately. 4. Choose distracters so that they are all equally plausible and attractive. 5. Phrase questions so that they will test higher order thinking skills. 6. Do not ask subjective questions or opinions for which there are no right or wrong answers. atid Siediliinasli/ASSESSWENT OF LeaRNNG OuTcoMEs (ASSESSvENT 1) EXERCISE II A. Construct a 10-item matching type to test this competency: Identify the computer system ~ i.e. parts, other components. B. Construct a 10-item supply type test to assess this competency: Identify farm tools according to use (Grade 7-8 Curriculum Guide; Agriculture Fishery.) . Justify each rule used in constructing an essay type of test. . Construct a 10-item data sufficiency test. . Ina 100-item test, what types of objective tests will you include? Justify your answer. F. In the sample essay “Plant Photosynthesis” given in this section, why would you give a zero (0) score to the student writing this essay? Justify your answer. Example: Write an essay on the topic: “Plant Photosynthesis” using the following keywords and phrases: chlorophyll, sunlight, water, carbon dioxide, oxygen, by-product, stomata. mya Plant Photosynthesis Nature has its own way of ensuring the balance between food producers and consumers. Plants are considered producers of food for animals. Plants produce food for animals through a process called photosynthesis. It is a complex process that combines various natural elements on earth into the final product which animals can consume in order to survive. Naturally, we all need to protect plants so that we will continue to have food on our table. We should discourage burning of grasses, cutting of trees and illegal logging. If the leaves of plants are destroyed, they cannot perform photosynthesis and animals will also perish. G. Give an example of a supply type of test that will measure higher order thinking skills (beyond mere recall of facts and information.) H. In what sense is a matching type test a variant of a multiple choice type of test? Justify your answer. I, In what sense is a supply type of a test considered a variant of multiple choice type of test? (Hint; In supply type, the choices are not explicitly given). Does this make the supply type of test more difficult than closed multiple choice type of test? How? . J. Choose learning competencies from the K to 12 Curriculum Guide. Construct aligned paper-and-pencil tests observing guidelines in test construction. Include all selected-response and constructed-response types of test. Do this in groups of 5.Chapter 6 item Analysis and Validation Cuarrer 6 ITEM ANALYSIS AND VALIDATION LEARNING OUTCOMES > Explain the meaning of item analysis, item validity, reliability, item. | difficulty, discrimination index > Determine the validity and reliability of given test items | > Determine the quality of a test item % its difficulty index, ibility of discrimination index and plausi options (for a selected - response test) INTRODUCTION The teacher normally prepares a draft of the test. Such a draft is subjected to item analysis and validation in order to ensure that the final version of the test would be useful and functional. First, the teacher tries out the draft test to a group of students of similar characteristics as the intended test takers (try-out phase). From the try-out group, each item will be analyzed in terms of its ability to discriminate between those who know and those who do not know and also its level of difficulty (item analysis phase). The item analysis will provide information that will allow the teacher to decide whether to revise or replace an item (item revision phase). Then, finally, the final draft of the test is subjected to validation if the intent is to make use of the test as a standard test for the particular unit or grading period. We shall be concerned with these concepts in this Chapter. 6.1. Item Analysis: Difficulty Index and Discrimination Index There are two important characteristics of an item that will be of interest to the teacher. These are: (a) item difficulty and 93‘Assessuent oF Leanne Ourcoves (Assessvenr 1) (6) discrimination index. We shall learn how to measure these characteristics and apply our knowledge in making a decision about the item in question. The difficulty of an item or item difficulty is defined as the number of students who are able to answer the item correctly divided by the total number of students. Thus: Item difficulty = number of students with correct answer/ total number of students The item difficulty is usually expressed in percentage. Example: What is the item difficulty index of an item if 25 students are unable to answer it correctly while 75 answered it correctly? Here, the total number of students is 100, hence the item difficulty index is 75/100 or 75%. Another example: 25 students answered the item correctly while 75 students did not. The total number of students is 100 so the difficulty index is 25/100 or 25 which is 25%. It is a more difficult test item than that one with a difficulty index of 75. A high percentage indicates an easy item/question while a low percentage indicates a difficult item. One problem with this type of difficulty index is that it may not actually indicate that the item is difficult (or easy). A student who does not know the subject matter will naturally be unable to answer the item correctly even if the question is easy. How do we decide on the basis of this index whether the item is too difficult or too easy? The following arbitrary rule is often used in the literature: Range of Difficulty Index Interpretation Action 0-0.25 Difficult Revise or disciard 0.26 — 0.75 . _ Right difficulty Retain 0.76 — above Easy Revise or discard‘Chepter 6 — Item Analysis and Validation Difficult itemis tend to discriminate between those who know and those who do not know the answer. Conversely, easy items cannot discriminate between these two groups of students. We are therefore interested in’deriving a measure that will tell us whether an item can discriminate between these two groups of students. Such a measure is called am index of discrimination. ‘An easy way to derive such a measure is to measure how difficult an item is with respect to those in the upper 25% of the class and how difficult it is with respect to those in the lower 25% of the class. If the upper 25% of the class found the item easy yet the lower 25% found it difficult, then the item can discriminate properly between these two groups. Thus: Index of discrimination = DU - DL (U — Upper group; L — Lower group) Example: Obtain the index of discrimination of an item if the upper 25% of the class had a difficulty index of 0.60 (i.e. 60% of the upper 25% got the correct answer) while the lower 25% of the class had a difficulty index of 0.20. Here, DU = 0.60 while DL = 0.20, thus index of discrimination = 60 - .20 = .40. Discrimination index is the difference between the proportion of the top scorers who got an item correct and the proportion of the lowest scorers who got the item right. The discrimination index range is between -1 and +1. The closer the discrimination index is to +1, the more effectively the item can discriminate or distinguish between - the two groups of students. A negative discrimination index means more from the lower group got the item correctly. The last item is not good and so must be discarded. Theoretically, the index of discrimination can range from -1.0 (when DU =0 and DL = 1) to 1.0 (when DU = 1 and DL = 0). When the index of discrimination is equal to -1, then this means that all of the lower 25% of the students got the correct answer while all of the upper 25% got the wrong answer. In a sense, such an index discriminates correctly between the two groups but the item itself is highly questionable. Why should the bright ones get the wrong answer and the poor ones get the right answer? On the other hand, if the index of discrimination is 1.0, then this means that all of the lower 25% failed to get the correct answer while all of the upper‘Assessuent oF Learwns Ourcoves (Assessvenr 1) 25% got the correct answer. This is a perfectly discriminating item and is the ideal item that should be included in the test. From these discussions, let us agree to discard or revise all items that have negative discrimination index for although they discriminate correctly between the upper and lower 25% of the class, the content of the item itself may be highly dubious or doubtful. As in the case of the index of difficulty, we have the following rule of thumb: . Index Range ~ Interpretation Action 1.0 --50 © Can discriminate Discard but item is questionable ~55,- 0.45 Non-discriminating Revise 0.46 — 1.0 Discriminating item Include Example: Consider a multiple choice type of test of which the following data were obtained: Item Options A oe Bie CoD 1 0 40 20 2 Total 0 15 5. 0 Upper 25% 0 St tO 5 Lower 25% The correct response is B. Let us compute the difficulty index and index of discrimination: Difficulty Index = no. of students getting correct response/total = 40/100 = 40%, within range of a “good item” The discrimination index can similarly be computed: DU = no. of students in upper 25% with correct response/no. of students in the upper 25% = 15/20 = .75 or 75% “DL = no. of students in lower 25% with correct response! no. of students in the lower 25% 4 = 5/20 = .25 or 25% ‘ Discrimination Index = DU - DL = .75 - 25 = 50 0r 50%, 9. * = Thus, the item also has a “good discriminating power.” It is also instructive to note that the distracter A is not an effective distracter since this was never selected by the students. It is an implausible disttacter. Distracters C and D appear to have good appeal as distracters. They are plausible distracters.‘Chapter 6 ~ tem Analysis and Validation * Index of Difficulty Fe ees Ru+Ri : P= ——— ,x100 Be Where: Ru — The number in the upper group who answered the item correctly. RL — The number in the lower group who answered the item correctly. T. — The total number who tried the item. Index of item Discriminating Power — Ru+RL . Where: : P — percentage who answered the item correctly (index of difficulty) R- — number who answered the item correctly T — total number who tried the item. 8 P= —— x 100= 40% 20 The smaller the percentage figure the more difficult the item Estimate the item discriminating power using the formula below: Ru — RL 6-2 Do “T 10 The discriminating power of an item is reported as a decimal fraction; maximum discriminating power is indicated by an index of 1.00. Maximum discrimination is usually found at the 50 percent level of difficulty 0,00 — 0.20 = Very difficult 0.21 — 0.80 = Moderately difficult 0.81 — 1.00 = Very easy *ee ee ‘Assesowenr oF Learwne Outcomes (Assesswent 1) For classroom achievement tests, most test constructors desire items with indices of difficulty no lower than 20 nor higher than 80, with an average index of difficulty from 30 or 40 to a maximum of 60. The INDEX OF DISCRIMINATION is the difference between the proportion of the upper group who got an item right and the proportion of the lower group who got the item right. This index is dependent upon the difficulty of an item. It may reach a maximum value of 100 for an item with an index of difficulty of 50, that is, when 100% of the upper group and none of the lower group answer the item correctly. For items of less than or greater than 50 difficulty, the index of discrimination has a maximum value of less than 100. More Sophisticated Discrimination Index Item discrimination refers to the ability of an item to differentiate among students on the basis of how well they know the material being tested. Various hand calculation procedures have traditionally been used to compare item responses to total test scores using high and low scoring groups of students. Computerized analyses provide more accurate assessment of the discrimination power of items because they take into account responses of all students rather than just high and low scoring groups. The item discrimination index provided by ScorePak® is a Pearson Product Moment correlation between student responses to a particular item and total scores on all other items on the test. This index is the equivalent of a point-biserial coefficient in this application. It provides an estimate of the degree to which an individual item is measuring the same thing as the rest of the items. Because the discrimination index reflects the degree to which an item and the test as a whole are measuring a unitary ability or attribute, values of the coefficient will tend to be lower for tests measuring a wide range of content areas than for more’ homogeneous tests. Item discrimination indices must always be inferpreted in the context of the type of test which is being’ analyzed. Itenis with low discrimination indices are often ambiguously worded and should be examined. Items with negative indices should be examined to determine why a negative value was obtained. For example, a negative value may indicate that the item was mis-keyed, so that‘Chapter 6 - tem Analysis and Validation students who knew the material tended to choose an unkeyed, but corgect, response option. ‘Tests with high internal consistency consist of items with mostly positive felationships with total test score. In practice, values of the discrimiriation index will seldom exceed .50 because of the differing shapes of item and total score distributions. ScorePak® classifies item discrimination as “good” if the index is above .30; “fair” if it is between. 10 and .30; and “poor” if it is below .10. SA good item is one that has good discriminating ability and has sufficient level of difficult (not too difficult nor too easy). At the end of the Item Analysis report, test items are listed according to their degrees of difficulty (easy, medium, hard) and discrimination (good, fair, poor). These distributions provide a quick overview of the test, and can be used to identify items which are not performing well and which can perhaps be improved or discarded. AAU The Item-Analysis Procedure for Norm-provides the following information: 1. The difficulty of the item; 2. The discriminating power of the item, and 3. The effectiveness of each alternative Some benefits derived from Item Analysis are: 1, It provides useful information for class discussion of the test. 2, It provides data which help students improve their learning, 3. It provides insights and skills that lead to the preparation for better tests in the future.100 Assessuen oF Learwne Ourcoues (Assesswtet 1) 6.2. Validation and Validity After performing the item analysis and revising the items which need revision, the next step is to validate the instrument. The purpose of validation is to determine the characteristics of the whole test itself, namely, the validity and reliability of the test. Validation is the process of collecting and analyzing evidence to support the meaningfulness and usefulness of the test. Validity. Validity is the extent to which a test measures what it purports to measure or as referring to the appropriateness, correctness, meaningfulness and usefulness of the specific decisions a teacher makes based on the test results. These two definitions of validity differ in the sense that the first definition refers to the test itself while the second refers to the decisions made by the teacher based on the test. A test is valid when it is aligned with the learning outcome. A teacher who conducts test validation might want to gather different kinds of evidence. There are essentially three main types of evidence that may be collected: content-related evidence of validity, criterion-related evidence of validity and construct-related evidence of validity, Content-related evidence of validity refers to the content and format of the instrument. How appropriate is the content? How comprehensive? Does it logically get at the intended variable? How adequately does the sample of items or questions represent the content to be assessed? Criterion-related evidence of validity refers to the relationship between scores obtained using the instrument and scores obtained using one or more other tests (often called criterion). How strong is this relationship? How well do such scores estimate present or predict future performance of a certain type? Construct-related evidence of validity refers to the nature of the psychological construct or characteristic being measured by the test. How well does a measure of the construct explain differences. in the behavior of the individuals or their performance on a certain task? , The usual procedure for determining content validity may be described as follows: The teacher writes out the objectives of.the test based on the Table of Specifications and then gives these together with the test to at least two (2) experts along with a description of the intended test takers. The experts look at the objectives, read overChapter 6 ~ tem Analysis and Validation the items in the test and place a check mark in front of each question or item that they feel does not measure one or more objectives. They also place a check mark in front of each objective not assessed by ‘any ‘item in the test. The teacher then rewrites any item checked and resubmits to the experts and/or writes new items to cover those objectives not covered by the existing test. This continues until the experts approve of all items and also until the experts agree that all of the objectives are sufficiently covered by the test. In order to obtain evidence of criterion-related validity, the teacher usually compares scores on the test in question with the scores on some other independent criterion test which presumably has already high validity. For example, if a test is designed to measure mathematics ability of students and it correlates highly with a standardized mathematics achievement test (external criterion), then we say we have high criterion-related evidence of validity. In particular, this type of criterion-related validity is called its concurrent validity. Another type of criterion-related validity is called predictive validity wherein the test scores in the instrument are correlated with scores on a later performance (criterion measure) of the students, For example, the mathematics ability test constructed by the teacher may be correlated with their later performance in a Division-wide mathematics achievement test. In summary content validity refers to how will the test items reflect the knowlege actually required for a given topic area (e.g. math). It requires the use of recognized subject matter experts to evaluate whether test items assess defined outcomes. Does a pre- employment test measure effectively and comprehensively the abilities required to perform the job? Does an English grammar test measure effectively the ability to write good English? Criterion-related validity is also known as concrete validity because criterion validity refers to a test’s correlation with a concrete outcome. In the case of pre-employment test, the two variables that are compared are test scores and employee performance. There are 2 main types of criterion validity-concurrent validity and predictive validity. Concurrent validity refers to a comparison between the measure in question and an outcome assessed at the same time. 101102 Assessuer oF Leanne Ourcoues (Assesswenr 1) An example of concurrent validity is a comparison of the scores with NAT Math exam with course grades in Grade 12 Math. In predictive validity, we ask this question: Do the scores in NAT Math exam predict the Math grade in Grade 12? 6.3, Reliability Reliability refers to the consistency of the scores obtained — how consistent they are for each individual from one administration of an instrument to another and from one set of items to another, We already gave the formula for computing the reliability of a test: for intemal consistency; for instance, we could use the split-half method or the Kuder-Richardson formulae (KR-20 or KR-21) Reliability and validity are related concepts. If an instrument is unreliable, it cannot get valid outcomes. As reliability improves, validity may improve (or it may not). However, if an instrument is shown scientifically to be valid then it is almost certain that it is also reliable. Predictive validity compares the question with an outcome assessed at a later time. An example of predicitve validity is a comparison of scores in the National Achievement Test (NAT) with first semester grade point average (GPA) in college. Do NAT scores predict college performance? Construct validity refers to the ability of a test to measure what it is supposed to measure. As researcher, you intend to measure depression but you actually measure anxiety so your research gets compromised.Chapter 6 - item Analysis and Validation 103 The following table is a standard followed almost universally in educational test and measurement. * Reliability ~ Interpretation 90 and, above Excellent reliability; at the level of the best . standardized tests 80 - 90 Very good for a classroom test Good for a classroom test; in the range of most. -70 - 80 There are probably a few items which could be improved. Somewhat low. This test needs to be supplemented by other measures (e.g., more tests) to determine grades. There are probably some items which could be improved. ¢ -60 - 70 Suggests need for revision of test, unless it is quite short (ten or fewer items). The test definitely needs , to be supplemented by other measures (e.g., more tests) for grading. -50 - 60 Questionable reliability. This test should not 50 or below contribute heavily to the course grade, and it needs revision. 6.4. Exercises Item Analysis = Discrimination and Difficulty Index A. Give the term described/explained. 1, Refers to a statistical technique that helps instructors identify the effectiveness of their test items 2. Refers to the proportion of students who got the test item correctly 3, Which is the difference between the proportion of the top scorers who got an item correct and the proportion of the bottom scorers who got the item right?Assessment oF Learwne Outcomes (Assessment 1) 4. Which one is concerened with how easy or difficult a test item is? 5. Which adjective describes an effective distracter? B. Write TRUE if the statement is correct and FALSE if it is wrong. 1. Difficulty index indicates the proportion of students who got the item right. 2. Difficulty index indicates the proportion of students who got the item wrong. 3. A high percentage indicates an easy item/question and a low percentage indicates a difficult item. 4. Authors agree, in general, that items should have values of difficulty no less than 20% correct and no greater than 80%. 5. Very difficult or very easy items contribute greatly to the discriminating power of a test. 6. The discrimination index range is between -1 and +2. 7. The farther the index is to +1, the more effectively the item distinguishes between the two groups of students. 8. When an item discriminates negatively, such item should be revised and eliminated from scoring. 9. A positive discrimination index indicates that the lower performing students actually selected the key or correct response more frequently than the top performers. 10. Ifno one selects a distracter it is important to revise the option and attempt to make the distracter a more plausible choice. C. Problem Solving 1. Solve for the difficulty index of each test item: Item No. No, of Correct Responses sine se] whe canis | | f |_|Chapter 6 - Item Analysis and Validation 1. Which is most difficult? Most easy? * 2. Which needs revision? Which should be discarded? Why? D. Solve for the discrimination indexes of the following test items: Pfeffer [oe [ie [ve [a | No, of Correc Responses 40 No. of Beret (ase e Based on the computed discrimination index, which are good test items? Not good test items? E. A multiple choice type of test has 5 options. The Table below indicates the number of examinees out of 50 who chose each option. - Correct answer e Which options. are plausible? Which ones are implausible? F. Study the following data, Compute for the difficulty index and the discrimination index of each set of scores. 1. N = 80, number of wrong answers: upper 25% = 2 lower 25% = 9 2. N = 30, number of wrong answers: upper 25% = 1 lower 25% = 6 3..N.= 50, number of wrong answers: upper 25% = 3 lower 25% = 8 4, N= 70, number of wrong answers, upper 25% = 4 lower 25% = 10 G. Write test items to check if the following learning outcomes were realized: Draw correct conclusions that are based on the information given;‘Assesouent oF Learwna Ourcouss (Assessutat 1) L Identify one or more logical implications that follow from a given point of view; State whether two ideas are identical, just similar, unrelated or contradictory. © Show your test to an expert and ask him to judge whether the constructed test items indeed assessed the learning outcomes. H. Enumerate the three types of validity evidence. Which of these types of validity is the most difficult to measure? Why? What is the relationship between validity and reliability? Can a test be reliable and yet not valid? Illustrate. Discuss the different measures of reliability. Justify the use of each measure in the context of measuring reliability. Can a test be valid and not reliable?. > Measures of Central -Tendency and Dispersion/Variability wv LEARNING OUTCOMES > Explain the meaning and function of the measures of central _ tendency and measures of dispersion/variability » Distinguish among the measures of central tendency and measures of variability/dispersion > Explain the meaning of normal and skewed score distribution INTRODUCTIOI A measure of central tendency is’a single value that attempts to describe a set of data (like scores) by identifying the central position within that set of data or scores. As such, measures of central tendency are sometimes called measures of central location. Central tendency tefers to the center of a distribution of observations. Where do scores tend to congregate? In a test of 100 items, where are most of the scores? Do they tend to group around the mean score of 50-or 802 There are three measures central tendency — the mean, the median and the mode. Perhaps you are most familiar with the mean (often called the average). But there are two other measures of central tendency, namely, the median and the mode. Is there such a thing as best measure of central tendency? If the measures of central tendency indicate where scores congregate, the measures of variability indicate how spread out a group of scores is or or how varied the scores are or how far they are from the mean? Common measures of dispersion or variability are range, interquartile range, Variance and standard deviation. 107‘Assessuent oF Leanne Ourooves (Assessuent 1) 7.1 The Measures of Central Tendency The mean, mode and median are valid measures of central tendency but under different conditions, one measure becomes more appropriate than the others. For example, if the scores are extremely high and extremely low, the median is a better measure of central tendency since mean is affected by extremely high and extremely low scores. The Mean (Arithmetic) ‘The mean (or average or arithmetic mean) is the most popular and most well known measure of central tendency. The mean is equal to the sum of all the values in the data set divided by the number of values in the data set. For example, 10 students in a Graduate School class got the following scores in a 100 — item test: 70, 72, 75,77, 78, 80, 84, 87, 90, 92. The mean score of the group of 10 students is the sum of all their scores divided by 10. The mean, therefore, is 805/10 equals 80.5. 80.5 is the average score of the group. There are 6 scores below the average score (mean) of the group (70, 72, 75,77,78, and 80) and there are 4 scores above the average score (mean) of the group (84, 87, 90 and 92). i ee When Not to Use the Mean The mean has one main disadvantage. It is particularly susceptible to the influence of outliers. These are values that are unusual compared to the rest of the data set by being especially small or large in numerical value. For example, consider the scores of 10 Grade 12 students in a 100-item Statistics test below: Pe ee ae ee The mean score for these ten Garde 12 students.is 62.1 However, inspecting the raw data suggests that this mean score may not be the best way to accurately reflect the score of the typical Grade 12 student., as most students have scores in the 5 to 95 rafige. The ingan is being skewed by the extremely low and extremely high scores. Therefore, in this situation, we would like to have a beiter measure of central tendency. As we will find out later, taking the median would be a better measure of central tendency in this situation.* ‘Chapter 7 - Measures of Central Tendency and Dispersion/Variablity Media The median is the middle score for a set of scores arranged from Flowest to highest. The mean is less affected by extremely low and extremely high scores. How do we find the median? Suppose we have the following data: 65 «(55 89 56 35 14 #56 55 87 45 92 To determine the median, first we have to rearrange the scores into order of magnitude (from smallest to largest). 14 #35 45 55 55 56 56 65 87 89 92 Our median is the score at the middle of the distribution, In this case, 56. It is the middle score. There are 5 scores before it and 5 scores after it. This works fine when you have an odd number of scores, but what happens when you have an even number of scores? What if you had 10 scores like the scores below? 65 55 89 56 35 14 56 5587) «45. Arrange that data according to order of magnitude (smallest to largest). Then take the middle two scores (55 and 56) and compute the average of the two scores. The median is 55.5. This gives us a more reliable picture of the tendency of the scores. There are indeed scores of 55 and 56 in the score distribution. Mode The mode is the most frequent score in our. data set. On a histogram or bar chart it represents the highest bar. If is a score of the number of times an optionis chosen in a multiple choice test You can, therefore, sometimes consider the mode as being the most popular option. Study the score distribution given below: 14 35 45 55 55 56 56 65 87 89 There are two most frequent scores 55 and 56. So we have a score distribution with two modes, hence a bimodal distribution. 7.2 Normal and Skewed Distributions A score distribution a sample has a “normal distribution” when most of the values are aggregated around the mean, and the number of values decrease as you move below or above the mean: the bar graph of frequencies of a “normally distributed” sample will look like a bell curve110 ‘Assessuent oF Learwnc Ourcoues (Assessuenr 1) Standard Normal Distribution http://kalnari.com/blog/an-interesting-muttipurposed-brown-sofa-by-marcin-wielgosz/ Figure 13. Normal Distribution © If mean is equal to the median and median is equal to the mode, the score distribution shows a perfectly normal distribution. This is illustrated by the perfect bell shape or normal curve shown in Figure 13. * If mean is less than the median and the mode, the score distribution is a negatively skewed distribution. See Figure 14. In a negatively skewed distribution the scores tend to congregate at the upper end of the score distribution. Median Burzle.com https://sciencestruck. coniypes-t slewedstrbuton-wihealttesxples Figure 14. Negatively Skewed DistributionChapter 7 - Measures of Central Tendency and Dispersion/Variabilty « Ifmean is greater than the median and the mode, the score distribution is a positivey skewed distribution. See Figure 15. In ". A positively skewed distribution the scores tend to congregate at the lower end of the score distribution. Bizzle bon, https://sciencestruck.com/types-of-skewed-distribution-with-real-life-examples Figure 15. Positively Skewed Distribution If scores tend to be high because teacher taught very well and students are highly motivated to learn, the score dsitribution tends to be negatively skewed, i.e. the scores will tend to be high. On the other hand, when teacher does not teach well and students are poorly motivated, the score distribution tends to be positively skewed which means that scores tend to below. So which score distribution should we work for? 7.3 Outcome-based Teaching-Learning and Score Distribution If teachers teach in accordance with the principles of outcome- based teaching-learning and so align content and assessment with the intended learning outcomes and re-teach till mastery what has/ have not been understood as revealed by the formative assessment process, then student scores in the assessment phase of the lesson will tend to congregate on the higher end of the score distribution. Score distribution will be positively skewed. 11112 ‘Assesswen oF Learnins Ourcowes (Assessvenr 1) On the other hand, if what teachers teach and assess are not aligned with the intended learning outcomes, the opposite will be true. Score distribution will be negatively skewed which means that scores tend to congregate on the lower end of the score distribution. 7.4 Measures of Dispersion or Variability If the measures of central tendency indicate where scores congregate, the measures of variability indicate how spread out a group of scores is or or how varied the scores are. Common measures of dispersion or variability are range, variance and standard deviation. Range What is variability? Variability refers to how “spread out” a group of scores is. The terms variability, spread, and dispersion are synonyms, and refer to how spread out a distribution is. Here are two sets of score distribution: A-5,5,5,.5,6, 6, 6, 6,6, 6 - Mean is 5,6 B-1,3,4,5,5, 6,7, 8, 8,9 - Mean is 5.6 The two score distributions have equal mean scores and yet the scores are varied: Score distribution A shows scores that are less varied than score distributioh B. That is what we mean by variability or dispersion. If we have to study both score distributions, assuming that the highest possible score in the quiz is 10, we can say that Groups A and B are equal iti terms of mean but Group A has more similar scores and are closer to the mean while Group B, while its mean is equal to the mean of Group A, students in Group B have more varied scores than Group A. Iti fact the lowest score is extremely low compared to Group A and the highest score is much higher than the highest score in Group A. To see more what we mean by spread out, consider graphs in Figure 1. These graphs fepresent the scores on two quizzes The mean score for each quiz is 7.0. Despite the equality of means, you can see that the distributions are quite different. Specifically, the seores,on Quiz 1 are more densely packed and those on Quiz 2 are ‘more spread out. The differences among students were much greater on Quiz 2 than on Quiz 1.Chapter 7 Measures of Central Tendency and Dispersion/Varibilty 113 Figure 16. Bar charts of two quizzes | ! \ : 4 s 6 7 5 9 0 Figure 17. Bar charts of two quizzes http://onlinestatbook.com/2/summarizing_distributions/variability. htm!Range The range is the most simple measure of variability. The range is simply the highest score minus the lowest score. Here are examples: Let's take a few examples. What is the range of the following group of scores: 10, 2, 5, 6, 7, 3, 4? The highest number is 10, and the lowest number is 2, so 10 - 2 = 8. The range is 8. ‘Assessvent oF Leawne Ourcoves (Assesswenr 1) Here are other examples: Here is a set of scores ina test: 99, 45, 23, 67, 45, 91, 82, 78, 62, 51. What is the range? The highest number is 99 and the lowest number is 23, so 99 - 23 equals 76; the range is 76. Here is another set of scores: 40, 40, 42, 50, 53, 56, 67, 68, 70, 89. What is the range? 89 minus 40 equals 49. The range is 49: The set of scores with a range of 76 is more varied or more spread than the set of scores with a range of 49. Variance Variability can also’be defined in terms of how close the scores in the distribution are to the middle of the distribution. Using the mean as the measure of the middle of the distribution, the variance is defined as the average squared difference of the scores from the mean. The data from Quiz | are shown in Table 1, The mean score is 7.0. Therefore, the column “Deviation from Mean” contains the score minus 7. The column “Squared Deviation” is simply the previous column squared.(Chapter 7 ~ Measures of Central Tendency and Dispersion/Variabiity ‘Table 6. Calculation of Variance for Quiz 1 scores. Squared Deviation One thing that is important to notice is that the mean deviation from the mean is 0, This will always be the case. The mean of the ° squared deviations is 1.5. Therefore, the variance is 1.5. The formula for the variance is: eee N 15 sail16 ‘Asgessvenr oF Letawne Ourcoues (Assessuenr 1) Standard Deviation To calculate the standard deviation of those numbers: 1. Work out the Mean (the simple average of the numbers). 2. Then for each number: subtract the Mean and square the result 3. Then work out the mean of those squared differences. 4. Take the square root of that and we are done! The Formula Explained First, let us have some example values to work on: Example: Sam has 20 rose bushes. The number of flowers on each bush is 9, 2, 5, 4, 12, 7, 8, 11, 9, 3, 7, 4, 12, 5, 4, 10, 9, 6,9, 4 Let’s solve for the Standard Deviation. Step 1. Work out the mean In the formula above p1 (the Greek letter “mu”) is the Mean of all our values... Example: 9, 2, 5, 4, 12, 7, 8, 11, 9, 3, 7, 4, 12, 5, 4, 10, 9, 6,9, 4 The mean is: G+24+54+4+12+748411+94347+4+12+5+4+ 1049+ 64944 20 = 140 20=7 So: p=7 Step 2. Then for each number: subtract the Mean and square the result. This is the part of the formula that says: 2 (xi — #) So what is x, ? They are the individual x values 9, 2, 5, 4, 12, 7, etc... In other words x, = 9,x,=2,x,=5, ete. - So it says “for each value, subtract the mean and square the result,” like this . as Example (continued): (@-78=(Q7=4 2-77 =(5 =25Chapter 7 ~ Measures of Central Tendency and Dispersion/Variablty (4-17=(3F=9 Q2- 77 = (6 =25 (7277 = OF =0 (8-7=(1R=1 ws CLC oe And we get these results: 4, 25, 4, 9, 25, 0, 1, 16, 4, 16, 0, 9, 25, 4,9, 9, 4, 1,4, 9 Step 3. Then work out the mean of those squared differences. To work out the mean, add up all the values then divide by how many. First add up all the values from the previous step. But how do we say “add them all up” in mathematics? We use “Sigma”: 5 The handy Sigma Notation says to sum up as many terms as ‘we want: i ‘We want to add up all the values from 1 to N, where N=20 in our case because there are 20 values: Example (continued): N > (ai — A)? eo Which means: Sum all values from (x,-7)? to (xy-7)? 1 We already calculated (x,-7)'=4 etc. in the previous step, so just sum them up: = 4+25+4+9425+0+1+16+4+16+0+9+25+4+9+ O+4+1+4+9=178 But that isn’t the mean yet, we need to divide by how many, which is done by multiplying by 1/N (the same as dividing by N): Example (continued): 1 N sees o> ween ES Wi - #) t=1 Mean of squared differences = (1/20) x 178 = 8.9 (Note: this value is called the “Variance”) 47Assessuent oF LeaRwnc Ourcowes (Assessvenr 1) Step 4. Take the square root of that: Example (concluded): o = V(8.9) = 2.983... Sample Standard Deviation But sometimes our data are only a sample of the whole population. Example: Sam has 20 rose bushes, but cu counted the flowers on 6 of them! The “population” is all 20 rose bushes, and the “sample” is the 6 bushes that Sam counted among the 20. Let us say Sam’s flower counts are: 9, 2, 5, 4, 12,7 We can still estimate the Standard Deviation. But when we use the sample as an estimate of the whole population, the Standard Deviation formula changes to this: The formula for Sample Standard Deviation: The important change is “N-1” instead of “N” (which is called “Bessel’s correction”). The symbols also change to reflect that we are ‘working ona sample instead of the whole population: * The mean is now x (for sample mean) insteat! of w (the population mean), : z «And the answer is s (for Sample Standard Devidtion) instead of 6, But that does not affect the calculations. Only Na instead of N changes the calculations.Chapter 7 - Measures of Central Tendency and Dispersior/Varibilty Here are the steps in calculating the Sample Standard Deviation: Step 1. Work out the mean ">, Sxample 2: Using sampled values 9, 2, 5, 4, 12,7 The mean is (9+2+5+4+12+7) / 6 = 39/6 = 6.5 So: x= 6.5 Step 2. Then for each number: subtract the Mean and square the result » Example 2 (continued): (9 - 6.5 = (2.5)? = 6.25 (2- 6.59 = (4.5)? = 20.25 (5 - 6.5) = (-1.5) = 2.25 (4 - 6.5 = (2.5) = 6.25 (12 - 6.5) = (5.5)? = 30.25 (7 - 6.5) = (0.5)? = 0.25 Step 3. Then work out the mean of those squared differences. To work out the mean, add up all the values then divide by how many. But hang on... we are calculating the Sample Standard Deviation, so instead of dividing by how many (N), we will divide by N-1 Example 2 (continued): Sum = 6.25 + 20.25 + 2.25 + 6,25 + 30.25 + 0.25 = 65.5 Divide by N-1: (1/5) x 65.5 = 13.1 (This value is called the “Sample Variance”) Step 4, Take the square root of that: Example 2 (concluded): s= (13.1) = 3.619.120 /Assesswenr oF Leanne Ourcoues (Assessuexr 1) 7.5 Comparing When we used the whole population we got: Mean = 7, Standard Deviation = 2.983... When we used the sample we got: Sample Mean = 6.5, Sample Standard Deviation = 3.619... Our Sample Mean was wrong by 7%, and our Sample Standard Deviation was wrong by 21%. Why Take a Sample? Mostly because it is easier and cheaper. Imagine you want to know what the whole university thinks... you can’t ask thousands of people, so instead you ask maybe only 300 people. Samuel Jolinson once said “You don’t have to eat the whole ox to know that the meat is tough.” (Source: https://www.mathsisfun.com/data/standard-deviation-formulas.htmi, Retrieved 1-24-19) 7.6 More Notes on-Standard Deviation The standard deviation is simply the square root of the variance. The standard deviation is an especially useful measure of variability when the distribution is normal or approximately normal because the proportion of the distribution within a given number of standard deviations from the mean can be calculated, For example, 68% of the distribution is within one standard deviation of the mean and approximately 95% of the distribution is within two standard deviations of the mean. Therefore, if you had a normal distribution with a mean of 50 and a standard deviation of 10, then 68% of the distribution would be between 50 - 10 = 40 and 50 +10 =60. Similarly, about 95% of the distribution would be between 50 - 2 x 10 = 30 and 50 + 2 x 10 = 70. The symbol for the population standard deviation is 0; - Figure 2 shows two normal distributions. The distribution (bold line) has a mean of 40 and a standard deviation of 5; The other distribution has a’ mean of 60 and a standard deviation of to. For the distribution (bold line), 68% of the distribution is between 35’ and 45; for the other distribution, 68% is between 50 and 70.‘Chapter 7 ~ Measures of Central Tendency and Dispersion/Variabilty Oo 10 20 3 40 50 60 7 80 90 100 http://onlinestatbook.com/2/summarizing_distributions/variability. htm! Figure 18. Normal distributions with standard deviations of 5 and 10. Standard Deviation is a measure of dispersion, the more dispersed the data, the less consistent the data are. A lower standard deviation means that the data are more clustered around the mean and hence the data set is more consistent. ‘You need to read your calculator instructions to see what notation your calculator uses for the standard deviation. An example. Standard deviation for a data set with frequency 1. Using the following data: 10 15 13 25 22 53 47 ‘We found the mean to be x = 26:4285714. You should also see from the same calculationthat the standard deviation (SD) = 16:98879182. (2008 ASU School of Mathematical & Statistical Sciences and Terri L. Miller, retrieved, 1-15-19) 7.7 Interpretation of Standard Deviation Let us use the standard deviation to compare two data sets. Let us use the standard deviation to interpret how consistent the data are. The lower the standard deviation, the more consistent the data are. Example - Two bowlers, Katie and Mike have the scores given below: Katie’s Scores 189 146 200 241 231 Mike’s Scores 235 201 217 168 186 Both sets of data have a mean (x) = 201.4, Does this mean they are equivalent bowlers? No, consider the standard deviations. Katie has a standard deviation of SD = 37.6470 and Mike has a standard deviation of SD = 26,1017. Since Mike has a smaller standard deviation, he is a more 121‘Assesswen oF Learwine Ourcones (Assessuenr 1) consistent bowler than Katie, i.e. Mike is more likely to get a score of 201.4. Let’s presume that Katie’s and Mike’s scores are scores in a long test: Katie’s Scores - 189 146 200 241 231 Mike’s Scores - 235 201 217 168 186 If you compute the mean for both sets of scores, you get 201. SD for Katie’s scores id 37.64.70 while that of Mike is 26.1017. Mike’s scores indicate greater consistency than those of Katie. This means that Mike tends to do better than Katie because his scores are more consistent than those of Katie. (Source: 2009 ASU School of Mathematical & Statistical Sciences and Terri L. Miller), Retrieved, 1-25-19) 7.8 Some Exercises 1. Which is referred to as average of scores? A. Mean C. Mode B. Median D. Standard Deviation 2. If scores are plotted in a histogram, which do you call that with the highest frequency? A. Mean C. Mode B. Median D. Standard Deviation 3. Which is the midpoint of a score distribution? A. Mean. C. Mode B. Median D. Standard Deviation 4. Which does NOT belong? “. A. Mean C. Mode S B. Median D. Standard Deviation _ 5. Which is a measure of variability? A. Range C. Mean B. Median D. ModeChapter 7 ~ Measure of Central Tendency and Disperion/Varility 123 6. Which ig @ measure of dispersion? A. Mean C. Mode “BB. Médian D. Variance 7. Whiich is a measure of the spread of scores ? A. Mean C. Standard Deviation B. Mode D. Median 8. You like to get a more reliable picture of the scores of your students in your Math class? Which will you compute? A. The mean B, The mean and the SD C. The difficulty index D. The discrimination index 9. Here is a score distribution of a quiz with 10 as the highest possible score: 2, 4, 5, 5, 6, 7, 7, 7, 8, 8. Which is the range? A 2 C.7 B. 6 D.8 10. Which score distribution do all teachers, parents and students wish? , A. Negatively skewed C. Positively skewed B. Bell curve D. That depends on the Mean 11. If there is not real teaching and learning that take place, which score distribution is most likely to come? A. Negatively skewed C. Positively skewed B. Bell curve D. That depends on the Mean 12. Among the measures of central tendency, which is most affected by outliers? A. Mean C. Median B. Mode D. Range 13. Ifa score distribution has_no outliers, which is most likely to be TRUE? A. The scores may not be so varied. B. The scores may be highly varied.124 ‘Assesswenr oF Leanwine Ourcowes (Assessuenrt 1) C. In this case, the median is the most reliable measure of central tendency. D. In this case, the mode is the best measure of central tendency. 14. Which is the mean of the squared deviation from the mean? A. Variance C. Standard Deviation B, Range D. Mean 15. Which is TRUE of scores that follow the normal distribution curve? A. The mean, the median and mode are equal. B. The median is higher than the mean. C. The mean is higher than the median. D. The mode is higher than the mean and the median. 16, Ifa score distribution has a Standard Deviation of zero, what does it mean? ‘A. Most scores are zero. B. The scores are the same. C. Most scores are high. D. Most scores are negative. 7.9 Problem Solving 1. Here is a set of scores: 1, 2,3,4,5,6,7. What is the mean? What is the median? What is the mode? What is the range? 2. A student has gotten the following grades on his fests: 87, 95, 76, and 88. He wants an 85 or better overall. “What is the minimum grade he must get on the last test in order to achiéve that average?(Chapter 7 - Measures of Central Tendency and Dispersion Variability 7 * 0 Key to Correction 1. Mean: 3.5 median: 3 |mode: none range: 6 find the average of all his grades (the known ones, plus the unknown one), we have to add up all the grades, and then divide by the number of grades. Since I don’t have a score for the last test yet, I’ll use a variable to stand for this unknown value: “x. Then computation to find the desired average is: (87 + 95 + 76 + 88 +x)+5=85 Multiplying through by 5 and simplifying, I get: 87+ 95 + 76 + 88 + x = 425 346 + x= 425 x= 19: He needs to get at least a 79 on the last test. (Source: https:/www.mathsisfun.com/data/standard-deviation-formulas.htm|, Retrieved 5-1-19) ze Solution: The minimum grade is what we need to find. To 125Cure 8 GRADING SYSTEMS anp THE GRADING SYSTEM or THe DepaRTMENT OF EDUCATION a LEARNING OUTCOMES : > Distinguish between norm-referenced and criterion-referenced ‘grading; cumulative and averaging grading system > Compute grades of students in various grade levels observing DepEd guidelines INTRODUCTION A seastiiend oteandent performance is essentially knowing how the student is progressing in a course (and, incidentally, how a teacher is also performing with respect to the teaching process). The first step in assessment is, of course, testing (either by some pencil- paper objective test or by some performance based testing procedure) followed by a decision to grade the performance of the student. Grading, therefore, is the next step after testing. Over the course of several years, grading systems had been evolved in different schools systems all over the world. In the American system, for instance, grades are expressed in terms of letters, A, B, B+, B-, C, C-, D or what is referred to as a seven-point system. In Philippine colleges and universities, the letters are replaced with numerical values: 1, 1.25, 1.50, 1.75, 2.0, 2.5, 3.0 and 4.0 or an eight-point system. In basic education, grades are expressed as percentages (of accomplishment) such as 80% or 75%. With the implementation of the K to 12 Basic Education curriculum, however, student’s performance is expressed in terms of level of proficiency. Regardless of grading system adopted, it is clear’that there appears to be a need to convert raw score values into the corresponding standard grading system. This Chapter is concerned with the underlying philosophy arid mechanigs of converting raw score values into standard grading formats.Chapter 8 — Grading Systems and the Grading Systems ofthe Department of Education 8.1. Norm-Referenced Grading +, Tie most commonly used grading system falls under the category of norm-referenced grading. Norm-referenced grading refers to a grading system where a student’s grade is placed in relation to the performance of a group. Thus, in this system, a grade of 80 means that the student performed better than or same as 80% of the class (or group). At first glance, there appears to be no problem with this type of grading system as it simply describes the performance of a student with reference to a particular group of learners. The following example shows some of the difficulties associated with norm-referenced grading: Example: Consider the following two sets of scores in an English 1 class for two sections of ten students each: A= { 30, 40,50, 55, 60, 65,70,75,80, 85 } B = { 60, 65, 70, 75, 80, 85, 90, 90, 95, 100 } In the first class, the student who got a raw score of 75 would get a grade of 80% while in the second class, the same grade of 80% would correspond to a raw score of 90. Indeed, if the test used for the two classes are the same, it would be a rather “unfair” system of grading. A wise student would opt to enroll in class A since it is easier to get higher grades in that class than in the other class (class B). The previous example illustrates one difficulty with using a norm-referenced grading system. This problem is called the problem of equivalency. Does a grade of 80 in one class represent the same achievement level as a grade of 80 in another class of the same subject? This problem is similar to the problem of trying to compare a Valedictorian from some remote rural high school with a Valedictorian from some very popular University in the urban area. Does one expect the same level of competence for these two valedictorians? As we have seen, norm-referenced grading systems are based on a pre-established formula regarding the percentage or ratio of students within a whole class who will be assigned each 127128 ‘Assessuent oF Leanne Ouroowes (ASSESswENT 1) grade or mark, It is therefore known in advance what percent of the students would pass or fail a given course. For this reason, many opponents to norm-referenced grading aver that such a grading system does not advance the cause of education and contradicts the principle of individual differences. In norm-referenced grading, the students, while they may work individually, are actually in competition to achieve a standard of performance that will classify them into the desired grade range. It essentially promotes competition among students or pupils in the same class. A student or pupil who happens to enroll in a class of gifted students in Mathematics will find that the norm-referenced grading system is rather worrisome. For example, a teacher may establish a grading policy whereby the top 15 percent of students will receive a mark of excellent or outstanding, which in a class of 100 enrolled students will be 15 persons. Such a grading policy is illustrated below: 1.0 (Excellent) = Top 15 % of Class 1.50 (Good) = Next 15 % of Class 2.0 (Average, Fair) = Next 45 % of Class 3.0 (Poor, Pass) = Next 15 % of Class 5.0 (Failure) = Bottom 10 % of Class The underlying assumption in norm-referenced grading is that the students have abilities (as reflected in their raw scores) that obey the normal distribution. The objective is to find out the best performers in this group. Norm-referenced systems are most often used for screening selected student populations in conditions where it is known that not all students can advance due to limitations such as available places, jobs, or other controlling factors. For example, in the Philippine setting, since not all high school students can actually advance-to college or university level because of financial constraints, the norm- referenced grading system. can be applied. r Ex Example: In a class of 100 students, the mean score in a test is 70 with a standard deviation of 5. Construct a norm- referenced grading table that would have seven-gtadeChapter 8 ~ Grading Systems and the Grading Systems ofthe Department of Education scales and such that students scoring between plus or minus one standard deviation from the mean receives an average grade. Solution: The following intervals of raw scores to grade equivalents are computed: Grade Equivalent Below 55 Only a few of the teachers who use norm-referenced grading apply it with complete consistency. When a teacher is faced with a particularly bright class, most of the time, he does not penalize good students for having the bad luck to enroll in a class with a cohort of other very capable students even if the grading system says he should fail a certain percentage of the class. On the other hand, it is also unlikely that a teacher would reduce the mean grade for a class when he observes a large proportion of poor performing students just to save them from failure. A serious problem with norm-referenced grading is that, no matter what the class level of knowledge and ability, and no matter how much they learn, a predictable proportion of students will receive each grade. Since its essential purpose is to sort students into categories based on relative performance, norm-referenced grading and evaluation is often used to weed out students for limited places in selective educational programs. Norm-teferenced grading indeed promotes competition to the extent that students would rather not help fellow students because by doing so, the mean of the class would be raised and consequently it would be more difficult to get higher grades. Similarly, students would do everything (legal) to pull down theASSESSwENT OF Leann OuTcowes (Assessment 1) scores of everyone else in order to lower the mean and thus assure hinvher of higher grades on the curve. A more subtle problem with norm-referenced grading is that a strict correspondence between the evaluation methods used and the course instructional goals is not necessary to yield the required grade distribution. The specific learning objectives of norm-referenced classes are often kept hidden, in part out of concem that instruction not “give away” the test or the teacher’s priorities, since this might tend to skew the curve. Since norm- referenced grading is replete with problems, what alternatives have been devised for grading the students? 8.2. Criterion-Referenced Grading Criterion-referenced grading systems are based on a fixed criterion measure. There is a fixed target and the students must achieve that target in order to obtain a passing grade in a course regardless of how the other students in the class perform. The scale does not change regardless of the quality, or lack thereof, of the students. For example, in a class of 100 students using the table below, no one might get a grade of excellent if no one scores 98 aboye or 85 above depending on the criterion used. There is no fixed percentage of students who are expected to get the various grades in the criterion-referenced grading system. 1.0 (Excellent) =98-100 or 85-100 1.5 Good) = 88-97 or 80-84 2.0 (air) Sd or 70-79 3.0 (Poor/Pass) = 65-74 or 60-69 5.0 (Failure) = below 65 or below 60 Criterion-referenced systems are often used in situations where the teachers are agreed on the meaning‘of a “standard of performance” in a subject but the quality of the students is unknown or uneven; where the work inVolves student collaboration or teamwork, and where there’ is no external driving factor such as needing to systematically reduce a pool of eligible students.Chapter 8 - Grading Systems and the Grading Systems ofthe Department of Education Note that in criterion-referenced grading system, students * can help a fellow student in a group work without necessarily wertying about lowering his grade in that course. This is because the criterion-referenced grading system does not require the mean (of the class) as basis for distributing grades among the students. It is therefore an ideal system to use in collaborative group work. When students are evaluated based on predefined criteria, they are freed to collaborate with one another and with the instructor. With criterion-referenced grading, a rich leaming environment is to everyone’s advantage, so students are rewarded for finding ways to help each other, and for contributing to class and small group discussions. Since the criterion measure used in criterion-referenced grading is a measure that ultimately rests with the teacher, it is logical to ask: What prevents teachers who use criterion- referenced grading from setting the performance criteria so low that everyone can pass with ease? There is a variety of measures used to prevent this situation from ever happening in the grading system. First, the criterion should not be based on only one teacher’s opinion or standard. It should be collaboratively arrived at. A group of teachers teaching the same subject must set the criterion together. Second, once the criterion is established, it must be made public and open to public scrutiny so that it does not become arbitrary and subject to the whim and caprices of the teacher. 8.3. Four Questions in Grading Marinila D. Svinicki (2007) of the Center for Teaching Effectiveness of the University of Texas in Austin poses four intriguing questions relative to grading. We share these questions here in this section and the corresponding opinion of Ms. Svinicki for your own reflection: J. Should grades reflect absolute achievement level or achievement relative to others in the same class? 2. Should grades reflect achievement only or nonacademic components such as attitude, speed and diligence? 3. Should grades report status achieved or amount of growth? 134 : ‘} —— ~_oe 132 ‘Assessuent oF Leanne Ourcowes (Assessvenr 1) 8.4, a 4. How can several grades on diverse skills combine to give a single mark? What Should Go into a Student’s Grade The grading system an instructor selects reflects his or her educational philosophy. There are no right or wrong systems, only systems which accomplish different objectives. The following are questions which an instructor may want to answer when choosing what will go into a student's grade. Should grades reflect absolute achievement level or achievement relative to others in the same class? This is often referred to as the controversy between norm- referenced versus criterion-referenced grading. In norm-referenced grading systems the letter grade a student receives is based on his or her standing in a class. A certain percentage of those at the top receive A’s, a specified percent of the next highest grades receive B’s and so on. Thus an outside person, looking at the grades, can decide which student in that group performed best under those circumstances. Such a system also takes into account circumstances beyond the students’ control which might adversely affect grades, such as poor teaching, bad tests or unexpected problems arising for the entire class. Presumably, these would affect all the students equally, so all performance would drop but the relative standing would stay the same. On the other hand, under such a system, an outside evaluator has little additional information about what a student actually knows since that will vary with the class. A student who has learned an average amount in a class of geniuses will probably know more than a student who is average in a class of low ability. Unless the instructor provides more information than just the grade, the external user of the grade is poorly informed. The system also assumes sufficient variability among student performances that the difference in learning between them justifies giving different grades. This may be true in large beginning classes, but is a shaky assumption where the student population is homogeneous such as in upper division classes.‘Chapter 8 - Grading Systems and the Grating Systems of the Department of Education 133 The ther most common grading system is the criterion- referenced system. In this case the instructor sets a standard of performance against which the students’ actual performance is" measured. All students achieving a given level receive the grade’ assigned to that level regardless of how many in the class receive the same grade. An outside evaluator, looking at the grade, knows only that the student has reached a certain level or set of objectives. The usefulness of that information to the outsider will depend on how much-information he or she is given on what behavior is represented by that grade. The grade, however, will always mean the same thing and will not vary from class to class. A possible problem with this is that outside factors such as those discussed under norm-referenced grading might influence the entire class and performance may drop. In such a case all the students would receive lower grades unless the instructor made special allowances for the circumstances. A second problem is that criterion-referenced grading does not provide “selection” information. There is no way to tell from the grading who the “best” students are, only that certain students have achieved certain levels. Whether one views this as positive or negative will depend on one’s individual philosophy. An advantage of this system is that the criteria for various grades are known from the beginning. This allows the student to take some responsibility for the level at which he or she is going to perform. Although this might result in some students working below their potential, it usually inspires students to work for a high grade. The instructor is then faced with the dilemma of a lot of students receiving high grades. Some people view this as a problem. A positive aspect of this foreknowledge is that much of the uncertainty which often accompanies grading for students is eliminated. Since they can plot their own progress toward the desired grade, the students have little uncertainty about where they stand. With competency-based teaching-learning or outcome -based teaching-learning observed in Philippines school the criterion-referenced system is what is used in the country. 2. Should grades reflect achievement only or non-academic components such as attitude, speed and diligence? It is a very common practice to incorporate such things as turning in assignments on time into the overall grade in a‘Assessnenr oF Leanne Ourcowes (Assesswent 1) course, primarily because the need to motivate students to get their work done is a real problem for instructors. Also it may be appropriate to the selection function of grading that such values as timeliness and diligence be reflected in the grades. External users of the grades may be interpreting the mark to include such factors as attitude and compliance in addition to competence in the material. The primary problem with such inclusion is that it makes grades even more ambiguous than they already are. It is very difficult to assess these nebulous traits accurately or consistently. Instructors must use real caution when incorporating such value judgments into final grade assignment. Two steps instructors should take are (1) to make students aware of this possibility well in advance of graded assignment and (2) to make clear what behavior is included in such qualities as prompt completion of work and neatness or completeness. In short, non-academic component such as attitude, speed and diligence may be refflected in the student’s grades provided they are informed in advance and that these qualities should be well understood. 3. Should grades report status achieved or amount of growth? This is a particularly difficult question to answer. In many beginning classes, the background of the students is so varied that some students can achieve the end objectives with little or no trouble while others with weak backgrounds will work twice as hard and still achieve only half as much. This dilemma results from the same problem as the previous question, that is, the feeling that we should be rewarding or punishing effort or attitude as well as knowledge gained. A positive aspect of this foreknowledge is that much of. the uncertainty which often accompanies grading for students is eliminated. Since they can plot their own progress toward the desired grade, the students have little uncertainty. about where they stand. si There are many problems with “growth” measures as a basis for change, most of them being related to sfatistical . artifacts. In some cases the ability to accurately measure entering and exiting levels is shaky enough to argue against change as a basis for grading. Also many courses are prerequisites to later courses and, therefore, are intended to provide the foundation for those courses. “Growth” scores in this case would be disastrous. a i es ee eR 6 alsChapter 8 - Grading Systems and the Grading Systems of the Department of Education Nevertheless, there is much to be said in favor of “growth” Ss as a component in grading. We would like to encourage hard ‘work and effort and to acknowledge the existence of different abilities, Unfortunately, there is no easy answer to this question. Each"instructor must review his or her own philosophy and contdat to determine if such factors are valid components of the grade. 4. How can several grades on diverse skills combine to give a single mark? The basic answer is that they can’t really. The results of instruction are so varied that the single mark is really a “Rube Goldberg” (doing something by an unnecessarily complicated means what could be done simply) as far as indicating what a student has achieved. It would be most desirable to be able to give multiple marks, one for each of the variety of skills which are learned. There are, of course, many problems with such a proposal. It would complicate an already complicated task. There might not be enough evidence to reliably grade any one skill. The “halo” effect of good performance in one area could spill over into others. And finally, most outsiders are looking for only one overall classification of each person so that they can choose the “best.” Our system requires that we produce one mark. Therefore, it is worth our while to see how that can be done even though currently the system does not lend itself to any satisfactory answers. 8.5, Standardized Test Scoring Test standardization is a process by which teacher or researcher-made tests are validated and item analyzed. After ‘a thorough process of validation, the test characteristics are established. These characteristics include: test validity, test reliability, test difficulty level and other characteristics as previously discussed. Each standardized test uses its own mathematical scoring system derived by the publisher and administrators, and these do not bear any relationship to academic grading systems. Standardized tests are psychometric instruments whose scoring systems are developed by norming the test using national samples of test-takers, centering the scoring.‘Assessvent oF Leawwne Ourcoves (Assessuenr 1) formula to assure that the likely score distribution describes a normal curve when graphed, and then using the resulting scoring system uniformly in a manner resembling a criterion-referenced approach. If you are interested in understanding and interpreting the scoring system of a specific standardized test, refer to the policies of the test’s producers. 8.6. Cumulative and Averaging Systems of Grading In the Philippines, there are two types of grading systems used: the averaging and the cumulative grading systems. In the averaging system, the grade of a student on a particular grading period equals the average of the grades obtained in the prior grading periods and the current grading period. Example: Student's grades are: 80 - Prelim 90 - Midterm 85 - Final 80+90+85 = 85 3 85 is the final grade for the semester The Department of Education makes use of the averaging grading system. In the cumulative grading system, the grade of a student in a grading period equals his current grading period grade which is assumed to have the cumulative effects of the previous grading periods. Example: 80 - Prelim 90 - Midterm Grade 80 - Tentative Final Grade Final Grade = 1/3 of Midterm Grade + 2/3 of Tentative Final Grade 1/3 of 90 + 2/3 of 80 = 30 + 53.33 = 83 In which grading system would there be more Soy sons observed in the students’ grades? How do these systems relate with either norm or criterion-referenced grading?Chapter 8 ~ Grading Systems and the Grading Systems of the Department of Education 8.7. Policy Guidelines on Classroom Assessment + forthe K to 12 Basic Education, DepEd = Order No. 8, s. 2015 Helow are some of the highlights of the new K to 12 Grading System which was implemented starting SY 2015-2016. These are all lifted from DepEd Order No. 8, s. 2015 Weights of the Components for the Different Grade Levels and Subjects The student’s grade is a function of three components:1) written work, 2) performance tasks and 3) quarterly assessment. The percentages vary across clusters of subjects, Languages, Araling Panlipunan (AP) and Edukasyon sa Pagpapahalaga (EsP) belong to one cluster and have the same grade percentages for written work, performance tasks and quarterly assessment. Science and Math are another cluster with the same component percentages. Music, Arts, Physical Education and Health (MAPEH) make up the third cluster with same component percentages. Among the three components, performance tasks are given the largest percentages. This means that the emphasis on assessment is on application of concepts learned. Table 7. Weight of the Components for Grades 1-10 Written Work Performance 1 to 10 | Tasks 50% oF a | 20% | 20% | 20% ssessment Table 7 presents the weights of the components for the Senior High School subjects which are grouped into 1) core subjects, 2) all other subjects (applied and specialization) and work immersion of the academic track, and 3) all other subjects (applied and specialization) and work immersion/research/exhibit/performance. An analysis of the figures reveal that among the components, performance tasks have the highest percentage contribution to the grade. This means that DepEd’s grading system consistently puts most emphasis on application of learned concepts and skills. 137‘Assessuer oF Leanne Ourcowes (Assessutent 1) 138 Table 8. Weight of the Components for SHS Technical-Vocational and Livelihood (TVL)/ Sports/Arts and Design Track Work Immersion/ All breguse Terres V Business All other other ; : Research/ : a Enterprise ‘subjects. ; subjects ate Exhibit Simulation! Bi oe Exhibit! ae Performance —) Written Work 35% 20% "1 a Forrmance | sow | 45% 40% 60% ‘cussed 25% | 30% 25% 20%Chapter 8 — Grading Systems and the Grading Systems of the Department of Education 8.8. Steps in Grade Computation Based on the:same DepEd Order (8, s. 2015), here are the steps to follow in computing grades,” Table 9. Steps for Computing Grades STEPS EXAMPLE Tearner’s Raw Highest Possible Score Score Written Work 1 18 20 Written Work 2 2 28 Written Work 3 20 20 Written Work 4 17 20 Written WorkS 23. 25 Written Work6 26 30 Written Work 7 19 20 TOTAL 145 160 1. Get the total Learner's Raw Highest Possible score for each ‘Score component, Performance Task 1 12 15 Performance Task2 13 15 Performance Task3 19 25 Performance Task4 15 20 Performance TaskS 16 20 Performance Task6 25 28 TOTAL 100 120 Learner's Raw Highest Possible Score Score Quarterly 40 50 Assessment Percentage Score (PS}= om x 100% 160 PS of Written Work is 90.63. 2. Divide the total raw score by the highest _| Percentage Score (PS)= x 100% possible score 120 then multiply the quotient by | PS of Performance Task is 83.33. 100%, Percentage Score (PS)= ae x 100% 30 PS of Quarterly Assessment is 80.00.Assessvent oF Learwn Outcowes (Assessnent 1) 140 STEPS EXAMPLE 3. Convert Written Work for English Grade 4 is 30% Percentage Weighted Score (WS}= 90.63 x 0.30 Scores to ‘The Weighted Score of Written Work in 27.19. Weighted ‘Scores. Performance Tasks for English Grade 4 is 50%. Multiply the Weighted Score (WS)= 83.33 x 0.50 Percentage The Weighted Score of Performance Task is 41.67. Score by the weight ofthe | Quarterly Assessment for English Grade 4 is 20%. component Weighted Score (WS)= 80.00 x 0.20 indicated in The Weighted Score of Quarterly Assessment is 16. Table 4 and Table 5. {The scores can be found in the sample class record on Table 6.) 4. Add the Component Weighted Score Weighted Written Work= 27.19 Scores ofeach | Performance Tasks= 41.67 component. _| Quarterly Asscssment=16.00 The result will be the Initial | TOTAL 84.86 Grade, ‘The Initial Grade is 84.86, 5. Transmute the | The Initial Grade is 84.86 Initial Grade using the ‘The Transmuted Grade is 90. ‘Transmutation Table in ‘The Quarterly Grade in English for the 1st Quarter is 90. Appendix B. is it lected in the Card, For MAPEH, individual grades are given to each area, namely, Music, Arts, Physical Education and Health. The quarterly grade for MAPEH is the average of the quarterly grades in the four areas. ® QG for Music + QG for Arts + QG for PE + QO for Health (Ga) sr maeEn. 2 een ee 4 8.9, Grade Computation . What follows is a description of how grades are computed based on Deplid Order 8, s. 2015. pet 2, For Kindergarten There are no numerical grades in Kindergarten. Descriptions of the learners’ progress in the various learning areas are represented using checklists artd student portfolios. These are presented to the parents at the end of each quarter for discussion. Additional guidelines on the Kindergarten program will be issued. *CChapler 8 ~ Grading Systems end the Grading Systems ofthe Department of Education 141 ” For Grades 1-10 =the average of the Quarterly Grades (QG) produces the Final Grade. pat wo Ist-quarter grade + Ind-quarter grade + Srd-quarter grade + 4th-quarter grade Learning 7 Ora ii Area The General Average is computed by dividing the sum of all final grades by the total number of learning areas. Each learning area has equal weight. : General = Sum of Final Grades of All ing Areas Average . ‘Total number of Learning Areas in a grade level ‘The Final Grade in each learning area and the General Average are reported as whole numbers, Table 10 shows an example of the Final Grades of the different learning areas and General Average of a Grade 4 student. Table 1 al Grades and General Average Learning Area Seo Filipino English Mathematics Science Aralin Panlipunan Edukasyon sa | Pagpapakatao Edukasyong Pantahanan at Pangkabuhayan MAPEH. For Grades 11 and 12 The two quarters determine the Final Grade in a semester. Table 11 shows an example in Grade 11, second semester for the Accounting Business and Management (ABM) strand.ASSESSMENT OF LEARNING OuTCOMES (ASSESSMENT 1) eee Table 11. Grade 11, 2nd Semester of ABM strand 142 Second Subjects et a sementer 3 ze Final Core Subjects r Resding sod Weng Stile | tang Teksto tungo sa Pananaliksik Statistics and Probability Physical Science Physical Education and Health Applied and Specialized Subjects | Empowerment Technologies: ICT | for Professional Tracks Business Math (Organization and Management Fundamentals of Accounting, Business and Management 1 (. How is the learner’s progress reported? The summary of learner progress is shown quarterly to parents and guardians through a parent-teacher conference, in which the report card is discussed. The grading scale, with-its corresponding descriptors, js in the Table 12. Remarks are given at the end of the grade level, Table 12. Descriptors, Grading Scale, and Remarks DESCRIPTOR GRADING SCALE REMARKS Outstanding 90-100 Passed Very Satisfactory 85-89 Passed Satisfactory 80-84 Passed Fairly Satisfactory 75-79 Passed Pree Below 75 Failed Using the sample class record in Table 14. LEARNER A received an-initial Grade of 84.86 in English for the First Quarter which, when transmuted to-a grade of 90, is equivalent to Outstanding, LEARNER B received a transmuted grade of 88 which is equivalent to Very Satisfactory. LEARNER C received a grade of 71. which means that the learner Did Not Meet Expectations in the First Quarter of Grade 4 English.Chapter 8 — Grading Systems and the Grading Systems ofthe Department of Education 143 When a learner’s raw scores are consistently below expectations in Written Work and Performance Tasks, the learner's parents or guardians must be informed not later than the fifth week of that quarter. This will enable them to help and guide their child to improve and prepare for the Quarterly Assessment. A learner who receives a grade below 75 in any subject in a quarter must be given intervention through remediation and extra lessons from the teacher/s of that subject. 8.11. Promotion and Retention at the End of the School Year These are what DepEd Order 8, s. 2015 say. A Final Grade of 75 or higher in all learning areas allows the student to be promoted to the next grade level, Table 13 specifies the guidelines to be followed for learner promotion and retention. Table 13. Learner Promotion and Retention Requirements Decision ] 1 Final Grade of at least 75 in all | Promoted to the next grade learning areas level af ‘Must pass remedial classes for learning areas with ee 2. Did Not Meet Expectations in not ee cmd Learners Be er ang oe Otherwise the learner is retained in the same grade level. 3. Did Not Meet Expectations in three | Retained in the same grade or more learning areas level T Final Grade of at least 75 in all | Promoted to the next grade learning areas level i ‘Must pass remedial dasses for learning areas with A sa 2. Did Not Meet Expectations in not ee eee cae Learners Sn ea Otherwise the leamer is retained in the same grade level. Z. Did Not Meet Expectations in three | Retained in the same grade or more learning areas level‘Assessvent oF Learwne Outcoves (ASsEssuENT 1) 8.12. Alternative Grading Systems Pass-Fail Systems. Other colleges and universities, faculties, schools and institutions use pass-fail grading systems in the Philippines, especially when the student’s work to be evaluated is highly subjective (as in the fine arts and music), there are no generally accepted standard gradations (as with independent studies), or the critical requirement is meeting a single satisfactory standard (as in some professional examinations and practicum). Non-Graded Evaluations. While not yet practised in Philippine schools, and institutions, non-graded evaluations do not assign numeric or letter grades as a matter of policy. This practice is usually based on a belief that grades introduce an inappropriate and distracting element of competition into the learning process, or that they are not as meaningful as measures of intellectual growth and development as are carefully: crafted faculty evaluations. Many faculty, schools, and institutions that follow a no-grade policy will, if requested, produce grades or convert their student evaluations into formulae acceptable to authorities who require traditional measures of performance. The. process of deciding on a grading system is a very complex one. The problems faced by teacher who tries to design a system which will be accurate and fair are common to any manager attempting to evaluate those for whom he or she is responsible. The problems of teachers and students with regard to grading are almost identical to those of administrators and faculty with regard to evaluation for promotion and tenure. The need for completeness and objectivity felt by teachers and administrators must be balanced against the need for fairness and clarity felt by students and faculty in their respective situations. The fact that the faculty member finds himself or herself in both the position of evaluator and evaluated should help to make him or her more thoughtful about the needs of each position.‘Chapter 8 Grading Systems and the Grading Systems ofthe Deperiment of Education 8.13, Exercises * i Define a:norm-referenced grading. What are some of the jssues’that confront a teacher using a norm-referenced grading system? Discuss. The following final grades are obtained in a class of Grave VI pupils: 80,81,82,83,84,82,80,81,79,77,88,83,89,90,91,90,78, 79,82,91, 92,90, 88,85,88,87,85,88,83,82,80,79,77,76,77,78,83,89,91,90,83,88, 86,83,80 a. Using a norm-referenced grading with a seven-point scale, determine the scores that would get a failing mark. What is your general impression on this? b. Using a norm-referenced grading with an 8-point grading scale, determine the scores that would get a failing mark. Compare this with the previous grading system above. Define a criterion-referenced grading. What are some of the issues that confront a teacher using a criterion-referenced grading system. Discuss. Using the data in Problem b, set a passing criterion of 78 and set equal intervals for all other grades above the passing criterion. How does your result compare with those of norm-referenced grading? In which grading system do you feel more comfortable? In a class of 100 pupils, the mean score in a test was determined to be 82 with a standard deviation of 7. Construct an 8-point grading scale using the standard normal curve in a norm-referenced grading. Discuss, in your own words, the four essential questions in grading provided by Svinicki. Do you agree or disagree with her own points of view? Justify. Would you use the norm-referenced grading system in your own class? Why or why not? When would a norm-referenced grading system be most appropriate to use? Similarly, when would a criterion- referenced grading system be most appropriate to use? —————————E———————E 145‘Assesswent oF Learns Ourcoves (Assesswent 1) 9. Compute the grade of a student in: a) Grade 9 English with the following raw scores Written work — 80 out of 100 Performance task — 60 out of 100 Score in Quarterly Test — 50 out of 100 b) Grade 11 student in Introduction to the Philosophy of the Human Person, a core subject in SHS with the following raw scores: Written work — 30 out of 50 Performance task — 42 out of 60 Quarterly test — 28 out of 40 c) Grade 3 student in the Mother Tongue subject: with the following raw scores: ‘ Written work — 20 out of 30 Performance task — 25 out of 40 Quarterly test — 22 out of 30Chapter 8 Grading Systems and the Grading Systems of the Department of Education APPENDIX A.° AREAS OF ANORMAL CURVE ee eeeeeSeEeESEeESESESsee # 0 1 2 3 4 6 6 7 8 o 00 © 000000404080 «0100- «0600109038 0279 03190888 01 ©0808» 40488S«w47B 05170887 «080606380675. OTIS 02 073 = 0882871810 08480887 1064 1064 1103141 09 © 17912171285 1203 138118881408 1H4. 14801817 04 1884. 1501 «1628-1664 1700 1772 = «180818441879 1738 08 101519801985 2019 205420882123 218721002224 229) 232423572388, 22 AAS 2349 06 1 Oe 24882518 07 2612 2642 «2673-2704 «27842764 «278428032882 08 2881 2910 ©2039-2967 2006 «3023305130780 «3183 09 «©8159-3186 3212 328832643280 $315 334033653389 10 34133488 HB] HBS 3808381 964 35773509321 Ll 36433665 3888-3708. 3728049 S110 3780 «38103880 12 384938803888 3007 3025 S044 3962 .9980 38074018 13 4032 40404064082 4000115 413141474162. AIT 14 4192 420742024296 425140654279 42024806419 15 488243454387 437043824904 4406 44184429 A] 16 448244834474 A484 44954508 4518 4528 4835 A5S, LT 488445644573 4582. 45914509 4608 4616 = 46254633 18 464148404056 «= 4G «4671 «46784686 480346094106 1 4113 4g 412632. ATOB CATH ATO ATG ATG TET 20 4772 «ATT. A7B3 4788 ATOS 4788. 4003 4808 «48124817 Ql» 4821 482648904834 48384798 4046 485048544857 22 ©4861 «48644888 AB7L «48754878 488148844887 «4800 23 ©4803. 48064808 490l. 40044006 4908 491149134916 a4. 4918 4020 «=«.4922« 492549274920 403]— 49924984 «4998 25 © 490840404041 4043 4045404648 4049496] 4952 26 ©4083-4055 4086 4957 4950 4960 «= 406149624983 4964 27 «498540854968 4968 4968 4970S A497] 4072—| 40734874 28 «49744075 4978-4077 0774078 «407949794980 «4081 29 496) «4082402 «493 40449844085 8084088288 #85 3a 4903 498349044004 4804 dtd 044005905 4995 «4005-4006. -4906« 4908 «490640964006 4007 34° 490749074987 4907 4907 490748074907 40074088 36 4008 «= 4008) 4008- 490849984098 «dang 400840984008 36 © 498d «999 ggg dg) 4000400949909 4000 37 —«.4990«s«wd909 ggg ggg) dogg) 00d dado 49094909 38 ©4909 409390994909 4009 4099-4909 4900 4008 39 5000 5000 8000 9000-5000 5000 800050005000‘Assessvent oF Learwnne Ourooues (Assessue 1) 148 APPENDIX B Transmutation Table 99 98 9 9360-9519] 98 92.00 - 93.59] 95 904 56,00 ~ 59.99 88,80 - 90.39| 93 52.00 - 55.99 85.60 — 87.19 40.00 - 43.99 70 8240-8399} 88 ~=——s | _:36.00 - 39.99 8 80.80 - 82.39 88 32.00 - 35.99 68 79.20 - 80.79 28.00 — 31.99 67 77.60 - 79.19 86 24.00 - 27.99 66 76.00 - 77.59 85 20.00 - 23.99 65 74,40 - 75.99 @ =|: 16.00 - 19.99 64 Z| 72.80 ~ 74.39 83 12.00 - 15.99 71.20 - 72.79 82 8.00 - 11.99 69.60 - 71.19 4.00 - 7.99 a 68.00 - 69.59 80 0-399 a149 ‘Chapter 8 Grading Systems and the Grading Systems ofthe Department of Education | APPENDIX C 8.8.1 Sample Report Cards for Grades 1 to 6 juauodmog Jo wfram x 8100g aBerUEOIOg = (uk) 98099 PANTAOM‘Assesswent oF Leanne Ourcowes (Assessuent 1) References Audet, R and Jordan, L. (2005) “Integrating inquires across the curriculum”. California: Sage Pub. Co. Baker, E. L., (1992) “The role of domain specifications in improving the technical quality of performance assessment” (CSE Tech. Rep.). Los Angeles: University of California, Center for Research on Evaluation, Standards, and Student Testing, Barton, P. E., and Coley, R.J., (1995) “Testing in America's schools policy informational report”, ETS, 46 p. Benner, Patricia, (1982) “From novice to expert,” American Journal of Nursing, 82 (3), 402-407 Bhatnagar, G and Sharma A. (1999) “Learn computing: a game plan”, Proceedings of the F13 conference, August 1999, NIIT Ltd., New Delhi. Brookhart, Susan M. (2013) How to create and use rubrics; for formative assessment and grading. Alexandria, VA:ASCD. Bryant, D., (1996) “A comparison of multiple choice versus alternative assessment: strengths and limitations”, The New York State Education Department, Albany, New York, (available at http://srww.nysed.govirscs/rschmult html) Darling - Hammond, L. and Bransford, J. (2005) Preparing teachers for a changing world. San Francisco: John Willey & Sons. Department of Education Order No. 8, s. 2015. Dunbar, S. B., Koretz, D., & Hoover H. D. (1991). “Quality control in the development and use of performance assessments”, Applied Measurement in Education, 4, 1991, 289-304. Gall, In. (2008) Educational Research: An Introduction. New York Allyn and Bacon. Herman, J., (1991) “Research in cognition and learning: implications for achievement testing,” In M. C. Wittrock & E. L. Baker (Eds.), Testing and cognition (pp. 154-165) Englewood Clifis, NI: Prentice Hall. Hemon, P, and Dugan, R. (2004) Outcomes assessment in higher education. Westport: Libraries Unlimited. Linn, R. L., Baker, E. L., & Dunbar, $, B., (1991) “Complex Sueiae ‘assessment: expectations and Validation Criteria”, Educational researcher, 20(8), 15-21. . Lipman, M., (1987) “Some thoughts on the formation of reflective education”, In teaching-thinking skills: theory and. practice, pp. 151-161. ated by 4B" Barqn and R. J. Stemberg. New York: W. H. Freeman. Lukhele, R., Thissen, D. and Wainer, H, (1994) “On the relative value.of multiple choice constructed response and examinee selected items on two- achievement tests”, Journal of Educational Measurement. 31 (3), 234-250 3ES ee eT ee ee eT eee ne ‘Chapter 8 ~ Grading Systems and the Grading Systéms of the Department of Education 151 Mehrens, W. A., (1992) “Using performance assessment for accountability purposes” Educational Measurement: Issues and Practice, 11(1), 3-20. > Schoehfeld, A. H. 1985) “Mathematical problem solving", Academic Press, Orlando FL, 19857 Shulman, L.’S and J. Shulman, (2004) “How and what teacherslearn”, Journal of Curriculum Strategies. 36 (2), 257 - 271. Seldon, A., Seldon, J., (1997) “What does it take to be an expert sproblem solver”, Research Sampler, MAA Online, No. 4.; (available at www.maa.org/t_and_1_/ sampleriresearch_sampler.html). Shavelson, R. J., Baxter, G. P, & Pine J,, (1992) “Performance assessments: Political rhetoric and measurement reality”, Educational Researcher, 21(4), 22-27. Tuckman B. (1993) “The essay test: A look at the advantages and disadvantages” NASSP Bulletin, 77, pp. 20-26. Zaremba, S., and Schultz, M., (1993),"An analysis of traditional classroom assessment techniques and discussion”, ED 365404, 1993, p. 13 Danielson, C. and Leslye Abrutyn (1997) Introduction to using portfolios in the classroom. ASCD: VA Bookhart, S. How to create and use rubrics for formative assessment and grading. (2013) ASCD:VA Anderson, L. W. & Krathwobl, D, R. (2001). 4 taxonomy for learning, teaching, and assessing. New York: Longman. Bloom, BS., (Ed.). 1956. Taxonomy of educational objectives: The classification of educational goals: Handbook I, cognitive domain. New York: Longman. Costa, A. L. (Bd). (2000). Developing minds: A resource book for teaching thinking. Alexandria, VA: ASCD. Marzano, R. J. (2000). Designing a new taxonomy of educational objectives. Thousand Oaks, CA: Corwin Press. Anderson, L. W. and Krathwohl, D. R., et al (Eds..) (2001) 4 Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives. Allyn & Bacon. Boston, MA (Pearson Education Group) Bloom, B.S. and Krathwohl, D. R. (1956) Taxonomy of Educational Objectives; The Classification of Educational Goals, by a committee of college and university examiners. Handbook I: Cognitive Domain. NY, NY: Longmans, Green Krathwobl, D. R. (2002) 4 Revision of Bloom's Taxonomy. (PDF) in Theory into Practice. V 41. #4. Autumn, 2002, Ohio State University. DR. Krathwohl, B.S. Bloom, B.B. Masia (1964) Taxonomy of Educational Objectives: Handbook U-Affective Domain, New York: David Mckay Co. Kaplan, RM. and Saccuzzo, DP. (2001) Psychological testing: principles, applications and issues. Singapore: Thomson Learning Asia.About the Authors ROSITA L. NAVARRO, LPT, PHD, is a career educator who stared as high school teacher of English and Mathematics after earning the BSE degree, Summa Cum Laude, in Centro Escolar Univers. She steadily rose the academic ladder through merit and regular honing of her expertise in teaching, research, curriculum evaluation and development, qualty assurance, and educational admnistration which culminated to ter appointment as sith President of Centro Escolar University, the first to rise from the ranks (2001-2007) Dr. Navarro reached the summit of her teaching career when she received two (2) prestigious awards “Outstanding Professional Teacher,” awarded in 2000 by the Professional Regulation Commission (PRC) and “Outstanding Professional in the Field of Teaching” awarded in 1997 by the Philippine Federation of Professional Associations (PFPA). She has produced more than a hundred researches, research-based articles, books and instructional materials ‘A recognized leader, she served as President of the folowing professional organizations: Philippine Assocation {or Teacher Education (PAFTE), Philippine Assocation for Graduate Educaton (PAGE), Philippine Soaety for Educational Research and Evaluaton (PSERE), Association of Southeast Asian Instiutons of Higher Learning (ASAIHL). At present she is Consultant on Quality Assurance and TOM of St. Jude College of Mania and Cavite, and Charman of the Phiippine Association of Colleges and Universities Commrssion on Accreditation (PACUCOA), ROSITA G. SANTOS, LPT, PhD, cared her doctorate in Mathematics Education (1987) and Master of Science in Mathematics Education (1970) from Centro Escolar Universi. Her Baccalaureate degree in Science Education major in Mathematics, minor in General Science (1966, Cum Laude), was also obtained from Centro Escolar University ‘She was Senior Assistant Vice President for Research and Evaluation (1995- 2006), Assistant Vice President for Research and Development (1993-1995); Presidential ‘Assistant for Research and Development (1982-1263), University Professor (Highest rank in CEU ranking System) and Graduate School Professor (1997-2008), Charman of the CEU Research and Development Foundation at Centro Escolar Univesity. Dr. Santos has served the Philippine Association for Teacher Education (PAFTE) in various positons as Treasurer and Director of PAFTE National and Philippine Society for Educational Research and Evaluation (PSERE) 1995-2004 as President At present, she is Senior Faculty Research Consultant at the Lyceum of the Philippines University (LPU) in Manila. Among her many contributions to research and development literature are research studies in teaching methodologies, predictive validity study, and pre-service elementary teacher training validaton of departmentaized fests. She has authoredico-authored books 1 mathematics, statstis, educatonal research, educational evaluation and measurement BRENDA B. CORPUZ, LPT, PhD, is a recipient of the Outstanding Professional of the Year Award in the field of Teaching ftom the Professional Regulation Commission on June 20, 2014. She brings into her writings the precipitate of years of teaching in basic and in higher education, administrative Experiences wth Coleges of Education and Graduate School, lectures delivered ‘and researches conducted and presented in the country and abroad, consultancy services with the Department of Education funded by World Bank and AusAID (now DFAT), the latest of which is the K to 12 Curriculum, membership in the Board for Professional Teachers and experiences as LET examiner for ten years and visits to schools in Asia, America and New Zealand. She was involved in teacher education 2s a presidential appointee to the Teacher Education Counc as representative of higher education institutions in Luzon, past chair of the Technical Committee for Educatonal Leadership and Foundation and past imember of the Technical Panel for Teacher Education of the Commission on Higher Education and past, member of the Continuing Professional Development Council for Teachers of the Professional Regulation Commission. ‘She Is a much-sought-after lecturer in K to 12 Curriculum, OBE and OBTL, Teaching Strategies, ‘Assessment and in other topios in education. She is the incumbent president of the, Philippine Association & for Teachers and Educators (2018-2010) and the Dean of the College of Education and Director of the Center for Teaching Excellence of the Technological Institute of the Philippines. i = ‘She is a product of ICM and CICM education having finished her Bachelor of Science in Secondary & = Education, major in English (Magna cum laude), Master of Arts in “Education «major in Biology (GPA z equivalent to Magna cum laude) and Doctor of Philosophy in Education malar in Educafonal Management ‘ {GPA equivalent to Magna cum laude) from St. Louis University, Baguio City. : — Published by re» ©2019 ISBN 978-621-8035-62-1