WHATS PROFICIENT?

THE NO CHILD LEFT BEHIND ACT AND THE MANY M EANINGS OF PROFICIENCY

Bella Rosenberg May 2004

WHATS PROFICIENT?
The No Child Left Behind Act and the Many Meanings of Proficiency

Each State shall establish a timeline for adequate yearly progress. The timeline shall ensure that not later than 12 years after the end of the 2001-2002 school year, all studentswill meet or exceed the States proficient level of academic achievements on the State assessments. No Child Left Behind Act of 2001 (emphasis added). proficient adj. Having or marked by an advanced degree of competence, as in an art, vocation, profession, or branch of learning. n. An expert; an adept. The American Heritage Dictionary of the English Language, Fourth Edition 2000.

Table of Contents
Introduction.. Different methods of setting performance standards (or achievement levels) yield different meanings of proficient. Different tests , even in the same subject, yield different percentages of proficient students Proficient typically means different things in different grades, even on the same test and in the same subject. Different percentages of proficient students in different subjects do not necessarily mean that students or schools are stronger or weaker in one subject than the other................................................... Proficient means different things in differe nt states. NAEP proficiency standa rds are set very high. NCLBs requirement of 100 percent proficiency is not the same thing as requiring that all students achieve at grade level. 3

6 7 9

10

The No Child Left Behind Act and the Many Meanings of Proficiency The goal of the No Child Left Behind Act (NCLB) is to have 100 percent of Americas public school students proficient by the year 2014, 12 years from the enactment of the law. Proficiency is to be measured through annual state-level tests in reading and math in each of grades 3-8 and at least once in high school. 1 In 2007-08, states must also test in science at least once in grades 3-5, 6-9, and 10-12. The goal of 100 percent proficiency is certainly a giant leap forward from Americas past expectations for minimal competency or universal attainment of basic skills or even being first in the world in international comparisons of student achievement. Indeed, NCLBs goal is globally unprecedented. Not even the highest-scoring nations that have participated in international tests of reading, math, and science are even close to attaining it. 2 Can 100 percent proficiency be accomplished? Setting aside iron-clad facts about individual variability, the unprecedented, uniquely ambitious nature of NCLBs goal is not necessarily a reason to doubt that it can be realized by 2014. After all, Americans have set extremely challenging goals before landing an astronaut on the moon comes to mind and have atta ined them. And certainly 100 percent proficiency is at least as important, beneficial, and even thrilling a goal as landing an astronaut on the moon. In the case of our moon goal, however, while most Americans didnt know what was involved in attaining it , we all knew what it meant and how we would know if it had been achieved: there would be an astronaut safely on the moon. But what does NCLBs goal of 100 percent proficiency mean? Whats proficient? Most parents and the public likely believe that 100 percent proficiency means that all students are on grade level. When U.S. Department of Education officials, as well as the media, say that the goal of NCLB and its adequate yearly progress (AYP) provision is having all students performing at grade level, they certainly encourage that belief.3 Are they right? The chances are also good that if you asked the proverbial man on the street if a student who scored proficient on one fourth-grade math test would also score proficient on a different fourthgrade math test, he would answer yes. Moreover, if you asked him what his conclusion would be if School X had a higher percentage of students who were proficient in reading than in math, the likely answer would be that the school was doing a better job with reading. Similarly, if you told him that 50 percent of the fourth-graders in School Y were proficient in reading but only 37 percent of the fifth-graders were, hed likely conclude that the fourth-grade teachers were clearly superior to the fifth-grade teachers at teaching the subject. True?

WHATS PROFICIENT?

States have until 2006 to test in these grades. Twenty states and the District of Columbia already do so. The rest may continue to test reading and math at least once between each of grades 4-6 and 6-8 and once in high school, as the previous version of the Elementary and Secondary Education Act required. 2 Robert L. Linn, Assessments and Accountability, Educational Researcher, 29, 2 (2000): pp. 4-16. 3 See, for example, the remarks of Eugene Hickok, acting deputy secretary of education, on the Lehrer NewsHour on March 15, 2004 (http://www.pbs.org/newshour/bb/education/jan-june04/schools_3-15.html); and Diana Jean Schemo, 14 States Ask U.S. To Revise Some Education Law Rules, New York Times, March 25, 2004, online edition.

And what about the inferences that are made about the quality of education when results from the National Assessment of Educational Progress (NAEP), the nations report card, come out?4 Who was not alarmed when, for example, former President Clinton reported that almost half of American fourth-graders were below basic in reading, or when presidential candidate George Bush ran ads in 2000 using similar NAEP results to say that almost half of our fourth-graders could barely handle a picture book? How many editorials decried our failing public schools when the most recent 12th -grade NAEP results showed only 36 percent of students at the proficient level in reading and an even paltrier 17 percent proficient in math? But are all these perfectly understandable inferences that parents and the public typically make and that politicians and the media so often promote correct? Not necessarily. It all depends on how proficient is defined. Welcome, then, to the many meanings of proficient and a closer look at NCLBs goal of 100 percent proficiency and making adequate yearly progress in getting there by 2014.

Different methods of setting performance standards (or achievement levels) yield different meanings of proficient.
Proficient is essentially a cut score (e.g., 90 out of 100 questions correct) on a test, though these days a cut score is often referred to as a performance standard or achievement level. There are a number of ways of deciding how to set that cut score, but under standards-based reform, its typically done by convening a committee, which usually includes many members from outside education, and putting them through one of many available standard-setting processes. These processes are more or less scientific, and each will yield a different result. Different committees using the same method may also come up with different results. In the end, however, where cut scores are set under standards-based reform is a matter of judgment about how students ought to perform on a test. Some committees set the cut scores for different levels of achievement (e.g., basic, proficient, advanced) very high, others lower, and others in between. So, for example, even if two different states used the same test, a student in one state might have to score 90 to be considered proficient, while in the other, the student could be proficient with a 60. All things being equal, states that have higher cut scores for proficiency will have higher AYP failure rates than states with less challenging cut scores.

Different tests, even in the same subject, yield different percentages of proficient students.
Below is a table displaying the percentage of students labeled proficient or advanced on three different achievement test batteries from major test publishers and NAEP. Each of these national tests actually uses the label proficient and advanced. But note the widely varying results in the percentages of students who score proficient on them.

NAEP is administered nationally to a representative group of students in the fourth, eighth, and 12th grades and has been tracking progress in key academic subjects since 1968. In 1990, NAEP was also given the authority to report state-level results, but participation by the states was voluntary. Under NCLB, state participation became mandatory. The idea is to use NAEP as a check on the states proficiency levels.

Table 1. Percent of Students in the Nation Labeled Proficient or Advanced by Three Test Batteries and NAEP Reading Comprehension Text X Test Y Test Z 64 43 14 58 28 41 56 37 14 55 40 24 49 32 33 47 31 18 47 39 25 47 39 33 49 39 11 50 34 15 53 23 25 54 22 29 Mathematics Test X Text Y 49 41 49 34 46 32 44 34 42 28 41 24 39 19 37 23 37 17 35 11 33 7 30 5

Grade 1 2 3 4 5 6 7 8 9 10 11 12

NAEP 1998

31

33

40

Test Z 12 40 5 15 30 12 21 33 9 12 24 27

NAEP 2000

26

27

17

Source: H.D. Hoover, Some Common Misconceptions about Tests and Testing, 2002 National Council on Measurement in Education Presidential Address, Educational Measurement: Issues and Practice (Spring 2003): pp. 5-14, Table 11, p.11.5

So, whats the real percentage of eighth-graders who are proficient in math? Test Xs 37 percent, test Ys 23 percent, test Zs 33 percent, or NAEPs 27 percent? The answer is that all these figures are real and equally meaningless without information about the content and skills these tests are measuring, the difficulty of the test items, and where the cut score was set. Whats clear, however, is that all other things being equal, a lower percentage of students would be found proficient, and a higher percentage of schools would fail AYP, in a state that used test Y or Z than in a state that used test X.

Proficient typically means different things in different grades, even on the same test and in the same subject.
Understandably, most people would think that proficient meant the same thing throughout the grades, even though they also know that different, or different levels of, content and skills are taught across the grades. Therefore, if they looked at Test X and Test Y in Table 1 for both reading and math, they would likely conclude that school quality deteriorates from the first to the 12th grade or that students get a lot dumber over time or both. And imagine explaining to parents test Zs wildly different percentages of proficient students in first, second, third, and other grades! The fact is, the criterion for proficient (or basic or advanced) is rarely comparable across the grades, even in the same subject. In order for there to be comparability, at least some, if not all, of the following would have to occur: the content and rigor of a states academic standards, its tests, and the cut points set on the tests would have to be calibrated across the grades. Few, if any, states can claim that. Therefore, parents and the public are likely, naturally enough, to reach all sorts of invalid conclus ions if, say, AYP failure rates are higher in high schools than they are in elementary schools. It just may, or may not, be because the content, tests, and cut points are higher for the high schools. Parents are in for an especially difficult time because they will get NCLB-mandated school report cards that must break out the percentage of proficient students by grade level. If the results bounce around anything like those on Tests X, Y, and Z and these are real results from real
5

Hoover writes (p.9) that tests X, Y and Z are produced (though not necessarily in the order Hoover lists them) by CTB/McGraw-Hill, Harcourt Educational Measurements, and Riverside Publishing Company.

tests theres likely to be a lot of understandable but misdirected anger towards teachers in some grades. As a recent report by the Northwest Evaluation Association (NWEA) put it, Standards that are not calibrated give students, parents, and educators an inaccurate perception about the childs standing relative to the expected level of performance. Students are reported as proficient in one grade who may not remain proficient in later grades even if they show normal growth.6

Different percentages of proficient students in different subjects do not necessarily mean that students or schools are stronger or weaker in one subject than the other.
No one would blink an eye if told that students were better in reading than in math, or vice versa. People are not surprised by differences in achievement in different subjects, and few parents have children who are equally strong or weak in all their school subjects. So, in looking back at Table 1, it would seem reasonable to conclude that in virtually every grade and on each of the four displayed tests, a higher percentage of students are proficient in reading than in math. Similarly, if the AYP status report of a certain elementary school showed that 50 percent of the students, on average, were proficient in math and 40 percent were proficient in reading, parents there would almost surely believe that the school was doing a better job in math than in reading. But they would not necessarily be correct. Is the reading and math content being measured by the reading and math tests of comparable difficulty? Are the reading and math test items of comparable difficulty? How can you even compare achievement in reading and math or in other subjects when these subjects are so different? The fact is you cannot. The best you can do is to compare where the cut points are set on tests of different subjects to see if it is harder to reach proficiency in one than the other. A recent study by the NWEA examined this question of whether proficiency levels are consistent among subject areas, even within the same state. The researchers took the results from reading and math tests given in 14 states and also administered an NWEA test to the same sample of students who had taken the different state tests; the point was to be able to place the results from different state tests onto a common measurement scale so that they could be compared. As Table 2 shows, in no state were the seventh- or eighth-grade cut scores for proficiency in reading and math set at the same level of difficulty. Moreover, in all states but Oregon, the cut score for math proficiency was more difficult than the standard for reading. In a few states, the difference was quite large. In Arizona, for example, the cut score for proficiency in math would not be reached by 75 percent of NWEAs norm group of students, while only 47 percent would not reach the proficient cut score in reading. 7

G. Gage Kingsbury, Allan Olson, John Cronin, Carl Hauser, and Ron Houser, The State of State Standards, Northwest Evaluation Association, http://www.nwea.org/research/statestudy.html, November 24, 2003, p. 15. 7 Ibid., pp. 18-21.

Table 2 Differences in Reading and Mathematics Percentile Cut Scores for the Proficient Level of Performance Grade 7 or Grade 8

State Arizona Colorado Wyoming South Carolina Idaho Washington Texas Minnesota Illinois Indiana California Montana Iowa Oregon

Math cut score (Percentile) 75 31 89 80 46 78 35 42 40 42 59 36 36 50

Reading cut score (Percentile) 47 12 74 68 32 67 24 32 32 35 54 35 35 58

Difference 28 19 15 12 14 11 11 10 8 7 5 1 1 -8

Source: Kingsbury et al, The State of State Standards, Northwest Evaluation Association (NWEA), November 24, 2003, http://www.nwea.org/research/statestudy.html, Table 7, p. 19.

Alas, the transparency afforded by the NWEA study is not readily available to parents, the public, and the media or even to most educators when they get school, district, state, or national proficiency results. Its easy to understand, then, why school board members who face tough decisions about how to deploy scarce resources would redirect funds from reading to math when the tests tell them that more students are proficient in reading than in math. Maybe that was the appropriate thing to do, and maybe not. What is certain, however, is that you cannot reach a valid conclusion about relative academic strength based only on the percentage of proficient students in one subject versus another.

Proficient means different things in different states.

Since states have different content standards and use different tests and different methods for setting cut scores, obviously the meaning of proficient varies among the states. Under NCLB, states are free to set their own standards for proficiency, which is one reason why AYP school failure rates vary so widely across the states. Its a lot harder for students to achieve proficiency in a state that has set that standard at a high level than it is in a state that has set it lower. Indeed, even if students in two schools in two different states have exactly the same achievement, one school could find itself on a failed-AYP list simply because it is located in the state whose standard for proficient is higher than the other states. But how great are the differences among the states in their standards for proficiency? The NWEA study also helps to address this question, while underscoring again the differences within states in the meaning of proficiency across grades and between math and reading. As Table 3 shows, states vary, sometimes dramatically, in the rigor of their proficiency standards. For example, the cut score for proficient in grade 4 reading ranged from the 73rd percentile in

Table 3 - Cut Scores Representing "Proficient" or "Meets Standards" Level of Performance on 14 State Assessments Reading
State SC CA MN OR ID MT IL IN IA AZ TX CO Grade 3 Cut Score 205 200 193 193 193 193 193 192 191 190 179 179 %ile 67 51 35 35 35 35 35 32 31 29 13 13 State WY SC WA CA ID MT IA CO Grade 4 Cut Score 214 213 207 205 200 196 196 191 %ile 73 70 53 46 34 26 26 18 State SC CA AZ OR IL MT ID IA MN TX CO Grade 5 Cut Score 220 214 210 209 207 206 206 205 204 204 197 %ile 73 54 45 42 37 35 35 32 30 30 18 State SC CA MT ID IN IA TX CO Grade 6 Cut Score 221 216 211 211 210 209 208 197 %ile 63 46 35 35 32 30 28 11 State SC WA CA MT IA ID TX CO Grade 7 Cut Score 227 226 221 218 216 215 210 206 %ile 70 67 50 43 37 35 24 18 State WY SC OR CA AZ IN MT IA ID IL MN CO Grade 8 Cut Score 232 230 227 226 224 219 219 219 218 218 218 206 %ile 74 68 58 54 49 35 35 35 32 32 32 12 State MT IA ID CO Grade 9 Cut Score 224 224 221 204 %ile 43 43 37 9 State OR WA ID MT IA CO CA Grade 10 Cut Score 236 227 224 224 223 209 208 %ile 77 51 44 44 42 15 14

Mathematics
State SC CA IN OR AZ MN MT IA ID IL Grade 3 Cut Score 208 204 201 199 199 198 197 197 196 193 %ile 75 60 50 46 46 42 39 39 36 29 State WY WA SC CA ID IA MT Grade 4 Cut Score 221 218 217 212 205 205 205 %ile 83 76 74 59 39 39 39 State SC CA AZ OR ID MT IA MN IL TX CO Grade 5 Cut Score 227 225 220 215 213 212 212 210 210 209 201 %ile 76 70 59 46 41 38 38 33 33 31 15 State SC CA IN ID IA MT CO Grade 6 Cut Score 235 230 221 219 218 218 207 %ile 78 67 47 42 40 40 19 State SC WA CA ID MT IA TX CO Grade 7 Cut Score 242 242 238 225 224 222 221 216 %ile 78 78 70 44 42 38 35 26 State WY SC AZ CA OR ID MN IN IL MT IA CO Grade 8 Cut Score 257 251 248 240 235 233 231 231 230 228 228 225 %ile 89 80 75 59 50 46 42 42 40 36 36 31 State MT IA ID CO Grade 9 Cut Score 242 241 240 235 %ile 47 44 42 32 State WA MT IA OR ID CO CA Grade 10 Cut Score 257 247 247 245 242 233 232 %ile 73 40 40 33 25 14 13

IN tests students in the fall. Their cut scores were adjusted to reflect equivalent spring performance. CO uses the partially proficient level of performance for NCLB reporting. To maintain consistency we report the level each state uses for NCLB reporting here. The TX estimate is based on the level for proficient performance that will be implemented in 2005. Source: Kingsbury et al, The State of State Standards, Northwest Evaluation Association, http://www.nwea.org/research/statestudy.html on Nov. 24, 2003, Table 4, p. 13.

Wyoming a very high standard to the 18th percentile in Colorado. 8 Similarly , the proficient cut score in grade 10 math ranged from the 73rd percentile in Washington to the 13th percentile in California. Does this mean that some states standards for proficient are right and others wrong? That is a matter of judgment. What is clear, however, is that the states have very different ideas about what it means to be proficient. Indeed, one of the ways parents can boost their chances of having proficient children is to move to another state! On the other hand, if they want their children to be extremely challenged, they should live in states like Wyoming, Arizona, or South Carolina, where, NWEA tells us, the standards are set at levels that may be beyond what is needed for a student to have good prospects for completing college.9

NAEP proficiency standards are set very high.

Under NCLB all states must administer NAEP every other year in reading and mathematics in grades 4 and 8, starting in 2003. The idea is to use NAEP as a check on states assessment results under NCLB or as a benchmark for judging states definitions of proficient. If, for example, a state reports a very high percentage of proficient students on its state math test but its performance on math NAEP reveals a low percentage of proficient students, the inference would be that this state has set a relatively easy standard for math proficiency and is trying to game NCLB. But how rigorous are the NAEP standards for proficient? 10 Very rigorous indeed. A few states proficiency standards come close, but NAEPs surpass them all. 11

Table 4. Percentile Ranks of the Proficient and the Advanced Levels on the Most Recent NAEP (Grades 4 & 8 2003; Grade 12 Reading 2002; Grade 12 Math 2000) Proficient Grade 4 8 12 Reading 69 68 64 Mathematics 68 71 83 Reading 92 97 95 Advanced Mathematics 96 95 98

Source: Robert L. Linn, Assessing Student Achievement: Requirements and Consequences of the No Child Left Behind Act of 2001, paper presented at the University of California at Davis, Feb. 23, 2004; see also, Robert L. Linn, Accountability: Responsibility and Reasonable Expectations, Educational Researcher 32, no. 7 (Oct ober 2003): pp. 3-13, p. 5, Table 1.

As Table 4 shows, the proficient level on NAEP for grade 4 and 8 reading is set at almost the 70th percentile, while for grade 12, its at the slightly less rigorous 64th percentile. The proficiency levels for grade 8 and 12 math are even more challenging. Note, too, the extreme rigor of NAEPs
8

Colorado folded its partially proficient standard into its proficient standard for NCLB purposes, which is why its standard is relatively low. The state retains its original proficient standard, which was set quite high, in its own accountability system. 9 Kingsbury et al, p. 21. 10 The lack of comparability of states and NAEPs achievement levels is not the only problem in using NAEP as a check on the states. Differences in state and NAEP content standards is another one. This discussion, however, is limited to the former issue. 11 D. McLaughlin and V. Bandeira de Mello, Comparison of State Elementary School Mathematics Achievement Standards Using NAEP 2000, paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA, April 2002.

advanced levels in reading and math in grades 4, 8 and 12, which range from the 92nd to the 98th percentile! It would not be unreasonable to think that the proficiency levels on NAEP represent a standard of achievement that is more commonly associated with fairly advanced students. But if NAEP is NCLBs check on the states, then NAEP proficiency levels are what all fourth-, eighth-, and 12th -graders are expected to achieve by 2014. Can it be done? The percentage of fourth-, eighth-, and 12th -grade students who scored at proficient or above on NAEP increased throughout the 1990s, which is good news. However, if past rates of progress on NAEP are a guide, it will not take 12 years to achieve 100 percent proficiency, as NCLB requires. Rather, in math, it will take 57 years for all fourth-graders to reach that goal, 61 years for eighth-graders, and 166 years for 12th -graders. It will take even longer to attain 100 percent proficiency in reading because, even though the percentage of proficient students is higher in reading than in math, the rate of progress on reading NAEP has been somewhat slower.12 Is an acceleration of past rates of progress on NAEP achievable? Likely so. But is it possible to progress four to twelve times faster than we have in math and even more in reading? As one of the worlds most respected testing and measurement experts concluded, Such rapid acceleration would be nothing short of miraculous.13

NCLBs requirement of 100 percent proficiency is not the same thing as requiring that all students achieve at grade level.
As we have seen, proficiency does not have a single meaning. It essentially represents judgments about where to set cut scores on tests. They can be set high or low or somewhere in between, depending on the standard-setting method used and the views of the standard setters. Grade level is an entirely different concept. It does not represent a judgment about what scores a student ought to achieve. Rather, it is an is statement: a description of how a student achieved in relation to typical students who took the same test in the same grade at the same time of year (the norm population, or the representative sample of students who took the test). So, while proficiency under NCLB is subjective, a judgment call, grade level is objective in the sense that it is based on the actual performance of students. You cannot, for example, decide that being on grade level in 12th -grade math should represent an Einstein-like or, at the other extreme, prenumerate level of achievement on a particular test if the average level of performance on that test was neither Einstein-like or pre-numerate.14 So, lets say Janes parents got a report that their fifth-grade daughter is on grade level in Reading Comprehension on the Iowa Test of Basic Skills (ITBS); Jane took the test in the spring of fifth grade and scored 5.4. What does that mean? According to ITBSs recommended definition, it means that Jane is on grade level because her grade equivalent score of 5.4 is in the range of 4.27.9, which corresponds to the performance of students scoring in the average range during the

12 13

Linn (2003), pp. 5-7. Ibid., p. 6. 14 Achievement standards, like proficiency standards, are increasingly being set on norm-referenced tests, a process that transforms such tests into criterion-referenced tests. This discussion is confined to traditional means of determining grade level.

10

spring months of grade 5.15 And Janes parents would very likely understand this because the definition is akin to the one they hear when their pediatrician tells them that their 54-inch, 10year-old is average in height because her height falls within the range of 50-59 inches that is typical or normal for 10-year-old girls.16 Now, contrast this with the report Janes parents must get under NCLB. Jane scored 54 on the states reading test and did not achieve the cut score for proficient, which was 59. Does that mean that Jane is below grade level in reading? Her parents have no way of knowing. Perhaps theyll get some description of what a student who is proficient in reading is supposed to know and be able to do. Theyll likely learn the percentage of reading-proficient students in Janes grade and school (if other grades were tested) or district or even in the state. But that still wont tell them whether or not Jane is at grade level. It would be like their pediatrician saying that, at 54 inches, Jane is short because she is not above 59 inches (which represents only five percent of 10-yearold girls); that still doesnt tell her parents whether Janes height is normal. But because Jane is not proficient in reading, her parents will no doubt be concerned. Janes classmate, John, did score at the proficient level on the states reading test, but his parents are likely to be concerned, too, because the school is on a list of schools that failed to make adequate yearly progress (AYP) under NCLB. Although 55 percent of its students were at the proficient level in reading and 50 percent were proficient in math, the AYP target set for every school in that state required that 45 percent of the schools students as a whole and in each of the schools subgroups 17 be at the proficient level in reading and 40 percent be at the proficient level in math that year. So why did John and Janes school fail AYP? The special education subgroup was 15 points below the states targets in both subjects, while the low-income student subgroup, which included Jane, was five points below. So even though the school as a whole and each of its subgroups made above-average progress from the previous year a commendable accomplishment the school failed AYP because it didnt precisely hit each of its AYP targets. But does that mean that 45 percent of the schools students as a whole are below grade level in reading and 50 percent are below grade level in math, while even heftier percentages of subgroup students fall below grade level? Not necessarily. In fact, even if all students in the school were on grade level, the school could still fail to make AYP. Students proficiency scores and their grade levels are not the same thing. Pointing out that proficiency and grade level are different is not an argument for the superiority of grade-level reporting. The range of achievement that constitutes grade-level achievement is so great that it is hardly a standard at all. 18 Indeed, there are excellent arguments for setting
15

H.D. Hoover et al, The Iowa Tests: Interpretive Guide for School Administrators, University of Iowa, 2003 (Riverside Publishing), p. 67. The authors also note that many people define average as the 50th percentile or, in the case of this example, 5.8, which, in contrast to Janes 5.4, is the exact average for students who took that test in the spring of fifth grade. But, they say, this is a very narrow viewthat does not take error and student variability into account. A more common and useful definition of average is performance between the 25th and 75th percentiles. This range.discourages the overinterpretation of small differences in student performances (p. 68). 16 http://sickkid.net/charts/pdf.charts/g-statage-wtage-e.pdf. 17 NCLB designates student subgroups by race/ethnicity; poverty; English language learners; disability; and migrant status. 18 See, for example, How Achievement Varies, ETS Policy Notes, Educational Testing Service, Princeton, NJ, Summer 1993, p. 1, 5-6.

11

standards/cut scores on the basis of ambitious, but reasonable , judgments about where student performance ought to be rather than according to how students actually perform. We certainly do it all the time when it comes to things like, say, weight, which, unlike achievement, we are frequently urged to reduce because Americans actual weight is far above ideal weights. But weight-reduction goals, unlike the goal of NCLB, are transparent. We all know, for example, what a pound is; its meaning and measurement do not vary across the states.19 And unlike the many meanings of academic proficiency, we can readily find the standards and ranges for ideal weights. We also know that extremely overweight people have a lot more pounds to lose to achieve their ideal weights, so it will take them longer and require more support than those who are merely a bit overweight. Yet NCLB does not similarly recognize the different academic starting points of students, including those whose achievement lags even before they start school; the interim achievement targets are the same for all students, whether they are well below or well above them. And so, unlike the praise we lavish on people who have lost a lot of weight in, say, a year, even though theyve fallen short of their targets, when schools fall short of AYP no matter how unreasonable their targets and no matter how great their progress they will be put on a failing list. The only comfort they will have is cold comfort indeed: like overweight Americans, they will be in the majority. Indeed, if the experts are right, in a couple of years, virtually every public school district in the nation will be on a failed-AYP list, and by 2014, the year 100 percent of our students are supposed to be proficient, almost every school will have failed to make AYP for one or more years. The reason isnt to be found in the near-universal deficiency of our public schools though there are certainly failures among them but in the deficiencies of the law.

19

This is not an argument either for national standards or national tests, but, rather, an analogy meant simply to illustrate the relative lack of public understanding about NCLB proficiency standards.

12