A Feel-Good Report Card Won’t Help Children

While cultural debates have roiled primary and secondary education, the National Assessment of Educational Progress has been largely free of ideological conflict. Known as the “nation’s report card,” NAEP tests students in math and reading and, less often, in civics, U.S. history, and science. These tests are not high-stakes student assessments because the NAEP does not calculate individual scores; rather, it measures school-system performance by state and large districts, and, as such, it is the nation’s best indicator of trends in student learning.

The NAEP’s current framework defines reading comprehension as “a dynamic cognitive process,” involving “Understanding written text”; “Developing and interpreting meaning”; and “Using meaning as appropriate to type of text, purpose, and situation.” The language is commonsensical and defines reading as a generic function common to any literate person. Reading goals may vary depending on context, but reading remains fundamentally the same activity from person to person, and the object of reading—discerning meaning—can be apprehended by anyone.

Now that might be changing. A proposal from the NAEP Governing Board to change the framework of the reading section threatens the status of the test as a reliable gauge of district performance. The new definition relativizes reading across cultures. Reading remains a “dynamic cognitive process,” but it’s now “expanded to a sociocultural model that positions the reader, the texts, and the activities in a sociocultural context.” Not only do we have readers exercising their skills on diverse texts and for different purposes, but readers themselves are also diversified, each one “positioned” by sociocultural background. Given that students from different backgrounds bring different frames of reference to passages on the exam, the new NAEP framework seeks to equalize—so far as possible—the readiness of each student to understand them.

The proposed new framework goes well beyond acknowledging the inarguable role that sociocultural background plays in the development of reading skills. It’s a given that students should be examined on their ability to read passages that reflect the diversity of a complex society. But the new NAEP definition asserts that the reading skill itself is sociocultural. In this understanding, the hows and whys and strengths and weaknesses of reading comprehension change from one sociocultural context to another, leaving some students in potentially “unfair” starting points, depending on the relationship between their backgrounds and the texts in question.

The NAEP exam’s architects have decided that they must no longer treat test-takers as generic readers and must instead account for their sociocultural origins. Hence, the designers of the new framework call for testing procedures that will “optimize the performance of the widest possible population of students in the NAEP Reading Assessment.” This means respecting the “linguistic, cognitive, and epistemological strengths that individuals develop across their diverse communities of practice and bring to their reading.” Such strengths may not include the skills or knowledge necessary to understand the test passages, so NAEP will “develop scaffolds that optimize comprehension performance for every reader.”

For example, the designers say, NAEP might include a “mini-probe,” asking test-takers about their interest in the topic of a passage and then tally that with their performance on the questions. Or NAEP may allow test-takers to watch a short video related to a passage if the test-takers have no prior knowledge of the topic. (The example included in the new framework is a video on weather balloons, which accompanies a passage on weather forecasting.) NAEP is also considering offering vocabulary assistance and motivational help, including digital “avatars” to assist candidates with questions.

At first blush, one might see a hopeful progressivism here. Why penalize youths because their background isn’t in tune with particular textual passages, even if they are taken from books that have been read for generations? Yet the whole point of the NAEP reading assessment is to tell educators how well states and school districts are teaching students to read the language they will encounter in the real world. That report, in turn, tells us how well prepared American high school seniors are for college and the workplace. If NAEP adopts the sociocultural model of reading, the scores for certain student populations will almost certainly improve, but the scores themselves will lack any predictive value.

One of the most powerful and consistent findings about reading comprehension is that it depends on the background knowledge that a student possesses. As E. D. Hirsch has argued, if you already know what a passage is about, you are more likely to understand it. An effective reading assessment, then, should reveal the gaps in knowledge among students. Our current mediocre results are a signal that English language arts teachers should focus instruction on the study of content-rich texts. If the NAEP reading exam doesn’t tell us how well we’re doing in this critically important endeavor, it will fail to indicate whether young Americans can handle reading demands at the next level.

In adapting the test to the students’ diverse backgrounds, the NAEP explicitly seeks to reduce the role of formal education and the knowledge that goes with it. In prizing each student’s culturally determined approach to reading as prima facie legitimate and in providing extra assistance to test-takers who need it, the new framework purposefully sets low expectations for students. Rather than allowing poor performance to serve as a signal that large knowledge gaps should be fixed through better education, we will simply lower the impact of background knowledge on the reported test performance.

For example, children from white rural backgrounds may have trouble understanding a passage from The Autobiography of Malcolm X. Why? One, because they haven’t grown up in a related sociocultural environment; and two, because their teachers haven’t introduced them to African-American literature. Is the remedy to give such students vocabulary and motivational help to “equalize” their ability to handle the passage? Or should poor scores alert teachers to broaden their reading lists? The same, surely, goes for students from inner-city public schools whose sociocultural background minimizes books and whose teachers provide a narrow literary education. Should we shrug and say to those students—and their teachers—never mind, your “authentic world experience” counts for more than what a strong education provides?

If NAEP follows this route, its assessment will no longer be a reading test that we can trust to demonstrate where students need more help—and where teachers should focus their efforts.

We wonder, then, if the proposed change is for the benefit of America’s children or for educators dismayed by our persistent achievement gaps. Because NAEP exams are not high-stakes like the SAT and ACT, a low score has no impact on an individual’s record. Instead, NAEP averages reveal how states and districts and different populations are doing from year to year. A high score on the revised reading test may let teachers and district superintendents breathe easier, while not actually indicating anything about how well the students can read. A feel-good report card will do nothing to improve the educational prospects of American children.

City Journal is a publication of the Manhattan Institute for Policy Research (MI), a leading free-market think tank. Are you interested in supporting the magazine? As a 501(c)(3) nonprofit, donations in support of MI and City Journal are fully tax-deductible as provided by law (EIN #13-2912529). SUPPORT