Wash. Post editorial cited study to show positive effects of No Child Left Behind, ignored authors' caveats

A July 2 Washington Post editorial on President Bush's "No Child Left Behind" (NCLB) education policy suggested that NCLB had led to improvements in reading and math test scores documented in a recent study. The editorial noted that a "recent report by outside experts showed students nationwide doing better on math and reading tests, as well as a narrowing of the achievement gap." The editorial contrasted these results with "the landscape before" NCLB went into effect: "No one was really focused on results, failure had no consequences, few people were talking about the achievement gap and parents had little choice if their child's schooling wasn't doing the job." But as a June 6 Washington Post news article noted, the authors of the report to which the Post editorial appeared to be referring -- a June 2007 Center on Education Policy (CEP) study titled "Has Student Achievement Increased Since No Child Left Behind?" -- "warned that it is difficult to say whether or how much the No Child Left Behind law is driving the achievement gains." Moreover, the study cautioned readers that its comparisons between pre- and post-NCLB data -- available for only 13 states -- should be treated as "suggestive."

The CEP study examined student scores since 2002 on the state tests required by NCLB. As Washington Monthly blogger Kevin Drum noted in a June 6 post, the study's authors emphasized that it is impossible to determine what effect, if any, NCLB (or any other federal, state, or local policy) had on the test-score trends. CEP offered this caveat in a section of the report titled "Limitations of This Study":

DIFFICULTY OF ATTRIBUTING CHANGES TO NCLB

This report focuses on whether student achievement has improved since the enactment of NCLB. It is very difficult to determine whether students are learning more because of NCLB. Isolating the cause-and-effect relationship of any education policy is often impracticable. With a policy as far-reaching as NCLB, it becomes nearly impossible when states, districts, and schools are simultaneously implementing so many different yet interconnected policies and programs. If student achievement has risen since NCLB took effect, is this due to NCLB, or to state or local reforms implemented at roughly the same time, or to both? If both, how much of the improvement is attributable to state or local policies and how much to federal policies? Using multiple methods of analyzing achievement will not tease out the causes of gains or declines.

In a similar vein, this study does not take a position on how well specific components of NCLB are working or whether the requirements in the current law are the most effective means to raise achievement and close test score gaps.

Furthermore, the study reported that only 13 states had sufficient data available to compare pre-NCLB testing trends to post-NCLB trends. While nine out of those 13 had greater gains after NCLB was passed, the study cautioned against deriving broad conclusions from these results: "It is difficult to say, however, whether the small sample of 13 states represents a true national trend of hastening progress after NCLB. For now, these comparisons should be taken as suggestive." Indeed, the June 6 Post article reported that one critic of the study, University of California at Berkeley professor Bruce Fuller, "said it made little sense to draw conclusions when so few states have adequate data."

In addition to the June 6 Post article, a June 6 New York Times article on the study quoted University of Colorado at Boulder education professor emeritus Robert Linn, a "frequent critic" of NCLB and a member of the advisory panel that advised CEP on the study, warning against drawing conclusions from the study. According to the article, Linn said he was "a little surprised that things were generally as positive as they were, so it may be that I would say that N.C.L.B. is contributing more positively than I had given it credit for." According to the article, Linn "urged readers to pay attention to the report's many caveats" because " '[t]he reason for all the caveats is that it is impossible to reach the conclusion that if scores go up, it is because of N.C.L.B. ... There are so many other factors that could lead to rising scores, including state efforts to raise achievement, and also, some of these gains may be artificial. So my worry is that people who come at it and don't read the caveats will come away with an exaggerated impression.' "

From the July 2 Washington Post editorial titled "'No Child' in the Crosshairs":

We've been unequivocal in our support of standards that have rigor and meaning. It's encouraging that Sen. Edward M. Kennedy (D-Mass.), a proponent of No Child Left Behind who chairs the education committee, has identified this as one of his priorities. Some promising ideas come from the nonprofit advocacy group Education Trust. One is to encourage states to raise their standards to a "college-and-career-ready level" with the trade-off of getting more time to reach more realistic goals of proficiency. The law's original goal of 100 percent proficiency by 2014, while laudatory, may be unrealistic.

Other areas cry out for improvement. Schools that are failing need help in the form of guidance and resources, rather than just sanctions, if they are to improve. Students most in need of quality teachers still aren't getting them. Provisions to get extra help for struggling students, such as private tutors, are not being applied the way they should.

Consider the landscape before No Child Left Behind. No one was really focused on results, failure had no consequences, few people were talking about the achievement gap and parents had little choice if their child's schooling wasn't doing the job. A recent report by outside experts showed students nationwide doing better on math and reading tests, as well as a narrowing of the achievement gap. To let states opt out of doing the hard, necessary work of raising standards is to turn back the clock on education reform.