Waiver Watch: School Accountability on a Curve

“All you have to do is beat the mean” was the mantra that helped me survive three years of undergraduate pre-med courses. Organic Chemistry, Physics, Botany, and the rest of the lot were horrible, but at least they were graded on a curve. To pass, I didn’t have to actually pass—I just had to do better than most of the other miserable students in the lecture hall with me.

All you have to do is beat the mean.

It turns out that school accountability today works in much the same way. February marks two years since states were first awarded waivers from No Child Left Behind (NCLB) and given the green light to experiment with new school accountability approaches. While all schools are still subject to NCLB-like performance targets, the consequences for missing those targets matter much less than before. What really matters under waivers is landing in the bottom 15 percent of schools: the priority and focus schools that must receive state interventions, supports, and resources to improve student outcomes. Clear the 15 percent bar, and schools escape the strictest consequences. In other words, a school’s absolute level of performance matters less than its performance relative to others—just like my Physics grade. I may have gotten only half of the questions right on the midterm, but that was good enough for a “B.”

What really matters under waivers is landing in the bottom 15 percent of schools: the priority and focus schools that must receive state interventions, supports, and resources to improve student outcomes. Clear the 15 percent bar, and schools escape the strictest consequences.

This new 15 percent accountability approach has had far-reaching effects. Using data collected from over 20,000 schools in 16 states during the transition from NCLB to waivers, the latest report from New America, “It’s All Relative,” finds that nearly 4,500, or 65 percent, of schools in NCLB improvement were eased from these interventions between the 2011–12 and 2012–13 school years. In these states, two in three so-called “failing” schools were no longer “failing”—at least not enough to warrant the priority or focus label. Further, these schools were often in the most serious NCLB improvement phases:

• Of all the schools previously in restructuring (those that had missed their performance targets for six consecutive years), over 50 percent were not identified for a seventh year of improvement in 2012–13. The same is true for over half of schools previously in corrective action (those that had missed their targets for four or five years).
• In 10 states, the majority of schools no longer identified had been in the first two years of improvement (DE, IN, MN, MS, MO, NJ, OK, OR, TN, VA). But in five, at least half of the schools relieved from interventions were previously in corrective action or restructuring (AZ, MA, NV, RI, SC).

The number of schools identified has also changed significantly in waiver states:

• Eleven classified fewer priority or focus schools in 2012–13 than the number of schools identified for NCLB improvement in 2011–12 (AZ, DE, FL, MA, MN, MO, NV, NJ, RI, SC, VA).
• This decrease is much larger in some states than in others. Using its waiver, Nevada identified over 85 percent fewer schools for interventions than it did the previous year, while Rhode Island identified 12 percent fewer schools.
• In five states, the number of schools identified for improvement reaches an all-time high under waivers (IN, MS, OK, OR, TN). But these states tended to have relatively smaller school improvement efforts under NCLB.

While the 15 percent strategy created a defined number of school improvement slots, states could also measure their performance in new ways, changing how they viewed a school’s relative success. The effects of these state choices are less dramatic than the switch to relative measures, but they do shape the kinds of schools that fare well in the new system. It’s like a university adopting a curved grading policy at the same time that professors in the Chemistry department adopt new Organic Chemistry course sequences, syllabi, and assessments. Both decisions affect student performance, but the new institutional grading policy probably matters more. States’ waiver choices—including student growth measures and new kinds of student subgroups—could influence whether the “right” schools made the 15 percent cut, but more and better data are needed to determine the scope of this influence, why certain schools did, and did not, get identified, and whether this changed a school’s long-term chances for improvement.

As relative school accountability becomes the norm (pun intended), states shouldn’t forget about student outcomes and absolute measures of school performance. Grading on a curve may have helped me pass my pre-med classes, but it didn’t mean I knew the material well enough to be a doctor. Then again, I would have also had to eventually pass a medical licensure exam to practice, a criterion-referenced assessment where absolute performance matters. If identification as a priority or focus school is a prerequisite for significant improvement and student success, states should ensure that the schools needing this level of support are included in the category—even if it means expanding their capacity or adding safeguards to their accountability systems to ensure certain kinds of schools are always identified.

“It’s All Relative” concludes with an ambitious federal and state research agenda for waiver implementation, especially as the Department begins to renew states’ waivers at the end of the 2013–14 school year. A 15 percent approach to accountability may make school improvement efforts more manageable for state departments of education, but does it lead to better outcomes? To more students—regardless of race, or income, or zip code—graduating high school prepared for college and the workforce?

We don’t have enough data yet to answer these questions, but it’s time to start asking them.

Click here to read the full report.”