What Ed Week's State-by-State Report Cards Can't Tell Us

Every year, the publisher of Education Week releases a detailed report on trends and progress in K-12 education, giving grades to each state. The report, "Quality Counts," provides reams of data in areas as varied as kindergarten enrollment levels and spending ratios per pupil. This year it also includes an excellent series of articles and analysis on students with limited English skills and how schools are absorbing them.

The first headlines about the report have focused on which states are bringing home the A's and B's, and which ones are nearly flunking out. Maryland tops the list with an overall score of 84.7, while Washington D.C. brings up the rear, with a weak 68.3.

But to really get a handle on what these scores mean, you have to dig in and look at what they measure. Not surprisingly, we wanted to zoom in on categories related to early childhood and the early elementary grades. What might this data be able to tell us, we wondered, about how particular states stack up in various early education categories?

(There are other good state-by-state resources to consider too, like "Votes Count," pre[k]now's report about legislative action expected this year, and its state profiles, as well as the "State Preschool Yearbook" published by the National Institute for Early Education Research. But they don't allow for an across-the-spectrum analysis that includes grades K-12.)

At first, the early-education data in "Quality Counts" look very promising.

"This year's survey finds that states are making great strides in early education," writes Amy M. Hightower, in her overview of the states' report cards. "Among the 50 states and the District of Columbia, only North Dakota lacks early-learning standards aligned with elementary-grade academic standards. For the first time, every state and the District of Columbia has aligned kindergarten learning expectations with elementary and secondary standards."

But we shouldn't be celebrating yet. Not because the data is wrong or unreliable -- on the contrary. The research center for Editorial Projects in Education, which publishes Education Week among other periodicals, has done a fine job incorporating information from many solid sources and extracting data on policies from all 50 states and the District of Columbia. It says it has also engaged in rigorous back-and-forth conversations with states to ensure that documents back up their claims. (The Pew Center on the States provided support.)

The problem is that the early-education data doesn't consider quality.

For example, states receive points for how well they measure up in a category called "transitions and alignment." We've advocated many times for more emphasis on alignment between preschool and the first three grades of elementary school, so the inclusion of this category makes us sit up and take notice. The E.P.E. asked states: Do you have early learning standards aligned with standards in the elementary grades? Are your kindergarten expectations lined up with your elementary and secondary standards? Do you have a formal definition of school readiness, a way to assess whether incoming kindergarteners match it and a plan of action if they don't?

All good questions. And this year, the states were ready with answers, especially on standards. That's why, as Hightower wrote, every one of them except North Dakota got points in that area. But what do those standards look like? Are they written so that first-grade teachers know exactly what concepts they should teach and second-grade teachers can be sure they were covered the year before? Are they vague and loosey-goosey? Are they too narrow and restrictive? (There's not much substance to saying, for example, "Students will learn about shapes," but on the other hand, imagine being pigeonholed into a requirement that students "demonstrate an ability to arrange circles and rectangles to create the outline of a school bus or other vehicle.")

Last spring, the American Federation of Teachers came out with a report, "Sizing Up State Standards," that evaluated state standards from K-12, stressing that they had to be coherent, specific and clear. We agreed completely. It found nine states that have no standards at all for numeracy and literacy in each of the kindergarten through second grades. Those states cluster them instead, saying "here's what kids need to know in the K-2 grades."

Several of the top states in "Quality Counts" are not the same as high-ranking states in the standards report. Virginia, for example, ranked fourth overall in "Quality Counts" and 14^th in the "transitions and alignment" category. According to the AFT, however, it ranks highest in standards, being the sole state to have strong standards across all grade levels and subject areas.

Meanwhile, Maryland - the highest scorer in "Quality Counts" - is one of a group of states with only half to three-quarters of its standards hitting the AFT quality bar.

It's worth noting that the producers of "Quality Counts" have used AFT research in the past to gather data about standards. (In fact, some scores on K-12 standards that appear in the report's online state-by-state maps use 2007 data from the teachers' union.) That collaboration, if it continues, could serve us all well. Then we might be able to see more clearly which states are positioned to make the most out of their early education programs. Until then, we'll keep pushing for more high-quality alignment in every state.