Examining the Data: Using Achievement Data to Compare State Standards

Whether we like it or not, student performance on standardized tests under the No Child Left Behind (NCLB) Act is the most widely available method of determining school district quality and success. Typically, this information is difficult to find all in one place – every state stores its school district student achievement data on a separate, but publically accessible, website. Luckily, the Federal Education Budget Project (FEBP), Ed Money Watch’s parent initiative, compiles achievement data from all 50 states for nearly 14,000 school districts and makes it available all in one place on its website, www.edbudgetproject.org. These data include the percent of 4th, 8th and high school students in each school district that score proficient and above in mathematics and reading on state No Child Left Behind accountability tests. Ed Money Watch took a closer look at what these data tell us about student achievement and found that state academic standards vary widely across the country.

It would be inaccurate to compare these standardized NCLB test results among school districts from different states because each state develops and administers its own NCLB test based on its own set of academic standards. It is possible, however, to compare achievement across states using student outcomes on the 2009 National Assessment of Educational Progress (NAEP) testing data. NAEP is a standardized test the U.S. Department of Education administers to a representative sample of 4^th and 8^th graders in each state every two years.

Additionally, we can use NAEP results to compare state academic standards to a common benchmark by calculating the difference between the percent of students scoring proficient or above on a state’s NCLB test and on the NAEP test for 4^th and 8^th grade reading and math. Using the most recent NAEP and NLCB state achievement data, Ed Money Watch did just that. The results suggest significant disparities between state standards and the standards measured by the NAEP test.

While an average of 33 to 42 percent more students scored proficient or above on a state’s test than on the NAEP test in each grade level and subject, some states saw significantly higher gaps. Tennessee’s state test outcomes, for example, were 60 percentage points higher than its NAEP outcomes for each grade level and subject area. It had the largest gap of all states for 4^th grade reading (62.3 percentage points), 4^th grade math (62.2 percentage points), and 8^th grade math (65.1 percentage points). Texas, however, had the greatest discrepancy between the two tests for 8^th grade reading (67.3 percentage points). This suggests that Tennessee’s standards are significantly different (and likely less rigorous) than the NAEP standards.

The only state to have a higher percentage of students score proficient or above on the NAEP test than on the NCLB test was Massachusetts with 9.1 percent more in 4^th grade math and 3 percent more in 8^th grade math. Massachusetts also had the smallest discrepancy between the two tests in 4^th grade reading (6.6 percentage points), while Missouri had the smallest discrepancy on 8^th grade reading (16.2 percentage points), suggesting that these states have standards that are most closely aligned with the NAEP standards.

Additionally, some states scored similarly to each other on the NAEP test, but differed greatly in their proficiency levels on state NCLB accountability tests. For example, 35 percent of students in both Maine and Nebraska scored proficient or above in NAEP 4^th grade reading. However, far more students scored proficient or above on Nebraska’s 4^th grade reading test (59.9 percent) than on Maine’s test (35.8 percent). This could indicate that while both state’s standards overlap with NAEP standards to a similar extent, the state standards include other aspects of reading for which their students have not completely mastered.

These results tell us a few things about testing in the states. First, with an average discrepancy of over 30 percentage points for every grade level and subject, it is clear that state standards are not aligned with NAEP standards.

Additionally, in every state except Massachusetts, more students score proficient or above on state tests than on the NAEP test for every indicator. Though one could argue that the state accountability tests are more aligned with the curriculum in each state causing students to have greater success, the magnitude of the scoring discrepancy would suggest that NAEP standards are set higher than those set by the states.

Finally, the results tell us that the NCLB proficiency standards vary vastly from state to state. Even in states where NAEP test results were similar, NCLB proficiency levels were sometimes very different. This tells us that the standards set by each state are not consistent, and implies that some states set higher proficiency standards for their students than others.

However, all this is likely to change with the recent push toward common standards across states. So far, 35 states and the District of Columbia have adopted the Common Core of Standards, a movement initiated by the National Governors Association and the Council of Chief State School Officers (CCSSO). As these states begin to implement these new standards and the tests that are aligned with them, state test outcomes and NAEP outcomes should begin to look more similar across the nation.

Ed Money Watch will continue to follow this process to see whether common standards succeed in lifting student achievement.

You can download a PDF of Ed Money Watch’s comparison of state results on the two tests here.