Q&A About Measuring Quality in Pre-K Classrooms

As enrollment in publicly funded pre-K programs continues to slowly increase, much attention is being paid to measuring the quality of the pre-K programs being offered. One popular measurement tool that is being increasingly used throughout the country to help measure quality is the Classroom Assessment Scoring System (CLASS). A new study explores the link between scores on this measurement tool and children’s gains in math, language, and executive function skills. To learn more about the study and its implications, I sent questions to Paola Guerrero-Rosada and Christina Weiland, two researchers from the University of Michigan who were involved in the study. In the following interview, they explain why more work is needed on defining and measuring the active ingredients that drive children’s school readiness gains in pre-K programs.

For readers who might not be familiar with CLASS, could you briefly explain what it is and how it’s used in early childhood education?

The CLASS® is an observational tool that measures the quality of interactions between teachers and children and between children and their peers in classroom settings. A trained, reliable observer typically watches about 80 minutes of classroom instruction and rates the classroom’s emotional support, organization, and instructional support, with scores averaged across four 20-minute observation cycles. Ratings range from 1 to 7, or from low to high.

The CLASS® has been shown to be effective as a professional development tool for preschool teachers in the U.S. It also provides a common metric for comparisons of programs within and across settings and can help identify system-wide strengths and weaknesses.

At the policy level in the U.S., the CLASS is widely used in early care and education quality rating and improvement systems in 21 states. Cities with large-scale public preschool programs like New York City and DC also use it to assess quality. The CLASS is also used in the monitoring and quality assurance process in Head Start as one criterion for identifying needed improvements and for determining which grantees have to recompete for funding, with potential loss of funding implications.

What research questions did you set out to answer with this study?

We explored whether CLASS scores predicted children’s gains in math, language, and executive function skills, within a sample of 263 children enrolled in the Boston Public Schools (BPS) Prekindergarten program in the 2016-2017 school year. We also examined whether CLASS scores were more predictive of gains for children enrolled in classrooms that had reached a higher threshold of quality. Finally, we tested whether children who started prekindergarten with lower vocabulary or math skills made larger gains in classrooms with higher CLASS scores.

Importantly, our study replicates and extends prior work in the Boston Public Schools Prekindergarten program that found that CLASS scores were not consistent predictors of children’s vocabulary and executive function skill gains. Given the widely noted replication crisis in the social sciences, we viewed it as important to undertake this replication study as BPS and the field broadly thinks more deeply about early childhood measurement.

And, speaking generally, what answers did you find to these questions through your research?

We found that, just as in the prior study, CLASS scores did not predict prekindergarteners ’ vocabulary, math, or executive function gains. Findings were also null when we tested whether CLASS scores were more predictive for children in classrooms in the higher range of quality, and when we tested whether CLASS scores were more predictive for children entering prekindergarten with lower vocabulary and math test scores than their peers.

Are there any limitations to the study that are worth pointing out?

Our study has several important limitations. The first one is that our sample is only representative of the Boston Public Schools district. Our findings may not generalize to other prekindergarten settings and programs. We also did not measure children’s literacy or socio-emotional skills and thus we can’t say whether the CLASS may have been more predictive of gains in those domains. CLASS scores in our sample also did not cover the full range of possible CLASS scores. It is possible that with a larger range, our results may have been different. Finally, our study is correlational.

Given that your research found no relation between CLASS scores and gains in children’s skills, can CLASS still be a useful tool for improving early childhood education?

In both this study and the prior one, we emphasize that the CLASS has many important strengths. It facilitates cross-program and within-program comparisons; serves as a PD tool for teachers; and shows small associations with child gains in some contexts. Right now, it is probably the best early childhood quality measurement tool we have.

What our work shows is simply that we’re not done yet. We need more work on defining and measuring the active ingredients that drive children’s school readiness gains in prekindergarten programs.

The study notes that, “It may be that process quality as measured by the CLASS does not capture teacher behaviors that are more predictive of preschool children’s gains.” What might some of these teacher behaviors be that are more predictive of children’s gains?

This is exactly the question that we need to work together to answer. Our team’s best bet is on measures that also capture the richness of instructional content and curriculum. In preschool education, we have tried approaches like measuring the quality of general instructional practices – or how teachers teach – and the amount of time spent on particular skills and in particular activity settings like whole group or small group. But we don’t really have validated observational tools that capture whether prekindergarteners are exposed to teaching that builds their background knowledge and to curriculum that matches the science of how young children learn. As highlighted in a recent review, multiple randomized trials show that some curricula are more effective than those used in most public preschool programs. But our quality measures do not capture what it is about effective curricula that lead to these gains. That’s the problem we think the field needs to solve together.

The study concludes by noting that “our findings support calls for a next generation of measurement work in early childhood education.” What might this next generation of measurement work look like? I know examining curricula is one possibility mentioned in the study.

The CLASS has been hugely influential in the field of early education and has led to stronger preschool experiences for many, many children. As we think about a next generation of measurement work, we think it is important to learn from and build on this success.

Looking forward, we’re hopeful that researchers and funders will prioritize work in this area. We know that programs are often very responsive to what we measure. We hope that additional work will tackle how to measure what it is about particular curricula that prepare children for kindergarten better than other curricula do. We need our best thinking as a field on this topic.

Is there anything else you’d like to tell us about the study or more generally about what readers should take away from the study?

Yes, two things. First, the questions that our paper raise are not new to the field. We cite other teams with similar findings throughout this paper and the previous one. Our work underscores and echoes calls from experts for a new generation of measurement work.

Second, our work with Boston takes place within an established research-practice partnership. We view this model as very important for advancing policy and practice in early education. Many of our best questions as a research team come directly from the ground, from our coaches and teachers. The next generation of measurement work should also prioritize deep collaborations between researchers and practitioners.

Interested in staying up to date on education and workforce policy? Subscribe to our newsletter to receive updates on the latest from our experts.