Reform #1: Measuring Quality Through Teacher-Child Interactions Alone

With Act 3 in place and a reform-oriented Department of Education taking the reins, state officials set out to bring consistency and alignment to early learning. A first step was determining whether programs were preparing children for kindergarten. In 2015, LDE rolled out a mandatory quality rating and improvement system (QRIS) for all publicly funded birth-to-five programs, aptly referred to as the Unified Rating System. The 2015–16 school year was the first year that all Type III ECE programs had to participate.

Like most states, Louisiana had a statewide QRIS called Quality Start in place for multiple years that rated programs on a series of inputs like teacher qualifications and staff-to-child-ratios.¹ Jenna Conway, former Louisiana assistant superintendent of Early Childhood and the brains behind many of the state’s early education reforms, told New America that under Quality Start there was a disconnect between programs scoring well and actually producing better outcomes for children. She said, “we had a good understanding of what the inputs were but still kids were not kindergarten-ready.” LDE decided to create a new QRIS unlike any other state’s—“a system that is indicative of outcomes, is equitable, and is consistent and scalable statewide,” according to Conway.

“We had a good understanding of what the inputs were but still kids were not kindergarten-ready.”

Based on the research and several years of piloting and gathering statewide data, LDE determined that the Classroom Assessment Scoring System (CLASS), a well-known tool for measuring the quality of child-adult interactions that is already used in Head Start programs across the country, would be the best way to meaningfully differentiate programs and encourage improvement.² Children’s learning depends on the quality of interactions and the relationships they form with their caregivers. As displayed below, CLASS measures the quality of interactions between adults and pre-K children in three domains: Emotional Support, Classroom Organization, and Instructional Support. CLASS is most commonly used in pre-K classrooms, but Louisiana also uses it in toddler classrooms statewide. It will begin piloting in infant classrooms statewide in 2020.

Source: Sara Diamond, “Transitioning CLASS from Pre-K to Infant and Toddler Classrooms,” Teachstone (website), December 29, 2015.

While CLASS is a factor in numerous quality rating systems throughout the country, Louisiana is the first state to rely on CLASS scores as the sole indicator of program quality. A 2017 report by RAND Corporation analyzing the validity of seven state QRISs recommended that states “Streamline QRIS rating rubrics to focus on fewer but most relevant aspects of quality.” The researchers argued that rating should focus on “fewer but more challenging criteria.”³

Research shows that a teacher’s interactions with children are a core determinant of whether children develop the social-emotional and cognitive skills that help them succeed in school, so it follows that they should be the ultimate focus. But should these interactions be the only focus? In other states, quality is measured by environmental factors, such as whether books are displayed at eye level or whether enough supplies are available.

One Lake Charles center director thinks that the sole focus on teacher interactions is the way to go. “I prefer the way it is now with the CLASS scores,” she said. “For me as a director, just seeing the differences, a teacher can really focus on social-emotional, communications, interactions, not on how many books we have, or the kinds of books. That was taking away from the importance of why we are really there—the interactions that build the foundations.” She said under Quality Start, “the teachers were so worried about the way we were changing a diaper, how many wipes we used, etc. instead of the way we were interacting with the children.”⁴

LDE officials also feel confident that CLASS is the right measure. “The focus and the specificity of the CLASS has led to the biggest transformation in Louisiana. It measures the things that matter most in a consistent and reliable way and provides really specific feedback to teachers and measures progress to decimal points,” said Jenna Conway. Erin Carroll, director of Classroom and Workforce Improvement in Louisiana’s Office of Early Childhood, agrees. She says having a system based solely on CLASS “really simplifies it for teachers and directors. It’s not a simple tool, but it’s one thing you can really sink your teeth into.”

Participation in the new QRIS is a condition of Type III licensure. Program participation is not optional as it is in most states, where only the stronger or more ambitious programs may choose to participate. CLASS scores place programs into one of four levels, and programs that score at the lowest level for two out of three years risk losing their funding. All classrooms receive a minimum of two CLASS observations conducted by local observers.⁵ At least half of the classrooms at every site must be observed by an external third party observer. When there is a significant discrepancy between the local observer’s score and the third party, the third party score prevails.⁶ Local observers can also be suspended temporarily when scores are consistently different from the third party. Conway acknowledged that this is a work in progress.⁷

The QRIS has created incentives to focus on teacher child-interactions and the classroom environment. The state has been providing free trainings over the last few years to get the workforce, both teachers and program administrators, up to speed on the CLASS tool. With state support, communities now have over 1,200 reliable CLASS observers.⁸ State funding allows staff members in every early learning center and elementary school with publicly funded ECE to access CLASS training and the supports that come with CLASS. While localities are responsible for offering professional learning, having a unified system has made it easy to share resources throughout the state. As Carroll told New America, “people can work with each other throughout the state now because we are all speaking the same language….It’s easy for school districts to share their PD [professional development] with others and collaborate. We do a big conference every spring in New Orleans with about 600 ECE professionals and over 60 ECE sessions. All are tied to CLASS. We picked the highest-performing sites to come and give trainings. Half of the teachers are from elementary schools, half are from child care.”

The QRIS has created incentives to focus on teacher child-interactions and the classroom environment.

If CLASS is going to be the only measure determining program quality, it needs to be an accurate one. Fortunately, the state is committed to using data to inform policy and has been open to bringing researchers on board. Daphna Bassok is part of a team at the University of Virginia⁹ that has been studying Louisiana’s QRIS to determine whether a state can accurately collect CLASS data from every single classroom. One goal of the study was to assess whether CLASS scores collected by local raters were similar to those collected by trained researchers, and to determine whether CLASS scores were predictive of children’s learning gains.

Bassok’s team collected CLASS data from classrooms serving four-year-olds in early learning centers, Head Start, and state-funded pre-K four times over a one-year period. When comparing researcher CLASS scores to the scores given by local observers, they found that “local raters gave programs systematically higher ratings on Instructional Support, and in turn the overall scores from the local raters were somewhat higher and more variable.” In the Emotional Support and Classroom Organization domains, the researchers found no difference in scores across reporter types.

They also found that “the CLASS total score, which is oftentimes the relevant measure in monitoring and accountability systems, was associated with children’s learning gains” for both groups. “For the local raters, this measure was positively linked to math, literacy, and average achievement gains. For the researcher group, the total score was significantly related to math and executive function gains.”¹⁰ This matters because research has shown limited or no correlation between state QRIS ratings and child outcomes. The aforementioned RAND report found “a modestly positive relationship between QRIS ratings and children’s developmental gains” at best and no relationship at worst.¹¹

Having this CLASS data across all ECE classrooms also allowed University of Virginia researchers to study early childhood teacher turnover in an unprecedented way. They found potentially promising trends when examining the relationship between teacher turnover and their classroom’s CLASS scores. As displayed below, the teachers who stayed at their centers had higher CLASS scores than those who left. And newly-hired teachers were performing better than those leaving their centers.¹²

Because this information is limited to CLASS scores, it is difficult to draw conclusions. But these findings suggest that quality is improving over time, with those staying in the field improving their CLASS scores and those who are not performing as well choosing to leave. In addition to helping policymakers understand trends in the workforce, CLASS scores can also be useful to help identify areas for improvement in teacher practice and inform professional learning.

Bassok told New America that transitioning to CLASS was challenging for teachers in child care programs, for which these types of observations were totally new. And even though pre-K teachers in public schools were already accustomed to classroom observations, they had been using the state’s observation system designed for kindergarten through fifth grades, so switching to CLASS was a big change. Because CLASS is working so well in pre-K, Bassok says the state sees this as an opportunity to expand the use of CLASS into kindergarten and possibly beyond. Some Louisiana elementary schools are already piloting CLASS in kindergarten. According to Bassok, the state is “low on resources, but is doing a lot to create systems. A lot of this is expensive and takes resources and time. It's hard to ask this workforce to do certain things when their resources are so low.”

While CLASS is popular among state policymakers and some practitioners, and implementation appears to be going well, it is not a perfect measure. The University of Virginia researchers admit in their report that “the associations between CLASS scores and child outcomes tend to be modest.”¹³ CLASS was originally designed to be a professional development tool, and not intended to be used in a high-stakes way. But according to Bassok, “what’s always super striking during these visits [to Louisiana] is just how focused the directors and leaders are on providing professional development in support of the CLASS. It is true they are using CLASS as an accountability tool, but it’s also true that there is a lot of focus on CLASS as a formative tool.”¹⁴

CLASS scores can also be useful to help identify areas for improvement in teacher practice and inform professional learning.

Implementation of this new system has been challenging. Familiarizing the workforce with CLASS and training local observers has been both expensive and time consuming. One Lake Charles center director said, “we don’t have the manpower to go in and score every single center. It takes a lot of time to give feedback and conduct the observations.” And one child care teacher shared her mixed feelings in a focus group: “once they explained it more to us and showed us exactly what they are looking for in each section it made it easier. Some aspects I still think are unrealistic. If you have a special needs child, it makes things a lot more difficult.” LDE acknowledges the importance of educator buy-in to make this system work, which was part of the rationale behind having local raters conduct the observations.¹⁵

Part of the challenge is that much of the ECE workforce in Louisiana (and in other states as well) does not have a strong background in child development and early learning and may be learning about how to engage in high-quality interactions for the first time with the CLASS training. Until recently, expectations for the quality and education of caregivers and teachers in early learning centers were low.