Recent Study Examines Effectiveness of North Carolina QRIS

While only 10 states had a Quality Rating & Improvement System (QRIS) in place in 2005, 38 states had a statewide QRIS and almost all others were planning or piloting a QRIS as of February 2017. QRIS took off in large part due to the federal Race to the Top - Early Learning Challenge (RTT-ELC) grant program that began in 2011 and included QRIS participation as a performance metric. (For a refresher on QRIS, click here). Despite robust growth in the number of states with QRIS, there has been limited evidence about whether QRIS actually lead to quality increases among participating providers or change the choices of parents seeking early care and education. Are QRIS investments that should be continued in the future? A recently released study helps to fill in the knowledge gap about the effectiveness of QRIS on spurring changes to program quality and enrollment levels.

The study authors examined the effectiveness of North Carolina’s QRIS, known as the Star Rated License program. “It’s the second-oldest QRIS in the country and that means it’s had enough time to really be in place. North Carolina gives us an opportunity to study a QRIS that’s fully developed,” said Daphna Bassok, one of the study’s authors, in a phone interview.

While most state QRIS are voluntary, all non-religious ECE programs are automatically enrolled in North Carolina’s QRIS. The current study used data for all licensed center-based ECE programs in North Carolina between 2007 and 2014 which the researchers narrowed down to about 3,000 center-based programs.

North Carolina’s QRIS assigns a rating of one to five stars to center-based programs. Programs are awarded points based on “education standards” which are determined by the education and experience levels of the teaching staff. Programs also earn points according to “program standards” which include measures of quality like child-staff ratios as well as observation scores from the Environment Rating Scale (ERS).

The ERS, currently used in 30 state QRIS, measures program features such as physical space and layout as well as student-teacher interactions and classroom activities. Because of the way the North Carolina QRIS is scored, ratings on the ERS have a significant impact on the ultimate star rating a program receives. For the current study the authors examined whether programs that earn a lower ERS score (and thus a lower star rating) will subsequently focus on improving their ERS score since it has such a significant impact on star ratings. The authors also looked into whether programs assigned a lower star rating would experience declines in enrollment due to parents using the ratings to influence their enrollment decisions.

The results of the study are heartening for supporters of the North Carolina QRIS. The authors found that earning lower star ratings led programs to substantially improve their performance as measured by their ERS score in subsequent years and that parents were less willing to enroll children in programs that received a lower star rating. “The big picture is that there’s been a huge investment in QRIS, but there hasn’t been any research until now around whether the QRIS are having an effect on making change,” said Bassok. “Do programs actually respond to this and do parents actually respond? What our study provides evidence of is programs and parents can and do respond. This is encouraging evidence that the basic premise of accountability systems can work.” Interestingly, the effects of improved quality and lower enrollment among programs receiving lower star ratings were concentrated in communities that had higher levels of competition between ECE providers.

The results of the study offer evidence that the North Carolina QRIS does, in fact, spur improvements among programs receiving lower initial star ratings. The study does not speak to the effectiveness of all the QRIS currently in operation in states other than North Carolina, however. As pointed out in the Transforming the Financing of Early Care and Education study report, the potential of QRIS are not yet realized in many states due to limited financial incentives for participating programs. It’s notable that unlike in most states, North Carolina ECE programs are automatically enrolled in the QRIS. The non-voluntary nature of North Carolina’s QRIS is a feature that other states might want to consider as well as the comparatively large amount of funds the state spends annually on QRIS administration. Investing in QRIS administration can enable states to provide regular quality monitoring and consistent technical assistance to help programs improve. The variations of QRIS that exist across states are one reason why more studies like this are needed.

Enjoy what you read? Subscribe to our newsletter to receive updates on what’s new in Education Policy!