The ‘Race to the Top Winners’: How States Plan to Assess Kindergarten Readiness

This is the fifth post in a series on winners of the Race to the Top – Early Learning Challenge (RTT-ELC), the Obama Administration’s competition to spur improvements in early learning for children up to age 5.  Two weeks ago, we wrote about states’ plans to develop the workforce. Prior to that, we dove into states’ plans to improve early learning standards, birth to 5. The first two posts in this series described states’ plans to use and evaluate quality rating & improvement systems. Today we explore states’  plans for kindergarten entry assessments.

One of the priorities for the RTT-ELC was “Measuring Outcomes and Progress.” States had two options under this category: kindergarten entry assessments (KEAs) or early learning data systems. Applicants could choose to describe their plans for one or both options. All the winners chose to write about their plans for KEAs. Six of the nine winners also explained their plans for building or enhancing their states’ early learning data systems. In addition to the 40 points available for this priority area, states could also earn 10 extra credit points for having a high-quality plan to implement a common, statewide KEA that informs instruction and services in the early grades of elementary school.

States’ descriptions had to show their current or planned KEA is:

• Aligned with the state’s early learning standards and covers all “essential domains of school readiness,” which include: language and literacy development, early math and science, approaches toward learning, physical development and social and emotional development;
• Valid, reliable and appropriate for the target population of kindergarten students, including English language learners and children with disabilities;
• Administered beginning no later than the start of the 2014-2015 school year;
• Reported to the state’s longitudinal data system and to the early learning data system if they are separate; and
• Funded, in significant part, with federal or state resources other than RTT-ELC grant funds.

All of the winners’ plans adequately addressed the requirements above. Maryland was the highest scoring winner with 19.6 out of a 20 possible points. The entire Measuring Outcomes and Progress section was worth 40 points. If states opted to develop plans for both subsections, they were worth 20 points each. Here’s how the winning states fared:

Delaware scored well below the other winners. Most reviewers scored its plan as “medium-quality” and “partially implemented,” resulting in the lower score. Non-winning states that were high-scorers in this section include Florida, Maine, Colorado, New Mexico and Connecticut.

Below are details on what winning states are doing or plan to do when it comes to kindergarten entry assessments.

Improving Work Sampling Systems

• Maryland and Ohio are planning to develop new KEAs, which will be field-tested in 2013 and given to all public school kindergarten students in 2014. Since 2001, Maryland has used an adaptation of the Work Sampling System as its readiness assessment. Each November, teachers conduct the observational assessment and children are categorized as “fully ready,” “approaching readiness” or “developing readiness.” Ohio also has an existing KEA, but it measures only children’s literacy abilities, while Maryland’s portfolio system also looks at social/emotional development, artistic development, and other skills.

Coupled with the development of a new KEA will be the creation of formative assessments for pre-K and kindergarten  teachers, to help them tailor their instruction to individual student needs. The various assessments will be computer-based. Data from the entry assessment, across all domains of learning, will be captured by an online system. The states plan to create professional development workshops for teachers on how to use the assessments. To support this face-to-face learning, the states are also planning to develop online tools and content and create professional learning communities. Additionally, Ohio and Maryland will establish a teacher certification process to ensure educators are able to reliably administer these assessments. Ohio also intends to link KEA results to children participating in publicly funded preschool programs, as well as programs participating in the state’s QRIS. Maryland and Ohio will share the costs for development of the KEA, and once ready, they plan to make it available to other interested states for purchase.

Both states, along with several other winners, are part of the Partnership for Assessment of Readiness for College and Career (PARCC) Consortium, one of two consortia receiving a federal grant to establish common state assessments aligned to the Common Core Standards. Maryland and Ohio will align their kindergarten through second grade assessments with PARCC’s system. The other national standards group, the Smarter Balanced Assessment Consortia, plans to develop assessments for grades 3-12 only.

• Similar to Maryland, Minnesota is currently using an adapted version of the Work Sampling System as its KEA. The state has given the assessment to a sample of children for the past 10 years, and has completed a validation study to determine how well the assessment predicts third grade achievement. (Read pages 195-196 of the application for more on the results.) Minnesota plans to expand and refine its system, linking KEA scores to the state’s longitudinal data system and convening a task force to discuss how the current KEA could be enhanced. Districts are encouraged to use KEA data, among other information, to identify staff development needs and to inform their local literacy plan to ensure every child is reading at or above grade level by the end of third grade. Moving forward, the state plans to use its current KEA for the 2011-2012 school year, and will begin planning to implement a new KEA statewide in 2014-2015.

State-created assessment

• In 2010, California contracted with the WestEd Center for Child and Family Studies and the University of California – Berkley Evaluation and Assessment Research Center to develop and test an assessment for school readiness, called the Desired Results Developmental Profile for School Readiness (DRDP-SR). This assessment is currently being implemented as part of the state’s existing “Desire Results System,” which was established in 2001 to improve quality in preschool programs. The state already uses a similar developmental assessment for preschool, which is aligned to the state’s early learning standards and covers all the essential domains of school readiness.

The DRDP-SR is an observational assessment designed to inform curriculum planning for individual children and support continuous program improvement. The instrument was piloted with four levels of readiness: developing, building, integrating and applying. After the pilot, however, officials determined some of the measures were too difficult for kindergarten students. As a result, the state plans to add an additional “early” level of readiness, called “practicing.” (For more on this challenge read pages 172-173 of the application.) The assessment will be finalized this spring and ready for implementation on a voluntary basis by the 2014-2015 school year, with early adopters using it this coming school year. California also plans to develop training materials and web-based tools for teachers on how to use the assessment.

Teaching Strategies GOLD

• Washington piloted and evaluated its KEA, the Washington Kindergarten Inventory of Developing Skills (WaKIDS), during the 2010-2011 school year. It will become mandatory for full-day kindergarten classrooms this September. Washington’s assessment includes three components: 1) the Teaching Strategies GOLD assessment to gather readiness data; 2) one-on-one or small group meetings between kindergarten teachers and families at or before the beginning of school year to discuss children’s strengths and to set goals for the year; and 3) collaboration between early learning providers and kindergarten teachers before the school year to share information, smoothing transitions for children.

Prior to full implementation for all kindergarteners during the 2014-2015 school year, the state will host a series of professional development trainings for districts, schools, teachers, and support staff. Through a partnership with Thrive by Five Washington, the state will introduce the public to the concept of KEAs. The University of Washington will conduct two validity and reliability evaluations to ensure GOLD results are accurate. (For information on these evaluations, see page 178 of the application.) The state will integrate WaKIDS data into its longitudinal data system and report results to the public.

• Delaware also plans to use Teaching Strategies GOLD for its pilot KEA, but this formative assessment won’t necessary become the state’s official KEA. Currently, many Delaware school districts limit their assessment of new kindergarteners to literacy readiness, but with the RTT-ELC grant the state plans to develop or select a KEA that measured all domains of school readiness.

Delaware’sGOLD pilot began in June 2011 and will run for two years. In the application, state education officials write the purpose of the pilot is to “provide policymakers with a deeper understanding of the challenges and supports required for kindergarten teachers to utilize a formative, multi-domain assessment to appraise the skills of each of their students upon school entry, and then to translate the data into improved instructional practice.” Following the completion of the GOLD pilot, Delaware plans to examine several other potential assessment tools, as well, to determine which is most appropriate.

Delaware envisions its KEA as two pieces: a formative assessment, like GOLD, and a family questionnaire. The state also plans to use a separate assessment to validate the results of the formative assessment to ensure it is a reliable measure that can inform state and local policy decisions. If the formative assessment is found to be valid, the DOE may phase out the additional assessment. Delaware is aiming for the KEA to be used in all classrooms by the 2014-2015 school year.

Building on existing work or starting fresh

• North Carolina currently conducts a KEA and an ongoing formative assessment in literacy and math in kindergarten, first and second grades. Due to weaknesses the state has identified in its current assessment strategies, North Carolina plans to use the RTT-ELC grant to move in a new direction for measuring kindergarten readiness. It plans to build on the existing assessment and create a new KEA that provides data to help close the achievement gap and helps improve instruction in the early grades.

North Carolina will broaden its current K-2 assessments to include all essential domains of school readiness and extend to third grade. The assessments will be developed with the goals of reducing the school readiness gap at kindergarten entry, as well as by the end of third grade. The state is currently revising its early learning standards, so officials plan to hold off on finalizing the content of the KEA until it can be aligned with the new standards.

In its application, North Carolina says, “We do not believe there is an existing measure that meets the criteria for reliability and validity, is aligned with its early learning standards, is appropriate for all children and matches the purposes required by the RTT-ELC.” Other winners seemed to lean more toward reviewing and selecting or adapting the best existing assessment tool rather than create their own. North Carolina also noted in its application that the state would be open to participating in a consortium of states to develop an appropriate assessment tool, although it didn’t name other states it had approached or might be interested.

North Carolina plans to fully implement the KEA during the 2014-2015 school year, beginning in schools with the largest number of high-need children. The state also plans to use a significant portion of the funds on professional development, teaching educators how to implement the new assessment tool and use the data it generates to guide instruction.

• Rhode Island does not currently have a common, statewide KEA. By law, however, school districts are required to screen children prior to school entry to determine how prepared they are in literacy and math. Districts must develop “personal literacy plans” for children who are reading below grade level, but individual districts can choose their own screening tools. In December 2010, Rhode Island began planning for a statewide KEA, establishing an assessment work team as part of the Rhode Island Early Learning Council. The state plans to pilot its newly developed KEA in select districts during the 2014-2015 school year. Additional school districts will be added, beginning with districts with the most high-need children, in the 2015 and 2016 school years. The costs of implementing this KEA are included in the state’s education funding formula. Rhode Island plans to use grant funds to develop or identify an appropriate assessment and develop training for teachers around it.

• Massachusetts will also implement a new KEA system. The state is unique among the winners in that it prioritizes local decision-making. Massachusetts will require participating schools to use an approved formative assessment tool that covers all essential domains of school readiness: the Work Sampling System, Teaching Strategies GOLD, High Scope COR, or any other tool the district chooses and for which it obtains state approval. Pre-K programs participating in the state’s tiered QRIS or that receive state pre-K funds are already required to use one of these assessment tools. This raises questions about alignment: It will be possible for a pre-K program to use work sampling, for instance, and for the public school it feeds into to choose another tool, such as GOLD. It isn’t clear whether elementary schools will be asked to consider what their feeder pre-K programs are already using when making their own assessment tool decisions. We hope they do. It would be beneficial for principals to connect with local public pre-K directors to discuss why they chose a certain assessment and challenges they encountered and to determine if it makes sense to use the same tool.

Massachusetts also plans to develop its own statewide measure of children’s school readiness by cross-walking the items on the three state approved assessments to find similarities. The state will contract a qualified institute of higher education to complete this work. The state expects that the common measure will provide data, regardless of assessment tool used by districts, to provide a statewide picture of children’s readiness for kindergarten. In year two of the grant, Massachusetts plans to release a series of reports, including one on how the common metric was created and a second on the results of the pilot study using the common metric.

In reviewing Massachusetts’s application and peer reviewers’ comments and scores, we found a scoring mistake. The state earned the 10 extra credit points available for implementing a kindergarten entry assessment, but it should not have. According to reviewers’ instructions, to earn the KEA extra credit points, a state had to either:

• Verify that the state has already implemented a KEA that meets RTT-ELC requirements, or
• Write a response to the kindergarten entry assessment section (E1) of the application that scored at least 70 percent of the maximum points available.

In Massachusetts’s case, there were 20 maximum points available. From the reviewers, Massachusetts earned three scores of 12 out of 20, a score of 16 and a score of 18. The three scores of 12 only equal 60 percent of the total points available, making the state ineligible for the points since it does not meet the other criteria of already having a qualifying assessment in place. Yet one of the reviewers who gave the state 12 points still granted it the extra credit anyway. The other two reviewers who scored the KEA section a 12 did not grant the extra credit. More than likely, this was simply a miscalculation by one of the peer reviewers.

And upon further review, we found this error would not have dropped Massachusetts’s score enough to take it out of the top nine. The state would still be a grant winner. We did, however, think it was worth noting that mistakes like this do happen. Federal officials didn’t catch this one either, as the 10 extra points were included in the state’s total score.

Our final post in this series will revisit one of the RTT-ELC’s invitational priorities: to sustain and build upon early learning outcomes throughout the early grades of elementary school. We wrote about the applicant’s plans back in December. Next week we’ll take another look at the winners’ proposals.

Be sure to visit our special page on the Race to the Top – Early Learning Challenge for continuing coverage.