It’s Not All Relative When Measuring Learning Gains And Losses

Dual language programs are offered in 42 states plus the District of Columbia. These programs provide instruction in English and a partner language such as Mandarin and offer students the chance to become bilingual and biliterate. Their popularity has grown in recent years due to research which has identified many benefits associated with bilingualism and has found positive correlations between dual language programs and academic achievement. For example, a 2017 study of dual language programs in Portland Public Schools found that both native English speakers and English learners (ELs) assigned to dual language programs outperformed their peers in reading by seven months in the 5th grade and nine months in 8th grade. While these studies focused on measuring relative gains from year-to-year, there is a gap in research on the relationship between dual language education and students’ shorter-term growth trajectories within and across grades.

A recent working paper by Angela Johnson, a researcher with NWEA, fills this gap by estimating what she calls the ‘seasonal learning patterns’ of Latinx students enrolled in dual language programs. This approach breaks from traditional study models by focusing on not only how effective dual language programs are at closing opportunity gaps, but also breaking down growth rates within each grade between 2nd and 8th including summer loss rates between each grade. Lastly, the study accounts for student-and-school characteristics that have been identified as affecting students’ academic achievement, such as EL identification, eligibility for free or reduced lunch, and enrollment in special education.

Johnson draws on data from the NWEA MAP Growth assessment from a large school district in the Midwest that is typically given three times a year (fall, winter, spring), to compare the performance of Latinx students enrolled in the dual language program with those of Latinx students who did not participate in the program. At the time of the study, over half of students in the district were Latinx, a large proportion of students (just under two-thirds) came from low-income families, and current ELs made up over 30 percent of its total student population, of which the majority (80 percent) spoke Spanish at home. Most importantly, the district has offered dual language instruction as the primary support program for Spanish-speaking ELs since 2011-12 which is available to students even after they are no longer classified as ELs. In total, the pool of data analyzed included 341,448 math and 341,333 reading scores representing 27,661 students in 51 schools.

Johnson uses the assessment data to investigate: a) the performance of dual language students relative to students in other programs; and, b) patterns in student learning during summer break to gauge dual language programs’ effectiveness at providing extended learning opportunities when students are out of school.

To that aim, Johnson’s research was driven by two research questions:

1. How does Latinx dual language participants’ achievement status compare to nonparticipants’?
2. How does Latinx dual language participants’ achievement growth within and across grades compare to nonparticipants’?

Among the study’s various findings, the data show that the opportunity gap between dual language students and the national average in math grew with the biggest dip between the spring of 6th and the spring of 7th grade, while non-dual language students shrank their gap. In reading, both groups managed to close their respective gaps.

In terms of long-term achievement (measured between fall of 2nd grade and winter of 8th grade), dual language students scored slightly lower than non-dual language students in both math and reading, and over time, dual language students remained slightly below non-participants. Overall, this initial low score proved difficult to overcome.

In math, for example, while both groups of students shared similar growth and loss trajectories from 6th to 8th grade, dual language students’ summer learning loss outpaced their relative growth between grades 2-5. In reading, both dual language and non-dual language students managed to shrink their respective gaps compared to the national average as they moved to higher grades. Despite having similar academic-year growth and summer loss rates, dual language students still remained slightly below the non-dual language group.

The study’s findings on month-to-month growth estimates provide a layer of detail that is often missing from the conversation. As Table 1 shows, dual language students’ growth trajectories differ from non-dual language students, especially in the early years, and between math and reading, students’ needs diverge almost immediately after 2nd grade. Lastly, it should be noted that while dual language students were the focus of the study, the study also breaks out growth estimates specific to ELs in both math and reading. The data show these estimates to be very similar to the full sample.

Enjoy what you read? Subscribe to our newsletter to receive updates on what’s new in Education Policy!

While previous studies have found resounding positive correlations between participation in dual language programs and academic achievement, Johnson’s study shows that students’ academic performance and associated needs are more nuanced when evaluated on a month-to-month basis. Focusing on these short-term academic gains and losses may allow schools to identify immediate learning loss before it hardens into a persistent opportunity gap.

In addition to its invaluable contribution to how the field monitors, evaluates, and attempts to address academic achievement gaps for racially and linguistically diverse students, the methodological approach of this study may prove useful in other contexts, particularly as education leaders grapple with how to assess COVID-related learning loss in a timely fashion.

At the beginning of the pandemic, education leaders warned that school closures would lead to learning loss on a scale larger than the usual ‘summer loss’ that students typically experience between academic years. Broadly referred to as the ‘COVID slide’, early projections estimated that a lack of continuity of instruction could result in students only retaining 70 percent of their reading progress relative to a typical school year, and lose between 50 - 100 percent of their annual growth in math compared to a normal year.

Of course, no one could have guessed that more than nine months after closures began schools would still be in limbo about whether to reopen their buildings, exacerbating students’ lack of exposure to regular instruction. Nevertheless, the 2020-21 school year is well on its way, whether remote or in person, and reports attempting to quantify COVID-related learning loss have started trickling in.

On the surface initial data indicate that students’ performance in reading and math hasn’t been as bad as expected. A closer look at the fine print, however, reveals that certain vulnerable groups of students, including ELs and racial minorities, were largely underrepresented in the data used to come to those conclusions. This is because these initial studies used data for students assessed in both fall of 2019 and 2020. The problem is, though, that because of COVID, the students who were able to test in the fall of this year were less racially diverse and tended to be enrolled in higher socioeconomic schools.

Despite being given broad flexibility to bypass federally mandated testing under the Every Student Succeeds Act for the 2019-20 school year, states have been advised not to expect the same for the 2020-21. That being said, state and federal academic assessments remain up in the air for 2020-21, which means the data needed to calculate these types of month-to-month, year-to-year learning gains and losses may not exist.

The Institute of Education Sciences recently launched ‘Operation Reverse the Loss’, a campaign intended to address pandemic-related academic loss. It is unclear what this body of work will entail, however, moving forward it is critically important that efforts strike the right balance between accountability and not placing undue pressure on children and families hit hardest by this pandemic and school closures.