Introduction

We are in the thick of the midterm season, typically a high-stress time of the year where college students across the country are preparing for exams and assignments to show how much of the course material they have retained so far. However, unless you have been a college student in the past decade, you may be unaware that this is also the time of year for midterm faculty surveys. Either mandatory or optional for faculty to complete, this is an opportunity for professors to identify students who are academically at-risk in a given course. It is also the opportunity for faculty to praise those who are performing well. This is typically done digitally through “flags” (for academically at-risk students) and “kudos” (for high-performing students).

But the jury is still out on whether receiving flags and kudos through early alert systems (EAS) are an effective technology to promote academic success, and while there is literature on the impact of EAS at four-year institutions, we lack widespread understanding of its effectiveness for community college students. For the research that does exist in a community college context, the findings are inconsistent and inconclusive.¹ As New America wrote last year, critics fear that student support technologies that use predictive analytics, such as EAS, run the risk of reifying racial discrimination and labeling for many college students.²

In response to the drastic community college enrollment declines over the past two years in the United States, community college leaders are under increasing pressure to retain students.³ As a result, many campus leaders are turning to technology to guide the distribution of limited resources to support student success. Yet despite limited empirical evidence demonstrating a positive impact on student outcomes, community college leaders across the country are readily investing in EAS.

Understanding that technology is not a simple solution to college retention and completion problems, we talked with community college leaders, practitioners, and experts in the field to understand the benefits and challenges of implementing an EAS, while paying attention to the importance of mitigating racial discrimination and labeling that are potential by-products of such technology. This report begins to fill in the knowledge gap with qualitative data so we might understand the components of the design/configuration, implementation, use, and perceived effectiveness of EAS at community colleges.

As of 2019, about four in five community colleges are adopting some variation of a congratulatory and alert system to promote student success.⁴ However, most of the research on EAS is oversaturated with insights applicable to four-year institutions, with very few studies evaluating the racial equity implications of the use of this technology. It is challenging for community college leaders to translate results from these studies to a context that offers diverse types of sub-baccalaureate (and some baccalaureate) credentials and serves more students of color, low-income, non-residential, and working students.

Since a large percentage of traditionally underserved students in the U.S. attend two-year institutions,⁵ it is critical to evaluate how EAS are used to improve and remove potential barriers to their academic success. Addressing this knowledge gap will help faculty, staff, and administrators ensure college students reach their academic goals.

In 2020, Joe Biden’s presidential campaign promised billions of dollars to invest in community colleges with evidence-based student success technologies, such as EAS. Although after his election Build Back Better Act did not materialize,⁶ this summer the U.S. Department of Education announced a $5 million investment to higher education institutions that primarily serve students of color, to invest in evidence-based initiatives that encourage postsecondary retention, transfer, and completion. This grant is known as the College Completion Fund for Postsecondary Student Success. Although it is aimed at four-year historically Black colleges and universities (HBCUs), tribal colleges and universities (TCUs), and minority-serving institutions (MSIs) such as Hispanic-serving institutions (HSIs), the Department is extending “invitational priority” to community colleges that experienced the brunt of enrollment declines during the pandemic.⁷

There is increased interest in empirical studies to inform student support practices as a result of a combination of federal investments and institutional motivation to recover from the impact of the pandemic. This study provides insight for community colleges who wish to use this technology equitably and effectively to promote student success.

It is important to understand what is and is not working to guide limited resources to support community college students. Our study provides insights that we gathered from interviewing various community college leaders, third-party EAS vendors, and experts in the field. Through our interviews, we identify the five most common challenges to implementing EAS at community colleges. We hope that our recommendations will help guide two-year campus leaders and third-party EAS vendors to address popular challenges amongst notification and intervention systems to ensure they are adequately serving all students and are promoting student success.

For this study, we are exclusively interested in understanding the challenges and experiences of implementing early alert systems (EAS), a component of a case management system, at community colleges.

From November 2021 to April 2022, we gathered qualitative data through semi-formal virtual video interviews, an invaluable source of information to explore community college leaders’ experiences, challenges, and nuances of implementing technology with EAS. We also interviewed third-party EAS vendors and experts in the field.

Community colleges, third-party EAS vendors, and experts were selected based on New America’s research, as well as suggestions from researchers and prominent experts in the field, including Achieving the Dream. All interviewees remain anonymous in the report.

Selection Criteria

Community colleges met three or more of the following criteria:

• Nonprofit community college
• Implemented EAS no later than 2016
• Used a third-party vendor’s EAS platform
• Used EAS to improve student retention and completion in associate degree programs
• Provided permission for New America to speak with campus leaders about EAS

The majority of community college leaders we spoke with hired an external technology vendor to facilitate their implementation of an EAS. A small percentage of our interviewees used an EAS tool that they either developed internally or personalized extensively to fit their unique student population.

Limitations

Due to the limited scope of the project, from our qualitative data, we are unable to account for students’ experiences with EAS at community colleges.

Limitations of interview data include, but are not limited to, the fact that interviewees may not feel encouraged to provide accurate, honest answers or may not be fully aware of their reasons for any given answer, all of which is beyond our ability to control.

The selection of interviewees is not nationally representative and thus cannot be generalized across all community colleges in the country. In addition, the qualitative findings cannot address causality of the effectiveness of EAS on student success outcomes. However, the findings shed light on some perceptions, beliefs, and experiences of implementing EAS at community colleges.