Five Common Challenges and Recommendations

Choosing an EAS can be time-consuming and difficult. It is a big decision that can cost thousands of dollars. It can lock a school into a particular system well into the future, even if the system turns out not to work as advertised. At the same time, administrators do not always know the best questions to ask vendors or the answers they should expect. Even more complicated, salespeople do not always know the answers to technical questions around data integration and data use.

Many college leaders believe salespeople pitch a product during the initial conversation that is good in principle but often falls short in practice. The community college practitioners we spoke to said that third-party EAS vendors can be misleading about the automated functionalities of a system, when in fact many processes require manual integrations. Or third-party vendors sometimes misinform community colleges to believe their EAS can link to other college platforms, which is not always possible in practice.

The community college needs to be sure that any tool it chooses will work well for its particular student population, have the flexibility to provide what the college needs in a timely manner, include data transparency, ensure data privacy and security, work interoperably with existing data systems, and support evaluation and professional development.

Recommendation: Make an informed decision between using a third-party vendor or developing an internal system, while being intentional of decision-makers at the table.

To decide what tool to procure, the college should bring together a diverse group of stakeholders to evaluate how the school plans to use it and how it will fit into existing workflows. This group should include someone in institutional leadership who is charged with leading the process, information technology staff members, institutional research staff, key faculty members, student advisors and navigators, other student support staff, student representatives, and communications staff. For many college leaders we interviewed, this diverse group of professionals participate at various stages of the procurement process. This group helps to decide:

1. What functionality the system should have. The leadership team should decide what the most important functions are. Many times it is tempting for colleges to think about how EAS can solve all of their student support problems. However, many of the most effective EAS are focused on solving one or two problems well. That is what colleges indicated in a 2017 Tyton Partners’ survey, which found that the majority of schools believe more focused solutions perform better than systems with a lot of different functionality.¹
2. How the system might need to be customized to the college. When the group decides on the most important functionality, it should also think through how much of that functionally will require customization. The more unusual the functionality, the more likely a college will need to customize its tool. That customization can be very expensive to create and maintain.
3. Whether the college should purchase a system or build its own. The questions around use and customization may help a college decide whether or not to build their own EAS rather than going with a vendor created system. But a community college must start with one question first: do they have the staff capacity to create and maintain an EAS? If the answer to that question is yes, the leadership team may want to explore some other reasons for creating a system in-house rather than buying a product. If a college wants to evaluate interventions that take discrete, nimble analysis, it can be easier to keep that analysis in-house rather than trying to work through a third party. If the team concludes that any system will need a lot of customization, that too could mean the college should build its own system. Cost of purchasing a system is another consideration. However, in-house systems are not always as inexpensive as they seem. Thousands of staff hours can go into creating and maintaining these systems and many colleges end up setting them aside for a purchased product because they are too difficult to maintain through staff turnover. People we spoke to at one college described the challenge of maintaining their in-house system when institutional knowledge was lost because of the retirement of staff.
4. What kind of privacy and security protections are needed. This team should think through who at the college will have access to which student data. Figuring out what kinds of permissions the system can have so it protects student privacy and supports the school’s information security is a key consideration for any system.
5. What kind of process the school needs around messaging alerts. We know that messages about seeking help can be discouraging to students who are struggling. As one administrator we talked to said, "a strong procurement process involves stakeholders of diverse backgrounds and experiences, where student voices are included as well. Students can share feedback on what different kinds of alerts mean to them to inform the type of language to use and whether systems are user-friendly for students." To avoid harm, messages to students should be accurate, timely, and framed as strength-based rather than deficit-based.²

Faculty members play a fundamental role in the use of EAS within community colleges. Professors see the different challenges that students face firsthand. One college interviewee said that their campus initially considered adopting an EAS because faculty collectively asked the college to do so, based on the needs they witnessed from their students. However, it is common for many community colleges to struggle with faculty buy-in when implementing an EAS.

Many college administrators shared reasons why faculty were reluctant to use an EAS. Some faculty did not believe in the promises of the system because of a lack of proof on short-term and long-term benefits of EAS on academic performance. Others had reservations about handling student data. Others were not interested in being held accountable to a system that they were not familiar with, while some were discouraged, from previous experiences with the system, about feeling left in the dark about what happens to students once they initiate a flag. Without adequate faculty buy-in, EAS will not work because these systems depend on faculty input for flagging alerts and engaging with student interventions.

Recommendation: Include faculty feedback during the procurement and pilot phase, require faculty training on how to use EAS that includes DEI and implicit bias training, and be considerate of faculty workload.

To develop faculty buy-in, it is important that colleges should involve faculty in the procurement and pilot phase prior to adopting an EAS or developing an in-house system. With the prevalence of faculty use in raising alerts, their input is imperative, and their feedback should be carefully considered. This can be done in the form of creating faculty focus groups or including faculty (adjunct faculty as well) on an EAS advisory committee during the pilot phase. This allows them to collaborate with administrators on ways to understand the context of signs that would cause them to initiate an alert. This inclusion would also help cultivate the faculty-administrator relationship that is critical to EAS implementation. Additionally, faculty can advise colleges on ways to personalize inputs of the system (e.g., pre-populated flag types based on common student behavior or options to personalize alerts) so that faculty members are not discouraged or overwhelmed by the types of flags to raise.

Faculty should be informed about the goals and outcomes administrators hope to achieve by implementing EAS so that institutional efforts are aligned across all offices and departments. Colleges can do this by disseminating problem statements and the vision with faculty or sharing evidence of the advantages of EAS and best practices adopted by other community colleges. This will help signal that this is not just another system adoption. Colleges should also implement mandatory professional development training so faculty can learn how to use the alert system and establish a form of accountability for faculty to continue to use the system and improve their practices. It is key that faculty members understand how and why flags are raised and how their participation feeds into institutional goals.

To cultivate equitable use of EAS, faculty training must include diversity, equity, and inclusion (DEI) and implicit bias training. Because community colleges enroll a large percentage of racially minoritized, low-income, and working students, alert notifications can sometimes be mistakenly triggered by faculty who are not knowledgeable about teaching students from diverse backgrounds.³ These alerts can have negative effects on students of color.⁴ Without these trainings, colleges run the risk of embedding biases into the system, resulting in flagging certain students at higher rates and inaccurately labeling students as academically at-risk. These trainings can open doors to inclusive opportunities for colleges to personalize features of their systems and equip faculty to meet the needs of minority students.

Another way to ensure faculty participation is that once faculty raise a flag, they remain informed through a feedback loop process to know what intervention(s) a student did or did not receive. For example, if a student is flagged for missing an assignment, the professor should be kept in the loop on what the advisor or assigned departmental staff is doing to intervene after the flag is raised. Tailoring EAS to update faculty members with notifications about the stages of an intervention institutes an additional layer of accountability and can help ensure students’ cases do not fall through the cracks. This will also allow professors to know how to continue to support their students and navigate next steps, depending on where students are in the EAS cycle.

Lastly, considering that many faculty members at community colleges are adjunct professors, administrators must be considerate about other responsibilities they have, so they do not disincentivize faculty from using the system. Flagging multiple students and keeping track of alerts can quickly become taxing, especially for faculty with bigger class sizes. It will be helpful to create built-in tools that can easily allow faculty to fulfill their role within EAS. Colleges should create an atmosphere where faculty members are not just using EAS to fulfill their responsibilities, but genuinely understand the results their participation can yield.⁵

Although EAS are designed to help colleges identify students who are academically at-risk or who need non-academic support services, administrators shared with us how connecting students with the right support services can be a challenge with their alert system. The top two reasons are:

1. Student support staff are overwhelmed. Some colleges do not have a clear or adequate process for handling all open cases across departments, which runs the risk of EAS staff overburdened with alerts. For some colleges, once a faculty member raises a flag, it takes a considerable amount of time for an EAS representative to review the flag, reach out to the student, and connect them to the appropriate support service. Unfortunately, as we heard from some EAS staff at community colleges, many students and faculty members assume there is a call center managing the influx of the alerts in a quick and efficient process. However, the reality is that it is usually one or two staff members managing the alerts, and they often have additional responsibilities outside of EAS. As a result of this limited staffing, flags sometimes fall through the cracks, with many students not receiving the support they need.
2. Messaging is ineffective. When faculty members send alerts and/or when staff members reach out to students to connect them to an intervention, both messaging approaches must be treated with care so that students' needs are adequately met.⁶ One administrator we interviewed shared how their college did not think much about the word “probation” when used in the context of financial aid or a student’s academic standing. However, this college primarily serves students of color, and so the students interpreted the term “probation” in a negative context that adversely affected their reception of the notifications.⁷ This resulted in decreased student engagement with the interventions the college was attempting to offer.

Recommendation: Build up staff capacity in monitoring early alerts, develop a process to streamline flags between offices, and establish inclusive communication practices with students.

One campus administrator told us that “one coordinator cannot manage all the interventions that need to be done with alerts.” This person candidly expressed that community colleges must address staffing needs to manage and monitor EAS flags, especially for institutions that enable manual alerts to be submitted throughout an academic term. Increasing staffing is a challenge for many community colleges that have limited financial resources to hire additional staff. Yet colleges must think of innovative funding mechanisms, through applying for grants and leveraging technical assistance organizations, such as Achieving the Dream, to support staffing needs to manage alerts.⁸ If colleges do not build staff capacity, rethink procedures, and create institutional policies to support EAS infrastructure, they risk students falling through the cracks and worse: a decline in student success outcomes.

It is imperative to develop an efficient process that streamlines flags initiated by faculty to connect students to appropriate interventions. One community college interviewee said that their process uses an intermediary staff member who receives the alert and then connects students to the appropriate interventions. Another community college leader told us that their school programmed their system to streamline flags directly from faculty in the classroom to the appropriate department that houses the intervention. For example, if a student is experiencing financial issues, the professor’s flag is immediately sent to a financial aid representative who then connects with the student to provide an emergency grant. Both options have pros and cons, yet both rely heavily on staffing. Without bolstering staff capacity, EAS are limited to the misconception that flags are lost in a black hole and students are not receiving support.

Lastly, colleges should carefully consider the type of language used in message notifications to students. A recurring theme from our conversations with community college administrators is the importance of effective messaging when sending early alerts to students. The choice of words used to alert or intervene is critical in effectively connecting with students and encouraging them to use appropriate student support services.⁹ Mindfulness about inclusive language when creating messages helps ensure that students are receptive to the notifications and interventions.¹⁰ One community college leader shared that their school hired an external consulting company to revamp their messaging for their EAS system. Although it was an expensive collaboration, it paid dividends in effectively communicating with their student population who are primarily students of color, the person we spoke to said. Not only did the consultants help the college implement inclusive language, which increased student engagement with the EAS process, but it was an eye-opening experience for administrators to reevaluate their positions of privilege to ensure they meet all of their students' needs. Effective messaging ensures that targeted approaches or support services are reaching the students they were designed to support. Activating the student voice in system design, configuration, testing, and implementation is critical for an effective and efficient EAS.

The effectiveness of an EAS is determined by how well it is used by faculty members. However, the laser focus on faculty inputs at many community colleges make the mistake of only engaging in the measurement and evaluation of EAS data based upon frequency of flags triggered by faculty and faculty participation. Limiting EAS evaluation with these superficial analyses poses a major challenge to fully understanding what is and is not working with a college’s alert system. The frequency of flags and the rate of faculty participation do not dig deep enough to understand the process of streamlining flags from faculty alerts to the connection of services to students. Looking only at these factors also does not help college leaders understand the impact interventions have on students’ academic and non-academic outcomes. Yet the majority of the colleges and vendors we spoke with focus their EAS evaluations on those two outcomes. This significantly restrains insight to a small aspect of the system and limits colleges from truly understanding whether the system is working to promote student success.

Almost all of the community colleges and vendors we spoke with had not previously evaluated flag frequency across key demographic characteristics such as race, gender, and full-time status, with many not thinking about this approach before our interview. To not disaggregate data is to admit ignorance about how sub-populations of students are faring. Wearing these blindfolds prevents colleges from implementing student-centered approaches with EAS and is a disservice to students. Many colleges do not uncover the full potential of EAS because of data evaluation limitations they impose on themselves.

Recommendation: Go beyond surface analyses of flag frequency and faculty participation, and disaggregate data by demographics, intervention type, and student outcomes.

The leadership team should think about what student success looks like for an EAS at their college and how to measure it. For many colleges represented in our sample, success was defined in terms of fall-to-fall retention, course completion rates, and graduation rates. Once colleges understand how they want to measure student success, they should come up with a measurement strategy to support regular evaluation. To guide interim evaluations, measurement strategies might include research questions like:

• Are certain types of students more likely to receive alerts than others? Why?
• How do students who receive alerts tend to perform compared to students who do not?
• What kinds of interventions in response to alerts seem to work best for students?
• How many alerts remain open without properly addressing students?
• How long are alerts opened before they are closed?
• Do closed alerts mean that students received adequate interventions? If not, what type of students are receiving interventions compared to those who are not?

A plan to disaggregate data by key demographic characteristics and answering these questions at regular intervals will help ensure the system is working for the college to reach their goals of student success. This approach can improve student-centered practices and streamline the process from faculty alerts to the provision of intervention(s) in a timely manner.

By the end of the 2020-21 academic year, community colleges felt the brunt¹¹ of the COVID-19 global health pandemic, causing enrollment rates to decline by 10 percent¹² and retention rates to decline by 2.1 percentage points.¹³ These unprecedented declines, coupled with the transition to emergency distance learning, made it difficult for colleges to seamlessly adopt and effectively use EAS. It was challenging for many community colleges to monitor students’ attendance and identify students’ needs in a virtual setting. For example, inconsistent attendance policies for remote learning made it difficult for faculty to input correct information regarding absences into the system. There was a general concern during the height of the pandemic that students were not flagged accurately because faculty members were unable to rely on traditional ways of identifying at-risk academic behaviors.

During the peak of the pandemic, there were countless reasons for absences and falling behind academically: testing positive for COVID-19, being exposed to the virus, caregiving for sick family members, homeschooling children, lacking access to broadband internet or not owning a laptop, and more. Many college administrators grappled with how to swiftly identify and connect students to appropriate interventions in a remote context. Furthermore, many colleges struggled with defining what a successful intervention looked like virtually. These challenges called for colleges to be flexible and discover new ways to meet students' needs. A third-party EAS vendor shared one example, about one of their community colleges hosting virtual advisor meetings on Saturdays to offer flexible slots for students with inconsistent weekly work schedules. This innovative thinking was one answer to pivot EAS interventions during the height of the pandemic.

Recommendation: Redefine student engagement and attendance policies, holistically support students by strengthening student support services, and maintain the personal touch within EAS.

Although most colleges are back to in-person classes, lessons from the pandemic are important as higher education increasingly moves to hybrid learning. Colleges must redefine what constitutes an alert in different learning modalities and ensure that institutional policies are aligned and clear for faculty to understand and implement an EAS in a hybrid environment.

The pandemic was an unexpected opportunity for community colleges to reevaluate and approach student success more holistically and to strengthen retention and completion efforts. We heard from some community college leaders that emergency distance learning caused a ripple effect, increasing student access to mental health counseling and one-on-one meetings with professors through virtual engagement. Once colleges addressed the disparities in access to broadband internet and laptops, online meetings reduced the equity gap in accessing campus resources and staff because they created more flexibility for more students (e.g. working students) and faculty to interact. Colleges must continue to provide hybrid support services to maximize the accessibility benefits that virtual interactions offer.

Colleges should also emphasize the importance of the use of EAS flags beyond signaling for academic support services. The rate of mental health services among community college students skyrocketed during the pandemic and more student support services were needed.¹⁴ An EAS vendor shared with us how "the pandemic forced colleges to not just see students as academic performers, but take a more holistic approach” in meeting their needs. Student support staff must improve their practices to adapt to the growing demand for non-academic support services.¹⁵

However, maintaining the personal touch when using EAS is key.¹⁶ Although EAS can be effective in promoting student success, students may be discouraged to engage with a notification or an intervention if they believe the notification came from a bot rather than a person. And with the reliance on technology increasing because of the pandemic, it is imperative that each flagged student receives some form of personalized engagement from faculty and staff.¹⁷