Part 1: The Science Behind Communicating

Imagine it’s your first year on your new college campus. There are so many new people to meet, things to learn, and institutional processes to wrap your head around. It’s been a tough few weeks, but you are confident that this year will be different, that your old habits—procrastination, working alone, not asking questions—will stay behind. You’ve even heard that your current institution has some cool new technology to help you do better in school. You recall that this technology was mentioned at orientation, but you don’t really remember much else about it.

Now it’s halfway through the semester and you just went through your first round of midterms. That was tough. Despite having done all the right things, like studying in groups, going to office hours, and practicing good time management, you still feel like you didn’t do as well on your exams as you would have liked. It was pretty hard to give school your all while you were juggling multiple jobs and stressed about cutting corners to stay within your budget. And the troubling news you got from back home in the middle of midterms didn’t leave you in the best headspace to concentrate. "Oh well", you think, you really did do your best, so you’re probably ok. It’s probably just new student jitters.

You wake up the day after your last midterm to your phone’s buzzing. There’s a message. You don’t recognize the sender, but curiosity gets the most of you, so you open it:

Image 1¹

“What. The. Heck. Who is this? How do they know my grades?” A million questions race through your head as you scramble to check your grades. Last you remember, you were doing ok. Not straight A’s, but you were definitely passing everything. Did your midterms really go that badly?

Shoot. You really are failing. Not all of your classes, but one really important course for your major, the one you’ve been struggling with the most. This message is just the cherry on top of a series of challenges, self-doubts, and obstacles during your education. Let’s face it—you were right when you told yourself you weren’t college material. This message is just proof of that.

This scenario shows the hazards of communicating predictive findings to students without care. Though sent with good intentions, this anonymous message is counterproductive, increasing this student’s anxiety, self-doubt, and even her likelihood of dropping out.

Predictive analytics has taken higher education by storm, with its promise of closing equity gaps, raising student retention rates, and increasing tuition revenue by keeping students enrolled. Many colleges and universities have made an investment in predictive analytics for student success initiatives, and even more are looking into implementing, expanding, and strengthening the technology.²

While having clean data, accurate algorithms, and strong ethical principles are vital to putting predictive systems in place to improve student success, these are only a start toward creating the institutional change necessary to help more students graduate. An algorithm alone cannot create change; action is necessary. Getting advisers and other end users to communicate the predictive system findings to students is a vital step in successfully using predictive analytics and doing so equitably is of utmost importance.

This report offers research-based guidelines to colleges for engaging in effective, ethical, and equitable communication about predictive analytic system findings to students. It covers how to approach the first engagement with students: how an early alert end user, such as a counselor or adviser, can tell students that a problem has been identified, connect them with resources, and create the behavior change needed for success. It is also a guide for institutional leadership to consider when working with students, faculty, and staff to implement predictive analytics at their institution.

Communicating with students can seem easy, intuitive, and low-stakes, and an entire report on the subject may seem like overkill. But as the example above shows, having good intentions does not guarantee success, and ineffective communication can cause serious harm. Extensive behavioral science research shows that how institutions communicate to students really matters.³ This is especially the case with communicating predictive system findings, as these messages relay potentially sensitive information to students. Carefully communicating with students matters for the following reasons.

It matters for student success

How colleges communicate predictive findings to students can help or hinder their success. Communicating effectively can serve to minimize barriers between students and resources and help them follow through with their intentions and goals. Effective communication can also serve to strengthen students’ sense of belonging and confidence in their abilities, an important aspect of student success. Poor communication, however, is unlikely to spur change in a student, and may, in fact, impede success. For example, unclear communication can leave students wondering what they are supposed to do if they fail a course. And as the example above shows, a carelessly worded message can make students feel like they are not meant for college, and inadvertently push them to drop out.

Students can also react negatively to messages communicating difficult information. Shannon Brady's dissertation for Stanford's Graduate School of Education on how students react to probation letters gives us two such examples.⁴ Students are likely to have similar negative reactions to predictive system messages that are critical of their academic performance.

Image 2: Student reactions to probation letters

It matters for your investment⁵⁶

Predictive analytics can cost a lot of money and social or political capital, so appropriately and effectively communicating findings to students is essential to ensuring your institution’s investment pays off. Effective, ethical, and equitable communications will keep students in school, helping institutions reach their retention, graduation, and revenue goals. Careless or ill-prepared messages can lead to unmet goals and an inefficient use of the capital invested in predictive analytics.

It matters for educational equity

How predictive system findings are communicated to students has serious equity implications. Today’s students are more diverse than ever: 42 percent are students of color and 24 percent have children or other dependents.⁷ Their needs and challenges differ from traditional college students who are younger, tend to be wealthier, and can focus exclusively on their academics. Poor communication can exacerbate challenges that students face. For example, a message without clear instructions on how to make an advising appointment could confuse a first-generation student who needs an appointment urgently but does not know how to schedule one or have relatives and friends who can help. And a message that lacks a growth mindset could exacerbate a student of color’s already strong sense of lack of belonging. Communicating predictive findings without care can have unintended consequences that add to the challenges today’s students face and hurt their chances of graduating. Institutions must use messages about predictive system findings as an opportunity to refer students to resources and better support students of color, low-income students, and other under-resourced or non-traditional students on their path to graduation.

Predictive analytics is the use of mathematical models that use patterns in past data to predict or forecast future events. Colleges often use predictive analytics to help retain and graduate students. Institutions do this by developing models based on the demographic, behavioral, and/or academic data of past students to calculate a score for each current student, which represents that student’s likelihood of graduating or dropping out. This predictive score, or risk score, can change depending on student behaviors, actions, or circumstances. For example, if a student fails to meet with an advisor, his or her risk score rises.

These risk scores are generally not communicated to students. Instead, an end user of the predictive analytics system, like an academic adviser, has access to what is commonly known as an early alert dashboard, which helps the adviser see if a risk score changes significantly. End users typically send messages to students urging them to take actions that will lower their risk score and effectively improve their chances of graduating.

Communicating predictive system findings is especially tricky because it aims to inform students of a potentially unpleasant fact while simultaneously persuading them to change their behavior. However, using concepts from several subfields within behavioral science can help end users and institutions create effective messages.

Behavioral economics and social psychology provide the key concepts and techniques that shape these communications. Behavioral economics helps explain how individuals make choices, and thus how a message can be structured to cause behavior change.⁸ Social psychology studies how individuals behave in a social context, or the way certain factors affect our behavior in relation to others, and informs how individuals may interpret a message.⁹ Using behavioral economics and social psychology together ensures that messages are effective, ethical, and equitable and support students in their educational goals.

Behavioral economics is a relatively new field of study that blends economics and psychology.¹⁰ Contrary to traditional economics, which operates on the premise that humans are rational decision makers, behavioral economics argues that they are irrational beings that make decisions within their context and available resources.¹¹ Concepts such as loss aversion, decision fatigue, and heuristics all come from behavioral economics.¹²

Behavioral economics helps explain how people’s surroundings lead them to make decisions, and more importantly, how to help them make better ones. When it comes to communicating predictive findings, behavioral economics can inform how a message is structured to help students be successful. What follows are several behavioral economics concepts that can make a message more effective.

Social Proof

People are more likely to do things their peers are doing.¹³ For predictive analytics purposes, you may be more successful in persuading students to take a certain action, like going to the writing center or meeting with an adviser, if you let them know other students are doing it too. Sending a message that says “60 percent of your peers in your major attended a writing center workshop and found it helpful” could convince the recipient to do the same. The ongoing project, "Nudges to the Finish Line," provides several examples of messages that exemplify the use of this concept and others. See Images 3 and 4 for good examples of social proofing.

Image 3¹⁴

Image 4¹⁵

Loss Aversion

Studies of human behavior have shown that people prefer to protect what they have over potential gains. This concept is known as loss aversion.¹⁶ Take this scenario: when given the option of choosing between having (A) a 100 percent chance of winning $250 or (B) a 25 percent chance of winning $1,000, most people choose option A because they would rather be certain of winning a small amount than taking the risk of not winning anything.¹⁷

For predictive analytic purposes, sending messages that make students aware of what they may lose as a result of inaction is likely to encourage them to change their behavior. Telling students that not enrolling in classes on time could cost them a spot in required major courses may encourage them to act more quickly, for example. However, institutions must use these types of loss aversion strategies in an ethical manner. Employing this technique in a dishonest way may lead to undesirable consequences for students, such as undue anxiety and a lack of sense of belonging.¹⁸ Coupling loss aversion with inclusionary and positive language can help students be aware of what they may lose while making them feel like an important part of the institution. Images 5 and 6 show how to employ loss aversion ethically in your messages to students, while Image 7 shows a bad example.

Image 5¹⁹

Image 6²⁰

Image 7

Minimizing Hassle

Students have a lot on their plates, so an effective message will make a desired change in behavior as easy to accomplish as possible. A predictive analytics message should provide clear and simple instructions so that students know exactly what they have to do, how to do it, and can act immediately. If you want students to set up an appointment with their major adviser, for example, include a link to make an appointment or have them reply to the message with times they are available and send a calendar invitation. To encourage students to enroll in courses on time, include instructions on how to do so in the message. In other words, the easier the better. Image 8 is an example of a message that includes clear instructions, reduces the mental energy necessary to take action, and therefore minimizes hassle.

Another benefit of designing messages to minimize hassle for students is that it may force an institution to simplify institutional processes that are more complicated than they need to be. Messages with clear instructions coupled with more efficient institutional processes make it easier for students to take action.

Image 8²¹

Timing

In addition to minimizing hassles, sending messages in a timely manner will make them more effective and increase the likelihood that students engage in the desired behavior change. Messages sent during high-leverage junctures make it more likely that students will take action (see Image 9).²² Periods such as course registration, midterms, or the FAFSA deadline may be the optimal time to contact students. On the other hand, trying to communicate with students during a holiday break or summer vacation is unlikely to be effective.

Image 9²³

Preserving Choice

When communicating predictive system findings to students and trying to influence their behavior, it is essential to preserve student choice. In other words, you do not want students to take a certain action because you left no other option. An example of how this can happen is by making something dependent on the completion of another task, like enrolling full time in order to receive a tuition discount. Some students may not be able to enroll full time, and without other options, could choose to not enroll in the institution at all.

Another example might be when course enrollment is tied to FAFSA completion. While this may seem like a two-for-one deal for the institution (students file the FAFSA and enroll in classes on time), it may put students in a challenging situation if they are ineligible to file. It could also leave students out of their education entirely: if they do not file the FAFSA, they are not able to enroll in classes or receive financial aid. Messages that try to change behavior while limiting student choice, like in Image 10, are unethical and ineffective.

Image 10

Social psychology is the study of “the nature and causes of individual behavior in social situations” and includes concepts such as the sense of the self, social influence, group processes, prejudice, and discrimination.²⁴ It explores how humans think and behave in the context of their relationships and greater society.

As it relates to predictive analytics, social psychology helps us understand how the content of a message can affect the receivers’ self-conception and their relationship to those around them. Using social psychology to create messages ensures that communication is both effective and ethical. Implement or be aware of the following social psychology concepts when communicating predictive findings to students in order to minimize harm.

Growth Mindset

Studies have shown that believing that one’s abilities can change and improve with dedicated effort can improve performance, and that students with this kind of growth mindset perform better on a task than those with a fixed mindset (believing that talent and ability is innate).²⁵ Communicating predictive findings with a growth mindset sends a message to students that they have the ability to change their behaviors and improve their academic outcomes instead of being stuck in their current ways. As a result, students feel more positive about their potential despite their academic challenges.

Image 11

Image 12

Image 13

Interdependent Focus

Some studies suggest that whether messages are worded to highlight independence or communal thinking can affect how students respond. One experiment compared admission letters with either an independent or interdependent tone (see Image 13 for an independent message followed by an interdependent message). An independent tone emphasizes individualism and assertiveness, while an interdependent tone emphasizes groups and collaborative behavior. Researchers found that letters with an interdependent, or more group-oriented, tone led to more low-income student enrollment as compared to an independent letter that highlighted individualism.²⁶ The interdependent letter highlighted values of community and collaboration that aligned with students’ own. Similarly, communication with a greater focus on the “we” as opposed to “me” could make it more likely that students change their behavior, especially if they come from more interdependent communities, like low-income, first-generation, or immigrant backgrounds, because these messages remind them that they are part of the school community and will be supported.²⁷

Table 1: Independent vs. Interdependent Messages

Belonging

Humans have an inherent need to feel that they belong to a group. Lacking a sense of belonging has shown to negatively affect peoples’ performance, including students.²⁸ A study on probation letters sent to students, for instance, found that they were more likely to persist at their institution when letters showed care and concern and expressed that students were an important part of the institution’s community.²⁹ Students who received a probation letter that only described the procedural process of probation without expressing concern were more likely to feel like they did not belong at the institution and were less likely to persist and earn a degree.

Communicating concern over behavior or academic performance from predictive system findings, like a drop in grades that qualifies someone for probation, can cause students to question their place at the institution, if not done with care. Messages like that in Image 14 express care for students and their outcomes and could reinforce their sense of belonging.

Image 14³⁰

Stereotyping and Stereotype Threat

In communicating predictive system findings to students, it is important to not make assumptions about why someone is struggling in school or what obstacles they face based on assumed or known characteristics, such as race, gender, or parenting status. Even if shared with good intentions, assuming things about students' experience in college based on stereotypes, as in Image 15, is likely to be hurtful to students, damage your relationship with them, and could imperil their academic progress.

Image 15

Profiling and stereotyping students can lead to other unfortunate outcomes beyond damaging a sense of belonging, such as tracking students into certain majors or professions that pay less. When acting upon a predictive finding, an end user may feel inclined to help students succeed by tracking them into “easier” majors, like public health instead of pre-med. While there is nothing wrong with switching majors, it is problematic to advise an “easier” route without first providing support for students in their chosen major, especially if the student is underrepresented in that field.

Stereotyping can also lead to stereotype threat, the social phenomenon where individuals are or believe they are at risk of conforming to a stereotype about their social group.³¹ Stereotype threat can hinder an individual’s performance on a task. In various studies, individuals who took an exam and held a marginalized social identity (such as a woman taking a science exam with two men, or one person of color taking an exam with two white people) actually performed worse on the task because they were stressed about confirming a stereotype.³²

End users should be careful to avoid communicating bias and stereotypes to students when sending messages about predictive system findings, whether unintentionally or not. If students receive a message saying that they are unlikely to graduate because they are low-income or a student of color, this message could trigger a series of thoughts, behaviors, and events that leads to underperforming. Bias training and thorough research and testing of messages can help prevent this.

Self-Fulfilling Prophecy

Stereotyping can lead to a self-fulfilling prophecy, the social psychology phenomenon where an individual thinks something will happen and then acts in a way that makes that outcome occur.³³ Self-fulfilling prophecies can be self-imposed or imposed by others. In the context of higher education, students who learn they have a low predictive score may take that to mean they are not “college material” and, as a result, study less, perform poorly, and eventually drop out. Although a low predictive score does not explicitly state that a student is not college material, the student can come to understand that and then engage in counterproductive behaviors that make failure seem inevitable. Poorly communicated predictive findings can lead to the very behaviors you are trying to change or prevent.