Introduction

Colleges are increasingly using models to predict student behavior and intervene to change that behavior. Because of this, when projects involve partnering with a vendor, it is more important than ever to make the right choice about which vendor. In some ways, partnering with a vendor to use predictive analytics is similar to procuring any other technology product. But the complexity of the algorithms—and the predictions they produce—add another layer to the decision-making process. As we talked to a number of colleges, we heard from some whose vendors had algorithms that showed all of their students were “at risk.” And we heard from others who were told that student GPA was the most significant predictor of student success. Neither of these are useful outcomes or productive partnerships.

Currently, many colleges choose vendors based on word of mouth. Colleges should certainly talk to other schools about their vendor experiences and ask vendors for their customer retention rates. But in this era of big data, colleges have additional ethical considerations.

Ideally, predictive analytics vendors can facilitate the ethical use of data all of the way through the student life cycle. Vendors can help ensure that data are complete and integrated correctly to diminish the chances of misidentifying students. They can be transparent about their algorithms and test them for disparate impact on student populations. They can be flexible with permissions and use reasonable security protocols to help preserve student privacy and security. They can help evaluate interventions to ensure they are not harming certain groups of students. And they can train staff on the correct interpretation of data and on the dangers of implicit bias.

Not all vendors, however, take this approach. Many do the bare minimum with data integration. Some consider their models and algorithms proprietary. Some are not focused on student privacy and security. Others do not support training and evaluation.

This guide will give administrators the tools to ask the right set of questions of predictive analytics vendors and provide a sense of what kind of answers they should expect. It will focus on ensuring that vendors use predictive analytics tools like early alert systems in an ethical manner.

Partnering with a vendor will not solve all problems around data and analytics. Instead, colleges must first determine what they want out of these tools, put the capacity in place to maintain and act on the information these tools provide, and get buy-in from key members of the school community for using these tools. In 2017, we published Predictive Analytics in Higher Education:, a guide for colleges looking to properly implement these tools with ethics in mind.¹ Before colleges think about partnering with a vendor, we recommend reading our guide, as well as exploring other useful resources, such as the paper Putting Data to Work, the book Creating a Data-Informed Culture in Community Colleges, the Data Maturity Framework from the University of Chicago Center for Data Science and Public Policy, and The Ada Center, which is dedicated to helping community colleges implement technology solutions.²

In the end, no vendor is going to do the hard work for your college. Many of the stories we have heard about unhappy partnerships between colleges and vendors come from misunderstanding what vendors and their products can unilaterally change on campus.