Goals and Strategies for Improving the Programs

To improve the success of any of these efforts—college access programs like TRIO and GEAR UP, the minority-serving institution grant programs, or even a competition like First in the World—the programs themselves must begin to change and adapt.

First, and most importantly, both Congress and the education secretary must make gathering and using evidence a priority across all Department programs. Grantees will not adopt evidence-based practices on their own, without incentives to do so. Nor will many of them even be clear on how to interpret research, carefully and faithfully implement it on their campuses, or evaluate how the practices worked in their contexts. The Department must promote measurement, evaluation, and continuous improvement by its grantees; provide support for practitioners in understanding and implementing research-based practices; and require that taxpayer dollars be spent to further encourage those practices.

Second, the goals of these programs—particularly the grants to minority-serving institutions, where the uses of federal dollars are so sprawling—should be refocused on improving student success. A grant program that allows for over a dozen different uses is likely to lead colleges to pursue self-serving interests over ones that are most beneficial to their students, or to continue using the funds exactly as they always have. If lawmakers want to spend money to improve student success, they must make their expectations clear that grantees should take proven and promising approaches to better student outcomes.

Finally, the programs must ensure a feedback loop, a way to keep trying new things and building the body of evidence, informing campus leaders about successes and failures, and driving the widespread adoption of policies that work for students. That includes a commitment by the Department’s research arm to gathering and assessing more evidence about what works and what does not in higher education, and to communicating that information to both policymakers and practitioners in order to change practices.

First, and most importantly, both Congress and the education secretary must make gathering and using evidence a priority across all Department programs.

Fortunately, there are plenty of opportunities within the law as it exists to improve these programs and expand the use of evidence to promote students’ enrollment in and completion of college, particularly through a greater focus on student interventions. Even better, policymakers at the Education Department need not wait for Congress to reauthorize the Higher Education Act before implementing some of the easiest suggestions here. Still, HEA reauthorization presents a once-in-a-decade opportunity to reorient these programs to ensure they serve students, not grantees, and lawmakers should be careful not to let that chance pass them by.

Continue to promote evidence-based practices: The Education Department has made significant progress in its efforts to encourage the use of evidence-based practices, incorporating priorities for research-backed interventions into many of its competitive grant competitions in higher education and PreK–12 programs in recent years. It should continue these efforts, asking that grantees use promising practices and evaluate their impact and assessing when to raise expectations for the level of rigor it expects grantees to meet based on the available research.

Additionally, the Education Department should consider how to emphasize evidence-based policymaking in the Secretary’s supplemental priorities, a grab-bag of incentives the Department can incorporate into future grant competitions. A version of the priorities released for public comment in 2017, for instance, included one for “promoting innovation and efficiency…with an increased focus on improving student outcomes.”¹ While the supplemental priorities are far from the only opportunity to incorporate evidence into higher education programs, this was an unfortunate missed opportunity to emphasize the use of evidence-based strategies and efforts to measure grantees’ success in ways that can contribute to the body of research on higher education efficiency. Future iterations of the supplemental priorities, as well as regulatory changes to the TRIO, GEAR UP, and/or MSI programs, should seek to signal the Department’s commitment to having colleges use proven and promising practices to help students succeed.

Evaluate programs and practices supported under the TRIO and GEAR UP programs: The Education Department has not comprehensively evaluated the TRIO programs in recent years, presumably due to the ban on conducting randomized controlled trials within these programs. However, the agency should consider the scope of the statutory ban and continue to support allowable research. For instance, it could have evaluators identify and assess the effectiveness of some of the most common practices of TRIO grantees, to determine whether and how particular interventions improved students’ outcomes. It did so recently, with a study of college advising in Upward Bound that revealed that using “Find the Fit” strategies with students increased the number and selectiveness of colleges to which students applied.² Or the Department could provide priorities for grantees willing to conduct rigorous assessments of their own programs. Even if the results are limited, conducting these kinds of evaluations may yield information useful to grantees and help to build support among lawmakers and key stakeholders for broader evaluations of the programs.

The Education Department should ask that grantees use promising practices and evaluate their impact.

Similarly, the Department should explore what types of evaluations are possible and useful for improving the GEAR UP program given that it is not subject to the prohibition on randomized controlled trials. While crafting large-scale evaluations has been a challenge in the past, the efforts and costs of conducting them are justified considering the stakes for the low-income students the program primarily serves. Supporting research on GEAR UP’s effectiveness is especially important given that Education Secretary Betsy DeVos cited the lack of evidence about the program’s effectiveness when she proposed eliminating the program and redirecting its funding into a new state formula grant program that would consolidate TRIO and GEAR UP funds to support college access activities. Before recommending such drastic change, the Department would be wise to determine whether or not GEAR UP works.

Improve data collection of higher education programs: The Department should ask more of grantees receiving taxpayer dollars by reevaluating its current performance report requirements and requesting additional data where needed to understand participants’ outcomes. TRIO is a model for high-quality data: the Department already makes available extensive grantee-level information on students’ outcomes within these programs. Meanwhile, there is little information available about how—and how much—minority-serving institutions are spending on various activities within their grants. The Department should audit its available data across all of its discretionary grant programs and consider what information it could make public prior to an HEA reauthorization that might inform lawmakers about necessary reforms—and what data it should begin to collect to improve its information.

Clearly publish and disseminate research relevant to grantees: The Education Department’s Institute of Education Sciences (IES) maintains the What Works Clearinghouse, which includes a list of available studies that have been reviewed according to IES evidence standards and assessed for their impact in improving student outcomes. However, the postsecondary research in that database is woefully inadequate and leaves out some of the research that exists in higher education improvement. IES should make a concerted effort to add more third-party studies and available research into the What Works Clearinghouse; and to clearly communicate to researchers how they can design studies eligible for inclusion, to ensure more qualifying research is conducted in the first place. And Department staff should drive grantees to those resources or make other resources (like brief summaries of research studies or information in the grant applications about the types of research that might meet evidence priorities, some of which it already produces) readily available and widely disseminated. Another resource the Department can access to help reach more K–12-based grantees is the Regional Education Laboratories, which conduct research and have connections with grantees and institutions across the country.

Consider large-scale reform to improve the efficacy of these programs: Hundreds of grantees around the country are engaged in distinct projects that are all working toward the same goal. As more research becomes available about the efficacy of certain types of interventions, models for reaching students and ensuring they feel connected, and best practices for improving college access and success, lawmakers may wish to rethink–without cutting spending on critical national efforts–how Congress finances and disseminates funds for these programs. A new version of these programs could narrow the scope of activities to those that are most effective when the evidence is robust or encourage the use of proven practices when the knowledge base is less developed. Congress should also encourage innovation to continue identifying new promising practices and the most cost-effective ways to ensure that the students most in need of additional resources get those supports. And lawmakers could award financial support to colleges or networks of colleges seeking to institutionalize the use of evidence by allowing funds to be used to identify evaluators, build data capacity, and build up institutional research staff.

However, based on the information available at this time, it is difficult to say exactly what that type of reform should look like. Greater investigation is needed, and any reforms should be informed by the types of research into the effectiveness of federally funded programs discussed throughout this paper. Congress should begin to lay the groundwork for a more robust public discussion with the other recommendations in this report.

Eliminate prior experience points: Lawmakers should do away with the antiquated and unfair system of prior experience points. The prior experience points in the TRIO program run counter to the very notion of evidence-based policymaking. A prospective grantee with a great idea backed by evidence stands little chance of winning a grant against a current grantee that gets a 15-point boost (on a 100-point scale)—more than double the point bonus used for evidence in the Department’s 2015 TRIO competition—simply for already having a grant.³

Congress should consider ways to ensure evidence-based practices are not just a priority, but the central priority.

Direct the Education Department to evaluate the effectiveness of the programs it supports: The Education Department is banned under the TRIO programs, and sometimes constrained by finances, from evaluating the effectiveness of the discretionary grant programs it supports with the most appropriate methods available. As a result, policymakers often are in the dark about what works, and grantees are unsure how they need to improve their projects. Congress should repeal the ban on conducting randomized controlled trials in the TRIO programs and direct the Department to conduct large-scale evaluations of both TRIO and GEAR UP.

Require the use of evidence in existing programs: Congress should consider ways to ensure evidence-based practices are not just a priority, but the central priority. The Department has already laid out a promising framework for incorporating evidence into existing programs and should continue to do so. However, assigning a one- or two-point priority for applicants proposing to conduct an evidence-based activity has led some applicants to propose the bare minimum, rather than taking action that would have a greater impact. For instance, the Department could require that a certain percentage of the grant dollars be reserved for the evidence-based activities the grantee proposes to incorporate. It could offer a more generous funding level for grantees that engage in the interventions with the biggest effects (typically more expensive interventions to conduct). Or it could establish a baseline share of grant spending each grantee must use on evidence-based activities.

Additionally, lawmakers should help facilitate a culture change in how grantees view these programs by rewriting the programs' purposes to emphasize the centrality of evidence-based practices. Doing so might have only modest effects, but will signal to the Education Department and grantees that Congress not only allows but expects experimentation will be a part of the program.

Increase available funding for evaluation: For a rigorous evaluation that yields usable and useful results, the Department will require funding. Those dollars can ensure that it provides grantees with technical assistance, gathers necessary data, and involves high-quality researchers. Congress should prioritize the need for such evaluations by appropriating additional funding that is earmarked for rigorous evaluations at the Institute for Education Sciences and/or under the Government Performance and Results Act (GPRA). GPRA provided hundreds of thousands of dollars each year for data collection and program evaluation in higher education until lawmakers defunded its budget completely in fiscal year 2015.⁴ Congress could also set aside a portion of funds for programs like the Titles III and V minority-serving institutions programs so that the Department uses that set-aside for evaluation purposes. Increased funds for measuring impact would complement the evaluation efforts already required by the Foundations for Evidence-Based Policymaking Act that became law earlier this year.⁵

Grant the Education Department pooled evaluation authority for its higher education dollars: As with the Education Department authority for K–12 programs established by the Every Student Succeeds Act, Congress should allow the Department to set aside a small percentage of program dollars to build up the necessary funding to evaluate its higher-education programs. Pooled evaluation authority is especially important if Congress fails to provide additional funding for evaluation through other sources and programs, because it will allow the Department to prioritize certain programs for evaluation and borrow from other funding streams to afford that research.

Improve the quality of higher education data: Congress should pass a law establishing a secure, privacy-protected student-level data network at the Education Department to improve the quality of information available to and from colleges about their outcomes. Evidence-based policymaking requires better data to understand where and with whom the greatest need exists. Data also help to benchmark the goals for grantees that are spending federal dollars and to measure improvement and progress. Yet Congress has explicitly banned the Education Department from collecting information that could fill significant gaps in federal data sources. Lawmakers must overturn that ban and direct the Department to collect and provide key information about college performance to institutions and the public. Growing momentum behind the College Transparency Act, which would direct the Department to create a student-level data network to answer critical questions about higher education, is indicative of the broad constituencies—including policymakers from across the political spectrum—clamoring for this information.

Evidence-based policymaking requires better data to understand where and with whom the greatest need exists.

Launch a new program that will continue building the body of evidence: Congress should commit to innovation and experimentation in higher education with a tiered evidence competition that strives to improve students’ chances of success in college with grants to institutions to test, replicate, and expand the most promising practices. There is a thirst among some colleges to experiment with promising new practices, and a commitment among others to evaluate what works and to expand the use of such practices. The First in the World competition may only have run for a few years, but the demand was great. In 2014, for instance, the Department received 459 eligible applications, but could fund only 24 of them using the $75 million Congress made available for the competition.⁶

Congress should capitalize on that demand and continue promoting evidence-based practices at more colleges by establishing a grant competition to help finance the interventions, like the Innovation Grants program in the FINISH Act would. Tiered-evidence programs, like the Education Innovation and Research (EIR) program established through the Every Student Succeeds Act in 2015 or the Maternal, Infant, and Early Childhood Home Visiting Program operated by the Department of Health and Human Services, can create much-needed momentum for the adoption of evidence-based and promising strategies. As Brookings Institution researchers wrote in a 2016 report, “applicants to traditional grant programs often lack incentives to identify and use approaches backed by strong evidence,” nor do they have much assistance in evaluating their programs or engaging in innovative and understudied efforts.⁷ A tiered-evidence grant solves for those problems and helps ensure that taxpayer dollars are spent as carefully as possible.