Establish a Tiered Evidence Fund for Innovation and Expansion

Authored by Roger Low and Karen Quarles, America Forward, and Kate Tromble, Results for America

Today’s postsecondary students are as varied as our nation, encompassing individuals just out of high school and older, working (full or part time) adults. Thirty-one percent are at or below the poverty line, only 46 percent live on campus, and nearly 30 percent have children of their own. Furthermore, these students are increasingly likely to come from Black and brown communities, and often face systemic barriers that make it harder to complete their postsecondary degrees, compared with many of their peers.

Each year 1.2 million low-income or first-generation students enroll in college, but only a quarter of them will leave with a job related to their studies or enter graduate school. Children who have never lived in poverty are ten times more likely to complete a college degree by age 25, compared with children who grow up persistently poor. Even at the most affordable community colleges, only about one-third of students graduate within one-and-a-half times the length of the program.

Well before the COVID-19 crisis brought about new challenges for students, it was clear that empowering students to succeed in postsecondary education and beyond was critical to increasing equitable economic mobility. With the spread of COVID-19, and the disparate impacts it has wrought on Black, brown and low-income communities across the country, the need for reform is more urgent than ever.

If we want to make higher education truly an engine of opportunity for low-income students and students of color¹ we need to ensure that colleges across the country are adopting evidence-based strategies to improve results. This requires multiple strategies. We need institutions to come up with new interventions and supports, and evaluate them to see if they work. Ideas with some evidentiary base should be evaluated in multiple contexts and settings to ensure they work in different environments. And those ideas that have been proven time and again to work need a path to adoption on a national scale.

The Department of Education can play a key role in facilitating the development and spread of innovative practices by adopting a tiered grant competition. In 2009, the American Recovery and Reinvestment Act (ARRA) was enacted to address the economic fallout from the Great Recession. The law went further though, encouraging and supporting innovation in education through the evidence-based Investing in Innovation (I3) Fund and its successor, the Education, Innovation and Research (EIR) program at the same time it addressed immediate economic needs.² Several years later, the First in the World (FITW) program was created with the goal of developing, replicating, and disseminating innovative solutions and evidence for what works in addressing persistent and widespread challenges in postsecondary education for vulnerable students. Given the burden the current pandemic is placing on our Black, brown and low-income communities in particular, the next Administration should build on these programs by prioritizing the proposed college completion grants, based on levels of evidence, in its first budget proposal.³

In recent years, a handful of interventions or approaches in the higher education domain have been rigorously evaluated, replicated, and shown to be effective at measurably improving student postsecondary degree attainment, particularly for low-income students and students of color. For instance:

• City University of New York (CUNY) Accelerated Study in Associate Programs (ASAP): The intervention is “a comprehensive set of supports that include financial assistance, advising and career counseling, tutoring, training in study skills, and reformed remedial courses,” aimed at first-time students, students of color, students in or near poverty, and students with high risk factors. A rigorous 2015 MDRC evaluation found that within three years, ASAP nearly doubled graduation rates, from 22 percent to 40 percent. The program was expanded to community colleges in Ohio; a second randomized evaluation from Ohio, equally rigorous, released in early 2020, shows equally positive results, with robust and comparable effects across white students and students of color.
• Bottom Line: Like CUNY ASAP, Bottom Line provides intensive, individualized guidance and support for low-income, first-generation students. Results for America found that a well-conducted randomized trial showed that students offered Bottom Line advising were substantially more likely to be enrolled in college at the three-year follow up with robust and comparable effects across white students and students of color.
• Project Quest: QUEST provides comprehensive supports for low-income individuals to enroll “full-time in occupational training programs at local community colleges, complete the training, pass certification exams, and enter well-paying careers in high-growth sectors of the local economy” (such as for instance the healthcare sector). This program also requires full-time enrollment, and includes “weekly group or individual counseling sessions, financial assistance for tuition and other school-related expenses, and remedial instruction for those requiring it.” A rigorous randomized trial found Quest “increased average annual earnings by $5,490, or 20 percent, nine years after program entry.”
• I-Best and Other Classroom-Based Interventions: The Washington State Integrated Basic Education and Skills Training (I-BEST) program “aims to increase access to and completion of college-level occupational training in a variety of in-demand occupational areas. Its signature feature is team teaching by a basic skills instructor and an occupational instructor during at least 50 percent of occupational training class time.” A rigorous randomized evaluation found that the I-BEST programs at the three technical and community colleges “increased participation in college level courses, number of credits earned and credential attainment.” In addition, certain other remedial education programs, such as CUNY Start’s remedial program, successfully increased student passage of gateway English and math courses in their first two years, according to a study with a matched comparison group.
• Additional Promising Programs: These include the Valley Initiative for Development and Advancement (VIDA), which combines career counseling, case management, and soft skills training; Detroit Promise Path, which combines a last dollar scholarship and advising; and Stay the Course, which combines comprehensive case management with emergency financial assistance. These programs take different but aligned approaches to ASAP. Notably, they also require full-time enrollment. VIDA and DPP have shown preliminary but promising results.

Other Evidence-Building And Evidence-Scaling Efforts: A number of other interventions and organizations offer a related comprehensive set of holistic supports for higher education students, including Braven, College Forward, One Million Degrees, uAspire, Year Up, Health Careers for All, Bridge to Employment in Healthcare, and Single Stop. These programs are in varying stages of the evidence-building continuum from some promising programs that have not yet been rigorously evaluated, to strong early evidence (for instance, One Million Degrees) to established strong evidence (for instance, Year Up).

There are no guarantees that interventions will succeed when it comes to something as complex as changing the trajectory of a student’s life. Different communities present different challenges. Demonstrating outcomes over one or even multiple evaluations is no guarantee of future success. Implementation matters. Community buy-in matters. Leadership and personnel matter. Any number of variables can complicate the scaling of even the most rigorously evaluated approaches — inadequate funds, fragmentation of the social innovation “ecosystem,” or gaps in the talent needed to scale, to name just a few.

With all of these caveats, the approaches summarized above, while intensive and expensive on the front end (ranging from $700 to $16,000 per participant per year), have demonstrated (or are in the process of seeking to demonstrate) a substantial return on investment (ROI) to the public sector and society in the long run. They offer hope that we can empower many more individuals to graduate from college and achieve greater economic security. And yet, although a handful of public and community colleges have funded these interventions, with impressive results, the vast majority of jurisdictions are not deploying public dollars to expand the supports identified by rigorous research as most effective at helping students with the highest barriers to successfully reach the collegiate finish line.

In a 2019 report, New America observed: “the research on what works to improve students’ odds of success in higher education remains limited. Only a handful of organizations and researchers even conduct such evaluations… and there has been a lack of coordination by the federal government to solve the research problems or tackle the challenges of increasing college completion rates and other problems. Moreover, even when promising interventions have been identified, colleges have few incentives to apply that research. Many of these interventions require significant upfront costs, and the benefits and savings to campuses, which are often realized through increased retention and graduation rates, may take a long time to achieve. Federal policy requires only that colleges avoid extremely high default rates and comply with federal rules and regulations, few of which are based on outcomes. Federal dollars to support improvement and quality assurance are limited… As a result, practitioners often lack the resources—or incentive—they need to implement such practices.”

Finally, as noted, today’s students are not the “traditional” college students that label calls to mind. They hold jobs, live off campus, face acute economic, housing, and transportation challenges, and have young children of their own. These students are also disproportionately not white, and they urgently need additional support. Yet, we have remarkably little rigorous evidence on approaches that successfully move the needle on degree attainment outcomes for part-time students.

• A revised and expanded version of the FINISH Act fund located in an Office of Evidence-Based Practices, a competitive evidence-based innovation and replication grant program focused on increasing completion rates. Based at the U.S. Department of Education, the fund should start with an $800 million federal investment and be overseen by an Office of Evidence-Based Practices. The Department currently maintains an evidence-based practices team within the Office of Elementary and Secondary Education. That team should be converted into an office serving both K-12 and postsecondary education and reporting directly to the Deputy Secretary.
• Tap the right talent: In addition, we urge the office to assemble a mix of permanent experienced full-time staff, supplemented with targeted outside experts in the evaluation, evidence, higher education and social innovation fields. The Department could draw this talent into the newly constituted office via a combination of direct hire authorities and extended fellowships, including Intergovernmental Personnel Act (IPA) partnerships, leveraging authorities available to the Secretary under current law (additional information on these authorities is available here).
• A well-structured fund: Theoretically, this fund could be placed within the Fund for the Improvement of Postsecondary Education (FIPSE), following the model of the FITW program. Our concern with adding dollars to FIPSE or the pre-existing FITW fund is three-fold:
  • First, those programs lack the intensive focus on college completion that we envision.
  • Second, they funded one-off, individual projects at institutions and nonprofit organizations, creating a myriad of pilots. More impact can be achieved by funding states working across their institutions in coordination and collaboration with nonprofit partners.
  • Third, and perhaps most importantly, in some respects, FITW may have been ahead of its time. At the time FITW grants were awarded, there simply were not many interventions in the higher education space that had convincingly reached a top evidence tier. While FITW helped spur innovation, in hindsight, its grant awards arguably were inadequate and insufficiently targeted toward moving the most promising evidence-based programs further along an evidence continuum. Since the FITW grant awards, a range of compelling higher education interventions summarized above have made exciting strides, with the release of a new tranche of randomized and quasi-experimental studies. We believe the field has advanced rapidly in the last six years, and the time is now ripe for a major federal investment in expanding these programs, and building evidence for a new cohort of innovative approaches, leveraging the important lessons learned. In short, we think starting fresh, given how much the field has advanced and how much we have learned from evidence-based programs over the last decade, will offer a better chance for success.
• Adequate funding: FITW received less than $150 million over its two years of funding. The Social Innovation Fund (SIF), which helped launch compelling community-based solutions in over 100 cities in 33 states and the District of Columbia, received approximately $400 million in federal investment. When combined with philanthropic funds and other dollars, SIF reached an impact of over $1 billion. We believe SIF’s total funding level offers more potential for success. And, although we think that level can eventually be achieved through a matching requirement, we believe additional investment is needed initially to achieve sufficient scale given the financial strain many states and institutions are experiencing currently and the scale of the problem the fund seeks to address. Grants would be awarded to states or state-entities in partnership with their institutions and nonprofits operating evidence-based programs, interventions or policies that increase completion rates for at-risk students such as counseling, mentoring, financial assistance, and other supports. As a tiered evidence fund, grants would be awarded in ranges based upon the amount and quality of causal evidence underlying the state’s proposed plan. Grants at the highest, or “strong evidence,”⁴ tier would be reserved for states proposing to scale an existing program that has met the What Works Clearinghouse’s (WWC) highest level of effectiveness rating. Programs would be required to submit their randomized or quasi-experimental evaluation evidence to the Institute of Education Sciences (IES), and for IES to review those evaluations within the first, planning year. Grants at the lowest, or “preliminary evidence,”⁵ tier would be for states proposing a program and strategic plan that would build evidence around how to increase completion for at-risk students. These grants would have to be accompanied by a plan for evaluating, in a rigorous manner, the impact of the proposed program, practice intervention or policy. The middle, or “promising evidence,”⁶ tier would be for states proposing a plan to replicate an existing strategy that has shown positive causal impacts, but only has short-term, non-experimental results, or has not yet been replicated.
• Planning Grants. Planning grants would be awarded in the range of $300,000 for a 12-month period to any state or state-entity interested in applying for a grant from the fund. Following a streamlined, rapid review and certification process by the U.S. Department of Education, any state or state-entity interested in submitting a competitive application would be eligible to automatically receive a planning grant, ensuring every state would have the resources, staff and capacity necessary to formulate a catalytic plan to build evidence for and scale effective higher education approaches. Recipients would develop a plan for either scaling an existing, WWC-reviewed, evidence-based approach or for building evidence of a new approach to increasing college completion; States could also elect to pursue a hybrid approach with a plan that did some of both. Plans would need to identify, among other things, the state’s plan for distributing funds among its institutions to focus on the most at-risk students, its evidence-based intervention or program, its technical assistance/ implementation partner, its goals for increased completion, and its plan for tracking and publicly reporting its performance against those goals.

• Focus on those most in need. State plans should specifically focus – as does the current FINISH Act – on first-generation students, students of color, and students in or near poverty, as well as students with high risk factors for not finishing college, such as those with children of their own and non-dependent, SNAP eligible students. To that end, state plans should prioritize partnerships with institutions that serve these students, namely community colleges, regional public universities, and minority-serving institutions. States that successfully prioritize these students, communities and institutions should receive favorable consideration in the competitive application process. Moreover, for grants at the early-phase or evidence-building tier, priority should be given to proposals to build evidence around increasing completion rates for part-time students with other vulnerability factors.
• Matching funds: Grantees also would need to provide matching dollars, which could be a combination of state, institution, and philanthropic dollars. States should be eligible for a waiver from the match requirement, through 2022 due to the current unfolding recession, with modifications as needed depending on the fiscal realities states face in 2022 and beyond.
• Funding for evaluation, performance management and capacity building: Costs associated with evidence-building and high-fidelity implementation of evidence-based programs (such as data system improvements, performance management improvements, technical assistance, evaluations and other training and capacity-building efforts) should be allowable uses of up to 10 percent of the full grant. (In the case of the planning grant, states should have maximal flexibility to use grant dollars for staff, planning, and capacity-building as needed to develop a coherent plan and application.)
• Deadlines for releasing guidance and selecting awards: So that the U.S. Department of Education prioritizes this initiative, legislation should include deadlines for the Department to release guidance, a Notice of Funding for the competitive grant process, and select awards. If the Department misses any of these deadlines by more than 60 days, the legislation should require an investigation by the Office of the Inspector General into the cause of the delay.

Prioritize an expanded FINISH Act in 117th Congress

In addition, the new Administration should work closely with the bipartisan FINISH Act sponsors, as well as additional, engaged members of both parties (in particular, members of the relevant education and appropriations Committees in the House and Senate) to introduce an expanded and retooled FINISH Act in 2021 that creates the evidence-based college completion fund proposed above.

In 2019, Senators Todd Young (R-IN), Michael Bennet (D-CO), Tim Scott (R-SC) in the Senate and Representatives Ben McAdams (D-UT) and Ron Wright (R-TX) in the House introduced the Fund for Innovation and Success in Higher Education Act (FINISH Act), which included a new tiered-evidence fund to improve graduation rates and boost the attainment of postsecondary credentials for today’s students. These members can be tapped as possible champions for the legislation.

The FINISH Act would amend the Fund for the Improvement of Secondary Education (FIPSE) within the Higher Education Act (HEA) to “allow grant recipients to use a portion of their funds for tiered, evidence-based grants to support initiatives that increase access to higher education for high-need students, increase degree attainment, and improve efficiency in our higher education systems.”