Searching for Accountability in Higher Education: A Balanced Framework of Goals & Metrics
Table of Contents
Abstract
This paper provides a brief overview of the current state of federal accountability and the need for change, and a summary of the landscape of state accountability reforms through funding structures for public colleges and universities. It explores key questions or decision points that must be addressed in designing a federal accountability system and provides recommendations for each, in an attempt to form a set of guiding principles for thinking about the system’s objectives. And it identifies recommendations both for what needs to be measured and how, with a focus on the goals of completion, equity, and safeguards against risk to borrowers and taxpayers. Overall, the paper argues for a combined system of consequential and report-card accountability, with the former including both bright lines for eligibility and graduated sanctions and rewards to drive continuous improvement.
Acknowledgments
New America would like to thank Arnold Ventures for its generous support of this project. The views expressed in this report are those of its author and do not necessarily represent the views of the foundation, their officers, or employees.
The author would also like to thank Clare McCann of New America for her thoughtful comments and suggestions.
Downloads
Introduction
Driven by concerns over cost, quality, and overall value, accountability in higher education is an area of great and growing interest among policymakers and stakeholders, but also of considerable ambiguity and debate. While featured prominently on the agenda for the upcoming reauthorization of the Higher Education Act (HEA), the lack of consensus on an appropriate accountability framework and associated metrics to assess the performance and value of colleges and universities has hindered progress. Despite numerous Congressional hearings and bills; research and policy papers; proposals; conferences; and convenings, the conversation has stalled in the face of considerable complexity, inadequate and imperfect data, the specter of intended and unintended consequences, and a period of retrenchment of the federal role in education.
Nevertheless, the conversation has moved from whether more accountability is needed to how to do it. The purpose of this paper is not to add to the chorus of calls for greater accountability, the need for which has been clearly established and endorsed by both sides of the aisle. Rather, it is to make a contribution to ongoing efforts to break through the logjam by suggesting a path forward for designing an effective federal higher education accountability system that is practically feasible and politically viable, to the extent that officially stated objectives and commitments to doing a much better job protecting student and taxpayer investments in higher education are genuine.
First, the paper provides a brief overview of the current state of federal accountability and the need for change. This is followed by a summary of the landscape of state accountability reforms through funding structures for public colleges and universities. The paper then examines key questions or decision points that must be addressed in designing a federal accountability system and provides recommendations for each, in an attempt to form a set of guiding principles for thinking about the system’s objectives. The final sections provide a set of recommendations for both what needs to be measured and how, with a focus on the goals of completion, equity, and safeguards against risk to borrowers and taxpayers. Overall, the paper argues for a combined system of consequential and report-card accountability, with the former including both bright lines for eligibility and graduated sanctions and rewards for driving continuous improvement.
Asleep at the Wheel: Federal Accountability
The current federal accountability system in higher education is widely recognized as being inadequate and ineffective, despite the massive federal investment of over $150 billion annually.1
Both macro-level outcomes, such as overall poor graduation rates and student loan repayment troubles, and micro-level outcomes, such as individual institutions with dismal educational track records, point to a systemic failure to hold colleges and universities accountable for their performance and to ensure minimum levels of quality throughout the system. The limited set of accountability provisions and metrics currently in the HEA (see Table 1) are wholly insufficient to meet the demands for better student outcomes and greater effectiveness and efficiency for federal student aid programs.
In terms of federal accountability mechanisms2 and metrics that apply to all institutions participating in Title IV programs, the cohort default rate (CDR) and Heightened Cash Monitoring (HCM), only the former serves as a proxy for student outcomes and overall educational value, while the latter focuses primarily on the financial responsibility composite scores and compliance issues. The outlived usefulness of the CDR, mostly as a result of gaming and manipulation by institutions and the explosion of income-driven repayment programs that helps struggling borrowers stay out of default, has been examined thoroughly and today there is broad consensus that it needs to be either replaced or reformed. The limited impact of both measures is clearly demonstrated by the number of schools affected: In the most recent period, only 12 schools were subject to sanctions under the CDR and only about 63 were placed on the highest Heightened Cash Monitoring 2 (HCM2) list, which involves significant restrictions, out of approximately 5,750 Title IV schools.
The remaining accountability mechanisms and metrics are similarly either weak or of limited scope. The 85-15 rule for for-profit school revenue was weakened to 90-10 and excludes significant sources of non-Title IV student aid, such as the GI Bill for veterans and Tuition Assistance dollars for servicemembers, while the Gainful Employment (GE) regulations for career-programs were recently rescinded. The financial responsibility composite scores for private schools, in need of a significant update, have been consistently inconsistent in identifying at-risk schools, including due to manipulation. Overall, the picture is bleak for accountability in higher education at a time when more than two out of five students fail to graduate within six years after enrolling in either a two- or four-year college3 and student loan debt has hit a historic record, at $1.5 trillion.4 As more than 1,200 campuses have shuttered their doors in the past five years, of which 88 percent were for-profit colleges,5 including national chains that collapsed amidst scandal and shockingly poor student outcomes, the growing calls for accountability are not surprising.
Table 1: Current Federal Accountability Provisions & Metrics
| Provision/Metric | Institutional Applicability | Legal Basis | Challenge |
|---|---|---|---|
| Cohort Default Rate | All Institutions | Statutory | Ineffective, gameable, increasingly irrelevant |
| Heightened Cash Monitoring | All Institutions | Statutory | Limited scope, outdated |
| 90-10 Revenue Percentages | For-Profit Institutions | Statutory | Does not include all federal aid; threshold too high |
| Financial Responsibility Composite Scores | Nonprofit & For-Profit Institutions | Statutory | Limited scope, outdated |
| Gainful Employment Debt-to-Earnings Rates | All Programs (For-Profit Institutions) Non-Degree Programs (Public & Nonprofit Institutions) |
Regulatory | Repealed |
Citations
- College Board (website), “Trends in Student Aid: Student Aid and Nonfederal Loans in Current Dollars over Time,” Table 2 (Washington, DC: The College Board, 2018), source
- This paper does not discuss the accreditation and state authorization components of the higher education triad. Its exclusive focus is on the accountability role of the U.S. Department of Education.
- Doug Shapiro, Afet Dundar, Faye Huie, Phoebe Khasiala Wakhungu, Ayesha Bhimdiwala, and Sean Eric Wilson, Completing College: A National View of Student Completion Rates, Signature Report No. 16 (Herndon, VA: National Student Clearinghouse Research Center, 2018), source
- Jillian Berman, “Student Debt Just Hit $1.5 Trillion,” MarketWatch, May 8, 2018, source
- Michael Vasquez and Dan Bauman, “How America’s College-Closure Crisis Leaves Families Devastated,” Chronicle of Higher Education, April 4, 2019, source
Test Vehicles for Accountability: State Developments
However, while the federal government failed to boost accountability, state governments have embarked on an ambitious, albeit controversial, effort to hold public institutions accountable through performance- or outcomes-based funding (OBF) mechanisms. At least 29 states had such policies in place last year, with several more actively considering, transitioning to, or adopting this approach.6 This is significant in light of the fact that almost three out of four students (73 percent) attend public institutions.7 While there is considerable debate about the impact of such efforts, both intended and unintended, their adoption signals the recognition for greater accountability at the state level. Moreover, given the historical role of states in education policy as laboratories of reform and innovation, this trend merits attention and could help contribute to accountability at the federal level by yielding insights and suggestions about the priorities and overall direction of an accountability system.
The growth of such policies reflects the ongoing search for higher education outcomes-driven accountability at the state level. In 2010, the National Governors Association (NGA) and Complete College America (CCA) announced a core set of metrics of college completion, including progress and outcomes data (see Table 2), which expanded on measures already required by the U.S. Department of Education for financial aid eligibility.8 The metrics are now reflected in the accountability reporting systems of 33 states9 and NGA recommended that states disaggregate performance data according to academic preparation, income, age, and race/ethnicity.
Table 2: Governors’ Task Force on College Completion (NGA/CCA) Metrics (2011)
| Progress Metrics | Outcome Metrics |
|---|---|
| Enrollment and success in remedial education programs | Degrees awarded (annual) |
| Success in first-year college courses (English and math) | Graduation rates |
| Credit accumulation | Transfer rates |
| Retention rates | Time and credits toward degrees |
| Course completion |
The states’ focus on completion is certainly reflected in OBF policies. Two recent analyses by Third Way and HCM Strategists (see Table 3) found that completion is the most prevalent outcome measured by states, followed closely by equity, with priority populations defined primarily as low-income students. Completion metrics are used in the OBF systems of almost all states that employ such a policy. Interestingly, labor market outcomes (such as wages and job placement) are not widely used, not even at the two-year college level; this could be in large part because of challenges in accessing the data necessary to accurately measure labor market success.
Table 3: Metrics Categories in State Outcome-Based Funding Models (2018)
| Four-Year (# of States) | Four-Year (# of States) | Two-Year (# of States) | Two-Year (# of States) | |
|---|---|---|---|---|
| Third Way | HCM | Third Way | HCM | |
| Completion | 20/21 | 17/18 | 25/25 | 19/22 |
| Equity | 17/21 | 16/18 | 19/25 | 19/22 |
| Labor market outcomes | 2/21 | 3/18 | 9/25 | 8/22 |
Overall, while the specifics vary widely among states implementing OBF policies, it is nonetheless quite significant that completion and equity are the top metrics used, reflecting the policy priorities for accountability. As we examine next the appropriate federal accountability framework and metrics, it is helpful to take into consideration the direction states have taken in choosing their categories of metrics and the priority areas chosen by state policymakers, as a significant portion of public colleges and universities operate under such OBF policies.
Citations
- College Board (website), “Trends in Student Aid: Student Aid and Nonfederal Loans in Current Dollars over Time,” Table 2 (Washington, DC: The College Board, 2018), source">source
- This paper does not discuss the accreditation and state authorization components of the higher education triad. Its exclusive focus is on the accountability role of the U.S. Department of Education.
- Doug Shapiro, Afet Dundar, Faye Huie, Phoebe Khasiala Wakhungu, Ayesha Bhimdiwala, and Sean Eric Wilson, Completing College: A National View of Student Completion Rates, Signature Report No. 16 (Herndon, VA: National Student Clearinghouse Research Center, 2018), source">source
- Jillian Berman, “Student Debt Just Hit $1.5 Trillion,” MarketWatch, May 8, 2018, source">source
- Michael Vasquez and Dan Bauman, “How America’s College-Closure Crisis Leaves Families Devastated,” Chronicle of Higher Education, April 4, 2019, source">source
- Amy Y. Li, Lessons Learned: A Case Study of Performance Funding in Higher Education (Washington, DC: Third Way, 2018), source; and Martha Snyder and Scott Boelscher, Driving Better Outcomes: Fiscal Year 2018 State Status & Typology Update, (Washington, DC: HCM Strategists, 2018),source
- Institute of Education Sciences, National Center for Education Statistics (website), “Digest of Education Statistics,” Table 303.20, November 2017, source
- Travis Reindl and Ryan Reyna, From Information to Action: Revamping Higher Education Accountability Systems (Washington, DC: National Governors Association Center for Best Practices, 2011), source
- Jane Wellman and Darcie Harvey, Recent Statewide Reforms in Higher Education Financing and Accountability: Emerging Lessons from the States (Oakland, CA: College Futures Foundation, 2016), source
Priming the Pump: Guiding Principles
Besides the state context, a set of guiding principles will help navigate the design of federal accountability and associated metrics. A review of the research and policy literature reveals a series of key decision points that must be addressed. This section seeks to help inform the choice process through a discussion and recommendation for each.
Report-Card or Consequential Accountability?
Deming and Figlio make the key distinction between accountability with and without consequences.10 The latter, which they call report-card accountability, entails making performance information and data public and letting students and families “vote with their feet” without attaching any consequences (rewards or sanctions). The College Scorecard is an example of such a no-stakes approach in higher education, which essentially is just a form of robust transparency that seeks to improve how the market functions, with informed consumers making rational choices based on higher quality and more information. In contrast, consequential accountability involves attaching specific rewards and sanctions to certain performance thresholds, such as CDR in the HEA or the GE regulations.
While there is a long history of transparency initiatives, from the 1990 Student Right-to-Know Act to the College Navigator and then to the College Affordability and Transparency Center and, most recently, the College Scorecard, the track record clearly demonstrates that transparency alone has failed to drive performance improvements. At the same time, it is also evident that the federal government has a responsibility to address the problems of imperfect information and information asymmetries in the higher education marketplace and to give students, families, policymakers, stakeholders, and the public accurate, relevant, and timely information in a user-friendly manner. However, just as in federal accountability policy in elementary and secondary education, such transparency must be coupled with consequential accountability, in order to hold colleges and universities responsible for their performance, protect students and taxpayers, and encourage improvements. This paper argues for a combination of both transparency and accountability, with the former providing information on a wide range of data and the latter focusing on a more limited set of metrics that reflect the top policy priorities. Transparency is a prerequisite, not a synonym, for accountability.
Progress or No Accountability?
Undoubtedly, designing a consequential accountability system is a complex endeavor, and given the massive federal investments in student aid, through direct and tax expenditures across several departments, any attempt will be imperfect and the subject of intense scrutiny and criticism. In light of the many issues regarding the quality and scope of data collected by the federal government (in part due to the absence of a student-level data network, thanks to the statutory ban); the value-laden choice process of what to measure and how; ideological differences among policymakers about the proper role of the federal government; and historical industry opposition to accountability—especially among institutions likely to be negatively impacted, any outcome will be viewed as imperfect. However, this should not act as a barrier to moving forward. Instead, we must acknowledge from the outset that any accountability system and metrics will be imperfect and will need to be tweaked and refined in the future. Simply put, it is essential to recognize that there is no perfect accountability system that will get it absolutely right the first time around. But replacing the current status quo of no accountability with a better system, albeit one that requires evaluation and adjustment, will be a step in the right direction.
Carrots or Sticks?
An accountability system that only rewards high performers is more feasible politically, for obvious reasons. However, such an approach will fail to provide incentives for continuous improvement to the large majority of schools that are and will remain ineligible for rewards, and it will not protect students and taxpayers from low-performing schools and bottom-feeders. Absent penalties, the latter will have no problem proceeding with business as usual, as their access to student aid dollars remains unchanged. At the same time, the current system of failing to recognize and reward high performance undermines continuous improvement efforts by treating all schools the same, regardless of their outcomes. Therefore, any accountability system must include both carrots and sticks, rewards and penalties, to incentivize stronger performance and improved outcomes.
Adjust for Risk or Not?
A major sticking point in efforts to design accountability systems, such as President Obama’s college ratings effort, has been the debate on whether and how to adjust data for student and institutional characteristics, given the enormous diversity among institutions and the populations they serve. Under-resourced schools and/or schools serving larger shares of at-risk students (low-income students, students of color, students with inadequate academic preparation, non-traditional students, etc.) feel that their performance should be risk-adjusted. While there is certainly merit to such an argument, it is important to remember that the sole federal accountability outcome-based metric, CDR, is not risk-adjusted and that schools have a responsibility to educate the students they enroll and charge for tuition. Also, risk-adjustment effectively sets lower expectations for at-risk students,11 directly contradicting the historical equity focus of the federal government’s role in higher education. Finally, an analysis of labor market outcomes found that the choice of metric and length of the follow-up data period—for instance, how many years after graduation earnings are measured—matter more than adjustments based on demographic and other factors.12
Therefore, instead of a lower bar, an alternative approach for recognizing different student profiles is to adjust indirectly and reward institutions by either weighing more heavily at-risk student outcomes, producing separate metrics for certain students (such as those eligible to receive Pell Grants), or a combination of both. Policymakers have had more success in implementing metrics such as these that balance the need to account for student characteristics with the need to be simple and understandable. This “bonus” approach, which is employed widely by states, balances the need to recognize student characteristics with the need for equity, and is also simpler, more straightforward, and easier to implement than the regression-adjusted metrics.13 A similar approach is already used in the CDR, which accounts for the borrowing rate to assess government exposure to risk from poor student loan performance, as opposed to school- or student-based characteristics, which is discussed more in the ensuing student loan accountability metrics section.
Institutional or Program-Level Accountability?
The GE regulations provided an enormous amount of outcome data at the program level and spurred calls for expanding the collection and release of program-level data across higher education, from both ends of the political spectrum. The simple fact that institution-level outcomes mask significant variation among individual programs by both field of study and credential level, illustrated by the shocking revelation that even Harvard’s graduate program in theater failed GE, is certainly a strong argument for the collection and dissemination of outcome data by program.14 Most recently, President Trump’s executive order to enhance the College Scorecard with program-level information,15 as well as the white paper released by Sen. Lamar Alexander (R-Tenn.), chair of the Senate education committee, that calls for program-level loan repayment,16 have further elevated the issue on the reauthorization agenda. However, there is growing concern about the limitations and “blind spots” of program-level data and their high risk of manipulation, as identified by industry associations,17 academics,18 and policy researchers.19 A balanced approach would use both program- and institutional-level outcome data, but for different accountability purposes: the former for report-card (low-stakes) accountability and the latter for consequential (high-stakes) accountability. An exception to this principle would be made for GE, in recognition of the fundamental differences between such programs and those originally supported by the HEA, as reflected in the law.
Universal or Targeted Accountability?
The choice between universal or targeted accountability includes questions of scope at the institutional, student, sector, and study levels. Should all accountability metrics apply to all institutions or only to some? Should accountability measures capture data from all students or only those receiving student aid? Should accountability apply just to the outcomes of undergraduate students or include graduate students as well?
In terms of which institutions should be subject to which accountability metrics, the question pertains primarily to the long-standing for-profit industry criticism of the HEA’s distinct treatment of its colleges through the statutory 90-10 rule and the GE regulations. In this paper, we will assume that this distinction will remain; in recognition of the fundamentally different incentives of for-profit entities, which have a legal, fiduciary responsibility to maximize profits for their owners or shareholders. However, we propose that the statutory 90-10 rule, which represents a market-based accountability test, should be restored to 85-15 and strengthened to include all other non-Title IV federal student aid revenues, but could also be waived as a reward for the highest performers within a new and strong accountability system that uses solid outcome metrics, as discussed in the following section, as well as under GE. The GE distinction—applying the requirement only to proprietary and postsecondary vocational programs—should be maintained, as it operates at the program level, and it also recognizes the distinct risk posed by such programs to those who finance their own job training with student loan debt. Simply put, the particular purpose of non-degree and career-college programs is different from the academic programs for which federal aid dollars were originally used.
In terms of the population whose outcomes should be included in an accountability system, some have argued that only students benefiting directly from Title IV student financial assistance should be included. However, all students should be included in principle, just as in the K–12 federal accountability system, where all states, districts, and schools receiving federal aid are held accountable for all of their students. Even students not receiving student loans and Pell Grants are likely benefiting from a variety of servicemember and veteran benefits, tax benefits, and/or research dollars through which taxpayers contribute to their education. At the same time, by definition, some metrics based on loans or grants would only capture a subset of students. This is both acceptable and desirable, given that the federal government does clearly have a particular interest in those students’ outcomes and (in the case of student loans) those students may be at risk if an institution’s quality is poor. Therefore, a balanced accountability system would include metrics capturing outcomes for all students, such as degree completion; metrics pertaining to student aid, such as loan-based metrics; and metrics addressing priority populations, such as Pell-based metrics. The inclusion of graduate students as a distinct category in federal accountability also seems necessary, given its outsized share of and impact on the federal student loan portfolio20 and recent enrollment trends that suggest the programs often act as a “cash cow” for colleges.21
Overall, the answer to the question of universal versus targeted accountability is that both must be incorporated in the federal accountability system through a mix of provisions and metrics that account for all schools and students but also measure a subset based on risk and government subsidies.
Bright Lines or Graduated Accountability?
Another major decision point involves the criticism of the all-or-nothing feature of the CDR and GE that is used as an argument against using bright lines or thresholds in any new accountability system. For example, the House PROSPER Act retains such an approach in its loan repayment metric. While bright lines indeed fail to provide an incentive for improvement to schools that barely miss the threshold, as well as schools that do not come close to hitting a trigger, they still provide an important function in identifying a minimum level of performance considered acceptable for participation in student aid programs, thus weeding out schools with consistently horrific outcomes.
However, a system of graduated accountability, under which sanctions gradually increase as performance decreases, also provides several advantages in providing meaningful incentives, limiting gaming, and ensuring that penalties (and rewards) are commensurate with performance. Thus, a balanced approach would involve both bright lines for identifying the worst as well as a system of gradual incentives. Setting a bright line for identifying the best performers (similar to Blue Ribbon schools in K–12 education) would also be beneficial in recognizing and rewarding excellence, sending a strong signal to consumers, and providing an added incentive for institutional improvements, especially if funding incentives are insufficient across the performance spectrum. This approach, used in the Senate’s bipartisan Student Protection and Success Act, would combine the best elements of both approaches and ensure a minimum threshold of acceptable performance for participation in student aid (above 15 percent repayment rate) and meaningful incentives for continuous improvement (risk-sharing fee based on non-repayment rate and bonus for schools doing a good job for low- and moderate-income students).
Obviously, the hardest part, both technically and politically, is deciding where to set the floor, followed by setting the performance thresholds for gradual incentives and disincentives. One potential way to move past this barrier is to adopt and adapt the approaches used in other areas, such as K–12 education, where the bottom 5 percent of schools and those with graduation rates under 67 percent require intervention. While the specific thresholds can be adjusted, the point is that federal policymakers have agreed on bright lines when it comes to elementary and secondary education, so setting bright lines in the postsecondary arena, notwithstanding key differences between the two areas, should be doable. The impetus should be even greater, given the outsized federal investment in higher education. The two largest federal programs for K–12, Title I and IDEA, are roughly equal to Pell expenditures.
Accuracy or Simplicity?
Finally, the trade-off between accuracy and simplicity is paramount. If there is one thing everyone seems to agree on is that any accountability system must be simple and understandable, which necessarily comes at the expense of precision. An overly complex system with multiple metrics, various adjustments, and groupings of institutions might provide a more accurate and nuanced picture of performance, but will reduce political and practical feasibility, expand the scope for strategic responses and gaming, and fail to provide user-friendly and accessible information. At the same time, reliance on a single metric, such as the current CDR, might be the simplest way forward, but it also presents significant risk in undermining accountability efforts. For example, a sole loan repayment rate to assess performance and risk for the $100 billion annual federal student loan investment risks weakening the accountability effort by failing to capture the return on the $30 billion annual federal investment in Pell Grants.22 Instead, a balanced approach would use a small number of metrics sufficient to quantify institutional performance on outcomes that represent priority policy objectives and capture a broader set of indicators, while ensuring overall simplicity.
Citations
- College Board (website), “Trends in Student Aid: Student Aid and Nonfederal Loans in Current Dollars over Time,” Table 2 (Washington, DC: The College Board, 2018), <a href="source">source">source
- This paper does not discuss the accreditation and state authorization components of the higher education triad. Its exclusive focus is on the accountability role of the U.S. Department of Education.
- Doug Shapiro, Afet Dundar, Faye Huie, Phoebe Khasiala Wakhungu, Ayesha Bhimdiwala, and Sean Eric Wilson, Completing College: A National View of Student Completion Rates, Signature Report No. 16 (Herndon, VA: National Student Clearinghouse Research Center, 2018), <a href="source">source">source
- Jillian Berman, “Student Debt Just Hit $1.5 Trillion,” MarketWatch, May 8, 2018, <a href="source">source">source
- Michael Vasquez and Dan Bauman, “How America’s College-Closure Crisis Leaves Families Devastated,” Chronicle of Higher Education, April 4, 2019, <a href="source">source">source
- Amy Y. Li, Lessons Learned: A Case Study of Performance Funding in Higher Education (Washington, DC: Third Way, 2018), source">source; and Martha Snyder and Scott Boelscher, Driving Better Outcomes: Fiscal Year 2018 State Status & Typology Update, (Washington, DC: HCM Strategists, 2018),source">source
- Institute of Education Sciences, National Center for Education Statistics (website), “Digest of Education Statistics,” Table 303.20, November 2017, source">source
- Travis Reindl and Ryan Reyna, From Information to Action: Revamping Higher Education Accountability Systems (Washington, DC: National Governors Association Center for Best Practices, 2011), source">source
- Jane Wellman and Darcie Harvey, Recent Statewide Reforms in Higher Education Financing and Accountability: Emerging Lessons from the States (Oakland, CA: College Futures Foundation, 2016), source">source
- David J. Deming and David Figlio, “Accountability in U.S. Education: Applying Lessons from K–12 Experience to Higher Education,” Journal of Economic Perspectives 30, no. 3 (Summer 2016): 33–56, source
- Trey Miller, Higher Education Outcomes-Based Funding Models and Academic Quality (Indianapolis, IN: Lumina Foundation, 2016), source
- Veronica Minaya and Judith Scott-Clayton, Labor Market Outcomes and Postsecondary Accountability, Working Paper 22880, (Cambridge, MA: National Bureau of Economic Research, 2016), source
- Using Federal Data to Measure and Improve the Performance of U.S. Institutions of Higher Education, (Washington, DC: U.S. Council of Economic Advisers, updated 2017), source
- Kevin Carey, “Programs That Are Predatory: It’s Not Just at For-Profit Colleges,” New York Times, January 13, 2017, source; and Jason Delisle, “Accountability for Higher Education,” National Affairs 40 (Summer 2019), source
- The White House (website), “Executive Order on Improving Free Inquiry, Transparency, and Accountability at Colleges and Universities,” March 21, 2019,source
- Senate Committee on Health, Education, Labor and Pensions, Higher Education Accountability, white paper (2018), source
- Peter McPherson (president of the Association of Public and Land-Grant Universities), “APLU Comments on Senate Help Committee’s ‘Higher Education Accountability’ White Paper,” letter to the Honorable Lamar Alexander, February 15 , 2018, source
- Robert Kelchen, “Comments on Accountability and the Higher Education Act,” Kelchen on Education (blog), February 15, 2018,source
- C. J. Libassi, Changes to Higher Ed Accountability Portend a Big Downside for Students of Color (Washington, DC: Center for American Progress, 2018), source
- Kevin Carey, “Biggest Offender in Outsize Debt: Graduate Schools,” New York Times, June 3, 2019, source
- Jon Marcus, “As Graduate School Costs Skyrocket, the Student Debt Problem Is Getting Worse,” Pacific Standard, April 23, 2019, source
- Student Aid Overview: Fiscal Year 2020 Budget Request (Washington, DC: U.S. Department of Education, 2019), source
The Objectives: What Gets Measured, Gets Done
If it is true that what gets measured gets done, then we need to figure out what we want to get done. First and foremost, a federal accountability system must assess risk and return on investment on its two primary and largest student aid programs, student loans and Pell Grants. While it could be argued that loan metrics could be used as proxies for both programs, as Pell Grant recipients represent nearly 90 percent of undergraduate loan defaulters,23 this would ignore the fact that 44 percent of Pell Grant recipients do not borrow federal student loans,24 while three out 10 non-borrowers received a Pell Grant.25 Pell is the bedrock of the nation’s student aid system and reflects the federal government’s historic role of promoting access for lower-income students in higher education. Unlike unsubsidized loans, Pell is very well targeted, as 68 percent of recipients have incomes under $30,000.26 A balanced approach would hold schools accountable for both federal student loans and the Pell Grant.
Second, completion is a key objective and priority, both from the student and government perspectives. The relationship between completion and student loan defaults and repayment has been well established. Non-completers are three times as likely to default on their loans compared to graduates27 and completers are at least 20 percentage points more likely than non-completers to pay down loan principal.28 Students also enroll and pay tuition and other college costs with the expectation that they will earn credentials; federal requirements provide federal aid eligibility only to students who are degree- or certificate-seeking. This fundamental public policy and individual objective is also reflected in governors’ commitments to the completion agenda and is featured prominently in state OBF systems, almost all of which include completion metrics. It is worth noting, however, an unintended consequence of this metric is some institutions, especially at the two-year level, may shift their offerings to certificates and shorter credentials that have higher completion rates but less value in the labor market. Accreditation and state oversight are critical in preventing such gaming. These dangers underscore the need to avoid a single metric federal accountability system.
Third, equity is another key federal objective and the primary impetus behind the HEA, which reflected the ambitious vision of college for all.29 However, while expanding access to higher education has been the role of the federal government, the need to move beyond access to success is imperative given the detrimental outcomes of dropping out for student loan borrowers.30 Low-income students are the priority population in the HEA and knowing whether federal programs are actually increasing the degree attainment rates for that population is essential. As a result, measuring this population’s access and success rates is critical for advancing equity. It is not surprising that states have also placed equity at the core of their OBF systems, along with completion.
Fourth, the growing interest in labor market outcomes, including employment, earnings, and job placement rates, cannot be ignored and research shows that even when imperfect such data can provide valuable information distinct from students’ academic outcomes, especially for two-year and non-degree institutions.31 At the same time, the research shows that the simplest labor market metrics are unreliable, especially when viewed in the near-term immediately following separation and cautions against reliance on any single metric. The low rate of usage in state OBF systems also underscores the limited feasibility of a large-scale federal consequential accountability metric, at least immediately. Since getting a job is a primary objective of prospective college students,32 labor market outcomes must certainly be provided for transparency. But given data quality and methodological concerns, it is more appropriate to initially rely on other metrics that serve as proxies for labor market success for consequential accountability purposes, such as the loan repayment rate, and include labor market data solely in report-card accountability while continuing to explore a reliable metric for consequential accountability.
Finally, report-card accountability must include fully disaggregated data, in order to allow for the analysis and identification of problems that are masked by aggregate performance metrics. This is critical for equity and has been tried and tested in K–12 education with great success, as it is the one area of clear consensus that emerged from the elementary and secondary accountability movement.
In summary, a new federal accountability system should protect federal investments and promote completion and equity through institutional-level consequential accountability, and strengthen institution- and program-level report-card accountability through the addition of labor market outcomes. At the same time, existing guardrails for higher-risk institutions and programs should be maintained, with flexible opportunities for high performers.
Citations
- College Board (website), “Trends in Student Aid: Student Aid and Nonfederal Loans in Current Dollars over Time,” Table 2 (Washington, DC: The College Board, 2018), <a href="<a href="source">source">source">source
- This paper does not discuss the accreditation and state authorization components of the higher education triad. Its exclusive focus is on the accountability role of the U.S. Department of Education.
- Doug Shapiro, Afet Dundar, Faye Huie, Phoebe Khasiala Wakhungu, Ayesha Bhimdiwala, and Sean Eric Wilson, Completing College: A National View of Student Completion Rates, Signature Report No. 16 (Herndon, VA: National Student Clearinghouse Research Center, 2018), <a href="<a href="source">source">source">source
- Jillian Berman, “Student Debt Just Hit $1.5 Trillion,” MarketWatch, May 8, 2018, <a href="<a href="source">source">source">source
- Michael Vasquez and Dan Bauman, “How America’s College-Closure Crisis Leaves Families Devastated,” Chronicle of Higher Education, April 4, 2019, <a href="<a href="source">source">source">source
- Amy Y. Li, Lessons Learned: A Case Study of Performance Funding in Higher Education (Washington, DC: Third Way, 2018), <a href="source">source">source; and Martha Snyder and Scott Boelscher, Driving Better Outcomes: Fiscal Year 2018 State Status & Typology Update, (Washington, DC: HCM Strategists, 2018),<a href="source">source">source
- Institute of Education Sciences, National Center for Education Statistics (website), “Digest of Education Statistics,” Table 303.20, November 2017, <a href="source">source">source
- Travis Reindl and Ryan Reyna, From Information to Action: Revamping Higher Education Accountability Systems (Washington, DC: National Governors Association Center for Best Practices, 2011), <a href="source">source">source
- Jane Wellman and Darcie Harvey, Recent Statewide Reforms in Higher Education Financing and Accountability: Emerging Lessons from the States (Oakland, CA: College Futures Foundation, 2016), <a href="source">source">source
- David J. Deming and David Figlio, “Accountability in U.S. Education: Applying Lessons from K–12 Experience to Higher Education,” Journal of Economic Perspectives 30, no. 3 (Summer 2016): 33–56, source">source
- Trey Miller, Higher Education Outcomes-Based Funding Models and Academic Quality (Indianapolis, IN: Lumina Foundation, 2016), source">source
- Veronica Minaya and Judith Scott-Clayton, Labor Market Outcomes and Postsecondary Accountability, Working Paper 22880, (Cambridge, MA: National Bureau of Economic Research, 2016), source">source
- Using Federal Data to Measure and Improve the Performance of U.S. Institutions of Higher Education, (Washington, DC: U.S. Council of Economic Advisers, updated 2017), source">source
- Kevin Carey, “Programs That Are Predatory: It’s Not Just at For-Profit Colleges,” New York Times, January 13, 2017, source">source; and Jason Delisle, “Accountability for Higher Education,” National Affairs 40 (Summer 2019), source">source
- The White House (website), “Executive Order on Improving Free Inquiry, Transparency, and Accountability at Colleges and Universities,” March 21, 2019,source">source
- Senate Committee on Health, Education, Labor and Pensions, Higher Education Accountability, white paper (2018), source">source
- Peter McPherson (president of the Association of Public and Land-Grant Universities), “APLU Comments on Senate Help Committee’s ‘Higher Education Accountability’ White Paper,” letter to the Honorable Lamar Alexander, February 15 , 2018, source">source
- Robert Kelchen, “Comments on Accountability and the Higher Education Act,” Kelchen on Education (blog), February 15, 2018,source">source
- C. J. Libassi, Changes to Higher Ed Accountability Portend a Big Downside for Students of Color (Washington, DC: Center for American Progress, 2018), source">source
- Kevin Carey, “Biggest Offender in Outsize Debt: Graduate Schools,” New York Times, June 3, 2019, source">source
- Jon Marcus, “As Graduate School Costs Skyrocket, the Student Debt Problem Is Getting Worse,” Pacific Standard, April 23, 2019, source">source
- Student Aid Overview: Fiscal Year 2020 Budget Request (Washington, DC: U.S. Department of Education, 2019), source">source
- Adam Looney, Reauthorizing the Higher Education Act: Strengthening Accountability to Protect Students and Taxpayers (Washington, DC: The Brookings Institution, 2019), source
- Author’s analysis of 2015–16 National Postsecondary Student Aid Study from National Center for Education Statistics, 2019.
- Student Financing of Undergraduate Education in 2015–16: Financial Aid by Type and Source (Washington, DC: National Center for Education Statistics, March 2019), source
- Student Financial Assistance: Fiscal Year 2020 Budget Request (Washington, DC: U.S. Department of Education, 2019), source
- U.S. Department of Education (website), “Archived Information,” “Toward a New Focus on Outcomes in Higher Education: Remarks by Secretary Arne Duncan,” July 27, 2015, source
- Michael Itzkowitz, Want More Students to Pay Down Their Loans? Help Them Graduate (Washington, DC: Third Way, 2018), source
- Kelly Field, “The Higher Education Act Just Turned 50. Has It Done What it Was Supposed To?” Chronicle of Higher Education, November 8, 2015, source
- Adam Looney and Constantine Yannelis, A Crisis in Student Loans? How Changes in the Characteristics of Borrowers and in the Institutions They Attended Contributed to Rising Loan Defaults (Washington, DC: The Brookings Institution, 2016), source
- Veronica Minaya and Judith Scott-Clayton, Labor Market Outcomes and Postsecondary Accountability, Working Paper 22880 (Cambridge, MA: National Bureau of Economic Research, 2016), source
- Jeffrey J. Selingo, “College Students Say They Want a Degree for a Job. Are They Getting What They Want?” Washington Post, September 1, 2018, source
The Metrics: Loan Repayment Rate, Pell Access & Success Rate, and Completion Rate
In terms of measuring risk for the federal student loan program, the loan repayment metric is widely acknowledged as being superior to CDR and should replace it, although further refinements are needed especially in the interactivity with repayment programs. A new federal accountability system would use an institutional loan repayment rate, which serves as a proxy for value, completion, and labor market outcomes, to hold schools accountable through both a bright line for eligibility loss and a risk-sharing system of graduated sanctions for low performers—who would be required to repay some of the under-paid amounts to the federal government as a penalty for leaving students not well off enough to comfortably repay their loans. Additionally, a program-level loan repayment rate would be used in GE accountability to hold the highest-risk programs accountable. One feature of the CDR that should be maintained is the mechanism for assessing risk based on the percentage of students who borrow loans. An institution with a higher borrowing rate presents a higher risk to the students it enrolls, and vice versa, and this must be reflected in the new system. A proposal put forth by the Institute for College Access and Success in 2016 for a Student Non-Repayment Risk Indicator (SNRI) demonstrates how this feature can be incorporated.33 This metric would serve the objectives of protecting the federal investment and promoting completion.
In regards to the second largest federal student aid program, the Pell Grant, institutions must be held accountable for enrolling and graduating low-income students through the use of an access and success metric that compares their completion rate with the college’s overall completion rate, to measure the achievement gap. This metric would not be used to draw a bright line for eligibility, but would be used for a sliding scale of rewards for institutional performance. Schools with high shares of and graduation rates for Pell recipients would receive increasingly sizable rewards, which would serve as an incentive for ineligible schools to become eligible and for eligible schools to gain additional rewards. To avoid gaming, perhaps higher thresholds could be established for shorter credentials, but this will need to be balanced against the need for simplicity. This metric would serve the objectives of completion and equity.
However, the two metrics that apply to federal student loan borrowers and Pell Grant recipients fail to capture the remaining college student population. An overall completion rate metric is essential for holding all federally aided schools accountable for serving all students well and working hard toward their graduation. Here, similar to the loan repayment rate, there would be an eligibility threshold for all federal student aid. Above that baseline threshold to further incent improvement, Congress should establish a system of sanctions that will require corrective action through improvement plans, with the loss of access to campus-based aid and other HEA institutional grant programs for persistently underperforming institutions.
These three metrics (see Table 4) would form the basis for institutional consequential accountability and advance the federal government’s objectives in protecting students and taxpayers against risk and promote completion and equity. While the devil is in the details and any system will be admittedly imperfect, the combined effect of these metrics will be a major improvement over the current system.
Table 4: Consequential Accountability Metrics
| Metric | Application | Incentives | Thresholds |
|---|---|---|---|
| Loan repayment rate | All institutions + GE programs | Loss of eligibility + graduated sanctions | Bright line + risk-sharing |
| Pell access & success rate | All institutions | Rewards | Tiered rewards |
| Completion rate | All institutions | Loss of eligibility + graduated sanctions | Bright line + tiered sanctions |
In terms of technical feasibility, the proposed metrics rank high, as they are all already collected and reported. While there are various proposals on the exact metrics, especially on the loan repayment rate, the differences are not insurmountable and consensus is within reach. Introduced legislation in both chambers of Congress already provide a menu of options that can be further refined and negotiated.
However, the real challenges center on the political feasibility and the need for both sides of the aisle to recognize that accountability necessarily will involve loss of eligibility for the lowest-performing institutions. One potential way to ameliorate the immediate impact and allow for institutions to plan accordingly would be—similar to the GE experience—to publish rates and provide a delayed or phased-in implementation; which would include a disclosure period followed by sanctions kicking in, thus providing enough time for adjustments. Another important element is the need to recognize up front that the metrics, and their thresholds, will also need to be revisited and adjusted in the future. Just as we now need to replace the CDR with a new metric, legislation could incorporate periodic reviews through special reports and/or commissions that monitor and examine the impact of the accountability system.
A quick note on the often-cited problem (or excuse) of unintended consequences: If there is one thing GE clearly demonstrated, it is that even the prospect of accountability changes behavior. Institutions responded to the data releases by discontinuing failing programs and making improvements in their program offerings. Any analysis that simply applies a proposed accountability framework while assuming unchanged behavior misses the mark. The “sky is falling” prognoses have been proven wrong in several instances of federal higher education legislation and regulation. While scare tactics might be an easy way to oppose change, policymakers should instead anticipate that institutions will respond to accountability by seeking to improve in order to avoid sanctions and/or obtain rewards.
Citations
- College Board (website), “Trends in Student Aid: Student Aid and Nonfederal Loans in Current Dollars over Time,” Table 2 (Washington, DC: The College Board, 2018), <a href="<a href="<a href="source">source">source">source">source
- This paper does not discuss the accreditation and state authorization components of the higher education triad. Its exclusive focus is on the accountability role of the U.S. Department of Education.
- Doug Shapiro, Afet Dundar, Faye Huie, Phoebe Khasiala Wakhungu, Ayesha Bhimdiwala, and Sean Eric Wilson, Completing College: A National View of Student Completion Rates, Signature Report No. 16 (Herndon, VA: National Student Clearinghouse Research Center, 2018), <a href="<a href="<a href="source">source">source">source">source
- Jillian Berman, “Student Debt Just Hit $1.5 Trillion,” MarketWatch, May 8, 2018, <a href="<a href="<a href="source">source">source">source">source
- Michael Vasquez and Dan Bauman, “How America’s College-Closure Crisis Leaves Families Devastated,” Chronicle of Higher Education, April 4, 2019, <a href="<a href="<a href="source">source">source">source">source
- Amy Y. Li, Lessons Learned: A Case Study of Performance Funding in Higher Education (Washington, DC: Third Way, 2018), <a href="<a href="source">source">source">source; and Martha Snyder and Scott Boelscher, Driving Better Outcomes: Fiscal Year 2018 State Status & Typology Update, (Washington, DC: HCM Strategists, 2018),<a href="<a href="source">source">source">source
- Institute of Education Sciences, National Center for Education Statistics (website), “Digest of Education Statistics,” Table 303.20, November 2017, <a href="<a href="source">source">source">source
- Travis Reindl and Ryan Reyna, From Information to Action: Revamping Higher Education Accountability Systems (Washington, DC: National Governors Association Center for Best Practices, 2011), <a href="<a href="source">source">source">source
- Jane Wellman and Darcie Harvey, Recent Statewide Reforms in Higher Education Financing and Accountability: Emerging Lessons from the States (Oakland, CA: College Futures Foundation, 2016), <a href="<a href="source">source">source">source
- David J. Deming and David Figlio, “Accountability in U.S. Education: Applying Lessons from K–12 Experience to Higher Education,” Journal of Economic Perspectives 30, no. 3 (Summer 2016): 33–56, <a href="source">source">source
- Trey Miller, Higher Education Outcomes-Based Funding Models and Academic Quality (Indianapolis, IN: Lumina Foundation, 2016), <a href="source">source">source
- Veronica Minaya and Judith Scott-Clayton, Labor Market Outcomes and Postsecondary Accountability, Working Paper 22880, (Cambridge, MA: National Bureau of Economic Research, 2016), <a href="source">source">source
- Using Federal Data to Measure and Improve the Performance of U.S. Institutions of Higher Education, (Washington, DC: U.S. Council of Economic Advisers, updated 2017), <a href="source">source">source
- Kevin Carey, “Programs That Are Predatory: It’s Not Just at For-Profit Colleges,” New York Times, January 13, 2017, <a href="source">source">source; and Jason Delisle, “Accountability for Higher Education,” National Affairs 40 (Summer 2019), <a href="source">source">source
- The White House (website), “Executive Order on Improving Free Inquiry, Transparency, and Accountability at Colleges and Universities,” March 21, 2019,<a href="source">source">source
- Senate Committee on Health, Education, Labor and Pensions, Higher Education Accountability, white paper (2018), <a href="source">source">source
- Peter McPherson (president of the Association of Public and Land-Grant Universities), “APLU Comments on Senate Help Committee’s ‘Higher Education Accountability’ White Paper,” letter to the Honorable Lamar Alexander, February 15 , 2018, <a href="source">source">source
- Robert Kelchen, “Comments on Accountability and the Higher Education Act,” Kelchen on Education (blog), February 15, 2018,<a href="source">source">source
- C. J. Libassi, Changes to Higher Ed Accountability Portend a Big Downside for Students of Color (Washington, DC: Center for American Progress, 2018), <a href="source">source">source
- Kevin Carey, “Biggest Offender in Outsize Debt: Graduate Schools,” New York Times, June 3, 2019, <a href="source">source">source
- Jon Marcus, “As Graduate School Costs Skyrocket, the Student Debt Problem Is Getting Worse,” Pacific Standard, April 23, 2019, <a href="source">source">source
- Student Aid Overview: Fiscal Year 2020 Budget Request (Washington, DC: U.S. Department of Education, 2019), <a href="source">source">source
- Adam Looney, Reauthorizing the Higher Education Act: Strengthening Accountability to Protect Students and Taxpayers (Washington, DC: The Brookings Institution, 2019), source">source
- Author’s analysis of 2015–16 National Postsecondary Student Aid Study from National Center for Education Statistics, 2019.
- Student Financing of Undergraduate Education in 2015–16: Financial Aid by Type and Source (Washington, DC: National Center for Education Statistics, March 2019), source">source
- Student Financial Assistance: Fiscal Year 2020 Budget Request (Washington, DC: U.S. Department of Education, 2019), source">source
- U.S. Department of Education (website), “Archived Information,” “Toward a New Focus on Outcomes in Higher Education: Remarks by Secretary Arne Duncan,” July 27, 2015, source">source
- Michael Itzkowitz, Want More Students to Pay Down Their Loans? Help Them Graduate (Washington, DC: Third Way, 2018), source">source
- Kelly Field, “The Higher Education Act Just Turned 50. Has It Done What it Was Supposed To?” Chronicle of Higher Education, November 8, 2015, source">source
- Adam Looney and Constantine Yannelis, A Crisis in Student Loans? How Changes in the Characteristics of Borrowers and in the Institutions They Attended Contributed to Rising Loan Defaults (Washington, DC: The Brookings Institution, 2016), source">source
- Veronica Minaya and Judith Scott-Clayton, Labor Market Outcomes and Postsecondary Accountability, Working Paper 22880 (Cambridge, MA: National Bureau of Economic Research, 2016), source">source
- Jeffrey J. Selingo, “College Students Say They Want a Degree for a Job. Are They Getting What They Want?” Washington Post, September 1, 2018, source">source
- Lindsay Ahlman, Debbie Cochrane, and Jessica Thompson, A New Approach to College Accountability: Balancing Sanctions and Rewards to Improve Student Outcomes (Oakland, CA: The Institute for College Access and Success, 2016), source
Conclusion
In the search for an effective accountability system, the key question remains whether there is political will to hold colleges and universities accountable for results. Any proposal will be modeled to estimate effects on institutions and thus get screened through the political lens. Each side of the aisle will be concerned about the impact on certain sectors and each member of Congress will look to see how the institutions in their jurisdictions would fare under each scenario, necessarily bringing political considerations. However, what needs to be agreed to from the outset is that change is necessary; business as usual is neither sustainable nor desirable. That means that an effective accountability system should lead to a significant shift in institutional priorities, practices, policies, and focus. Unless there is agreement on this premise, and thus on the need to compromise ideological and political bright lines, then the exercise is academic and ultimately futile.
A new accountability system, albeit imperfect, is desperately needed, not to punish, but to drive system-wide improvement while weeding out the worst performers to protect federal investments in student aid, and by extent, students, families, and taxpayers. We have assumed in this paper that there is consensus on the need to reverse long-standing trends of runaway costs and questionable value, poor student outcomes, and virtually no accountability. While the discussion on the specifics will involve major disagreements and vigorous debate, hopefully a balanced approach can emerge that focuses on the major federal policy priorities and objectives, adheres to the principle of simplicity, and anticipates the need for further refinement. By building on developments in the states and focusing on completion, equity, and risk, the proposed framework here represents a modest attempt to chart a path toward consensus.
Citations
- College Board (website), “Trends in Student Aid: Student Aid and Nonfederal Loans in Current Dollars over Time,” Table 2 (Washington, DC: The College Board, 2018), <a href="<a href="<a href="<a href="source">source">source">source">source">source
- This paper does not discuss the accreditation and state authorization components of the higher education triad. Its exclusive focus is on the accountability role of the U.S. Department of Education.
- Doug Shapiro, Afet Dundar, Faye Huie, Phoebe Khasiala Wakhungu, Ayesha Bhimdiwala, and Sean Eric Wilson, Completing College: A National View of Student Completion Rates, Signature Report No. 16 (Herndon, VA: National Student Clearinghouse Research Center, 2018), <a href="<a href="<a href="<a href="source">source">source">source">source">source
- Jillian Berman, “Student Debt Just Hit $1.5 Trillion,” MarketWatch, May 8, 2018, <a href="<a href="<a href="<a href="source">source">source">source">source">source
- Michael Vasquez and Dan Bauman, “How America’s College-Closure Crisis Leaves Families Devastated,” Chronicle of Higher Education, April 4, 2019, <a href="<a href="<a href="<a href="source">source">source">source">source">source
- Amy Y. Li, Lessons Learned: A Case Study of Performance Funding in Higher Education (Washington, DC: Third Way, 2018), <a href="<a href="<a href="source">source">source">source">source; and Martha Snyder and Scott Boelscher, Driving Better Outcomes: Fiscal Year 2018 State Status & Typology Update, (Washington, DC: HCM Strategists, 2018),<a href="<a href="<a href="source">source">source">source">source
- Institute of Education Sciences, National Center for Education Statistics (website), “Digest of Education Statistics,” Table 303.20, November 2017, <a href="<a href="<a href="source">source">source">source">source
- Travis Reindl and Ryan Reyna, From Information to Action: Revamping Higher Education Accountability Systems (Washington, DC: National Governors Association Center for Best Practices, 2011), <a href="<a href="<a href="source">source">source">source">source
- Jane Wellman and Darcie Harvey, Recent Statewide Reforms in Higher Education Financing and Accountability: Emerging Lessons from the States (Oakland, CA: College Futures Foundation, 2016), <a href="<a href="<a href="source">source">source">source">source
- David J. Deming and David Figlio, “Accountability in U.S. Education: Applying Lessons from K–12 Experience to Higher Education,” Journal of Economic Perspectives 30, no. 3 (Summer 2016): 33–56, <a href="<a href="source">source">source">source
- Trey Miller, Higher Education Outcomes-Based Funding Models and Academic Quality (Indianapolis, IN: Lumina Foundation, 2016), <a href="<a href="source">source">source">source
- Veronica Minaya and Judith Scott-Clayton, Labor Market Outcomes and Postsecondary Accountability, Working Paper 22880, (Cambridge, MA: National Bureau of Economic Research, 2016), <a href="<a href="source">source">source">source
- Using Federal Data to Measure and Improve the Performance of U.S. Institutions of Higher Education, (Washington, DC: U.S. Council of Economic Advisers, updated 2017), <a href="<a href="source">source">source">source
- Kevin Carey, “Programs That Are Predatory: It’s Not Just at For-Profit Colleges,” New York Times, January 13, 2017, <a href="<a href="source">source">source">source; and Jason Delisle, “Accountability for Higher Education,” National Affairs 40 (Summer 2019), <a href="<a href="source">source">source">source
- The White House (website), “Executive Order on Improving Free Inquiry, Transparency, and Accountability at Colleges and Universities,” March 21, 2019,<a href="<a href="source">source">source">source
- Senate Committee on Health, Education, Labor and Pensions, Higher Education Accountability, white paper (2018), <a href="<a href="source">source">source">source
- Peter McPherson (president of the Association of Public and Land-Grant Universities), “APLU Comments on Senate Help Committee’s ‘Higher Education Accountability’ White Paper,” letter to the Honorable Lamar Alexander, February 15 , 2018, <a href="<a href="source">source">source">source
- Robert Kelchen, “Comments on Accountability and the Higher Education Act,” Kelchen on Education (blog), February 15, 2018,<a href="<a href="source">source">source">source
- C. J. Libassi, Changes to Higher Ed Accountability Portend a Big Downside for Students of Color (Washington, DC: Center for American Progress, 2018), <a href="<a href="source">source">source">source
- Kevin Carey, “Biggest Offender in Outsize Debt: Graduate Schools,” New York Times, June 3, 2019, <a href="<a href="source">source">source">source
- Jon Marcus, “As Graduate School Costs Skyrocket, the Student Debt Problem Is Getting Worse,” Pacific Standard, April 23, 2019, <a href="<a href="source">source">source">source
- Student Aid Overview: Fiscal Year 2020 Budget Request (Washington, DC: U.S. Department of Education, 2019), <a href="<a href="source">source">source">source
- Adam Looney, Reauthorizing the Higher Education Act: Strengthening Accountability to Protect Students and Taxpayers (Washington, DC: The Brookings Institution, 2019), <a href="source">source">source
- Author’s analysis of 2015–16 National Postsecondary Student Aid Study from National Center for Education Statistics, 2019.
- Student Financing of Undergraduate Education in 2015–16: Financial Aid by Type and Source (Washington, DC: National Center for Education Statistics, March 2019), <a href="source">source">source
- Student Financial Assistance: Fiscal Year 2020 Budget Request (Washington, DC: U.S. Department of Education, 2019), <a href="source">source">source
- U.S. Department of Education (website), “Archived Information,” “Toward a New Focus on Outcomes in Higher Education: Remarks by Secretary Arne Duncan,” July 27, 2015, <a href="source">source">source
- Michael Itzkowitz, Want More Students to Pay Down Their Loans? Help Them Graduate (Washington, DC: Third Way, 2018), <a href="source">source">source
- Kelly Field, “The Higher Education Act Just Turned 50. Has It Done What it Was Supposed To?” Chronicle of Higher Education, November 8, 2015, <a href="source">source">source
- Adam Looney and Constantine Yannelis, A Crisis in Student Loans? How Changes in the Characteristics of Borrowers and in the Institutions They Attended Contributed to Rising Loan Defaults (Washington, DC: The Brookings Institution, 2016), <a href="source">source">source
- Veronica Minaya and Judith Scott-Clayton, Labor Market Outcomes and Postsecondary Accountability, Working Paper 22880 (Cambridge, MA: National Bureau of Economic Research, 2016), <a href="source">source">source
- Jeffrey J. Selingo, “College Students Say They Want a Degree for a Job. Are They Getting What They Want?” Washington Post, September 1, 2018, <a href="source">source">source
- Lindsay Ahlman, Debbie Cochrane, and Jessica Thompson, A New Approach to College Accountability: Balancing Sanctions and Rewards to Improve Student Outcomes (Oakland, CA: The Institute for College Access and Success, 2016), source">source