What the Washington Post Gainful Employment Fact Check (and the Original Stat) Miss

One of the facts that particularly stood out when the U.S. Department of Education released its latest proposal for defining gainful employment was that 72 percent of programs at for-profit colleges "produced graduates who on average earned less than high school dropouts." It was a stark claim based upon creating an annual earnings figure from the weekly earnings of high school dropouts from the Bureau of Labor Statistics.

But in a Friday fact check column, the Washington Post challenged that claim, saying the threshold is too high. It then suggested three other figures, which represent a range of earnings between $22,860 and $18,580. The fact check concludes that if you use the lowest census figure that 72 percent figure drops to about 49 percent. The Post also raises a few other critiques, such as basing the figures on programs, not size, including anyone regardless of whether they are working, and using an earnings comparison that includes people who may have left school decades earlier.

This weekend, the Department released a lengthy and detailed response to the Post, challenging some data assumptions as well as making a bigger case for the need for the regulation and refuting many claims line by line.

What's funny about this debate is that the concept of the original comparison chosen was quite charitable, even if there are different levels that could have been used--a point that the Department touches on briefly deep in its  response. The Department's initial comparison was to high school dropouts. The Post is saying that high school dropouts actually earn less than what the Department said.

But gainful employment isn't looking at the earnings of high school or even college dropouts. It's of college graduates. Given that a major premise of higher education is that it improves incomes, comparing the gainful results to the population it is supposed to mirror would only look worse. For example, the same National Center for Education Statistic source suggested by the fact check column puts students with some college but no degree at $31,990 in median earnings and associate degree graduates at $37,030. The Post's BLS source, which had the lowest results, estimates $31,580 for those who have a postsecondary non-degree award and $34,760 for an associate degree graduate.

So instead of arguing whether the earnings return is close to what we would expect for those who earned postsecondary credentials, the debate is now over whether 49 percent or just under three-quarters of programs have earnings below those of a high school dropout?

Credentials Matter

Let's set aside contextual thresholds for the moment and take a step back to consider what these data are and what they tell us. The results are for graduates of programs who received a federal investment in the form of grants or loans to help pay for their education. Some may not be working, some may have been working before. But it's designed to be a picture of how these graduates--the ones who did what was expected of them to earn a credential from that institution--are faring compared to their debt levels. The results are the higher of the mean or median. That means by definition, the level reported represents the earnings for at least half of the graduates receiving federal investments. (If the mean is higher than the median, then it's more than half, if it's the median than it's half).

So how much money are these graduates actually earning? If you weight the data by the number of reported completers, then the average earnings are $27,625. But that masks a huge spread, from graduates with typical earnings of $1,812 to $162,280. To get a better sense, the chart below breaks down the weighted average earnings by credential levels and the type of institution.

This table shows what both the Post column and the original statistic failed to acknowledge: a big chunk of the gainful employment problem is about credentialing.  Colleges created and expanded a series of programs, particularly certificates and to a lesser extent associate degrees, tied to very low-wage occupations. These are programs in areas like cosmetology, medical assisting, and some other entry-level health care and business occupations where even the best case scenario outcome for a young adult is somewhere in the $18,000 to $25,000 range. For these programs, the issue may not even be that they're producing exceptionally low earnings compared to expectations, though that certainly can happen. The problem is that what they are expected to do in the first place does not merit the price being charged and the debt used to finance it. That's why in many cases the programs that fail could almost have almost been predicted ahead of time.

To be clear, the issue is not that all programs fall into this problem, it's that too many are associated with low expected outcomes.

Look for example, at the earnings for certificates. The weighted average annual earnings by number of completers is $17,309 at for-profit colleges. In 2011, someone making that much money would find themselves about $1,000 above 150 percent of the poverty level for a single person and underneath it if they had more than one person in their household. Someone at that level with student loan debt likely would make minimal payments at best under income-based repayment, since their adjusted gross income minus 150 percent of the poverty level would be close to $0 if not below.

A big chunk of the gainful employment problem is about credentialing.

These results may be shocking, but they shouldn't be. The two largest programs in the data are in medical assisting (weighted average earnings $15,308) and cosmetology ($12,272). Those two program types alone count for nearly one-third of the certificate completers in the data. And they show just the type of expansion mentioned above. In 2000, about 36,000 students earned certificates in medical assisting; by 2012, the annual total was up to 101,000. For cosmetology certificates, the number of graduates increased by 131 percent to nearly 88,000. Over three-quarters of these graduates attended for-profit colleges.

But clearly for-profit colleges can generate pretty good outcomes. Consider post-baccalaureate certificates. There, the weighted earnings are pretty similar across all types of institutions and they're all quite high. And though the results for the bachelor's or other graduate degrees do not have comparison figures at public or private nonprofit colleges, they too show pretty good income.  For those programs, the issue is much less likely to be an earnings problem and maybe some combination of completion or overly high debt levels.

The story is less clear for associate degrees. On the one hand, the weighted earnings are about $10,000 above that of certificate programs. On the other, the level is still below that of certificate completers at public colleges. Part of that is because some of the same programs with poor earnings returns at the certificate level are also offered at the associate degree level. On a completer basis, the two largest associate programs in the data are for medical assisting (a profession that requires no more than a certificate) and culinary arts. The difference is these programs account for only 16 percent of students, allowing some higher paying programs to occupy a larger share.

The Importance of Programmatic Accountability

What these data reinforce is the importance of the programmatic approach of gainful employment given how much results can vary. Consider, Four-D College in Colton, Calif. It had four certificate programs with earnings data: medical assisting; medical insurance coding specialist; pharmacy technician; and licensed practical/vocational nursing. The first four are 8-month certificates, the last is one year. Annual earnings for the first three were $9,965, $10,356, and $10,883, respectively. Graduates from the vocational nursing program, meanwhile, earned $26,855, more than two times the other programs. That program does not have a great default rate, but at 29 percent it's still below the figures of 55 percent, 48 percent, and 41 percent posted by the other three. One has to wonder how many more students might avoid default and earn more money had they simply been shifted into a more valuable credential program at the exact same school?

The programmatic approach is also crucial for righting the problems exposed by the data. When aggregated up, the results look so-so (for-profit colleges, for example have weighted average earnings of $27,290). But a breakdown of the data reveals that we're really looking at multiple stories. On the one hand, are some programs, particularly at the graduate level, that have pretty good earnings. They may struggle with debt, but graduates appear to be doing OK. On the other end are a set of sub-baccalaureate programs with very low earnings, often tied to occupations where one would expect to see low earnings anyway. For these programs, the question is not how much better or worse they are doing than the likely struggling high school dropouts. The issue is are they in programs that will help them earn enough money to justify the program expense and put them in a position for some degree of economic security? For certificate programs, the answer to that question all too frequently appears to be "no."