Appendix A: Methodology

Data Source

Prior to 2016 when the U.S. PIAAC Prison Survey data were released, it was nearly impossible to evaluate college and job training in prisons nationwide because a comprehensive dataset was nonexistent. The 2014 U.S. PIAAC Prison Survey used a nationally representative sample on the prison population to examine the skills of incarcerated adults in relationship to educational attainment and job training while in prison.¹

The U.S. PIAAC Household & Prison² survey instruments collected information from adults over a broad range of abilities from basic reading to complex problem solving. To evaluate the broad spectrum of skills, the survey assessed adults over four domains: literacy, numeracy, problem solving in technology-rich environments, and reading components. The prison survey was administered in both computer and paper-and-pencil modalities. The paper-and-pencil modality only assessed literacy, numeracy, and reading, while the computer-administered survey assessed literacy, numeracy, and problem solving.³ Sixty-one percent of the incarcerated population took the computer assessment, while 37 percent completed the paper-based assessment. Because almost 40 percent of the prison population lack data on problem-solving, this report focuses primarily on literacy and numeracy skills in order to include the entire prison population across both modalities, paper-and-pencil and computer.

Variables

Two methods to report the results of the PIAAC skills assessment include scale scores on a range from 0–500 or proficiency levels within each of the domains.⁴ We used the scale scores to provide the range of literacy and numeracy scores, as well as mean and median scores for the adult population covered by the study. Proficiency levels are the primary method we used to report the skills for both incarcerated adults and the general public, analyzing the percentages of adults who reach the various proficiency levels for each of the literacy and numeracy domains.

Following the same designation of proficiency levels as reported by the Organization for Economic Cooperation and Development (OECD), we provide findings across the five proficiency levels for literacy and numeracy: Below Level 1, Level 1, Level 2, Level 3, and Level 4/5. Below Level 1 and Level 1 are considered low proficiency levels, Level 2 is designated as the basic proficiency level, and Level 3 and beyond are the higher proficiency levels.⁵ See Appendix B and Appendix C for the literacy and numeracy proficiency levels as described by OECD. The appendices describe the types of skills that can be performed at each respective proficiency level. For consistency with the OECD international report, we combine Levels 4 and 5.

Methods

To evaluate the U.S. PIAAC data, sampling weights were used. Sampling weights are used to account for the selection probabilities of the participating adults and to adjust for non-responses within particular groups. Statistical comparisons presented here were based on a two-tailed t-Test with significance at the 0.05 level or exceeding a t-value of 1.96. To analyze complex relationships between reentry status, participation and/or completion in postsecondary education, and/or job training with skill level, linear and logistic regression analyses were used.

Limitations

Selection bias refers to unobservable characteristics among incarcerated adults who participate in postsecondary education and/or job training programs that make them intrinsically different from those who do not participate. In other words, individuals who complete/participate in postsecondary education and/or job training while incarcerated may have certain qualities and motivation to improve their skill level regardless of whether they received the programming. These unobservable characteristics could explain the differences in skill level between the two groups. Because of this, care should be taken when interpreting the results of the effect of correctional programming. In efforts to address issues with selection bias, our statistical modeling includes assumptions that the correctional programming remained constant. Although selection bias is a limitation of the study, control variables were used in statistical modeling to identify statistically significant relationships for participating in correctional programming.

Observations, Interviews, and Focus Groups

This report is primarily a quantitative analysis on the skills of incarcerated individuals, using the U.S. PIAAC Household Survey and Prison Survey. While the data were an invaluable source of information on the skills of incarcerated adults in relationship to educational attainment and job training while in prison, there are limits to what the quantitative data can speak to. Visits to multiple prisons addressed those limitations and incorporated the human aspect by providing context to some of the unexplainable nuances observed in the data. The site visits also ensured a holistic evaluation of these correctional programs. Prison site visits provided an inclusive narrative to explain the availability, participation and interest in, and challenges and barriers to postsecondary education and job training programs for individuals behind bars. Furthermore, these prison site visits provided insight on how these programs are implemented; they were not an attempt to identify best practices or ineffective programs.

To include the voices of those most directly impacted by these correctional programs, we observed, interviewed, and led focus groups at selected federal and state prisons. We collected qualitative data from over 200 individuals, including formerly and currently incarcerated students, federal and state correctional administrators, college and job training programming staff, instructors, college presidents, and family members of currently incarcerated students.

Correctional Facility Selection

Facilities were selected based on meeting three or more of the following criteria: federal or state prison, men’s or women’s correctional facility, postsecondary education program available, job training program available, sites from multiple regions across the country, Second Chance Pell Experimental site, program provided by a two-year or four-year institution, and permission provided to observe an academic and/or job training course.

Facilities were selected based on New America analyst research and conversations with formerly incarcerated students, correctional education administrators, and prominent experts in the field—including federal officials—who suggested some of the selected facilities.

Content of Prison Site Visits

The site visits included informal, semi-structured interviews with the program facilitator(s), currently incarcerated students, college and job training instructors, affiliated college presidents, federal and state department of corrections administrators, and family of currently incarcerated students. Observations of the college and job training courses were conducted as well as tours of the programming facilities. All visits complied with the safety procedures and pre-screening requirements for visitor access. The site visits were conducted by the lead analyst for the project over the course of three months (May through July of 2019).

Anonymity of Correctional Facilities

All correctional facilities remain anonymous in this report. The names of individuals, the programs, correctional facilities, and their locations are intentionally not included in the report because the purpose of the site visits was to add context to provide a comprehensive understanding of college-in-prison and job training programs, not to single out facilities based on best practices or programming processes.