Appendix: Methodology
Varying Degrees was administered by NORC at the University of Chicago using NORC’s AmeriSpeak® Panel for the sample source. This was the sixth annual survey seeking to better understand the general population’s opinions on higher education. This year’s survey placed an additional emphasis on collecting detailed information about individuals with student loans and how they are managing repayment during the COVID-19 pandemic.
A general population sample 18 years and older was selected from NORC’s AmeriSpeak Panel for this study. Additional African Americans (referred to as "Black Americans" in this report), Asian/Pacific Islander Americans ("Asian Americans"), and Hispanic Americans ("Latinx Americans") were also sampled in order to hit client-specific targets in these subgroups. Lastly, additional sample flagged as likely to be student loan borrowers (using previously collected AmeriSpeak Panel profile data) was included to hit client-specific targets for this subgroup. Sampled panelists in this final subgroup were asked at the beginning of the survey if they had student loan debt for loans in their name or in someone else’s name. In order to be eligible for the survey, these individuals must have answered ‘Yes’ to having student loans in their name or someone else’s name.
The sample for a specific study is selected from the AmeriSpeak Panel using sampling strata based on age, race/Hispanic ethnicity, education, and gender (48 sampling strata in total). Sample selection takes into account the expected differential survey completion rates across the sampling strata. The size of the selected sample per stratum is determined such that the distribution of the complete surveys across the strata matches that of the target population as represented by census data. If a panel household has more than one active adult panel member, only one adult panel member is selected at random. When panelists are selected for an AmeriSpeak survey, the selection process, within each sampling strata, favors those who were not selected in the most recent previous AmeriSpeak survey. This selection process is designed to minimize the number of surveys any one panelist is exposed to and maximize the rotation of all panelists across AmeriSpeak surveys.
The survey was offered in English and Spanish, and it was administered in two modes depending on the preference of the respondent provided during the panel recruitment: 1) self-administered by the respondent online via the Web; or 2) administered over the telephone by a live interviewer.
A small sample of English-speaking AmeriSpeak web-mode panelists was invited on Thursday, March 31 for a pretest. In total, NORC collected 52 pretest interviews. The initial data from the pretest were reviewed by NORC and delivered to New America. No changes were made before fielding the Main survey to collect the surveys interviews used for the final data. Pretest interviews are not included in the final data.
For the main survey, a sub-sample of AmeriSpeak web-mode panelists was invited to the survey on Tuesday April 19 in a soft-launch. The remainder of sampled AmeriSpeak panelists were invited to the survey on Thursday April 21.
NORC collected 2,174 final interviews, 2,093 by web mode and 81 by phone mode, but these numbers do not account for interviews that may have been removed for data quality purposes.
This final collection of survey completers includes specific oversamples of Non-Hispanic African Americans (155 of completions), Hispanic Americans (157 of completions), Non-Hispanic Asian/Pacific Islander Americans (160 of completions), and Student Loan Borrowers (657 of completions) to ensure adequate sample size of those groups for analysis.
Data Collection Procedures
If invited, AmeriSpeak panelists can take the survey online through the password-protected AmeriSpeak Mobile App, the password-protected AmeriSpeak Web portal, or by following a link in the e-mail invitation sent to them. To encourage study cooperation, NORC sent the initial invitation and nine email reminders to sampled web-mode panelists.
To administer the phone survey, NORC dialed sampled panelists who prefer to take surveys on the phone from Thursday, April 21 to Wednesday, May 18. Although most panelists who have stated a preference to take the survey on the phone do take them in that mode, they also have the option of taking the survey online via the web portal or the AmeriSpeak App or can ask the interviewer to e-mail them an invite instead. These rare phone-preferred panelists who end up taking the survey online are coded in the data based on the mode they took the survey, not their previously stated mode preference. Panelists were offered the cash equivalent of $3 for completing this survey.
Design Effect and Sampling Margin of Error Calculations
General Population Sample Size: 1,517 Margin of Error: ±3.47% Design Effect: 1.97