Appendix: Methodology
Varying Degrees was administered by NORC at the University of Chicago using NORC’s AmeriSpeak Panel for the sample source. This was the seventh annual survey seeking to better understand the general population’s opinions about higher education in the U.S. This year’s survey examined attitudes and opinions about a variety of areas, including the affordability of colleges and universities, the individual and societal value of earning postsecondary credentials, and the racial diversity and equity in higher education.
The survey was offered in English and Spanish, and it was administered in two modes, depending on the preference of the respondent expressed during the panel recruitment: (1) self-administered by the respondent online via the World Wide Web; or (2) administered over the telephone by a live interviewer.
A general population sample, 18 years and older, was selected from NORC’s AmeriSpeak Panel for this study. Additional African Americans (referred to as "Black Americans" in this report), Asian/Pacific Islander Americans ("Asian Americans"), and Hispanic Americans ("Latinx Americans") were also sampled in order to hit our targets in these subgroups. The sample for a specific study is selected from the AmeriSpeak Panel using sampling strata based on age, race/Hispanic ethnicity, education, and gender (48 sampling strata in total).
The sample selection accounts for the expected differential survey completion rates across the sampling strata. The size of the selected sample per stratum is determined such that the distribution of the complete surveys across the strata matches that of the target population as represented by census data. If a panel household has more than one active adult member, only one is selected at random. When panelists are selected for an AmeriSpeak survey, the selection process, within each sampling strata, favors those who were not selected in the most recent (previous) AmeriSpeak survey. This selection process is designed to minimize the number of surveys any one panelist is exposed to and maximize the rotation of all panelists across AmeriSpeak surveys.
A small sample of English-speaking AmeriSpeak web-mode panelists were invited for a pretest on Friday, March 3, 2023. NORC collected 57 pretest interviews. The initial data from the pretest were reviewed by NORC and delivered to New America. No changes were made before fielding the main survey. Pretest interviews are not included in the final data.
For the main survey, a sub-sample of AmeriSpeak web-mode panelists were invited to the survey on Thursday, March 23, 2023, in a soft launch. The initial data from the soft launch were once again reviewed to confirm that there were no processing or programming errors. Once reviewed, the remainder of AmeriSpeak panelists were invited to the survey on Monday, March 27, 2023.
In total, NORC collected 1,497 final interviews, 1,422 by web mode and 75 by phone mode. This does not include interviews that may have been removed for data quality purposes (i.e., speeding, high refusal rates, straight-lining). The final collection of survey completers includes specific oversamples of non-Hispanic African Americans (268 of completions), Hispanics (348 of completions), and non-Hispanic Asian/Pacific Islander Americans (223 of completions) to ensure adequate sample size of those groups for analysis. These oversampled groups are weighted down to match their respective proportion in the population.
Summary
General population sample size: 1,497
Margin of error: ±3.46 percentage points
Design effect: 1.86
Fielding period: March 23–May 8, 2023