Methodology

This brief is part of a larger project that attempts to understand the implementation and impact of the TAACCCT investment through a systematic review of third-party grant evaluation reports. The project involves a team of six researchers from Bragg & Associates and New America who have collected and reviewed 220 available evaluation reports available from 256 TAACCCT projects. It has three phases: scanning and scoring, systematic review, and semi-structured interviews with evaluators and grantee institutions.

In the first phase, our team collected 220 evaluation reports available from the four rounds of TAACCCT. Each report was reviewed at least twice and graded by two different team members on three elements: theory of change, implementation analysis, and impact analysis. Fifty-six reports with the top quartile total scores and implementation scores were chosen for the second phase.¹

For this brief, we reviewed those 56 reports again, this time looking for efforts related to improving or expanding PLA practices. We later added one additional report suggested by experts as having substantial PLA policy implementation, bringing the number of reports to 57. We then selected nine reports for further investigation, conducting interviews with their evaluators and grant staff.