Youth Apprenticeship Data Framework, high quality full report_compressed.pdf
Acknowledgments
We would like to thank the PAYA National Partners (Advance CTE, CareerWise Colorado, Charleston Regional Youth Apprenticeships at Trident Technical College, Education Strategy Group, JFF, the National Alliance for Partnerships in Equity, and the National Governors Association), as well as the PAYA Accelerator Cohort Grantees for their expertise and insights that informed this toolkit. In particular, we would like to thank Austin Estes at Advance CTE and Hollis Salway at CareerWise Colorado for reviewing the toolkit and offering extensive suggestions.
We would also like to thank Sabrina Detlef for her expert editorial insight; and Julie Brosnan, Riker Pasterkiewicz, Fabio Murgia, and Naomi Morduch Toubman for their help in the design, publication, and dissemination of this toolkit.
Finally, we are thankful for our funders, the Annie E. Casey Foundation, Ballmer Group, Bloomberg Philanthropies, Carnegie Corporation of New York, JP Morgan Chase & Co., Ralph C. Wilson, Jr. Foundation, Siemens Foundation, Smidt Foundation, and the Walton Family Foundation for their generous support of the Partnership to Advance Youth Apprenticeship.
Downloads
Overview of the Youth Apprenticeship Data Framework
Youth apprenticeship is a strategy for building a more inclusive economy by creating affordable, reliable, and equitable pathways from high school to good jobs and college degrees. Youth apprenticeship programs are growing in states and communities across the country, as partnerships of employers, educators, community leaders, and state and local policymakers discover that apprenticeship can expand economic opportunity, while meeting the needs of businesses across a range of industries.
As youth apprenticeship programs develop and expand across the U.S., it is critical that practitioners develop plans for capturing, analyzing, and leveraging data at every stage of program development. Robust data allow state and local leaders to monitor, evaluate and demonstrate the impact of youth apprenticeship programs, to drive continuous improvement practices, and to identify best practices to support improvement, replication, and scale.
After extensive consultation with leading youth apprenticeship practitioners and education and workforce experts, the Partnership to Advance Youth Apprenticeship has developed the Youth Apprenticeship Data Framework. This framework is designed to help program leaders identify relevant, actionable information and data they can collect to document their processes, monitor apprentice progress, and track outcomes for apprentices, employers, and other key stakeholders. Based on stakeholder feedback, the Framework has been organized into three categories–Partnership Data, Pathway Data, and Participant Data–each of which is described in greater detail below.
About the Youth Apprenticeship Data Framework
The Youth Apprenticeship Data Framework is intended to be a practical and adaptable guide to support youth apprenticeship partnerships’ efforts to build data systems that monitor the effectiveness of youth apprenticeship programs and inform continuous improvement activities. The framework can be used as a foundation that supports the data capacity of leaders and organizers working together to identify, develop, and scale up effective, equitable youth apprenticeship programs around the country. Programs should use this Framework to start conversations with partners about what data are being collected and by whom; to identify and address gaps in data collection; to develop and expand data sharing agreements; and to determine what data infrastructure will be needed to improve data capacity as programs expand.
At the same time, however, the Data Framework is also designed to set an aspirational bar for data collection in the field of youth apprenticeship and to establish a consistent vision for how the field defines, measures, and communicates its progress and success. PAYA recognizes that many programs—especially pilots and early-stage programs—will not be ready to collect or access all of the indicators in the Framework. Program leaders should use this framework to identify indicators that are available at their particular stage of program development, and then to determine what resources and actions will be necessary to collect additional indicators as the program evolves. As the field improves its capacity to collect and report on the indicators included in the Data Framework, the evidence base for youth apprenticeship will grow, informing innovation and improvement at scale.
Although this tool has been largely developed for use at the local or regional program level, it can also inform state-level efforts to collect and report youth apprenticeship data, which requires careful alignment across multiple systems. For example, state system leaders can use the Youth Apprenticeship Data Framework to provide guidance to local and regional programs to improve their ability to report consistent, quality data and to ensure program-level data collection efforts are aligned to regional and state priorities for youth apprenticeship.
Navigating the Youth Apprenticeship Data Framework
Based on feedback from youth apprenticeship practitioners and experts from across the fields of education and workforce development, the indicators in the Youth Apprenticeship Data Framework have been organized into three categories:
Partnership Data: These indicators collect information about partner organizations, including information about their engagement in the development, design, and implementation of the youth apprenticeship programs. In addition, indicators in this category provide important context about the communities and systems in which youth apprenticeship programs operate, which can inform programs' efforts to set and track goals for participation and apprentice outcomes that directly address local equity challenges.
Pathway Data: These indicators collect information about the design, structure, requirements, outcomes, and employers involved in youth apprenticeship programs. PAYA recommends this information be collected for each occupational pathway, so program leaders have a clear picture of how program structures, outputs, and outcomes may vary within and across their programs.
Participant Data: These indicators collect participant-level information about the students who apply to and enroll in apprenticeship programs, including apprentice demographics, progress measures, and outcomes. These data will allow program leaders to monitor participation, progress, and completion, and help them take steps to ensure programs expand opportunity and lead to improved education and employment outcomes.
For each category, the Framework includes a table with multiple indicators program leaders should consider collecting to document or monitor partner engagement, apprentice participation and progress, and program design and outcomes. The Framework also includes a definition for each indicator.
The Youth Apprenticeship Data Framework also includes additional information about each indicator:
PAYA Principles for High-Quality Youth Apprenticeship: Each indicator in the Data Framework has been tagged with one or multiple of the PAYA Principles for High-Quality Youth Apprenticeship. While each program is likely to collect its own mix of indicators from the Framework initially, program leaders should ensure they are collecting multiple indicators for each PAYA Principle to monitor and improve alignment to those foundational design principles.
Collection Phase: To help program leaders prioritize data to collect as their programs launch, grow, and evolve, each indicator in the framework has also been labeled with a stage of program development. Programs building partnerships and beginning to design pilot pathways should initially prioritize the collection of the indicators labeled Pre-Launch, many of which relate to key elements of partnership and program development. As programs launch pilots and begin enrolling apprentices, program leaders should build on the information and data collected during the pre-launch phase by collecting indicators labeled Pilot. Finally, as programs move into the later stages of a pilot program (e.g. additional cohorts of apprentices enroll and/or an initial cohort completes the program) and become more established, program leaders should be prepared to collect indicators labeled Established to collect outcomes data and other key information. These phases are intended to be cumulative. In other words, as a program matures and expands its capacity to collect indicators labeled Established, it should continue collecting indicators labeled Pre-Launch and Pilot, as well.
Accountability Requirement: The Accountability Requirement column denotes indicators that are included in federal accountability requirements and, as such, are currently being collected by local and/or state partners. Data that is required to be reported under the Every Student Succeeds Act (ESSA), Workforce Innovation and Opportunity Act (WIOA), Perkins V, and by the Registered Apprenticeship Partners Information Management Data System (RAPIDS) have been labeled in an effort to reduce duplication and streamline collection efforts. (Please note the collecting agencies will vary by state, and therefore have not been listed here. Program leaders should consult education and workforce partners to learn more.)
Readers can explore each category of indicators in the following pages. Each table of indicators can be searched, sorted, and downloaded as a PDF. Below each table of indicators, PAYA has provided a customizable spreadsheet program leaders can download and adapt to facilitate data collection processes informed by this Framework. A PDF version of the full Youth Apprenticeship Data Framework, along with other tools and resources to support program-level data planning, collection, and use, is available in the Appendix section of this report.
Partnership Data
High-quality youth apprenticeship programs are built on strong partnerships that include intermediary organizations, employers, high schools, providers of postsecondary education, and other community partners and leaders.
Indicators in this section of the framework will help program leaders understand and document:
- How partners are contributing to program development and implementation
- How satisfied partners and other stakeholders are with the program and its outcomes
- Student demographics and outcomes of partner school districts and postsecondary institutions
- How apprentice demographics and outcomes compare to those of non-apprentice peers
This information can help program leaders make decisions to strengthen the partnerships that undergird high-quality youth apprenticeship programs, while also gathering information to ensure that they are delivering positive results for employers, apprentices, and core partners.
Download the Partnership Data Collection Tool spreadsheet.
Pathway Data
High-quality youth apprenticeship programs enroll students in on-the-job training, combined with academic learning for specific occupational pathways. This unique combination prepares them to be successful in a full range of postsecondary education options and to begin their careers, broadening their options for the future.
Indicators in this section of the framework will help program leaders understand and document:
- The requirements for enrollment in each pathway
- The journey of a youth apprentice through each pathway, including courses, on-the-job training, and the estimated time to complete the program
- The course credits and credentials required and/or offered within each pathway
- The accessibility and transferability of related instruction credits towards an associate or bachelor’s degree
- The names, types, and size of employers participating in each occupational pathway
- How a program’s apprenticeship pathways are connected to careers that offer family-sustaining wages and opportunities for advancement
This information can help program leaders document and explain the core elements of each occupational pathway, and identify strengths and areas for improvement within particular pathways, or across the program as a whole.
Download the Pathway Data Collection tool spreadsheet.
Participant Data
High-quality youth apprenticeship programs use data to continually monitor and support apprentices’ progress and success, and to assess the degree to which the program is delivering equitable outcomes for individual youth apprentices.
Indicators in this section will help program leaders understand and document:
- The demographics of students applying to and participating in the program
- Hiring patterns by employers in specific occupations and industries
- Apprentices' wage progression over time
- Progress in and completion of different components of the program for each apprentice
- Long-term employment and educational outcomes following the completion of the program for each apprentice
Robust apprentice-level data can help programs identify strengths and gaps in participation, retention, and completion rates—especially those underrepresented by gender, race and ethnicity, or disability status—in youth apprenticeship programs. Participant data should be disaggregated to uncover barriers that may be limiting the progress or success of particular sub-groups of students.
When collecting and sharing participant-level data, program leaders should ensure they and their partners take necessary steps to protect student privacy by complying with local, state, and federal data privacy requirements. PAYA encourages program leaders to initiate conversations about data privacy early, as different partners may be subject to different data privacy requirements that affect their ability to collect and share particular indicators at the apprentice level.
Download the Participant Data Collection Tool spreadsheet.
Conclusion
Youth apprenticeship allows students to complete high school, start their postsecondary education at no (or low) cost, get paid work experience alongside a mentor, and start along a path that broadens their options for the future. Youth apprenticeship programs rely on close collaboration across education institutions, employers, intermediary organizations, and other community stakeholders who share responsibility for ensuring the success of the program and the continuous improvement of the program as a whole.
The Youth Apprenticeship Data Framework is a tool to support program leaders' efforts to identify and collect the information and data they need to effectively document their processes and programs; monitor progress; evaluate the outcomes of apprentices, employers, and other key stakeholders; and engage in continuous improvement efforts. Robust data collection in youth apprenticeship requires alignment and cooperation across multiple partners and systems, each of which will have its own processes and priorities for data. Establishing a shared plan for data collection and data sharing is a complex but critical task for program leaders. The Youth Apprenticeship Data Framework provides direction and a useful foundation for this critical, collaborative process.
At the same time, however, the Youth Apprenticeship Data Framework sets an ambitious standard for the future of data collection and use across the field of youth apprenticeship. As the field continues to grow and improve, this Framework will promote a shared culture around the collection and use of data, and promote programmatic alignment to the PAYA Principles for High-Quality Youth Apprenticeship. With these norms in place across the field, the evidence base for youth apprenticeship will continue to expand and strengthen, informing growth, innovation, and investment in the field over time.
Appendix
Youth Apprenticeship Data Framework – Partnership, Pathway, and Participant Data
Below, you will find a table that includes indicators listed across all three categories of the Youth Apprenticeship Data Framework. A PDF of this table can be downloaded via this link.
Data Collection Tool Spreadsheets
To download, click one of the links above and select the "File" menu. Then select "Download" from the dropdown list to download the file as an Excel Spreadsheet. To copy the file for use as a Google Sheet, simply select "Make a Copy" in the "File" menu. PAYA encourages program leaders to modify and share these files to support data collection and sharing. A Google Drive folder with all three of the Data Collection Tool spreadsheets is available at this link.
Additional Resources from PAYA
The Role of Data and Accountability in Growing Youth Apprenticeship Program (Advance CTE)
Improving Youth Apprenticeship Data Quality: Challenges and Opportunities (Advance CTE)
Sample Youth Apprenticeship Data Sharing Agreement – Coming Soon!
Check back soon for additional tools and resources to support your program's data collection efforts.
Questions?
Please submit any suggested modifications, additions, or technical issues to paya@newamerica.org.