Safe and Drug Free Schools PART Assessment

Over the past few weeks we’ve been looking at the PART on a macro-level. Now it’s time to take a closer look at PART results for an individual program. The Safe and Drug Free Schools program (SDFS) is a block grant that provides funding to states and school districts for drug and violence prevention in schools. In 2006 it was given a “results not demonstrated” rating. Why?

Let’s first examine how SDFS fared on each section of the PART assessment:

Program Purpose and Design – SDFS Score 60%

The program received full credit for three of the five questions in this section – whether the program’s purpose is clear, whether the program addresses a specific problem, and whether the program is duplicative of other programs.

It received no credit for two questions. The PART assessment found that SDFS has a design flaw that limits its effectiveness. Because SDFS is a block grant, funding is thinly distributed across all 50 states. As a result, “two-thirds of all school districts receive allocations of less than $10,000, amounts typically too small to mount comprehensive and effective drug prevention and school safety programs.” Though the PART points out the design problem, at the end of the day, this is really a funding issue.

The PART assessment also found that SDFS does not target resources effectively. States allocate funds based on Title I funding and enrollment, not on the prevalence of drugs or violence in specific districts or schools.

Strategic Planning – SDFS Score 62%

SDFS received full credit for five of the eight questions in this section, including whether the program has annual and long term performance measures. However, SDFS received no credit for “ambitious targets and timeframes” for either the annual or long-term measures. While SDFS does have targets for five of its seven goals and is currently conducting research to set targets for the final two, their assessment is not complete.

SDFS also lost points because its budget requests are not explicitly tied to the accomplishment of performance goals and are not derived from estimates of what is needed to achieve these outcomes. As discussed in a prior post, it is unclear whether any program’s budget requests are really tied to performance goals. This criticism is not one unique to SDFS.

Program Management – SDFS Score 78%

SDFS received full credit for seven of the nine questions in this section including whether the program has strong financial management, is coordinated with related programs, engages in oversight activities, and spends funds in a timely manner. SDFS lost credit because it does not collect timely and credible performance information and does not make performance data available to the public in a transparent way. While the explanation indicates that SDFS does collect some performance information, it does not provide baseline data by which to compare current data.

Program Results/Accountability – SDFS Score 8%

SDFS received a “no” or “not applicable” on four out of five questions in this section. The program did receive a small amount of credit for meeting some of its long term performance targets. (Oddly, the program met three out of five goals for both annual and long term performance, but only received credit for the long term section.)

SDFS received no credit for the question pertaining to evidence of effectiveness from independent evaluations. While a 2001 RAND study found the program’s structure was flawed, it did not assess the program’s effectiveness. A new study will assess whether grantees use research-based prevention programs and is implementing them according to the research. Still, it’s not clear whether this study will ultimately assess the program’s effectiveness.

Most Department of Education Programs fare the worst on this section. Only two programs out of 93 received 100 percent in this section, and only 17 of the 93 programs scored better than 50 percent.

The PART Score and Improvement Plans

The scores from the four sections are weighted to produce a final score. Program Purpose makes up 20% of the total score, Strategic Planning is 10%, Program Management is 20%, and Program Results is 50%. When programs, such as SDFS, lack annual or long term performance goals, baseline information, or performance data, they are rated “Results Not Demonstrated.” Almost half of the Department of Education’s programs receive this rating.

Each program rated by the PART receives an improvement plan. SDFS’s plan includes providing training to states on the quality and use of data, posting performance data, and working with Congress to authorize a more effective program.

What did we learn from the Safe and Drug Free Schools’ PART Assessment?

• PART can be useful in highlighting issues with program design or targeting. The PART can reveal weaknesses in program design that can be changed by legislation.
• PART can be useful in identifying planning and management issues. The PART can help reveal weaknesses in planning and management that the Department can address.
• There is limited opportunity for partial credit or work that is in progress. The PART takes a snapshot of a program at a specific time, but does not capture ongoing work or give partial credit for progress.
• Education programs often score poorly on the Results section, which makes up 50 percent of the PART score. Many Department of Education programs do not have rigorous evaluations or performance information on annual and long term measures as required by this section. The 93 education programs with PART assessments score an average of 25 percent in the Results section (compared to 63 percent in Planning, 73 percent in Management, and 84 percent in Program Purpose and Design).

Up next in the Ed Money Watch PART Series: The Child Care Access Means Parents in Schools (CCAMPIS) PART assessment.