Program Assessment Rating Tool PART

1
Program Assessment Rating Tool(PART)

• Office of Management and Budget

November 2003
2
Presidents Management Agenda

• Government should be results-oriented
  guided not by process but guided by performance.
  There comes a time when every program must be
  judged either a success or a failure. Where we
  find success, we should repeat it, share it, and
  make it the standard. And where we find failure,
  we must call it by its name. Government action
  that fails in its purpose must be reformed or
  ended.
• Governor George W. Bush
• Philadelphia
• June 9, 2000

3
TODAYS AGENDA

• Introduction
• What is the Program Assessment Rating Tool
  (PART)?
• Discussion of Performance Measures
• Discussion of PART Questions and Format

4
Introduction

• The Program Assessment Rating Tool (PART) is a
  component of the Presidents Management Agenda
  element focusing on Budget and Performance
  Integration.
• The PART builds on the Government Performance and
  Results Act (GPRA).
• The PART promotes efforts to achieve concrete and
  measurable results.

5
What is the PART?

• A set of questions that evaluates program
  performance in four areas
• Program Purpose and Design
• Strategic Planning
• Program Management
• Program Results and Accountability
• A tool to assess performance using evidence.
• Provides a consistent approach to evaluating
  programs across the Federal government.

6
What is the PART?

• Revised for FY2005 budget development based on
  comments from agencies, outside organizations,
  and others
• Roughly 25-30 questions explanations and
  evidence are required
• Most questions are Yes/No format
• Section IV questions are Yes/Large Extent/Small
  Extent/No format
• Question weights can be adjusted to emphasize key
  program factors
• Completed by agency and OMB staff

7
PART History

• - FY 2003 Presidents Budget
• OMB published performance ratings for about 130
  programs.
• Good start, but needed refinement.
• - FY 2004 Presidents Budget
• Developed PART to improve assessment system by
  making it more consistent, transparent, and
  defensible.
• Used PART to evaluate performance of 234 programs
  representing about one fifth of federal spending.
• - Goals for FY 2005 Presidents Budget
• Assess the next 20 of federal programs (over 170
  programs, 455 billion).
• Increase use of performance information for
  budgeting.
• Address program weaknesses identified by FY 2004
  PARTs, i.e, ensure follow-up on last years
  recommendations.
• Revisit selected FY 2004 PARTs for re-evaluation.

8
Program Rating Assessment Tool (PART)

• Asks whether
• Long-term and annual goals with measures and
  targets are in place
• Information on performance is being collected
• The data show the program is achieving results.
• Results inform the budget process, but do not by
  themselves determine budget decisions.
• Supports Government Performance and Results Act
  concepts at the program level.

9
Linking the PART to Agencies Budget
Justifications

• Agencies budget requests to OMB and then to
  Congress should highlight PART findings.
• Measures and targets should be included in the
  documents.
• Should discuss PART recommendations
• Budgetary
• Policy
• Legislative
• Management

10
PART Program Types

• Direct Federal
• Competitive Grant
• Block/Formula Grant
• Regulatory Based
• Capital Assets and Service Acquisition
• Credit
• Research and Development

11
PART Guidance

• Available at www.omb.gov/PART
• Role of PART Guidance
• Explains purpose of question
• Explains what is required for a Yes
• Explains what type of Evidence/Data are required
  
• Answers most questions about the PART.
  

12
FY 2005 PART Timeline

• Early March OMB consults with agencies to
  identify
• programs for analysis.
• March Agencies and OMB start work on PARTs
• Mid May Agencies complete PART drafts.
• Late June OMB completes PARTs and provides to
  agencies for budget deliberations.
• July-August Consistency checks (OMB, with
  assistance from the National Academy of Public
  Administration).
• July-August OMB resolves appeals on PARTs.
• September Agency budget submissions to OMB.
• December Budget settlement with agencies.
• December PART summaries finalized by OMB.

13
Performance Measures

• The PART focuses on performance measures because
  they are central to assessing program
  effectiveness.
• Performance measures should capture the most
  important aspects of a programs mission and
  priorities.
• Key issues to consider
• 1) performance measures and targets.
• 2) outputs and outcomes.
• 3) annual and long-term timeframes.

14
Performance Measures

• Performance goals may focus on outputs or
  outcomes.
• Outputs are what the program produced or
  provided.
• Outcomes describe the intended result or effect
  that will occur from carrying out a program.
• The PART strives for measuring outcomes, as these
  are the ultimate objective of the program. When
  outputs are measured, they should support
  outcomes in a logical fashion.

15
Performance Measures

• Strategic goals are broad statements of purpose.
• Performance goals are the target levels of
  performance expressed as a tangible, measurable
  objective, against which actual achievement can
  be compared.
• Performance goals include 1) the performance
  measure and 2) targets and timeframes for the
  performance measure.
• (Performance goal performance measure target)

16
Discussion of PART Questions

• Section I Program Purpose Design (20)
• Section II Strategic Planning (10)
• Section III Program Management (20)
• Section IV Program Results/Accountability (50)

17
Section I Program Purpose Design (20)

• Questions include
• Is the program purpose clear?
• Does the program address a specific and existing
  problem?
• Is the program designed so that it is not
  redundant or duplicative of any other Federal,
  State, local or private effort?
• Is the program design free of major flaws that
  would limit the programs effectiveness or
  efficiency?
• Is the program effectively targeted, so that
  resources will reach intended beneficiaries?

18
Section II Strategic Planning (10)

• Questions include
• Does the program have
• A limited number of long-term performance
  measures that focus on outcomes and meaningfully
  reflect the programs purpose?
• Ambitious targets and timeframes for its
  long-term measures?
• A limited number of specific annual performance
  measures that can demonstrate progress toward
  achieving the programs long-term goals?
• Baselines and ambitious targets for its annual
  measures?
• Are budget requests tied to accomplishment of the
  annual and long-term performance goals, and are
  the resource needs presented in a complete and
  transparent manner in the programs budget?

19
Section III Program Management (20)

• Questions focus on
• Effective management of the program
• Financial oversight
• Evaluation of program improvements
• Data collection
• Accountability of Federal managers and program
  partners (including grantees, contractors, etc.)

20
Section IV Program Results/Accountability (50)

• Questions include
• Has the program demonstrated adequate progress in
  achieving its long-term performance goals?
• Does the program (including program partners)
  achieve its annual performance goals?
• Does the program demonstrate improved
  efficiencies or cost effectiveness in achieving
  program goals each year?
• Does the performance of the program compare
  favorably to other programs, including
  government, private, etc., with similar purpose
  and goals?
• Do independent evaluations of sufficient scope
  and quality indicate that the program is
  effective and achieving results?

21
2004 PARTs held programs to high standards

• 6 of programs effective
• 24 moderately effective
• 14.5 adequate
• 5.1 ineffective
• 50.4 results not demonstrated (lacked adequate
  performance goals or data)

22
Conclusions

• The PART is still in its early stages and has
  some challenges.
• The PART seems to be advancing the consideration
  of performance information in budgeting.
• The PART also provides a framework for developing
  and following up on recommendations.