Managing for Results CommunityBased Abstinence Education Grantees Conference - PowerPoint PPT Presentation

1 / 31
About This Presentation
Title:

Managing for Results CommunityBased Abstinence Education Grantees Conference

Description:

Budget submissions include PART recommendations for program and ... Abstinence Education might be PARTed this year... Page 14. Research and Program Evaluation ... – PowerPoint PPT presentation

Number of Views:50
Avg rating:3.0/5.0
Slides: 32
Provided by: acf75
Category:

less

Transcript and Presenter's Notes

Title: Managing for Results CommunityBased Abstinence Education Grantees Conference


1
Managing for Results Community-Based Abstinence
Education Grantees Conference
Vijaya ChannahSorah, Ph.D. ACF
Performance Management Operations
2
Managing for Results
  • Uses logic modeling to relate
  • gt tracking program implementation
  • gt measuring outputs and outcomes
  • gt budgeting
  • gt planning research/evaluation priorities
  • The purpose is to improve program outcomes and
    service to program recipients.

3
The World a few Years ago
  • Focus on incremental budgeting
  • traditional management
  • Focus on reporting processes and outputs

4
Traditional Budgeting
  • Focus on inputs, such as increased funding,
    rather than outcomes
  • Changes in inputs at the margin (for example, X
    more dollars than last year will fund x more
    grants)

5
The World of Managing for Results
  • Strategic Planning
  • Performance Budgeting
  • Performance Measurement / Reporting
  • PART
  • Presidents Management Agenda (PMA)
  • Perform. Accountability Rept.
  • Research / Program Evaluation
  • Logic Modeling to link
  • all these (how the puzzle fits) !

6
A New Way of Budgeting
  • Budget/performance information is integrated
    for the first time in 2006 into the
    Congressional budget request
  • Budget justifications include discussions of
    goals, measures, and performance
  • Budget submissions include PART recommendations
    for program and management improvement and how
    those findings influenced budget decisions
  • Example 500,000 was included in LIHEAP request
    for feasibility study for national evaluation
    (PART recom.)

7
What Does this Mean forProgram Funding?
  • Bottom Line Program effectiveness may affect
    budget decisions
  • Reduced funds, or in some cases no funding, might
    be requested for poor performers
  • High performing programs might have a better
    opportunity to garner additional funds
  • Example Refugee program demonstrated
    significant improvements in refugee employment
    which reinforced ORR request for 68M

8
PERFORMANCE INFORMATION
  • Key outcome performance measures for programs
  • Output measures
  • Key efficiency measures
  • Program Assessment Rating Tool results
  • Research and Evaluation

9
How Performance Information is Integrated into
Budget Request
  • Performance information (including PART results)
    is now included in
  • - the Overview in the Budget Request and
  • - each programs section of Budget Request
  • ALL the ACF detailed Performance Information
    (tables, data issues, etc.) included in the
    performance budget as well.

10
Challenges in Performance Management
  • Defining efficiency for human services programs
    (e.g., abstinence education)
  • Defining outcomes (e.g., family violence
    prevention)
  • Timely, reliable data

11
Program Assessment Rating Tool (PART)
  • 25 questions about
  • Program Purpose Design (20)
  • Strategic Planning (10)
  • Program Management (20)
  • Program Results (50)
  • And approximately 5 supplemental questions
    tailored to the type of program, e.g., grant,
    direct service delivery, research, etc.
  • Ratings can be Inadequate, Results not
    Demonstrated, Adequate, Moderately Effective, or
    Effective

12
Program Assessment Rating Tool
  • PART experience 22 of our ACF programs have
    participated in PART
  • FY 2004 3 programs
  • Head Start, Foster Care, Refugee Resettlement
    (Social Services and Targeted Assistance Formula
    Grants)
  • Scores ranged from 37 to 68 (RND to Adequate)
  • FY 2005 5 programs
  • CSBG, LIHEAP, Child Support, ADD, RHY
  • Scores ranged from 35 to 93 Foster Care was
    reassessed, increased from 33 to 54
  • FY 2006 6 programs
  • AFI, CAPTA, CBCAP, Child Care, Indep Living, VCR
  • Scores ranged from 39 to 81 (Ratings RND to
    Moderately Effective)

13
Program Assessment Rating Tool
  • FY 2007 8 programs
  • - ORR/TAMS - effective
  • - ORR/Human Trafficking, Adoption
    Assistance, TANF - moderately effective
  • - Adoption Incentives Adoption
    Opportunities adequate
  • - Mentoring Children of Prisoners SSBG
    results not demonstrated
  • Abstinence Education might be PARTed this year

14
Research and Program Evaluation
  • Outcome measurement can show
  • Trends in outcomes
  • Comparisons between actual and desired outcomes
  • Rigorous evaluation can show
  • Outcomes relative to what they would have been in
    the absence of the program
  • The programs causal contribution to the observed
    outcome

15

16
Overview of Logic Models
  • A logic model tracks how we get from our
    challenges to our solutions and desired long term
    outcomes

CHALLENGE
INPUTS
ACTIVITIES
OUTPUTS
DEMOGRAPHICS
LONG TERM OUTCOMES (GOALS)
OUTCOMES
EXT. FACTORS
CONSTRAINTS
ASSUMPTIONS
17
Logic Model Defined
  • What is a Logic Model?
  • A graphic representation of a program.
  • It shows what the program is designed to
    accomplish, including the services it delivers,
    expected results of those services, and the
    linkages between services and program goals.

18
Logic Model Formal Definition
  • A logic model is a picture of how your
    organization does or plans to do its work it
    links outcomes (both short and long term) with
    program activities/processes and the theoretical
    assumptions/principles of a program

19
Logic Models
  • can go both ways
  • X -------------------------------------------
    --?Y
  • (e.g., how can we make a raft float?)
  • X lt- ----------------------------------------
    ----Y
  • (e.g., why did the raft sink?)

20
Logic Model Uses
  • Use for program
  • Design
  • Budgeting
  • Implementation
  • Evaluation
  • Communication
  • Marketing
  • So a logic model can be used to help plan a
    program, to help manage a program, and to guide
    the evaluation of a program.

21
Logic Model Construction Process
  • Model Development
  • Think about what it is that you are really trying
    to do
  • Establish the scope and context
  • Determine challenge(s), outcomes, inputs,
    activities, outputs, and measurement components
  • Create model draft
  • Express relationships among/between key
    components
  • Determine evaluation needs/points using dotted
    line arrows (solid arrows show known
    relationships)

22
Logic Model Development
  • 1. Identify CHALLENGE/Social ill What do we want
    to improve in the population?
  • Out of wedlock births
  • Relationships before marriage
  • Diseases (STDs)
  • Challenges are often expressed as statements of
    fact, based on empirical data/statistics.
  • There may be multiple challenges addressed by a
    single program.

23
Logic Model Development
  • 2. Identify LONG-TERM OUTCOMES The ultimate end
    goals of your program, how your service
    population will look after your interventions
    have taken place
  • Decrease out of wedlock births
  • Increase proportion of abstinent youths
  • Decrease preventable disease (STDs)
  • Long-term outcomes/goals (as well as all other
    outcomes) are usually expressed as changes you
    will use words such as improved, increased,
    decreased, etc.
  • Ultimate long-term outcomes/goals are sometimes
    pie in the sky or utopian.

24
Logic Model Development
  • The remainder of the logic model elaborates how
    we get from the CHALLENGE to the LONG TERM
    OUTCOMES.

CHALLENGE
INPUTS
ACTIVITIES
OUTPUTS
DEMOGRAPHICS
LONG TERM OUTCOMES (GOALS)
OUTCOMES
EXT. FACTORS
CONSTRAINTS
ASSUMPTIONS
25
Logic Model Development
  • 3. Identify Inputs/Resources (personnel, funds,
    laws/regulations, creative ideas, etc.)
  • 4. Identify Activities
  • Train the trainer
  • Delivering abstinence education in churches,
    community centers, schools

26
Logic Model Development
  • 5. Identify Outputs, such as number of trainers
    trained, number of training courses developed and
    administered.
  • 6. Identify/develop Key Outcomes and Measures
  • Decrease the rate of births to unmarried teenage
    girls ages 15-19 (35.4 in 2002) Target 35 in
    2003
  • Decrease the proportion of youth ages 15-19 who
    have engaged in sexual intercourse (46.7 in
    2003) Target 45.5 for 2004
  • and set Targets

27
Logic Model Development
  • 7. Determine where program evaluation/research
    needs to take place (and depict arrows
    accordingly)
  • - - - - - - gt plausible causal
    relationship (or desired effects/results)
  • ----------? known causal
    relationship (based on scientific
    research/program evaluation)

28
Logic Model Development
  • Complementary Tools
  • Strategic plans
  • Flowcharts
  • Process diagrams
  • Related logic models
  • Performance budgets containing global Agency
    performance measure information

29
Linking ! How does all this relate?
  • Results Act (requires performance measurement
    reporting in the Performance Budget)
  • PMA (requires integrating this performance with
    budget requests and with financial reporting)
  • PART (very detailed analysis of Agency
    management, measurement, reporting)
  • Logic Modeling (gives a clear picture of what
    outcome measures needed, program evaluation
    needed, etc.)

30
In Conclusion
  • ACF is increasingly focusing on results
  • We have lots of information! how do we assemble
    it to help us manage?
  • Use Logic Modeling to track and improve
    performance at Federal, state, and grantee levels

31
Questions / Discussion
  • Questions on any aspect of Managing by Results?
  • Is it clear how all aspects relate to one another
    and how logic modeling can be used as a powerful
    tool?
Write a Comment
User Comments (0)
About PowerShow.com