APF for Program Improvement Program performance over three years

1
APF for Program ImprovementProgram performance
over three years

• AEFLA Program Directors Meeting
• August 3-4, 2006

2
APF Components (a refresher)

• Enrollment
• Total to projected enrollment
• MIN to total enrollment
• Expenditure per learner hour of instruction
• Relative to similar size programs
• Level completion
• Total levels made to total served
• Overall completion rate to state target
• Learners completing 2 levels to learners
  completing 1
• GED completion
• Pre/post-test

3
Component Weights

• Enrollment 20
• Total to projected enrollment 10
• MIN to total enrollment 10
• Expenditure per learner hour of instruction 25
• Relative to similar size programs
• Level completion 40
• Total levels made to total served 20
• Overall completion rate to state target 15
• Learners completing 2 levels to learners
  completing 1 5
• GED completion 7.5
• Pre/post-test 7.5

4
Standards FY05 Results

• High Performing .95
• Acceptable minimum .70
• FY05 impact on FY06
• 2 programs with .95 scores earned a PAIR pass
• 6 programs with scores lt70 submitted program
  improvement plans and mid-year updates
• FY05 APF results drove decisions about FY06
  program expansions
• FY05 APF distribution informed revisions to FY06
  formula

5
FY06 APF Revisions

• Programs awarded full enrollment points (.10) if
  within /- 5 of enrollment projections
• FY02 expense per learner hour of instruction data
  removed from curve calculation
• ? Completion factor from 40 to 45
• 20 to 22.5 for levels made to levels served
• 15 to 20 for overall completion percentage to
  state target
• ? 2 levels to 1 level from 5 to 2.5
• ? GED completion and pre/post-test factors by
  2.5 each

6
FY06 APF Revisions

• Programs awarded full enrollment points (.10) if
  within /- 5 of enrollment projections
• FY03 expense per learner hour of instruction data
  removed from curve calculation
• ? Completion factor from 40 to 45
• 20 to 22.5 for levels made to levels served
• 15 to 20 for overall completion percentage to
  state target
• ? 2 levels to 1 level from 5 to 2.5
• ? GED completion and pre/post-test factors by
  2.5 each

7
Implications of DAEL Monitoring Visit

• Finding 2 Assessment policy must indicate when
  (after how many hours of instruction) programs
  must post-test learners (and not before)
• ? Level completion targets (10 levels dropped
  from 5-17 pts. 1 level (ASE Low increased 1)
• ? Overall level completion targets (42 for FY07
  compared to 49 in FY06)
• Likely that program post-testing percentages will
  decline (after implementation)
• Finding 3 Colorado must improve local programs
  goal-setting processes significantly to account
  for the actual number of students with goals on
  WIA core outcome measures
• Number of learners with GED, post-secondary ed.,
  and employment goals likely to ?
• Are factors included in APF to encourage
  employment and post-secondary ed.?

8
Decided Possibilities for FY07

• Remove ASE High from pre/post-test factor (not
  part of level completion factor in FY05 and FY06)
  
• Continue to use the APF as the primary measure
  when considering program expansion proposals
• Incorporate the APF into the evaluation of
  program proposals upon reauthorization of WIA
• Add data quality, LIA, and PD factors
• Create a CAESAR APF report and APF calculator
• ? Acceptable Minimum score from .70 to .75