Core Indicators, Reports, Negotiated Performance Targets, and Perkins Funds Regional Workshops

1
Core Indicators, Reports,Negotiated Performance
Targets,and Perkins FundsRegional Workshops
Carl D Perkins (Perkins IV) Career and Technical
Education Act

• Chuck Wiseley
• Spring 2008

2
Agenda

• Why are we here why do we care?
• Perkins IV
• Legislative requirements
• State Local requirements
• Negotiating Meeting Targets
• Not meeting targets targeting funds
• Perkins funds - improving student success.
• Core Indicators, Reports, Performance Targets
• Setting Targets Perkins IV Local Application.

3
Session 1 Goals

• Perkins funds - improving student success.
• What are the Rules?
• Negotiating Meeting Targets
• Tools for analysis
• Not meeting targets targeting funds

4
Session 2 Goals Objectives

• Update on development
• Core Indicators, Reports, Performance Targets
  Perkins IV Local Application.
• Participants will
• View/Review Perkins IV Indicators Reports
• what meaning can be derived
• changing performance with Perkins funds
• Discuss and Set Proposed Targets
• negotiated targets (who when)
• strategies for negotiating targets.
• strategies for program improvement,
• strategies for getting buy-in
• Other necessary topics for inclusion

5
2004 Crystal Ball

• 109th Congress
• Some changes in Accountability
• Strengthened
• Common measures
• Wage gain
• Negotiated performance State Local

6
Perkins IV

• Carl D. Perkins Career and Technical Education
  Improvement Act of 2006
• P.L. 109-270
• August 12, 2006.
• 2007-8 Transition Year
• 2008-9 First Year under the Act

7
Negotiating Targets

• Perkins III
• State negotiates targets USDE
• Perkins IV
• State negotiates targets USDE
• Targets for 1 year and then 2 years (3rd 5th)
• Locals either
• Accept those targets
• Negotiate local targets
• Included in Local Plans
• Targets for next year

8
Sanctions in Law

• Perkins III
• State
• Bundled WIA ESEA
• Improvement plan
• Two years below, Secretary MAY withhold funds
• Perkins IV
• State Locals
• 90 of Perkins Targets
• Improvement Plan
• Year 2 no improvement, lt90 for 3 years for any
  single target
• Feds or State SHALL provide technical assistance
• MAY withhold funds (all or part)

9
CCC Sanction PlanMeeting Performance Targets

• Application is Improvement Plan
• Accountability Determination
• In compliance at or above 90
• Expenditures need not be linked to the areas with
  the largest performance gaps
• Focused Improvement Status - lt 90
• Focused Improvement section
• Year 2 No Improvement Diagnostic Study
• Programs funded must address individual program
  performance gaps.
• Priority Focused Improvement 3rd Yr lt 90
• Diagnostic Study and Action Plan

10
Priority Focused Improvement

• Focused Improvement section
• Diagnostic Study
• Action Plan
• Expected outcomes,
• the Perkins and other funding dedicated to each
  strategy, and
• the implementation dates expected for each
  strategy.

11
Odds of being in

• Focused Improvement?
• Depends on how well the negotiation goes.
• Improving after Focused Improvement?
• Depends on last year.
• Priority Focused Improvement?
• Depends on how well the negotiation goes.
• Unlikely to occur?

12
The Negotiation Process

• Negotiation Workbook
• Online
• Tables
• Charts
• Negotiation Worksheet
• Local Application

13
Negotiation Tables
Regional Workshops
14
Mention Office 2003 v 2007
15
Problem? 90 x 55.6 50
What has occurred? Outcomes Diagnostics
16
Faupl
17
Local Application District
18
(No Transcript)
19
Perkins IV funds

• Improving student success.
• Nine Requirements
• 18 Permissive Uses
• Should address performance gaps
• Perkins IV Local Application.
• Improvement Plans
• Negotiating Meeting Targets
• Progressive Oversight, Improvement Plans, and
  Risk Monitoring

20
With or Without Sanctions

• Basic criteria for expending Perkins funds
• Expenditures must
• Meet the purpose of the Act
• Be necessary and reasonable
• Expenditures may not be used for
• General purposes
• Maintenance of existing programs

21
What federal rules apply?

• Carl D. Perkins Career and Technical Education
  Act (Perkins IV)
• EDGAR (Education Department General
  Administration Regulations)

22
What federal rules apply? (Continued)

• OMB Circulars (Office of Management and Budget)
• Circular A-87 Cost Principals State and Local
• Circular A-21 Cost Principals Education
  Institutions
• Circular A-133 Single Audit Requirements
• Circular A-102 Grants and Cooperative Agreements
  with State and Local Governments
• Circular A-110 Uniform Administration
  Requirements for Grants and Agreements with
  Institutions of Higher Education, Hospitals and
  Other Non-Profit Organizations

23
What types of costs are generally considered
eligible?

• Administrative Costs (5)
• Personnel Services (time records)
• Operating Expenses
• Stipends
• Consultants
• Instructional Materials
• Travel
• Instructional Equipment

24
What costs are NOT eligible?

1. Student expenses or direct assistance to students
   
2. College tuition, fees, books
3. Entertainment
4. Awards and memorabilia
5. Individual memberships
6. Membership with orgs. that lobby
7. Fines and penalties
8. Insurance/self-insurance

• Expenses that supplant
• Audits except single audit
• Contributions and donations
• Contingencies
• Facilities and furniture
• General advertising
• Alcohol
• Fund raising
• General administration

25
What are the 9 requirements for Perkins IV
expenditures?
Funds made available to eligible recipients under
this part shall be used to support career and
technical education programs that-

1. Strengthen academic technical skills of
   students
2. Link secondary and postsecondary CTE programs (at
   least one program of study)
3. Provide students with strong experience and
   understanding all aspects of an industry (WBL)
4. Develop, improve, expand use of technology
5. Professional development

1. Evaluate programs with emphasis in meeting needs
   of spec. pops.
2. Initiate, improve, expand and modernize quality
   programs
3. Provide services activities of sufficient,
   size, scope and quality
4. Provide activities to prepare special pops. for
   high skill, high wage, or high demand occupations
   leading to self-sufficiency.

26
20 permissive uses of funds

• Involve parents, business and labor in planning
  operation
• Career guidance academic counseling
• Business Partnerships - Work-related experience
  students faculty
• Programs for spec. pops.
• CTE student organizations
• Mentoring support services
• Upgrading equipment
• Teacher prep. programs
• Improving and developing new CTE courses
  including distance ed.
• assist transition to BA degree programs

• support entrepreneurship education
• initiatives for secondary students obtaining
  postsecondary credit to count towards an AA/AS or
  BA/BS degree
• support small CTE learning communities
• Family consumer sciences
• Adult CTE programs
• Job placement programs
• Support Nontraditional activities
• Automotive technologies
• Pooling funds
• Teacher prep,
• data accountability,
• assessments
• Support other CTE programs

27
Effects of not achieving targets

• Above 90 of targets
• Freedom to fund any program improvements
• Encouraged to address low performance
• Below 90 - year 1
• Focused Improvement section
• Requires some analysis to write the narrative
• Must address low performance in funded Programs
• Encouraged to start a diagnostic study

28
Effects of not achieving targets

• continued...
• Below 90 - year 2 or no improvement
• Focused Improvement section
• Diagnostic study
• Propose Effective Practice solutions
• Probably a more district level approach to
  expenditures

29
Effects of not achieving targets

• continued...
• Below 90 of target - 3 years
• Focused Improvement section
• Submit Diagnostic study w Action Plan to CO
• Action Plan
• Proposed Effective Practice solutions
  Implementation dates
• Probably an even more district level approach to
  expenditures
• Risk Monitoring

30
Break time

• I am the messenger
• dont kill the messenger
• Questions
• 20 minute break

31
Data Why?

• Funding
• Outcomes Analysis
• Telling the story

32
Defining the Data

• SAM Codes
• TOP Codes
• Data Elements
• Core Indicators
• The Law
• Definitions
• Negotiated Performance Targets
• Measurement Approaches/Formulas

Funding
33
Student Accountability Model (SAM) Taxonomy of
Programs (TOP)

• Priority A - Apprenticeship
• Must have the approval of the Division of
  Apprenticeship Standards
• Priority B Advanced Vocational
• Used sparingly, no more than two courses in any
  one program
• B level courses must have a C prerequisite in
  the same program area
• Priority "C" Clearly Occupational
• Generally taken in the middle stage of a program,
  detracts "drop-ins."

34
Student Accountability Model (SAM) Taxonomy of
Programs (TOP), Continued

• Priority "D" Possibly Occupational
• Taken by students in the beginning stages of
  their occupational programs
• Can be survey course
• Priority E Non-Occupational
• Vocational Flag on TOP code
• Designed to identify vocational Programs for
  federal reporting () - see Taxonomy of Programs,
  Sixth Edition, July 2007

35
Data ElementsMIS System

• Students, Courses, Degrees, Services
• Student VTEA Data Elements
• Economically Disadvantaged
• Single Parent
• Displaced Homemaker
• Cooperative Work Experience Education
• Tech Prep
• Migrant Worker

New 2 Years
36
Accountability Requirements Section 113(b)

• 5 core indicators of performance
• Student attainment of technical skill
  proficiencies
• Student attainment of credential, certificate, or
  degree
• Student retention in postsecondary education or
  transfer
• Student placement in military, apprenticeship, or
  employment
• Student participation/completion of
  non-traditional training
• State and Local adjusted levels of performance
• Levels of performance negotiated with USDE /
  State
• Results reported annually

37
Perkins III (1998)Core Indicators

• Skill AttainmentStudent Achievement Successful
  course completion
• Academic
• Vocational and Technical
• Completions Student program completion
  Certificates Degrees
• Placement and (3b) Retention Placement and
  retention in employment
• Equity -- Non-Traditional Employment
  Participation/Completion of non-traditional
  training
• 4a - Participation
• 4b Completion

38
Core Indicators

1. Technical Skill AttainmentSuccessful CTE course
   completion
2. Completions Program completionCertificate,
   Degree Transfer Ready
3. Persistence TransferStudent persistence in
   Higher Ed
4. Placement Placement in apprenticeship,
   employment, military, fed gov
5. Equity -- Non-Traditional Employment
   Participation (5a)/Completion (5b) -
   non-traditional training

39
Cohort Definitions Used for Measurement

• Participant NT Participation
• Perkins III -Any enrollment in a CTE course (TOP)
• Perkins IV concentrator using assigned major
• Concentrator All other indicators
• Cohort of students enrolled during the cohort
  year and
• Successfully completed at least one course in the
  middle or end of a program (SAM A-C) and 12
  vocational units within a single discipline (two
  digit TOP) or
• Program completion as indicated by receipt of ANY
  vocational credit certificate or degree
• Leavers Not enrolled in year following cohort
  year
• 2P1 - Completions
• 4P1 - Placement (Employment)

40
Assigning a Program to a Student

• Award TOP code of CTE Certificate or Degree
• Concentrators
• Hierarchy based on SAM Priority code
• Assigned to the TOP where most CTE units occurred

41
What is not decided

• Concentrator
• How long to accumulate 12 units in a two digit
  TOP (discipline)
• Two, three, or five years
• Completions
• Transfer Ready or Prepared
• Employment
• Which period for employed?
• Feds want quarter 2 - Oct-Dec
• Any of 4 quarter
• NT Participation
• CRCC

42
Negotiating Targets

• Perkins III
• State negotiates targets USDE
• Perkins IV
• State negotiates targets USDE
• Targets for 1 year and then 2 years (3rd 5th)
• Locals either
• Accept those targets
• Negotiate local targets
• Included in Local Plans
• Targets for next year
• Negotiations complete by May 15

43
Sanctions in Law

• Perkins III
• State Bundled with WIA, ESEA, Adult Ed
• Improvement plan
• Two years below, Secretary MAY withhold funds
• Perkins IV
• State Perkins each indicator
• 90 of Target
• 3 years for any single target
• Locals
• 90 of Targets
• Improvement Plan
• Year 2 no improvement, 3 years lt90 for any
  single target
• State SHALL provide technical assistance
• State MAY withhold funds (all or part)

44
CCC Sanction PlanMeeting Performance Targets

• Application is Improvement Plan
• Accountability Determination
• In compliance at or above 90
• Focused Improvement Status - lt 90
• Focused Improvement section
• Year 2 No Improvement Diagnostic Study
• Priority Focused Improvement 3rd Yr lt 90
• Diagnostic Study and Action Plan

45
Timeline for Outcomes Outputs

• Negotiating Performance 2008-09
• Spring 2008
• Cohort Year (2006-07)
• 1 yr for outcomes (2007-08)
• Transfer
• Persistence
• Employment
• Spring 2009
• Outcomes have already occurred
• Target low performance now!

46
Local Application District
47
Spring 2008 Sessions

• 7 Regional Sessions for District/College Teams
• Content so far from this session
• TOP SAM Codes
• Data Elements
• Core Indicators
• The Law
• Definitions Negotiated Performance Levels
• CCC Strategies
• CI Measurement Approaches/Formulas/Targets
• Reports
• Negotiation Strategies
• Compliance Local App
• Program Improvement

48
Core Indicators Targets

1. Technical Skill AttainmentSuccessful CTE course
   completion
2. Completions Program completionCertificate,
   Degree Transfer ready
3. Persistence TransferStudent persistence in
   Higher Ed
4. Placement Placement in apprenticeship,
   employment, military, fed gov
5. Equity -- Non-Traditional Employment
   Participation (5a)/Completion (5b) -
   non-traditional training

49
Core Indicator 1Technical Skill Attainment

• Successful Course Completions
• Technical Skill Proficiencies
• Vocational (CTE) Courses
• SAM A-C
• Vocational TOP
• G.P.A. Concentrators
• Grade reports (moved to Data Mart)

50
Core Indicator 1 Measurement Performance
Levels
SAM A - C Courses Students with GPA gt
2.00 Students with Grades A - F
Year Target Actual 2002-03
77.76 84.01 2003-04
78.76 84.73 2004-05
79.76 84.70 2005-06
84.02 83.69 2006-07 83.70 ??.??
99.6
Based on Perkins III data.
51
Core Indicator 1 Measurement Performance
Levels (Analysis)
SAM A - C Courses Grades A-C Cr
Grades AF, Cr-NoCr, W, I
52
Core Indicator 2Program Completions

• Completers (numerator)
• Award in Current Year
• AA/AS degrees
• Certificates
• Transfer Ready (or Prepared)
• or Equivalent
• Leavers Completers (denominator)
• Left system (college) for one year and/or
• Award in Current Year
• AA/AS degrees
• Certificates
• Transfer Ready

53
CI 2-Completions Measurement Performance
Levels
Certificate/Degree/Transfer Ready All
Concentrators (Leavers Completers)
Year Target Actual 2002-03
59.82 76.36 2003-04
59.82 74.89 2004-05
60.82 73.42 2005-06
73.00 73.73 2006-07 73.00 ??.??
Based on prior indicator definition.
54
Core Indicator 3 Persistence Transfer
New

• Concentrators who were not leavers in the year
  following the cohort year
• Or
• Transfers (CSU/UC/Clearinghouse)
• All Concentrators

55
Core Indicator 4 Placement

• Placement
• UI wage file match 1st year out
• Apprenticeship, Military, Fed Gov
• Minus Continuing in Four Year Institution
• CSU, UC, Clearinghouse
• Not resolved
• Four quarters in Academic Yr after leaving
• Quarter 2

56
CI 4 Placement Measurement Performance Levels
Leavers Completers in UI covered employment or
Apprenticeship, Military, Fed Gov All Leavers
Completers
Year Target Actual 2002-03 83.19 84.08 2003
-04 83.19 82.53 2004-05
83.19 82.07 2005-06 82.07 81.76
98.6599.62
Based on prior indicator definition.
57
Core Indicator 5Gender Equity

• Training leading to Nontraditional Employment
• (e.g., Men in Nursing Women in Auto)
• 75 / 25 from 2000 census employment data
• Job codes (SOC) mapped to 2000 Census data
• SOC codes mapped to CIP (USDE)
• NAPE developed Nontraditional CIP table
• CIP codes mapped to TOP (CCC)

58
Core Indicator 5Gender Equity

• In Programs leading to Nontraditional Employment
• Nontraditional Gender Students
• All Students in NT Program

59
CI 5a Participation Measurement Performance
Levels
Nontraditional participants enrolled in a
nontraditional TOP Code All participants
enrolled in a nontraditional TOP Code
Year Target Actual 2002-03
27.98 36.98 2003-04
28.98 36.80 2004-05
29.98 35.31 2005-06 35.31 37.16
60
CI 5b Completion Measurement Performance
Levels
Nontraditional completers of nontraditional
programs All completers of nontraditional
programs
Year Target Actual 2002-03 26.95
24.42 2003-04 26.95
24.21 2004-05 25.05 23.25 2005-06
23.25 25.66
61
CI 5b Completion Measurement Performance
Levels
Nontraditional completers of nontraditional
programs All completers of nontraditional
programs
Year Target Actual 2002-03 26.95
24.42 2003-04 26.95
24.21 2004-05 25.05 23.25 2005-06
23.25 25.66
90.6 89.8 92.8
62
Do these indicators really measure what education
in CCC does?
63
Do these indicators really measure what education
in CCC does?

• They are only indicators.

64
Are they meaningless?
65
Are they meaningless?

• No. Student success matters.

66
How can we use the indicators to improve student
success?
67
How can we use the indicators to improve student
success?

• Understand What is happening to students.

68
How can we use the indicators to improve student
success?

• Understand What is happening to students.
• Why is it happening?
• SLOs, CATs, Research, etc.

69
How important is setting an appropriate target ?
70
Accessing the Data

• Access
• Report Formats
• Report Structures
• Report Contents
• Report Examples

71
Reports

• New reports
• Developed for negotiations
• District and College Level
• Special Pops detail
• 8 years history
• Final version mid February 2008
• All Reports
• Easier access Excel or SQL reports
• ??? Late February 2008 ???
• Progressive scrutiny on use of reports
• Below 90 of Targets in

72
Access

• http//www.cccco.edu
• Important Documentation
• Quick Reference
• Negotiation Report Instructions
• System Documentation
• System Requirements
• No more Installing plug-in
• No Administrative rights requirements

73
Report Formats

• Print Version
• For administrators, faculty, students PIOs
• Printing from the WWW
• Local Applications and Final Reports
• For exporting to
• word processor
• spreadsheet
• Data Download Version
• For Researchers, IS Technophiles
• CSV For Export to
• Excel
• SPSS, SAS or other Stat Package
• BRIO
• Access or other database software

74
Data Download
75
Negotiation Workbooks

• Instructions
• Counts
• Percentages
• Charts
• Rates
• Counts
• Data work tables

76
Negotiation Tables
77
Negotiation Charts
78
(No Transcript)
79
Report Structures

• Negotiation Workbooks
• Forms
• Perkins IC - Local Application
• Perkins IC - Final Report
• Summaries
• All five Indicators on one page
• Answer Sheet Style
• Detailed
• Percents and counts
• Detailed breakouts for each Indicator component

80
CI Report Contents

• CI Reports
• State, District, College
• 2-4-6 digit TOP
• Special Populations breakouts
• Identified on each report
• Performance below negotiated levels
• Performance below is highlighted
• Low numbers (denominator lt 10)
• Blue-italics

81
CI Contents (continued)

• Summary
• College, District State
• Identifying Low Performance Low Numbers
• Comparing Performance across Programs
• Detailed
• Program Area (6 digit TOP)
• Three Year Trends
• Identifying Low Performance areas
• Comparing Performance within Programs

82
Report Examples

• Final Report Form
• College by TOP Form
• Summary College
• Summary - 5 digit TOP
• Summary Detail
• Core 2 Completions
• Core 2 - 5 digit TOP
• Course Voc Status/SAM Code

83
Negotiation Report Contents

• District, College?
• Special Populations breakouts
• Numerators, Denominators, Rates
• Charts for analysis

84
College Local App -Final Report Form
85
Dist Local App - Final Report Form
Regional Workshops
86
Final Report Form
87
College by TOP Form
88
Summary - College
89
Summary College 5 Digit
90
Summary - Detail
91
Core 2 - Completions
92
Core 2 - Completions 5 digit
93
Course Voc Status/SAM Code
94
Using the Data

• Negotiate or accept state-adjusted levels
• Trends over time for your college
• By program, special population, Tech Prep
• Workshops, Committees Faculty Information
• Circulate to faculty for their use input
• Factor in allocation decision-making for Local
  Plans
• Supplement MIS data with your own knowledge and
  follow-up of students

95
Practice using the Data

• Negotiation Exercise
• Review FAUPL Worksheet
• Walk through common scenarios
• Review the pre-printed report for a sample
  college
• Conduct a guided practice using first indicators
• Negotiate or Accept State Targets

96
Faupl
97
Negotiation Workbook
98
Discussion

• Improving Session Content
• Order Negotiations earlier or later
• Detail
• Strengthen funding ties or are sanction threats
  sufficient?
• Other?
• Resources JSPAC, IWITTS, NAPE
• Chuck Wiseley, cwiseley_at_cccco.edu, 916.327.5895