Measurement Standardization in Perkins The Perspective from the Integrated Performance Information (IPI) Project Data Quality Institute June 14, 2005 Bryan Wilson, Ph.D. Associate Director for Policy and Research Washington State Workforce Board

1
Measurement Standardization in PerkinsThe
Perspective from the Integrated Performance
Information (IPI) ProjectData Quality
InstituteJune 14, 2005Bryan Wilson,
Ph.D.Associate Director for Policy and
ResearchWashington State Workforce Board
2
Measurement Standardization in Perkins

• One Criticism from the Office of Office of
  Management and Budgets (OMB) Program Assessment
  Rating Tool (PART)
• Lack of Standardization with Other Workforce
  Development Programs

3
Integrated Performance Information (IPI) Project

• A Nationwide Project to Design a Blueprint for
  States to Create Integrated Performance
  Information for Workforce Development Programs.

4
Integrated Performance Information (IPI)

• What is Integrated Performance Information?
• Consistent information on performance results
  across programs, across levels (from institutions
  to local areas to state), or for programs as a
  system.

5
Workforce Development Programs

• Secondary Career and Technical Education
• Postsecondary Career and Technical Education
• The Employment Service, Workforce Investment Act
  (WIA) Title III
• WIA Title I-B
• Adult Education and Family Literacy, WIA Title II
• Vocational Rehabilitation, WIA Title IV
• Trade Adjustment Assistance Act
• Temporary Assistance for Needy Families Work
  Program
• Apprenticeship
• Other Programs

6
Integrated Performance Information (IPI)
What are the Benefits of IPI? Accountability
to Policy Makers Improved Strategic Planning
Better Research Administrative Efficiency
Better Collaboration Sense of Shared
Responsibility
7
IPI Project

• Supported by the Employment and Training
  Administration of the U.S. Department of Labor
• Teams from Florida, Michigan, Montana, Oregon,
  Texas, Washington

8
IPI Project

• National meetings of policy and technical teams
  including over 80 people from 6 states
  representing
• Governors Offices
• State and Local Workforce Investment Boards
• State Workforce Agencies
• Community and Technical Colleges
• Secondary Career and Technical Education
• Vocational Rehabilitation
• TANF
• Adult Education and Family Literacy

9
IPI Project

• Consensus Process
• State Institutes 10 more states, 55 more people
• Vetted by National Experts in Research and
  Evaluation
• Vetted by Representatives of National Stakeholder
  Groups
• End Product A Blueprint Guide to States on
  Creating Integrated Performance Information

10
Integrated Performance Information (IPI) Blueprint

• Performance Measures
• What are the best performance measures for
  workforce development if the same measures are
  applied horizontally and vertically within the
  system, including programs that are funded mostly
  by the states and programs that are funded mostly
  by Congress?
• Policy not Management Level Measures

11
What Do Policy Makers Want to Know About
Performance?

• Labor Market Results for Program Participants
• Do people get jobs?
• What are they paid?
• Skill Gains
• To what extent do education levels increase?
• Results for Employers and the Economy
• Are we meeting the needs of employers?
• Return on Investment
• What is the return on the investment?

12
What Makes for a Good Performance Measure?

• Easily Explainable to a Lay Audience
• Level Playing Field Among Programs and Service
  Strategies
• Meaningful for Each Program
• Scaleable and Divisible
• Methodologically Sound
• Not Easily Gamed
• Inexpensive
• There is no perfect set of performance measures.

13
IPI Performance Measures

• Two Categories of Measures
• 5 Accountability Measures Possible Targets and
  Consequences
• 3 Performance Indicators Indicators of General
  Performance

14
IPI Accountability MeasuresLabor Market Results
for Program Participants

• 1. Short-term Employment Rate The percentage of
  participants who are employed during the second
  quarter after exit.
• 2. Long-term Employment Rate The percentage of
  participants who are employed during the fourth
  quarter after exit.
• For youth, the measures include exiters
  enrolled in education or employed.

15
IPI Accountability MeasuresLabor Market Results
for Program Participants

• 3. Earnings Level Median earnings during the
  second quarter after exit among all exiters with
  any earnings. (For youth, individuals enrolled in
  education should be excluded from the measure.)

16
IPI Accountability MeasuresSkill Gains

• 4. Credential Completion Rate The percentage
  of exiters who have completed a certificate,
  degree, diploma, licensure, or other
  industry-recognized credential during
  participation or within one year of exit.

17
IPI Accountability MeasuresResults for Employers
and the Economy

• 5. Repeat Employer Customers The percentage of
  employers who are served who return to the same
  program for service within one year.

18
IPI Performance IndicatorsResults for Employers
and the Economy

• 6. Employer Market Penetration The percentage
  of all employers who are served during the year.

19
IPI Performance IndicatorsReturn on Investment

• 7. Taxpayer Return on Investment The net
  impact on tax revenue and social-welfare payments
  compared to the cost of the services.
• 8. Participant Return on Investment The net
  impact on participant earnings and
  employer-provided benefits compared to the cost
  of the services.

20
IPI Performance Measures

• 8 Measures that answer the questions of policy
  makers, and
• Meet the criteria for good performance measures
  for the workforce development system.
• OMB Favorable Response
• IPI Blueprint http//www.wtb.wa.gov/IPI.PDF