WIA Youth Common Measures and Program Reporting Requirements

1
WIA Youth Common Measures and Program Reporting
Requirements
1
2
Agenda

• Clarifying PY 2005 Accountability
• Reviewing Key Terms and Concepts
• Calculating Youth Common Measures
• Youth Program Reporting Requirements

2
3
Clarifying PY 2005 Accountability
3
4
PY 2005 Accountability

• Grantees are held accountable for performance
  outcomes on the seven WIA youth performance
  measures)
• In the absence of a reauthorized WIA, the
  statutory measures must be used for
  accountability purposes
• Grantees are required to report on two of the
  three youth common measures Placement in
  Employment/Education and Degree Attainment
• Data collected through revised reports (i.e.,
  WIASRD, Quarterly Report, and Annual Report)
• No negotiation of youth common
  measures took place for PY 2005

4
5
PY 2005 Accountability (contd)

• Grantees are encouraged to report on the
  Literacy/ Numeracy common measure but are not
  required to do so until PY 2006
• This allows a transition year (or two)
  of data collection for the
  youth common measures
• What are some advantages to beginning reporting
  on this common measure during PY 2005?

5
6
Current WIA Youth Measures

• OLDER YOUTH (19-21)
• Entered Employment
• Retention
• Credential Rate
• Earnings Change
• YOUNGER YOUTH (14-18)
• Skill Attainment Rate
• Diploma Attainment Rate
• Retention Rate

6
7
Current vs. Common MeasuresIn General

• States should continue to ensure success with
  current measures, while transitioning to a focus
  on common measures
• Major difference between the two sets of measures
  is the lack of ISY exclusion for placement and
  certificate common measures
• States should focus on serving ISY who will need
  program services until graduation/GED attainment
• Consistent with new youth vision to
  serve the neediest youth

7
8
Reviewing Key Terms and Concepts
8
9
Reviewing Key Terms/Concepts

• A participant is an individual who is determined
  eligible to participate in a program and who
  receives a service funded by the program in
  either a physical location or through electronic
  technologies
• An individual must be a participant in order to
  count in performance calculations
• Note Self-service only does not initiate
  participation in the WIA youth program
• Program exit occurs when a participant does not
  receive a service (funded by the program or a
  partner) for 90 consecutive days and no future
  services are scheduled
• No more hard exit

9
10
No More Hard Exit

• Translation An exit cannot be officially
  recorded until the 90 days has elapsed, but its
  still the last date of service!
• Local programs should know when a service is
  expected to be the last service
• Does not mean that the case isnt closed from
  the case managers perspective (in fact,
  states/locals could have a case closure code
  separate from the exit code to make this
  distinction if a youth needs additional services
  beyond follow-up, the case could then be
  re-opened during this window)
• Does not mean case managers must wait
  before providing follow-up services

10
11
Clarifying Follow-Up Services

• Youth are required to receive at least 12 months
  of follow-up services, which are triggered at
  exit
• With a soft exit approach, do you lose the
  first 3 months of follow-up? NO
• Follow-up begins after the expected last service
  if only follow-up services are provided for
  next 90 days, we have an exit
• However, if youth need additional services,
  this 90 days provides a window
  to better ensure success (because
  no official exit was
  recorded)

11
12
Excluding Individuals From Performance
Calculations

• Those specifically excluded by statute
• Although this applies to WIA, it applies only to
  adults and dislocated workers who receive
  self-service and informational activities only
• 2. Other Circumstances
• Invalid or missing SSN
• Institutionalized
• Health/Medical or Family Care
• Deceased
• Reservists called to active duty
• For youth only Relocated to a
  residential or non-residential program
  (such as foster care)

12
13
Global Exclusion Impacts

• Exit due to exclusion causes all outcomes to be
  lost
• Participant had a success in the 1st year and
  then they were excluded in the 2nd year
• Impact Positive or negative outcome would be
  lost
• Relevant to real-time and exit-based outcomes
• Impacts
• Skill Attainment new to this measure
• Literacy or Numeracy measure

14
Gaps in Service

• Nomenclature changed from planned gaps to gap
  in recognition of the fact that some service
  interruptions cannot be planned in advance
• Participants w/ planned gap in service of greater
  than 90 days should not be exited if gap is due
  to one of the following
• Delay before beginning of training
• Health/medical condition
• Temporary move from area that prevents
  individual from participating (e.g., youth
  living with another parent
  over the summer)

14
15
Data Sources

• UI Wage Records and supplemental data sources are
  allowable for the two employment related youth
  common measures
• Administrative records (including appropriate
  assessment instruments) will be the data source
  for non-employment placements, degree or
  certificate attainment, and literacy/numeracy

15
16
Calculating Youth Common Measures
16
17
Three Youth Common Measures

• Placement in Employment, Education or Training
• Attainment of a Degree/Certificate
• Literacy/Numeracy Gains
• NOTE The measures apply to youth 14-21
  no more older/younger distinction!

17
18
PLACEMENT IN EMPLOYMENT OR EDUCATION

• Definition Of those not in post-secondary
  education, employment, or the military at
  participation, the percentage in employment, the
  military or enrolled in post-secondary education
  and/or advanced training/occupational skills
  training in the 1st quarter after exit

18
19
PLACEMENT IN EMPLOYMENT OR EDUCATION

• Calculation

Of those not employed, in the military, or in
post-secondary education at participation
Number of youth employed, in the military or
enrolled in post-secondary education and/or
advanced training or occupational skills training
in the 1st quarter after exit Number of exiters
19
20
Highlights of Youth Placement in Employment or
Education

• Excludes youth in employment, the military or
  post-secondary education at participation
• Employment, military and education status at
  participation is based on information collected
  from the individual
• Employment and military status in the 1st quarter
  after exit is based on wage records (supplemental
  data allowed)
• Education status in the 1st quarter after exit is
  based on administrative records
• Youth in secondary school at exit are included,
  consistent with ETAs vision to ensure youth
  successfully complete their secondary
  education

20
21
Defining Specific Terms

• Post-Secondary Education a program at an
  accredited degree-granting institution leading to
  an academic degree. Programs offered by
  degree-granting institutions that do not lead to
  an academic degree (such as certificate programs)
  do not count as a placement in post-secondary
  education but may count as a placement in
  advanced training/occupational skills training

21
22
Defining Specific Terms (contd)

• Advanced Training/Occupational Skills Training
• An organized program of study that provides
  specific vocational skills leading to proficiency
  in performing actual tasks and technical
  functions required by certain occupational fields
  at entry, intermediate or advanced levels. Such
  training should 1) be outcome-oriented and
  focused on a long-term goal as specified in the
  ISS, 2) coincide with exit rather than
  short-term training, and 3) result in attainment
  of a certificate
• Much tighter definition!

22
23
PLACEMENT IN EMPLOYMENT OR EDUCATION
Qualifying Outcomes employment, military,
enrolled in post-secondary education, advanced
training, or occupational skills training
23
24
ATTAINMENT OF DEGREE OR CERTIFICATE

• Definition Of those enrolled in education at or
  during participation, the percentage who attained
  a diploma, GED or certificate by the end of the
  3rd quarter after exit

24
25
ATTAINMENT OF DEGREE OR CERTIFICATE

• Calculation

Of those enrolled in education at participation
or any time during the program
Number of youth who attained a diploma, GED or
certificate by the end of the 3rd quarter after
exit Number of exiters
25
26
Highlights of Youth Attainment of Degree or
Certificate

• Youth in secondary school at exit are included,
  which is consistent with ETAs vision to ensure
  youth successfully complete their secondary
  education
• Diplomas, GEDs or certificates can be obtained
  during participation or at any point by the end
  of the 3rd quarter after exit
• Work readiness certificates will not be
  accepted for this measure

26
27
Defining Specific Terms

• Education participation in secondary or
  post-secondary school, adult education programs,
  or any other organized program of study leading
  to a degree or certificate
• Diploma any credential accepted by the State
  educational agency as equivalent to a HS diploma
  also includes post-secondary degrees
• Certificate awarded in recognition of an
  individuals attainment of technical or
  occupational skills by specified institutions
  such as the State educational agency or
  institution of higher education
• Does not include work readiness certificates or
  certificates awarded by local
  Boards

27
28
Certificates Approved Awarding Institutions

• A state educational agency, or a state agency
  responsible for administering vocational and
  technical education within a state
• Institution of higher education (including
  community colleges)
• A professional, industry, or employer
  organization or a product manufacturer or
  developer
• A registered apprenticeship program
• A public regulatory agency (e.g., FAA aviation
  mechanic certification)
• A program approved by DVA (Veterans Affairs) to
  offer education and training to veterans and
  other eligible persons under the Montgomery GI
  Bill
• Office of Job Corps
• Institutions of higher education which are
  formally controlled, or formally
  sanctioned or chartered by
  the governing body of an Indian tribe(s)

28
29
ATTAINMENT OF DEGREE OR
CERTIFICATE
29
30
LITERACY/NUMERACY GAINS

• Definition Of those out of-school youth who are
  basic skills deficient, the percentage who
  increase one or more educational functioning
  levels within one year of participation

30
31
LITERACY/NUMERACY GAINS

• Calculation

Of those out-of-school youth who are basic skills
deficient
Number of youth participants who increase one or
more educational functioning levels
Number of youth who completed a year of
participation (based on date of 1st youth service)
Number of youth who exit before completing a year
of participation

31
32
Highlights of Literacy/Numeracy Gains

• Only common measure that is not exit-based
• Excludes in-school youth and out-of-school youth
  who are not basic skills deficient
• Includes individuals with learning disabilities
• A positive outcome means the youth must advance
  one or more Adult Basic Education (ABE) or
  English as a Second Language (ESL) functioning
  levels
• Gains can occur in literacy or numeracy (programs
  can pre-test at different levels in each
  category)
• Educational levels are consistent with
  Department of Educations National
  Reporting System (NRS)

32
33
About the Assessments

• All out-of-school youth must be assessed in basic
  reading, writing and math
• Pre-testing must occur within 60 days of the
  first youth program service can use pre-test
  from up to six months prior to date of first
  youth service
• The same standardized assessment must be used for
  pre- and post-testing
• Youth should be post-tested by the end of one
  year of participation and compared to pre-test
  results obtained during initial assessment

33
34
About the Assessments (contd)

• Youth should be post-tested and included in the
  measure at the completion of the 2nd year if they
  complete two years in the program
• If a youth continues to be basic skills deficient
  after the first 12 months of participation, they
  should continue to receive training in literacy
  and/or numeracy skills
• NOTE On the agenda there is a Youth
  Literacy/Numeracy measure workshop

34
35
Youth Program Reporting Implications of the New
Youth Vision and Program Design
35
36
Program Design Implication 1 Long-Term
Participation

• Common Measure outcomes are unlikely to be met
  unless youth participate for extended periods
• In-school younger youth must remain engaged until
  they graduate or receive a GED and go onto
  employment or post-secondary education
• Out-of-school youth who are basic skills
  deficient will likely need time to improve one or
  more EFLs
• Summer employment programs that enroll WIA
  youth for the summer only are
  unlikely to achieve positive outcomes

36
37
Long-Term ParticipationEngagement and Retention
Strategies

• Keeping youth connected to an organization or
  program
• Establishing a sense of self-worth through
  program participation (e.g., they get paid,
  receive positive feedback their contributions
  matter, etc.)
• Providing incentives
• Opportunities to demonstrate skills to family and
  peers field trips and events stipends,
  opportunities to serve and lead supportive
  services recognition
• Youth-centered programs
• Youth-friendly intake procedures interpersonal
  support in programs from personnel, parents,
  peers honesty and authenticity in program
  (provide whats promised) culturally competent
  staff and culturally
  relevant programs

37
38
Program Design Implication 2 Network of
Services

• Since long-term participation is needed to
  achieve outcomes, more than ever, WIA youth must
  have access to a network of services, not a
  single program or provider
• Strategic case management strategies will be
  needed to connect youth to multiple programs
• Develop short-term attainable goals for youth and
  youth programs to support long-term positive
  outcomes

38
39
Network of ServicesStrategic Case Management

• Case management is key
• Create and maintain positive relationships with
  youth have youth see case manager role as
  connector to a network of services use team
  approach to helping youth

39
40
Network of ServicesShort-Term Outcomes

• For Youth
• Setting intermediate goals (outcomes) helps them
  see progress, which contributes to engagement and
  retention
• Think SMART Set goals WITH the youth Make it
  VISUAL
• For WIA Youth Programs
• WIA outcomes arent appropriate to use as program
  outcomes for one-year contracts develop
  intermediate measures to monitor program success
• Including participation rates, skill/goal
  attainment rates
• Balance outcomes with outputs

40
41
Program Design Implication 3 Assessment

• When done well, assessment sets the stage for all
  future services across programs when done
  poorly, it can have a negative impact on
  individual youth success and youth program success

41
42
Program Design Implication 3 Assessment

• Youth friendly assessments
• Shorter versions when possible (subtests),
  pleasant testing environments, dont send for a
  test right from the start
• Tell youth why they are being tested and how test
  will be used dont retest too soon
• Consider other assessments style,
  self-awareness, learning style remove the
  high-stakes test pressure

42
43
Resources
ETAs Office of Workforce Investment Youth
Services http//www.doleta.gov/youth_services/ Na
tional Reporting System http//www.nrsweb.org Spe
cial Thanks To Diana Jackson Darcie
Milazzo LearningWork Connection
http//www.learningworkconnection.org
43
44
Remember www.doleta.gov/performance
Thank You !
44