WorkKeys Innovations: A Holistic Solution

1
WorkKeys Innovations A Holistic Solution
presented at the 2007 Michigan WorkKeys
Conference

• Steve Robbins, AVP, Applied Research, ACT, Inc.

2
Overview

• Why we should care about combining cognitive- and
  personality-based measures
• WorkKeys Pyramid for Success
• The National Career Readiness Certificate Plus
  and Personal Skills Assessments
• Differential test strategies along the entire
  continuum of employment

3
What We Know from Research Literature on
Incremental Test Validity

• Cognitive and foundational skill tests rule
• Adverse impact issues can be ameliorated with
  specific tests
• Task analysis
• Job-specific tests
• Combinational use of cognitive and non-cognitive
  tests
• Personality and career tests add incremental
  validity

4
General vs. specific test effect sizes
Brown, Le, Schmidt (2006) Salgado et al.
(2003)
5
Combining Personality Cognitive Ability Tests

• Level of correlations are low
• GMA x C .02 GMA x ES .17
• Math x C -.15 Math x ES .17
• Reading x C -.05 Reading x ES .11

(Ackerman Heggestad, 1997)
6
Personality TestsAdd Value

• Validity Estimates
• 1 From Schmidt et al. (2007) using indirect
  range restriction

7
Combining Personality Cognitive Ability Tests

• Creating opportunity for incremental validity
  especially as criteria vary

Correlations between general cognitive ability
and personality tests and measures of job
performance in Project A Cog Pers Both Criteria
.63 .26 .67 Core technical proficiency .65 .25 .70
General Soldiering Proficiency .31 .33 .44 Effort
and Leadership .16 .32 .37 Personal
Discipline .20 .37 .42 Physical fitness and
military bearing
(McHenry, Hough, Toquam, Hanson, Ashworth, 1990)
8
Why Now?

• Market Need SHRM National Study
• Body of research informs how to optimize Personal
  Skill Assessments
• ACT strength in cognitive non-cognitive
  assessment (John Holland, VP Research in 1960s)
• Solution-focused approach

9
Businesses want integration of Cognitive
Personality Constructs

• SHRM Applied Skills and Basic Knowledge
• Combining and Ranking
• For new entrants with a two-year
  college/technical school diploma, applied skills
  are four of the top five very important skills
  in combined ranking with basic knowledge and
  skills.

Casner-Lotto, J. Barrington, L. (2006)
10
Solutions are Need Driven Continuum of Employment
11
WorkKeys Assessment Solutions Pyramid for Success

• Ensure work and training readiness (WorkKeys
  Foundational Skills)
• Give a snap shot of strengths and areas of
  improvement across key response tendencies or
  domains (Talent)

12
Pyramid for Success

• FIT
• Match individual interests/values to work
  environment
• Enhance job persistence satisfaction
• Develop Talent pool to meet needs

• TALENT
• Benchmarking for selection
• Coaching Development
• Compound Indices Sales, Managerial,
  Leadership, Safety

• PERFORMANCE
• General Work Performance Productivity,
  Absenteeism, Complaints about conduct
• Safety and Risk Reduction

• FOUNDATIONAL SKILLS
• Job Analysis identifies the skills and skill
  levels needed to be successful on the job
• Assessments show the current skill levels of an
  individual
• Training helps individuals and employers
  correct skill gaps

13
National Career Readiness Certificate
The WorkKeys system assesses foundational skills,
allowing individuals to seek a certificate, and
when necessary to fill in the gaps.
Silver Certificate Goal
4
4
4
3
3
Applied Math
Locating Information
Reading
14
National Career Readiness Certificate Plus

• A flexible solution to meet state and system
  needs
• Georgia example
• Talent for coaching development
• Alterable variables allow for intervention
  guidance

15
It was about here, wasnt it, Ed, when you came
on board as sales manager?
Harvard Business Review. March 2007. p. 90
16
Talent Score Report
17
TALENT Development Worksheet
18
Behavioral Scales, Targets, and Representative
Behaviors
19
Behavioral Scales, Targets, and Representative
Behaviors
20
Behavioral Scales, Targets, and Representative
Behaviors
21
A Comparison of Talent Scale Scores for Computer
Programmers vs. Sales Representatives Sales
Managers
70
65
60
Mean Scores
55
50
45
Order
Savvy
Stability
Striving
Goodwill
Influence
Discipline
Optimism
Sociability
Creativity
Carefulness
Cooperation
Talent Scales
22
(No Transcript)
23
FIT Score Report
24
Employer Report
2 of 3
Fit Assessment
Report for Abbatoir Industries Site Iowa City,
IA Test Date 3/30/07
Examinee Alvin C. Tracey Examinee ID
XXXXX7890
Top 10 Occupations Ranked by Fit The top 10
occupations for the examinee, ranked by Fit
Index, are shown below. This is based on all of
the occupations in the WorkKeys Fit database.
Examinee-specified occupations, if any, are in
BOLD.
Employer Report Top 10 Occupations and Interest
Inventory Results
Title
Code
Sales Agents, Financial Services
41-3031.02
Loan Counselors
13-2071.00
Transportation, Storage, and Distribution Managers
11-3071.00
Insurance Sales Agents
41-3021.00
Administrative Services Managers
11-3011.00
Real Estate Sales Agents
41-9022.00
Sales Managers
11-2022.00
Real Estate Brokers
41-9021.00
Middle School Teachers, Except Special
Vocational Educ
25-2022.00
Appraisers, Real Estate
13-2021.02
Interest Results
The Interest Inventory obtains scores on six
scales. The examinee's score profile and highest
scales are shown.
Interest Score Profile Standard Score (20-80)
20 30 40 50 60
70 80
Administration Sales Persuading, influencing,
or motivating others via sales, management, etc.
Business Operations Maintaining accurate/
orderly files/accounts via systematic
procedures.
Technical Working with tools, instruments,
machines, etc.
Science Technology Studying phenomena through
research, reading, etc.
Arts Expressing oneself via painting, singing,
writing, etc.
Social Service Helping or serving others via
teaching, counseling, etc.
Highest Interest Scales Administration Sales
Science Technology Business Operations
25
3 of 3
Employer Report
Fit Assessment
Report for Abbatoir Industries Site Iowa City,
IA Test Date 3/30/07
Examinee Alvin C. Tracey Examinee ID
XXXXX7890
Work Values Results Examinee's highest 5 and
lowest 2 work values are shown. The definition of
each value, as it appears in the Work Values
Inventory, is provided.
Employer Report Work Values Inventory Results
Highest Values Variety Using many different types
of skills in my work. Public Contact Interacting
with customers (as in sales) or the public (as in
police work). Authority Telling people what to
do controlling the behavior of
others. Precision Being exact or very accurate
in the work I do. Helping People Improving
the lives of others through activities such as
teaching, physically assisting, or mentoring.
Lowest Values Social Status Being looked up to
by others in my company or my community because
of my job. Physical Activity Moving around in my
work by walking, bending, lifting, etc.
For more information go to http//www.act.org/work
keys/assess/fit
26
Selection Solutions

• Reducing Risk
• Task Competence through WorkKeys and job
  profiling
• General Work Safety
• Increasing Tenure
• Task Competence through WorkKeys and job
  profiling
• Fit
• Getting the Right Person
• Talent Benchmarking
• Past Work Performance Record

27
If they staged a slowdown, how would we know?
Harvard Business Review. March 2007. p. 90
28
Coaching Development Solutions

• Leadership Development
• Talent
• Fit
• Teamwork
• Talent

29
Return on Investment Approximations under Various
Scenarios
Notes Selection the percentage of the
candidate pool selected for hire, Candidate
Success the percentage of the candidate pool
that would be successful if hired, Selected
Success the percentage of the selected
candidate pool that will be successful, Cost per
failure the average cost for each unsuccessful
employee relative to each successful employee,
ROI per 100 candidates the average return on
investment for the selection procedure assuming a
15 fee per candidate.
30
Final Thoughts

• Adverse Impact may be reduced when combining
  tests
• Still recommend multiple hurdles approach
• Incremental Validity Research underway
• Combination of Task Personality measures
• Differential work outcomes
• ROI x Solution

31
References

• Ackerman, P. L., Heggestad, E. D. (1997).
  Intelligence, personality, and interests
  Evidence for overlapping traits. Psychological
  Bulletin, 121, 219-245.
• Brown, K. G., Le, H., Schmidt, F. L. (2006).
  Specific aptitude theory revisited Is there
  incremental validity for training performance?
  International Journal of Selection and
  Assessment, 14, 87-100.
• Casner-Lotto, J. Barrington, L. (2006). Are
  they really ready to Work? Society for Human
  Resource Management. http//www.shrm.org/hrresourc
  es/surveys_published
• McHenry, J. J., Hough, L. M., Toquam, J. L.,
  Hanson, M. A., Ashworth, S. (1990). Project A
  validity results The relationship between
  predictor and criterion domains. Personnel
  Psychology, 43, 335-354.
• Robbins, S., Allen, J., Casillas, A., Peterson,
  C., Le, H. (2006). Unraveling the differential
  effects of motivational and skills, social, and
  self-management measures from traditional
  predictors of college outcomes. Journal of
  Educational Psychology, 98, 598-616.
• Rotundo, M., Sackett, P. R. (2002). The
  relative importance of task, citizenship, and
  counterproductive performance to global ratings
  of job performance A policy-capturing approach.
  Journal of Applied Psychology, 87, 66-80.
• Salgado, J. F., Anderson, N., Moscoso, S.,
  Bertua, C., de Fruyt, F. (2003). International
  validity generalization of GMA and cognitive
  abilities A European community meta-analysis.
  Personnel Psychology, 56, 573-605.
• Schmidt, F. L., Hunter, J. E. (1998). The
  validity and utility of selection methods in
  personnel psychology Practical and theoretical
  implications of 85 years of research findings.
  Psychological Bulletin, 124, 262-274.
• Schmidt, F. L., Shaffer, J., Oh. I. (2007).
  Reassessing the Relative Importance of Cognitive
  Ability and Personality in Job Performance and
  Training Performance Some Surprising New
  Research Findings. Paper presented at the 2007
  ATP conference, Palm Springs, CA. Feb. 6.

32
Incorporating Foundational and Soft Skill
Assessments
For questions regarding this presentation or for
further information contact Steve Robbins at
319-337-1227 or steve.robbins_at_act.org, Gary
Nolan at 319-337-1526 or gary.nolan_at_act.org,