Title: WorkKeys Innovations: A Holistic Solution for WIRED West Michigan
1WorkKeys Innovations A Holistic Solution for
WIRED West Michigan
- Steve Robbins, AVP, Applied Research, ACT, Inc.
2Overview
- Why we should care about combining cognitive- and
personality-based measures - National Career Readiness Certificate Plus
- What we know from workforce and educational
literatures - Differential test strategies along the entire
continuum of employment
3Why Now?
- Market Need SHRM National Study
- Body of research informs how to optimize Personal
Skill Assessments - ACT strength in cognitive non-cognitive
assessment (John Holland, VP Research in 1960s) - Solution-focused approach
4Businesses want integration of Cognitive
Personality Constructs
- SHRM Applied Skills and Basic Knowledge
- Combining and Ranking
- For new entrants with a two-year
college/technical school diploma, applied skills
are four of the top five very important skills
in combined ranking with basic knowledge and
skills.
Casner-Lotto, J. Barrington, L. (2006)
5Solutions are Need Driven Continuum of Employment
6WorkKeys Assessment Solutions Pyramid for Success
- Ensure work and training readiness (WorkKeys
Foundational Skills) - Provide employers another source of information
for selection (Performance) - Give a snap shot of strengths and areas of
improvement across key response tendencies or
domains (Talent) - Promote career exploration of job fit (Fit)
7Pyramid for Success
- FIT
- Match individual interests/values to work
environment - Enhance job persistence satisfaction
- Develop Talent pool to meet needs
- TALENT
- Benchmarking for selection
- Coaching Development
- Compound Indices Sales, Managerial,
Leadership, Safety
- PERFORMANCE
- General Work Performance Productivity,
Absenteeism, Complaints about conduct - Safety and Risk Reduction
- FOUNDATIONAL SKILLS
- Job Analysis identifies the skills and skill
levels needed to be successful on the job - Assessments show the current skill levels of an
individual - Training helps individuals and employers
correct skill gaps
8National Career Readiness Certificate
Currently, the WorkKeys system assesses
foundational skills, such as reading and locating
information, using a three-step process of job
profiling, assessment, and skill-gap training.
9National Career Readiness Certificate Plus
- A flexible solution to meet state and system
needs - Georgia example
- Talent for coaching development
- Fit for career exploration
- Alterable variables allow for intervention
guidance
10Cognitive Ability TestsRule
- General vs. specific test effect sizes
- Adverse impact issues can be ameliorated with
specific tests - Task analysis
- Job-specific tests
- Combinational use of cognitive and non-cognitive
tests
Brown, Le, Schmidt (2006) Salgado et al.
(2003)
11Personality TestsAdd Value
- Validity Estimates
-
- 1 From Schmidt et al. (2007) using indirect
range restriction
12What are We Talking About? Personality Test
Constructs
- The Big Five
- Emotional Stability, Extraversion,
Agreeableness, Conscientiousness, Openness to
Experience - Specific or Facet-level Traits
- Carefulness, Discipline, Influence, Order,
Sociability, Drive, Creativity - Compound Traits
- Service Orientation, Integrity, Managerial
Potential, Teamwork
13Personality Test Issues
- Selection vs. Coaching and Development as
Differential Applications - Selection Approach
- Recommend multiple hurdles and/or top-down
approach - Adverse Impact Limited
- Differential Outcomes (Rotundo Sackett, 2002)
- Task Performance (Technical Core)
- Organizational Citizenship Behavior (Contextual
Performance) - Counterproductive Work Behavior
- Add Satisfaction/Tenure as another Key Outcome
14Combining Personality Cognitive Ability Tests
- Level of correlations are low
- GMA x C .02 GMA x ES .17
- Math x C -.15 Math x ES .17
- Reading x C -.05 Reading x ES .11
(Ackerman Heggestad, 1997)
15Combining Personality Cognitive Ability Tests
- Creating opportunity for incremental validity
especially as criteria vary
Correlations between general cognitive ability
and personality tests and measures of job
performance in Project A Cog Pers Both Criteria
.63 .26 .67 Core technical proficiency .65 .25 .70
General Soldiering Proficiency .31 .33 .44 Effort
and Leadership .16 .32 .37 Personal
Discipline .20 .37 .42 Physical fitness and
military bearing
(McHenry, Hough, Toquam, Hanson, Ashworth, 1990)
16Educational Examples
1 Correlation of fitted probabilities and
dichotomous retention outcome 2 SRI scale scores
and ACT Composite score used as predictors
17Educational Examples
1 Multiple R from linear regression model 2 SRI
scale scores and ACT Composite score used as
predictors
18ACT Foundational and Personal Skills Assessments
measure Different Applied Job Skills
19If they staged a slowdown, how would we know?
Harvard Business Review. March 2007. p. 90
20Performance Score Report
21It was about here, wasnt it, Ed, when you came
on board as sales manager?
Harvard Business Review. March 2007. p. 90
22Talent Score Report
23(No Transcript)
24FIT Score Report
25(No Transcript)
26Selection Solutions
- Reducing Risk
- Task Competence through WorkKeys and job
profiling - General Work Safety
- Increasing Tenure
- Task Competence through WorkKeys and job
profiling - Fit
- Getting the Right Person
- Talent Benchmarking
- Past Work Performance Record
27Coaching Development Solutions
- Career Exploration
- Task Competency
- Fit
- Leadership Development
- Talent
- Fit
- Teamwork
- Talent
28Return on Investment Approximations under Various
Scenarios
Notes Selection the percentage of the
candidate pool selected for hire, Candidate
Success the percentage of the candidate pool
that would be successful if hired, Selected
Success the percentage of the selected
candidate pool that will be successful, Cost per
failure the average cost for each unsuccessful
employee relative to each successful employee,
ROI per 100 candidates the average return on
investment for the selection procedure assuming a
15 fee per candidate.
29Final Thoughts
- Adverse Impact may be reduced when combining
tests - Still recommend multiple hurdles approach
- Incremental Validity Research underway
- Combination of Task Personality measures
- Differential work outcomes
- ROI x Solution
30References
- Ackerman, P. L., Heggestad, E. D. (1997).
Intelligence, personality, and interests
Evidence for overlapping traits. Psychological
Bulletin, 121, 219-245. - Brown, K. G., Le, H., Schmidt, F. L. (2006).
Specific aptitude theory revisited Is there
incremental validity for training performance?
International Journal of Selection and
Assessment, 14, 87-100. - Casner-Lotto, J. Barrington, L. (2006). Are
they really ready to Work? Society for Human
Resource Management. http//www.shrm.org/hrresourc
es/surveys_published - McHenry, J. J., Hough, L. M., Toquam, J. L.,
Hanson, M. A., Ashworth, S. (1990). Project A
validity results The relationship between
predictor and criterion domains. Personnel
Psychology, 43, 335-354. - Robbins, S., Allen, J., Casillas, A., Peterson,
C., Le, H. (2006). Unraveling the differential
effects of motivational and skills, social, and
self-management measures from traditional
predictors of college outcomes. Journal of
Educational Psychology, 98, 598-616. - Rotundo, M., Sackett, P. R. (2002). The
relative importance of task, citizenship, and
counterproductive performance to global ratings
of job performance A policy-capturing approach.
Journal of Applied Psychology, 87, 66-80. - Salgado, J. F., Anderson, N., Moscoso, S.,
Bertua, C., de Fruyt, F. (2003). International
validity generalization of GMA and cognitive
abilities A European community meta-analysis.
Personnel Psychology, 56, 573-605. - Schmidt, F. L., Hunter, J. E. (1998). The
validity and utility of selection methods in
personnel psychology Practical and theoretical
implications of 85 years of research findings.
Psychological Bulletin, 124, 262-274. - Schmidt, F. L., Shaffer, J., Oh. I. (2007).
Reassessing the Relative Importance of Cognitive
Ability and Personality in Job Performance and
Training Performance Some Surprising New
Research Findings. Paper presented at the 2007
ATP conference, Palm Springs, CA. Feb. 6.
31Incorporating Foundational and Soft Skill
Assessments
For questions regarding this presentation or for
further information contact Steve Robbins at
319-337-1227 or steve.robbins_at_act.org, Gary
Nolan at 319-337-1526 or gary.nolan_at_act.org,