Training and Experience

1
Training and Experience
Dr. Mike Aamodt Radford University maamodt_at_radford
.edu
Updated 12/06/2004
2
Personnel Selection Methods

• Skills
• Work Samples
• Assessment Centers
• References
• Personality Character
• Personality Tests
• Integrity Tests
• Medical
• Medical Exams
• Psychological Exams
• Drug Testing

• Training Education
• Experience
• Applications/Resumes
• Biodata
• Interviews
• Knowledge
• Ability
• Cognitive
• Physical
• Perceptual

3
Ratings of Training

• Education
• Work-Related Training
• Military

4
Education as a Predictor of Job Performance

• Education can be broken further into 4
  categories
• Degree
• Years of education
• Major
• GPA

5
Does Education Predict Performance?
6
Summary of Meta-Analyses
Meta-analysis Occupation K N ?
Aamodt (2004) Police 54 9,120 .28
Vineberg Joyner (1982) Military 35 .25
Hunter (1980) Hunter Hunter (1984) Schmidt Hunter (1998) USES data base 425 32,124 .10
Dunnette (1972) Entry level petroleum 15 .00
7
Validity of Education for PoliceOur Updated
Meta-Analysis

• Academy Supervisor Discipline
• Grades Ratings Problems
• ________ _________ ________
• Number of studies 32 54 54
• Sample Size 6,153 9,120 21,416
• Uncorrected validity .26 .17 - .08
• Corrected validity .38 .28 - .12
• Sampling error 100 80 73

8
Our Meta-Analysis (Cont.)

• Academy Grades Patrol Performance
• _____________ _______________
• Education level N Score N Score
• ______________ _____ _____ _____ _____
• GED/HS Diploma 376 -.23c 915 -.12a
• 12-64 college hours 100 -.05ac 391
  .02b
• AAS 98 .16ab 139 .07bc
• 64 college hours 62 .01ac 45
  .29bc
• Bachelors 255 .32b 441 .18c

9
Interaction of Education and ExperienceRatings
of Patrol Performance
Experience Experience
Educational Level lt 2 years 2 Years Total
High School Diploma M - .09ab - .16a - .13a
High School Diploma SD (.92) (.98)
Some college or associates degree M - .26a .07b - .09a
Some college or associates degree SD (1.02) (.94)
B.A. or higher M .04ab .34c .19b
B.A. or higher SD (1.02) (1.12)
10
Education and Academy GradesEffect of Academy
Length
K N r ? SE
Overall 32 6,153 .26 .38 100
Academy Length
12-17 weeks 16 2,477 .31 .46 100
gt 20 weeks 15 3,473 .25 .36 100
11
Does Education Add Incremental Validity Over
Cognitive Ability?
12
Education and Incremental Validity

• Schmidt Hunter (1998) say no
• Cognitive ability (r .51)
• Cognitive ability and education (r .52)

13
Education and Incremental Validity

• Our meta-analysis for police says yes
• Academy Grades
• Cognitive ability (r .33)
• Cognitive ability and education (r .34)
• Patrol Performance (uncorrected)
• Cognitive ability (r .16)
• Cognitive ability and education (r .29)

14
Can You Substitute Education for a Cognitive
Ability Test?

• Education has less adverse impact
• But
• Cognitive ability and education are only slightly
  correlated
• .29 in our police data
• .00 assumption by Schmidt Hunter (1998) of
  applicants applying for a particular job
• .55 estimate by Schmidt Hunter (1998) for the
  general population
• Education adds incremental validity for law
  enforcement
• Our dataset contains plenty of college graduates
  with low cognitive ability

15
Does Grade Point Average Predict Performance?
16
Validity of GPA

• GPA is a valid predictor of performance on the
  job, training performance, starting salary,
  promotions, and grad school performance
• GPA is most predictive in the first few years
  after graduation (Roth et al., 1996)
• GPA will result in high levels (d.78) of adverse
  impact (Roth Bobko, 2000)
• People with high GPAs
• Are intelligent (r.50 Jensen, 1980)
• Are conscientious (r.34 Bevier et al., 1998)

17
Validity of GPAMeta-Analysis Results
r ?
Work-Related Criteria
Job performance (Roth et al., 1996) .16 .36
Training performance (Dye Reck, 1989) .29
Promotions (Cohen, 1984) .16
Salary (Roth Clarke, 1996)
Starting salary .13 .20
Current salary .18 .28
Graduate School Performance (Kuncel et al., 2001)
Grades .28 .30
Faculty ratings .25 .35
18
Predicting Graduate School Performance
Graduate GPA Teaching Ratings
GRE Score .33 .00
GPA Overall .31 .02
GPA Psych .32 .03
GPA Junior/Senior .40 .09
Faculty References
Mental agility .25 - .08
Vigor - .26 - .07
Extroversion .03 .24
Agreeableness - .11 .06
Conscientiousness .16 - .04
19
Does College Major Predict Performance?
20
College Major as a Predictor of Job Performance

• Aamodt (2004) meta-analysis
• Validity of having a criminal justice major
• ___________________________________
• r k n
• ____ ___ _____
• Patrol performance - .02 9 1,706
• Academy grades .01 6 976
• Discipline problems .00 10 2,621
• ___________________________________

21
Will Education Requirements Result in Adverse
Impact?
22
Adverse Impact
Technique White-Black White-Hispanic Meta-analysis
Cognitive ability 1.10 .72 Roth et al. (2001)
GPA .78 Roth Bobko (2000)
Biodata .33 Bobko et al. (1999)
Reference Ratings .25 Aamodt (2002)
Structured interview .23 Huffcutt Roth (1998)
Recommendations .22 Aamodt (2002)
HS diploma .20 .52 Aamodt (2002)
Bachelors degree .18 .26 Aamodt (2002)
Personality .09 Schmitt et al. (1996)
Integrity tests .07 -.05 Ones Viswesvaran (1998)
23
Lingering Questions

• Is the validity of education job specific?
• What is the actual incremental validity of
  education over cognitive ability?
• Why would education predict performance?
• Knowledge
• Liberal arts skills
• Mental ability
• Motivation

24
Work-Related Training as a Predictor of Job
PerformanceAamodt (2000) Meta-Analysis

• Performance in the academy predicts probationary
  performance (?.44)
• Performance in the academy predicts patrol
  performance (? .38)
• Performance in the probationary period predicts
  patrol performance (? .44)

25
Military Training as a Predictor of Job
Performance

• It is common for Police departments to give
  preferential treatment to candidates with
  Military experience for two reasons
• Reward veterans for serving their country
• They believe veterans will be better cops because
  the training that theyve gone through and that
  of police are similar

26
Validity of Military Experience and Police
Performance
Academy Grades Patrol Performance Commendations
K 9 16 8
N 2,061 4,090 1,337
r .02 - .03 - .07
? .04 - .05 - .10
SE 90 23 81

27
Military Experience in Candidate Selection

• In selecting applicants, the employer must be
  careful because
• Minorities tend to have a higher rate of
  undesirable discharges therefore any questions
  about military service dates, type of discharge,
  and discipline received while in the service can
  result in adverse impact.
• - Gatewood Feild (2001)

28
Experience
29
Experience

• Past behavior predicts future behavior
• Experience is a valid predictor of future
  performance (r .27 Quinones et al., 1995)
• Types of Experience
• Work
• Life

30
Experience

• Evaluated through
• Application blanks
• Resumes
• Interviews
• Reference checks
• Biodata instruments

31
Experience

• Considerations
• How much experience?
• How well did the person perform?
• How related is it to the current job?

32
Experience Predicts Best

• Credit prior work experience only
• In the same occupational area as that in which
  performance is to be predicted
• In the performance of tasks or functions that
  have direct application on the job
• Recency of experience should be used as a
  decision rule for awarding credit only when
  justified on a case-by-case basis
• Credit for duration of work experience should be
  limited to a few years.
• High prediction up to about 3 years of
  experience, declining to low prediction for more
  than 12 years of experience.

33
Experience for Selection Some Concerns

• Sullivan (2000) claims that experience in
  solving past problems is rapidly losing its
  applicability to current and future problems.
• Organizations will increase their applicant pool
  if they delete the ancient history requirements
  (i.e. Ten years experience required).

34
Sullivan (2000)

1. Reduce or eliminate the number of years required
   in your ads and replace them with the
   demonstrated ability to solve problems with our
   required level of difficulty.
2. Use simulations and actual problems to assess
   applicants.
3. Develop future-oriented questions for
   applicants.
4. Train evaluators and compensation professionals
   to put less weight on experience of candidates.
5. Revise job descriptions to include level of
   difficulty.
6. Identify the amount and type of experience and
   competencies that would predict job performance.
7. Check to see if there is a correlation between
   the number of years of experience an employee has
   and their success in your firm.

35
Experience Some More Concerns

• Performance matters
• Havent done doesnt mean cant do
• Experience has a shelf life
• Listing something on a resume is not experience
• Where you get your experience matters
• Experience does not guarantee success
• Experience is expensive
• More experience might be bad (old ways and ideas)