The Situational Interview

1
The Situational Interview

• www.mubeena.net

2
Definition

• Job-related situational questions
• Situations are specific on-the-job activities
• Applicants put in hypothetical situations
• Questions can refer to past experience or future
  intentions
• Differs from structured format by benchmarking
  applicant responses

3
EvolutionTree
Redefinition of Selection Tool A valid test
cannot be developed until the organization agrees
upon an acceptable definition (measure) of
employee behavior. Latham Wexley, 1982
Workforce Prediction Quality of workforce will
deteriorate Dyer, 1981
Legal Issues Selection Process Dilemma Griggs vs.
Duke Power, 1971
4
Research Literature

• Latham, Saari, Pursell, Campion, 1980
• Examined reliability and validity of situational
  interview by conducting three studies
• Study 1 entry-level position, Study 2
  first-line supervisory position, Study 3
  concurrent predictive validity
• Results for Study 1 Interview scores were
  significantly correlated with every performance
  criterion including the overall global rating
• Results for Study 2 Interview scores correlated
  significantly with three of four criteria
  safety, work habits, org commitment
• Results for Study 3 Interview scores
  significantly correlated with composite job
  performance scores, even for women and black
  employees

5
Research Literature

• Conclusions of Latham et al. (1980)
• Intentions correlate with behavior
• Comprehensive job analysis reflect content
  validity
• Interviewee motivation influenced by face
  validity
• Inter-rater reliability high because interviewers
  themselves developed scoring key
• Emphasis on critical behaviors rather than traits
• Past behavior based items could lessen dishonest
  responses, it can be verified by previous
  employer. However, adverse impact must be kept in
  check

6
Research Literature

• Pulakos Schmitt, 1995
• Compared validities of experience-based (past
  performance) questions and situational (future
  intentions) questions
• Results Past performance is better predictor of
  job performance
• Lack of Validity for Situational Items Attributed
  to
• Responses were evaluated at end of interview. It
  is necessary to make the ratings immediately
• Prior studies indicate that situational
  interviews involve lower-level jobs. In this
  study, the job was complex and demanding
• Major conclusion was that interviews result in
  lower levels of adverse impact because they
  measure cognitive as well as non-cognitive
  performance dimensions

7
Research Literature

• Maurer Fay, 1988
• Two hypotheses Rater training
  Inter-rater agreement
• Situational
  Inter-rater agreement
• Interpretation of results Situational interviews
  are more effective in producing higher
  inter-rater agreement, even with little or no
  training, because of specific rating scales
• There was so significant main effect of training
• Situational interview is more cost-effective
  strategy in comparison with conventional
  structured interview

8
Critical Incident Technique

• Was first developed and used by John Flanagan and
  his students at the University of Pittsburgh in
  the late 1940s and early 1950s
• Used to identify job behaviors that differentiate
  successful performance from unsuccessful
  performance
• Can be useful in job analysis as well as training
  and performance appraisal
• The development of the graduate situational
  interview was based on this technique

9
Critical Incident Development

• Critical incidents were identified by reviewing
  the job analysis (CMQ)
• Behaviors that could exemplify either good or
  poor behaviors of graduate students were chosen
• Fourteen job dimensions were identified
• Prioritizing Leadership
• Stress management Ethics
• Ability to work with others Public speaking
• Receiving feedback Altruism
• Self Assessment Diversity
• Decision making Retention
• Group involvement
• Participation

10
Development of Situational Interview Questions

• The critical incidents were used to form the
  questions to be asked in the situational
  interview
• The beginning of each question stem entailed a
  description of a circumstance involving a
  critical incident
• Example Critical Incident Decision Making
• Example You are placed at a company in which
  your supervisor is critical and provides no
  direction to complete a task
• The questions ended with a general proposition
• Example How would you handle this situation?
• Seventeen questions were written using this
  format
• Each critical incident that we identified had a
  corresponding question
• Two of the critical incidents had more that one
  corresponding question, namely stress management
  and ethics

11
BARS Development

• To facilitate the scoring process a behaviorally
  anchored rating scale was developed for each
  question
• The behavioral scales were developed by
  brainstorming and determining KSA relevance and
  importance as rated by SMEs (job incumbents)
• Our anticipation of the responses that we would
  receive was used to develop a five point scale
• The five point scales included examples of high,
  average and low responses
• The examples that we agreed represented high,
  average and low responses were used as behaviors
  on the scale

12
Example

• 5 Applicant tries to talk to the boss
• 4 Applicant tries to set own goals and
  direction no resolve of conflict.
• 3 Applicant says nothing.
• 2 Applicant complains, does not start the
  job until the issue is addressed.
• 1 Applicant leaves the job/talks to boss in
  a negative fashion.

13
Focus Groups

• Our questions and behavioral scales were
  presented to a focus group
• The focus group (subject matter experts)
  recommended the following
• - add multitasks to the scales
• - reword ambiguous and vague questions
• - make sure that each question corresponds to
  the critical incident it is describing in a
  situation

14
Scoring of the Situational Interview

• Each response on the behavioral scale had a
  corresponding score
• Scores were rated from one to five
• - a 1 or 2 represented a low response
• - a 3 represented an average response
• - a 4 or 5 represented a high response
• The interviewers choose the response that best
  represents the interviewees answer
• A total score for the interview can be obtained
  by summing the ratings for each question

15
Development of Cutoff Score

• The Ebel method was used to determine the cutoff
  score
• - rate each item (1-5)
• - determine the percentage of items a minimally
  qualified candidate would respond to correctly (a
  correct response to each item represented a score
  of 3 or more)
• - multiply this percentage by the number of
  items
• (3 x 17 51)
• - the cutoff score becomes 51 for the
  situational interview

16
Administration

• Participants 30 I/O graduate students
• Two groups of raters 3 and 2
• Selection Tool was administered twice over a 2
  week period.
• The Tool was administered in a classroom setting
  with other selection tests being administered at
  the same time.

17
Possible Threats To Validity

• Random Error
• Noise/Disruption Effects
• Exposure to test questions before administration
• Fakeablity
• Rater effects(Halo, Contrasts, Leniency Effects)

18
Descriptive Data

• Mean64.21
• Standard Deviation5.78
• Range27.67
• Skewness-.845

19
Score Distribution
20
Validity

• Correlation with
• -Graduate GPA? r.284, p.064
• -Psychology 249? r.360, p.025
• -Psychology 283A? r.381, p.019
• -Undergraduate GPA? r.252, p.089
• -GRE? r -.007, p.485

21
Near Future

• Standardize scores
• Reliability and more validity analysis
• Analyze individual questions

22
Distant Future

• How can we modify the interview to make it a
  better instrument for selecting graduate
  students?
• separate information gathering and evaluation,
  evaluate after gathering
• modify behavioral anchors
• re-categorize questions so that questions which
  co-vary are in the same category