Establishing a Worldwide

1
Establishing a Worldwide Driving Simulator
Scenario Library Matthew Rizzo Division of
Neuroergonomics University of Iowa, Department
of Neurology Iowa City, Iowa USA HFES, New
Orleans September 22, 2004

2
Introduction

• This brief talk outlines advantages of developing
  a standardized shared library of driving
  simulator scenarios -- and hopefully serves as a
  call to action.
• Having standards for driving simulator a common
  library of standardized driving simulator
  scenarios is essential.
• Why?

3
Introduction

• Simulators are an important means for safe and
  objective assessment of performance capabilities
  in normal and impaired automobile drivers.
• Yet, substantial problems that limit the utility
  of simulator research.

4
Introduction

• Simulation is advancing, yet it remains a cottage
  industry of home grown devices with little
  overall standardization. This is a major
  hindrance to research.
• Key problems facing simulator users are related
  to
• simulator graphics, audio, and movement
• scenario design and validation
• subject adaptation and comfort
• selection of performance measures
• technical standards for reporting experimental
  set-up and results.

5
We Need Standard Scenarios

• Standards for simulation and and simulator
  scenarios can facilitate comparisons of operator
  performance across different simulator platforms.
• Availability of standards for simulation and
  simulator scenarios can facilitate collection of
  large amounts of data at different institutions
  with greater power for addressing worldwide
  public health issues related to driving.
• Comparisons against standards can clearly reveal
  missing descriptors and other weaknesses in ever
  mounting numbers of driving simulator research
  reports.

6
Issues in Standardization

• How well can most simulator scenarios be
  described within a common framework?
• Should researchers be expected to implement a key
  scenario just one way?
• Can levels of fidelity be adequately specified
  for cross-platform comparisons?
• How can we overcome vested interests and
  entrenched opinions on simulators?

7
Clinical Trials

• We need simulator standards to implement clinical
  trials in drivers with a variety of medical
  impairments.
• NIH definition A clinical trial .. is a
  research study in human volunteers to answer
  specific health questions. Clinical trials are
  the fastest and safest way to find treatments
  that improve health. Interventional trials
  determine whether experimental treatments .. are
  safe and effective under controlled environments.
  Observational trials address health issues in
  large groups of people or populations in natural
  settings.
• http//www.clinicaltrials.gov/ct/info/whatiswhati
  s
• A clinical has "some formal structure of an
  experiment, particularly control over treatment
  assignment by the investigator" (Piantadosi,
  1997, "Clinical Trials A Methodologic
  Perspective).

8
Many dimensions to consider

• Dr. Jeff Greenberg, Chief of VIRTTEX at Ford's
  Research Laboratory, reviewed issues in simulator
  fidelity relevant to standardization (SUG _at_ TRB,
  1/13/04)
• http//www.uiowa.edu/neuroerg/index.html).  
• Simulator fidelity and infidelitycan be
  characterized along 21 key dimensions along 3 key
  domains
• Cueing systems Visual, Auditory, Haptic or
  Tactile, and Vestibular
• Physical environment Cab, Laboratory, Virtual
  environment
• Visual world Terrain and roadways, and Vehicle
  Models

9
There is hope

• Some differences in physical fidelity may not
  affect psychological fidelity
• The driver in the loop may behave similarly
  across a range of small physical differences in
  simulation
• Meaningful measures may even be had in a surreal
  simulation
• Standards are within our reach, with a promising
  payoff in new research.

10
Clinical trials in driving safety examples

• Cervical immobility
• Reduced visual fields
• Ecstasy, marijuana and other illicit drugs
• Parkinsons Disease
• Shoulder (rotator cuff) surgery with sling, leg
  casting
• Dialysis
• Attention Deficit Disorder
• Cirrhosis
• Detrussor spasm in women
• Low back disorders
• Ambulatory surgical/anesthesia restrictions
• Post abdominal surgery
• Anti-hypertensive medication effects
• NPH pre and post-shunting
• Post ambulatory anesthesia

11
Things to agree on

• Scenarios
• Measures
• Standards of reporting in journals
• Subject selection criteria
• Reporting of independent measures
• Standards for physiologic recording
• Visuals, audio, haptic cues
• Training criteria
• Simulator adaptation
• What is a simulation
• Criteria for validation

12
HASTE effort

• Mandatory driving performance measures
•  Speed
• Speed variation
• Lateral position
• Lateral position variation
• Lane exceedences
• Time to line crossing
• Reversal rate
• Time To Collision, Time headway and Distance
  headway Brake reaction time

13
HASTE effort

• Optional driving performance measures
• High frequency component of steering wheel angle
  variation  
• Steering entropy
• High risk overtakings
• Abrupt onsets of brakes
• Post Encroachment Time

14
HASTE effort

• Mandatory workload measures
• Rating Scale Mental Effort (RSME)
•  Glance frequency
• Glance duration
• S-IVIS performance
• Situation Awareness Global Assessment Technique
  (SAGAT)
• Optional workload measures
• Heart Rate (HR) and Heart Rate Variability (HRV)
• Respiration
• Skin Conductance

15
Rationale for selecting preliminary scenarios for
a standard library

• Importance in the epidemiologic record -- focus
  on key situations conveying potential high crash
  risk, e.g.
• Run off the road on curves
• Rear end collision
• Intersection incursion avoidance
• Interaction with emergency vehicles/pedestrians
• Merging with the potential for side impact
  collisions
• Scenarios that address behavioral effects of
  using in-vehicle devices, e.g., cell phone,
  information displays

16
Sample Format presented at SUG 1/13/04

• Sample Format for Describing a Simulator
  Scenario Lane Change with Slow Traffic (SUG
  1/13/04)
• Specifications
• Script
• Cognitive constructs stressed
• Dependent measures
• Independent variables
• Implementation challenges
• Testing Validity of Scenario
• Bibliography

17
Sample Scenario Description

• Script The driver is traveling on a road with
  two lanes of traffic, each moving at different
  speeds. At different times, one lane of traffic
  is moving more advantageously (faster) although
  overall this may be the slower lane. The drivers
  task is to pass through traffic as quickly as
  possible.
• This task would be similar to the Gambling Task
  (Bechara et al, 1994) in which an individual has
  to overcome an immediate reward to ensure
  long-term benefit.
• Variations can be made on this script the two
  lanes of traffic could average the same speed or
  could even be moving at the same speed. The
  drivers perception that one lane is moving
  faster may be a visual and cognitive illusion.

18
Sample Scenario Description

• Cognitive constructs stressed Attention,
  perception, and decision-making.
• Dependent measures Time it takes the driver to
  maneuver through the traffic to arrive at a
  destination, number of navigation errors, and
  number of moving violations or safety errors
  (e.g., excessive speed, near misses, and
  collisions) could be recorded.
• Independent variables The number of vehicles
  involved, number of lanes, speed of the different
  lanes and final destination, and travel
  contingencies could be manipulated (e.g., the
  driver could be instructed to get off at a
  specific exit for a hospital).

19
Sample Scenario Description

• Implementation challenges
• It may be difficult to create a realistic feel.
  The problem may be lessened by giving the driver
  external instructions, thus altering driver
  expectations and rewards or incentives. For
  instance, the instructions could be to drive as
  if you were taking a pregnant woman or a
  critically sick person to the emergency room.
• Is it possible for the driver to be able to
  change lanes when desired?
• How will the surrounding cars respond to the
  driver?

20
Sample Scenario Description

• Measurement challenges
• Some drivers will not try to get in the faster
  lane, if they think it makes little difference in
  the long run.
• A questionnaire following the task may be helpful
  in assessing how fast the driver felt each lane
  was moving, and whether or not they the driver
  have changed lanes if given the opportunity.
• Variations in personalities would have to be
  considered.

21
Sample Scenario Description

• Testing Validity of Scenario An instrumented
  vehicle could be used to study lane change
  behavior during times of high- density traffic,
  yet the environmental variables could not be
  easily controlled.
• Bibliography
• Bechara , A., Damasio, A. R., Damasio, H.,
  Anderson, S. W. Cognition , 50, 7-15 (1994)
• Redelmeier, D.A. Tibshirani, R.J. Nature 401,
  35 (1999).

22
Sample Scenario Description

• Testing Validity of Scenario An instrumented
  vehicle could be used to study lane change
  behavior during times of high- density traffic,
  yet the environmental variables could not be
  easily controlled.
• Bibliography
• Bechara , A., Damasio, A. R., Damasio, H.,
  Anderson, S. W. Cognition , 50, 7-15 (1994)
• Redelmeier, D.A. Tibshirani, R.J. Nature 401,
  35 (1999).

23
Value of scenario plots for visualizing driver
response
What would you do if you saw a police car parked
on the side of a 2-lane highway with no vehicles
coming in the other lane?
24
Value of task analysis Response to darting dog
ahead, honking SUV behind
e-dog
Brake application
25
Honking SUV in rear view
Gas reaction 0Nothing, 1 Release, 2Press
Frame
At-fault safety errors Quantitative vehicular
control measures
1s
Brake reaction 0Nothing, 1 Release, 2Press
Frame
Dog darts to road from right
Parkinsonism 0No change, 1Worse, 2Better Frame
Frame, speed
Gas reaction 0Nothing, 1 Release, 2Press
Frame
At-fault safety errors Quantitative vehicular
control measures
Brake reaction 0Nothing, 1 Release, 2Press
Frame
Steering reaction 0Straight, 1 Right, 2Left
Frame
Parkinsonism 0No change, 1Worse, 2Better Frame
Outcome (Frame, speed) 1Avoids dog, no crash
with oncoming traffic 2Hits dog, no crash with
oncoming traffic 3Avoids dog, but crashes with
oncoming traffic 4Hits dog and crashes with
oncoming traffic
26
Summary

• General standards are lacking for scenario design
  and dependent measures.
• Lack of standards limits research and testing
  replication.
• Representations of simulator scenarios in a task
  analysis format help to clarify the logical
  structure of scenarios and related issues such as
  operational definitions and measurements of
  variables.
• Joint consideration of these issues will aid in
  the creation of a proposed driving scenario
  library.
• The effort will contribute to an infrastructure
  for worldwide research on critical public health
  issues concerning driver performance and vehicle
  design, with the support of major funding
  agencies such the NIH and NHTSA.

27
How shall we proceed?

• We for your expert opinions and advice
• Which key scenarios are most worth implementing
  across different simulator platforms for clinical
  trials.
• How should these scenarios be represented?
• Please Contact
• Matt Rizzo (matthew-rizzo_at_uiowa.edu) or
• Jeff Caird (jkcaird_at_ucalgary.ca)

28
Action Items

• Continue to examine key topics on
  standardization, physical and psychological
  fidelity, reporting of methods, selection of
  variables and scenarios, and simulator discomfort
  in simulation research.
• Identify funding sources for addressing these
  concerns in anticipation of future research
  trials (e.g., involving impaired drivers or
  patients and on board safety devices).
• Prepare a white paper outlining suggestions for
  standards
• Continued efforts to spread the word
• TRB Simulator Workshop 2005 (Fisher, Mourant,
  Rizzo)
• Driving Assessment 2005, including SUG meeting
• (Rockport ME, June 27-30).