SHRP 2 Safety Research Program Naturalistic Driving Study

1
SHRP 2 Strategic Highway
Research Program

Charles Fay, Sr. Program Officer
Big Data Meets Computer Vision
Dec. 7, 2012
2
Looking for Big data well have
some

• Like challenges?
• Then you should be excited by SHRP 2 NDS
• 4 petabytes of data that need to be
  post-processed
• 1 million hrs of video
• 3000 subjects, 5 million trips, gt18 million
  miles driven, 4 billion GPS points
• Real world - automotive conditions (daylight
  variance nighttime IR) low quality cameras
  images
• Data compressed(H264) and saved at 15 Hz 
• PII (personal identifiable information)
  protection of privacy
• Patience with getting access to data- working out
  details
•  

3
National Academy of Sciences
Advisors to the Nation on Science, Engineering,
and Medicine To "investigate, examine,
experiment, and report upon any subject of
science or art - whenever called upon to do so by
any department of the government Transportation
Research Board (TRB) is one of six major
divisions
Est. 1863
4
Content

• Whats the Problem(s)?
• Preview video data
• Naturalistic Driving Study (NDS)
• Roadway Information Database(RID)
• FHWA Exploratory Advanced Research Program
• Goal today
• promote interest in mining these data
• making these data more usable

Ultimately saving lives/ reducing severity of
injury
5
What is the problem(s)?

• Public Health Highway Safety
• Crashes leading cause of death
• for 4-34 year old (US)
• 40,000 total deaths in US/year
• 2.5-3.0 million injuries /yr in US
• Estimated costs 230 billion/ yr in US
• Driver behavior has been identified as the major
  factor in 90 -95 of roadway crashes (know very
  little about behavior )
• Major issue around the world Naturalistic
  driving studies in EU, China, Australia others
  in development- way of the future

wot.motortrend.com
6
What is the problem(s)?

• Computer Vision
• Before analysts can use the full NDS dataset
  more usable form that is where you come in
• Lots of data from 3000 participants
• 4 petabytes 1 million hrs video other
  sensor data 5 million trips gt 18 million miles
• Saved video poorer quality relative to what you
  are used to analyzing.
• PII (personal identifiable information) data
  access (working out details-patience please)
• recording continuously GPS face video

7
Sample areas for extraction from video

• Driver related

• Context related

• Driver behavior
• Distraction
• Head pose
• Eye gaze
• Fatigue/drowsiness
• Mobile device use
• Hand position
• Foot/pedal

• Traffic signal state
• Roadside information
• Weather, pavement conditions
• Bike/ Pedestrian
• Other vehicles (brake lights) traffic

8
Ready for a mission?

• Your challenge should you choose to accept
• working on post processing these data in an
  efficient manner to gain meaningful information

kmnnz.wordpress.com
http//kellypuffs.wordpress.com
9
Benefits of the Study (safety related)

• These data are not available one of a kind
  database(s) decades of use
• Almost Everyone (OEMs, DOTs, researchers) eager
  to get hands on these data
• Intelligent / automated/connected vehicles
  transportation
• Improved understanding of baseline driving
  behaviors
• Trip characteristics
• Driver performance profiles
• Adherence to laws and basic safety practices
• Improved understanding of unsafe behaviors and
  traffic events
• Assess circumstances and motivations for
  speeding, red light running, etc.
• Deconstruct crashes and near-misses and examine
  causality
• How do driver, vehicle, roadway, and
  environmental factors influence behavior and
  impact crash risk?
• Improved ability to develop safety
  countermeasures for
• Education and training
• Roadway design and traffic engineering

10
Camera Image Samples
What can be done post-processed? Video saved _at_
15Hz H 264 compression
Driver Face Rotated for max pixel efficiency
Forward View - color
Right-Rear View
Center stack Pedal Interactions hands
Periodic still cabin image, permanently blurred
for passenger anonymity (child safety seat use?)
11
480x360 Scaled full to 480x360
240x360 Scaled full to 360x240 or cropped at
360x240 and Rotated 90 degrees
360x120 Scale Vertical by ¼ horizontal by 1/2
360x120 Crop 25 off top and Bottom then Scale by
1/2
12
Specs camera and saved video
13

SHRP 2 Safety Research Program
Naturalistic Driving Study
14
SHRP2 Naturalistic Driving Study (NDS)
Roadway Information Database (RID)
NDS
RID
15
NDS data linkable to Roadway Information
database (RID) A systems
approach driver, vehicle, and context look
at Interactions
Existing Data characterize the environment in
which the participant/ DAS operates roadway,
crash, safety campaigns, laws, traffic, weather,
work zones linked to roadway segment
Query
NDS Data
RID (GIS)
(DAS GPS is Link)
1950 DAS 3000 participants 5 million
trips Passenger Car, Van, SUV, Pickup
Requested Data
New Roadway Data Collected and QA
Analysis
Application of Results Safety Countermeasures
16
Six NDS Data Collection Sites across the U.S.
One Coordinator
NY Data Collection
IN Data Collection
PA Data Collection
NC Data Collection
FL Data Collection
17
NDS Data

• Driving data from instrumentation on vehicle
• Driver data from questionnaires, tests
• Vehicle data vehicle inspection CANbus-vehicle
  network
• Crash data detailed investigation of selected
  crashes
• Will include both restricted and non-restricted
  data requiring various levels of access
• Restricted data that which may be used to
  identify a participant, such as face video or
  GPS. Requires high level of physical and
  electronic security, data access agreements,
  ethics review, oversight. Working on specifics
  for data access (remote enclave(s) being
  considered)
• Non-restricted data can be disseminated more
  widely via web access, summarized data sets,
  numerical variables

18
Data Acquisition System (DAS)
18
19
DAS Overview
DAS Overview

• Illuminance sensor
• Infrared illumination
• Passive alcohol sensor
• Incident push button
• Audio (only on incident push button)
• Turn signals
• Vehicle network data
• Accelerator
• Brake pedal activation
• ABS
• Gear position
• Steering wheel angle
• Speed
• Horn
• Seat Belt Information
• Airbag deployment
• Many more variables

• Multiple Videos
• Machine Vision
• Eyes Forward Monitor
• Lane Tracker
• Accelerometer Data (3 axis)
• Rate Sensors (3 axis)
• GPS
• Latitude, Longitude, Elevation, Time, Velocity
• Forward Radar
• X and Y positions
• X and Y Velocities
• Cell Phone
• ACN, health checks, location notification
• Health checks, remote upgrades

20
Estimated Completed Collection
3500-3900 total vehicle years
21
Roadway Information database
3 Mix/Combo
3 Mix/Combo
4 Mix/Combo
Paved Shoulder
N/A
Median
Flush Paint.
N/A
N/A
N/A
Flush Paint.
Flush (Painted)
Flush (Painted)
Lanes
Thru Lane 1 (11) Right Turn 1
Thru Lane 1 (12)
Thru Lane 1 (12)
Thru Lane 1 (14) Deccel. Lane 1
Thru Lane 2 (11) Deccel. Lane 1
Thru Lane 1 (12) Left Turn Lane 1
Thru Lane 1 (21)
Thru Lane 1 (12) Accel. Lane 1
Lanes
N/A
N/A
N/A
Flush Paint.
Flush (Painted)
Flush (Painted)
Flush Paint.
Median
2 Mix/Combo
0 Mix/Combo
3 Mix/Combo
2 Mix/Combo
Paved Shoulder
Grade, Cross Slope
Unpaved Shoulder N/A
Rumble Strips N/A
Lighting N/A
22
New Data Collected at Highway Speed by
S04B(Fugro-Roadware)

• Horizontal Curvature Radius , Length ,PC , PT
  ,Direction
• Grade
• Cross Slope/ Super Elevation
• Lane in terms of the number, width, and type (
  turn, passing, acceleration, car pool, etc)
• Shoulder type/curb paved width if exists
• Intersection location , number of approaches, and
  control (uncontrolled, all-way stop, two-way
  stop, yield, signalized, roundabout). Ramp
  termini are considered intersections
• Posted speed limit sign and location (R2-4
  Series)
• Median presence(Y/N), type (depressed, raised,
  flush, barrier)
• Rumble Strip presence(Y/N) location (centerline,
  edgeline, shoulder)
• Lighting presence( Y/N)
• FHWA determining if additional data types will be
  processed (e.g., All MUTCD signs barriers - TBD)
  

23
1920 x 1080 every 21 feet on 25K miles
Front ROW Images
Route Name
Direction
Chainage
State
Collection Date
24
Context (supplemental data)
Item Priority
1 Crash Data 1
2 Traffic Information - AADT 1
8 Aerial Imagery 1
9 Speed Limit Data 1
10 Speed Limit Laws 1
11 Cell phone and text messaging laws 1
12 Automated enforcement laws 1
13 Alcohol-Impaired and Drugged Drivers laws 1
14 Graduated driver licensing (GDL) laws 1
15 State motor cycle helmet use laws 1
16 Seat Belt Use laws 1
5 Local Climatological Data (LCD) NOAA 1
17 Cooperative Weather Observer/Other Sources 1
4 Winter Road Conditions (DOT) 2
3 Work Zone 2
24 511 Information 2
18 Traffic Data - Continuous Counts (ATR) 2
19 Traffic Data -Short Duration Counts 2
21 Changes to existing infrastructure condition 2
22 Roadway Capacity Improvements 2
6 Nonrecurring Congestion 3
20 Automated Enforcement 3
7 Travel Time Data 3
23 Innovative Treatments 4
25 Recurring Congestion 4
25
SHRP2 Data Sharing Websitehttp//forums.shrp2nds.
us/
26
Sample data set-spring 2013

• Working on providing data from 24 individuals
• 45 min per driver
• Variety of facial features
• Glasses/sunglasses
• Daytime/ nighttime conditions
• IRB consent form allow data to be shared for
  research purposes
• May need your IRB approval- most likey expedited
  review

27
Federal Highway (FHWA)

• Exploratory Advanced Research Program
• Video analytics workshop 10/10-11/2012
• summary report by January 2013
• http//www.fhwa.dot.gov/advancedresearch/
• David.Kuehn_at_dot.gov
• Lincoln.Cobb_at_dot.gov

28
Questions

• Charles Fay
• cfay _at_nas.edu
• 202-334-1817