Institute of Integrated Sensor Systems Department of Electrical and Computer Engineering

1
Institute of Integrated Sensor
SystemsDepartment of Electrical and Computer
Engineering
Multi-Sensor Soft-Computing System for Driver
Drowsiness Detection Li Li, Klaudius Werber,
Carlos F. Calvillo, Khac Dong Dinh, Ander
Guarde and Andreas König 10-Dec-2012

• Introduction
• Driving Scene Modeling and Hardware Setup
• Software Components and Algorithms
• Experimental Results
• Conclusion and Future Work

2
Introduction

• Major factor in 20 percent of all accidents in
  the United States in 2006
• The second most frequent cause of serious truck
  accidents on German highways
• Major damage caused by drowsy truck or bus drivers

Enhance active safety with advanced driver
assistance
3
Hardware Setup

• DeCaDrive System
• Multi-sensing interfaces
• Depth camera
• Steering angle sensor
• Pulse rate sensor
• PC-based soft-computing subsystem
• PC-based driving simulator

4
Hardware Setup

• SoA depth camera
• Extention of 2D image with distance
• Wide field of view
• Relatively low computational cost
• Robust to lighting variations (active sensing)
• Non-intrusive and non-obstructive (eye-safe NIR
  light source)

5
Hardware Setup

• Steering angle sensor
• Steering behavior of driver
• Correlation with driver status and driver
  intention

• Pulse rate sensor
• Heart health and fitness
• Time domain analysis
• Frequency domain analysis

6
Software Components and Algorithms

• Overview of the data processing flow

7
Feature Computation

• Features being computed from multiple sensor
  measurements

8
Experimental Results

• Test subjects
• Five male test subjects
• 22 to 25 years old (mean 23.6, std1.1)
• All have drivers license for at least 4 years
• No alcohol drinking before test
• Experiments
• One hour driving simulation for each test subject
• 588-minute driving sequence recorded
• Ground truth not drowsy, a little drowsy, deep
  drowsy
• Through self-rated score and response time

9
Experimental Results

• Examples of different sensor features

blink frequency
low steering percentage
mean pulse rate
10
Experimental Results

• Screenshot of online processing of various sensor
  data

Eye pupil and corners
Depth image
11
Experimental Results

• Results of ANN based classifier with two training
  algorithms

12
Experimental Results

• Confusion matrix of LM
• 80 hidden neurons
• 10-fold cross-validation

• Confusion matrix of SCG
• 40 hidden neurons
• 10-fold cross-validation

13
Experimental Results

• Drowsiness level classification accuracy
  depending on selected features

14
Conclusion and Future Work

• Contribution
• Emerging framework for driver status monitoring
  and intention detection with multi-sensor
  soft-computing system
• Classification of three different drowsiness
  levels with up to 98.9 accuracy based on data
  sets of ?ve test subjects.
• Future work
• Validation with more statistics and with data
  from real vehicles
• Variance compensation by adaptive learning
• Optimization of feature selection with
  sophisticated heuristics
• Utilization of other advanced classification
  techniques, e.g., SVM
• Integration of more embedded sensors with
  wireless technology

15
Thank you!