1

Mexico / 30 - 31, May 2016

Transport Systems and Data Science / Guadalajara,
International Virtual Concept Workshop on Intelligent
Advanced Driver Monitoring
for Assistance System
(ADMAS) based on
emotions
AUTHORS: JAVIER IZQUIERDO-REYES, RICARDO A. MENDOZA-RAMIREZ, MARTIN
R. BUSTAMANTE-BELLO
 2
Agenda

1- Introduction
2- ADAS and its classification
3- Driver Monitoring system
4- Investigation precedent
5- Future work

International Virtual Concept Workshop on Intelligent

Transport Systems and Data Science / Guadalajara, Mexico
/ 30 - 31, May 2016
 3
1- Introduction

 Driving safety is a really important issue that is taking a great impact in

research and industrial applications because is a generalized activity that
in some cases implies risk for the driver.

 The topic deals with many aspects involved in this activity such as the
urban environment, the mechanical-electrical status of the car and the
health-emotional state of the driver.

International Virtual Concept Workshop on Intelligent

Transport Systems and Data Science / Guadalajara, Mexico
/ 30 - 31, May 2016
 4
1.1- Introduction to ADAS

 In general, ADAS are monitoring some external factors like the road, the luminescence,
the distance with other cars, etc.

 Main purpose of ADAS is to

 give some alerts to the driver in case of risk (passive mode)
 react and try to avoid an accident (active mode) [AG1].

International Virtual Concept Workshop on Intelligent

Transport Systems and Data Science / Guadalajara, Mexico
/ 30 - 31, May 2016
 5
2- ADAS and its classification

ADAS are classified according to the kind of action that

execute.
 Anders and Chen consider four levels to classify these
systems:
1. Only informative
2. Warning for an imminent risk
3. A guide for the driver to improve performance
4. ADAS takes an action or override the execution of driver
[AC1].

International Virtual Concept Workshop on Intelligent

Transport Systems and Data Science / Guadalajara, Mexico
/ 30 - 31, May 2016
 Jamson [JB1], divided the ADAS according to its characteristics and
6
field of action as follows:

1. Driver monitoring
8. Vision enhancements
2. Reversing/Parking aids

7. Intelligent speed adaptation

3. Lateral control

6. Longitudinal control
4. Road surface/ Low friction warning
& Avoidance

International Virtual Concept Workshop on Intelligent

5. Pre-crash systems
Transport Systems and Data Science / Guadalajara, Mexico
/ 30 - 31, May 2016
 7
Mobileye

 Mobileye is a technology that use artificial vision to provide some ADAS in one
device that analyses the road to detect and alert driver of hazards as pedestrian
collision, Forward collision, headway monitoring, speed limit indication and lane
departure warning

International Virtual Concept Workshop on Intelligent

Transport Systems and Data Science / Guadalajara, Mexico
/ 30 - 31, May 2016
 8
Mobileye Features

1. Left/Right Deviation Alert 1

2. Pedestrian Alert 4
3. Headway Visual Alert (Car icon) 2
4. Headway-distance meter 5
5. Zero Speed indication
6. IHNC Indication 6
3

International Virtual Concept Workshop on Intelligent

Transport Systems and Data Science / Guadalajara, Mexico
/ 30 - 31, May 2016
  In ADAS research field driver monitoring is an issue that has been studied 9
since some years ago by a specific ADAS called driver monitoring systems
[GB1], [YA1] that include drowsiness, fatigue and other physiological
variables.

Temperature sensing

Ambient Temperature

Heart rate monitoring Respiration

http://www.dailymail.co.uk/sciencetech/article-2360694/Ford-car-monitor-HEALTH-
International Virtual Concept Workshop on Intelligent drive--ill-fall-asleep.html
Transport Systems and Data Science / Guadalajara, Mexico
/ 30 - 31, May 2016
 10
3- Driver Monitoring system

 Driver monitoring is subdivided in others classifications according to the field of action in:

1. Driver Behaviour Monitoring

2. Driver Health Monitoring
3. Emotions while driving

International Virtual Concept Workshop on Intelligent

Transport Systems and Data Science / Guadalajara, Mexico
/ 30 - 31, May 2016
 11
Driver Behaviour Monitoring

 Driver behaviour is defined as an uncertain

context characterized by constant interaction
between the driver, the vehicle and the
environment [GS1].

 Most common driver states are: drunk, wayward,

rash, fatigued/drowsy and acceptable driving
[IY1].

International Virtual Concept Workshop on Intelligent

Transport Systems and Data Science / Guadalajara, Mexico
/ 30 - 31, May 2016
 12
Driver Health Monitoring

 This is the most studied monitoring system due to the number of accidents derived of drowsiness,
fatigue and distraction states in subjects while driving.

 Usually this kind of monitoring is made in two Accidents and alcohol 2007 to 2012
ways [LW1]:
1. Contact method: is by physiological features
such as breathing, electroencephalogram
(EEG), electrocardiogram (ECG),
electromyography (EMG), Galvanic Skin
Response (GSR), etc.
2. Contactless: is done by eye tracking, head
movement and facial expressions.

INEGI 2012
International Virtual Concept Workshop on Intelligent
Transport Systems and Data Science / Guadalajara, Mexico
/ 30 - 31, May 2016
 13
Emotions while
driving

 Automotive industry is
following the trend, focusing
in sensations of the driver
aside from the use and the
satisfaction of needs

 Companies want to offer a better environment inside the car to make safe the way
and had a user friendly interface.

 Since psychological features are strongly correlated to physiological characteristics

of driver, the same tools used in driver health monitoring can be used to recognize
emotions.[G1] [LS1] [LS2].
International Virtual Concept Workshop on Intelligent
Transport Systems and Data Science / Guadalajara, Mexico
/ 30 - 31, May 2016
 14
4- Investigation precedent

 An experimentation process developed by B. Sc. Sergio

Navarro was made in emotions recognition using facial
expressions [N1].

 Individual was requested to build a vehicle with Lego™

pieces which does not require more than thirteen steps
for completion.

 The analysis of expression during activity was made using

the FACET module of the iMotions™ tool.

International Virtual Concept Workshop on Intelligent

Transport Systems and Data Science / Guadalajara, Mexico
/ 30 - 31, May 2016
 15
Results of precedent

 Emotions tend to return to its basal status in a period no greater than three
seconds upon the activity completion.

 Also a significant difference in the impact that red light has over fear and
surprise emotions, whereas blue light induced a sadness emotion over the
individuals.

International Virtual Concept Workshop on Intelligent

Transport Systems and Data Science / Guadalajara, Mexico
/ 30 - 31, May 2016
 16
5- Future work

 A new experimental protocol will be done in real driving environment.

 Subjects will be filmed while executing

driving tasks at the same time that stimulus
are applied to try to generate a change in
emotions additional to alerts emitted by
the Mobileye™ system.

International Virtual Concept Workshop on Intelligent

Transport Systems and Data Science / Guadalajara, Mexico
/ 30 - 31, May 2016
 17

 In this experiment factors such as braking,

throttle and steering wheel obtained by
OpenXC system, from an instrumented car will
also be registered to characterize driving style
of Mexican young people.

 The objective of this new experiment is to

detect driver state and synchronize with
vehicle factors in real time to propose a novel
and efficient Advance Driver Monitoring for
Assistance System

International Virtual Concept Workshop on Intelligent

Transport Systems and Data Science / Guadalajara, Mexico
/ 30 - 31, May 2016
 18
References:

 [AC1] L. Anders and F. Chen, State of the Art Analysis: An Overview of Advanced Driver Assistance Systems (ADAS)
and Possible Human Factors Issues. 2007.
 [LS1] Z. Lan, O. Sourina, L. Wang, and Y. Liu, “Stability of Features in Real-Time EEG-based Emotion Recognition
 [AG1] M. Akamatsu, P. Green, and K. Bengler, “Review Article Automotive Technology and Human Factors
Algorithm,” 2014 Int. Conf. Cyberworlds, pp. 137–144, 2014.
Research: Past, Present, and Future,” Int. J. Veh. Technol., vol. 2013, 2013.
 [LS2] Y. Liu and O. Sourina, “EEG databases for emotion recognition,” Proc. - 2013 Int. Conf. Cyberworlds, CW 2013,
 [GB1] R. Grace, V. E. Byrne, D. M. Bierman, J.-M. Legrand, D. Gricourt, B. K. Davis, J. J. Staszewski, and B. Carnahan,
pp. 302–309, 2013.
“A drowsy driver detection system for heavy vehicles,” 17th DASC. AIAA/IEEE/SAE. Digit. Avion. Syst. Conf. Proc. (Cat.
 No.98CH36267), vol. 2,Z.1998.
[LW1] S. Li, L. Wang, Yang, B. Ji, F. Qiao, and Z. Yang, “An active driver fatigue identification technique using
multiple physiological features,” Proc. 2011 Int. Conf. Mechatron. Sci. Electr. Eng. Comput. MEC 2011, pp. 733–737,
 [GS1] A. Gupta, V. Sharma, N. K. Ruparam, S. Jain, A. Alhammad, and M. A. K. Ripon, “Context-awareness based
2011.
intelligent driver behavior detection: Integrating Wireless Sensor networks and Vehicle ad hoc networks,” 2014 Int.
 Conf. Adv. Comput.
[N1] Navarro S DirectCommun. Informatics, pp. 2155–2162, 2014.
Communication

 [IY1]
[YA1]T.G.
Imamura, H. Yamashita,
Yannis and C. Antoniou, Z. “State
Zhang,ofR.art
Othman, and T. Driver
of Advanced Miyake, “A StudySystems,”
Assistance of Classification
The rolefor
of Driver Conditions
Advanced Driver
using Driving Behaviors,” Syst. Man Cybern. 2008. SMC 2008.
Assistance Systems on trafficc safety and efficiency. 2000. IEEE Int. Conf., no. Figure 4, pp. 1506–1511, 2008.
 [JB1] S. Jamson, R. Batley, V. Portouli, V. Papakostopoulos, A. Tapani, J. Lundgren, Y.-H. Huang, E. Hollnagel, and W.
Janssen, “AIDE IST-1-507674-IP - Obtaining the functions describing the relations between behaviour and risk,” no.
March, 2005.
International Virtual Concept Workshop on Intelligent
Transport Systems and Data Science / Guadalajara, Mexico
/ 30 - 31, May 2016