Detecting Driver Use of Mobile Phone Based on In-car Camera
Dan Wang
1
, Mingtao Pei
1
and Lan Zhu
2
1
Beijing Laboratory of Intelligent Information Technology
School of Computer Science, Beijing Institute of Technology, Beijing 100081, P.R. China
{wangdan5086, peimt}@bit.edu.cn
2
China North Vehicle Research Institute
Beijing 100072, P.R. China
Abstract—It is dangerous for drivers to make a call while
driving, as it could easily divert the drivers’ attention. In this
paper, we present a novel method to detect the driver use of
mobile phone based on an in-car camera. The in-car camera
is mounted on the front windshield to capture the video of
the driver during driving, and an activity parsing algorithm
is employed to identify whether the driver is using a mobile
phone. We decompose the phoning activity into three actions
and use the And-Or Graph (AoG) to represent the
hierarchically compositions of the phoning activity and the
temporal relationship between the actions. An online parsing
algorithm for AoG based on Earley’s parser is implemented
to parse the video and detect the driver use of mobile phone.
Experiment results on collected video data set shows that the
proposed method can detect the driver use of mobile phone
precisely. This method could be used
as a supplement of the
safety device inside the vehicle.
Keywords-driver assistant system; in-car camera; driver
use of mobile phone
I. INTRODUCTION
Many studies [1-2] show that driving while talking on
hand-held phone increases the risk of accident. When
drivers are using mobile phone, their reaction time to
outside events are thirty percent longer than driving with
a blood alcohol level over the legal limit [3]. Other
studies have shown that almost 70% of fatal traffic
accidents are attributed to driver’s distraction, and
hand-held phone using is one of the major reasons of the
distraction. Although many counties have made laws to
penalize the use of mobile phone while driving [4], the
results are not optimistic, as the supervising of mobile
phone use is difficult.
Currently, the driver use of mobile phone is mainly
supervised by law enforcement officials through direct
observation. It was time-consuming and inefficient.
Therefore more effective and automatic supervising
methods were needed to detect this illegal behavior.
However, little work has been done on this aspect. Okuda
[5] proposed a method to set a receiver outside the car in
a stationary place to receive signals from the driver; José
Manuel et al. [6] suggested put antennas on the seat inside
the car. However, their methods were easily affected by
many disturbances, such as the weather and the
passenger’s phoning activity.
Our paper presents a novel method to detect driver use
of mobile phone based on the video captured by an in-car
camera. As far as we know, there is no study on detecting
driver’s phoning activity in computer vision aspect.
When driver makes a phone call by a mobile phone,
usually his hand will be near his ear with the phone, and
his lip will be open for some time during the phoning
process. Therefore we use the AdaBoost algorithm and
skin color to detect the driver’s face and hand, and then
we can detect the phoning activity by driver’s gesture and
lip state. However, as the car is moving, the background
is complex, it is possible to contain many changing
background areas in the image. When the background
area, locating near the ears, has the similar color as the
skin color, it may be detected as hand, and causes many
false detections. As we can observe that the phoning
activity is a sequential behavior, it is usually composed of
three actions, those are putting hand up, phoning and
putting hand down. We can use the putting hand up and
down actions to filter the false detections. Based on the
above observation, we model the phoning activity by the
And-Or Graph (AoG), which can represent the
hierarchically compositions of the activity and the
temporal relationships between the actions, and employed
an activity parsing algorithm to identify whether the
driver is using a phone.
This method could be used as a supplement of the
safety device inside the public vehicles, such as taxi, bus,
truck, train and so on. Nowadays, many of these vehicles
have mounted in-car cameras in front of the driver for
safety especially in the shuttle buses; these cameras can
be used directly for phoning detection. If the driver is
phoning while driving, the system will alert the driver,
and will transport a photo as evidence to the traffic
official if the driver continues phoning.
The rest of the paper is organized as follows: Section2
introduces the AoG representation of the phoning activity.
Section3 and Section4 describe the detection of the three
actions in phoning activity. Section 5 discusses the
detection process. Section 6 presents the experiment
results, and Section 7 concludes this paper.
II. MODELING PHONING ACTIVITY BY AOG
The And-Or Graph(AoG) was first introduced to
computer vision in [8] and [9] and has been utilized in [10]
to parse video events. AoG is composed of And-nodes,
Or-nodes, Leaf-nodes and temporal relations between the
nodes. And-nodes represent the composition and
2014 10th International Conference on Computational Intelligence and Security
978-1-4799-7434-4/14 $31.00 © 2014 IEEE
DOI 10.1109/.11
148
2014 10th International Conference on Computational Intelligence and Security
978-1-4799-7434-4/14 $31.00 © 2014 IEEE
DOI 10.1109/CIS.2014.12
148
2014 Tenth International Conference on Computational Intelligence and Security
978-1-4799-7434-4/14 $31.00 © 2014 IEEE
DOI 10.1109/CIS.2014.12
148