Resumen de Face tracking with active models for a driver monitoring application
- español
La falta de atención durante la conducción es una de las principales causas de accidentes de tráfico. La monitorización del conductor para detectar inatención es un problema complejo, que incluye elementos fisiológicos y de comportamiento. Un sistema de Visión Computacional para detección de inatención se compone de varios etapas de procesado, y esta tesis se centra en el seguimiento de la cara del conductor. La tesis doctoral propone un nuevo conjunto de vídeos de conductores, grabados en un vehículo real y en dos simuladores realistas, que contienen la mayoría de los comportamientos presentes en la conducción, incluyendo gestos, giros de cabeza, interacción con el sistema de sonido y otras distracciones, y somnolencia. Esta base de datos, RS-DMV, se emplea para evaluar el rendimiento de los métodos que propone la tesis y otros del estado del arte. La tesis analiza el rendimiento de los Modelos Activos de Forma (ASM), y de los Modelos Locales Restringidos (CLM), por considerarlos a priori de interés. En concreto, se ha evaluado el método Stacked Trimmed ASM (STASM), que integra una serie de mejoras sobre el ASM original, mostrando una alta precisión en todas las pruebas cuando la cara es frontal a la cámara, si bien no funciona con la cara girada y su velocidad de ejecución es muy baja. CLM es capaz de ejecutarse con mayor rapidez, pero tiene una precisión mucho menor en todos los casos. El tercer método a evaluar es el Modelado y Seguimiento Simultáneo (SMAT), que caracteriza la forma y la textura de manera incremental, a partir de muestras encontradas previamente. La textura alrededor de cada punto de la forma que define la cara se modela mediante un conjunto de grupos (clusters) de muestras pasadas. El trabajo de tesis propone 3 métodos de clustering alternativos al original para la textura, y un modelo de forma entrenado off-line con una función de ajuste robusta. Los métodos alternativos propuestos obtienen una amplia mejora tanto en la precisión del seguimiento como en la robustez de éste frente a giros de cabeza, oclusiones, gestos y cambios de iluminación. Los métodos propuestos tienen, además, una baja carga computacional, y son capaces de ejecutarse a velocidades en torno a 100 imágenes por segundo en un computador de sobremesa.
- English
Driver inattention is one of the main causes of traffic accidents. Monitoring a driver to detect inattention is a complex problem that involves physiological and behavioural elements. Different approaches have been made, and among them Computer Vision has the potential of monitoring the person behind the wheel without interfering with her driving. An accurate estimation of the state of the driver can be obtained by analyzing facial expressions, eye movements and actions like blinking and gaze fixation. A computer vision system for driving monitoring uses face location and tracking as the first processing stages. Relaying on adequate techniques in these stages is key for the correct operation of the rest of the system. This thesis presents a method for face tracking, using active models to characterize a face. Three different models are tested on the newly composed RobeSafe Driver Monitoring Video (RS-DMV) dataset, proposed in this thesis. The dataset contains sequences of drivers in real scenarios and two realistic simulators. The first sequences present subjects performing the most common actions in everyday driving. Drivers on the first simulator were presented with situations that highlighted distracted behaviours. Finally, the drivers in the second simulator had been deprived of sleep, and were in fatigue and showed signs of drowsiness. The aim of RS-DMV is to create a test set containing situations that appear frequently an application of driver monitoring. This set can be used to compare the performance of models and methods, and the fulfillment of the requirements and restrictions of a production system: real-time execution, robustness to occlusions and head turns, facial expressions and changing illumination, and users of diverse appearance. The performance of Active Shape Models (ASM) and Constrained Local Models (CLM) have been tested, as they have been considered of interest a priori. More precisely, the Stacked Trimmed ASM (STASM), which integrates a number of extensions to the original ASM, has demonstrated great accuracy in all tests when the face is frontal. Its main drawbacks is that it does not work when the face is rotated, and processing times are far from the required real-time execution. CLM runs in real-time, but it is less accurate than STASM, and needs periodic repositioning. It is not able to handle head turns properly. The third method evaluated is Simultaneous Modeling and Tracking (SMAT), which characterizes shape and texture incrementally from samples previously encountered. The texture around each point of the shape is modeled with a set of clusters of previous samples. The shape is also described with its own set of clusters. This dynamic face model is well suited for the requirements of the application, and has been used as the base for our proposal. Several contributions have been made on this method. Three alternative clustering methods have been developed to model texture, and in the test they obtain lower fitting error, with better reliability. A shape model built from a training set as the one used by ASM has been integrated in SMAT. An M-estimator has been included for robust shape fitting, and a remarkable improvement in the overall robustness of the model to head turns and occlusions has been achieved. We have denominated this new method Robust SMAT (R-SMAT). The improvements of R-SMAT over the original SMAT have been evaluated, and the performance of R-SMAT and STASM has been compared on the sequences in RS-DMV. In conclusion, R-SMAT is able to process more than 100 frames per second, obtains similar accuracy to STASM, and much better than the original SMAT.
