Deep neural networks for vehicle driving characterization by means of smartphone sensors

• Autores: Sara Hernández
• Directores de la Tesis: Luis Alfonso Hernández Gómez (dir. tes.), Rubén Fernández Pozo (codir. tes.)
• Lectura: En la Universidad Politécnica de Madrid ( España ) en 2020
• Idioma: español
• Materias:
  • Ciencias tecnológicas
• Enlaces
  • Tesis en acceso abierto en: Archivo Digital UPM
• Resumen

  • This Thesis analyzes the driving characterization by means of the accelerometers present in drivers' smartphones, applying Deep Learning techniques. This research studies both the accelerometer possibilities to address the characterization, and the ability of Deep Learning tools to learn these attributes.

    Most research have addressed the driving characterization employing a large number of sensors, generating in many cases the need for both the installation of extra equipment in order to capture these signals, and the access to the vehicle information. Although accelerometer signals are widely used, for example for activity recognition tasks or intelligent assistance systems, these are often complemented by others to different nature. In particular, in the driving task, most works use information from the Controller Area Network (CAN) bus of the vehicle, such as signals from the gas and brake pedals, information from the steering wheel, engine or fuel, among others. It is also common the use of location signals, such as the Global Positioning System (GPS), or motion sensors, as the gyroscope and the magnetometer.

    Neural Networks have become the state-of-the-art for many Machine Learning problems. These networks consist of neurons or neuron networks, where each of them acts as a computational unit. How the neurons are connected is related to the learning algorithm used for the training. There are mainly three types: single layer feedforward networks, multilayer feedforward networks and recurrent networks. For our research in the Thesis we have focused on multilayer and recurrent networks. More specifically in Convolutional Multilayer Perceptron, or Convolutional Neural Networks (CNN) as these are commonly known, and in Long Short-Term Memory Networks (LSTM) and Gated Recurrent Units (GRU), within the Recurrent Neural Networks (RNN). Each one of these types of Neural Network has different properties to recognize patterns. CNNs are especially designed to recognize two-dimensional shapes with a high degree of invariance to different forms of distortion (such as translation or scaling), using three common steps: feature extraction, feature mapping, and subsampling. RNNs are non-linear systems, characterized by presenting at least one feedback loop. These are very effective at extracting patterns when the data attributes are highly dependent on each other, since these networks share parameters over time. In (Bengio & LeCun, 2007), it is argued that deep architectures have great potential to generalize in a nonlocal way, which is very important in the design of Machine Learning algorithms applicable to complex tasks. We consider that driving characterization is a highly complex task, therefore we hope these deep networks will be a good tool for pattern extraction and driver authentication.

    In this work we have faced two dual problems in order to address the driving characterization: the driver behavior description and the driver authentication. On the basis of the hypothesis that each driver has a unique behavior, we believe that the extraction of their characteristic patterns allows both to analyze the type of maneuvers or events performed, and to recognize the driver against others. Generally this authentication or recognition includes both identification and verification of the driver.

    We have collected two different databases according to the task under analysis. The first one, for the maneuver characterization, is composed of more than 60000 real driving journeys, of more than 300 different drivers. For the second one, employed for driver authentication, there are more than 23000 journeys out of a total of 83 drivers.

    The results obtained during the Thesis demonstrate that the driving characterization is possible using only the accelerometer signals from drivers' smartphones. Few works have addressed this characterization optimizing the number of signals employed, as well as using sensors that promote both energy efficiency and costs. Even works that have carried out the characterization using exclusively the accelerometers include additional conditions, such as the need to place the smartphone in a fixed position in order to identify the orientation directions during the driving. We offer an alternative system to traditional rotation matrices, which allows mapping from the smartphone coordinate system to the vehicle coordinate system. By means of the procedures presented in the Thesis, different maneuver classification techniques have been proposed. Using methods that allow obtaining the longitudinal and transversal accelerations from the original raw accelerometers, we have achieved accuracies of 90.07% in the assignment of these signals. For driver recognition, network architectures commonly used in other tasks such as image classification or speech recognition have also been analyzed. Many pre-trained models of the literature as well as many data augmentation techniques have been developed for images, however few works have applied these techniques on time series. Through our tests we contribute both to the study of transformation techniques for 1-D time signals to 2-D images, in order to use powerful pre-trained state-of-the-art models, as well as to the study of different techniques to increase data in temporal signals. Our experiments have achieved results in the field of identification of almost 72% of accuracy for the baseline database, and almost 76% for another pubic database of the literature. Whereas verification rates have reached almost 80% of precision and 74% of F1 score.

    This work opens possible future lines to continue with the driving characterization task, in order to improve driver assistance systems and to contribute to the autonomous driving, improving safety, mobility and environmental effects.