Integration of a Sign Language Interface in a Socially Assistive Robot

• Sohaila Chtioui Bouhou ^[1] ; Alberto Tudela ; José Galeas ; Oscar Pons ; Antonio Bandera ; A. Romero Garcés
  1. [1] Universidad de Málaga

     Universidad de Málaga

     Málaga, España

     
• Localización: Proceedings of the XXIV Workshop of Physical Agents: September 5-6, 2024 / coord. por Miguel Cazorla Quevedo, Francisco Gómez Donoso, Félix Escalona Moncholi, 2024, ISBN 978-84-09-63822-2, págs. 158-168
• Idioma: español
• Enlaces
  • Texto completo
• Resumen

  • This work aims to design and implement a Sign Language interface to improve the communication of deaf people in hospital and telemedical environments. Using Mediapipe to detect landmarks and a Random Forest classifier to interpret gestures, a system capable of recognising and translating signs in real time has been developed. The interface includes an intuitive graphical application developed in Tkinter that allows the visualisation of the recognised signs and facilitates interaction with the user. In addition, detection modes for both letters and numbers have been implemented, and mechanisms for remote communication and system updating have been considered. The interface was integrated into a prototype robot to test its functionality in real environments.