Reconocimiento de voz para un sistema de interacción humano máquina orientado a personas con limitaciones motrices

Abstract

The development of this degree work is based on the problems faced by people with motor limitations when connecting to the world via the Internet, since it is not possible for them to use the mouse and keyboard in a traditional way. In this order of ideas, this degree work generated contributions in this area by developing an interface that allows interaction between a person with motor limitations and the WhatsApp Desktop application through voice commands.

The degree work used an inductive method, taking a qualitative approach, with exploratory, explanatory and experimental scopes. In which research was conducted on related systems that use voice signals to command human-machine interfaces, identifying techniques and experiments used in the scientific community. Based on the information extracted, the WhatsApp Desktop application was defined as a test instrument to implement the interface, the software components that allow basic navigation in the application were designed and later the algorithms for voice processing and interaction with the computer operating system were implemented using Python language.

The scope and limitations of the interface were quantified through tests and surveys of a group of seven people. Initially, users' familiarization with the Internet, the WhatsApp application and speech recognition systems was characterized. Subsequently, a set of three tests were performed; the first test evaluated the system's performance in recognizing a set of 26 commands where each user independently pronounced each command three times, evaluating a total of 546 commands and obtaining a total success rate of 91.76%.

The second test evaluated the performance of the system to recognize dictations. For this purpose, a dictation of 29 words was defined, which each of the seven users pronounced three times, evaluating a total of 609 words in dictation and obtaining a 93% success rate.

The third test evaluated the performance of the interface directly on the WhatsApp Desktop application. For this, a test protocol was designed that combines commands with dictations and allows the user to open the application, select a contact and send a message. This test was repeated three times by each of the seven users and obtained a 91.16% success rate.

Finally, a survey was conducted to quantify the users' perception after interacting with the developed interface. In general, 40.6% and 23.2% of the test subjects agree and totally agree that the interface is a useful tool and improves the quality of life of people with motor limitations. Similarly, they expressed as future work the need to integrate punctuation marks and accents for better quality of the text sent as a message.