Abstract

Sign languages are natural languages native to the Deaf or Hard-of-Hearing communities. They are not mere transpositions of an oral language, but have their own origin and evolution, as well as their own lexicon, grammar, and idiosyncrasy.

This makes them a legitimate object of study for the fields of Computational Linguistics and Natural Language Processing. This thesis describes research framed within this study, with an emphasis on Spanish Sign Language and its SignWriting. The doctoral research conducted is articulated in six publications, developed within the framework of a competitively funded research project led by the author, “Visualizing SignWriting” (VisSE).

An unifying discussion is included that establishes and justifies the objectives of the research. A critical analysis of the current state of the question is also conducted, considering essential aspects for sign language research, such as methods for their automatic recognition, their grammar and structure, and their written representation. It is precisely this last topic that motivates the main technical objective of the thesis: improving the automatic processing of SignWriting.

In this context, an original corpus of SignWriting samples has been collected, for which a new and complex annotation scheme based on its linguistics has been developed. Based on this corpus, Artificial Intelligence algorithms, specifically deep learning neural networks, have been trained to recognize and interpret SignWriting. The effectiveness of these algorithms has been enhanced by creating an expert system that combines them with logical rules, derived from the theoretical analysis performed. These developments have been made possible thanks to a Python library built for this research, Quevedo, which allows managing data corpora as well as visualizing them, annotating them and processing them. A web application has also been created in order to transfer the research results to society. This application converts SignWriting into textual explanations and includes a three-dimensional model of the hand.

Additionally, this thesis has not only contributed technical advances in the processing of SignWriting but has also generated in-depth knowledge about Spanish Sign Language, materialized in an underlying model and in various publications and computational applications not included in the compendium but referenced in this document.

The software and data generated during the research have been released under open source licenses, thus allowing future researchers to continue advancing in this area without having to start from scratch.