Combining the user friendliness of SignWriting with the precision of linguistic parameters
Antonio F. G. Sevilla
afgs@ucm.es
José María Lahoz-Bengoechea
jmlahoz@ucm.es
Sandra Conde González
saconde@ucm.es
Alberto Díaz Esteban
adiazest@ucm.es
Pablo Folgueira Galán
pablofol@ucm.es
Julia de la Calle Pérez
judelaca@ucm.es
SignWriting
Phonetic
- Universal
- Lossless
+
Iconic
- Intuitive
- Expressive
=
User-friendly
Signotation
Phonological
- Significant
- Parametric
+
Formal
- Well defined
- NLP-ready
=
Precise
TraduSE
TraduSE (Translating SignWriting) is a progressive web app developed as a bachelor’s thesis.
The app converts images of SignWriting (handwritten, or captured) into a
parametric representation (Signotation) which is then used to search for
matching signs in the Spanish Sign Language Signary.
Due to the intuitiveness of SignWriting, only a small learning curve is needed to be able to transcribe basic information. The hands, body and movements are transcribed in a very iconic way, allowing users with imperfect knowledge to query the system. SignWriting can be directly drawn in the app, or scanned via a device camera.
The app converts the SignWriting to Signotation, a parametric system in the style of Stokoe notation or HamNoSys. This coding represents the different sign parameters independently, allowing incomplete or imperfect queries. This helps in finding similar signs, or for example uninflected forms of user searches.
Screen capture of the app, showing the input SignWriting and the list of recovered
videos.
VisSE
Project VisSE (“Automatic SignWriting Recognition”, IEEE Access, 2023) implements an artificial vision pipeline to decompose SignWriting into meaningful units.
TraduSE: The missing link
Both SignWriting and systems based on linguistic parameters contain all the information needed, they just need translation. TraduSE uses a purpose-built module to convert the graphical information encoded in SignWriting logograms into the analytical parameters of Signotation.
SSL Signary
The Spanish Sign Language Signary contains videos transcribed in Signotation and indexed in an ElasticSearch database with a custom tokenizer.