| Abstract |
Sign languages are viso-gestual languages, using space and movement to convey
meaning. To be able to transcribe them, SignWriting uses an iconic system of
symbols meaningfully arranged in the page. This two-dimensional system,
however, is very different to traditional writing systems, so its automatic
processing poses a novel challenge for computational linguistics. In this
article, we present a novel problem for the state of the art in artificial
intelligence: automatic SignWriting recognition. We examine the problem, model
the underlying data domain, and present a first solution in the form of an
expert system that exploits the domain knowledge encoded in the data
modelization. This system uses an adaptable pipeline of neural networks and
deterministic processing, overcoming the challenges posed by the novelty and
originality of the problem. Thanks to our data modelization, it improves the
accuracy compared to a straight-forward deep learning approach by 17%. All of
our data and code are publicly available, and our approach may be useful not
only for SignWriting processing but also for other similar graphical data.
|
| Citation |
@ARTICLE{sevilla_23_automaticsignwritingrecognition,
author={Sevilla, Antonio F. G. and Esteban, Alberto Díaz and Lahoz-Bengoechea, José María},
journal={IEEE Access},
title={Automatic SignWriting Recognition: Combining Machine Learning and Expert Knowledge to Solve a Novel Problem},
year={2023},
volume={11},
number={},
pages={13211-13222},
doi={10.1109/ACCESS.2023.3242203}}
|