In a lot of recent research, attention has been drawn to recognizing sequences of lexical signs in continuous Sign Language corpora, often artificial. However, as SLs are structured through the use of space and iconicity, focusing on lexicon only prevents the field of Continuous Sign Language Recognition (CSLR) from extending to Sign Language Understanding and Translation. In this article, we propose a new formulation of the CSLR problem and discuss the possibility of recognizing higher-level linguistic structures in SL videos, like classifier constructions. These structures show much more variability than lexical signs, and are fundamentally different than them in the sense that form and meaning can not be disentangled. Building on the recently published French Sign Language corpus Dicta-Sign-LSF-v2, we discuss the performance and relevance of a simple recurrent neural network trained to recognize illustrative structures.
Keywords
Language and the Brain – Experimental methods using or producing sign language resources
Machine / Deep Learning – Machine Learning methods both in the visual domain and on linguistic annotation of sign language data
Experiences from linguistic research using corpora
Language and the Brain – Sign language processing applications
Sign language corpus mining
Proposals for standards for linguistic annotation or for metadata descriptions
In the Service of the Language Community – What is the value of sign language resources for the sign language community?
Use of (parallel) corpora and lexicons in translation studies and machine translation
@inproceedings{belissen:20028:sign-lang:lrec,
author = {Belissen, Valentin and Gouiff{\`e}s, Mich{\`e}le and Braffort, Annelies},
title = {Improving and Extending Continuous Sign Language Recognition: Taking Iconicity and Spatial Language into Account},
pages = {7--12},
editor = {Efthimiou, Eleni and Fotinea, Stavroula-Evita and Hanke, Thomas and Hochgesang, Julie A. and Kristoffersen, Jette and Mesch, Johanna},
booktitle = {Proceedings of the {LREC2020} 9th Workshop on the Representation and Processing of Sign Languages: Sign Language Resources in the Service of the Language Community, Technological Challenges and Application Perspectives},
maintitle = {12th International Conference on Language Resources and Evaluation ({LREC} 2020)},
publisher = {{European Language Resources Association (ELRA)}},
address = {Marseille, France},
day = {16},
month = may,
year = {2020},
isbn = {979-10-95546-54-2},
language = {english},
url = {https://www.sign-lang.uni-hamburg.de/lrec/pub/20028.pdf}
}