Lost in Expression: Diagnosing Systemic Challenges with Non-Manual Generalization in Sign Language Understanding Tasks
Sazonov, Dmitriy 
| Gurbuz, Sevgi | Malaia, Evie A. | Martetschläger, Julia | Schwameder, Hermann | Roehm, Dietmar | Wilbur, Ronnie B.
- Volume:
- Proceedings of the LREC2026 12th Workshop on the Representation and Processing of Sign Languages: Language in Motion
- Venue:
- Palma, Mallorca, Spain
- Date:
- 16 May 2026
- Pages:
- 438–449
- Publisher:
- European Language Resources Association (ELRA)
- Licence:
- CC BY-NC 4.0
- sign-lang ID:
- 26056
- ISBN:
- 978-2-493814-82-1
Abstract
Incorporation of non-manual information is one of the most challenging aspects of Sign Language Understanding (SLU), as these features contribute to the semantic, syntactic, and pragmatic structure of signed communication as a critical feature of compositional meaning at sign, phrase and sentence level. Despite their key linguistic role, non-manuals are often an afterthought in SLU model and dataset design, with many recent models still neglecting to implement non-manual analysis or evaluate how articulators beyond the hands are contributing to the model prediction. In this work, we identify and analyze the challenges relating to recognition of non-manuals and generalization of their linguistic roles encountered by SLU models, offering new explanations for failures to properly model non-manual behavior. We perform a case study on the subtasks of Continuous Sign Language Recognition and Sign Language Translation by applying the Uni-Sign model to Isharah-1000, a Saudi Sign Language dataset. Using controlled partitioning and feature attribution, we further analyze model behavior and failure cases. With this work we hope to set the stage for the creation of diagnostic frameworks for generalization of non-manuals.Document Download
Paper PDF BibTeX File + Abstract
Cite as
Citation in ACL Citation Format
Dmitriy Sazonov, Sevgi Gurbuz, Evie A. Malaia, Julia Martetschläger, Hermann Schwameder, Dietmar Roehm, Ronnie B. Wilbur. 2026. Lost in Expression: Diagnosing Systemic Challenges with Non-Manual Generalization in Sign Language Understanding Tasks. In Proceedings of the LREC2026 12th Workshop on the Representation and Processing of Sign Languages: Language in Motion, pages 438–449, Palma, Mallorca, Spain. European Language Resources Association (ELRA).BibTeX Export
@inproceedings{sazonov:26056:sign-lang:lrec,
author = {Sazonov, Dmitriy and Gurbuz, Sevgi and Malaia, Evie A. and Martetschl{\"a}ger, Julia and Schwameder, Hermann and Roehm, Dietmar and Wilbur, Ronnie B.},
title = {Lost in Expression: Diagnosing Systemic Challenges with Non-Manual Generalization in Sign Language Understanding Tasks},
pages = {438--449},
editor = {Efthimiou, Eleni and Fotinea, Stavroula-Evita and Hanke, Thomas and Hochgesang, Julie A. and Mesch, Johanna and Schulder, Marc},
booktitle = {Proceedings of the {LREC2026} 12th Workshop on the Representation and Processing of Sign Languages: Language in Motion},
maintitle = {15th International Conference on Language Resources and Evaluation ({LREC} 2026)},
publisher = {{European Language Resources Association (ELRA)}},
address = {Palma, Mallorca, Spain},
day = {16},
month = may,
year = {2026},
isbn = {978-2-493814-82-1},
language = {english},
url = {https://www.sign-lang.uni-hamburg.de/lrec/pub/26056.html}
}