sign-lang@LREC Anthology

Improving Signer Independent Sign Language Recognition for Low Resource Languages

Holmes, Ruth | Rushe, Ellen | Fowley, Frank | Ventresque, Anthony


Volume:
Proceedings of the 7th International Workshop on Sign Language Translation and Avatar Technology: The Junction of the Visual and the Textual: Challenges and Perspectives
Venue:
Marseille, France
Date:
24 June 2022
Pages:
45–52
Publisher:
European Language Resources Association (ELRA)
License:
CC BY-NC 4.0
ACL ID:
2022.sltat-1.7
ISBN:
979-10-95546-82-5

Content Categories

Projects:
SignOn
Languages:
Irish Sign Language
Corpora:
ISL-HS

Abstract

The reliance of deep learning algorithms on large scale datasets represents a significant challenge when learning from low resource sign language datasets. This challenge is compounded when we consider that, for a model to be effective in the real world, it must not only learn the variations of a given sign, but also learn to be invariant to the person signing. In this paper, we first illustrate the performance gap between signer-independent and signer-dependent models on Irish Sign Language manual hand shape data. We then evaluate the effect of transfer learning, with different levels of fine-tuning, on the generalisation of signer independent models, and show the effects of different input representations, namely variations in image data and pose estimation. We go on to investigate the sensitivity of current pose estimation models in order to establish their limitations and areas in need of improvement. The results show that accurate pose estimation outperforms raw RGB image data, even when relying on pre-trained image models. Following on from this, we investigate image texture as a potential contributing factor to the gap in performance between signer-dependent and signer-independent models using counterfactual testing images and discuss potential ramifications for low-resource sign languages. Keywords: Sign language recognition, Transfer learning, Irish Sign Language, Low-resource languages

Document Download

Paper PDF BibTeX File+ Abstract

BibTeX Export

@inproceedings{holmes:70025:sltat:lrec,
  author    = {Holmes, Ruth and Rushe, Ellen and Fowley, Frank and Ventresque, Anthony},
  title     = {Improving Signer Independent Sign Language Recognition for Low Resource Languages},
  pages     = {45--52},
  editor    = {Efthimiou, Eleni and Fotinea, Stavroula-Evita and Hanke, Thomas and McDonald, John C. and Shterionov, Dimitar and Wolfe, Rosalee},
  booktitle = {Proceedings of the 7th International Workshop on Sign Language Translation and Avatar Technology: The Junction of the Visual and the Textual: Challenges and Perspectives},
  maintitle = {13th International Conference on Language Resources and Evaluation ({LREC} 2022)},
  publisher = {{European Language Resources Association (ELRA)}},
  address   = {Marseille, France},
  day       = {24},
  month     = jun,
  year      = {2022},
  isbn      = {979-10-95546-82-5},
  language  = {english},
  url       = {http://www.lrec-conf.org/proceedings/lrec2022/workshops/sltat/pdf/2022.sltat-1.7}
}
Something missing or wrong?