Generating Sign Language Poses from HamNoSys and Natural Language Descriptions

Máximo, Santiago | Chiruzzo, Luis

Volume:: Proceedings of the 15th International Conference on Language Resources and Evaluation (LREC 2026)
Venue:: Palma, Mallorca, Spain
Date:: 11 to 16 May 2026
Pages:: 9358–9367
Publisher:: European Language Resources Association (ELRA)
Licence:: CC BY-NC 4.0
DOI:: 10.63317/466di7tv7dpd
ISBN:: 978-2-493814-49-4

Abstract

One of the steps involved in the process of sign language generation is generating a sequence of poses that represent the signs. This paper presents a method for using textual information to improve the translation of signs in HamNoSys format into sequences of poses. The method comprises a description generator that translates HamNoSys into a textual description, an LLM fine-tuned to the task of predicting a pose sequence from a HamNoSys description, and a VQ-VAE network that encodes and decodes pose sequences as a list of discrete symbols. Our experiments found that even using simple dictionary descriptions of HamNoSys, it is possible to improve the predictions of pose sequences by leveraging the information from a pretrained LLM.

Document Download

Paper PDF BibTeX File + Abstract

Cite as

Citation in ACL Citation Format

Santiago Máximo, Luis Chiruzzo. 2026. Generating Sign Language Poses from HamNoSys and Natural Language Descriptions. In Proceedings of the 15th International Conference on Language Resources and Evaluation (LREC 2026), pages 9358–9367, Palma, Mallorca, Spain. European Language Resources Association (ELRA).

BibTeX Export

@inproceedings{maximo-chiruzzo-2026-poses:lrec,
  author    = {M{\'a}ximo, Santiago and Chiruzzo, Luis},
  title     = {Generating Sign Language Poses from {HamNoSys} and Natural Language Descriptions},
  pages     = {9358--9367},
  editor    = {Piperidis, Stelios and Bel, N{\'u}ria and van den Heuvel, Henk and Ide, Nancy and Krek, Simon and Toral, Antonio},
  booktitle = {15th International Conference on Language Resources and Evaluation ({LREC} 2026)},
  publisher = {{European Language Resources Association (ELRA)}},
  address   = {Palma, Mallorca, Spain},
  day       = {11--16},
  month     = may,
  year      = {2026},
  isbn      = {978-2-493814-49-4},
  language  = {english},
  url       = {https://lrec.elra.info/lrec2026-main-735},
  doi       = {10.63317/466di7tv7dpd}
}