Bridging Text-to-Sign Translation via Codebook-Oriented Pretraining

Phuangchoke, Ninlawat | Polprasert, Chantri

Volume:: Proceedings of the 15th International Conference on Language Resources and Evaluation (LREC 2026)
Venue:: Palma, Mallorca, Spain
Date:: 11 to 16 May 2026
Pages:: 9504–9513
Publisher:: European Language Resources Association (ELRA)
Licence:: CC BY-NC 4.0
DOI:: 10.63317/2s9976y7ibcu
ISBN:: 978-2-493814-49-4

Abstract

Sign Language Production (SLP), the automatic translation from spoken to sign languages, faces several challenges due to the intricate mapping between linguistic semantics and the spatial–temporal motion domain. Existing SLP methods employing a transformer model with a Vector Quantization (VQ) method exhibit poor translation performance due to weak semantic alignment between the codebook and the text representation. In this work, we propose a novel text-to-sign translation based on model pretraining, which enhances semantic alignment by inheriting codebook-oriented prior knowledge from masked self-supervised models. Our approach involves two stages: (i) transforming sign language into discrete values by employing VQ with masked self-attention learning to create pre-tasks that bridge the semantic gap between text and codebook representations, (ii) constructing an end-to-end architecture with an encoder-decoder-like structure that inherits the parameters of the model from the first stage. The integration of these designs forms a robust sign language representation and significantly improves the translation model, which surpass prior baselines.

Document Download

Paper PDF BibTeX File + Abstract

Cite as

Citation in ACL Citation Format

Ninlawat Phuangchoke, Chantri Polprasert. 2026. Bridging Text-to-Sign Translation via Codebook-Oriented Pretraining. In Proceedings of the 15th International Conference on Language Resources and Evaluation (LREC 2026), pages 9504–9513, Palma, Mallorca, Spain. European Language Resources Association (ELRA).

BibTeX Export

@inproceedings{phuangchoke-polprasert-2026-codebook:lrec,
  author    = {Phuangchoke, Ninlawat and Polprasert, Chantri},
  title     = {Bridging Text-to-Sign Translation via Codebook-Oriented Pretraining},
  pages     = {9504--9513},
  editor    = {Piperidis, Stelios and Bel, N{\'u}ria and van den Heuvel, Henk and Ide, Nancy and Krek, Simon and Toral, Antonio},
  booktitle = {15th International Conference on Language Resources and Evaluation ({LREC} 2026)},
  publisher = {{European Language Resources Association (ELRA)}},
  address   = {Palma, Mallorca, Spain},
  day       = {11--16},
  month     = may,
  year      = {2026},
  isbn      = {978-2-493814-49-4},
  language  = {english},
  url       = {https://lrec.elra.info/lrec2026-main-746},
  doi       = {10.63317/2s9976y7ibcu}
}