sign-lang@LREC Anthology


Automatic Gloss-level Data Augmentation for Sign Language Translation

Jang, Jin Yea | Park, Han-Mu | Shin, Saim | Shin, Suna | Yoon, Byungcheon | Gweon, Gahgene


Volume:
Proceedings of the 13th International Conference on Language Resources and Evaluation (LREC 2022)
Venue:
Marseille, France
Date:
20 to 25 June 2022
Pages:
6808–6813
Publisher:
European Language Resources Association (ELRA)
License:
CC BY-NC 4.0
ACL ID:
2022.lrec-1.734

Abstract

Securing sufficient data to enable automatic sign language translation modeling is challenging. The data insufficiency issue exists in both video and text modalities; however, fewer studies have been performed on text data augmentation compared to video data. In this study, we present three methods of augmenting sign language text modality data, comprising 3,052 Gloss-level Korean Sign Language (GKSL) and Word-level Korean Language (WKL) sentence pairs. Using each of the three methods, the following number of sentence pairs were created: blank replacement 10,654, sentence paraphrasing 1,494, and synonym replacement 899. Translation experiment results using the augmented data showed that when translating from GKSL to WKL and from WKL to GKSL, Bi-Lingual Evaluation Understudy (BLEU) scores improved by 0.204 and 0.170 respectively, compared to when only the original data was used. The three contributions of this study are as follows. First, we demonstrated that three different augmentation techniques used in existing Natural Language Processing (NLP) can be applied to sign language. Second, we propose an automatic data augmentation method which generates quality data by utilizing the Korean sign language gloss dictionary. Lastly, we publish the Gloss-level Korean Sign Language 13k dataset (GKSL13k), which has verified data quality through expert reviews.

Document Download

Paper PDF BibTeX File BibTeX File w/Abstract

BibTeX Export

@inproceedings{jang-etal-2022-augmentation:lrec,
  author    = {Jang, Jin Yea and Park, Han-Mu and Shin, Saim and Shin, Suna and Yoon, Byungcheon and Gweon, Gahgene},
  title     = {Automatic Gloss-level Data Augmentation for Sign Language Translation},
  pages     = {6808--6813},
  editor    = {Calzolari, Nicoletta and Fr{\'e}d{\'e}ric B{\'e}chet and Blache, Philippe and Choukri, Khalid and Cieri, Christopher and Declerck, Thierry and Goggi, Sara and Isahara, Hitoshi and Maegaard, Bente and Mariani, Joseph and Mazo, H{\'e}l{\`e}ne and Odijk, Jan and Piperidis, Stelios},
  booktitle = {13th International Conference on Language Resources and Evaluation ({LREC} 2022)},
  publisher = {{European Language Resources Association (ELRA)}},
  address   = {Marseille, France},
  day       = {20--25},
  month     = jun,
  year      = {2022},
  language  = {english},
  url       = {http://www.lrec-conf.org/proceedings/lrec2022/pdf/2022.lrec-1.734}
}
Something missing or wrong?