KoSign Sign Language Translation Project: Introducing The NIASL2021 Dataset

Huerta-Enochian, Mathew | Lee, Du Hui | Myung, Hye Jin | Byun, Kang Suk | Lee, Jun Woo

Volume:: Proceedings of the 7th International Workshop on Sign Language Translation and Avatar Technology: The Junction of the Visual and the Textual: Challenges and Perspectives
Venue:: Marseille, France
Date:: 24 June 2022
Pages:: 59–66
Publisher:: European Language Resources Association (ELRA)
License:: CC BY-NC 4.0
ACL ID:: 2022.sltat-1.9
ISBN:: 979-10-95546-82-5

Content Categories

Projects:: KoSign
Languages:: Korean Sign Language, Korean
Corpora:: NIASL2021

Abstract

We introduce a new sign language production (SLP) and sign language translation (SLT) dataset, NIASL2021, consisting of 201,026 Korean-KSL data pairs. KSL translations of Korean source texts are represented in three formats: video recordings, keypoint position data, and time-aligned gloss annotations for each hand (using a 7,989 sign vocabulary) and for eight different non-manual signals (NMS). We evaluated our sign language elicitation methodology and found that text-based prompting had a negative effect on translation quality in terms of naturalness and comprehension. We recommend distilling text into a visual medium before translating into sign language or adding a prompt-blind review step to text-based translation methodologies.

Document Download

Paper PDF BibTeX File + Abstract

Cite as

Citation in ACL Citation Format

Mathew Huerta-Enochian, Du Hui Lee, Hye Jin Myung, Kang Suk Byun, Jun Woo Lee. 2022. KoSign Sign Language Translation Project: Introducing The NIASL2021 Dataset. In Proceedings of the 7th International Workshop on Sign Language Translation and Avatar Technology: The Junction of the Visual and the Textual: Challenges and Perspectives, pages 59–66, Marseille, France. European Language Resources Association (ELRA).

BibTeX Export

@inproceedings{huertaenochian:70012:sltat:lrec,
  author    = {Huerta-Enochian, Mathew and Lee, Du Hui and Myung, Hye Jin and Byun, Kang Suk and Lee, Jun Woo},
  title     = {{KoSign} Sign Language Translation Project: Introducing The {NIASL2021} Dataset},
  pages     = {59--66},
  editor    = {Efthimiou, Eleni and Fotinea, Stavroula-Evita and Hanke, Thomas and McDonald, John C. and Shterionov, Dimitar and Wolfe, Rosalee},
  booktitle = {Proceedings of the 7th International Workshop on Sign Language Translation and Avatar Technology: The Junction of the Visual and the Textual: Challenges and Perspectives},
  maintitle = {13th International Conference on Language Resources and Evaluation ({LREC} 2022)},
  publisher = {{European Language Resources Association (ELRA)}},
  address   = {Marseille, France},
  day       = {24},
  month     = jun,
  year      = {2022},
  isbn      = {979-10-95546-82-5},
  language  = {english},
  url       = {http://www.lrec-conf.org/proceedings/lrec2022/workshops/sltat/pdf/2022.sltat-1.9}
}