Deep JSLC: A Multimodal Corpus Collection for Data-driven Generation of Japanese Sign Language Expressions

Brock, Heike | Nakadai, Kazuhiro

Volume:: Proceedings of the 11th International Conference on Language Resources and Evaluation (LREC 2018)
Venue:: Miyazaki, Japan
Date:: 7 to 12 May 2018
Pages:: 4247–4252
Publisher:: European Language Resources Association (ELRA)
Licence:: CC BY-NC 4.0
ACL ID:: L18-1670
ISBN:: 979-10-95546-00-9

Content Categories

Languages:: Japanese Sign Language
Corpora:: Deep JSLC

Abstract

The three-dimensional visualization of spoken or written information in Sign Language (SL) is considered a potential tool for better inclusion of deaf or hard of hearing individuals with low literacy skills. However, conventional technologies for such CG-supported data display are not able to depict all relevant features of a natural signing sequence such as facial expression, spatial references or inter-sign movement, leading to poor acceptance amongst speakers of sign language. The deployment of fully data-driven, deep sequence generation models that proved themselves powerful in speech and text applications might overcome this lack of naturalness. Therefore, we collected a corpus of continuous sentence utterances in Japanese Sign Language (JSL) applicable to the learning of deep neural network models. The presented corpus contains multimodal content information of high resolution motion capture data, video data and both visual and gloss-like mark up annotations obtained with the support of fluent JSL signers. Furthermore, all annotations were encoded under three different encoding schemes with respect to directions, intonation and non-manual information. Currently, the corpus is employed to learn first sequence-to-sequence networks where it shows the ability to train relevant language features.

Document Download

Paper PDF BibTeX File + Abstract

Cite as

Citation in ACL Citation Format

Heike Brock, Kazuhiro Nakadai. 2018. Deep JSLC: A Multimodal Corpus Collection for Data-driven Generation of Japanese Sign Language Expressions. In Proceedings of the 11th International Conference on Language Resources and Evaluation (LREC 2018), pages 4247–4252, Miyazaki, Japan. European Language Resources Association (ELRA).

BibTeX Export

@inproceedings{brock-nakadai-2018-deep:lrec,
  author    = {Brock, Heike and Nakadai, Kazuhiro},
  title     = {Deep {JSLC}: A Multimodal Corpus Collection for Data-driven Generation of {J}apanese {S}ign {L}anguage Expressions},
  pages     = {4247--4252},
  editor    = {Calzolari, Nicoletta and Choukri, Khalid and Cieri, Christopher and Declerck, Thierry and Goggi, Sara and Hasida, Koiti and Isahara, Hitoshi and Maegaard, Bente and Mariani, Joseph and Mazo,  H{\'e}l{\`e}ne and Moreno, Asuncion and Odijk, Jan and Piperidis, Stelios and Tokunaga, Takenobu},
  booktitle = {11th International Conference on Language Resources and Evaluation ({LREC} 2018)},
  publisher = {{European Language Resources Association (ELRA)}},
  address   = {Miyazaki, Japan},
  day       = {7--12},
  month     = may,
  year      = {2018},
  isbn      = {979-10-95546-00-9},
  language  = {english},
  url       = {https://aclanthology.org/L18-1670}
}