We have developed a Japanese-to-Japanese Sign Language (JSL) translation system to expand sign language services for the Deaf. Although recording the motion data of isolated JSL by motion capture (MoCap) and avatar animation driven by MoCap data is effective for capturing the more natural movements of sign language, the disadvantage is that they lack the flexibility to reproduce the contextual modification of signs. We therefore propose a sign language motion data editing method based on the Hamburg Notation System for Sign Languages (HamNoSys) for use in a hybrid system that combines a MoCap data-driven technique and a phonological generation technique. The proposed method enables the editing of handshape, hand orientation, and location of the motion data based on HamNoSys components to generate contextual modifications for motion-captured citation form signs in translated gloss sequences. Experimental results demonstrate that our method achieves the flexibility to generate contextual modifications and new movements while preserving natural human-like movements without the need for additional MoCap processes.
Tsubasa Uchida, Taro Miyazaki, Hiroyuki Kaneko. 2024. HamNoSys-based Motion Editing Method for Sign Language. In Proceedings of the LREC-COLING 2024 11th Workshop on the Representation and Processing of Sign Languages: Evaluation of Sign Language Resources, pages 90–99, Torino, Italy. ELRA Language Resources Association (ELRA) and the International Committee on Computational Linguistics (ICCL).
BibTeX Export
@inproceedings{uchida:24011:sign-lang:lrec,
author = {Uchida, Tsubasa and Miyazaki, Taro and Kaneko, Hiroyuki},
title = {{HamNoSys-based} Motion Editing Method for Sign Language},
pages = {90--99},
editor = {Efthimiou, Eleni and Fotinea, Stavroula-Evita and Hanke, Thomas and Hochgesang, Julie A. and Mesch, Johanna and Schulder, Marc},
booktitle = {Proceedings of the {LREC-COLING} 2024 11th Workshop on the Representation and Processing of Sign Languages: Evaluation of Sign Language Resources},
maintitle = {2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation ({LREC-COLING} 2024)},
publisher = {{ELRA Language Resources Association (ELRA) and the International Committee on Computational Linguistics (ICCL)}},
address = {Torino, Italy},
day = {25},
month = may,
year = {2024},
isbn = {978-2-493814-30-2},
language = {english},
url = {https://www.sign-lang.uni-hamburg.de/lrec/pub/24011.pdf}
}
| Miyazaki, Taro