Video Analysis for Continuous Sign Language Recognition

Piater, Justus | Hoyoux, Thomas | Du, Wei

Volume:: Proceedings of the LREC2010 4th Workshop on the Representation and Processing of Sign Languages: Corpora and Sign Language Technologies
Venue:: Valletta, Malta
Date:: 22 and 23 May 2010
Pages:: 192–195
Publisher:: European Language Resources Association (ELRA)
License:: CC BY-NC
sign-lang ID:: 10052

Content Categories

Projects:: SignSpeak
Languages:: German Sign Language, Sign Language of the Netherlands
Corpora:: Corpus NGT, RWTH-BOSTON-104

Abstract

The recognition of continuous, natural signing is very challenging due to the multimodal nature of the visual cues (fingers, lips, facial expressions, body pose, etc.), as well as technical limitations such as spatial and temporal resolution and unreliable depth cues. On the other hand, signing gestures are designed to be robustly discernible. We therefore argue in favor of an integrative approach to sign language recognition that aims to extract sufficient aggregate information for robust sign language recognition, even if many of the individual cues are unreliable. Our strategy to implement such an integrated system currently rests on two modules, for which we will show initial results. The first module uses active appearance models for detailed face tracking, allowing the quantification of facial expressions such as mouth and eye aperture and eyebrow raise. The second module is dedicated to hand tracking using color and appearance. A third module will be concerned with tracking upper-body articulated pose, linking the face to the hands for increased overall robustness.

Document Download

Paper PDF BibTeX File + Abstract

Cite as

Citation in ACL Citation Format

Justus Piater, Thomas Hoyoux, Wei Du. 2010. Video Analysis for Continuous Sign Language Recognition. In Proceedings of the LREC2010 4th Workshop on the Representation and Processing of Sign Languages: Corpora and Sign Language Technologies, pages 192–195, Valletta, Malta. European Language Resources Association (ELRA).

BibTeX Export

@inproceedings{piater:10052:sign-lang:lrec,
  author    = {Piater, Justus and Hoyoux, Thomas and Du, Wei},
  title     = {Video Analysis for Continuous Sign Language Recognition},
  pages     = {192--195},
  editor    = {Dreuw, Philippe and Efthimiou, Eleni and Hanke, Thomas and Johnston, Trevor and Mart{\'i}nez Ruiz, Gregorio and Schembri, Adam},
  booktitle = {Proceedings of the {LREC2010} 4th Workshop on the Representation and Processing of Sign Languages: Corpora and Sign Language Technologies},
  maintitle = {7th International Conference on Language Resources and Evaluation ({LREC} 2010)},
  publisher = {{European Language Resources Association (ELRA)}},
  address   = {Valletta, Malta},
  day       = {22--23},
  month     = may,
  year      = {2010},
  language  = {english},
  url       = {https://www.sign-lang.uni-hamburg.de/lrec/pub/10052.pdf}
}