sign-lang@LREC Anthology

Computer-based recognition of facial expressions in ASL: from face tracking to linguistic interpretation

Michael, Nicholas | Neidle, Carol | Metaxas, Dimitris

Proceedings of the LREC2010 4th Workshop on the Representation and Processing of Sign Languages: Corpora and Sign Language Technologies
Valletta, Malta
22 and 23 May 2010
European Language Resources Association (ELRA)
sign-lang ID:

Content Categories

American Sign Language


Most research in the field of sign language recognition has focused on the manual component of signing, despite the fact that there is critical grammatical information expressed through facial expressions and head gestures. We, therefore, propose a novel framework for robust tracking and analysis of nonmanual behaviors, with an application to sign language recognition.
Our method uses computer vision techniques to track facial expressions and head movements from video, in order to recognize such linguistically significant expressions. The methods described here have relied crucially on the use of a linguistically annotated video corpus that is being developed, as the annotated video examples have served for training and testing our machine learning models. We apply our framework to continuous recognition of three classes of grammatical expressions, namely wh-questions, negative expressions, and topics.
Our method is signer-independent, utilizing spatial pyramids and Hidden Markov Models (HMMs) to model the temporal variations of facial shape and appearance.

Document Download

Paper PDF BibTeX File+ Abstract

BibTeX Export

  author    = {Michael, Nicholas and Neidle, Carol and Metaxas, Dimitris},
  title     = {Computer-based recognition of facial expressions in {ASL}: from face tracking to linguistic interpretation},
  pages     = {164--167},
  editor    = {Dreuw, Philippe and Efthimiou, Eleni and Hanke, Thomas and Johnston, Trevor and Mart{\'i}nez Ruiz, Gregorio and Schembri, Adam},
  booktitle = {Proceedings of the {LREC2010} 4th Workshop on the Representation and Processing of Sign Languages: Corpora and Sign Language Technologies},
  maintitle = {7th International Conference on Language Resources and Evaluation ({LREC} 2010)},
  publisher = {{European Language Resources Association (ELRA)}},
  address   = {Valletta, Malta},
  day       = {22--23},
  month     = may,
  year      = {2010},
  language  = {english},
  url       = {}
Something missing or wrong?