sign-lang@LREC Anthology

Toward an computer-aided sign segmentation

Lefebvre-Albaret, François | Gianni, Frédérick | Dalle, Patrice


Volume:
Proceedings of the LREC2008 3rd Workshop on the Representation and Processing of Sign Languages: Construction and Exploitation of Sign Language Corpora
Venue:
Marrakech, Morocco
Date:
1 June 2008
Pages:
123–128
Publisher:
European Language Resources Association (ELRA)
License:
CC BY-NC
sign-lang ID:
08007

Content Categories

Projects:
SESCA
Languages:
French Sign Language

Abstract

Processing sentences of a sign language corpus requires a first step of temporal segmentation, which is long and tedious. To realize this segmentation more quickly, we propose an innovating method of computer-aided segmentation. This method processes motions of the dominated and dominating hands during the sign realisation. The video treatments are applied in four steps. The first one consists in tracking the hands in a video sequence using particles filtering. Then, in a second step, an operator watches the video sequence and indicates for each sign a time stamp during the sign realisation. Using this information and the trajectories of each hand, our method is able to find the beginning and the end of each sign in a third step. At the end, the operator can eventually apply some rectifications and validate the segmentation.
The presented article explains the different steps, from the calculation of the head and hands 2D positions to the computer-aided determination of the temporal segmentation of the signs. The segmentation exploits a model of French Sign Language and focuses especially on the characteristics of manual sign movements. Our method detects in the video several dynamic properties as the relative hands movement (symmetries, static hands) and the movement primitives (simple or double repetition, uniform or accelerated straight movement). We also detect time spaces between two consecutive signs. Those transitions must be economical, considering the necessary energy to realize these: a movement with a complex realization will contain a sign. Those elements are then combined to each other to determine the most plausible temporal segmentation of the signed sentences. The result can be represented as a succession of signs and transitions segments.
Other observations can be taken into account to obtain the temporal segmentation. We can mention the determination of the elbows 2D positions, the characterization of hand configurations and the head orientation measurement. We describe how those elements could be used to improve the segmentation reliability.
The proposed method is based on motion analysis and does not use any knowledge about the words used in the processed sentences. Using the characteristics shared by the majority of French Sign Language’s signs, it is possible to detect not only standard signs but also other manual iconic signs.
Our segmentation results are finally compared with a traditional manual segmentation produced with an annotation software named AnColin. This comparison exhibits several possible error sources. We focus on the problem of granularity and precision of the segmentation. We also discuss about other qualitative problems such as the detection criteria of the signs start and end. The evaluation protocol of a temporal segmentation is also adressed. Finally we will raise several problems to overcome, to realize a fully automatic segmentation.

Document Download

Paper PDF BibTeX File+ Abstract

BibTeX Export

@inproceedings{lefebvrealbaret:08007:sign-lang:lrec,
  author    = {Lefebvre-Albaret, Fran{\c c}ois and Gianni, Fr{\'e}d{\'e}rick and Dalle, Patrice},
  title     = {Toward an computer-aided sign segmentation},
  pages     = {123--128},
  editor    = {Crasborn, Onno and Efthimiou, Eleni and Hanke, Thomas and Thoutenhoofd, Ernst D. and Zwitserlood, Inge},
  booktitle = {Proceedings of the {LREC2008} 3rd Workshop on the Representation and Processing of Sign Languages: Construction and Exploitation of Sign Language Corpora},
  maintitle = {6th International Conference on Language Resources and Evaluation ({LREC} 2008)},
  publisher = {{European Language Resources Association (ELRA)}},
  address   = {Marrakech, Morocco},
  day       = {1},
  month     = jun,
  year      = {2008},
  language  = {english},
  url       = {https://www.sign-lang.uni-hamburg.de/lrec/pub/08007.pdf}
}
Something missing or wrong?