In a 5-year project, we compare expressions in the spatial domain between two sign languages (German Sign Language and Turkish Sign Language), the co-speech gestures accompanying two spoken languages (German and Turkish), and the pantomime-like structures used by hearing people (German and Turkish) asked to convey information without speaking. The aim is to discover the similarities and differences in the use of space in expressing referent location and motion between the sign languages, and between the signing, co-speech gesture and no-speech pantomime modes. To this end, we are building a large video corpus of task-related discourse data (about 90 minutes per 15 participants per condition). The data will be described using the IMDI metadata standard and linguistically annotated using ELAN. Parts of the data will be made accessible for research and educational purposes on the Browsable Corpus based at the MPI for Psycholinguistics. In this presentation, we report the annotation conventions we have been developing based on collected data. There are two levels of annotation: (i) a descriptive level where we gloss signs and gestures according to the movements/positions of the hands, head, face, and body; and (ii) an analytic/coding level where each sign or gesture is analyzed with respect to the function of establishing and/or maintaining reference in discourse (e.g. through the use of pronouns, classifier predicates, modified verbs, and role shift in signing, and similar forms in gestures). Our conventions combine aspects from other annotation and coding systems developed for sign and gesture (e.g. the ECHO, Corpus NGT, and Auslan Corpus conventions; HamNoSys; gesture coding conventions as developed by Kita, Van Gijn and Van der Hulst), but go beyond them in placing special emphasis on coding both systems with the same parameters. On the descriptive level, we developed a 3-dimensional scheme to identify for hand orientation, location, and direction of signs and gestures, allowing comparison across languages. On the analytic/coding level, we devised ways of categorizing how the various spatial expressions in sign and gesture map onto different coreference devices in discourse. Parts of the sign and gesture data have now been annotated. We will present some generalizations and conclusions drawn from using our annotation conventions regarding cross-linguistic and sign-gesture comparison. Furthermore, based on our annotation experiences, we will discuss the advantages as well as the shortcomings of our annotation scheme and suggest specific improvements, which the linguistic community needs to consider in terms of ways they can be implemented in the technology of annotation software (such as ELAN).
@inproceedings{zwitserlood:08027:sign-lang:lrec,
author = {Zwitserlood, Inge and {\"O}zy{\"u}rek, Asli and Perniss, Pamela},
title = {Annotation of Sign and Gesture Cross-linguistically},
pages = {185--190},
editor = {Crasborn, Onno and Efthimiou, Eleni and Hanke, Thomas and Thoutenhoofd, Ernst D. and Zwitserlood, Inge},
booktitle = {Proceedings of the {LREC2008} 3rd Workshop on the Representation and Processing of Sign Languages: Construction and Exploitation of Sign Language Corpora},
maintitle = {6th International Conference on Language Resources and Evaluation ({LREC} 2008)},
publisher = {{European Language Resources Association (ELRA)}},
address = {Marrakech, Morocco},
day = {1},
month = jun,
year = {2008},
language = {english},
url = {https://www.sign-lang.uni-hamburg.de/lrec/pub/08027.pdf}
}