The goal of sign language synthesis is to create an avatar which uses sign languge as main communication form. In order to emulate human behaviour during signing the avatar has to express manual components (hand position, hand shape) and non-manual components (face expression, lip articulation) of the performed signs. The task of sign language synthesis is implemented in several steps. Since the sign language has different grammar than the spoken language, the source sentence has to be translated into corresponding sequence of isolated signs. Those signs are synthesized in sequence and create output sentence in sign language. Non-manual components are synthesized by already developed Czech talking head which is able to articulate words and sentences in Czech language. Face expressions can be manually set. The synthesis process of manual movements is based on HamNoSys 3.0 notation. This notation is used for deterministic and suitable processing of the sign speech. The methodology of the notation allows precise and also extensible expression of the sign description. Firstly, our synthesis system automatically carries out the syntactic analysis of symbolic string (in HamNoSys notation) and generates a tree structure. The tree structure is suitable for conversion of the symbols to tra jectories with application parse rules. The parsing rules were manually formed to cover all HamNoSys notation variants. There are 39 rule actions forming complete animation tra jectories. For this purpose 138 HamNoSys symbols are currently adopted. The processing of the tree is carried out by several tree walks whilst the size of the tree is reduced. The final animation tra jectories in the root node are transformed by an inverse kinematics technique to control the joints of avatar animation model. The analysis of HamNoSys symbols allows us to animate hands and the upper half-body. Thus a single sign is encoded by corresponding sequence of HamNoSys symbols. We have developed an interactive tool which purpose is to extend our database of signs. The main application window contains list of symbols which can be clicked and added into the sequence. This sequence can be immediately converted into the movement of the avatar which is shown in the second window. This allows fast production of symbol sequences for new signs and easy modification of existing signs since the changes are directly visible. In addition it allows people who have no high experince with HamNoSys to learn it faster. At present our database contains about 300 signs which are encoded as sequeces of HamNoSys symbols. This first database is targeted to the information system for train connections. Further expansion of the database will add new areas where the avatar can be used.
@inproceedings{kanis:08012:sign-lang:lrec,
author = {Kanis, Jakub and Kr{\v n}oul, Zden{\v e}k},
title = {Interactive {HamNoSys} Notation Editor for Signed Speech Annotation},
pages = {88--93},
editor = {Crasborn, Onno and Efthimiou, Eleni and Hanke, Thomas and Thoutenhoofd, Ernst D. and Zwitserlood, Inge},
booktitle = {Proceedings of the {LREC2008} 3rd Workshop on the Representation and Processing of Sign Languages: Construction and Exploitation of Sign Language Corpora},
maintitle = {6th International Conference on Language Resources and Evaluation ({LREC} 2008)},
publisher = {{European Language Resources Association (ELRA)}},
address = {Marrakech, Morocco},
day = {1},
month = jun,
year = {2008},
language = {english},
url = {https://www.sign-lang.uni-hamburg.de/lrec/pub/08012.pdf}
}