2. Methodology

2.1 Suppliers / consumers
2.2 The product

2.2.1 Management information
2.2.2 Reference information
2.2.3 Costs
2.2.4 Content
2.2.5 Language information
2.2.6 Sign language presentation
2.2.7 Visual information
2.2.8 Captions and subtitles
2.2.9 Sound information
2.2.10 Editing and special effects
2.2.11 Navigation
2.2.12 Overall usability
2.2.13 Usability for visually impaired persons

2.3 The production and distribution process

2.4 Selection of aspects to be researched in WP 5

Workpackage 3 started in month 1 of the project (January 1998) and lasted until month 7 (July 1998). Each of the three research partners in the project focused on a specific target group. FODOK was to collect data on signing books for children, Hamburg University on signing books for students, and City University on signing books for deaf adults.

At the same time, each partner was responsible for collecting information about signing books for all age groups, in his/her own country or language area (FODOK: NL and BE-NL, Hamburg University: DE and AT, City University: GB and IE).

2.1 Suppliers / consumers

Each partner set up a database of persons and organisations for and of the deaf, that were to be contacted.

Each partner sent information about the Signing Books project and an initial questionnaire (see the appendix) to the persons and organisations in their respective database, to inform them of the Signing Books project, to ask for information about signing books available in that country and for names and addresses of the producers.(2)

Names and addresses were compiled from existing networks, by literature search (relevant journals, and proceedings of relevant conferences), and by searching the Internet.

The first questionnaire was also published (in three languages) on the Signing Books Internet site.

Communication with several of the persons/organisations has proved to be difficult, partly because of language barriers (several of the organisations in southern European countries have indicated that they are unable to read and write in English). A number of videos were ordered in early 1998 but have not yet been received. Much of the information that has been received, e.g. catalogues from publishers, has had to be translated before it could be used by us. Only a few of the larger publishers have catalogues of their video productions; others seem to have little or no printed information available (in any language). Organisations of and for the deaf are often unaware of signing books that are available in their own country. Only Germany (Hamburg University) has a central library which, as well as substantial holdings of signing books, has information all the signing books that are available in that country, as well as on signing books from other countries.

Because of these problems the process of data collecting is still ongoing and the results in this chapter are only preliminary. The video database will be updated during the project, as new information is received, and an updated issue of this deliverable will be published at the end of the project.

2.2 The product

On the basis of the information that was received in response to the first questionnaire, sample videos were purchased by each of the partners. Using these newly purchased signing books as well as those that were already available at the beginning of the project, a general 'framework for description' was developed. The framework for description is a standardised way to describe those aspects of signing books that are relevant for the objectives of this workpackage (also see paragraph 1.3): the general description of what is currently available, the selection of good and bad examples, and the selection of 'unresolved issues' to be researched in this project.

The framework for description was set up as a database, and all three partners entered information about the signing books each had details of (also see the appendix for the structure and codes used for the database, and for a printout of the database). For printing purposes, the database was later split into two parts: a database with the reference data for all the signing books available in EU-countries (the bibliographic database), and a database for those signing books that were viewed and described in detailed by consortium partners (the detailed database).

Information in the bibliographic database falls into several categories:

2.2.1 Management information

The date of information input to the database; the name of the person(s) who input the data; information on availability of that video.

2.2.2 Reference information

The title of the video; a translation of the title in English.

The name of the producer and/or publisher, the year of publication, and the country of publication. Also whether the publisher is a deaf organisation, a commercial company, a government body or some other organisation.

The medium: videotape (VHS-PAL or SECAM), CD-ROM, or CD-I; the duration of the production in minutes, and the number of cassettes or CD-ROMs that a production consists of.

2.2.3 Costs

The price of the production (both in national currency and in ECU) and whether it is a national or international production.

2.2.4 Content

Information about the content of the production; for what age group the production is intended; whether it is fiction or non-fiction; the subject of the video; keywords and a short description of the contents.

In general, two very broad categories of signing books were distinguished: those that are intended to make information from a hearing community or culture accessible to deaf people by presenting it in sign language, in contrast to those which are made to give hearing and deaf persons access to information about a deaf community or culture.

The attractiveness and effectiveness of the story or script on which a video production is based is a determining factor for the attractiveness and effectiveness of the final production. At the moment, very little is known about what determines the attractiveness and effectiveness of a story or script for deaf persons. Incidental comments by deaf actors and authors (e.g. those cited by Ahlgren & Bergman, 1990; Bragg, 1989) indicate that there is indeed a deaf way of story telling, which differs from European hearing story-telling conventions. The hearing way of storytelling is linear and very much influenced by the fact that stories are mostly presented in print. The deaf way of storytelling is said to be spatial and can be compared to the storytelling that is practised in non-literate communities (Branson, Miller & Gede Marsaja, 1996, page 39-57). A field was therefore added to the database for notes about 'remarkable' stories or scripts.

Finally, in relation to content, information is registered as to whether a video was an original sign language production, or a translation or adaptation from a hearing source (book, video, film). Original productions are productions specifically made for deaf viewers, with a scenario and script specifically written for presentation in sign language. Examples are signing books with sign language poetry, and news-magazines for the deaf. Translations are productions based on an original 'hearing' source, with a translation of the spoken or printed text in sign language but no other adaptations made. Examples are an educational video with an insert (box) that shows an interpreter presenting the spoken information in sign language, or a printed book that is translated and recorded on video in sign language. Adaptations are also based on a hearing source, but the source is rewritten or re-edited to be more accessible for deaf viewers. These include, for example, a fairy tale that is not only translated into sign language and recorded on video, but that is also dramatised by actors.

2.2.5 Language information

Which sign language(s) are included in the video, and/or what forms of sign supported speech. Since there is as yet no internationally agreed way to indicate national sign languages, a standard coding scheme was developed (see the appendix). Sign languages are indicated with SL- and the ISO country code, sign supported speech with SS and the ISO country code. Belgium and Spain each have two different sign languages, as does GB.

Many signing books include the spoken language. This can either be the original soundtrack, or a voice over of (parts of) the signed information. Information in the spoken language is helpful for (hearing or hard-of-hearing) users without a full command of sign language, and when the video is watched in mixed audiences of deaf and hearing persons - e.g. a deaf child with hearing siblings and/or parents.

Finally, the spoken language can be presented through captions or subtitles. In this document, the term captions is used for all printed texts that are visible on screen. The term subtitles is used to refer only to those lines of text on the screen that are direct representations of spoken or signed texts in a programme.

For each video information was registered as to what language codes were included, on the basis of information in publishers' catalogues and advertisements. More detailed information about each language code was entered for those signing books that were viewed by a member of the consortium.

2.2.6 Sign language presentation

The main focus of the project is the way sign language is presented on video. One of the most important factors contributing to the effectiveness of a sign language video is of course whether or not the sign language presentation is correct. The sign language presenter must use the sign language correctly (with correct syntax, vocabulary, register and pragmatics). If the production is a translation or adaptation, the original spoken or printed text must have been translated correctly into sign language. Since we have described signing books from all countries of the European Union, with source and sign languages that are unfamiliar to the consortium, we have not addressed this aspect when describing signing books for this workpackage. In later workpackages, data will be collected with respect to this issue.

Many other aspects may contribute to the effectiveness and attractiveness of a sign language presentation. Some have to do with the sign language presenter him- or herself. Is s/he the main character in the video, or only included as a neutral "sign-over" (comparable to the "voice over"). If the sign language presenter is included as a "sign-over", is s/he the same gender as the speaking presenter? In talking books for the blind, a male voice is often preferred for books written by men, and a female voice for books written by women. Sign language interpreters, however, are rarely if ever selected on the basis of gender. It is not known whether it is distracting to watch a female signer present an obviously 'male' text or story, and vice versa.

What is the role of the sign language presenter: does s/he only interpret spoken texts, or does s/he add information, for instance descriptions of what is not visible (e.g. environmental sounds). Does s/he add explanations, commentary etc. and act as an intermediary between the viewer and the speaker / text?

When a text that is presented in sign language is based on a spoken or written source, the text can be translated in sign language in several stages, by one or several sign language users, linguists and/or interpreters. The translation can be recorded on video or can be put on paper by means of glosses. The person who presents the text in sign language can use this sign language version as his/her script. S/he can learn the translation by heart before the actual presentation, and/or can present the sign language text on the basis of printed or signed cues. In live presentations and many other situations, the presenter has to interpret written or spoken texts directly into sign language, without the assistance of translators.

If a sign language video is intended for national distribution, the signer will have to take care to use signs that are nationally used, instead of regional variants. If infrequent or new signs are used - such as name signs or technical vocabulary - the signer may first introduce the signs and explain their meaning.

A very relevant characteristic of the sign language presenter may be whether he or she is deaf or a hearing sign language interpreter. Deaf signers - especially native signers (those who learned sign language at an early age as a first language) are usually described as signing more naturally, more fluently, and more expressively than hearing signers. Also, it may be easier for deaf signers to find the right register and sign vocabulary for deaf viewers. Whether the same applies to hearing native signers (hearing children in signing deaf families) is unknown.

On the other hand, during the recording of a video hearing signers can be much more easily cued as to what they have to sign. The original spoken text can be presented auditorily to the signer, and interpreted immediately (that is: translated and presented in real time) in sign language in front of the camera. A deaf signer, on the other hand, will have to memorise his or her text, or will have to be cued visually: either by reading printed text (written not in sign language, but in English or glosses) or by shadowing a second signer who is off-camera. These methods may interfere with the signing speed and the expressiveness and the use of eye-gaze by the signer.

Also relevant is how the signer is recorded by the camera. Does s/he look directly at the camera or at an angle? How much of the signer is visible: signing space (the area extending from the top of the head to the hips) or full-body? This is of particular importance if a signer is not presented in the main television frame (see below) but added in a small(ish) rectangular or oval frame, where the camera may record only signing space. However, if the signer then moves while signing, or produces signs that are larger than normal, his or her hands may be "cut off". "Cut offs" can of course also occur when a signer is presented in the main screen.

The colour of the clothes worn by the presenter and the colour of the background both contribute to the attractiveness and the visibility of the sign language presentation (SIGNBASE, 1997).

The angle of the signer to the camera (eye-line) is relevant for the visibility of the signs and the face of the signer, as well as for the 'register' of the presentation. A sign language presenter who signs directly to the camera may seem to address the viewer(s) personally, engaging the viewer as a direct communication partner; a signer who addresses a visible or invisible audience in the studio may seem more impersonal, with the viewer being an unobserved bystander. Which of the two is more effective, will of course also depend on the subject of the video.

If visuals (graphics, photos, films) are used in a video, it is relevant to register the interaction between the sign language presenter and the visuals. The interaction can be negative, for instance when the sign language presentation blocks out part of the visuals, neutral with no interaction, or positive with the signer actively using the visuals to support his or her signing and vice versa.

The sign language presenter can be presented on the screen in different ways. S/he may be shown in the main screen, s/he can be added to the screen in a box, circle or egg using picture-in-picture technology, or can be shown in front of and/or to the side of a main video using ChromaKey technology. If ChromaKey is used, than the signer and the background visuals can be completely overlapping, or the visuals can be less then full-screen, leaving space to the right or left for the signer. In that case, there can be partial overlap, or none at all. The location of the signer with respect to any visuals and or captions has been shown to be relevant in earlier research (Woll, 1991).

If two or more signers are presented on screen (e.g. in an interview or a discussion) the location of the signers with respect to each other, and (if two or more cameras are used) the switching from one signer to the other are important factors that may affect the visibility of individual signs and the correct interpretation of spatial parameters used by the signer.

Finally, a relevant aspect is whether a sign language presenter uses a great deal of finger-spelling. Finger-spelling is a system in which each letter of the alphabet is represented by a manual configuration. Words can then be spelled "in the air". Finger-spelling is visually less redundant than (most) signs and may be difficult to read if far from the camera or if the contrast between the hands and the signer's clothing and/or the background is insufficient.

2.2.7 Visual information

A sign language video can include visual information other than the sign language presentation. The visual information may be an original film or video-track, illustrations from a book, or films or other images that were specifically made for the sign language video.

The quality of the visuals is important as well as the interaction between the visuals and the sign language presentation. If the visuals are too dominant, they may interfere with the information that is presented in sign language: the deaf viewer will have to choose between watching the sign language presentation or the visuals. If there are more than 2 'active channels' on screen (that is: windows on the screen showing relevant and/or dynamic information) it may be even more difficult for the viewer to decide on what part of the screen to focus.

On the other hand, visuals can support understanding of the story or content of the sign language presentation, especially for children or adults with limited command of sign language or of the specific sign vocabulary on a video.

Visuals can also be used to make a production more attractive - more visually interesting. Finally, they can be used to give cues to the structure of the video (e.g.: when visuals are used to indicate the end of a chapter or story) and may help in navigation (see below).

2.2.8 Captions and subtitles

Captions can be important for viewers who are not fluent signers. Captions can also be important if the subject that is covered in a production requires many unfamiliar signs, or regional signs that may be unfamiliar to some viewers. Captions can then clarify the meaning of these signs. In educational signing books, captions can help bridge the gap between the sign language video and the printed book. In interactive productions, subtitles and captions can be used for navigation purposes (hypertext, searching), for adding bookmarks and for quotations Captions can also be used for supplementary information.

As was noted with respect to sign language interpretation, captions have to represent the spoken or signed text correctly. Because many of the signing books studied for this workpackage are in languages that are unknown to the consortium, this aspect has not been covered.

Other factors may influence the effectiveness of captions: the percentage of the spoken/signed text that is captioned, and whether the subtitles are a literal representation of the spoken/signed texts or an abridged and/or simplified version.

Visual factors that may affect the effectiveness of captions are the colour and size of captions; contrast between the captions and the background; the location of the captions on the screen, and the amount of screen that is covered by captions. Finally, the captions may vary in how they are presented on the screen: the length of time that they are visible; whether they appear on the screen letter-by-letter, word-by-word, or in blocks); how many lines are visible at a time; and how new lines are added (scrolling, rolling right to left or vice versa, rolling top to bottom or vice versa) (van Son, Verboom & Balkom, 1998).

2.2.9 Sound information

The presence and quality of sound will contribute to the attractiveness of a sign language video for (hard-of) hearing viewers. Sound may include voices (original voices or voice-over), environmental sounds, sound effects, music. In cases of environmental sounds and sound-effects, it is important to note whether or not these have been visualised for deaf viewers.

In case of a voice-over, the accuracy of the spoken text is an important issue that was not addressed in this workpackage. As with a 'sign-over' it may be preferable to have the speaker be of the same gender as the signer. In case of more than one signer, one voice can be used for all signers, or a different voice for each signer.

The register of the voice-over, the amount of intonation and the use of special effects may also be important.

Other aspects that may determine the effectiveness and attractiveness of a voice-over are the sound level and quality, also with respect to background music or sounds. In interactive productions, it may be possible to select what auditory information is presented, and/or it may be possible to amplify one channel (speech) independently of other channels (background sounds, music).

2.2.10 Editing and special effects

After a video has been recorded, it can be edited in many different ways. Editing can either add to, or subtract from the attractiveness and/or effectiveness of a video. For each video, average length of shot was noted, ranging from the entire video shot from one fixed camera-position with no cuts or changes, to a patchwork of bits and pieces (MTV-style). What is most effective will of course depend on the age of the target group and the objective of the video; however, if the content is limited to conveying information, long shots may well be preferable. Often where there is no or only limited use of visuals, the visual presentation may seem boring - to viewers who are unfamiliar with the sign language. However, it may well be that a viewer who does know sign language may actually prefer these 'boring' visual set-ups, because s/he can focus on the sign language without distraction by frills that may have nothing to add to the story that is being told. Whether this is also the case for children, and for children whose command of sign language is still limited, is an issue for further research.

2.2.11 Navigation

A sign language video can be as short as a few minutes, or as long as several hours. The longer the video is, the more important it is that there are navigation aids. It is inefficient for a viewer to watch hours of video, before being able to conclude that the information that was sought is not included in that production. A table of contents can be printed on the video box, and/or can be shown (in sign language and/or in text) at the beginning of the video. A clear visual demarcation of changes in subjects, chapters or stories enables the viewer to use the fast-forward option while searching for information on the video. A table of contents or a map of the entire video - with an indication of 'you are here' - used at regular intervals (e.g. to indicate changes in topics or stories), will help the user find his/her place while searching - and can help as a memory aide when a video is not watched in its entirety in one sitting.

The navigation possibilities in interactive productions are of course much greater than in analogue videos. Interactive productions have the possibility of hypertext, searching for text, adding bookmarks, adding notes, and/or copying (printed or signed) quotes from a production.

2.2.12 Overall usability

Research on the usability of sign language videos for deaf users is almost non-existent (exceptions are Woll, 1991 and Allsop et al, 1990). In review articles, different aspects are often mentioned. In the database, information on the usability of specific signing books was input, with a reference to source of the information (review article or evaluation study).

2.2.13 Usability for visually impaired persons

The incidence of visual problems in deaf persons is higher than in the hearing population. A large percentage of signers are therefore at risk for visual problems that may interfere with the effectiveness and/or attractiveness of signing books. As in the hearing population, there is an even higher incidence of vision problems in the elderly deaf.

One of the objectives of this project is to pay special attention to the needs of visually impaired signers. A number of aspects that may specifically influence the effectiveness or attractiveness of a video for low vision signers, is specified in the database. These are: contrast of the signer and signs to background and clothing; overall visibility of the signers, overall visibility of visuals (pictures, film, animations), and overall visibility of captions (also see: Video for Visually Impaired Learners, 1998).

2.3 The production and distribution process

To get more background information about the production and distribution process of signing books, a list of questions was made that could be used as a guideline for interviewing producers of signing books. The questions were meant to get information about:

Scenario

Questions

Someone, somewhere decided to make a video production in Sign Language:

  • Who made this decision?
  • Why, what were the motives?
  • When was this decision made?
  • What kind of production?
  • For whom was the production intended?
  • What was the objective of the production?
  • Was it a first production? Was it one in a series?
  • Was it a commercial production, or not-for-profit?

The content was selected:

  • What content?
  • Why this content?
  • Who selected the content?
  • Was it existing material?
  • What was the original material (book, film, software)
  • Was the content adapted?
  • How & why?
  • Was (any of) the original material used?
  • How were copyright issues for the original material dealt with?

A producer/production company was selected

  • Who/what company, for what reasons?
  • Deaf/hearing?
  • Previous experience?
  • Large/small scale company?
  • Price range?
  • Facilities?

A format was decided on:

  • Tape - disc?
  • Duration of the production?
  • Visuals?
  • Visuals: still pictures vs. film?
  • Content of the visuals?
  • Visuals vs. text vs. sign language: simultaneous, sequential, interactive?
  • Sign language presented how
  • Captions and subtitles?
  • Voice-over?
  • Music, sound effects?

A script was written:

  • Who wrote the script?
  • Was it an original story, adaptation, or translation?
  • Were deaf people involved? How? Translation into sign language - how? "Standard" sign language or regional dialect? Literal or free translation? Difficult words, translated how? Was the language level of the viewers (e.g. children) taken into account?
  • How were decisions made about what was to be presented in Sign Language, what through film, etc. What about the order information was to be presented, and the 'story-line'?
  • What was specified in the script about camera angle, position, etc.

Presentation:

  • How was/were the signer(s) selected? What criteria?
  • Hearing/deaf?
  • How was s/he coached?
  • One or more signers?
  • Male or female?
  • Clothes, looks, costumes?
  • se of props?
  • Use of expression: neutral presenter, or with emotion, make up?
  • Average signing speed? (fast, normal, slow)
  • Breast-up, waist-up, knees-up, feet-up?
  • Standing / sitting?
  • How was the signer cued?

Recording:

 

 

 

  • Inside / on location?
  • Live or to be edited?
  • Lights - how many, what kind?
  • Cameras - how many, what kind?
  • Sound
  • Contrast
  • Angle
  • Distance
  • Other people involved?
  • Audience

Editing + 'information load':

 

 

  • Who edited the tape?
  • How was the tape edited?
  • What was the story schema, was their a detailed script or storyboard
  • Content presented fast, average or slow (Average length of time on a topic?)
  • Average number of changes in camera-focus per minute?
  • Average number of shots per minute?
  • What was done to catch the attention of the viewer, and to hold the attention?
  • Special effects?

Quality Control:

  • What was done to make sure the production was 'usable': user involvement during the production?
  • Quality standards?
  • Were usability tests done before /during / after the production?
  • Were low vision users taken into account? If so, how?

Marketing:

 

  • Who marketed the video (commissioner, producer, 3rd party/)
  • How: advertisements, direct mail, ...?
  • Budget?
  • Copyright issues video-material
  • When was the video marketed.

Sales / distribution:

  • One point of sale or more?

One time only project, or one of many?

  • How many?
  • How many years of experience?
  • Plans for future productions?

Time schedule:

  • How long did it take from the decision to make the production, until the actual publication?
  • What took the most time?

Financing:

 

 

  • Was the production subsidised?
  • What was the sales price?
  • Was it a commercial or not-for-profit production?
  • What was paid for, what was volunteer work; what were the rates for 'paid for' work (commercial, or charity rates?).

Co-operation, national, inter-national:

  • Was the production a co-production?
  • What parties were involved?
  • In case of international co-operation: what was done nationally, what internationally?
  • Difficulties / advantages of co-operation

Evaluation, any other business:

  • What is "remarkable", "special" or "innovative" about this video according to the makers, what was the motivation, was it successful, worth repeating?
  • Major problems that had to be solved during the production...
  • Advice, wishes, regrets,. etc.

2.4 Selection of aspects to be researched in workpackage 5

On the basis of a qualitative and quantitative analysis of the results of workpackage 3, at the beginning of workpackage 5 aspects of sign language productions will be selected that have presented themselves as 'unresolved issues'- i.e. issues for which the data offer insufficient basis for the development of guidelines. A first selection of aspects will be made for the three target groups separately. A second selection will then be made on the basis of relevancy for all three target groups. A third and final selection will be made on the basis of practicality and feasibility, considering the resources available for this project.



Footnote

(2) Because of the delay in the signing of the consortium's contract by the EC, City University was unable to start until 1 April 1998 and the GB-questionnaire was not sent out until mid April.