Automatic Translation of German Train Announcements into Swiss German Sign Language

Trainslate Project: Goals

In the Trainslate (=train+translate) project we are developing a system that automatically translates German train announcements of the Swiss Federal Railways (Schweizerische Bundesbahnen, SBB) into Swiss German Sign Language. The idea for such a system was suggested to us by Deaf signers in Switzerland.

DSGS sign for GEBÄRDENSPRACHKURS (‘sign language course’) in HamNoSys

On the target-language side the announcements are represented in the Hamburg Notation System for Sign Languages (HamNoSys) (Prillwitz et al., 1989). HamNoSys is machine-readable: it offers an XML representation, the Signing Gesture Markup Language (SiGML) (Elliott et al., 2000), which can be used to drive an avatar.

Avatar

The avatar of our system is displayed in a smartphone application. The corresponding German text is shown as a subtitle beneath the avatar. Announcements will remain available for a certain time so that they can be replayed. Hence, the target group of the application are Deaf and hearing-impaired users on the one hand, and hearing users who are looking for a replay functionality for train announcements on the other hand.

Data and Resources

We received data from the Swiss Federal Railways (Schweizerische Bundesbahnen, SBB). Following is an example of a train announcement.

Ausfallmeldung zum RegioExpress nach Olten. Der RegioExpress nach Olten, Abfahrt um 6 Uhr 41, fällt aus.
‘Notice of cancellation of the regional express to Olten: The RegioExpress to Olten, scheduled to leave at 6:41, has been cancelled.’

For our system we are using an already existing lexicon of DSGS signs. Work on the lexicon was started in 1996 by Penny Boyes Braem, Director of the Center for Sign Language Research. Currently it contains about 9000 signs.

Project Phases

The project is divided into three phases:

  1. In the first phase a fixed number of announcements are manually translated into Swiss German Sign Language. This consists of the following steps:
    • Signing the announcements in front of a camera
    • Notating the announcements in HamNoSys based on the video recordings
  2. In this phase the translation system is developed. We apply statistical methods. Our department has extensive expertise in the area of statistical machine translation.
  3. Here we develop the smartphone application.

 

Team

Project head:

Researcher:

Bibliography

Penny Boyes Braem. A multimedia bilingual database for the lexicon of Swiss German Sign Language. Sign Language & Linguistics, 4:133–143, 2001.

R. Elliott, J. R. W. Glauert, J. R. Kennaway, and I. Marshall. The development of language processing support for the ViSiCAST project. In Proceedings of the Fourth International ACM Conference on Assistive Technologies, Assets'00, pages 101–108, Arlington, Virginia, United States, 2000. ACM.

Siegmund Prillwitz, Regina Leven, Heiko Zienert, Thomas Hanke, and Jan Henning. HamNoSys: Version 2.0: Hamburg Notation System for Sign Languages. An introductory guide. Signum, Hamburg, 1989.