Trainslate: Automatic Translation of German Train Announcements into Swiss German Sign Language

In the Trainslate (=train+translate) project we developed a system that automatically translates German train announcements of the Swiss Federal Railways (Schweizerische Bundesbahnen, SBB) into Swiss German Sign Language.

On the target-language side the announcements are represented in the Hamburg Notation System for Sign Languages (HamNoSys) (Prillwitz et al., 1989). HamNoSys is machine-readable: it offers an XML representation, the Signing Gesture Markup Language (SiGML) (Elliott et al., 2000), which can be used to drive an avatar.

The avatar of our system is displayed in a smartphone application. The corresponding German text is shown as a subtitle beneath the avatar. Announcements will remain available for a certain time so that they can be replayed. Hence, the target group of the application are Deaf and hearing-impaired users on the one hand, and hearing users who are looking for a replay functionality for train announcements on the other hand.

We received data from the Swiss Federal Railways (Schweizerische Bundesbahnen, SBB). Following is an example of a train announcement.

Ausfallmeldung zum RegioExpress nach Olten. Der RegioExpress nach Olten, Abfahrt um 6 Uhr 41, fällt aus.
‘Notice of cancellation of the regional express to Olten: The RegioExpress to Olten, scheduled to leave at 6:41, has been cancelled.’

For our system we are using an already existing lexicon of DSGS signs. Work on the lexicon was started in 1996 by Penny Boyes Braem, Director of the Center for Sign Language Research. Currently it contains about 9000 signs.

The project is divided into three phases:

1. In the first phase a fixed number of announcements are manually translated into Swiss German Sign Language. This consists of the following steps:
   • Signing the announcements in front of a camera
   • Notating the announcements in HamNoSys based on the video recordings
2. In this phase the translation system is developed. We apply statistical methods. Our department has extensive expertise in the area of statistical machine translation.
3. Here we develop the smartphone application.

Project head:

Prof. Dr. Martin Volk

Researcher:

Prof. Dr. Sarah Ebling

Penny Boyes Braem. A multimedia bilingual database for the lexicon of Swiss German Sign Language. Sign Language & Linguistics, 4:133–143, 2001.

R. Elliott, J. R. W. Glauert, J. R. Kennaway, and I. Marshall. The development of language processing support for the ViSiCAST project. In Proceedings of the Fourth International ACM Conference on Assistive Technologies, Assets'00, pages 101–108, Arlington, Virginia, United States, 2000. ACM.

Siegmund Prillwitz, Regina Leven, Heiko Zienert, Thomas Hanke, and Jan Henning. HamNoSys: Version 2.0: Hamburg Notation System for Sign Languages. An introductory guide. Signum, Hamburg, 1989.