SIGMA: Fundamental linguistic tools for sign language machine translation

Diagram of the project structure showing the five research questions situated between the modalities speech and sign as well as the disciplines involved in the project: automatic speech processing, computer vision, natural language processing, spoken language linguistics, sign langauge linguistics, spoken language assessment, second language acquisition, and sign language assessment. Further, the two proficiency levels involved (L1 and L2) are shown.

SIGMA aims to provide fundamental tools for automatically processing sign languages, with the main goal of improving a higher-level task: sign language machine translation. Improving machine translation for sign languages is relevant for Swiss and international societies. While for hearing persons translation technology is mature and usable in every-day situations, such tools do not exist for Deaf sign language users. Deaf individuals can only be full members of a digital society if translation technology is afforded to them. The project is situated at the intersection between computer science, linguistics and accessibility studies, and therefore is highly interdisciplinary.

The tools we work on include: