Navigation auf uzh.ch
In this thesis you are going to develop a language model for sign languages, based on human pose estimation.
Language models are an important technology for text-based NLP systems. For sign languages, which are usually not represented as text, language models do not exist. The goal of this work is to develop one.
Instead of working with video data, sign language data will be represented as human pose estimation data. Widely used systems for pose estimation are OpenPose (https://github.com/CMU-Perceptual-Computing-Lab/openpose) and Mediapipe Holistic (https://google.github.io/mediapipe/solutions/holistic.html).
You will conduct a thorough review of how current languages models are trained and evaluated, to see which components and principles can be applied to non-textual representations of language.
Then your task is to develop a training procedure and evaluation method that make sense for modeling sign language data. Your approach can also be inspired by recent advances in sign language processing.