Kolloquiumsplan HS 2020

Kolloquium HS 2020: Berichte aus der aktuellen Forschung am Institut, Bachelor- und Master-Arbeiten, Programmierprojekte, Gastvorträge

Zeit & Ort: alle 14 Tage dienstags von 10.15 Uhr bis 12.00 Uhr, BIN-2.A.01 (Karte)

Verantwortlich: Duygu Ataman


Vortragende & Thema


  • Einführung ins Kolloquium, Duygu Ataman & Martin Volk
  • Nicolas Spring, BA Thesis presentation: "Probing Tasks for Noised Back-Translation"
  • Prof. Dr. Lena Jäger, "What eye movements can and cannot tell us: Challenges in eye-tracking research"
  • Olga Sozinova, PhD research presentation
  • Adrian van der Lek, MA Thesis presentation: "Evaluating the cognitive plausibility of sentence-level embeddings"


  • Invited Talk by Dr. Reto Gubelmann
  • Patrick Haller, "Investigating variability in letter - speech sound association learning. A model-based fMRI approach."
  • Invited Talk by Prof. Dr. Jan Niehues, Maastricht University, "(Simultaneous) Speech Translation: Challenges and Techniques"


  • Marek Kostrzewa, PhD research presentation
  • Jan Deriu, PhD research presentation


  • Invited Talk by Dr. Garrett Smith, the University of Potsdam (virtual)
  • Tannon Kew, PhD research presentation


  • Noëmi Aepli, PhD research presentation
  • Tatiana Ruzsics, PhD research presentation


  • Nicolas Spring, Probing Tasks for Noised Back-Translation, 15.09.2020
    When using back-translation, adding artificial noise to the synthetic source data leads to better model performance. This led to the hypothesis that noise can serve as an implicit label, signaling to the model that a given sentence has been back-translated. In this thesis, we use probing tasks, an adaptable model introspection technique, to verify this hypothesis, and we investigate the way in which the outputs of a model trained with explicitly tagged back-translation change when decoding a source sentence with and without an explicit label. We show that sentences with noise can be distinguished from genuine sentences with the information present in model states, thus confirming that noise can serve as an implicit label. Furthermore, we discover that decoding sentences with an explicit label yields translations that are lexically more diverse, while no clear changes can be observed in word order in respect to the source sentences. BLEU scores of hypotheses produced with an explicit label are lower than that of their standard-decoding counterparts, indicating that the interaction between lexical diversity and translation quality measured in BLEU is not yet fully understood.

  • Prof. Dr. Lena Jäger, What eye movements can and cannot tell us: Challenges in eye-tracking research, 15.09.2020
    Eye-tracking methodology is considered the gold standard in psycholinguistic reading reseearch (Rayner et al., 2006). Moreover, eye-tracking has been attracting increasing attention from researchers developing various kinds of technological applications, including language technology. Unfortunately, the usage of eye-tracking data as a dependent variable for psyhcolinguistic research or as (additional) features for language models entails methodological challenges at the level of the hardware, the data preprocessing, the representation of the data, the model architecture, as well as the statistical practices. In this talk, I will discuss the potential, the challenges and the limitations of using eye-tracking for linguistic research. 

  • Adrian van der Lek, Evaluating the cognitive plausibility of sentence-level embeddings, 29.09.2020
    In order to assess different approaches to obtaining word and sentence embedding vectors, and to form a basis upon which new methods can developed, formal evaluation is necessary. Since their inception, embeddings have been evaluated using extrinsic means, i.e. in downstream tasks, which serve as proxies to real world applications. More recently, intrinsic evaluation methods have been proposed, which strive to investigate inherent properties of embedding vectors, but typically only investigate very specific phenomena and can be subject to individual bias. An alternative is to leverage processes occurring in the human brain whilst reading or processing speech, in order to obtain a measure of the cognitive plausiblity of embedding approaches. Hollenstein et al. (2019) have shown that word embedding vectors can be evaluated in a neural regression setting, where embeddings predict cognitive signals aggregated on the word-level. Such signals can be obtained through recordings of physiological monitoring methods such as eye-tracking, EEG and fMRI. Tested approaches differ significantly in how well they predict cognitive signals and rankings correlate between datasets, modalities, as well as with results of extrinsic evaluations. In my thesis, I applied the approach by Hollenstein et al. (2019) to sentence embeddings and sentence-level cognitive signals, with necessary adaptations. I evaluated eight sentence embedding approaches of varying complexity, using cognitive datasets which offer sufficient data on the sentence level. Between approaches, I observed distinct rankings, which differ considerably between the modalities eye-tracking and EEG. I also informally assessed correlation between cognitive and previous intrinsic and extrinsic evaluation results. Results point toward a potential relationship between EEG and tasks measuring semantic relatedness and textual similarity, and to a lesser extent, between eye-tracking and linguistic probing tasks.

  • Prof. Dr. Jan Niehues, (Simultaneous) Speech Translation: Challenges and Techniques, 27.10.2020
    In today’s globalized world, we have the opportunity to communicate with people all over the world. However, often the language barrier still poses a challenge and prevents communication. Machines that automatically translate the speech from one language into another one are a long dream of humankind. In this presentation, we will start with an overview on the different uses cases and difficulties of speech translation. We will continue with a review of state-of-the-art methods to build speech translation system. We will start with reviewing the translation approach of spoken language translation, a cascade of an automatic speech recognition system and a machine translation system. We will highlight the challenges when combining both systems. Secondly, we will discuss end-to-end speech translation, which attracted a rising research interest with the success of neural models in both areas. In the final part of the lecture, we will highlight several challenges of simultaneous speech translation: Latency, sentence segmentation and stream decoding and present techniques that address these challenges.