Pretrained Embeddings

Student: N.N.

Supervisor: Mathias Müller, Annette Rios

Introduction

Experiment with pretrained embeddings, e.g.:

  • pretrain (subword) embeddings for NMT on monolingual data
  • initialize and train normally
  • initialize and freeze embeddings
  • maybe: train with dual updates (more advanced experiment)
  • evaluate influence of learning rate
  • compare word vs. subword embeddings
  • compare vanilla embeddings vs. NMT trained embeddins