Deep Learning and Neural Machine Translation

If you start from scratch with neural networks and neural machine translation, here are my personal ("biased" - if you understand this pun already, skip the introductions to neural networks in general!) recommendations.

Did I miss something important or do you have a comment? Please write to mmueller AT


Before we even get to neural networks, here are some pointers to stuff deep learning is building on:

Neural Networks in General

Resources that introduce you to neural networks in general, largely without applying them to NLP problems yet:

Suggestions for papers that discuss specific methods that 1) are very widely used and 2) are what actually makes deep learning work well:

Neural networks in NLP

Moving on to how neural networks are used in NLP:

Neural Machine Translation

Neural networks are used extensively in machine translation. Here are resources you should read to get up to speed. It is assumed that you are familiar with machine translation in general.

Suggestions for papers about specific methods (usually within the Encoder-Decoder framework, but independent of the type of network (RNN or not)):

Deep Learning Libraries (and actual code!)

You are encouraged to scrutinize and write actual code as you learn about NMT: