Learn about the complete journey of a neural machine.
We just posted a course on the freecodecamp.org YouTube channel that is a comprehensive journey through the evolution of Continuity Models and Neural Machine Translation (NMT). It combines historical developments, architectural innovations, mathematical insights, and Petrarch’s transcriptions of historical papers that have shaped modern NLP and AI.
Course Features:
A detailed narrative tracing the history and developments of RNN, LSTMS, GRASS, SEQ2SEQ, ATTENTION, GNMT, and multilingual NMT.
Copies of 7 historical NMT papers in pytorch, so learners can code and reconstruct step-by-step history.
Explanation of the mathematics behind RNNS, LSTMS, GRUS, and transformers.
Architectural comparisons, visual descriptions, and conceptual explanations with interactive demos like the Transformers playground.
All parts of the course are:
Evolution of RNN
The evolution of machine translation
Machine translation techniques
Long-Short-Term Memory (Review)
Learning Phrase Representations Using RNN (Encoder-Decoder for SMT)
Learning Phrase Representations (Replication of Pytorch Lab – CHO et al., 2014)
Seq2seq learning with neural networks
Seq2Seq (Pytorch Lab – Replication of Sutskever et al., 2014)
NMT joint alignment learning (Behdnau et al., 2015).
NMT co-aligning and learning to translate (Patrich Lab – Replication of Behdanau et al., 2015)
On the use of very large target words
Large Vocabulary NMT (Pytorch Lab – Replication of Jean ET, 2015)
Effective approaches to attention (Long et al., 2015)
Attention approach (replicating Pytorch Lab – Long et al., 2015)
Long-Short-Term Memory Network (Deep Description)
Attention is all you need (Vaswani et al., 2017).
Google Neural Machine Translation System (GNMT – Wu et al., 2016)
GNMT (Pytorch Lab – Adapted from Wu et al., 2016)
Google’s Multilingual NMT (Johnson et al., 2017)
Multilingual NMT (Pytorch Lab – Replication of Johnson et al., 2017)
Transformer vs. GPT vs. Brit architectures
Transformers Playground (Tool Demo)
SEQ2SEQ idea from Google Translate tool
RNN, LSTM, GRU Architecture (Comparison)
LSTM and Guru equation
View the full course freecodecamp.org YouTube channel (7 hour clock)