NLP

Milestones

  • 2013: Word2vec and N-grams
  • 2014: RNN/LSTM
  • 2015: Attention mechanism
  • 2017: Large Pre-Trained Language Models (Transformers)
  • 2018: BERT
  • 2019: T5
  • 2020: GPT-3