Sources et lectures complémentaires

Numérotées dans l'ordre de leur première apparition dans cet article.

  1. Gage, P. (1994). A new algorithm for data compression. The C Users Journal, 12(2).
  2. Sennrich, R., Haddow, B. and Birch, A. (2016). Neural machine translation of rare words with subword units. https://arxiv.org/abs/1508.07909
  3. Templeton, A. et al., Anthropic (2024). Scaling monosemanticity: extracting interpretable features from Claude 3 Sonnet. https://transformer-circuits.pub/2024/scaling-monosemanticity/
  4. Mikolov, T. et al. (2013). Efficient estimation of word representations in vector space. https://arxiv.org/abs/1301.3781
  5. Kaplan, J. et al. (2020). Scaling laws for neural language models. https://arxiv.org/abs/2001.08361
  6. Vaswani, A. et al. (2017). Attention is all you need. https://arxiv.org/abs/1706.03762
  7. Llama Team, AI at Meta (2024). The Llama 3 herd of models. https://arxiv.org/abs/2407.21783
  8. Clark, K. et al. (2019). What does BERT look at? An analysis of BERT's attention. https://arxiv.org/abs/1906.04341
  9. Levesque, H., Davis, E. and Morgenstern, L. (2012). The Winograd schema challenge. Proceedings of KR 2012.
  10. Geva, M. et al. (2021). Transformer feed-forward layers are key-value memories. https://arxiv.org/abs/2012.14913
  11. Liu, N. F. et al. (2023). Lost in the middle: how language models use long contexts. Transactions of the ACL, 12, 157-173. https://arxiv.org/abs/2307.03172