Generative AI · Part 1 of 3
How a Neural Network WorksNeurons, layers and learning, the building blocks of Claude, Gemini and ChatGPT
Summary
Claude, Gemini, ChatGPT, Mistral: every large language model (LLM) is built on the same architecture, the transformer, introduced by a Google team in 2017. This series of three articles explains how it works, without formulas, using examples small enough to follow by hand.
This first article sets the scene and lays the foundations. It starts with a short history: why the neural networks of the 1980s and 1990s struggled with language, and what the 2017 paper "Attention Is All You Need" changed. It then explains what an artificial neuron computes, why neurons are stacked in layers that are really tables of numbers, and how a network finds those numbers by itself, with a spam filter trained round by round. Every number can be checked, by hand or in the companion spreadsheet.
The companion spreadsheet opens in Excel, LibreOffice or Google Sheets. It was built with the help of Claude, Anthropic's AI assistant.
Along the way come two ideas that matter for any executive deploying AI: a model holds three kinds of numbers, chosen, learned and computed; and its reliability is measured on examples, never proven. The second article uses these foundations to follow a sentence through a language model; the third explains how such models are trained, and why that requires GPUs.