03How a network learns

The training loop, three ways to learn, backpropagation, and why a network is never right every time.

The training loop

Think of a sound engineer at a mixing desk with billions of knobs, told after each recording which way to turn every one of them, and by how much. No single knob understands music, yet after enough recordings the desk plays in tune. Training a neural network works exactly like this. The weights in chapter 2 were chosen by us; in practice, nobody chooses them. Training finds them, by repeating the same loop until the network stops improving:

The training loopStart with random weights, then repeat: forward pass, error, gradient, adjust. Startrandom weights Forward passrun the examples Errorvs right answers Gradientwhich way, how much Adjusta small step ↻ repeat, until the error stops going down Our spam neuron: 60 emails, 445 rounds. A large language model: trillions of words, about a million rounds.
  • Start. All the weights are set to random values. The network knows nothing, and its answers are no better than chance.
  • Forward pass. The examples are run through the network, layer after layer, table after table, and the network produces its answers.
  • Error. Each answer is compared with the right one, which a person supplied when labelling the examples. The gap is the error.
  • Gradient. For each weight, the training works out which way it should turn, and how strongly, to reduce the error. That pair, direction and strength, for every weight at once, is called the gradient.
  • Adjust. Every weight takes a small step in the direction its gradient indicates. Then the loop starts again.

The usual image for the gradient is a hiker lost in fog on a mountainside, trying to reach the valley. They cannot see where it is, but they can feel the slope under their feet, and if they always step downhill, they will get there. The landscape is the error, the hiker's position is the set of weights, and the slope is the gradient. That is why the method is called gradient descent.

One question remains: how long should each stride be? That is the step size (specialists also say learning rate), and it is chosen by the engineers, not learned. The gradient says which way is downhill and how steep it is; the step size says how far to go. Too short, and the hiker is safe but painfully slow. Too long, and the hiker overshoots: one giant stride carries them past the bottom of the valley and up the opposite slope, the next one carries them back past it again, and they never settle at the bottom. For the network, that means an error that jumps up and down instead of falling.

Training, round by round

Our spam neuron now looks at five clues: the word "free", an unknown sender, my name in the email, the word "urgent", and a link. To train it, we have 80 emails, each labelled by a person. We use 60 to train it, and keep 20 aside. The neuron never learns from those 20; they serve only to check its progress.

Training seen through the weight tableA five-clue spam neuron at rounds 0, 20, 100 and 445: weights, two example scores, and accuracy on kept-aside emails.Spam email: contains "free", unknown sender, "urgent", a link. Right answer: spam (1).Colleague's email: uses my name, nothing else. Right answer: not spam (0).Weights, one per clueScoresRound"free"unknownsendermyname"urgent"linkbiasspamemailcolleagueemailcorrect onkept-aside0random−0.2−0.30.1−0.40.0−0.10.26 ✗0.50 ?50%200.30.4−0.20.00.6−0.60.68 ✓0.31 ✓65%1001.01.5−0.70.51.5−2.10.92 ✓0.06 ✓80%445stop1.82.4−1.10.72.6−3.70.98 ✓0.01 ✓80%20 rounds of adjusting80 more rounds345 more roundsSame table at every round: only the weights change. Score above 0.5 = flagged as spam.Stopped at round 445: from there on, the error on the kept-aside emails no longer falls.
Real training run: 80 emails, 60 used for training and 20 kept aside, random starting weights, step size 0.5.

Round 0: random weights

The weights are drawn at random. The neuron's verdicts are no better than a coin toss: right on 50% of the kept-aside emails. The obvious spam email in the table gets a score of 0.26, so the neuron lets it through. The colleague's email gets 0.50: the neuron has no idea.

The first adjustment, by hand

For a single neuron, the rule is simple. The error is the right answer (1 for spam, 0 otherwise) minus the score. Each clue present in the email has its weight moved by that error, times a small step size; clues absent from the email are left alone, and the bias moves like a present clue. Take the obvious spam email: its score is 0.26 where it should be 1, so the error is +0.74. With a step size of 0.5, the four clues it contains ("free", "unknown sender", "urgent", "link") each gain 0.5 × 0.74 = 0.37. "My name", absent, does not move. The neuron now leans towards spam for emails like this one.

Each round

In practice, the adjustments of all 60 training emails are averaged and applied together, once per round. Round after round, the weights take shape. "free", "unknown sender" and "link" grow positive. "My name" turns negative. The bias moves down, which raises the bar for calling something spam. By round 20, the neuron is right 65% of the time. It reaches 80% around round 30 and is still there at round 100.

Training the five-clue spam neuronAccuracy on training and kept-aside emails rises from about 50 to 80 percent, after a dip in the first rounds, and plateaus; training stops at round 445.0%50%100%0125102030501002004451000stop: round 445early rounds: a dipTraining round (uneven scale)Correct on the 60 training emailsCorrect on the 20 kept-aside emails
Same training run as the table above. The plateau at 80% is the best these five clues allow on this data.

Three things are worth noticing in this curve:

  • The error falls steadily, the score does not. The average error on the kept-aside emails falls at every round, from 0.77 to 0.43. The share of correct answers moves in jumps, and even dips in the first rounds. Training follows the error: each round makes the neuron a little less wrong on average, even when an individual verdict flips the wrong way.
  • Nobody set the weights. They came out of the 60 emails. In this sample, links happened to be very common in spam, so the neuron gave "link" its largest weight. Another sample would give somewhat different weights. A network is a summary of its data, with its quirks.
  • Progress slows down. The neuron reaches 80% by round 30. After that, it keeps refining its scores but changes no verdict.

When to stop

Training does not stop when the neuron is perfect, because it never will be. It stops when the kept-aside emails show no further progress. Here, the error on those emails falls until round 445, then stops falling, and even starts to creep back up very slightly. That is the signal. Continuing would only teach the neuron the quirks of its 60 training emails, which do not carry over to new ones. Specialists call this overfitting, and the stopping rule early stopping. In a real project, a second test applies: whether the network is accurate enough for its purpose. That is a business decision. A spam filter right 80% of the time may be acceptable; a medical triage tool may need far more, and may never get there with the data available.

Try it. In the companion spreadsheet, the "Training a neuron" tab replays these rounds with live formulas. Change the step size from 0.5 to 0.05 and our spam neuron takes far longer to become a seasoned professional: after 500 rounds, it is still learning. Raise it to 20 and it overshoots, like the hiker taking giant strides: the error on the kept-aside emails jumps up and down and never settles.

Three ways to learn

The spam neuron learned from emails that a person had labelled "spam" or "not spam". Learning from examples that come with their right answer is called supervised learning. It is the most common form of machine learning, and its main limitation is the labelling itself: someone has to provide the answers, which is slow and expensive at scale. Two other families exist:

  • Unsupervised learning works without any answers. The network is given data and looks for structure in it on its own: for example, grouping customers into segments that nobody defined in advance.
  • Reinforcement learning works without examples of the right answer, but with a score after the fact. The network tries something, receives a reward or a penalty, and adjusts. This is how programs learned to play games at superhuman level.

Large language models are trained in a way that does not fit neatly into these boxes. The third article explains why.

Through several layers: backpropagation

For a single neuron, the gradient is easy to find, as the rule above shows. In a network with several layers, it is harder. When the final verdict is wrong, how much is each weight of the first layer to blame, given that it only affects the verdict through the layers above it?

The answer is a method called backpropagation, popularised in 1986[2]. It starts from the error at the output and works backward, layer by layer, sharing out the responsibility: the last table first, then the one before it, down to the first. The result is a gradient for every weight in the network, at a cost similar to that of the forward pass. The details involve calculus, but the principle is the one above: work out which way each weight should turn, then take a small step. This method trains every neural network in use today, the largest language models included.

Why never 100%

In our spam detection example, after 445 rounds of training, the neuron is still wrong on one email in five, and no amount of further training will fix that. Among our 80 emails, seven share exactly the same clues: "free", an unknown sender, my name, no "urgent", no link. Two of them are spam, five are not. To the neuron they are identical, so whatever its weights, it must get some of them wrong. The information needed to tell them apart is simply not in the clues. The 80% plateau is the best any weights could achieve with these five clues on this data.

Real networks, with millions of neurons, face the same limits on a larger scale:

  • The inputs never hold all the information. Some cases look identical to the network while having different answers.
  • The examples contain errors. The people labelling them disagree or make mistakes, and the network learns those too.
  • The world changes. Spammers adapt their wording; what was learned last year fits this year's emails less well.
  • Rare cases are poorly learned. A situation that appears a handful of times in millions of examples carries little weight.
  • Every threshold is a trade-off. Raise the line from 0.5 to 0.9 and fewer genuine emails are blocked, but more spam gets through. Somebody has to decide which mistake is worse.

Who sets the numbers?

Think of a new employee. Before she arrived, management drew up the org chart and her job description. Over the years, she built up her know-how through experience and feedback. And in this morning's meeting, she makes a decision on the spot, based on the case in front of her.

A neural network holds the same three kinds of numbers:

  • chosen by the engineers before training: how big the tables are, how many layers, how big each adjustment step is. That is the org chart. Specialists call these hyperparameters.
  • learned during training: every weight in every table, starting from random and adjusted, round after round, by the loop above. That is the know-how. When people say a model has 400 billion parameters, these are the numbers they count.
  • computed on the spot: everything the network works out for one particular input, like a score, a share or a verdict. That is this morning's decision, recalculated every time and never kept.

From here on, every diagram in this series carries these small tags. Whenever you see learned, the story is always the same: random at first, then nudged into shape by training.