Takeaways
- Transformers replaced recurrent networks because they handle word order through attention and can be trained in parallel, on hardware built to do thousands of calculations at once.
- A neuron multiplies its inputs by weights, adds them up and applies a bend. A layer is a table of weights, a matrix; a network is a stack of them. Everything comes down to multiplying and adding.
- Nobody sets the weights: training finds them, by running examples through, measuring the error, and adjusting every weight a little in the direction that reduces it (gradient descent). The step size is a choice: too small is slow, too large overshoots.
- Training stops when examples kept aside stop improving, not when the network is perfect. It never is.
- Reliability is measured, not proven. That, not poor engineering, is why AI answers need checking wherever an error would matter.
- A model holds three kinds of numbers: chosen by engineers, learned by training, and computed on the spot for each input.
Continue with part 2: What Claude, Gemini and ChatGPT Actually Are