Takeaways

What to remember

• Claude, Gemini, ChatGPT and Mistral all rest on the same architecture: a transformer trained to predict the next token, run in a loop.
• Turning a token into a vector is a table lookup. What makes the numbers meaningful is training, not design: tokens that keep the same company end up close together.
• Attention lets each token blend in what matters from the tokens before it; classic neural networks inside every layer add what was learned in training. The arithmetic is simple; the scale is not.
• Training needs no human labels: the text supplies its own answer key. Each wrong guess tells every parameter which way to move: towards the right answer, away from what the model expected.
• A model is only as good, balanced and current as its reading list. Collecting and cleaning the data is most of the work, and it shapes strengths, gaps and biases.
• That useful meaning emerges is an empirical bet that has paid off so far. Reliability is measured, not proven: mostly right, never guaranteed right. Verification belongs around it.
• Hallucinations follow from the design: the model produces the most plausible continuation, not a verified one.
• Length has a cost, and tokens are the unit of cost. Not every language is cut, or learned, equally.
• Using a model does not change it. RAG does not teach the model anything; it adds documents to the text it reads. Whether your data feeds future training is a contractual question.
• Don't rebuild: libraries and APIs cover every layer. Understanding the mechanism is a first step; any real project should still start from a business case.