09See for yourself how it works

An animated walkthrough of everything above, step by step.

Everything described in this article, in one place, with the numbers of our toy model. Press Play, or step through at your own pace, and watch "I speak Esperanto" become tokens, IDs and vectors, then watch "anto" gather context through attention, get corrected by the classic network, and produce a prediction. Every number on screen is one you can find, and check, in chapters 02 and 05.

Animated walkthroughStep by step, I speak Esperanto becomes tokens, IDs and vectors; anto gathers context through attention, is corrected by the classic network, and the model predicts fluently.I speak EsperantoIID 0[0.0, 0.1, 0.9]␣speakID 1[0.3, 0.9, 0.0]␣EsperID 2[0.6, 0.2, 0.0]antoID 3[0.5, 0.1, 0.0]2%22%49%27% itselfcorrection [−0.4, +0.4, 0.0]Probability of coming next␣fluently76%␣speak15%␣English5%␣French2%␣loudly1%␣fluentlyappended