What Is ChatGPT Doing ... and Why Does It Work?
I put this at #32 because Wolfram explains language models from the machinery upward: prediction, tokens, neural networks, embeddings, and why coherent language can emerge from choosing the next piece. It is compact enough to reopen whenever the magic starts replacing the mechanism.
The long version
Stephen Wolfram explains large language models by starting with next-token prediction and building upward through neural networks, learned representations, probability, and the structure of human language. He shows how a system trained to continue text can produce coherent passages without consulting a stored book of answers. The account is compact and sometimes simplified, but it gives the reader enough machinery to replace mystery with better questions about what the model computes, what training supplies, and why plausible language is not the same as truth.
Why it is here
Read it before the longer engineering manuals when the mechanism still feels magical. It gives the AI shelf a small conceptual hinge between language and computation. I keep it because I can reopen it quickly whenever fluent output tempts me to invent a mind behind the sentence.
How to read it
Follow one sentence token by token and make a few possible continuations of your own. Do not get trapped in every diagram; keep asking what input enters, what is learned, and what output is selected. Then test the explanation against a current model’s confident mistake.
What it taught me
- 01
A language model repeatedly assigns probabilities to possible next tokens from the context it has received.
- 02
Training adjusts many numerical parameters so that useful patterns and representations emerge across examples.
- 03
Syntactic fluency and factual reliability are different achievements and must be evaluated separately.
Before and after
What makes it easier
More about the author
Stephen Wolfram is a British-American computer scientist, mathematician, and entrepreneur known for work in computational science and for creating Mathematica and Wolfram Alpha. His writing often explains complex systems through computation and simple generative rules.
More by the same hand
A New Kind of Science
Stephen Wolfram
Idea Makers
Stephen Wolfram
Adventures of a Computational Explorer
Stephen Wolfram