A teaching page · measured 2026-11-06
The Parrot Test
A parrot copies what it hears. People say AI language models, like the one behind ChatGPT, do the same. Here is a test you can do in your head.
"When Kate and Mark went to the market, Mark gave a hammer to ___"
You said Kate. Swap the names and you'd say Mark. A machine that only replays the last few words can't get both right, because those words, gave a hammer to, are the same both times. A small AI model from 2019, GPT-2, got both right almost every time.
The short answer. These models do more than store sentences and play them back. But they can recite from memory, and they say false things as smoothly as true ones. And nothing here shows they understand anything.
Two ways through. Five minutes: the name test, where the critics are right, and what you can fairly say. Twenty minutes: read on, and see the machine itself.
What would a parrot be?
People mean two different things by "parrot". Most arguments slide between them, so keep them apart.
Claim 1: it only copiesIt stores sentences it has seen, and plays them back or stitches them together. We can test this, and this page does.
Claim 2: it doesn't know what words meanIt might say "the sky is blue" with no idea what a sky is. This is closer to what a famous 2021 paper meant by "parrot". Nothing on this page settles it.
The tests of how a language model works all use GPT-2, an early relative of the models behind ChatGPT, made by the same company in 2019. It's small and weak, and it can't even add. I used it because it's open and runs on a laptop, so anyone can check. What it shows may not hold for newer models.
More about the paper and the method
The authors are Bender, Gebru, McMillan-Major and Mitchell. "Stochastic" means roughly "by chance". They meant a system that strings words together by probability "without any reference to meaning". They were also writing about the risks of ever larger models, including cost and bias. GPT-2 small has 124 million weights. Each test here comes with a simpler rival that could embarrass it, and some did.
The first test
Swap two names and the answer flips
The point: the model uses the whole sentence to pick its answer. Replaying short phrases can't do that.
To compare, I built a stand-in parrot: a store of every short phrase from a big pile of writing. To guess the next word, it looks back at the last four words. Here is what it sees:
When Kate and Mark went to the market, Mark gave a hammer to ___
When Mark and Kate went to the market, Kate gave a hammer to ___
The lit words are all it can use. They're identical both times and contain no name, so it must give the same answer twice. One of the two has to be wrong.
Now the real model. Each pair of bars shows how likely it thinks each name is to come next. Look at the top pair, then the bottom pair: when the names swap, the winner swaps. (They don't add up to 100% because any other word could come next too.)
GPT-2's probability for each name as the next word. measured
How often the right name beat the wrong one
600 sentences. measured
What it rules outReplaying short phrases. A copier with a much longer memory isn't tested here. Nobody wrote this rule into GPT-2. It came out of training on text.
What it doesn't proveThat it's reasoning. A one-line rule, "answer the name that appears once", scores 100%. It may have picked up something that simple.
More detail: how fair is this test?
There are 300 swapped pairs using 24 names. GPT-2 got both versions right in 98.3% of pairs, and the phrase store in 0%. The store holds 5.3 million words, which aren't GPT-2's training data, and its scores are guaranteed by the picture above, so this is a demonstration of what replaying short phrases can't do rather than a fair contest. GPT-2's training data isn't public, so I can't prove it never saw these sentences. I can only say the combinations make that unlikely.
It handles made-up words too
"The blicks near the cabinet ___" should take are, even though blicks means nothing. GPT-2 chose the right verb 98.0% of the time. Even when the word right before the blank pointed the wrong way, like "cabinet" here, it was right 96.8% of the time. A phrase store has never seen these words, so it just guesses. measured Again, a simple rule would pass too: plural if the word ends in s.
More detail: invented words
Across 864 sentences it scored 98.0%, and 96.8% when the nearest noun pointed the wrong way, where "copy the nearest noun" would always fail. The phrase store scored 50%: it gives one fixed answer and is right half the time. The model sees pieces rather than words: it reads "blicks" as "b + licks", and those pieces are common. So it has met the parts before, just not this phrase.
What the machine is
What a neural network looks like
To see how it passes that test, we have to open it up. You don't need any maths. Ten short steps, using the real GPT-2 reading one sentence: "The teller walked down to the bank". (A teller works at a bank.) Drag any 3D picture to turn it.
1. Text becomes pieces
The model chops text into pieces called tokens.
They come from a fixed list of 50,257. Common words are one piece. Here "teller" happens to be two. This sentence is 8 pieces.
2. Each piece has a list of numbers
A computer can't do sums on the word "bank". It can do sums on numbers. So every piece gets its own list of 768 numbers, like a fingerprint.
Everything the model does later is sums on these lists. Here is the real list for "bank", drawn as coloured squares. Nobody chose these numbers. In training, the model read a huge amount of text and kept guessing the next word, like a student checking the answers. Each guess was scored, and every number in the model, these lists included, got nudged a little toward the right answer. Then they were all fixed for good.
3. A list of numbers is an address
Think of a theatre ticket: row 5, seat 12. Two numbers pick out one seat. A token's list does the same with 768 numbers, and all the possible addresses are called the space.
Add a third number, and the address is a spot in a room. These addresses aren't on a neat grid, and no single number means anything by itself. Nobody can draw 768 directions, so every picture here shows three. Think of the space as a room.
Here's why it matters: similar words get nearby addresses. The picture ends with real words from GPT-2's lists. Monday's closest neighbours are Tuesday, Thursday, Wednesday. Nobody told it that.
More detail: the neighbours and the picture
Closest to doctor: doctors and physician. Closest to river, leaving out its own spellings: lake and creek. The picture is a 3D shadow of those 16 words, and those three directions keep 52% of how they differ. measured
4. Dots and lines: a tiny network
A neural network is a machine that turns some numbers into other numbers, using dots and lines.
This toy uses made-up numbers. Follow the top dot: 1 × 1.5 is 1.5, and 1 × −0.5 is −0.5. Add them and you get 1. Then 1 × 2 gives the answer, 2.
One more rule: each dot is a gate. If its total is zero or less, the gate shuts and passes nothing on. That's why the bottom dot shows 0.
Move the slider to −1. Now the top gate shuts and the bottom one opens, and the answer is 3, higher than the 2 you started with. So the answer can rise when you push the first number up, and also when you push it far enough down. (Drag slowly: it dips first.) Without gates it could only go one way. That's what gates buy: the network can treat different inputs differently.
5. The same trick, much bigger
This is what a real one looks like. Same dots and lines, just far more of them. Don't try to follow the lines.
Watch the moving dots. A word's list goes in on the left, where the colours are the real list for "bank". The lines multiply, the dots in the middle add up, and a new list comes out on the right. The moving dots show the direction of flow, not real values.
More detail: the real sizes
There are 768 numbers going in, 3,072 dots in the middle, and 768 numbers coming out. It has 4.7 million lines, and I drew 500. The wide middle gives it more room to work, and the real gates are softer than the toy's: they close gradually.
6. Twelve floors: follow one word up
Each word climbs a tower of twelve floors, called layers. On each floor its numbers change a little.
Each column is one word, and the coloured one is "bank". It starts at the bottom with its stored list. On each floor it does two things, then moves up: (1) it looks back at the earlier words, which is the next step, and (2) it goes through a network like the one in step 5. Every word does the same, and every word on a floor shares that floor's network.
At the very top, the model uses the numbers of the last word, "bank", to guess the word that comes after it. The other columns matter because "bank" has been looking back at them on the way up. For this sentence it guesses .
7. Attention: looking back at earlier words
On each floor, a word takes some numbers from the words before it. This is called attention.
That's how "bank" finds out what came before it. Watch numbers flow in along the arcs, and the bar below fill with the mix. The arcs show the real shares for "bank" on floor 1, averaged across the 12 readers GPT-2 runs side by side. Thicker means it takes more. Some of its attention goes to itself. What it gathers is added to what it already holds. It only ever looks back, like reading left to right. The shares are worked out from the words themselves, so they change from sentence to sentence.
More detail: the other floors, and what these shares are
They're averaged over the 12 readers (called heads) GPT-2 runs side by side on each floor. On floor 1, the two pieces of "teller" get 6% and 3%: small, but it's one way that word reaches "bank". Higher floors mostly look back at "The". This page doesn't test why.
8. Roads and traffic
The weights are the roads. The numbers are the traffic.
Watch the two counters. The sentence climbs the tower as traffic, then it vanishes and the next sentence starts. The roads never change. The moving picture is a diagram of this idea, with the real counts.
Training built the 124 million weights, and they don't change when you use the model. The numbers in the columns exist only while this sentence is being read, and then they're gone. Your sentence doesn't get saved.
So there's no shelf of sentences in the tower. It can still produce some famous passages word for word, as you'll see. They aren't kept as text on a shelf. They are folded into the weights, a bit like a tune you can hum without being able to point to where it's stored.
9. Change one word and watch it climb
So "bank" ends up with different numbers depending on who walked there.
Swap "teller" for "angler" (someone who fishes), and compare the two sentences. Each square shows how much that word's list changed, as a percentage, and more colour means more change. "bank" starts out identical, because its stored list is the same either way. By the top, its list has shifted by about 13%: small, but real. "The" never changes, because it comes first and has nothing to look back at.
More detail: what the percentage means
The size of the change as a share of the list's size (treating the list as an arrow and measuring its length), so the swapped word itself can pass 100%: 0% at the start, 8% after one layer, 13% after twelve. No table in the model holds both sentences. Attention carried the change across.
10. The tower is the machine. The room is the space.
Every floor moves the word around in the same room. It doesn't go to a new room.
Think of a delivery route. The route sheet lists the stops in order, and the city is where the van goes. Stop 5 isn't a different city. The tower (left) is the route sheet. The room (right) is the city. The dot is "bank", and each floor nudges it somewhere new.
Why care? Because it means one word has one address that moves. So we can follow a word floor by floor and watch it change. That's how the rest of this page works.
More detail: how far each layer moves a word
The jump into layer 1 is large. From there on, each layer changes a word's direction only slightly (average cosine similarity 0.95, where 1 means the same direction), while its distance from the centre keeps growing. measured What a given direction means may differ from floor to floor, and other network designs don't all work like this.
What you just saw
- Token
- One piece of text.
- List
- The 768 numbers a token has at any moment.
- Space
- Every possible list, as an address. The room.
- Weight
- A number fixed by training. The roads.
- Layer
- One floor of the tower. There are 12.
A picture that looks like proof
Similar things sit close together
The point: this map looks like the machine grasps meaning, but a plain word counter does even better. A good picture isn't evidence.
This is a real map of 3,190 pages of one writer's research notes. The topics don't matter here. A separate program (not GPT-2) turned each page into a list of numbers, an address just like step 3. Nobody told it the topics, yet pages on one topic land near each other. Drag it to see the clumps.
Here's the test. The writer had linked related pages by hand, so we know which pages belong together. Take one linked pair and one random pair. How often does the program put the linked pair closer?
91%the trained program behind this map
97%a program that only counts shared words
50% would be a coin flip. measured
Nobody thinks a word counter understands anything, so clumps like these can't prove understanding. Keep the idea of a space: topics can become positions you can measure. Just don't treat the picture as proof.
More detail: the numbers
The test used 13,113 linked pairs. With all links deleted from the text first, the word counter still scored 94%. My guess is that linked pages share names, though I didn't test that. assessed
From stored list to word in use
One word, many meanings
The point: every "bank" starts from one stored list, but in a sentence it gets different numbers depending on the meaning.
I wrote 40 sentences in pairs that differ by one word. Each line below is one sentence, followed from the bottom of the tower to the top. Left means the numbers look more like the first meaning, and right more like the second.
Tap or hover a line to read its sentence. Left and right are a direction learned from the other sentences, never the one being plotted. Each floor is rescaled to fit, so read the side, not the distance. measured
Keep this one idea: the same word gets different numbers in different sentences. Don't read more into it. Most of the split happens on the first floor, and a trick with no tower at all does just as well: average the stored lists of all the words in the sentence. River sentences contain river-like words, and those probably pull the average one way.
More detail: the chart, the other views, and how this was tested
In 768 dimensions, an axis fitted to these same 40 points would separate them whatever they meant, which is why the direction above is learned from other sentences.
The same sentences in 3D
Tap a point to read its sentence.
- The whole trip is the plain geography: the 3 directions out of 768 along which these paths vary most (83% to 86% of the variation). Layer 1 throws every use far from the stored list, then they drift out together, so the meanings hardly show.
- Ride along moves the camera with the group, keeping 34% to 41% of what's left. The meanings separate clearest for cell and pitch and stay faint for bank. I don't know why bank is the hard one.
Can the meaning be read off the numbers?
A small program learned from some sentences which numbers go with which meaning, then guessed on sentences it had never seen. Shuffled labels are a check that the method isn't fooling itself. measured
After one layer the meaning can be read about 9 times in 10 (90% for bank, 95% for cell, 88% for pitch). Averaging stored lists does as well (90%, 95%, 93%), and counting words doesn't (48%, 50%, 55%), because counting needs the very same word to have turned up before. "Prisoner" sits closer to "inmate" (0.69) than to "biologist" (0.34), on a scale where 1 means the same direction.
The same test on real text. The company name Palantir appears in the notes on domestic systems and on empire abroad. Across 192 uses on 64 pages, holding out whole pages, the final layer got 73% right. Always guessing the commoner thread gets 62%, the best of 36 word-counter settings gets 65%, and averaging gets 69%. Shuffling labels page by page reached 64% at its 95th percentile (p ≈ 0.01). Treat this as suggestive, confidence about 0.5: the lead over averaging is small enough to be noise, I chose the final layer after seeing every layer (the one before scored slightly higher), and the label is only the folder a page was filed in. measured
The strongest evidence
Follow the answer through the machine
The point: inside the model, the answer depends first on one word and then on another, and you can catch it moving.
Imagine the words as people in a row, each holding a note (its list of numbers). On every floor, each person can look back at anyone earlier and rewrite their own note. The answer comes out at the end. Here is how to find out which note it depends on.
- Run "When Kate and Mark went to the market, Mark gave a hammer to". The answer is Kate.
- Run its twin, with one word changed: "When Kate and Mark went to the market, Kate gave a hammer to". The answer is Mark.
- Pick one person, on one floor, in the Kate run. Swap their note for the matching note from the Mark run.
- Let it finish. If the answer flips to Mark, it depended on that note. If nothing changes, either it didn't, or another note was holding the same thing.
Do this for every word on every floor, and you get a map of where the answer depends on the difference between the two sentences.
Where the answer lives
Dark means swapping a note here flips the answer. Read from floor 1 down. Up to floor 7 or so it depends on the second "Mark". Over floors 8 and 9 it moves to the last word, "to". Blank doesn't mean a word is unused: words before the second "Mark" hold the same note in both runs, so swapping changes nothing.
Hover or tap a square to try that swap above.
Average of 60 sentence pairs. measured
Why this matters. A store of phrases has nothing like this inside. This model does, and those notes do a real job. Change one and the answer changes. The trail moves across floors 8 and 9, in a part of the model you can point to. A more detailed study of this kind of sentence points to the same floors. third-party
What you can take from it: there is a real inside, and it works in stages. What you can't: that this is thinking, or that bigger models work the same way. It's one task on one small model.
More detail: the numbers, and the checks
Part of the picture is guaranteed and proves nothing: the blanks on the left (the sentences don't differ yet), the top of the repeated-name column, and the bottom-right corner. The finding is the route between them, because nothing forced a clean hop. The information stays on the repeated name through layer 7 (0.90), then crosses to the sentence's final word over layers 8 and 9 (0.57 and 0.29 at layer 8; 0.02 and 0.87 at layer 9). Copying that same final word's state at layer 7 does almost nothing (0.01). These are shares of the answer moved, so they don't have to add up to 1.
In 2022, Wang and colleagues found attention heads reading from the repeated name at the final word (heads 7.3, 7.9, 8.6, 8.10). Researchers count layers from 0, so those are layers 8 and 9 here. Their method is finer and their contrast differs from mine, so this is agreement about where, not a replication. They also say their account has gaps.
Two checks are built into the run, and it stops if either fails: copying the last state must switch the answer fully, and copying a run into itself must change nothing (it read 0.000).
Others found similar things in other models. Llama-2 holds internal numbers that track where places are and when events happened (Gurnee and Tegmark, 2023). Claude 3.5 Haiku appears to pick a rhyme word before writing the line that ends on it (Anthropic, 2025), which comes from the model's own maker and uses a simplified stand-in model. third-party
The neatest example from another lab: a small model was fed only lists of moves from the board game Othello, with no rules and no picture of a board. Researchers looked inside and found it had built its own picture of the board. When they edited that picture, its next move changed (Li et al., 2022). third-party That was a different small model in a simple game. It doesn't show that GPT-2 or a chatbot grasps the world.
Where the critics are right
It really does parrot, sometimes
Give the same model famous openings and plain questions. Below is GPT-2's actual output from one run, always taking its top guess, unedited. measured
- It can recite from memory. Researchers got hundreds of passages word for word out of the largest version of GPT-2 (Carlini et al., 2021). third-party It isn't reliable either: above, the Gettysburg Address falls apart.
- Smooth isn't true. "Sydney" comes out in the same confident voice a right answer would. Newer models get questions like these right far more often and still state false things smoothly, though nothing here measures that. assessed
- Working parts don't mean being right. A model can have the inside you just saw and still fail at 47 + 38. Both are true.
What you can fairly say
You can say: it's not just a parrot. It does more than store sentences and play them back, and you can follow an answer as it moves through it.
You can also say: it does recite, and it says false things smoothly. "Parrot" gets those right.
You can't say: it understands. If you leave this page thinking "so it understands", the page has failed.
Why it stops there. Everything above could still be patterns in how words go together. Nobody has an agreed test for understanding. One common view is that it needs words tied to the world: this model has read "hammer" countless times and never held one. No test here touches that. This is close to what the 2021 paper was saying, and it's a fair reading.
How far to trust this. The tests of how the model works used one model from 2019, using sentences I wrote by hand, and the page itself was written by Claude, an AI model, which has an obvious stake in the question. Weigh it accordingly. You can try the name test on any chatbot yourself.
My position, at more length
"Parrot" is the wrong picture of the mechanism, and this evidence doesn't reach the question of understanding, intent, belief or consciousness. "Statistical" and "stored" are different things. These tests separate them and go no further. A rule learned from text is a pattern. A list that carries a name is a mechanism, not a meaning.