No-Hype AI Part 3B: Picking the Words to Number

I split this post into two parts. If you missed the first part, go read that first. If none of this rings a bell, check out the intro post to find out what this post series is, and what I mean by “no hype”.

It’s a Mad, Mad, Mad Libs World

Remember the foreshadowing in Part 2, where I said something like “it’s not like data labels itself”? After the first part of this post, you probably guessed how we’re going to get around that, but let’s make it explicit: we’re going to take bits of text, automatically pluck a word out, and train a neural network to try to guess that word given the context.

What is that context?

When we human beings learn our first words, the context for those words is all the stuff around us: faces, pets, colors, toys, sounds, sensations. When learning to read, at first we’re usually trying to map those funny drawings to words we already know and speak. Later, though, we increasingly learn new words according to other words on the page. (Never make fun of someone for mispronouncing a word, because they learned it by reading!)

Here, “context” can only be other words. Words tend to travel in packs, conveniently, but neural networks are rigid things. We couldn’t just throw different-size images at our handwriting recognition network; we had to know ahead of time exactly how many pixels there would be. By the same token, as unnatural as it’s going to feel, we’re going to at least temporarily have to come up with a rule by which a computer can reliably pick out “the context” for any given training word.

The simplest context for a word is the word before it. Which cat? The tabby cat. The black cat. That cat. A single word of context is easy, but runs into trouble quickly: What comes after “hoist”? The word “petard” is strongly associated with “hoist” but there are always words in between. Even with less-set phrases, there are often words in between. And anyway, that might teach me about “petard” but never about “hoist”. Picking the word after has similar problems: Quick, what comes after “the”? (Or more subtly, what never comes after “the”?)

Still, the immediately-surrounding words do make sense. Let’s take a page from Mad Libs, a game many of us played as kids: We’d have a book full of short stories with words missing. For example, “The brown fox (adverb) jumped over the lazy (noun).” As kids, our role was to figure out how to turn “fart” into an adverb, then hold in the giggles until a sighing adult would read them all back to us. We’re going to make our neural network do that.

This broadly shadows a 2013 technique developed at Google called Word2Vec1. The essence of Word2Vec is to train a simple neural network to learn word vectors using variations on that Mad Libs approach. First. there’s a sliding snippet of text of some length, maybe five words, and the middle word of that snippet is your blank.

You take all the words around the blank and try to guess the blank based on all of them together, an approach called “Bag of Words”. Say I have a sentence, “Both of my cats are criminals.” I’ll blank out one word (“cats”) and just some number of words on either side into my bag. If the number is 1, I get “my” and “are” for my context. If that number is 2, then I add: “of” and “criminals”. (If I run off the beginning or end, we can go to the previous or next sentences, or leave it empty.)

I want to train my neural network so that, given that bag of words, however big it is, the word “cat” is more likely to show up in the output. The words in the bag will give me clues: the “are” hints at plurality, along with “criminals” instead of “criminal”. The adjective “my” suggests a noun’s in the mix. I’m discarding the order here, and trying to find a sweet spot where the bag has enough words to clue me in as to what could go there, but not so many as to lead me astray. Lots of words could be the fifth word in the middle, though; this could have been from something like, “All criminals are fond of my novel.” Not even the same part of speech.

That’s the Bag of Words approach: Mad Libs-style guess the blank. (Presumably with less scatology.) Losing the sentence order plainly loses you something, but there turns out to be some good reasons to do that when it comes time to build the neural network.

Or, if we’re going to lose the order anyway, you can take the word in the blank and try to guess what words might be around it (an approach called Skip-gram2). That’s a little harder to wrap your head around, a little more like graded free association, but that feels a little easier. Going from “cats” to “my” and “criminals” seems a bit more natural, right?

Don’t be fooled.

For the neural network itself: We don’t have our embedding yet, so we need to dedicate inputs and output neurons to individual words, pixel-style. So, that one-hot encoding comes back for now. We’re just going to number every damn word we can think of. Input 1 is “the”. Input 2 is “and”, and so forth. Output 1, mirroring that, covers “the”ishness, etc. Literally just listing words.

If I have ten thousand words, then I have ten thousand inputs and ten thousand output neurons. Yes, only ten thousand inputs: If we cast aside sequence order, we can just treat those inputs as checkmarks and light up the inputs for all four words at once. We’re essentially playing Mad Libs with those food lists, and we can check off as many as we like for context3.

In between, I have a single hidden layer. That’s it. I just want the one. The weights of that hidden layer become my embedding; everything else gets thrown away. So, the embedding vector for “the” is the list of its input weights for each neuron in the hidden layer. However many neurons I have in the hidden layer, that’s the number of dimensions4.

Training will be a matter of producing input/output pairs out of a large text corpus: “hoist” together with “by” “own” and “petard” for example. I can produce negative examples too: since I have the whole training corpus, I can check that “hoist” never appears near “Picard” or “canard”. The error function for Bag of Words will be whether it got the one blank word right for the list of context words. For Skip-gram it will be comparing how the output list compares.

The point isn’t really to come up with something that predicts the target or context words. It will pretty much always get the output wrong during training, except for very common pairings and by accident. What I want is to just go through the process so that the gradual erosion/deposition effect of neural network gradient descent training builds up those input weights over time so that it’s much more likely to produce those words than it is to produce random other words. Then I steal the inputs for my embeddings.

How do I know if the embeddings are any good? Any embedding worth its salt would let us do analogy tests. I might not have any idea what kind of wacky embedding dimensions this neural network came up with, but the difference between “big” and “bigger” ought to be comparable to “small” and “smaller”.

That is, whatever steps I need to take to get from “big” to “bigger”, if I start from “small” and take those exact same steps, I should end up close to “smaller”. If you remember back to the SATs, you might remember this being written out “big : bigger :: small : ?”

Just for fun, I downloaded the embeddings for GPT-2 and tried three different analogies5:

  • France : Paris :: Japan : ?
  • big : bigger :: small : ?

I found the steps it takes to go from, France to Paris; then I took those steps starting from Japan and looked around to find the closest words. “Tokyo” was the closest, but close on after that are “Japanese” and “Kyoto”.

It’s a mistake to read knowledge into this, and the big : bigger example helps show why. It’s Coke and Pepsi all over again: are they synonyms? Antonyms? Irrelevant! They substitute for each other because they appear in such similar contexts. “Tokyo” is used in relation to “Japan” in a similar way that “Paris” is used in relation to “France” but apparently so is “Japanese”. I can make up stories by which this makes sense, but it might just be an accident of placement: it was Japan-ish enough to be placed in the neighborhood, and happens to be close to where we wound up when we wandered in that direction.

Still, there is something in that high dimensional embedding space that’s something like meaning. Here’s a few more6.

The heart of the “stochastic parrot” critique of LLMs is that true surtext, true meaning as we native language speakers understand it, is simply not available at any stage of the training. Whatever GPT-2 learned, it learned only from context. But it did learn an alien something from that context. That’s not how human beings usually learn language, because we often learn in a much richer contextual environment.

But then there’s words like “vorpal” and “Jubjub” where what we human readers do feels a lot closer to what the machine does for everything. The neural network arranged word labels in space in such a way that it can start from them and find words we consider relevant. Even just at the input stage, before something complicated like an LLM is run, something about these words is encoded in this space.

Embeddings are a core part of neural networks that work with text7. LLM designers think in terms of embeddings when they design their networks. Understanding the concept of embeddings will help us finally understand: what is a neural network actually doing when it selects a word to output?

Neural networks discard the words they work with immediately, substituting them for these embedding vectors that can be more naturally added to, subtracted from, compared, and divided. Any of those things immediately shifts the vector off of a known word and into space between words. Vectors are combined like those mutant restaurants we talked about, and recombined.

Then to output a word, we look at the embedding vector that’s been produced, and cast about for the closest word

So where does that leave us? We have a basic approach here for converting words into locations on some wild high-dimensional map, such that a word’s location tells us something – hopefully a lot – about how that word is used.

Just one last little question, then:

What’s a Word?

“What’s a word?” is a weird question for a writer to ask, and maybe it’s a little late to think of that. But seriously: what constitutes a word?

Words on a page seem like nice, neat little things. A bunch of letters together, split up by gaps or dots or lines.

Is that it, then? Is a word just everything that’s separated by a space or punctuation? In that last sentence, what is the last word: “punctuation”, “punctuation?”, or “?”? Do I need to treat “the” and “The” as separate words, or can I just always output “the” and if it happens to be the first word in a sentence, fix it in post?

For that matter, are spaces so important? Is “free fall” one word or two? Are “free fall” and “freefall” the same word? How do I tell the computer my decision?

I kind of glossed over that whole Jubjub bird thing in the prior post: it’s great that the neural network can work with the vector for a rare or nonsense word, but what do I do if I don’t already have an embedding for a word?

The answer to all of those is a technique called tokenizing. Tokenizing can be a simple process: for example, that groups of letters divided by spaces or punctuation are all words, and we make everything lowercase. So, in the sentence, “My cats are a bunch of criminals.” The tokens become (“my”, “cats”, “are”, “a”, “bunch”, “of”, “criminals”).

The trouble is that we need to enumerate all the words it’s possible to encounter, or else eventually we’ll come across something we’ve never seen and not be able to embed it.

We’ve got some options here. What if we just declared that every letter is a token? Punctuation and spaces too. (“M”, ”y”, “ ”, “c”, “a”,…, “s”, “.”) Until people start using characters with accents, characters from other alphabets or writing systems, or emoji, that kind of works. In fact, Andrej Karpathy did exactly this in his example video building GPT from scratch using the works of Shakespeare as his text source.

Letter-level tokenizing has the virtue of being compact, but it doesn’t work very well when we get to embeddings. There’s just not that much to learn at the individual letter level. There might be some connections that can be drawn (e.g. “q” and “u”) and vowel/consonant relationships, but individual letters are just too low-level in English8. We lose all of the contextual meaning we worked so hard for. I drew pictures and everything!

At the same time, I cannot literally assign a number to every word in the English language: new words are being invented all the time, some words are vanishingly rare, and anyway there are typos all over the place. Plus, we really do need to consider the role that spaces, new lines, and punctuation play in how we interpret what we read.

The individual letters have the great virtue that nothing gets past them: Anything I can throw at a tokenizer can be broken down into letters. The virtue of individual words or even multi-word phrases is that our embeddings will work best with them. We need something in between to try to balance these virtues.

So, we’ll take a fallback approach to our tokenizing: we can have tokens for the most common sequences, like “ the” and “ computer” and “ token” (note the leading space, which is super common), and we can also have tokens for all the letters of the alphabet so that words like “jubjub” can still be taken up into the network even if we won’t have specific embeddings for them. Worst case, we can spell out words like when it’s time to take the dog to the v-e-t.

In fact, we can have multiple fallbacks, because some words appear in other words. “Snozzleberry” might not be common enough to be worth setting aside a whole token for, but “berry” sure is: it’s token number 8396 out of 50,257 in the GPT-2 token list9. Some combinations of letters, like “st”, appear so frequently as to be worth making a token for them (token 301, in fact). So I can now encode “Jubjub” as a series of tokens, with “J” and “j” treated differently:

J (41) – ub (549) – j (73) – ub (549)

Punctuation, too: spaces appear so often that most of the full-word tokens in the GPT-2 token list are preceded by spaces. If I change that to “Jubjub bird” the space comes along with “ bird” so it’s just one more token: 6512.

There are a number of techniques for figuring out your tokens, but what the designers of a tokenizer really want is for any given text to be expressed in as few tokens as reasonably possible. Essentially, the process of building up a token library is compression. Finding out what the most common sequences actually are requires yet another romp through a giant dataset of text, then, for example using a technique called “Byte-pair encoding”. The Byte-pair encoding approach works from the bottom up. You pick how big your vocabulary is going to be10 and then tokenize with single-letter tokens.

Then you rip through and count every pair you come upon. So for “banana” you’d have one “ba”, two “an”s and two “na”s. Then you find the most common pair, and turn that pair into a token. So if “an” turns out, across the whole text, to be the next most common pair, then “banana” might become “bXXa” where X is the new token. You just keep doing that until you’ve hit your target, basically. If you do that purely mechanically, the tokens tend to split in odd ways, including multi-word tokens and tokens with spaces in the middle. Pre-splitting on spaces and punctuation before doing this process lets them put a thumb on the scale and make sure the tokenizer is filled at least with word-like things whose embeddings will be more meaningful.

Tokenizing is absolutely necessary for working with text, but even taking pains to keep the tokens word-like imposes some hideous penalties once we have our tokens and need to come up with embedding vectors: many of our words aren’t words anymore. They become sequences of unintelligible tokens barely a step up from individual letters. We still have those embedding vectors describing that Plato-like latent space of pure meaning where little balls of context frolic in green fields… but only for the most common words in our vocabulary, and some of them have twins. There might be a little nesting space set aside for “Jubjub birds” in our embedding space, but we can’t get there directly from its tokens.

Jubjub birds and… Magikarp?

So why not just have an enormous token space? After all, it’s the size of the embedding space that really causes weight counts in the neural network to blow up. (It’s only a token, Michael, what can it cost, $1011?) Having the embedding for each of those tokens seems like it would be useful—way more useful than st-raw-berry, you know?

The trouble is, you need to see a token a lot in order to actually get an embedding for it. Each occurrence in the training set is just a grain of sand in terms of the erosion and deposition by which backpropagation and gradient descent shape a neural network, and thus the embedding weights. Just because a token is a real word doesn’t mean that it will appear often enough to be embedded. Even “strawberry” (without a space in front of it or a capital letter) might be too rare to properly embed.

To some extent, an embedding still acts like a label. Even a random embedding has some use, in theory, just as a label for a token. But if the token is uncommon in the embedding training, it’s probably going to be uncommon in the dataset for whatever neural network you’re using it for, too, and it’ll wind up acting like those undertrained pixels we saw last time: when it does appear, there’s no telling how it’ll be interpreted. In other words, a rare token isn’t just dead weight, it’s potentially a bomb waiting to go off.

I don’t want to get too into LLM chatbots yet, because we have a lot ahead of us before we get there. But this under-trained token problem isn’t just academic: this has been observed. Researchers studying GPT-212 identified a number of low-probability tokens in its vocabulary—things like ” SolidGoldMagikarp”, ” Dragonbound”, and ” RandomRedditor”—that when incorporated into GPT-2 queries, caused bizarre responses. Asking it to repeat a sequence containing these tokens resulted in responses ranging from “You’re a nerd” to “You can’t see me,” to (my favorite) “N-O-T-H-I-N-G-I-S-F-A-I-R-I-N-T-H-I-S-W-O-R-L-D-O-F-M-A-D-N-E-S-S!” This is believed to have happened because, like we saw with the shifted numerals in the last post, those paths through the neural network just weren’t trained much. And again: when neural networks fail, they fail weird.

Since we’re talking weird failures with LLMs that involve tokens, we’re also well-equipped now to think about that time cartoonist Rob DenBleyker asked ChatGPT a simple question:

What’s going on there? We think of language differently than neural networks do, because we interact with text in starkly different ways. We fluent readers of a language have the remarkable ability to see the forest for the trees, and to switch back and forth at will. We engage with text at a very chunk-y level, but can relatively seamlessly step back from words to the letters that make them up. Not so a system that relies on tokenizing once and using that to select a different representation that it will actually work with.

The GPT-4o tokens are a larger space than GPT-3: 200,000 instead of 50,000. We can download them and look at that word “strawberry.” Actually “that word” is already a mistake: We may think of “strawberry” as one word, but if it’s not preceded by a space it gets encoded as:

st (token 302) – raw (1618) – berry (19772)

With a leading space, it depends whether we’re talking about a “ strawberry” (101830) or a “ Strawberry” (138322).

Neural networks like LLMs, working with the mathematical embeddings of individual tokens, are unable to inherently reshape and recontextualize their own inputs13. From the neural network’s perspective, filtered through the tokenizer, there are no “r”s in “strawberry”. “strawberry” is just a 302, a 1618, and a 19772 – no “r” (81) or “R” (49) in sight. The neural network had no access to the original text, and no ability to retokenize as letters. That’s just at the tokenizing level: The embedding space likely doesn’t have the embedding for token 81 anywhere near the tokens for any of the different alternatives for “strawberry”, though “r” and “R” are probably very close.

It’s the kind of question that a neural network was not equipped to answer, then, because neural networks fundamentally engage with written text in a very different way than human beings do. Even asking the question to a person who can’t read would be more likely to get a reasoned correct answer based on the ability to engage with the question text by sounding the word out.

No shade for asking the question, though. None of these failings were made clear to the user. It’s reasonable to wonder why the chatbot attempted to answer in the first place, how (or whether) it got the answer right on the second try, and why its own explanation of its error is not only not right, but as Wolfgang Pauli put it: It’s not even wrong.

Tokenizing and embedding go a long way toward getting a neural network to work with written text in the first place, but if we’re going to get a sense of what’s going on in that chat log, we’ll have to dig into what’s going on in an LLM: what happens to those embeddings between the input and the output, and how it selects what to generate. Something in there was able to stitch together the tokens in st-raw-berry and equate them to the single token ” strawberry”. To understand that, we’ll need to talk about the token “T” in ChatGPT: the Transformer, and the attention mechanism it relies on. Conveniently, that’ll be the topic of the next post.

What does this look like in fiction?

Science fiction has many staples related to machines receiving text as input. Universal translators, heck, any translators; computers you write or speak to in natural language instead of programming. A lot of stories use these technologies as a way to make language matter less. (I don’t want the intricacies of alien communication, I want space kablooie!) But a lot of us writers have in common a fascination with language, and now we have some more tools for writing about how language intersects with technology.

Fantasy writers use language in fascinating ways. What is a magic system if not a way by which the world understands and responds to human language? Magic runes, inscribed circles, and other mystical writing are staples of fantasy stories. Even if you’re not thinking about a story in which an automated system reads those things, hopefully you have some new ways to think about old standbys. What if that high-dimensional embedding isn’t just a mathematical convenience, but a kind of true Platonic representation of a concept, and manipulating it has real effect? If you had a character trying to learn an old and forgotten system, might they try a brute force approach like those spreadsheets of food?

We made a lot of assumptions about the role of context in written or transcribed language in order to convert text labels into measurements in a latent space. The approach described here elevates context above subtext and even surtext. I’m still occasionally stunned that it works at all, let alone as well as it does, with the ability to draw reasonable analogies mechanically.

Surely, though, there are languages for which those assumptions fail in important ways? These embedding techniques were developed around the notion of sequences; even where the exact sequence is discarded, sequence still defines context. There exist methods designed for non-linear structure, like Graph Neural Networks, but the same question applies: what constitutes context, and how much context is needed?

Even if a fictional translator doesn’t use a neural network or doesn’t specify a technology, we know something will need to convert written symbols into a format computers can use: tokens, or something very much like them. Understanding those tokens means understanding context. We can do a LOT with context.

Consider a language constructed in spider webs, where context might be literal connections of silk. Perhaps the silk has different chemical properties, to catch the light in different ways at different times of day so that the same web contains a message that deliberately evolves over time. What might constitute a “token” here? What constitutes context? Where does it fall down, especially in self-referential cases like spelling?

Or consider a language that incorporates secret shared knowledge. Why must a language be readily understandable to anyone listening in? One might imagine a creature whose communications involve some kind of analogue to public key cryptography: it might flash messages for the world to see across its skin like an octopus, but only those who are privy to some shared secret (a private color pattern hidden by a veil? a combination of pheromones only detectable up close?) can understand the message or its full subtext. Parts of the language might be broadly understood, and parts secret—somewhat like references to history, literature, and shared experience, but even more so. What, then, is a token and context, if your characters seek to make a machine learn this language? How do they create a dataset?

For that matter, how well do these assumptions hold up for human languages, even English? I keep using “words” or “language” as a shorthand for what’s being done here, but what I’m describing as the basis for text embeddings is very much a subset of English-language communication. My training data might feel comprehensive, but it’s not. It can’t be. For example, transcribed speech: it seems like that would work fine here, but verbal speech has very different context from written speech. Human beings soak in context to such extent that don’t notice most of it.

I attended the AI@70 conference a few weeks back, commemorating the 1956 conference at which the term “artificial intelligence” was introduced. Consider these experiences I had:

  • A conversation with a former professor about people we both knew.
  • A panel I attended in person, where the panelists included my PhD advisor and a former classmate, on a topic I knew well and where both people alluded to research I was familiar with.
  • Another panel I attended on a topic I knew little about, with panelists I’d never heard of.
  • A talk I watched live on my laptop.
  • A set of short talks I listened to on my phone while doing dishes.
  • Two auto-generated transcripts (one for a talk I attended, another for a talk I didn’t).

Listed like that it’s a bit easier to think through just how much context was available and is lost at each step away from the in-person conversation. Visual context like slides and hand gestures. Facial expressions. Tone of voice. Little things you pick up from knowing someone for decades. Awareness of where you are and who is in the audience with you. Even where you’re sitting, the volume level, or the video screen size all matter for what you pick up and don’t.

Don’t get me wrong: written text is still a very rich source of context. When I read the transcripts, I understood them. I can read transcripts of Richard Feynman’s lectures, now sixty years later13 and get a lot out of them. Communication intended to be read has lots of tricks to add that context back in.

There’s a “drunk looking for keys14” aspect to all this that’s worth exploring. One reason this post is so late is that I tore up my plans for it after watching a talk by Molly Crockett, a psychology professor15. I was struck by the extent to which, baked into the field of AI going back at least 70 years, there’s a tendency to proceed as though what we can measure is all there is16. “This is what we can do,” goes the implicit (if unintentional) logic, “therefore it’s what matters; it’s what’s worth doing.”

Alan Turing proposed tests where subjects interacted with a machine through a proxy to have conversation or play chess. That setup did not stem from a belief that it captured intelligence entirely; it was inspired more by the need to shied the test subjects from the wealth of context that a human conversationalist or chess player would add. There is information in the delay of a chess move, in the confidence with which a piece is placed on the board, that would instantly give away a human player. Playing or chatting through a proxy evened the playing field by making it harder for humans to recognize each other. Today, talking to each other through little glowing rectangles full of enraging snippets of text, that format still serves to make it hard for humans to recognize each other.

Many of the decisions that go into what to try to capture in a neural network are the natural consequences of following a line of academic inquiry to see how far it goes; we can’t do everything, so we make choices according to what’s interesting, obvious, or easy. Sometimes the challenge is just figuring out how to get the machine to interact with the person at all.

When academic inquiry becomes engineering, though, when it becomes a product, those choices become a blind spot. Yes, it turns out that you can learn a lot—a stunning amount—about words purely from the company they keep. There is genuine meaning here, even just at the embedding level before we add the structures that make up a large language model.

But that’s not all there is to learn. We must therefore be careful how we think and talk about what we have learned. We haven’t learned about “words”; we learned about “tokens as they tend to appear in a specific (if humongous) collection of written communication.” There’s much worth learning in the space of things we can measure, but it’s not everything.

There is tension between what there is to know, and what we value knowing. That’s true of all science, but it’s especially true of something as close to human identity as language. If I’ve rambled at length here it’s because I think this is an important place for fiction writers to stake out some ground and make themselves heard. Writers know language, but also the inadequacies of language.

The two posts in this sequence covered both very little ground and a lot. We broke text down into tokens. We embedded those tokens like constellations in a high-dimensional latent space sky. We captured recognizable meaning in an incomprehensible way, akin to those neuron images in the last post. We have an alien map of familiar territory.

Just remember: The map is not the territory. There are no Rs in “strawberry”.


  1. “Word to vector”. I’ve used the term “vector” a few times without defining it: a vector is just an ordered list of numbers that can be treated as coordinates. If I write down the latitude, longitude, and altitude of a location, that’s a vector. Order matters: I need to avoid adding latitude to altitude or the results will be nonsensical. ↩︎
  2. If you want to dig into some code and math a bit, try this: https://www.geeksforgeeks.org/python/implement-your-own-word2vecskip-gram-model-in-python/ ↩︎
  3. Actually these are going to be counts, so that if the same word appears multiple times we account for that. Typically we then divide by the total to “normalize” the input and make the whole thing add up to 1. ↩︎
  4. Mathematically, this is a sleight-of-hand trick: the inputs and their weights can essentially be swapped. It’s easiest to see this with Skip-gram, where one input is 1 and the rest 0. ↩︎
  5. If you try this at home, omit leading spaces; you’ll see why in the next section. ↩︎
  6. “Yosh” should make a little more sense in the next section. ↩︎
  7. Text embeddings aren’t restricted to neural networks. Notably, they underlie search technology, and much of this approach was developed by Google with that in mind. When a search engine indexes a web page, it embeds the page’s text in a high-dimensional vector. These embedding vectors are a convenient and compact way to store a kind of summary. When you search for text like “how do I stop an arterial bleed?” the embedding vectors for the individual works are combined into a single search vector that encapsulates the general meaning. That vector can be matched against vectors produced from whole web pages, and it’s fast to determine “hey, these vectors are pointing in basically the same direction” so that pages with the closest-match vectors can be returned as the highest-ranking results. ↩︎
  8. An information-dense writing system like Chinese, on the other hand, where individual symbols are more strongly tied to concepts, probably works much better. ↩︎
  9. The first token is “!”. I don’t know what to read into that. ↩︎
  10. GPT-2 used 50k tokens; GPT-4 used 200k. As far as I can tell, 200k seems to be the modern sweet spot; the details are more locked-up than they used to be, but I’m not aware of any model using much more than 250k, and most are half that. ↩︎
  11. This joke will never not be funny to me. Might as well make a whole token out of “what can it cost, $10” right now. ↩︎
  12. They published in a series of posts starting here; including a painstaking investigation of what many of these so-called “gitch tokens” originally referred to, giving insight into the sources from which GPT-2 was trained. ↩︎
  13. This has not inherently changed, even if more recent LLMs appear to be able to handle this question with ease, but understanding how that’s done requires a bit more explanation, in a later post. ↩︎
  14. There’s another loss of context! ↩︎
  15. Goes the joke: Someone helps a drunk look for his keys under a lamppost until finally asking, “Are you sure you dropped them here?” “No, but this is where the light is.” ↩︎
  16. Online here: https://www.youtube.com/live/xm916h_6XNY and Crockett’s website is here: https://psych.princeton.edu/people/molly-crockett ↩︎
  17. Phrasing is deliberate there; as much as I want to shoehorn in a discussion of Daniel Kahneman’s “What you see is all there is” fallacy, this post is already super long. ↩︎

One thought on “No-Hype AI Part 3B: Picking the Words to Number”

  1. The explanation of context through missing-word prediction makes the abstraction unusually concrete. It also helps show both the strength and the limitation of the method: statistical context can capture rich relationships between words, while still being different from the sensory and social context through which people first learn meaning.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.