Andrew Stellman 分析 AI 为何写出 Yoda 式倒装句

I use AI for writing all the time. But not because it’s good at writing.

In fact, just the opposite is true. AI is so reliably bad at writing that I’ve actually taken advantage of that, and made its bad writing an integral part of my writing process.

I’ve written a lot of books and articles over the years, and while I used to start with an outline, I don’t anymore. These days I do a lot of brain dumping and brainstorming and talking out loud to an AI, working out what I think about something. I accumulate a lot of notes and ideas, and run a lot of experiments, and at some point, if I’m lucky, I decide there might be an article somewhere in all of that.

Then I ask the AI to take everything I’ve given it and write a draft.

And that AI-generated draft is universally terrible. Like, some of the worst writing you can imagine.

Which is exactly what I want.

Instead of an outline, I get an entire “article” or “chapter” (or, at least, a weird facsimile of one), with paragraphs that express complete thoughts that I gave the AI. Granted, those paragraphs are full of obvious AI tics, like how it uses em dashes to squish two unrelated sentences together, or weird staccato sentences one after another, or these insufferable parades of sentence fragments. Maybe the worst tic is that half of the paragraphs AI generates have an extra sentence that doesn’t actually say anything, but just delivers a forced conclusion with fake “punchiness.” I guess maybe AIs were trained on a lot of bad writing? Well, whatever, it works for me.

The reason I love that type of obviously bad AI writing is because when I’m looking at one of its paragraphs, it’s immediately obvious that an AI wrote it. I’ve been thinking of it as a kind of an improved “lorem ipsum” that lets me see how my article will look, while still serving as a functional placeholder. That lets me use it the way I would use an outline: I can move parts of the article around, make sure it flows correctly, and then replace ALL of the bad AI writing with (hopefully) decent human writing, just like I used to do with bullets in my old outlines.

One weird side-effect of writing this way is that it’s made me very sensitive to the different kinds of AI tics and tells. And I think the weirdest and most frustrating one is “Yoda-speak,” where the AI literally talks like Yoda. It’s weird, kind of hilarious, annoying, and pervasive, but most of all, it’s fascinating. Why does the AI keep talking like Yoda? What causes it? Can we do anything about it? (Do we even want to? It certainly makes it easy to identify AI-generated text, and also, it’s often unintentionally hilarious.)

I’ve got a theory about why this happens, and I ran an experiment that generated some evidence to support it. But first… what, exactly, is Yoda-speak?

Backwards, this bad writing is

I’ve been working on my latest book. It’s called Be the Model, and it teaches you how LLMs work by having you build interactive pages that let you literally do the job of a model to get an intuitive understanding of how they work. So I’ve been writing a lot, which means I’ve been confronted with a lot of Yoda-speak.

Here’s a great example from the chapter I’m currently working on (the “pond sentences” are a small training text), from a description of a page that you’ll build to learn how training works:

The eight pond sentences fill the left region, every word clickable.

That’s weird, right? Nobody talks like that.

To be perfectly fair, I’ll need you to suspend your disbelief a little. Yoda would probably say something more like, “The left region, filled by the eight pond sentences is. Clickable, every word must be.” But the AI-generated version is still very weird, and in a pattern that is recognizably Yoda-like.

That’s a pattern I’ve seen a lot. At one point I was looking through some AI-generated text, trying to figure out a particularly unintelligible sentence. When I asked Claude Opus for help, it came up with the term “Yoda-speak,” and I’ve since found that using that term is really useful when trying to get an AI to help decipher sentences like that.

The interesting thing is that AIs are perfectly capable of detecting and correcting Yoda-speak. You can try this yourself: start a new ChatGPT session, paste in that “pond” sentence, tell it: “An AI generated that sentence. That’s Yoda-speak.” I just tried that in an incognito window, and here’s the answer it just gave me:

Yep —that’s unmistakably Yoda-speak. A natural version would be:

“The eight pond sentences fill the left region, with every word being clickable.”

Or, more naturally for UI instructions:

“The left region contains the eight pond sentences, and every word is clickable.”

The original sounds like the AI took normal English, rearranged the syntax, and then forgot to put it back.

This particularly ugly sentence, from one of those AI-generated outlines of a later chapter that I haven’t started writing yet about the attention mechanism (the part of the LLM that understands context and relationships between words), is the example that made me want to write this article:

What’s left is to let the two heads’ work combine, so run the combined eight numbers through one more small table, eight in and eight out, square, called the output projection.

Ugh.

The problem with that sentence, and what made it hard to diagnose what was going on, is that when you first look at it you can basically understand what it’s saying (sort of?), and it’s hard to put your finger on exactly what’s wrong with it. It’s obviously terrible writing, but that’s not unique. AI is a universally awful writer.

What’s uniquely wrong with that sentence is its weird, comma-laden structure. Different parts of the sentence are split up into small comma-separated phrases, and AIs seem especially prone to using them to rearrange sentences, often at the expense of readability. That particular sentence is about running a combined set of eight numbers through one additional small 8×8 table. But the part of the sentence that should have described the table was in the second half of the sentence, and then you learn at the very end that the whole thing is called an output projection (but only after you’ve already been told to run numbers through it). Somehow the components of the sentence were arranged in a way that made it almost unintelligible.

One word at a time, it writes

An LLM is a next-word predictor. It works by taking your prompt, all of the context you’ve fed it, and everything it’s generated so far, generating the next word in its response, and then repeating the whole process for the next word, then the next one, and so on. It’s counterintuitive to think that AIs could possibly work that way, because these things can produce long, complex, and generally coherent and helpful responses. It seems very strange that they do all of that work one word at a time, without following a master plan for a response.

But that’s literally how the AI works. Once an LLM generates a word, it’s part of the response. It generally can’t go back and rearrange the first half of a sentence, even though it hasn’t finished generating it.

And that brings me back to that especially weird “eight in and eight out, square” barely intelligible sentence. Here’s what I think happened. The model generated the sentence up through “run the combined eight numbers through one more small table,” so those words were already baked into the response. Then it figured out that it had three more things it wanted to say about that table. But by that time it was too late to go back and put those details where they belonged, and apparently there was a shortage of periods that week, or possibly a surplus of commas (there’s no way to know for sure), so it just smashed each of those thoughts onto the end of an already awkward sentence.

I did some research to try to figure out if anyone has looked into this phenomenon before. The closest thing I found to an answer is a 2024 paper by Wilson Wu, John X. Morris, and Lionel Levine called “Do language models plan ahead for future tokens?” They were attempting to determine whether a model does real advance work for words it hasn’t written yet, or whether the information that pays off later is a byproduct of whatever was useful at the time.

The experiment that Wu et al. did was actually pretty clever. In normal training, when the model is being corrected for mispredicting a word several steps ahead, that correction reaches back and shapes what it computed at earlier positions. That way, the model learns to prepare information at an early word for the benefit of words further along, which is the “planning ahead” the paper is testing for. So they trained one model normally, and a second one with a modified process that cut that out: each position could only be trained to help predict the very next word. They kept next-word prediction but removed the training signal from words two or more steps ahead.

Then the researchers compared how well the two models predicted text: if the normally trained model did better, that would be evidence that preparing information in advance was actually helping. With ordinary language, the difference was fairly small for GPT-2, although it became larger as they tested bigger models.

So there’s evidence that a model can sometimes prepare information that will be useful later, which means my story about it only figuring things out as it goes is probably too simple. But anticipating is not revising. Whatever the model sees coming, it still can’t reach back and reorder the words it already committed to, and the words it already committed to are where the damage shows up.

Better, the second answer often is

I have a theory that the Yoda-speak phenomenon is closely related to something I see all the time (and I expect you do, too). Say you’re asking an AI for help with an email, and it gives you a draft of a sentence. Then, immediately after it gives you that sentence, it says something like this: “A better way to phrase that would be:” and gives you a second version. Sometimes there’s even a third option. And it almost always tells you it prefers the last one that it generated.

Coincidentally, I ran into a good example of that “here’s a better version” phenomenon when I asked ChatGPT to make the pond sentence more Yoda-like. I literally gave it the prompt make it more yoda-like. Here’s what it said:

More Yoda-like

The left region, the eight pond sentences fill. Clickable, every word is.

Or even more Yoda:

Extra Yoda-like

The left region, filled by the eight pond sentences is. Clickable, every word must be.

At this point, you can probably guess what I think happened. It generated the first version, and now that version was part of the context it had to work with. Then it generated a second version that was even more Yoda-like. But the first version was already there, so it couldn’t replace it. All it could do was give me another one. I see AIs do this kind of self-correction all the time.

If that’s what’s happening, then self-correction and Yoda-speak may be two versions of the same basic limitation. The model can’t go back and change what it already wrote. Sometimes that means the better answer comes afterward. Other times it means a thought that belonged earlier gets smashed onto the end of a sentence.

My own experiment, I ran

After looking at these two related phenomena (the AI generating Yoda-speak and constantly generating a second or even a third option), I got thinking. I started to wonder if this wasn’t just an AI fluke but actually something that I could reproduce and even potentially control. It would be really interesting if the way we give facts to the AI and structure the input could actually have an outsized effect on how it generates its output, especially if seemingly innocuous changes could cause it to create really weird results.

This was a real experiment, because this approach isn’t obviously something that should work. In theory, if you give a model a set of facts, it should be able to produce coherent output no matter how you order those facts. But I’ve been looking at how attention works in LLMs lately because I need to explain it for Be the Model in straightforward terms that anyone can understand. It seemed to me that changing the order of the facts could actually change what the model was paying attention to.

It turns out that there are two different mechanisms at work inside the LLM, and my theory was that they each played a part in both phenomena, the Yoda-speak and the unprompted second answers:

• Attention is about what the model is looking at in the input. Every time the AI generates a new word, it looks back over the whole prompt and everything it has written so far, and weights the parts differently depending on their content and their shape. That weighting is a big part of what makes it good at predicting the next word. I had a theory that three facts written as complete sentences would get interpreted as facts to reorganize, while the same exact facts written as bare fragments might look like pieces that could be pasted right after a noun, and attention would weight them that way and send the model down a bad path.
  
• Autoregression is about what the model has already written. The AI is a “predict the next word” machine: it generates one word at a time, and each of those words becomes part of what it’s looking at when it picks the next one. Once it’s written one more small table, it can’t go back and insert square before table, so if any facts about the table are still unexpressed at that moment, the only place left for them is later in the sentence. If attention had already sent the model down a bad path, autoregression would keep it from repairing the opening.

My guess was that the “I prefer the second answer” text that the AI keeps generating is essentially a mechanism for recovering from weird phrasing, and that the Yoda-speak or other weird phrasing was the underlying phenomenon. I felt like a good experiment would concentrate on reproducing just that weird phrasing.

Coincidentally, OpenAI had just released their new model, Astra, and I was looking for an excuse to kick its tires, so I decided to use it to run an experiment to see if I could reproduce any of the phenomena I’d been seeing. I couldn’t actually verify specifically that autoregression or attention were causing them, because those are internal to the LLM. But I felt like I could design an experiment that could provide some evidence, by structuring prompts in a way that would take advantage of how those mechanisms work. If I could get an AI to produce weird phrasing that showed some of the same problems as Yoda-speak, it would be evidence that these mechanisms were at least a potential root cause.

I used Astra as an orchestrator, and had it spin up a bunch of Luna sub-agents and give them almost identical prompts. Each consecutive sub-agent would get a small tweak in its prompt. I asked Astra to look at the output and see if there were any weird grammatical constructions that looked similar to Yoda-speak, then iterate on that prompt to make another small tweak, and keep going to produce increasingly weird output.

The first test was deliberately simple. Astra gave a fresh Luna writer this prompt:

Write one concise, natural paragraph that communicates every supplied fact. Use only the supplied facts. Do not pad the paragraph or repeat information to make it longer. Do not exceed 100 words. Do not mention these instructions, show a plan, use bullets, or explain your choices. Return only the paragraph.

Facts: The bicycle is blue. The bicycle has a narrow frame. The bicycle has a silver bell. The bicycle has a worn leather seat. The bicycle leans beside the garage. Rina rides the bicycle to work on weekdays. Rina locks the bicycle to the rack outside her office.

Luna turned that into a totally normal paragraph:

The blue bicycle has a narrow frame, a silver bell, and a worn leather seat. It leans beside the garage, and Rina rides it to work on weekdays, locking it to the rack outside her office.

Then Astra gave another fresh Luna writer exactly the same facts, with exactly the same wording, but changed their order. It moved the fact that the bicycle leans beside the garage and the fact that Rina rides it to work ahead of the color, frame, bell, and seat.

Luna mostly defeated the experiment. It had all of the facts before it started writing, and it was perfectly capable of moving them back where they belonged.

So Astra dropped the bicycle and wrote a fresh set of facts about something Luna had never seen, a completely made-up device with a steel slot called a “feed throat” that you push clay tiles through. It sent them with the same prompt as the bicycle:

Facts: The slot is made of steel. A clay tile slides through the slot. The slot is a hand’s width across. The slot is tapered. The slot is called the feed throat.

With no rule attached, Luna handled them the way it handled the bicycle:

The feed throat is a tapered steel slot, a hand’s width across, through which a clay tile slides.

Then Astra added two rules: the answer had to be one sentence, and it had to start with a fixed phrase. That phrase ends with the word slot, so everything else about the slot has to come after it.

The sentence must begin with exactly this text and continue from it:

Slide the clay tile through one more steel slot

Facts: The slot is a hand’s width across. The slot is tapered. The slot is called the feed throat.

Luna still tidied it up. Every time, it reached for a relative clause:

Slide the clay tile through one more steel slot, which is a hand’s width across, tapered, and called the feed throat.

Then Astra made one more small change. It kept the same opening and the same three facts, but wrote the facts as short pieces instead of complete sentences:

Facts: a hand’s width across tapered called the feed throat

And Luna wrote this:

Slide the clay tile through one more steel slot, a hand’s width across, tapered, called the feed throat.

The only thing that changed between those two runs was how the facts were written down. Astra tried the same setup on a completely different subject:

Rest the cracked bowl inside one more padded cradle, two palms wide, crescent-shaped, called the handling saddle.

That’s the sentence I’d been chasing. Once the model had committed to the noun and the remaining facts arrived as loose pieces instead of sentences, it tacked them on in the order they came.

The shape of its input, the model copies. Reorder the words it has already written, it cannot.

That still doesn’t tell me whether this is what happened when the model wrote one more small table, eight in and eight out, square, called the output projection. In these experiments, we deliberately pushed Luna into that kind of situation. In the original sentence, the model got there on its own.

I think the experiment shows that the model copies the shape of its input, and that fits my theory about the underlying attention and autoregression causes. When the three facts went in as complete sentences, Luna wrote a relative clause. The same facts sent in as bare fragments came back as bare fragments after the noun.

There are some obvious limitations to this experiment. I’m watching the model from the outside, so I don’t have any visibility into how the model weighted those tokens, or what its attention mechanism was doing. I also can’t say whether the model already had those three facts in mind when it wrote slot, but even if it did, it couldn’t put them anywhere but after the text it already generated. But the outputs match what those two mechanisms would do. Attention responds to how the prompt is written, as well as to what it says. Once the model has written the noun, it can’t go back and put anything in front of it.

I suspect that’s also why the AI keeps offering us that second answer. It can’t fix the Yoda-speak that it already generated. It can only write another sentence and hope that’s good enough.