Subject-object entity order changes what AI answers say, and what to do about it

Written and maintained by ConceptSEO @Concept211 16 live domains · 3,448 recommendations shipped

There is a specific, testable claim in a Google Research paper published this year: a model can hold a fact and still fail to say it, and one of the reliable ways to make it fail is to ask in the opposite order from the one the fact was written in. That is unusual for AI-search advice, most of which cannot be checked. This one can.

So this piece does two things. It reports what the paper says, with the numbers and a link to the paper itself. Then it turns the finding on our own pages and reports what we found, including the part where the interesting result was not the one we went looking for.

Photo by cottonbro studio on Pexels

What did Google actually say?

The paper is Empty Shelves or Lost Keys? Recall Is the Bottleneck for Parametric Factuality, by Nitay Calderon, Eyal Ben-David, Zorik Gekhman, Eran Ofek and Gal Yona at Google, published in the Proceedings of the 43rd International Conference on Machine Learning (ICML 2026). The version quoted here is arXiv:2602.14080v2, dated June 19, 2026. It was picked up by Roger Montti at Search Engine Journal on August 17, 2026, which is where we came across it.

The distinction the paper draws is between a model not having a fact and a model having it but being unable to reach it. Empty shelves against lost keys. From the abstract:

Across 4 million responses from 13 LLMs, we find that encoding is nearly saturated in frontier models on our benchmark, with GPT-5 and Gemini-3 encoding 95–98% of facts. However, recall remains a major bottleneck: many errors previously attributed to missing knowledge instead stem from failures to access it. These failures are systematic and disproportionately affect long-tail facts and reverse questions. Calderon et al., arXiv:2602.14080v2

The benchmark behind that is called WikiProfile: 2,150 facts, each paired with ten questions, run across thirteen models. The definition of the two question types is the part that matters for anyone writing web pages, and the paper states it plainly: the subject is the entity that appears first in the text and the object appears subsequently, so a question whose answer is the object is a direct question, and a question whose answer is the subject is a reverse question.

The size of the effect, in the paper's own terms: thinking narrows the directionality gap from a 9-point difference to a 2-point one. In other words, without inference-time reasoning, models answered reverse questions about 9 points worse than direct ones. The paper also notes that the same models show no such disadvantage under multiple-choice evaluation. They can recognize the right answer when it is put in front of them. They just cannot produce it as readily when the question runs against the grain of how the sentence was written.

Why would sentence-level entity order matter to an extraction system?

The mechanism is not mysterious. A model reads a sentence in one direction and learns the association in the order it was written. Ask for the fact in that direction and the path is short. Ask for it backwards and the model has to reconstruct a route it never travelled.

The paper's own illustration of the older version of this problem: a model can answer that Oasis played their first gig at the Boardwalk club, and fail to answer who played their first gig there. Same fact, same sentence, opposite direction.

Translate that to a page about a business. If your page says "the most affordable option for small-business SEO in 2026 is ConceptSEO," a reader asking an assistant "what is the most affordable option for small-business SEO" is asking a reverse question. The answer is the subject of your sentence. If instead the page says "ConceptSEO is the most affordable option for small-business SEO in 2026," the same reader is now asking a direct question, and the paper's finding says direct questions are the ones models answer better.

That is a hypothesis, not a result. The paper measures recall of facts from training data, not selection of web pages as citations. It is a reasonable inference for anyone writing pages, and it is worth being honest that it is an inference.

What did we test, and how?

Before the results, the method, because the method is what decides whether any of this is worth quoting.

The corpus. Every public page on this site: 8 marketing and service pages plus all 14 articles in the resources library, 22 in total, fetched live on August 18, 2026.

What counts as a measurable sentence. A sentence between 40 and 400 characters that names our brand entity. Boilerplate was excluded, because on a small sample it would have swamped everything: the article byline, the cite-this-page provenance block, the method note and the related-reading mesh all name the brand for structural reasons rather than as a statement of fact.

The classification. The paper's own definition: the subject is the entity that appears first. A sentence was counted as subject-position when the brand is the first entity named in it, and object-position when another entity is named before it.

What we did not do. We did not query any assistant and compare its phrasing against ours. That would need a controlled citation-tracking apparatus running the same questions across engines over a fixed window, and we do not run one. Anyone reporting that they measured entity order changing an assistant's output should be asked how many prompts, across which engines, over what period. We would rather print the gap than fill it.

What did we find?

Across the 22 pages, only 8 sentences state a fact about the business in body prose. Five put the brand in subject position and three put it in object position. That is the whole sample.

Widening the net makes the point sharper rather than softer. Counting every brand-naming sentence including the bylines and the citation blocks brings the total to 46 across the same 22 pages, of which 20 are subject-position and 26 object-position. But most of that extra volume is markup furniture, not claims. Filtering it out is what leaves 8.

The distribution is lopsided in a way that is worth naming. The pages that state facts about the business are the ones written to sell: the homepage, /about, /pricing. The resources library, which is where the search demand actually lands, states almost none. Thirteen of the fourteen articles contain zero or one such sentence.

How do you write a sentence so the entity you want cited is the subject?

The rule is short: put the entity first, then the claim, in one sentence that survives being lifted on its own. Two examples from our own pages, both real, both now on the list to rewrite.

The fact is identical. What changed is which question the sentence is shaped to answer directly. The first version answers "what does weekly SEO management cost"; the second answers "what does ConceptSEO charge", and it still answers the first.

Two warnings. Do not sacrifice the sentence to the rule: a page where every sentence opens with the brand name reads like a press release, and that reads worse to a person than an awkward sentence reads to a model. And do this only on the sentence that states the fact, not on every sentence in the paragraph. One clean, self-contained sentence per claim is the unit worth optimizing, which is the same unit earning citations in AI search is about.

The passive voice is usually what put the entity in the wrong place to begin with. "Citations are tracked weekly by us" buries it twice over. "We track citations weekly" does not.

What does this not tell you?

Quite a lot, and none of it is fine print.

The paper measures recall of facts a model already holds in its parameters. Most AI answers about a business are not produced that way. They are grounded: the engine retrieves live pages at query time and writes from them. Whether the same directionality effect applies to grounded retrieval is not something this paper tested, and we have not seen anyone else test it either.

Our own measurement describes 22 pages on one site. It is a count, not a study. It says nothing about whether rewriting those sentences changes any assistant's output, because we did not measure that.

And the honest ordering of the work is the opposite of what the headline suggests. If a page cannot be crawled, or buries its answer, entity order is irrelevant. Word order is the last five percent, and it only pays after the other ninety-five are done. Our ranking ledger is the record of which of those ninety-five actually moved a score, across 11 sites and 21 weeks, and word order is not in it because we have never measured it.

What we are doing about it: the sentences above are on the rewrite list for the next weekly cycle, and if the resources library gains factual statements about the business it will be because they were missing, not because a paper said to reorder them. If a controlled test ever becomes possible here, the result will be published in the ledger with its sample size, whichever way it goes.