Something genuinely changed. Search results increasingly answer the question directly, sometimes citing sources, and a share of the traffic that used to arrive from informational queries now does not.
A great deal of nonsense grew up around that change. Vendors selling AI optimization packages, techniques presented as new that are ordinary SEO with a fresh label, and at least one product category built around a metric nobody can define.
This is an attempt to separate the two, with the primary sources attached so you can check rather than take our word for it.
What Google actually says
This matters because a lot of AEO advice is built on assumptions about how these systems work rather than on what the operators have published.
On special files and markup. Google’s guidance is direct: you do not need to create machine
readable files, AI text files, markup or Markdown to appear in Google Search including its
generative features, because Search does not use them. That includes llms.txt, which Google has
said neither helps nor hurts rankings and is not used by Google Search.
This does not make llms.txt harmful. Other systems may choose to use it. It makes it optional, and
it makes anyone selling it as the key to AI visibility either mistaken or dishonest.
On what does work. Google’s position is that normal SEO practice is what makes content eligible for AI features. Crawlable pages, useful content, clear structure. Not a separate discipline.
On FAQ markup. Worth noting because it is still being sold. Google removed the FAQ rich result feature from Search, announced in the May 2026 changelog and effective 7 May 2026. Prior to that it had already been limited to well-known government and health sites since 2023.
The practical consequence: write FAQs because readers have questions, not to earn a rich result that no longer exists.
What genuinely is different
Four things have actually changed, and they are worth working on.
The answer has to be extractable
A page that buries its conclusion under six paragraphs of preamble was always worse for readers. It is now also harder to quote.
A direct answer near the top, in 40 to 90 words, that makes sense when lifted out of the page, is the single most useful structural change. It serves a skimming human and a retrieval system identically, which is a good sign it is a real improvement rather than a trick.
Self-contained sections
If a section only makes sense having read the two before it, it does not survive being retrieved on its own. Writing sections that stand up independently is a discipline that helps both audiences.
This is not the same as “chunking,” which Google specifically lists among the tactics you can ignore. The difference is that one is writing clearly and the other is formatting for an imagined parser.
Entity clarity carries more weight
These systems have to establish who you are before they can present you as a source. Inconsistent company details across your site, your profiles and third-party listings make that harder.
A single maintained facts page, consistent naming, and structured data that matches what is visible is cheap to do and genuinely helps. This is not new advice. It is old advice that now matters more.
Being worth citing is the actual differentiator
A page that summarizes publicly available information has no reason to be selected over the source it summarized. This is the part that cannot be shortcut, and it is why the volume-based content strategy transfers badly to this environment.
First-hand information, original measurements, real failure modes, configuration examples, explicitly stated limits. Material that exists nowhere else because it came from doing the work.
What has not changed
- Crawlability. A page that cannot be retrieved cannot be cited. This remains the largest practical gap on most sites.
- Rendering. Content that only exists after JavaScript runs is a risk with no upside when serving real HTML is available.
- Internal linking. Still how importance is communicated and how pages are discovered.
- Page speed and Core Web Vitals. Still a floor rather than a differentiator.
- Editorial judgement. Fewer, better pages still beat more, thinner ones.
If a vendor’s AI optimization proposal does not start with these, they are selling the interesting part before the necessary part.
The measurement problem, stated honestly
This is where most of the dishonesty in the market sits.
What you can measure:
- Referral traffic from AI products, where it appears in analytics with a recognizable source
- Branded search growth, which often rises when people encounter you in an answer and then search for you directly
- Non-brand impressions in Search Console, which move before rankings do
- Manual checking of a defined set of queries, run consistently over time
What you cannot currently measure:
- Comprehensive citation frequency across systems
- Whether a specific answer used your page
- Impressions inside an AI answer, in any comparable way
- Attribution when someone reads an answer, does not click, and buys from you three weeks later
Anyone offering a “GEO score” should be asked for the methodology in writing. If the answer is not a reproducible calculation over data you can inspect, it is a number invented to justify a retainer.
We would rather tell a client that a channel is partly unmeasurable than produce a confident figure that means nothing.
Deciding your crawler policy properly
This is two decisions and they get conflated constantly.
Retrieval crawlers fetch pages to answer a user’s question now. They can cite you and send
referral traffic. Examples include OAI-SearchBot, PerplexityBot, Claude-SearchBot and
Claude-User.
Training crawlers collect data used to train models. They will not send you traffic.
Most businesses want retrieval allowed. Whether they want training allowed is a separate commercial and ethical judgement, and reasonable organizations decide it differently.
The failure mode is a blanket rule aimed at training crawlers that also blocks retrieval bots. The result is that you cannot be cited at all, which is rarely what anyone intended.
# Retrieval allowed, so we can be cited
User-agent: OAI-SearchBot
Allow: /
User-agent: PerplexityBot
Allow: /
User-agent: Claude-SearchBot
Allow: /
User-agent: Claude-User
Allow: /
# Training crawlers are a separate, deliberate decision.
# Whatever you choose, write down why.Check what your current file actually does. We regularly find blanket blocks added by a plugin or a previous agency that nobody decided on.
What we would actually do
If you asked us to improve how well answer engines can use your content, in order:
- Make it retrievable. Server-rendered HTML, correct status codes, sensible internal linking. This is usually where the real gap is.
- Fix the entity. One maintained facts page, consistent details everywhere, structured data matching visible content.
- Put the answer at the top of the pages that matter, in a form that survives extraction.
- Add what only you know. Original evidence, cited sources, explicit limits. This is the slow part and the part that matters.
- Decide the crawler policy deliberately and record the reasoning.
- Measure what is measurable and say plainly what is not.
Notice how much of that is ordinary good practice. That is the honest summary of this whole topic: the fundamentals did not change, the consequences of getting them wrong extended to more surfaces, and the emphasis shifted toward being genuinely worth quoting.
The integrity line
We can improve how clearly your expertise is structured, sourced and crawlable. We can make your entity unambiguous and your answers extractable.
We cannot guarantee that a specific answer engine will cite a specific page for a specific query. Nobody controls that, the systems change frequently, and any guarantee offered is either a misunderstanding of how they work or a deliberate misrepresentation.
If that sounds like a weaker pitch than the alternative, it is. It is also the only version that survives contact with the next six months.