The pitch shows up in every inbox now. Generate hundreds of articles with AI, publish them all, watch the traffic roll in. Some sites try it and see a brief spike. Most then watch the whole thing flatten or slide a few months later. The reason is not a mystery, and it is not a penalty in the dramatic sense. It is economics. Google spends a finite amount of effort crawling the web, and it spends that effort where it expects to find value. Publish a thousand thin pages and you are asking for crawl budget you have not earned.
What crawl economics actually means
Every site gets an implicit crawl budget: how often and how deeply Google's crawler visits. That budget is not set by your ambitions. It scales with how much demand and value Google has historically found on your domain. Fast, healthy, frequently useful sites get crawled more. Slow sites, and sites full of pages that turn out to be low value, get crawled less, because continuing to crawl them is a poor use of Google's resources.
Publishing at scale runs straight into this. A thousand new URLs is a thousand new things asking to be crawled, indexed, and ranked. If the domain has not earned the budget, most of those pages get crawled slowly and indexed partially or not at all. The ones that do get in tend to dilute the site's overall quality signal rather than add to it.
Why scaled AI content fails specifically
The problem is rarely that the text reads badly. Modern models write fluent prose. The problem is what the pages are: near-duplicates of each other and of what already exists, assembled from the same training data every competitor is drawing from, with no first-hand information and nothing a reader could not get from the model directly.
Google's systems are built to find exactly this. The helpful-content signals and the core ranking systems reward content that shows first-hand experience and expertise, and they discount content that is derivative at scale. A page that summarizes what already ranks does not deserve to outrank the sources it summarized, and increasingly it does not. There is a second cost that is easy to miss. A large block of low-value pages drags on how the whole domain is assessed, so the thin content can pull the pages you actually care about down with it.
What separates content that earns crawl from content that gets ignored
The pages that keep earning crawl and citations have a few things in common, and none of them come out of a bulk generator:
- First-hand input. Original data, real examples, tests you ran, a point of view a model cannot produce on its own.
- Entity depth. Coverage thorough enough to resolve the questions and entities around a topic, instead of skimming the surface of it.
- Internal support. New pages linked from relevant existing pages, so crawl and authority actually reach them.
- Evidence of use. Pages people find, read, and come back to, which is the signal everything above is meant to produce.
That work does not scale to a thousand pages a week, which is the whole point. The output that earns crawl is the output you cannot mass-produce.
What to do instead
Publish fewer pages, each of which deserves to exist. Before you ship one, ask what it contains that the current top results do not. If the answer is nothing, the page is not going to win, and publishing it anyway spends budget and reputation you would rather keep.
Use AI where it is genuinely good: drafting from an outline you control, restructuring passages so they are easier to cite, generating variations to test, catching gaps you missed. Then add the part it cannot supply, which is your data, your experience, and your specific take. Maintain what you publish instead of abandoning it, because continued usefulness is part of why a page keeps its crawl. This is slower than the mass-production pitch. It is also the version that still has traffic in a year.
We run content this way on the domains we manage. Each piece is implemented and maintained as part of a weekly loop rather than dumped in a batch. The honest weekly SEO routine shows what that cadence looks like day to day.
Where this leaves you
AI has not made content cheap in the way the mass-production pitch implies. It has made fluent text cheap, which is a different thing. The scarce input is still judgment: what is worth writing, what a page needs to contain to earn its place, and whether it delivered once it went live. That is the part worth spending on. The pricing page lays out how we fold it into a flat weekly plan.