My agent system read a real company’s website end to end: 884 pages, 6,839 automated checks, about 18 minutes. It came back with 203 proposals, each with its own evidence. It found the flagship product page hidden from Google in four languages, 41 localized pages carrying English descriptions, and 206 pages missing from the sitemap. Before any human saw the results, three refutation agents attacked them and cut the biggest number nearly in half.
Nobody has ever read your whole site
Here is an uncomfortable fact about almost every company website: no one has ever read it. Not the founder, not the agency, not the SEO tool. Sites accumulate pages for a decade; humans read samples. Crawlers check technical boxes on each URL but never sit down and read the site the way an editor would.
I’m co-building a system that does exactly that: a site manager, not a report generator. It reads every page deeply: what the page is about, how it’s written, who links to it, whether anyone searches for it. Then it proposes specific actions and waits for a human yes or no. Nothing executes on its own. It’s the same principle I described in my post on autonomy.
Recently it read the site of a mid-size iGaming B2B vendor, end to end: 884 pages found, 848 read in depth, a graph of 69,201 internal links assembled, every page checked against the company’s 51 closest search competitors. Eighteen minutes of machine time. What came out was not flattering, and that raised the real question first.
Before you see the findings: we tried to break them
An AI that reads 884 pages will also hallucinate confidently about them unless something forces it not to. So before anything left the system, three independent agents with one mandate, refute this, attacked the findings from three sides: the stored data, the live site, and the structural claims. They broke things. One headline number did not survive, and I’ll show you exactly which one and why after you’ve seen the findings themselves. Keep that in mind as you read: everything below survived an attack designed to kill it.
Which pages were hiding? Four findings
Why was the flagship product page invisible to Google in four languages?
The company’s main product page carried an explicit instruction to search engines, in line 6 of its source code: <meta name="robots" content="noindex, nofollow" />. In plain words: “do not show this page, do not follow its links.” Not once: on the English, Spanish, Portuguese, and Turkish versions alike. A buyer googling this product cannot find its page, because the page itself asks not to be found. Nobody had noticed, because nobody reads source code in four languages.
How did 41 localized pages end up speaking English to Google?
The site maintains Spanish, Portuguese, and Turkish sections. On 41 of those localized pages, the meta description (the text a searcher actually sees in results) is byte-identical English. A Spanish page declares lang="es-ES" and then describes itself to Spanish searchers in English. The translation project covered the visible page body and stopped before the layer search engines read first.
Where did 206 pages disappear from the sitemap?
The system compared the full crawl against the sitemap the site reports to Google and found 206 real, working pages the sitemap never mentions, including entire localized sections and a glossary. Google learns about these pages by accident, not from the site. A separate group of 138 pages has zero internal links pointing to them: they exist, but no path on the site leads there.
How far behind was the site in search, measured honestly?
Against 51 companies ranking for the same searches, this site’s estimated search visibility sat roughly 5.6× below the niche median, ahead of only 14 of the 51. Method, in one breath: bought ranking data (DataForSEO, July 2026), the peer pool auto-selected by shared search terms plus competitors the company itself would name, the comparison taken at the median so a few giant B2C domains can’t skew it. These are estimates from third-party data, and we label them that way. Only the company’s own analytics could make them exact.
The full picture in one table
| Finding | Scale | Why a standard crawler misses it |
|---|---|---|
| Product page noindexed | 4 language versions | flags the tag on one URL; doesn’t connect it across hreflang variants of the money page |
| English meta on localized pages | 41 pages | needs language detection per field, not per page |
| Pages absent from sitemap | 206 pages | needs crawl-vs-sitemap set comparison, cleaned of URL aliases |
| Zero inbound internal links | 138 pages | needs the full 69,201-edge link graph, not per-page checks |
| Over-optimization signals | 519 of 848 pages | needs inbound anchor-text distribution per page (worked example: 58.5% exact-match anchors on one post) |
| Thin content | 145 pages | word counts exist in crawlers; judging main content apart from theme boilerplate doesn’t |
The number that didn’t survive: 382 → 206
Now the promised confession. The missing-from-sitemap finding originally said 382. The refutation pass re-derived every number from raw inputs: it recomputed the competitor benchmarks from the vendor’s raw responses (they reproduced to the last digit) and re-implemented the scoring pipeline independently (it matched to four decimal places). Then it hit the sitemap-gap count and found a real bug.
The engine was counting URL strings, not pages. The same page with and without a trailing slash: two missing pages. Cleaned honestly, the raw 382 decomposed into 14 trailing-slash twins, 22 foreign subdomains, 55 pagination pages, and 85 tracking-parameter aliases, leaving 206 genuinely missing pages.
The system had overstated its most dramatic finding by nearly two times, and the number looked completely plausible the whole way.
I now call this the «382 of what?» rule: before you trust any count from an AI, a crawler, or a dashboard, ask what unit it is actually counting. The most durable errors in analytics aren’t wrong numbers; they’re right numbers counting the wrong thing. Nobody outside would ever have caught ours, because 382 flattered the product. That’s exactly why the examiner has to live outside the examinee. It’s the same lesson as the runaway-agent story, at audit scale.
And here is the part I want to defend: we did not quietly fix the number. The corrected 206 went into the client-facing document with a footnote showing the raw 382 and exactly what was cleaned out. If you sell analysis, you sell trust in numbers, and a disclosed self-correction is the strongest proof of trust you can offer. The error you surface yourself builds credibility; the one your reader finds destroys it.
What this system can’t do
Honesty cuts both ways, so the limits, plainly. The 203 proposals are not 203 truths: they are evidence-backed candidates for a human decision, and some will be rejected on context the machine can’t see. Visibility numbers are estimates from bought data, not the site’s analytics. And the system reads public pages only: it can’t see conversion data, business priorities, or the reason a page exists that no one searches for.
What happened next
Nothing, by design. The 203 proposals sit in an approval queue, each waiting for a human yes or no, each reversible, each carrying its evidence. Nothing has been changed on the site. A system like this earns the right to act the same way it earned the numbers above: by being checkable at every step. The read takes 18 minutes; the trust is built the slow way.
I’m building this site manager now, in a closed early-access circle. If you run a content-heavy site and want it read end to end, say hello.
Written by Serhii Kravchenko. Drafted in dialogue with the site’s agent from the case’s source material, edited and approved by him.
