We scanned 104 websites. The median “Markdown” response was basically HTML.

We have now recorded 1,904 page results across 104 domains. The clearest finding is not that most websites are bad at serving AI agents.

It is that the middle is almost empty.

In each domain’s latest completed audit, 68% of sites scored below 40. Another 30% scored 70 or higher. Just two sites landed between those groups.

The median score was 15. The 75th percentile was 80.

That is not a smooth adoption curve. It is a cliff.

The “Markdown” response was usually the HTML response

Every page in our audit gets fetched twice:

GET /page
Accept: text/html

GET /page
Accept: text/markdown, text/html, */*

If a site supports content negotiation, the second request should return a distinct, useful Markdown representation. Across the 706 pages in each domain’s latest completed scan:

That middle row deserves attention. A different response is not necessarily an intentional Markdown implementation. Dynamic HTML, rotating content, and request-time rendering can all change the body between two requests. If the server still identifies the result as HTML—and the body still looks like HTML—the agent did not get the representation it asked for.

The payload sizes tell the same story. For the 695 pages where we could compare both responses, the median “Markdown” body was 99.8% the size of the HTML body. On 43% of pages, it was the same size or larger. Only 36% returned a body less than half the size of the browser response.

The format preference changed. The payload usually did not.

Good implementations were good all the way down

Here is the encouraging part: once a page returned a distinct response with the correct content type, the rest of the implementation was usually strong.

Of those 219 pages:

• 92% returned content that actually looked like Markdown
• 97% removed navigation and browser chrome
• 99% cut the payload below half the size of the HTML
• 79% included useful YAML frontmatter
• 79% had clean, descriptive links

This is why the score distribution is so polarized. Content negotiation is rarely one isolated header change. The teams that treat it as a rendering path tend to solve the whole path: format, structure, metadata, links, and payload size.

The teams that do not have that path send the browser document again.

The hard part is not converting HTML

It is tempting to frame this as a serialization problem:

1. Receive HTML.
2. Run it through a converter.
3. Return Markdown.

That can produce Markdown syntax. It does not automatically produce good agent content.

Agents do not need the mobile menu, cookie controls, footer columns, social icons, “read more” links, or every piece of text that happened to exist in the DOM. A useful agent response needs an editorial boundary: what is the page actually about, and what context helps a machine use it?

Our latest scans show the gap:

• 43% of pages had navigation and chrome successfully stripped
• 25% included useful frontmatter
• 32% passed link-quality checks
• 22% of homepages acted as a useful Markdown site index

Some of those checks depend on having Markdown in the first place, so they should not be read as independent web-wide failure rates. They are better understood as layers. If the representation never changes, metadata and Markdown link quality cannot rescue it.

The order matters.

Agent readiness is usually a system property

Only 21 of the 99 domains with a completed scan negotiated Markdown correctly on every page we tested. Another 15 supported it on some pages. The remaining 63 did not pass the negotiation check on any tested page.

That split looks familiar if you have maintained a large website. Platform behavior is consistent; one-off page behavior is not.

A middleware rule can cover every route. A shared content renderer can strip the same shell from every template. A Vary: Accept policy can make cache behavior explicit across the site.

By contrast, hand-authoring a Markdown endpoint for one documentation page creates a demo, not a capability.

The mixed group matters because partial coverage is easy to miss. A homepage can work while product pages fall through to HTML. A blog template can negotiate correctly while documentation lives on another stack. One successful URL does not prove the site is agent-ready.

Test the routes that come from different systems.

What we would implement first

The data points to a practical order of operations.

1. Make negotiation real

Inspect the Accept header and return a genuinely different representation when text/markdown is preferred. Send an accurate Content-Type, and include Vary: Accept so caches do not serve Markdown to browsers or HTML to agents.

2. Render from content, not from the finished page

If possible, generate both HTML and Markdown from the same structured source. Converting the final DOM means starting with every browser concern and trying to subtract the right ones later.

When DOM conversion is the only realistic option, define the main-content boundary explicitly. Do not ask a generic converter to make product decisions for you.

3. Add identity and navigation

Frontmatter should answer basic questions without inference: title, description, canonical URL, date, and author where relevant.

The homepage should also help an agent decide where to go next. A short, organized set of descriptive links is more useful than reproducing a visual mega-menu.

4. Test consistency, not one showcase URL

Pick representative routes from each rendering system: homepage, product, docs, blog, pricing, and any separately hosted section. Check both the response headers and body. Compare payload sizes. Follow the links.

If support disappears at a template boundary, the implementation is not done.

This is a snapshot, not a census

The full dataset covers 247 audit runs and 1,904 page results collected between March 15 and August 30, 2026. Because some domains were scanned repeatedly, the percentages in this post use only the latest completed audit for each domain: 99 domains and 706 pages. Five domains did not have a completed run.

The sample is self-selected. People audit sites they are curious about, responsible for, or trying to improve. We crawl up to 10 pages per run, not every URL. These numbers describe what our scanner observed, not the entire web.

They are still useful because the implementation pattern is so sharp.

Most sites are not sitting halfway through a polished Markdown rollout. They either have a real representation layer, or the agent gets the browser page again. Once that layer exists, clean Markdown, smaller payloads, and stripped navigation tend to arrive together.

The biggest opportunity is not a better converter.

It is making agent content a platform capability.

Check your own site

Run a free audit to test up to 10 pages across your site. You will see the response headers, payload difference, page-level checks, and the evidence behind the score.

No signup. No black-box grade. If the agent gets HTML again, we will show you exactly where.