NewIntroducing Agent Layer: serve agent ready content and see how agents use itRead the post
← All posts
·4 min read

We audit Markdown responses. Here is how we serve our own.

A practical walkthrough of AgentReady.dev’s content-negotiation setup using a Next.js proxy and one Markdown route.

engineeringcontent-negotiationmarkdownnextjs

There is one especially embarrassing way to build AgentReady.dev: publish an audit for Markdown support, then fail it ourselves.

So yes, this site serves Markdown. When a client asks for text/markdown, it gets the article or page content without the React shell, navigation, scripts, or styling.

The implementation is small: a Next.js proxy, one route handler, and a generator for each public page type. Here is how it fits together.

The pattern

The goal: one URL, two useful representations. Browsers get HTML. Clients that prefer text/markdown get a purpose-built Markdown response.

We needed this for three routes: / (home), /blog (post index), and /blog/[slug] (individual posts).

Step 1: The proxy spots Markdown requests

Our Next.js proxy checks the Accept header before the page route runs. Matching public paths are internally rewritten to a dedicated API handler.

// proxy.ts
const MARKDOWN_PATHS = new Set(["/", "/blog"]);

function isMarkdownPath(path: string): boolean {
  const normalized = path.replace(/\/$/, "") || "/";
  if (MARKDOWN_PATHS.has(normalized)) return true;
  if (normalized.startsWith("/blog/") && normalized.split("/").length === 3) return true;
  return false;
}

export function proxy(request: NextRequest) {
  const accept = request.headers.get("accept") || "";

  if (!accept.includes("text/markdown")) {
    return NextResponse.next();
  }

  const path = request.nextUrl.pathname;

  if (!isMarkdownPath(path)) {
    return NextResponse.next();
  }

  const url = request.nextUrl.clone();
  url.pathname = "/api/md";

  return NextResponse.rewrite(url, {
    request: {
      headers: new Headers({
        ...Object.fromEntries(request.headers),
        "x-original-path": path,
      }),
    },
  });
}

export const config = {
  matcher: ["/", "/blog", "/blog/:slug*"],
};

The slightly awkward but important part is x-original-path. The internal rewrite points every Markdown request to /api/md; the header tells that shared handler which page the client originally wanted.

Step 2: The API handler generates the markdown

/api/md reads that forwarded path and calls the matching generator. One route, three page shapes.

// app/api/md/route.ts
export async function GET(request: NextRequest) {
  const originalPath = request.headers.get("x-original-path") || "/";
  const path = originalPath.replace(/\/$/, "") || "/";

  let markdown: string | null = null;

  if (path === "/") {
    markdown = homeMarkdown();
  } else if (path === "/blog") {
    markdown = blogIndexMarkdown();
  } else if (path.startsWith("/blog/")) {
    const slug = path.replace("/blog/", "");
    markdown = blogPostMarkdown(slug);
  }

  if (!markdown) {
    return NextResponse.json({ error: "Not found" }, { status: 404 });
  }

  return new NextResponse(markdown, {
    status: 200,
    headers: {
      "Content-Type": "text/markdown; charset=utf-8",
      "Cache-Control": "public, s-maxage=3600, stale-while-revalidate=86400",
    },
  });
}

Step 3: Generating the markdown content

Each page type has its own generator. They all follow the same contract: useful YAML frontmatter, one clear H1, and content with no browser wrapper.

The homepage becomes a compact directory. That is more useful to an agent than translating a visual marketing layout card by card, and it earns the audit’s Sitemap / Index check:

export function homeMarkdown(): string {
  const posts = getAllPosts();
  const postLinks = posts
    .map((p) => `- [${p.title}](${BASE_URL}/blog/${p.slug})`)
    .join("\n");

  return `---
title: "AgentReady.dev — Is Your Website Agent Ready?"
description: "Audit how well websites handle AI agent requests."
url: "${BASE_URL}"
---

# AgentReady.dev

...

## Site Pages

- [Home](${BASE_URL}/)
- [Blog](${BASE_URL}/blog)
${postLinks}
`;
}

Blog posts are simpler because the source is already MDX. We reuse the body and add a small, predictable envelope:

export function blogPostMarkdown(slug: string): string | null {
  const post = getPostBySlug(slug);
  if (!post) return null;

  return `---
title: "${post.meta.title}"
description: "${post.meta.description}"
date: "${post.meta.date}"
url: "${BASE_URL}/blog/${slug}"
---

# ${post.meta.title}

${post.content}
`;
}

Trust, but curl

You can verify it works with curl:

curl -H "Accept: text/markdown" https://agentready.dev/
curl -H "Accept: text/markdown" https://agentready.dev/blog/how-we-serve-markdown

You can also run AgentReady.dev on itself. We do. A perfect score is more persuasive when the product is willing to grade its own homework.

What this approach gets right

A few things worth calling out:

No browser chrome. The generators emit page content, not a lossy conversion of the full HTML document. Navigation, headers, footers, and scripts never enter the response.

Frontmatter on every response. Title, description, date when relevant, and canonical URL. The response says what it is before asking a client to interpret it.

The homepage is a map. An agent hitting the root gets a structured list of important pages instead of a flattened version of a visual hero.

Cache-aware. Markdown responses get a one-hour CDN cache with stale-while-revalidate, just like the HTML pages.

None of this is novel infrastructure. That is the point. Content negotiation is ordinary HTTP applied to a new reader, and the cleanest implementation is often the least dramatic one.