How to serve documentation for agents

Content negotiation allows clients to request different representations of the same resource using the HTTP-standard Accept header, rather than requiring different URLs.

For example, the Vercel documentation uses this technique to respond with markdown when an agent requests docs:

Terminal

# Responds with HTML
curl https://vercel.com/docs

# Responds with markdown
curl -H "Accept: text/markdown" https://vercel.com/docs

This guide assumes that your content is authored using markdown in a Next.js app, but can be adapted to your content authoring strategy and framework by converting your content to markdown on-the-fly.

Copy link to headingCreate a Route Handler for markdown responses

First, create a Route Handler that returns markdown responses when it is requested:

app/docs/md/[[...slug]]/route.ts

import { notFound } from 'next/navigation';
import { getMarkdownContent } from '@/lib/content';

export async function GET(
  _req: Request,
  { params }: { params: Promise<{ slug?: string[] }> }
) {
  const { slug } = await params;
  const content = getMarkdownContent(slug?.join('/') ?? 'index');

  if (!content) {
    notFound();
  }

  return new Response(content, {
    headers: {
      'Content-Type': 'text/markdown',
    },
  });
}

Copy link to headingCreating conditional responses

Add a function to your next.config.ts that handles the Accept header and use it to rewrite to the Route Handler you created previously:

import type { NextConfig } from 'next';

const nextConfig: NextConfig = {
  rewrites() {
    function markdownRewrite(prefix: string) {
      return {
        source: `${prefix}/:path*`,
        has: [
          {
            type: 'header',
            key: 'accept',
            value: '(.*)text/markdown(.*)',
          },
        ],
        destination: `${prefix}/:path*.md`,
      };
    }
    return {
      beforeFiles: [
        markdownRewrite('/docs'),
      ],
    };
  },
};

export default nextConfig;

Copy link to headingAdd a sitemap.md

Leave a discovery point to the rest of your documentation in these responses, so the LLM knows where to continue searching for helpful content if it doesn’t find what it needs on the first try.

For example, see the sitemap.md for the Vercel documentation.

In the following snippet, a table of contents is created to show the LLM all paths where it can retrieve more information in markdown:

app/docs/sitemap.md/route.ts

import { createTableOfContents, TocItem } from "@lib/content";

function renderTocItems(items: TocItem[], indent = '') {
  let sitemap = '';

  for (const item of items) {
    sitemap += `${indent}- [${item.title}](/${item.path})\n`;
    if (item.children) {
      sitemap += renderTocItems(item.children, `${indent}  `);
    }
  }

  return sitemap;
}

export async function GET() {
  const tableOfContents = createTableOfContents(`content`);

  const sitemap = `# Documentation sitemap\n\n${renderTocItems(tableOfContents)}`

  return new Response(sitemap, { headers: { 'Content-Type': 'text/markdown' } });
}

Copy link to headingTest it out

You can now use curl to receive different response types of your content.

Terminal

# Responds with HTML
curl https://your-domain.com/docs

# Responds with markdown
curl -H "Accept: text/markdown" https://your-domain.com/docs

Build with AI

Deploy & scale

Operate & protect

Resources