Serving static files

Serve tenant-specific robots.txt, sitemap.xml, and llms.txt files using route handlers.

Multi-tenant applications need tenant-specific versions of static files like robots.txt, sitemap.xml, and agent discovery files like llms.txt. You can use route handlers to serve these files dynamically per tenant.

Next.js provides built-in metadata file conventions for robots.txt, sitemap.xml, and other common files. Use route handlers when you need to serve files not covered by the metadata API, like llms.txt or custom discovery files.

Route handler

Create a route handler that resolves the tenant and returns the appropriate content:

app/[domain]/robots.txt/route.ts

import { NextRequest, NextResponse } from "next/server";

export async function GET(
  request: NextRequest,
  { params }: { params: { domain: string } }
) {
  const { domain } = params;
  const tenant = await getTenant(domain);

  if (!tenant) {
    return new NextResponse("Not found", { status: 404 });
  }

  const content = `User-agent: *
Allow: /
Sitemap: https://${tenant.domain}/sitemap.xml`;

  return new NextResponse(content, {
    headers: {
      "Content-Type": "text/plain",
      "Cache-Control": "public, max-age=3600",
    },
  });
}

Proxy integration

Your Proxy must allow static file paths to reach route handlers instead of rewriting them:

proxy.ts

const STATIC_FILE_PATHS = [
  "/robots.txt",
  "/sitemap.xml",
  "/llms.txt",
  "/.well-known",
];

export function proxy(request: NextRequest) {
  const { pathname } = request.nextUrl;

  if (STATIC_FILE_PATHS.some((path) => pathname.startsWith(path))) {
    return NextResponse.next();
  }

  // Your other logic
}

In Next.js 16+, the middleware.ts file was renamed to proxy.ts. See the proxy.ts documentation for more details.

Content types

Set the correct Content-Type header for each file type:

Extension	Content-Type
`.txt`	`text/plain`
`.xml`	`application/xml`
`.html`	`text/html`

Caching

Use the Cache-Control or CDN-Cache-Control header to cache responses in the Vercel CDN cache. Invalidate your resource in the CDN cache when tenant content changes.

return new NextResponse(content, {
  headers: {
    "Content-Type": "text/plain",
    "CDN-Cache-Control": "s-maxage=3600",
  },
});

The main difference between these headers is that CDN-Cache-Control allows you to control cache behavior separately from browser cache behavior. For a more thorough explanation, read more about cache control options on Vercel.

When not to use this pattern

Truly static assets: Use /public for files that don't change per tenant
Large files or media: Use dedicated blob storage