Serving Static Files

Multi-tenant applications need tenant-specific versions of static files like robots.txt, sitemap.xml, and agent discovery files like llms.txt. You can use route handlers to serve these files dynamically per tenant.

Next.js provides built-in metadata file conventions for robots.txt, sitemap.xml, and other common files. Use route handlers when you need to serve files not covered by the metadata API, like llms.txt or custom discovery files.

Create a route handler that resolves the tenant from the request hostname and returns the appropriate content:

import { NextRequest, NextResponse } from 'next/server';
 
export async function GET(request: NextRequest) {
  const hostname = request.headers.get('host') ?? '';
  const tenant = await getTenant(hostname);
 
  if (!tenant) {
    return new NextResponse('Not found', { status: 404 });
  }
 
  const content = `User-agent: *
Allow: /
Sitemap: https://${tenant.domain}/sitemap.xml`;
 
  return new NextResponse(content, {
    headers: {
      'Content-Type': 'text/plain',
      'Cache-Control': 'public, max-age=3600',
    },
  });
}

Your Proxy must allow static file paths to reach route handlers instead of rewriting them:

import { NextRequest, NextResponse } from 'next/server';
 
const STATIC_FILE_PATHS = [
  '/robots.txt',
  '/sitemap.xml',
  '/llms.txt',
  '/.well-known',
];
 
export function proxy(request: NextRequest) {
  const { pathname } = request.nextUrl;
 
  if (STATIC_FILE_PATHS.some((path) => pathname.startsWith(path))) {
    return NextResponse.next();
  }
 
  // Your other logic
}

Set the correct Content-Type header for each file type:

Use the Cache-Control or CDN-Cache-Control header to cache responses in the Vercel CDN cache. Invalidate your resource in the CDN cache when tenant content changes.

return new NextResponse(content, {
  headers: {
    'Content-Type': 'text/plain',
    'CDN-Cache-Control': 's-maxage=3600',
  },
});

The main difference between these headers is that CDN-Cache-Control allows you to control cache behavior separately from browser cache behavior. For a more thorough explanation, read more about cache control options on Vercel.

• Truly static assets: Use /public for files that don't change per tenant
• Large files or media: Use dedicated blob storage