OpenAI Responses Compaction with AI Gateway

Long agent sessions eventually fill the model's context window. Compaction lets you hand the conversation so far to the model and get back a shorter version you can carry forward: your user messages, followed by one opaque compaction item that stands in for everything else.

AI Gateway serves OpenAI's compaction endpoint at the same path as OpenAI:

Requests are forwarded to OpenAI unchanged apart from the model ID, and OpenAI's response is returned as is. Coding agents that support server-side compaction, such as OpenAI Codex, call this endpoint automatically when you point them at AI Gateway.

Send the conversation you want to shrink as input. The response contains a compaction item you can pass back in the input of your next request in place of the original history:

import OpenAI from 'openai';
 
const client = new OpenAI({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: 'https://ai-gateway.vercel.sh/v1',
});
 
const compacted = await client.responses.compact({
  model: 'openai/gpt-6-astra',
  input: [
    {
      role: 'user',
      content: 'Refactor the auth module to use the new session store.',
    },
    {
      role: 'assistant',
      content: 'I updated session.ts and login.ts to read from SessionStore...',
    },
    {
      role: 'user',
      content: 'Now add tests for the logout path.',
    },
  ],
});
 
// Continue the conversation from the compacted history
const next = await client.responses.create({
  model: 'openai/gpt-6-astra',
  input: [
    ...(compacted.output as OpenAI.Responses.ResponseInputItem[]),
    {
      role: 'user',
      content: 'Also cover the token refresh path.',
    },
  ],
});
 
console.log(next.output_text);

import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.getenv('AI_GATEWAY_API_KEY'),
    base_url='https://ai-gateway.vercel.sh/v1',
)
 
compacted = client.responses.compact(
    model='openai/gpt-6-astra',
    input=[
        {'role': 'user', 'content': 'Refactor the auth module to use the new session store.'},
        {'role': 'assistant', 'content': 'I updated session.ts and login.ts to read from SessionStore...'},
        {'role': 'user', 'content': 'Now add tests for the logout path.'},
    ],
)
 
# Continue the conversation from the compacted history
next_response = client.responses.create(
    model='openai/gpt-6-astra',
    input=[
        *compacted.output,
        {'role': 'user', 'content': 'Also cover the token refresh path.'},
    ],
)
 
print(next_response.output_text)

curl -X POST "https://ai-gateway.vercel.sh/v1/responses/compact" \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-6-astra",
    "input": [
      { "role": "user", "content": "Refactor the auth module to use the new session store." },
      { "role": "assistant", "content": "I updated session.ts and login.ts to read from SessionStore..." },
      { "role": "user", "content": "Now add tests for the logout path." }
    ]
  }'

The response is a response.compaction object:

{
  "id": "resp_0f3a...",
  "object": "response.compaction",
  "created_at": 1756800000,
  "output": [
    { "type": "message", "role": "user", "content": [{ "type": "input_text", "text": "Refactor the auth module..." }] },
    { "type": "message", "role": "user", "content": [{ "type": "input_text", "text": "Now add tests for the logout path." }] },
    { "type": "compaction", "id": "cmp_8b2e...", "encrypted_content": "gAAAAAB..." }
  ],
  "usage": {
    "input_tokens": 1240,
    "output_tokens": 310,
    "total_tokens": 1550
  }
}

The compaction item's encrypted_content is opaque to you. Replay it in input on later turns and the model recovers the compacted context. You can compact again later; a new compaction item stacks on top of the previous one.

Compaction runs on OpenAI's servers, so the model must be an OpenAI model. Requests for models from other providers return 400:

{
  "error": {
    "type": "invalid_request_error",
    "message": "Model \"anthropic/claude-sonnet-5\" is not available on OpenAI; /v1/responses/compact is only supported for OpenAI-routed models"
  }
}

AI Gateway does not fail over to another provider for compaction requests. The compaction item is OpenAI server-side state that other providers can't read.

A compaction request runs the model over your conversation, and you pay for it like any other Responses API request: input, cached input, reasoning, and output tokens from the response's usage block, at the model's listed rates. The request appears in your AI Gateway usage alongside your other requests to the same model.

With your own OpenAI key, OpenAI bills the tokens to your account and AI Gateway charges nothing.

Compaction works with your own OpenAI key configured at the team level, including any custom base URL and model mappings on the credential. Without a team-level OpenAI credential, requests use AI Gateway's OpenAI credentials.

Per-request keys passed in providerOptions.gateway.byok aren't supported for compaction. Configure the credential at the team level instead.

Compaction is designed for clients that don't rely on OpenAI storing conversation state: everything the model needs to continue lives in the compaction item you carry in input. This makes it a good fit for store: false and for accounts under zero data retention.