AI SDK for Python with AI Gateway

The AI SDK for Python is a toolkit for building LLM-powered applications and agents in Python. It uses AI Gateway to route requests by default.

See the Python beta getting-started guide for package setup and your first request.

The SDK requires Python 3.12 or later. Install the ai package:

uv add ai

pip install ai

See the Python streaming guide for stream events and final messages.

Create a model with a string model ID and pass it to ai.stream to get a streaming response. You can omit the provider prefix or use gateway: to route through AI Gateway.

import asyncio
import ai
 
async def main() -> None:
    model = ai.get_model('anthropic/claude-sonnet-5')
    messages = [ai.user_message('Explain quantum computing in one paragraph.')]
 
    async with ai.stream(model, messages) as stream:
        async for event in stream:
            if isinstance(event, ai.events.TextDelta):
                print(event.chunk, end='', flush=True)
 
asyncio.run(main())

Responses always stream. After iteration, the full response is available on stream.text, and token counts on stream.usage.

See structured output in the Python beta for Pydantic models and validated results.

Pass a Pydantic model as output_type to get validated, structured data. After the stream finishes, stream.output returns an instance of your model:

import asyncio
import ai
import pydantic
 
class Person(pydantic.BaseModel):
    name: str
    age: int
    city: str
 
async def main() -> None:
    model = ai.get_model('anthropic/claude-sonnet-5')
    messages = [
        ai.user_message('Extract: John is 30 years old and lives in NYC.')
    ]
 
    async with ai.stream(model, messages, output_type=Person) as stream:
        async for event in stream:
            pass
 
    print(stream.output)  # Person(name='John', age=30, city='NYC')
 
asyncio.run(main())

See the Python beta agents and tools guides for execution and conversation loops.

Define tools as Python functions with the @ai.tool decorator. An ai.Agent runs a loop: it streams model output and executes requested tools until the model returns a final answer.

import asyncio
import ai
 
@ai.tool
async def get_weather(city: str) -> str:
    """Get the current weather for a city."""
    return 'Sunny, 72F'
 
async def main() -> None:
    model = ai.get_model('anthropic/claude-sonnet-5')
    agent = ai.Agent(tools=[get_weather])
    messages = [ai.user_message("What's the weather in Tokyo?")]
 
    async with agent.run(model, messages) as stream:
        async for event in stream:
            if isinstance(event, ai.events.TextDelta):
                print(event.chunk, end='', flush=True)
 
    history = stream.messages  # Full conversation, including tool results
 
asyncio.run(main())

The function signature and docstring become the tool schema. The model sees the tool name, description, and typed parameters.

See Inputs & Tools for complete vision, PDF, audio, and video examples across API formats.

See the Python multimodal-message guide for file parts and media types.

Pass file parts alongside text in a message. The model must support the file's media type:

import asyncio
from pathlib import Path
import ai
 
async def main():
    model = ai.get_model("anthropic/claude-sonnet-5")
    messages = [ai.user_message(
        "Describe this image in one sentence.",
        ai.file_part(Path("diagram.png").read_bytes(), media_type="image/png"),
    )]
    async with ai.stream(model, messages) as stream:
        async for event in stream:
            if isinstance(event, ai.events.TextDelta):
                print(event.chunk, end="", flush=True)
    print()
 
asyncio.run(main())

Replace the image part with ai.file_part(Path("report.pdf").read_bytes(), media_type="application/pdf") to send a PDF. See modalities for dedicated image, video, audio, embedding, and reranking operations through ai.ops.

See Python provider-specific parameters for model request configuration.

Use ai.InferenceRequestParams with extra_body to pass providerOptions.gateway. See the Python tabs in provider routing, model fallbacks, and reasoning. The Python beta has a separate version cycle from AI SDK 7 for TypeScript.

The SDK can use AI_GATEWAY_API_KEY environment variable to authenticate:

export AI_GATEWAY_API_KEY="your_ai_gateway_api_key"

When deploying to Vercel or using vercel dev for local development, you are encouraged to use OIDC instead of an API key. Install the vercel extra and the SDK will handle authentication automatically:

uv add "ai[vercel]"

See Authentication for more details.

• Explore the full AI SDK for Python documentation for agents, subagents, human-in-the-loop hooks, and custom loops
• Browse examples on GitHub, from single-file scripts to end-to-end demos
• Learn about model routing and fallbacks
• Building with TypeScript? See the AI SDK page