# How to build an on-demand voice agent with Vercel Sandbox

**Author:** Ismael Rumzan

---

Have you ever enabled voice mode in ChatGPT and lost track of time chatting with the agent? The natural back-and-forth of voice conversation creates an experience that text alone can't match. If you want to bring this same conversational magic to your web application, this guide shows you how.

Voice AI agents require long-running processes to maintain WebSocket connections, process audio streams, and run machine learning models. Traditional serverless functions can't handle these requirements due to execution time limits and cold start issues. [Vercel Sandbox](https://vercel.com/sandbox) solves this by providing on-demand, isolated environments that run for the duration of a voice conversation and automatically clean up when finished.

This guide shows you how to build a voice agent application where each user session creates its own ephemeral agent environment, eliminating the need to manage persistent servers while maintaining the performance needed for real-time voice interactions.

## Overview

In this guide, you will learn:

- How to create on-demand Vercel sandboxes for LiveKit voice agents.
  
- The architecture of a Next.js frontend that manages sandbox lifecycle.
  
- How to handle real-time progress feedback during application initialization.
  

## Application architecture

These steps describe the interaction flow between the user and the Live voice agent in Vercel Sandbox:

1. A user clicks **Start call** in their browser
   
2. The Next.js frontend calls `/api/sandbox/create`
   
3. The Next.js backend initiates the creation of a Vercel Sandbox
   
4. The sandbox clones the [Livekit python agent repository](https://github.com/livekit-examples/agent-starter-python), installs dependencies, and starts the agent
   
5. The agent connects to LiveKit Cloud and waits in the room
   
6. The frontend polls `/api/sandbox/status` for progress
   
7. When ready, the frontend connects the user to the LiveKit room
   
8. The user and agent communicate through LiveKit Cloud
   
9. After 10 minutes (configurable), the sandbox expires automatically
   

### How Vercel Sandbox works for voice agents

Vercel Sandbox creates isolated compute environments that can run long-running processes. Unlike Functions, which are designed for request-response patterns and have strict execution time limits, sandboxes can maintain websocket connections and run processes for extended periods.

Each sandbox is isolated, ensuring users don't interfere with each other's sessions.

## Deploy the application

### Prerequisites

Before you begin, ensure you have:

- Node.js 18+ installed
  
- A [Vercel account](https://vercel.com/signup) with access to Vercel Sandbox API
  
- A [LiveKit cloud account](https://cloud.livekit.io/) with API credentials
  

Use the [Voice Agent with Vercel Sandbox example repository](https://github.com/vercel/examples/tree/main/guides/sandbox-voice-agent) in one of the following two methods:

### Method 1: One-click deploy

Deploy directly on Vercel by [using the one-click deploy with Vercel](https://vercel.com/new/clone?repository-url=https://github.com/vercel/examples/tree/main/guides/sandbox-voice-agent&project-name=sandbox-voice-agent&repository-name=sandbox-voice-agent&env=LIVEKIT_API_KEY,LIVEKIT_API_SECRET,LIVEKIT_URL,NEXT_PUBLIC_USE_SANDBOX,AGENT_REPO_URL,AGENT_RUNTIME,SANDBOX_TIMEOUT&envDefaults=%7B%22NEXT_PUBLIC_USE_SANDBOX%22%3A%22true%22%2C%22AGENT_REPO_URL%22%3A%22https%3A%2F%2Fgithub.com%2Flivekit-examples%2Fagent-starter-python.git%22%2C%22AGENT_RUNTIME%22%3A%22python3.13%22%2C%22SANDBOX_TIMEOUT%22%3A%22600000%22%7D).

1. Follow the steps in the browser to connect with your Github account
   
2. Enter the environment variables `LIVEKIT_API_KEY` , `LIVEKIT_API_SECRET` and `LIVEKIT_URL` by creating a [Livekit cloud account](https://cloud.livekit.io/) and a free project to get these values
   
3. Vercel will deploy a new project with these values
   
4. Open the deployment URL and test the voice agent. Click **Start call**. You will need wait for the sandbox to be created after which you can have a voice conversation with the agent
   

### Method 2: Clone and deploy

Clone and deploy from your terminal

`npx degit vercel/examples/guides/sandbox-voice-agent sandbox-voice-agent cd sandbox-voice-agent`

### Link your repository to a new project

Install the Vercel CLI if you haven't already:

`pnpm install -g vercel`

Run the following command to link your application with a new project and follow the prompts:

`vercel link`

Run the `vc env` to pull the value of `VERCEL_OIDC_TOKEN` locally. This will create your `.env.local` file.

`vercel env pull`

### Add Livekit environment variables

Obtain the `LIVEKIT_API_KEY` , `LIVEKIT_API_SECRET` and `LIVEKIT_URL` environment variables by creating a [Livekit cloud account](https://cloud.livekit.io/) and a free project to get these values.

Copy the contents of `env.example` below the Vercel sandbox section to `.env.local` in your Next.js project and update the values.

`# Vercel Sandbox VERCEL_OIDC_TOKEN=<your_vercel_OIDC_token> # LiveKit credentials - Required for voice agent functionality LIVEKIT_API_KEY=<your_api_key> LIVEKIT_API_SECRET=<your_api_secret> LIVEKIT_URL=wss://<project-subdomain>.livekit.cloud # Enable Vercel Sandbox mode NEXT_PUBLIC_USE_SANDBOX=true # Agent Configuration AGENT_REPO_URL=https://github.com/livekit-examples/agent-starter-python.git AGENT_RUNTIME=python3.13 SANDBOX_TIMEOUT=600000`

### Deploy and test your application

Test your application locally with:

`pnpm install pnpm dev`

Browse to `http://localhost:3000` and click **Start Call**. Wait for your sandbox to be ready and start talking to the voice agent!

Deploy using the Vercel CLI to test in a preview deployment:

`vercel vercel --prod // For production`

Open the deployment URL and test the voice agent. Click **Start call**.

## Understanding the code

### The sandbox manager

The Python agent runs inside Vercel Sandbox and is automatically cloned from the [agent-starter-python](https://github.com/livekit-examples/agent-starter-python) repository.

When you create a sandbox, it goes through several stages: creating, installing dependencies, starting the agent, and finally becoming ready. The sandbox manager tracks which stage each sandbox is in using an in-memory Map:

`class SandboxManager { private sandboxes: Map<string, SandboxInfo> = new Map(); register(sandbox: SandboxInfo): void { this.sandboxes.set(sandbox.id, sandbox); } get(sandboxId: string): SandboxInfo | undefined { return this.sandboxes.get(sandboxId); } updateStatus(sandboxId: string, status: SandboxStatus): void { const sandbox = this.sandboxes.get(sandboxId); if (sandbox) sandbox.status = status; } isExpired(sandboxId: string): boolean { const sandbox = this.sandboxes.get(sandboxId); return !sandbox || Date.now() > sandbox.expiresAt; } } export const sandboxManager = new SandboxManager();`

This pattern lets your frontend display progress updates while the sandbox initializes. Sandboxes expire after their configured timeout (default 10 minutes).

### The sandbox creation endpoint

This API endpoint creates a new Vercel Sandbox, clones the Python agent code, installs dependencies, and starts the agent. Since this takes 20-40 seconds, the endpoint returns immediately and continues setup in the background.

The key part is the `Sandbox.create()` call, which provisions the isolated environment:

`import { Sandbox } from '@vercel/sandbox'; export async function POST(req: Request) { const sandbox = await Sandbox.create({ source: { url: 'https://github.com/livekit-examples/agent-starter-python.git', type: 'git' }, runtime: 'python3.13', resources: { vcpus: 4 }, timeout: 600000, }); const sandboxId = sandbox.sandboxId; setupSandbox(sandbox, sandboxId).catch((error) => { sandboxManager.updateStatus(sandboxId, 'failed', error.message); }); return NextResponse.json({ success: true, sandboxId }); }`

The `setupSandbox` function runs in the background. It uses `sandbox.runCommand()` to execute shell commands inside the sandbox:

`async function setupSandbox(sandbox, sandboxId) { sandboxManager.updateStatus(sandboxId, 'installing'); await sandbox.runCommand({ cmd: 'pip', args: ['install', '--user', '.'] }); sandboxManager.updateStatus(sandboxId, 'starting'); await sandbox.runCommand({ cmd: 'python3', args: ['src/agent.py', 'start'], detached: true, env: { LIVEKIT_API_KEY: process.env.LIVEKIT_API_KEY, LIVEKIT_API_SECRET: process.env.LIVEKIT_API_SECRET, LIVEKIT_URL: process.env.LIVEKIT_URL, }, }); sandboxManager.updateStatus(sandboxId, 'ready'); }` The `detached: true` option keeps the agent running after the command returns. Environment variables pass LiveKit credentials to the agent process. ### The status polling endpoint The frontend polls this endpoint every second to check if the sandbox is ready: `export async function GET(req: Request) { const { searchParams } = new URL(req.url); const sandboxId = searchParams.get('sandboxId'); const sandbox = sandboxManager.get(sandboxId); if (!sandbox) { return NextResponse.json({ error: 'Not found' }, { status: 404 }); } return NextResponse.json({ sandbox, ready: sandbox.status === 'ready' && !sandboxManager.isExpired(sandboxId), }); }` The endpoint returns the full sandbox object including its current status `creating`, `installing`, `starting`, `ready`, or `failed`) and a `ready` boolean that indicates when the frontend should connect to LiveKit. ### Sandbox creation and management hook The React hook manages sandbox lifecycle on the frontend. It handles creating sandboxes, polling for status updates, and tracking progress through different states. The core pattern is a polling loop that checks the sandbox status every second until it's ready: ``export type SandboxCreationStatus = | 'idle' | 'creating' | 'installing' | 'downloading' | 'starting' | 'ready' | 'error'; export function useSandboxCreation() { const [status, setStatus] = useState<SandboxCreationStatus>('idle'); const [sandbox, setSandbox] = useState<SandboxInfo | null>(null); const createSandbox = useCallback(async () => { setStatus('creating'); const res = await fetch('/api/sandbox/create', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ config: {} }), }); const { sandbox: created } = await res.json(); for (let i = 0; i < 60; i++) { const statusRes = await fetch( `/api/sandbox/status?sandboxId=${created.id}` ); const { sandbox: current } = await statusRes.json(); if (current.status === 'ready') { setStatus('ready'); setSandbox(current); return current; } if (current.status === 'failed') { throw new Error(current.error); } setStatus(current.status); await new Promise((r) => setTimeout(r, 1000)); } }, []); return { status, sandbox, createSandbox }; }`` The hook also stores the sandbox ID in localStorage so users can resume their session after a page reload (within 10 minutes). See the full implementation in the [example repository](https://github.com/vercel/examples/tree/main/guides/sandbox-voice-agent) for session persistence and progress animation.

### Welcome view component

The welcome screen uses the hook to create sandboxes and auto-connects users when ready:

`'use client'; import { useEffect } from 'react'; import { useSandboxCreation } from '@/hooks/useSandboxCreation'; export const WelcomeView = ({ onSandboxReady }) => { const { status, sandbox, createSandbox } = useSandboxCreation(); useEffect(() => { if (status === 'ready' && sandbox) { onSandboxReady(sandbox.id, sandbox.roomName); } }, [status, sandbox, onSandboxReady]); if (status !== 'idle') { return <SandboxLoading status={status} />; } return ( <Button onClick={createSandbox}> Start call </Button> ); };` When the user clicks **Start call**, the component calls `createSandbox()`. The effect watches for the `ready` status and automatically triggers `onSandboxReady`, which connects the user to the LiveKit room to enable the conversation with the voice agent. ## Monitoring sandboxes After deploying your application, you can monitor active sandboxes in the Vercel dashboard. This helps you understand usage patterns, debug issues, and track costs. ### View active sandboxes Navigate to your project in the Vercel dashboard and click on the **Sandboxes** tab. Here you'll see: - **Active sandboxes**: Currently running sandbox instances    - **Sandbox ID**: Unique identifier for each sandbox    - **Created**: When the sandbox was created    - **Status**: Current state (running, stopped, or failed)    - **Runtime**: The runtime environment (e.g., python3.13)    - **Resources**: Allocated vCPUs and memory    ### Check sandbox logs Click on any sandbox to view its logs. This shows: - Standard output from your agent process    - Error messages if the agent fails    - Dependency installation progress    - LiveKit connection status    ## Best practices ### Optimize cold start time Agent initialization takes 20-40 seconds due to model downloads. Reduce this by pre-loading models when the agent server starts in the Python agent code: `def prewarm(proc: JobProcess): proc.userdata["vad"] = silero.VAD.load()` ### Customize the agent Fork the [agent-starter-python](https://github.com/livekit-examples/agent-starter-python) repository to customize:

- Voice model and personality
  
- Response latency settings
  
- Custom tools and function calling
  
- STT/TTS provider selection
  

Review the [LiveKit Voice Agent settings documentation](https://docs.livekit.io/agents/plugins/openai/) to see all available configuration options for speech recognition, language models, and text-to-speech.

Update `AGENT_REPO_URL` in your environment variables to use your forked repository.

## Conclusion

You've built a voice AI application that creates isolated agent environments on-demand using Vercel Sandbox. This architecture provides the benefits of cloud scaling while maintaining the performance characteristics needed for real-time voice conversations.

To learn more, explore

- [Vercel Sandbox API documentation](https://vercel.com/docs/sandbox)
  
- [LiveKit Agents framework](https://docs.livekit.io/agents/)

---

[View full KB sitemap](/kb/sitemap.md)