WebCrawlerAPI
Use WebCrawlerAPI with AI apps and agents through Vercel Connect. Follow a complete setup guide for secure runtime access.
Install the skill
$ npx skills add vercel/vercel-plugin --skill vercel-connectConnect to
Use its data in agents(opens in new tab) and apps(opens in new tab)
import { getToken } from '@vercel/connect';const token = await getToken('webcrawlerapi/acme-webcrawlerapi');
Use WebCrawlerAPI in apps and agents
Crawl and scrape websites, run web agents, and monitor website feeds.
Read the WebCrawlerAPI developer docs(opens in new tab)Why use Vercel Connect with WebCrawlerAPI?
Credentials on demand
Request WebCrawlerAPI credentials only when your application or agent needs them instead of copying secrets into application code.
Access for the right subject
Keep app-scoped WebCrawlerAPI access behind a reusable connector controlled by your team.
Centralized credential management
Manage WebCrawlerAPI credentials in one place and retrieve them through a consistent runtime API.
Vercel-native controls
Bind connections to the projects and environments that need them while Vercel OIDC authenticates each runtime request.
Use WebCrawlerAPI with your stack
Follow a complete setup for the Connect SDK or a supported agent framework.
Add WebCrawlerAPI to an existing Vercel application with the Connect SDK.
Install the Connect SDK
npm install @vercel/connectCreate the WebCrawlerAPI connector
Run these commands from the project that will use the connection.
vercel linkvercel connect create webcrawlerapi --name acme-webcrawlerapivercel env pullRequest WebCrawlerAPI credentials
app/api/route.tsimport { getToken } from '@vercel/connect';const token = await getToken('webcrawlerapi/acme-webcrawlerapi');
