Vercel Logo

Turn on Bot Protection

Three scripts still have the run of the truck. Pinchy's disguised scraper reads today's spot every morning, the preorder bot waits out the rate limit and takes 10 more, and the fake-order bot keeps ordering lunch for totally.real people. None of them is a browser, and two are pretending to be one. Let's turn on the layer that checks.

Outcome

Bot Protection challenges the disguised scraper and both bots while your browser gets straight in, and the AI Bots ruleset denies AI crawlers and allows AI assistants.

Two managed rulesets

So far every rule has been one we wrote. The Firewall also ships managed rulesets that Vercel writes and keeps up to date. Two of them are on every plan, and neither was acting on traffic when you checked in 1.2 (Bot Protection said Off, AI Bots said Allow):

  • Bot Protection challenges traffic that's unlikely to be a browser, including a script that claims to be Chrome. It skips verified bots like search engine crawlers automatically.
  • AI Bots acts on known AI bots in two categories you set separately: AI Crawlers, which collect content for training, and AI Assistants, which fetch pages because a person asked them to.

Managed rulesets run after your custom rules. A request that Pinchy's scraper denies never reaches them.

Bot Protection, on log

Same habit as 2.1: log first, then act. Managed rules are edited, never added:

vercel firewall rules update bot-protection --action log --yes
vercel firewall publish --confirm-name claw-and-order
export SINCE=$(date -u +%Y-%m-%dT%H:%M:%SZ)

Wait out the Preorder rush window if you ran the bot in the last 10 minutes. Then restock, send all three scripts, and load the home page in your browser:

npm run restock
npm run scalper -- scrape --target "$URL" --count 5 --disguise
npm run scalper -- preorder --target "$URL" --count 20
npm run scalper -- order --target "$URL" --count 5

Everything still succeeds, because log never blocks. Read what Bot Protection would have done:

vercel firewall traffic list --since "$SINCE" --action log --dimension path --dimension user-agent

All three scripts are there: /location, /api/preorder, and /api/order. Two of them send a Chrome user agent, and Bot Protection logged them anyway. Your browser isn't in the list. That's the result we wanted before switching it on.

Bot Protection, on challenge

The log looks clean, so switch it on:

vercel firewall rules update bot-protection --action challenge --yes
vercel firewall publish --confirm-name claw-and-order

Send the disguised scraper again:

npm run scalper -- scrape --target "$URL" --count 5 --disguise

Challenged. A browser would solve the JavaScript challenge without the person noticing. A Node.js script can't, so Pinchy's gets a challenge page instead of today's spot. The user agent they wrote doesn't help. Bot Protection judges how the client behaves, and a borrowed name doesn't change that.

The preorder bot and the order bot do no better. Wait out the Preorder rush window, then send them both:

npm run scalper -- preorder --target "$URL" --count 20
npm run scalper -- order --target "$URL" --count 5

Preorder rush still counts the preorder bot's first 10, then Bot Protection challenges the ones the rate limit let through. The order bot never reaches /api/order. Load the home page and the truck still has its rolls.

AI Bots: which ones are customers

Marguerite runs the office at Harborview Insurance. On Thursdays she tells her AI assistant to "order lobster rolls for the team, 30 of them, pickup at noon," and goes to a meeting. That assistant is a bot, and it's a customer. A crawler that copies the menu into someone's training data is a bot too, and it has never once ordered lunch.

The AI Bots ruleset lets you treat them differently. Its default is Allow for both. We'll deny crawlers and keep assistants allowed:

vercel firewall rules update ai-bots --category ai-crawlers --action deny --yes
vercel firewall rules update ai-bots --category ai-assistants --action allow --yes
vercel firewall diff
vercel firewall publish --confirm-name claw-and-order

Which bots count as customers is a business call. A site that wants to show up in AI-generated answers might allow both. The bots.fyi catalog lists which bots land in each category.

We skipped log mode here, and only because there's nothing to log yet. Verified AI bots are recognized by where they connect from and how they sign requests, so a script on your laptop can't pretend to be one. On a site with real traffic, set the category to Log for a day and read the verified bots list before you deny:

vercel firewall rules update ai-bots --category ai-crawlers --action log --yes
vercel firewall publish --confirm-name claw-and-order
vercel firewall traffic list --since 24h --dimension bot

Hands-on exercise 4.1

Turn on both managed rulesets, the careful way.

Requirements:

  1. Set Bot Protection to log and publish.
  2. Restock, then run the disguised scraper, the preorder bot, and the order bot. Load the site in your browser.
  3. Confirm Bot Protection logged all three scripts and not your browser.
  4. Set Bot Protection to challenge and publish. Confirm the disguised scraper, the preorder bot, and the order bot are challenged.
  5. Set AI Bots to deny AI crawlers and allow AI assistants, and publish.
  6. Snapshot your rules and commit.

Implementation hints:

  • vercel firewall bot-management prints the current action for each managed ruleset.
  • rules add, remove, enable, disable, and move refuse the managed slugs. Use rules update <slug> --action.
  • The scripts send small counts here on purpose. You're checking what gets logged, and the truck keeps most of its rolls.

Try It

Check the managed settings and your custom rules together:

vercel firewall bot-management
vercel firewall rules list

Bot Protection is on challenge, AI Bots denies crawlers and allows assistants, and your custom rules are still Pinchy's scraper and Preorder rush.

Now the before and after for Pinchy's:

npm run scalper -- scrape --target "$URL" --count 3 --disguise
curl -s -o /dev/null -w "curl %{http_code}\n" "$URL/location"

The scraper is challenged. curl is challenged too, since curl isn't a browser either. Open /location in your browser and the truck loads with no challenge page in sight.

Section 5 puts this on log for drills

Every script in attack/ is now challenged, including the friendly ones you'll meet later. Section 5 switches Bot Protection back to log for its drills, then back to challenge at the end. In production, leave it on challenge.

Commit

vercel firewall rules list --json > firewall/rules.json
git add firewall/rules.json
git commit -m "chore(firewall): Bot Protection on challenge, deny AI crawlers" \
  -m "Logged first: only the attack scripts matched. AI assistants stay allowed; they place real orders."

Done-When

  • Bot Protection logged the three scripts and not your browser before you switched it
  • Bot Protection is set to challenge
  • The disguised scraper is challenged on /location
  • The preorder bot and the order bot are challenged
  • AI Bots denies AI crawlers and allows AI assistants
  • Your browser loads the site without a challenge page
  • firewall/rules.json is committed

Troubleshooting

Your browser gets a challenge page on every visit. Try a private window with extensions turned off. If you've put a reverse proxy like another CDN in front of the deployment, Bot Protection can't read the signals it needs, so remove the proxy or leave the ruleset on log.

The log report doesn't show the order bot. Custom rules run first, and a request a custom rule acted on doesn't reach the managed ruleset. Check that nothing in vercel firewall rules list matches /api/order. Then widen the window with --since 15m.

rules update ai-bots complains about the category. Check the spelling: ai-crawlers and ai-assistants, plural. If it says --category is an unknown option, your CLI is older than the flag. Update it with npm i -g vercel@latest, as in Deploy the Truck, and run the command again.

Solution

Both rulesets, configured and published:

# Bot Protection: log, read, challenge
vercel firewall rules update bot-protection --action log --yes
vercel firewall publish --confirm-name claw-and-order
export SINCE=$(date -u +%Y-%m-%dT%H:%M:%SZ)
npm run restock
npm run scalper -- scrape --target "$URL" --count 5 --disguise
npm run scalper -- preorder --target "$URL" --count 20
npm run scalper -- order --target "$URL" --count 5
vercel firewall traffic list --since "$SINCE" --action log --dimension path --dimension user-agent
vercel firewall rules update bot-protection --action challenge --yes
vercel firewall publish --confirm-name claw-and-order
npm run scalper -- scrape --target "$URL" --count 5 --disguise
# wait out the Preorder rush window, then:
npm run scalper -- preorder --target "$URL" --count 20
npm run scalper -- order --target "$URL" --count 5
 
# AI Bots: deny crawlers, allow assistants
vercel firewall rules update ai-bots --category ai-crawlers --action deny --yes
vercel firewall rules update ai-bots --category ai-assistants --action allow --yes
vercel firewall publish --confirm-name claw-and-order
 
vercel firewall rules list --json > firewall/rules.json
git add firewall/rules.json
git commit -m "chore(firewall): Bot Protection on challenge, deny AI crawlers"

Pinchy's can call its scraper anything it likes. It still has to act like a browser to get past the door.

Was this helpful?

supported.