Lorixen.
EN

Bot protection blocking your webhooks, app or API? Fix it safely

Short answer: Bot protection works by challenging suspicious visitors with a check that a real browser can pass. Payment providers sending webhooks, your own mobile app, uptime monitors and partner integrations aren't browsers, so they fail the challenge and their requests never reach your site. Find the blocked requests in your security event log, then exempt those specific paths or sources with a narrow rule, such as skipping bot checks for /webhooks/stripe from the provider's published IP addresses, while keeping protection on everything else. Some simple bot modes can't be exempted at all; if so, you may need to switch them off and use rules you can control.

What breaks, and how it shows up

Turning on a "block bots" or "bot fight" switch at your CDN or firewall feels like an easy win. A day later:

  • Payment webhooks fail. Orders are paid but never marked as paid in your shop, because the provider's notifications are being challenged. The provider's dashboard shows failed deliveries with 403 or 503 responses.
  • The mobile app stops loading data. Its API calls get an HTML challenge page instead of JSON.
  • Integrations go quiet. Accounting, shipping or CRM connections that call your API start erroring.
  • Uptime monitors report outages that customers don't see.

Browsers mostly get through, so the website itself looks fine, which is why this can go unnoticed for days.

Why non-browser traffic fails

A challenge asks the client to run a script or show it behaves like a real browser. A server sending a webhook can't do that. Cloudflare's documentation for its Bot Fight Mode says plainly that although these products are designed to fight malicious actors, they "may challenge API or mobile app traffic". It also notes that Bot Fight Mode protects entire domains and can't be customised or adjusted with custom WAF rules.

Other CDNs and firewalls behave similarly when bot protection is set to challenge.

Step 1: confirm what's being blocked

  1. Open your CDN or firewall's security events log.
  2. Filter by the time of a failed webhook or app error.
  3. Look at the path, source IP and user agent of challenged or blocked requests.
  4. Compare with the webhook delivery log in your payment provider's dashboard.

You'll usually see the provider's IP addresses being challenged on your webhook URL, or your app's requests to /api/ being challenged.

Step 2: exempt narrowly

Avoid turning everything off. Create the smallest exception that lets legitimate traffic through:

  • By path: webhook endpoints such as /webhooks/payments and API paths such as /api/v1/.
  • By source, where published: Stripe, for example, publishes the IP addresses its webhooks come from and recommends making sure webhook events originate from them. Combine path and source in one rule.
  • By method: webhooks are POST requests; API reads may be GET.

Cloudflare's WAF documentation describes custom rules with a Skip action, intended for cases like allowing a trusted API client through rate limiting rules. It lists several security features that can be skipped, including rate limiting, managed rules and Super Bot Fight Mode rules, but notes you cannot skip Bot Fight Mode on the free plan. If you're using that mode, your choices are to switch it off or move to a plan or setup where exceptions are possible.

Step 3: keep the endpoints protected another way

Exempting a path from bot checks doesn't mean leaving it open:

  • Verify webhook signatures. Every major payment and platform provider signs webhooks. Reject anything without a valid signature. This is your real defence, and it works regardless of IP addresses.
  • Authenticate API calls from your app with tokens, and rate-limit per token rather than by bot score.
  • Return quickly from webhook handlers and do heavy work in the background.

Step 4: recover what was missed

Once traffic flows again, go back to the provider's webhook log and resend failed events, or run a reconciliation that compares paid orders in the provider's dashboard with your shop's order statuses. Check integrations that may have skipped records during the outage.

When the provider doesn't publish IP addresses

Many services send webhooks from changing cloud addresses and don't publish a list. In that case, exempt by path and method only, keep the path hard to guess where the provider allows a custom URL, and rely on signature verification and a rate limit on that path. Log every rejected signature, so a spike of failures tells you someone is probing the endpoint.

Prevent it next time

  • Before enabling any bot or WAF setting, list every non-browser client that talks to your site: payment and shipping webhooks, mobile apps, partner APIs, monitoring, feed fetchers for marketplaces and search.
  • Test in "log only" or simulate mode first, if your provider offers it, and review what would have been blocked.
  • Set an alert for repeated webhook failures in your payment provider.

A worked example

An illustrative Nelson outdoor gear shop switched on its CDN's free bot protection after a wave of fake sign-ups. Two days later, customers emailed asking why their paid orders showed as "pending". The payment provider's dashboard showed webhook deliveries failing with 403 responses, and the CDN's event log showed them being challenged. Because that bot mode couldn't be exempted, the shop turned it off and instead added a rate limit and challenge rule on the sign-up page only, plus a skip rule for the webhook path from the provider's IP addresses. They resent the failed webhooks, and orders updated correctly.

FAQ

Is it safe to exempt webhook URLs?

Yes, if your code verifies each webhook's signature. That check matters more than where the request came from.

Why does the website work but the app doesn't?

Browsers can pass challenges; apps and servers usually can't.

Should I allowlist the provider's IPs only?

It's a useful extra layer, but IP lists change. Signature checks are the dependable part.

Will this affect search engines?

Major search crawlers are usually recognised by bot protection, but check your search tools for crawl errors after changes.

If bot protection is breaking your payments, app or integrations, tell us what changed and what's failing and we'll help you set safe exceptions.

Sources

WhatsAppBook a call