Bot Management
Bot Management lets you control how detected bot traffic uses your source’s rendering capacity. Rules stop new renders for matching requests; already-cached images may still be served by the CDN.
Enabling Bot Management
Bot Management is off by default and must be enabled per source. To enable it, open the source’s edit page and toggle Enable Bot Management in the Bot Management panel in the bottom-right of the page.
Once enabled, the full set of rules becomes available.
Category Rules
Category rules let you set a single rule for an entire group of bots, including any bots added to that category in the future. Each category has a dropdown with four options: No rule, Allow, Block, or Rate limit. You can view the full list of bots in each category in your source’s dashboard settings.
| Category | Description |
|---|---|
| AI training crawlers | Crawlers that collect images to train models. AI search engines and AI assistants are separate categories and are not affected. |
| Search engines | Crawlers operated by traditional search engines that index web content for search results (e.g. Googlebot, Bingbot, DuckDuckBot). |
| AI search engines | Crawlers used by AI-powered search products to build their indexes (e.g. PerplexityBot, Brave Search). |
| AI assistants | Bots that access content on behalf of AI chatbots and assistants to answer user queries (e.g. ChatGPT, Claude, Gemini). |
| SEO tools | Third-party crawlers used to audit, analyze, or track search performance (e.g. Ahrefs, Semrush, Screaming Frog). |
| Social previews | Bots that fetch pages to generate link previews on social platforms and messaging apps (e.g. Twitterbot, facebookexternalhit, Slackbot). |
| Monitoring | Automated clients that check availability and performance of web services (e.g. Pingdom, UptimeRobot, Datadog). |
| Service agents | Automated clients acting on behalf of legitimate third-party services, such as feed readers, RSS aggregators, and workflow integrations. |
| Suspected scrapers | Clients exhibiting scraping behavior that are not definitively identified as a known scraper. |
| Scraper fleets | Known scraper networks engaged in bulk data harvesting. |
Important notes:
- A named bot rule always overrides the bot’s category rule. If you set a rule for a specific bot under Bot Rules, that rule takes precedence over the category setting.
- A category rate limit is a shared budget across all bots in the category — it is not a per-bot limit.
Bot Rules
Bot rules let you override the category setting for a specific known bot. To add a rule, click + Add Bot Rule, select the bot from the list, and choose the action.
A named bot rule always takes precedence over the bot’s category rule.
Your Own User-Agents
You can define custom rules for clients not covered by the bot categories — for example, your own backend fetchers or scrapers you have identified yourself.
Rules are matched from the start of the User-Agent string and are case-insensitive. A prefix like MyCompany-Sync/ will match every version of that client. To add a rule, click + Add User-Agent Rule.
Custom User-Agent rules take precedence over all category and bot rules.
Fallback Rules
Fallback rules determine how Imgix handles traffic that does not match any of your category, bot, or User-Agent rules.
| Rule | Default | Options |
|---|---|---|
| All other bots | Allow | Allow / Block / Rate limit |
| Requests with no User-Agent | Allow | Allow / Block / Rate limit |
| Plain HTTP requests | Allow | Allow / Redirect to HTTPS / Block |
Rate Limits
Rate limits use rolling budgets with continuous refill. For example, a rolling 24-hour budget can be spent in a burst and refills gradually throughout the day. Budgets never reset at a calendar boundary.