Cloudflare added more nuance to their AI bot controls to give users more ways to manage AI traffic:

• Search - Bots that scan your site to help it appear in search engine results
• Training - Crawlers that scrape your content to train AI models
• Agent - Bots that retrieve info from your site to answer user questions

The options across all 3 use cases are: Allow, Block, or Block on pages with ads. The reasoning is that ads are meant to be seen by people, and this monetization funds the business that publishes the content.

Starting September 15, 2026 there will be new default settings for these controls:

• Training & Agent will be blocked by default on pages that display ads
• Search will be allowed by default
• Multipurpose crawlers that combine Search + Training (e.g. Googlebot, Applebot, Bingbot) will be blocked if the user blocks Training

They're also creating bot management capability based on content use, with three settings:

• Immediate - go ahead and interact, but you can't store or reuse anything
• Reference (default) - index, excerpt, and reference back
• Full - go ahead and summarize/reproduce my content

These content use values can be combined with the bot classifications to refine the rules to your liking.

Cloudflare is currently testing a new `use` signal that will live in your robots.txt but it will only signal a preference, it won't block things.

There's more in this article too, about verified bots and "transitive trust" and things for the future they are thinking about.