Cloudflare, which sits in front of a large share of the world’s websites, has changed how it manages AI bots. If your site uses Cloudflare, this affects whether AI tools like ChatGPT, Google AI Overviews, and Perplexity can read and cite your content, which matters for your visibility in AI search. Here is what changed in the July 2026 update, and what your business should do about it.
What changed
Cloudflare replaced its old all-or-nothing “block AI bots” switch with granular controls. Site owners can now manage three types of AI crawler separately:
- Search: crawlers that collect and index your content to answer questions about it later.
- Agent: automated activity acting in real time for a person, such as chat fetch bots and browser-use agents.
- Training: crawlers taking your content to train or fine-tune an AI model.
The key change to note: from 15 September 2026, Cloudflare sets new defaults for new domains. Bots classified as Training or Agent will be blocked on pages that display ads, while Search crawlers stay allowed. Mixed-purpose crawlers that combine Search and Training are blocked by any setting that blocks training.
Cloudflare is also evolving its Pay Per Crawl tool, which lets sites charge AI bots to scrape them, into Pay Per Use, so publishers can charge AI companies when their content actually creates value, not just when a bot fetches it.
Why this matters for your SEO and AI visibility
This is not just a security story. It is a visibility and monetisation decision.
- If you block AI crawlers, you may vanish from AI answers. As buyers increasingly ask AI tools to research and recommend brands, being blocked means AI cannot cite you. That undercuts your GEO and AI search visibility.
- If you allow everything, others may profit from your content. Training crawlers can use your work to build models without sending you traffic or payment.
- The new controls let you choose. You can keep Search crawlers allowed, so your content stays eligible for AI answers, while blocking or charging for Training and Agent access.
The practical takeaway: allow the crawlers that help you get discovered, and control the ones that only take. For most businesses that want AI visibility, that means keeping Search allowed.
The trade-off, in plain terms
Think of it as three questions:
- Do you want to appear in AI answers? Then keep Search crawlers allowed.
- Do you want to control or monetise how AI models train on your content? Then use the Training controls or Pay Per Use.
- Do you want AI agents transacting on your site in real time? Then decide on Agent access based on your business.
Blocking everything is the safe-looking option that quietly costs you AI visibility. Allowing everything gives away control. The middle path, now possible, is usually the right one.
What your business should do
- Check whether you use Cloudflare and what your current AI bot settings are.
- Keep Search crawlers allowed if AI visibility matters to you, which for most brands it does.
- Decide on Training and Agent access based on whether you want to protect or monetise your content.
- Align it with your GEO strategy. There is no point optimising to be cited by AI if your own settings block the crawlers that cite you.
- Review after 15 September 2026, when new defaults take effect, especially if you run ads on your pages.
Frequently asked questions
Does this affect my website?
Only if your site is behind Cloudflare. If it is, your AI bot settings decide whether AI tools can read and cite your content.
Will blocking AI bots hurt my SEO?
It does not directly affect classic Google rankings, but blocking Search-type AI crawlers can remove you from AI answers and AI-assisted search, which is a growing source of visibility.
Should I block AI crawlers?
Most businesses that want to be found should keep Search crawlers allowed. You can still block or charge for Training crawlers if you want to protect your content from model training.
What is Pay Per Crawl or Pay Per Use?
Cloudflare tools that let you charge AI companies for access to your content, moving from charging per crawl toward charging when your content creates value.
What happens on 15 September 2026?
Cloudflare sets new defaults for new domains: Training and Agent bots blocked on pages with ads, Search still allowed. Review your settings if this affects you.
Final thoughts
Cloudflare’s update turns a blunt on-off switch into a real choice: which AI systems can use your content, and on what terms. For any business that cares about being found in AI search, the sensible default is to keep Search crawlers allowed and control the rest. Do not let a bot setting quietly erase you from the AI answers your customers now rely on.
If you want to make sure your site is both protected and visible to AI search, our GEO and AI search team can align your crawler settings with your SEO strategy. Talk to MediaPlus Digital and claim a free RM300 audit of your AI and search visibility.
