Post by SEO

🧭 Cloudflare just handed publishers the biggest lever yet over who gets to train AI on their content. Cloudflare is splitting crawler permissions into three separate controls: Search for traditional indexing, AI Agent for real-time user-initiated fetches, and AI Training for crawling that trains models. For the first time you can say yes to being found and no to being trained on, or the reverse. The bigger news is the default. Starting September 15, AI training crawlers will be blocked by default across Cloudflare's network unless a site opts in, and that reportedly includes Google's own AI training crawls. Robots.txt never drew this line. Cloudflare is drawing it at the infrastructure layer, in front of a large share of the web. Here is the trap for SEO and GEO teams: the three categories are easy to conflate. If you block AI Training but accidentally also restrict Search or AI Agent, you can cut your AI Overviews and AI Mode citation traffic while thinking you only stopped model training. What to actually do: if your site sits behind Cloudflare, audit your bot settings before September 15. Confirm which of the three buckets each major AI crawler falls into, and set them deliberately rather than letting the new default decide for you. The wider shift is that control over AI access is moving from a text file you edit to a platform that sits in front of your site, and the defaults there now matter as much as your own choices. What is your take on this? 🤔💬 Want to stay ahead of the future of search? 🔔 Follow Search Shift for the daily signal on SEO, GEO, and AEO (trusted by 79,404 followers). 📬 Join 7,535 readers of our newsletter and get today's SEO and GEO moves decoded: https://lnkd.in/d7axEeXK