Bot Traffic Exceeds Humans: Agentic AI Drives 57.5% of Web

For the first time ever, automated systems now generate most of the requests on the internet. Cloudflare reports that machines account for 57.5% of HTTP requests worldwide, compared to 42.5% from people. This shift is driven by agentic AI, which are semi-autonomous programs browsing on behalf of AI assistants. For example, an AI assistant can hit thousands of pages for a task where a human might open only a few. Cloudflare's data shows crawler activity climbing sharply, with OpenAI's GPTBot growing 305% in a single year. This increased bot activity is already expensive for some organizations. The Wikimedia Foundation saw bandwidth for multimedia surge 50% since January 2024, largely due to scrapers harvesting data. Cloudflare is addressing this with new systems that use an HTTP 402 "Payment Required" status code as a negotiation signal for crawlers. This change impacts anyone who publishes online, pays for hosting, or uses AI assistants, as the web's economic models are being rewritten.