Cloudflare June 2025: Workers KV failed, taking Workers AI and AI Gateway with it

Original: Cloudflare incident analysis, 12 June 2025. Start 17:52 UTC; impact end 20:28 UTC (2h 36m by those timeline timestamps; report text says 2h 28m; preserve the discrepancy rather than silently reconcile). Cloudflare attributed the failure to one third-party storage provider underpinning Workers KV, then used as a central store for cold reads and writes. The postmortem does not name that provider. Cached reads survived, but 90.22% of KV requests failed. Its AI inference hardware did not need to fail: globally distributed Workers AI required KV to distribute configuration and route inference requests, so all Workers AI inference requests failed in the incident window. AutoRAG was unavailable through Workers AI; AI Gateway error rates peaked at 97% until dependencies recovered.

Measured comparator and mechanism: Cloudflare’s CDN, DNS, cache, proxy and WAF were not directly impacted, nor was its v4 API. Access failed 100% of identity-based logins (not service-token, mTLS or bypass policies); it intentionally failed closed when it could not read policy or identity. This is an explicit availability–security trade-off, not evidence that disabling authentication is a safe fallback. After storage returned around 20:23 UTC, repopulating caches caused additional rate limits until 20:28 UTC. Cloudflare reported no data loss and no attack. Only availability, not downstream patient safety, productivity loss or provider-wide model outages, is measured here.

Cloudflare’s August 2025 architectural follow-up supplies the crucial comparator over time: KV formerly used two active-active storage providers; it had been deliberately reduced to one due to consistency/operations trade-offs, temporarily raising single-provider failure risk. Cloudflare says it subsequently moved all data and request serving onto its own infrastructure with other provider(s) for redundancy. Treat 2025 topology as historical, not proof the same bottleneck still holds in October 2026. Compare Google’s simultaneous independent original with explicit caveat that Cloudflare did not identify its storage supplier. The synthesis tests what failure propagation is established.