Cloudflare added a Web Search API to AI Gateway on October 2, 2026, with Ceramic.ai, Exa and Linkup as the first providers. This article explains what changed, where web retrieval fits in an AI-agent stack, and what AiFolder plans to test independently. Benchmark status: pending.
Why this matters for AI agents
An agent that needs current information has a retrieval problem before it has a reasoning problem. A strong model can still fail when it starts from stale training data or weak search results. Cloudflare says the API supports REST and Workers bindings. Requests can use AI Gateway credits, appear in observability logs and inherit gateway access controls. The company says provider pricing is passed through at list API pricing without additional markup, and BYOK is supported. Ceramic, Exa and Linkup share an integration point, but retrieval quality still needs independent measurement. A useful evaluation should separate freshness, relevance, source quality, coverage, latency and cost.
Retrieval begins with an answerable question
A search call is most useful when the agent knows which fact is missing. Consider an assistant comparing two documentation pages after an API release. Searching for the product name alone may produce a landing page, announcements, tutorials, and outdated examples. A better query describes the missing fact: the endpoint, the option name, and the relevant version. The application should retain that query and the retrieval timestamp, because a later reader needs to understand which evidence the answer was built from.
Cloudflare's current provider documentation lists provider selection and defaults; verify those settings rather than assuming every request uses the same search backend. The gateway can give your application one integration surface, but a consistent endpoint does not make the returned evidence interchangeable. A query about current limits, a search for one exact page, and broad background research have different acceptance conditions. For the API example, success means finding the authoritative documentation that actually contains the option, not merely returning many plausible-looking links.
Separate discovery, reading and synthesis
A result snippet can help decide which page to open. It should not substitute for reading the passage that supports a precise claim. Build retrieval as three stages: discover candidate pages, obtain useful source content, and synthesize an answer constrained by that content. Preserve the URL and relevant passage through the pipeline. If a page is unreachable, stale, or contradictory, expose that uncertainty instead of silently treating another snippet as equivalent evidence.
For a hypothetical pricing question, the agent should favor the vendor's current pricing documentation over an old announcement and distinguish an introductory rate from the ordinary rate. For a technical migration question, an official changelog may establish when a feature arrived while the reference documentation defines how to configure it. Those sources perform different jobs. Compare search providers on your actual query set using evidence relevance and reproducibility, with the same downstream reader. This article does not provide a measured provider ranking; any diagrams here represent the proposed retrieval workflow.
Architecture, privacy and testing
A practical research-agent flow is: user question → query planner → web search → source filter → model → cited answer. Search grounding reduces hallucination risk, but it does not make every retrieved page trustworthy. Technical claims should prefer primary documentation and preserve source provenance. Cloudflare says participating providers must meet its verified-bot requirements, respect robots.txt and return the source location. AI Gateway will identify providers supporting Zero Data Retention. Teams handling private queries should still inspect each provider's own policy. Cloudflare AI Search is designed for managed retrieval over content you index. Web Search API reaches the live public web. In simple terms: AI Search is your corpus; Web Search is the Internet.
Planned AiFolder test
AiFolder plans to run the same query set against each provider across fresh news, official API documentation, niche developer questions and ambiguous research queries. We will record raw responses, latency, duplicate domains and source relevance. Benchmark status: pending. No provider winner is claimed here. Related AiFolder reading: AI Agent Permissions Explained, Best AI Tools for Research in 2026, and the AiFolder AI Hub. These map to existing internal routes and should be linked by the site renderer.
Put a limit around research loops
Search agents can keep issuing slightly different queries without moving closer to an answer. Give a task a retrieval budget and a stop condition. In the API example, stop when the authoritative page and a working configuration example have been found, or when the budget has been exhausted with the missing fact clearly stated. Record errors separately from empty results: a rate-limit response says something different from a search that returned no useful page.
Evaluate the integration with several deliberately awkward cases: a renamed feature, a deprecated endpoint, conflicting release dates, and a query whose answer is absent. The desired behavior on the last case is to admit the gap, not fill it with a confident guess. Gateway observability can help attribute calls and failures, but avoid logging private search terms or full confidential documents unnecessarily. A small structured record of queries, provider, status, sources selected, and final confidence gives operators useful evidence without turning search logs into a second sensitive content store. A saved evidence record also lets an editor refresh the answer when a source changes, without guessing which passage originally supported it.
Sources
Cloudflare Blog — “Introducing Web Search API via AI Gateway,” October 2, 2026.
Practical takeaway
Cloudflare Web Search API turns live search into an infrastructure primitive for agents. The final provider choice should be based on measured retrieval quality, privacy requirements and total query cost.
