# ChatGPT finds pages with no links thanks to Cloudflare: what to fix on your site

> Cloudflare now flags new, changed and useless pages to ChatGPT: hundreds of millions indexed in the pilot. How the signals work and what to fix on your site.

URL: https://geosnap.ai/en/blog/chatgpt-trova-pagine-senza-link-segnali-cloudflare-cosa-fare
Language: English
Version IT: https://geosnap.ai/blog/chatgpt-trova-pagine-senza-link-segnali-cloudflare-cosa-fare
Author: Rinald Sefa, CMO Geosnap · Category: Guides
Publisher: Geosnap (Maind Group S.r.l.), https://geosnap.ai

## In short

On October 15, 2026 Cloudflare and OpenAI said a pilot on about 1% of Cloudflare's network let ChatGPT discover hundreds of millions of new pages, including useful pages with few inbound links such as manuals, opening hours and prices. The signals cover freshness, traffic quality and real changes; they grant no access to blocked pages and do not guarantee a citation. You cannot opt in yet, but you can let OAI-SearchBot through, remove generic redirects and duplicates, and keep the HTML current.

- The Cloudflare-OpenAI pilot, started in July 2026 on about 1% of the network, let OpenAI index hundreds of millions of new pages for ChatGPT search.
- The signals tell the crawler which pages exist, which changed and which to skip (missing addresses, homepage redirects, duplicates); they grant no permission to access blocked pages.
- The pages that benefit are useful but rarely linked: manuals, requirements and fees, menus and opening hours, FAQs and price lists.
- There is no sign-up form: Cloudflare picks the cohort and there is no date for a wider rollout, announced only in general terms for 2026.
- On your site: allow OAI-SearchBot even if you block GPTBot (about 24 hours to take effect), remove generic redirects, consolidate duplicate URLs and update the real content, not the date.

Written by the Geosnap AI agent, reviewed and approved by Rinald Sefa.

Translated from the Italian original. [Read the original](https://geosnap.ai/blog/chatgpt-trova-pagine-senza-link-segnali-cloudflare-cosa-fare)

Until now a crawler found a page in one way only: by following a link. A page with no inbound links, such as a product manual or a shop's opening hours, stayed invisible even when it answered a precise question. On October 15, 2026 Cloudflare and OpenAI published the first results of a project that changes that rule for ChatGPT: Cloudflare's network tells OpenAI's search crawler which pages exist, which have changed and which are not worth a visit. Here is what those signals do, what they do not promise, and what to fix on your site.

## What did Cloudflare and OpenAI announce?

A research pilot, started in July 2026 on a cohort of about 1% of Cloudflare's network, that let OpenAI [discover and index hundreds of millions of new web pages](https://blog.cloudflare.com/cloudflare-openai-discovery-agentic-web/) for ChatGPT search. The original announcement dates from [July 8, 2026](https://marketchameleon.com/PressReleases/i/2333779/NET/cloudflare-announces-research-pilot-with-openai): Cloudflare, which sits in front of more than 20% of the web, shares "real-time network insights" with OpenAI on three things: content freshness, traffic quality and actual page changes. OpenAI brings its models, its search system and real user queries.

The key point is the kind of page that surfaces. The October 15 post gives three examples: [the manual for a specific washing machine, the current requirements and fees for a passport, a restaurant's latest menu and opening hours](https://blog.cloudflare.com/cloudflare-openai-discovery-agentic-web/). Pages that help someone with a question but have few links from other sites, which a classic crawler reaches late or never. One common mix-up to avoid: 20% is the share of the web behind Cloudflare; the pilot involved about 1% of that network.

## How do network signals work for ChatGPT's crawler?

They do three things: point to new pages worth visiting, say when a known page has changed, and help skip useless requests. Cloudflare describes [change signals](https://blog.cloudflare.com/cloudflare-openai-discovery-agentic-web/) that tell the crawler "when a page is worth another visit, and when it isn't". On the other side, it filters the traps that inflate crawl work: addresses that do not exist, sites that redirect almost everything to the homepage, different URLs serving the same content. Signals that flag potentially harmful pages are also being tested, for example a fake login form planted on one page of a legitimate site: the aim is to keep out that page, not the whole domain.

Two limits are stated plainly. First: ["a signal about a page doesn't grant permission to access it"](https://blog.cloudflare.com/cloudflare-openai-discovery-agentic-web/). The domain owner's choices (robots.txt, blocks, Cloudflare itself) apply as before. Second: the project uses [metadata about the web for search and discovery](https://blog.cloudflare.com/cloudflare-openai-discovery-agentic-web/), rather than sharing page content for model training. And once discovered, pages are evaluated and ranked by OpenAI independently: being found is not being cited.

| Signal | What it tells the crawler | Effect on your site |
| --- | --- | --- |
| Content freshness | The page changed since the last visit | Updates reach the index sooner |
| Traffic quality | Visits come from people, not bots | Pages with real traffic but few links become candidates |
| Actual page changes | What really changed in the served content | Fewer repeat visits to pages that stand still |
| Error and duplicate filters | Missing address, generic redirect, copy | The crawler stops wasting requests on empty pages |

## What changes for a site with few linked pages?

Its "service" pages can enter ChatGPT's index sooner, without depending on links. Think of a car repair shop in Leeds: the page with service prices, the FAQ on vehicle pickup, a page for each service. Nobody links to them, but they answer questions people ask ChatGPT. If the site sits behind Cloudflare and is included, freshness and traffic signals can bring them up.

The flip side is practical. Cloudflare warns that [finding new pages can initially increase crawling](https://blog.cloudflare.com/cloudflare-openai-discovery-agentic-web/) while they are evaluated and indexed: if your server is near its limit, you will notice in the logs. And indexing does not translate into visibility by itself: as [Search Engine Journal](https://www.searchenginejournal.com/chatgpt-search-can-draw-on-more-pages-after-a-cloudflare-pilot/593720/) notes, neither the post nor the July announcement says which sites took part, how many of the discovered pages are then cited in answers, or how many visits they sent. No figures on crawl volume or server load either.

## Is my site in? How do I join?

You cannot sign up today: the pilot used a cohort chosen by Cloudflare, and none of the published documents describes an opt-in for an individual site. [PPC Land](https://ppc.land/cloudflare-gives-openai-network-signals-covering-20-of-the-web/) had already noted this in July: no transmission protocol, no opt-in, no price or timeline. In a July 1 post Cloudflare stated its intention to [provide broad access to this capability during the rest of 2026](https://www.searchenginejournal.com/chatgpt-search-can-draw-on-more-pages-after-a-cloudflare-pilot/593720/), but the October 15 post gives no date. The next phase, the two companies write, will improve page selection, test faster change signals and measure what happens after discovery.

What you can do is make sure the door is open when the crawler arrives. In the Cloudflare dashboard, [AI Crawl Control](https://developers.cloudflare.com/ai-crawl-control/) is available on all plans: it shows which AI services access your site, with the number of requests and robots.txt violations per crawler, and lets you choose [Allow or Block](https://developers.cloudflare.com/ai-crawl-control/features/manage-ai-crawlers/index.md) for each one. On the free plan crawlers are identified by user-agent only, which is enough for self-declared ones such as OAI-SearchBot. A block answers with 403, or 402 if you want to signal that access is paid.

## What to fix so discovered pages actually get indexed

Five checks, in order of importance.

1. **Let OAI-SearchBot through, even if you block GPTBot.** OpenAI documents [two independent robots.txt directives](https://developers.openai.com/api/docs/bots): OAI-SearchBot decides whether you appear in ChatGPT search answers, GPTBot concerns model training. You can allow the first and disallow the second. A site that blocks OAI-SearchBot does not appear in answers, at most as a navigational link. After a robots.txt change it takes about 24 hours for OpenAI's systems to adjust. If you run a firewall, also check the IP ranges OpenAI publishes (searchbot.json).
2. **Remove generic redirects to the homepage.** If a non-existent address sends visitors to the homepage with a 200 or a 301, the crawler reads an "interesting" page that does not exist. That is one of the very patterns Cloudflare now filters: a clean 404 is better.
3. **One content, one address.** Tracking parameters, versions with and without a trailing slash, capital letters: every duplicate is a wasted request and a confusing signal. Use redirects and canonicals.
4. **Really update the pages that matter.** The freshness signal measures changes in the served content, not the date printed at the bottom. Prices, hours, availability and FAQs need to be kept current in the HTML.
5. **Check that the content is in the initial HTML.** Being discovered is useless if the crawler finds an empty page because the text arrives via JavaScript: [here is how to verify it](https://geosnap.ai/en/blog/sito-javascript-chatgpt-vede-contenuti-come-verificarlo).

If the site is already fine on these points, what remains is the usual work on answer quality: the page has to state directly what the person is looking for, with explicit data, prices and conditions. For robots.txt and firewall details see the [guide to AI crawlers](https://geosnap.ai/en/blog/il-tuo-sito-blocca-chatgpt-guida-crawler-ai); in its technical analysis Geosnap also checks robots.txt and whether your brand appears in ChatGPT's answers for the questions you monitor, which is the most concrete way to know whether discovered pages are then used.

## Sources

- [Cloudflare and OpenAI: Making AI discovery better for everyone](https://blog.cloudflare.com/cloudflare-openai-discovery-agentic-web/) (Cloudflare, 2026-10-15)
- [ChatGPT Search Can Draw On More Pages After A Cloudflare Pilot](https://www.searchenginejournal.com/chatgpt-search-can-draw-on-more-pages-after-a-cloudflare-pilot/593720/) (Search Engine Journal, 2026-10-20)
- [Cloudflare Announces Research Pilot with OpenAI](https://marketchameleon.com/PressReleases/i/2333779/NET/cloudflare-announces-research-pilot-with-openai) (Cloudflare (comunicato Business Wire), 2026-07-08)
- [Cloudflare gives OpenAI network signals covering 20% of the web](https://ppc.land/cloudflare-gives-openai-network-signals-covering-20-of-the-web/) (PPC Land, 2026-07-12)
- [Overview of OpenAI Crawlers](https://developers.openai.com/api/docs/bots) (OpenAI Developers, 2026-10-22)
- [AI Crawl Control](https://developers.cloudflare.com/ai-crawl-control/) (Cloudflare Docs, 2026-08-14)
- [Manage AI crawlers](https://developers.cloudflare.com/ai-crawl-control/features/manage-ai-crawlers/index.md) (Cloudflare Docs, 2026-07-28)

## Frequently asked questions

### Does my site need to be behind Cloudflare to be found by ChatGPT?

No. Network signals are an extra discovery channel, not the only one: OAI-SearchBot keeps following links and reading pages as before. If your site does not use Cloudflare, clear internal links to your service pages and a robots.txt that lets the search crawler through matter even more.

### Can I enrol my site in the Cloudflare-OpenAI pilot?

Not at the moment. The pilot used a cohort of about 1% of Cloudflare's network, chosen by the company, and no published document describes an opt-in mechanism. Cloudflare has said it intends to widen access during 2026, with no date. In the meantime you can check in AI Crawl Control that OAI-SearchBot is not blocked.

### If a page is discovered through the signals, will ChatGPT cite it?

Not necessarily. Cloudflare writes that OpenAI evaluates and ranks pages independently after discovery, and neither the post nor the July announcement reports how many discovered pages are then cited or how many visits they send. Discovery removes an obstacle; the citation depends on how well the page answers the question.

### Does Cloudflare pass the content of my pages to OpenAI?

According to the October 15, 2026 post, the project uses metadata about the web for search and discovery and does not share page content for model training. A signal about a page does not grant permission to access it: if you block a crawler in robots.txt or in Cloudflare, the block still applies. For training, the directive to manage is GPTBot's.

### How do I know whether ChatGPT actually uses my pages?

Server logs or AI Crawl Control show OAI-SearchBot requests, that is, crawling. For the part that matters, the answers, you need to ask ChatGPT your customers' questions and see whether and how your brand appears and which sources are cited. Geosnap runs this check repeatedly on ChatGPT, Gemini and Google AI Mode for the prompts you choose to monitor.