Is Your Brewery Website Invisible to ChatGPT? Signs It's Not AI-Ready

OpenAI runs three different crawlers and they do three different jobs. One trains models, one builds the index ChatGPT searches, and one fetches your page live when somebody asks about you. Blocking the training crawler is a reasonable choice that costs you nothing in ChatGPT answers. Blocking the search crawler takes you out of them. Across a sample of 4,242 robots.txt files, the training crawler was blocked outright on 8.11% of sites while the search crawler was allowed more often than it was blocked, which means the distinction is real and plenty of sites still get it backwards. This guide is for brewery operators who want to check, in about twenty minutes, whether ChatGPT can actually reach their site.

Introduction: Before You Rewrite Anything, Find Out Whether the Door Is Locked

The brewery version of this question almost always arrives the same way. Somebody types their own taproom into ChatGPT, gets a vague answer or a competitor, and concludes the website needs rewriting. Sometimes it does. But rewriting a page that no crawler can reach is an expensive way to change nothing, and the version BeerSoft runs into most often is a brewery a month deep into content work before anyone has opened the robots.txt file.

So this is a retrieval guide rather than a content one. It asks whether ChatGPT can fetch your pages, not whether it likes them. Those are separate questions with separate fixes, and only one of them can be answered with certainty today.

That last part matters more than the AI-readiness checklists usually admit. A lot of what circulates as settled fact about AI crawlers rests on a single study that did not test the thing it is quoted for. We will get to which parts hold up and which do not, because knowing the difference is what stops you spending a quarter on the wrong work.

Which OpenAI Bot Are You Actually Blocking?

Probably not the one you think, and this is the only item on the list that is documented rather than inferred. OpenAI publishes four crawlers, and the three that matter here have genuinely different jobs.

GPTBot crawls content that may be used to train OpenAI's foundation models. OpenAI's own wording is that disallowing it "indicates a site's content should not be used in training generative AI foundation models." That is a legitimate business decision and it is the one most publishers have made.

OAI-SearchBot builds the search index behind ChatGPT's search features. OpenAI recommends allowing it, and says that opting out keeps a site out of ChatGPT search answers, though it may still appear as a navigational link. Changes to robots.txt take around 24 hours to register.

ChatGPT-User is the one that fetches your page live when ChatGPT browses on someone's behalf. OpenAI says it is "not used for crawling the web in an automatic fashion," and because the action is user-initiated, robots.txt rules may not apply to it at all.

Here is the trap. These three names all look like the same thing to anyone pasting a block list into robots.txt, and a brewery that decided "we don't want AI training on our beer descriptions" very reasonably blocks everything with an OpenAI name on it. Blocking GPTBot gets you exactly what you wanted. Blocking OAI-SearchBot in the same stroke removes you from the answers, which is almost never what anyone intended.

The wider web has mostly worked this out. In a snapshot of 4,242 robots.txt files taken on 28 September 2026, GPTBot appeared in 520 Disallow rules against 326 Allow rules, a ratio of 1.60 to 1, and was fully disallowed on 8.11% of sites. OAI-SearchBot ran the other way, 268 Disallow against 295 Allow, a ratio of 0.91 to 1. Publishers are blocking the training crawler and letting the answering crawler through. The question is whether your site is on the right side of that split, and the only way to know is to look.

If you want to check one thing after reading this, make it this one. Open yoursite.com/robots.txt in a browser and search the page for "OAI-SearchBot" and "GPTBot". Twenty seconds, no tools, and it settles the most consequential item on the list.

Key Takeaway: Blocking GPTBot and allowing OAI-SearchBot is a coherent position: no training use, full presence in answers. Blocking both is the common accident, and it is a two-line fix.

Did Your CDN Make This Decision Without You?

Possibly, and this is the version of the problem that no amount of content work will fix. On 1 July 2025, Cloudflare announced it had become the first infrastructure provider to block AI crawlers accessing content "without permission or compensation, by default."

The detail matters more than the headline, because the headline is routinely overstated. What Cloudflare's own announcement describes is that new domains are asked, at signup, whether to allow AI crawlers, and that "every new domain starts with the default of control." It does not say existing domains were switched over. Separately, Cloudflare says more than a million customers had already chosen its one-click AI blocking option, introduced in September 2024, and that it handles traffic for about 20% of the web.

So the realistic brewery scenario is narrower than "Cloudflare blocked everyone," and it is still worth checking. If your site launched or moved hosts after July 2025, somebody answered that question during setup, and it may not have been you. If a developer turned on the one-click block in 2024 because it sounded prudent, that setting is still there. Neither leaves a trace in your content, which is exactly why content audits miss it.

Key Takeaway: If the site is behind Cloudflare, open the dashboard's bot settings before touching a single page. A block applied at the edge overrides anything your robots.txt says.

How Much of the Usual AI-Readiness Advice Actually Holds Up?

Less than it appears, and a brewery operator deciding where to spend should know which warnings are documented and which are inference. Two of the most repeated claims in this space are weaker than the confidence around them suggests.

"AI crawlers don't execute JavaScript." You will see this stated as settled fact. It traces back substantially to one Vercel traffic analysis from December 2024, which observed that OpenAI's and Anthropic's crawlers were downloading JavaScript files, with roughly 11.5% of ChatGPT's crawler fetches and 23.8% of Claude's being JavaScript. The industry read downloading as evidence of not executing. That is an inference from indirect data, not a controlled rendering test, and the study is now nearly two years old. The direction is probably right and it is still the prudent assumption. It is not the proven fact it gets quoted as.

"Content has a one-year half-life in AI search." This comes from a Q1 2026 analysis of 194,077 sources across ChatGPT, AI Overviews, Gemini and Perplexity, which found that URLs from 2024 appeared at 0.58 times the rate of 2025 URLs, 2023 at 0.33, and 2022 at 0.23. Those numbers are real measurements. The "half-life" framing is the authors' own label for a year-over-year ratio of URL counts, not a measured decay of individual pages. The analysis is cross-sectional, so it cannot separate content age from how much was published in each year, and age was inferred from a year appearing in the URL path, which only 17.4% of URLs had. The remaining 82.6% were excluded.

Neither caveat means ignore the advice. It means the honest version is "older content is retrieved less, by a meaningful margin, and we do not know the mechanism," which is a different instruction from "rewrite everything every twelve months."

Key Takeaway: Spend first on the item that is documented and binary, which is robots.txt. Treat the rendering and freshness advice as sensible precaution rather than as measured law.

What Does Stale Information Actually Cost a Brewery?

More than it costs most businesses, because a brewery's most-asked facts are the ones that change most often. Nobody asks an assistant for your brand story. They ask whether you are open, whether the patio takes dogs, whether the kitchen is running tonight, and whether the hazy IPA is still on.

That is the practical version of the freshness finding, and it does not require believing any half-life claim. If your hours page still describes the winter schedule in June, an assistant that retrieves it will answer confidently and wrongly, and the person asking will go somewhere else without ever knowing they were misinformed. The failure is silent at your end, which is what makes it persistent.

The pattern worth stealing here is one BeerSoft applies to every site it looks at: separate the facts that expire from the copy that does not, and keep the expiring ones on pages that someone owns. Hours, the current tap list, event dates and seasonal closures belong somewhere a staff member updates without needing a developer. Everything else can sit still for two years and lose nothing.

Key Takeaway: List the five facts a stranger most often needs from you, then check how old each one is on the live site. If any is wrong today, fix that before writing anything new.

What Should a Brewery Check First, and in What Order?

In the order that makes each later check meaningful, which is infrastructure before content, every time. The checks below take roughly twenty minutes end to end and none needs a developer.

Start with robots.txt, because it is binary and it governs everything after it. Then the CDN or host bot settings, because an edge block silently overrides what robots.txt permits. Then confirm the facts a stranger actually asks for are present in the HTML and currently true. Only then is there any point looking at structure, and at that stage our guide to what actually breaks on a brewery website covers schema gaps, load speed and FAQ structure in more depth than this post does, and our GEO guide for craft breweries covers how to shape content once the door is open.

The ordering is not arbitrary, and it is the part most audits get wrong by starting at the bottom. A schema fix on a page that OAI-SearchBot is disallowed from reaching changes nothing, because the markup is only useful to a crawler that is permitted to fetch the page in the first place. The same is true of load speed: a page nobody is allowed to request cannot be requested slowly. Every item in the lower half of this list is conditional on the upper half passing, which is why a content-led audit can run for weeks and report steady progress while the actual blocker sits untouched two lines into a text file.

There is one more reason to work in this order, and it is about cost rather than logic. The first two checks are free and take minutes. The last two are projects. Running them in sequence means that when an edge block or a single robots.txt line turns out to be the whole problem, the fix costs an afternoon rather than a quarter, and you find that out on day one instead of after the invoice.

Key Takeaway: Run the checks in order and stop at the first failure. A site that fails check one cannot benefit from anything you do about check four.

Frequently Asked Questions

How do I know if my website is blocked from ChatGPT?

Open yoursite.com/robots.txt in a browser and look for OAI-SearchBot. If it is disallowed, your site is opted out of ChatGPT's search answers, though it may still appear as a navigational link. Also check your CDN or host for an AI bot blocking setting, because a block applied at that level overrides robots.txt. Changes to robots.txt take around 24 hours to register with OpenAI.

What is the difference between GPTBot and OAI-SearchBot?

GPTBot crawls content that may be used to train OpenAI's foundation models. OAI-SearchBot builds the index behind ChatGPT's search features. They are separate crawlers with separate purposes, so blocking GPTBot does not remove you from ChatGPT answers, while blocking OAI-SearchBot does. OpenAI recommends allowing OAI-SearchBot.

Should a brewery block AI crawlers?

Blocking the training crawler is a defensible choice if you do not want your content used that way, and it costs you nothing in ChatGPT answers. Blocking the search crawler means giving up your presence in those answers, which for a brewery competing on local discovery is usually a poor trade. Most publishers now split the difference: block training, allow answering.

Do AI crawlers read JavaScript?

The common advice is that they do not, and it is prudent to assume so. That advice rests largely on a December 2024 Vercel traffic analysis which inferred non-execution from crawlers downloading JavaScript files, rather than on a controlled rendering test. The safe approach is to make sure your hours, location, tap list and contact details appear in the page HTML rather than loading only after a script runs.

How often should a brewery update its website for AI search?

Update what is wrong rather than everything on a schedule. Research across 194,077 sources found URLs from 2024 retrieved at 0.58 times the rate of 2025 URLs, so age does appear to matter, but that analysis is cross-sectional and inferred age from a year in the URL. For a brewery the practical rule is simpler: the facts that expire, meaning hours, tap list, events and seasonal closures, should never be wrong on the live site.

Conclusion: Check the Lock Before You Redecorate

Almost every brewery that asks why ChatGPT does not know them assumes the answer is about writing. Sometimes it is. But the writing only matters once the retrieval works, and retrieval is governed by two settings that take twenty minutes to check and that no content audit will surface.

If the robots.txt file and the CDN both come back clean and the answers are still wrong, then the problem genuinely is content, and that is a different and much larger job. Working out which of the two you actually have is the first thing BeerSoft does on any SEO and AI optimization engagement, because the two look identical from the outside and cost wildly different amounts to fix. If you have checked both and want a second opinion on what the answers are saying about you, get in touch.

Sources

1. OpenAI. Bots. Developer documentation describing OAI-SearchBot, OAI-AdsBot, GPTBot and ChatGPT-User, their purposes and robots.txt guidance.
2. TechnologyChecker. We Analyzed robots.txt Across Cloudflare's Network. Snapshot of 4,242 robots.txt files dated 28 September 2026, giving Disallow and Allow counts and full disallow shares per crawler.
3. Cloudflare. Cloudflare Just Changed How AI Crawlers Scrape the Internet-at-Large. Press release, 1 July 2025, on default AI crawler blocking and Pay Per Crawl.
4. Gander. How Content Freshness Drives Visibility in AI Search. Analysis of 194,077 unique sources from Q1 2026 across ChatGPT, AI Overviews, Gemini and Perplexity, with age inferred from a year in the URL path on 17.4% of URLs.
5. SEOBRO. Do AI Crawlers Execute JavaScript? What the Evidence Says. Review of the December 2024 Vercel traffic analysis and the per-crawler claims drawn from it.