Outrings
AI visibility and answer engines

Why is my website not showing on ChatGPT?

The four things that decide whether ChatGPT can find, read and cite your site — and how to tell which one is stopping you.

5 min read
Short answer

Almost always one of four causes: your robots.txt blocks OpenAI's crawlers, your content only appears after JavaScript runs, the site has no presence in the search index ChatGPT retrieves from, or the page simply is not the best available source for that question. The first two are configuration faults you can fix today. The last two take longer.

ChatGPT uses three different crawlers, and they are not interchangeable

This trips up almost everyone. OpenAI does not have one bot; it has three, each with a separate job and a separate name in robots.txt. Blocking the wrong one produces exactly the outcome people complain about.

CrawlerWhat it doesBlocking it means
GPTBotCollects text used to train future modelsYour content is not used in training. Has no effect on whether you are cited today.
OAI-SearchBotBuilds the index ChatGPT search retrieves fromYou cannot appear in ChatGPT results at all. This is the one that matters.
ChatGPT-UserFetches a page live because a user asked about itSomeone who pastes your URL into ChatGPT gets nothing back.

A great many sites added User-agent: GPTBot / Disallow: / during the training-data backlash and then blocked all three by copying a stricter snippet from a blog post. If you want the traffic but not the training, that distinction is the whole game.

Check what your robots.txt actually says

Not what you think it says — what it says. Open yoursite.com/robots.txt in a browser. Then read it carefully, because two rules commonly bite:

  • A Disallow: / under User-agent: * blocks every crawler that does not have its own more specific group. Adding User-agent: OAI-SearchBot / Allow: / fixes it, because the most specific matching group wins.
  • Some hosting platforms and CDNs inject or override robots.txt at the edge. If the file you uploaded is not the file being served, that is your answer. Fetch it and compare.
A useful rule to hold onto: robots.txt is not access control. It is a request, honoured by the major operators and ignored by everyone else. Blocking a crawler there does not protect anything — it only removes you from the products that respect it.

If the page needs JavaScript, assume it is invisible

Ask for your page the way a crawler does, with no browser involved:

curl -sL https://yoursite.com | grep -i "some text from your page"

If your actual content does not come back, it is not in the HTML — it is assembled in the browser afterwards. Googlebot renders JavaScript, slowly and selectively. AI crawlers overwhelmingly do not. A React or Vue app with client-side rendering serves an empty <div id="root"> to every one of them.

The fix is server-side rendering or static generation, which every major framework supports. This is the single highest-impact change for AI visibility and it is invisible in a browser, which is why it goes unnoticed for years.

Being crawlable is not the same as being cited

Once the mechanics are right, you are competing on merit. What answer engines pull from, in rough order of influence:

  • Directly answering a specific question. Pages structured as question-and-answer get quoted because a passage can be lifted whole. A page that buries the answer in paragraph nine does not.
  • Being the primary source. Original data, first-hand detail and specifics get cited. Summaries of other people's articles get skipped in favour of the article.
  • Corroboration elsewhere. Models weigh whether other sources agree with you. A claim that appears nowhere else reads as unreliable, however true.
  • Clear attribution. A named author, a visible organisation, a date. Anonymous content is harder to cite responsibly, so it gets cited less.

Work through it in this order

  1. Fetch yoursite.com/robots.txt and confirm OAI-SearchBot and ChatGPT-User are not blocked, by name or by a catch-all.
  2. Fetch the page with curl and confirm your real content is in the HTML.
  3. Search for a distinctive sentence from your page in Bing. ChatGPT search leans on that index; absence there is a strong signal.
  4. Ask ChatGPT directly about a topic you cover and see who it does cite. That tells you what a citation-worthy source looks like in your area.
  5. Only then change the writing.

Steps one and two take ten minutes and account for most cases. Do not rewrite anything before you have done them.

What our audit reports about this

Every item below is measured directly, not inferred. Run it against your own site and the result names the exact rule or header responsible.

  • Whether GPTBot, OAI-SearchBot and ChatGPT-User are allowed or blocked — naming the exact robots.txt rule responsible, not just the verdict.
  • How much of your visible text is present in the raw HTML versus added by JavaScript.
  • Whether structured data is present and valid, and whether an llms.txt exists.
  • Whether the page has a clear author, organisation and date that a model can attribute.

For agents and scripts, the same measurement is at /api/v1/ai?url=yoursite.com — see the API documentation.

Related questions

Does blocking GPTBot stop ChatGPT citing me?

No. GPTBot collects training data. Citations in ChatGPT come from its search index, which is built by OAI-SearchBot, and from live fetches by ChatGPT-User. You can block training while staying fully visible in the product — block GPTBot and allow the other two.

How long until ChatGPT picks up my new content?

There is no published figure and it varies widely. Search-index inclusion is typically days to weeks. Training data is far slower and tied to model releases, so months at minimum. Live fetches by ChatGPT-User are immediate, which is why a URL pasted into a chat works even for a page published minutes ago.

Do I need an llms.txt file?

It will not hurt, and it is trivial to add, but no major operator has committed to reading it. Treat it as cheap insurance rather than a fix. Correct robots.txt rules and server-rendered HTML are what actually determine whether you are readable today.

My competitor gets cited and I do not. Why?

Check their robots.txt and their raw HTML first — the difference is often purely mechanical. If both of you are readable, the difference is usually that their page answers a specific question directly and yours covers the topic broadly. Answer engines extract passages, not pages.

Read next

All 50 guides · How every check works · API for agents