Why is my website not showing on ChatGPT?
The four things that decide whether ChatGPT can find, read and cite your site — and how to tell which one is stopping you.
Almost always one of four causes: your robots.txt blocks OpenAI's crawlers, your content only appears after JavaScript runs, the site has no presence in the search index ChatGPT retrieves from, or the page simply is not the best available source for that question. The first two are configuration faults you can fix today. The last two take longer.
ChatGPT uses three different crawlers, and they are not interchangeable
This trips up almost everyone. OpenAI does not have one bot; it has three, each with a separate job and a separate name in robots.txt. Blocking the wrong one produces exactly the outcome people complain about.
| Crawler | What it does | Blocking it means |
|---|---|---|
GPTBot | Collects text used to train future models | Your content is not used in training. Has no effect on whether you are cited today. |
OAI-SearchBot | Builds the index ChatGPT search retrieves from | You cannot appear in ChatGPT results at all. This is the one that matters. |
ChatGPT-User | Fetches a page live because a user asked about it | Someone who pastes your URL into ChatGPT gets nothing back. |
A great many sites added User-agent: GPTBot / Disallow: / during the training-data backlash and then blocked all three by copying a stricter snippet from a blog post. If you want the traffic but not the training, that distinction is the whole game.
Check what your robots.txt actually says
Not what you think it says — what it says. Open yoursite.com/robots.txt in a browser. Then read it carefully, because two rules commonly bite:
- A
Disallow: /underUser-agent: *blocks every crawler that does not have its own more specific group. AddingUser-agent: OAI-SearchBot / Allow: /fixes it, because the most specific matching group wins. - Some hosting platforms and CDNs inject or override
robots.txtat the edge. If the file you uploaded is not the file being served, that is your answer. Fetch it and compare.
If the page needs JavaScript, assume it is invisible
Ask for your page the way a crawler does, with no browser involved:
curl -sL https://yoursite.com | grep -i "some text from your page"If your actual content does not come back, it is not in the HTML — it is assembled in the browser afterwards. Googlebot renders JavaScript, slowly and selectively. AI crawlers overwhelmingly do not. A React or Vue app with client-side rendering serves an empty <div id="root"> to every one of them.
The fix is server-side rendering or static generation, which every major framework supports. This is the single highest-impact change for AI visibility and it is invisible in a browser, which is why it goes unnoticed for years.
Being crawlable is not the same as being cited
Once the mechanics are right, you are competing on merit. What answer engines pull from, in rough order of influence:
- Directly answering a specific question. Pages structured as question-and-answer get quoted because a passage can be lifted whole. A page that buries the answer in paragraph nine does not.
- Being the primary source. Original data, first-hand detail and specifics get cited. Summaries of other people's articles get skipped in favour of the article.
- Corroboration elsewhere. Models weigh whether other sources agree with you. A claim that appears nowhere else reads as unreliable, however true.
- Clear attribution. A named author, a visible organisation, a date. Anonymous content is harder to cite responsibly, so it gets cited less.
Work through it in this order
- Fetch
yoursite.com/robots.txtand confirmOAI-SearchBotandChatGPT-Userare not blocked, by name or by a catch-all. - Fetch the page with
curland confirm your real content is in the HTML. - Search for a distinctive sentence from your page in Bing. ChatGPT search leans on that index; absence there is a strong signal.
- Ask ChatGPT directly about a topic you cover and see who it does cite. That tells you what a citation-worthy source looks like in your area.
- Only then change the writing.
Steps one and two take ten minutes and account for most cases. Do not rewrite anything before you have done them.
What our audit reports about this
Every item below is measured directly, not inferred. Run it against your own site and the result names the exact rule or header responsible.
- Whether
GPTBot,OAI-SearchBotandChatGPT-Userare allowed or blocked — naming the exactrobots.txtrule responsible, not just the verdict. - How much of your visible text is present in the raw HTML versus added by JavaScript.
- Whether structured data is present and valid, and whether an
llms.txtexists. - Whether the page has a clear author, organisation and date that a model can attribute.
For agents and scripts, the same measurement is at
/api/v1/ai?url=yoursite.com —
see the API documentation.
Related questions
Does blocking GPTBot stop ChatGPT citing me?
No. GPTBot collects training data. Citations in ChatGPT come from its search index, which is built by OAI-SearchBot, and from live fetches by ChatGPT-User. You can block training while staying fully visible in the product — block GPTBot and allow the other two.
How long until ChatGPT picks up my new content?
There is no published figure and it varies widely. Search-index inclusion is typically days to weeks. Training data is far slower and tied to model releases, so months at minimum. Live fetches by ChatGPT-User are immediate, which is why a URL pasted into a chat works even for a page published minutes ago.
Do I need an llms.txt file?
It will not hurt, and it is trivial to add, but no major operator has committed to reading it. Treat it as cheap insurance rather than a fix. Correct robots.txt rules and server-rendered HTML are what actually determine whether you are readable today.
My competitor gets cited and I do not. Why?
Check their robots.txt and their raw HTML first — the difference is often purely mechanical. If both of you are readable, the difference is usually that their page answers a specific question directly and yours covers the topic broadly. Answer engines extract passages, not pages.
Read next
Why can't Claude see my website?
Anthropic runs three separate crawlers with three separate purposes. Blocking the wrong one is the usual reason a site is invisible.
ReadHow do I get my site cited by AI assistants?
The mechanics that must be right, then the content decisions that actually earn the citation — in the order worth doing them.
ReadDoes JavaScript hurt AI crawlers?
Yes, more than it hurts search engines. If your content only exists after JavaScript runs, most AI crawlers see an empty page.
ReadWhat is llms.txt and do I need it?
A proposed convention for pointing language models at your best content. Cheap to add, unproven, and no substitute for the things that do work.
Read