Can I trust what ChatGPT says about my website?
A practical guide to which parts of an AI answer about your site are reliable, which are guesses, and how to tell them apart in seconds.
Trust it on questions about text it can see, and distrust it on questions about configuration it cannot. Content, tone, structure and clarity: reliable if it fetched the page. Security headers, certificate expiry, DNS, email authentication, load time, crawler access: not observable through a browsing tool, so anything specific it says about them is reconstruction rather than observation.
Sort the question before you weigh the answer
The reliability of an AI answer about your website depends almost entirely on which category the question falls into. This is a more useful frame than "is the model good".
| Ask about | Reliability | Why |
|---|---|---|
| Wording, structure, clarity, tone | Good | The page text is genuinely in front of it, and judging prose is what it is best at |
| Whether a page answers a question well | Good | Same — this is a reading task |
| Whether content looks thin or padded | Fair | Reading task, but the judgement is subjective and it tends to be generous |
| Security headers, CSP, HSTS | Poor | Never in a fetch tool's output. Answers are reconstructed. |
| Certificate expiry, TLS version | Poor | Requires the handshake, which no browsing tool surfaces |
| DNS, SPF, DMARC, MX | Poor | Requires a resolver, not an HTTP fetch |
| Page speed, Core Web Vitals | Poor | Requires timing instrumentation or field data |
| Whether AI crawlers can read the site | Mixed | It may read robots.txt; correctly resolving precedence rules is another matter |
The two-question test
You do not need to know how the assistant is built. Two follow-ups will tell you what you are dealing with.
- "What exactly did you observe that told you this?" A grounded answer names an artefact — a header and its value, a line from robots.txt, a certificate field. An ungrounded one restates the conclusion in different words or explains the general principle.
- "What were you unable to check?" A grounded answer has a real list and gives it. An ungrounded one either claims to have checked everything or produces a vague caveat about verifying independently.
Question two is the more revealing of the pair. Almost nothing can check everything, so a tool that names its limits is describing reality, and one that does not is describing an ambition.
Three answers that should always raise an eyebrow
- "Your site appears to be secure." Appears to whom, on what basis? Security is a set of specific headers and transport settings, each individually checkable. An answer that does not name one has not examined any.
- "I could not find any issues." Not finding issues and not looking produce identical sentences. Ask which checks ran.
- "Your site loads in about two seconds." A specific number with no measurement behind it. Timing requires instrumentation; a browsing tool does not report it.
Getting a better answer
Two approaches, in order of effort.
- Bring the measurement to the model. Run a real audit, paste the structured result into the conversation, then ask your questions. The model is now reasoning over facts, which is what it is genuinely good at — and it will interpret and prioritise them better than most dashboards.
- Give the model a tool. If your assistant supports the Model Context Protocol, connecting a measurement server means it fetches the facts itself, every time, without you brokering. See Outrings for AI agents.
Both work. The first requires nothing but a copy and paste and is the right place to start.
What our audit reports about this
Every item below is measured directly, not inferred. Run it against your own site and the result names the exact rule or header responsible.
- Every header, certificate field and DNS record an assistant cannot observe on its own — returned with the raw value attached.
- Per-crawler robots.txt resolution, naming the exact rule that produced each verdict rather than summarising the file.
- A Markdown report written specifically to be pasted into an assistant, structured so the model reasons over facts rather than filling gaps.
- An explicit statement of what was not covered, so the model has the material to qualify its answer.
For agents and scripts, the same measurement is at
/api/v1/summary?url=yoursite.com —
see the API documentation.
Related questions
Is ChatGPT wrong about my website on purpose?
No. It is producing the most plausible answer available given what it was given, which for configuration questions is nothing about your site specifically. The confidence in the phrasing is a property of how language models write, not a claim about certainty.
What if it used a browsing tool — is it reliable then?
For page content, yes. A browsing tool returns body text and discards response headers, so the security, transport and DNS parts of the answer are exactly as unfounded as before, now surrounded by well-founded content observations.
Should I ignore AI advice about my site entirely?
No — models are strong at interpreting findings, weighing priorities and explaining why something matters. Give it real measurements and its analysis is genuinely valuable. The failure is asking it to be the instrument as well as the analyst.
How do I paste an audit into a chat usefully?
Fetch the Markdown report at /api/v1/llm?url=yoursite.com and paste it whole. It is written to be read by a model — findings, evidence, what was not covered — and it includes the endpoints so the assistant can re-measure after you make a change.
Read next
Why do LLMs hallucinate website audits?
The specific reason a model invents plausible findings about your site, and the two design choices that stop it.
ReadHow do AI agents inspect websites?
What actually happens when you ask an assistant about a website — and why the answer is usually inferred rather than observed.
ReadWhy is AI-generated SEO advice usually wrong?
Four reasons an assistant gives you outdated, generic or inapplicable SEO advice — and the one change that fixes most of it.
ReadHow do I check an AI agent's claims about a website?
Verifying an assistant's findings yourself, in a browser or one terminal command, without taking anything on trust.
Read