What is evidence-based website analysis?
The difference between a tool that tells you a verdict and one that shows you what it saw — and why the second is worth insisting on.
Analysis is evidence-based when every conclusion arrives with the observation that produced it. Not "your security is weak" but "no Content-Security-Policy header was present in the response". The first requires trust; the second can be checked in ten seconds and is either true or not.
Verdicts versus observations
Most analysis tools give you conclusions. A conclusion is a compressed thing — it has thrown away what it was computed from, and once thrown away it cannot be recovered or argued with.
| Verdict | Evidence |
|---|---|
| "Security needs improvement" | "No Content-Security-Policy header in the response" |
| "Poor AI visibility" | "robots.txt line 4, Disallow: / under User-agent: *, and no group for OAI-SearchBot" |
| "Certificate issue" | "Certificate covers *.example.com; requested host was example.com" |
| "Slow page" | "Time to first byte 1,840 ms across three requests" |
The right-hand column has three properties the left does not: you can verify it yourself, you can act on it without further diagnosis, and you can tell when it is wrong.
Why it matters practically
- You can check it. A named header is confirmable in a devtools panel. A verdict has to be taken on faith.
- It is actionable without a second step. "Poor AI visibility" starts an investigation. A named robots.txt line at a named line number ends one.
- Disagreements resolve. When two tools conflict, evidence settles it. Two verdicts just contradict each other.
- An agent can quote the reason. Given a verdict, a model paraphrases and drifts. Given an observation, it repeats a fact.
- You can detect a wrong tool. If evidence and verdict disagree, the ruleset is wrong. Without evidence, you would never know.
What has to accompany the evidence
Evidence alone is not sufficient. Three companion properties make it trustworthy:
- Provenance. Was this measured directly, derived by rule from a measurement, or inferred heuristically? A heuristic presented like an observation is the quiet failure mode.
- Coverage. What was never examined at all. Without it, the absence of a finding reads as a clean result.
- Undetermined, separately. Checks that reached no verdict, with the reason, kept out of both the passes and the failures.
Where evidence is genuinely hard
Being fair about this matters, because a tool that claims evidence for everything is claiming too much. Some things simply are not directly observable:
- Content quality. Readability scores are proxies and should be labelled as such. Whether writing is any good is not a measurement.
- Machine-generated text. A likelihood signal over stylistic features, wrong in both directions, and honest only when labelled inferred.
- Design quality. Layout stability and responsiveness are measurable; whether a page looks good is not.
- Real-world speed. Server response time is measurable. What a specific user experiences depends on their device and network.
The right response is not to omit these — they are useful — but to mark them clearly so nobody weighs a heuristic like a header.
What our audit reports about this
Every item below is measured directly, not inferred. Run it against your own site and the result names the exact rule or header responsible.
- The observed value returned with every verdict — the header, the certificate field, the robots.txt line, the timing figure.
- A provenance label per check distinguishing measured, derived, inferred and external.
- An explicit coverage statement and a separate undetermined bucket with reasons.
- Every check documented publicly, so the rule behind a verdict can be read rather than guessed at.
For agents and scripts, the same measurement is at
/api/v1/summary?url=yoursite.com —
see the API documentation.
Related questions
Why do most tools not show evidence?
Partly presentation — a verdict is tidier than a header dump — and partly that some tools have less evidence than their confidence implies. Showing the observation makes a wrong rule visible, which is uncomfortable but is exactly the accountability worth having.
Is evidence useful if I am not technical?
Yes, in two ways. You can hand it to whoever is technical and they can act immediately without re-diagnosing. And you can paste it into an assistant, which will explain it accurately because it is reasoning over facts rather than filling gaps.
What if the evidence and the verdict disagree?
Then the rule is wrong, and you have found something worth reporting. This is only possible to notice when the evidence is published — which is one of the better arguments for publishing it.
Does this make reports longer?
Somewhat, which is why response shaping exists — compact mode strips evidence when you only want verdicts. The default keeps it, because the default should be the honest one.
Read next
What is web provenance and why does it matter?
Knowing where a claim about a website came from — and why "measured", "derived" and "inferred" must never be presented as the same thing.
ReadHow do I prove my website is secure?
What you can honestly demonstrate about your site's security, what nobody can prove, and how to make the claim checkable.
ReadWhat is a deterministic website audit?
Why running the same audit twice should give the same answer, what breaks that property, and why it matters more than accuracy.
ReadHow do I check an AI agent's claims about a website?
Verifying an assistant's findings yourself, in a browser or one terminal command, without taking anything on trust.
Read