How do I add a website check to my CI pipeline?

Assert on findings, not on the score

The instinct is to gate on a number: fail if the score drops below 85. In practice this produces a build that breaks for content changes and passes while something genuinely important regresses, because a single critical failure can be masked by improvements elsewhere.

Gate on the things that must never be true instead. They are stable, they never fire spuriously, and every one of them is worth stopping a deploy for.

• Any noindex on a page that should be indexed.
• A robots.txt that blocks a crawler you rely on.
• A missing security header you had previously set.
• A certificate expiring inside your renewal window.
• A canonical pointing at a host that is not production.
• A new critical-severity finding of any kind.

GitHub Actions

name: Post-deploy site check

on:
  workflow_run:
    workflows: ["Deploy"]
    types: [completed]

jobs:
  inspect:
    runs-on: ubuntu-latest
    steps:
      - name: Measure the deployed site
        run: |
          set -euo pipefail
          result=$(curl -sf "https://outrings.com/api/v1/summary?url=${{ vars.SITE_URL }}&refresh=1")

          # Anything critical is a hard stop.
          critical=$(jq -r '[.priorityActions[] | select(.severity == "critical")] | length' <<<"$result")
          if [ "$critical" -gt 0 ]; then
            jq -r '.priorityActions[] | select(.severity == "critical") | "::error::\(.title) — \(.action)"' <<<"$result"
            exit 1
          fi

          # Indexability is the expensive silent failure.
          jq -e '.categories[] | select(.id == "discoverability") | .score > 70' <<<"$result" > /dev/null \
            || { echo "::error::Discoverability regressed — check robots.txt and noindex"; exit 1; }

          echo "::notice::Site scored $(jq -r '.score' <<<"$result")"

GitLab CI

site-check:
  stage: verify
  image: alpine:latest
  before_script:
    - apk add --no-cache curl jq
  script:
    - |
      set -euo pipefail
      result=$(curl -sf "https://outrings.com/api/v1/summary?url=$SITE_URL&refresh=1")
      critical=$(jq -r '[.priorityActions[] | select(.severity == "critical")] | length' <<<"$result")
      if [ "$critical" -gt 0 ]; then
        jq -r '.priorityActions[] | select(.severity == "critical") | "CRITICAL: \(.title)"' <<<"$result"
        exit 1
      fi
      echo "Score: $(jq -r '.score' <<<"$result")"
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

Comparing against the previous deploy

Absolute assertions catch known-bad states. To catch anything that got worse, compare instead:

result=$(curl -sf "https://outrings.com/api/v1/changes?url=$SITE_URL")

if [ "$(jq -r '.comparable' <<<"$result")" = "true" ]; then
  # newProblems entries carry an id and a status of fail or warn.
  # Gate on outright failures; warnings are worth reading, not blocking on.
  new=$(jq -r '[.newProblems[]? | select(.status == "fail") | .id] | join("; ")' <<<"$result")
  [ -z "$new" ] || { echo "Regression introduced: $new" >&2; exit 1; }
fi

Practical notes

• Run it after deploy, against the real URL. Auditing a preview host tells you about the preview host — different headers, different certificate, often different robots rules.
• Make the first runs non-blocking. Use continue-on-error for a week and read the output before you let it fail builds. You will find pre-existing findings you did not know about.
• Do not gate on flaky signals. Latency-derived checks vary with network conditions; assert on configuration, which does not.
• Mind the rate limit. Forty audits an hour is generous for deploys and easy to exhaust from a matrix build fanning out across environments.