UK Public Sector GitHub Observatory

UK public sector / Domain security

7,918 .gov.uk domains: DNS, SPF, DMARC, HTTPS/HSTS and robots/security.txt/sitemap posture, refreshed weekly.

Every domain in GOV.UK's register of .gov.uk domains, probed weekly for DNS, email-spoofing defences and basic web hygiene. marks a domain that actually appears in commit metadata elsewhere on this site.

7,918
DOMAINS PROBED
95%
HAVE SPF
77%
HAVE DMARC
4,103 enforced (quarantine/reject)
65%
SERVE HTTPS
10 domains behind harvested commit addresses have no enforced DMARC

Of the 360 .gov.uk domains that actually appear in commit metadata across this estate (fork-excluded - the population the rest of this site counts as real committers, not the whole register), 10 resolve to no DMARC record or a policy of p=none, covering 32 harvested committer addresses. A domain in that state cannot reject a spoofed email claiming to be from it - the exact gap Finding 04 of the security read describes. Subdomains are judged on the policy a receiver would actually apply, which is the organisational domain's record and its sp= tag where the subdomain has none of its own. A further 47 are not registered .gov.uk domains at all - overwhelmingly typos committed into git configs (cabinetofice.gov.uk, companieshgouse.gov.uk). Nobody can publish a DMARC record for a domain nobody owns, and an unregistered lookalike of a government domain is available for anyone to register.

Domain HTTPS HSTS SPF DMARC NS provider robots.txt security.txt sitemap.xml
Loading 7,918 domains…

Method and limits

Probed by pipeline/probe-domains.py, identifying itself honestly (no browser-spoofed User-Agent) and rate-limited across a shared pool rather than hammering any one domain. DMARC is read from _dmarc.<domain>, not the apex TXT record, which is where an earlier one-off version of this scan silently got it wrong. WHOIS is not collected - registrant data for .gov.uk domains is almost always institutional boilerplate and not worth the run time at this scale. A domain that fails to answer is not overwritten with a false "no SPF/no DMARC" reading; last known posture is kept until a probe actually confirms a change. Full data, refreshed every Friday alongside the rest of the site: the open dataset - 7,918 domains as posture.csv, the full probe records, and the append-only log of every posture change since the first run.

This table is served from data/domains.json (551 kB, gzipped in transit) and rendered a screenful at a time. Sorting and filtering happen in the browser across all 7,918 rows, not just the ones on screen.