Adds a Data Breach History check, via Have I Been Pwned - #334
Open
Eales wants to merge 5 commits into
Open
Conversation
Web Check can tell you how a site is configured today, but nothing about whether the service behind it has already lost its users' data. This adds a check against the public Have I Been Pwned breach catalogue, which is free and needs no API key, so it works on every self-hosted instance without configuration. Only the registrable domain is ever sent upstream. No email address, account or password leaves the server, and the check makes no claim about any individual user — that would need HIBP's authenticated account API, which is deliberately not used here. Three things the raw feed does not give you for free: - HIBP indexes breaches against the registrable domain, so a lookup for store.adobe.com returns nothing while adobe.com returns the breach. The hostname is folded down with psl, as the whois and subdomain checks already do. - Breach descriptions contain real markup, including anchors. Rendering third-party HTML is the exact flaw this tool reports on other people's sites, so the tags are stripped server-side. - The catalogue mixes confirmed incidents with unverified, fabricated, spam-list and stealer-log entries. Those are kept, but flagged, so a fabricated entry is never presented with the weight of a real breach. Exposed data classes are tiered by how damaging they are, so credentials and financial records sort above plain identifiers. Adds 28 unit tests, run with `yarn test` via the built-in Node test runner, and a CI job to go with them.
Shows each catalogued incident with its date, the number of accounts affected, and the classes of data exposed, colour-coded by how damaging each class is. Confirmed incidents are badged as verified; unverified, fabricated, spam-list and stealer-log entries carry an explicit caveat saying which, so a reader can weigh them accordingly. A domain with nothing on record says so rather than hiding the card, which matches how the vulnerabilities card reports a clean host. HIBP publishes under CC BY 4.0, so the attribution and licence link travel with the data rather than being hardcoded in the component.
A catalogued breach is history, not a live misconfiguration, so it is deliberately never reported as critical: putting a 2012 breach next to "MySQL exposed to the internet" would misrepresent what needs attention today. It is still worth surfacing, because it says credentials for this service have already circulated. A breach that exposed credentials is reported as an issue with advice to force a reset and watch for credential stuffing, other confirmed breaches as a warning, and unconfirmed entries as info. A domain with nothing on record earns a pass.
✅ Deploy Preview for web-check ready!Built without sensitive environment variables
To edit notification comments on pull requests, go to your Netlify project configuration. |
Scanning morele.pl reported it clean, which was wrong: it 301s straight to morele.net, and the 2018 breach of 2.5 million accounts is catalogued against morele.net. Looking up only what the user typed produces a false negative for every site whose primary domain differs from the one it is branded on. The destination is now resolved alongside the first lookup, so the extra work is close to free, and both domains are queried when they differ. Resolution is best-effort: a site that refuses HEAD, hangs or fails just falls back to the domain that was typed. The two domains are reported, never merged into one identity. A parked domain pointing at a large site must not inherit that site's breaches, so each entry keeps the domain HIBP recorded it against, the card says which that was, and a note explains that the redirect is why a second domain was checked.
DocContent rendered <img src={doc.screenshot}> unconditionally, so the
four checks with no screenshot on file — get-ip, vulnerabilities,
subdomains and breach-history — showed an Example section containing a
broken image.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Web Check tells you how a site is configured today. It says nothing about whether the service behind it has already lost its users' data — which is often the more useful thing to know when you are judging a vendor, or working out whether credential stuffing is a live risk for a login page.
This adds a Data Breach History check against the public Have I Been Pwned breach catalogue: which incidents are on record for the domain, when, how many accounts, and which classes of data were exposed.
No API key. The
/api/v3/breaches?domain=endpoint is unauthenticated, so this works on every self-hosted instance out of the box, with nothing to configure.Privacy — deliberately the domain-level catalogue only
HIBP exposes three easily-confused things. This uses only the first:
/breaches?domain=— the breach catalogue/breachedaccount/{email}Only the registrable domain leaves the server. No email address, account or password is submitted, and the check makes no claim about any individual user's data.
Three things the raw feed does not give you for free
1. Subdomains silently miss. HIBP indexes against the registrable domain, so
store.adobe.comreturns[]whileadobe.comreturns the breach. The hostname is folded down withpsl, the same waywhois.jsandsubdomains.jsalready do it.2. Descriptions contain real markup, including anchors with
target="_blank":Rendering third-party HTML is the exact flaw this tool reports on other people's sites, so tags are stripped server-side rather than trusted downstream.
3. The catalogue is not all equally credible. Of the 1026 entries currently published, 42 are unverified, 3 fabricated, 16 spam lists and 6 stealer logs. Those are kept — hiding them would be its own distortion — but flagged, so a fabricated entry is never shown with the weight of a confirmed one.
Exposed data classes are tiered by how damaging they are (credentials and financial records, then data enabling targeted attacks, then plain identifiers), which drives both the ordering and the colour coding.
Example
linkedin.com, through the live endpoint:The faked 2023 entry sorts last and is badged "unconfirmed" rather than presented as a breach.
Advisory integration
A catalogued breach is history, not a live misconfiguration, so it is deliberately never reported as critical — putting a 2012 breach beside "MySQL exposed to the internet" would misrepresent what actually needs attention today. Instead:
Tests
28 unit tests using the built-in Node test runner — no new dependencies, no config file. They cover domain folding (including multi-part suffixes like
.co.uk), HTML stripping and entity decoding, data-class tiering, sorting, the trust flags, and the malformed-input paths.A
🧪 Unit Testsjob is wired into.github/workflows/ci.ymlalongside the existing lint and typecheck jobs.Verification
yarn test— 28/28yarn lint— cleanyarn typecheck(astro check) — 0 errors, 0 warnings, 0 hintsyarn build— completesprettier --checkon every touched file — cleanadobe.com,store.adobe.com(proving the subdomain fold), a fullhttps://www.linkedin.com/feedURL, a clean domain, and an IP (correctly skipped)Notes
testscript topackage.jsonand the same CI job. Whichever lands first, the other reduces to a trivial conflict. Everything else here is new files or additive registry entries.