Repository navigation
Detect Chinese Resident ID numbers - #187
Merged
Merged
Conversation
Chinese Resident Identity Card numbers were not detected, so most of them reached the provider unmasked. The roughly one in ten valid IDs that also pass Luhn were reported as CREDIT_CARD instead. Add a CN_RESIDENT_ID detector to the deterministic layer. Candidates are 17 digits plus a digit or X check character, validated with python-stdnum (check digit, birth date and region code). Boundaries only reject ASCII letters and digits, because the numbers often sit directly next to CJK characters, which are \w in Python regex. The detector runs before the credit-card detector. Filter deterministic spans to the requested entity types before overlap resolution. Configs with an explicit entity list that does not include CN_RESIDENT_ID keep masking Luhn-valid IDs as CREDIT_CARD. Enable CN_RESIDENT_ID by default and document it in the example config, README, docs and benchmark corpus. Closes #184
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Automations to automatically generate PRs for you. |
sgasser
enabled auto-merge
November 7, 2026 10:14
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #184
What changed
CN_RESIDENT_IDentity in the detector's deterministic layer. Candidates are 17 digits plus a digit orX/xcheck character, validated withstdnum.cn.ric.is_valid(check digit, birth date and region code).身份证号是360426199101010071请核对).CN_RESIDENT_IDinstead ofCREDIT_CARD.detect_deterministicnow drops entity types the request did not ask for before resolving overlaps. Configs with an explicitentitieslist that lacksCN_RESIDENT_IDkeep masking Luhn-valid IDs asCREDIT_CARD, as before.CN_RESIDENT_IDis enabled by default and added toconfig.example.yaml, the README, the PII docs and the benchmark corpus.The second example in the issue,
110105199001010017, has a wrong check digit (it should end in0), so it is not a valid ID. It passes Luhn and is still reported asCREDIT_CARD. The tests use110105199001010256, a valid ID that also passes Luhn.Upgrade note
Existing configs with an explicit
pii_detection.entitieslist need to addCN_RESIDENT_IDto mask these IDs.Verification
bun test(513 tests),bun run typecheck,bun run typecheck:benchmarksandbun run checkpass.pytestpasses, andruff checkandruff format --checkare clean. New tests cover CJK-adjacent IDs,X/xcheck characters, priority overCREDIT_CARD, invalid check digit, birth date and region code, IDs inside longer tokens, and the requested-entities filter through/analyze./api/mask, OpenAI (non-streaming and streaming) and Anthropic routes forward only[[CN_RESIDENT_ID_n]]placeholders and restore the IDs in the response. An ID with a wrong check digit stays unmasked. The SQLite request log contains no raw IDs.CN_RESIDENT_ID: the Luhn-valid ID is still masked asCREDIT_CARD.config.example.yaml: IDs are masked asCN_RESIDENT_ID.Reviewed with sol-loop: 1 round with gpt-6.1-sol on high. Sol verdict: approved