Skip to content

Detect Chinese Resident ID numbers - #187

Merged
sgasser merged 2 commits into
mainfrom
184-detect-chinese-resident-id
Nov 7, 2026
Merged

sgasser merged 2 commits into
mainfrom
184-detect-chinese-resident-id

Conversation

@sgasser

@sgasser sgasser commented Nov 7, 2026

Copy link
Copy Markdown
Owner

Closes #184

What changed

  • New CN_RESIDENT_ID entity in the detector's deterministic layer. Candidates are 17 digits plus a digit or X/x check character, validated with stdnum.cn.ric.is_valid (check digit, birth date and region code).
  • Boundaries only reject ASCII letters and digits, so IDs directly next to Chinese characters are detected (身份证号是360426199101010071请核对).
  • The detector runs before the credit-card detector, so a valid ID that also passes Luhn (about 1 in 10) is labeled CN_RESIDENT_ID instead of CREDIT_CARD.
  • detect_deterministic now drops entity types the request did not ask for before resolving overlaps. Configs with an explicit entities list that lacks CN_RESIDENT_ID keep masking Luhn-valid IDs as CREDIT_CARD, as before.
  • CN_RESIDENT_ID is enabled by default and added to config.example.yaml, the README, the PII docs and the benchmark corpus.

The second example in the issue, 110105199001010017, has a wrong check digit (it should end in 0), so it is not a valid ID. It passes Luhn and is still reported as CREDIT_CARD. The tests use 110105199001010256, a valid ID that also passes Luhn.

Upgrade note

Existing configs with an explicit pii_detection.entities list need to add CN_RESIDENT_ID to mask these IDs.

Verification

  • bun test (513 tests), bun run typecheck, bun run typecheck:benchmarks and bun run check pass.
  • Detector: pytest passes, and ruff check and ruff format --check are clean. New tests cover CJK-adjacent IDs, X/x check characters, priority over CREDIT_CARD, invalid check digit, birth date and region code, IDs inside longer tokens, and the requested-entities filter through /analyze.
  • Live test with the all-in-one Docker image built from this branch (real GLiNER backend), using a local mock upstream that records what the proxy forwards:
    • Default config: /api/mask, OpenAI (non-streaming and streaming) and Anthropic routes forward only [[CN_RESIDENT_ID_n]] placeholders and restore the IDs in the response. An ID with a wrong check digit stays unmasked. The SQLite request log contains no raw IDs.
    • Explicit entity list without CN_RESIDENT_ID: the Luhn-valid ID is still masked as CREDIT_CARD.
    • Built-in config.example.yaml: IDs are masked as CN_RESIDENT_ID.

Reviewed with sol-loop: 1 round with gpt-6.1-sol on high. Sol verdict: approved

Chinese Resident Identity Card numbers were not detected, so most of
them reached the provider unmasked. The roughly one in ten valid IDs
that also pass Luhn were reported as CREDIT_CARD instead.

Add a CN_RESIDENT_ID detector to the deterministic layer. Candidates
are 17 digits plus a digit or X check character, validated with
python-stdnum (check digit, birth date and region code). Boundaries
only reject ASCII letters and digits, because the numbers often sit
directly next to CJK characters, which are \w in Python regex. The
detector runs before the credit-card detector.

Filter deterministic spans to the requested entity types before
overlap resolution. Configs with an explicit entity list that does not
include CN_RESIDENT_ID keep masking Luhn-valid IDs as CREDIT_CARD.

Enable CN_RESIDENT_ID by default and document it in the example
config, README, docs and benchmark corpus.

Closes #184
@mintlify

mintlify Bot commented Nov 7, 2026 •

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
PasteGuard 🟢 Ready View Preview Nov 7, 2026, 10:12 AM

💡 Tip: Enable Automations to automatically generate PRs for you.

@sgasser
sgasser merged commit 8481c59 into main Nov 7, 2026
4 checks passed
@sgasser
sgasser deleted the 184-detect-chinese-resident-id branch November 7, 2026 14:43
@sgasser sgasser mentioned this pull request Nov 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Chinese Resident ID numbers are not detected (and some are labeled CREDIT_CARD)

1 participant