OpenTender: identifier-first lookup primitive (fetch_by_registration) - #74
Open
Nitjsefnie wants to merge 1 commit into
Open
OpenTender: identifier-first lookup primitive (fetch_by_registration)#74Nitjsefnie wants to merge 1 commit into
Nitjsefnie wants to merge 1 commit into
Conversation
Add fetch_by_registration() to OpenTenderAdapter so the source can be queried by the national registration number the lookup pipeline already derives from the GLEIF anchor (siren / cz_ico / ee_registry_code / fi_business_id …) instead of a name-keyed FTS MATCH. This is the fix for the core of issue StephenAbbott#29: an FTS MATCH on "Orange" returns genuine but unrelated bodies ("Red-Orange e.U.", "Orange controls s.r.o."), because relevance ranking is not identity. Keying on the indexed body_ids table (id_value, scoped to the subject's country, restricted to the id types that carry registration numbers — ORGANIZATION_ID / TRADE_REGISTER / HEADER_ICO / TAX_ID, excluding the internal SOURCE_ID / BVD_ID / ETALON_ID keys) returns only tenders the same legal entity actually took part in — the precise inverse of the name search's behaviour. The registration query mirrors the search/fetch corrupt-DB hardening (quick_check defence in depth: a DatabaseError drops the connection and returns empty rather than raising up the pipeline) and normalises the id into exact / digits-only / leading-zero-stripped forms (mirrors eiti._norm_forms) to bridge GLEIF/DIGIWHIST formatting drift. Tests pin the StephenAbbott#29 regressions: the Orange-SIREN identifier hit, the "Orange ≠ Red-Orange e.U." name-collision rejection, country scoping, internal-id-type exclusion, id-form normalisation, and graceful empty degradation (no DB / blank inputs / corrupt DB). Scope: adapter method + tests only. Wiring into REGISTRY / the lookup dispatch is deliberately deferred — the issue notes registration is separately blocked on artifact size (Render cold-start budget), and it is a dispatch-framework change outside this change's scope. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
Author
|
No rush on this — just flagging it's still clean and mergeable against current One thing that might make it cheaper to review: the registry and lookup-router wiring is deliberately deferred here, which is the remainder of #29. If you'd rather have this as two smaller pieces, I'm happy to split the OpenTender source changes from the dispatch wiring, or to close it and re-send it in whatever shape fits better alongside the MCP SDK v2 work in #88. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
fetch_by_registration(country, registration_number, legal_name="")toOpenTenderAdapter— the identifier-first lookup primitive from #29, steps 1 and 3.Changes
backend/opencheck/sources/opentender.py(+136) — queries the indexedbody_idstable onid_valuescoped to the subject's country, restricted to the id types that genuinely carry registration numbers (ORGANIZATION_ID/TRADE_REGISTER/HEADER_ICO/TAX_ID) and excluding the internalSOURCE_ID/BVD_ID/ETALON_IDkeys, joined back totenders._id_forms()normalises exact / digits-only / zero-stripped forms to bridge GLEIF↔DIGIWHIST formatting drift. Corrupt-DB hardening follows this adapter's own_db_searchpattern (drop connection, return empty).backend/tests/test_opentender.py(+242/−9) — 9 new tests.Shapes mirror the merged EITI precedent (#46), with two deliberate divergences, open to your override: it returns
list[SourceHit]via the existing_tender_hitfactory (a procurement subject maps to many tenders, not one org record), and no name-equivalence gate is applied to identifier hits — the identifier is the identity match, and a name filter would wrongly drop a subsidiary whose registered name differs.Testing
uv run pytest tests/test_opentender.py -q— 48 passed (9 new). Full backenduv run pytest -q— 2496 passed, 16 skipped, 5 xfailed.generate_okf.py --checkin sync.Verification notes:
body_ids.id_valuestores the raw DIGIWHIST id (hence forms-matching, confirmed againstextract_opentender.py:375), andtenders.countryis already ISO-normalised at extract time (UK→GB), so the query scopes on the GLEIF jurisdiction directly.Follow-ups / Known Limitations
Registry and lookup-router wiring are deliberately deferred. The issue itself notes that registering the source before the artifact-slimming work would break Render cold starts, and the existing tests
test_adapter_is_registered/test_warm_opentender_db_noop_when_not_registeredassertopentenderstays out ofREGISTRY— both still pass. This lands the identity-safe primitive so the router wiring becomes a small follow-up once the artifact issue is resolved.Addresses #29
Footer
Generated by Claude Fable 5 (brief, review), Claude Opus 4.8 (implementation)