Skip to content

chore(sqlite-out): the retired engine's name reaches zero (20 files -> 0), and two committed databases go with it - #372

Merged
ywatanabe1989 merged 2 commits into
developfrom
chore/sqlite-eradication
Aug 31, 2026
Merged

chore(sqlite-out): the retired engine's name reaches zero (20 files -> 0), and two committed databases go with it#372
ywatanabe1989 merged 2 commits into
developfrom
chore/sqlite-eradication

Conversation

@ywatanabe1989

Copy link
Copy Markdown
Collaborator

Why

We do not use the retired embedded engine. Runtime state lives in PostgreSQL on
55432. Its name nevertheless survived in 20 tracked files here — including two
live database files committed into the tree.

Standing ruling applied: nothing outside docs/adr/ may keep the name, and
no guard test asserting its absence — so none is added.

Deleted outright

  • .scitex/clew/runtime/db.sqlite and
    examples/_legacy/scitex/clew/multi_parent/scitex/clew.db — two real database
    files committed into the repository.
  • examples/_legacy/notebooks/19_scitex_db.ipynb — a tutorial for the retired
    engine end to end: raw client connections, SELECT ... FROM sqlite_master
    introspection, "SQLite for local work" as the closing advice. Nothing survives
    removing the engine from it, so the notebook goes, and the master index's two
    links to it go with it.

Rewritten to name PostgreSQL

  • README's scitex.db section (summary line, the worked example, the closing
    paragraph).
  • docs/05_ADDITIONAL_MODULES.md module table row.
  • The clew provenance prose in docs/sphinx/core_concepts.rst and
    docs/sphinx/modules/io.rst.
  • examples/_legacy/scitex/io/schematic_io_concept.py and its recorded _out/
    YAML — the schematic's store box, kept consistent between source and output.

Tests

  • tests/integration/test_integration.py no longer asserts the engine's wrapper
    class is exported from stx.db; it still asserts PostgreSQL and
    check_health.
  • tests/integration/test_cross_package_imports.py drops
    scitex_io._load_modules._sqlite3. That module was deleted in scitex-io
    0.3.0, so the entry has silently skipped ever since — it was testing
    nothing.

Also

  • The name leaves .gitignore (both copies of the Django block), the clew
    comment in pyproject.toml, the stdlib table in
    scripts/maintenance/_pypi_packages.py (behaviour-neutral: the name is not in
    KNOWN_PACKAGES either, so get_package_name still returns None), the
    CrossRef env example, and the legacy io-module guideline's format table.
  • examples/_legacy/notebooks/27_scitex_scholar.ipynb no longer opens a raw
    client against the third-party impact-factor database. Scholar's own
    enrich_by_default=True already performs that enrichment — the removed blocks
    were labelled as fallbacks for when it did not.

A finding this PR cannot fix

Running the suite after deleting the two committed database files re-created
.scitex/clew/runtime/clew.db — 84 KB, first sixteen bytes SQLite format 3.
The writer is scitex-clew==0.17.0, an external pin; this repo cannot stop the
write. What it can stop is the file being committed again, so the second commit
adds an ignore rule for that path. Nothing was ignoring it before — the previous
file was tracked, so it never appeared as untracked noise and no rule was ever
added. The engine still runs at runtime via scitex-clew; that belongs in the
scitex-clew leaf, not here.

Verification

Positive control first — the same search, same root, for a string known to be
present, so the zero below cannot be a broken search:

$ git grep -il scitex HEAD | wc -l
2054

Then the actual query:

$ git grep -il sqlite HEAD
$ echo $?
1

Zero matches across the whole tree. docs/adr/ holds none in this repo either,
so no exemption is being leaned on.

561 passed, 101 skipped, 44 xfailed, 17 xpassed locally, 0 failed.

CI may still go red on the py3.13 leg for a reproducible segfault
(Fatal Python error: Segmentation fault, no JUnit report) that predates this
branch and is tracked separately — it is not caused by anything here.

Overlap

#362, #364, #365, #366 and #368 are open against the same subject. This branch
is cut fresh from origin/develop and takes the count to a measured zero; if it
lands, those should close.

…s/adr

We do not use the retired embedded engine; runtime state lives in
PostgreSQL. Its name survived in 20 tracked files. This clears all of
them, keeping only docs/adr/ (untouched) and the Zotero/Optuna
integrations (which carry none).

Deleted outright:

- `.scitex/clew/runtime/db.sqlite` and
  `examples/_legacy/scitex/clew/multi_parent/scitex/clew.db` — two live
  database files committed into the tree.
- `examples/_legacy/notebooks/19_scitex_db.ipynb` — a tutorial for the
  retired engine end to end (raw client connections, `sqlite_master`
  introspection). Nothing survives removing the engine from it, so the
  notebook goes and the master index's two links to it go with it.

Rewritten to name PostgreSQL, which is what we actually run:

- README's `scitex.db` section, `docs/05_ADDITIONAL_MODULES.md`, and the
  clew provenance prose in `docs/sphinx/core_concepts.rst` /
  `docs/sphinx/modules/io.rst`.
- `examples/_legacy/scitex/io/schematic_io_concept.py` and its recorded
  `_out/` YAML — the schematic's store box.

Also:

- `tests/integration/test_integration.py` no longer asserts the engine's
  wrapper class is exported; it still asserts `PostgreSQL` and
  `check_health`.
- `tests/integration/test_cross_package_imports.py` drops
  `scitex_io._load_modules._sqlite3` — that module was deleted in
  scitex-io 0.3.0, so the entry has silently skipped ever since.
- The engine's name leaves `.gitignore` (both copies), `pyproject.toml`'s
  clew comment, the stdlib table in `scripts/maintenance/_pypi_packages.py`,
  the CrossRef env example, and the legacy io-module guideline's format
  table.
- `examples/_legacy/notebooks/27_scitex_scholar.ipynb` no longer opens a
  raw client against the third-party impact-factor database; Scholar's
  own `enrich_by_default=True` already does that enrichment.

No guard test asserting the name's absence is added, deliberately.
Running the suite after deleting the two committed database files
re-created `.scitex/clew/runtime/clew.db` (84 KB, magic bytes
`SQLite format 3`). The writer is the `scitex-clew` package pinned in
`pyproject.toml`, so this repo cannot stop the write — but it can stop
the file from being re-committed, which is how it got in.

Nothing was ignoring that path: the previous file was TRACKED, so it
never showed up as untracked noise and nobody added a rule.
@ywatanabe1989
ywatanabe1989 merged commit a365d8a into develop Aug 31, 2026
5 of 6 checks passed
@ywatanabe1989
ywatanabe1989 deleted the chore/sqlite-eradication branch August 31, 2026 14:45
@github-actions github-actions Bot locked and limited conversation to collaborators Aug 31, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant