chore(deps): scitex-clew becomes a floor (>=0.20.0), not an exact pin - #375
Open
ywatanabe1989 wants to merge 5 commits into
Open
chore(deps): scitex-clew becomes a floor (>=0.20.0), not an exact pin#375ywatanabe1989 wants to merge 5 commits into
ywatanabe1989 wants to merge 5 commits into
Conversation
All three occurrences in pyproject.toml — the core dependency (L82), the `clew` extra (L796) and the `dev` extra (L951) — read `scitex-clew==0.17.0`, a build tagged 2026-07-06. That exact pin is why a completed migration never arrived. clew moved its four stores off the retired embedded file engine onto the per-host PostgreSQL and released it as 0.20.0; the `==` meant this package kept resolving the July build regardless. The symptom is observable in this repo: running the suite re-creates `.scitex/clew/runtime/clew.db`, whose first sixteen bytes are `SQLite format 3`. A floor also fixes the general failure the pin caused, not just this instance of it — subsequent clew fixes now reach consumers without a manual bump here. The version chosen is the first clew release that carries the migration. Also corrects the `clew` extra's comment: since 0.20.0 clew resolves its stores through `scitex_dev.store.host_store()`, so it is no longer a standard-library-only extra.
…ithout it
The first commit moved scitex-clew to a floor. CI then showed the floor
alone is not enough: clew's stores resolve through
`scitex_dev.store.host_store()`, and the pinned `scitex-dev==0.28.0` has
no `scitex_dev.store` at all.
The failure mode is the same silent one the clew pin caused. The resolve
SUCCEEDED; the break landed at import:
[scitex-dev linter] WARNING: failed to load plugin 'clew':
ModuleNotFoundError: No module named 'scitex_dev.store'
with TestClewIntegration::test_status, ::test_hash_file and
TestCrossModuleWorkflow::test_io_save_then_clew_hash failing on py3.11,
3.12 and 3.13.
clew 0.20.1 (scitex-ai/scitex-clew#158) declares `scitex-dev>=0.49.2`, so
the bad combination is refused at resolve time from now on. That floor is
measured by a real write-then-read, not by importing: 0.43.1 is the first
release carrying `scitex_dev.store` and `host_store`, imports every symbol
clew imports, and still raises `TypeError: tuple indices must be integers
or slices, not str` inside `Store.rows()`. 0.28.0 FAIL, 0.43.1 FAIL,
0.46.0 FAIL, 0.48.0 FAIL, 0.49.0 FAIL, 0.49.2 OK, 0.49.3 OK, 0.50.0 OK,
0.57.0 OK.
0.57.0 is PyPI's newest, matching this file's established practice of
pinning the umbrella at latest. The clew floor moves to >=0.20.1 in all
three places for the same reason.
The previous local commit fixed the resolve-vs-import gap (clew needs scitex_dev.store.host_store(), scitex-dev==0.28.0 has none) by bumping scitex-dev to 0.57.0 — but kept it as an exact `==` pin in all three places (core dependency, `dev` extra, `linter` extra). That repeats the exact bug this PR exists to fix for clew: an exact `==` silently freezes a dependency and blocks every subsequent fix from reaching this package without a manual PR, which is what let scitex-clew==0.17.0 sit for two months while clew migrated off the retired storage engine. scitex-dev>=0.57.0 in all three occurrences. Verified against PyPI (JSON API + simple index) immediately before this commit: 0.57.0 is still the newest scitex-dev release, and it satisfies clew 0.20.1's declared `scitex-dev>=0.49.2` floor.
…floor Pushing the scitex-dev>=0.57.0 floor and re-running CI (run 33416452001 et al., all on py3.11/3.12/3.13) surfaced two more unresolvable transitive conflicts, not visible until the resolver actually ran: Because scitex-dev>=0.57.0 depends on scitex-scholar>=1.4.3 and scitex==2.30.8 depends on scitex-dev>=0.57.0, we can conclude that scitex==2.30.8 depends on scitex-scholar>=1.4.3. And because scitex==2.30.8 depends on scitex-scholar==1.4.2, ... unsatisfiable. Because scitex-dev>=0.57.0 depends on scitex-ssh>=1.1.0 and scitex depends on scitex-dev>=0.57.0, ... scitex depends on scitex-ssh==1.0.1, ... unsatisfiable. Both are minimal bumps to the first release that satisfies scitex-dev's own declared floor: scitex-scholar==1.4.2 -> ==1.4.3, scitex-ssh==1.0.1 -> ==1.1.0. Checked each new version's own requires_dist on PyPI before bumping - neither pulls in anything that conflicts with the rest of this file's pins. Verified locally with uv pip compile --extra all --extra dev pyproject.toml: resolves cleanly now, locking scitex-clew==0.20.1, scitex-dev==0.57.0, scitex-scholar==1.4.3, scitex-ssh==1.1.0.
…for CI Re-running the pytest matrix with the clew/scitex-dev floors resolving surfaced two more failures in exactly the three named tests, neither a pin problem: 1. test_hash_file / test_io_save_then_clew_hash asserted `len(h) == 32`. clew's hash_file() (see scitex-clew's clew-fix-truncated-hash-comparison) intentionally stopped truncating its SHA-256 digest to 32 hex chars — that truncation silently broke every external hash comparison. The full 64-char digest is the corrected, intended behaviour; the assertion here was written against the old truncated output and is now updated to match. 2. test_status hit `scitex_dev.store._errors.StoreTargetError: Cannot connect to Postgres store 'runs' at postgres[host=/home/runner/.scitex/pg/run ...]`. clew>=0.20.1's four stores resolve through scitex_dev.store.host_store(), which by default targets the fleet's central Postgres — unreachable (and not something a test should touch) from this off-fleet GitHub Actions runner. scitex-scholar, scitex-writer and scitex-dev's own CI hit the identical wall on 2026-08-29; scitex-dev shipped scitex_dev.store.testing.ephemeral_cluster_dsn() as the shared fix. Added a module-scoped `_clew_store_dsn` fixture that starts a throwaway local cluster via that helper and a `_clew_store` fixture that points SCITEX_STORE_DSN at it for the duration of one test; wired into all three of TestClewIntegration::test_status, TestClewIntegration::test_hash_file and TestCrossModuleWorkflow::test_io_save_then_clew_hash. Skips (not fails) if the runner has no initdb/pg_ctl — verified locally that path raises cleanly; GitHub's ubuntu-latest image ships PostgreSQL 16 server binaries under /usr/lib/postgresql/16/bin, which ephemeral_cluster_dsn() already knows to look for.
Collaborator
Author
|
Re-triggering the CLA check after promoting The check was failing for a reason unrelated to this PR: GitHub resolves
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
All three occurrences in
pyproject.toml— the core dependency (L82), theclewextra (L796) and thedevextra (L951) — readscitex-clew==0.17.0, a build tagged 2026-07-06.That exact pin is why a completed migration never arrived. clew moved its four stores off the retired embedded file engine onto the per-host PostgreSQL; the
==meant this package kept resolving the July build regardless. The symptom is observable in this repo today: running the suite re-creates.scitex/clew/runtime/clew.db, 84 KB, first sixteen bytesSQLite format 3.What ships this
scitex-clew0.20.0 is on PyPI as of 2026-08-31 (scitex_clew-0.20.0-py3-none-any.whl), released from scitex-ai/scitex-clew#156 + tagv0.20.0. It is the first release carrying the migration — 0.18.0/0.19.0/0.19.1 were cut frommainwhile the migration was being built ondevelop, and 0.19.2 was never published at all.Verified against the published artifact, not the version string — installed
scitex-clew==0.20.0from PyPI into a clean venv:_db/_core.pyimportsfrom scitex_dev.store import ANY_REVISION, Store, StoreTarget, WriterPolicy, host_storeand resolves targets viahost_store(pkg="scitex_clew", name=...).pyin the shipped package finds zerosqlite3importsset_db,use_db,get_active_db_path,resolve_db_path,VerificationDB.db_pathexport_manuscript_hints,is_claim_grounded,GROUNDING_REASONS,find_sessions_by_hash,hosts_for_hashall present.scitex/clew/runtime/and no file beginningSQLite format 3anywhere under the project rootFloor, not pin
A floor also fixes the general failure the pin caused, not just this instance of it: subsequent clew fixes reach consumers without a manual bump here. The
clewextra's comment is corrected too — since 0.20.0 clew resolves its stores throughscitex_dev.store.host_store(), so it is no longer a standard-library-only extra.