Skip to content

fix(lerobot): namespace cache by resolved revision - #328

Merged
kstonekuan merged 4 commits into
Hebbian-Robotics:mainfrom
mikemikimike:fix/lerobot-cache-by-revision
Sep 1, 2026
Merged

kstonekuan merged 4 commits into
Hebbian-Robotics:mainfrom
mikemikimike:fix/lerobot-cache-by-revision

Conversation

@mikemikimike

@mikemikimike mikemikimike commented Aug 31, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Namespace LeRobot source caches by the resolved immutable repository SHA.
  • Prevent reused output directories from mixing artifacts across revisions.
  • Preserve same-SHA cache reuse and manifest provenance.

Closes #294

Validation

  • ruff check, ruff format --check, ty check
  • focused LeRobot converter tests
  • full pytest suite: 1250 passed, 8 skipped

@github-actions

Copy link
Copy Markdown

👋 Hi @mikemikimike — thank you so much for your first contribution to HFlow!

A maintainer will review your pull request as soon as possible. In the meantime:

💡 Tip: one open pull request per contributor at a time. Issues with an assignee are taken; everything else is fair game.

We are excited to have you here and appreciate your help making the project better! 🙌

@kstonekuan kstonekuan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, and welcome. The fix is the right one-liner and the test earns its place: routing tag-a and branch-a to the same sha and asserting the marker file survives proves reuse and separation in the same assertion, rather than just checking two directory names differ.

One thing before merge, and it's a consequence of your change rather than something you inherited.

Until this PR the cache path was a constant. Now it ends in a string that came off the network, and nothing checks its shape. _hf_repo_info only requires a non-empty str:

resolved_revision = repository_information.get("sha")
if not isinstance(resolved_revision, str) or not resolved_revision.strip():

So DoD 5 ("the cache remains entirely below output_dir / "_lerobot_cache"") is stated but not enforced:

sha='8f2c1ab9...'                      -> /tmp/probe-328-out/_lerobot_cache/8f2c1ab9...   below: True
sha='../../../../tmp/probe-328-escaped' -> /tmp/probe-328-escaped                          below: False
sha='/tmp/probe-328-absolute'           -> /tmp/probe-328-absolute                         below: False

To be clear about the size of this: it needs Hugging Face to return a hostile sha, so it is not a live path traversal and I don't want the commit message calling it one. It matters because the DoD names it, and because your change is what turned a constant path segment into a remote-controlled one.

Refuse a sha that isn't a commit hash where it's resolved, next to the check above. A commit sha is hex, so something like this is enough:

if not re.fullmatch(r"[0-9a-f]{7,64}", resolved_revision):
    raise ValueError(f"Hugging Face returned a malformed commit sha for {repo_id}@{revision}")

Then a case in your existing parametrized style asserting ../../evil is refused. Your test stubs _hf_repo_info, so this one wants to go at the real function.

Gate is otherwise clean on your head: 1252 passed / 6 skipped.

@kstonekuan kstonekuan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, merging. DoD 5 is enforced now rather than assumed:

ACCEPTED '8f2c1ab9d4e5f60718293a4b5c6d7e8f90a1b2c3'  -> below _lerobot_cache: True
ACCEPTED '8f2c1ab'                                   -> below _lerobot_cache: True
refused  '../../../../tmp/probe-328-escaped'
refused  '/tmp/probe-328-absolute'
refused  '..'
refused  'C:/windows'
refused  '8f2c1ab9/../../../etc'

Putting it in _hf_repo_info rather than at the join is the right place: the sha is wrong there, not just unsafe as a path, and every future caller of the resolved revision gets it. The match="malformed commit sha" in your test matters too, since a bare pytest.raises(ValueError) there would also pass on the pre-existing empty-sha refusal.

I merged current main in and pushed as 62c3156, because main moved a long way under this and #326 landed in both files you touch. That surfaced a real interaction, which is why I mention it rather than just fixing it quietly:

FAILED tests/test_lerobot_converter.py::test_hf_repo_info_valid_json_still_resolves

#326's fixture uses {"sha": "abc123"}, six hex characters, one short of your {7,64} minimum. Your constraint is right and the fixture was the unrealistic part, so I changed it to abc1234. Nothing about either change is wrong; it's just a collision only visible on the merged result, which is the argument for the bound you picked being a real one.

Full gate on the merged result: clean, 1340 passed / 6 skipped.

One small thing, no action needed. Your namespacing test uses sha-a and sha-b, which the validator you just added would reject. It passes because that test stubs _hf_repo_info wholesale, so it never sees them. Fine as is, but if you touch it again, hex fixtures would keep the two tests telling the same story.

@kstonekuan
kstonekuan merged commit 8116404 into Hebbian-Robotics:main Sep 1, 2026
7 checks passed
ayam04 added a commit to ayam04/hflow that referenced this pull request Sep 1, 2026
main Hebbian-Robotics#328 namespaces the materialized archive under _lerobot_cache/<resolved sha>; export resolved the flat path and would fail on a fresh materialization. Resolve the single namespaced subdirectory after the import and fail loudly on an unexpected layout. The test fake mirrors the namespaced layout.

Signed-off-by: Ayamullah Khan <ayamullahkhan04@gmail.com>
ayam04 added a commit to ayam04/hflow that referenced this pull request Sep 2, 2026
main Hebbian-Robotics#328 namespaces the materialized archive under _lerobot_cache/<resolved sha>; export resolved the flat path and would fail on a fresh materialization. Resolve the single namespaced subdirectory after the import and fail loudly on an unexpected layout. The test fake mirrors the namespaced layout.

Signed-off-by: Ayamullah Khan <ayamullahkhan04@gmail.com>
kstonekuan pushed a commit that referenced this pull request Sep 2, 2026
…t Dataset v3 (fixes #190) (#273)

* feat(lerobot): export an HFlow-curated selection as a loadable v3 dataset (fixes #190)

* fix(lerobot): repoint export at the public importer API

Review follow-up on #273:

- Tests run for real now: dropped pytest.importorskip("examples.lerobot.prepare")
  (module-scope skip, the whole file collapsed into one silent skip).
  The suite uses the sys.path namespace-import approach the repo already
  uses in examples/egosuite_evaluation/tests; 8 tests execute, no skips.
- export.py no longer touches examples/lerobot/prepare private helpers.
  The source archive is materialized through the public
  hflow.import_lerobot_dataset entry point (hflow import lerobot) and
  consumed from the _lerobot_cache artifacts it writes; chunk fetches
  became existence checks against that archive. --camera-keys is now
  required so the importer validates the requested cameras (the old
  "default: all" branch could not resolve keys on the public surface).
- Rebased onto current main (covers the first-class importer move).
- README export examples updated for --camera-keys.

Validation: pytest tests/test_lerobot_export.py -> 8 passed (0 skipped);
ruff check/format clean; ty check clean; full suite 1256 passed, 6
skipped, 2 pre-existing failures (catalog_ui flake re-passed; end_to_end
ffmpeg filter_script:v unsupported by the WSL ffmpeg build, reproduced
on clean main without this change).

Signed-off-by: Ayamullah Khan <ayamullahkhan04@gmail.com>

* test(lerobot): pin provenance digests and v3 structural validation

Second review round found _sha256 and _validate_v3 could both be neutered with no test noticing. Add a digest test asserting each provenance digest equals the sha256 of the file it names and that distinct files get distinct digests, plus one rejection test per structural check (incoherent info.json, window != length, missing video). RED-verified: with both checks neutered the four new tests fail while the eight existing ones still pass.

Signed-off-by: Ayamullah Khan <ayamullahkhan04@gmail.com>

* fix(lerobot): consume the revision-namespaced importer cache

main #328 namespaces the materialized archive under _lerobot_cache/<resolved sha>; export resolved the flat path and would fail on a fresh materialization. Resolve the single namespaced subdirectory after the import and fail loudly on an unexpected layout. The test fake mirrors the namespaced layout.

Signed-off-by: Ayamullah Khan <ayamullahkhan04@gmail.com>

* fix(lerobot): carry source (chunk, file) identity through the curated export

Round-3 export-logic findings from review, plus the #353 cache coupling.

The exporter wrote every camera to chunk-000/file-000.mp4 and skipped the
copy when that path existed, so a selection spanning more than one source
video chunk published the wrong bytes while the episode rows still carried
their own window offsets. The destination now keeps the source (chunk,
file) identity: distinct source files land in distinct output files, each
episode row references the file its frames actually come from, and the
copy is deduplicated by identity instead of by first-write.

The importer now separates video cache files by index (#353), so the cache
lookup reads the {camera}-chunk{chunk}-file{file}.mp4 name and the window
reader keeps a 0 chunk/file index instead of dropping it as falsy.

Episode data refs are plain integers again and one data parquet is written
per selected episode, so the exported dataset reads back with its own
reader and the data_path template formats over every row. Sources without
the v3 index column are refused loudly instead of silently writing a
global frame_index over the per-episode one. The dead official-API
validation pass (lerobot never installed anywhere) is dropped; the
structural checks now own validation, including that every data and video
reference formats through the dataset templates and resolves to an
existing file. total_videos and the provenance digests are per-file.

Signed-off-by: Ayamullah Khan <ayamullahkhan04@gmail.com>

* test(lerobot): multi-chunk exports, read-back, and template-format refusals

Every round-3 finding is pinned by at least one test that fails on the old
code and passes now.

The fixture writes the #353 cache names (chunk and file index) and gains a
second video chunk with distinguishable bytes. test_export_multi_chunk_videos
asserts both source files land byte-exact in distinct destinations, the
second chunk is no longer absent, episode rows reference the file their
frames come from, total_videos counts distinct files, and the provenance
digests differ per file. test_export_output_readback feeds the exported
episode parquet through the exporter own reader and substitutes the
data_path template, the exact loadability contract the review described.
test_export_validation_failure_leaves_no_destination pins the atomic
publish guarantee: a validator refusal propagates and leaves a
pre-existing destination untouched and no new destination behind.
test_validate_v3_rejects_non_integer_data_refs stages the review exact
chunk-000 string and asserts the refusal. The noncontiguous test now
asserts the plain data file indexes and that the per-episode data file
exists with a per-episode frame_index restart.

Signed-off-by: Ayamullah Khan <ayamullahkhan04@gmail.com>

---------

Signed-off-by: Ayamullah Khan <ayamullahkhan04@gmail.com>
rakesh0x added a commit to rakesh0x/hflow that referenced this pull request Sep 2, 2026
_stop_duckdb_ui_server called stop_ui_server() whenever the server start
hook ran, but the function only exists once the ui extension is loaded.
When it is not (e.g. the lifecycle test monkeypatches the start hook),
DuckDB auto-installs the extension from the network during shutdown --
seconds on a slow connection, and pointless: there is no server to stop
if the extension was never loaded. That network fetch inside a teardown
another thread waits on with a fixed timeout is almost certainly the
flaky test_catalog_ui timing failure reported on Hebbian-Robotics#326, Hebbian-Robotics#327 and Hebbian-Robotics#328.

Guard the call with a duckdb_extensions() loaded check: production
always LOADs ui before the server starts, so the stop call is preserved,
and a connection that never loaded the extension now closes directly.

Pin both shutdown branches with tests that record the statements issued
on the connection (Hebbian-Robotics#336 DoD): the stubbed-starter path must issue no
stop_ui_server() call, and a session whose ui extension is loaded must
still stop its server. The recording proxy does not delegate the stop
call to DuckDB -- on a connection without the loaded extension that
would itself trigger the network auto-install this fix removes, and
asserting DuckDB's install behavior is not the point.

Closes Hebbian-Robotics#336
kstonekuan pushed a commit that referenced this pull request Sep 2, 2026
_stop_duckdb_ui_server called stop_ui_server() whenever the server start
hook ran, but the function only exists once the ui extension is loaded.
When it is not (e.g. the lifecycle test monkeypatches the start hook),
DuckDB auto-installs the extension from the network during shutdown --
seconds on a slow connection, and pointless: there is no server to stop
if the extension was never loaded. That network fetch inside a teardown
another thread waits on with a fixed timeout is almost certainly the
flaky test_catalog_ui timing failure reported on #326, #327 and #328.

Guard the call with a duckdb_extensions() loaded check: production
always LOADs ui before the server starts, so the stop call is preserved,
and a connection that never loaded the extension now closes directly.

Pin both shutdown branches with tests that record the statements issued
on the connection (#336 DoD): the stubbed-starter path must issue no
stop_ui_server() call, and a session whose ui extension is loaded must
still stop its server. The recording proxy does not delegate the stop
call to DuckDB -- on a connection without the loaded extension that
would itself trigger the network auto-install this fix removes, and
asserting DuckDB's install behavior is not the point.

Closes #336
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LeRobot cache can mix revisions when an output directory is reused

2 participants