fix(egocentric): derive episode identity and operator from the source - #522
Open
Sagar-024 wants to merge 2 commits into
Open
fix(egocentric): derive episode identity and operator from the source#522Sagar-024 wants to merge 2 commits into
Sagar-024 wants to merge 2 commits into
Conversation
Sagar-024
force-pushed
the
fix/519-episode-identity
branch
from
September 13, 2026 02:48
a945ab5 to
b9fe5c7
Compare
kstonekuan
requested changes
Sep 14, 2026
kstonekuan
left a comment
Contributor
There was a problem hiding this comment.
The invariance proof and the near-miss note are the right way to report this.
Two mutations pass that should not:
digest dropped from the id entirely 4 passed
sidecar field validation removed 4 passed
Your two shards use different member stems (factory002_worker001_00000, factory012_worker003_00000), so {source_stem}-{episode_number} already separates them and the digest never does the work. Its stated job is that source basenames are not identities. Either cover the case where that bites, or drop it.
The sidecar refusals have no case at all.
Not a gap, for the record: dropping the window start from the seed also passes, and that is correct now, since episode_number is global and disambiguates on its own. Your near-miss is not reachable by that route any more.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fix(egocentric): derive episode identity and operator from the source
Closes #519. Found in the multi-factory field run (discussion #503): the example hardcodes every episode id to
factory_051and the operator tofactory_051_worker_001, so a second factory's shard silently overwrites the first corpus and the survivors carry provenance from a worker who never recorded them.What changed
archive sha256, the member name, and the episode's window start. This mirrors the collision-resistant pattern
App.processuses for run directories (_source_artifact_directory_name, app.py:311), extended with the window start because one source member yields many excerpt windows. Two shards or two windows can never share a landing filename.operatoris composed from the per-clip sidecar'sfactory_idandworker_id. The sidecar rides in the same pinned tar as the video (identical stem,.jsonsuffix) and is read in the same_extract_source_videospass: one extract call, no new IO. A missing or unusable sidecar fails loudly, naming the member and the field.factorymetadata field recordsfactory_idexplicitly on everyepisode.
Three assumptions, redirect me if any are wrong
operatorkeeps thefactory_X_worker_Ycomposition so existing consumers see the same shape, now truthful per source. 3. A newfactorykey is added toepisode/v1. The record already carries example-specific keys (injected_fault,task_completion), and the canonical transform plus checks accept the field (the suite covers the full canonical path with it present).Invariance proof
The MCAP message payloads are byte-identical before and after the change. The full default-shard corpus (96 episodes) was rebuilt with the new code and compared against the existing landing episode-for-episode, matched by source member and window start: 96 of 96 payload byte-identical. Only ids, operator, the new
factoryfield, and filenames changed.Honest note: the first implementation of this fix collapsed those 96 windows into 11 files (one per member, the exact silent-overwrite shape this issue describes, reintroduced by my own id scheme). The default-shard rebuild caught it, and the window start moved into the digest seed.
Tests
two coexisting episodes with distinct derived ids and truthful provenance (the old code produced one, silently destroying the other).
source member stem.
Validation: ruff check, ruff format, ty on the changed files; 4 passed in tests/test_egocentric_prepare.py; zero non-ASCII and zero em dashes in the changed files.