Skip to content

Group sync reuse witness fields into one typed value #319

Closed

Description

@kstonekuan

Current behavior

_SyncCompletion describes one optional reuse witness with three independent optional fields at src/hflow/app.py:247-274:

source_digest: str | None
ffmpeg_version: str | None
transform_kind: _TransformKind | None

The current writer always supplies the complete set. The lenient marker parser can produce every partial combination at src/hflow/app.py:355-400. The reuse gate then checks the fields separately at src/hflow/app.py:1154-1206.

The behavior is fail-closed today, but the type does not express the rule the code relies on: a reuse witness is complete or absent.

Verified against main at fbfd553.

Pattern to copy

RetainedDatasetSnapshotBackup groups two values that only have meaning together, and the report carries that group as one optional value.

What to build

Introduce a frozen _SyncReuseWitness holding source_digest, ffmpeg_version, and transform_kind. Change _SyncCompletion to carry reuse_witness: _SyncReuseWitness | None.

Parse a complete, known witness into _SyncReuseWitness. Treat a missing, partial, malformed, or future witness as absent so old markers remain readable for non-sync stages and sync still retranscodes on any doubt.

Keep the JSON marker shape byte-for-byte compatible. This is an internal representation change.

Definition of done

  1. _SyncCompletion cannot represent a partial reuse witness.
  2. The current marker still contains the same six keys with the same JSON bytes and ordering.
  3. A marker with all valid witness fields can reuse an unchanged default transform.
  4. A legacy marker with no witness remains readable and retranscodes once.
  5. Every partial witness combination remains readable and cannot trigger reuse.
  6. An unknown transform_kind remains readable and cannot trigger reuse.
  7. Transform overrides still never reuse.
  8. No transform behavior version bump is needed because canonical bytes and marker bytes stay unchanged.

Non-goals

  • Migrating existing marker files
  • Versioning transform overrides
  • Changing the marker format
  • Changing which canonical episodes qualify for reuse

Validation

uv run ruff check --fix
uv run ruff format
uv run ty check
uv run pytest -q tests/test_sync_reuse.py
uv run pytest -q

Activity

  1. added
    enhancementNew feature or request
    help wantedExtra attention is needed
    advancedNeeds codebase familiarity; not a starter issue
    on Aug 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    advancedNeeds codebase familiarity; not a starter issueenhancementNew feature or requesthelp wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions