Skip to content

feat(dart): add provider-neutral voice capture coordinator - #51

Open
Pushkraj-Space wants to merge 3 commits into
october-dev:mainfrom
Pushkraj-Space:issue-36-capture-coordinator
Open

Pushkraj-Space wants to merge 3 commits into
october-dev:mainfrom
Pushkraj-Space:issue-36-capture-coordinator

Conversation

@Pushkraj-Space

Copy link
Copy Markdown
Contributor

Closes #36

Stacked on #50 (issue-34-idiomatic-runtime-interfaces): the coordinator builds on VoiceConnector and VoiceSession, so this diff includes that commit until #50 merges. Review the top commit (0262058) for the changes specific to this PR.

Summary

Adds the two runtime contracts from docs/voice-runtime.md to the pure-Dart murmur_protocol package:

  • VoiceProvider / ProviderSession (lib/src/provider.dart): the recognition adapter. Serialized addFrame whose awaited future is the backpressure signal, a warm setInputGate distinct from terminal finalize, a single-subscription ProviderEvent stream (partial / final / rejected / readiness / amplitude / failure / closed), and idempotent stop. ProviderCapability declares optional behaviors, each with a defined absent behavior.
  • VoiceCaptureCoordinator (lib/src/coordinator.dart): the single owner of one capture generation. Per generation it connects a fresh VoiceSession, opens a ProviderSession, gates and finalizes input, assembles the utterance once, and emits ordered murmur.v1 events. CaptureSnapshot is the small immutable view a UI renders.
  • Scheduler seam (lib/src/scheduler.dart): SystemScheduler for production, ManualScheduler for deterministic tests under virtual time. No fake_async or any other new dependency.

No proto or wire-model change. SessionControl, every RuntimeEvent arm, and SessionState (including WARM_MUTED) already exist. The existing fixtureSets corpus and the Python, TypeScript, and Rust runners are untouched.

Invariants enforced

  • One owner. Every async continuation checks the generation guard; late connects, provider opens, transcript chunks, and timers from a superseded generation never publish or deliver an utterance. Late acquisitions are released.
  • Gate immediately, keep the tail, keep the head. release() closes the coordinator gate synchronously. Frames accepted before the gate closes, pre-roll frames buffered while the provider opens, and gate/finalize control calls all flow through one ordered lane to the provider, so nothing reorders and a warm resume cannot reopen the gate before the previous flush completes. Pre-roll is bounded by PreRoll{maxFrames,maxBytes} with drop-oldest.
  • Finalize once. Concurrent finalize() calls share one future; onUtteranceFinalized fires at most once per finalization, with rejected and empty finals never buffered.
  • Stop safely. stop() is idempotent before, during, and after startup; repeated calls share one bounded cleanup.
  • Bound every wait. CaptureTimeouts{startup, endpoint, finalize, shutdown, warmHold} with typed outcomes: startup timeout is a retryable startup_timeout error, finalize timeout is graceful (FinalizationOutcome.timedOut), shutdown timeout detaches cleanup that then never publishes.
  • Honest readiness. Provider readiness when the provider declares it, otherwise an injectable CaptureReadinessPolicy (default: PCM sample threshold, never "any frame").

Modes: tap-to-speak, hold-to-talk (warm push-to-talk when the provider has warmGate, terminal finalize otherwise), hands-free (finals combined across endpoint, one utterance). wakePhrase and unspecified are rejected.

Shared lifecycle fixtures

conformance/manifest.json gains scenarioManifestVersion, scenarioTimeoutsMs, and scenarioSets. Each scenario is a JSONL file of ordered steps: a language-neutral driver vocabulary for the connector/provider boundary (start, release, connect_ok, provider_open_fail, frame, final, provider_closed, advanceMs, …) and emit steps carrying the exact murmur.v1 RuntimeEvent the coordinator must produce next. The emit contract (1-based per-session sequence, clock-driven monotonicTimeUs, session id scheme, explicit default booleans) is documented in conformance/README.md so a second SDK reproduces the lifecycle byte for byte.

Scenarios: release-during-startup, late-final, empty-final, provider-disconnect, finalize-timeout, rapid-restart, repeated-stop, warm-mute-cycle, plus startup-timeout and startup-failure so every failure step in the vocabulary is exercised.

test/scenario_test.dart replays them through the coordinator with scriptable fakes under ManualScheduler; tool/check_conformance.py validates the manifest shape, step vocabulary, each emit body via the existing diagnose(), and the emit contract.

Decisions beyond the written plan

  • Added the warmHold timeout; the four planned waits did not bound warm push-to-talk parking.
  • release() during startup cancels the generation (start() completes with cancelled), mirroring the [Core] Finish runtime interfaces on murmur.v1 #34 rule that a terminator wins against a pending start.
  • start() while warm-muted for the same source and mode resumes the generation instead of superseding it.
  • An unsolicited VoiceSession stopped while listening finalizes the utterance, so file and stored-audio sources end cleanly; error fails the generation.
  • Expected utterances live in the manifest per scenario rather than as ordered steps, because the callback is not on the wire.

Verification

  • dart format --set-exit-if-changed, dart analyze, dart test: 68 tests pass (race matrix in test/coordinator_test.dart, 10 scenario replays, existing runtime and conformance tests).
  • python3 tool/check_conformance.py: 15 sets, 44 lines; 10 scenarios, 145 steps.
  • Python and TypeScript conformance runners pass unchanged.
  • Six deliberately malformed scenarios (bad sequence, bad clock, unknown step, invalid emit body, non-synthetic text, extra argument) are each rejected by the checker.

Acceptance criteria

Criterion Where
Pure Dart, no Flutter/connector/vendor/product-command dependency lib/src/provider.dart, lib/src/coordinator.dart
Cross-process state/events map to murmur.v1 without Dart-only fields emit steps are RuntimeEvent bodies only
Shared fixtures cover the lifecycle another SDK must reproduce scenarioSets + emit contract
Hold release gates new input immediately, retains accepted tail release() + ordered lane
Cancelled/superseded session cannot update state or dispatch generation guard
Concurrent finalize shares one result, cannot dispatch twice single-flight finalize()
Startup, endpointing, finalization waits bounded CaptureTimeouts
Stop safe before/during/after startup, repeatable idempotent stop()
Tests for release-during-startup, late/empty final, disconnect, timeout, rapid restart, repeated stop unit matrix + scenarios

🤖 Generated with Claude Code

@harshsaver harshsaver left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this, @Pushkraj-Space. The coordinator tracks docs/voice-runtime.md closely, the generation guard and ordered lane are careful, and the scenario fixtures are exactly what #36 asked for. Locally: format/analyze are clean, 68/68 Dart tests pass, and check_conformance.py, Python and TS all pass.

Process: #50 is now merged, so please rebase so this diff is only 0262058. #52 also edits tool/check_conformance.py, so whichever lands second needs a small rebase.

Blocking

  1. Warm push-to-talk drops the second utterance on a fast re-press (lib/src/coordinator.dart:643-670, 558-560).

    • finalizePending is one flag shared across parks.
    • The sequence: hold → release() (flush 1 pending) → start() (resume) → speak → release() (flush 2 queued).
    • When flush 1 completes, _completeWarmFlush clears finalizePending, takes the utterance and arms warmHold while flush 2 is still running at the provider.
    • The second hold's finals then hit _gateOpen == false && !finalizePending and are discarded, and flush 2's completion returns early.
    • Repro with _Harness: frame 1 → release() → start(holdToTalk) → frame 2 → release() → Final('synthetic one') + completeFlush() → Final('synthetic two') + completeFlush(). This delivers ['synthetic one']; expected both.

    Please track flushes per park (an outstanding-flush count or a per-_parkWarm token), and only clear finalizePending or arm warmHold when the latest park's flush settles. Also store the _parkWarm deadline timer (:648) in a field so resume() and end() cancel it; today a flush timeout that fires after a resume delivers the resumed hold's finals early. Add this sequence as a unit test, and ideally as a warm-rapid-rehold scenario.

Nits (optional)

  • coordinator.dart:208-211: the broadcast events/stateChanges controllers are never closed. Consider a dispose().
  • coordinator.dart:489-495: drop-oldest in the pre-roll/backpressure lane is silent. A warning event or counter would help debugging.
  • coordinator.dart:533-538: a throwing finalize()/setInputGate() is swallowed and reports timedOut: false. Either document that providers must also emit ProviderFailure, or surface the error.
  • coordinator.dart:609-613: a source reporting stopped during startup waits for startup_timeout. Failing fast would be friendlier.

Pushkraj-Space added a commit to Pushkraj-Space/murmur that referenced this pull request Sep 28, 2026
Address the blocking review on october-dev#51 by tracking each warm park separately,
settling its tail inside the ordered provider lane, and allowing only the
latest park to clear finalizePending or start warmHold. Cancel the stored
flush deadline on resume, terminal finalization, and end; ignore duplicate
or overdue acknowledgements.

Address all four optional comments: add idempotent coordinator disposal,
expose droppedFrames in the snapshot, surface provider control failures,
and fail promptly when the source stops during startup.

Add deterministic regression tests and the shared warm-rapid-rehold
scenario. Dart: 80 tests pass; format and analysis are clean. Shared
conformance, Python, and TypeScript checks and tests pass.
@Pushkraj-Space

Copy link
Copy Markdown
Contributor Author

Addressed the blocking issue and all four optional comments:

  1. Rapid warm re-hold: each park now has an increasing token and a completion watermark. An earlier flush delivers its own tail without clearing a later park's finalizePending, cancelling its deadline, or starting warmHold. Flush completion runs inside the ordered lane before input reopens. The stored warm-flush timer is cancelled on resume, terminal finalization, and end. Late acknowledgements after a timeout cannot deliver text twice. Added unit coverage for rapid re-hold, deadline ownership, resume/stop cancellation, and overlapping flush timeouts, plus warm-rapid-rehold in the shared scenario manifest. In a temporary copy using the original coordinator, that scenario fails with ['synthetic one']; the fixed coordinator delivers ['synthetic one', 'synthetic two'].

  2. Stream lifetime: added idempotent dispose(). It stops capture with bounded cleanup and closes both broadcast controllers. Paused listeners do not delay disposal, and future start() calls throw StateError. Tests cover stream completion, repeated disposal, disposal before start, and paused listeners.

  3. Silent drop-oldest: added CaptureSnapshot.droppedFrames, a per-generation counter covering both pre-roll and backpressure drops. Tests cover frame and byte limits and resetting the count for a new generation. No transcript or audio content is logged.

  4. Throwing provider controls: control exceptions now fail capture and publish an error event and error snapshot, even without a separate ProviderFailure. Typed VoiceError values are preserved; other exceptions use provider_control_failed with a fixed message. Failed capture does not deliver an utterance. The provider contract documents this behavior, and tests cover synchronous gate errors, asynchronous finalize errors, and typed errors. timedOut remains a deadline indicator; failures are reported through the error event/snapshot.

  5. Source stops during startup: this now immediately fails with retryable source_stopped, cancels the startup timer, and cleans up. A test also verifies that a provider opening afterward is stopped without resurrecting the session.

Process: the local main history includes merged #50 (f5510eb), whose tree matches the stacked prerequisite 354919b. I have not rebased: the task explicitly says not to commit or push, and rebasing would rewrite/create commits. The committing step should replay the coordinator change and these fixes onto main while excluding the already-merged prerequisite. These review fixes leave tool/check_conformance.py unchanged; if #52 lands first, reconcile that file's existing PR changes during the rebase and rerun conformance checks.

Verification: Dart 3.13.3: offline dependency resolution with the lockfile enforced, format check (15 files unchanged), dart analyze (no issues), and dart test (80 passed). python3 tool/check_conformance.py passes with 15 fixture sets, 44 lines, 11 scenarios, and 167 steps. Python conformance passes. TypeScript npm run check and npm test pass using existing locally installed dependencies; network access was unavailable for a fresh npm install. git diff --check passes. No dependency lockfiles were changed, and no commit or push was made.

@Pushkraj-Space

Copy link
Copy Markdown
Contributor Author

The rapid re-hold fix remains in place: each warm park owns its flush completion, obsolete deadlines are cancelled, and both utterances survive a fast re-press. The four optional review points are addressed by idempotent dispose(), per-generation droppedFrames, surfaced provider control failures, and immediate source_stopped failure during startup. The shared warm-rapid-rehold.jsonl scenario is tracked.

The audit's callback regression is now fixed and tested. A throwing onUtteranceFinalized reports the original exception and stack asynchronously to the delivery zone's uncaught-error handler. It neither becomes a provider failure nor disappears during terminal finalization; release/finalize still complete, delivery is not retried, and warm capture remains usable. Four tests cover warm and terminal acknowledgement/timeout paths and late acknowledgements. The two acknowledgement cases fail on the previous code. ProviderSession.stop() now documents failure during terminal finalization versus cleanup; the README and changelog cover disposal, dropped frames, and callback error handling.

The scratch branch has been rebased onto main 73ab6ea, with only the coordinator commit and original review-fix commit; 354919b is absent and both original messages are preserved. This corrects the earlier comment that a later step would perform the rebase. No files conflicted, so no manual resolutions were needed. Main's contents are retained. The audit independently verified remote main at this same commit and confirmed #52 is open, not merged; this pass's fetch retry still failed DNS. If #52 lands first, another rebase must preserve both sets of changes to tool/check_conformance.py.

Verification on the rebased tree plus these fixes: Dart format (15 files, 0 changed), analysis, and all 84 tests pass, including 11 scenario replays. Shared conformance: 15 fixture sets, 44 lines; 11 scenarios, 167 steps. Python: 1 test passes. TypeScript check and 1 test pass using available local dependencies. The committed scratch branch merges cleanly into 73ab6ea; the audit-fix patch applies cleanly to that tip and reproduces the tested files. The owner must import the rebase and apply the saved patch before committing. Nothing was pushed.

Pushkraj-Space and others added 3 commits September 29, 2026 02:34
Add VoiceProvider/ProviderSession recognition contracts and the
VoiceCaptureCoordinator, the single owner of one capture generation:
per-generation session and provider ownership, a synchronous input gate
on release, a bounded pre-roll drained through one ordered provider lane,
single-flight finalization with an exactly-once utterance callback,
warm push-to-talk parking, hands-free endpointing, and idempotent stop.

Every wait (startup, endpoint, finalize, shutdown, warm hold) is bounded
through an injected Scheduler with a ManualScheduler for deterministic
tests; no new dependency.

Freeze the lifecycle as language-neutral scenarioSets in the conformance
manifest with a documented driver vocabulary and emit contract, replay
them in the Dart scenario test, and validate them in the repository
checker. All changes are additive to murmur.v1 and the existing fixtures.

Closes october-dev#36

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Address the blocking review on october-dev#51 by tracking each warm park separately,
settling its tail inside the ordered provider lane, and allowing only the
latest park to clear finalizePending or start warmHold. Cancel the stored
flush deadline on resume, terminal finalization, and end; ignore duplicate
or overdue acknowledgements.

Address all four optional comments: add idempotent coordinator disposal,
expose droppedFrames in the snapshot, surface provider control failures,
and fail promptly when the source stops during startup.

Add deterministic regression tests and the shared warm-rapid-rehold
scenario. Dart: 80 tests pass; format and analysis are clean. Shared
conformance, Python, and TypeScript checks and tests pass.
Address the audit's callback-exception regression in warm and terminal
capture. Report original app errors and stack traces asynchronously to
the delivery zone without failing capture or swallowing finalization
errors. Keep release/finalize completion and exactly-once delivery intact.

Add four regression tests for acknowledgements and timeouts, including
continued warm capture and late acknowledgements. Document stop failures,
callback error handling, dispose(), and per-generation droppedFrames.

Dart: 84 tests pass; format and analysis clean. Shared conformance:
11 scenarios, 167 steps. Python and TypeScript checks/tests pass.

@harshsaver harshsaver left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Pushkraj-Space. Re-review of 5647abb: the blocking item is resolved, approving.

  • Rapid warm re-hold: I re-ran the original repro (frame 1 -> release -> start(holdToTalk) -> frame 2 -> release -> Final one + flush -> Final two + flush). It now delivers ['synthetic one', 'synthetic two'], clears finalizePending, and parks warm-muted until warmHold. A three-press variant also delivers all three. The per-park token (coordinator.dart:692-731), the stored _warmFlushTimer cancelled in resume/finalizeTerminal/end, the unit test (coordinator_test.dart:320) and the warm-rapid-rehold scenario cover it well.
  • All four nits are addressed.
  • Local: dart format/analyze clean, 84/84 tests, check_conformance.py passes (11 scenarios, 167 steps).

To merge: #52 and #54 just landed, so this now conflicts in tool/check_conformance.py (both sides add functions; keep both and combine the final summary print) and CHANGELOG.md (keep both). Please merge main and push; I checked all three combined locally (103/103 tests, conformance passes), so it's mechanical. We'll merge as soon as CI is green.

Also heads-up: your #55 adds a competing lib/src/provider.dart (mute()/unmute(), void finalize()) vs this PR's contract (setInputGate, finalize() returning FinalizationOutcome). #55 should build on this one's contract.

Non-blocking, fine as follow-ups:

  • coordinator.dart:692-710: the second park's deadline starts at release, but its flush can't be sent until the first flush acks and the gate reopens, so a slow first flush uses up the second park's budget and a later final is dropped. Defensible as a total latency bound; a one-line doc comment saying so would help.
  • test/support/fake_connector.dart:98-100 uses broadcast(sync: true), the pattern we just removed from #52's example. Harmless today, but switching to async delivery (or to #54's public fake) would keep the fakes consistent; :10-28 also duplicates #54's format/payload helpers.
  • CHANGELOG.md:9-16: nothing has been released, so the "Preserve both utterances..." and "Report utterance callback exceptions..." bullets describe review fixes to an unreleased feature. Consider folding them into the feature entry.

@harshsaver

Copy link
Copy Markdown
Member

Correction to my note above about #55: I had the direction backwards. #55 implements the dedicated provider-contract issue (#7), and its description already says this PR will rebase onto it. #55 is now merged, so please:

  1. Rebase onto main: replace this PR's provisional lib/src/provider.dart and test/support provider fake with feat(dart): add streaming transcription provider contract #55's contract and fake — start(format) -> createSession + start(), setInputGate -> mute/unmute, ProviderClosed -> ProviderFailure + stream done, as feat(dart): add streaming transcription provider contract #55's description lays out.
  2. Resolve the docs(connectors): complete the voice connector authoring guide #52/feat(dart): add a deterministic fake voice connector #54 conflicts (tool/check_conformance.py: keep both sides and combine the summary print; CHANGELOG.md: keep both). feat(licensing): add default-deny engine and model licensing audit #57 may also require fixture registration if it lands first.
  3. Keep the warm re-hold fix working under feat(dart): add streaming transcription provider contract #55's finalize() semantics — the per-park flush tracking currently relies on FinalizationOutcome, so please make sure the warm-rapid-rehold scenario and coordinator_test.dart:320 still pass.

Since the rebase changes the coordinator's provider integration, I'll re-review once it's pushed. The approval above covers the coordinator logic as reviewed at 5647abb.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Core] Implement the reliable voice capture coordinator

2 participants