Skip to content

fix(voice): clean up turn variables when committing a user turn manually#1438

Closed
toubatbrian wants to merge 1 commit intomainfrom
claude/quirky-galileo-kLqAD
Closed

fix(voice): clean up turn variables when committing a user turn manually#1438
toubatbrian wants to merge 1 commit intomainfrom
claude/quirky-galileo-kLqAD

Conversation

@toubatbrian
Copy link
Copy Markdown
Contributor

Summary

Automated port of livekit/agents#5671 into agents-js.

When clearUserTurn() is invoked (the path used by manual turn detection to abandon an in-progress turn), it previously left several turn-scoped fields stale:

  • lastFinalTranscriptTime, lastSpeakingTime, speechStartTime would carry over into the next turn, corrupting the EOU transcriptionDelay / endOfUtteranceDelay metrics and the preemptive-generation startedSpeakingAt for the next turn.
  • The current user_turn span was kept open and would be reused for the next user turn instead of starting a fresh one. Any provider STT request ids collected so far would also be flushed onto the wrong span.

This fix resets those turn variables and ends the in-progress user_turn span so the next speech starts a fresh span and metrics window.

cc @toubatbrian @livekit/agent-devs

Ported features

  • agents/src/voice/audio_recognition.ts — extends clearUserTurn() to:
    • reset lastFinalTranscriptTime to 0, speechStartTime and lastSpeakingTime to undefined (mirrors _last_final_transcript_time = None, _speech_start_time = None, _last_speaking_time = None in Python)
    • end the in-progress userTurnSpan (if recording) and clear the cached span + accumulated sttRequestIds, so the next speech starts a fresh span and the next ended span doesn't inherit stale provider request ids

Implementation nuances vs. Python

  • _vad_speech_started field has no JS equivalent. In Python, _vad_speech_started is a separate boolean used to gate _speech_start_time assignment in the VAD START_OF_SPEECH handler. The Python diff resets both fields together (self._speech_start_time = None, self._vad_speech_started = False).

    In agents-js, that gate is implemented directly as if (this.speechStartTime === undefined) in the VAD INFERENCE_DONE handler — there is no separate vadSpeechStarted field. Resetting speechStartTime = undefined therefore re-arms the same gate, so the Python _vad_speech_started = False line maps to a no-op in JS. No new field was introduced.

  • lastFinalTranscriptTime sentinel value. Python uses float | None; agents-js types this as number and uses 0 as the "not set" sentinel (already established in bounceEOUTask's lastFinalTranscriptTime !== 0 check and in the existing endUserTurn reset path that assigns this.lastFinalTranscriptTime = 0). The port resets to 0 for consistency rather than introducing undefined.

  • update_stt(None) / update_stt(stt) reset. The existing restartStt() IIFE already mirrors the Python update_stt reset (it stops the STT tasks, closes the pipeline, then re-starts the pipeline), so no change was needed there. It's preserved as-is and runs after the new resets.

Test plan

  • pnpm build:agents succeeds.
  • pnpm exec eslint agents/src/voice/audio_recognition.ts clean.
  • pnpm exec prettier --check clean for the touched file and changeset.
  • pnpm exec vitest run agents/src/voice/audio_recognition_span.test.ts (4 passed).
  • Manual verification with manual turn detection: trigger clearUserTurn mid-turn, confirm the next turn produces a fresh user_turn span and correct EOU metrics.

https://claude.ai/code/session_01P6kkyq5V2krvxdChbADKzg


Generated by Claude Code

Port of livekit/agents#5671 to agents-js.

Resets last final-transcript time, speech-start time, and last-speaking
time alongside the existing transcript/confidence/committed reset, and
ends any in-progress user_turn span (clearing collected STT request ids)
so the next speech starts a fresh span and metrics window.
Copy link
Copy Markdown
Contributor Author

/cc @toubatbrian @livekit/agent-devs for review.


Generated by Claude Code

@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented May 8, 2026

🦋 Changeset detected

Latest commit: e4aa7ce

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 31 packages
Name Type
@livekit/agents Patch
@livekit/agents-plugin-anam Patch
@livekit/agents-plugin-assemblyai Patch
@livekit/agents-plugin-baseten Patch
@livekit/agents-plugin-bey Patch
@livekit/agents-plugin-cartesia Patch
@livekit/agents-plugin-cerebras Patch
@livekit/agents-plugin-deepgram Patch
@livekit/agents-plugin-elevenlabs Patch
@livekit/agents-plugin-fishaudio Patch
@livekit/agents-plugin-google Patch
@livekit/agents-plugin-hedra Patch
@livekit/agents-plugin-hume Patch
@livekit/agents-plugin-inworld Patch
@livekit/agents-plugin-lemonslice Patch
@livekit/agents-plugin-liveavatar Patch
@livekit/agents-plugin-livekit Patch
@livekit/agents-plugin-minimax Patch
@livekit/agents-plugin-mistral Patch
@livekit/agents-plugin-mistralai Patch
@livekit/agents-plugin-neuphonic Patch
@livekit/agents-plugin-openai Patch
@livekit/agents-plugin-phonic Patch
@livekit/agents-plugin-resemble Patch
@livekit/agents-plugin-rime Patch
@livekit/agents-plugin-runway Patch
@livekit/agents-plugin-sarvam Patch
@livekit/agents-plugin-silero Patch
@livekit/agents-plugins-test Patch
@livekit/agents-plugin-trugen Patch
@livekit/agents-plugin-xai Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@CLAassistant
Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 3 additional findings.

Open in Devin Review

@u9g u9g closed this May 8, 2026
@u9g u9g deleted the claude/quirky-galileo-kLqAD branch May 8, 2026 18:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants