to-dev: fix(webrtc): reuse existing Receiver when remote switches payload type mid-session - #2375
Open
misecichin wants to merge 1 commit into
Open
to-dev: fix(webrtc): reuse existing Receiver when remote switches payload type mid-session#2375misecichin wants to merge 1 commit into
misecichin wants to merge 1 commit into
Conversation
…e mid-session Some WebRTC producers (observed with the Nest/SDM source) start sending media on a different payload type than the one negotiated at connection setup, without a full renegotiation. Since Conn.GetTrack deduplicates Receivers by exact *Codec pointer identity, a payload-type change for the same Media creates a brand-new, unwired Receiver instead of reusing the existing one - permanently starving every already-connected consumer (RTSP/WebRTC senders) of video, even though the producer is actively receiving valid frames. Reproduced against a live Nest doorbell (WEB_RTC protocol): go2rtc negotiates H264 on payload type 96, then the Nest media server begins transmitting on payload type 98 a few seconds into the session. Before this fix, /api/streams shows two H264 receivers - the wired one stuck at 0 bytes forever, and an orphan one accumulating real data with no consumer attached. ffmpeg consuming the RTSP restream fails with: Could not find codec parameters for stream 0 (Video: h264, none): unspecified size After this fix, there is a single Receiver correctly wired to the consumer, and ffmpeg reports a valid stream (H264 High profile, 384x512 @ 30fps) with megabytes of real video flowing through. Reproduced and fix verified on v1.9.0 and v1.9.10; the bug is present in both, so this isn't a version-specific regression.
Author
|
Current docker with the fix which i'm using now for the Nest Doorbell 2: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug
Some WebRTC producers start sending media on a different RTP payload type than the one negotiated at connection setup, without a full SDP renegotiation. I hit this consistently with a Nest doorbell (
nest:source,protocols=WEB_RTC).Conn.GetTrack(pkg/webrtc/producer.go) deduplicatesReceivers by exact*Codecpointer identity:When the remote peer switches payload type for the same
Media,getMediaCodecresolves a different*Codecpointer (matching the new payload type), so this lookup misses and a brand-new, unwiredReceiveris created — permanently starving every already-connected consumer (RTSP/WebRTC senders) of video, even though the producer is actively receiving valid frames on the new payload type.Reproduction
go2rtc negotiates H264 on payload type 96 with the Nest media server. A few seconds into the session, Nest begins transmitting on payload type 98 instead (same media, no renegotiation). Before this fix,
/api/streamsshows:ffmpeg consuming the RTSP restream fails with:
I reproduced this identically on v1.9.0 and v1.9.10, and also with an unmodified
alexxit/go2rtc:latest— so it isn't a version-specific regression, and adding explicitvideo=h264&audio=opushints to the source URL doesn't change the behavior either.Fix
Reuse the existing
Receiverfor the sameMediawhen no exact codec-pointer match is found, instead of creating an orphan:This is safe because
Receiver.Inputjust forwards raw RTP payload bytes to its children — it doesn't depend on which declared payload type the packet arrived on, and outgoing payload-type stamping is handled separately by theSenderside. I don't see simulcast/multi-encoding selection logic in this producer path that this would conflict with, but happy to adjust if I'm missing a case this affects.Verification
After the fix,
/api/streamsshows a singleReceivercorrectly wired to the consumer, and ffmpeg reports a fully valid stream:15MB+ of real video flowing through in a 12s test, versus 0 bytes on the wired receiver before the fix. Also validated against a real Frigate deployment (ffmpeg 7.0) pulling the RTSP restream in production, stable with no drops over multiple minutes, versus crash-looping every ~20s before.