Skip to content

to-dev: fix(webrtc): reuse existing Receiver when remote switches payload type mid-session - #2375

Open
misecichin wants to merge 1 commit into
AlexxIT:devfrom
misecichin:fix-webrtc-payload-type-switch
Open

to-dev: fix(webrtc): reuse existing Receiver when remote switches payload type mid-session#2375
misecichin wants to merge 1 commit into
AlexxIT:devfrom
misecichin:fix-webrtc-payload-type-switch

Conversation

@misecichin

Copy link
Copy Markdown

Bug

Some WebRTC producers start sending media on a different RTP payload type than the one negotiated at connection setup, without a full SDP renegotiation. I hit this consistently with a Nest doorbell (nest: source, protocols=WEB_RTC).

Conn.GetTrack (pkg/webrtc/producer.go) deduplicates Receivers by exact *Codec pointer identity:

for _, track := range c.Receivers {
    if track.Codec == codec {
        return track, nil
    }
}

When the remote peer switches payload type for the same Media, getMediaCodec resolves a different *Codec pointer (matching the new payload type), so this lookup misses and a brand-new, unwired Receiver is created — permanently starving every already-connected consumer (RTSP/WebRTC senders) of video, even though the producer is actively receiving valid frames on the new payload type.

Reproduction

go2rtc negotiates H264 on payload type 96 with the Nest media server. A few seconds into the session, Nest begins transmitting on payload type 98 instead (same media, no renegotiation). Before this fix, /api/streams shows:

"receivers": [
  {"id": 3, "codec": {"codec_name": "h264"}, "childs": [4]},                    // wired to the RTSP sender, 0 bytes forever
  {"id": 7, "codec": {"codec_name": "h264"}, "bytes": 787424, "packets": 730}   // orphan, no consumer, real data
]

ffmpeg consuming the RTSP restream fails with:

Could not find codec parameters for stream 0 (Video: h264, none): unspecified size

I reproduced this identically on v1.9.0 and v1.9.10, and also with an unmodified alexxit/go2rtc:latest — so it isn't a version-specific regression, and adding explicit video=h264&audio=opus hints to the source URL doesn't change the behavior either.

Fix

Reuse the existing Receiver for the same Media when no exact codec-pointer match is found, instead of creating an orphan:

case core.ModePassiveProducer, core.ModeActiveProducer:
    for _, track := range c.Receivers {
        if track.Media == media {
            return track, nil
        }
    }

This is safe because Receiver.Input just forwards raw RTP payload bytes to its children — it doesn't depend on which declared payload type the packet arrived on, and outgoing payload-type stamping is handled separately by the Sender side. I don't see simulcast/multi-encoding selection logic in this producer path that this would conflict with, but happy to adjust if I'm missing a case this affects.

Verification

After the fix, /api/streams shows a single Receiver correctly wired to the consumer, and ffmpeg reports a fully valid stream:

Stream #0:0: Video: h264 (High), yuv420p(progressive), 384x512, 30 fps, 15 tbr, 90k tbn
Stream #0:1: Audio: opus, 48000 Hz, stereo, fltp

15MB+ of real video flowing through in a 12s test, versus 0 bytes on the wired receiver before the fix. Also validated against a real Frigate deployment (ffmpeg 7.0) pulling the RTSP restream in production, stable with no drops over multiple minutes, versus crash-looping every ~20s before.

…e mid-session

Some WebRTC producers (observed with the Nest/SDM source) start sending
media on a different payload type than the one negotiated at connection
setup, without a full renegotiation. Since Conn.GetTrack deduplicates
Receivers by exact *Codec pointer identity, a payload-type change for
the same Media creates a brand-new, unwired Receiver instead of reusing
the existing one - permanently starving every already-connected
consumer (RTSP/WebRTC senders) of video, even though the producer is
actively receiving valid frames.

Reproduced against a live Nest doorbell (WEB_RTC protocol): go2rtc
negotiates H264 on payload type 96, then the Nest media server begins
transmitting on payload type 98 a few seconds into the session. Before
this fix, /api/streams shows two H264 receivers - the wired one stuck
at 0 bytes forever, and an orphan one accumulating real data with no
consumer attached. ffmpeg consuming the RTSP restream fails with:

  Could not find codec parameters for stream 0 (Video: h264, none): unspecified size

After this fix, there is a single Receiver correctly wired to the
consumer, and ffmpeg reports a valid stream (H264 High profile,
384x512 @ 30fps) with megabytes of real video flowing through.

Reproduced and fix verified on v1.9.0 and v1.9.10; the bug is present
in both, so this isn't a version-specific regression.
@misecichin

Copy link
Copy Markdown
Author

Current docker with the fix which i'm using now for the Nest Doorbell 2:
misecichin/go2rtc-nest:1.9.10-nestfix

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant