nest: request keyframes + advertise sprop-parameter-sets (fix slow/failed consumer opens) - #2368
Open
ajplotkin wants to merge 2 commits into
Open
nest: request keyframes + advertise sprop-parameter-sets (fix slow/failed consumer opens)#2368ajplotkin wants to merge 2 commits into
ajplotkin wants to merge 2 commits into
Conversation
pkg/webrtc/conn.go sends an RTCP PictureLossIndication every 2s to keep the source emitting keyframes, but only for ModePassiveProducer (WHIP/browser push). Nest is an active-pull WebRTC source (ModeActiveProducer), so it was excluded -- its keyframe interval drifts long when idle, and any consumer that joins mid-GOP (RTSP live view, snapshot grab) waits up to a full keyframe interval to start. Enable the keyframe request for the Nest source, gated on FormatName "nest/webrtc" rather than the mode so other ModeActiveProducer WebRTC sources (ring/tuya battery cams etc.) aren't forced into 2s IDRs, which would be battery- and bandwidth-hostile. PLI is media-plane RTCP -- no SDM API quota impact. Also add defer ticker.Stop() (the upstream pattern relies on GC of an unreferenced ticker, fine on go>=1.23 but explicit is safer). Measured on a Nest doorbell: keyframe interval ~3.3s -> ~1.8s; on-demand frame.jpeg grabs drop to ~1s. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…IBE) The Nest WebRTC source's H264 codec FmtpLine comes from Google's SDP answer, which carries profile-level-id but no sprop-parameter-sets (WebRTC sends SPS/PPS in-band). go2rtc's RTSP server copies FmtpLine verbatim into the DESCRIBE SDP, so an RTSP consumer (e.g. ffmpeg for HomeKit live view) gets no SPS/PPS in the SDP and can't learn video dimensions until an in-band keyframe arrives -- forcing a large -probesize and slow starts (or failing outright at low probesize). Capture SPS (NAL 7) / PPS (NAL 8) from the incoming H264 RTP in the OnTrack receive loop (handling STAP-A bundling, which is how libwebrtc packs SPS+PPS+IDR) and append sprop-parameter-sets to the codec FmtpLine once. Every consumer that clones the codec (RTSP) or reads GetParameterSet (MSE/MP4) then benefits. Gated on FormatName "nest/webrtc" like the keyframe-request patch. Upstream does the equivalent for H265 in pkg/dvrip. With preload + the 2s keyframe request, SPS/PPS are captured well before any DESCRIBE. This lets RTSP consumers use a small -probesize safely (dimensions come from the SDP). Note it does not by itself make stream-copy opens instant: ffmpeg still aligns copy output to the next keyframe, so the ~2s keyframe interval (bounded by the PLI patch) remains the floor. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ajplotkin
force-pushed
the
nest-keyframe-sprop
branch
from
July 18, 2026 09:13
e428c99 to
133c89b
Compare
ajplotkin
added a commit
to ajplotkin/go2rtc
that referenced
this pull request
Jul 18, 2026
… math, udp4/sync caveats, credit adopted PRs - conn.go: inline note that the sprop FmtpLine append is intentionally lock-free (captured before any DESCRIBE in practice; mirrors upstream H265; Codec.Match ignores FmtpLine). Comment-only, no behavior change. - NEST-HOMEKIT-SNAPSHOTS.md: M7 (plugin-version fingerprint 1.1.23 + explicit install order plugin->PR#212->patches), L1 (udp4 drops TCP ICE tradeoff), L3 (per-camera vs per-project quota scoping for many cameras), L9 (sync only on camera-set change + a go2rtc restart drops warm streams), execSync->fs refresh matching the deployed code, and credit for adopted PRs AlexxIT#2351/AlexxIT#2327/ AlexxIT#2193/AlexxIT#2194/AlexxIT#2368. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two small, self-contained improvements to the Nest (
nest:) WebRTC source, both gated onFormatName == "nest/webrtc"so nothing else is affected. Together they make Nest cameras usable as a warm RTSP/HLS/consumer source — right now a consumer that joins a Nest stream can wait a long time to start, or fail to determine the video size at all.1. Request periodic keyframes for the Nest source
OnTrackinpkg/webrtc/conn.goalready sends an RTCPPictureLossIndicationevery 2s to keep the source emitting keyframes — but only forModePassiveProducer(WHIP/browser push). Nest is an active-pull source (ModeActiveProducer), so it was excluded, and an idle Nest camera lets its keyframe interval drift long. Any consumer that joins mid-GOP (RTSP live view, an on-demandframe.jpeg, MSE) then waits up to a full keyframe interval to start.This enables the existing keyframe-request loop for the Nest source, gated on
FormatName(not the mode) so otherModeActiveProducerWebRTC sources — e.g. battery cameras where a forced 2s IDR would be power/bandwidth hostile — are untouched. PLI is media-plane RTCP, so there's no impact on the Nest/SDM API command quota.Measured on a Nest doorbell: keyframe interval ~3.3s → ~1.8s; on-demand
frame.jpeggrabs drop from several seconds to ~1s.Also added
defer t.Stop()on the ticker (the existing pattern relies on GC of an unreferenced ticker — fine on Go ≥1.23, but explicit is safer).2. Advertise
sprop-parameter-setsin the RTSP SDPThe Nest source's H264 codec
FmtpLinecomes from Google's WebRTC SDP answer, which carriesprofile-level-idbut nosprop-parameter-sets(WebRTC sends SPS/PPS in-band). The RTSP server copiesFmtpLineverbatim intoDESCRIBE, so an RTSP consumer gets no SPS/PPS in the SDP and can't determine the video dimensions until an in-band keyframe arrives — which forces a large-probesizefor ffmpeg, or fails outright at a small one.This captures SPS (NAL 7) / PPS (NAL 8) from the incoming H264 RTP in the
OnTrackreceive loop — handling STAP-A, which is how libwebrtc typically bundles SPS+PPS+IDR — and appendssprop-parameter-setsto the codecFmtpLineonce. Every consumer that clones the codec (RTSP) or readsGetParameterSet(MSE/MP4) then benefits. This mirrors whatpkg/dvripalready does for H265.Verified in a live
DESCRIBEafter the change:Notes / honesty
FormatName == "nest/webrtc"; no behavior change for any other source.Codec.MatchignoresFmtpLine, so appending sprop can't affect media matching/renegotiation.FmtpLineappend happens on the receive goroutine and is written once; with the keyframe request above, SPS/PPS are captured before any consumer'sDESCRIBE. A-racebuild could flag a consumer connecting during the sub-2s warm-up window; the existing H265 append inpkg/dvripis unguarded in the same way. Happy to wrap it in a small mutex or capture-into-locals-then-assign if preferred.Context: I'm using this to serve Nest cameras to Apple HomeKit via Homebridge over local RTSP, so a single warm go2rtc stream is shared instead of each viewer opening its own concurrent Google WebRTC stream (Nest enforces a concurrent-stream limit). Both patches were needed to make that path reliable.