Add AV1 codec support for WebRTC - #2334
Open
duckysownyou wants to merge 1 commit into
Open
Conversation
Register AV1 in the pion MediaEngine and add RTP depacketize/repacketize handlers (pkg/av1) built on pion/rtp's AV1Depacketizer/AV1Payloader, mirroring the existing H264/H265 handler structure: AV1 RTP from RTSP sources is reassembled into temporal units and re-fragmented at MTU 1200. Two burst-loss fixes that large AV1 temporal units surface (a 4K keyframe spans hundreds of RTP packets; blasting them in one loop overflowed UDP socket buffers with ~50% observed loss on a gigabit LAN): - pace AV1 sends lightly (2ms per 16 packets) inside the payload handler; each sender runs on its own goroutine so this never delays the producer - request larger socket buffers on the WebRTC UDP mux listeners Tested end-to-end with a UniFi Protect G6 PTZ (4K AV1 via rtspx) playing in Chrome over WebRTC, direct and through a coturn TURN relay.
Collaborator
|
@AlexxIT the magic of codecs is beyond my competence. I can add to dev for testing after your positive review |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds AV1 as a negotiable video codec for WebRTC consumers.
pkg/webrtc/api.go— register AV1 in the pion MediaEngine (PT 105, sameRTCP feedback set as the other video codecs), and request larger socket
buffers on the WebRTC UDP mux listeners (see below).
pkg/av1/rtp.go(new) —RTPDepay/RTPPayhandlers mirroring the existingH264/H265 structure, built on
pion/rtp'sAV1Depacketizer/AV1Payloader:incoming AV1 RTP is reassembled into temporal units, then re-fragmented at
MTU 1200 (RTSP sources are often TCP-interleaved and can carry RTP packets
larger than the UDP/TURN-safe MTU).
pkg/webrtc/consumer.go— wire the AV1 case intoAddTrack.Why the pacing + socket buffers
A 4K AV1 keyframe temporal unit spans hundreds of RTP packets. Writing them
in one tight loop overflowed UDP socket buffers — ~50% packet loss measured on
a gigabit LAN with the kernel-default (~208KB) send buffer. Two mitigations:
runs on its own goroutine, so this never delays the producer or other
consumers,
SetWriteBuffer(4MB)/SetReadBuffer(1MB)on the UDP mux listeners (thisalso benefits large H265 keyframes).
Tested
rtspx://→ Chrome 148: AV1 negotiatedfirst in the answer, 3840x2160 frames decoded, direct and through a coturn
TURN relay.
(MSE + WebRTC negotiation unchanged).
Not included
MSE/fMP4 AV1 (
av01muxing) — separate concern, tracked in #2260. Browsersthat can't receive AV1 over WebRTC still get the "codecs not matched" error
as before.