Re: [FFmpeg-devel] [PATCH] avformat: add AV1 RTP depacketizer and packetizer

[Tristan Matthews via ffmpeg-devel]

Sent with Proton Mail secure email.

On Thursday, December 12th, 2024 at 4:18 AM, Chris Hodges 
<chris.hod...@axis.com> wrote:

> Hi Tristan,
>
> I have a draft reply ongoing the last couple of days, but I got
> sidetracked with other work, sorry. Will send it out without a new patch
> so we can discuss the changes and it gives me more time to look into the
> issue below.
>
> On 12/11/24 23:53, Tristan Matthews via ffmpeg-devel wrote:
>
> > Aside from my minor comments on the patch, one thing that came up in terms 
> > of testing is that while the packetization seems to be working well, I'm 
> > unable to depacketize a stream being sent by ffmpeg with ffmpeg as the 
> > receiver.
>
>
> There was a bug that triggered on fragmented OBUs >= 16384 bytes if it
>
> was not the last OBU in the TU:
>
> // move existing data for OBU back a bit
> memmove(lebptr + extra_bytes, lebptr,
> pkt->size - extra_bytes - data->frag_pkt_leb_pos);
>
> + // move pktpos further down for following OBUs in same
> packet.
> + pktpos += extra_bytes;
>
> This might have been one of the problems.
>
> > This is fairly easy to reproduce locally, start an RTSP server:
> > $ docker run --rm -it --network=host bluenviron/mediamtx:latest
> >
> > Push an RTP stream:
> > $ ./ffmpeg -re -i Chimera-AV1-8bit-1280x720-3363kbps.ivf -c copy -strict 
> > experimental -f rtsp rtsp://127.0.0.1:8554/foobar
> >
> > Pull the same stream:
> > $ ./ffmpeg -i rtsp://127.0.0.1:8554/foobar -copy -f matroska foobar.mkv
>
>
> Thanks for these examples, helps a lot!
>
> > This fails with a bunch of errors on trying decode e.g.:
> > [libdav1d @ 0x5c50bb8d1580] No sequence header available: unable to decode 
> > frame header.
> > [libdav1d @ 0x5c50bb8d1580] obu_forbidden_bit out of range: 1, but must be 
> > in [0,0].
> > etc.
>
>
> I don't seem to be able to replay the Chimera video directly with my
> local build of ffplay, get's all choppy and skips big deals of the
> video. Might be an outdated libdav1d?
>
> Anyway, the AV1 RTP decoder currently does not wait for a keyframe
> before sending down frames. Should it do that? Also, should it wait for
> another keyframe if any frame or packet had been dropped (the GOP is
> going to be useless for the decoder anyway)?

That's a good question, I don't believe any of the RTP depacketizers in 
libavformat do this currently so maybe best left as a future enhancement 
(behind an option/flag). 

On a related note, one feature that only the VP8 depacketizer supports now is 
signalling via RTCP that it needs a keyframe (in the event of loss or the start 
of a stream), to do this you just have to implement the .need_keyframe callback 
(see 
https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/eb79c316c7b205f45e7003e91f77c253818e6685:/libavformat/rtpdec_vp8.c#l274)

>
> > The good news is that I can pull that same stream and save it to disk with 
> > e.g. gstreamer so it's clearly not that far off. It's not clear to me yet 
> > if this is an issue on the packetization side, the intermediate RTSP server 
> > side, the depacketization side or some combination therein.
>
>
> I haven't been working with such complex frame hierarchies, but I will
> try to get the Chimera working.

FWIW I was seeing similar behaviour with smaller/simpler files as well.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".