On Fri, May 23, 2025 at 11:33:40AM +0200, Timothée wrote: > On 2025-05-23T02:57:36.000+02:00, Michael Niedermayer > <mich...@niedermayer.cc> wrote: > > > On Fri, May 23, 2025 at 02:45:59AM +0200, Michael Niedermayer wrote: > > > >> Hi Ronald On Thu, May 22, 2025 at 07:59:06AM -0400, Ronald S. > >> Bultje wrote: > >> > >>> Hi, On Wed, May 21, 2025 at 9:34 AM Timothée < > >>> timothee.informati...@regaud-chapuy.fr> wrote: > >>> > >>>> Hello, I am interested in expanding ffmpeg's capabilities to > >>>> extract low-level data from video codecs. Specifically, I'd > >>>> like to implement functionality that would allow exporting > >>>> frame data, macroblock information, quantization tables, and > >>>> similar codec-specific elements to binary files for further > >>>> analysis. After searching through the documentation and > >>>> existing features, I haven't found similar functionality, > >>>> though I may have missed something. Has this been implemented > >>>> before, or are there related features I should examine? > >>> > >>> Some older codecs implement minor variants for this, e.g. grep > >>> for AV_FRAME_DATA_MOTION_VECTORS, which attaches a frame's > >>> motion vectors to the picture data. I believe there's an example > >>> app and possibly a filter to overlay MVs on top of the video > >>> frame based on this concept. You could extend this to cover > >>> other (macro)block info. There used to be a variant of this for > >>> quant-tables also but I can't find it, maybe it was removed. > >> > >> For motion vectors: ./ffplay -flags2 +export_mvs -i > >> matrixbench_mpeg2.mpg -vf codecview=mv=pf+bf+bb For macroblock > >> segmentation and type vissualization + also motion vectors: > >> ffplay-3.4.13 -debug vis_mb_type matrixbench_mpeg2.mpg -vf > >> codecview=mv=pf+bf+bb For QP vissualization + also motion vectors: > >> ffplay-3.4.13 -debug vis_qp matrixbench_mpeg2.mpg -vf > >> codecview=mv=pf+bf+bb For qp values dumped on the console ./ffplay > >> -debug qp -i matrixbench_mpeg2.mpg > > > > And this can easily be extended to other codecs, ATM it should work > > with all 16x16 MB based codecs like > > msmpeg4*/wmv*/mpeg1/2/4/h263/h264 mbtype and qp vissualization need > > codecview to be extended or versions around 3.4 which implemented it > > differently Implementing vissualization as done currently with > > sidedata and codecview is simple and efficient. It also would allow > > exporting the data to json by writing a codec2json filter in place > > of codecview Also all decoders already have all this data parsed and > > available so its simpler than trying to do it in a decoder > > independant way I would thus suggest implementations of this for > > modern codecs to follow the same path as the existing code. thx > > Thanks for the helpful pointers! > > I will work on the codec2json filter. > > Looking at the code, I see where I can access sidedata but extracting > qb table seems to fail. (in codecview.c l.233: ff_qp_table_extract() > return 0 and qp_table is empty) (I use ./ffmpeg -flags2 +export_mvs -i > input.mp4 -vf codecview=qp=1 output.mp4 -y) > > Is is qp extraction not implemented yet? Or is it because I have h264 > video? If it's not implemented, I'm curious why there’s already code > that appears to handle it.
look at ff_print_debug_info2() theres probably something missing The QP and MB type code was changed from being inside arrays of pictures to sidedata. Something likely was lost/forgotten in the process -debug qp works with h264 so likely teh export into sidedata is not fully implemented thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB "Nothing to hide" only works if the folks in power share the values of you and everyone you know entirely and always will -- Tom Scott
signature.asc
Description: PGP signature
_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
