On 2025-05-23T02:57:36.000+02:00, Michael Niedermayer <mich...@niedermayer.cc> wrote:
> On Fri, May 23, 2025 at 02:45:59AM +0200, Michael Niedermayer wrote: > >> Hi Ronald On Thu, May 22, 2025 at 07:59:06AM -0400, Ronald S. >> Bultje wrote: >> >>> Hi, On Wed, May 21, 2025 at 9:34 AM Timothée < >>> timothee.informati...@regaud-chapuy.fr> wrote: >>> >>>> Hello, I am interested in expanding ffmpeg's capabilities to >>>> extract low-level data from video codecs. Specifically, I'd >>>> like to implement functionality that would allow exporting >>>> frame data, macroblock information, quantization tables, and >>>> similar codec-specific elements to binary files for further >>>> analysis. After searching through the documentation and >>>> existing features, I haven't found similar functionality, >>>> though I may have missed something. Has this been implemented >>>> before, or are there related features I should examine? >>> >>> Some older codecs implement minor variants for this, e.g. grep >>> for AV_FRAME_DATA_MOTION_VECTORS, which attaches a frame's >>> motion vectors to the picture data. I believe there's an example >>> app and possibly a filter to overlay MVs on top of the video >>> frame based on this concept. You could extend this to cover >>> other (macro)block info. There used to be a variant of this for >>> quant-tables also but I can't find it, maybe it was removed. >> >> For motion vectors: ./ffplay -flags2 +export_mvs -i >> matrixbench_mpeg2.mpg -vf codecview=mv=pf+bf+bb For macroblock >> segmentation and type vissualization + also motion vectors: >> ffplay-3.4.13 -debug vis_mb_type matrixbench_mpeg2.mpg -vf >> codecview=mv=pf+bf+bb For QP vissualization + also motion vectors: >> ffplay-3.4.13 -debug vis_qp matrixbench_mpeg2.mpg -vf >> codecview=mv=pf+bf+bb For qp values dumped on the console ./ffplay >> -debug qp -i matrixbench_mpeg2.mpg > > And this can easily be extended to other codecs, ATM it should work > with all 16x16 MB based codecs like > msmpeg4*/wmv*/mpeg1/2/4/h263/h264 mbtype and qp vissualization need > codecview to be extended or versions around 3.4 which implemented it > differently Implementing vissualization as done currently with > sidedata and codecview is simple and efficient. It also would allow > exporting the data to json by writing a codec2json filter in place > of codecview Also all decoders already have all this data parsed and > available so its simpler than trying to do it in a decoder > independant way I would thus suggest implementations of this for > modern codecs to follow the same path as the existing code. thx Thanks for the helpful pointers! I will work on the codec2json filter. Looking at the code, I see where I can access sidedata but extracting qb table seems to fail. (in codecview.c l.233: ff_qp_table_extract() return 0 and qp_table is empty) (I use ./ffmpeg -flags2 +export_mvs -i input.mp4 -vf codecview=qp=1 output.mp4 -y) Is is qp extraction not implemented yet? Or is it because I have h264 video? If it's not implemented, I'm curious why there’s already code that appears to handle it. Timothée _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
