On Sun, 2022-06-26 at 23:41 +0000, ffmpegagent wrote: > Missing SEI information has always been a major drawback when using the QSV > decoders. I used to think that there's no chance to get at the data without > explicit implementation from the MSDK side (or doing something weird like > parsing in parallel). It turned out that there's a hardly known api method > that provides access to all SEI (h264/hevc) or user data (mpeg2video). > > This allows to get things like closed captions, frame packing, display > orientation, HDR data (mastering display, content light level, etc.) without > having to rely on those data being provided by the MSDK as extended buffers. > > The commit "Implement SEI parsing for QSV decoders" includes some hard-coded > workarounds for MSDK bugs which I reported: > https://github.com/Intel-Media-SDK/MediaSDK/issues/2597#issuecomment-1072795311 > > But that doesn't help. Those bugs exist and I'm sharing my workarounds, > which are empirically determined by testing a range of files. If someone is > interested, I can provide private access to a repository where we have been > testing this. Alternatively, I could also leave those workarounds out, and > just skip those SEI types. > > In a previous version of this patchset, there was a concern that payload > data might need to be re-ordered. Meanwhile I have researched this carefully > and the conclusion is that this is not required. > > My detailed analysis can be found here: > https://gist.github.com/softworkz/36c49586a8610813a32270ee3947a932 > > v3 > > * frame.h: clarify doc text for av_frame_copy_side_data() > > v2 > > * qsvdec: make error handling consistent and clear > * qsvdec: remove AV_CODEC_ID_MPEG1VIDEO constants > * hevcdec: rename function to ff_hevc_set_side_data(), add doc text > > v3 > > * qsvdec: fix c/p error > > softworkz (6): > avutil/frame: Add av_frame_copy_side_data() and > av_frame_remove_all_side_data() > avcodec/vpp_qsv: Copy side data from input to output frame > avcodec/mpeg12dec: make mpeg_decode_user_data() accessible > avcodec/hevcdec: make set_side_data() accessible > avcodec/h264dec: make h264_export_frame_props() accessible > avcodec/qsvdec: Implement SEI parsing for QSV decoders > > doc/APIchanges | 4 + > libavcodec/h264_slice.c | 98 ++++++++------- > libavcodec/h264dec.h | 2 + > libavcodec/hevcdec.c | 117 +++++++++--------- > libavcodec/hevcdec.h | 9 ++ > libavcodec/mpeg12.h | 28 +++++ > libavcodec/mpeg12dec.c | 40 +----- > libavcodec/qsvdec.c | 234 +++++++++++++++++++++++++++++++++++ > libavfilter/qsvvpp.c | 6 + > libavfilter/vf_overlay_qsv.c | 19 ++- > libavutil/frame.c | 67 ++++++---- > libavutil/frame.h | 32 +++++ > libavutil/version.h | 2 +- > 13 files changed, 485 insertions(+), 173 deletions(-) > > > base-commit: 6a82412bf33108111eb3f63076fd5a51349ae114 > Published-As: > https://github.com/ffstaging/FFmpeg/releases/tag/pr-ffstaging-31%2Fsoftworkz%2Fsubmit_qsv_sei-v4 > Fetch-It-Via: git fetch https://github.com/ffstaging/FFmpeg pr-ffstaging- > 31/softworkz/submit_qsv_sei-v4 > Pull-Request: https://github.com/ffstaging/FFmpeg/pull/31 > > Range-diff vs v3: > > 1: c442597a35 ! 1: 7656477360 avutil/frame: Add av_frame_copy_side_data() > and av_frame_remove_all_side_data() > @@ doc/APIchanges: libavutil: 2021-04-27 > > API changes, most recent first: > > -+2022-05-26 - xxxxxxxxx - lavu 57.26.100 - frame.h > ++2022-05-26 - xxxxxxxxx - lavu 57.28.100 - frame.h > + Add av_frame_remove_all_side_data(), av_frame_copy_side_data(), > + AV_FRAME_TRANSFER_SD_COPY, and AV_FRAME_TRANSFER_SD_FILTER. > + > - 2022-05-23 - xxxxxxxxx - lavu 57.25.100 - avutil.h > - Deprecate av_fopen_utf8() without replacement. > - > + 2022-06-12 - xxxxxxxxxx - lavf 59.25.100 - avio.h > + Add avio_vprintf(), similar to avio_printf() but allow to use it > + from within a function taking a variable argument list as input. > > ## libavutil/frame.c ## > @@ libavutil/frame.c: FF_ENABLE_DEPRECATION_WARNINGS > @@ libavutil/frame.h: int av_frame_copy(AVFrame *dst, const AVFrame > *src); > + * @param src a frame from which to copy the side data. > + * @param flags a combination of AV_FRAME_TRANSFER_SD_* > + * > -+ * @return >= 0 on success, a negative AVERROR on error. > ++ * @return 0 on success, a negative AVERROR on error. > + * > + * @note This function will create new references to side data buffers > in src, > + * unless the AV_FRAME_TRANSFER_SD_COPY flag is passed. > @@ libavutil/version.h > */ > > #define LIBAVUTIL_VERSION_MAJOR 57 > --#define LIBAVUTIL_VERSION_MINOR 25 > -+#define LIBAVUTIL_VERSION_MINOR 26 > +-#define LIBAVUTIL_VERSION_MINOR 27 > ++#define LIBAVUTIL_VERSION_MINOR 28 > #define LIBAVUTIL_VERSION_MICRO 100 > > #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, > \ > 2: 6f50d0bd57 = 2: 06976606c5 avcodec/vpp_qsv: Copy side data from input to > output frame > 3: f682b1d695 = 3: 320a8a535c avcodec/mpeg12dec: make > mpeg_decode_user_data() accessible > 4: 995d835233 = 4: e58ad6564f avcodec/hevcdec: make set_side_data() > accessible > 5: ac8dc06395 = 5: a57bfaebb9 avcodec/h264dec: make > h264_export_frame_props() accessible > 6: 27c3dded4d = 6: 3f2588563e avcodec/qsvdec: Implement SEI parsing for QSV > decoders
Patchset LGTM, -Haihao _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
