Re: [FFmpeg-devel] [PATCH] avcodec: uninit hwaccel in case of software decoder
Ping.
On Fri, Nov 29, 2024, at 11:44, Thomas Guillem via ffmpeg-devel wrote:
> avcodec_get_hw_frames_parameters(), called by the user from get_format,
> is allocating ctx->internal->hwaccel_priv_data. But the hardware
> decoding setup may fail on the user side and it may fallback to software
> decoding. In that case, ctx->internal->hwaccel_priv_data is still
> allocated but not used anymore.
>
> Fixes the following assert:
>
> Assertion p_dst->hwaccel_threadsafe || (!dst->hwaccel &&
> !dst->internal->hwaccel_priv_data) failed at
> src/libavcodec/pthread_frame.c:426
> ---
> libavcodec/decode.c | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/libavcodec/decode.c b/libavcodec/decode.c
> index d1e10223f2..365fed68e5 100644
> --- a/libavcodec/decode.c
> +++ b/libavcodec/decode.c
> @@ -1346,6 +1346,8 @@ int ff_get_format(AVCodecContext *avctx, const
> enum AVPixelFormat *fmt)
> if (!hw_config) {
> // No config available, so no extra setup required.
> ret = user_choice;
> +// Remove a potential failed hwaccel context
> +ff_hwaccel_uninit(avctx);
> break;
> }
> config = &hw_config->public;
> --
> 2.45.2
>
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v3 2/2] lavc/rv40dsp: fix RISC-V chroma_mc
On Thu, 05 Dec 2024 21:41:51 +0800 uk7b-at-foxmail@ffmpeg.org wrote: > From: sunyuechi > > --- > libavcodec/riscv/rv40dsp_rvv.S | 116 ++--- > 1 file changed, 78 insertions(+), 38 deletions(-) > > diff --git a/libavcodec/riscv/rv40dsp_rvv.S b/libavcodec/riscv/rv40dsp_rvv.S > index ca431eb8ab..d4beb7f1e9 100644 > --- a/libavcodec/riscv/rv40dsp_rvv.S > +++ b/libavcodec/riscv/rv40dsp_rvv.S > @@ -20,15 +20,30 @@ > > #include "libavutil/riscv/asm.S" > > -.macro manual_avg dst src1 src2 > -vadd.vv \dst, \src1, \src2 > -vadd.vi \dst, \dst, 1 > -vsrl.vi \dst, \dst, 1 > -.endm > +const rv40_bias > +.byte 0, 16, 32, 16 > +.byte 32, 28, 32, 28 > +.byte 0, 32, 16, 32 > +.byte 32, 28, 32, 28 > +endconst > > .macro do_chroma_mc type unroll > -csrwi vxrm, 2 > +csrwi vxrm, 0 > +addisp, sp, -16 > +#if __riscv_xlen == 32 > +sw s2, (sp) > +#elif __riscv_xlen == 64 > +sd s2, (sp) > +#else > +sq s2, (sp) > +#endif You can use the macro `sx` from libavcodec/riscv/h264qpel_rvv.S (I suggest to move them into some common include path) > +lla t4, rv40_bias > +srlit5, a5, 1 > +sh2add t4, t5, t4 > +srlit5, a4, 1 > +add t5, t4, t5 > sllit2, a5, 3 > +lb s2, (t5) > mul t1, a5, a4 > sh3add a5, a4, t2 > sllia4, a4, 3 > @@ -80,17 +95,19 @@ > vwmulu.vx v12, v14, a6 > vwmaccu.vx v10, t1, v15 > vwmaccu.vx v12, a7, v15 > -vnclipu.wi v15, v8, 6 > +vwaddu.wx v20, v8, s2 > +vnsrl.wiv15, v20, 6 >.ifc \type,avg > vle8.v v9, (a0) > -manual_avg v15, v15, v9 > +vaaddu.vv v15, v15, v9 >.endif > vse8.v v15, (a0) > add a0, a0, a2 > -vnclipu.wi v8, v10, 6 > +vwaddu.wx v20, v10, s2 > +vnsrl.wiv8, v20, 6 >.ifc \type,avg > vle8.v v9, (a0) > -manual_avg v8, v8, v9 > +vaaddu.vv v8, v8, v9 >.endif > add t4, t4, t3 > vse8.v v8, (a0) > @@ -115,17 +132,19 @@ > vslide1down.vx v14, v14, t5 > vsetvli zero, t6, e8, m1, ta, ma > vwmaccu.vx v16, t1, v14 > -vnclipu.wi v8, v12, 6 > +vwaddu.wx v20, v12, s2 > +vnsrl.wiv8, v20, 6 >.ifc \type,avg > vle8.v v9, (a0) > -manual_avg v8, v8, v9 > +vaaddu.vv v8, v8, v9 >.endif > vse8.v v8, (a0) > add a0, a0, a2 > -vnclipu.wi v8, v16, 6 > +vwaddu.wx v20, v16, s2 > +vnsrl.wiv8, v20, 6 >.ifc \type,avg > vle8.v v9, (a0) > -manual_avg v8, v8, v9 > +vaaddu.vv v8, v8, v9 >.endif > vse8.v v8, (a0) > add a0, a0, a2 > @@ -159,18 +178,20 @@ > vwmaccu.vx v10, t0, v8 > add a4, a4, a7 > vwmaccu.vx v12, t0, v9 > -vnclipu.wi v15, v10, 6 > +vwaddu.wx v20, v10, s2 > +vnsrl.wiv15, v20, 6 > vwmulu.vx v10, v9, a6 > -vnclipu.wi v9, v12, 6 > +vwaddu.wx v20, v12, s2 > +vnsrl.wiv9, v20, 6 >.ifc \type,avg > vle8.v v16, (a0) > -manual_avg v15, v15, v16 > +vaaddu.vv v15, v15, v16 >.endif > vse8.v v15, (a0) > add a0, a0, a2 >.ifc \type,avg > vle8.v v16, (a0) > -manual_avg v9, v9, v16 > +vaaddu.vv v9, v9, v16 >.endif > vse8.v v9, (a0) > add a0, a0, a2 > @@ -179,18 +200,20 @@ > vle8.v v14, (a5) > vwmaccu.vx v10, t0, v8 > vwmulu.vx v12, v8, a6 > -vnclipu.wi v8, v10, 6 > +vwaddu.wx v20, v10, s2 > +vnsrl.wiv8, v20, 6 > vwmaccu.vx v12, t0, v14 >.ifc \type,avg > vle8.v v16, (a0) > -manual_avg v8, v8, v16 > +vaaddu.vv v8, v8, v16 >.endif > vse8.v v8, (a0) > add a0, a0, a2 > -vnclipu.wi v8, v12, 6 > +vwaddu.wx v20, v12, s2 > +vnsrl.wiv8, v20, 6 >.ifc \type,avg > vle8.v v16, (a0) > -manual_avg v8, v8, v16 > +vaaddu.vv v8, v8, v16 >.endif > vse8.v v8, (a0) > add a0, a0, a2 > @@ -226,17 +249,1
Re: [FFmpeg-devel] [PATCH 0/7] RFC: complete rework of s337m support
>De : Anton Khirnov >Envoyé : jeudi 5 décembre 2024 15:06 >Hi, >I believe a similar approach was tried by Gyan earlier this year and >unanimously rejected by the TC [1-3]. > >[1] >https://patchwork.ffmpeg.org/project/ffmpeg/patch/20240127103854.9971-1-ffm...@gyani.pro/ >[2] Message-Id <9be400cf-949f-4eb1-93a1-b352a1b80...@gyani.pro> >https://lists.ffmpeg.org/pipermail/ffmpeg-devel/2024-February/321564.html >[3] Message-Id <20240412122920.gb3...@haasn.xyz> >https://lists.ffmpeg.org/pipermail/ffmpeg-devel/2024-April/325588.html Anton, Yes, of course, I followed this work. I thought I had highlighted many new elements (including many s337m issues in current code), and my work is not focused on s302m, so I would have expected some discussions to start. You certainely noticed that I posted this as an "RFC", so it is more "open". To take an example, I will quote one point of the TC: "The opposition to the patch was based on the opinion that this implementation of S302M should be handled by libavformat, not libavcodec." In my patch serie, the s302m decoder IS moved to libavformat, which is what was asked. So: even if there are some strong/hard points against the s337m decoder I designed, maybe part of this work can be helpful. Maybe just the s302m decoder moved to libavformat, I don't know. Nicolas ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 0/7] RFC: complete rework of s337m support
>De : ffmpeg-devel de la part de Lynne via >ffmpeg-devel >Envoyé : jeudi 5 décembre 2024 15:15 >I strongly object to using a resampler here. We *never* covert audio or >video in decoders from the native codec samplerate or pixel format. >This includes Opus. Opus carries the original samplerate of the audio >encoded into its extradata. We ignore it, whilst libopusfile does not. I've >been >on #opus for a long time, and the only thing I've ever heard about >libopusfile's >behaviour has been confusion and criticism from its users. > >You should expose this, and let users make an informed decision. Most users >will have a resampler anyway, and they may be *particularly sensitive* about >any potential latency you introduce, so applying mandatory resampler here is >both heavy-handed and problematic. I was not aware of this, thank you for your information. Users: I still think the situation is different with Dolby E: from a user's perspective, only the "original" aes sample_rate "exists" at all. See here: https://forum.videohelp.com/threads/400100-Dolby-E-Decoding-FFmpeg "It might be an FFmpeg bug, so parameter -ar 48000 is used below to 'fix' this issue via resampling" Internal sample_rate: In some cases, the native decoder sample rate is an integer-rounded value, so there is no way for the user to avoid a slight drift over time. Design: I am confused, maybe I missunderstood something, because when working on this, I was inspired by opus/dec.c which actually uses a resampler. So I thought there would be no hard objection about the technical design. Latency: is obviously a good point and in my case I add about 30/40ms which is much, but I don't think there are "very low" latency scenarios with dolby_e which is not a generic nor affordable codec, so it more about live sport events at most, but no interactive talks or so, I would say. Moreover, the "official" dolby delay indeed is 1 full video/audio frame to keep synced with the start of the video, so no user should expect a low delay. Requirement: As I described in my cover later but maybe I was not clear, the use of a resampler is absolutely necessary to fix multiple sync issues, and the tuning of it is somewhat tricky, it is not just about sample rate conversion. So, if the resampler is not included, it must be inserted somewhere downstream with advanced fine tuned settings. Anyway, I can try to go forward. And I also read that nested decoders should be avoided. But at the end, it would mean that the output of s337m would be a "string of packets" with pts adjusted packet by packet and having discontinuities (small overlap and/or big or small gaps) between each packet. Is it really feasible and desirable ? Well, there are currently so many issues specific to s337m and it seems impossible to me to map this in ffmpeg. The dolby_e decoder itself has no active maintainer. Maybe it should be recorded that s337m/dolby_e is an experimental thing and has to be handled in a separate broadcast fork. Nicolas ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [RFC PATCH] tests/color_utils: Reduce precision requirement to resolve test failure
On Fri, 6 Dec 2024, Zhao Zhili wrote:
From: Zhao Zhili
---
libavutil/tests/color_utils.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/libavutil/tests/color_utils.c b/libavutil/tests/color_utils.c
index 27ba8b529e..b8200e91fa 100644
--- a/libavutil/tests/color_utils.c
+++ b/libavutil/tests/color_utils.c
@@ -70,7 +70,7 @@ int main(int argc, char *argv[])
printf("trc=%s calling func(%f) expected=%f roundtrip=%f\n",
name, test_data[i], result, roundtrip);
-if (result > 0.0 && fabs(roundtrip - test_data[i]) > 1e-8) {
+if (result > 0.0 && fabs(roundtrip - test_data[i]) > 1e-7) {
printf(" FAIL\n");
return 1;
}
--
2.46.0
LGTM
// Martin
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 0/7] RFC: complete rework of s337m support
>De : ffmpeg-devel de la part de Kieran >Kunhya via ffmpeg-devel >Envoyé : jeudi 5 décembre 2024 19:31 > >> I resample the decoded data. The s337m decoder requires the dolby_e decoder >> (same way the current ftr decoder requires the aac decoder, so should not be >> an issue). > >Why not just send the packet with the actual PTS (including the guard >band) to Dolby E and then resample there? >The resampler needs to handle the NTSC 1601/1602 cadence. > >Kieran In my design, indeed, the resampler handles the cadence. More specifically, it "forwards" the input cadence. This could be done in the Dolby E decoder, for sure. That would fix the decoder itself. But another resampler will still be required for silence fallback for example, this one will still be in the s337m decoder or inserted downstream by a user. Now, the issue is that there are currently objections to both: - nested decoders - use of resamplers in decoders So, see if it is confirmed, but I will not continue my work in such a case... Nicolas ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [FFmpeg-cvslog] avfilter/vf_scale: switch to new swscale API
Hi On Mon, Nov 25, 2024 at 10:11:49AM +, Niklas Haas wrote: > ffmpeg | branch: master | Niklas Haas | Fri Jun 28 21:42:23 > 2024 +0200| [04ce01df0bb2d66e143bcfcea439afc2a1b8d96e] | committer: Niklas > Haas > > avfilter/vf_scale: switch to new swscale API > > Most logic from this filter has been co-opted into swscale itself, > allowing the resulting filter to be substantially simpler as it no > longer has to worry about context initialization, interlacing, etc. > > Sponsored-by: Sovereign Tech Fund > Signed-off-by: Niklas Haas > > > http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=04ce01df0bb2d66e143bcfcea439afc2a1b8d96e > --- > > libavfilter/vf_scale.c | 351 > +++-- > 1 file changed, 77 insertions(+), 274 deletions(-) It seems this broke: ./ffmpeg -i mm-short.mpg -vcodec prores -vframes 3 -bitexact -an prores.mkv ./ffplay prores.mkv [swscaler @ 0x7f76380068c0] Unsupported input (Operation not supported): fmt:yuv422p10le csp:gbr prim:reserved trc:reserved -> fmt:yuv420p csp:bt709 prim:reserved trc:reserved thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB You can kill me, but you cannot change the truth. signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [FFmpeg-cvslog] avfilter/vf_scale: switch to new swscale API
On Fri, 06 Dec 2024 14:31:43 +0100 Michael Niedermayer wrote: > Hi > > On Mon, Nov 25, 2024 at 10:11:49AM +, Niklas Haas wrote: > > ffmpeg | branch: master | Niklas Haas | Fri Jun 28 > > 21:42:23 2024 +0200| [04ce01df0bb2d66e143bcfcea439afc2a1b8d96e] | > > committer: Niklas Haas > > > > avfilter/vf_scale: switch to new swscale API > > > > Most logic from this filter has been co-opted into swscale itself, > > allowing the resulting filter to be substantially simpler as it no > > longer has to worry about context initialization, interlacing, etc. > > > > Sponsored-by: Sovereign Tech Fund > > Signed-off-by: Niklas Haas > > > > > http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=04ce01df0bb2d66e143bcfcea439afc2a1b8d96e > > --- > > > > libavfilter/vf_scale.c | 351 > > +++-- > > 1 file changed, 77 insertions(+), 274 deletions(-) > > It seems this broke: > > ./ffmpeg -i mm-short.mpg -vcodec prores -vframes 3 -bitexact -an prores.mkv > ./ffplay prores.mkv > > [swscaler @ 0x7f76380068c0] Unsupported input (Operation not supported): > fmt:yuv422p10le csp:gbr prim:reserved trc:reserved -> fmt:yuv420p csp:bt709 > prim:reserved trc:reserved What is the meaning of AVCOL_*_RESERVED and why should we accept it here? If you look at e.g. H.273, it clearly reserves these values for future use, and so IMHO treatind them as invalid inputs is not entirely incorrect. Could we fix the prores decoder to instead output UNSPECIFIED? > > thx > > [...] > -- > Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB > > You can kill me, but you cannot change the truth. > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 3/3] avcodec/cuvid: introduce a ringbuffer to reattach additional data
Hi Hendrik, On Tue, 3 Dec 2024 at 19:05, Clément Péron wrote: > > Hi Hendrik, > > On Tue, 3 Dec 2024 at 17:47, Devin Heitmueller > wrote: > > > > On Tue, Dec 3, 2024 at 9:57 AM Hendrik Leppkes wrote: > > > The real question is, why are you using cuviddec instead of nvdec? > > > side data would already pass through natively with nvdec, and other > > > hwaccel-based decoders. > > I think it's a good question and I will try to benchmark performance > and latency of the nvdec against cuvid for my use case. > > Are you sure that side data would be natively supported with nvdec? > > Reading the source code, the only function I saw is > ff_decode_frame_props(), which will make the side data not properly > sync for my use case. I have tested the nvdec and the side_data is properly reattached to the correct packet! But Why? I'm not sure to understand where this logic is done in FFmpeg. Could you point me where this is done? My test shows that the CUVID H264 decode frame latency is ~103ms with σ 35 CUDA seems much faster; the decode time is less than 10ms. My GPU is a RTX4070 but I will run this on a prod machine on a L4. So I will respin this series and drop this patch. Thanks for your remark! > > > > I suspect there could be benefits if your workload involves alot of > > decoding and the GPU only has a single nvdec unit on it. Even if it's > > less efficient to do it with CUDA, getting a graphics card with lots > > of CUDA units could very well be cheaper than a card with multiple > > nvdec fixed units on it (each of which might only be able to do 10-15 > > decodes depending on codec/resolution/framerate). > > > > Devin > > > > -- > > Devin Heitmueller, Senior Software Engineer > > LTN Global Communications > > o: +1 (301) 363-1001 > > w: https://ltnglobal.com e: devin.heitmuel...@ltnglobal.com > > ___ > > ffmpeg-devel mailing list > > ffmpeg-devel@ffmpeg.org > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > > To unsubscribe, visit link above, or email > > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v3 1/2] libavutil/frame: Introduce a new AV_FRAME_DATA_PRFT
This will be used to propagate the Producer Reference Timestamp.
Signed-off-by: Clément Péron
---
libavcodec/decode.c | 1 +
libavfilter/f_sidedata.c | 1 +
libavutil/frame.c| 1 +
libavutil/frame.h| 5 +
4 files changed, 8 insertions(+)
diff --git a/libavcodec/decode.c b/libavcodec/decode.c
index 3380cb27cf..c00e7a62fc 100644
--- a/libavcodec/decode.c
+++ b/libavcodec/decode.c
@@ -1523,6 +1523,7 @@ int ff_decode_frame_props_from_pkt(const AVCodecContext
*avctx,
{ AV_PKT_DATA_S12M_TIMECODE, AV_FRAME_DATA_S12M_TIMECODE
},
{ AV_PKT_DATA_SKIP_SAMPLES, AV_FRAME_DATA_SKIP_SAMPLES },
{ AV_PKT_DATA_LCEVC, AV_FRAME_DATA_LCEVC },
+{ AV_PKT_DATA_PRFT, AV_FRAME_DATA_PRFT },
{ AV_PKT_DATA_NB }
};
diff --git a/libavfilter/f_sidedata.c b/libavfilter/f_sidedata.c
index 23dffec803..c5691a1030 100644
--- a/libavfilter/f_sidedata.c
+++ b/libavfilter/f_sidedata.c
@@ -81,6 +81,7 @@ static const AVOption filt_name##_options[] = { \
{ "DYNAMIC_HDR_VIVID", "", 0, AV_OPT_TYPE_CONST,
{.i64 = AV_FRAME_DATA_DYNAMIC_HDR_VIVID }, 0, 0, FLAGS, .unit = "type"
}, \
{ "AMBIENT_VIEWING_ENVIRONMENT","", 0, AV_OPT_TYPE_CONST,
{.i64 = AV_FRAME_DATA_AMBIENT_VIEWING_ENVIRONMENT}, 0, 0, FLAGS, .unit = "type"
}, \
{ "VIDEO_HINT", "", 0, AV_OPT_TYPE_CONST,
{.i64 = AV_FRAME_DATA_VIDEO_HINT }, 0, 0, FLAGS, .unit = "type"
}, \
+{ "PRFT", "", 0, AV_OPT_TYPE_CONST,
{.i64 = AV_FRAME_DATA_PRFT }, 0, 0, FLAGS, .unit = "type"
}, \
{ NULL } \
}
diff --git a/libavutil/frame.c b/libavutil/frame.c
index f0a0dba018..8cfd21df5b 100644
--- a/libavutil/frame.c
+++ b/libavutil/frame.c
@@ -48,6 +48,7 @@ static const AVSideDataDescriptor sd_props[] = {
[AV_FRAME_DATA_DOVI_METADATA] = { "Dolby Vision Metadata" },
[AV_FRAME_DATA_LCEVC] = { "LCEVC NAL data" },
[AV_FRAME_DATA_VIEW_ID] = { "View ID" },
+[AV_FRAME_DATA_PRFT]= { "Producer time reference"
},
[AV_FRAME_DATA_STEREO3D]= { "Stereo 3D",
AV_SIDE_DATA_PROP_GLOBAL },
[AV_FRAME_DATA_REPLAYGAIN] = { "AVReplayGain",
AV_SIDE_DATA_PROP_GLOBAL },
[AV_FRAME_DATA_DISPLAYMATRIX] = { "3x3 displaymatrix",
AV_SIDE_DATA_PROP_GLOBAL },
diff --git a/libavutil/frame.h b/libavutil/frame.h
index f7806566d5..25657e153f 100644
--- a/libavutil/frame.h
+++ b/libavutil/frame.h
@@ -243,6 +243,11 @@ enum AVFrameSideDataType {
* The data is an int storing the view ID.
*/
AV_FRAME_DATA_VIEW_ID,
+
+/**
+ * Producer time reference data in the form of the AVProducerReferenceTime
struct.
+ */
+AV_FRAME_DATA_PRFT,
};
enum AVActiveFormatDescription {
--
2.47.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v3 0/2] Propagate PRFT side data
Hi, This is in the continuity of the v1 that was sent in September 2023. The idea is to be able to access the Producer Timestamp aka PRFT of a RTSP stream in the encoder to be able to write it in the final file. In my case, the PRFT is a timestamp that is produced by the camera to give the precise time when the image has been captured. The V2 included a ringBuffer to support CUVID hardware decoder, but CUVID is almost deprecated and the Side data is properly supported in NVDEC so it's no more required. Thanks for your comment, Regards Changes since v2: - Drop cuvid ringbuffer Changes since v1: - Rebase on master - Add cuvid ringbuffer Clément Péron (2): libavutil/frame: Introduce a new AV_FRAME_DATA_PRFT avcodec/rawenc: propagate the Producer Reference time libavcodec/decode.c | 1 + libavcodec/rawenc.c | 12 libavfilter/f_sidedata.c | 1 + libavutil/frame.c| 1 + libavutil/frame.h| 5 + 5 files changed, 20 insertions(+) -- 2.47.1 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v3 2/2] avcodec/rawenc: propagate the Producer Reference time
The Producer Reference time contains the source time when the frame
has been produced. This is usefull in the muxer so propagate it.
Signed-off-by: Clément Péron
---
libavcodec/rawenc.c | 12
1 file changed, 12 insertions(+)
diff --git a/libavcodec/rawenc.c b/libavcodec/rawenc.c
index 8c577006d9..f238c8e165 100644
--- a/libavcodec/rawenc.c
+++ b/libavcodec/rawenc.c
@@ -49,6 +49,8 @@ static av_cold int raw_encode_init(AVCodecContext *avctx)
static int raw_encode(AVCodecContext *avctx, AVPacket *pkt,
const AVFrame *frame, int *got_packet)
{
+AVFrameSideData *side_data;
+
int ret = av_image_get_buffer_size(frame->format,
frame->width, frame->height, 1);
@@ -78,6 +80,16 @@ static int raw_encode(AVCodecContext *avctx, AVPacket *pkt,
}
}
*got_packet = 1;
+
+// Forward the PRFT to Mux
+side_data = av_frame_get_side_data(frame, AV_FRAME_DATA_PRFT);
+if (side_data && side_data->size) {
+uint8_t *buf = av_packet_new_side_data(pkt, AV_PKT_DATA_PRFT,
side_data->size);
+if (!buf)
+return AVERROR(ENOMEM);
+memcpy(buf, side_data->data, side_data->size);
+}
+
return 0;
}
--
2.47.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2] avfilter/af_ashowinfo: fix scaling factor of replaygain peak values
Commit 8542f9c4f17125d483c40c0c5723842f1c982f81 changed replaygain peak values to use 100k instead of UINT32_MAX as peak, with possibility of overflow. af_ashowinfo was never updated to reflect this, so we update it now. Fixes: 8542f9c4f17125d483c40c0c5723842f1c982f81 Signed-off-by: Kacper Michajłow --- libavfilter/af_ashowinfo.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavfilter/af_ashowinfo.c b/libavfilter/af_ashowinfo.c index b83847b866..de94d4c7c2 100644 --- a/libavfilter/af_ashowinfo.c +++ b/libavfilter/af_ashowinfo.c @@ -120,7 +120,7 @@ static void print_peak(AVFilterContext *ctx, const char *str, uint32_t peak) if (!peak) av_log(ctx, AV_LOG_INFO, "unknown"); else -av_log(ctx, AV_LOG_INFO, "%f", (float)peak / UINT32_MAX); +av_log(ctx, AV_LOG_INFO, "%f", peak / 10.0f); av_log(ctx, AV_LOG_INFO, ", "); } -- 2.45.1 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 0/3] Make fate tests succeed with zlib-ng
Hi On Sun, Dec 01, 2024 at 09:13:56PM +0100, Alexander Strasser via ffmpeg-devel wrote: > This is a fixed up version of the series I sent before. > > This worked for me on Ubuntu 20.04 but probably will break > with older zlib versions as Hendrik pointed out in the > previous thread. Either we must update zlib on affected > FATE clients or add more .alt files to them as well. > > We could also go the further the "no_file_checksums" as > was demonstrated by James' series. > > I still prefer this way because it's simpler and retains > the value of the tests. > > Of course I'm not opposing to implement zlib compression > in FFmpeg, but until that happens FATE should also work > on zlib-ng systems IMHO. > > > Best regards, > Alexander > > Alexander Strasser (3): > Reapply "tests/fate: disable compression for zlib-based codecs" > fate: Make it possible to have alternative reference files > fate: Add .alt files so all tests pass on Ubuntu 20.04 not reviewed, but it seems working on my box thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Homeopathy is like voting while filling the ballot out with transparent ink. Sometimes the outcome one wanted occurs. Rarely its worse than filling out a ballot properly. signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] tests/swscale: fix sscanf return value check
On Wed, Dec 04, 2024 at 11:43:47AM +0100, Niklas Haas wrote: > From: Niklas Haas > > We only parse 12 values, so this check always failed. Regression caused by > a change to the print format. > > Fixes: 59c39a79cafdcb46972380aac5644f84059cd2a8 > Signed-off-by: Niklas Haas > Sponsored-by: Sovereign Tech Fund > --- > libswscale/tests/swscale.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) LGTM thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB If you think the mosad wants you dead since a long time then you are either wrong or dead since a long time. signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 11/16] swscale/csputils: add internal colorspace math helpers
On 12/6/24 9:32 AM, Niklas Haas wrote:
From: Niklas Haas
Logic is, for the most part, a straight port of similar logic in
liplacebo's colorspace.c, with some general edits and refactors.
---
libswscale/Makefile | 1 +
libswscale/csputils.c | 412 ++
libswscale/csputils.h | 111
3 files changed, 524 insertions(+)
create mode 100644 libswscale/csputils.c
create mode 100644 libswscale/csputils.h
diff --git a/libswscale/Makefile b/libswscale/Makefile
index 81f32f4dd7..08036634b7 100644
--- a/libswscale/Makefile
+++ b/libswscale/Makefile
@@ -6,6 +6,7 @@ HEADERS = swscale.h
\
version_major.h \
OBJS = alphablend.o \
+ csputils.o \
hscale.o \
hscale_fast_bilinear.o \
gamma.o \
diff --git a/libswscale/csputils.c b/libswscale/csputils.c
new file mode 100644
index 00..479caff5dd
--- /dev/null
+++ b/libswscale/csputils.c
@@ -0,0 +1,412 @@
+/*
+ * Copyright (C) 2024 Niklas Haas
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "libavutil/csp.h"
+
+#include "csputils.h"
+#include "utils.h"
+
+void ff_sws_matrix3x3_mul(SwsMatrix3x3 *a, const SwsMatrix3x3 *b)
+{
+float a00 = a->m[0][0], a01 = a->m[0][1], a02 = a->m[0][2],
+ a10 = a->m[1][0], a11 = a->m[1][1], a12 = a->m[1][2],
+ a20 = a->m[2][0], a21 = a->m[2][1], a22 = a->m[2][2];
+
+for (int i = 0; i < 3; i++) {
+a->m[0][i] = a00 * b->m[0][i] + a01 * b->m[1][i] + a02 * b->m[2][i];
+a->m[1][i] = a10 * b->m[0][i] + a11 * b->m[1][i] + a12 * b->m[2][i];
+a->m[2][i] = a20 * b->m[0][i] + a21 * b->m[1][i] + a22 * b->m[2][i];
+}
+}
+
+void ff_sws_matrix3x3_rmul(const SwsMatrix3x3 *a, SwsMatrix3x3 *b)
+{
+float b00 = b->m[0][0], b01 = b->m[0][1], b02 = b->m[0][2],
+ b10 = b->m[1][0], b11 = b->m[1][1], b12 = b->m[1][2],
+ b20 = b->m[2][0], b21 = b->m[2][1], b22 = b->m[2][2];
+
+for (int i = 0; i < 3; i++) {
+b->m[i][0] = a->m[i][0] * b00 + a->m[i][1] * b10 + a->m[i][2] * b20;
+b->m[i][1] = a->m[i][0] * b01 + a->m[i][1] * b11 + a->m[i][2] * b21;
+b->m[i][2] = a->m[i][0] * b02 + a->m[i][1] * b12 + a->m[i][2] * b22;
+}
+}
+
+void ff_sws_matrix3x3_invert(SwsMatrix3x3 *mat)
+{
+double m00 = mat->m[0][0], m01 = mat->m[0][1], m02 = mat->m[0][2],
+ m10 = mat->m[1][0], m11 = mat->m[1][1], m12 = mat->m[1][2],
+ m20 = mat->m[2][0], m21 = mat->m[2][1], m22 = mat->m[2][2];
+
+// calculate the adjoint
+double a00 = (m11 * m22 - m21 * m12);
+double a01 = -(m01 * m22 - m21 * m02);
+double a02 = (m01 * m12 - m11 * m02);
+double a10 = -(m10 * m22 - m20 * m12);
+double a11 = (m00 * m22 - m20 * m02);
+double a12 = -(m00 * m12 - m10 * m02);
+double a20 = (m10 * m21 - m20 * m11);
+double a21 = -(m00 * m21 - m20 * m01);
+double a22 = (m00 * m11 - m10 * m01);
+
+// calculate the determinant (as inverse == 1/det * adjoint,
+// adjoint * m == identity * det, so this calculates the det)
+double det = m00 * a00 + m10 * a01 + m20 * a02;
+det = 1.0 / det;
+
+mat->m[0][0] = det * a00;
+mat->m[0][1] = det * a01;
+mat->m[0][2] = det * a02;
+mat->m[1][0] = det * a10;
+mat->m[1][1] = det * a11;
+mat->m[1][2] = det * a12;
+mat->m[2][0] = det * a20;
+mat->m[2][1] = det * a21;
+mat->m[2][2] = det * a22;
+}
+
+void ff_sws_matrix3x3_apply(const SwsMatrix3x3 *mat, float vec[3])
+{
+float x = vec[0], y = vec[1], z = vec[2];
+
+for (int i = 0; i < 3; i++)
+vec[i] = mat->m[i][0] * x + mat->m[i][1] * y + mat->m[i][2] * z;
+}
+
+/* Recovers (X / Y) from a CIE xy value. */
+static inline AVRational cie_X(AVCIExy xy)
+{
+return av_div_q(xy.x, xy.y);
+}
+
+/* Recovers (Z / Y) from a CIE xy value. */
+static inline AVRational cie_Z(AVCIExy xy)
+{
+return av_div_q(av_sub_q(av_sub_q(av_make_q(1, 1), xy.x), xy.y), xy.y);
+}
+
+SwsMatrix3x3 ff_sws_rgb2
Re: [FFmpeg-devel] [PATCH v2 10/16] avfilter/vf_scale: strip metadata when changing colorspace
On 12/6/24 9:32 AM, Niklas Haas wrote:
From: Niklas Haas
This is no longer relevant after a change in color space.
---
libavfilter/vf_scale.c | 12
1 file changed, 12 insertions(+)
diff --git a/libavfilter/vf_scale.c b/libavfilter/vf_scale.c
index e33617468a..a56d452c6c 100644
--- a/libavfilter/vf_scale.c
+++ b/libavfilter/vf_scale.c
@@ -868,6 +868,18 @@ scale:
if (scale->out_transfer != AVCOL_TRC_UNSPECIFIED)
out->color_trc = scale->out_transfer;
+if (in->color_primaries != out->color_primaries ||
+in->color_trc != out->color_trc)
+{
+av_frame_remove_side_data(out,
AV_FRAME_DATA_MASTERING_DISPLAY_METADATA);
+av_frame_remove_side_data(out, AV_FRAME_DATA_DYNAMIC_HDR_PLUS);
+av_frame_remove_side_data(out, AV_FRAME_DATA_DYNAMIC_HDR_VIVID);
+av_frame_remove_side_data(out, AV_FRAME_DATA_CONTENT_LIGHT_LEVEL);
+av_frame_remove_side_data(out, AV_FRAME_DATA_ICC_PROFILE);
+av_frame_remove_side_data(out, AV_FRAME_DATA_DOVI_METADATA);
+av_frame_remove_side_data(out, AV_FRAME_DATA_DOVI_RPU_BUFFER);
+}
+
av_reduce(&out->sample_aspect_ratio.num, &out->sample_aspect_ratio.den,
(int64_t)in->sample_aspect_ratio.num * outlink->h * link->w,
(int64_t)in->sample_aspect_ratio.den * outlink->w * link->h,
I think Mastering Display Metadata may still be relevant after a change
in TRC but not primaries (e.g. after linearization), since it only
references the gamut of the mastering display.
- Leo Izen (Traneptora)
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] avformat/movenc: log stream index for invalid pkt duration
LGTM ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v3 2/2] lavc/rv40dsp: fix RISC-V chroma_mc
Le 6 décembre 2024 13:19:29 GMT+01:00, Niklas Haas a écrit : >On Thu, 05 Dec 2024 21:41:51 +0800 uk7b-at-foxmail@ffmpeg.org wrote: >> From: sunyuechi >> >> --- >> libavcodec/riscv/rv40dsp_rvv.S | 116 ++--- >> 1 file changed, 78 insertions(+), 38 deletions(-) >> >> diff --git a/libavcodec/riscv/rv40dsp_rvv.S b/libavcodec/riscv/rv40dsp_rvv.S >> index ca431eb8ab..d4beb7f1e9 100644 >> --- a/libavcodec/riscv/rv40dsp_rvv.S >> +++ b/libavcodec/riscv/rv40dsp_rvv.S >> @@ -20,15 +20,30 @@ >> >> #include "libavutil/riscv/asm.S" >> >> -.macro manual_avg dst src1 src2 >> -vadd.vv \dst, \src1, \src2 >> -vadd.vi \dst, \dst, 1 >> -vsrl.vi \dst, \dst, 1 >> -.endm >> +const rv40_bias >> +.byte 0, 16, 32, 16 >> +.byte 32, 28, 32, 28 >> +.byte 0, 32, 16, 32 >> +.byte 32, 28, 32, 28 >> +endconst >> >> .macro do_chroma_mc type unroll >> -csrwi vxrm, 2 >> +csrwi vxrm, 0 >> +addisp, sp, -16 >> +#if __riscv_xlen == 32 >> +sw s2, (sp) >> +#elif __riscv_xlen == 64 >> +sd s2, (sp) >> +#else >> +sq s2, (sp) >> +#endif > >You can use the macro `sx` from libavcodec/riscv/h264qpel_rvv.S > >(I suggest to move them into some common include path) As we need to backport this, should we merge this to stable and the sx/lx version to master? Or should we merge this to master so that we can cherry-pick the exact patch to master? > >> +lla t4, rv40_bias >> +srlit5, a5, 1 >> +sh2add t4, t5, t4 >> +srlit5, a4, 1 >> +add t5, t4, t5 >> sllit2, a5, 3 >> +lb s2, (t5) >> mul t1, a5, a4 >> sh3add a5, a4, t2 >> sllia4, a4, 3 >> @@ -80,17 +95,19 @@ >> vwmulu.vx v12, v14, a6 >> vwmaccu.vx v10, t1, v15 >> vwmaccu.vx v12, a7, v15 >> -vnclipu.wi v15, v8, 6 >> +vwaddu.wx v20, v8, s2 >> +vnsrl.wiv15, v20, 6 >>.ifc \type,avg >> vle8.v v9, (a0) >> -manual_avg v15, v15, v9 >> +vaaddu.vv v15, v15, v9 >>.endif >> vse8.v v15, (a0) >> add a0, a0, a2 >> -vnclipu.wi v8, v10, 6 >> +vwaddu.wx v20, v10, s2 >> +vnsrl.wiv8, v20, 6 >>.ifc \type,avg >> vle8.v v9, (a0) >> -manual_avg v8, v8, v9 >> +vaaddu.vv v8, v8, v9 >>.endif >> add t4, t4, t3 >> vse8.v v8, (a0) >> @@ -115,17 +132,19 @@ >> vslide1down.vx v14, v14, t5 >> vsetvli zero, t6, e8, m1, ta, ma >> vwmaccu.vx v16, t1, v14 >> -vnclipu.wi v8, v12, 6 >> +vwaddu.wx v20, v12, s2 >> +vnsrl.wiv8, v20, 6 >>.ifc \type,avg >> vle8.v v9, (a0) >> -manual_avg v8, v8, v9 >> +vaaddu.vv v8, v8, v9 >>.endif >> vse8.v v8, (a0) >> add a0, a0, a2 >> -vnclipu.wi v8, v16, 6 >> +vwaddu.wx v20, v16, s2 >> +vnsrl.wiv8, v20, 6 >>.ifc \type,avg >> vle8.v v9, (a0) >> -manual_avg v8, v8, v9 >> +vaaddu.vv v8, v8, v9 >>.endif >> vse8.v v8, (a0) >> add a0, a0, a2 >> @@ -159,18 +178,20 @@ >> vwmaccu.vx v10, t0, v8 >> add a4, a4, a7 >> vwmaccu.vx v12, t0, v9 >> -vnclipu.wi v15, v10, 6 >> +vwaddu.wx v20, v10, s2 >> +vnsrl.wiv15, v20, 6 >> vwmulu.vx v10, v9, a6 >> -vnclipu.wi v9, v12, 6 >> +vwaddu.wx v20, v12, s2 >> +vnsrl.wiv9, v20, 6 >>.ifc \type,avg >> vle8.v v16, (a0) >> -manual_avg v15, v15, v16 >> +vaaddu.vv v15, v15, v16 >>.endif >> vse8.v v15, (a0) >> add a0, a0, a2 >>.ifc \type,avg >> vle8.v v16, (a0) >> -manual_avg v9, v9, v16 >> +vaaddu.vv v9, v9, v16 >>.endif >> vse8.v v9, (a0) >> add a0, a0, a2 >> @@ -179,18 +200,20 @@ >> vle8.v v14, (a5) >> vwmaccu.vx v10, t0, v8 >> vwmulu.vx v12, v8, a6 >> -vnclipu.wi v8, v10, 6 >> +vwaddu.wx v20, v10, s2 >> +vnsrl.wiv8, v20, 6 >> vwmaccu.vx v12, t0, v14 >>.ifc \type,avg >> vle8.v v16, (a0) >> -manual_avg v8, v8, v16 >> +vaaddu.vv v8, v8, v16 >>.endif >> vse8.v v8, (a0) >>
Re: [FFmpeg-devel] [FFmpeg-cvslog] avfilter/vf_scale: switch to new swscale API
Hi On Fri, Dec 06, 2024 at 02:40:06PM +0100, Niklas Haas wrote: > On Fri, 06 Dec 2024 14:31:43 +0100 Michael Niedermayer > wrote: > > Hi > > > > On Mon, Nov 25, 2024 at 10:11:49AM +, Niklas Haas wrote: > > > ffmpeg | branch: master | Niklas Haas | Fri Jun 28 > > > 21:42:23 2024 +0200| [04ce01df0bb2d66e143bcfcea439afc2a1b8d96e] | > > > committer: Niklas Haas > > > > > > avfilter/vf_scale: switch to new swscale API > > > > > > Most logic from this filter has been co-opted into swscale itself, > > > allowing the resulting filter to be substantially simpler as it no > > > longer has to worry about context initialization, interlacing, etc. > > > > > > Sponsored-by: Sovereign Tech Fund > > > Signed-off-by: Niklas Haas > > > > > > > http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=04ce01df0bb2d66e143bcfcea439afc2a1b8d96e > > > --- > > > > > > libavfilter/vf_scale.c | 351 > > > +++-- > > > 1 file changed, 77 insertions(+), 274 deletions(-) > > > > It seems this broke: > > > > ./ffmpeg -i mm-short.mpg -vcodec prores -vframes 3 -bitexact -an prores.mkv > > ./ffplay prores.mkv > > > > [swscaler @ 0x7f76380068c0] Unsupported input (Operation not supported): > > fmt:yuv422p10le csp:gbr prim:reserved trc:reserved -> fmt:yuv420p csp:bt709 > > prim:reserved trc:reserved > > What is the meaning of AVCOL_*_RESERVED and why should we accept it here? > > If you look at e.g. H.273, it clearly reserves these values for future use, > and so IMHO treatind them as invalid inputs is not entirely incorrect. > > Could we fix the prores decoder to instead output UNSPECIFIED? the decoder should probably output the value stored in the file. I have to say its a little odd a file generated by ffmpeg contains a reserved value but how do i know without a prores spec ... Now even if theres an argument for the prores decoder not to output that value here. Another decoder and other file could still have exactly that reserved value. And in that case that decoder should output that thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Good people do not need laws to tell them to act responsibly, while bad people will find a way around the laws. -- Plato signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2 02/16] swscale/utils: add HDR metadata to SwsFormat
From: Niklas Haas
Only add the condensed values that we actually care about. Group them into
a new struct to make it easier to discard or replace this metadata.
Define a special comparison function that does not choke on undefined/unknown
metadata.
---
libswscale/swscale.c | 7 ---
libswscale/utils.c | 28 +++---
libswscale/utils.h | 48 +++-
3 files changed, 63 insertions(+), 20 deletions(-)
diff --git a/libswscale/swscale.c b/libswscale/swscale.c
index 3c4637c0a1..7e77e82c8d 100644
--- a/libswscale/swscale.c
+++ b/libswscale/swscale.c
@@ -1362,7 +1362,8 @@ int sws_frame_setup(SwsContext *ctx, const AVFrame *dst,
const AVFrame *src)
}
/* TODO: remove once implemented */
-if ((dst_fmt.prim != src_fmt.prim || dst_fmt.trc != src_fmt.trc) &&
+if ((dst_fmt.color.prim != src_fmt.color.prim ||
+ dst_fmt.color.trc != src_fmt.color.trc) &&
!s->color_conversion_warned)
{
av_log(ctx, AV_LOG_WARNING, "Conversions between different
primaries / "
@@ -1407,9 +1408,9 @@ int sws_frame_setup(SwsContext *ctx, const AVFrame *dst,
const AVFrame *src)
" fmt:%s csp:%s prim:%s trc:%s\n",
err_msg, av_err2str(ret),
av_get_pix_fmt_name(src_fmt.format),
av_color_space_name(src_fmt.csp),
- av_color_primaries_name(src_fmt.prim),
av_color_transfer_name(src_fmt.trc),
+ av_color_primaries_name(src_fmt.color.prim),
av_color_transfer_name(src_fmt.color.trc),
av_get_pix_fmt_name(dst_fmt.format),
av_color_space_name(dst_fmt.csp),
- av_color_primaries_name(dst_fmt.prim),
av_color_transfer_name(dst_fmt.trc));
+ av_color_primaries_name(dst_fmt.color.prim),
av_color_transfer_name(dst_fmt.color.trc));
for (int i = 0; i < FF_ARRAY_ELEMS(s->graph); i++)
sws_graph_free(&s->graph[i]);
diff --git a/libswscale/utils.c b/libswscale/utils.c
index 43805b9679..428cf1c7f5 100644
--- a/libswscale/utils.c
+++ b/libswscale/utils.c
@@ -2659,11 +2659,13 @@ SwsFormat ff_fmt_from_frame(const AVFrame *frame, int
field)
.height = frame->height,
.format = frame->format,
.range = frame->color_range,
-.prim = frame->color_primaries,
-.trc= frame->color_trc,
.csp= frame->colorspace,
.loc= frame->chroma_location,
.desc = desc,
+.color = {
+.prim = frame->color_primaries,
+.trc = frame->color_trc,
+},
};
av_assert1(fmt.width > 0);
@@ -2676,12 +2678,14 @@ SwsFormat ff_fmt_from_frame(const AVFrame *frame, int
field)
fmt.range = AVCOL_RANGE_JPEG;
} else if (desc->flags & AV_PIX_FMT_FLAG_XYZ) {
fmt.csp = AVCOL_SPC_UNSPECIFIED;
-fmt.prim = AVCOL_PRI_SMPTE428;
-fmt.trc = AVCOL_TRC_SMPTE428;
+fmt.color = (SwsColor) {
+.prim = AVCOL_PRI_SMPTE428,
+.trc = AVCOL_TRC_SMPTE428,
+};
} else if (desc->nb_components < 3) {
/* Grayscale formats */
-fmt.prim = AVCOL_PRI_UNSPECIFIED;
-fmt.csp = AVCOL_SPC_UNSPECIFIED;
+fmt.color.prim = AVCOL_PRI_UNSPECIFIED;
+fmt.csp= AVCOL_SPC_UNSPECIFIED;
if (desc->flags & AV_PIX_FMT_FLAG_FLOAT)
fmt.range = AVCOL_RANGE_UNSPECIFIED;
else
@@ -2756,12 +2760,12 @@ static int test_loc(enum AVChromaLocation loc)
int ff_test_fmt(const SwsFormat *fmt, int output)
{
-return fmt->width > 0 && fmt->height > 0&&
- sws_test_format(fmt->format, output) &&
- sws_test_colorspace(fmt->csp,output) &&
- sws_test_primaries (fmt->prim, output) &&
- sws_test_transfer (fmt->trc,output) &&
- test_range (fmt->range) &&
+return fmt->width > 0 && fmt->height > 0&&
+ sws_test_format(fmt->format, output) &&
+ sws_test_colorspace(fmt->csp,output) &&
+ sws_test_primaries (fmt->color.prim, output) &&
+ sws_test_transfer (fmt->color.trc, output) &&
+ test_range (fmt->range) &&
test_loc (fmt->loc);
}
diff --git a/libswscale/utils.h b/libswscale/utils.h
index 4d204ef6cc..1263e3f8ed 100644
--- a/libswscale/utils.h
+++ b/libswscale/utils.h
@@ -21,26 +21,55 @@
#ifndef SWSCALE_UTILS_H
#define SWSCALE_UTILS_H
+#include "libavutil/csp.h"
#include "libavutil/pixdesc.h"
#include "swscale.h"
+/* Like av_cmp_q but considers 0/0 == 0/0 */
+static inline int ff_q_equal(const AVRational a, const AVRational b)
+{
+return (a.den || b.den) ? !av_cmp_q(a, b) : a.den == b.den;
+}
+
+static inline int ff_cie_xy_equal(const AVCIExy a, const AVCIExy b)
+{
+return ff_q_equal(a.x, b.x) && ff_q_equal(a.y, b.y);
+}
+
+static inline i
[FFmpeg-devel] [PATCH v2 16/16] swscale: remove primaries/trc change warning
From: Niklas Haas
This is now supported when using the new API.
---
libswscale/swscale.c | 11 ---
1 file changed, 11 deletions(-)
diff --git a/libswscale/swscale.c b/libswscale/swscale.c
index 7e77e82c8d..42efccd12b 100644
--- a/libswscale/swscale.c
+++ b/libswscale/swscale.c
@@ -1361,17 +1361,6 @@ int sws_frame_setup(SwsContext *ctx, const AVFrame *dst,
const AVFrame *src)
goto fail;
}
-/* TODO: remove once implemented */
-if ((dst_fmt.color.prim != src_fmt.color.prim ||
- dst_fmt.color.trc != src_fmt.color.trc) &&
-!s->color_conversion_warned)
-{
-av_log(ctx, AV_LOG_WARNING, "Conversions between different
primaries / "
- "transfer functions are not currently implemented, expect "
- "wrong results.\n");
-s->color_conversion_warned = 1;
-}
-
if (!ff_test_fmt(&src_fmt, 0)) {
err_msg = "Unsupported input";
ret = AVERROR(ENOTSUP);
--
2.47.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2 05/16] swscale/utils: read dynamic HDR10+ metadata from AVFrame
From: Niklas Haas
Logic ported from libplacebo's AVFrame helpers. The basic idea is to use the
provided MaxRGB/MaxSCL values to infer what the actual luminance would have
been, which HDR10+ metadata does not provide directly. It's worth pointing out
that this gives us an *upper* bound on the true maximum luminance, so any
error in the estimation cannot result in clipping.
---
libswscale/utils.c | 50 ++
1 file changed, 50 insertions(+)
diff --git a/libswscale/utils.c b/libswscale/utils.c
index 3d3fcadacc..b097ec51b2 100644
--- a/libswscale/utils.c
+++ b/libswscale/utils.c
@@ -43,6 +43,7 @@
#include "libavutil/cpu.h"
#include "libavutil/csp.h"
#include "libavutil/emms.h"
+#include "libavutil/hdr_dynamic_metadata.h"
#include "libavutil/imgutils.h"
#include "libavutil/intreadwrite.h"
#include "libavutil/libm.h"
@@ -2745,6 +2746,55 @@ SwsFormat ff_fmt_from_frame(const AVFrame *frame, int
field)
}
}
+if ((sd = av_frame_get_side_data(frame, AV_FRAME_DATA_DYNAMIC_HDR_PLUS))) {
+const AVDynamicHDRPlus *dhp = (const AVDynamicHDRPlus *) sd->data;
+const AVHDRPlusColorTransformParams *pars = &dhp->params[0];
+const AVRational nits = av_make_q(1, 1);
+AVRational maxrgb = pars->maxscl[0];
+
+if (!dhp->num_windows || dhp->application_version > 1)
+goto skip_hdr10;
+
+/* Maximum of MaxSCL components */
+if (av_cmp_q(pars->maxscl[1], maxrgb) > 0)
+maxrgb = pars->maxscl[1];
+if (av_cmp_q(pars->maxscl[2], maxrgb) > 0)
+maxrgb = pars->maxscl[2];
+
+if (maxrgb.num > 0) {
+/* Estimate true luminance from MaxSCL */
+const AVLumaCoefficients *luma =
av_csp_luma_coeffs_from_avcsp(fmt.csp);
+if (!luma)
+goto skip_hdr10;
+fmt.color.frame_peak = av_add_q(av_mul_q(luma->cr,
pars->maxscl[0]),
+ av_add_q(av_mul_q(luma->cg,
pars->maxscl[1]),
+av_mul_q(luma->cb,
pars->maxscl[2])));
+/* Scale the scene average brightness by the ratio between the
+ * maximum luminance and the MaxRGB values */
+fmt.color.frame_avg = av_mul_q(pars->average_maxrgb,
+ av_div_q(fmt.color.frame_peak,
maxrgb));
+} else {
+/**
+ * Calculate largest value from histogram to use as fallback for
+ * clips with missing MaxSCL information. Note that this may end
+ * up picking the "reserved" value at the 5% percentile, which in
+ * practice appears to track the brightest pixel in the scene.
+ */
+for (int i = 0; i < pars->num_distribution_maxrgb_percentiles;
i++) {
+const AVRational pct = pars->distribution_maxrgb[i].percentile;
+if (av_cmp_q(pct, maxrgb) > 0)
+maxrgb = pct;
+fmt.color.frame_peak = maxrgb;
+fmt.color.frame_avg = pars->average_maxrgb;
+}
+}
+
+/* Rescale to nits */
+fmt.color.frame_peak = av_mul_q(nits, fmt.color.frame_peak);
+fmt.color.frame_avg = av_mul_q(nits, fmt.color.frame_avg);
+}
+skip_hdr10:
+
/* PQ is always scaled down to absolute zero, so ignore mastering metadata
*/
if (fmt.color.trc == AVCOL_TRC_SMPTE2084)
fmt.color.min_luma = av_make_q(0, 1);
--
2.47.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2 01/16] swscale/utils: check for supported color transfers
From: Niklas Haas
We will use the av_csp_itu_eotf() functions to decode these internally, so
check this function to see if it succeeds.
---
libswscale/utils.c | 6 --
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/libswscale/utils.c b/libswscale/utils.c
index 32f90e366e..43805b9679 100644
--- a/libswscale/utils.c
+++ b/libswscale/utils.c
@@ -41,6 +41,7 @@
#include "libavutil/attributes.h"
#include "libavutil/avassert.h"
#include "libavutil/cpu.h"
+#include "libavutil/csp.h"
#include "libavutil/emms.h"
#include "libavutil/imgutils.h"
#include "libavutil/intreadwrite.h"
@@ -2738,8 +2739,9 @@ int sws_test_primaries(enum AVColorPrimaries prim, int
output)
int sws_test_transfer(enum AVColorTransferCharacteristic trc, int output)
{
-return trc > AVCOL_TRC_RESERVED0 && trc < AVCOL_TRC_NB &&
- trc != AVCOL_TRC_RESERVED;
+av_csp_eotf_function eotf = output ? av_csp_itu_eotf_inv(trc)
+ : av_csp_itu_eotf(trc);
+return trc == AVCOL_TRC_UNSPECIFIED || eotf != NULL;
}
static int test_range(enum AVColorRange range)
--
2.47.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2 03/16] swscale/utils: set static/implied HDR metadata
From: Niklas Haas
Provide default values for the fields added in the previous commit.
---
libswscale/utils.c | 21 +
1 file changed, 21 insertions(+)
diff --git a/libswscale/utils.c b/libswscale/utils.c
index 428cf1c7f5..182f92396a 100644
--- a/libswscale/utils.c
+++ b/libswscale/utils.c
@@ -2654,6 +2654,8 @@ int ff_range_add(RangeList *rl, unsigned int start,
unsigned int len)
SwsFormat ff_fmt_from_frame(const AVFrame *frame, int field)
{
const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(frame->format);
+const AVColorPrimariesDesc *primaries;
+
SwsFormat fmt = {
.width = frame->width,
.height = frame->height,
@@ -2710,6 +2712,25 @@ SwsFormat ff_fmt_from_frame(const AVFrame *frame, int
field)
fmt.interlaced = 1;
}
+/* Set luminance and gamut information */
+fmt.color.min_luma = av_make_q(0, 1);
+switch (fmt.color.trc) {
+case AVCOL_TRC_SMPTE2084:
+fmt.color.max_luma = av_make_q(1, 1); break;
+case AVCOL_TRC_ARIB_STD_B67:
+fmt.color.max_luma = av_make_q( 1000, 1); break; /* HLG reference
display */
+default:
+fmt.color.max_luma = av_make_q( 203, 1); break; /* SDR reference
brightness */
+}
+
+primaries = av_csp_primaries_desc_from_id(fmt.color.prim);
+if (primaries)
+fmt.color.gamut = primaries->prim;
+
+/* PQ is always scaled down to absolute zero, so ignore mastering metadata
*/
+if (fmt.color.trc == AVCOL_TRC_SMPTE2084)
+fmt.color.min_luma = av_make_q(0, 1);
+
return fmt;
}
--
2.47.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2 11/16] swscale/csputils: add internal colorspace math helpers
From: Niklas Haas
Logic is, for the most part, a straight port of similar logic in
liplacebo's colorspace.c, with some general edits and refactors.
---
libswscale/Makefile | 1 +
libswscale/csputils.c | 412 ++
libswscale/csputils.h | 111
3 files changed, 524 insertions(+)
create mode 100644 libswscale/csputils.c
create mode 100644 libswscale/csputils.h
diff --git a/libswscale/Makefile b/libswscale/Makefile
index 81f32f4dd7..08036634b7 100644
--- a/libswscale/Makefile
+++ b/libswscale/Makefile
@@ -6,6 +6,7 @@ HEADERS = swscale.h
\
version_major.h \
OBJS = alphablend.o \
+ csputils.o \
hscale.o \
hscale_fast_bilinear.o \
gamma.o \
diff --git a/libswscale/csputils.c b/libswscale/csputils.c
new file mode 100644
index 00..479caff5dd
--- /dev/null
+++ b/libswscale/csputils.c
@@ -0,0 +1,412 @@
+/*
+ * Copyright (C) 2024 Niklas Haas
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "libavutil/csp.h"
+
+#include "csputils.h"
+#include "utils.h"
+
+void ff_sws_matrix3x3_mul(SwsMatrix3x3 *a, const SwsMatrix3x3 *b)
+{
+float a00 = a->m[0][0], a01 = a->m[0][1], a02 = a->m[0][2],
+ a10 = a->m[1][0], a11 = a->m[1][1], a12 = a->m[1][2],
+ a20 = a->m[2][0], a21 = a->m[2][1], a22 = a->m[2][2];
+
+for (int i = 0; i < 3; i++) {
+a->m[0][i] = a00 * b->m[0][i] + a01 * b->m[1][i] + a02 * b->m[2][i];
+a->m[1][i] = a10 * b->m[0][i] + a11 * b->m[1][i] + a12 * b->m[2][i];
+a->m[2][i] = a20 * b->m[0][i] + a21 * b->m[1][i] + a22 * b->m[2][i];
+}
+}
+
+void ff_sws_matrix3x3_rmul(const SwsMatrix3x3 *a, SwsMatrix3x3 *b)
+{
+float b00 = b->m[0][0], b01 = b->m[0][1], b02 = b->m[0][2],
+ b10 = b->m[1][0], b11 = b->m[1][1], b12 = b->m[1][2],
+ b20 = b->m[2][0], b21 = b->m[2][1], b22 = b->m[2][2];
+
+for (int i = 0; i < 3; i++) {
+b->m[i][0] = a->m[i][0] * b00 + a->m[i][1] * b10 + a->m[i][2] * b20;
+b->m[i][1] = a->m[i][0] * b01 + a->m[i][1] * b11 + a->m[i][2] * b21;
+b->m[i][2] = a->m[i][0] * b02 + a->m[i][1] * b12 + a->m[i][2] * b22;
+}
+}
+
+void ff_sws_matrix3x3_invert(SwsMatrix3x3 *mat)
+{
+double m00 = mat->m[0][0], m01 = mat->m[0][1], m02 = mat->m[0][2],
+ m10 = mat->m[1][0], m11 = mat->m[1][1], m12 = mat->m[1][2],
+ m20 = mat->m[2][0], m21 = mat->m[2][1], m22 = mat->m[2][2];
+
+// calculate the adjoint
+double a00 = (m11 * m22 - m21 * m12);
+double a01 = -(m01 * m22 - m21 * m02);
+double a02 = (m01 * m12 - m11 * m02);
+double a10 = -(m10 * m22 - m20 * m12);
+double a11 = (m00 * m22 - m20 * m02);
+double a12 = -(m00 * m12 - m10 * m02);
+double a20 = (m10 * m21 - m20 * m11);
+double a21 = -(m00 * m21 - m20 * m01);
+double a22 = (m00 * m11 - m10 * m01);
+
+// calculate the determinant (as inverse == 1/det * adjoint,
+// adjoint * m == identity * det, so this calculates the det)
+double det = m00 * a00 + m10 * a01 + m20 * a02;
+det = 1.0 / det;
+
+mat->m[0][0] = det * a00;
+mat->m[0][1] = det * a01;
+mat->m[0][2] = det * a02;
+mat->m[1][0] = det * a10;
+mat->m[1][1] = det * a11;
+mat->m[1][2] = det * a12;
+mat->m[2][0] = det * a20;
+mat->m[2][1] = det * a21;
+mat->m[2][2] = det * a22;
+}
+
+void ff_sws_matrix3x3_apply(const SwsMatrix3x3 *mat, float vec[3])
+{
+float x = vec[0], y = vec[1], z = vec[2];
+
+for (int i = 0; i < 3; i++)
+vec[i] = mat->m[i][0] * x + mat->m[i][1] * y + mat->m[i][2] * z;
+}
+
+/* Recovers (X / Y) from a CIE xy value. */
+static inline AVRational cie_X(AVCIExy xy)
+{
+return av_div_q(xy.x, xy.y);
+}
+
+/* Recovers (Z / Y) from a CIE xy value. */
+static inline AVRational cie_Z(AVCIExy xy)
+{
+return av_div_q(av_sub_q(av_sub_q(av_make_q(1, 1), xy.x), xy.y), xy.y);
+}
+
+SwsMatrix3x3 ff_sws_rgb2xyz(const AVColorPrimariesDesc *desc)
+{
+SwsMatrix3x
[FFmpeg-devel] [PATCH v2 12/16] swscale/cms: add color management subsystem
From: Niklas Haas
The underlying color mapping logic was ported as straightforwardly as possible
from libplacebo, although the API and glue code has been very heavily
refactored / rewritten. In particular, the generalization of gamut mapping
methods is replaced by a single ICC intent selection, and constants have been
hard-coded.
To minimize the amount of overall operations, this gamut mapping LUT now embeds
a direct end-to-end transformation to the output color space; something that
libplacebo does in shaders, but which is prohibitively expensive in software.
In order to preserve compatibility with dynamic tone mapping without severely
regressing performance, we add the ability to generate a pair of "split" LUTS,
one for encoding the input and output to the perceptual color space, and a
third to embed the tone mapping operation. Additionally, this intermediate
space could be used for additional subjective effect (e.g. changing
saturation or brightness).
The big downside of the new approach is that generating a static color mapping
LUT is now fairly slow, as the chromaticity lobe peaks have to be recomputed
for every single RGB value, since correlated RGB colors are not necessarily
aligned in ICh space. Generating a split 3DLUT significantly alleviates this
problem because the expensive step is done as part of the IPT input LUT, which
can share the same hue peak calculation at least for all input intensities.
---
libswscale/Makefile | 1 +
libswscale/cms.c| 766
libswscale/cms.h| 105 ++
3 files changed, 872 insertions(+)
create mode 100644 libswscale/cms.c
create mode 100644 libswscale/cms.h
diff --git a/libswscale/Makefile b/libswscale/Makefile
index 08036634b7..c4e45d494e 100644
--- a/libswscale/Makefile
+++ b/libswscale/Makefile
@@ -6,6 +6,7 @@ HEADERS = swscale.h
\
version_major.h \
OBJS = alphablend.o \
+ cms.o\
csputils.o \
hscale.o \
hscale_fast_bilinear.o \
diff --git a/libswscale/cms.c b/libswscale/cms.c
new file mode 100644
index 00..7793a19b87
--- /dev/null
+++ b/libswscale/cms.c
@@ -0,0 +1,766 @@
+/*
+ * Copyright (C) 2024 Niklas Haas
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include
+#include
+
+#include "libavutil/attributes.h"
+#include "libavutil/avassert.h"
+#include "libavutil/csp.h"
+#include "libavutil/slicethread.h"
+
+#include "cms.h"
+#include "csputils.h"
+#include "libswscale/swscale.h"
+#include "utils.h"
+
+bool sws_color_map_noop(const SwsColorMap *map)
+{
+/* If the encoding space is different, we must go through a conversion */
+if (map->src.prim != map->dst.prim || map->src.trc != map->dst.trc)
+return false;
+
+/* If the black point changes, we have to perform black point compensation
*/
+if (av_cmp_q(map->src.min_luma, map->dst.min_luma))
+return false;
+
+switch (map->intent) {
+case SWS_INTENT_ABSOLUTE_COLORIMETRIC:
+case SWS_INTENT_RELATIVE_COLORIMETRIC:
+return ff_prim_superset(&map->dst.gamut, &map->src.gamut) &&
+ av_cmp_q(map->src.max_luma, map->dst.max_luma) <= 0;
+case SWS_INTENT_PERCEPTUAL:
+case SWS_INTENT_SATURATION:
+return ff_prim_equal(&map->dst.gamut, &map->src.gamut) &&
+ !av_cmp_q(map->src.max_luma, map->dst.max_luma);
+default:
+av_assert0(!"Invalid gamut mapping intent?");
+return true;
+}
+}
+
+/* Approximation of gamut hull at a given intensity level */
+static const float hull(float I)
+{
+return ((I - 6.0f) * I + 9.0f) * I;
+}
+
+/* For some minimal type safety, and code cleanliness */
+typedef struct RGB {
+float R, G, B; /* nits */
+} RGB;
+
+typedef struct IPT {
+float I, P, T;
+} IPT;
+
+typedef struct ICh {
+float I, C, h;
+} ICh;
+
+static av_always_inline ICh ipt2ich(IPT c)
+{
+return (ICh) {
+.I = c.I,
+.C = sqrtf(c.P * c.P + c.T * c.T),
+.h = atan2f(c.T,
[FFmpeg-devel] Decoding performance -f rawvideo pipe:1 vs BMP images output
Hi, I am trying to convert a RTSP stream to a series of frames that I send to a stdout PIPE with a low latency . I first tried this command. "ffmpeg -hwaccel cuda -flags +low_delay -fflags +nobuffer -nostats -debug_ts -re -rtsp_flags prefer_tcp -rtsp_transport tcp -i RTSP_STREAM -f rawvideo -fps_mode passthrough -pix_fmt rgb24 pipe:1 > /dev/null" Instead of dev/null I plug my app. Interestingly, most of the time is spent in the encode part. latency(total:136.931ms, decode 3.355ms/2%, decode-filter: 8.426ms/6%, filter 3.191ms/2%, encode 120.774ms/88%) latency(total:73.519ms, decode 1.592ms/2%, decode-filter: 2.047ms/2%, filter 2.928ms/3%, encode 66.856ms/90%) latency(total:139.766ms, decode 1.885ms/1%, filter 1.898ms/1%, encode 135.023ms/96%) latency(total:71.03ms, decode 3.524ms/4%, decode-filter: 1.503ms/2%, filter 1.189ms/1%, encode 64.743ms/91%) latency(total:134.037ms, decode 1.935ms/1%, encode 130.176ms/97%) < If I compare this to multiple BMP files "ffmpeg -hwaccel cuda -flags +low_delay -fflags +nobuffer -debug_ts -nostats -re -rtsp_flags prefer_tcp -rtsp_transport tcp -i RTSP_STREAM -fps_mode passthrough -pix_fmt rgb24 output_image_%03d.bmp" latency(total:18.478ms, decode 3.222ms/17%, decode-filter: 9.715ms/52%, filter 2.576ms/13%, encode 2.771ms/14%) latency(total:13.019ms, decode 1.857ms/14%, decode-filter: 2.184ms/16%, filter 3.348ms/25%, encode 5.468ms/42%) latency(total:6.565ms, decode 1.642ms/25%, decode-filter: 1.1ms/16%, filter 0.63ms/9%, encode 3.105ms/47%) latency(total:6.628ms, decode 2.116ms/31%, decode-filter: 1.131ms/17%, filter 1.851ms/27%, filter-encode: 0.085ms/1%, encode 1.324ms/19%, encode-mux: 0.075ms/1%) latency(total:3.932ms, decode 1.642ms/41%, decode-filter: 0.588ms/14%, filter 0.863ms/21%, encode 0.779ms/19%) latency(total:4.528ms, decode 1.91ms/42%, decode-filter: 0.766ms/16%, filter 1.061ms/23%, encode 0.694ms/15%) Then the encoded time is much more acceptable. Do you know why such a difference exists ? Thanks for your help, Clement ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2 09/16] avfilter/vf_scale: add colorspace and transfer property options
From: Niklas Haas
In the long run, it would be ideal if we could add these to the avfilter
negotiation as well, but for now, this is a good start.
---
doc/filters.texi | 56 +++
libavfilter/vf_scale.c | 75 +-
2 files changed, 130 insertions(+), 1 deletion(-)
diff --git a/doc/filters.texi b/doc/filters.texi
index 9068a8a4e7..eb7463db84 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -21189,6 +21189,62 @@ is used by default. Possible values:
@item bottom
@end table
+@item in_primaries
+@item out_primaries
+Set in/output RGB primaries.
+
+This allows the autodetected value to be overridden as well as allows forcing
+a specific value used for the output and encoder. Possible values:
+
+@table @samp
+@item auto/unknown
+Choose automatically.
+
+@item bt709
+@item bt470m
+@item bt470bg
+@item smpte170m
+@item smpte240m
+@item film
+@item bt2020
+@item smpte428
+@item smpte431
+@item smpte432
+@item jedec-p22
+@item ebu3213
+@end table
+
+@item in_transfer
+@item out_transfer
+Set in/output transfer response curve (TRC).
+
+This allows the autodetected value to be overridden as well as allows forcing
+a specific value used for the output and encoder. Possible values:
+
+@table @samp
+@item auto/unknown
+Choose automatically.
+
+@item bt709
+@item bt470m
+@item gamma22
+@item bt470bg
+@item gamma28
+@item smpte170m
+@item smpte240m
+@item linear
+@item iec61966-2-1
+@item srgb
+@item iec61966-2-4
+@item xvycc
+@item bt1361e
+@item bt2020-10
+@item bt2020-12
+@item smpte2084
+@item smpte428
+@item arib-std-b67
+@end table
+
@item force_original_aspect_ratio
Enable decreasing or increasing output video width or height if necessary to
keep the original aspect ratio. Possible values:
diff --git a/libavfilter/vf_scale.c b/libavfilter/vf_scale.c
index d86e50f79b..e33617468a 100644
--- a/libavfilter/vf_scale.c
+++ b/libavfilter/vf_scale.c
@@ -160,7 +160,10 @@ typedef struct ScaleContext {
int in_color_matrix;
int out_color_matrix;
-
+int in_primaries;
+int out_primaries;
+int in_transfer;
+int out_transfer;
int in_range;
int out_range;
@@ -387,6 +390,30 @@ static av_cold int init(AVFilterContext *ctx)
if (ret < 0)
return ret;
+if (scale->in_primaries != -1 && !sws_test_primaries(scale->in_primaries,
0)) {
+av_log(ctx, AV_LOG_ERROR, "Unsupported input primaries '%s'\n",
+ av_color_primaries_name(scale->in_primaries));
+return AVERROR(EINVAL);
+}
+
+if (scale->out_primaries != -1 &&
!sws_test_primaries(scale->out_primaries, 1)) {
+av_log(ctx, AV_LOG_ERROR, "Unsupported output primaries '%s'\n",
+ av_color_primaries_name(scale->out_primaries));
+return AVERROR(EINVAL);
+}
+
+if (scale->in_transfer != -1 && !sws_test_transfer(scale->in_transfer, 0))
{
+av_log(ctx, AV_LOG_ERROR, "Unsupported input transfer '%s'\n",
+ av_color_transfer_name(scale->in_transfer));
+return AVERROR(EINVAL);
+}
+
+if (scale->out_transfer != -1 && !sws_test_transfer(scale->out_transfer,
1)) {
+av_log(ctx, AV_LOG_ERROR, "Unsupported output transfer '%s'\n",
+ av_color_transfer_name(scale->out_transfer));
+return AVERROR(EINVAL);
+}
+
if (scale->in_color_matrix != -1 &&
!sws_test_colorspace(scale->in_color_matrix, 0)) {
av_log(ctx, AV_LOG_ERROR, "Unsupported input color matrix '%s'\n",
av_color_space_name(scale->in_color_matrix));
@@ -815,6 +842,10 @@ scale:
if (scale->in_color_matrix != -1)
in->colorspace = scale->in_color_matrix;
+if (scale->in_primaries != -1)
+in->color_primaries = scale->in_primaries;
+if (scale->in_transfer != -1)
+in->color_trc = scale->in_transfer;
if (scale->in_range != AVCOL_RANGE_UNSPECIFIED)
in->color_range = scale->in_range;
in->chroma_location = scale->in_chroma_loc;
@@ -832,6 +863,10 @@ scale:
out->colorspace = outlink->colorspace;
if (scale->out_chroma_loc != AVCHROMA_LOC_UNSPECIFIED)
out->chroma_location = scale->out_chroma_loc;
+if (scale->out_primaries != AVCOL_PRI_UNSPECIFIED)
+out->color_primaries = scale->out_primaries;
+if (scale->out_transfer != AVCOL_TRC_UNSPECIFIED)
+out->color_trc = scale->out_transfer;
av_reduce(&out->sample_aspect_ratio.num, &out->sample_aspect_ratio.den,
(int64_t)in->sample_aspect_ratio.num * outlink->h * link->w,
@@ -1084,6 +1119,44 @@ static const AVOption scale_options[] = {
{"top", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCHROMA_LOC_TOP},
0, 0, FLAGS, .unit = "chroma_loc"},
{"bottomleft",NULL, 0, AV_OPT_TYPE_CONST,
{.i64=AVCHROMA_LOC_BOTTOMLEFT}, 0, 0, FLAGS, .unit = "chroma_loc"},
{"bottom",NULL, 0, AV_OPT_TYPE_CONST,
{.i64=AVCHROMA_LOC_BOTTOM}, 0, 0, FLAGS, .unit = "chro
[FFmpeg-devel] [PATCH v2 08/16] swscale: add ICC intent enum and option
From: Niklas Haas
This setting can be used to infuence the type of tone and gamut mapping used
internally when color space conversions are required. As discussed at VDD'24,
the default was set to relative colorimetric clipping, which is approximately
associative, surjective and idempotent. As such, it roundtrips well, although
it is strictly speaking not associative on out-of-gamut colors.
---
doc/APIchanges| 3 +++
doc/filters.texi | 31 +++
libswscale/graph.c| 11 ++-
libswscale/options.c | 6 ++
libswscale/swscale.h | 13 +
libswscale/swscale_internal.h | 2 +-
libswscale/version.h | 2 +-
libswscale/x86/output.asm | 2 +-
8 files changed, 62 insertions(+), 8 deletions(-)
diff --git a/doc/APIchanges b/doc/APIchanges
index b9af3de933..2f1101fc92 100644
--- a/doc/APIchanges
+++ b/doc/APIchanges
@@ -2,6 +2,9 @@ The last version increases of all libraries were on 2024-03-07
API changes, most recent first:
+2024-12-xx - xx - lsws 8.13.100 - swscale.h
+ Add enum SwsIntent and SwsContext.intent.
+
2024-12-05 - xx - lavu 59.49.100 - csp.h
Add av_csp_itu_eotf() and av_csp_itu_eotf_inv().
diff --git a/doc/filters.texi b/doc/filters.texi
index 900baf2f45..9068a8a4e7 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -21071,7 +21071,38 @@ Set libswscale input parameters for scaling algorithms
that need them. See
complete documentation. If not explicitly specified the filter applies
empty parameters.
+@item intent
+Set the ICC rendering intent to use when transforming between different color
+spaces. It accepts the following values:
+@table @samp
+@item perceptual
+Use a perceptually guided tone and gamut mapping curve. The exact details of
+the mapping used may change at any time and should not be relied on as stable.
+This intent is recommended for final viewing of image/video content in typical
+viewing settings.
+
+@item relative_colorimetric
+Statically clip out-of-gamut colors using a colorimetric clipping curve which
+attempts to find the colorimetrically least dissimilar in-gamut color. This
+intent performs white point adaptation and black point adaptation. This is
+the default. This intent is recommended wherever faithful color reproduction
+is of the utmost importance, even at the cost of clipping.
+
+@item absolute_colorimetric
+Hard clip out-of-gamut colors with no attempt at white or black point
+reproduction. This intent will reproduce in-gamut colors 1:1 on the output
+display as they would appear on the reference display, assuming the output
+display is appropriately calibrated.
+
+@item saturation
+Performs saturation mapping - that is, stretches the input color volume
+directly onto the output color volume, in non-linear fashion that preserves the
+original signal appearance as much as possible. This intent is recommended for
+signal content evaluation, as it will not lead to any clipping. It is roughly
+analogous to not performing any color mapping, although it still takes into
+account the mastering display primaries and any differences in encoding TRC.
+@end table
@item size, s
Set the video size. For the syntax of this option, check the
diff --git a/libswscale/graph.c b/libswscale/graph.c
index fbad1fe8c3..bf2f54b979 100644
--- a/libswscale/graph.c
+++ b/libswscale/graph.c
@@ -570,15 +570,16 @@ void sws_graph_free(SwsGraph **pgraph)
/* Tests only options relevant to SwsGraph */
static int opts_equal(const SwsContext *c1, const SwsContext *c2)
{
-return c1->flags == c2->flags &&
- c1->threads == c2->threads &&
- c1->dither == c2->dither &&
- c1->alpha_blend == c2->alpha_blend &&
- c1->gamma_flag == c2->gamma_flag &&
+return c1->flags == c2->flags &&
+ c1->threads == c2->threads &&
+ c1->dither== c2->dither&&
+ c1->alpha_blend == c2->alpha_blend &&
+ c1->gamma_flag== c2->gamma_flag&&
c1->src_h_chr_pos == c2->src_h_chr_pos &&
c1->src_v_chr_pos == c2->src_v_chr_pos &&
c1->dst_h_chr_pos == c2->dst_h_chr_pos &&
c1->dst_v_chr_pos == c2->dst_v_chr_pos &&
+ c1->intent== c2->intent&&
!memcmp(c1->scaler_params, c2->scaler_params,
sizeof(c1->scaler_params));
}
diff --git a/libswscale/options.c b/libswscale/options.c
index 5eef26de06..08d369e620 100644
--- a/libswscale/options.c
+++ b/libswscale/options.c
@@ -84,6 +84,12 @@ static const AVOption swscale_options[] = {
{ "threads", "number of threads", OFFSET(threads),
AV_OPT_TYPE_INT, {.i64 = 1 }, .flags = VE, .unit = "threads", .max = INT_MAX
},
{ "auto","automatic selection", 0,
AV_OPT_TYPE_CONST, {.i64 = 0 }, .flags = VE, .unit = "threads" },
[FFmpeg-devel] [PATCH v2 06/16] swscale/utils: add helper function to infer colorspace metadata
From: Niklas Haas
Logic is loosely on equivalent decisions in libplacebo. The basic idea is to try
and be a bit conservative by treating AVCOL_*_UNSPECIFIED as a no-op, unless the
other primaries set are non-standard / wide-gamut or HDR. This helps avoid
unintended or unexpected colorspace conversions, while forcing it in cases where
we are almost certain it is needed. The major departure from libplacebo
semantics
is that we no default to a 1000:1 contrast ration for SDR displays, instead
modelling
them as idealized devices with an infinite contrast ratio.
In either case, setting SWS_STRICT overrides this behavior in favor of always
requiring explicit colorspace metadata.
---
libswscale/utils.c | 66 ++
libswscale/utils.h | 3 +++
2 files changed, 69 insertions(+)
diff --git a/libswscale/utils.c b/libswscale/utils.c
index b097ec51b2..adbfda10a5 100644
--- a/libswscale/utils.c
+++ b/libswscale/utils.c
@@ -2802,6 +2802,72 @@ skip_hdr10:
return fmt;
}
+static int infer_prim_ref(SwsColor *csp, const SwsColor *ref)
+{
+if (csp->prim != AVCOL_PRI_UNSPECIFIED)
+return 0;
+
+/* Re-use the reference gamut only for "safe", similar primaries */
+switch (ref->prim) {
+case AVCOL_PRI_BT709:
+case AVCOL_PRI_BT470M:
+case AVCOL_PRI_BT470BG:
+case AVCOL_PRI_SMPTE170M:
+case AVCOL_PRI_SMPTE240M:
+csp->prim = ref->prim;
+csp->gamut = ref->gamut;
+break;
+default:
+csp->prim = AVCOL_PRI_BT709;
+csp->gamut = av_csp_primaries_desc_from_id(csp->prim)->prim;
+break;
+}
+
+return 1;
+}
+
+static int infer_trc_ref(SwsColor *csp, const SwsColor *ref)
+{
+if (csp->trc != AVCOL_TRC_UNSPECIFIED)
+return 0;
+
+/* Pick a suitable SDR transfer function, to try and minimize conversions
*/
+switch (ref->trc) {
+case AVCOL_TRC_UNSPECIFIED:
+/* HDR curves, never default to these */
+case AVCOL_TRC_SMPTE2084:
+case AVCOL_TRC_ARIB_STD_B67:
+csp->trc = AVCOL_TRC_BT709;
+csp->min_luma = av_make_q(0, 1);
+csp->max_luma = av_make_q(203, 1);
+break;
+default:
+csp->trc = ref->trc;
+csp->min_luma = ref->min_luma;
+csp->max_luma = ref->max_luma;
+break;
+}
+
+return 1;
+}
+
+int ff_infer_colors(SwsColor *src, SwsColor *dst)
+{
+int incomplete = 0;
+
+incomplete |= infer_prim_ref(dst, src);
+incomplete |= infer_prim_ref(src, dst);
+av_assert0(src->prim != AVCOL_PRI_UNSPECIFIED);
+av_assert0(dst->prim != AVCOL_PRI_UNSPECIFIED);
+
+incomplete |= infer_trc_ref(dst, src);
+incomplete |= infer_trc_ref(src, dst);
+av_assert0(src->trc != AVCOL_TRC_UNSPECIFIED);
+av_assert0(dst->trc != AVCOL_TRC_UNSPECIFIED);
+
+return incomplete;
+}
+
int sws_test_format(enum AVPixelFormat format, int output)
{
return output ? sws_isSupportedOutput(format) :
sws_isSupportedInput(format);
diff --git a/libswscale/utils.h b/libswscale/utils.h
index 1263e3f8ed..69ae7da39f 100644
--- a/libswscale/utils.h
+++ b/libswscale/utils.h
@@ -112,4 +112,7 @@ static inline int ff_fmt_align(enum AVPixelFormat fmt)
int ff_test_fmt(const SwsFormat *fmt, int output);
+/* Returns 1 if the formats are incomplete, 0 otherwise */
+int ff_infer_colors(SwsColor *src, SwsColor *dst);
+
#endif /* SWSCALE_UTILS_H */
--
2.47.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2 15/16] swscale/graph: allow dynamically updating HDR metadata
From: Niklas Haas
Without triggering a full graph reinit.
---
libswscale/graph.c | 21 -
libswscale/graph.h | 12 +---
libswscale/utils.h | 6 ++
3 files changed, 35 insertions(+), 4 deletions(-)
diff --git a/libswscale/graph.c b/libswscale/graph.c
index 7160458a5f..f390e53d24 100644
--- a/libswscale/graph.c
+++ b/libswscale/graph.c
@@ -475,6 +475,14 @@ static void free_lut3d(void *priv)
sws_lut3d_free(&lut);
}
+static void setup_lut3d(const SwsImg *out, const SwsImg *in, const SwsPass
*pass)
+{
+SwsLut3D *lut = pass->priv;
+
+/* Update dynamic frame metadata from the original source frame */
+sws_lut3d_update(lut, &pass->graph->src.color);
+}
+
static void run_lut3d(const SwsImg *out_base, const SwsImg *in_base,
int y, int h, const SwsPass *pass)
{
@@ -543,6 +551,7 @@ static int adapt_colors(SwsGraph *graph, SwsFormat src,
SwsFormat dst,
sws_lut3d_free(&lut);
return AVERROR(ENOMEM);
}
+pass->setup = setup_lut3d;
pass->free = free_lut3d;
*output = pass;
@@ -678,16 +687,26 @@ static int opts_equal(const SwsContext *c1, const
SwsContext *c2)
int sws_graph_reinit(SwsContext *ctx, const SwsFormat *dst, const SwsFormat
*src,
int field, SwsGraph **out_graph)
{
-const SwsGraph *graph = *out_graph;
+SwsGraph *graph = *out_graph;
if (graph && ff_fmt_equal(&graph->src, src) &&
ff_fmt_equal(&graph->dst, dst) &&
opts_equal(ctx, &graph->opts_copy))
+{
+sws_graph_update_metadata(graph, &src->color);
return 0;
+}
sws_graph_free(out_graph);
return sws_graph_create(ctx, dst, src, field, out_graph);
}
+void sws_graph_update_metadata(SwsGraph *graph, const SwsColor *color)
+{
+if (!color)
+return;
+
+ff_color_update_dynamic(&graph->src.color, color);
+}
void sws_graph_run(SwsGraph *graph, uint8_t *const out_data[4],
const int out_linesize[4],
diff --git a/libswscale/graph.h b/libswscale/graph.h
index 989695043d..758060ca12 100644
--- a/libswscale/graph.h
+++ b/libswscale/graph.h
@@ -134,9 +134,15 @@ int sws_graph_create(SwsContext *ctx, const SwsFormat
*dst, const SwsFormat *src
void sws_graph_free(SwsGraph **graph);
/**
- * Wrapper around sws_graph_create that does nothing if the format is
- * unchanged. Must be called after changing any of the fields in `ctx`, or else
- * they will have no effect.
+ * Update dynamic per-frame HDR metadata without requiring a full reinit.
+ */
+void sws_graph_update_metadata(SwsGraph *graph, const SwsColor *color);
+
+/**
+ * Wrapper around sws_graph_create() that reuses the existing graph if the
+ * format is compatible. This will also update dynamic per-frame metadata.
+ * Must be called after changing any of the fields in `ctx`, or else they will
+ * have no effect.
*/
int sws_graph_reinit(SwsContext *ctx, const SwsFormat *dst, const SwsFormat
*src,
int field, SwsGraph **graph);
diff --git a/libswscale/utils.h b/libswscale/utils.h
index 69ae7da39f..1a45e61307 100644
--- a/libswscale/utils.h
+++ b/libswscale/utils.h
@@ -60,6 +60,12 @@ typedef struct SwsColor {
AVRational frame_avg;/* per-frame/scene average luminance, or 0 */
} SwsColor;
+static inline void ff_color_update_dynamic(SwsColor *dst, const SwsColor *src)
+{
+dst->frame_peak = src->frame_peak;
+dst->frame_avg = src->frame_avg;
+}
+
/* Subset of AVFrame parameters that uniquely determine pixel representation */
typedef struct SwsFormat {
int width, height;
--
2.47.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2 14/16] swscale/graph: add color mapping pass
From: Niklas Haas
This leverages the previously introduced color management subsystem in order
to adapt between transfer functions and color spaces, as well as for HDR tone
mapping.
Take special care to handle grayscale formats without a colorspace
gracefully.
---
libswscale/graph.c | 95 +-
1 file changed, 93 insertions(+), 2 deletions(-)
diff --git a/libswscale/graph.c b/libswscale/graph.c
index bf2f54b979..7160458a5f 100644
--- a/libswscale/graph.c
+++ b/libswscale/graph.c
@@ -30,6 +30,8 @@
#include "libswscale/swscale.h"
#include "libswscale/utils.h"
+#include "cms.h"
+#include "lut3d.h"
#include "swscale_internal.h"
#include "graph.h"
@@ -463,6 +465,89 @@ static int add_legacy_sws_pass(SwsGraph *graph, SwsFormat
src, SwsFormat dst,
return 0;
}
+/**
+ * Gamut and tone mapping *
+ **/
+
+static void free_lut3d(void *priv)
+{
+SwsLut3D *lut = priv;
+sws_lut3d_free(&lut);
+}
+
+static void run_lut3d(const SwsImg *out_base, const SwsImg *in_base,
+ int y, int h, const SwsPass *pass)
+{
+SwsLut3D *lut = pass->priv;
+const SwsImg in = shift_img(in_base, y);
+const SwsImg out = shift_img(out_base, y);
+
+sws_lut3d_apply(lut, in.data[0], in.linesize[0], out.data[0],
+out.linesize[0], pass->width, h);
+}
+
+static int adapt_colors(SwsGraph *graph, SwsFormat src, SwsFormat dst,
+SwsPass *input, SwsPass **output)
+{
+enum AVPixelFormat fmt_in, fmt_out;
+SwsColorMap map = {0};
+SwsLut3D *lut;
+SwsPass *pass;
+int ret;
+
+/**
+ * Grayspace does not really have primaries, so just force the use of
+ * the equivalent other primary set to avoid a conversion. Technically,
+ * this does affect the weights used for the Grayscale conversion, but
+ * in practise, that should give the expected results more often than not.
+ */
+if (isGray(dst.format)) {
+dst.color = src.color;
+} else if (isGray(src.format)) {
+src.color = dst.color;
+}
+
+/* Fully infer color spaces before color mapping logic */
+graph->incomplete |= ff_infer_colors(&src.color, &dst.color);
+
+map.intent = graph->ctx->intent;
+map.src= src.color;
+map.dst= dst.color;
+
+if (sws_color_map_noop(&map))
+return 0;
+
+lut = sws_lut3d_alloc();
+if (!lut)
+return AVERROR(ENOMEM);
+
+fmt_in = sws_lut3d_pick_pixfmt(src, 0);
+fmt_out = sws_lut3d_pick_pixfmt(dst, 1);
+if (fmt_in != src.format) {
+SwsFormat tmp = src;
+tmp.format = fmt_in;
+ret = add_legacy_sws_pass(graph, src, tmp, input, &input);
+if (ret < 0)
+return ret;
+}
+
+ret = sws_lut3d_generate(lut, fmt_in, fmt_out, &map);
+if (ret < 0) {
+sws_lut3d_free(&lut);
+return ret;
+}
+
+pass = pass_add(graph, lut, fmt_out, src.width, src.height,
+input, 1, run_lut3d);
+if (!pass) {
+sws_lut3d_free(&lut);
+return AVERROR(ENOMEM);
+}
+pass->free = free_lut3d;
+
+*output = pass;
+return 0;
+}
/***
* Main filter graph construction code *
@@ -470,11 +555,17 @@ static int add_legacy_sws_pass(SwsGraph *graph, SwsFormat
src, SwsFormat dst,
static int init_passes(SwsGraph *graph)
{
-const SwsFormat src = graph->src;
-const SwsFormat dst = graph->dst;
+SwsFormat src = graph->src;
+SwsFormat dst = graph->dst;
SwsPass *pass = NULL; /* read from main input image */
int ret;
+ret = adapt_colors(graph, src, dst, pass, &pass);
+if (ret < 0)
+return ret;
+src.format = pass ? pass->format : src.format;
+src.color = dst.color;
+
if (!ff_fmt_equal(&src, &dst)) {
ret = add_legacy_sws_pass(graph, src, dst, pass, &pass);
if (ret < 0)
--
2.47.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2 10/16] avfilter/vf_scale: strip metadata when changing colorspace
From: Niklas Haas
This is no longer relevant after a change in color space.
---
libavfilter/vf_scale.c | 12
1 file changed, 12 insertions(+)
diff --git a/libavfilter/vf_scale.c b/libavfilter/vf_scale.c
index e33617468a..a56d452c6c 100644
--- a/libavfilter/vf_scale.c
+++ b/libavfilter/vf_scale.c
@@ -868,6 +868,18 @@ scale:
if (scale->out_transfer != AVCOL_TRC_UNSPECIFIED)
out->color_trc = scale->out_transfer;
+if (in->color_primaries != out->color_primaries ||
+in->color_trc != out->color_trc)
+{
+av_frame_remove_side_data(out,
AV_FRAME_DATA_MASTERING_DISPLAY_METADATA);
+av_frame_remove_side_data(out, AV_FRAME_DATA_DYNAMIC_HDR_PLUS);
+av_frame_remove_side_data(out, AV_FRAME_DATA_DYNAMIC_HDR_VIVID);
+av_frame_remove_side_data(out, AV_FRAME_DATA_CONTENT_LIGHT_LEVEL);
+av_frame_remove_side_data(out, AV_FRAME_DATA_ICC_PROFILE);
+av_frame_remove_side_data(out, AV_FRAME_DATA_DOVI_METADATA);
+av_frame_remove_side_data(out, AV_FRAME_DATA_DOVI_RPU_BUFFER);
+}
+
av_reduce(&out->sample_aspect_ratio.num, &out->sample_aspect_ratio.den,
(int64_t)in->sample_aspect_ratio.num * outlink->h * link->w,
(int64_t)in->sample_aspect_ratio.den * outlink->w * link->h,
--
2.47.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2 07/16] swscale/utils: fix XYZ primaries tagging
From: Niklas Haas
Swscale currently handles XYZ by embedding a forced conversion to
BT.709 RGB with a hardcoded matrix. This is not ideal, but to preserve the
status quo and avoid any unexpected changes in behavior, this patch merely
fixes the inferred primaries tag to match the reality.
In the future, I would like to handle XYZ properly, via direct conversion
to the target colorspace (or possibly simply by using a more fitting
RGB intermediate like SMPTE428), but for now just keep the status quo.
---
libswscale/utils.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/libswscale/utils.c b/libswscale/utils.c
index adbfda10a5..01c21c74d8 100644
--- a/libswscale/utils.c
+++ b/libswscale/utils.c
@@ -2684,7 +2684,7 @@ SwsFormat ff_fmt_from_frame(const AVFrame *frame, int
field)
} else if (desc->flags & AV_PIX_FMT_FLAG_XYZ) {
fmt.csp = AVCOL_SPC_UNSPECIFIED;
fmt.color = (SwsColor) {
-.prim = AVCOL_PRI_SMPTE428,
+.prim = AVCOL_PRI_BT709, /* swscale currently hard-codes this XYZ
matrix */
.trc = AVCOL_TRC_SMPTE428,
};
} else if (desc->nb_components < 3) {
--
2.47.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] Decoding performance -f rawvideo pipe:1 vs BMP images output
Hi,
On Fri, 6 Dec 2024 at 18:55, Clément Péron wrote:
>
> Hi,
>
> I am trying to convert a RTSP stream to a series of frames that I send
> to a stdout PIPE with a low latency .
>
> I first tried this command.
>
> "ffmpeg -hwaccel cuda -flags +low_delay -fflags +nobuffer -nostats
> -debug_ts -re -rtsp_flags prefer_tcp -rtsp_transport tcp -i
> RTSP_STREAM -f rawvideo -fps_mode passthrough -pix_fmt rgb24 pipe:1 >
> /dev/null"
> Instead of dev/null I plug my app.
>
> Interestingly, most of the time is spent in the encode part.
>
> latency(total:136.931ms, decode 3.355ms/2%, decode-filter: 8.426ms/6%,
> filter 3.191ms/2%, encode 120.774ms/88%)
> latency(total:73.519ms, decode 1.592ms/2%, decode-filter: 2.047ms/2%,
> filter 2.928ms/3%, encode 66.856ms/90%)
> latency(total:139.766ms, decode 1.885ms/1%, filter 1.898ms/1%, encode
> 135.023ms/96%)
> latency(total:71.03ms, decode 3.524ms/4%, decode-filter: 1.503ms/2%,
> filter 1.189ms/1%, encode 64.743ms/91%)
> latency(total:134.037ms, decode 1.935ms/1%, encode 130.176ms/97%)
> <
>
> If I compare this to multiple BMP files
> "ffmpeg -hwaccel cuda -flags +low_delay -fflags +nobuffer -debug_ts
> -nostats -re -rtsp_flags prefer_tcp -rtsp_transport tcp -i RTSP_STREAM
> -fps_mode passthrough -pix_fmt rgb24 output_image_%03d.bmp"
>
>
> latency(total:18.478ms, decode 3.222ms/17%, decode-filter:
> 9.715ms/52%, filter 2.576ms/13%, encode 2.771ms/14%)
> latency(total:13.019ms, decode 1.857ms/14%, decode-filter:
> 2.184ms/16%, filter 3.348ms/25%, encode 5.468ms/42%)
> latency(total:6.565ms, decode 1.642ms/25%, decode-filter: 1.1ms/16%,
> filter 0.63ms/9%, encode 3.105ms/47%)
> latency(total:6.628ms, decode 2.116ms/31%, decode-filter: 1.131ms/17%,
> filter 1.851ms/27%, filter-encode: 0.085ms/1%, encode 1.324ms/19%,
> encode-mux: 0.075ms/1%)
> latency(total:3.932ms, decode 1.642ms/41%, decode-filter: 0.588ms/14%,
> filter 0.863ms/21%, encode 0.779ms/19%)
> latency(total:4.528ms, decode 1.91ms/42%, decode-filter: 0.766ms/16%,
> filter 1.061ms/23%, encode 0.694ms/15%)
>
>
> Then the encoded time is much more acceptable.
> Do you know why such a difference exists ?
After investigating it seems that the AV_CODEC_CAP_FRAME_THREADS
impact the processing time a lot!
+++ b/libavcodec/rawenc.c
@@ -98,7 +98,7 @@ const FFCodec ff_rawvideo_encoder = {
.p.id = AV_CODEC_ID_RAWVIDEO,
-.p.capabilities = AV_CODEC_CAP_DR1 | AV_CODEC_CAP_FRAME_THREADS |
+.p.capabilities = AV_CODEC_CAP_DR1 |
AV_CODEC_CAP_ENCODER_REORDERED_OPAQUE,
.init = raw_encode_init,
>>>
latency(total:7.312ms, decode 2.347ms/32%, decode-filter: 0.901ms/12%,
filter 1.293ms/17%, encode 2.71ms/37%)
latency(total:7.982ms, decode 3.369ms/42%, decode-filter: 0.617ms/7%,
filter 2.406ms/30%, encode 1.461ms/18%)
latency(total:4.63ms, decode 2.401ms/51%, decode-filter: 0.733ms/15%,
filter 0.912ms/19%, encode 0.524ms/11%)
latency(total:4.291ms, decode 2.154ms/50%, decode-filter: 0.568ms/13%,
filter 1.041ms/24%, encode 0.475ms/11%)
latency(total:5.103ms, decode 2.37ms/46%, decode-filter: 0.75ms/14%,
filter 1.238ms/24%, encode 0.652ms/12%)
latency(total:5.658ms, demux-decode: 0.112ms/1%, decode 2.283ms/40%,
decode-filter: 1.28ms/22%, filter 1.25ms/22%, filter-encode:
0.077ms/1%, encode 0.634ms/11%)
I tried to redo without patch and adding "-threads 1" to my command
but the performance is still a bit lower.
latency(total:15.234ms, decode 2.591ms/17%, decode-filter: 0.807ms/5%,
filter 5.803ms/38%, encode 5.893ms/38%)
latency(total:36.815ms, decode 5.125ms/13%, decode-filter: 1.445ms/3%,
filter 27.749ms/75%, encode 2.193ms/5%)
latency(total:13.314ms, decode 2.449ms/18%, decode-filter: 0.647ms/4%,
filter 8.811ms/66%, encode 1.319ms/9%)
latency(total:12.199ms, decode 2.211ms/18%, decode-filter: 0.561ms/4%,
filter 7.701ms/63%, encode 1.64ms/13%)
latency(total:11.915ms, decode 2.35ms/19%, decode-filter: 0.59ms/4%,
filter 8.014ms/67%, encode 0.881ms/7%)
latency(total:21.685ms, decode 2.493ms/11%, decode-filter: 1.024ms/4%,
filter 16.372ms/75%, encode 1.62ms/7%)
latency(total:12.756ms, decode 2.366ms/18%, decode-filter: 0.63ms/4%,
filter 8.206ms/64%, encode 1.457ms/11%)
latency(total:10.902ms, decode 2.163ms/19%, decode-filter: 0.865ms/7%,
filter 6.629ms/60%, encode 1.169ms/10%)
latency(total:24.758ms, decode 3.356ms/13%, decode-filter: 0.962ms/3%,
filter 19.548ms/78%, encode 0.788ms/3%)
latency(total:10.619ms, decode 2.122ms/19%, decode-filter: 0.652ms/6%,
filter 7.097ms/66%, encode 0.685ms/6%)
Is this impact when threading is enabled expected?
Is there another cmd param to disable the threading?
Thanks
>
> Thanks for your help,
> Clement
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] (no subject)
Changes since v1: - Fix an edge case with XYZ, YUVJ and RGB0 handling - Fix compilation with asm enabled - Fix crash in fate test - Fix unnecessary reinitialization per frame in some cases - Fix vf_scale accidentally rejecting untagged files Note: For ease of testing, this series can also be found at: https://github.com/haasn/FFmpeg/tree/sws_cms ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2 13/16] swscale/lut3d: add 3DLUT dispatch system
From: Niklas Haas
This is a lightweight wrapper around the underlying color management system,
whose job it is merely to manage the 3DLUT state and apply them to the frame
data. This is where we might add platform-specific optimizations in the future.
I also plan on adding support for more pixel formats in the future. In
particular, we could support YUV or XYZ input formats directly using only
negligible additional code in the 3DLUT setup functions. This would eliminate
the major source of slowdown, which is currently the roundtrip to RGBA64.
---
libswscale/Makefile | 1 +
libswscale/lut3d.c | 290
libswscale/lut3d.h | 98 +++
3 files changed, 389 insertions(+)
create mode 100644 libswscale/lut3d.c
create mode 100644 libswscale/lut3d.h
diff --git a/libswscale/Makefile b/libswscale/Makefile
index c4e45d494e..267952d870 100644
--- a/libswscale/Makefile
+++ b/libswscale/Makefile
@@ -14,6 +14,7 @@ OBJS = alphablend.o \
graph.o \
half2float.o \
input.o \
+ lut3d.o \
options.o\
output.o \
rgb2rgb.o\
diff --git a/libswscale/lut3d.c b/libswscale/lut3d.c
new file mode 100644
index 00..e62820b244
--- /dev/null
+++ b/libswscale/lut3d.c
@@ -0,0 +1,290 @@
+/*
+ * Copyright (C) 2024 Niklas Haas
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include
+#include
+
+#include "libavutil/attributes.h"
+#include "libavutil/avassert.h"
+#include "libavutil/mem.h"
+
+#include "cms.h"
+#include "csputils.h"
+#include "lut3d.h"
+
+SwsLut3D *sws_lut3d_alloc(void)
+{
+SwsLut3D *lut3d = av_malloc(sizeof(*lut3d));
+if (!lut3d)
+return NULL;
+
+lut3d->dynamic = false;
+return lut3d;
+}
+
+void sws_lut3d_free(SwsLut3D **plut3d)
+{
+av_freep(plut3d);
+}
+
+bool sws_lut3d_test_fmt(enum AVPixelFormat fmt, int output)
+{
+return fmt == AV_PIX_FMT_RGBA64;
+}
+
+enum AVPixelFormat sws_lut3d_pick_pixfmt(SwsFormat fmt, int output)
+{
+return AV_PIX_FMT_RGBA64;
+}
+
+/**
+ * v0 and v1 are 'black' and 'white'
+ * v1 and v2 are closest RGB/CMY vertices
+ * x >= y >= z are relative weights
+ */
+static av_always_inline
+v3u16_t barycentric(int shift, int x, int y, int z,
+v3u16_t v0, v3u16_t v1, v3u16_t v2, v3u16_t v3)
+{
+const int a = (1 << shift) - x;
+const int b = x - y;
+const int c = y - z;
+const int d = z;
+av_assert2(x >= y);
+av_assert2(y >= z);
+
+return (v3u16_t) {
+(a * v0.x + b * v1.x + c * v2.x + d * v3.x) >> shift,
+(a * v0.y + b * v1.y + c * v2.y + d * v3.y) >> shift,
+(a * v0.z + b * v1.z + c * v2.z + d * v3.z) >> shift,
+};
+}
+
+static av_always_inline
+v3u16_t tetrahedral(const SwsLut3D *lut3d, int Rx, int Gx, int Bx,
+int Rf, int Gf, int Bf)
+{
+const int shift = 16 - INPUT_LUT_BITS;
+const int Rn = FFMIN(Rx + 1, INPUT_LUT_SIZE - 1);
+const int Gn = FFMIN(Gx + 1, INPUT_LUT_SIZE - 1);
+const int Bn = FFMIN(Bx + 1, INPUT_LUT_SIZE - 1);
+
+const v3u16_t c000 = lut3d->input[Bx][Gx][Rx];
+const v3u16_t c111 = lut3d->input[Bn][Gn][Rn];
+if (Rf > Gf) {
+if (Gf > Bf) {
+const v3u16_t c100 = lut3d->input[Bx][Gx][Rn];
+const v3u16_t c110 = lut3d->input[Bx][Gn][Rn];
+return barycentric(shift, Rf, Gf, Bf, c000, c100, c110, c111);
+} else if (Rf > Bf) {
+const v3u16_t c100 = lut3d->input[Bx][Gx][Rn];
+const v3u16_t c101 = lut3d->input[Bn][Gx][Rn];
+return barycentric(shift, Rf, Bf, Gf, c000, c100, c101, c111);
+} else {
+const v3u16_t c001 = lut3d->input[Bn][Gx][Rx];
+const v3u16_t c101 = lut3d->input[Bn][Gx][Rn];
+return barycentric(shift, Bf, Rf, Gf, c000, c001, c101, c111);
+}
+} else {
+if (Bf > Gf) {
+const v3u16_t c001 = lut3d->input[Bn][Gx][Rx];
+
[FFmpeg-devel] [PATCH v2 04/16] swscale/utils: read HDR mastering metadata from AVFrame
From: Niklas Haas
---
libswscale/utils.c | 18 ++
1 file changed, 18 insertions(+)
diff --git a/libswscale/utils.c b/libswscale/utils.c
index 182f92396a..3d3fcadacc 100644
--- a/libswscale/utils.c
+++ b/libswscale/utils.c
@@ -46,6 +46,7 @@
#include "libavutil/imgutils.h"
#include "libavutil/intreadwrite.h"
#include "libavutil/libm.h"
+#include "libavutil/mastering_display_metadata.h"
#include "libavutil/mathematics.h"
#include "libavutil/mem.h"
#include "libavutil/opt.h"
@@ -2655,6 +2656,7 @@ SwsFormat ff_fmt_from_frame(const AVFrame *frame, int
field)
{
const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(frame->format);
const AVColorPrimariesDesc *primaries;
+AVFrameSideData *sd;
SwsFormat fmt = {
.width = frame->width,
@@ -2727,6 +2729,22 @@ SwsFormat ff_fmt_from_frame(const AVFrame *frame, int
field)
if (primaries)
fmt.color.gamut = primaries->prim;
+if ((sd = av_frame_get_side_data(frame,
AV_FRAME_DATA_MASTERING_DISPLAY_METADATA))) {
+const AVMasteringDisplayMetadata *mdm = (const
AVMasteringDisplayMetadata *) sd->data;
+if (mdm->has_luminance) {
+fmt.color.min_luma = mdm->min_luminance;
+fmt.color.max_luma = mdm->max_luminance;
+}
+
+if (mdm->has_primaries) {
+/* Ignore mastering display white point as it has no bearance on
+ * the underlying content */
+fmt.color.gamut.r = (AVCIExy) { mdm->display_primaries[0][0],
mdm->display_primaries[0][1] };
+fmt.color.gamut.g = (AVCIExy) { mdm->display_primaries[1][0],
mdm->display_primaries[1][1] };
+fmt.color.gamut.b = (AVCIExy) { mdm->display_primaries[2][0],
mdm->display_primaries[2][1] };
+}
+}
+
/* PQ is always scaled down to absolute zero, so ignore mastering metadata
*/
if (fmt.color.trc == AVCOL_TRC_SMPTE2084)
fmt.color.min_luma = av_make_q(0, 1);
--
2.47.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] avformat: add AV1 RTP depacketizer and packetizer
Hi (also apologies if my client mangles the inline version of the patch, it's
the first time I've tried to review an attached patch with it)...
On Monday, December 2nd, 2024 at 9:57 AM, Chris Hodges
wrote:
> Hi,
>
> On 11/25/24 18:47, Tristan Matthews via ffmpeg-devel wrote:
>
> > One nit I'd add is that since the RTP AV1 spec is still in draft (according
> > to https://aomediacodec.github.io/av1-rtp-spec/) this feature should
> > probably be marked experimental as is done for VP9 in RTP, see:
> > https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/f8e91ab05ff3d111626ab8a3b5d570865a934f07:/libavformat/rtpenc.c#l221
> >
> > in which case CLI users will have to add `-strict experimental` to their
> > options.
>
>
> I've added this as suggested.
>
> > For the keyframe detection issue I'm not sure if this is something missing
> > in FFMPEG's RTP stack (e.g. I've noticed that both GStreamer and libwebrtc
> > signal that a buffer contains a keyframe at a higher level), but if not
> > could you set it if you're dealing with a FRAME OBU of type 0 (keyframe) or
> > 2 (intra-only)? You'd need to parse the OBU to extract that however.
>
>
> It turns out this was a lapsus on my side, when I used the flags field
> in the RTPMuxContext structure instead of the flags in AVPacket. After
> transferring the key frame information to the encoder function, I could
> remove the workaround.
>
> I also added the key frame info to the AVPacket during demuxing that was
> missing before.
>
> New patch attached.
>
> --
> Best regards, Chris___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
> From: Chris Hodges
> Date: Mon, 26 Aug 2024 11:29:29 +0200
> Subject: [PATCH] avformat: add AV1 RTP depacketizer and packetizer
> X-Unsent: 1
> To: ffmpeg-devel@ffmpeg.org
> Content-Type: text/plain
> MIME-Version: 1.0
>
> Add RTP packetizer and depacketizer according to (most)
> of the official AV1 RTP specification. This enables
> streaming via RTSP between ffmpeg and ffmpeg and has
> also been tested to work with AV1 RTSP streams via
> GStreamer.
>
> It also adds the required SDP attributes for AV1.
>
> AV1 RTP encoding is marked as experimental due to
> draft specification status (courtesy of Tristan).
>
> Change-Id: Ie7f984b97be54d86d06bc73fa97c6faa8ffabf89
> Signed-off-by: Chris Hodges
> ---
> libavformat/Makefile | 2 +
> libavformat/demux.c | 1 +
> libavformat/rtp_av1.h| 128 +++
> libavformat/rtpdec.c | 1 +
> libavformat/rtpdec_av1.c | 421 +++
> libavformat/rtpdec_formats.h | 1 +
> libavformat/rtpenc.c | 14 ++
> libavformat/rtpenc.h | 1 +
> libavformat/rtpenc_av1.c | 294
> libavformat/sdp.c| 30 +++
> 10 files changed, 893 insertions(+)
> create mode 100644 libavformat/rtp_av1.h
> create mode 100644 libavformat/rtpdec_av1.c
> create mode 100644 libavformat/rtpenc_av1.c
>
> diff --git a/libavformat/Makefile b/libavformat/Makefile
> index 7ca68a7036..1200668a2f 100644
> --- a/libavformat/Makefile
> +++ b/libavformat/Makefile
> @@ -47,6 +47,7 @@ OBJS-$(CONFIG_RTPDEC)+= rdt.o
> \
> rtpdec_ac3.o\
> rtpdec_amr.o\
> rtpdec_asf.o\
> +rtpdec_av1.o\
> rtpdec_dv.o \
> rtpdec_g726.o \
> rtpdec_h261.o \
> @@ -515,6 +516,7 @@ OBJS-$(CONFIG_RTP_MUXER) += rtp.o
> \
> rtpenc_aac.o \
> rtpenc_latm.o\
> rtpenc_amr.o \
> +rtpenc_av1.o \
> rtpenc_h261.o\
> rtpenc_h263.o\
> rtpenc_h263_rfc2190.o \
> diff --git a/libavformat/demux.c b/libavformat/demux.c
> index cba1f2e4df..8357a3bff1 100644
> --- a/libavformat/demux.c
> +++ b/libavformat/demux.c
> @@ -111,6 +111,7 @@ static int set_codec_from_probe_data(AVFormatContext *s,
> AVStream *st,
> { "aac",AV_CODEC_ID_AAC, AVMEDIA_TYPE_AUDIO},
> { "ac3",AV_CODEC_ID_AC3, AVMEDIA_TYPE_AUDIO},
> { "aptx", AV_CODEC_ID_APTX, AVMEDIA_TYPE_AUD
Re: [FFmpeg-devel] [FFmpeg-cvslog] avutil/csp: add av_csp_trc_inv_from_id()
Hi On Thu, Dec 05, 2024 at 11:28:54AM +, Niklas Haas wrote: > ffmpeg | branch: master | Niklas Haas | Mon Nov 25 13:35:34 > 2024 +0100| [bf0a6c484ffab9f157db7ee7c8a0f59df077] | committer: Niklas > Haas > > avutil/csp: add av_csp_trc_inv_from_id() > > Mathematical inverse of av_csp_trc_from_id(), plus testing to make sure it > roundtrips correctly with the corresponding TRC. > > > http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=bf0a6c484ffab9f157db7ee7c8a0f59df077 > --- > > doc/APIchanges| 3 + > libavutil/csp.c | 155 ++- > libavutil/csp.h | 5 + > libavutil/tests/color_utils.c | 13 +- > libavutil/version.h | 4 +- > tests/ref/fate/color_utils| 608 > +- > 6 files changed, 470 insertions(+), 318 deletions(-) Breaks x86-32 TESTcolor_utils --- src/tests/ref/fate/color_utils 2024-12-06 21:02:29.850814721 +0100 +++ tests/data/fate/color_utils 2024-12-06 21:02:42.958917771 +0100 @@ -263,42 +263,4 @@ trc=smpte2084 calling func(15123.456700) expected=1.042921 roundtrip=15123.456700 trc=smpte2084 calling func(19845.889230) expected=1.070677 roundtrip=19845.889230 trc=smpte2084 calling func(98678.423100) expected=1.225908 roundtrip=98678.423100 -trc=smpte2084 calling func(9.88) expected=1.227127 roundtrip=9.88 -trc=smpte428 calling func(-0.10) expected=0.00 roundtrip=0.00 -trc=smpte428 calling func(-0.018054) expected=0.00 roundtrip=0.00 -trc=smpte428 calling func(-0.01) expected=0.00 roundtrip=0.00 -trc=smpte428 calling func(-0.004490) expected=0.00 roundtrip=0.00 -trc=smpte428 calling func(0.00) expected=0.00 roundtrip=0.00 -trc=smpte428 calling func(0.003162) expected=0.105659 roundtrip=0.003162 -trc=smpte428 calling func(0.005000) expected=0.126018 roundtrip=0.005000 -trc=smpte428 calling func(0.009000) expected=0.157985 roundtrip=0.009000 -trc=smpte428 calling func(0.015000) expected=0.192284 roundtrip=0.015000 -trc=smpte428 calling func(0.10) expected=0.398869 roundtrip=0.10 -trc=smpte428 calling func(1.00) expected=0.967043 roundtrip=1.00 -trc=smpte428 calling func(52.37) expected=4.432321 roundtrip=52.37 -trc=smpte428 calling func(125.098765) expected=6.195572 roundtrip=125.098765 -trc=smpte428 calling func(1999.111230) expected=17.988639 roundtrip=1999.111230 -trc=smpte428 calling func(6945.443000) expected=29.041734 roundtrip=6945.443000 -trc=smpte428 calling func(15123.456700) expected=39.174525 roundtrip=15123.456700 -trc=smpte428 calling func(19845.889230) expected=43.490646 roundtrip=19845.889230 -trc=smpte428 calling func(98678.423100) expected=80.593559 roundtrip=98678.423100 -trc=smpte428 calling func(9.88) expected=81.006971 roundtrip=9.88 -trc=arib-std-b67 calling func(-0.10) expected=0.00 roundtrip=0.00 -trc=arib-std-b67 calling func(-0.018054) expected=0.00 roundtrip=0.00 -trc=arib-std-b67 calling func(-0.01) expected=0.00 roundtrip=0.00 -trc=arib-std-b67 calling func(-0.004490) expected=0.00 roundtrip=0.00 -trc=arib-std-b67 calling func(0.00) expected=0.00 roundtrip=0.00 -trc=arib-std-b67 calling func(0.003162) expected=0.097400 roundtrip=0.003162 -trc=arib-std-b67 calling func(0.005000) expected=0.122474 roundtrip=0.005000 -trc=arib-std-b67 calling func(0.009000) expected=0.164317 roundtrip=0.009000 -trc=arib-std-b67 calling func(0.015000) expected=0.212132 roundtrip=0.015000 -trc=arib-std-b67 calling func(0.10) expected=0.544089 roundtrip=0.10 -trc=arib-std-b67 calling func(1.00) expected=1.00 roundtrip=1.00 -trc=arib-std-b67 calling func(52.37) expected=1.712092 roundtrip=52.37 -trc=arib-std-b67 calling func(125.098765) expected=1.867862 roundtrip=125.098765 -trc=arib-std-b67 calling func(1999.111230) expected=2.363502 roundtrip=1999.111230 -trc=arib-std-b67 calling func(6945.443000) expected=2.586219 roundtrip=6945.443000 -trc=arib-std-b67 calling func(15123.456700) expected=2.725380 roundtrip=15123.456700 -trc=arib-std-b67 calling func(19845.889230) expected=2.773978 roundtrip=19845.889230 -trc=arib-std-b67 calling func(98678.423100) expected=3.060803 roundtrip=98678.423100 -trc=arib-std-b67 calling func(9.88) expected=3.063182 roundtrip=9.88 + FAIL Test color_utils failed. Look at tests/data/fate/color_utils.err for details. make: *** [src/tests/Makefile:311: fate-color_utils] Error 1 [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB It is what and why we do it that matters, not just one of them. signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscri
Re: [FFmpeg-devel] [FFmpeg-cvslog] avutil/tests/color_utils: add tests for av_csp_itu_eotf
Hi
On Thu, Dec 05, 2024 at 11:28:58AM +, Niklas Haas wrote:
> ffmpeg | branch: master | Niklas Haas | Mon Nov 25 14:10:46
> 2024 +0100| [7b73ea501dc738c52daf2a2fef0c41b4ce95928f] | committer: Niklas
> Haas
>
> avutil/tests/color_utils: add tests for av_csp_itu_eotf
>
> > http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=7b73ea501dc738c52daf2a2fef0c41b4ce95928f
> ---
>
> libavutil/tests/color_utils.c | 110 ++
> tests/ref/fate/color_utils| 2650
> +
> 2 files changed, 2760 insertions(+)
does not work on x86-32
--- src/tests/ref/fate/color_utils 2024-12-06 21:09:00.541865625 +0100
+++ tests/data/fate/color_utils 2024-12-06 21:10:00.406330276 +0100
@@ -263,2692 +263,4 @@
trc=smpte2084 calling func(15123.456700) expected=1.042921
roundtrip=15123.456700
trc=smpte2084 calling func(19845.889230) expected=1.070677
roundtrip=19845.889230
trc=smpte2084 calling func(98678.423100) expected=1.225908
roundtrip=98678.423100
-trc=smpte2084 calling func(9.88) expected=1.227127
roundtrip=9.88
-trc=smpte428 calling func(-0.10) expected=0.00 roundtrip=0.00
-trc=smpte428 calling func(-0.018054) expected=0.00 roundtrip=0.00
-trc=smpte428 calling func(-0.01) expected=0.00 roundtrip=0.00
-trc=smpte428 calling func(-0.004490) expected=0.00 roundtrip=0.00
-trc=smpte428 calling func(0.00) expected=0.00 roundtrip=0.00
...
-trc=arib-std-b67 EOTF(50, 1.5, {0.3}) = {4.33808}
-trc=arib-std-b67 EOTF(50, 1.5, {0.4}) = {5.84815}
-trc=arib-std-b67 EOTF(50, 1.5, {0.5}) = {8.08169}
-trc=arib-std-b67 EOTF(50, 1.5, {0.6}) = {11.3853}
-trc=arib-std-b67 EOTF(50, 1.5, {0.7}) = {16.2716}
-trc=arib-std-b67 EOTF(50, 1.5, {0.8}) = {23.4989}
-trc=arib-std-b67 EOTF(50, 1.5, {0.9}) = {34.1888}
-trc=arib-std-b67 EOTF(50, 1.5, {1}) = {50}
-trc=arib-std-b67 EOTF(100, 1.5, {0}) = {1.5}
-trc=arib-std-b67 EOTF(100, 1.5, {0.1}) = {2.82113}
-trc=arib-std-b67 EOTF(100, 1.5, {0.2}) = {4.55608}
-trc=arib-std-b67 EOTF(100, 1.5, {0.3}) = {6.70485}
-trc=arib-std-b67 EOTF(100, 1.5, {0.4}) = {9.31566}
-trc=arib-std-b67 EOTF(100, 1.5, {0.5}) = {13.1594}
-trc=arib-std-b67 EOTF(100, 1.5, {0.6}) = {19.1309}
-trc=arib-std-b67 EOTF(100, 1.5, {0.7}) = {28.4082}
-trc=arib-std-b67 EOTF(100, 1.5, {0.8}) = {42.8211}
-trc=arib-std-b67 EOTF(100, 1.5, {0.9}) = {65.2128}
-trc=arib-std-b67 EOTF(100, 1.5, {1}) = {100}
-trc=arib-std-b67 EOTF(203, 1.5, {0}) = {1.5}
-trc=arib-std-b67 EOTF(203, 1.5, {0.1}) = {3.70512}
-trc=arib-std-b67 EOTF(203, 1.5, {0.2}) = {6.89058}
-trc=arib-std-b67 EOTF(203, 1.5, {0.3}) = {11.0564}
-trc=arib-std-b67 EOTF(203, 1.5, {0.4}) = {16.2025}
-trc=arib-std-b67 EOTF(203, 1.5, {0.5}) = {23.1645}
-trc=arib-std-b67 EOTF(203, 1.5, {0.6}) = {34.3482}
-trc=arib-std-b67 EOTF(203, 1.5, {0.7}) = {52.3484}
-trc=arib-std-b67 EOTF(203, 1.5, {0.8}) = {81.3198}
-trc=arib-std-b67 EOTF(203, 1.5, {0.9}) = {127.949}
-trc=arib-std-b67 EOTF(203, 1.5, {1}) = {203}
-trc=arib-std-b67 EOTF(1000, 1.5, {0}) = {1.5}
-trc=arib-std-b67 EOTF(1000, 1.5, {0.1}) = {5.88189}
-trc=arib-std-b67 EOTF(1000, 1.5, {0.2}) = {13.9736}
-trc=arib-std-b67 EOTF(1000, 1.5, {0.3}) = {26.3592}
-trc=arib-std-b67 EOTF(1000, 1.5, {0.4}) = {43.5217}
-trc=arib-std-b67 EOTF(1000, 1.5, {0.5}) = {67.6771}
-trc=arib-std-b67 EOTF(1000, 1.5, {0.6}) = {109.808}
-trc=arib-std-b67 EOTF(1000, 1.5, {0.7}) = {184.977}
-trc=arib-std-b67 EOTF(1000, 1.5, {0.8}) = {319.786}
-trc=arib-std-b67 EOTF(1000, 1.5, {0.9}) = {562.396}
-trc=arib-std-b67 EOTF(1000, 1.5, {1}) = {1000}
-trc=arib-std-b67 EOTF(1, 1.5, {0}) = {1.5}
-trc=arib-std-b67 EOTF(1, 1.5, {0.1}) = {9.61469}
-trc=arib-std-b67 EOTF(1, 1.5, {0.2}) = {31.0816}
-trc=arib-std-b67 EOTF(1, 1.5, {0.3}) = {73.4178}
-trc=arib-std-b67 EOTF(1, 1.5, {0.4}) = {144.725}
-trc=arib-std-b67 EOTF(1, 1.5, {0.5}) = {262.813}
-trc=arib-std-b67 EOTF(1, 1.5, {0.6}) = {505.341}
-trc=arib-std-b67 EOTF(1, 1.5, {0.7}) = {1022.34}
-trc=arib-std-b67 EOTF(1, 1.5, {0.8}) = {2142.19}
-trc=arib-std-b67 EOTF(1, 1.5, {0.9}) = {4594.07}
-trc=arib-std-b67 EOTF(1, 1.5, {1}) = {1}
+ FAIL
Test color_utils failed. Look at tests/data/fate/color_utils.err for details.
threads=1
make: *** [src/tests/Makefile:311: fate-color_utils] Error 1
[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
Some people wanted to paint the bikeshed green, some blue and some pink.
People argued and fought, when they finally agreed, only rust was left.
signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] avcodec/vvc decode: ALF filtering without CC-ALF
On Wed, Dec 4, 2024 at 5:20 AM Chris Warrington via ffmpeg-devel < ffmpeg-devel@ffmpeg.org> wrote: > Nuo Mi wrote: > > This will introduce two writes for all blocks, even if there is no CC > ALF. > How about checking the sps_ccalf_enabled_flag in ff_vvc_alf_filter? > > That makes sense too, but I'd think you'd need to check both > sps_ccalf_enabled_flag and the slice header > sh_alf_cc_cb_enabled_flag/sh_alf_cc_cr_enabled_flag or the similar picture > header values. No, it's not needed. If sps_ccalf_enabled_flag is true, all ctb_cc_idc values will be initialized. > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2] avformat/mov: use dvdclut for YUV to RGB conversion of DVD subtitle palettes
Will push on v3 Sunday ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 3/6] lavc/vvc: Store MIP information over entire CU area
On Wed, Dec 4, 2024 at 2:09 AM Frank Plowman wrote:
> Hi,
>
> Thanks for your review.
>
> On 03/12/2024 10:04, Nuo Mi wrote:
> > Hi Frank,
> > Thank you for the patch
> >
> > On Sat, Nov 30, 2024 at 8:13 PM Frank Plowman
> wrote:
> >
> >> On 30/11/2024 06:46, Nuo Mi wrote:
> >>> On Fri, Nov 29, 2024 at 6:19 AM Frank Plowman
> >> wrote:
> >>>
> Previously, the code only stored the MIP mode and transpose flag in
> the
> relevant tables at the top-left corner of the CU. This information
> ends
> up being retrieved in ff_vvc_intra_pred_* not based on the CU position
> but instead the transform unit position (specifically, using the x0
> and
> y0 from get_luma_predict_unit). There might be multiple transform
> units
> in a CU, hence the top-left corner of the transform unit might not
> coincide with the top-left corner of the CU. Consequently, we need to
> store the MIP information at all positions in the CU, not only its
> top-left corner, as we already do for the MIP flag.
>
> Signed-off-by: Frank Plowman
> ---
> libavcodec/vvc/ctu.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/libavcodec/vvc/ctu.c b/libavcodec/vvc/ctu.c
> index 1e06119cfd..0030938cf5 100644
> --- a/libavcodec/vvc/ctu.c
> +++ b/libavcodec/vvc/ctu.c
> @@ -975,8 +975,8 @@ static void intra_luma_pred_modes(VVCLocalContext
> >> *lc)
> for (int y = 0; y < (cb_height>>log2_min_cb_size); y++) {
> int width = cb_width>>log2_min_cb_size;
> memset(&fc->tab.imf[x], cu->intra_mip_flag, width);
> -fc->tab.imtf[x] = intra_mip_transposed_flag;
> -fc->tab.imm[x] = intra_mip_mode;
> +memset(&fc->tab.imtf[x], intra_mip_transposed_flag,
> width);
> +memset(&fc->tab.imm[x], intra_mip_mode, width);
> >>>
> >>> intra_mip_mode is 4 bits, 2 flags are 2 bits. maybe we can use a
> uint8_t
> >>> for 3 fields,
> >>> We only need 1 memset and save 2/3 memory.
> >>
> >> I've implemented this (patch attached, to be applied atop the set), but
> >> it's not as straightforward as it may seem. In particular, because the
> >> tables are read directly from when determining which CABAC context to
> >> use for these flags, we have to add quite a lot of extra code in cabac.c
> >> to support this special case where the MIP information is a bit field.
> >> In my implementation, this was done by adding this coerce_to_bool
> >> parameter to get_inc and get_top. This does actually save a moderate
> >> amount of memory though, ~1MB for 4K and ~256kB for 1080p.
> >>
> > A coding block can be as small as 4x4, so for a 1080p frame, the memory
> > required is approximately 2*1920*1080/(4*4) ~= 256 kB.
> > However, with a maximum delay of 16 frames, the total memory usage will
> be
> > 256kB * 16=4 MB.
> >
> > In your patch, coerce_to_bool is set to 1 only when we are in
> > ff_vvc_intra_mip_flag.
> > How about defining get_mip_inc like this? This way, we can avoid changing
> > get_inc
> >
> > static av_always_inline
> > uint8_t get_mip_inc (VVCLocalContext *lc, const uint8_t *ctx)
> > {
> > uint8_t left = 0, top = 0;
> > get_left_top(lc, &left, &top, lc->cu->x0, lc->cu->y0, ctx, ctx);
> > return !!left + !!top;
> > }
>
> What about instead taking a function pointer as an argument to
> get_inc/get_left_top which can be used to modify the value read from the
> table before using it, or NULL to apply no such modification?
Function pointers have a cost
> At the
> moment, this would only be used for the MIP flag, but if we wanted to
> pack other tables in order to save more memory, we could then reuse that
argument in other ways and avoid duplicating the functions for every
> case where the in-memory representation differs. It's not that bad for
> get_inc, but if we ever wanted to change the representation of something
> which uses get_left_top directly, we would have to duplicate a fair
> amount of code.
>
We can change get_mip_inc to
int get_masked_inc (VVCLocalContext *lc, const uint8_t *ctx, uint8_t mask,
int shift)
{
uint8_t left = 0, top = 0;
get_left_top(lc, &left, &top, lc->cu->x0, lc->cu->y0, ctx, ctx);
return ((left & mask) + (right & mask)) >> shift;
}
>
> >
> > BTW: Using pack_mip_info/unpack_mip_info might be shorter and more
> concise
> > than structure_mip_info/destructure_mip_info.
>
> Agreed, will do.
>
> >
> >>
> >>>
>
> >>>
> >>> x += pps->min_cb_width;
> }
> cu->intra_pred_mode_y = intra_mip_mode;
> --
> 2.47.0
>
> >>
> >>
> >> --
> >> Frank
> >>
>
>
> --
> Frank
>
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-
Re: [FFmpeg-devel] [PATCH] avformat/movenc: log stream index for invalid pkt duration
On 2024-12-06 09:58 pm, Marth64 wrote: LGTM Thanks. Pushed as 914f89dbc61781eab00c05919b552b3170517e16 Regards, Gyan ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] avformat/mpegts: correctly skip TP_extra_header in m2ts
On Sat, Nov 30, 2024 at 8:30 PM llyyr wrote: > > On 8/8/24 1:44 AM, llyyr wrote: > > On 6/7/24 1:29 PM, Hendrik Leppkes wrote: > >> On Fri, Jun 7, 2024 at 9:47 AM Hendrik Leppkes > >> wrote: > >>> > >>> On Mon, May 27, 2024 at 3:47 PM llyyr wrote: > > instead of just resyncing and skipping a bunch of TS packets, > leading to > a loss of frames. > > Before this, a stray byte with the value of 0x47 in TP_extra_header > would throw off the detection of where TS packets start. > > A typical file that could cause issues would look like this: > > 0300: 238f 4780 4750 1110 01e0 84c0 > ^^ ^^ > The first four bytes here are TP_extra_header and the actual TS packet > starts at offset 0x304 > > FFmpeg would try to read a packet at 0x300 but since nothing skips the > 4 byte TP_extra_header, find that the first byte is not 0x47 and > immediately go into mpegts_resync, and incorrectly detect the stray > 0x47 > in the TP_extra_header at 0x302 as the new sync byte. > > Fix this by correctly skipping the first 4 bytes if the source packet > is 192 bytes. > > Signed-off-by: llyyr > --- > libavformat/mpegts.c | 12 +++- > 1 file changed, 11 insertions(+), 1 deletion(-) > > diff --git a/libavformat/mpegts.c b/libavformat/mpegts.c > index c66a1ea6ede0..0d80ad80f1aa 100644 > --- a/libavformat/mpegts.c > +++ b/libavformat/mpegts.c > @@ -2944,6 +2944,12 @@ static int read_packet(AVFormatContext *s, > uint8_t *buf, int raw_packet_size, > AVIOContext *pb = s->pb; > int len; > > +// 192 bytes source packet that start with a 4 bytes > TP_extra_header > +// followed by 188 bytes of TS packet. The sync byte is at > offset 4, so skip > +// the first 4 bytes otherwise we'll end up syncing to the > wrong packet. > +if (raw_packet_size == TS_DVHS_PACKET_SIZE) > +avio_skip(pb, 4); > + > >>> > >>> I think this doesn't work with mpegts_resync, since it always resyncs > >>> for the packet start to be on the 0x47 marker, not 4 bytes before it. > >>> So if sync is lost, it would never recover, if I read this right. > >>> > >> > >> Since we're dealing with a special case for 192 byte packets here > >> anyway, maybe a special case in mpegts_resync that just checks if raw > >> size = 192 && [4] = 0x47 would handle this with less potential > >> fallout? > > > > I've looked at this again and I don't see a way to gracefully handle it > > in mpegts_resync. It's much cleaner to just skip the header that's not > > relevant to us so we line up with the first packet being the sync byte. > > > > FWIW this can be reproduced with > > > > `ffmpeg -i -c copy out.h264` > > > > Sample file: https://0x0.st/XVJi.m2ts > > ping > > I don't think handling it in mpegts_resync is the right fix because it's > not a "special case" for specific files when m2ts is a standard and > commonly found in blurays and HDV cameras. I don't think there should be > any fallout from this. I've already tested the patch and it passes FATE, > along with other ts/m2ts samples I had. > > if it must be done in mpegts_resync, then it would need to accompany > other changes in read_packet as well and would result in a much larger > patch anyway. I went over the logic again and cleared up one of my concerns, as well as tested with a variety of samples. Couldn't find any regressions. So LGTM. - Hendrik ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
