Re: [FFmpeg-devel] [PATCH 3/5] avformat/mvdec: Do not set invalid sample rate

2021-09-16 Thread Peter Ross
On Wed, Sep 15, 2021 at 10:00:46PM +0200, Michael Niedermayer wrote:
> Fixes: signed integer overflow: -682581959642593728 * 16 cannot be 
> represented in type 'long'
> Fixes: 
> 37883/clusterfuzz-testcase-minimized-ffmpeg_dem_MV_fuzzer-5311691517198336
> 
> Found-by: continuous fuzzing process 
> https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> Signed-off-by: Michael Niedermayer 
> ---
>  libavformat/mvdec.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/libavformat/mvdec.c b/libavformat/mvdec.c
> index b1450e08da9..7573087c7cc 100644
> --- a/libavformat/mvdec.c
> +++ b/libavformat/mvdec.c
> @@ -156,9 +156,10 @@ static int parse_audio_var(AVFormatContext *avctx, 
> AVStream *st,
>  } else if (!strcmp(name, "NUM_CHANNELS")) {
>  return set_channels(avctx, st, var_read_int(pb, size));
>  } else if (!strcmp(name, "SAMPLE_RATE")) {
> -st->codecpar->sample_rate = var_read_int(pb, size);
> -if (st->codecpar->sample_rate <= 0)
> +int sample_rate = var_read_int(pb, size);
> +if (sample_rate <= 0)
>  return AVERROR_INVALIDDATA;
> +st->codecpar->sample_rate = sample_rate;
>  avpriv_set_pts_info(st, 33, 1, st->codecpar->sample_rate);
>  } else if (!strcmp(name, "SAMPLE_WIDTH")) {
>  uint64_t bpc = var_read_int(pb, size) * (uint64_t)8;

please apply

-- Peter
(A907 E02F A6E5 0CD2 34CD 20D2 6760 79C5 AC40 DD6B)


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


[FFmpeg-devel] [PATCH 1/2] libavfilter/x86/vf_gblur: fixed the fate-test failed on MacOS

2021-09-16 Thread Wu Jianhua
Signed-off-by: Wu Jianhua 
---
 libavfilter/x86/vf_gblur.asm | 6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/libavfilter/x86/vf_gblur.asm b/libavfilter/x86/vf_gblur.asm
index c0d57cc82b..64c067538a 100644
--- a/libavfilter/x86/vf_gblur.asm
+++ b/libavfilter/x86/vf_gblur.asm
@@ -455,12 +455,13 @@ cglobal horiz_slice, 4, 9, 9, ptr, width, height, steps, 
nu, bscale, x, y, step,
 mov nuq, localbufm
 DEFINE_ARGS buffer, width, height, steps, \
 localbuf, x, y, step, stride, remain, ptr, mask
-MOVSXDIFNIDN width, height, steps
 %else
 VBROADCASTSSm0, xmm0 ; nu
 VBROADCASTSSm1, xmm1 ; bscale
 %endif
 
+MOVSXDIFNIDN width, height, steps
+
 %if cpuflag(avx512)
 vpbroadcastdm2, widthd
 INIT_WORD_MASK  k6, k5, k4, k3, k2, k1
@@ -861,11 +862,12 @@ cglobal verti_slice, 6, 12, 9, 0-mmsize*2, buffer, width, 
height, cbegin, cend,
 VBROADCASTSS m1, bscalem
 DEFINE_ARGS buffer, width, height, cbegin, cend, \
 steps, x, y, cwidth, step, ptr, stride
-MOVSXDIFNIDN width, height, cbegin, cend, steps
 %else
 VBROADCASTSS m0, xmm0 ; nu
 VBROADCASTSS m1, xmm1 ; bscale
 %endif
+MOVSXDIFNIDN width, height, cbegin, cend, steps
+
 mov cwidthq, cendq
 sub cwidthq, cbeginq
 lea strideq, [widthq * 4]
-- 
2.17.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


[FFmpeg-devel] [PATCH 2/2] libavfilter/x86/vf_gblur: correct the order of loop step

2021-09-16 Thread Wu Jianhua
The problem was caused by if the width of the processed block
minus 1 is a multiple of the aligned number the instruction
jle .bscale_scalar would skip the Optimized Loop Step, which
will lead to an incorrect sampling when specifying steps more
than 1. Move the Optimized Loop Step after .bscale_scalar to
ensure the loop step is enabled.

Signed-off-by: Wu Jianhua 
---
 libavfilter/x86/vf_gblur.asm | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/libavfilter/x86/vf_gblur.asm b/libavfilter/x86/vf_gblur.asm
index 64c067538a..16e802e002 100644
--- a/libavfilter/x86/vf_gblur.asm
+++ b/libavfilter/x86/vf_gblur.asm
@@ -524,9 +524,8 @@ cglobal horiz_slice, 4, 9, 9, ptr, width, height, steps, 
nu, bscale, x, y, step,
 cmp xq,0
 jg .loop_x_scalar
 
-OPTIMIZED_LOOP_STEP
-
 .bscale_scalar:
+OPTIMIZED_LOOP_STEP
 sub ptrq, 4
 sub localbufq, mmsize
 mulps m3, m1
-- 
2.17.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 0/3] Detect field_order when probing MP4/H264

2021-09-16 Thread Nicolas Gaullier
>Nicolas Gaullier (3):
>  avcodec/h264_parser: Set AVCodecContext.framerate
>  avformat/utils: Use r_frame_rate in compute_frame_duration if 
> codec_framerate is unknown
>  avformat/mov: Set AVSTREAM_PARSE_HEADERS flag for H264

Hello, I have received no feedback on this serie I posted last week.
Could someone find some time for a review ?

Thanks to you
Nicolas
(ngaullier@irc)
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 1/2] libavfilter/x86/vf_gblur: fixed the fate-test failed on MacOS

2021-09-16 Thread Wu, Jianhua
Jianhua wrote:
> From: Wu, Jianhua 
> Sent: Thursday, September 16, 2021 3:34 PM
> To: ffmpeg-devel@ffmpeg.org
> Cc: Wu, Jianhua 
> Subject: [PATCH 1/2] libavfilter/x86/vf_gblur: fixed the fate-test failed on
> MacOS
> 
> Signed-off-by: Wu Jianhua 
> ---

Hi Zhili,

I have submitted the patches fixing the fate-test that failed on macOS. Could
you help try to apply the patches to check if they work? Here is the link:
http://ffmpeg.org/pipermail/ffmpeg-devel/2021-September/285376.html

And the illegal instructions:4 occurred might be the version of your GCC
installed less than 4.7, which could be fixed by specific a compiler option,
-mmacosx-version-min=10.x, by the way, the exact I used is 10.10. To enable
it on FFmpeg, you might have to specify in the --extra-cflags option when
configuring. But I recommend updating the GCC to the newest version, which
can fix this as well. 

If there is another problem, feel free to let me know.

Best regards,
Jianhua

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 01/12] lavf/concat: refactor parsing

2021-09-16 Thread Nicolas George
Nicolas George (12021-09-12):
> Will push this series soon.

Series pushed.

ffmpeg + lsdvd + dvd2concat can now encode DVDs reasonably well.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 1/4] libavcodec/adts_header: add frame_length field and avpriv function to parse AAC ADTS header

2021-09-16 Thread Steven Liu
Nachiket Tarate  于2021年9月1日周三 下午10:39写道:
>
> These will be used by HLS demuxer in case of sample decryption.
>
> Signed-off-by: Nachiket Tarate 
> ---
>  libavcodec/adts_header.c |  1 +
>  libavcodec/adts_header.h | 15 +++
>  libavcodec/adts_parser.c | 31 +++
>  3 files changed, 47 insertions(+)
>
> diff --git a/libavcodec/adts_header.c b/libavcodec/adts_header.c
> index 0889820f8a..e4454529c4 100644
> --- a/libavcodec/adts_header.c
> +++ b/libavcodec/adts_header.c
> @@ -66,6 +66,7 @@ int ff_adts_header_parse(GetBitContext *gbc, 
> AACADTSHeaderInfo *hdr)
>  hdr->sample_rate= avpriv_mpeg4audio_sample_rates[sr];
>  hdr->samples= (rdb + 1) * 1024;
>  hdr->bit_rate   = size * 8 * hdr->sample_rate / hdr->samples;
> +hdr->frame_length   = size;
>
>  return size;
>  }
> diff --git a/libavcodec/adts_header.h b/libavcodec/adts_header.h
> index f615f6a9f9..166a28ffc9 100644
> --- a/libavcodec/adts_header.h
> +++ b/libavcodec/adts_header.h
> @@ -34,6 +34,7 @@ typedef struct AACADTSHeaderInfo {
>  uint8_t  sampling_index;
>  uint8_t  chan_config;
>  uint8_t  num_aac_frames;
> +uint32_t frame_length;
>  } AACADTSHeaderInfo;
>
>  /**
> @@ -47,4 +48,18 @@ typedef struct AACADTSHeaderInfo {
>   */
>  int ff_adts_header_parse(GetBitContext *gbc, AACADTSHeaderInfo *hdr);
>
> +/**
> + * Parse the ADTS frame header contained in the buffer, which is
> + * the first 54 bits.
> + * @param[in]  buf  Pointer to buffer containing the first 54 bits of the 
> frame.
> + * @param[in]  size Size of buffer containing the first 54 bits of the frame.
> + * @param[out] phdr Pointer to pointer to struct AACADTSHeaderInfo for which
> + * memory is allocated and header info is written into it. After using the 
> header
> + * information, the allocated memory must be freed by using av_free().
> + * @return Returns 0 on success, -1 if there is a sync word mismatch,
> + * -2 if the version element is invalid, -3 if the sample rate
> + * element is invalid, or -4 if the bit rate element is invalid.
> + */
> +int avpriv_adts_header_parse(AACADTSHeaderInfo **phdr, const uint8_t *buf, 
> size_t size);
> +
>  #endif /* AVCODEC_ADTS_HEADER_H */
> diff --git a/libavcodec/adts_parser.c b/libavcodec/adts_parser.c
> index 5c9f8ff6f2..4a1a8fd5f4 100644
> --- a/libavcodec/adts_parser.c
> +++ b/libavcodec/adts_parser.c
> @@ -42,3 +42,34 @@ int av_adts_header_parse(const uint8_t *buf, uint32_t 
> *samples, uint8_t *frames)
>  return AVERROR(ENOSYS);
>  #endif
>  }
> +
> +int avpriv_adts_header_parse(AACADTSHeaderInfo **phdr, const uint8_t *buf, 
> size_t size)
> +{
> +#if CONFIG_ADTS_HEADER
> +int ret = 0;
> +GetBitContext gb;
> +
> +if (!phdr || !buf || size < AV_AAC_ADTS_HEADER_SIZE)
> +return AVERROR_INVALIDDATA;
> +
> +*phdr = av_mallocz(sizeof(AACADTSHeaderInfo));
> +if (!*phdr)
> +return AVERROR(ENOMEM);
> +
> +ret = init_get_bits8(&gb, buf, AV_AAC_ADTS_HEADER_SIZE);
> +if (ret < 0) {
> +av_freep(phdr);
> +return ret;
> +}
> +
> +ret = ff_adts_header_parse(&gb, *phdr);
> +if (ret < 0) {
> +av_freep(phdr);
> +return ret;
> +}
> +
> +return 0;
> +#else
> +return AVERROR(ENOSYS);
> +#endif
> +}
> --
> 2.17.1
>
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


looks have been fixed the suggestion from zhili.
Will apply patchset if no more comments
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 2/2] avformat/mxf: support MCA audio information

2021-09-16 Thread Tomas Härdin
ons 2021-09-15 klockan 12:14 +0200 skrev Marc-Antoine Arnaud:

+static int mxf_read_mca_sub_descriptor(void *arg, AVIOContext *pb, int
tag, int size, UID uid, int64_t klv_offset)
+{
+    MXFMCASubDescriptor *mca_sub_descriptor = arg;
+
+    if (IS_KLV_KEY(uid, mxf_mca_prefix)) {
+    if (IS_KLV_KEY(uid, mxf_mca_label_dictionnary_id)) {
+    avio_read(pb, mca_sub_descriptor-
>mca_label_dictionnary_id, 16);
+    }
+    if (IS_KLV_KEY(uid, mxf_mca_link_id)) {
+    avio_read(pb, mca_sub_descriptor->mca_link_id, 16);
+    }
+    if (IS_KLV_KEY(uid, mxf_soundfield_group_link_id)) {
+    avio_read(pb, mca_sub_descriptor->mca_group_link_id, 16);
+    }
+    }

This nesting of ifs is unnecessary

Rest of the patch looks fine. Passes FATE. Do you have a sample or two?

/Tomas

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH v5 00/12] Subtitle Filtering

2021-09-16 Thread Nicolas George
Hendrik Leppkes (12021-09-15):
> Or perhaps some people have a day job, a life and other obligations
> that prevent them from spending time on FFmpeg every day, especially
> outside the weekend.
> But no, that can't be it, surely we are all just evil. /s

> Nothing here is being invented. You are trying to push a major API
> change to the core functionality of our libraries, these rules on how
> to do API changes on that level (or any level, really) have been in
> place and followed by everyone for years and years.
> If you don't believe that, feel free to check how any other major API
> change was handled in the past. Because the overall pattern is always
> the same.
> 
> Do you think I'm the only one thinking that way, only because I spoke
> up? The set certainly wouldn't have been applied if I hadn't said
> anything, it would've just sat there. Maybe someone else might've come
> forward eventually.

Thank you for having the patience and taking the time to explain this.

> As for the actual subject:
> - Part1, add subtitles to AVFrame in avutil. You can move
> enums/structs to avutil as needed (eg AVSubtitleRect or whatever), as
> long as they are still available the same way (eg. avcodec.h should
> include any new avutil header with them, so user code works without
> changes), but moving functions is a more tricky business, so we
> usually make new functions and cleanup their naming convention in the
> process, but I don't think any functions are involved here right now.
> 
> To dive a bit deeper into this, redundant fields should be removed,
> actual subtitle data should be refcounted (with AVBuffers in
> AVFrame->buf), all frame functionality need to properly account for
> the new data type.

There is another point to consider when designing subtitles in AVFrame:
since we intend it not only as a general API cleanup but as a prelude to
extending the API with filtering and such, we must not only think about
what is needed now, we must think about what may be needed later.

The few examples I have, not excluding questions I have not thought of:

- Right now, we have text subtitles and bitmap subtitles. But do we want
  to be able to handle mixed text and bitmap subtitles?

  To this, I would say: probably no. But we should give it a thought.

- Number of rectangles. Currently, the decoders usually output a very
  limited number of rectangles. OTOH, libass may output three alpha maps
  per glyph, that makes potentially hundreds of rectangles.

  Will the data structure be able to handle this efficiently?

  Consider also the issue of rectangles overlapping.

- Speaking of overlapping, we need a system to signal whether a new
  subtitle frame should replace the current one (like dvdsub, srt, etc.)
  or overlap with it (like ASS).

- Colorspace. All current bitmap subtitles formats are paletted. But
  palette pixel formats are bad for many treatments. Inside a filter
  chain, it would probably make sense to have them in a true color
  format.

- Global styles. ASS subtitles, in particular, contain reference to
  globally-defined styles. How do we handle that in libavfilter? I have
  no idea.

- Sparseness. Subtitles streams have gaps, and synchronization with
  other streams requires a next frame, that can be minutes away or never
  come. This needs to be solved in a way compatible with processing.

> - Part3, avfilter support for subtitles in AVFrames. At this point we
> have a defined structure to store subtitles in AVFrames, and actual
> code that can generate or consume them. When approaching this, the
> same rules apply as before, existing subtitle functionality, as crude
> as it may be, has to remain functional as exposed to the user.
> Internal under-the-hood changes are fine, as long as the user does not
> notice.
> 
> I'm not involved in internal avfilter design, so at this point you'll
> have to get Nicolas on board.

The most salient point is negotiation. I have already said it and I say
it again, because the format negotiation in libavfilter is a very
important aspect of what makes it work well, and it is also a very
tricky piece of code.

We need to decide which aspects of the subtitles formats are negotiated.

At least, obviously, the text or bitmap aspect will be, with a
conversion filter inserted automatically where needed.

But depending on the answers to the questions in part 1, we may need to
negotiate the pixel format and colorspace too.

Unfortunately, the current negotiation code is messy and fragile. We
cannot afford to pile new code on top of it. However good the new code
may be, adding it on top of messy code would only make it harder to
clean up and maintain later. I absolutely oppose that.

Therefore, if anybody wants to add something to libavfilter that
involves the negotiation, they should consider helping cleaning it up.
It would make your work easier afterwards, so you should do it if only
for that reason.

The first task, as I have mentioned, would probably be to add FATE tes

[FFmpeg-devel] [PATCH 1/5] avcodec/libsvtav1: Fix redundant setting of caps_internal

2021-09-16 Thread lance . lmwang
From: Limin Wang 

Signed-off-by: Limin Wang 
---
 libavcodec/libsvtav1.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/libavcodec/libsvtav1.c b/libavcodec/libsvtav1.c
index fabc4e6..82ae2b9 100644
--- a/libavcodec/libsvtav1.c
+++ b/libavcodec/libsvtav1.c
@@ -561,12 +561,11 @@ const AVCodec ff_libsvtav1_encoder = {
 .receive_packet = eb_receive_packet,
 .close  = eb_enc_close,
 .capabilities   = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_OTHER_THREADS,
-.caps_internal  = FF_CODEC_CAP_AUTO_THREADS,
+.caps_internal  = FF_CODEC_CAP_AUTO_THREADS | FF_CODEC_CAP_INIT_CLEANUP,
 .pix_fmts   = (const enum AVPixelFormat[]){ AV_PIX_FMT_YUV420P,
 AV_PIX_FMT_YUV420P10,
 AV_PIX_FMT_NONE },
 .priv_class = &class,
 .defaults   = eb_enc_defaults,
-.caps_internal  = FF_CODEC_CAP_INIT_CLEANUP,
 .wrapper_name   = "libsvtav1",
 };
-- 
1.8.3.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


[FFmpeg-devel] [PATCH 2/5] avcodec/libsvtav1: make intra_refresh_type configurable

2021-09-16 Thread lance . lmwang
From: Limin Wang 

Signed-off-by: Limin Wang 
---
 libavcodec/libsvtav1.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/libavcodec/libsvtav1.c b/libavcodec/libsvtav1.c
index 82ae2b9..8c2c970 100644
--- a/libavcodec/libsvtav1.c
+++ b/libavcodec/libsvtav1.c
@@ -210,7 +210,8 @@ static int config_enc_params(EbSvtAv1EncConfiguration 
*param,
 param->min_qp_allowed   = avctx->qmin;
 }
 
-param->intra_refresh_type   = 2; /* Real keyframes only */
+/* 2 = IDR, closed GOP, 1 = CRA, open GOP */
+param->intra_refresh_type = avctx->flags & AV_CODEC_FLAG_CLOSED_GOP ? 2 : 
1;
 
 if (svt_enc->la_depth >= 0)
 param->look_ahead_distance  = svt_enc->la_depth;
@@ -548,6 +549,7 @@ static const AVCodecDefault eb_enc_defaults[] = {
 { "g", "-1"},
 { "qmin",  "0" },
 { "qmax",  "63"},
+{ "flags", "+cgop" },
 { NULL },
 };
 
-- 
1.8.3.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


[FFmpeg-devel] [PATCH 3/5] doc: update for libsvtav1 encoder

2021-09-16 Thread lance . lmwang
From: Limin Wang 

Signed-off-by: Limin Wang 
---
 doc/encoders.texi | 44 
 1 file changed, 44 insertions(+)

diff --git a/doc/encoders.texi b/doc/encoders.texi
index 8fccd73..64d604e 100644
--- a/doc/encoders.texi
+++ b/doc/encoders.texi
@@ -1750,12 +1750,56 @@ You need to explicitly configure the build with 
@code{--enable-libsvtav1}.
 @table @option
 @item profile
 Set the encoding profile.
+@table @samp
+@item main
+@item high
+@item professional
+@end table
 
 @item level
 Set the operating point level.
+@table @samp
+@item 2.0
+@item 2.1
+@item 2.2
+@item 2.3
+@item 3.0
+@item 3.1
+@item 3.2
+@item 3.3
+@item 4.0
+@item 4.1
+@item 4.2
+@item 4.3
+@item 5.0
+@item 5.1
+@item 5.2
+@item 5.3
+@item 6.0
+@item 6.1
+@item 6.2
+@item 6.3
+@item 7.0
+@item 7.1
+@item 7.2
+@item 7.3
+@end table
+
+@item hielevel
+Set the Hierarchical prediction levels.
+@table @samp
+@item 3level
+@item 4level, This is the default.
+@end table
 
 @item tier
 Set the operating point tier.
+@table @samp
+@item main
+The main tier was designed for most applications. This is the default.
+@item high
+The high tier was designed for very demanding applications.
+@end table
 
 @item rc
 Set the rate control mode to use.
-- 
1.8.3.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


[FFmpeg-devel] [PATCH 4/5] avcodec/libsvtav1: Fix CQP mode doesn't work as expection

2021-09-16 Thread lance . lmwang
From: Limin Wang 

Signed-off-by: Limin Wang 
---
 libavcodec/libsvtav1.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/libavcodec/libsvtav1.c b/libavcodec/libsvtav1.c
index 8c2c970..1f1f86b 100644
--- a/libavcodec/libsvtav1.c
+++ b/libavcodec/libsvtav1.c
@@ -208,6 +208,8 @@ static int config_enc_params(EbSvtAv1EncConfiguration 
*param,
 if (param->rate_control_mode) {
 param->max_qp_allowed   = avctx->qmax;
 param->min_qp_allowed   = avctx->qmin;
+} else {
+param->enable_tpl_la = 0; /* CQP need turn off enable_tp_la */
 }
 
 /* 2 = IDR, closed GOP, 1 = CRA, open GOP */
-- 
1.8.3.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


[FFmpeg-devel] [PATCH 5/5] avcodec/libsvtav1: support constant quality mode

2021-09-16 Thread lance . lmwang
From: Limin Wang 

Signed-off-by: Limin Wang 
---
 doc/encoders.texi  |  7 ++-
 libavcodec/libsvtav1.c | 10 +-
 libavcodec/version.h   |  2 +-
 3 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/doc/encoders.texi b/doc/encoders.texi
index 64d604e..2300bb9 100644
--- a/doc/encoders.texi
+++ b/doc/encoders.texi
@@ -1807,9 +1807,11 @@ Set the rate control mode to use.
 Possible modes:
 @table @option
 @item cqp
-Constant quantizer: use fixed values of qindex (dependent on the frame type)
+Constant quantizer(not set crf): use fixed values of qindex (dependent on the 
frame type)
 throughout the stream.  This mode is the default.
 
+Constant quality(set crf): maintain a constant QP throughout the stream.
+
 @item vbr
 Variable bitrate: use a target bitrate for the whole stream.
 
@@ -1826,6 +1828,9 @@ Set the minimum quantizer to use when using a bitrate 
mode.
 @item qp
 Set the quantizer used in cqp rate control mode (0-63).
 
+@item crf
+Select the quality for constant quality mode (0-63).
+
 @item sc_detection
 Enable scene change detection.
 
diff --git a/libavcodec/libsvtav1.c b/libavcodec/libsvtav1.c
index 1f1f86b..29be4bd 100644
--- a/libavcodec/libsvtav1.c
+++ b/libavcodec/libsvtav1.c
@@ -66,6 +66,7 @@ typedef struct SvtContext {
 int rc_mode;
 int scd;
 int qp;
+int crf;
 
 int tier;
 
@@ -210,6 +211,10 @@ static int config_enc_params(EbSvtAv1EncConfiguration 
*param,
 param->min_qp_allowed   = avctx->qmin;
 } else {
 param->enable_tpl_la = 0; /* CQP need turn off enable_tp_la */
+if ( svt_enc->crf > 0) {
+param->qp = svt_enc->crf;
+param->enable_tpl_la = 1;
+}
 }
 
 /* 2 = IDR, closed GOP, 1 = CRA, open GOP */
@@ -523,13 +528,16 @@ static const AVOption options[] = {
 
 { "rc", "Bit rate control mode", OFFSET(rc_mode),
   AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 3, VE , "rc"},
-{ "cqp", "Constant quantizer", 0, AV_OPT_TYPE_CONST, { .i64 = 0 },  
INT_MIN, INT_MAX, VE, "rc" },
+{ "cqp", "Constant quantizer(not set crf), Constant quality(set crf)", 
0, AV_OPT_TYPE_CONST, { .i64 = 0 },  INT_MIN, INT_MAX, VE, "rc" },
 { "vbr", "Variable Bit Rate, use a target bitrate for the entire 
stream", 0, AV_OPT_TYPE_CONST, { .i64 = 1 },  INT_MIN, INT_MAX, VE, "rc" },
 { "cvbr", "Constrained Variable Bit Rate, use a target bitrate for 
each GOP", 0, AV_OPT_TYPE_CONST,{ .i64 = 2 },  INT_MIN, INT_MAX, VE, "rc" },
 
 { "qp", "Quantizer to use with cqp rate control mode", OFFSET(qp),
   AV_OPT_TYPE_INT, { .i64 = 50 }, 0, 63, VE },
 
+{ "crf", "Select the quality for constant quality mode", OFFSET(crf),
+  AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 63, VE },
+
 { "sc_detection", "Scene change detection", OFFSET(scd),
   AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, VE },
 
diff --git a/libavcodec/version.h b/libavcodec/version.h
index 4b4fe54..b0a741b 100644
--- a/libavcodec/version.h
+++ b/libavcodec/version.h
@@ -29,7 +29,7 @@
 
 #define LIBAVCODEC_VERSION_MAJOR  59
 #define LIBAVCODEC_VERSION_MINOR   7
-#define LIBAVCODEC_VERSION_MICRO 103
+#define LIBAVCODEC_VERSION_MICRO 104
 
 #define LIBAVCODEC_VERSION_INT  AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \
LIBAVCODEC_VERSION_MINOR, \
-- 
1.8.3.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 1/5] avcodec/libsvtav1: Fix redundant setting of caps_internal

2021-09-16 Thread Andreas Rheinhardt
lance.lmw...@gmail.com:
> From: Limin Wang 
> 
> Signed-off-by: Limin Wang 
> ---
>  libavcodec/libsvtav1.c | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
> 
> diff --git a/libavcodec/libsvtav1.c b/libavcodec/libsvtav1.c
> index fabc4e6..82ae2b9 100644
> --- a/libavcodec/libsvtav1.c
> +++ b/libavcodec/libsvtav1.c
> @@ -561,12 +561,11 @@ const AVCodec ff_libsvtav1_encoder = {
>  .receive_packet = eb_receive_packet,
>  .close  = eb_enc_close,
>  .capabilities   = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_OTHER_THREADS,
> -.caps_internal  = FF_CODEC_CAP_AUTO_THREADS,
> +.caps_internal  = FF_CODEC_CAP_AUTO_THREADS | FF_CODEC_CAP_INIT_CLEANUP,
>  .pix_fmts   = (const enum AVPixelFormat[]){ AV_PIX_FMT_YUV420P,
>  AV_PIX_FMT_YUV420P10,
>  AV_PIX_FMT_NONE },
>  .priv_class = &class,
>  .defaults   = eb_enc_defaults,
> -.caps_internal  = FF_CODEC_CAP_INIT_CLEANUP,
>  .wrapper_name   = "libsvtav1",
>  };
> 
This is not redundant; the second initialization overrides the first. If
I read validate_thread_parameters in pthread.c correctly, this means
that 7d09579190d broke the ability to forward the threading parameters
to the actual encoder. But it never did this anyway, so it doesn't
matter. Should be fixed nevertheless.

- Andreas
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


[FFmpeg-devel] Plans for libavfilter

2021-09-16 Thread Nicolas George
Since it is not healthy to keep everything for myself, here is a summary
of the projects I have in mind for the enhancement of libavfilter.

The thing is, there are a lot of dependencies between these projects.
Working on them in a proper order would make achieving the goals easier.
On the other hand, if task B depends on task A, then working B without
addressing A involves a little more work for B, but also a lot more work
for A, and a lot more work for everything else that depends on A.

This is one reason I am writing this mail: to make clear what needs to
be done, and how tasks depend on each-other and fit together.

The earliest tasks are not very sexy. They are about making the code
clearer or more robust and generic rather than adding features. It is
always boring. But they are necessary, because parts of the process is
quite shaky, and we cannot afford to build on shaky code.

If you want to help, please consider this and you will be welcome.

Now that is established the specifics:

- Multi-stage negotiation.

  That means negotiating properties that depend on other properties. I
  think the colorspace stuff is in that case, it cannot be negotiated
  until the pixel format is known.

- Media type negotiation.

  That means that filters that do not care about the contents of what
  they are filtering, filters that only act on the frame information
  fields, can also not care if they are filtering audio or video or
  other. There are a lot of these filters: pts manipulation, concat,
  split, selection, etc.

  We cannot afford to have to update every one of them if we want to add
  a new media type, so this is a prerequisite. And it is an aspect of
  multi-stage negotiation, since pixel format or sample format can only
  be negotiated once we know the media type.

- Partial graph configuration.

  Right now, the negotiation happens in steps, first query_formats then
  merge, then pick. But pick can be done on some links as soon as the
  formats are merged to a singleton while other filters are still
  delaying their query_formats, and once pick is done, the graph can
  start running. That would allow filters with constraints more complex
  than what AVFilterFormats can express. (We have a mechanism for that,
  used by amerge and one or two other filters; I introduced it early and
  it was a mistake, it is much too fragile and only converges in the
  simplest cases.)

- Partial graph re-configuration.

  If we can run a graph when not all filters are configured, then we can
  de-configure and re-configure part of a graph. That would bring us the
  support for changes in pixel format.

- Global running API.

  Right now, we make libavfilter activate filters by pumping
  request_frame() on one of the buffersinks, more or less randomly, and
  we hope it will result in something useful. We need a better API, one
  where we say libavfilter "start running these graphs", we push frames
  on input as we have them, we are notified when frames arrive on
  output.

- Inter-filter threading.

  This is a big one, and quite self-explanatory. It goes hand to hand
  with a global running API, because "start running these graphs" also
  means "start as many threads as you need and keep them ready".

  Note that the network protocols in libavformat requires the same
  thing. And I really do not want yet two other frigging threading
  subsystems. This is why I have started slowly working on an unique
  system suited for all needs, starting with both libavformat and
  libavfilter but keeping also libavcodec in mind.

- Out-of-band side-data.

  This is a mechanism to notify filters that the work they are currently
  doing may become irrelevant, or other kinds of urgent information.
  Filters can ignore it, and just do the work for nothing. But smarter
  filters can take it into account and just skip until it becomes
  relevant again.

- Seeking.

  This is when applications notify the outputs they want to jump at a
  specified time, and all the filters in the graph make sure the
  instruction reach the inputs.

  This is probably rather easy and does not depend on many other things.
  Out-of-band side-data is meant for that, among other things: notify a
  filter "you can skip processing the frames you have queued, because we
  are seeking anyway"; and if seeking fails, notify the user as fast as
  possible "we tried seeking, it will not happen".

  A few things need ironing out, though. Do we want to handle it in
  activate() or do we want a separate callback? Do we want a specific
  mechanism for seeking or something more generic for all the messages
  that go backwards in the graph? Do we want a queue for messages that
  go backwards or would a single message be enough?

- Subtitles.

  This one has been under the spotlight recently. What it means is
  rather obvious. But it if far from trivial.

  For starters, it requires the negotiation of media type, because we
  cannot afford to update all the utility filters for

Re: [FFmpeg-devel] Plans for libavfilter

2021-09-16 Thread Paul B Mahol
On Thu, Sep 16, 2021 at 1:55 PM Nicolas George  wrote:

> Since it is not healthy to keep everything for myself, here is a summary
> of the projects I have in mind for the enhancement of libavfilter.
>
> The thing is, there are a lot of dependencies between these projects.
> Working on them in a proper order would make achieving the goals easier.
> On the other hand, if task B depends on task A, then working B without
> addressing A involves a little more work for B, but also a lot more work
> for A, and a lot more work for everything else that depends on A.
>
> This is one reason I am writing this mail: to make clear what needs to
> be done, and how tasks depend on each-other and fit together.
>
> The earliest tasks are not very sexy. They are about making the code
> clearer or more robust and generic rather than adding features. It is
> always boring. But they are necessary, because parts of the process is
> quite shaky, and we cannot afford to build on shaky code.
>
> If you want to help, please consider this and you will be welcome.
>
> Now that is established the specifics:
>
> - Multi-stage negotiation.
>
>   That means negotiating properties that depend on other properties. I
>   think the colorspace stuff is in that case, it cannot be negotiated
>   until the pixel format is known.
>
> - Media type negotiation.
>
>   That means that filters that do not care about the contents of what
>   they are filtering, filters that only act on the frame information
>   fields, can also not care if they are filtering audio or video or
>   other. There are a lot of these filters: pts manipulation, concat,
>   split, selection, etc.
>
>   We cannot afford to have to update every one of them if we want to add
>   a new media type, so this is a prerequisite. And it is an aspect of
>   multi-stage negotiation, since pixel format or sample format can only
>   be negotiated once we know the media type.
>

This is short sighted and irrelevant and also source  of confusion for
users.

I really see no way to tell you to stop insisting on this nonsense. So I
kindly ask you
yet again to stop further work you planned on libavfilter for everyone else
good outcome.



> - Partial graph configuration.
>
>   Right now, the negotiation happens in steps, first query_formats then
>   merge, then pick. But pick can be done on some links as soon as the
>   formats are merged to a singleton while other filters are still
>   delaying their query_formats, and once pick is done, the graph can
>   start running. That would allow filters with constraints more complex
>   than what AVFilterFormats can express. (We have a mechanism for that,
>   used by amerge and one or two other filters; I introduced it early and
>   it was a mistake, it is much too fragile and only converges in the
>   simplest cases.)
>
> - Partial graph re-configuration.
>
>   If we can run a graph when not all filters are configured, then we can
>   de-configure and re-configure part of a graph. That would bring us the
>   support for changes in pixel format.
>
> - Global running API.
>
>   Right now, we make libavfilter activate filters by pumping
>   request_frame() on one of the buffersinks, more or less randomly, and
>   we hope it will result in something useful. We need a better API, one
>   where we say libavfilter "start running these graphs", we push frames
>   on input as we have them, we are notified when frames arrive on
>   output.
>
> - Inter-filter threading.
>
>   This is a big one, and quite self-explanatory. It goes hand to hand
>   with a global running API, because "start running these graphs" also
>   means "start as many threads as you need and keep them ready".
>
>   Note that the network protocols in libavformat requires the same
>   thing. And I really do not want yet two other frigging threading
>   subsystems. This is why I have started slowly working on an unique
>   system suited for all needs, starting with both libavformat and
>   libavfilter but keeping also libavcodec in mind.
>
>
This is the only useful entry so far.


> - Out-of-band side-data.
>
>   This is a mechanism to notify filters that the work they are currently
>   doing may become irrelevant, or other kinds of urgent information.
>   Filters can ignore it, and just do the work for nothing. But smarter
>   filters can take it into account and just skip until it becomes
>   relevant again.
>
> - Seeking.
>
>   This is when applications notify the outputs they want to jump at a
>   specified time, and all the filters in the graph make sure the
>   instruction reach the inputs.
>
>   This is probably rather easy and does not depend on many other things.
>   Out-of-band side-data is meant for that, among other things: notify a
>   filter "you can skip processing the frames you have queued, because we
>   are seeking anyway"; and if seeking fails, notify the user as fast as
>   possible "we tried seeking, it will not happen".
>
>   A few things need ironing out, though. Do we wan

Re: [FFmpeg-devel] [PATCH v2 05/14] avfilter/af_aresample: Use preinit instead of init_dict

2021-09-16 Thread Nicolas George
Andreas Rheinhardt (12021-09-14):
> By using preinit, the SwrContext already exists directly after
> allocating the filter, so that the filter's AVClass's child_next
> becomes usable for setting options with the AV_OPT_SEARCH_CHILDREN
> search flag. This means that it is no longer necessary to use
> the init_dict callback for this filter.
> 
> Furthermore, the earlier code did not abide by the documentation
> of the init_dict callback at all: Instead of only returning the
> options that have not been recognized it always returned all options
> on any av_opt_set() error and errored out in this case; yet if
> the error was just caused by an unrecognized option, it should not
> error out at all and instead return said option.
> 
> This behaviour has been inherited by avfilter_init_dict(),
> contradicting its documentation. This is also fixed by this commit.
> 
> Signed-off-by: Andreas Rheinhardt 
> ---
>  libavfilter/af_aresample.c | 27 +++
>  1 file changed, 7 insertions(+), 20 deletions(-)

This looks like a good change. Thanks.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH v2 14/14] avfilter/avfilter: Report all unrecognized options in avfilter_init_str

2021-09-16 Thread Nicolas George
Andreas Rheinhardt (12021-09-14):
> Signed-off-by: Andreas Rheinhardt 
> ---
>  libavfilter/avfilter.c | 5 ++---
>  1 file changed, 2 insertions(+), 3 deletions(-)

Looks ok.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] Plans for libavfilter

2021-09-16 Thread Nicolas George
Paul B Mahol (12021-09-16):
> This is short sighted and irrelevant and also source  of confusion for
> users.
> 
> I really see no way to tell you to stop insisting on this nonsense. So I
> kindly ask you
> yet again to stop further work you planned on libavfilter for everyone else
> good outcome.

If you want to discuss politely why you think this is a bad idea, I am
ready to listen to your arguments and present mine.

Otherwise, I do not care.

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH v2 11/14] avfilter/avfilter: Use AV_DICT_DONT_STRDUP_(KEY|VAL) when possible

2021-09-16 Thread Nicolas George
Andreas Rheinhardt (12021-09-14):
> Signed-off-by: Andreas Rheinhardt 
> ---
>  libavfilter/avfilter.c | 7 +++
>  1 file changed, 3 insertions(+), 4 deletions(-)

LGTM.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 1/5] avformat/sbgdec: Check for t0 overflow in expand_tseq()

2021-09-16 Thread Nicolas George
Michael Niedermayer (12021-09-15):
> Fixes: signed integer overflow: 4611686025627387904 + 4611686025627387904 
> cannot be represented in type 'long'
> Fixes: 
> 35489/clusterfuzz-testcase-minimized-ffmpeg_dem_SBG_fuzzer-4862678601433088
> 
> Found-by: continuous fuzzing process 
> https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> Signed-off-by: Michael Niedermayer 
> ---
>  libavformat/sbgdec.c | 3 +++
>  1 file changed, 3 insertions(+)

No objection.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH v2 12/14] avfilter/avfilter: Honour the short options documentation

2021-09-16 Thread Nicolas George
Andreas Rheinhardt (12021-09-14):
> The documentation for filter arguments states that short options must
> precede long options (i.e. those of the form key=value). Yet if
> process_options() encounters arguments not abiding by this, it simply
> treats short options after a long option as if it were parsing short
> options for the first time. In particular, it overwrites options already
> set earlier, possibly via other short options. This is not how it is
> intended (as a comment in the code indicates).
> 
> This commit modifies the code to reject further shorthand options
> after a long option has been encountered. After all, avfilter_init_str()
> errors out upon unrecognized options, so it is intended to be picky.
> 
> Signed-off-by: Andreas Rheinhardt 
> ---
>  libavfilter/avfilter.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)

Since you have a good reason to use a pointer instead of a integer
boolean, then ok.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH v2 13/14] avfilter/avfilter: Don't fail upon options for filter without AVClass

2021-09-16 Thread Nicolas George
Andreas Rheinhardt (12021-09-14):
> Commit 62549f9655c48f0ec061087fa33a96040ce01145 added a check to
> (the predecessor of) avfilter_init_str() to error out if options
> were provided to a filter without options (or rather, without private
> class). This was fine at the time, yet soon afterwards commit
> fdd93eabfb2644f541f7aac9943abce26776ea73 added a generic option
> for all AVFilterContexts and since then it is wrong to error out
> in case options have been provided to a filter without AVClass.
> 
> To workaround this issue, several filters with timeline support
> added AVClasses and empty options; these will be removed in subsequent
> commits. Furthermore, the super2xsai filter supports slice threading,
> but no options and so has no AVClass, making it impossible to set
> the number of threads when using avfilter_init_str() (and therefore
> from the ffmpeg-tool). This is fixed by this commit, too.
> 
> Signed-off-by: Andreas Rheinhardt 
> ---
> Now with a comment explaining the two functions of checking for priv.
> 
>  libavfilter/avfilter.c | 9 +++--
>  1 file changed, 3 insertions(+), 6 deletions(-)

Ok, I suppose.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH v2 09/14] avfilter/vf_scale: Honour the AVFilter.init_dict documentation

2021-09-16 Thread Nicolas George
Andreas Rheinhardt (12021-09-14):
> The documentation states that unrecognized options need to be returned
> (just as av_opt_set_dict() would do). Yet the scale and scale2ref
> filters didn't abide by this: They simply copied all options
> and in case it contained an unrecognized option was among them,
> they would error out later in config_props. This violates the
> documentation of av_filter_init_dict().
> 
> Fix this by only keeping the recognized options and returning
> the unrecognized ones.
> 
> Signed-off-by: Andreas Rheinhardt 
> ---
>  libavfilter/vf_scale.c | 16 +---
>  1 file changed, 13 insertions(+), 3 deletions(-)
> 
> diff --git a/libavfilter/vf_scale.c b/libavfilter/vf_scale.c
> index a1902a13cf..c31b92b847 100644
> --- a/libavfilter/vf_scale.c
> +++ b/libavfilter/vf_scale.c
> @@ -269,6 +269,8 @@ revert:
>  
>  static av_cold int init_dict(AVFilterContext *ctx, AVDictionary **opts)
>  {
> +const AVClass *class = sws_get_class();
> +const AVDictionaryEntry *entry = NULL;
>  ScaleContext *scale = ctx->priv;
>  int ret;
>  
> @@ -312,15 +314,23 @@ static av_cold int init_dict(AVFilterContext *ctx, 
> AVDictionary **opts)
>  scale->flags = 0;
>  
>  if (scale->flags_str && *scale->flags_str) {
> -const AVClass *class = sws_get_class();
>  const AVOption*o = av_opt_find(&class, "sws_flags", NULL, 0,
> AV_OPT_SEARCH_FAKE_OBJ);
>  int ret = av_opt_eval_flags(&class, o, scale->flags_str, 
> &scale->flags);
>  if (ret < 0)
>  return ret;
>  }
> -scale->opts = *opts;
> -*opts = NULL;
> +FFSWAP(AVDictionary *, *opts, scale->opts);
> +/* Now move all the unrecognized options back to opts. */
> +while (entry = av_dict_get(scale->opts, "", entry, 
> AV_DICT_IGNORE_SUFFIX)) {
> +if (!av_opt_find(&class, entry->key, NULL, 0, 
> AV_OPT_SEARCH_FAKE_OBJ)) {
> +if ((ret = av_dict_set(opts, entry->key, entry->value, 0)) < 0 ||
> +(ret = av_dict_set(&scale->opts, entry->key, NULL, 0)) < 0)
> +return ret;
> +/* Removing the entry from scale->opts invalidated entry. */
> +entry = NULL;
> +}
> +}
>  
>  return 0;
>  }

I managed to understand what this code does, but I find it quite
confusing.

First, correct me if I am wrong, but I think the entry parameter to
av_dict_get() is always NULL.

Second, I think if you swap removing the option from scale->opts and
adding it to opts you can avoid strduping the key.

And is entry->value not leaking?

But more importantly, is it not a convoluted implementation that would
be simplified by using preinit like you did for sws?

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH v2 01/14] avfilter/vsrc_testsrc: Deduplicate AVClasses

2021-09-16 Thread Nicolas George
Andreas Rheinhardt (12021-09-14):
> Signed-off-by: Andreas Rheinhardt 
> ---
> I am not resending all the deduplication patches;
> they can be e.g. found here:
> https://github.com/mkver/FFmpeg/commits/avfilter_deduplication
> 
>  libavfilter/vsrc_testsrc.c | 47 +-
>  1 file changed, 16 insertions(+), 31 deletions(-)

The deduplication patches are ok IMHO.

I do not maintain zscale nor spp but see nothing obviously wrong with
the patch.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] regarding hiding AVStream.first_dts member into AVStreamInternal

2021-09-16 Thread James Almer
Chromium could instead keep track of that timestamp in their own code. 
It's what i did to remove the internal fields usage from our CLI tools. 
See https://github.com/FFmpeg/FFmpeg/commit/ab4f299e23


On 9/16/2021 8:24 AM, Marek Behún wrote:

Hello James, liberato,

James, your commit 591b88e6787c [1] in FFmpeg hid the first_dts and
cur_dts members of AVStream structure into the internal member, making
them inaccessible from third party code.

The reasoning in the commit message says:
   They are private fields, no reason to have them exposed in a public
   header.

But the first_dts member is used in Chromium, in
media/filters/ffmpeg_demuxer.cc [2] in method FFmpegDemuxer::InitializeTask.

Chromium solved this by adding av_stream_get_first_dts() method into
their bundled ffmpeg copy [3].

This makes it now impossible to link Chromium with system-ffmpeg, which
was till now a possibility in Gentoo, and which we would like to have
also in the future.

James, what do you think the best approach is here?
Should the FFmpeg commit be reverted to unhide this member?
Or should Chromium be fixed not to use this member? (I confess I did
not study the code much, so I don't know why it is needed, besides that
Chromium tries to compute start_time_ variable with it.)

Thank you.

Marek

[1] https://github.com/FFmpeg/FFmpeg/commit/591b88e6787c

[2]
https://chromium.googlesource.com/chromium/+/refs/heads/trunk/media/filters/ffmpeg_demuxer.cc

[3]
https://chromium.googlesource.com/chromium/third_party/ffmpeg/+/95aab0fd83619408995720ce53d7a74790580220%5E%21/



___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


[FFmpeg-devel] [PATCH] avcodec/mmaldec: fix decoder freeze when flushing

2021-09-16 Thread Ho Ming Shun
When mmal_flush is called before any packets are sent,
enabling/disabling the MMAL ports seems to cause the decoder to not
output any packets after flushing.

Bug is triggered when using 'mpv --hwdec=mmal --start=1 test.mp4'.

Proposed workaround in MPV: https://github.com/mpv-player/mpv/pull/9189

Signed-off-by: Ho Ming Shun 
---
 libavcodec/mmaldec.c | 38 +++---
 1 file changed, 23 insertions(+), 15 deletions(-)

diff --git a/libavcodec/mmaldec.c b/libavcodec/mmaldec.c
index 8c7d749742..6c3e5d99b6 100644
--- a/libavcodec/mmaldec.c
+++ b/libavcodec/mmaldec.c
@@ -145,19 +145,21 @@ static int ffmmal_set_ref(AVFrame *frame, FFPoolRef *pool,
 return 0;
 }
 
-static void ffmmal_stop_decoder(AVCodecContext *avctx)
+static void ffmmal_stop_decoder(AVCodecContext *avctx, int flush_ports)
 {
 MMALDecodeContext *ctx = avctx->priv_data;
 MMAL_COMPONENT_T *decoder = ctx->decoder;
 MMAL_BUFFER_HEADER_T *buffer;
 
-mmal_port_disable(decoder->input[0]);
-mmal_port_disable(decoder->output[0]);
-mmal_port_disable(decoder->control);
+if(flush_ports) {
+mmal_port_disable(decoder->input[0]);
+mmal_port_disable(decoder->output[0]);
+mmal_port_disable(decoder->control);
 
-mmal_port_flush(decoder->input[0]);
-mmal_port_flush(decoder->output[0]);
-mmal_port_flush(decoder->control);
+mmal_port_flush(decoder->input[0]);
+mmal_port_flush(decoder->output[0]);
+mmal_port_flush(decoder->control);
+}
 
 while ((buffer = mmal_queue_get(ctx->queue_decoded_frames)))
 mmal_buffer_header_release(buffer);
@@ -185,7 +187,7 @@ static av_cold int ffmmal_close_decoder(AVCodecContext 
*avctx)
 MMALDecodeContext *ctx = avctx->priv_data;
 
 if (ctx->decoder)
-ffmmal_stop_decoder(avctx);
+ffmmal_stop_decoder(avctx, 1);
 
 mmal_component_destroy(ctx->decoder);
 ctx->decoder = NULL;
@@ -456,14 +458,20 @@ static void ffmmal_flush(AVCodecContext *avctx)
 MMAL_COMPONENT_T *decoder = ctx->decoder;
 MMAL_STATUS_T status;
 
-ffmmal_stop_decoder(avctx);
+// MMAL will freeze if ports are enabled/disabled/flushed before
+// buffers are sent
+int flush_ports = ctx->packets_sent || ctx->extradata_sent;
 
-if ((status = mmal_port_enable(decoder->control, control_port_cb)))
-goto fail;
-if ((status = mmal_port_enable(decoder->input[0], input_callback)))
-goto fail;
-if ((status = mmal_port_enable(decoder->output[0], output_callback)))
-goto fail;
+ffmmal_stop_decoder(avctx, flush_ports);
+
+if(flush_ports) {
+if ((status = mmal_port_enable(decoder->control, control_port_cb)))
+goto fail;
+if ((status = mmal_port_enable(decoder->input[0], input_callback)))
+goto fail;
+if ((status = mmal_port_enable(decoder->output[0], output_callback)))
+goto fail;
+}
 
 return;
 
-- 
2.33.0

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 1/5] avcodec/libsvtav1: Fix redundant setting of caps_internal

2021-09-16 Thread lance . lmwang
On Thu, Sep 16, 2021 at 01:32:00PM +0200, Andreas Rheinhardt wrote:
> lance.lmw...@gmail.com:
> > From: Limin Wang 
> > 
> > Signed-off-by: Limin Wang 
> > ---
> >  libavcodec/libsvtav1.c | 3 +--
> >  1 file changed, 1 insertion(+), 2 deletions(-)
> > 
> > diff --git a/libavcodec/libsvtav1.c b/libavcodec/libsvtav1.c
> > index fabc4e6..82ae2b9 100644
> > --- a/libavcodec/libsvtav1.c
> > +++ b/libavcodec/libsvtav1.c
> > @@ -561,12 +561,11 @@ const AVCodec ff_libsvtav1_encoder = {
> >  .receive_packet = eb_receive_packet,
> >  .close  = eb_enc_close,
> >  .capabilities   = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_OTHER_THREADS,
> > -.caps_internal  = FF_CODEC_CAP_AUTO_THREADS,
> > +.caps_internal  = FF_CODEC_CAP_AUTO_THREADS | 
> > FF_CODEC_CAP_INIT_CLEANUP,
> >  .pix_fmts   = (const enum AVPixelFormat[]){ AV_PIX_FMT_YUV420P,
> >  AV_PIX_FMT_YUV420P10,
> >  AV_PIX_FMT_NONE },
> >  .priv_class = &class,
> >  .defaults   = eb_enc_defaults,
> > -.caps_internal  = FF_CODEC_CAP_INIT_CLEANUP,
> >  .wrapper_name   = "libsvtav1",
> >  };
> > 
> This is not redundant; the second initialization overrides the first. If
> I read validate_thread_parameters in pthread.c correctly, this means
> that 7d09579190d broke the ability to forward the threading parameters
> to the actual encoder. But it never did this anyway, so it doesn't
> matter. Should be fixed nevertheless.
Yes, It seems that the commit message is misleading, it's the second overrides 
the first.
I'll change it to " Fix override setting of caps_internal".

> 
> - Andreas
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

-- 
Thanks,
Limin Wang
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 1/2] libavfilter/x86/vf_gblur: fixed the fate-test failed on MacOS

2021-09-16 Thread Zhao Zhili



> On Sep 16, 2021, at 3:48 PM, Wu, Jianhua  wrote:
> 
> Jianhua wrote:
>> From: Wu, Jianhua 
>> Sent: Thursday, September 16, 2021 3:34 PM
>> To: ffmpeg-devel@ffmpeg.org
>> Cc: Wu, Jianhua 
>> Subject: [PATCH 1/2] libavfilter/x86/vf_gblur: fixed the fate-test failed on
>> MacOS
>> 
>> Signed-off-by: Wu Jianhua 
>> ---
> 
> Hi Zhili,
> 
> I have submitted the patches fixing the fate-test that failed on macOS. Could
> you help try to apply the patches to check if they work? Here is the link:
> http://ffmpeg.org/pipermail/ffmpeg-devel/2021-September/285376.html
> 
> And the illegal instructions:4 occurred might be the version of your GCC
> installed less than 4.7, which could be fixed by specific a compiler option,
> -mmacosx-version-min=10.x, by the way, the exact I used is 10.10. To enable
> it on FFmpeg, you might have to specify in the --extra-cflags option when
> configuring. But I recommend updating the GCC to the newest version, which
> can fix this as well. 

Thanks, the patch fixed both issues. By the way, the default toolchain
on macOS is clang, gcc has been dropped by Apple long time ago.

> 
> If there is another problem, feel free to let me know.
> 
> Best regards,
> Jianhua
> 
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
> 

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 4/5] avcodec/apedec: Fix integer overflow in intermediate

2021-09-16 Thread Michael Niedermayer
On Wed, Sep 15, 2021 at 10:09:49PM +0200, Andreas Rheinhardt wrote:
> Michael Niedermayer:
> > Fixes: signed integer overflow: 559334865 * 4 cannot be represented in type 
> > 'int'
> > Fixes: 
> > 37929/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-6751932295806976
> > 
> > Found-by: continuous fuzzing process 
> > https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> > Signed-off-by: Michael Niedermayer 
> > ---
> >  libavcodec/apedec.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/libavcodec/apedec.c b/libavcodec/apedec.c
> > index bf481ba3549..23318be0613 100644
> > --- a/libavcodec/apedec.c
> > +++ b/libavcodec/apedec.c
> > @@ -1337,7 +1337,7 @@ static void do_apply_filter(APEContext *ctx, int 
> > version, APEFilter *f,
> >  absres = FFABSU(res);
> >  if (absres)
> >  *f->adaptcoeffs = APESIGN(res) *
> > -  (8 << ((absres > f->avg * 3) + (absres > 
> > f->avg * 4 / 3)));
> > +  (8 << ((absres > f->avg * 3) + (absres > 
> > f->avg * 4LL / 3)));
> >  /* equivalent to the following code
> >  if (absres <= f->avg * 4 / 3)
> >  *f->adaptcoeffs = APESIGN(res) * 8;
> > 
> How about using f->avg + f->avg / 3? Given that you haven't modified the
> f->avg * 3 before, this should be save.

will apply this variant

thx

[...]

-- 
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

The day soldiers stop bringing you their problems is the day you have stopped 
leading them. They have either lost confidence that you can help or concluded 
you do not care. Either case is a failure of leadership. - Colin Powell


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 3/5] avformat/mvdec: Do not set invalid sample rate

2021-09-16 Thread Michael Niedermayer
On Thu, Sep 16, 2021 at 05:02:53PM +1000, Peter Ross wrote:
> On Wed, Sep 15, 2021 at 10:00:46PM +0200, Michael Niedermayer wrote:
> > Fixes: signed integer overflow: -682581959642593728 * 16 cannot be 
> > represented in type 'long'
> > Fixes: 
> > 37883/clusterfuzz-testcase-minimized-ffmpeg_dem_MV_fuzzer-5311691517198336
> > 
> > Found-by: continuous fuzzing process 
> > https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> > Signed-off-by: Michael Niedermayer 
> > ---
> >  libavformat/mvdec.c | 5 +++--
> >  1 file changed, 3 insertions(+), 2 deletions(-)
> > 
> > diff --git a/libavformat/mvdec.c b/libavformat/mvdec.c
> > index b1450e08da9..7573087c7cc 100644
> > --- a/libavformat/mvdec.c
> > +++ b/libavformat/mvdec.c
> > @@ -156,9 +156,10 @@ static int parse_audio_var(AVFormatContext *avctx, 
> > AVStream *st,
> >  } else if (!strcmp(name, "NUM_CHANNELS")) {
> >  return set_channels(avctx, st, var_read_int(pb, size));
> >  } else if (!strcmp(name, "SAMPLE_RATE")) {
> > -st->codecpar->sample_rate = var_read_int(pb, size);
> > -if (st->codecpar->sample_rate <= 0)
> > +int sample_rate = var_read_int(pb, size);
> > +if (sample_rate <= 0)
> >  return AVERROR_INVALIDDATA;
> > +st->codecpar->sample_rate = sample_rate;
> >  avpriv_set_pts_info(st, 33, 1, st->codecpar->sample_rate);
> >  } else if (!strcmp(name, "SAMPLE_WIDTH")) {
> >  uint64_t bpc = var_read_int(pb, size) * (uint64_t)8;
> 
> please apply

will do

thx

[...]

-- 
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Concerning the gods, I have no means of knowing whether they exist or not
or of what sort they may be, because of the obscurity of the subject, and
the brevity of human life -- Protagoras


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 1/5] avformat/sbgdec: Check for t0 overflow in expand_tseq()

2021-09-16 Thread Michael Niedermayer
On Thu, Sep 16, 2021 at 02:52:10PM +0200, Nicolas George wrote:
> Michael Niedermayer (12021-09-15):
> > Fixes: signed integer overflow: 4611686025627387904 + 4611686025627387904 
> > cannot be represented in type 'long'
> > Fixes: 
> > 35489/clusterfuzz-testcase-minimized-ffmpeg_dem_SBG_fuzzer-4862678601433088
> > 
> > Found-by: continuous fuzzing process 
> > https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> > Signed-off-by: Michael Niedermayer 
> > ---
> >  libavformat/sbgdec.c | 3 +++
> >  1 file changed, 3 insertions(+)
> 
> No objection.

will apply

thx

[...]

-- 
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Dictatorship naturally arises out of democracy, and the most aggravated
form of tyranny and slavery out of the most extreme liberty. -- Plato


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH v5 00/12] Subtitle Filtering

2021-09-16 Thread Michael Niedermayer
On Thu, Sep 16, 2021 at 12:20:17PM +0200, Nicolas George wrote:
> Hendrik Leppkes (12021-09-15):
> > Or perhaps some people have a day job, a life and other obligations
> > that prevent them from spending time on FFmpeg every day, especially
> > outside the weekend.
> > But no, that can't be it, surely we are all just evil. /s
> 
> > Nothing here is being invented. You are trying to push a major API
> > change to the core functionality of our libraries, these rules on how
> > to do API changes on that level (or any level, really) have been in
> > place and followed by everyone for years and years.
> > If you don't believe that, feel free to check how any other major API
> > change was handled in the past. Because the overall pattern is always
> > the same.
> > 
> > Do you think I'm the only one thinking that way, only because I spoke
> > up? The set certainly wouldn't have been applied if I hadn't said
> > anything, it would've just sat there. Maybe someone else might've come
> > forward eventually.
> 
> Thank you for having the patience and taking the time to explain this.
> 
> > As for the actual subject:
> > - Part1, add subtitles to AVFrame in avutil. You can move
> > enums/structs to avutil as needed (eg AVSubtitleRect or whatever), as
> > long as they are still available the same way (eg. avcodec.h should
> > include any new avutil header with them, so user code works without
> > changes), but moving functions is a more tricky business, so we
> > usually make new functions and cleanup their naming convention in the
> > process, but I don't think any functions are involved here right now.
> > 
> > To dive a bit deeper into this, redundant fields should be removed,
> > actual subtitle data should be refcounted (with AVBuffers in
> > AVFrame->buf), all frame functionality need to properly account for
> > the new data type.
> 
> There is another point to consider when designing subtitles in AVFrame:
> since we intend it not only as a general API cleanup but as a prelude to
> extending the API with filtering and such, we must not only think about
> what is needed now, we must think about what may be needed later.
> 
> The few examples I have, not excluding questions I have not thought of:
> 
> - Right now, we have text subtitles and bitmap subtitles. But do we want
>   to be able to handle mixed text and bitmap subtitles?
> 
>   To this, I would say: probably no. But we should give it a thought.

This question can also be asked seperatly for API and implementation,
for example we may support mixed bitmap/text subtitles in the API but
not the implementation. The advantage would be that the API does not
need to be changed if we want to add support on the implementation side

[...]


-- 
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Let us carefully observe those good qualities wherein our enemies excel us
and endeavor to excel them, by avoiding what is faulty, and imitating what
is excellent in them. -- Plutarch


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH v2 09/14] avfilter/vf_scale: Honour the AVFilter.init_dict documentation

2021-09-16 Thread Andreas Rheinhardt
Nicolas George:
> Andreas Rheinhardt (12021-09-14):
>> The documentation states that unrecognized options need to be returned
>> (just as av_opt_set_dict() would do). Yet the scale and scale2ref
>> filters didn't abide by this: They simply copied all options
>> and in case it contained an unrecognized option was among them,
>> they would error out later in config_props. This violates the
>> documentation of av_filter_init_dict().
>>
>> Fix this by only keeping the recognized options and returning
>> the unrecognized ones.
>>
>> Signed-off-by: Andreas Rheinhardt 
>> ---
>>  libavfilter/vf_scale.c | 16 +---
>>  1 file changed, 13 insertions(+), 3 deletions(-)
>>
>> diff --git a/libavfilter/vf_scale.c b/libavfilter/vf_scale.c
>> index a1902a13cf..c31b92b847 100644
>> --- a/libavfilter/vf_scale.c
>> +++ b/libavfilter/vf_scale.c
>> @@ -269,6 +269,8 @@ revert:
>>  
>>  static av_cold int init_dict(AVFilterContext *ctx, AVDictionary **opts)
>>  {
>> +const AVClass *class = sws_get_class();
>> +const AVDictionaryEntry *entry = NULL;
>>  ScaleContext *scale = ctx->priv;
>>  int ret;
>>  
>> @@ -312,15 +314,23 @@ static av_cold int init_dict(AVFilterContext *ctx, 
>> AVDictionary **opts)
>>  scale->flags = 0;
>>  
>>  if (scale->flags_str && *scale->flags_str) {
>> -const AVClass *class = sws_get_class();
>>  const AVOption*o = av_opt_find(&class, "sws_flags", NULL, 0,
>> AV_OPT_SEARCH_FAKE_OBJ);
>>  int ret = av_opt_eval_flags(&class, o, scale->flags_str, 
>> &scale->flags);
>>  if (ret < 0)
>>  return ret;
>>  }
>> -scale->opts = *opts;
>> -*opts = NULL;
>> +FFSWAP(AVDictionary *, *opts, scale->opts);
>> +/* Now move all the unrecognized options back to opts. */
>> +while (entry = av_dict_get(scale->opts, "", entry, 
>> AV_DICT_IGNORE_SUFFIX)) {
>> +if (!av_opt_find(&class, entry->key, NULL, 0, 
>> AV_OPT_SEARCH_FAKE_OBJ)) {
>> +if ((ret = av_dict_set(opts, entry->key, entry->value, 0)) < 0 
>> ||
>> +(ret = av_dict_set(&scale->opts, entry->key, NULL, 0)) < 0)
>> +return ret;
>> +/* Removing the entry from scale->opts invalidated entry. */
>> +entry = NULL;
>> +}
>> +}
>>  
>>  return 0;
>>  }
> 
> I managed to understand what this code does, but I find it quite
> confusing.
> 
> First, correct me if I am wrong, but I think the entry parameter to
> av_dict_get() is always NULL.

It is not. It is only reset when the last entry key was not recognized
as option; if it was recognized, entry is not NULL. In particular, in
the (hopefully) common case where all options are recognized, the
dictionary is just traversed linearly.

> 
> Second, I think if you swap removing the option from scale->opts and
> adding it to opts you can avoid strduping the key.
> 

You seem to think about "stealing" the key from the returned entry. This
does not work at all here, because for av_dict_set(&scale->opts,
entry->key, NULL, 0) to actually find the entry to remove, the entry
still needs to have the key. But if it has the key, it will free the key
upon deleting the entry.
More generally, stealing a key/value is forbidden: "The returned entry
key or value must not be changed, or it will cause undefined behavior."
Several parts of the AVDict API itself as well as user code requires
this to be so.
(I know you do this in tee.c where it happens to work; btw: I have a few
open patches for it that you may look at.)

> And is entry->value not leaking?

No, default behaviour for av_dict_set() (the flags AV_DICT_MULTIKEY,
AV_DICT_DONT_OVERWRITE, AV_DICT_APPEND modify this) is to replace the
first current entry whose key matches with the new entry (or just delete
said entry when the new value is NULL). When an entry is freed, both of
its key and value are freed, too.

> 
> But more importantly, is it not a convoluted implementation that would
> be simplified by using preinit like you did for sws?
> 

Unfortunately, the situation here is more complicated: We have not one
object on which we can simply apply options, but several (there are
separate scalers for interlaced pictures); and they are freed and
recreated on parameter changes. So one would at least need a persistent
master scale context which is used to apply the initial user options and
also used to copy options from it to the other scalers. At that point I
looked at the place where the options from the dictionary get applied
and saw that it is in the middle of applying other options, so I
believed that I could not maintain current behaviour by using a master
context and instead opted for this.

Now I look at this again and see that it is possible to override certain
parameters which should not be overrideable at all: E.g. overriding
dstw/dsth/srcw/srch easily leads to crashes. So this should be reordered
and in this case the master scaler approach mig

Re: [FFmpeg-devel] [PATCH v5 00/12] Subtitle Filtering

2021-09-16 Thread Soft Works



> -Original Message-
> From: ffmpeg-devel  On Behalf Of
> Nicolas George
> Sent: Thursday, 16 September 2021 12:20
> To: FFmpeg development discussions and patches  de...@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v5 00/12] Subtitle Filtering
> 

> There is another point to consider when designing subtitles in
> AVFrame:
> since we intend it not only as a general API cleanup but as a prelude
> to
> extending the API with filtering and such, we must not only think
> about
> what is needed now, we must think about what may be needed later.
> 
> The few examples I have, not excluding questions I have not thought
> of:
> 
> - Right now, we have text subtitles and bitmap subtitles. But do we
> want
>   to be able to handle mixed text and bitmap subtitles?
> 
>   To this, I would say: probably no. But we should give it a thought.

Each rect has a field of type enum AVSubtitleType, so this remains to
be possible, but I agree that it's unlikely to be required.

> - Number of rectangles. Currently, the decoders usually output a very
>   limited number of rectangles. OTOH, libass may output three alpha
> maps
>   per glyph, that makes potentially hundreds of rectangles.
> 
>   Will the data structure be able to handle this efficiently?

You are right about the amount of temporary images that libass
is creating (can go up to 100k for complex subs with animation).
But I don't think that there's any case where the data would 
leave a filter that way.

I have already:

- overlay_graphicsubs: The bitmaps are blended on a video 
  (2 inputs: video, subs[format=bitmap], 1 output: video)
- overlay_graphicsubs: The bitmaps are blended on a new
  empty transparent frame 
  (1 input, subs[format=bitmap], 1 output: video[with alpha])   

What might make sense as well, would be a conversion from 
text subs to graphic subs, but this can't be done by having 
a separate bitmap for each glyph which is most likely off-spec
for any graphic subtitle format.
So, even in that case, the individual libass rects won't leave
the filter as subtitle rects.

>   Consider also the issue of rectangles overlapping.
> 
> - Speaking of overlapping, we need a system to signal whether a new
>   subtitle frame should replace the current one (like dvdsub, srt,
> etc.)
>   or overlap with it (like ASS).

I don't think anything new is required as the existing logic is
preserved, no matter whether the subtitle frames drive through 
a filter chain or are forwarded to encoding directly.


> - Colorspace. All current bitmap subtitles formats are paletted. But
>   palette pixel formats are bad for many treatments. Inside a filter
>   chain, it would probably make sense to have them in a true color
>   format.

There are cases where this makes sense, for example to feed them
for overlay into a hardware filtering chain (via hwupload).

For these cases, I have graphicsub2video and textsub2video which
output transparent RGBA images.


For local (non-hardware) overlay it doesn't make sense to convert 
the palette bitmaps. 
For example, in overlay_graphicsubs, when overlaying over a yuv 
format, I'm just converting the palette to yuv for blending.

> - Global styles. ASS subtitles, in particular, contain reference to
>   globally-defined styles. How do we handle that in libavfilter? I
> have
>   no idea.

There's a shared reference pointing at the global ass header (string)
which each subtitle AVFrame is carrying.
If that is not available, ass_get_default_header (iirc) can be called.


> - Sparseness. Subtitles streams have gaps, and synchronization with
>   other streams requires a next frame, that can be minutes away or
> never
>   come. This needs to be solved in a way compatible with processing.

I have kept the heartbeat logic from your sub2video implementation.
It makes sense, is required and I can't think of any better way to
handle this. It's just renamed to subtitle_heartbeat and used for 
all subtitle formats.


> > - Part3, avfilter support for subtitles in AVFrames. At this point
> we
> > have a defined structure to store subtitles in AVFrames, and actual
> > code that can generate or consume them. When approaching this, the
> > same rules apply as before, existing subtitle functionality, as
> crude
> > as it may be, has to remain functional as exposed to the user.

Check.

> We need to decide which aspects of the subtitles formats are
> negotiated.
> 
> At least, obviously, the text or bitmap aspect will be, with a
> conversion filter inserted automatically where needed.

I'm inserting the graphicsub2video filter for keeping compatibility
with sub2video command lines, but I'm not a fan of any other 
automatic filter insertion. 
Let's talk about this in a separate conversation. 


> But depending on the answers to the questions in part 1, we may need
> to
> negotiate the pixel format and colorspace too.

At current, bitmap subtitles are always PAL8 and that should be
(remain to be) the meaning if SUBTITLE_BITMAP. 

It will be easy to add

Re: [FFmpeg-devel] Plans for libavfilter

2021-09-16 Thread Soft Works



> -Original Message-
> From: ffmpeg-devel  On Behalf Of
> Nicolas George
> Sent: Thursday, 16 September 2021 13:55
> To: ffmpeg-devel@ffmpeg.org
> Subject: [FFmpeg-devel] Plans for libavfilter
> 


> 
> 
> - Media type negotiation.
> 
>   That means that filters that do not care about the contents of what
>   they are filtering, filters that only act on the frame information
>   fields, can also not care if they are filtering audio or video or
>   other. There are a lot of these filters: pts manipulation, concat,
>   split, selection, etc.
> 
>   We cannot afford to have to update every one of them if we want to
> add
>   a new media type, so this is a prerequisite. And it is an aspect of
>   multi-stage negotiation, since pixel format or sample format can
> only
>   be negotiated once we know the media type.

I don't see any benefit in having filters that are handling variable 
media types. 
Even when the filter names are the same, the semantic of what these 
are doing often varies significantly.
They would often need to have options where some only apply to a certain
media type only.
Further, this increases error and confusion for command line use, as
this would lead to filters connecting in ways that aren't intended
at all, while making it hard for users to understand what's happening
and why it's failing.
It's already hard or impossible to see which formats are actually
negotiated between filters. Having filters that can do multiple 
media types on a single pad would cause ultimate confusion for users
and make things happen that are undesired, unexpected and intransparent.

Kind regards,
softworkz


___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] Plans for libavfilter

2021-09-16 Thread Xiang Xiao
We are facing similar issues after working with libavfilter for more than
one year. We are implementing a private solution for:

   - Partial graph configuration
   - Partial graph re-configuration
   - Global running API

It's glad to see other people have the same feeling of the shortcoming
design in avfilter and improve it in a general way.

On Thu, Sep 16, 2021 at 7:55 PM Nicolas George  wrote:

> Since it is not healthy to keep everything for myself, here is a summary
> of the projects I have in mind for the enhancement of libavfilter.
>
> The thing is, there are a lot of dependencies between these projects.
> Working on them in a proper order would make achieving the goals easier.
> On the other hand, if task B depends on task A, then working B without
> addressing A involves a little more work for B, but also a lot more work
> for A, and a lot more work for everything else that depends on A.
>
> This is one reason I am writing this mail: to make clear what needs to
> be done, and how tasks depend on each-other and fit together.
>
> The earliest tasks are not very sexy. They are about making the code
> clearer or more robust and generic rather than adding features. It is
> always boring. But they are necessary, because parts of the process is
> quite shaky, and we cannot afford to build on shaky code.
>
> If you want to help, please consider this and you will be welcome.
>
> Now that is established the specifics:
>
> - Multi-stage negotiation.
>
>   That means negotiating properties that depend on other properties. I
>   think the colorspace stuff is in that case, it cannot be negotiated
>   until the pixel format is known.
>
> - Media type negotiation.
>
>   That means that filters that do not care about the contents of what
>   they are filtering, filters that only act on the frame information
>   fields, can also not care if they are filtering audio or video or
>   other. There are a lot of these filters: pts manipulation, concat,
>   split, selection, etc.
>
>   We cannot afford to have to update every one of them if we want to add
>   a new media type, so this is a prerequisite. And it is an aspect of
>   multi-stage negotiation, since pixel format or sample format can only
>   be negotiated once we know the media type.
>
> - Partial graph configuration.
>
>   Right now, the negotiation happens in steps, first query_formats then
>   merge, then pick. But pick can be done on some links as soon as the
>   formats are merged to a singleton while other filters are still
>   delaying their query_formats, and once pick is done, the graph can
>   start running. That would allow filters with constraints more complex
>   than what AVFilterFormats can express. (We have a mechanism for that,
>   used by amerge and one or two other filters; I introduced it early and
>   it was a mistake, it is much too fragile and only converges in the
>   simplest cases.)
>
> - Partial graph re-configuration.
>
>   If we can run a graph when not all filters are configured, then we can
>   de-configure and re-configure part of a graph. That would bring us the
>   support for changes in pixel format.
>
> - Global running API.
>
>   Right now, we make libavfilter activate filters by pumping
>   request_frame() on one of the buffersinks, more or less randomly, and
>   we hope it will result in something useful. We need a better API, one
>   where we say libavfilter "start running these graphs", we push frames
>   on input as we have them, we are notified when frames arrive on
>   output.
>
> - Inter-filter threading.
>
>   This is a big one, and quite self-explanatory. It goes hand to hand
>   with a global running API, because "start running these graphs" also
>   means "start as many threads as you need and keep them ready".
>
>   Note that the network protocols in libavformat requires the same
>   thing. And I really do not want yet two other frigging threading
>   subsystems. This is why I have started slowly working on an unique
>   system suited for all needs, starting with both libavformat and
>   libavfilter but keeping also libavcodec in mind.
>
> - Out-of-band side-data.
>
>   This is a mechanism to notify filters that the work they are currently
>   doing may become irrelevant, or other kinds of urgent information.
>   Filters can ignore it, and just do the work for nothing. But smarter
>   filters can take it into account and just skip until it becomes
>   relevant again.
>
> - Seeking.
>
>   This is when applications notify the outputs they want to jump at a
>   specified time, and all the filters in the graph make sure the
>   instruction reach the inputs.
>
>   This is probably rather easy and does not depend on many other things.
>   Out-of-band side-data is meant for that, among other things: notify a
>   filter "you can skip processing the frames you have queued, because we
>   are seeking anyway"; and if seeking fails, notify the user as fast as
>   possible "we tried seeking, it will not happen".
>
>   A fe

Re: [FFmpeg-devel] Plans for libavfilter

2021-09-16 Thread Michael Niedermayer
On Thu, Sep 16, 2021 at 01:55:17PM +0200, Nicolas George wrote:
> Since it is not healthy to keep everything for myself, here is a summary
> of the projects I have in mind for the enhancement of libavfilter.
> 
> The thing is, there are a lot of dependencies between these projects.
> Working on them in a proper order would make achieving the goals easier.
> On the other hand, if task B depends on task A, then working B without
> addressing A involves a little more work for B, but also a lot more work
> for A, and a lot more work for everything else that depends on A.
> 
> This is one reason I am writing this mail: to make clear what needs to
> be done, and how tasks depend on each-other and fit together.
> 
> The earliest tasks are not very sexy. They are about making the code
> clearer or more robust and generic rather than adding features. It is
> always boring. But they are necessary, because parts of the process is
> quite shaky, and we cannot afford to build on shaky code.
> 
> If you want to help, please consider this and you will be welcome.
> 
> Now that is established the specifics:
> 
> - Multi-stage negotiation.
> 
>   That means negotiating properties that depend on other properties. I
>   think the colorspace stuff is in that case, it cannot be negotiated
>   until the pixel format is known.
> 

> - Media type negotiation.
> 
>   That means that filters that do not care about the contents of what
>   they are filtering, filters that only act on the frame information
>   fields, can also not care if they are filtering audio or video or
>   other. There are a lot of these filters: pts manipulation, concat,
>   split, selection, etc.

+1
the current need to duplicate filters per media type is ugly.
Also "negotiation" is a strong term for this. Filters in general
have either one media type on their pads or pass the type from
input to output and support any. In that sense this is not hard
to solve, it should not be dependant on anything else


[...]

> 
> - Partial graph configuration.
> 
>   Right now, the negotiation happens in steps, first query_formats then
>   merge, then pick. But pick can be done on some links as soon as the
>   formats are merged to a singleton while other filters are still
>   delaying their query_formats, and once pick is done, the graph can
>   start running. That would allow filters with constraints more complex
>   than what AVFilterFormats can express. (We have a mechanism for that,
>   used by amerge and one or two other filters; I introduced it early and
>   it was a mistake, it is much too fragile and only converges in the
>   simplest cases.)
> 
> - Partial graph re-configuration.
> 
>   If we can run a graph when not all filters are configured, then we can
>   de-configure and re-configure part of a graph. That would bring us the
>   support for changes in pixel format.

Maybe some orthogonal direction could be considered too
completely seperating graph configuration and using some more generic
algorithm to solve it.
What we have basically are a bunch of constraints and then we need to
color the links between vertexes so as to minimize a weighted average
of computational complexity and quality loss. And each link can have
also different colors on both ends which would correspond to an inserted
convert filter. The "color" here would be a vector of all elements that
need configuration.

This view if it works would also allow very easy changes to the things
needing configuration. For example in this framework one could replace
a colorspace enum by a AxB element matrix and nothing would change on
the solver it would just need a different quality estimation function.

That means all the heuristics would be isolated in the function which
is to be minimized. While the solver would just deal with generic 
values of some kind like ints


[...]

-- 
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

The smallest minority on earth is the individual. Those who deny 
individual rights cannot claim to be defenders of minorities. - Ayn Rand


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


[FFmpeg-devel] [PATCH] avformat/mvdec: propagate the errors returned by parse_audio_var()

2021-09-16 Thread James Almer
Signed-off-by: James Almer 
---
 libavformat/mvdec.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavformat/mvdec.c b/libavformat/mvdec.c
index 7573087c7c..0b4aa1f18f 100644
--- a/libavformat/mvdec.c
+++ b/libavformat/mvdec.c
@@ -387,7 +387,7 @@ static int mv_read_header(AVFormatContext *avctx)
 if (!ast)
 return AVERROR(ENOMEM);
 ast->codecpar->codec_type = AVMEDIA_TYPE_AUDIO;
-if ((read_table(avctx, ast, parse_audio_var)) < 0)
+if ((ret = read_table(avctx, ast, parse_audio_var)) < 0)
 return ret;
 if (mv->acompression == 100 &&
 mv->aformat == AUDIO_FORMAT_SIGNED &&
-- 
2.33.0

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH] avformat/mvdec: propagate the errors returned by parse_audio_var()

2021-09-16 Thread Andreas Rheinhardt
James Almer:
> Signed-off-by: James Almer 
> ---
>  libavformat/mvdec.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/libavformat/mvdec.c b/libavformat/mvdec.c
> index 7573087c7c..0b4aa1f18f 100644
> --- a/libavformat/mvdec.c
> +++ b/libavformat/mvdec.c
> @@ -387,7 +387,7 @@ static int mv_read_header(AVFormatContext *avctx)
>  if (!ast)
>  return AVERROR(ENOMEM);
>  ast->codecpar->codec_type = AVMEDIA_TYPE_AUDIO;
> -if ((read_table(avctx, ast, parse_audio_var)) < 0)
> +if ((ret = read_table(avctx, ast, parse_audio_var)) < 0)
>  return ret;
>  if (mv->acompression == 100 &&
>  mv->aformat == AUDIO_FORMAT_SIGNED &&
> 

The title is misleading: it should be something like "Don't signal
success upon error". Just reading your title would indicate that we are
just making error codes up instead of forwarding them.

- Andreas
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH] avformat/mvdec: propagate the errors returned by parse_audio_var()

2021-09-16 Thread James Almer

On 9/16/2021 4:39 PM, Andreas Rheinhardt wrote:

James Almer:

Signed-off-by: James Almer 
---
  libavformat/mvdec.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavformat/mvdec.c b/libavformat/mvdec.c
index 7573087c7c..0b4aa1f18f 100644
--- a/libavformat/mvdec.c
+++ b/libavformat/mvdec.c
@@ -387,7 +387,7 @@ static int mv_read_header(AVFormatContext *avctx)
  if (!ast)
  return AVERROR(ENOMEM);
  ast->codecpar->codec_type = AVMEDIA_TYPE_AUDIO;
-if ((read_table(avctx, ast, parse_audio_var)) < 0)
+if ((ret = read_table(avctx, ast, parse_audio_var)) < 0)
  return ret;
  if (mv->acompression == 100 &&
  mv->aformat == AUDIO_FORMAT_SIGNED &&



The title is misleading: it should be something like "Don't signal
success upon error". Just reading your title would indicate that we are
just making error codes up instead of forwarding them.


I don't really think it's misleading, but sure, I'll use your suggestion.



- Andreas
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".



___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


[FFmpeg-devel] [PATCH v6 00/11] Subtitle Filtering

2021-09-16 Thread Soft Works
v6 Update:

- Implements all given feedback from reviews (thanks!)
- Add AVSubtitle fields to AVFrame, keeping AVSubtitle unchanged
  for compatibility and future deprecation
- Use subtitle frames internally and for filtering 
- Retain compatibility with legacy subtitle implementation
  through a number of compatibility conversion methods
- Reordered and reorganized commits, each one passing 
  FATE now
- Document new public API functions
- Many more detail changes


v5 Update:

- Merge AVSubtitle into AVFrame
- Move FATE test adjustments to corresponding commit
- Move documentation updates to corresponding filter commits
- Remove mediatype parameter from av_frame_get_buffer2

(still need some advice for splitting the commits in a way that 
every single one will pass FATE)

v4 Update:

- Re-Sending due to Patchwork having failed to parse my patchset
  There seems to be a bug in Patchwork when parallel processing is 
  enabled. 
  This time, I'll send the e-mails slowly, one after another.

v3 Update:

- Reworked, revised, rebased, reorganized, refactored
- No more prototype/test-style code
- Additional subtitle filters added
- Filter documentation added
- Adjusted FATE tests


This patchset is about introducing filtering support for subtitles.

The current sub2video "hack" implementation is slow, ineffective and 
limited in capabilities.

=> This patchset introduces true subtitle filtering

These new filters are included:

- overlay_graphicsubs (VS -> V)
  Overlay graphic subtitles onto a video stream

- graphicsub2video {S -> V)
  Converts graphic subtitles to (transparent) video frames
  
- overlay_textsubs {VS -> V)
  Overlay text subtitles onto a video stream.
  
- textsubs2video {S -> V)
  Converts text subtitles to video frames
  
- textmod {S -> S)
  Modify subtitle text in a number of ways

- stripstyles {S -> S)
  Remove all inline styles from subtitle events

Regards,
softworkz


softworkz (11):
  global: Prepare AVFrame for subtitle handling
  fftools/play,probe: Adjust for subtitle changes
  avfilter/subtitles: Add subtitles.c for subtitle frame allocation
  avfilter/avfilter: Handle subtitle frames
  avfilter/sbuffer: Add sbuffersrv and sbuffersink filters
  avfilter/overlay_graphicsubs: Add overlay_graphicsubs and
graphicsub2video filters
  fftools/ffmpeg: Replace sub2video with subtitle frame filtering
  avfilter/overlay_textsubs: Add overlay_textsubs and textsubs2video
filters
  avfilter/textmod: Add textmod filter
  avcodec/ass_split: Extend ass dialog parsing
  avfilter/stripstyles: Add stripstyles filter

 configure |   4 +-
 doc/filters.texi  | 261 
 fftools/ffmpeg.c  | 459 ++
 fftools/ffmpeg.h  |  14 +-
 fftools/ffmpeg_filter.c   | 198 --
 fftools/ffmpeg_hw.c   |   2 +-
 fftools/ffmpeg_opt.c  |   3 +-
 fftools/ffplay.c  |  50 +-
 fftools/ffprobe.c |  49 +-
 libavcodec/ass_split.c|  12 +-
 libavcodec/ass_split.h|   2 +
 libavcodec/avcodec.c  |  19 -
 libavcodec/avcodec.h  | 105 ++--
 libavcodec/decode.c   |  24 +-
 libavcodec/pgssubdec.c|   1 +
 libavcodec/utils.c|  11 +
 libavfilter/Makefile  |   9 +
 libavfilter/allfilters.c  |  16 +-
 libavfilter/avfilter.c|  30 +-
 libavfilter/avfiltergraph.c   |   5 +
 libavfilter/buffersink.c  |  63 ++
 libavfilter/buffersink.h  |  15 +
 libavfilter/buffersrc.c   |  72 +++
 libavfilter/buffersrc.h   |   1 +
 libavfilter/formats.c |  14 +
 libavfilter/formats.h |   3 +
 libavfilter/internal.h|   1 +
 libavfilter/sf_stripstyles.c  | 211 +++
 libavfilter/sf_textmod.c  | 372 +++
 libavfilter/subtitles.c   |  61 ++
 libavfilter/subtitles.h   |  44 ++
 libavfilter/version.h |   2 +-
 libavfilter/vf_overlay_graphicsubs.c  | 727 ++
 libavfilter/vf_overlay_textsubs.c | 613 ++
 libavfilter/vf_subtitles.c|  28 +-
 libavformat/utils.c   |   1 +
 libavutil/Makefile|   2 +
 libavutil/frame.c | 184 +-
 libavutil/frame.h |  93 ++-
 libavutil/subfmt.c| 221 +++
 libavutil/subfmt.h| 186 ++
 libavutil/version.h   |   2 +-
 tests/ref/fate/filter-overlay-dvdsub-2397 | 181 +++---
 tests/ref/fate/sub-dvb| 162 ++---
 tests/ref/fate/sub2video  |  44 -

[FFmpeg-devel] [PATCH v6 01/11] global: Prepare AVFrame for subtitle handling

2021-09-16 Thread Soft Works
Root commit for adding subtitle filtering capabilities.
In detail:

- Add type (AVMediaType) field to AVFrame
  Replaces previous way of distinction which was based on checking
  width and height to determine whether a frame is audio or video
- Add subtitle fields to AVFrame
- Add new struct AVSubtitleArea, similar to AVSubtitleRect, but different
  allocation logic. Cannot and must not be used interchangeably, hence
  the new struct
- Move enum AVSubtitleType, AVSubtitle and AVSubtitleRect to avutil
- Add public-named members to enum AVSubtitleType (AV_SUBTITLE_FMT_)
- Add avcodec_decode_subtitle3 which takes subtitle frames,
  serving as compatibility shim to legacy subtitle decoding
- Add additional methods for conversion between old and new API

Signed-off-by: softworkz 
---
 libavcodec/avcodec.c   |  19 
 libavcodec/avcodec.h   | 105 +++---
 libavcodec/decode.c|  24 +++-
 libavcodec/pgssubdec.c |   1 +
 libavcodec/utils.c |  11 ++
 libavfilter/vf_subtitles.c |  28 ++---
 libavformat/utils.c|   1 +
 libavutil/Makefile |   2 +
 libavutil/frame.c  | 184 +++---
 libavutil/frame.h  |  93 +++-
 libavutil/subfmt.c | 221 +
 libavutil/subfmt.h | 186 +++
 libavutil/version.h|   2 +-
 13 files changed, 757 insertions(+), 120 deletions(-)
 create mode 100644 libavutil/subfmt.c
 create mode 100644 libavutil/subfmt.h

diff --git a/libavcodec/avcodec.c b/libavcodec/avcodec.c
index 2dd7dd84e0..963f52c4bd 100644
--- a/libavcodec/avcodec.c
+++ b/libavcodec/avcodec.c
@@ -426,25 +426,6 @@ void avcodec_flush_buffers(AVCodecContext *avctx)
 av_bsf_flush(avci->bsf);
 }
 
-void avsubtitle_free(AVSubtitle *sub)
-{
-int i;
-
-for (i = 0; i < sub->num_rects; i++) {
-av_freep(&sub->rects[i]->data[0]);
-av_freep(&sub->rects[i]->data[1]);
-av_freep(&sub->rects[i]->data[2]);
-av_freep(&sub->rects[i]->data[3]);
-av_freep(&sub->rects[i]->text);
-av_freep(&sub->rects[i]->ass);
-av_freep(&sub->rects[i]);
-}
-
-av_freep(&sub->rects);
-
-memset(sub, 0, sizeof(*sub));
-}
-
 av_cold int avcodec_close(AVCodecContext *avctx)
 {
 int i;
diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index ffd58c333f..de23067d29 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -35,6 +35,7 @@
 #include "libavutil/frame.h"
 #include "libavutil/log.h"
 #include "libavutil/pixfmt.h"
+#include "libavutil/subfmt.h"
 #include "libavutil/rational.h"
 
 #include "codec.h"
@@ -1670,7 +1671,7 @@ typedef struct AVCodecContext {
 
 /**
  * Header containing style information for text subtitles.
- * For SUBTITLE_ASS subtitle type, it should contain the whole ASS
+ * For AV_SUBTITLE_FMT_ASS subtitle type, it should contain the whole ASS
  * [Script Info] and [V4+ Styles] section, plus the [Events] line and
  * the Format line following. It shouldn't include any Dialogue line.
  * - encoding: Set/allocated/freed by user (before avcodec_open2())
@@ -2233,63 +2234,8 @@ typedef struct AVHWAccel {
  * @}
  */
 
-enum AVSubtitleType {
-SUBTITLE_NONE,
-
-SUBTITLE_BITMAP,///< A bitmap, pict will be set
-
-/**
- * Plain text, the text field must be set by the decoder and is
- * authoritative. ass and pict fields may contain approximations.
- */
-SUBTITLE_TEXT,
-
-/**
- * Formatted text, the ass field must be set by the decoder and is
- * authoritative. pict and text fields may contain approximations.
- */
-SUBTITLE_ASS,
-};
-
 #define AV_SUBTITLE_FLAG_FORCED 0x0001
 
-typedef struct AVSubtitleRect {
-int x; ///< top left corner  of pict, undefined when pict is not 
set
-int y; ///< top left corner  of pict, undefined when pict is not 
set
-int w; ///< widthof pict, undefined when pict is not 
set
-int h; ///< height   of pict, undefined when pict is not 
set
-int nb_colors; ///< number of colors in pict, undefined when pict is not 
set
-
-/**
- * data+linesize for the bitmap of this subtitle.
- * Can be set for text/ass as well once they are rendered.
- */
-uint8_t *data[4];
-int linesize[4];
-
-enum AVSubtitleType type;
-
-char *text; ///< 0 terminated plain UTF-8 text
-
-/**
- * 0 terminated ASS/SSA compatible event line.
- * The presentation of this is unaffected by the other values in this
- * struct.
- */
-char *ass;
-
-int flags;
-} AVSubtitleRect;
-
-typedef struct AVSubtitle {
-uint16_t format; /* 0 = graphics */
-uint32_t start_display_time; /* relative to packet pts, in ms */
-uint32_t end_display_time; /* relative to packet pts, in ms */
-unsigned num_rects;
-AVSubtitleRect **rects;
-int64_t pts;   

[FFmpeg-devel] [PATCH v6 02/11] fftools/play, probe: Adjust for subtitle changes

2021-09-16 Thread Soft Works
Signed-off-by: softworkz 
---
 fftools/ffplay.c  | 50 +++
 fftools/ffprobe.c | 49 ++
 2 files changed, 62 insertions(+), 37 deletions(-)

diff --git a/fftools/ffplay.c b/fftools/ffplay.c
index ccea0e4578..08e5d53f9c 100644
--- a/fftools/ffplay.c
+++ b/fftools/ffplay.c
@@ -152,7 +152,6 @@ typedef struct Clock {
 /* Common struct for handling all types of decoded data and allocated render 
buffers. */
 typedef struct Frame {
 AVFrame *frame;
-AVSubtitle sub;
 int serial;
 double pts;   /* presentation timestamp for the frame */
 double duration;  /* estimated duration of the frame */
@@ -586,7 +585,7 @@ static int decoder_init(Decoder *d, AVCodecContext *avctx, 
PacketQueue *queue, S
 return 0;
 }
 
-static int decoder_decode_frame(Decoder *d, AVFrame *frame, AVSubtitle *sub) {
+static int decoder_decode_frame(Decoder *d, AVFrame *frame) {
 int ret = AVERROR(EAGAIN);
 
 for (;;) {
@@ -654,7 +653,7 @@ static int decoder_decode_frame(Decoder *d, AVFrame *frame, 
AVSubtitle *sub) {
 
 if (d->avctx->codec_type == AVMEDIA_TYPE_SUBTITLE) {
 int got_frame = 0;
-ret = avcodec_decode_subtitle2(d->avctx, sub, &got_frame, d->pkt);
+ret = avcodec_decode_subtitle3(d->avctx, frame, &got_frame, 
d->pkt);
 if (ret < 0) {
 ret = AVERROR(EAGAIN);
 } else {
@@ -683,7 +682,6 @@ static void decoder_destroy(Decoder *d) {
 static void frame_queue_unref_item(Frame *vp)
 {
 av_frame_unref(vp->frame);
-avsubtitle_free(&vp->sub);
 }
 
 static int frame_queue_init(FrameQueue *f, PacketQueue *pktq, int max_size, 
int keep_last)
@@ -981,7 +979,7 @@ static void video_image_display(VideoState *is)
 if (frame_queue_nb_remaining(&is->subpq) > 0) {
 sp = frame_queue_peek(&is->subpq);
 
-if (vp->pts >= sp->pts + ((float) sp->sub.start_display_time / 
1000)) {
+if (vp->pts >= sp->pts + ((float) sp->frame->subtitle_start_time / 
1000)) {
 if (!sp->uploaded) {
 uint8_t* pixels[4];
 int pitch[4];
@@ -993,8 +991,8 @@ static void video_image_display(VideoState *is)
 if (realloc_texture(&is->sub_texture, 
SDL_PIXELFORMAT_ARGB, sp->width, sp->height, SDL_BLENDMODE_BLEND, 1) < 0)
 return;
 
-for (i = 0; i < sp->sub.num_rects; i++) {
-AVSubtitleRect *sub_rect = sp->sub.rects[i];
+for (i = 0; i < sp->frame->num_subtitle_areas; i++) {
+AVSubtitleArea *sub_rect = 
sp->frame->subtitle_areas[i];
 
 sub_rect->x = av_clip(sub_rect->x, 0, sp->width );
 sub_rect->y = av_clip(sub_rect->y, 0, sp->height);
@@ -1041,13 +1039,15 @@ static void video_image_display(VideoState *is)
 int i;
 double xratio = (double)rect.w / (double)sp->width;
 double yratio = (double)rect.h / (double)sp->height;
-for (i = 0; i < sp->sub.num_rects; i++) {
-SDL_Rect *sub_rect = (SDL_Rect*)sp->sub.rects[i];
-SDL_Rect target = {.x = rect.x + sub_rect->x * xratio,
-   .y = rect.y + sub_rect->y * yratio,
-   .w = sub_rect->w * xratio,
-   .h = sub_rect->h * yratio};
-SDL_RenderCopy(renderer, is->sub_texture, sub_rect, &target);
+for (i = 0; i < sp->frame->num_subtitle_areas; i++) {
+AVSubtitleArea *area = sp->frame->subtitle_areas[i];
+SDL_Rect sub_rect = { .x = area->x, .y = area->y,
+  .w = area->w, .h = area->h};
+SDL_Rect target = {.x = rect.x + sub_rect.x * xratio,
+   .y = rect.y + sub_rect.y * yratio,
+   .w = sub_rect.w * xratio,
+   .h = sub_rect.h * yratio};
+SDL_RenderCopy(renderer, is->sub_texture, &sub_rect, &target);
 }
 #endif
 }
@@ -1651,13 +1651,13 @@ retry:
 sp2 = NULL;
 
 if (sp->serial != is->subtitleq.serial
-|| (is->vidclk.pts > (sp->pts + ((float) 
sp->sub.end_display_time / 1000)))
-|| (sp2 && is->vidclk.pts > (sp2->pts + ((float) 
sp2->sub.start_display_time / 1000
+|| (is->vidclk.pts > (sp->pts + ((float) 
sp->frame->subtitle_end_time / 1000)))
+|| (sp2 && is->vidclk.pts > (sp2->pts + ((float) 
sp2->frame->subtitle_start_time / 1000
 {
 if (sp->uploaded) {
 int i;
-for (i = 0; i < sp->sub.num_rects; i++) {
-AVSubtitleRect *sub_rect = sp->sub.rects[i]

[FFmpeg-devel] [PATCH v6 03/11] avfilter/subtitles: Add subtitles.c for subtitle frame allocation

2021-09-16 Thread Soft Works
Analog to avfilter/video.c and avfilter/audio.c

Signed-off-by: softworkz 
---
 libavfilter/Makefile|  1 +
 libavfilter/avfilter.c  |  4 +++
 libavfilter/internal.h  |  1 +
 libavfilter/subtitles.c | 61 +
 libavfilter/subtitles.h | 44 +
 5 files changed, 111 insertions(+)
 create mode 100644 libavfilter/subtitles.c
 create mode 100644 libavfilter/subtitles.h

diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index f059f3fef8..041d3c5382 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -19,6 +19,7 @@ OBJS = allfilters.o   
  \
framequeue.o \
graphdump.o  \
graphparser.o\
+   subtitles.o  \
video.o  \
 
 OBJS-$(HAVE_THREADS) += pthread.o
diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
index c614eb0740..ac46bde782 100644
--- a/libavfilter/avfilter.c
+++ b/libavfilter/avfilter.c
@@ -43,6 +43,7 @@
 #include "formats.h"
 #include "framepool.h"
 #include "internal.h"
+#include "subtitles.h"
 
 #include "libavutil/ffversion.h"
 const char av_filter_ffversion[] = "FFmpeg version " FFMPEG_VERSION;
@@ -1474,6 +1475,9 @@ int ff_inlink_make_frame_writable(AVFilterLink *link, 
AVFrame **rframe)
 case AVMEDIA_TYPE_AUDIO:
 out = ff_get_audio_buffer(link, frame->nb_samples);
 break;
+case AVMEDIA_TYPE_SUBTITLE:
+out = ff_get_subtitles_buffer(link, link->format);
+break;
 default:
 return AVERROR(EINVAL);
 }
diff --git a/libavfilter/internal.h b/libavfilter/internal.h
index e7c154aff0..8977dda2b3 100644
--- a/libavfilter/internal.h
+++ b/libavfilter/internal.h
@@ -90,6 +90,7 @@ struct AVFilterPad {
 union {
 AVFrame *(*video)(AVFilterLink *link, int w, int h);
 AVFrame *(*audio)(AVFilterLink *link, int nb_samples);
+AVFrame *(*subtitle)(AVFilterLink *link, int format);
 } get_buffer;
 
 /**
diff --git a/libavfilter/subtitles.c b/libavfilter/subtitles.c
new file mode 100644
index 00..fd2e7c3c0f
--- /dev/null
+++ b/libavfilter/subtitles.c
@@ -0,0 +1,61 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "libavutil/common.h"
+
+#include "subtitles.h"
+#include "avfilter.h"
+#include "internal.h"
+
+
+AVFrame *ff_null_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+return ff_get_subtitles_buffer(link->dst->outputs[0], format);
+}
+
+AVFrame *ff_default_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+AVFrame *frame;
+
+frame = av_frame_alloc();
+if (!frame)
+return NULL;
+
+frame->format = format;
+frame->type = AVMEDIA_TYPE_SUBTITLE;
+
+if (av_frame_get_buffer2(frame, 0) < 0)
+return NULL;
+
+return frame;
+}
+
+AVFrame *ff_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+AVFrame *ret = NULL;
+
+if (link->dstpad->get_buffer.subtitle)
+ret = link->dstpad->get_buffer.subtitle(link, format);
+
+if (!ret)
+ret = ff_default_get_subtitles_buffer(link, format);
+
+return ret;
+}
diff --git a/libavfilter/subtitles.h b/libavfilter/subtitles.h
new file mode 100644
index 00..d3d5491652
--- /dev/null
+++ b/libavfilter/subtitles.h
@@ -0,0 +1,44 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Pub

[FFmpeg-devel] [PATCH v6 04/11] avfilter/avfilter: Handle subtitle frames

2021-09-16 Thread Soft Works
Signed-off-by: softworkz 
---
 libavfilter/avfilter.c  |  8 +---
 libavfilter/avfiltergraph.c |  5 +
 libavfilter/formats.c   | 14 ++
 libavfilter/formats.h   |  3 +++
 4 files changed, 27 insertions(+), 3 deletions(-)

diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
index ac46bde782..3f303fbafa 100644
--- a/libavfilter/avfilter.c
+++ b/libavfilter/avfilter.c
@@ -57,7 +57,8 @@ void ff_tlog_ref(void *ctx, AVFrame *ref, int end)
 ref->linesize[0], ref->linesize[1], ref->linesize[2], 
ref->linesize[3],
 ref->pts, ref->pkt_pos);
 
-if (ref->width) {
+switch(ref->type) {
+case AVMEDIA_TYPE_VIDEO:
 ff_tlog(ctx, " a:%d/%d s:%dx%d i:%c iskey:%d type:%c",
 ref->sample_aspect_ratio.num, ref->sample_aspect_ratio.den,
 ref->width, ref->height,
@@ -65,12 +66,13 @@ void ff_tlog_ref(void *ctx, AVFrame *ref, int end)
 ref->top_field_first ? 'T' : 'B',/* Top / Bottom */
 ref->key_frame,
 av_get_picture_type_char(ref->pict_type));
-}
-if (ref->nb_samples) {
+break;
+case AVMEDIA_TYPE_AUDIO:
 ff_tlog(ctx, " cl:%"PRId64"d n:%d r:%d",
 ref->channel_layout,
 ref->nb_samples,
 ref->sample_rate);
+break;
 }
 
 ff_tlog(ctx, "]%s", end ? "\n" : "");
diff --git a/libavfilter/avfiltergraph.c b/libavfilter/avfiltergraph.c
index 45b028cd9c..7a5a4ea419 100644
--- a/libavfilter/avfiltergraph.c
+++ b/libavfilter/avfiltergraph.c
@@ -314,6 +314,8 @@ static int filter_link_check_formats(void *log, 
AVFilterLink *link, AVFilterForm
 return ret;
 break;
 
+case AVMEDIA_TYPE_SUBTITLE:
+return 0;
 default:
 av_assert0(!"reached");
 }
@@ -444,6 +446,9 @@ static int query_formats(AVFilterGraph *graph, void 
*log_ctx)
 if (!link)
 continue;
 
+if (link->type == AVMEDIA_TYPE_SUBTITLE)
+continue;
+
 neg = ff_filter_get_negotiation(link);
 av_assert0(neg);
 for (neg_step = 1; neg_step < neg->nb_mergers; neg_step++) {
diff --git a/libavfilter/formats.c b/libavfilter/formats.c
index 1bf7d36195..704774d763 100644
--- a/libavfilter/formats.c
+++ b/libavfilter/formats.c
@@ -19,6 +19,7 @@
  * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
  */
 
+#include "libavcodec/avcodec.h"
 #include "libavutil/avassert.h"
 #include "libavutil/channel_layout.h"
 #include "libavutil/common.h"
@@ -430,6 +431,12 @@ int ff_add_channel_layout(AVFilterChannelLayouts **l, 
uint64_t channel_layout)
 return 0;
 }
 
+int ff_add_subtitle_type(AVFilterFormats **avff, int64_t fmt)
+{
+ADD_FORMAT(avff, fmt, ff_formats_unref, int, formats, nb_formats);
+return 0;
+}
+
 AVFilterFormats *ff_all_formats(enum AVMediaType type)
 {
 AVFilterFormats *ret = NULL;
@@ -447,6 +454,13 @@ AVFilterFormats *ff_all_formats(enum AVMediaType type)
 return NULL;
 fmt++;
 }
+} else if (type == AVMEDIA_TYPE_SUBTITLE) {
+if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_BITMAP) < 0)
+return NULL;
+if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_ASS) < 0)
+return NULL;
+if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_TEXT) < 0)
+return NULL;
 }
 
 return ret;
diff --git a/libavfilter/formats.h b/libavfilter/formats.h
index 471cb42bc4..25ae511fc1 100644
--- a/libavfilter/formats.h
+++ b/libavfilter/formats.h
@@ -180,6 +180,9 @@ int ff_set_common_formats_from_list(AVFilterContext *ctx, 
const int *fmts);
 av_warn_unused_result
 int ff_add_channel_layout(AVFilterChannelLayouts **l, uint64_t channel_layout);
 
+av_warn_unused_result
+int ff_add_subtitle_type(AVFilterFormats **avff, int64_t fmt);
+
 /**
  * Add *ref as a new reference to f.
  */
-- 
2.30.2.windows.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


[FFmpeg-devel] [PATCH v6 05/11] avfilter/sbuffer: Add sbuffersrv and sbuffersink filters

2021-09-16 Thread Soft Works
Signed-off-by: softworkz 
---
 configure|  2 +-
 libavfilter/allfilters.c | 10 +++---
 libavfilter/buffersink.c | 63 +++
 libavfilter/buffersink.h | 15 +
 libavfilter/buffersrc.c  | 72 
 libavfilter/buffersrc.h  |  1 +
 libavfilter/version.h|  2 +-
 7 files changed, 159 insertions(+), 6 deletions(-)

diff --git a/configure b/configure
index 7ac23123c7..74c5a41be1 100755
--- a/configure
+++ b/configure
@@ -7718,7 +7718,7 @@ print_enabled_components(){
 fi
 done
 if [ "$name" = "filter_list" ]; then
-for c in asrc_abuffer vsrc_buffer asink_abuffer vsink_buffer; do
+for c in asrc_abuffer vsrc_buffer ssrc_sbuffer asink_abuffer 
vsink_buffer ssink_sbuffer; do
 printf "&ff_%s,\n" $c >> $TMPH
 done
 fi
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index ddd6404228..154eba5bb2 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -533,10 +533,12 @@ extern const AVFilter ff_avsrc_movie;
  * they are formatted to not be found by the grep
  * as they are manually added again (due to their 'names'
  * being the same while having different 'types'). */
-extern  const AVFilter ff_asrc_abuffer;
-extern  const AVFilter ff_vsrc_buffer;
-extern  const AVFilter ff_asink_abuffer;
-extern  const AVFilter ff_vsink_buffer;
+extern const AVFilter ff_asrc_abuffer;
+extern const AVFilter ff_vsrc_buffer;
+extern const AVFilter ff_ssrc_sbuffer;
+extern const AVFilter ff_asink_abuffer;
+extern const AVFilter ff_vsink_buffer;
+extern const AVFilter ff_ssink_sbuffer;
 extern const AVFilter ff_af_afifo;
 extern const AVFilter ff_vf_fifo;
 
diff --git a/libavfilter/buffersink.c b/libavfilter/buffersink.c
index 8b46dcb15e..55cbaff3d5 100644
--- a/libavfilter/buffersink.c
+++ b/libavfilter/buffersink.c
@@ -29,6 +29,8 @@
 #include "libavutil/internal.h"
 #include "libavutil/opt.h"
 
+#include "libavcodec/avcodec.h"
+
 #define FF_INTERNAL_FIELDS 1
 #include "framequeue.h"
 
@@ -57,6 +59,10 @@ typedef struct BufferSinkContext {
 int *sample_rates;  ///< list of accepted sample 
rates, terminated by -1
 int sample_rates_size;
 
+/* only used for subtitles */
+enum AVSubtitleType *subtitle_types; ///< list of accepted subtitle 
types, must be terminated with -1
+int subtitle_types_size;
+
 AVFrame *peeked_frame;
 } BufferSinkContext;
 
@@ -168,6 +174,15 @@ AVABufferSinkParams *av_abuffersink_params_alloc(void)
 return NULL;
 return params;
 }
+
+AVSBufferSinkParams *av_sbuffersink_params_alloc(void)
+{
+AVSBufferSinkParams *params = av_mallocz(sizeof(AVSBufferSinkParams));
+
+if (!params)
+return NULL;
+return params;
+}
 #endif
 
 static av_cold int common_init(AVFilterContext *ctx)
@@ -305,6 +320,28 @@ static int asink_query_formats(AVFilterContext *ctx)
 return 0;
 }
 
+static int ssink_query_formats(AVFilterContext *ctx)
+{
+BufferSinkContext *buf = ctx->priv;
+AVFilterFormats *formats = NULL;
+unsigned i;
+int ret;
+
+CHECK_LIST_SIZE(subtitle_types)
+if (buf->subtitle_types_size) {
+for (i = 0; i < NB_ITEMS(buf->subtitle_types); i++)
+if ((ret = ff_add_subtitle_type(&formats, buf->subtitle_types[i])) 
< 0)
+return ret;
+if ((ret = ff_set_common_formats(ctx, formats)) < 0)
+return ret;
+} else {
+if ((ret = ff_default_query_formats(ctx)) < 0)
+return ret;
+}
+
+return 0;
+}
+
 #define OFFSET(x) offsetof(BufferSinkContext, x)
 #define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM
 static const AVOption buffersink_options[] = {
@@ -322,9 +359,16 @@ static const AVOption abuffersink_options[] = {
 { NULL },
 };
 #undef FLAGS
+#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_SUBTITLE_PARAM
+static const AVOption sbuffersink_options[] = {
+{ "subtitle_types", "set the supported subtitle formats", 
OFFSET(subtitle_types), AV_OPT_TYPE_BINARY, .flags = FLAGS },
+{ NULL },
+};
+#undef FLAGS
 
 AVFILTER_DEFINE_CLASS(buffersink);
 AVFILTER_DEFINE_CLASS(abuffersink);
+AVFILTER_DEFINE_CLASS(sbuffersink);
 
 static const AVFilterPad avfilter_vsink_buffer_inputs[] = {
 {
@@ -363,3 +407,22 @@ const AVFilter ff_asink_abuffer = {
 FILTER_INPUTS(avfilter_asink_abuffer_inputs),
 .outputs   = NULL,
 };
+
+static const AVFilterPad avfilter_ssink_sbuffer_inputs[] = {
+{
+.name = "default",
+.type = AVMEDIA_TYPE_SUBTITLE,
+},
+};
+
+AVFilter ff_ssink_sbuffer = {
+.name  = "sbuffersink",
+.description   = NULL_IF_CONFIG_SMALL("Buffer subtitle frames, and make 
them available to the end of the filter graph."),
+.priv_class= &sbuffersink_class,
+.priv_size = sizeof(BufferSinkContext),
+.init  = common_init,
+.query_formats = ssink_query_formats,
+.activate  = 

[FFmpeg-devel] [PATCH v6 06/11] avfilter/overlay_graphicsubs: Add overlay_graphicsubs and graphicsub2video filters

2021-09-16 Thread Soft Works
- overlay_graphicsubs (VS -> V)
  Overlay graphic subtitles onto a video stream

- graphicsub2video {S -> V)
  Converts graphic subtitles to video frames (with alpha)
  Gets auto-inserted for retaining compatibility with
  sub2video command lines

Signed-off-by: softworkz 
---
 doc/filters.texi | 104 
 libavfilter/Makefile |   2 +
 libavfilter/allfilters.c |   2 +
 libavfilter/vf_overlay_graphicsubs.c | 727 +++
 4 files changed, 835 insertions(+)
 create mode 100644 libavfilter/vf_overlay_graphicsubs.c

diff --git a/doc/filters.texi b/doc/filters.texi
index 8f20ccf8c6..e281af7155 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25079,6 +25079,110 @@ tools.
 
 @c man end VIDEO SINKS
 
+@chapter Subtitle Filters
+@c man begin SUBTITLE FILTERS
+
+When you configure your FFmpeg build, you can disable any of the
+existing filters using @code{--disable-filters}.
+
+Below is a description of the currently available subtitle filters.
+
+@section graphicsub2video
+
+Renders graphic subtitles as video frames. 
+
+This filter replaces the previous "sub2video" hack which did the conversion 
implicitly and up-front as subtitle filtering wasn't possible at that time.
+To retain compatibility with earlier sub2video command lines, this filter is 
being auto-inserted in those cases.
+
+For overlaying graphicsal subtitles it is recommended to use the 
'overlay_graphicsubs' filter which is more efficient and takes less processing 
resources.
+
+This filter is still useful in cases where the overlay is done with hardware 
acceleration (e.g. overlay_qsv, overlay_vaapi, overlay_cuda) for preparing the 
overlay frames.
+
+It accepts the following parameters:
+
+@table @option
+@item size, s
+Set the size of the output video frame.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Overlay PGS subtitles
+(not recommended - better use overlay_graphicsubs)
+@example
+ffmpeg -i 
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
 -filter_complex "[0:1]graphicsub2video[subs];[0:0][subs]overlay" output.mp4
+@end example
+
+@item
+Overlay PGS subtitles implicitly 
+The graphicsub2video is inserted automatically for compatibility with legacy 
command lines. 
+@example
+ffmpeg -i 
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
 -filter_complex "[0:0][0:1]overlay" output.mp4
+@end example
+@end itemize
+
+@section overlay_graphicsubs
+
+Overlay graphic subtitles onto a video stream.
+
+This filter can blend graphical subtitles on a video stream directly, i.e. 
without creating full-size alpha images first.
+The blending operation is limited to the area of the subtitle rectangles, 
which also means that no processing is done at times where no subtitles are to 
be displayed.
+
+
+It accepts the following parameters:
+
+@table @option
+@item x
+@item y
+Set the expression for the x and y coordinates of the overlaid video
+on the main video. Default value is "0" for both expressions. In case
+the expression is invalid, it is set to a huge value (meaning that the
+overlay will not be displayed within the output visible area).
+
+@item eof_action
+See @ref{framesync}.
+
+@item eval
+Set when the expressions for @option{x}, and @option{y} are evaluated.
+
+It accepts the following values:
+@table @samp
+@item init
+only evaluate expressions once during the filter initialization or
+when a command is processed
+
+@item frame
+evaluate expressions for each incoming frame
+@end table
+
+Default value is @samp{frame}.
+
+@item shortest
+See @ref{framesync}.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Overlay PGS subtitles
+@example
+ffmpeg -i 
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
 -filter_complex "[0:1]graphicsub2video[subs];[0:0][subs]overlay" output.mp4
+@end example
+
+@item
+Overlay PGS subtitles implicitly 
+The graphicsub2video is inserted automatically for compatibility with legacy 
command lines. 
+@example
+ffmpeg -i 
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
 -filter_complex "[0:0][0:1]overlay" output.mp4
+@end example
+@end itemize
+@c man end SUBTITLE FILTERS
+
 @chapter Multimedia Filters
 @c man begin MULTIMEDIA FILTERS
 
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 041d3c5382..8fcc25989e 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -290,6 +290,7 @@ OBJS-$(CONFIG_FSPP_FILTER)   += vf_fspp.o 
qp_table.o
 OBJS-$(CONFIG_GBLUR_FILTER)  += vf_gblur.o
 OBJS-$(CONFIG_GEQ_FILTER)+= vf_geq.o
 OBJS-$(CONFIG_GRADFUN_FILTER)+= vf_gradfun.o
+OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER)   += vf_overlay_graphicsubs.o 
framesync.o
 OBJS-$(CONFIG_GRAPHMONITOR_FILTER)   += f_graphmonitor.o
 OBJS-$(CONFIG_GRAYWORLD_FILTER)  += vf_grayworld.o
 

[FFmpeg-devel] [PATCH v6 08/11] avfilter/overlay_textsubs: Add overlay_textsubs and textsubs2video filters

2021-09-16 Thread Soft Works
- overlay_textsubs {VS -> V)
  Overlay text subtitles onto a video stream.

- textsubs2video {S -> V)
  Converts text subtitles to video frames

Signed-off-by: softworkz 
---
 configure |   2 +
 doc/filters.texi  |  69 
 libavfilter/Makefile  |   2 +
 libavfilter/allfilters.c  |   2 +
 libavfilter/avfilter.c|  18 +-
 libavfilter/vf_overlay_textsubs.c | 613 ++
 6 files changed, 701 insertions(+), 5 deletions(-)
 create mode 100644 libavfilter/vf_overlay_textsubs.c

diff --git a/configure b/configure
index 74c5a41be1..6d6a12754d 100755
--- a/configure
+++ b/configure
@@ -3624,6 +3624,7 @@ openclsrc_filter_deps="opencl"
 overlay_opencl_filter_deps="opencl"
 overlay_qsv_filter_deps="libmfx"
 overlay_qsv_filter_select="qsvvpp"
+overlay_textsubs_filter_deps="avcodec libass"
 overlay_vulkan_filter_deps="vulkan_lib libglslang"
 owdenoise_filter_deps="gpl"
 pad_opencl_filter_deps="opencl"
@@ -3669,6 +3670,7 @@ superequalizer_filter_deps="avcodec"
 superequalizer_filter_select="rdft"
 surround_filter_deps="avcodec"
 surround_filter_select="rdft"
+textsub2video_filter_deps="avcodec libass"
 tinterlace_filter_deps="gpl"
 tinterlace_merge_test_deps="tinterlace_filter"
 tinterlace_pad_test_deps="tinterlace_filter"
diff --git a/doc/filters.texi b/doc/filters.texi
index e281af7155..1c38ecd8ec 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25181,6 +25181,75 @@ The graphicsub2video is inserted automatically for 
compatibility with legacy com
 ffmpeg -i 
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
 -filter_complex "[0:0][0:1]overlay" output.mp4
 @end example
 @end itemize
+
+@section overlay_textsubs
+
+Overlay text subtitles onto a video stream.
+
+This filter supersedes the classic @ref{subtitles} filter opposed to which it 
does no longer require to open and access the source stream separately, which 
is often causing problems or doesn't even work for non-local or slow sources.
+
+Inputs:
+- 0: Video [YUV420P, YUV422P, YUV444P, ARGB, RGBA, ABGR, BGRA, RGB24, BGR24]
+- 1: Subtitles [text]
+
+Outputs:
+- 0: Video (same as input)
+
+It accepts the following parameters:
+
+@table @option
+
+@item alpha
+Process alpha channel, by default alpha channel is untouched.
+
+@item fonts_dir
+Set a directory path containing fonts that can be used by the filter.
+These fonts will be used in addition to whatever the font provider uses.
+
+@item force_style
+Override default style or script info parameters of the subtitles. It accepts a
+string containing ASS style format @code{KEY=VALUE} couples separated by ",".
+
+@end table
+
+@section textsub2video
+
+Converts text subtitles to video frames.
+
+For overlaying text subtitles onto video frames it is recommended to use the 
overlay_textsubs filter.
+The textsub2video is useful for for creating transparent text-frames when 
overlay is done via hw acceleration 
+
+Inputs:
+- 0: Subtitles [text]
+
+Outputs:
+- 0: Video [ARGB, RGBA, ABGR, BGRA]
+
+It accepts the following parameters:
+
+@table @option
+
+@item rate, r
+Set the framerate for updating overlay frames.
+Normally, overlay frames will only be updated each time when the subtitles to 
display are changing.
+In cases where subtitles include advanced features (like animation), this 
parameter determines the frequency by which the overlay frames should be 
updated.
+
+@item size, s
+Set the output frame size.
+Allows to override the size of output video frames.
+
+@item alpha
+Process alpha channel, by default alpha channel is untouched.
+
+@item fonts_dir
+Set a directory path containing fonts that can be used by the filter.
+These fonts will be used in addition to whatever the font provider uses.
+
+@item force_style
+Override default style or script info parameters of the subtitles. It accepts a
+string containing ASS style format @code{KEY=VALUE} couples separated by ",".
+
+@end table
 @c man end SUBTITLE FILTERS
 
 @chapter Multimedia Filters
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 8fcc25989e..c0b1cc7001 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -365,6 +365,7 @@ OBJS-$(CONFIG_OVERLAY_OPENCL_FILTER) += 
vf_overlay_opencl.o opencl.o \
 opencl/overlay.o framesync.o
 OBJS-$(CONFIG_OVERLAY_QSV_FILTER)+= vf_overlay_qsv.o framesync.o
 OBJS-$(CONFIG_OVERLAY_GRAPHICSUBS_FILTER)+= vf_overlay_graphicsubs.o 
framesync.o
+OBJS-$(CONFIG_OVERLAY_TEXTSUBS_FILTER)   += vf_overlay_textsubs.o
 OBJS-$(CONFIG_OVERLAY_VULKAN_FILTER) += vf_overlay_vulkan.o vulkan.o
 OBJS-$(CONFIG_OWDENOISE_FILTER)  += vf_owdenoise.o
 OBJS-$(CONFIG_PAD_FILTER)+= vf_pad.o
@@ -453,6 +454,7 @@ OBJS-$(CONFIG_SWAPRECT_FILTER)   += 
vf_swaprect.o
 OBJS-$(CONFIG_SWAPUV_FILTER) += vf_swapuv.o
 OBJS-$(CONFIG_TBLEND_FILTER)  

[FFmpeg-devel] [PATCH v6 07/11] fftools/ffmpeg: Replace sub2video with subtitle frame filtering

2021-09-16 Thread Soft Works
This commit actually enables subtitle filtering in ffmpeg by
sending and receiving subtitle frames to and from a filtergraph.

The heartbeat functionality from the previous sub2video implementation
is retained and applied to all subtitle frames (bitmap, text, ..).

The other part of sub2video functionality is retained by
auto-insertion of the new graphicsub2video filter.

Signed-off-by: softworkz 
---
 fftools/ffmpeg.c  | 459 ++
 fftools/ffmpeg.h  |  14 +-
 fftools/ffmpeg_filter.c   | 198 +++---
 fftools/ffmpeg_hw.c   |   2 +-
 fftools/ffmpeg_opt.c  |   3 +-
 tests/ref/fate/filter-overlay-dvdsub-2397 | 181 +
 tests/ref/fate/sub-dvb| 162 
 tests/ref/fate/sub2video  |  44 ---
 tests/ref/fate/sub2video_basic|  93 +++--
 tests/ref/fate/sub2video_time_limited |   4 +-
 10 files changed, 571 insertions(+), 589 deletions(-)

diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c
index 5365f711e5..80ed9461a3 100644
--- a/fftools/ffmpeg.c
+++ b/fftools/ffmpeg.c
@@ -169,163 +169,6 @@ static int restore_tty;
 static void free_input_threads(void);
 #endif
 
-/* sub2video hack:
-   Convert subtitles to video with alpha to insert them in filter graphs.
-   This is a temporary solution until libavfilter gets real subtitles support.
- */
-
-static int sub2video_get_blank_frame(InputStream *ist)
-{
-int ret;
-AVFrame *frame = ist->sub2video.frame;
-
-av_frame_unref(frame);
-ist->sub2video.frame->width  = ist->dec_ctx->width  ? ist->dec_ctx->width  
: ist->sub2video.w;
-ist->sub2video.frame->height = ist->dec_ctx->height ? ist->dec_ctx->height 
: ist->sub2video.h;
-ist->sub2video.frame->format = AV_PIX_FMT_RGB32;
-if ((ret = av_frame_get_buffer(frame, 0)) < 0)
-return ret;
-memset(frame->data[0], 0, frame->height * frame->linesize[0]);
-return 0;
-}
-
-static void sub2video_copy_rect(uint8_t *dst, int dst_linesize, int w, int h,
-AVSubtitleRect *r)
-{
-uint32_t *pal, *dst2;
-uint8_t *src, *src2;
-int x, y;
-
-if (r->type != SUBTITLE_BITMAP) {
-av_log(NULL, AV_LOG_WARNING, "sub2video: non-bitmap subtitle\n");
-return;
-}
-if (r->x < 0 || r->x + r->w > w || r->y < 0 || r->y + r->h > h) {
-av_log(NULL, AV_LOG_WARNING, "sub2video: rectangle (%d %d %d %d) 
overflowing %d %d\n",
-r->x, r->y, r->w, r->h, w, h
-);
-return;
-}
-
-dst += r->y * dst_linesize + r->x * 4;
-src = r->data[0];
-pal = (uint32_t *)r->data[1];
-for (y = 0; y < r->h; y++) {
-dst2 = (uint32_t *)dst;
-src2 = src;
-for (x = 0; x < r->w; x++)
-*(dst2++) = pal[*(src2++)];
-dst += dst_linesize;
-src += r->linesize[0];
-}
-}
-
-static void sub2video_push_ref(InputStream *ist, int64_t pts)
-{
-AVFrame *frame = ist->sub2video.frame;
-int i;
-int ret;
-
-av_assert1(frame->data[0]);
-ist->sub2video.last_pts = frame->pts = pts;
-for (i = 0; i < ist->nb_filters; i++) {
-ret = av_buffersrc_add_frame_flags(ist->filters[i]->filter, frame,
-   AV_BUFFERSRC_FLAG_KEEP_REF |
-   AV_BUFFERSRC_FLAG_PUSH);
-if (ret != AVERROR_EOF && ret < 0)
-av_log(NULL, AV_LOG_WARNING, "Error while add the frame to buffer 
source(%s).\n",
-   av_err2str(ret));
-}
-}
-
-void sub2video_update(InputStream *ist, int64_t heartbeat_pts, AVSubtitle *sub)
-{
-AVFrame *frame = ist->sub2video.frame;
-int8_t *dst;
-int dst_linesize;
-int num_rects, i;
-int64_t pts, end_pts;
-
-if (!frame)
-return;
-if (sub) {
-pts   = av_rescale_q(sub->pts + sub->start_display_time * 1000LL,
- AV_TIME_BASE_Q, ist->st->time_base);
-end_pts   = av_rescale_q(sub->pts + sub->end_display_time   * 1000LL,
- AV_TIME_BASE_Q, ist->st->time_base);
-num_rects = sub->num_rects;
-} else {
-/* If we are initializing the system, utilize current heartbeat
-   PTS as the start time, and show until the following subpicture
-   is received. Otherwise, utilize the previous subpicture's end time
-   as the fall-back value. */
-pts   = ist->sub2video.initialize ?
-heartbeat_pts : ist->sub2video.end_pts;
-end_pts   = INT64_MAX;
-num_rects = 0;
-}
-if (sub2video_get_blank_frame(ist) < 0) {
-av_log(ist->dec_ctx, AV_LOG_ERROR,
-   "Impossible to get a blank canvas.\n");
-return;
-}
-dst  = frame->data[0];
-dst_linesize = frame->linesize[0];
-for (i = 0; i < num_rects; i++)
-sub2video_copy_rect(dst, dst_linesi

[FFmpeg-devel] [PATCH v6 09/11] avfilter/textmod: Add textmod filter

2021-09-16 Thread Soft Works
- textmod {S -> S)
  Modify subtitle text in a number of ways

Signed-off-by: softworkz 
---
 doc/filters.texi |  64 +++
 libavfilter/Makefile |   3 +
 libavfilter/allfilters.c |   1 +
 libavfilter/sf_textmod.c | 372 +++
 4 files changed, 440 insertions(+)
 create mode 100644 libavfilter/sf_textmod.c

diff --git a/doc/filters.texi b/doc/filters.texi
index 1c38ecd8ec..1832dc8847 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25087,6 +25087,70 @@ existing filters using @code{--disable-filters}.
 
 Below is a description of the currently available subtitle filters.
 
+@section textmod
+
+Modify subtitle text in a number of ways.
+
+It accepts the following parameters:
+
+@table @option
+@item mode
+The kind of text modification to apply
+
+Supported operation modes are:
+
+@table @var
+@item 0, leet
+Convert subtitle text to 'leet speak'. It's primarily useful for testing as 
the modification will be visible with almost all text lines.
+@item 1, to_upper
+Change all text to upper case. Might improve readability.
+@item 2, to_lower
+Change all text to lower case.
+@item 3, replace_chars
+Replace one or more characters. Requires the find and replace parameters to be 
specified. 
+Both need to be equal in length.
+The first char in find is replaced by the first char in replace, same for all 
subsequent chars.
+@item 4, remove_chars
+Remove certain characters. Requires the find parameter to be specified. 
+All chars in the find parameter string will be removed from all subtitle text.
+@item 5, replace_words
+Replace one or more words. Requires the find and replace parameters to be 
specified. Multiple words must be separated by the delimiter char specified vie 
the separator parameter (default: ','). 
+The number of words in the find and replace parameters needs to be equal.
+The first word in find is replaced by the first word in replace, same for all 
subsequent words
+@item 6, remove_words
+Remove certain words. Requires the find parameter to be specified. Multiple 
words must be separated by the delimiter char specified vie the separator 
parameter (default: ','). 
+All words in the find parameter string will be removed from all subtitle text.
+@end table
+
+@item find
+Required for replace_chars, remove_chars, replace_words and remove_words.
+
+@item replace
+Required for replace_chars and replace_words.
+
+@item separator
+Delimiter character for words. Used with replace_words and remove_words- Must 
be a single character.
+The default is '.'.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Change all characters to upper case while keeping all styles and animations:
+@example
+ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv"; 
-filter_complex "[0:s]textmod=mode=to_upper" -map 0 -y out.mkv
+@end example
+@item
+Mark the 100-pixel-wide region on the left edge of the frame as very
+uninteresting (to be encoded at much lower quality than the rest of
+the frame).
+@example
+addroi=0:0:100:ih:+1/5
+@end example
+@end itemize
+
 @section graphicsub2video
 
 Renders graphic subtitles as video frames. 
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index c0b1cc7001..630f701a5f 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -536,6 +536,9 @@ OBJS-$(CONFIG_YUVTESTSRC_FILTER) += 
vsrc_testsrc.o
 
 OBJS-$(CONFIG_NULLSINK_FILTER)   += vsink_nullsink.o
 
+# subtitle filters
+OBJS-$(CONFIG_TEXTMOD_FILTER)+= sf_textmod.o
+
 # multimedia filters
 OBJS-$(CONFIG_ABITSCOPE_FILTER)  += avf_abitscope.o
 OBJS-$(CONFIG_ADRAWGRAPH_FILTER) += f_drawgraph.o
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 8543fa22e9..63e41b637d 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -526,6 +526,7 @@ extern const AVFilter ff_avf_showvolume;
 extern const AVFilter ff_avf_showwaves;
 extern const AVFilter ff_avf_showwavespic;
 extern const AVFilter ff_vaf_spectrumsynth;
+extern const AVFilter ff_sf_textmod;
 extern const AVFilter ff_svf_graphicsub2video;
 extern const AVFilter ff_svf_textsub2video;
 
diff --git a/libavfilter/sf_textmod.c b/libavfilter/sf_textmod.c
new file mode 100644
index 00..6a29a2d3c0
--- /dev/null
+++ b/libavfilter/sf_textmod.c
@@ -0,0 +1,372 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Publ

[FFmpeg-devel] [PATCH v6 10/11] avcodec/ass_split: Extend ass dialog parsing

2021-09-16 Thread Soft Works
Signed-off-by: softworkz 
---
 libavcodec/ass_split.c | 12 ++--
 libavcodec/ass_split.h |  2 ++
 2 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/libavcodec/ass_split.c b/libavcodec/ass_split.c
index 05c5453e53..e58585d3da 100644
--- a/libavcodec/ass_split.c
+++ b/libavcodec/ass_split.c
@@ -497,8 +497,8 @@ int ff_ass_split_override_codes(const ASSCodesCallbacks 
*callbacks, void *priv,
 while (*buf == '\\') {
 char style[2], c[2], sep[2], c_num[2] = "0", tmp[128] = {0};
 unsigned int color = 0x;
-int len, size = -1, an = -1, alpha = -1;
-int x1, y1, x2, y2, t1 = -1, t2 = -1;
+int len, size = -1, an = -1, alpha = -1, scale = 0;
+int x1, y1, x2, y2, t1 = -1, t2 = -1, accel = 1;
 if (sscanf(buf, "\\%1[bisu]%1[01\\}]%n", style, c, &len) > 1) {
 int close = c[0] == '0' ? 1 : c[0] == '1' ? 0 : -1;
 len += close != -1;
@@ -546,6 +546,14 @@ int ff_ass_split_override_codes(const ASSCodesCallbacks 
*callbacks, void *priv,
 } else if (sscanf(buf, "\\org(%d,%d)%1[\\}]%n", &x1, &y1, sep, 
&len) > 2) {
 if (callbacks->origin)
 callbacks->origin(priv, x1, y1);
+} else if (sscanf(buf, "\\t(%d,%d,%1[\\}]%n", &t1, &t2, sep, 
&len) > 2 ||
+   sscanf(buf, "\\t(%d,%d,%d,%1[\\}]%n", &t1, &t2, 
&accel, sep, &len) > 3) {
+if (callbacks->animate)
+callbacks->animate(priv, t1, t2, accel, tmp);
+} else if (sscanf(buf, "\\p%1[\\}]%n", sep, &len) > 0 ||
+   sscanf(buf, "\\p%u%1[\\}]%n", &scale, sep, &len) > 
1) {
+if (callbacks->drawing_mode)
+callbacks->drawing_mode(priv, scale);
 } else {
 len = strcspn(buf+1, "\\}") + 2;  /* skip unknown code */
 }
diff --git a/libavcodec/ass_split.h b/libavcodec/ass_split.h
index a45fb9b8a1..bda7bb27db 100644
--- a/libavcodec/ass_split.h
+++ b/libavcodec/ass_split.h
@@ -156,7 +156,9 @@ typedef struct {
  * @{
  */
 void (*move)(void *priv, int x1, int y1, int x2, int y2, int t1, int t2);
+void (*animate)(void *priv, int t1, int t2, int accel, char *style);
 void (*origin)(void *priv, int x, int y);
+void (*drawing_mode)(void *priv, int scale);
 /** @} */
 
 /**
-- 
2.30.2.windows.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


[FFmpeg-devel] [PATCH v6 11/11] avfilter/stripstyles: Add stripstyles filter

2021-09-16 Thread Soft Works
- stripstyles {S -> S)
  Remove all inline styles from subtitle events

Signed-off-by: softworkz 
---
 doc/filters.texi |  24 
 libavfilter/Makefile |   1 +
 libavfilter/allfilters.c |   1 +
 libavfilter/sf_stripstyles.c | 211 +++
 4 files changed, 237 insertions(+)
 create mode 100644 libavfilter/sf_stripstyles.c

diff --git a/doc/filters.texi b/doc/filters.texi
index 1832dc8847..c8bf444d2c 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25087,6 +25087,30 @@ existing filters using @code{--disable-filters}.
 
 Below is a description of the currently available subtitle filters.
 
+@section stripstyles
+
+Remove all inline styles from subtitle events.
+
+It accepts the following parameters:
+
+@table @option
+@item remove_animated
+Also remove text which is subject to animation (default: true)
+Usually, animated text elements are used used in addition to static subtitle 
lines for creating effects, so in most cases it is safe to remove the animation 
content.
+If subtitle text is missing, try setting this to false.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Change all characters to upper case while keeping all styles and animations:
+@example
+ffmpeg -i 
"https://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv"; 
-filter_complex "[0:1]stripstyles" -map 0 output.mkv
+@end example
+@end itemize
+
 @section textmod
 
 Modify subtitle text in a number of ways.
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 630f701a5f..94ec11c4ce 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -538,6 +538,7 @@ OBJS-$(CONFIG_NULLSINK_FILTER)   += 
vsink_nullsink.o
 
 # subtitle filters
 OBJS-$(CONFIG_TEXTMOD_FILTER)+= sf_textmod.o
+OBJS-$(CONFIG_STRIPSTYLES_FILTER)+= sf_stripstyles.o
 
 # multimedia filters
 OBJS-$(CONFIG_ABITSCOPE_FILTER)  += avf_abitscope.o
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 63e41b637d..88dd97adf1 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -526,6 +526,7 @@ extern const AVFilter ff_avf_showvolume;
 extern const AVFilter ff_avf_showwaves;
 extern const AVFilter ff_avf_showwavespic;
 extern const AVFilter ff_vaf_spectrumsynth;
+extern const AVFilter ff_sf_stripstyles;
 extern const AVFilter ff_sf_textmod;
 extern const AVFilter ff_svf_graphicsub2video;
 extern const AVFilter ff_svf_textsub2video;
diff --git a/libavfilter/sf_stripstyles.c b/libavfilter/sf_stripstyles.c
new file mode 100644
index 00..15c00e73b1
--- /dev/null
+++ b/libavfilter/sf_stripstyles.c
@@ -0,0 +1,211 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/**
+ * @file
+ * text subtitle filter which removes inline-styles from subtitles
+ */
+
+#include 
+
+#include "libavutil/avassert.h"
+#include "libavutil/opt.h"
+#include "avfilter.h"
+#include "internal.h"
+#include "libavcodec/avcodec.h"
+#include "libavcodec/ass_split.h"
+
+typedef struct StripStylesContext {
+const AVClass *class;
+enum AVSubtitleType format;
+int remove_animated;
+} StripStylesContext;
+
+typedef struct DialogContext {
+const AVClass *class;
+const StripStylesContext* ss_ctx;
+AVBPrint buffer;
+int drawing_scale;
+int is_animated;
+} DialogContext;
+
+static void dialog_text_cb(void *priv, const char *text, int len)
+{
+DialogContext *s = priv;
+if (!s->drawing_scale && (!s->is_animated || !s->ss_ctx->remove_animated))
+av_bprint_append_data(&s->buffer, text, len);
+}
+
+static void dialog_new_line_cb(void *priv, int forced)
+{
+DialogContext *s = priv;
+if (!s->drawing_scale && !s->is_animated)
+av_bprint_append_data(&s->buffer, forced ? "\\N" : "\\n", 2);
+}
+
+static void dialog_drawing_mode_cb(void *priv, int scale)
+{
+DialogContext *s = priv;
+s->drawing_scale = scale;
+}
+
+static void dialog_animate_cb(void *priv, int t1, int t2, int accel, char 
*style)
+{
+DialogContext *s = priv;
+s->is_animated = 1;
+}
+
+static void dialog_move_cb(void *priv, int x1, int y1, int x2, int y2, int t1, 
int t2)
+{
+DialogContext *s = priv;
+if (t1 >= 0 || t2 >= 0)
+s->is_anima

Re: [FFmpeg-devel] [PATCH v5 00/12] Subtitle Filtering

2021-09-16 Thread Soft Works



> -Original Message-
> From: ffmpeg-devel  On Behalf Of
> Hendrik Leppkes
> Sent: Wednesday, 15 September 2021 11:21
> To: FFmpeg development discussions and patches  de...@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v5 00/12] Subtitle Filtering
> 
> On Wed, Sep 15, 2021 at 9:39 AM Soft Works 
> wrote:
> >
> > In my upcoming patchset, I have added a new struct AVSubtitleArea
> > which is similar to AVSubtitleRect with the difference that
> >
> > - It's not AVSubtitleRect (which can be deprecated then)
> > - It uses refcounted buffers for the image data
> >
> > AVFrame gets an array of AVSubtitleArea structs. The image data
> > is attached to AVSubtitleArea (data, buf, linesize).
> >
> > I'm not sure whether it would make sense to store the image data in
> > buf/data of AVFrame like
> >
> > AVSubtitleArea[0] <=> buf[0], buf[1]
> > AVSubtitleArea[1] <=> buf[2], buf[3]
> >
> 
> I suppose the main problem is that there can be more areas then we
> have buf entries?
> There is just one small wrinkle, there is an expectation in various
> areas of the code that an empty frame has buf[0] zero, and conversely
> a filled one does not.

You made re-think this and there are quite a number of places that
do that buf[0] checking. Replacing those checks with an API method
might make sense, but I think it's beyond the scope of this patchset.

The context above was about whether storing bitmap buffers in
individual rects or inside the buffers of AVFrame. 
We need to be aware though, that storing the buffers in AVFrame
wouldn't help with the checks for frame->buf[0], because:

- For text subtitles, the texts are not stored in buffers
- Subtitle frames that do not contain any rects at all are 
  still valid

For those reasons I resorted to adding a simple 1-byte buffer to
subtitle frames in buf[0]. This allows to remain compatible with all 
existing frame handling logic.
The meaning of buf[0]=!0 is the same as for other frames: "has data"
as it will only get set on calling av_frame_get_buffer2.

Kind regards,
softworkz


___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 5/5] avcodec/libsvtav1: support constant quality mode

2021-09-16 Thread Jan Ekström
On Thu, Sep 16, 2021 at 1:55 PM  wrote:
>
> From: Limin Wang 
>
> Signed-off-by: Limin Wang 
> ---

For the record, Christopher Degawa also posted something similar
(https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210310224859.1726693-1-c...@randomderp.com/),
and I finally recently had gotten to noting on IRC that the rate
control options should be brought more in line with, say, libx264. Set
AVOption-specific defaults (f.ex., libx264 sets CRF to its default
value by default), and then if any of the other rate control modes is
nonzero the appropriate rate control would get utilized in the
wrapper.

Jan
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


[FFmpeg-devel] [PATCH 2/2] avcodec/wmaprodec: Check that the EOF frame was allocated before decoding into it

2021-09-16 Thread Michael Niedermayer
Fixes: NULL pointer dereference
Fixes: 
38125/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_XMA1_fuzzer-5151909422432256

Found-by: continuous fuzzing process 
https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer 
---
 libavcodec/wmaprodec.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavcodec/wmaprodec.c b/libavcodec/wmaprodec.c
index a4b48b13a8c..7b489943f43 100644
--- a/libavcodec/wmaprodec.c
+++ b/libavcodec/wmaprodec.c
@@ -1855,7 +1855,7 @@ static int xma_decode_packet(AVCodecContext *avctx, void 
*data,
 eof = 1;
 
 for (i = 0; i < s->num_streams; i++) {
-if (!s->xma[i].eof_done) {
+if (!s->xma[i].eof_done && s->frames[i]->data[0]) {
 ret = decode_packet(avctx, &s->xma[i], s->frames[i],
 &got_stream_frame_ptr, avpkt);
 }
-- 
2.17.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 2/5] avcodec/libsvtav1: make intra_refresh_type configurable

2021-09-16 Thread Jan Ekström
Hi

On Thu, Sep 16, 2021 at 1:54 PM  wrote:
>
> From: Limin Wang 
>
> Signed-off-by: Limin Wang 
> ---

I think something like:

avcodec/libsvtav1: make coded GOP type configurable

Might be a bit better as a commit message?
>  libavcodec/libsvtav1.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/libavcodec/libsvtav1.c b/libavcodec/libsvtav1.c
> index 82ae2b9..8c2c970 100644
> --- a/libavcodec/libsvtav1.c
> +++ b/libavcodec/libsvtav1.c
> @@ -210,7 +210,8 @@ static int config_enc_params(EbSvtAv1EncConfiguration 
> *param,
>  param->min_qp_allowed   = avctx->qmin;
>  }
>
> -param->intra_refresh_type   = 2; /* Real keyframes only */
> +/* 2 = IDR, closed GOP, 1 = CRA, open GOP */
> +param->intra_refresh_type = avctx->flags & AV_CODEC_FLAG_CLOSED_GOP ? 2 
> : 1;
>

Ugh... I see they have EbIntraRefreshType for an enum (NO_REFRESH |
CRA_REFRESH | IDR_REFRESH), but I guess those are not available
through the public API?

>  if (svt_enc->la_depth >= 0)
>  param->look_ahead_distance  = svt_enc->la_depth;
> @@ -548,6 +549,7 @@ static const AVCodecDefault eb_enc_defaults[] = {
>  { "g", "-1"},
>  { "qmin",  "0" },
>  { "qmax",  "63"},
> +{ "flags", "+cgop" },

Let's keep the alphabetic ordering since this module is not yet a mess
in that sense :) Thus this would go after "b" .

Otherwise LGTM.

Jan
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


[FFmpeg-devel] [PATCH 1/2] avformat/mpegts: use actually read packet size in mpegts_resync special case

2021-09-16 Thread Michael Niedermayer
Fixes: infinite loop
Fixes: 
37986/clusterfuzz-testcase-minimized-ffmpeg_dem_MPEGTSRAW_fuzzer-5292311517462528
 -

Found-by: continuous fuzzing process 
https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer 
---
 libavformat/mpegts.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libavformat/mpegts.c b/libavformat/mpegts.c
index fe89d4fb9f3..bd13118f7f7 100644
--- a/libavformat/mpegts.c
+++ b/libavformat/mpegts.c
@@ -2865,8 +2865,8 @@ static int mpegts_resync(AVFormatContext *s, int 
seekback, const uint8_t *curren
 int64_t back = FFMIN(seekback, pos);
 
 //Special case for files like 01c56b0dc1.ts
-if (current_packet[0] == 0x80 && current_packet[12] == 0x47) {
-avio_seek(pb, 12 - back, SEEK_CUR);
+if (current_packet[0] == 0x80 && current_packet[12] == 0x47 && pos >= 
TS_PACKET_SIZE) {
+avio_seek(pb, 12 - TS_PACKET_SIZE, SEEK_CUR);
 return 0;
 }
 
-- 
2.17.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 2/5] avcodec/libsvtav1: make intra_refresh_type configurable

2021-09-16 Thread lance . lmwang
On Fri, Sep 17, 2021 at 01:38:46AM +0300, Jan Ekström wrote:
> Hi
> 
> On Thu, Sep 16, 2021 at 1:54 PM  wrote:
> >
> > From: Limin Wang 
> >
> > Signed-off-by: Limin Wang 
> > ---
> 
> I think something like:
> 
> avcodec/libsvtav1: make coded GOP type configurable
> 
> Might be a bit better as a commit message?

Yes, it's more better, I'll update it.


> >  libavcodec/libsvtav1.c | 4 +++-
> >  1 file changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/libavcodec/libsvtav1.c b/libavcodec/libsvtav1.c
> > index 82ae2b9..8c2c970 100644
> > --- a/libavcodec/libsvtav1.c
> > +++ b/libavcodec/libsvtav1.c
> > @@ -210,7 +210,8 @@ static int config_enc_params(EbSvtAv1EncConfiguration 
> > *param,
> >  param->min_qp_allowed   = avctx->qmin;
> >  }
> >
> > -param->intra_refresh_type   = 2; /* Real keyframes only */
> > +/* 2 = IDR, closed GOP, 1 = CRA, open GOP */
> > +param->intra_refresh_type = avctx->flags & AV_CODEC_FLAG_CLOSED_GOP ? 
> > 2 : 1;
> >
> 
> Ugh... I see they have EbIntraRefreshType for an enum (NO_REFRESH |
> CRA_REFRESH | IDR_REFRESH), but I guess those are not available
> through the public API?

Yes, it's not public API.

> 
> >  if (svt_enc->la_depth >= 0)
> >  param->look_ahead_distance  = svt_enc->la_depth;
> > @@ -548,6 +549,7 @@ static const AVCodecDefault eb_enc_defaults[] = {
> >  { "g", "-1"},
> >  { "qmin",  "0" },
> >  { "qmax",  "63"},
> > +{ "flags", "+cgop" },
> 
> Let's keep the alphabetic ordering since this module is not yet a mess
> in that sense :) Thus this would go after "b" .

OK, I'll change the order.

> 
> Otherwise LGTM.
> 
> Jan
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

-- 
Thanks,
Limin Wang
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 5/5] avcodec/libsvtav1: support constant quality mode

2021-09-16 Thread lance . lmwang
On Fri, Sep 17, 2021 at 01:10:33AM +0300, Jan Ekström wrote:
> On Thu, Sep 16, 2021 at 1:55 PM  wrote:
> >
> > From: Limin Wang 
> >
> > Signed-off-by: Limin Wang 
> > ---
> 
> For the record, Christopher Degawa also posted something similar
> (https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210310224859.1726693-1-c...@randomderp.com/),

I think It's very confusing for the configure for crf, it's 0 as cqp, user had 
to enable_tpl_la and
configure crf also. So I prefer to my proposal.

> and I finally recently had gotten to noting on IRC that the rate
> control options should be brought more in line with, say, libx264. Set
> AVOption-specific defaults (f.ex., libx264 sets CRF to its default
> value by default), and then if any of the other rate control modes is
> nonzero the appropriate rate control would get utilized in the
> wrapper.

As rate control options is supported already, so it's better to make crf mode
works also. User can use AVOption-specific if can working as expect in future.

If CRF is preferable default, I can change the default crf value to 50 instead
of -1.

> 
> Jan
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

-- 
Thanks,
Limin Wang
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH v3] libavfilter: add a gblur_vulkan filter

2021-09-16 Thread Wu, Jianhua
Ping.

> From: Wu, Jianhua 
> Sent: Thursday, September 9, 2021 1:45 PM
> To: ffmpeg-devel@ffmpeg.org
> Cc: Wu, Jianhua 
> Subject: [PATCH v3] libavfilter: add a gblur_vulkan filter
> 
> This commit adds a powerful and customizable gblur Vulkan filter, which
> provides a maximum 127x127 kernel size of Gaussian Filter.
> The size could be adjusted by requirements on quality or performance.
> 
> The following command is on how to apply gblur_vulkan filter:
> 
> ffmpeg -init_hw_device vulkan=vul:0 -filter_hw_device vul -i input.264 -vf
> hwupload=extra_hw_frames=16,gblur_vulkan,hwdownload,format=yuv420
> p
> output.264
> 
> Signed-off-by: Wu Jianhua 
> ---
>  configure |   1 +
>  libavfilter/Makefile  |   1 +
>  libavfilter/allfilters.c  |   1 +
>  libavfilter/vf_gblur_vulkan.c | 511
> ++
>  4 files changed, 514 insertions(+)

Hi  there,

Any update?
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


[FFmpeg-devel] [PATCH 01/13] avcodec/elbg: Remove avoidable buffer

2021-09-16 Thread Andreas Rheinhardt
Signed-off-by: Andreas Rheinhardt 
---
 libavcodec/elbg.c | 9 +++--
 1 file changed, 3 insertions(+), 6 deletions(-)

diff --git a/libavcodec/elbg.c b/libavcodec/elbg.c
index d012d9a384..795fc83f16 100644
--- a/libavcodec/elbg.c
+++ b/libavcodec/elbg.c
@@ -376,7 +376,6 @@ int avpriv_do_elbg(int *points, int dim, int numpoints, int 
*codebook,
 elbg_data elbg_d;
 elbg_data *elbg = &elbg_d;
 int i, j, k, steps = 0, ret = 0;
-int *dist_cb = av_malloc_array(numpoints, sizeof(int));
 int *size_part = av_malloc_array(numCB, sizeof(int));
 cell *list_buffer = av_malloc_array(numpoints, sizeof(cell));
 cell *free_cells;
@@ -394,7 +393,7 @@ int avpriv_do_elbg(int *points, int dim, int numpoints, int 
*codebook,
 elbg->utility_inc = av_malloc_array(numCB, sizeof(*elbg->utility_inc));
 elbg->scratchbuf = av_malloc_array(5*dim, sizeof(int));
 
-if (!dist_cb || !size_part || !list_buffer || !elbg->cells ||
+if (!size_part || !list_buffer || !elbg->cells ||
 !elbg->utility || !elbg->utility_inc || !elbg->scratchbuf) {
 ret = AVERROR(ENOMEM);
 goto out;
@@ -423,9 +422,8 @@ int avpriv_do_elbg(int *points, int dim, int numpoints, int 
*codebook,
 }
 }
 elbg->nearest_cb[i] = best_idx;
-dist_cb[i] = best_dist;
-elbg->error += dist_cb[i];
-elbg->utility[elbg->nearest_cb[i]] += dist_cb[i];
+elbg->error += best_dist;
+elbg->utility[elbg->nearest_cb[i]] += best_dist;
 free_cells->index = i;
 free_cells->next = elbg->cells[elbg->nearest_cb[i]];
 elbg->cells[elbg->nearest_cb[i]] = free_cells;
@@ -453,7 +451,6 @@ int avpriv_do_elbg(int *points, int dim, int numpoints, int 
*codebook,
 (steps < max_steps));
 
 out:
-av_free(dist_cb);
 av_free(size_part);
 av_free(elbg->utility);
 av_free(list_buffer);
-- 
2.30.2

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


[FFmpeg-devel] [PATCH 02/13] avcodec/elbg: Move avpriv_init_elbg() down

2021-09-16 Thread Andreas Rheinhardt
It will avoid a forward declaration later.

Signed-off-by: Andreas Rheinhardt 
---
 libavcodec/elbg.c | 71 +++
 1 file changed, 35 insertions(+), 36 deletions(-)

diff --git a/libavcodec/elbg.c b/libavcodec/elbg.c
index 795fc83f16..3ca67b400c 100644
--- a/libavcodec/elbg.c
+++ b/libavcodec/elbg.c
@@ -332,42 +332,6 @@ static void do_shiftings(elbg_data *elbg)
 }
 }
 
-#define BIG_PRIME 433494437LL
-
-int avpriv_init_elbg(int *points, int dim, int numpoints, int *codebook,
- int numCB, int max_steps, int *closest_cb,
- AVLFG *rand_state)
-{
-int i, k, ret = 0;
-
-if (numpoints > 24*numCB) {
-/* ELBG is very costly for a big number of points. So if we have a lot
-   of them, get a good initial codebook to save on iterations   */
-int *temp_points = av_malloc_array(dim, (numpoints/8)*sizeof(int));
-if (!temp_points)
-return AVERROR(ENOMEM);
-for (i=0; iscratchbuf);
 return ret;
 }
+
+#define BIG_PRIME 433494437LL
+
+int avpriv_init_elbg(int *points, int dim, int numpoints, int *codebook,
+ int num_cb, int max_steps, int *closest_cb,
+ AVLFG *rand_state)
+{
+int ret = 0;
+
+if (numpoints > 24LL * num_cb) {
+/* ELBG is very costly for a big number of points. So if we have a lot
+   of them, get a good initial codebook to save on iterations   */
+int *temp_points = av_malloc_array(dim, 
(numpoints/8)*sizeof(*temp_points));
+if (!temp_points)
+return AVERROR(ENOMEM);
+for (int i = 0; i < numpoints / 8; i++) {
+int k = (i*BIG_PRIME) % numpoints;
+memcpy(temp_points + i*dim, points + k*dim, dim * 
sizeof(*temp_points));
+}
+
+ret = avpriv_init_elbg(temp_points, dim, numpoints / 8, codebook,
+   num_cb, 2 * max_steps, closest_cb, rand_state);
+if (ret < 0) {
+av_freep(&temp_points);
+return ret;
+}
+ret = avpriv_do_elbg(temp_points, dim, numpoints / 8, codebook,
+ num_cb, 2 * max_steps, closest_cb, rand_state);
+av_free(temp_points);
+} else  // If not, initialize the codebook with random positions
+for (int i = 0; i < num_cb; i++)
+memcpy(codebook + i * dim, points + ((i*BIG_PRIME)%numpoints)*dim,
+   dim * sizeof(*codebook));
+return ret;
+}
-- 
2.30.2

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


[FFmpeg-devel] [PATCH 05/13] avcodec/elbg: Rename elbg_data to ELBGContext

2021-09-16 Thread Andreas Rheinhardt
It is in line with our naming conventions for types.
Also change numCB to num_cb for the same reason.

Signed-off-by: Andreas Rheinhardt 
---
 libavcodec/elbg.c | 84 +++
 libavcodec/elbg.h |  4 +--
 2 files changed, 44 insertions(+), 44 deletions(-)

diff --git a/libavcodec/elbg.c b/libavcodec/elbg.c
index b563254bbc..ac5c53161d 100644
--- a/libavcodec/elbg.c
+++ b/libavcodec/elbg.c
@@ -43,10 +43,10 @@ typedef struct cell_s {
 /**
  * ELBG internal data
  */
-typedef struct elbg_data {
+typedef struct ELBGContext {
 int64_t error;
 int dim;
-int numCB;
+int num_cb;
 int *codebook;
 cell **cells;
 int64_t *utility;
@@ -55,7 +55,7 @@ typedef struct elbg_data {
 int *points;
 AVLFG *rand_state;
 int *scratchbuf;
-} elbg_data;
+} ELBGContext;
 
 static inline int distance_limited(int *a, int *b, int dim, int limit)
 {
@@ -80,7 +80,7 @@ static inline void vect_division(int *res, int *vect, int 
div, int dim)
 
 }
 
-static int eval_error_cell(elbg_data *elbg, int *centroid, cell *cells)
+static int eval_error_cell(ELBGContext *elbg, int *centroid, cell *cells)
 {
 int error=0;
 for (; cells; cells=cells->next)
@@ -89,11 +89,12 @@ static int eval_error_cell(elbg_data *elbg, int *centroid, 
cell *cells)
 return error;
 }
 
-static int get_closest_codebook(elbg_data *elbg, int index)
+static int get_closest_codebook(ELBGContext *elbg, int index)
 {
-int i, pick=0, diff, diff_min = INT_MAX;
-for (i=0; inumCB; i++)
+int pick = 0;
+for (int i = 0, diff_min = INT_MAX; i < elbg->num_cb; i++)
 if (i != index) {
+int diff;
 diff = distance_limited(elbg->codebook + i*elbg->dim, 
elbg->codebook + index*elbg->dim, elbg->dim, diff_min);
 if (diff < diff_min) {
 pick = i;
@@ -103,17 +104,17 @@ static int get_closest_codebook(elbg_data *elbg, int 
index)
 return pick;
 }
 
-static int get_high_utility_cell(elbg_data *elbg)
+static int get_high_utility_cell(ELBGContext *elbg)
 {
 int i=0;
 /* Using linear search, do binary if it ever turns to be speed critical */
 uint64_t r;
 
-if (elbg->utility_inc[elbg->numCB-1] < INT_MAX) {
-r = av_lfg_get(elbg->rand_state) % (unsigned 
int)elbg->utility_inc[elbg->numCB-1] + 1;
+if (elbg->utility_inc[elbg->num_cb - 1] < INT_MAX) {
+r = av_lfg_get(elbg->rand_state) % (unsigned 
int)elbg->utility_inc[elbg->num_cb - 1] + 1;
 } else {
 r = av_lfg_get(elbg->rand_state);
-r = (av_lfg_get(elbg->rand_state) + (r<<32)) % 
elbg->utility_inc[elbg->numCB-1] + 1;
+r = (av_lfg_get(elbg->rand_state) + (r<<32)) % 
elbg->utility_inc[elbg->num_cb - 1] + 1;
 }
 
 while (elbg->utility_inc[i] < r) {
@@ -128,7 +129,7 @@ static int get_high_utility_cell(elbg_data *elbg)
 /**
  * Implementation of the simple LBG algorithm for just two codebooks
  */
-static int simple_lbg(elbg_data *elbg,
+static int simple_lbg(ELBGContext *elbg,
   int dim,
   int *centroid[3],
   int newutility[3],
@@ -169,7 +170,7 @@ static int simple_lbg(elbg_data *elbg,
 return newutility[0] + newutility[1];
 }
 
-static void get_new_centroids(elbg_data *elbg, int huc, int *newcentroid_i,
+static void get_new_centroids(ELBGContext *elbg, int huc, int *newcentroid_i,
   int *newcentroid_p)
 {
 cell *tempcell;
@@ -205,7 +206,7 @@ static void get_new_centroids(elbg_data *elbg, int huc, int 
*newcentroid_i,
  * @param indexes  {luc, huc, cluc}
  * @param newcentroid  A vector with the position of the new centroids
  */
-static void shift_codebook(elbg_data *elbg, int *indexes,
+static void shift_codebook(ELBGContext *elbg, int *indexes,
int *newcentroid[3])
 {
 cell *tempdata;
@@ -233,20 +234,19 @@ static void shift_codebook(elbg_data *elbg, int *indexes,
 }
 }
 
-static void evaluate_utility_inc(elbg_data *elbg)
+static void evaluate_utility_inc(ELBGContext *elbg)
 {
-int i;
 int64_t inc=0;
 
-for (i=0; i < elbg->numCB; i++) {
-if (elbg->numCB*elbg->utility[i] > elbg->error)
+for (int i = 0; i < elbg->num_cb; i++) {
+if (elbg->num_cb * elbg->utility[i] > elbg->error)
 inc += elbg->utility[i];
 elbg->utility_inc[i] = inc;
 }
 }
 
 
-static void update_utility_and_n_cb(elbg_data *elbg, int idx, int newutility)
+static void update_utility_and_n_cb(ELBGContext *elbg, int idx, int newutility)
 {
 cell *tempcell;
 
@@ -262,7 +262,7 @@ static void update_utility_and_n_cb(elbg_data *elbg, int 
idx, int newutility)
  * @param elbg  Internal elbg data
  * @param idx   {luc (low utility cell, huc (high utility cell), cluc (closest 
cell to low utility cell)}
  */
-static void try_shift_candidate(elbg_data *elbg, int idx[3])
+static void try_shift_candidate(ELBGContext *elbg, int idx[3])
 {
 int j, k, cont=0;
 int

[FFmpeg-devel] [PATCH 03/13] avcodec/elbg: Merge avpriv_init_elbg() into avpriv_do_elbg()

2021-09-16 Thread Andreas Rheinhardt
These functions are always called directly after another with
the exact same arguments. This avoids exporting a symbol;
it also avoids having to perform two calls for every caller.

Signed-off-by: Andreas Rheinhardt 
---
 libavcodec/a64multienc.c |  4 
 libavcodec/cinepakenc.c  |  1 -
 libavcodec/elbg.c| 29 +
 libavcodec/elbg.h| 12 
 libavcodec/msvideo1enc.c |  3 ---
 libavcodec/roqvideoenc.c |  4 
 libavfilter/vf_elbg.c|  3 ---
 7 files changed, 25 insertions(+), 31 deletions(-)

diff --git a/libavcodec/a64multienc.c b/libavcodec/a64multienc.c
index e2dd85b756..9ee0b07463 100644
--- a/libavcodec/a64multienc.c
+++ b/libavcodec/a64multienc.c
@@ -333,10 +333,6 @@ static int a64multi_encode_frame(AVCodecContext *avctx, 
AVPacket *pkt,
 buf = pkt->data;
 
 /* calc optimal new charset + charmaps */
-ret = avpriv_init_elbg(meta, 32, 1000 * c->mc_lifetime, best_cb,
-   CHARSET_CHARS, 50, charmap, &c->randctx);
-if (ret < 0)
-return ret;
 ret = avpriv_do_elbg(meta, 32, 1000 * c->mc_lifetime, best_cb,
  CHARSET_CHARS, 50, charmap, &c->randctx);
 if (ret < 0)
diff --git a/libavcodec/cinepakenc.c b/libavcodec/cinepakenc.c
index 41da231dfb..8e8b73ce1d 100644
--- a/libavcodec/cinepakenc.c
+++ b/libavcodec/cinepakenc.c
@@ -761,7 +761,6 @@ static int quantize(CinepakEncContext *s, int h, uint8_t 
*data[4],
 if (i < size)
 size = i;
 
-avpriv_init_elbg(s->codebook_input, entry_size, i, codebook, size, 1, 
s->codebook_closest, &s->randctx);
 avpriv_do_elbg(s->codebook_input, entry_size, i, codebook, size, 1, 
s->codebook_closest, &s->randctx);
 
 // set up vq_data, which contains a single MB
diff --git a/libavcodec/elbg.c b/libavcodec/elbg.c
index 3ca67b400c..b563254bbc 100644
--- a/libavcodec/elbg.c
+++ b/libavcodec/elbg.c
@@ -332,7 +332,7 @@ static void do_shiftings(elbg_data *elbg)
 }
 }
 
-int avpriv_do_elbg(int *points, int dim, int numpoints, int *codebook,
+static int do_elbg(int *points, int dim, int numpoints, int *codebook,
 int numCB, int max_steps, int *closest_cb,
 AVLFG *rand_state)
 {
@@ -426,7 +426,14 @@ out:
 
 #define BIG_PRIME 433494437LL
 
-int avpriv_init_elbg(int *points, int dim, int numpoints, int *codebook,
+/**
+ * Initialize the codebook vector for the elbg algorithm.
+ * If numpoints < 8*numCB this function fills codebook with random numbers.
+ * If not, it calls do_elbg for a (smaller) random sample of the points in
+ * points.
+ * @return < 0 in case of error, 0 otherwise
+ */
+static int init_elbg(int *points, int dim, int numpoints, int *codebook,
  int num_cb, int max_steps, int *closest_cb,
  AVLFG *rand_state)
 {
@@ -443,13 +450,13 @@ int avpriv_init_elbg(int *points, int dim, int numpoints, 
int *codebook,
 memcpy(temp_points + i*dim, points + k*dim, dim * 
sizeof(*temp_points));
 }
 
-ret = avpriv_init_elbg(temp_points, dim, numpoints / 8, codebook,
+ret = init_elbg(temp_points, dim, numpoints / 8, codebook,
num_cb, 2 * max_steps, closest_cb, rand_state);
 if (ret < 0) {
 av_freep(&temp_points);
 return ret;
 }
-ret = avpriv_do_elbg(temp_points, dim, numpoints / 8, codebook,
+ret = do_elbg  (temp_points, dim, numpoints / 8, codebook,
  num_cb, 2 * max_steps, closest_cb, rand_state);
 av_free(temp_points);
 } else  // If not, initialize the codebook with random positions
@@ -458,3 +465,17 @@ int avpriv_init_elbg(int *points, int dim, int numpoints, 
int *codebook,
dim * sizeof(*codebook));
 return ret;
 }
+
+int avpriv_do_elbg(int *points, int dim, int numpoints,
+   int *codebook, int num_cb, int max_steps,
+   int *closest_cb, AVLFG *rand_state)
+{
+int ret;
+
+ret = init_elbg(points, dim, numpoints, codebook,
+num_cb, max_steps, closest_cb, rand_state);
+if (ret < 0)
+return ret;
+return do_elbg (points, dim, numpoints, codebook,
+num_cb, max_steps, closest_cb, rand_state);
+}
diff --git a/libavcodec/elbg.h b/libavcodec/elbg.h
index f48aa3b443..3b9c2931f3 100644
--- a/libavcodec/elbg.h
+++ b/libavcodec/elbg.h
@@ -42,16 +42,4 @@ int avpriv_do_elbg(int *points, int dim, int numpoints, int 
*codebook,
int numCB, int num_steps, int *closest_cb,
AVLFG *rand_state);
 
-/**
- * Initialize the **codebook vector for the elbg algorithm. If you have already
- * a codebook and you want to refine it, you shouldn't call this function.
- * If numpoints < 8*numCB this function fills **codebook with random numbers.
- * If not, it calls avpriv_do_elbg for a (smaller) random sample o

[FFmpeg-devel] [PATCH 06/13] avcodec/elbg: Add persistent ELBGContext

2021-09-16 Thread Andreas Rheinhardt
It will be used in future commits to avoid having to allocate and free
all the buffers used.

Signed-off-by: Andreas Rheinhardt 
---
 libavcodec/a64multienc.c |  8 ++--
 libavcodec/cinepakenc.c  |  5 -
 libavcodec/elbg.c| 12 +++-
 libavcodec/elbg.h| 16 +---
 libavcodec/msvideo1enc.c | 12 +---
 libavcodec/roqvideoenc.c |  7 +--
 libavfilter/vf_elbg.c|  5 -
 7 files changed, 52 insertions(+), 13 deletions(-)

diff --git a/libavcodec/a64multienc.c b/libavcodec/a64multienc.c
index 9ee0b07463..71a620b4ff 100644
--- a/libavcodec/a64multienc.c
+++ b/libavcodec/a64multienc.c
@@ -43,6 +43,7 @@
 
 typedef struct A64Context {
 /* variables for multicolor modes */
+struct ELBGContext *elbg;
 AVLFG randctx;
 int mc_lifetime;
 int mc_use_5col;
@@ -195,6 +196,9 @@ static void render_charset(AVCodecContext *avctx, uint8_t 
*charset,
 static av_cold int a64multi_close_encoder(AVCodecContext *avctx)
 {
 A64Context *c = avctx->priv_data;
+
+avpriv_elbg_free(&c->elbg);
+
 av_freep(&c->mc_meta_charset);
 av_freep(&c->mc_best_cb);
 av_freep(&c->mc_charmap);
@@ -333,8 +337,8 @@ static int a64multi_encode_frame(AVCodecContext *avctx, 
AVPacket *pkt,
 buf = pkt->data;
 
 /* calc optimal new charset + charmaps */
-ret = avpriv_do_elbg(meta, 32, 1000 * c->mc_lifetime, best_cb,
- CHARSET_CHARS, 50, charmap, &c->randctx);
+ret = avpriv_elbg_do(&c->elbg, meta, 32, 1000 * c->mc_lifetime,
+ best_cb, CHARSET_CHARS, 50, charmap, 
&c->randctx);
 if (ret < 0)
 return ret;
 
diff --git a/libavcodec/cinepakenc.c b/libavcodec/cinepakenc.c
index 8e8b73ce1d..2984b93de3 100644
--- a/libavcodec/cinepakenc.c
+++ b/libavcodec/cinepakenc.c
@@ -127,6 +127,7 @@ typedef struct CinepakEncContext {
 int min_min_strips;
 int max_max_strips;
 int strip_number_delta_range;
+struct ELBGContext *elbg;
 } CinepakEncContext;
 
 #define OFFSET(x) offsetof(CinepakEncContext, x)
@@ -761,7 +762,8 @@ static int quantize(CinepakEncContext *s, int h, uint8_t 
*data[4],
 if (i < size)
 size = i;
 
-avpriv_do_elbg(s->codebook_input, entry_size, i, codebook, size, 1, 
s->codebook_closest, &s->randctx);
+avpriv_elbg_do(&s->elbg, s->codebook_input, entry_size, i, codebook,
+   size, 1, s->codebook_closest, &s->randctx);
 
 // set up vq_data, which contains a single MB
 vq_data[0] = vq_pict_buf;
@@ -1161,6 +1163,7 @@ static av_cold int cinepak_encode_end(AVCodecContext 
*avctx)
 CinepakEncContext *s = avctx->priv_data;
 int x;
 
+avpriv_elbg_free(&s->elbg);
 av_frame_free(&s->last_frame);
 av_frame_free(&s->best_frame);
 av_frame_free(&s->scratch_frame);
diff --git a/libavcodec/elbg.c b/libavcodec/elbg.c
index ac5c53161d..9eac802688 100644
--- a/libavcodec/elbg.c
+++ b/libavcodec/elbg.c
@@ -466,12 +466,17 @@ static int init_elbg(int *points, int dim, int numpoints, 
int *codebook,
 return ret;
 }
 
-int avpriv_do_elbg(int *points, int dim, int numpoints,
+int avpriv_elbg_do(ELBGContext **elbgp, int *points, int dim, int numpoints,
int *codebook, int num_cb, int max_steps,
int *closest_cb, AVLFG *rand_state)
 {
+ELBGContext *const elbg = *elbgp ? *elbgp : av_mallocz(sizeof(*elbg));
 int ret;
 
+if (!elbg)
+return AVERROR(ENOMEM);
+*elbgp = elbg;
+
 ret = init_elbg(points, dim, numpoints, codebook,
 num_cb, max_steps, closest_cb, rand_state);
 if (ret < 0)
@@ -479,3 +484,8 @@ int avpriv_do_elbg(int *points, int dim, int numpoints,
 return do_elbg (points, dim, numpoints, codebook,
 num_cb, max_steps, closest_cb, rand_state);
 }
+
+av_cold void avpriv_elbg_free(ELBGContext **elbgp)
+{
+av_freep(elbgp);
+}
diff --git a/libavcodec/elbg.h b/libavcodec/elbg.h
index a9a19aa5e4..abeeb4ff44 100644
--- a/libavcodec/elbg.h
+++ b/libavcodec/elbg.h
@@ -23,11 +23,16 @@
 
 #include "libavutil/lfg.h"
 
+struct ELBGContext;
+
 /**
  * Implementation of the Enhanced LBG Algorithm
  * Based on the paper "Neural Networks 14:1219-1237" that can be found in
  * http://citeseer.ist.psu.edu/patan01enhanced.html .
  *
+ * @param ctx  A pointer to a pointer to an already allocated ELBGContext
+ * or a pointer to NULL. In the latter case, this function
+ * will allocate an ELBGContext and put a pointer to it in `*ctx`.
  * @param points Input points.
  * @param dim Dimension of the points.
  * @param numpoints Num of points in **points.
@@ -38,8 +43,13 @@
  * @param rand_state A random number generator state. Should be already 
initialized by av_lfg_init().
  * @return < 0 in case of error, 0 otherwise
  */
-int avpriv_do_elbg(int *points, int dim, int numpoints, int *codebook,
-   int num_cb, int num_steps, int *closest

[FFmpeg-devel] [PATCH 04/13] avfilter/vf_elbg: Rename ELBGContext->ELBGFilterContext

2021-09-16 Thread Andreas Rheinhardt
The former name will be used for a context for avpriv_do_elbg().

Signed-off-by: Andreas Rheinhardt 
---
 libavfilter/vf_elbg.c | 18 +-
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/libavfilter/vf_elbg.c b/libavfilter/vf_elbg.c
index bf78030ffe..79797ee25f 100644
--- a/libavfilter/vf_elbg.c
+++ b/libavfilter/vf_elbg.c
@@ -33,7 +33,7 @@
 #include "internal.h"
 #include "video.h"
 
-typedef struct ELBGContext {
+typedef struct ELBGFilterContext {
 const AVClass *class;
 AVLFG lfg;
 int64_t lfg_seed;
@@ -46,9 +46,9 @@ typedef struct ELBGContext {
 const AVPixFmtDescriptor *pix_desc;
 uint8_t rgba_map[4];
 int pal8;
-} ELBGContext;
+} ELBGFilterContext;
 
-#define OFFSET(x) offsetof(ELBGContext, x)
+#define OFFSET(x) offsetof(ELBGFilterContext, x)
 #define FLAGS AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_FILTERING_PARAM
 
 static const AVOption elbg_options[] = {
@@ -66,7 +66,7 @@ AVFILTER_DEFINE_CLASS(elbg);
 
 static av_cold int init(AVFilterContext *ctx)
 {
-ELBGContext *elbg = ctx->priv;
+ELBGFilterContext *const elbg = ctx->priv;
 
 if (elbg->pal8 && elbg->codebook_length > 256) {
 av_log(ctx, AV_LOG_ERROR, "pal8 output allows max 256 codebook 
length.\n");
@@ -82,7 +82,7 @@ static av_cold int init(AVFilterContext *ctx)
 
 static int query_formats(AVFilterContext *ctx)
 {
-ELBGContext *elbg = ctx->priv;
+ELBGFilterContext *const elbg = ctx->priv;
 int ret;
 
 static const enum AVPixelFormat pix_fmts[] = {
@@ -109,7 +109,7 @@ static int query_formats(AVFilterContext *ctx)
 static int config_input(AVFilterLink *inlink)
 {
 AVFilterContext *ctx = inlink->dst;
-ELBGContext *elbg = ctx->priv;
+ELBGFilterContext *const elbg = ctx->priv;
 
 elbg->pix_desc = av_pix_fmt_desc_get(inlink->format);
 elbg->codeword_length = inlink->w * inlink->h;
@@ -140,7 +140,7 @@ static int config_input(AVFilterLink *inlink)
 
 static int filter_frame(AVFilterLink *inlink, AVFrame *frame)
 {
-ELBGContext *elbg = inlink->dst->priv;
+ELBGFilterContext *const elbg = inlink->dst->priv;
 int i, j, k;
 uint8_t *p, *p0;
 
@@ -221,7 +221,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame 
*frame)
 
 static av_cold void uninit(AVFilterContext *ctx)
 {
-ELBGContext *elbg = ctx->priv;
+ELBGFilterContext *const elbg = ctx->priv;
 
 av_freep(&elbg->codebook);
 av_freep(&elbg->codeword);
@@ -248,7 +248,7 @@ static const AVFilterPad elbg_outputs[] = {
 const AVFilter ff_vf_elbg = {
 .name  = "elbg",
 .description   = NULL_IF_CONFIG_SMALL("Apply posterize effect, using the 
ELBG algorithm."),
-.priv_size = sizeof(ELBGContext),
+.priv_size = sizeof(ELBGFilterContext),
 .priv_class= &elbg_class,
 .query_formats = query_formats,
 .init  = init,
-- 
2.30.2

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


[FFmpeg-devel] [PATCH 08/13] avcodec/elbg: Keep buffers to avoid allocations and frees

2021-09-16 Thread Andreas Rheinhardt
Up until now, each call to avpriv_elbg_do() would result
in at least six allocations. And this function is called a lot:
A typical FATE run results in 52213653 calls to av_malloc; of these,
34974671 originate from av_malloc_array and from these 34783679
originate from avpriv_elbg_do; the msvideo1 encoder tests are behind
most of these.

This commit changes this by keeping the buffers and only reallocating
them when needed. E.g. for the encoding part of fate-vsynth1-msvideo1
total heap usage went down from 11,407,939 allocs and frees with
468,106,207 bytes allocated to 3,149 allocs and frees with 13,181,847
bytes allocated. The time for one encode2-call went down by 69%.

Signed-off-by: Andreas Rheinhardt 
---
 libavcodec/elbg.c | 84 ++-
 1 file changed, 54 insertions(+), 30 deletions(-)

diff --git a/libavcodec/elbg.c b/libavcodec/elbg.c
index 24c6f06f54..4397bff1ef 100644
--- a/libavcodec/elbg.c
+++ b/libavcodec/elbg.c
@@ -53,8 +53,20 @@ typedef struct ELBGContext {
 int64_t *utility_inc;
 int *nearest_cb;
 int *points;
+int *size_part;
 AVLFG *rand_state;
 int *scratchbuf;
+cell *cell_buffer;
+
+/* Sizes for the buffers above. Pointers without such a field
+ * are not allocated by us and only valid for the duration
+ * of a single call to avpriv_elbg_do(). */
+unsigned utility_allocated;
+unsigned utility_inc_allocated;
+unsigned size_part_allocated;
+unsigned cells_allocated;
+unsigned scratchbuf_allocated;
+unsigned cell_buffer_allocated;
 } ELBGContext;
 
 static inline int distance_limited(int *a, int *b, int dim, int limit)
@@ -332,32 +344,19 @@ static void do_shiftings(ELBGContext *elbg)
 }
 }
 
-static int do_elbg(ELBGContext *elbg, int *points, int numpoints,
-   int max_steps)
+static void do_elbg(ELBGContext *elbg, int *points, int numpoints,
+int max_steps)
 {
-int i, j, steps = 0, ret = 0;
-int *size_part = av_malloc_array(elbg->num_cb, sizeof(int));
-cell *list_buffer = av_malloc_array(numpoints, sizeof(cell));
-cell *free_cells;
+int *const size_part = elbg->size_part;
+int i, j, steps = 0;
 int best_idx = 0;
 int64_t last_error;
 
 elbg->error = INT64_MAX;
-elbg->cells   = av_malloc_array(elbg->num_cb, sizeof(cell *));
-elbg->utility = av_malloc_array(elbg->num_cb, sizeof(*elbg->utility));
 elbg->points = points;
-elbg->utility_inc = av_malloc_array(elbg->num_cb, 
sizeof(*elbg->utility_inc));
-elbg->scratchbuf = av_malloc_array(5 * elbg->dim, sizeof(int));
-
-if (!size_part || !list_buffer || !elbg->cells ||
-!elbg->utility || !elbg->utility_inc || !elbg->scratchbuf) {
-ret = AVERROR(ENOMEM);
-goto out;
-}
-
 
 do {
-free_cells = list_buffer;
+cell *free_cells = elbg->cell_buffer;
 last_error = elbg->error;
 steps++;
 memset(elbg->utility, 0, elbg->num_cb * sizeof(*elbg->utility));
@@ -408,15 +407,6 @@ static int do_elbg(ELBGContext *elbg, int *points, int 
numpoints,
 
 } while(((last_error - elbg->error) > DELTA_ERR_MAX*elbg->error) &&
 (steps < max_steps));
-
-out:
-av_free(size_part);
-av_free(elbg->utility);
-av_free(list_buffer);
-av_free(elbg->cells);
-av_free(elbg->utility_inc);
-av_free(elbg->scratchbuf);
-return ret;
 }
 
 #define BIG_PRIME 433494437LL
@@ -450,13 +440,13 @@ static int init_elbg(ELBGContext *elbg, int *points, int 
numpoints,
 av_freep(&temp_points);
 return ret;
 }
-ret =   do_elbg(elbg, temp_points, numpoints / 8, 2 * max_steps);
+do_elbg(elbg, temp_points, numpoints / 8, 2 * max_steps);
 av_free(temp_points);
 } else  // If not, initialize the codebook with random positions
 for (int i = 0; i < elbg->num_cb; i++)
 memcpy(elbg->codebook + i * dim, points + 
((i*BIG_PRIME)%numpoints)*dim,
dim * sizeof(*elbg->codebook));
-return ret;
+return 0;
 }
 
 int avpriv_elbg_do(ELBGContext **elbgp, int *points, int dim, int numpoints,
@@ -476,13 +466,47 @@ int avpriv_elbg_do(ELBGContext **elbgp, int *points, int 
dim, int numpoints,
 elbg->num_cb = num_cb;
 elbg->dim= dim;
 
+#define ALLOCATE_IF_NECESSARY(field, new_elements, multiplicator)\
+if (elbg->field ## _allocated < new_elements) {  \
+av_freep(&elbg->field);  \
+elbg->field = av_malloc_array(new_elements,  \
+  multiplicator * sizeof(*elbg->field)); \
+if (!elbg->field) {  \
+elbg->field ## _allocated = 0;   \
+return AVERROR(ENOMEM);  \
+} 

[FFmpeg-devel] [PATCH 11/13] avcodec/msvideo1enc: Check all calls to avpriv_elbg_do()

2021-09-16 Thread Andreas Rheinhardt
Signed-off-by: Andreas Rheinhardt 
---
 libavcodec/msvideo1enc.c | 20 +---
 1 file changed, 13 insertions(+), 7 deletions(-)

diff --git a/libavcodec/msvideo1enc.c b/libavcodec/msvideo1enc.c
index d43013ba5f..79810ec8c3 100644
--- a/libavcodec/msvideo1enc.c
+++ b/libavcodec/msvideo1enc.c
@@ -118,8 +118,10 @@ static int encode_frame(AVCodecContext *avctx, AVPacket 
*pkt,
 }
 // try to find optimal value to fill whole 4x4 block
 score = 0;
-avpriv_elbg_do(&c->elbg, c->block, 3, 16, c->avg,
-   1, 1, c->output, &c->rnd);
+ret = avpriv_elbg_do(&c->elbg, c->block, 3, 16, c->avg,
+ 1, 1, c->output, &c->rnd);
+if (ret < 0)
+return ret;
 if(c->avg[0] == 1) // red component = 1 will be written as skip 
code
 c->avg[0] = 0;
 for(j = 0; j < 4; j++){
@@ -138,8 +140,10 @@ static int encode_frame(AVCodecContext *avctx, AVPacket 
*pkt,
 }
 // search for optimal filling of 2-color block
 score = 0;
-avpriv_elbg_do(&c->elbg, c->block, 3, 16, c->codebook,
-   2, 1, c->output, &c->rnd);
+ret = avpriv_elbg_do(&c->elbg, c->block, 3, 16, c->codebook,
+ 2, 1, c->output, &c->rnd);
+if (ret < 0)
+return ret;
 // last output value should be always 1, swap codebooks if needed
 if(!c->output[15]){
 for(i = 0; i < 3; i++)
@@ -164,9 +168,11 @@ static int encode_frame(AVCodecContext *avctx, AVPacket 
*pkt,
 // search for optimal filling of 2-color 2x2 subblocks
 score = 0;
 for(i = 0; i < 4; i++){
-avpriv_elbg_do(&c->elbg, c->block2 + i * 4 * 3, 3, 4,
-   c->codebook2 + i * 2 * 3, 2, 1,
-   c->output2 + i*4, &c->rnd);
+ret = avpriv_elbg_do(&c->elbg, c->block2 + i * 4 * 3, 3, 4,
+ c->codebook2 + i * 2 * 3, 2, 1,
+ c->output2 + i * 4, &c->rnd);
+if (ret < 0)
+return ret;
 }
 // last value should be always 1, swap codebooks if needed
 if(!c->output2[15]){
-- 
2.30.2

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


[FFmpeg-devel] [PATCH 13/13] avcodec/elbg: Add flags to avpriv_elbg_do()

2021-09-16 Thread Andreas Rheinhardt
This is currently unused and it is only added to enable changes
while maintaining ABI compatibility. The type is uintptr_t in order
to potentially accept a pointer argument.

Signed-off-by: Andreas Rheinhardt 
---
 libavcodec/a64multienc.c | 2 +-
 libavcodec/cinepakenc.c  | 2 +-
 libavcodec/elbg.c| 2 +-
 libavcodec/elbg.h| 4 +++-
 libavcodec/msvideo1enc.c | 6 +++---
 libavcodec/roqvideoenc.c | 2 +-
 libavfilter/vf_elbg.c| 2 +-
 7 files changed, 11 insertions(+), 9 deletions(-)

diff --git a/libavcodec/a64multienc.c b/libavcodec/a64multienc.c
index 71a620b4ff..c368ac218a 100644
--- a/libavcodec/a64multienc.c
+++ b/libavcodec/a64multienc.c
@@ -338,7 +338,7 @@ static int a64multi_encode_frame(AVCodecContext *avctx, 
AVPacket *pkt,
 
 /* calc optimal new charset + charmaps */
 ret = avpriv_elbg_do(&c->elbg, meta, 32, 1000 * c->mc_lifetime,
- best_cb, CHARSET_CHARS, 50, charmap, 
&c->randctx);
+ best_cb, CHARSET_CHARS, 50, charmap, 
&c->randctx, 0);
 if (ret < 0)
 return ret;
 
diff --git a/libavcodec/cinepakenc.c b/libavcodec/cinepakenc.c
index d1bcf2b2d5..edb553f0db 100644
--- a/libavcodec/cinepakenc.c
+++ b/libavcodec/cinepakenc.c
@@ -764,7 +764,7 @@ static int quantize(CinepakEncContext *s, int h, uint8_t 
*data[4],
 size = i;
 
 ret = avpriv_elbg_do(&s->elbg, s->codebook_input, entry_size, i, codebook,
- size, 1, s->codebook_closest, &s->randctx);
+ size, 1, s->codebook_closest, &s->randctx, 0);
 if (ret < 0)
 return ret;
 
diff --git a/libavcodec/elbg.c b/libavcodec/elbg.c
index 2bacf5b773..712c125a58 100644
--- a/libavcodec/elbg.c
+++ b/libavcodec/elbg.c
@@ -445,7 +445,7 @@ static void init_elbg(ELBGContext *elbg, int *points, int 
*temp_points,
 
 int avpriv_elbg_do(ELBGContext **elbgp, int *points, int dim, int numpoints,
int *codebook, int num_cb, int max_steps,
-   int *closest_cb, AVLFG *rand_state)
+   int *closest_cb, AVLFG *rand_state, uintptr_t flags)
 {
 ELBGContext *const elbg = *elbgp ? *elbgp : av_mallocz(sizeof(*elbg));
 
diff --git a/libavcodec/elbg.h b/libavcodec/elbg.h
index abeeb4ff44..34d96846b1 100644
--- a/libavcodec/elbg.h
+++ b/libavcodec/elbg.h
@@ -21,6 +21,7 @@
 #ifndef AVCODEC_ELBG_H
 #define AVCODEC_ELBG_H
 
+#include 
 #include "libavutil/lfg.h"
 
 struct ELBGContext;
@@ -41,11 +42,12 @@ struct ELBGContext;
  * @param num_steps The maximum number of steps. One step is already a good 
compromise between time and quality.
  * @param closest_cb Return the closest codebook to each point. Must be 
allocated.
  * @param rand_state A random number generator state. Should be already 
initialized by av_lfg_init().
+ * @param flags Currently unused; must be set to 0.
  * @return < 0 in case of error, 0 otherwise
  */
 int avpriv_elbg_do(struct ELBGContext **ctx, int *points, int dim,
int numpoints, int *codebook, int num_cb, int num_steps,
-   int *closest_cb, AVLFG *rand_state);
+   int *closest_cb, AVLFG *rand_state, uintptr_t flags);
 
 /**
  * Free an ELBGContext and reset the pointer to it.
diff --git a/libavcodec/msvideo1enc.c b/libavcodec/msvideo1enc.c
index 79810ec8c3..a8761bdd2e 100644
--- a/libavcodec/msvideo1enc.c
+++ b/libavcodec/msvideo1enc.c
@@ -119,7 +119,7 @@ static int encode_frame(AVCodecContext *avctx, AVPacket 
*pkt,
 // try to find optimal value to fill whole 4x4 block
 score = 0;
 ret = avpriv_elbg_do(&c->elbg, c->block, 3, 16, c->avg,
- 1, 1, c->output, &c->rnd);
+ 1, 1, c->output, &c->rnd, 0);
 if (ret < 0)
 return ret;
 if(c->avg[0] == 1) // red component = 1 will be written as skip 
code
@@ -141,7 +141,7 @@ static int encode_frame(AVCodecContext *avctx, AVPacket 
*pkt,
 // search for optimal filling of 2-color block
 score = 0;
 ret = avpriv_elbg_do(&c->elbg, c->block, 3, 16, c->codebook,
- 2, 1, c->output, &c->rnd);
+ 2, 1, c->output, &c->rnd, 0);
 if (ret < 0)
 return ret;
 // last output value should be always 1, swap codebooks if needed
@@ -170,7 +170,7 @@ static int encode_frame(AVCodecContext *avctx, AVPacket 
*pkt,
 for(i = 0; i < 4; i++){
 ret = avpriv_elbg_do(&c->elbg, c->block2 + i * 4 * 3, 3, 4,
  c->codebook2 + i * 2 * 3, 2, 1,
- c->output2 + i * 4, &c->rnd);
+ c->output2 + i * 4, &c->rnd, 0);
 if (ret < 0)
 return ret;
 }
diff --git a/libavcodec/roqvideoenc.c b/libavcodec/roqvideoenc

[FFmpeg-devel] [PATCH 07/13] avcodec/elbg: Move arguments to the context early if possible

2021-09-16 Thread Andreas Rheinhardt
This affects all the arguments that don't change during a call
to avpriv_elbg_do(); doing so makes it easily recognizable which
arguments change upon recursive calls.

Signed-off-by: Andreas Rheinhardt 
---
 libavcodec/elbg.c | 73 +++
 1 file changed, 35 insertions(+), 38 deletions(-)

diff --git a/libavcodec/elbg.c b/libavcodec/elbg.c
index 9eac802688..24c6f06f54 100644
--- a/libavcodec/elbg.c
+++ b/libavcodec/elbg.c
@@ -332,30 +332,22 @@ static void do_shiftings(ELBGContext *elbg)
 }
 }
 
-static int do_elbg(int *points, int dim, int numpoints, int *codebook,
-   int num_cb, int max_steps, int *closest_cb,
-AVLFG *rand_state)
+static int do_elbg(ELBGContext *elbg, int *points, int numpoints,
+   int max_steps)
 {
-int dist;
-ELBGContext elbg_d;
-ELBGContext *elbg = &elbg_d;
 int i, j, steps = 0, ret = 0;
-int *size_part = av_malloc_array(num_cb, sizeof(int));
+int *size_part = av_malloc_array(elbg->num_cb, sizeof(int));
 cell *list_buffer = av_malloc_array(numpoints, sizeof(cell));
 cell *free_cells;
-int best_dist, best_idx = 0;
+int best_idx = 0;
 int64_t last_error;
 
 elbg->error = INT64_MAX;
-elbg->dim = dim;
-elbg->num_cb = num_cb;
-elbg->codebook = codebook;
-elbg->cells = av_malloc_array(num_cb, sizeof(cell *));
-elbg->utility = av_malloc_array(num_cb, sizeof(*elbg->utility));
-elbg->nearest_cb = closest_cb;
+elbg->cells   = av_malloc_array(elbg->num_cb, sizeof(cell *));
+elbg->utility = av_malloc_array(elbg->num_cb, sizeof(*elbg->utility));
 elbg->points = points;
-elbg->utility_inc = av_malloc_array(num_cb, sizeof(*elbg->utility_inc));
-elbg->scratchbuf = av_malloc_array(5*dim, sizeof(int));
+elbg->utility_inc = av_malloc_array(elbg->num_cb, 
sizeof(*elbg->utility_inc));
+elbg->scratchbuf = av_malloc_array(5 * elbg->dim, sizeof(int));
 
 if (!size_part || !list_buffer || !elbg->cells ||
 !elbg->utility || !elbg->utility_inc || !elbg->scratchbuf) {
@@ -363,23 +355,26 @@ static int do_elbg(int *points, int dim, int numpoints, 
int *codebook,
 goto out;
 }
 
-elbg->rand_state = rand_state;
 
 do {
 free_cells = list_buffer;
 last_error = elbg->error;
 steps++;
-memset(elbg->utility, 0, num_cb * sizeof(*elbg->utility));
-memset(elbg->cells,   0, num_cb * sizeof(*elbg->cells));
+memset(elbg->utility, 0, elbg->num_cb * sizeof(*elbg->utility));
+memset(elbg->cells,   0, elbg->num_cb * sizeof(*elbg->cells));
 
 elbg->error = 0;
 
 /* This loop evaluate the actual Voronoi partition. It is the most
costly part of the algorithm. */
 for (i=0; i < numpoints; i++) {
-best_dist = distance_limited(elbg->points + i*elbg->dim, 
elbg->codebook + best_idx*elbg->dim, dim, INT_MAX);
+int best_dist = distance_limited(elbg->points   + i * elbg->dim,
+ elbg->codebook + best_idx * 
elbg->dim,
+ elbg->dim, INT_MAX);
 for (int k = 0; k < elbg->num_cb; k++) {
-dist = distance_limited(elbg->points + i*elbg->dim, 
elbg->codebook + k*elbg->dim, dim, best_dist);
+int dist = distance_limited(elbg->points   + i * elbg->dim,
+elbg->codebook + k * elbg->dim,
+elbg->dim, best_dist);
 if (dist < best_dist) {
 best_dist = dist;
 best_idx = k;
@@ -396,9 +391,9 @@ static int do_elbg(int *points, int dim, int numpoints, int 
*codebook,
 
 do_shiftings(elbg);
 
-memset(size_part, 0, num_cb * sizeof(*size_part));
+memset(size_part,  0, elbg->num_cb * sizeof(*size_part));
 
-memset(elbg->codebook, 0, elbg->num_cb * dim * 
sizeof(*elbg->codebook));
+memset(elbg->codebook, 0, elbg->num_cb * elbg->dim * 
sizeof(*elbg->codebook));
 
 for (i=0; i < numpoints; i++) {
 size_part[elbg->nearest_cb[i]]++;
@@ -433,13 +428,13 @@ out:
  * points.
  * @return < 0 in case of error, 0 otherwise
  */
-static int init_elbg(int *points, int dim, int numpoints, int *codebook,
- int num_cb, int max_steps, int *closest_cb,
- AVLFG *rand_state)
+static int init_elbg(ELBGContext *elbg, int *points, int numpoints,
+ int max_steps)
 {
+int dim = elbg->dim;
 int ret = 0;
 
-if (numpoints > 24LL * num_cb) {
+if (numpoints > 24LL * elbg->num_cb) {
 /* ELBG is very costly for a big number of points. So if we have a lot
of them, get a good initial codebook to save on iterations   */
 int *temp_points = av_malloc_array(dim, 
(numpoints/8)*sizeof(*temp_points));
@@ -450,19 +445,17 @@ static

[FFmpeg-devel] [PATCH 09/13] avcodec/elbg: Also allocate buffers for recursion only once

2021-09-16 Thread Andreas Rheinhardt
This is possible because the number of elements needed in each
recursion step decreases geometrically, so the geometric series
provides an upper bound for the sum of number of elements of
the needed buffers.

Signed-off-by: Andreas Rheinhardt 
---
 libavcodec/elbg.c | 38 --
 1 file changed, 20 insertions(+), 18 deletions(-)

diff --git a/libavcodec/elbg.c b/libavcodec/elbg.c
index 4397bff1ef..2bacf5b773 100644
--- a/libavcodec/elbg.c
+++ b/libavcodec/elbg.c
@@ -53,6 +53,7 @@ typedef struct ELBGContext {
 int64_t *utility_inc;
 int *nearest_cb;
 int *points;
+int *temp_points;
 int *size_part;
 AVLFG *rand_state;
 int *scratchbuf;
@@ -67,6 +68,7 @@ typedef struct ELBGContext {
 unsigned cells_allocated;
 unsigned scratchbuf_allocated;
 unsigned cell_buffer_allocated;
+unsigned temp_points_allocated;
 } ELBGContext;
 
 static inline int distance_limited(int *a, int *b, int dim, int limit)
@@ -416,37 +418,29 @@ static void do_elbg(ELBGContext *elbg, int *points, int 
numpoints,
  * If numpoints <= 24 * num_cb this function fills codebook with random 
numbers.
  * If not, it calls do_elbg for a (smaller) random sample of the points in
  * points.
- * @return < 0 in case of error, 0 otherwise
  */
-static int init_elbg(ELBGContext *elbg, int *points, int numpoints,
- int max_steps)
+static void init_elbg(ELBGContext *elbg, int *points, int *temp_points,
+  int numpoints, int max_steps)
 {
 int dim = elbg->dim;
-int ret = 0;
 
 if (numpoints > 24LL * elbg->num_cb) {
 /* ELBG is very costly for a big number of points. So if we have a lot
of them, get a good initial codebook to save on iterations   */
-int *temp_points = av_malloc_array(dim, 
(numpoints/8)*sizeof(*temp_points));
-if (!temp_points)
-return AVERROR(ENOMEM);
 for (int i = 0; i < numpoints / 8; i++) {
 int k = (i*BIG_PRIME) % numpoints;
 memcpy(temp_points + i*dim, points + k*dim, dim * 
sizeof(*temp_points));
 }
 
-ret = init_elbg(elbg, temp_points, numpoints / 8, 2 * max_steps);
-if (ret < 0) {
-av_freep(&temp_points);
-return ret;
-}
+/* If anything is changed in the recursion parameters,
+ * the allocated size of temp_points will also need to be updated. */
+init_elbg(elbg, temp_points, temp_points + numpoints / 8 * dim,
+  numpoints / 8, 2 * max_steps);
 do_elbg(elbg, temp_points, numpoints / 8, 2 * max_steps);
-av_free(temp_points);
 } else  // If not, initialize the codebook with random positions
 for (int i = 0; i < elbg->num_cb; i++)
 memcpy(elbg->codebook + i * dim, points + 
((i*BIG_PRIME)%numpoints)*dim,
dim * sizeof(*elbg->codebook));
-return 0;
 }
 
 int avpriv_elbg_do(ELBGContext **elbgp, int *points, int dim, int numpoints,
@@ -454,7 +448,6 @@ int avpriv_elbg_do(ELBGContext **elbgp, int *points, int 
dim, int numpoints,
int *closest_cb, AVLFG *rand_state)
 {
 ELBGContext *const elbg = *elbgp ? *elbgp : av_mallocz(sizeof(*elbg));
-int ret;
 
 if (!elbg)
 return AVERROR(ENOMEM);
@@ -487,10 +480,18 @@ int avpriv_elbg_do(ELBGContext **elbgp, int *points, int 
dim, int numpoints,
 ALLOCATE_IF_NECESSARY(size_part,   num_cb,1)
 ALLOCATE_IF_NECESSARY(cell_buffer, numpoints, 1)
 ALLOCATE_IF_NECESSARY(scratchbuf,  dim,   5)
+if (numpoints > 24LL * elbg->num_cb) {
+/* The first step in the recursion in init_elbg() needs a buffer with
+* (numpoints / 8) * dim elements; the next step needs numpoints / 8 / 8
+* * dim elements etc. The geometric series leads to an upper bound of
+* numpoints / 8 * 8 / 7 * dim elements. */
+uint64_t prod = dim * (uint64_t)(numpoints / 7U);
+if (prod > INT_MAX)
+return AVERROR(ERANGE);
+ALLOCATE_IF_NECESSARY(temp_points, prod, 1)
+}
 
-ret = init_elbg(elbg, points, numpoints, max_steps);
-if (ret < 0)
-return ret;
+init_elbg(elbg, points, elbg->temp_points, numpoints, max_steps);
 do_elbg (elbg, points, numpoints, max_steps);
 return 0;
 }
@@ -507,6 +508,7 @@ av_cold void avpriv_elbg_free(ELBGContext **elbgp)
 av_freep(&elbg->cells);
 av_freep(&elbg->utility_inc);
 av_freep(&elbg->scratchbuf);
+av_freep(&elbg->temp_points);
 
 av_freep(elbgp);
 }
-- 
2.30.2

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


[FFmpeg-devel] [PATCH 10/13] avcodec/cinepakenc: Check all calls to avpriv_elbg_do()

2021-09-16 Thread Andreas Rheinhardt
Signed-off-by: Andreas Rheinhardt 
---
 libavcodec/cinepakenc.c | 28 ++--
 1 file changed, 22 insertions(+), 6 deletions(-)

diff --git a/libavcodec/cinepakenc.c b/libavcodec/cinepakenc.c
index 2984b93de3..d1bcf2b2d5 100644
--- a/libavcodec/cinepakenc.c
+++ b/libavcodec/cinepakenc.c
@@ -709,6 +709,7 @@ static int quantize(CinepakEncContext *s, int h, uint8_t 
*data[4],
 uint8_t vq_pict_buf[(MB_AREA * 3) / 2];
 uint8_t *sub_data[4], *vq_data[4];
 int  sub_linesize[4],  vq_linesize[4];
+int ret;
 
 for (mbn = i = y = 0; y < h; y += MB_SIZE) {
 for (x = 0; x < s->w; x += MB_SIZE, ++mbn) {
@@ -762,8 +763,10 @@ static int quantize(CinepakEncContext *s, int h, uint8_t 
*data[4],
 if (i < size)
 size = i;
 
-avpriv_elbg_do(&s->elbg, s->codebook_input, entry_size, i, codebook,
-   size, 1, s->codebook_closest, &s->randctx);
+ret = avpriv_elbg_do(&s->elbg, s->codebook_input, entry_size, i, codebook,
+ size, 1, s->codebook_closest, &s->randctx);
+if (ret < 0)
+return ret;
 
 // set up vq_data, which contains a single MB
 vq_data[0] = vq_pict_buf;
@@ -888,8 +891,10 @@ static int rd_strip(CinepakEncContext *s, int y, int h, 
int keyframe,
 if (mode == MODE_V1_ONLY) {
 info.v1_size = v1_size;
 // the size may shrink even before optimizations if the 
input is short:
-info.v1_size = quantize(s, h, data, linesize, 1,
-&info, ENC_UNCERTAIN);
+if ((new_v1_size = quantize(s, h, data, linesize, 1,
+&info, ENC_UNCERTAIN)) < 0)
+return new_v1_size;
+info.v1_size = new_v1_size;
 if (info.v1_size < v1_size)
 // too few eligible blocks, no sense in trying bigger 
sizes
 v1enough = 1;
@@ -902,8 +907,11 @@ static int rd_strip(CinepakEncContext *s, int y, int h, 
int keyframe,
 
 if (mode == MODE_V1_V4) {
 info.v4_size = v4_size;
-info.v4_size = quantize(s, h, data, linesize, 0,
-&info, ENC_UNCERTAIN);
+new_v4_size = quantize(s, h, data, linesize, 0,
+   &info, ENC_UNCERTAIN);
+if (new_v4_size < 0)
+return new_v4_size;
+info.v4_size = new_v4_size;
 if (info.v4_size < v4_size)
 // too few eligible blocks, no sense in trying 
bigger sizes
 v4enough = 1;
@@ -921,11 +929,15 @@ static int rd_strip(CinepakEncContext *s, int y, int h, 
int keyframe,
 // we assume we _may_ come here with more blocks to encode 
than before
 info.v1_size = v1_size;
 new_v1_size = quantize(s, h, data, linesize, 1, &info, 
ENC_V1);
+if (new_v1_size < 0)
+return new_v1_size;
 if (new_v1_size < info.v1_size)
 info.v1_size = new_v1_size;
 // we assume we _may_ come here with more blocks to encode 
than before
 info.v4_size = v4_size;
 new_v4_size = quantize(s, h, data, linesize, 0, &info, 
ENC_V4);
+if (new_v4_size < 0)
+return new_v4_size;
 if (new_v4_size < info.v4_size)
 info.v4_size = new_v4_size;
 // calculate the resulting score
@@ -942,12 +954,16 @@ static int rd_strip(CinepakEncContext *s, int y, int h, 
int keyframe,
 if (v1shrunk) {
 info.v1_size = v1_size;
 new_v1_size = quantize(s, h, data, linesize, 1, 
&info, ENC_V1);
+if (new_v1_size < 0)
+return new_v1_size;
 if (new_v1_size < info.v1_size)
 info.v1_size = new_v1_size;
 }
 if (v4shrunk) {
 info.v4_size = v4_size;
 new_v4_size = quantize(s, h, data, linesize, 0, 
&info, ENC_V4);
+if (new_v4_size < 0)
+return new_v4_size;
 if (new_v4_size < info.v4_size)
 info.v4_size = new_v4_size;
 }
-- 
2.30.2

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or 

[FFmpeg-devel] [PATCH 12/13] avfilter/vf_elbg: Check call to avpriv_elbg_do()

2021-09-16 Thread Andreas Rheinhardt
Signed-off-by: Andreas Rheinhardt 
---
 libavfilter/vf_elbg.c | 13 +
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/libavfilter/vf_elbg.c b/libavfilter/vf_elbg.c
index 2c9c861d02..fac3b6f7fe 100644
--- a/libavfilter/vf_elbg.c
+++ b/libavfilter/vf_elbg.c
@@ -142,7 +142,7 @@ static int config_input(AVFilterLink *inlink)
 static int filter_frame(AVFilterLink *inlink, AVFrame *frame)
 {
 ELBGFilterContext *const elbg = inlink->dst->priv;
-int i, j, k;
+int i, j, k, ret;
 uint8_t *p, *p0;
 
 const uint8_t r_idx  = elbg->rgba_map[R];
@@ -164,9 +164,14 @@ static int filter_frame(AVFilterLink *inlink, AVFrame 
*frame)
 }
 
 /* compute the codebook */
-avpriv_elbg_do(&elbg->ctx, elbg->codeword, NB_COMPONENTS, 
elbg->codeword_length,
-   elbg->codebook, elbg->codebook_length, elbg->max_steps_nb,
-   elbg->codeword_closest_codebook_idxs, &elbg->lfg);
+ret = avpriv_elbg_do(&elbg->ctx, elbg->codeword, NB_COMPONENTS,
+ elbg->codeword_length, elbg->codebook,
+ elbg->codebook_length, elbg->max_steps_nb,
+ elbg->codeword_closest_codebook_idxs, &elbg->lfg);
+if (ret < 0) {
+av_frame_free(&frame);
+return ret;
+}
 
 if (elbg->pal8) {
 AVFilterLink *outlink = inlink->dst->outputs[0];
-- 
2.30.2

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH v1 1/1] avformat/amr: Return PATCHWELCOME on stereo files

2021-09-16 Thread Sun Zhenliang
Ping.

Anyone could review this patch?

Thx.
在 2021年9月16日 +0800 11:24,sunzhenliang ,写道:
> Signed-off-by: sunzhenliang 
> ---
> libavformat/amr.c | 22 ++
> 1 file changed, 18 insertions(+), 4 deletions(-)
>
> diff --git a/libavformat/amr.c b/libavformat/amr.c
> index 836b276fd5..2762010ebe 100644
> --- a/libavformat/amr.c
> +++ b/libavformat/amr.c
> @@ -36,8 +36,10 @@ typedef struct {
> uint64_t block_count;
> } AMRContext;
>
> -static const char AMR_header[] = "#!AMR\n";
> -static const char AMRWB_header[] = "#!AMR-WB\n";
> +static const char AMR_header[] = "#!AMR\n";
> +static const char AMR_MC_header[] = "#!AMR_MC1.0\n";
> +static const char AMRWB_header[] = "#!AMR-WB\n";
> +static const char AMRWB_MC_header[] = "#!AMR-WB_MC1.0\n";
>
> static const uint8_t amrnb_packed_size[16] = {
> 13, 14, 16, 18, 20, 21, 27, 32, 6, 1, 1, 1, 1, 1, 1, 1
> @@ -82,7 +84,7 @@ static int amr_read_header(AVFormatContext *s)
> {
> AVIOContext *pb = s->pb;
> AVStream *st;
> - uint8_t header[9];
> + uint8_t header[15];
>
> if (avio_read(pb, header, 6) != 6)
> return AVERROR_INVALIDDATA;
> @@ -94,7 +96,19 @@ static int amr_read_header(AVFormatContext *s)
> if (avio_read(pb, header + 6, 3) != 3)
> return AVERROR_INVALIDDATA;
> if (memcmp(header, AMRWB_header, 9)) {
> - return -1;
> + if (avio_read(pb, header + 6 + 3, 3) != 3)
> + return AVERROR_INVALIDDATA;
> + if (memcmp(header, AMR_MC_header, 12)) {
> + if (avio_read(pb, header + 6 + 3 + 3, 3) != 3)
> + return AVERROR_INVALIDDATA;
> + if (memcmp(header, AMRWB_MC_header, 15)) {
> + return -1;
> + }
> + avpriv_report_missing_feature(s, "multi-channel AMRWB");
> + return AVERROR_PATCHWELCOME;
> + }
> + avpriv_report_missing_feature(s, "multi-channel AMR");
> + return AVERROR_PATCHWELCOME;
> }
>
> st->codecpar->codec_tag = MKTAG('s', 'a', 'w', 'b');
> --
> 2.25.1
>
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


[FFmpeg-devel] [PATCH] avfilter/vf_zscale: fix output color_range discrepancy

2021-09-16 Thread Pavel Koshevoy
This filter chain was supposed to convert from narrow range
to full range yuv444p, but didn't:

buffer=width=1280:height=720:pix_fmt=yuv444p:frame_rate=25/1:\
time_base=1/25:sar=1/1,zscale=min=709:rin=limited:pin=709:\
tin=709:t=linear,format=gbrpf32le,zscale=tin=linear:p=709:m=709:\
r=full:t=709,format=pix_fmts=yuv444p,buffersink
---
 libavfilter/vf_zscale.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavfilter/vf_zscale.c b/libavfilter/vf_zscale.c
index dfea00f9eb..06a025e6e6 100644
--- a/libavfilter/vf_zscale.c
+++ b/libavfilter/vf_zscale.c
@@ -676,7 +676,7 @@ static int filter_frame(AVFilterLink *link, AVFrame *in)
 out->color_primaries = (int)s->dst_format.color_primaries;
 
 if (s->range != -1)
-out->color_range = (int)s->dst_format.pixel_range;
+out->color_range = (int)s->dst_format.pixel_range + 1;
 
 if (s->trc != -1)
 out->color_trc = (int)s->dst_format.transfer_characteristics;
-- 
2.26.2

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH] avutil/buffer: Avoid allocation of AVBuffer when using buffer pool

2021-09-16 Thread Andreas Rheinhardt
Andreas Rheinhardt:
> Do this by putting an AVBuffer structure into BufferPoolEntry and
> reuse it for all subsequent uses of said BufferPoolEntry.
> 
> Signed-off-by: Andreas Rheinhardt 
> ---
>  libavutil/buffer.c  | 44 +
>  libavutil/buffer_internal.h | 11 ++
>  2 files changed, 41 insertions(+), 14 deletions(-)
> 
> diff --git a/libavutil/buffer.c b/libavutil/buffer.c
> index b13eeadffb..4d9ccf74b7 100644
> --- a/libavutil/buffer.c
> +++ b/libavutil/buffer.c
> @@ -26,16 +26,11 @@
>  #include "mem.h"
>  #include "thread.h"
>  
> -AVBufferRef *av_buffer_create(uint8_t *data, size_t size,
> -  void (*free)(void *opaque, uint8_t *data),
> -  void *opaque, int flags)
> +static AVBufferRef *buffer_create(AVBuffer *buf, uint8_t *data, size_t size,
> +  void (*free)(void *opaque, uint8_t *data),
> +  void *opaque, int flags)
>  {
>  AVBufferRef *ref = NULL;
> -AVBuffer*buf = NULL;
> -
> -buf = av_mallocz(sizeof(*buf));
> -if (!buf)
> -return NULL;
>  
>  buf->data = data;
>  buf->size = size;
> @@ -47,10 +42,8 @@ AVBufferRef *av_buffer_create(uint8_t *data, size_t size,
>  buf->flags = flags;
>  
>  ref = av_mallocz(sizeof(*ref));
> -if (!ref) {
> -av_freep(&buf);
> +if (!ref)
>  return NULL;
> -}
>  
>  ref->buffer = buf;
>  ref->data   = data;
> @@ -59,6 +52,23 @@ AVBufferRef *av_buffer_create(uint8_t *data, size_t size,
>  return ref;
>  }
>  
> +AVBufferRef *av_buffer_create(uint8_t *data, size_t size,
> +  void (*free)(void *opaque, uint8_t *data),
> +  void *opaque, int flags)
> +{
> +AVBufferRef *ret;
> +AVBuffer *buf = av_mallocz(sizeof(*buf));
> +if (!buf)
> +return NULL;
> +
> +ret = buffer_create(buf, data, size, free, opaque, flags);
> +if (!ret) {
> +av_free(buf);
> +return NULL;
> +}
> +return ret;
> +}
> +
>  void av_buffer_default_free(void *opaque, uint8_t *data)
>  {
>  av_free(data);
> @@ -117,8 +127,12 @@ static void buffer_replace(AVBufferRef **dst, 
> AVBufferRef **src)
>  av_freep(dst);
>  
>  if (atomic_fetch_sub_explicit(&b->refcount, 1, memory_order_acq_rel) == 
> 1) {
> +/* b->free below might already free the structure containing *b,
> + * so we have to read the flag now to avoid use-after-free. */
> +int free_avbuffer = !(b->flags_internal & BUFFER_FLAG_NO_FREE);
>  b->free(b->opaque, b->data);
> -av_freep(&b);
> +if (free_avbuffer)
> +av_free(b);
>  }
>  }
>  
> @@ -378,11 +392,13 @@ AVBufferRef *av_buffer_pool_get(AVBufferPool *pool)
>  ff_mutex_lock(&pool->mutex);
>  buf = pool->pool;
>  if (buf) {
> -ret = av_buffer_create(buf->data, pool->size, pool_release_buffer,
> -   buf, 0);
> +memset(&buf->buffer, 0, sizeof(buf->buffer));
> +ret = buffer_create(&buf->buffer, buf->data, pool->size,
> +pool_release_buffer, buf, 0);
>  if (ret) {
>  pool->pool = buf->next;
>  buf->next = NULL;
> +buf->buffer.flags_internal |= BUFFER_FLAG_NO_FREE;
>  }
>  } else {
>  ret = pool_alloc_buffer(pool);
> diff --git a/libavutil/buffer_internal.h b/libavutil/buffer_internal.h
> index 839dc05f8f..bdff1b5b32 100644
> --- a/libavutil/buffer_internal.h
> +++ b/libavutil/buffer_internal.h
> @@ -30,6 +30,11 @@
>   * The buffer was av_realloc()ed, so it is reallocatable.
>   */
>  #define BUFFER_FLAG_REALLOCATABLE (1 << 0)
> +/**
> + * The AVBuffer structure is part of a larger structure
> + * and should not be freed.
> + */
> +#define BUFFER_FLAG_NO_FREE   (1 << 1)
>  
>  struct AVBuffer {
>  uint8_t *data; /**< data described by this buffer */
> @@ -73,6 +78,12 @@ typedef struct BufferPoolEntry {
>  
>  AVBufferPool *pool;
>  struct BufferPoolEntry *next;
> +
> +/*
> + * An AVBuffer structure to (re)use as AVBuffer for subsequent uses
> + * of this BufferPoolEntry.
> + */
> +AVBuffer buffer;
>  } BufferPoolEntry;
>  
>  struct AVBufferPool {
> 

Will apply this tomorrow unless there are objections.

- Andreas
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH v5 00/12] Subtitle Filtering

2021-09-16 Thread Soft Works



> -Original Message-
> From: ffmpeg-devel  On Behalf Of
> Soft Works
> Sent: Thursday, 16 September 2021 19:46
> To: FFmpeg development discussions and patches  de...@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v5 00/12] Subtitle Filtering
> 

[..]

> > - Sparseness. Subtitles streams have gaps, and synchronization with
> >   other streams requires a next frame, that can be minutes away or
> > never
> >   come. This needs to be solved in a way compatible with
> processing.
> 
> I have kept the heartbeat logic from your sub2video implementation.
> It makes sense, is required and I can't think of any better way to
> handle this. It's just renamed to subtitle_heartbeat and used for
> all subtitle formats.

I'd like to add a few more details about that subject.

The known and retained sub2video mechanism is used for feeding
subtitle frames into the graph: it sends an initial empty frame
(unless there's any actual one) and it repeats the most recent
subtitle frame (or an empty subtitle frame) at a certain minimum
interval into the graph.

What's happening inside the graph is a different story and depends
on each individual filter's implementation. For an example, let's
look at the new textsub2video filter which renders ass subtitles
(input) on transparent video frames (output).

On the input side, the filter receives subtitle frames containing 
ass lines ("ass events"), but with libass, there's no relation 
between the time when its getting fed event lines and the output.
These are totally independent: at the input side, you could
feed the whole set of lines at once, while at the output side, 
you are specifying the ecact point in time for which you would
like the subtitle images to get rendered.

As such, the textsub2video filter cannot work in a way like 
producing one video frame at the output side for each subtitle 
text frame at the input side.
That could be either too many or also too few frames, depending
on the available compute resources, the source material and 
the desired results:

Normally, subtitle text is changing at a rather low frequency,
e.g. no more often than once or twice per second. That means
that a filter like textsub2video would need to generate a new 
overlay image no more than once or twice per second.

But wait - ass subtitles can also have animations; in that case,
every single frame rendered by libass will be different.
That will And animations
require a much higher frame-rate for smooth appearance.
 


Essentially, this leads to a similar problem like the heartbeat
topic. 





> 
> 
> > > - Part3, avfilter support for subtitles in AVFrames. At this
> point
> > we
> > > have a defined structure to store subtitles in AVFrames, and
> actual
> > > code that can generate or consume them. When approaching this,
> the
> > > same rules apply as before, existing subtitle functionality, as
> > crude
> > > as it may be, has to remain functional as exposed to the user.
> 
> Check.
> 
> > We need to decide which aspects of the subtitles formats are
> > negotiated.
> >
> > At least, obviously, the text or bitmap aspect will be, with a
> > conversion filter inserted automatically where needed.
> 
> I'm inserting the graphicsub2video filter for keeping compatibility
> with sub2video command lines, but I'm not a fan of any other
> automatic filter insertion.
> Let's talk about this in a separate conversation.
> 
> 
> > But depending on the answers to the questions in part 1, we may
> need
> > to
> > negotiate the pixel format and colorspace too.
> 
> At current, bitmap subtitles are always PAL8 and that should be
> (remain to be) the meaning if SUBTITLE_BITMAP.
> 
> It will be easy to add additional subtitle formats which could
> use a different kind of bitmap format.
> 
> 
> > Unfortunately, the current negotiation code is messy and fragile.
> We
> > cannot afford to pile new code on top of it. However good the new
> > code
> > may be, adding it on top of messy code would only make it harder to
> > clean up and maintain later. I absolutely oppose that.
> 
> The situation may be messy for audio and video, but for subtitles
> format negotiation is really simple (SUBTITLE_BITMAP, SUBTITLE_ASS
> or SUBTITLE_TEXT).
> 
> The improvements you are planning can easily be done afterwards
> as the subtitle format negotiation really doesn't add any significant
> technical debt to the situation.
> 
> 
> Kind regards,
> softworkz
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH v5 00/12] Subtitle Filtering

2021-09-16 Thread Soft Works



> -Original Message-
> From: ffmpeg-devel  On Behalf Of Soft Works
> Sent: Friday, 17 September 2021 06:35
> To: FFmpeg development discussions and patches 
> Subject: Re: [FFmpeg-devel] [PATCH v5 00/12] Subtitle Filtering
> 
> 
> 
> > -Original Message-
> > From: ffmpeg-devel  On Behalf Of
> > Soft Works
> > Sent: Thursday, 16 September 2021 19:46
> > To: FFmpeg development discussions and patches  > de...@ffmpeg.org>
> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/12] Subtitle Filtering
> >
> 
> [..]
> 
> > > - Sparseness. Subtitles streams have gaps, and synchronization with
> > >   other streams requires a next frame, that can be minutes away or
> > > never
> > >   come. This needs to be solved in a way compatible with
> > processing.
> >
> > I have kept the heartbeat logic from your sub2video implementation.
> > It makes sense, is required and I can't think of any better way to
> > handle this. It's just renamed to subtitle_heartbeat and used for
> > all subtitle formats.
> 
> I'd like to add a few more details about that subject.
> 
> The known and retained sub2video mechanism is used for feeding
> subtitle frames into the graph: it sends an initial empty frame
> (unless there's any actual one) and it repeats the most recent
> subtitle frame (or an empty subtitle frame) at a certain minimum
> interval into the graph.
> 
> What's happening inside the graph is a different story and depends
> on each individual filter's implementation. For an example, let's
> look at the new textsub2video filter which renders ass subtitles
> (input) on transparent video frames (output).
> 
> On the input side, the filter receives subtitle frames containing
> ass lines ("ass events"), but with libass, there's no relation
> between the time when its getting fed event lines and the output.
> These are totally independent: at the input side, you could
> feed the whole set of lines at once, while at the output side,
> you are specifying the ecact point in time for which you would
> like the subtitle images to get rendered.
> 
> As such, the textsub2video filter cannot work in a way like
> producing one video frame at the output side for each subtitle
> text frame at the input side.
> That could be either too many or also too few frames, depending
> on the available compute resources, the source material and
> the desired results:
> 
> Normally, subtitle text is changing at a rather low frequency,
> e.g. no more often than once or twice per second. That means
> that a filter like textsub2video would need to generate a new
> overlay image no more than once or twice per second.
> 
> But wait - ass subtitles can also have animations; in that case,
> every single frame rendered by libass will be different.
> That will And animations
> require a much higher frame-rate for smooth appearance.
> 

Sorry - accidentally sent before completion...


The new textsub2video filter has a 'framerate' parameter. 
This allows to control the frequency at which an overlay 
frame is recreated.

For example:

ffmpeg -i INPUT -filter_complex 
"[0:2]textsub2video=r=2[sub];[0:1][sub]overlay=repeatlast=0" output.ts

will provide better performance due to overlay frames getting 
recreated only twice per second while

ffmpeg -i INPUT -filter_complex 
"[0:2]textsub2video=r=25[sub];[0:1][sub]overlay=repeatlast=0" output.ts

will provide smooth animation due to frequent recreation of
overlay frames.


There's no epic conclusion - just another example for the 
wide range of new possibilities.

Regards,
softworkz




___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH] avfilter/vf_zscale: fix output color_range discrepancy

2021-09-16 Thread Paul B Mahol
I doubt this is correct, something is fishy here.

On Fri, Sep 17, 2021 at 4:17 AM Pavel Koshevoy  wrote:

> This filter chain was supposed to convert from narrow range
> to full range yuv444p, but didn't:
>
> buffer=width=1280:height=720:pix_fmt=yuv444p:frame_rate=25/1:\
> time_base=1/25:sar=1/1,zscale=min=709:rin=limited:pin=709:\
> tin=709:t=linear,format=gbrpf32le,zscale=tin=linear:p=709:m=709:\
> r=full:t=709,format=pix_fmts=yuv444p,buffersink
> ---
>  libavfilter/vf_zscale.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/libavfilter/vf_zscale.c b/libavfilter/vf_zscale.c
> index dfea00f9eb..06a025e6e6 100644
> --- a/libavfilter/vf_zscale.c
> +++ b/libavfilter/vf_zscale.c
> @@ -676,7 +676,7 @@ static int filter_frame(AVFilterLink *link, AVFrame
> *in)
>  out->color_primaries = (int)s->dst_format.color_primaries;
>
>  if (s->range != -1)
> -out->color_range = (int)s->dst_format.pixel_range;
> +out->color_range = (int)s->dst_format.pixel_range + 1;
>
>  if (s->trc != -1)
>  out->color_trc = (int)s->dst_format.transfer_characteristics;
> --
> 2.26.2
>
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
>
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 2/2] avcodec/wmaprodec: Check that the EOF frame was allocated before decoding into it

2021-09-16 Thread Paul B Mahol
lgtm
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 11/13] avcodec/msvideo1enc: Check all calls to avpriv_elbg_do()

2021-09-16 Thread Paul B Mahol
lgtm
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 01/13] avcodec/elbg: Remove avoidable buffer

2021-09-16 Thread Paul B Mahol
lgtm
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH v1 1/1] avformat/amr: Return PATCHWELCOME on stereo files

2021-09-16 Thread Paul B Mahol
Are stereo files available somewhere? Known way to encode?

On Fri, Sep 17, 2021 at 4:14 AM Sun Zhenliang 
wrote:

> Ping.
>
> Anyone could review this patch?
>
> Thx.
> 在 2021年9月16日 +0800 11:24,sunzhenliang ,写道:
> > Signed-off-by: sunzhenliang 
> > ---
> > libavformat/amr.c | 22 ++
> > 1 file changed, 18 insertions(+), 4 deletions(-)
> >
> > diff --git a/libavformat/amr.c b/libavformat/amr.c
> > index 836b276fd5..2762010ebe 100644
> > --- a/libavformat/amr.c
> > +++ b/libavformat/amr.c
> > @@ -36,8 +36,10 @@ typedef struct {
> > uint64_t block_count;
> > } AMRContext;
> >
> > -static const char AMR_header[] = "#!AMR\n";
> > -static const char AMRWB_header[] = "#!AMR-WB\n";
> > +static const char AMR_header[] = "#!AMR\n";
> > +static const char AMR_MC_header[] = "#!AMR_MC1.0\n";
> > +static const char AMRWB_header[] = "#!AMR-WB\n";
> > +static const char AMRWB_MC_header[] = "#!AMR-WB_MC1.0\n";
> >
> > static const uint8_t amrnb_packed_size[16] = {
> > 13, 14, 16, 18, 20, 21, 27, 32, 6, 1, 1, 1, 1, 1, 1, 1
> > @@ -82,7 +84,7 @@ static int amr_read_header(AVFormatContext *s)
> > {
> > AVIOContext *pb = s->pb;
> > AVStream *st;
> > - uint8_t header[9];
> > + uint8_t header[15];
> >
> > if (avio_read(pb, header, 6) != 6)
> > return AVERROR_INVALIDDATA;
> > @@ -94,7 +96,19 @@ static int amr_read_header(AVFormatContext *s)
> > if (avio_read(pb, header + 6, 3) != 3)
> > return AVERROR_INVALIDDATA;
> > if (memcmp(header, AMRWB_header, 9)) {
> > - return -1;
> > + if (avio_read(pb, header + 6 + 3, 3) != 3)
> > + return AVERROR_INVALIDDATA;
> > + if (memcmp(header, AMR_MC_header, 12)) {
> > + if (avio_read(pb, header + 6 + 3 + 3, 3) != 3)
> > + return AVERROR_INVALIDDATA;
> > + if (memcmp(header, AMRWB_MC_header, 15)) {
> > + return -1;
> > + }
> > + avpriv_report_missing_feature(s, "multi-channel AMRWB");
> > + return AVERROR_PATCHWELCOME;
> > + }
> > + avpriv_report_missing_feature(s, "multi-channel AMR");
> > + return AVERROR_PATCHWELCOME;
> > }
> >
> > st->codecpar->codec_tag = MKTAG('s', 'a', 'w', 'b');
> > --
> > 2.25.1
> >
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
>
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 06/13] avcodec/elbg: Add persistent ELBGContext

2021-09-16 Thread Paul B Mahol
lgtm
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 04/13] avfilter/vf_elbg: Rename ELBGContext->ELBGFilterContext

2021-09-16 Thread Paul B Mahol
lgtm
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 02/13] avcodec/elbg: Move avpriv_init_elbg() down

2021-09-16 Thread Paul B Mahol
lgtm
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 10/13] avcodec/cinepakenc: Check all calls to avpriv_elbg_do()

2021-09-16 Thread Paul B Mahol
lgtm
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 05/13] avcodec/elbg: Rename elbg_data to ELBGContext

2021-09-16 Thread Paul B Mahol
lgtm
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 12/13] avfilter/vf_elbg: Check call to avpriv_elbg_do()

2021-09-16 Thread Paul B Mahol
lgtm
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 09/13] avcodec/elbg: Also allocate buffers for recursion only once

2021-09-16 Thread Paul B Mahol
probably fine
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 01/13] avcodec/elbg: Remove avoidable buffer

2021-09-16 Thread Tomas Härdin
fre 2021-09-17 klockan 04:02 +0200 skrev Andreas Rheinhardt:
> Signed-off-by: Andreas Rheinhardt 
> ---
>  libavcodec/elbg.c | 9 +++--
>  1 file changed, 3 insertions(+), 6 deletions(-)
> 
> diff --git a/libavcodec/elbg.c b/libavcodec/elbg.c
> index d012d9a384..795fc83f16 100644
> --- a/libavcodec/elbg.c
> +++ b/libavcodec/elbg.c
> @@ -376,7 +376,6 @@ int avpriv_do_elbg(int *points, int dim, int
> numpoints, int *codebook,
>  elbg_data elbg_d;
>  elbg_data *elbg = &elbg_d;
>  int i, j, k, steps = 0, ret = 0;
> -    int *dist_cb = av_malloc_array(numpoints, sizeof(int));
>  int *size_part = av_malloc_array(numCB, sizeof(int));
>  cell *list_buffer = av_malloc_array(numpoints, sizeof(cell));
>  cell *free_cells;
> @@ -394,7 +393,7 @@ int avpriv_do_elbg(int *points, int dim, int
> numpoints, int *codebook,
>  elbg->utility_inc = av_malloc_array(numCB, sizeof(*elbg-
> >utility_inc));
>  elbg->scratchbuf = av_malloc_array(5*dim, sizeof(int));
>  
> -    if (!dist_cb || !size_part || !list_buffer || !elbg->cells ||
> +    if (!size_part || !list_buffer || !elbg->cells ||
>  !elbg->utility || !elbg->utility_inc || !elbg->scratchbuf) {
>  ret = AVERROR(ENOMEM);
>  goto out;
> @@ -423,9 +422,8 @@ int avpriv_do_elbg(int *points, int dim, int
> numpoints, int *codebook,
>  }
>  }
>  elbg->nearest_cb[i] = best_idx;
> -    dist_cb[i] = best_dist;
> -    elbg->error += dist_cb[i];
> -    elbg->utility[elbg->nearest_cb[i]] += dist_cb[i];
> +    elbg->error += best_dist;
> +    elbg->utility[elbg->nearest_cb[i]] += best_dist;
>  free_cells->index = i;
>  free_cells->next = elbg->cells[elbg->nearest_cb[i]];
>  elbg->cells[elbg->nearest_cb[i]] = free_cells;
> @@ -453,7 +451,6 @@ int avpriv_do_elbg(int *points, int dim, int
> numpoints, int *codebook,
>  (steps < max_steps));
>  
>  out:
> -    av_free(dist_cb);
>  av_free(size_part);
>  av_free(elbg->utility);
>  av_free(list_buffer);

Nice catch. Looks good to me

/Tomas

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 13/13] avcodec/elbg: Add flags to avpriv_elbg_do()

2021-09-16 Thread Paul B Mahol
lgtm
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 08/13] avcodec/elbg: Keep buffers to avoid allocations and frees

2021-09-16 Thread Paul B Mahol
lgtm
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 07/13] avcodec/elbg: Move arguments to the context early if possible

2021-09-16 Thread Paul B Mahol
probably fine
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".