Re: [FFmpeg-devel] [PATCH 3/5] avformat/mvdec: Do not set invalid sample rate
On Wed, Sep 15, 2021 at 10:00:46PM +0200, Michael Niedermayer wrote: > Fixes: signed integer overflow: -682581959642593728 * 16 cannot be > represented in type 'long' > Fixes: > 37883/clusterfuzz-testcase-minimized-ffmpeg_dem_MV_fuzzer-5311691517198336 > > Found-by: continuous fuzzing process > https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg > Signed-off-by: Michael Niedermayer > --- > libavformat/mvdec.c | 5 +++-- > 1 file changed, 3 insertions(+), 2 deletions(-) > > diff --git a/libavformat/mvdec.c b/libavformat/mvdec.c > index b1450e08da9..7573087c7cc 100644 > --- a/libavformat/mvdec.c > +++ b/libavformat/mvdec.c > @@ -156,9 +156,10 @@ static int parse_audio_var(AVFormatContext *avctx, > AVStream *st, > } else if (!strcmp(name, "NUM_CHANNELS")) { > return set_channels(avctx, st, var_read_int(pb, size)); > } else if (!strcmp(name, "SAMPLE_RATE")) { > -st->codecpar->sample_rate = var_read_int(pb, size); > -if (st->codecpar->sample_rate <= 0) > +int sample_rate = var_read_int(pb, size); > +if (sample_rate <= 0) > return AVERROR_INVALIDDATA; > +st->codecpar->sample_rate = sample_rate; > avpriv_set_pts_info(st, 33, 1, st->codecpar->sample_rate); > } else if (!strcmp(name, "SAMPLE_WIDTH")) { > uint64_t bpc = var_read_int(pb, size) * (uint64_t)8; please apply -- Peter (A907 E02F A6E5 0CD2 34CD 20D2 6760 79C5 AC40 DD6B) signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 1/2] libavfilter/x86/vf_gblur: fixed the fate-test failed on MacOS
Signed-off-by: Wu Jianhua --- libavfilter/x86/vf_gblur.asm | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/libavfilter/x86/vf_gblur.asm b/libavfilter/x86/vf_gblur.asm index c0d57cc82b..64c067538a 100644 --- a/libavfilter/x86/vf_gblur.asm +++ b/libavfilter/x86/vf_gblur.asm @@ -455,12 +455,13 @@ cglobal horiz_slice, 4, 9, 9, ptr, width, height, steps, nu, bscale, x, y, step, mov nuq, localbufm DEFINE_ARGS buffer, width, height, steps, \ localbuf, x, y, step, stride, remain, ptr, mask -MOVSXDIFNIDN width, height, steps %else VBROADCASTSSm0, xmm0 ; nu VBROADCASTSSm1, xmm1 ; bscale %endif +MOVSXDIFNIDN width, height, steps + %if cpuflag(avx512) vpbroadcastdm2, widthd INIT_WORD_MASK k6, k5, k4, k3, k2, k1 @@ -861,11 +862,12 @@ cglobal verti_slice, 6, 12, 9, 0-mmsize*2, buffer, width, height, cbegin, cend, VBROADCASTSS m1, bscalem DEFINE_ARGS buffer, width, height, cbegin, cend, \ steps, x, y, cwidth, step, ptr, stride -MOVSXDIFNIDN width, height, cbegin, cend, steps %else VBROADCASTSS m0, xmm0 ; nu VBROADCASTSS m1, xmm1 ; bscale %endif +MOVSXDIFNIDN width, height, cbegin, cend, steps + mov cwidthq, cendq sub cwidthq, cbeginq lea strideq, [widthq * 4] -- 2.17.1 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 2/2] libavfilter/x86/vf_gblur: correct the order of loop step
The problem was caused by if the width of the processed block minus 1 is a multiple of the aligned number the instruction jle .bscale_scalar would skip the Optimized Loop Step, which will lead to an incorrect sampling when specifying steps more than 1. Move the Optimized Loop Step after .bscale_scalar to ensure the loop step is enabled. Signed-off-by: Wu Jianhua --- libavfilter/x86/vf_gblur.asm | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/libavfilter/x86/vf_gblur.asm b/libavfilter/x86/vf_gblur.asm index 64c067538a..16e802e002 100644 --- a/libavfilter/x86/vf_gblur.asm +++ b/libavfilter/x86/vf_gblur.asm @@ -524,9 +524,8 @@ cglobal horiz_slice, 4, 9, 9, ptr, width, height, steps, nu, bscale, x, y, step, cmp xq,0 jg .loop_x_scalar -OPTIMIZED_LOOP_STEP - .bscale_scalar: +OPTIMIZED_LOOP_STEP sub ptrq, 4 sub localbufq, mmsize mulps m3, m1 -- 2.17.1 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 0/3] Detect field_order when probing MP4/H264
>Nicolas Gaullier (3): > avcodec/h264_parser: Set AVCodecContext.framerate > avformat/utils: Use r_frame_rate in compute_frame_duration if > codec_framerate is unknown > avformat/mov: Set AVSTREAM_PARSE_HEADERS flag for H264 Hello, I have received no feedback on this serie I posted last week. Could someone find some time for a review ? Thanks to you Nicolas (ngaullier@irc) ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/2] libavfilter/x86/vf_gblur: fixed the fate-test failed on MacOS
Jianhua wrote: > From: Wu, Jianhua > Sent: Thursday, September 16, 2021 3:34 PM > To: ffmpeg-devel@ffmpeg.org > Cc: Wu, Jianhua > Subject: [PATCH 1/2] libavfilter/x86/vf_gblur: fixed the fate-test failed on > MacOS > > Signed-off-by: Wu Jianhua > --- Hi Zhili, I have submitted the patches fixing the fate-test that failed on macOS. Could you help try to apply the patches to check if they work? Here is the link: http://ffmpeg.org/pipermail/ffmpeg-devel/2021-September/285376.html And the illegal instructions:4 occurred might be the version of your GCC installed less than 4.7, which could be fixed by specific a compiler option, -mmacosx-version-min=10.x, by the way, the exact I used is 10.10. To enable it on FFmpeg, you might have to specify in the --extra-cflags option when configuring. But I recommend updating the GCC to the newest version, which can fix this as well. If there is another problem, feel free to let me know. Best regards, Jianhua ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 01/12] lavf/concat: refactor parsing
Nicolas George (12021-09-12): > Will push this series soon. Series pushed. ffmpeg + lsdvd + dvd2concat can now encode DVDs reasonably well. Regards, -- Nicolas George signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/4] libavcodec/adts_header: add frame_length field and avpriv function to parse AAC ADTS header
Nachiket Tarate 于2021年9月1日周三 下午10:39写道: > > These will be used by HLS demuxer in case of sample decryption. > > Signed-off-by: Nachiket Tarate > --- > libavcodec/adts_header.c | 1 + > libavcodec/adts_header.h | 15 +++ > libavcodec/adts_parser.c | 31 +++ > 3 files changed, 47 insertions(+) > > diff --git a/libavcodec/adts_header.c b/libavcodec/adts_header.c > index 0889820f8a..e4454529c4 100644 > --- a/libavcodec/adts_header.c > +++ b/libavcodec/adts_header.c > @@ -66,6 +66,7 @@ int ff_adts_header_parse(GetBitContext *gbc, > AACADTSHeaderInfo *hdr) > hdr->sample_rate= avpriv_mpeg4audio_sample_rates[sr]; > hdr->samples= (rdb + 1) * 1024; > hdr->bit_rate = size * 8 * hdr->sample_rate / hdr->samples; > +hdr->frame_length = size; > > return size; > } > diff --git a/libavcodec/adts_header.h b/libavcodec/adts_header.h > index f615f6a9f9..166a28ffc9 100644 > --- a/libavcodec/adts_header.h > +++ b/libavcodec/adts_header.h > @@ -34,6 +34,7 @@ typedef struct AACADTSHeaderInfo { > uint8_t sampling_index; > uint8_t chan_config; > uint8_t num_aac_frames; > +uint32_t frame_length; > } AACADTSHeaderInfo; > > /** > @@ -47,4 +48,18 @@ typedef struct AACADTSHeaderInfo { > */ > int ff_adts_header_parse(GetBitContext *gbc, AACADTSHeaderInfo *hdr); > > +/** > + * Parse the ADTS frame header contained in the buffer, which is > + * the first 54 bits. > + * @param[in] buf Pointer to buffer containing the first 54 bits of the > frame. > + * @param[in] size Size of buffer containing the first 54 bits of the frame. > + * @param[out] phdr Pointer to pointer to struct AACADTSHeaderInfo for which > + * memory is allocated and header info is written into it. After using the > header > + * information, the allocated memory must be freed by using av_free(). > + * @return Returns 0 on success, -1 if there is a sync word mismatch, > + * -2 if the version element is invalid, -3 if the sample rate > + * element is invalid, or -4 if the bit rate element is invalid. > + */ > +int avpriv_adts_header_parse(AACADTSHeaderInfo **phdr, const uint8_t *buf, > size_t size); > + > #endif /* AVCODEC_ADTS_HEADER_H */ > diff --git a/libavcodec/adts_parser.c b/libavcodec/adts_parser.c > index 5c9f8ff6f2..4a1a8fd5f4 100644 > --- a/libavcodec/adts_parser.c > +++ b/libavcodec/adts_parser.c > @@ -42,3 +42,34 @@ int av_adts_header_parse(const uint8_t *buf, uint32_t > *samples, uint8_t *frames) > return AVERROR(ENOSYS); > #endif > } > + > +int avpriv_adts_header_parse(AACADTSHeaderInfo **phdr, const uint8_t *buf, > size_t size) > +{ > +#if CONFIG_ADTS_HEADER > +int ret = 0; > +GetBitContext gb; > + > +if (!phdr || !buf || size < AV_AAC_ADTS_HEADER_SIZE) > +return AVERROR_INVALIDDATA; > + > +*phdr = av_mallocz(sizeof(AACADTSHeaderInfo)); > +if (!*phdr) > +return AVERROR(ENOMEM); > + > +ret = init_get_bits8(&gb, buf, AV_AAC_ADTS_HEADER_SIZE); > +if (ret < 0) { > +av_freep(phdr); > +return ret; > +} > + > +ret = ff_adts_header_parse(&gb, *phdr); > +if (ret < 0) { > +av_freep(phdr); > +return ret; > +} > + > +return 0; > +#else > +return AVERROR(ENOSYS); > +#endif > +} > -- > 2.17.1 > > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". looks have been fixed the suggestion from zhili. Will apply patchset if no more comments ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 2/2] avformat/mxf: support MCA audio information
ons 2021-09-15 klockan 12:14 +0200 skrev Marc-Antoine Arnaud: +static int mxf_read_mca_sub_descriptor(void *arg, AVIOContext *pb, int tag, int size, UID uid, int64_t klv_offset) +{ + MXFMCASubDescriptor *mca_sub_descriptor = arg; + + if (IS_KLV_KEY(uid, mxf_mca_prefix)) { + if (IS_KLV_KEY(uid, mxf_mca_label_dictionnary_id)) { + avio_read(pb, mca_sub_descriptor- >mca_label_dictionnary_id, 16); + } + if (IS_KLV_KEY(uid, mxf_mca_link_id)) { + avio_read(pb, mca_sub_descriptor->mca_link_id, 16); + } + if (IS_KLV_KEY(uid, mxf_soundfield_group_link_id)) { + avio_read(pb, mca_sub_descriptor->mca_group_link_id, 16); + } + } This nesting of ifs is unnecessary Rest of the patch looks fine. Passes FATE. Do you have a sample or two? /Tomas ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/12] Subtitle Filtering
Hendrik Leppkes (12021-09-15): > Or perhaps some people have a day job, a life and other obligations > that prevent them from spending time on FFmpeg every day, especially > outside the weekend. > But no, that can't be it, surely we are all just evil. /s > Nothing here is being invented. You are trying to push a major API > change to the core functionality of our libraries, these rules on how > to do API changes on that level (or any level, really) have been in > place and followed by everyone for years and years. > If you don't believe that, feel free to check how any other major API > change was handled in the past. Because the overall pattern is always > the same. > > Do you think I'm the only one thinking that way, only because I spoke > up? The set certainly wouldn't have been applied if I hadn't said > anything, it would've just sat there. Maybe someone else might've come > forward eventually. Thank you for having the patience and taking the time to explain this. > As for the actual subject: > - Part1, add subtitles to AVFrame in avutil. You can move > enums/structs to avutil as needed (eg AVSubtitleRect or whatever), as > long as they are still available the same way (eg. avcodec.h should > include any new avutil header with them, so user code works without > changes), but moving functions is a more tricky business, so we > usually make new functions and cleanup their naming convention in the > process, but I don't think any functions are involved here right now. > > To dive a bit deeper into this, redundant fields should be removed, > actual subtitle data should be refcounted (with AVBuffers in > AVFrame->buf), all frame functionality need to properly account for > the new data type. There is another point to consider when designing subtitles in AVFrame: since we intend it not only as a general API cleanup but as a prelude to extending the API with filtering and such, we must not only think about what is needed now, we must think about what may be needed later. The few examples I have, not excluding questions I have not thought of: - Right now, we have text subtitles and bitmap subtitles. But do we want to be able to handle mixed text and bitmap subtitles? To this, I would say: probably no. But we should give it a thought. - Number of rectangles. Currently, the decoders usually output a very limited number of rectangles. OTOH, libass may output three alpha maps per glyph, that makes potentially hundreds of rectangles. Will the data structure be able to handle this efficiently? Consider also the issue of rectangles overlapping. - Speaking of overlapping, we need a system to signal whether a new subtitle frame should replace the current one (like dvdsub, srt, etc.) or overlap with it (like ASS). - Colorspace. All current bitmap subtitles formats are paletted. But palette pixel formats are bad for many treatments. Inside a filter chain, it would probably make sense to have them in a true color format. - Global styles. ASS subtitles, in particular, contain reference to globally-defined styles. How do we handle that in libavfilter? I have no idea. - Sparseness. Subtitles streams have gaps, and synchronization with other streams requires a next frame, that can be minutes away or never come. This needs to be solved in a way compatible with processing. > - Part3, avfilter support for subtitles in AVFrames. At this point we > have a defined structure to store subtitles in AVFrames, and actual > code that can generate or consume them. When approaching this, the > same rules apply as before, existing subtitle functionality, as crude > as it may be, has to remain functional as exposed to the user. > Internal under-the-hood changes are fine, as long as the user does not > notice. > > I'm not involved in internal avfilter design, so at this point you'll > have to get Nicolas on board. The most salient point is negotiation. I have already said it and I say it again, because the format negotiation in libavfilter is a very important aspect of what makes it work well, and it is also a very tricky piece of code. We need to decide which aspects of the subtitles formats are negotiated. At least, obviously, the text or bitmap aspect will be, with a conversion filter inserted automatically where needed. But depending on the answers to the questions in part 1, we may need to negotiate the pixel format and colorspace too. Unfortunately, the current negotiation code is messy and fragile. We cannot afford to pile new code on top of it. However good the new code may be, adding it on top of messy code would only make it harder to clean up and maintain later. I absolutely oppose that. Therefore, if anybody wants to add something to libavfilter that involves the negotiation, they should consider helping cleaning it up. It would make your work easier afterwards, so you should do it if only for that reason. The first task, as I have mentioned, would probably be to add FATE tes
[FFmpeg-devel] [PATCH 1/5] avcodec/libsvtav1: Fix redundant setting of caps_internal
From: Limin Wang Signed-off-by: Limin Wang --- libavcodec/libsvtav1.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/libavcodec/libsvtav1.c b/libavcodec/libsvtav1.c index fabc4e6..82ae2b9 100644 --- a/libavcodec/libsvtav1.c +++ b/libavcodec/libsvtav1.c @@ -561,12 +561,11 @@ const AVCodec ff_libsvtav1_encoder = { .receive_packet = eb_receive_packet, .close = eb_enc_close, .capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_OTHER_THREADS, -.caps_internal = FF_CODEC_CAP_AUTO_THREADS, +.caps_internal = FF_CODEC_CAP_AUTO_THREADS | FF_CODEC_CAP_INIT_CLEANUP, .pix_fmts = (const enum AVPixelFormat[]){ AV_PIX_FMT_YUV420P, AV_PIX_FMT_YUV420P10, AV_PIX_FMT_NONE }, .priv_class = &class, .defaults = eb_enc_defaults, -.caps_internal = FF_CODEC_CAP_INIT_CLEANUP, .wrapper_name = "libsvtav1", }; -- 1.8.3.1 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 2/5] avcodec/libsvtav1: make intra_refresh_type configurable
From: Limin Wang Signed-off-by: Limin Wang --- libavcodec/libsvtav1.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/libavcodec/libsvtav1.c b/libavcodec/libsvtav1.c index 82ae2b9..8c2c970 100644 --- a/libavcodec/libsvtav1.c +++ b/libavcodec/libsvtav1.c @@ -210,7 +210,8 @@ static int config_enc_params(EbSvtAv1EncConfiguration *param, param->min_qp_allowed = avctx->qmin; } -param->intra_refresh_type = 2; /* Real keyframes only */ +/* 2 = IDR, closed GOP, 1 = CRA, open GOP */ +param->intra_refresh_type = avctx->flags & AV_CODEC_FLAG_CLOSED_GOP ? 2 : 1; if (svt_enc->la_depth >= 0) param->look_ahead_distance = svt_enc->la_depth; @@ -548,6 +549,7 @@ static const AVCodecDefault eb_enc_defaults[] = { { "g", "-1"}, { "qmin", "0" }, { "qmax", "63"}, +{ "flags", "+cgop" }, { NULL }, }; -- 1.8.3.1 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 3/5] doc: update for libsvtav1 encoder
From: Limin Wang Signed-off-by: Limin Wang --- doc/encoders.texi | 44 1 file changed, 44 insertions(+) diff --git a/doc/encoders.texi b/doc/encoders.texi index 8fccd73..64d604e 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -1750,12 +1750,56 @@ You need to explicitly configure the build with @code{--enable-libsvtav1}. @table @option @item profile Set the encoding profile. +@table @samp +@item main +@item high +@item professional +@end table @item level Set the operating point level. +@table @samp +@item 2.0 +@item 2.1 +@item 2.2 +@item 2.3 +@item 3.0 +@item 3.1 +@item 3.2 +@item 3.3 +@item 4.0 +@item 4.1 +@item 4.2 +@item 4.3 +@item 5.0 +@item 5.1 +@item 5.2 +@item 5.3 +@item 6.0 +@item 6.1 +@item 6.2 +@item 6.3 +@item 7.0 +@item 7.1 +@item 7.2 +@item 7.3 +@end table + +@item hielevel +Set the Hierarchical prediction levels. +@table @samp +@item 3level +@item 4level, This is the default. +@end table @item tier Set the operating point tier. +@table @samp +@item main +The main tier was designed for most applications. This is the default. +@item high +The high tier was designed for very demanding applications. +@end table @item rc Set the rate control mode to use. -- 1.8.3.1 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 4/5] avcodec/libsvtav1: Fix CQP mode doesn't work as expection
From: Limin Wang Signed-off-by: Limin Wang --- libavcodec/libsvtav1.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/libavcodec/libsvtav1.c b/libavcodec/libsvtav1.c index 8c2c970..1f1f86b 100644 --- a/libavcodec/libsvtav1.c +++ b/libavcodec/libsvtav1.c @@ -208,6 +208,8 @@ static int config_enc_params(EbSvtAv1EncConfiguration *param, if (param->rate_control_mode) { param->max_qp_allowed = avctx->qmax; param->min_qp_allowed = avctx->qmin; +} else { +param->enable_tpl_la = 0; /* CQP need turn off enable_tp_la */ } /* 2 = IDR, closed GOP, 1 = CRA, open GOP */ -- 1.8.3.1 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 5/5] avcodec/libsvtav1: support constant quality mode
From: Limin Wang Signed-off-by: Limin Wang --- doc/encoders.texi | 7 ++- libavcodec/libsvtav1.c | 10 +- libavcodec/version.h | 2 +- 3 files changed, 16 insertions(+), 3 deletions(-) diff --git a/doc/encoders.texi b/doc/encoders.texi index 64d604e..2300bb9 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -1807,9 +1807,11 @@ Set the rate control mode to use. Possible modes: @table @option @item cqp -Constant quantizer: use fixed values of qindex (dependent on the frame type) +Constant quantizer(not set crf): use fixed values of qindex (dependent on the frame type) throughout the stream. This mode is the default. +Constant quality(set crf): maintain a constant QP throughout the stream. + @item vbr Variable bitrate: use a target bitrate for the whole stream. @@ -1826,6 +1828,9 @@ Set the minimum quantizer to use when using a bitrate mode. @item qp Set the quantizer used in cqp rate control mode (0-63). +@item crf +Select the quality for constant quality mode (0-63). + @item sc_detection Enable scene change detection. diff --git a/libavcodec/libsvtav1.c b/libavcodec/libsvtav1.c index 1f1f86b..29be4bd 100644 --- a/libavcodec/libsvtav1.c +++ b/libavcodec/libsvtav1.c @@ -66,6 +66,7 @@ typedef struct SvtContext { int rc_mode; int scd; int qp; +int crf; int tier; @@ -210,6 +211,10 @@ static int config_enc_params(EbSvtAv1EncConfiguration *param, param->min_qp_allowed = avctx->qmin; } else { param->enable_tpl_la = 0; /* CQP need turn off enable_tp_la */ +if ( svt_enc->crf > 0) { +param->qp = svt_enc->crf; +param->enable_tpl_la = 1; +} } /* 2 = IDR, closed GOP, 1 = CRA, open GOP */ @@ -523,13 +528,16 @@ static const AVOption options[] = { { "rc", "Bit rate control mode", OFFSET(rc_mode), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 3, VE , "rc"}, -{ "cqp", "Constant quantizer", 0, AV_OPT_TYPE_CONST, { .i64 = 0 }, INT_MIN, INT_MAX, VE, "rc" }, +{ "cqp", "Constant quantizer(not set crf), Constant quality(set crf)", 0, AV_OPT_TYPE_CONST, { .i64 = 0 }, INT_MIN, INT_MAX, VE, "rc" }, { "vbr", "Variable Bit Rate, use a target bitrate for the entire stream", 0, AV_OPT_TYPE_CONST, { .i64 = 1 }, INT_MIN, INT_MAX, VE, "rc" }, { "cvbr", "Constrained Variable Bit Rate, use a target bitrate for each GOP", 0, AV_OPT_TYPE_CONST,{ .i64 = 2 }, INT_MIN, INT_MAX, VE, "rc" }, { "qp", "Quantizer to use with cqp rate control mode", OFFSET(qp), AV_OPT_TYPE_INT, { .i64 = 50 }, 0, 63, VE }, +{ "crf", "Select the quality for constant quality mode", OFFSET(crf), + AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 63, VE }, + { "sc_detection", "Scene change detection", OFFSET(scd), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, VE }, diff --git a/libavcodec/version.h b/libavcodec/version.h index 4b4fe54..b0a741b 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -29,7 +29,7 @@ #define LIBAVCODEC_VERSION_MAJOR 59 #define LIBAVCODEC_VERSION_MINOR 7 -#define LIBAVCODEC_VERSION_MICRO 103 +#define LIBAVCODEC_VERSION_MICRO 104 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ LIBAVCODEC_VERSION_MINOR, \ -- 1.8.3.1 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/5] avcodec/libsvtav1: Fix redundant setting of caps_internal
lance.lmw...@gmail.com: > From: Limin Wang > > Signed-off-by: Limin Wang > --- > libavcodec/libsvtav1.c | 3 +-- > 1 file changed, 1 insertion(+), 2 deletions(-) > > diff --git a/libavcodec/libsvtav1.c b/libavcodec/libsvtav1.c > index fabc4e6..82ae2b9 100644 > --- a/libavcodec/libsvtav1.c > +++ b/libavcodec/libsvtav1.c > @@ -561,12 +561,11 @@ const AVCodec ff_libsvtav1_encoder = { > .receive_packet = eb_receive_packet, > .close = eb_enc_close, > .capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_OTHER_THREADS, > -.caps_internal = FF_CODEC_CAP_AUTO_THREADS, > +.caps_internal = FF_CODEC_CAP_AUTO_THREADS | FF_CODEC_CAP_INIT_CLEANUP, > .pix_fmts = (const enum AVPixelFormat[]){ AV_PIX_FMT_YUV420P, > AV_PIX_FMT_YUV420P10, > AV_PIX_FMT_NONE }, > .priv_class = &class, > .defaults = eb_enc_defaults, > -.caps_internal = FF_CODEC_CAP_INIT_CLEANUP, > .wrapper_name = "libsvtav1", > }; > This is not redundant; the second initialization overrides the first. If I read validate_thread_parameters in pthread.c correctly, this means that 7d09579190d broke the ability to forward the threading parameters to the actual encoder. But it never did this anyway, so it doesn't matter. Should be fixed nevertheless. - Andreas ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] Plans for libavfilter
Since it is not healthy to keep everything for myself, here is a summary of the projects I have in mind for the enhancement of libavfilter. The thing is, there are a lot of dependencies between these projects. Working on them in a proper order would make achieving the goals easier. On the other hand, if task B depends on task A, then working B without addressing A involves a little more work for B, but also a lot more work for A, and a lot more work for everything else that depends on A. This is one reason I am writing this mail: to make clear what needs to be done, and how tasks depend on each-other and fit together. The earliest tasks are not very sexy. They are about making the code clearer or more robust and generic rather than adding features. It is always boring. But they are necessary, because parts of the process is quite shaky, and we cannot afford to build on shaky code. If you want to help, please consider this and you will be welcome. Now that is established the specifics: - Multi-stage negotiation. That means negotiating properties that depend on other properties. I think the colorspace stuff is in that case, it cannot be negotiated until the pixel format is known. - Media type negotiation. That means that filters that do not care about the contents of what they are filtering, filters that only act on the frame information fields, can also not care if they are filtering audio or video or other. There are a lot of these filters: pts manipulation, concat, split, selection, etc. We cannot afford to have to update every one of them if we want to add a new media type, so this is a prerequisite. And it is an aspect of multi-stage negotiation, since pixel format or sample format can only be negotiated once we know the media type. - Partial graph configuration. Right now, the negotiation happens in steps, first query_formats then merge, then pick. But pick can be done on some links as soon as the formats are merged to a singleton while other filters are still delaying their query_formats, and once pick is done, the graph can start running. That would allow filters with constraints more complex than what AVFilterFormats can express. (We have a mechanism for that, used by amerge and one or two other filters; I introduced it early and it was a mistake, it is much too fragile and only converges in the simplest cases.) - Partial graph re-configuration. If we can run a graph when not all filters are configured, then we can de-configure and re-configure part of a graph. That would bring us the support for changes in pixel format. - Global running API. Right now, we make libavfilter activate filters by pumping request_frame() on one of the buffersinks, more or less randomly, and we hope it will result in something useful. We need a better API, one where we say libavfilter "start running these graphs", we push frames on input as we have them, we are notified when frames arrive on output. - Inter-filter threading. This is a big one, and quite self-explanatory. It goes hand to hand with a global running API, because "start running these graphs" also means "start as many threads as you need and keep them ready". Note that the network protocols in libavformat requires the same thing. And I really do not want yet two other frigging threading subsystems. This is why I have started slowly working on an unique system suited for all needs, starting with both libavformat and libavfilter but keeping also libavcodec in mind. - Out-of-band side-data. This is a mechanism to notify filters that the work they are currently doing may become irrelevant, or other kinds of urgent information. Filters can ignore it, and just do the work for nothing. But smarter filters can take it into account and just skip until it becomes relevant again. - Seeking. This is when applications notify the outputs they want to jump at a specified time, and all the filters in the graph make sure the instruction reach the inputs. This is probably rather easy and does not depend on many other things. Out-of-band side-data is meant for that, among other things: notify a filter "you can skip processing the frames you have queued, because we are seeking anyway"; and if seeking fails, notify the user as fast as possible "we tried seeking, it will not happen". A few things need ironing out, though. Do we want to handle it in activate() or do we want a separate callback? Do we want a specific mechanism for seeking or something more generic for all the messages that go backwards in the graph? Do we want a queue for messages that go backwards or would a single message be enough? - Subtitles. This one has been under the spotlight recently. What it means is rather obvious. But it if far from trivial. For starters, it requires the negotiation of media type, because we cannot afford to update all the utility filters for
Re: [FFmpeg-devel] Plans for libavfilter
On Thu, Sep 16, 2021 at 1:55 PM Nicolas George wrote: > Since it is not healthy to keep everything for myself, here is a summary > of the projects I have in mind for the enhancement of libavfilter. > > The thing is, there are a lot of dependencies between these projects. > Working on them in a proper order would make achieving the goals easier. > On the other hand, if task B depends on task A, then working B without > addressing A involves a little more work for B, but also a lot more work > for A, and a lot more work for everything else that depends on A. > > This is one reason I am writing this mail: to make clear what needs to > be done, and how tasks depend on each-other and fit together. > > The earliest tasks are not very sexy. They are about making the code > clearer or more robust and generic rather than adding features. It is > always boring. But they are necessary, because parts of the process is > quite shaky, and we cannot afford to build on shaky code. > > If you want to help, please consider this and you will be welcome. > > Now that is established the specifics: > > - Multi-stage negotiation. > > That means negotiating properties that depend on other properties. I > think the colorspace stuff is in that case, it cannot be negotiated > until the pixel format is known. > > - Media type negotiation. > > That means that filters that do not care about the contents of what > they are filtering, filters that only act on the frame information > fields, can also not care if they are filtering audio or video or > other. There are a lot of these filters: pts manipulation, concat, > split, selection, etc. > > We cannot afford to have to update every one of them if we want to add > a new media type, so this is a prerequisite. And it is an aspect of > multi-stage negotiation, since pixel format or sample format can only > be negotiated once we know the media type. > This is short sighted and irrelevant and also source of confusion for users. I really see no way to tell you to stop insisting on this nonsense. So I kindly ask you yet again to stop further work you planned on libavfilter for everyone else good outcome. > - Partial graph configuration. > > Right now, the negotiation happens in steps, first query_formats then > merge, then pick. But pick can be done on some links as soon as the > formats are merged to a singleton while other filters are still > delaying their query_formats, and once pick is done, the graph can > start running. That would allow filters with constraints more complex > than what AVFilterFormats can express. (We have a mechanism for that, > used by amerge and one or two other filters; I introduced it early and > it was a mistake, it is much too fragile and only converges in the > simplest cases.) > > - Partial graph re-configuration. > > If we can run a graph when not all filters are configured, then we can > de-configure and re-configure part of a graph. That would bring us the > support for changes in pixel format. > > - Global running API. > > Right now, we make libavfilter activate filters by pumping > request_frame() on one of the buffersinks, more or less randomly, and > we hope it will result in something useful. We need a better API, one > where we say libavfilter "start running these graphs", we push frames > on input as we have them, we are notified when frames arrive on > output. > > - Inter-filter threading. > > This is a big one, and quite self-explanatory. It goes hand to hand > with a global running API, because "start running these graphs" also > means "start as many threads as you need and keep them ready". > > Note that the network protocols in libavformat requires the same > thing. And I really do not want yet two other frigging threading > subsystems. This is why I have started slowly working on an unique > system suited for all needs, starting with both libavformat and > libavfilter but keeping also libavcodec in mind. > > This is the only useful entry so far. > - Out-of-band side-data. > > This is a mechanism to notify filters that the work they are currently > doing may become irrelevant, or other kinds of urgent information. > Filters can ignore it, and just do the work for nothing. But smarter > filters can take it into account and just skip until it becomes > relevant again. > > - Seeking. > > This is when applications notify the outputs they want to jump at a > specified time, and all the filters in the graph make sure the > instruction reach the inputs. > > This is probably rather easy and does not depend on many other things. > Out-of-band side-data is meant for that, among other things: notify a > filter "you can skip processing the frames you have queued, because we > are seeking anyway"; and if seeking fails, notify the user as fast as > possible "we tried seeking, it will not happen". > > A few things need ironing out, though. Do we wan
Re: [FFmpeg-devel] [PATCH v2 05/14] avfilter/af_aresample: Use preinit instead of init_dict
Andreas Rheinhardt (12021-09-14): > By using preinit, the SwrContext already exists directly after > allocating the filter, so that the filter's AVClass's child_next > becomes usable for setting options with the AV_OPT_SEARCH_CHILDREN > search flag. This means that it is no longer necessary to use > the init_dict callback for this filter. > > Furthermore, the earlier code did not abide by the documentation > of the init_dict callback at all: Instead of only returning the > options that have not been recognized it always returned all options > on any av_opt_set() error and errored out in this case; yet if > the error was just caused by an unrecognized option, it should not > error out at all and instead return said option. > > This behaviour has been inherited by avfilter_init_dict(), > contradicting its documentation. This is also fixed by this commit. > > Signed-off-by: Andreas Rheinhardt > --- > libavfilter/af_aresample.c | 27 +++ > 1 file changed, 7 insertions(+), 20 deletions(-) This looks like a good change. Thanks. Regards, -- Nicolas George signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 14/14] avfilter/avfilter: Report all unrecognized options in avfilter_init_str
Andreas Rheinhardt (12021-09-14): > Signed-off-by: Andreas Rheinhardt > --- > libavfilter/avfilter.c | 5 ++--- > 1 file changed, 2 insertions(+), 3 deletions(-) Looks ok. Regards, -- Nicolas George signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] Plans for libavfilter
Paul B Mahol (12021-09-16): > This is short sighted and irrelevant and also source of confusion for > users. > > I really see no way to tell you to stop insisting on this nonsense. So I > kindly ask you > yet again to stop further work you planned on libavfilter for everyone else > good outcome. If you want to discuss politely why you think this is a bad idea, I am ready to listen to your arguments and present mine. Otherwise, I do not care. -- Nicolas George signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 11/14] avfilter/avfilter: Use AV_DICT_DONT_STRDUP_(KEY|VAL) when possible
Andreas Rheinhardt (12021-09-14): > Signed-off-by: Andreas Rheinhardt > --- > libavfilter/avfilter.c | 7 +++ > 1 file changed, 3 insertions(+), 4 deletions(-) LGTM. Regards, -- Nicolas George signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/5] avformat/sbgdec: Check for t0 overflow in expand_tseq()
Michael Niedermayer (12021-09-15): > Fixes: signed integer overflow: 4611686025627387904 + 4611686025627387904 > cannot be represented in type 'long' > Fixes: > 35489/clusterfuzz-testcase-minimized-ffmpeg_dem_SBG_fuzzer-4862678601433088 > > Found-by: continuous fuzzing process > https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg > Signed-off-by: Michael Niedermayer > --- > libavformat/sbgdec.c | 3 +++ > 1 file changed, 3 insertions(+) No objection. Regards, -- Nicolas George signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 12/14] avfilter/avfilter: Honour the short options documentation
Andreas Rheinhardt (12021-09-14): > The documentation for filter arguments states that short options must > precede long options (i.e. those of the form key=value). Yet if > process_options() encounters arguments not abiding by this, it simply > treats short options after a long option as if it were parsing short > options for the first time. In particular, it overwrites options already > set earlier, possibly via other short options. This is not how it is > intended (as a comment in the code indicates). > > This commit modifies the code to reject further shorthand options > after a long option has been encountered. After all, avfilter_init_str() > errors out upon unrecognized options, so it is intended to be picky. > > Signed-off-by: Andreas Rheinhardt > --- > libavfilter/avfilter.c | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) Since you have a good reason to use a pointer instead of a integer boolean, then ok. Regards, -- Nicolas George signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 13/14] avfilter/avfilter: Don't fail upon options for filter without AVClass
Andreas Rheinhardt (12021-09-14): > Commit 62549f9655c48f0ec061087fa33a96040ce01145 added a check to > (the predecessor of) avfilter_init_str() to error out if options > were provided to a filter without options (or rather, without private > class). This was fine at the time, yet soon afterwards commit > fdd93eabfb2644f541f7aac9943abce26776ea73 added a generic option > for all AVFilterContexts and since then it is wrong to error out > in case options have been provided to a filter without AVClass. > > To workaround this issue, several filters with timeline support > added AVClasses and empty options; these will be removed in subsequent > commits. Furthermore, the super2xsai filter supports slice threading, > but no options and so has no AVClass, making it impossible to set > the number of threads when using avfilter_init_str() (and therefore > from the ffmpeg-tool). This is fixed by this commit, too. > > Signed-off-by: Andreas Rheinhardt > --- > Now with a comment explaining the two functions of checking for priv. > > libavfilter/avfilter.c | 9 +++-- > 1 file changed, 3 insertions(+), 6 deletions(-) Ok, I suppose. Regards, -- Nicolas George signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 09/14] avfilter/vf_scale: Honour the AVFilter.init_dict documentation
Andreas Rheinhardt (12021-09-14): > The documentation states that unrecognized options need to be returned > (just as av_opt_set_dict() would do). Yet the scale and scale2ref > filters didn't abide by this: They simply copied all options > and in case it contained an unrecognized option was among them, > they would error out later in config_props. This violates the > documentation of av_filter_init_dict(). > > Fix this by only keeping the recognized options and returning > the unrecognized ones. > > Signed-off-by: Andreas Rheinhardt > --- > libavfilter/vf_scale.c | 16 +--- > 1 file changed, 13 insertions(+), 3 deletions(-) > > diff --git a/libavfilter/vf_scale.c b/libavfilter/vf_scale.c > index a1902a13cf..c31b92b847 100644 > --- a/libavfilter/vf_scale.c > +++ b/libavfilter/vf_scale.c > @@ -269,6 +269,8 @@ revert: > > static av_cold int init_dict(AVFilterContext *ctx, AVDictionary **opts) > { > +const AVClass *class = sws_get_class(); > +const AVDictionaryEntry *entry = NULL; > ScaleContext *scale = ctx->priv; > int ret; > > @@ -312,15 +314,23 @@ static av_cold int init_dict(AVFilterContext *ctx, > AVDictionary **opts) > scale->flags = 0; > > if (scale->flags_str && *scale->flags_str) { > -const AVClass *class = sws_get_class(); > const AVOption*o = av_opt_find(&class, "sws_flags", NULL, 0, > AV_OPT_SEARCH_FAKE_OBJ); > int ret = av_opt_eval_flags(&class, o, scale->flags_str, > &scale->flags); > if (ret < 0) > return ret; > } > -scale->opts = *opts; > -*opts = NULL; > +FFSWAP(AVDictionary *, *opts, scale->opts); > +/* Now move all the unrecognized options back to opts. */ > +while (entry = av_dict_get(scale->opts, "", entry, > AV_DICT_IGNORE_SUFFIX)) { > +if (!av_opt_find(&class, entry->key, NULL, 0, > AV_OPT_SEARCH_FAKE_OBJ)) { > +if ((ret = av_dict_set(opts, entry->key, entry->value, 0)) < 0 || > +(ret = av_dict_set(&scale->opts, entry->key, NULL, 0)) < 0) > +return ret; > +/* Removing the entry from scale->opts invalidated entry. */ > +entry = NULL; > +} > +} > > return 0; > } I managed to understand what this code does, but I find it quite confusing. First, correct me if I am wrong, but I think the entry parameter to av_dict_get() is always NULL. Second, I think if you swap removing the option from scale->opts and adding it to opts you can avoid strduping the key. And is entry->value not leaking? But more importantly, is it not a convoluted implementation that would be simplified by using preinit like you did for sws? Regards, -- Nicolas George signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 01/14] avfilter/vsrc_testsrc: Deduplicate AVClasses
Andreas Rheinhardt (12021-09-14): > Signed-off-by: Andreas Rheinhardt > --- > I am not resending all the deduplication patches; > they can be e.g. found here: > https://github.com/mkver/FFmpeg/commits/avfilter_deduplication > > libavfilter/vsrc_testsrc.c | 47 +- > 1 file changed, 16 insertions(+), 31 deletions(-) The deduplication patches are ok IMHO. I do not maintain zscale nor spp but see nothing obviously wrong with the patch. Regards, -- Nicolas George signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] regarding hiding AVStream.first_dts member into AVStreamInternal
Chromium could instead keep track of that timestamp in their own code. It's what i did to remove the internal fields usage from our CLI tools. See https://github.com/FFmpeg/FFmpeg/commit/ab4f299e23 On 9/16/2021 8:24 AM, Marek Behún wrote: Hello James, liberato, James, your commit 591b88e6787c [1] in FFmpeg hid the first_dts and cur_dts members of AVStream structure into the internal member, making them inaccessible from third party code. The reasoning in the commit message says: They are private fields, no reason to have them exposed in a public header. But the first_dts member is used in Chromium, in media/filters/ffmpeg_demuxer.cc [2] in method FFmpegDemuxer::InitializeTask. Chromium solved this by adding av_stream_get_first_dts() method into their bundled ffmpeg copy [3]. This makes it now impossible to link Chromium with system-ffmpeg, which was till now a possibility in Gentoo, and which we would like to have also in the future. James, what do you think the best approach is here? Should the FFmpeg commit be reverted to unhide this member? Or should Chromium be fixed not to use this member? (I confess I did not study the code much, so I don't know why it is needed, besides that Chromium tries to compute start_time_ variable with it.) Thank you. Marek [1] https://github.com/FFmpeg/FFmpeg/commit/591b88e6787c [2] https://chromium.googlesource.com/chromium/+/refs/heads/trunk/media/filters/ffmpeg_demuxer.cc [3] https://chromium.googlesource.com/chromium/third_party/ffmpeg/+/95aab0fd83619408995720ce53d7a74790580220%5E%21/ ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH] avcodec/mmaldec: fix decoder freeze when flushing
When mmal_flush is called before any packets are sent, enabling/disabling the MMAL ports seems to cause the decoder to not output any packets after flushing. Bug is triggered when using 'mpv --hwdec=mmal --start=1 test.mp4'. Proposed workaround in MPV: https://github.com/mpv-player/mpv/pull/9189 Signed-off-by: Ho Ming Shun --- libavcodec/mmaldec.c | 38 +++--- 1 file changed, 23 insertions(+), 15 deletions(-) diff --git a/libavcodec/mmaldec.c b/libavcodec/mmaldec.c index 8c7d749742..6c3e5d99b6 100644 --- a/libavcodec/mmaldec.c +++ b/libavcodec/mmaldec.c @@ -145,19 +145,21 @@ static int ffmmal_set_ref(AVFrame *frame, FFPoolRef *pool, return 0; } -static void ffmmal_stop_decoder(AVCodecContext *avctx) +static void ffmmal_stop_decoder(AVCodecContext *avctx, int flush_ports) { MMALDecodeContext *ctx = avctx->priv_data; MMAL_COMPONENT_T *decoder = ctx->decoder; MMAL_BUFFER_HEADER_T *buffer; -mmal_port_disable(decoder->input[0]); -mmal_port_disable(decoder->output[0]); -mmal_port_disable(decoder->control); +if(flush_ports) { +mmal_port_disable(decoder->input[0]); +mmal_port_disable(decoder->output[0]); +mmal_port_disable(decoder->control); -mmal_port_flush(decoder->input[0]); -mmal_port_flush(decoder->output[0]); -mmal_port_flush(decoder->control); +mmal_port_flush(decoder->input[0]); +mmal_port_flush(decoder->output[0]); +mmal_port_flush(decoder->control); +} while ((buffer = mmal_queue_get(ctx->queue_decoded_frames))) mmal_buffer_header_release(buffer); @@ -185,7 +187,7 @@ static av_cold int ffmmal_close_decoder(AVCodecContext *avctx) MMALDecodeContext *ctx = avctx->priv_data; if (ctx->decoder) -ffmmal_stop_decoder(avctx); +ffmmal_stop_decoder(avctx, 1); mmal_component_destroy(ctx->decoder); ctx->decoder = NULL; @@ -456,14 +458,20 @@ static void ffmmal_flush(AVCodecContext *avctx) MMAL_COMPONENT_T *decoder = ctx->decoder; MMAL_STATUS_T status; -ffmmal_stop_decoder(avctx); +// MMAL will freeze if ports are enabled/disabled/flushed before +// buffers are sent +int flush_ports = ctx->packets_sent || ctx->extradata_sent; -if ((status = mmal_port_enable(decoder->control, control_port_cb))) -goto fail; -if ((status = mmal_port_enable(decoder->input[0], input_callback))) -goto fail; -if ((status = mmal_port_enable(decoder->output[0], output_callback))) -goto fail; +ffmmal_stop_decoder(avctx, flush_ports); + +if(flush_ports) { +if ((status = mmal_port_enable(decoder->control, control_port_cb))) +goto fail; +if ((status = mmal_port_enable(decoder->input[0], input_callback))) +goto fail; +if ((status = mmal_port_enable(decoder->output[0], output_callback))) +goto fail; +} return; -- 2.33.0 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/5] avcodec/libsvtav1: Fix redundant setting of caps_internal
On Thu, Sep 16, 2021 at 01:32:00PM +0200, Andreas Rheinhardt wrote: > lance.lmw...@gmail.com: > > From: Limin Wang > > > > Signed-off-by: Limin Wang > > --- > > libavcodec/libsvtav1.c | 3 +-- > > 1 file changed, 1 insertion(+), 2 deletions(-) > > > > diff --git a/libavcodec/libsvtav1.c b/libavcodec/libsvtav1.c > > index fabc4e6..82ae2b9 100644 > > --- a/libavcodec/libsvtav1.c > > +++ b/libavcodec/libsvtav1.c > > @@ -561,12 +561,11 @@ const AVCodec ff_libsvtav1_encoder = { > > .receive_packet = eb_receive_packet, > > .close = eb_enc_close, > > .capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_OTHER_THREADS, > > -.caps_internal = FF_CODEC_CAP_AUTO_THREADS, > > +.caps_internal = FF_CODEC_CAP_AUTO_THREADS | > > FF_CODEC_CAP_INIT_CLEANUP, > > .pix_fmts = (const enum AVPixelFormat[]){ AV_PIX_FMT_YUV420P, > > AV_PIX_FMT_YUV420P10, > > AV_PIX_FMT_NONE }, > > .priv_class = &class, > > .defaults = eb_enc_defaults, > > -.caps_internal = FF_CODEC_CAP_INIT_CLEANUP, > > .wrapper_name = "libsvtav1", > > }; > > > This is not redundant; the second initialization overrides the first. If > I read validate_thread_parameters in pthread.c correctly, this means > that 7d09579190d broke the ability to forward the threading parameters > to the actual encoder. But it never did this anyway, so it doesn't > matter. Should be fixed nevertheless. Yes, It seems that the commit message is misleading, it's the second overrides the first. I'll change it to " Fix override setting of caps_internal". > > - Andreas > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". -- Thanks, Limin Wang ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/2] libavfilter/x86/vf_gblur: fixed the fate-test failed on MacOS
> On Sep 16, 2021, at 3:48 PM, Wu, Jianhua wrote: > > Jianhua wrote: >> From: Wu, Jianhua >> Sent: Thursday, September 16, 2021 3:34 PM >> To: ffmpeg-devel@ffmpeg.org >> Cc: Wu, Jianhua >> Subject: [PATCH 1/2] libavfilter/x86/vf_gblur: fixed the fate-test failed on >> MacOS >> >> Signed-off-by: Wu Jianhua >> --- > > Hi Zhili, > > I have submitted the patches fixing the fate-test that failed on macOS. Could > you help try to apply the patches to check if they work? Here is the link: > http://ffmpeg.org/pipermail/ffmpeg-devel/2021-September/285376.html > > And the illegal instructions:4 occurred might be the version of your GCC > installed less than 4.7, which could be fixed by specific a compiler option, > -mmacosx-version-min=10.x, by the way, the exact I used is 10.10. To enable > it on FFmpeg, you might have to specify in the --extra-cflags option when > configuring. But I recommend updating the GCC to the newest version, which > can fix this as well. Thanks, the patch fixed both issues. By the way, the default toolchain on macOS is clang, gcc has been dropped by Apple long time ago. > > If there is another problem, feel free to let me know. > > Best regards, > Jianhua > > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 4/5] avcodec/apedec: Fix integer overflow in intermediate
On Wed, Sep 15, 2021 at 10:09:49PM +0200, Andreas Rheinhardt wrote: > Michael Niedermayer: > > Fixes: signed integer overflow: 559334865 * 4 cannot be represented in type > > 'int' > > Fixes: > > 37929/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-6751932295806976 > > > > Found-by: continuous fuzzing process > > https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg > > Signed-off-by: Michael Niedermayer > > --- > > libavcodec/apedec.c | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/libavcodec/apedec.c b/libavcodec/apedec.c > > index bf481ba3549..23318be0613 100644 > > --- a/libavcodec/apedec.c > > +++ b/libavcodec/apedec.c > > @@ -1337,7 +1337,7 @@ static void do_apply_filter(APEContext *ctx, int > > version, APEFilter *f, > > absres = FFABSU(res); > > if (absres) > > *f->adaptcoeffs = APESIGN(res) * > > - (8 << ((absres > f->avg * 3) + (absres > > > f->avg * 4 / 3))); > > + (8 << ((absres > f->avg * 3) + (absres > > > f->avg * 4LL / 3))); > > /* equivalent to the following code > > if (absres <= f->avg * 4 / 3) > > *f->adaptcoeffs = APESIGN(res) * 8; > > > How about using f->avg + f->avg / 3? Given that you haven't modified the > f->avg * 3 before, this should be save. will apply this variant thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB The day soldiers stop bringing you their problems is the day you have stopped leading them. They have either lost confidence that you can help or concluded you do not care. Either case is a failure of leadership. - Colin Powell signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 3/5] avformat/mvdec: Do not set invalid sample rate
On Thu, Sep 16, 2021 at 05:02:53PM +1000, Peter Ross wrote: > On Wed, Sep 15, 2021 at 10:00:46PM +0200, Michael Niedermayer wrote: > > Fixes: signed integer overflow: -682581959642593728 * 16 cannot be > > represented in type 'long' > > Fixes: > > 37883/clusterfuzz-testcase-minimized-ffmpeg_dem_MV_fuzzer-5311691517198336 > > > > Found-by: continuous fuzzing process > > https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg > > Signed-off-by: Michael Niedermayer > > --- > > libavformat/mvdec.c | 5 +++-- > > 1 file changed, 3 insertions(+), 2 deletions(-) > > > > diff --git a/libavformat/mvdec.c b/libavformat/mvdec.c > > index b1450e08da9..7573087c7cc 100644 > > --- a/libavformat/mvdec.c > > +++ b/libavformat/mvdec.c > > @@ -156,9 +156,10 @@ static int parse_audio_var(AVFormatContext *avctx, > > AVStream *st, > > } else if (!strcmp(name, "NUM_CHANNELS")) { > > return set_channels(avctx, st, var_read_int(pb, size)); > > } else if (!strcmp(name, "SAMPLE_RATE")) { > > -st->codecpar->sample_rate = var_read_int(pb, size); > > -if (st->codecpar->sample_rate <= 0) > > +int sample_rate = var_read_int(pb, size); > > +if (sample_rate <= 0) > > return AVERROR_INVALIDDATA; > > +st->codecpar->sample_rate = sample_rate; > > avpriv_set_pts_info(st, 33, 1, st->codecpar->sample_rate); > > } else if (!strcmp(name, "SAMPLE_WIDTH")) { > > uint64_t bpc = var_read_int(pb, size) * (uint64_t)8; > > please apply will do thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Concerning the gods, I have no means of knowing whether they exist or not or of what sort they may be, because of the obscurity of the subject, and the brevity of human life -- Protagoras signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/5] avformat/sbgdec: Check for t0 overflow in expand_tseq()
On Thu, Sep 16, 2021 at 02:52:10PM +0200, Nicolas George wrote: > Michael Niedermayer (12021-09-15): > > Fixes: signed integer overflow: 4611686025627387904 + 4611686025627387904 > > cannot be represented in type 'long' > > Fixes: > > 35489/clusterfuzz-testcase-minimized-ffmpeg_dem_SBG_fuzzer-4862678601433088 > > > > Found-by: continuous fuzzing process > > https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg > > Signed-off-by: Michael Niedermayer > > --- > > libavformat/sbgdec.c | 3 +++ > > 1 file changed, 3 insertions(+) > > No objection. will apply thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Dictatorship naturally arises out of democracy, and the most aggravated form of tyranny and slavery out of the most extreme liberty. -- Plato signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/12] Subtitle Filtering
On Thu, Sep 16, 2021 at 12:20:17PM +0200, Nicolas George wrote: > Hendrik Leppkes (12021-09-15): > > Or perhaps some people have a day job, a life and other obligations > > that prevent them from spending time on FFmpeg every day, especially > > outside the weekend. > > But no, that can't be it, surely we are all just evil. /s > > > Nothing here is being invented. You are trying to push a major API > > change to the core functionality of our libraries, these rules on how > > to do API changes on that level (or any level, really) have been in > > place and followed by everyone for years and years. > > If you don't believe that, feel free to check how any other major API > > change was handled in the past. Because the overall pattern is always > > the same. > > > > Do you think I'm the only one thinking that way, only because I spoke > > up? The set certainly wouldn't have been applied if I hadn't said > > anything, it would've just sat there. Maybe someone else might've come > > forward eventually. > > Thank you for having the patience and taking the time to explain this. > > > As for the actual subject: > > - Part1, add subtitles to AVFrame in avutil. You can move > > enums/structs to avutil as needed (eg AVSubtitleRect or whatever), as > > long as they are still available the same way (eg. avcodec.h should > > include any new avutil header with them, so user code works without > > changes), but moving functions is a more tricky business, so we > > usually make new functions and cleanup their naming convention in the > > process, but I don't think any functions are involved here right now. > > > > To dive a bit deeper into this, redundant fields should be removed, > > actual subtitle data should be refcounted (with AVBuffers in > > AVFrame->buf), all frame functionality need to properly account for > > the new data type. > > There is another point to consider when designing subtitles in AVFrame: > since we intend it not only as a general API cleanup but as a prelude to > extending the API with filtering and such, we must not only think about > what is needed now, we must think about what may be needed later. > > The few examples I have, not excluding questions I have not thought of: > > - Right now, we have text subtitles and bitmap subtitles. But do we want > to be able to handle mixed text and bitmap subtitles? > > To this, I would say: probably no. But we should give it a thought. This question can also be asked seperatly for API and implementation, for example we may support mixed bitmap/text subtitles in the API but not the implementation. The advantage would be that the API does not need to be changed if we want to add support on the implementation side [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Let us carefully observe those good qualities wherein our enemies excel us and endeavor to excel them, by avoiding what is faulty, and imitating what is excellent in them. -- Plutarch signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 09/14] avfilter/vf_scale: Honour the AVFilter.init_dict documentation
Nicolas George: > Andreas Rheinhardt (12021-09-14): >> The documentation states that unrecognized options need to be returned >> (just as av_opt_set_dict() would do). Yet the scale and scale2ref >> filters didn't abide by this: They simply copied all options >> and in case it contained an unrecognized option was among them, >> they would error out later in config_props. This violates the >> documentation of av_filter_init_dict(). >> >> Fix this by only keeping the recognized options and returning >> the unrecognized ones. >> >> Signed-off-by: Andreas Rheinhardt >> --- >> libavfilter/vf_scale.c | 16 +--- >> 1 file changed, 13 insertions(+), 3 deletions(-) >> >> diff --git a/libavfilter/vf_scale.c b/libavfilter/vf_scale.c >> index a1902a13cf..c31b92b847 100644 >> --- a/libavfilter/vf_scale.c >> +++ b/libavfilter/vf_scale.c >> @@ -269,6 +269,8 @@ revert: >> >> static av_cold int init_dict(AVFilterContext *ctx, AVDictionary **opts) >> { >> +const AVClass *class = sws_get_class(); >> +const AVDictionaryEntry *entry = NULL; >> ScaleContext *scale = ctx->priv; >> int ret; >> >> @@ -312,15 +314,23 @@ static av_cold int init_dict(AVFilterContext *ctx, >> AVDictionary **opts) >> scale->flags = 0; >> >> if (scale->flags_str && *scale->flags_str) { >> -const AVClass *class = sws_get_class(); >> const AVOption*o = av_opt_find(&class, "sws_flags", NULL, 0, >> AV_OPT_SEARCH_FAKE_OBJ); >> int ret = av_opt_eval_flags(&class, o, scale->flags_str, >> &scale->flags); >> if (ret < 0) >> return ret; >> } >> -scale->opts = *opts; >> -*opts = NULL; >> +FFSWAP(AVDictionary *, *opts, scale->opts); >> +/* Now move all the unrecognized options back to opts. */ >> +while (entry = av_dict_get(scale->opts, "", entry, >> AV_DICT_IGNORE_SUFFIX)) { >> +if (!av_opt_find(&class, entry->key, NULL, 0, >> AV_OPT_SEARCH_FAKE_OBJ)) { >> +if ((ret = av_dict_set(opts, entry->key, entry->value, 0)) < 0 >> || >> +(ret = av_dict_set(&scale->opts, entry->key, NULL, 0)) < 0) >> +return ret; >> +/* Removing the entry from scale->opts invalidated entry. */ >> +entry = NULL; >> +} >> +} >> >> return 0; >> } > > I managed to understand what this code does, but I find it quite > confusing. > > First, correct me if I am wrong, but I think the entry parameter to > av_dict_get() is always NULL. It is not. It is only reset when the last entry key was not recognized as option; if it was recognized, entry is not NULL. In particular, in the (hopefully) common case where all options are recognized, the dictionary is just traversed linearly. > > Second, I think if you swap removing the option from scale->opts and > adding it to opts you can avoid strduping the key. > You seem to think about "stealing" the key from the returned entry. This does not work at all here, because for av_dict_set(&scale->opts, entry->key, NULL, 0) to actually find the entry to remove, the entry still needs to have the key. But if it has the key, it will free the key upon deleting the entry. More generally, stealing a key/value is forbidden: "The returned entry key or value must not be changed, or it will cause undefined behavior." Several parts of the AVDict API itself as well as user code requires this to be so. (I know you do this in tee.c where it happens to work; btw: I have a few open patches for it that you may look at.) > And is entry->value not leaking? No, default behaviour for av_dict_set() (the flags AV_DICT_MULTIKEY, AV_DICT_DONT_OVERWRITE, AV_DICT_APPEND modify this) is to replace the first current entry whose key matches with the new entry (or just delete said entry when the new value is NULL). When an entry is freed, both of its key and value are freed, too. > > But more importantly, is it not a convoluted implementation that would > be simplified by using preinit like you did for sws? > Unfortunately, the situation here is more complicated: We have not one object on which we can simply apply options, but several (there are separate scalers for interlaced pictures); and they are freed and recreated on parameter changes. So one would at least need a persistent master scale context which is used to apply the initial user options and also used to copy options from it to the other scalers. At that point I looked at the place where the options from the dictionary get applied and saw that it is in the middle of applying other options, so I believed that I could not maintain current behaviour by using a master context and instead opted for this. Now I look at this again and see that it is possible to override certain parameters which should not be overrideable at all: E.g. overriding dstw/dsth/srcw/srch easily leads to crashes. So this should be reordered and in this case the master scaler approach mig
Re: [FFmpeg-devel] [PATCH v5 00/12] Subtitle Filtering
> -Original Message- > From: ffmpeg-devel On Behalf Of > Nicolas George > Sent: Thursday, 16 September 2021 12:20 > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/12] Subtitle Filtering > > There is another point to consider when designing subtitles in > AVFrame: > since we intend it not only as a general API cleanup but as a prelude > to > extending the API with filtering and such, we must not only think > about > what is needed now, we must think about what may be needed later. > > The few examples I have, not excluding questions I have not thought > of: > > - Right now, we have text subtitles and bitmap subtitles. But do we > want > to be able to handle mixed text and bitmap subtitles? > > To this, I would say: probably no. But we should give it a thought. Each rect has a field of type enum AVSubtitleType, so this remains to be possible, but I agree that it's unlikely to be required. > - Number of rectangles. Currently, the decoders usually output a very > limited number of rectangles. OTOH, libass may output three alpha > maps > per glyph, that makes potentially hundreds of rectangles. > > Will the data structure be able to handle this efficiently? You are right about the amount of temporary images that libass is creating (can go up to 100k for complex subs with animation). But I don't think that there's any case where the data would leave a filter that way. I have already: - overlay_graphicsubs: The bitmaps are blended on a video (2 inputs: video, subs[format=bitmap], 1 output: video) - overlay_graphicsubs: The bitmaps are blended on a new empty transparent frame (1 input, subs[format=bitmap], 1 output: video[with alpha]) What might make sense as well, would be a conversion from text subs to graphic subs, but this can't be done by having a separate bitmap for each glyph which is most likely off-spec for any graphic subtitle format. So, even in that case, the individual libass rects won't leave the filter as subtitle rects. > Consider also the issue of rectangles overlapping. > > - Speaking of overlapping, we need a system to signal whether a new > subtitle frame should replace the current one (like dvdsub, srt, > etc.) > or overlap with it (like ASS). I don't think anything new is required as the existing logic is preserved, no matter whether the subtitle frames drive through a filter chain or are forwarded to encoding directly. > - Colorspace. All current bitmap subtitles formats are paletted. But > palette pixel formats are bad for many treatments. Inside a filter > chain, it would probably make sense to have them in a true color > format. There are cases where this makes sense, for example to feed them for overlay into a hardware filtering chain (via hwupload). For these cases, I have graphicsub2video and textsub2video which output transparent RGBA images. For local (non-hardware) overlay it doesn't make sense to convert the palette bitmaps. For example, in overlay_graphicsubs, when overlaying over a yuv format, I'm just converting the palette to yuv for blending. > - Global styles. ASS subtitles, in particular, contain reference to > globally-defined styles. How do we handle that in libavfilter? I > have > no idea. There's a shared reference pointing at the global ass header (string) which each subtitle AVFrame is carrying. If that is not available, ass_get_default_header (iirc) can be called. > - Sparseness. Subtitles streams have gaps, and synchronization with > other streams requires a next frame, that can be minutes away or > never > come. This needs to be solved in a way compatible with processing. I have kept the heartbeat logic from your sub2video implementation. It makes sense, is required and I can't think of any better way to handle this. It's just renamed to subtitle_heartbeat and used for all subtitle formats. > > - Part3, avfilter support for subtitles in AVFrames. At this point > we > > have a defined structure to store subtitles in AVFrames, and actual > > code that can generate or consume them. When approaching this, the > > same rules apply as before, existing subtitle functionality, as > crude > > as it may be, has to remain functional as exposed to the user. Check. > We need to decide which aspects of the subtitles formats are > negotiated. > > At least, obviously, the text or bitmap aspect will be, with a > conversion filter inserted automatically where needed. I'm inserting the graphicsub2video filter for keeping compatibility with sub2video command lines, but I'm not a fan of any other automatic filter insertion. Let's talk about this in a separate conversation. > But depending on the answers to the questions in part 1, we may need > to > negotiate the pixel format and colorspace too. At current, bitmap subtitles are always PAL8 and that should be (remain to be) the meaning if SUBTITLE_BITMAP. It will be easy to add
Re: [FFmpeg-devel] Plans for libavfilter
> -Original Message- > From: ffmpeg-devel On Behalf Of > Nicolas George > Sent: Thursday, 16 September 2021 13:55 > To: ffmpeg-devel@ffmpeg.org > Subject: [FFmpeg-devel] Plans for libavfilter > > > > - Media type negotiation. > > That means that filters that do not care about the contents of what > they are filtering, filters that only act on the frame information > fields, can also not care if they are filtering audio or video or > other. There are a lot of these filters: pts manipulation, concat, > split, selection, etc. > > We cannot afford to have to update every one of them if we want to > add > a new media type, so this is a prerequisite. And it is an aspect of > multi-stage negotiation, since pixel format or sample format can > only > be negotiated once we know the media type. I don't see any benefit in having filters that are handling variable media types. Even when the filter names are the same, the semantic of what these are doing often varies significantly. They would often need to have options where some only apply to a certain media type only. Further, this increases error and confusion for command line use, as this would lead to filters connecting in ways that aren't intended at all, while making it hard for users to understand what's happening and why it's failing. It's already hard or impossible to see which formats are actually negotiated between filters. Having filters that can do multiple media types on a single pad would cause ultimate confusion for users and make things happen that are undesired, unexpected and intransparent. Kind regards, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] Plans for libavfilter
We are facing similar issues after working with libavfilter for more than one year. We are implementing a private solution for: - Partial graph configuration - Partial graph re-configuration - Global running API It's glad to see other people have the same feeling of the shortcoming design in avfilter and improve it in a general way. On Thu, Sep 16, 2021 at 7:55 PM Nicolas George wrote: > Since it is not healthy to keep everything for myself, here is a summary > of the projects I have in mind for the enhancement of libavfilter. > > The thing is, there are a lot of dependencies between these projects. > Working on them in a proper order would make achieving the goals easier. > On the other hand, if task B depends on task A, then working B without > addressing A involves a little more work for B, but also a lot more work > for A, and a lot more work for everything else that depends on A. > > This is one reason I am writing this mail: to make clear what needs to > be done, and how tasks depend on each-other and fit together. > > The earliest tasks are not very sexy. They are about making the code > clearer or more robust and generic rather than adding features. It is > always boring. But they are necessary, because parts of the process is > quite shaky, and we cannot afford to build on shaky code. > > If you want to help, please consider this and you will be welcome. > > Now that is established the specifics: > > - Multi-stage negotiation. > > That means negotiating properties that depend on other properties. I > think the colorspace stuff is in that case, it cannot be negotiated > until the pixel format is known. > > - Media type negotiation. > > That means that filters that do not care about the contents of what > they are filtering, filters that only act on the frame information > fields, can also not care if they are filtering audio or video or > other. There are a lot of these filters: pts manipulation, concat, > split, selection, etc. > > We cannot afford to have to update every one of them if we want to add > a new media type, so this is a prerequisite. And it is an aspect of > multi-stage negotiation, since pixel format or sample format can only > be negotiated once we know the media type. > > - Partial graph configuration. > > Right now, the negotiation happens in steps, first query_formats then > merge, then pick. But pick can be done on some links as soon as the > formats are merged to a singleton while other filters are still > delaying their query_formats, and once pick is done, the graph can > start running. That would allow filters with constraints more complex > than what AVFilterFormats can express. (We have a mechanism for that, > used by amerge and one or two other filters; I introduced it early and > it was a mistake, it is much too fragile and only converges in the > simplest cases.) > > - Partial graph re-configuration. > > If we can run a graph when not all filters are configured, then we can > de-configure and re-configure part of a graph. That would bring us the > support for changes in pixel format. > > - Global running API. > > Right now, we make libavfilter activate filters by pumping > request_frame() on one of the buffersinks, more or less randomly, and > we hope it will result in something useful. We need a better API, one > where we say libavfilter "start running these graphs", we push frames > on input as we have them, we are notified when frames arrive on > output. > > - Inter-filter threading. > > This is a big one, and quite self-explanatory. It goes hand to hand > with a global running API, because "start running these graphs" also > means "start as many threads as you need and keep them ready". > > Note that the network protocols in libavformat requires the same > thing. And I really do not want yet two other frigging threading > subsystems. This is why I have started slowly working on an unique > system suited for all needs, starting with both libavformat and > libavfilter but keeping also libavcodec in mind. > > - Out-of-band side-data. > > This is a mechanism to notify filters that the work they are currently > doing may become irrelevant, or other kinds of urgent information. > Filters can ignore it, and just do the work for nothing. But smarter > filters can take it into account and just skip until it becomes > relevant again. > > - Seeking. > > This is when applications notify the outputs they want to jump at a > specified time, and all the filters in the graph make sure the > instruction reach the inputs. > > This is probably rather easy and does not depend on many other things. > Out-of-band side-data is meant for that, among other things: notify a > filter "you can skip processing the frames you have queued, because we > are seeking anyway"; and if seeking fails, notify the user as fast as > possible "we tried seeking, it will not happen". > > A fe
Re: [FFmpeg-devel] Plans for libavfilter
On Thu, Sep 16, 2021 at 01:55:17PM +0200, Nicolas George wrote: > Since it is not healthy to keep everything for myself, here is a summary > of the projects I have in mind for the enhancement of libavfilter. > > The thing is, there are a lot of dependencies between these projects. > Working on them in a proper order would make achieving the goals easier. > On the other hand, if task B depends on task A, then working B without > addressing A involves a little more work for B, but also a lot more work > for A, and a lot more work for everything else that depends on A. > > This is one reason I am writing this mail: to make clear what needs to > be done, and how tasks depend on each-other and fit together. > > The earliest tasks are not very sexy. They are about making the code > clearer or more robust and generic rather than adding features. It is > always boring. But they are necessary, because parts of the process is > quite shaky, and we cannot afford to build on shaky code. > > If you want to help, please consider this and you will be welcome. > > Now that is established the specifics: > > - Multi-stage negotiation. > > That means negotiating properties that depend on other properties. I > think the colorspace stuff is in that case, it cannot be negotiated > until the pixel format is known. > > - Media type negotiation. > > That means that filters that do not care about the contents of what > they are filtering, filters that only act on the frame information > fields, can also not care if they are filtering audio or video or > other. There are a lot of these filters: pts manipulation, concat, > split, selection, etc. +1 the current need to duplicate filters per media type is ugly. Also "negotiation" is a strong term for this. Filters in general have either one media type on their pads or pass the type from input to output and support any. In that sense this is not hard to solve, it should not be dependant on anything else [...] > > - Partial graph configuration. > > Right now, the negotiation happens in steps, first query_formats then > merge, then pick. But pick can be done on some links as soon as the > formats are merged to a singleton while other filters are still > delaying their query_formats, and once pick is done, the graph can > start running. That would allow filters with constraints more complex > than what AVFilterFormats can express. (We have a mechanism for that, > used by amerge and one or two other filters; I introduced it early and > it was a mistake, it is much too fragile and only converges in the > simplest cases.) > > - Partial graph re-configuration. > > If we can run a graph when not all filters are configured, then we can > de-configure and re-configure part of a graph. That would bring us the > support for changes in pixel format. Maybe some orthogonal direction could be considered too completely seperating graph configuration and using some more generic algorithm to solve it. What we have basically are a bunch of constraints and then we need to color the links between vertexes so as to minimize a weighted average of computational complexity and quality loss. And each link can have also different colors on both ends which would correspond to an inserted convert filter. The "color" here would be a vector of all elements that need configuration. This view if it works would also allow very easy changes to the things needing configuration. For example in this framework one could replace a colorspace enum by a AxB element matrix and nothing would change on the solver it would just need a different quality estimation function. That means all the heuristics would be isolated in the function which is to be minimized. While the solver would just deal with generic values of some kind like ints [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB The smallest minority on earth is the individual. Those who deny individual rights cannot claim to be defenders of minorities. - Ayn Rand signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH] avformat/mvdec: propagate the errors returned by parse_audio_var()
Signed-off-by: James Almer --- libavformat/mvdec.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavformat/mvdec.c b/libavformat/mvdec.c index 7573087c7c..0b4aa1f18f 100644 --- a/libavformat/mvdec.c +++ b/libavformat/mvdec.c @@ -387,7 +387,7 @@ static int mv_read_header(AVFormatContext *avctx) if (!ast) return AVERROR(ENOMEM); ast->codecpar->codec_type = AVMEDIA_TYPE_AUDIO; -if ((read_table(avctx, ast, parse_audio_var)) < 0) +if ((ret = read_table(avctx, ast, parse_audio_var)) < 0) return ret; if (mv->acompression == 100 && mv->aformat == AUDIO_FORMAT_SIGNED && -- 2.33.0 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] avformat/mvdec: propagate the errors returned by parse_audio_var()
James Almer: > Signed-off-by: James Almer > --- > libavformat/mvdec.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/libavformat/mvdec.c b/libavformat/mvdec.c > index 7573087c7c..0b4aa1f18f 100644 > --- a/libavformat/mvdec.c > +++ b/libavformat/mvdec.c > @@ -387,7 +387,7 @@ static int mv_read_header(AVFormatContext *avctx) > if (!ast) > return AVERROR(ENOMEM); > ast->codecpar->codec_type = AVMEDIA_TYPE_AUDIO; > -if ((read_table(avctx, ast, parse_audio_var)) < 0) > +if ((ret = read_table(avctx, ast, parse_audio_var)) < 0) > return ret; > if (mv->acompression == 100 && > mv->aformat == AUDIO_FORMAT_SIGNED && > The title is misleading: it should be something like "Don't signal success upon error". Just reading your title would indicate that we are just making error codes up instead of forwarding them. - Andreas ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] avformat/mvdec: propagate the errors returned by parse_audio_var()
On 9/16/2021 4:39 PM, Andreas Rheinhardt wrote: James Almer: Signed-off-by: James Almer --- libavformat/mvdec.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavformat/mvdec.c b/libavformat/mvdec.c index 7573087c7c..0b4aa1f18f 100644 --- a/libavformat/mvdec.c +++ b/libavformat/mvdec.c @@ -387,7 +387,7 @@ static int mv_read_header(AVFormatContext *avctx) if (!ast) return AVERROR(ENOMEM); ast->codecpar->codec_type = AVMEDIA_TYPE_AUDIO; -if ((read_table(avctx, ast, parse_audio_var)) < 0) +if ((ret = read_table(avctx, ast, parse_audio_var)) < 0) return ret; if (mv->acompression == 100 && mv->aformat == AUDIO_FORMAT_SIGNED && The title is misleading: it should be something like "Don't signal success upon error". Just reading your title would indicate that we are just making error codes up instead of forwarding them. I don't really think it's misleading, but sure, I'll use your suggestion. - Andreas ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v6 00/11] Subtitle Filtering
v6 Update: - Implements all given feedback from reviews (thanks!) - Add AVSubtitle fields to AVFrame, keeping AVSubtitle unchanged for compatibility and future deprecation - Use subtitle frames internally and for filtering - Retain compatibility with legacy subtitle implementation through a number of compatibility conversion methods - Reordered and reorganized commits, each one passing FATE now - Document new public API functions - Many more detail changes v5 Update: - Merge AVSubtitle into AVFrame - Move FATE test adjustments to corresponding commit - Move documentation updates to corresponding filter commits - Remove mediatype parameter from av_frame_get_buffer2 (still need some advice for splitting the commits in a way that every single one will pass FATE) v4 Update: - Re-Sending due to Patchwork having failed to parse my patchset There seems to be a bug in Patchwork when parallel processing is enabled. This time, I'll send the e-mails slowly, one after another. v3 Update: - Reworked, revised, rebased, reorganized, refactored - No more prototype/test-style code - Additional subtitle filters added - Filter documentation added - Adjusted FATE tests This patchset is about introducing filtering support for subtitles. The current sub2video "hack" implementation is slow, ineffective and limited in capabilities. => This patchset introduces true subtitle filtering These new filters are included: - overlay_graphicsubs (VS -> V) Overlay graphic subtitles onto a video stream - graphicsub2video {S -> V) Converts graphic subtitles to (transparent) video frames - overlay_textsubs {VS -> V) Overlay text subtitles onto a video stream. - textsubs2video {S -> V) Converts text subtitles to video frames - textmod {S -> S) Modify subtitle text in a number of ways - stripstyles {S -> S) Remove all inline styles from subtitle events Regards, softworkz softworkz (11): global: Prepare AVFrame for subtitle handling fftools/play,probe: Adjust for subtitle changes avfilter/subtitles: Add subtitles.c for subtitle frame allocation avfilter/avfilter: Handle subtitle frames avfilter/sbuffer: Add sbuffersrv and sbuffersink filters avfilter/overlay_graphicsubs: Add overlay_graphicsubs and graphicsub2video filters fftools/ffmpeg: Replace sub2video with subtitle frame filtering avfilter/overlay_textsubs: Add overlay_textsubs and textsubs2video filters avfilter/textmod: Add textmod filter avcodec/ass_split: Extend ass dialog parsing avfilter/stripstyles: Add stripstyles filter configure | 4 +- doc/filters.texi | 261 fftools/ffmpeg.c | 459 ++ fftools/ffmpeg.h | 14 +- fftools/ffmpeg_filter.c | 198 -- fftools/ffmpeg_hw.c | 2 +- fftools/ffmpeg_opt.c | 3 +- fftools/ffplay.c | 50 +- fftools/ffprobe.c | 49 +- libavcodec/ass_split.c| 12 +- libavcodec/ass_split.h| 2 + libavcodec/avcodec.c | 19 - libavcodec/avcodec.h | 105 ++-- libavcodec/decode.c | 24 +- libavcodec/pgssubdec.c| 1 + libavcodec/utils.c| 11 + libavfilter/Makefile | 9 + libavfilter/allfilters.c | 16 +- libavfilter/avfilter.c| 30 +- libavfilter/avfiltergraph.c | 5 + libavfilter/buffersink.c | 63 ++ libavfilter/buffersink.h | 15 + libavfilter/buffersrc.c | 72 +++ libavfilter/buffersrc.h | 1 + libavfilter/formats.c | 14 + libavfilter/formats.h | 3 + libavfilter/internal.h| 1 + libavfilter/sf_stripstyles.c | 211 +++ libavfilter/sf_textmod.c | 372 +++ libavfilter/subtitles.c | 61 ++ libavfilter/subtitles.h | 44 ++ libavfilter/version.h | 2 +- libavfilter/vf_overlay_graphicsubs.c | 727 ++ libavfilter/vf_overlay_textsubs.c | 613 ++ libavfilter/vf_subtitles.c| 28 +- libavformat/utils.c | 1 + libavutil/Makefile| 2 + libavutil/frame.c | 184 +- libavutil/frame.h | 93 ++- libavutil/subfmt.c| 221 +++ libavutil/subfmt.h| 186 ++ libavutil/version.h | 2 +- tests/ref/fate/filter-overlay-dvdsub-2397 | 181 +++--- tests/ref/fate/sub-dvb| 162 ++--- tests/ref/fate/sub2video | 44 -
[FFmpeg-devel] [PATCH v6 01/11] global: Prepare AVFrame for subtitle handling
Root commit for adding subtitle filtering capabilities. In detail: - Add type (AVMediaType) field to AVFrame Replaces previous way of distinction which was based on checking width and height to determine whether a frame is audio or video - Add subtitle fields to AVFrame - Add new struct AVSubtitleArea, similar to AVSubtitleRect, but different allocation logic. Cannot and must not be used interchangeably, hence the new struct - Move enum AVSubtitleType, AVSubtitle and AVSubtitleRect to avutil - Add public-named members to enum AVSubtitleType (AV_SUBTITLE_FMT_) - Add avcodec_decode_subtitle3 which takes subtitle frames, serving as compatibility shim to legacy subtitle decoding - Add additional methods for conversion between old and new API Signed-off-by: softworkz --- libavcodec/avcodec.c | 19 libavcodec/avcodec.h | 105 +++--- libavcodec/decode.c| 24 +++- libavcodec/pgssubdec.c | 1 + libavcodec/utils.c | 11 ++ libavfilter/vf_subtitles.c | 28 ++--- libavformat/utils.c| 1 + libavutil/Makefile | 2 + libavutil/frame.c | 184 +++--- libavutil/frame.h | 93 +++- libavutil/subfmt.c | 221 + libavutil/subfmt.h | 186 +++ libavutil/version.h| 2 +- 13 files changed, 757 insertions(+), 120 deletions(-) create mode 100644 libavutil/subfmt.c create mode 100644 libavutil/subfmt.h diff --git a/libavcodec/avcodec.c b/libavcodec/avcodec.c index 2dd7dd84e0..963f52c4bd 100644 --- a/libavcodec/avcodec.c +++ b/libavcodec/avcodec.c @@ -426,25 +426,6 @@ void avcodec_flush_buffers(AVCodecContext *avctx) av_bsf_flush(avci->bsf); } -void avsubtitle_free(AVSubtitle *sub) -{ -int i; - -for (i = 0; i < sub->num_rects; i++) { -av_freep(&sub->rects[i]->data[0]); -av_freep(&sub->rects[i]->data[1]); -av_freep(&sub->rects[i]->data[2]); -av_freep(&sub->rects[i]->data[3]); -av_freep(&sub->rects[i]->text); -av_freep(&sub->rects[i]->ass); -av_freep(&sub->rects[i]); -} - -av_freep(&sub->rects); - -memset(sub, 0, sizeof(*sub)); -} - av_cold int avcodec_close(AVCodecContext *avctx) { int i; diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index ffd58c333f..de23067d29 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -35,6 +35,7 @@ #include "libavutil/frame.h" #include "libavutil/log.h" #include "libavutil/pixfmt.h" +#include "libavutil/subfmt.h" #include "libavutil/rational.h" #include "codec.h" @@ -1670,7 +1671,7 @@ typedef struct AVCodecContext { /** * Header containing style information for text subtitles. - * For SUBTITLE_ASS subtitle type, it should contain the whole ASS + * For AV_SUBTITLE_FMT_ASS subtitle type, it should contain the whole ASS * [Script Info] and [V4+ Styles] section, plus the [Events] line and * the Format line following. It shouldn't include any Dialogue line. * - encoding: Set/allocated/freed by user (before avcodec_open2()) @@ -2233,63 +2234,8 @@ typedef struct AVHWAccel { * @} */ -enum AVSubtitleType { -SUBTITLE_NONE, - -SUBTITLE_BITMAP,///< A bitmap, pict will be set - -/** - * Plain text, the text field must be set by the decoder and is - * authoritative. ass and pict fields may contain approximations. - */ -SUBTITLE_TEXT, - -/** - * Formatted text, the ass field must be set by the decoder and is - * authoritative. pict and text fields may contain approximations. - */ -SUBTITLE_ASS, -}; - #define AV_SUBTITLE_FLAG_FORCED 0x0001 -typedef struct AVSubtitleRect { -int x; ///< top left corner of pict, undefined when pict is not set -int y; ///< top left corner of pict, undefined when pict is not set -int w; ///< widthof pict, undefined when pict is not set -int h; ///< height of pict, undefined when pict is not set -int nb_colors; ///< number of colors in pict, undefined when pict is not set - -/** - * data+linesize for the bitmap of this subtitle. - * Can be set for text/ass as well once they are rendered. - */ -uint8_t *data[4]; -int linesize[4]; - -enum AVSubtitleType type; - -char *text; ///< 0 terminated plain UTF-8 text - -/** - * 0 terminated ASS/SSA compatible event line. - * The presentation of this is unaffected by the other values in this - * struct. - */ -char *ass; - -int flags; -} AVSubtitleRect; - -typedef struct AVSubtitle { -uint16_t format; /* 0 = graphics */ -uint32_t start_display_time; /* relative to packet pts, in ms */ -uint32_t end_display_time; /* relative to packet pts, in ms */ -unsigned num_rects; -AVSubtitleRect **rects; -int64_t pts;
[FFmpeg-devel] [PATCH v6 02/11] fftools/play, probe: Adjust for subtitle changes
Signed-off-by: softworkz --- fftools/ffplay.c | 50 +++ fftools/ffprobe.c | 49 ++ 2 files changed, 62 insertions(+), 37 deletions(-) diff --git a/fftools/ffplay.c b/fftools/ffplay.c index ccea0e4578..08e5d53f9c 100644 --- a/fftools/ffplay.c +++ b/fftools/ffplay.c @@ -152,7 +152,6 @@ typedef struct Clock { /* Common struct for handling all types of decoded data and allocated render buffers. */ typedef struct Frame { AVFrame *frame; -AVSubtitle sub; int serial; double pts; /* presentation timestamp for the frame */ double duration; /* estimated duration of the frame */ @@ -586,7 +585,7 @@ static int decoder_init(Decoder *d, AVCodecContext *avctx, PacketQueue *queue, S return 0; } -static int decoder_decode_frame(Decoder *d, AVFrame *frame, AVSubtitle *sub) { +static int decoder_decode_frame(Decoder *d, AVFrame *frame) { int ret = AVERROR(EAGAIN); for (;;) { @@ -654,7 +653,7 @@ static int decoder_decode_frame(Decoder *d, AVFrame *frame, AVSubtitle *sub) { if (d->avctx->codec_type == AVMEDIA_TYPE_SUBTITLE) { int got_frame = 0; -ret = avcodec_decode_subtitle2(d->avctx, sub, &got_frame, d->pkt); +ret = avcodec_decode_subtitle3(d->avctx, frame, &got_frame, d->pkt); if (ret < 0) { ret = AVERROR(EAGAIN); } else { @@ -683,7 +682,6 @@ static void decoder_destroy(Decoder *d) { static void frame_queue_unref_item(Frame *vp) { av_frame_unref(vp->frame); -avsubtitle_free(&vp->sub); } static int frame_queue_init(FrameQueue *f, PacketQueue *pktq, int max_size, int keep_last) @@ -981,7 +979,7 @@ static void video_image_display(VideoState *is) if (frame_queue_nb_remaining(&is->subpq) > 0) { sp = frame_queue_peek(&is->subpq); -if (vp->pts >= sp->pts + ((float) sp->sub.start_display_time / 1000)) { +if (vp->pts >= sp->pts + ((float) sp->frame->subtitle_start_time / 1000)) { if (!sp->uploaded) { uint8_t* pixels[4]; int pitch[4]; @@ -993,8 +991,8 @@ static void video_image_display(VideoState *is) if (realloc_texture(&is->sub_texture, SDL_PIXELFORMAT_ARGB, sp->width, sp->height, SDL_BLENDMODE_BLEND, 1) < 0) return; -for (i = 0; i < sp->sub.num_rects; i++) { -AVSubtitleRect *sub_rect = sp->sub.rects[i]; +for (i = 0; i < sp->frame->num_subtitle_areas; i++) { +AVSubtitleArea *sub_rect = sp->frame->subtitle_areas[i]; sub_rect->x = av_clip(sub_rect->x, 0, sp->width ); sub_rect->y = av_clip(sub_rect->y, 0, sp->height); @@ -1041,13 +1039,15 @@ static void video_image_display(VideoState *is) int i; double xratio = (double)rect.w / (double)sp->width; double yratio = (double)rect.h / (double)sp->height; -for (i = 0; i < sp->sub.num_rects; i++) { -SDL_Rect *sub_rect = (SDL_Rect*)sp->sub.rects[i]; -SDL_Rect target = {.x = rect.x + sub_rect->x * xratio, - .y = rect.y + sub_rect->y * yratio, - .w = sub_rect->w * xratio, - .h = sub_rect->h * yratio}; -SDL_RenderCopy(renderer, is->sub_texture, sub_rect, &target); +for (i = 0; i < sp->frame->num_subtitle_areas; i++) { +AVSubtitleArea *area = sp->frame->subtitle_areas[i]; +SDL_Rect sub_rect = { .x = area->x, .y = area->y, + .w = area->w, .h = area->h}; +SDL_Rect target = {.x = rect.x + sub_rect.x * xratio, + .y = rect.y + sub_rect.y * yratio, + .w = sub_rect.w * xratio, + .h = sub_rect.h * yratio}; +SDL_RenderCopy(renderer, is->sub_texture, &sub_rect, &target); } #endif } @@ -1651,13 +1651,13 @@ retry: sp2 = NULL; if (sp->serial != is->subtitleq.serial -|| (is->vidclk.pts > (sp->pts + ((float) sp->sub.end_display_time / 1000))) -|| (sp2 && is->vidclk.pts > (sp2->pts + ((float) sp2->sub.start_display_time / 1000 +|| (is->vidclk.pts > (sp->pts + ((float) sp->frame->subtitle_end_time / 1000))) +|| (sp2 && is->vidclk.pts > (sp2->pts + ((float) sp2->frame->subtitle_start_time / 1000 { if (sp->uploaded) { int i; -for (i = 0; i < sp->sub.num_rects; i++) { -AVSubtitleRect *sub_rect = sp->sub.rects[i]
[FFmpeg-devel] [PATCH v6 03/11] avfilter/subtitles: Add subtitles.c for subtitle frame allocation
Analog to avfilter/video.c and avfilter/audio.c Signed-off-by: softworkz --- libavfilter/Makefile| 1 + libavfilter/avfilter.c | 4 +++ libavfilter/internal.h | 1 + libavfilter/subtitles.c | 61 + libavfilter/subtitles.h | 44 + 5 files changed, 111 insertions(+) create mode 100644 libavfilter/subtitles.c create mode 100644 libavfilter/subtitles.h diff --git a/libavfilter/Makefile b/libavfilter/Makefile index f059f3fef8..041d3c5382 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -19,6 +19,7 @@ OBJS = allfilters.o \ framequeue.o \ graphdump.o \ graphparser.o\ + subtitles.o \ video.o \ OBJS-$(HAVE_THREADS) += pthread.o diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c index c614eb0740..ac46bde782 100644 --- a/libavfilter/avfilter.c +++ b/libavfilter/avfilter.c @@ -43,6 +43,7 @@ #include "formats.h" #include "framepool.h" #include "internal.h" +#include "subtitles.h" #include "libavutil/ffversion.h" const char av_filter_ffversion[] = "FFmpeg version " FFMPEG_VERSION; @@ -1474,6 +1475,9 @@ int ff_inlink_make_frame_writable(AVFilterLink *link, AVFrame **rframe) case AVMEDIA_TYPE_AUDIO: out = ff_get_audio_buffer(link, frame->nb_samples); break; +case AVMEDIA_TYPE_SUBTITLE: +out = ff_get_subtitles_buffer(link, link->format); +break; default: return AVERROR(EINVAL); } diff --git a/libavfilter/internal.h b/libavfilter/internal.h index e7c154aff0..8977dda2b3 100644 --- a/libavfilter/internal.h +++ b/libavfilter/internal.h @@ -90,6 +90,7 @@ struct AVFilterPad { union { AVFrame *(*video)(AVFilterLink *link, int w, int h); AVFrame *(*audio)(AVFilterLink *link, int nb_samples); +AVFrame *(*subtitle)(AVFilterLink *link, int format); } get_buffer; /** diff --git a/libavfilter/subtitles.c b/libavfilter/subtitles.c new file mode 100644 index 00..fd2e7c3c0f --- /dev/null +++ b/libavfilter/subtitles.c @@ -0,0 +1,61 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/common.h" + +#include "subtitles.h" +#include "avfilter.h" +#include "internal.h" + + +AVFrame *ff_null_get_subtitles_buffer(AVFilterLink *link, int format) +{ +return ff_get_subtitles_buffer(link->dst->outputs[0], format); +} + +AVFrame *ff_default_get_subtitles_buffer(AVFilterLink *link, int format) +{ +AVFrame *frame; + +frame = av_frame_alloc(); +if (!frame) +return NULL; + +frame->format = format; +frame->type = AVMEDIA_TYPE_SUBTITLE; + +if (av_frame_get_buffer2(frame, 0) < 0) +return NULL; + +return frame; +} + +AVFrame *ff_get_subtitles_buffer(AVFilterLink *link, int format) +{ +AVFrame *ret = NULL; + +if (link->dstpad->get_buffer.subtitle) +ret = link->dstpad->get_buffer.subtitle(link, format); + +if (!ret) +ret = ff_default_get_subtitles_buffer(link, format); + +return ret; +} diff --git a/libavfilter/subtitles.h b/libavfilter/subtitles.h new file mode 100644 index 00..d3d5491652 --- /dev/null +++ b/libavfilter/subtitles.h @@ -0,0 +1,44 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Pub
[FFmpeg-devel] [PATCH v6 04/11] avfilter/avfilter: Handle subtitle frames
Signed-off-by: softworkz --- libavfilter/avfilter.c | 8 +--- libavfilter/avfiltergraph.c | 5 + libavfilter/formats.c | 14 ++ libavfilter/formats.h | 3 +++ 4 files changed, 27 insertions(+), 3 deletions(-) diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c index ac46bde782..3f303fbafa 100644 --- a/libavfilter/avfilter.c +++ b/libavfilter/avfilter.c @@ -57,7 +57,8 @@ void ff_tlog_ref(void *ctx, AVFrame *ref, int end) ref->linesize[0], ref->linesize[1], ref->linesize[2], ref->linesize[3], ref->pts, ref->pkt_pos); -if (ref->width) { +switch(ref->type) { +case AVMEDIA_TYPE_VIDEO: ff_tlog(ctx, " a:%d/%d s:%dx%d i:%c iskey:%d type:%c", ref->sample_aspect_ratio.num, ref->sample_aspect_ratio.den, ref->width, ref->height, @@ -65,12 +66,13 @@ void ff_tlog_ref(void *ctx, AVFrame *ref, int end) ref->top_field_first ? 'T' : 'B',/* Top / Bottom */ ref->key_frame, av_get_picture_type_char(ref->pict_type)); -} -if (ref->nb_samples) { +break; +case AVMEDIA_TYPE_AUDIO: ff_tlog(ctx, " cl:%"PRId64"d n:%d r:%d", ref->channel_layout, ref->nb_samples, ref->sample_rate); +break; } ff_tlog(ctx, "]%s", end ? "\n" : ""); diff --git a/libavfilter/avfiltergraph.c b/libavfilter/avfiltergraph.c index 45b028cd9c..7a5a4ea419 100644 --- a/libavfilter/avfiltergraph.c +++ b/libavfilter/avfiltergraph.c @@ -314,6 +314,8 @@ static int filter_link_check_formats(void *log, AVFilterLink *link, AVFilterForm return ret; break; +case AVMEDIA_TYPE_SUBTITLE: +return 0; default: av_assert0(!"reached"); } @@ -444,6 +446,9 @@ static int query_formats(AVFilterGraph *graph, void *log_ctx) if (!link) continue; +if (link->type == AVMEDIA_TYPE_SUBTITLE) +continue; + neg = ff_filter_get_negotiation(link); av_assert0(neg); for (neg_step = 1; neg_step < neg->nb_mergers; neg_step++) { diff --git a/libavfilter/formats.c b/libavfilter/formats.c index 1bf7d36195..704774d763 100644 --- a/libavfilter/formats.c +++ b/libavfilter/formats.c @@ -19,6 +19,7 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ +#include "libavcodec/avcodec.h" #include "libavutil/avassert.h" #include "libavutil/channel_layout.h" #include "libavutil/common.h" @@ -430,6 +431,12 @@ int ff_add_channel_layout(AVFilterChannelLayouts **l, uint64_t channel_layout) return 0; } +int ff_add_subtitle_type(AVFilterFormats **avff, int64_t fmt) +{ +ADD_FORMAT(avff, fmt, ff_formats_unref, int, formats, nb_formats); +return 0; +} + AVFilterFormats *ff_all_formats(enum AVMediaType type) { AVFilterFormats *ret = NULL; @@ -447,6 +454,13 @@ AVFilterFormats *ff_all_formats(enum AVMediaType type) return NULL; fmt++; } +} else if (type == AVMEDIA_TYPE_SUBTITLE) { +if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_BITMAP) < 0) +return NULL; +if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_ASS) < 0) +return NULL; +if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_TEXT) < 0) +return NULL; } return ret; diff --git a/libavfilter/formats.h b/libavfilter/formats.h index 471cb42bc4..25ae511fc1 100644 --- a/libavfilter/formats.h +++ b/libavfilter/formats.h @@ -180,6 +180,9 @@ int ff_set_common_formats_from_list(AVFilterContext *ctx, const int *fmts); av_warn_unused_result int ff_add_channel_layout(AVFilterChannelLayouts **l, uint64_t channel_layout); +av_warn_unused_result +int ff_add_subtitle_type(AVFilterFormats **avff, int64_t fmt); + /** * Add *ref as a new reference to f. */ -- 2.30.2.windows.1 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v6 05/11] avfilter/sbuffer: Add sbuffersrv and sbuffersink filters
Signed-off-by: softworkz --- configure| 2 +- libavfilter/allfilters.c | 10 +++--- libavfilter/buffersink.c | 63 +++ libavfilter/buffersink.h | 15 + libavfilter/buffersrc.c | 72 libavfilter/buffersrc.h | 1 + libavfilter/version.h| 2 +- 7 files changed, 159 insertions(+), 6 deletions(-) diff --git a/configure b/configure index 7ac23123c7..74c5a41be1 100755 --- a/configure +++ b/configure @@ -7718,7 +7718,7 @@ print_enabled_components(){ fi done if [ "$name" = "filter_list" ]; then -for c in asrc_abuffer vsrc_buffer asink_abuffer vsink_buffer; do +for c in asrc_abuffer vsrc_buffer ssrc_sbuffer asink_abuffer vsink_buffer ssink_sbuffer; do printf "&ff_%s,\n" $c >> $TMPH done fi diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index ddd6404228..154eba5bb2 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -533,10 +533,12 @@ extern const AVFilter ff_avsrc_movie; * they are formatted to not be found by the grep * as they are manually added again (due to their 'names' * being the same while having different 'types'). */ -extern const AVFilter ff_asrc_abuffer; -extern const AVFilter ff_vsrc_buffer; -extern const AVFilter ff_asink_abuffer; -extern const AVFilter ff_vsink_buffer; +extern const AVFilter ff_asrc_abuffer; +extern const AVFilter ff_vsrc_buffer; +extern const AVFilter ff_ssrc_sbuffer; +extern const AVFilter ff_asink_abuffer; +extern const AVFilter ff_vsink_buffer; +extern const AVFilter ff_ssink_sbuffer; extern const AVFilter ff_af_afifo; extern const AVFilter ff_vf_fifo; diff --git a/libavfilter/buffersink.c b/libavfilter/buffersink.c index 8b46dcb15e..55cbaff3d5 100644 --- a/libavfilter/buffersink.c +++ b/libavfilter/buffersink.c @@ -29,6 +29,8 @@ #include "libavutil/internal.h" #include "libavutil/opt.h" +#include "libavcodec/avcodec.h" + #define FF_INTERNAL_FIELDS 1 #include "framequeue.h" @@ -57,6 +59,10 @@ typedef struct BufferSinkContext { int *sample_rates; ///< list of accepted sample rates, terminated by -1 int sample_rates_size; +/* only used for subtitles */ +enum AVSubtitleType *subtitle_types; ///< list of accepted subtitle types, must be terminated with -1 +int subtitle_types_size; + AVFrame *peeked_frame; } BufferSinkContext; @@ -168,6 +174,15 @@ AVABufferSinkParams *av_abuffersink_params_alloc(void) return NULL; return params; } + +AVSBufferSinkParams *av_sbuffersink_params_alloc(void) +{ +AVSBufferSinkParams *params = av_mallocz(sizeof(AVSBufferSinkParams)); + +if (!params) +return NULL; +return params; +} #endif static av_cold int common_init(AVFilterContext *ctx) @@ -305,6 +320,28 @@ static int asink_query_formats(AVFilterContext *ctx) return 0; } +static int ssink_query_formats(AVFilterContext *ctx) +{ +BufferSinkContext *buf = ctx->priv; +AVFilterFormats *formats = NULL; +unsigned i; +int ret; + +CHECK_LIST_SIZE(subtitle_types) +if (buf->subtitle_types_size) { +for (i = 0; i < NB_ITEMS(buf->subtitle_types); i++) +if ((ret = ff_add_subtitle_type(&formats, buf->subtitle_types[i])) < 0) +return ret; +if ((ret = ff_set_common_formats(ctx, formats)) < 0) +return ret; +} else { +if ((ret = ff_default_query_formats(ctx)) < 0) +return ret; +} + +return 0; +} + #define OFFSET(x) offsetof(BufferSinkContext, x) #define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM static const AVOption buffersink_options[] = { @@ -322,9 +359,16 @@ static const AVOption abuffersink_options[] = { { NULL }, }; #undef FLAGS +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_SUBTITLE_PARAM +static const AVOption sbuffersink_options[] = { +{ "subtitle_types", "set the supported subtitle formats", OFFSET(subtitle_types), AV_OPT_TYPE_BINARY, .flags = FLAGS }, +{ NULL }, +}; +#undef FLAGS AVFILTER_DEFINE_CLASS(buffersink); AVFILTER_DEFINE_CLASS(abuffersink); +AVFILTER_DEFINE_CLASS(sbuffersink); static const AVFilterPad avfilter_vsink_buffer_inputs[] = { { @@ -363,3 +407,22 @@ const AVFilter ff_asink_abuffer = { FILTER_INPUTS(avfilter_asink_abuffer_inputs), .outputs = NULL, }; + +static const AVFilterPad avfilter_ssink_sbuffer_inputs[] = { +{ +.name = "default", +.type = AVMEDIA_TYPE_SUBTITLE, +}, +}; + +AVFilter ff_ssink_sbuffer = { +.name = "sbuffersink", +.description = NULL_IF_CONFIG_SMALL("Buffer subtitle frames, and make them available to the end of the filter graph."), +.priv_class= &sbuffersink_class, +.priv_size = sizeof(BufferSinkContext), +.init = common_init, +.query_formats = ssink_query_formats, +.activate =
[FFmpeg-devel] [PATCH v6 06/11] avfilter/overlay_graphicsubs: Add overlay_graphicsubs and graphicsub2video filters
- overlay_graphicsubs (VS -> V) Overlay graphic subtitles onto a video stream - graphicsub2video {S -> V) Converts graphic subtitles to video frames (with alpha) Gets auto-inserted for retaining compatibility with sub2video command lines Signed-off-by: softworkz --- doc/filters.texi | 104 libavfilter/Makefile | 2 + libavfilter/allfilters.c | 2 + libavfilter/vf_overlay_graphicsubs.c | 727 +++ 4 files changed, 835 insertions(+) create mode 100644 libavfilter/vf_overlay_graphicsubs.c diff --git a/doc/filters.texi b/doc/filters.texi index 8f20ccf8c6..e281af7155 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -25079,6 +25079,110 @@ tools. @c man end VIDEO SINKS +@chapter Subtitle Filters +@c man begin SUBTITLE FILTERS + +When you configure your FFmpeg build, you can disable any of the +existing filters using @code{--disable-filters}. + +Below is a description of the currently available subtitle filters. + +@section graphicsub2video + +Renders graphic subtitles as video frames. + +This filter replaces the previous "sub2video" hack which did the conversion implicitly and up-front as subtitle filtering wasn't possible at that time. +To retain compatibility with earlier sub2video command lines, this filter is being auto-inserted in those cases. + +For overlaying graphicsal subtitles it is recommended to use the 'overlay_graphicsubs' filter which is more efficient and takes less processing resources. + +This filter is still useful in cases where the overlay is done with hardware acceleration (e.g. overlay_qsv, overlay_vaapi, overlay_cuda) for preparing the overlay frames. + +It accepts the following parameters: + +@table @option +@item size, s +Set the size of the output video frame. + +@end table + +@subsection Examples + +@itemize +@item +Overlay PGS subtitles +(not recommended - better use overlay_graphicsubs) +@example +ffmpeg -i "https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv"; -filter_complex "[0:1]graphicsub2video[subs];[0:0][subs]overlay" output.mp4 +@end example + +@item +Overlay PGS subtitles implicitly +The graphicsub2video is inserted automatically for compatibility with legacy command lines. +@example +ffmpeg -i "https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv"; -filter_complex "[0:0][0:1]overlay" output.mp4 +@end example +@end itemize + +@section overlay_graphicsubs + +Overlay graphic subtitles onto a video stream. + +This filter can blend graphical subtitles on a video stream directly, i.e. without creating full-size alpha images first. +The blending operation is limited to the area of the subtitle rectangles, which also means that no processing is done at times where no subtitles are to be displayed. + + +It accepts the following parameters: + +@table @option +@item x +@item y +Set the expression for the x and y coordinates of the overlaid video +on the main video. Default value is "0" for both expressions. In case +the expression is invalid, it is set to a huge value (meaning that the +overlay will not be displayed within the output visible area). + +@item eof_action +See @ref{framesync}. + +@item eval +Set when the expressions for @option{x}, and @option{y} are evaluated. + +It accepts the following values: +@table @samp +@item init +only evaluate expressions once during the filter initialization or +when a command is processed + +@item frame +evaluate expressions for each incoming frame +@end table + +Default value is @samp{frame}. + +@item shortest +See @ref{framesync}. + +@end table + +@subsection Examples + +@itemize +@item +Overlay PGS subtitles +@example +ffmpeg -i "https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv"; -filter_complex "[0:1]graphicsub2video[subs];[0:0][subs]overlay" output.mp4 +@end example + +@item +Overlay PGS subtitles implicitly +The graphicsub2video is inserted automatically for compatibility with legacy command lines. +@example +ffmpeg -i "https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv"; -filter_complex "[0:0][0:1]overlay" output.mp4 +@end example +@end itemize +@c man end SUBTITLE FILTERS + @chapter Multimedia Filters @c man begin MULTIMEDIA FILTERS diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 041d3c5382..8fcc25989e 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -290,6 +290,7 @@ OBJS-$(CONFIG_FSPP_FILTER) += vf_fspp.o qp_table.o OBJS-$(CONFIG_GBLUR_FILTER) += vf_gblur.o OBJS-$(CONFIG_GEQ_FILTER)+= vf_geq.o OBJS-$(CONFIG_GRADFUN_FILTER)+= vf_gradfun.o +OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlay_graphicsubs.o framesync.o OBJS-$(CONFIG_GRAPHMONITOR_FILTER) += f_graphmonitor.o OBJS-$(CONFIG_GRAYWORLD_FILTER) += vf_grayworld.o
[FFmpeg-devel] [PATCH v6 08/11] avfilter/overlay_textsubs: Add overlay_textsubs and textsubs2video filters
- overlay_textsubs {VS -> V) Overlay text subtitles onto a video stream. - textsubs2video {S -> V) Converts text subtitles to video frames Signed-off-by: softworkz --- configure | 2 + doc/filters.texi | 69 libavfilter/Makefile | 2 + libavfilter/allfilters.c | 2 + libavfilter/avfilter.c| 18 +- libavfilter/vf_overlay_textsubs.c | 613 ++ 6 files changed, 701 insertions(+), 5 deletions(-) create mode 100644 libavfilter/vf_overlay_textsubs.c diff --git a/configure b/configure index 74c5a41be1..6d6a12754d 100755 --- a/configure +++ b/configure @@ -3624,6 +3624,7 @@ openclsrc_filter_deps="opencl" overlay_opencl_filter_deps="opencl" overlay_qsv_filter_deps="libmfx" overlay_qsv_filter_select="qsvvpp" +overlay_textsubs_filter_deps="avcodec libass" overlay_vulkan_filter_deps="vulkan_lib libglslang" owdenoise_filter_deps="gpl" pad_opencl_filter_deps="opencl" @@ -3669,6 +3670,7 @@ superequalizer_filter_deps="avcodec" superequalizer_filter_select="rdft" surround_filter_deps="avcodec" surround_filter_select="rdft" +textsub2video_filter_deps="avcodec libass" tinterlace_filter_deps="gpl" tinterlace_merge_test_deps="tinterlace_filter" tinterlace_pad_test_deps="tinterlace_filter" diff --git a/doc/filters.texi b/doc/filters.texi index e281af7155..1c38ecd8ec 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -25181,6 +25181,75 @@ The graphicsub2video is inserted automatically for compatibility with legacy com ffmpeg -i "https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv"; -filter_complex "[0:0][0:1]overlay" output.mp4 @end example @end itemize + +@section overlay_textsubs + +Overlay text subtitles onto a video stream. + +This filter supersedes the classic @ref{subtitles} filter opposed to which it does no longer require to open and access the source stream separately, which is often causing problems or doesn't even work for non-local or slow sources. + +Inputs: +- 0: Video [YUV420P, YUV422P, YUV444P, ARGB, RGBA, ABGR, BGRA, RGB24, BGR24] +- 1: Subtitles [text] + +Outputs: +- 0: Video (same as input) + +It accepts the following parameters: + +@table @option + +@item alpha +Process alpha channel, by default alpha channel is untouched. + +@item fonts_dir +Set a directory path containing fonts that can be used by the filter. +These fonts will be used in addition to whatever the font provider uses. + +@item force_style +Override default style or script info parameters of the subtitles. It accepts a +string containing ASS style format @code{KEY=VALUE} couples separated by ",". + +@end table + +@section textsub2video + +Converts text subtitles to video frames. + +For overlaying text subtitles onto video frames it is recommended to use the overlay_textsubs filter. +The textsub2video is useful for for creating transparent text-frames when overlay is done via hw acceleration + +Inputs: +- 0: Subtitles [text] + +Outputs: +- 0: Video [ARGB, RGBA, ABGR, BGRA] + +It accepts the following parameters: + +@table @option + +@item rate, r +Set the framerate for updating overlay frames. +Normally, overlay frames will only be updated each time when the subtitles to display are changing. +In cases where subtitles include advanced features (like animation), this parameter determines the frequency by which the overlay frames should be updated. + +@item size, s +Set the output frame size. +Allows to override the size of output video frames. + +@item alpha +Process alpha channel, by default alpha channel is untouched. + +@item fonts_dir +Set a directory path containing fonts that can be used by the filter. +These fonts will be used in addition to whatever the font provider uses. + +@item force_style +Override default style or script info parameters of the subtitles. It accepts a +string containing ASS style format @code{KEY=VALUE} couples separated by ",". + +@end table @c man end SUBTITLE FILTERS @chapter Multimedia Filters diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 8fcc25989e..c0b1cc7001 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -365,6 +365,7 @@ OBJS-$(CONFIG_OVERLAY_OPENCL_FILTER) += vf_overlay_opencl.o opencl.o \ opencl/overlay.o framesync.o OBJS-$(CONFIG_OVERLAY_QSV_FILTER)+= vf_overlay_qsv.o framesync.o OBJS-$(CONFIG_OVERLAY_GRAPHICSUBS_FILTER)+= vf_overlay_graphicsubs.o framesync.o +OBJS-$(CONFIG_OVERLAY_TEXTSUBS_FILTER) += vf_overlay_textsubs.o OBJS-$(CONFIG_OVERLAY_VULKAN_FILTER) += vf_overlay_vulkan.o vulkan.o OBJS-$(CONFIG_OWDENOISE_FILTER) += vf_owdenoise.o OBJS-$(CONFIG_PAD_FILTER)+= vf_pad.o @@ -453,6 +454,7 @@ OBJS-$(CONFIG_SWAPRECT_FILTER) += vf_swaprect.o OBJS-$(CONFIG_SWAPUV_FILTER) += vf_swapuv.o OBJS-$(CONFIG_TBLEND_FILTER)
[FFmpeg-devel] [PATCH v6 07/11] fftools/ffmpeg: Replace sub2video with subtitle frame filtering
This commit actually enables subtitle filtering in ffmpeg by sending and receiving subtitle frames to and from a filtergraph. The heartbeat functionality from the previous sub2video implementation is retained and applied to all subtitle frames (bitmap, text, ..). The other part of sub2video functionality is retained by auto-insertion of the new graphicsub2video filter. Signed-off-by: softworkz --- fftools/ffmpeg.c | 459 ++ fftools/ffmpeg.h | 14 +- fftools/ffmpeg_filter.c | 198 +++--- fftools/ffmpeg_hw.c | 2 +- fftools/ffmpeg_opt.c | 3 +- tests/ref/fate/filter-overlay-dvdsub-2397 | 181 + tests/ref/fate/sub-dvb| 162 tests/ref/fate/sub2video | 44 --- tests/ref/fate/sub2video_basic| 93 +++-- tests/ref/fate/sub2video_time_limited | 4 +- 10 files changed, 571 insertions(+), 589 deletions(-) diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index 5365f711e5..80ed9461a3 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -169,163 +169,6 @@ static int restore_tty; static void free_input_threads(void); #endif -/* sub2video hack: - Convert subtitles to video with alpha to insert them in filter graphs. - This is a temporary solution until libavfilter gets real subtitles support. - */ - -static int sub2video_get_blank_frame(InputStream *ist) -{ -int ret; -AVFrame *frame = ist->sub2video.frame; - -av_frame_unref(frame); -ist->sub2video.frame->width = ist->dec_ctx->width ? ist->dec_ctx->width : ist->sub2video.w; -ist->sub2video.frame->height = ist->dec_ctx->height ? ist->dec_ctx->height : ist->sub2video.h; -ist->sub2video.frame->format = AV_PIX_FMT_RGB32; -if ((ret = av_frame_get_buffer(frame, 0)) < 0) -return ret; -memset(frame->data[0], 0, frame->height * frame->linesize[0]); -return 0; -} - -static void sub2video_copy_rect(uint8_t *dst, int dst_linesize, int w, int h, -AVSubtitleRect *r) -{ -uint32_t *pal, *dst2; -uint8_t *src, *src2; -int x, y; - -if (r->type != SUBTITLE_BITMAP) { -av_log(NULL, AV_LOG_WARNING, "sub2video: non-bitmap subtitle\n"); -return; -} -if (r->x < 0 || r->x + r->w > w || r->y < 0 || r->y + r->h > h) { -av_log(NULL, AV_LOG_WARNING, "sub2video: rectangle (%d %d %d %d) overflowing %d %d\n", -r->x, r->y, r->w, r->h, w, h -); -return; -} - -dst += r->y * dst_linesize + r->x * 4; -src = r->data[0]; -pal = (uint32_t *)r->data[1]; -for (y = 0; y < r->h; y++) { -dst2 = (uint32_t *)dst; -src2 = src; -for (x = 0; x < r->w; x++) -*(dst2++) = pal[*(src2++)]; -dst += dst_linesize; -src += r->linesize[0]; -} -} - -static void sub2video_push_ref(InputStream *ist, int64_t pts) -{ -AVFrame *frame = ist->sub2video.frame; -int i; -int ret; - -av_assert1(frame->data[0]); -ist->sub2video.last_pts = frame->pts = pts; -for (i = 0; i < ist->nb_filters; i++) { -ret = av_buffersrc_add_frame_flags(ist->filters[i]->filter, frame, - AV_BUFFERSRC_FLAG_KEEP_REF | - AV_BUFFERSRC_FLAG_PUSH); -if (ret != AVERROR_EOF && ret < 0) -av_log(NULL, AV_LOG_WARNING, "Error while add the frame to buffer source(%s).\n", - av_err2str(ret)); -} -} - -void sub2video_update(InputStream *ist, int64_t heartbeat_pts, AVSubtitle *sub) -{ -AVFrame *frame = ist->sub2video.frame; -int8_t *dst; -int dst_linesize; -int num_rects, i; -int64_t pts, end_pts; - -if (!frame) -return; -if (sub) { -pts = av_rescale_q(sub->pts + sub->start_display_time * 1000LL, - AV_TIME_BASE_Q, ist->st->time_base); -end_pts = av_rescale_q(sub->pts + sub->end_display_time * 1000LL, - AV_TIME_BASE_Q, ist->st->time_base); -num_rects = sub->num_rects; -} else { -/* If we are initializing the system, utilize current heartbeat - PTS as the start time, and show until the following subpicture - is received. Otherwise, utilize the previous subpicture's end time - as the fall-back value. */ -pts = ist->sub2video.initialize ? -heartbeat_pts : ist->sub2video.end_pts; -end_pts = INT64_MAX; -num_rects = 0; -} -if (sub2video_get_blank_frame(ist) < 0) { -av_log(ist->dec_ctx, AV_LOG_ERROR, - "Impossible to get a blank canvas.\n"); -return; -} -dst = frame->data[0]; -dst_linesize = frame->linesize[0]; -for (i = 0; i < num_rects; i++) -sub2video_copy_rect(dst, dst_linesi
[FFmpeg-devel] [PATCH v6 09/11] avfilter/textmod: Add textmod filter
- textmod {S -> S) Modify subtitle text in a number of ways Signed-off-by: softworkz --- doc/filters.texi | 64 +++ libavfilter/Makefile | 3 + libavfilter/allfilters.c | 1 + libavfilter/sf_textmod.c | 372 +++ 4 files changed, 440 insertions(+) create mode 100644 libavfilter/sf_textmod.c diff --git a/doc/filters.texi b/doc/filters.texi index 1c38ecd8ec..1832dc8847 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -25087,6 +25087,70 @@ existing filters using @code{--disable-filters}. Below is a description of the currently available subtitle filters. +@section textmod + +Modify subtitle text in a number of ways. + +It accepts the following parameters: + +@table @option +@item mode +The kind of text modification to apply + +Supported operation modes are: + +@table @var +@item 0, leet +Convert subtitle text to 'leet speak'. It's primarily useful for testing as the modification will be visible with almost all text lines. +@item 1, to_upper +Change all text to upper case. Might improve readability. +@item 2, to_lower +Change all text to lower case. +@item 3, replace_chars +Replace one or more characters. Requires the find and replace parameters to be specified. +Both need to be equal in length. +The first char in find is replaced by the first char in replace, same for all subsequent chars. +@item 4, remove_chars +Remove certain characters. Requires the find parameter to be specified. +All chars in the find parameter string will be removed from all subtitle text. +@item 5, replace_words +Replace one or more words. Requires the find and replace parameters to be specified. Multiple words must be separated by the delimiter char specified vie the separator parameter (default: ','). +The number of words in the find and replace parameters needs to be equal. +The first word in find is replaced by the first word in replace, same for all subsequent words +@item 6, remove_words +Remove certain words. Requires the find parameter to be specified. Multiple words must be separated by the delimiter char specified vie the separator parameter (default: ','). +All words in the find parameter string will be removed from all subtitle text. +@end table + +@item find +Required for replace_chars, remove_chars, replace_words and remove_words. + +@item replace +Required for replace_chars and replace_words. + +@item separator +Delimiter character for words. Used with replace_words and remove_words- Must be a single character. +The default is '.'. + +@end table + +@subsection Examples + +@itemize +@item +Change all characters to upper case while keeping all styles and animations: +@example +ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv"; -filter_complex "[0:s]textmod=mode=to_upper" -map 0 -y out.mkv +@end example +@item +Mark the 100-pixel-wide region on the left edge of the frame as very +uninteresting (to be encoded at much lower quality than the rest of +the frame). +@example +addroi=0:0:100:ih:+1/5 +@end example +@end itemize + @section graphicsub2video Renders graphic subtitles as video frames. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index c0b1cc7001..630f701a5f 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -536,6 +536,9 @@ OBJS-$(CONFIG_YUVTESTSRC_FILTER) += vsrc_testsrc.o OBJS-$(CONFIG_NULLSINK_FILTER) += vsink_nullsink.o +# subtitle filters +OBJS-$(CONFIG_TEXTMOD_FILTER)+= sf_textmod.o + # multimedia filters OBJS-$(CONFIG_ABITSCOPE_FILTER) += avf_abitscope.o OBJS-$(CONFIG_ADRAWGRAPH_FILTER) += f_drawgraph.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 8543fa22e9..63e41b637d 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -526,6 +526,7 @@ extern const AVFilter ff_avf_showvolume; extern const AVFilter ff_avf_showwaves; extern const AVFilter ff_avf_showwavespic; extern const AVFilter ff_vaf_spectrumsynth; +extern const AVFilter ff_sf_textmod; extern const AVFilter ff_svf_graphicsub2video; extern const AVFilter ff_svf_textsub2video; diff --git a/libavfilter/sf_textmod.c b/libavfilter/sf_textmod.c new file mode 100644 index 00..6a29a2d3c0 --- /dev/null +++ b/libavfilter/sf_textmod.c @@ -0,0 +1,372 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Publ
[FFmpeg-devel] [PATCH v6 10/11] avcodec/ass_split: Extend ass dialog parsing
Signed-off-by: softworkz --- libavcodec/ass_split.c | 12 ++-- libavcodec/ass_split.h | 2 ++ 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/libavcodec/ass_split.c b/libavcodec/ass_split.c index 05c5453e53..e58585d3da 100644 --- a/libavcodec/ass_split.c +++ b/libavcodec/ass_split.c @@ -497,8 +497,8 @@ int ff_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, while (*buf == '\\') { char style[2], c[2], sep[2], c_num[2] = "0", tmp[128] = {0}; unsigned int color = 0x; -int len, size = -1, an = -1, alpha = -1; -int x1, y1, x2, y2, t1 = -1, t2 = -1; +int len, size = -1, an = -1, alpha = -1, scale = 0; +int x1, y1, x2, y2, t1 = -1, t2 = -1, accel = 1; if (sscanf(buf, "\\%1[bisu]%1[01\\}]%n", style, c, &len) > 1) { int close = c[0] == '0' ? 1 : c[0] == '1' ? 0 : -1; len += close != -1; @@ -546,6 +546,14 @@ int ff_ass_split_override_codes(const ASSCodesCallbacks *callbacks, void *priv, } else if (sscanf(buf, "\\org(%d,%d)%1[\\}]%n", &x1, &y1, sep, &len) > 2) { if (callbacks->origin) callbacks->origin(priv, x1, y1); +} else if (sscanf(buf, "\\t(%d,%d,%1[\\}]%n", &t1, &t2, sep, &len) > 2 || + sscanf(buf, "\\t(%d,%d,%d,%1[\\}]%n", &t1, &t2, &accel, sep, &len) > 3) { +if (callbacks->animate) +callbacks->animate(priv, t1, t2, accel, tmp); +} else if (sscanf(buf, "\\p%1[\\}]%n", sep, &len) > 0 || + sscanf(buf, "\\p%u%1[\\}]%n", &scale, sep, &len) > 1) { +if (callbacks->drawing_mode) +callbacks->drawing_mode(priv, scale); } else { len = strcspn(buf+1, "\\}") + 2; /* skip unknown code */ } diff --git a/libavcodec/ass_split.h b/libavcodec/ass_split.h index a45fb9b8a1..bda7bb27db 100644 --- a/libavcodec/ass_split.h +++ b/libavcodec/ass_split.h @@ -156,7 +156,9 @@ typedef struct { * @{ */ void (*move)(void *priv, int x1, int y1, int x2, int y2, int t1, int t2); +void (*animate)(void *priv, int t1, int t2, int accel, char *style); void (*origin)(void *priv, int x, int y); +void (*drawing_mode)(void *priv, int scale); /** @} */ /** -- 2.30.2.windows.1 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v6 11/11] avfilter/stripstyles: Add stripstyles filter
- stripstyles {S -> S) Remove all inline styles from subtitle events Signed-off-by: softworkz --- doc/filters.texi | 24 libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/sf_stripstyles.c | 211 +++ 4 files changed, 237 insertions(+) create mode 100644 libavfilter/sf_stripstyles.c diff --git a/doc/filters.texi b/doc/filters.texi index 1832dc8847..c8bf444d2c 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -25087,6 +25087,30 @@ existing filters using @code{--disable-filters}. Below is a description of the currently available subtitle filters. +@section stripstyles + +Remove all inline styles from subtitle events. + +It accepts the following parameters: + +@table @option +@item remove_animated +Also remove text which is subject to animation (default: true) +Usually, animated text elements are used used in addition to static subtitle lines for creating effects, so in most cases it is safe to remove the animation content. +If subtitle text is missing, try setting this to false. + +@end table + +@subsection Examples + +@itemize +@item +Change all characters to upper case while keeping all styles and animations: +@example +ffmpeg -i "https://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv"; -filter_complex "[0:1]stripstyles" -map 0 output.mkv +@end example +@end itemize + @section textmod Modify subtitle text in a number of ways. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 630f701a5f..94ec11c4ce 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -538,6 +538,7 @@ OBJS-$(CONFIG_NULLSINK_FILTER) += vsink_nullsink.o # subtitle filters OBJS-$(CONFIG_TEXTMOD_FILTER)+= sf_textmod.o +OBJS-$(CONFIG_STRIPSTYLES_FILTER)+= sf_stripstyles.o # multimedia filters OBJS-$(CONFIG_ABITSCOPE_FILTER) += avf_abitscope.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 63e41b637d..88dd97adf1 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -526,6 +526,7 @@ extern const AVFilter ff_avf_showvolume; extern const AVFilter ff_avf_showwaves; extern const AVFilter ff_avf_showwavespic; extern const AVFilter ff_vaf_spectrumsynth; +extern const AVFilter ff_sf_stripstyles; extern const AVFilter ff_sf_textmod; extern const AVFilter ff_svf_graphicsub2video; extern const AVFilter ff_svf_textsub2video; diff --git a/libavfilter/sf_stripstyles.c b/libavfilter/sf_stripstyles.c new file mode 100644 index 00..15c00e73b1 --- /dev/null +++ b/libavfilter/sf_stripstyles.c @@ -0,0 +1,211 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * text subtitle filter which removes inline-styles from subtitles + */ + +#include + +#include "libavutil/avassert.h" +#include "libavutil/opt.h" +#include "avfilter.h" +#include "internal.h" +#include "libavcodec/avcodec.h" +#include "libavcodec/ass_split.h" + +typedef struct StripStylesContext { +const AVClass *class; +enum AVSubtitleType format; +int remove_animated; +} StripStylesContext; + +typedef struct DialogContext { +const AVClass *class; +const StripStylesContext* ss_ctx; +AVBPrint buffer; +int drawing_scale; +int is_animated; +} DialogContext; + +static void dialog_text_cb(void *priv, const char *text, int len) +{ +DialogContext *s = priv; +if (!s->drawing_scale && (!s->is_animated || !s->ss_ctx->remove_animated)) +av_bprint_append_data(&s->buffer, text, len); +} + +static void dialog_new_line_cb(void *priv, int forced) +{ +DialogContext *s = priv; +if (!s->drawing_scale && !s->is_animated) +av_bprint_append_data(&s->buffer, forced ? "\\N" : "\\n", 2); +} + +static void dialog_drawing_mode_cb(void *priv, int scale) +{ +DialogContext *s = priv; +s->drawing_scale = scale; +} + +static void dialog_animate_cb(void *priv, int t1, int t2, int accel, char *style) +{ +DialogContext *s = priv; +s->is_animated = 1; +} + +static void dialog_move_cb(void *priv, int x1, int y1, int x2, int y2, int t1, int t2) +{ +DialogContext *s = priv; +if (t1 >= 0 || t2 >= 0) +s->is_anima
Re: [FFmpeg-devel] [PATCH v5 00/12] Subtitle Filtering
> -Original Message- > From: ffmpeg-devel On Behalf Of > Hendrik Leppkes > Sent: Wednesday, 15 September 2021 11:21 > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/12] Subtitle Filtering > > On Wed, Sep 15, 2021 at 9:39 AM Soft Works > wrote: > > > > In my upcoming patchset, I have added a new struct AVSubtitleArea > > which is similar to AVSubtitleRect with the difference that > > > > - It's not AVSubtitleRect (which can be deprecated then) > > - It uses refcounted buffers for the image data > > > > AVFrame gets an array of AVSubtitleArea structs. The image data > > is attached to AVSubtitleArea (data, buf, linesize). > > > > I'm not sure whether it would make sense to store the image data in > > buf/data of AVFrame like > > > > AVSubtitleArea[0] <=> buf[0], buf[1] > > AVSubtitleArea[1] <=> buf[2], buf[3] > > > > I suppose the main problem is that there can be more areas then we > have buf entries? > There is just one small wrinkle, there is an expectation in various > areas of the code that an empty frame has buf[0] zero, and conversely > a filled one does not. You made re-think this and there are quite a number of places that do that buf[0] checking. Replacing those checks with an API method might make sense, but I think it's beyond the scope of this patchset. The context above was about whether storing bitmap buffers in individual rects or inside the buffers of AVFrame. We need to be aware though, that storing the buffers in AVFrame wouldn't help with the checks for frame->buf[0], because: - For text subtitles, the texts are not stored in buffers - Subtitle frames that do not contain any rects at all are still valid For those reasons I resorted to adding a simple 1-byte buffer to subtitle frames in buf[0]. This allows to remain compatible with all existing frame handling logic. The meaning of buf[0]=!0 is the same as for other frames: "has data" as it will only get set on calling av_frame_get_buffer2. Kind regards, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 5/5] avcodec/libsvtav1: support constant quality mode
On Thu, Sep 16, 2021 at 1:55 PM wrote: > > From: Limin Wang > > Signed-off-by: Limin Wang > --- For the record, Christopher Degawa also posted something similar (https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210310224859.1726693-1-c...@randomderp.com/), and I finally recently had gotten to noting on IRC that the rate control options should be brought more in line with, say, libx264. Set AVOption-specific defaults (f.ex., libx264 sets CRF to its default value by default), and then if any of the other rate control modes is nonzero the appropriate rate control would get utilized in the wrapper. Jan ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 2/2] avcodec/wmaprodec: Check that the EOF frame was allocated before decoding into it
Fixes: NULL pointer dereference Fixes: 38125/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_XMA1_fuzzer-5151909422432256 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer --- libavcodec/wmaprodec.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavcodec/wmaprodec.c b/libavcodec/wmaprodec.c index a4b48b13a8c..7b489943f43 100644 --- a/libavcodec/wmaprodec.c +++ b/libavcodec/wmaprodec.c @@ -1855,7 +1855,7 @@ static int xma_decode_packet(AVCodecContext *avctx, void *data, eof = 1; for (i = 0; i < s->num_streams; i++) { -if (!s->xma[i].eof_done) { +if (!s->xma[i].eof_done && s->frames[i]->data[0]) { ret = decode_packet(avctx, &s->xma[i], s->frames[i], &got_stream_frame_ptr, avpkt); } -- 2.17.1 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 2/5] avcodec/libsvtav1: make intra_refresh_type configurable
Hi On Thu, Sep 16, 2021 at 1:54 PM wrote: > > From: Limin Wang > > Signed-off-by: Limin Wang > --- I think something like: avcodec/libsvtav1: make coded GOP type configurable Might be a bit better as a commit message? > libavcodec/libsvtav1.c | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/libavcodec/libsvtav1.c b/libavcodec/libsvtav1.c > index 82ae2b9..8c2c970 100644 > --- a/libavcodec/libsvtav1.c > +++ b/libavcodec/libsvtav1.c > @@ -210,7 +210,8 @@ static int config_enc_params(EbSvtAv1EncConfiguration > *param, > param->min_qp_allowed = avctx->qmin; > } > > -param->intra_refresh_type = 2; /* Real keyframes only */ > +/* 2 = IDR, closed GOP, 1 = CRA, open GOP */ > +param->intra_refresh_type = avctx->flags & AV_CODEC_FLAG_CLOSED_GOP ? 2 > : 1; > Ugh... I see they have EbIntraRefreshType for an enum (NO_REFRESH | CRA_REFRESH | IDR_REFRESH), but I guess those are not available through the public API? > if (svt_enc->la_depth >= 0) > param->look_ahead_distance = svt_enc->la_depth; > @@ -548,6 +549,7 @@ static const AVCodecDefault eb_enc_defaults[] = { > { "g", "-1"}, > { "qmin", "0" }, > { "qmax", "63"}, > +{ "flags", "+cgop" }, Let's keep the alphabetic ordering since this module is not yet a mess in that sense :) Thus this would go after "b" . Otherwise LGTM. Jan ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 1/2] avformat/mpegts: use actually read packet size in mpegts_resync special case
Fixes: infinite loop Fixes: 37986/clusterfuzz-testcase-minimized-ffmpeg_dem_MPEGTSRAW_fuzzer-5292311517462528 - Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer --- libavformat/mpegts.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libavformat/mpegts.c b/libavformat/mpegts.c index fe89d4fb9f3..bd13118f7f7 100644 --- a/libavformat/mpegts.c +++ b/libavformat/mpegts.c @@ -2865,8 +2865,8 @@ static int mpegts_resync(AVFormatContext *s, int seekback, const uint8_t *curren int64_t back = FFMIN(seekback, pos); //Special case for files like 01c56b0dc1.ts -if (current_packet[0] == 0x80 && current_packet[12] == 0x47) { -avio_seek(pb, 12 - back, SEEK_CUR); +if (current_packet[0] == 0x80 && current_packet[12] == 0x47 && pos >= TS_PACKET_SIZE) { +avio_seek(pb, 12 - TS_PACKET_SIZE, SEEK_CUR); return 0; } -- 2.17.1 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 2/5] avcodec/libsvtav1: make intra_refresh_type configurable
On Fri, Sep 17, 2021 at 01:38:46AM +0300, Jan Ekström wrote: > Hi > > On Thu, Sep 16, 2021 at 1:54 PM wrote: > > > > From: Limin Wang > > > > Signed-off-by: Limin Wang > > --- > > I think something like: > > avcodec/libsvtav1: make coded GOP type configurable > > Might be a bit better as a commit message? Yes, it's more better, I'll update it. > > libavcodec/libsvtav1.c | 4 +++- > > 1 file changed, 3 insertions(+), 1 deletion(-) > > > > diff --git a/libavcodec/libsvtav1.c b/libavcodec/libsvtav1.c > > index 82ae2b9..8c2c970 100644 > > --- a/libavcodec/libsvtav1.c > > +++ b/libavcodec/libsvtav1.c > > @@ -210,7 +210,8 @@ static int config_enc_params(EbSvtAv1EncConfiguration > > *param, > > param->min_qp_allowed = avctx->qmin; > > } > > > > -param->intra_refresh_type = 2; /* Real keyframes only */ > > +/* 2 = IDR, closed GOP, 1 = CRA, open GOP */ > > +param->intra_refresh_type = avctx->flags & AV_CODEC_FLAG_CLOSED_GOP ? > > 2 : 1; > > > > Ugh... I see they have EbIntraRefreshType for an enum (NO_REFRESH | > CRA_REFRESH | IDR_REFRESH), but I guess those are not available > through the public API? Yes, it's not public API. > > > if (svt_enc->la_depth >= 0) > > param->look_ahead_distance = svt_enc->la_depth; > > @@ -548,6 +549,7 @@ static const AVCodecDefault eb_enc_defaults[] = { > > { "g", "-1"}, > > { "qmin", "0" }, > > { "qmax", "63"}, > > +{ "flags", "+cgop" }, > > Let's keep the alphabetic ordering since this module is not yet a mess > in that sense :) Thus this would go after "b" . OK, I'll change the order. > > Otherwise LGTM. > > Jan > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". -- Thanks, Limin Wang ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 5/5] avcodec/libsvtav1: support constant quality mode
On Fri, Sep 17, 2021 at 01:10:33AM +0300, Jan Ekström wrote: > On Thu, Sep 16, 2021 at 1:55 PM wrote: > > > > From: Limin Wang > > > > Signed-off-by: Limin Wang > > --- > > For the record, Christopher Degawa also posted something similar > (https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210310224859.1726693-1-c...@randomderp.com/), I think It's very confusing for the configure for crf, it's 0 as cqp, user had to enable_tpl_la and configure crf also. So I prefer to my proposal. > and I finally recently had gotten to noting on IRC that the rate > control options should be brought more in line with, say, libx264. Set > AVOption-specific defaults (f.ex., libx264 sets CRF to its default > value by default), and then if any of the other rate control modes is > nonzero the appropriate rate control would get utilized in the > wrapper. As rate control options is supported already, so it's better to make crf mode works also. User can use AVOption-specific if can working as expect in future. If CRF is preferable default, I can change the default crf value to 50 instead of -1. > > Jan > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". -- Thanks, Limin Wang ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v3] libavfilter: add a gblur_vulkan filter
Ping. > From: Wu, Jianhua > Sent: Thursday, September 9, 2021 1:45 PM > To: ffmpeg-devel@ffmpeg.org > Cc: Wu, Jianhua > Subject: [PATCH v3] libavfilter: add a gblur_vulkan filter > > This commit adds a powerful and customizable gblur Vulkan filter, which > provides a maximum 127x127 kernel size of Gaussian Filter. > The size could be adjusted by requirements on quality or performance. > > The following command is on how to apply gblur_vulkan filter: > > ffmpeg -init_hw_device vulkan=vul:0 -filter_hw_device vul -i input.264 -vf > hwupload=extra_hw_frames=16,gblur_vulkan,hwdownload,format=yuv420 > p > output.264 > > Signed-off-by: Wu Jianhua > --- > configure | 1 + > libavfilter/Makefile | 1 + > libavfilter/allfilters.c | 1 + > libavfilter/vf_gblur_vulkan.c | 511 > ++ > 4 files changed, 514 insertions(+) Hi there, Any update? ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 01/13] avcodec/elbg: Remove avoidable buffer
Signed-off-by: Andreas Rheinhardt --- libavcodec/elbg.c | 9 +++-- 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/libavcodec/elbg.c b/libavcodec/elbg.c index d012d9a384..795fc83f16 100644 --- a/libavcodec/elbg.c +++ b/libavcodec/elbg.c @@ -376,7 +376,6 @@ int avpriv_do_elbg(int *points, int dim, int numpoints, int *codebook, elbg_data elbg_d; elbg_data *elbg = &elbg_d; int i, j, k, steps = 0, ret = 0; -int *dist_cb = av_malloc_array(numpoints, sizeof(int)); int *size_part = av_malloc_array(numCB, sizeof(int)); cell *list_buffer = av_malloc_array(numpoints, sizeof(cell)); cell *free_cells; @@ -394,7 +393,7 @@ int avpriv_do_elbg(int *points, int dim, int numpoints, int *codebook, elbg->utility_inc = av_malloc_array(numCB, sizeof(*elbg->utility_inc)); elbg->scratchbuf = av_malloc_array(5*dim, sizeof(int)); -if (!dist_cb || !size_part || !list_buffer || !elbg->cells || +if (!size_part || !list_buffer || !elbg->cells || !elbg->utility || !elbg->utility_inc || !elbg->scratchbuf) { ret = AVERROR(ENOMEM); goto out; @@ -423,9 +422,8 @@ int avpriv_do_elbg(int *points, int dim, int numpoints, int *codebook, } } elbg->nearest_cb[i] = best_idx; -dist_cb[i] = best_dist; -elbg->error += dist_cb[i]; -elbg->utility[elbg->nearest_cb[i]] += dist_cb[i]; +elbg->error += best_dist; +elbg->utility[elbg->nearest_cb[i]] += best_dist; free_cells->index = i; free_cells->next = elbg->cells[elbg->nearest_cb[i]]; elbg->cells[elbg->nearest_cb[i]] = free_cells; @@ -453,7 +451,6 @@ int avpriv_do_elbg(int *points, int dim, int numpoints, int *codebook, (steps < max_steps)); out: -av_free(dist_cb); av_free(size_part); av_free(elbg->utility); av_free(list_buffer); -- 2.30.2 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 02/13] avcodec/elbg: Move avpriv_init_elbg() down
It will avoid a forward declaration later. Signed-off-by: Andreas Rheinhardt --- libavcodec/elbg.c | 71 +++ 1 file changed, 35 insertions(+), 36 deletions(-) diff --git a/libavcodec/elbg.c b/libavcodec/elbg.c index 795fc83f16..3ca67b400c 100644 --- a/libavcodec/elbg.c +++ b/libavcodec/elbg.c @@ -332,42 +332,6 @@ static void do_shiftings(elbg_data *elbg) } } -#define BIG_PRIME 433494437LL - -int avpriv_init_elbg(int *points, int dim, int numpoints, int *codebook, - int numCB, int max_steps, int *closest_cb, - AVLFG *rand_state) -{ -int i, k, ret = 0; - -if (numpoints > 24*numCB) { -/* ELBG is very costly for a big number of points. So if we have a lot - of them, get a good initial codebook to save on iterations */ -int *temp_points = av_malloc_array(dim, (numpoints/8)*sizeof(int)); -if (!temp_points) -return AVERROR(ENOMEM); -for (i=0; iscratchbuf); return ret; } + +#define BIG_PRIME 433494437LL + +int avpriv_init_elbg(int *points, int dim, int numpoints, int *codebook, + int num_cb, int max_steps, int *closest_cb, + AVLFG *rand_state) +{ +int ret = 0; + +if (numpoints > 24LL * num_cb) { +/* ELBG is very costly for a big number of points. So if we have a lot + of them, get a good initial codebook to save on iterations */ +int *temp_points = av_malloc_array(dim, (numpoints/8)*sizeof(*temp_points)); +if (!temp_points) +return AVERROR(ENOMEM); +for (int i = 0; i < numpoints / 8; i++) { +int k = (i*BIG_PRIME) % numpoints; +memcpy(temp_points + i*dim, points + k*dim, dim * sizeof(*temp_points)); +} + +ret = avpriv_init_elbg(temp_points, dim, numpoints / 8, codebook, + num_cb, 2 * max_steps, closest_cb, rand_state); +if (ret < 0) { +av_freep(&temp_points); +return ret; +} +ret = avpriv_do_elbg(temp_points, dim, numpoints / 8, codebook, + num_cb, 2 * max_steps, closest_cb, rand_state); +av_free(temp_points); +} else // If not, initialize the codebook with random positions +for (int i = 0; i < num_cb; i++) +memcpy(codebook + i * dim, points + ((i*BIG_PRIME)%numpoints)*dim, + dim * sizeof(*codebook)); +return ret; +} -- 2.30.2 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 05/13] avcodec/elbg: Rename elbg_data to ELBGContext
It is in line with our naming conventions for types. Also change numCB to num_cb for the same reason. Signed-off-by: Andreas Rheinhardt --- libavcodec/elbg.c | 84 +++ libavcodec/elbg.h | 4 +-- 2 files changed, 44 insertions(+), 44 deletions(-) diff --git a/libavcodec/elbg.c b/libavcodec/elbg.c index b563254bbc..ac5c53161d 100644 --- a/libavcodec/elbg.c +++ b/libavcodec/elbg.c @@ -43,10 +43,10 @@ typedef struct cell_s { /** * ELBG internal data */ -typedef struct elbg_data { +typedef struct ELBGContext { int64_t error; int dim; -int numCB; +int num_cb; int *codebook; cell **cells; int64_t *utility; @@ -55,7 +55,7 @@ typedef struct elbg_data { int *points; AVLFG *rand_state; int *scratchbuf; -} elbg_data; +} ELBGContext; static inline int distance_limited(int *a, int *b, int dim, int limit) { @@ -80,7 +80,7 @@ static inline void vect_division(int *res, int *vect, int div, int dim) } -static int eval_error_cell(elbg_data *elbg, int *centroid, cell *cells) +static int eval_error_cell(ELBGContext *elbg, int *centroid, cell *cells) { int error=0; for (; cells; cells=cells->next) @@ -89,11 +89,12 @@ static int eval_error_cell(elbg_data *elbg, int *centroid, cell *cells) return error; } -static int get_closest_codebook(elbg_data *elbg, int index) +static int get_closest_codebook(ELBGContext *elbg, int index) { -int i, pick=0, diff, diff_min = INT_MAX; -for (i=0; inumCB; i++) +int pick = 0; +for (int i = 0, diff_min = INT_MAX; i < elbg->num_cb; i++) if (i != index) { +int diff; diff = distance_limited(elbg->codebook + i*elbg->dim, elbg->codebook + index*elbg->dim, elbg->dim, diff_min); if (diff < diff_min) { pick = i; @@ -103,17 +104,17 @@ static int get_closest_codebook(elbg_data *elbg, int index) return pick; } -static int get_high_utility_cell(elbg_data *elbg) +static int get_high_utility_cell(ELBGContext *elbg) { int i=0; /* Using linear search, do binary if it ever turns to be speed critical */ uint64_t r; -if (elbg->utility_inc[elbg->numCB-1] < INT_MAX) { -r = av_lfg_get(elbg->rand_state) % (unsigned int)elbg->utility_inc[elbg->numCB-1] + 1; +if (elbg->utility_inc[elbg->num_cb - 1] < INT_MAX) { +r = av_lfg_get(elbg->rand_state) % (unsigned int)elbg->utility_inc[elbg->num_cb - 1] + 1; } else { r = av_lfg_get(elbg->rand_state); -r = (av_lfg_get(elbg->rand_state) + (r<<32)) % elbg->utility_inc[elbg->numCB-1] + 1; +r = (av_lfg_get(elbg->rand_state) + (r<<32)) % elbg->utility_inc[elbg->num_cb - 1] + 1; } while (elbg->utility_inc[i] < r) { @@ -128,7 +129,7 @@ static int get_high_utility_cell(elbg_data *elbg) /** * Implementation of the simple LBG algorithm for just two codebooks */ -static int simple_lbg(elbg_data *elbg, +static int simple_lbg(ELBGContext *elbg, int dim, int *centroid[3], int newutility[3], @@ -169,7 +170,7 @@ static int simple_lbg(elbg_data *elbg, return newutility[0] + newutility[1]; } -static void get_new_centroids(elbg_data *elbg, int huc, int *newcentroid_i, +static void get_new_centroids(ELBGContext *elbg, int huc, int *newcentroid_i, int *newcentroid_p) { cell *tempcell; @@ -205,7 +206,7 @@ static void get_new_centroids(elbg_data *elbg, int huc, int *newcentroid_i, * @param indexes {luc, huc, cluc} * @param newcentroid A vector with the position of the new centroids */ -static void shift_codebook(elbg_data *elbg, int *indexes, +static void shift_codebook(ELBGContext *elbg, int *indexes, int *newcentroid[3]) { cell *tempdata; @@ -233,20 +234,19 @@ static void shift_codebook(elbg_data *elbg, int *indexes, } } -static void evaluate_utility_inc(elbg_data *elbg) +static void evaluate_utility_inc(ELBGContext *elbg) { -int i; int64_t inc=0; -for (i=0; i < elbg->numCB; i++) { -if (elbg->numCB*elbg->utility[i] > elbg->error) +for (int i = 0; i < elbg->num_cb; i++) { +if (elbg->num_cb * elbg->utility[i] > elbg->error) inc += elbg->utility[i]; elbg->utility_inc[i] = inc; } } -static void update_utility_and_n_cb(elbg_data *elbg, int idx, int newutility) +static void update_utility_and_n_cb(ELBGContext *elbg, int idx, int newutility) { cell *tempcell; @@ -262,7 +262,7 @@ static void update_utility_and_n_cb(elbg_data *elbg, int idx, int newutility) * @param elbg Internal elbg data * @param idx {luc (low utility cell, huc (high utility cell), cluc (closest cell to low utility cell)} */ -static void try_shift_candidate(elbg_data *elbg, int idx[3]) +static void try_shift_candidate(ELBGContext *elbg, int idx[3]) { int j, k, cont=0; int
[FFmpeg-devel] [PATCH 03/13] avcodec/elbg: Merge avpriv_init_elbg() into avpriv_do_elbg()
These functions are always called directly after another with the exact same arguments. This avoids exporting a symbol; it also avoids having to perform two calls for every caller. Signed-off-by: Andreas Rheinhardt --- libavcodec/a64multienc.c | 4 libavcodec/cinepakenc.c | 1 - libavcodec/elbg.c| 29 + libavcodec/elbg.h| 12 libavcodec/msvideo1enc.c | 3 --- libavcodec/roqvideoenc.c | 4 libavfilter/vf_elbg.c| 3 --- 7 files changed, 25 insertions(+), 31 deletions(-) diff --git a/libavcodec/a64multienc.c b/libavcodec/a64multienc.c index e2dd85b756..9ee0b07463 100644 --- a/libavcodec/a64multienc.c +++ b/libavcodec/a64multienc.c @@ -333,10 +333,6 @@ static int a64multi_encode_frame(AVCodecContext *avctx, AVPacket *pkt, buf = pkt->data; /* calc optimal new charset + charmaps */ -ret = avpriv_init_elbg(meta, 32, 1000 * c->mc_lifetime, best_cb, - CHARSET_CHARS, 50, charmap, &c->randctx); -if (ret < 0) -return ret; ret = avpriv_do_elbg(meta, 32, 1000 * c->mc_lifetime, best_cb, CHARSET_CHARS, 50, charmap, &c->randctx); if (ret < 0) diff --git a/libavcodec/cinepakenc.c b/libavcodec/cinepakenc.c index 41da231dfb..8e8b73ce1d 100644 --- a/libavcodec/cinepakenc.c +++ b/libavcodec/cinepakenc.c @@ -761,7 +761,6 @@ static int quantize(CinepakEncContext *s, int h, uint8_t *data[4], if (i < size) size = i; -avpriv_init_elbg(s->codebook_input, entry_size, i, codebook, size, 1, s->codebook_closest, &s->randctx); avpriv_do_elbg(s->codebook_input, entry_size, i, codebook, size, 1, s->codebook_closest, &s->randctx); // set up vq_data, which contains a single MB diff --git a/libavcodec/elbg.c b/libavcodec/elbg.c index 3ca67b400c..b563254bbc 100644 --- a/libavcodec/elbg.c +++ b/libavcodec/elbg.c @@ -332,7 +332,7 @@ static void do_shiftings(elbg_data *elbg) } } -int avpriv_do_elbg(int *points, int dim, int numpoints, int *codebook, +static int do_elbg(int *points, int dim, int numpoints, int *codebook, int numCB, int max_steps, int *closest_cb, AVLFG *rand_state) { @@ -426,7 +426,14 @@ out: #define BIG_PRIME 433494437LL -int avpriv_init_elbg(int *points, int dim, int numpoints, int *codebook, +/** + * Initialize the codebook vector for the elbg algorithm. + * If numpoints < 8*numCB this function fills codebook with random numbers. + * If not, it calls do_elbg for a (smaller) random sample of the points in + * points. + * @return < 0 in case of error, 0 otherwise + */ +static int init_elbg(int *points, int dim, int numpoints, int *codebook, int num_cb, int max_steps, int *closest_cb, AVLFG *rand_state) { @@ -443,13 +450,13 @@ int avpriv_init_elbg(int *points, int dim, int numpoints, int *codebook, memcpy(temp_points + i*dim, points + k*dim, dim * sizeof(*temp_points)); } -ret = avpriv_init_elbg(temp_points, dim, numpoints / 8, codebook, +ret = init_elbg(temp_points, dim, numpoints / 8, codebook, num_cb, 2 * max_steps, closest_cb, rand_state); if (ret < 0) { av_freep(&temp_points); return ret; } -ret = avpriv_do_elbg(temp_points, dim, numpoints / 8, codebook, +ret = do_elbg (temp_points, dim, numpoints / 8, codebook, num_cb, 2 * max_steps, closest_cb, rand_state); av_free(temp_points); } else // If not, initialize the codebook with random positions @@ -458,3 +465,17 @@ int avpriv_init_elbg(int *points, int dim, int numpoints, int *codebook, dim * sizeof(*codebook)); return ret; } + +int avpriv_do_elbg(int *points, int dim, int numpoints, + int *codebook, int num_cb, int max_steps, + int *closest_cb, AVLFG *rand_state) +{ +int ret; + +ret = init_elbg(points, dim, numpoints, codebook, +num_cb, max_steps, closest_cb, rand_state); +if (ret < 0) +return ret; +return do_elbg (points, dim, numpoints, codebook, +num_cb, max_steps, closest_cb, rand_state); +} diff --git a/libavcodec/elbg.h b/libavcodec/elbg.h index f48aa3b443..3b9c2931f3 100644 --- a/libavcodec/elbg.h +++ b/libavcodec/elbg.h @@ -42,16 +42,4 @@ int avpriv_do_elbg(int *points, int dim, int numpoints, int *codebook, int numCB, int num_steps, int *closest_cb, AVLFG *rand_state); -/** - * Initialize the **codebook vector for the elbg algorithm. If you have already - * a codebook and you want to refine it, you shouldn't call this function. - * If numpoints < 8*numCB this function fills **codebook with random numbers. - * If not, it calls avpriv_do_elbg for a (smaller) random sample o
[FFmpeg-devel] [PATCH 06/13] avcodec/elbg: Add persistent ELBGContext
It will be used in future commits to avoid having to allocate and free all the buffers used. Signed-off-by: Andreas Rheinhardt --- libavcodec/a64multienc.c | 8 ++-- libavcodec/cinepakenc.c | 5 - libavcodec/elbg.c| 12 +++- libavcodec/elbg.h| 16 +--- libavcodec/msvideo1enc.c | 12 +--- libavcodec/roqvideoenc.c | 7 +-- libavfilter/vf_elbg.c| 5 - 7 files changed, 52 insertions(+), 13 deletions(-) diff --git a/libavcodec/a64multienc.c b/libavcodec/a64multienc.c index 9ee0b07463..71a620b4ff 100644 --- a/libavcodec/a64multienc.c +++ b/libavcodec/a64multienc.c @@ -43,6 +43,7 @@ typedef struct A64Context { /* variables for multicolor modes */ +struct ELBGContext *elbg; AVLFG randctx; int mc_lifetime; int mc_use_5col; @@ -195,6 +196,9 @@ static void render_charset(AVCodecContext *avctx, uint8_t *charset, static av_cold int a64multi_close_encoder(AVCodecContext *avctx) { A64Context *c = avctx->priv_data; + +avpriv_elbg_free(&c->elbg); + av_freep(&c->mc_meta_charset); av_freep(&c->mc_best_cb); av_freep(&c->mc_charmap); @@ -333,8 +337,8 @@ static int a64multi_encode_frame(AVCodecContext *avctx, AVPacket *pkt, buf = pkt->data; /* calc optimal new charset + charmaps */ -ret = avpriv_do_elbg(meta, 32, 1000 * c->mc_lifetime, best_cb, - CHARSET_CHARS, 50, charmap, &c->randctx); +ret = avpriv_elbg_do(&c->elbg, meta, 32, 1000 * c->mc_lifetime, + best_cb, CHARSET_CHARS, 50, charmap, &c->randctx); if (ret < 0) return ret; diff --git a/libavcodec/cinepakenc.c b/libavcodec/cinepakenc.c index 8e8b73ce1d..2984b93de3 100644 --- a/libavcodec/cinepakenc.c +++ b/libavcodec/cinepakenc.c @@ -127,6 +127,7 @@ typedef struct CinepakEncContext { int min_min_strips; int max_max_strips; int strip_number_delta_range; +struct ELBGContext *elbg; } CinepakEncContext; #define OFFSET(x) offsetof(CinepakEncContext, x) @@ -761,7 +762,8 @@ static int quantize(CinepakEncContext *s, int h, uint8_t *data[4], if (i < size) size = i; -avpriv_do_elbg(s->codebook_input, entry_size, i, codebook, size, 1, s->codebook_closest, &s->randctx); +avpriv_elbg_do(&s->elbg, s->codebook_input, entry_size, i, codebook, + size, 1, s->codebook_closest, &s->randctx); // set up vq_data, which contains a single MB vq_data[0] = vq_pict_buf; @@ -1161,6 +1163,7 @@ static av_cold int cinepak_encode_end(AVCodecContext *avctx) CinepakEncContext *s = avctx->priv_data; int x; +avpriv_elbg_free(&s->elbg); av_frame_free(&s->last_frame); av_frame_free(&s->best_frame); av_frame_free(&s->scratch_frame); diff --git a/libavcodec/elbg.c b/libavcodec/elbg.c index ac5c53161d..9eac802688 100644 --- a/libavcodec/elbg.c +++ b/libavcodec/elbg.c @@ -466,12 +466,17 @@ static int init_elbg(int *points, int dim, int numpoints, int *codebook, return ret; } -int avpriv_do_elbg(int *points, int dim, int numpoints, +int avpriv_elbg_do(ELBGContext **elbgp, int *points, int dim, int numpoints, int *codebook, int num_cb, int max_steps, int *closest_cb, AVLFG *rand_state) { +ELBGContext *const elbg = *elbgp ? *elbgp : av_mallocz(sizeof(*elbg)); int ret; +if (!elbg) +return AVERROR(ENOMEM); +*elbgp = elbg; + ret = init_elbg(points, dim, numpoints, codebook, num_cb, max_steps, closest_cb, rand_state); if (ret < 0) @@ -479,3 +484,8 @@ int avpriv_do_elbg(int *points, int dim, int numpoints, return do_elbg (points, dim, numpoints, codebook, num_cb, max_steps, closest_cb, rand_state); } + +av_cold void avpriv_elbg_free(ELBGContext **elbgp) +{ +av_freep(elbgp); +} diff --git a/libavcodec/elbg.h b/libavcodec/elbg.h index a9a19aa5e4..abeeb4ff44 100644 --- a/libavcodec/elbg.h +++ b/libavcodec/elbg.h @@ -23,11 +23,16 @@ #include "libavutil/lfg.h" +struct ELBGContext; + /** * Implementation of the Enhanced LBG Algorithm * Based on the paper "Neural Networks 14:1219-1237" that can be found in * http://citeseer.ist.psu.edu/patan01enhanced.html . * + * @param ctx A pointer to a pointer to an already allocated ELBGContext + * or a pointer to NULL. In the latter case, this function + * will allocate an ELBGContext and put a pointer to it in `*ctx`. * @param points Input points. * @param dim Dimension of the points. * @param numpoints Num of points in **points. @@ -38,8 +43,13 @@ * @param rand_state A random number generator state. Should be already initialized by av_lfg_init(). * @return < 0 in case of error, 0 otherwise */ -int avpriv_do_elbg(int *points, int dim, int numpoints, int *codebook, - int num_cb, int num_steps, int *closest
[FFmpeg-devel] [PATCH 04/13] avfilter/vf_elbg: Rename ELBGContext->ELBGFilterContext
The former name will be used for a context for avpriv_do_elbg(). Signed-off-by: Andreas Rheinhardt --- libavfilter/vf_elbg.c | 18 +- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/libavfilter/vf_elbg.c b/libavfilter/vf_elbg.c index bf78030ffe..79797ee25f 100644 --- a/libavfilter/vf_elbg.c +++ b/libavfilter/vf_elbg.c @@ -33,7 +33,7 @@ #include "internal.h" #include "video.h" -typedef struct ELBGContext { +typedef struct ELBGFilterContext { const AVClass *class; AVLFG lfg; int64_t lfg_seed; @@ -46,9 +46,9 @@ typedef struct ELBGContext { const AVPixFmtDescriptor *pix_desc; uint8_t rgba_map[4]; int pal8; -} ELBGContext; +} ELBGFilterContext; -#define OFFSET(x) offsetof(ELBGContext, x) +#define OFFSET(x) offsetof(ELBGFilterContext, x) #define FLAGS AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_FILTERING_PARAM static const AVOption elbg_options[] = { @@ -66,7 +66,7 @@ AVFILTER_DEFINE_CLASS(elbg); static av_cold int init(AVFilterContext *ctx) { -ELBGContext *elbg = ctx->priv; +ELBGFilterContext *const elbg = ctx->priv; if (elbg->pal8 && elbg->codebook_length > 256) { av_log(ctx, AV_LOG_ERROR, "pal8 output allows max 256 codebook length.\n"); @@ -82,7 +82,7 @@ static av_cold int init(AVFilterContext *ctx) static int query_formats(AVFilterContext *ctx) { -ELBGContext *elbg = ctx->priv; +ELBGFilterContext *const elbg = ctx->priv; int ret; static const enum AVPixelFormat pix_fmts[] = { @@ -109,7 +109,7 @@ static int query_formats(AVFilterContext *ctx) static int config_input(AVFilterLink *inlink) { AVFilterContext *ctx = inlink->dst; -ELBGContext *elbg = ctx->priv; +ELBGFilterContext *const elbg = ctx->priv; elbg->pix_desc = av_pix_fmt_desc_get(inlink->format); elbg->codeword_length = inlink->w * inlink->h; @@ -140,7 +140,7 @@ static int config_input(AVFilterLink *inlink) static int filter_frame(AVFilterLink *inlink, AVFrame *frame) { -ELBGContext *elbg = inlink->dst->priv; +ELBGFilterContext *const elbg = inlink->dst->priv; int i, j, k; uint8_t *p, *p0; @@ -221,7 +221,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *frame) static av_cold void uninit(AVFilterContext *ctx) { -ELBGContext *elbg = ctx->priv; +ELBGFilterContext *const elbg = ctx->priv; av_freep(&elbg->codebook); av_freep(&elbg->codeword); @@ -248,7 +248,7 @@ static const AVFilterPad elbg_outputs[] = { const AVFilter ff_vf_elbg = { .name = "elbg", .description = NULL_IF_CONFIG_SMALL("Apply posterize effect, using the ELBG algorithm."), -.priv_size = sizeof(ELBGContext), +.priv_size = sizeof(ELBGFilterContext), .priv_class= &elbg_class, .query_formats = query_formats, .init = init, -- 2.30.2 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 08/13] avcodec/elbg: Keep buffers to avoid allocations and frees
Up until now, each call to avpriv_elbg_do() would result in at least six allocations. And this function is called a lot: A typical FATE run results in 52213653 calls to av_malloc; of these, 34974671 originate from av_malloc_array and from these 34783679 originate from avpriv_elbg_do; the msvideo1 encoder tests are behind most of these. This commit changes this by keeping the buffers and only reallocating them when needed. E.g. for the encoding part of fate-vsynth1-msvideo1 total heap usage went down from 11,407,939 allocs and frees with 468,106,207 bytes allocated to 3,149 allocs and frees with 13,181,847 bytes allocated. The time for one encode2-call went down by 69%. Signed-off-by: Andreas Rheinhardt --- libavcodec/elbg.c | 84 ++- 1 file changed, 54 insertions(+), 30 deletions(-) diff --git a/libavcodec/elbg.c b/libavcodec/elbg.c index 24c6f06f54..4397bff1ef 100644 --- a/libavcodec/elbg.c +++ b/libavcodec/elbg.c @@ -53,8 +53,20 @@ typedef struct ELBGContext { int64_t *utility_inc; int *nearest_cb; int *points; +int *size_part; AVLFG *rand_state; int *scratchbuf; +cell *cell_buffer; + +/* Sizes for the buffers above. Pointers without such a field + * are not allocated by us and only valid for the duration + * of a single call to avpriv_elbg_do(). */ +unsigned utility_allocated; +unsigned utility_inc_allocated; +unsigned size_part_allocated; +unsigned cells_allocated; +unsigned scratchbuf_allocated; +unsigned cell_buffer_allocated; } ELBGContext; static inline int distance_limited(int *a, int *b, int dim, int limit) @@ -332,32 +344,19 @@ static void do_shiftings(ELBGContext *elbg) } } -static int do_elbg(ELBGContext *elbg, int *points, int numpoints, - int max_steps) +static void do_elbg(ELBGContext *elbg, int *points, int numpoints, +int max_steps) { -int i, j, steps = 0, ret = 0; -int *size_part = av_malloc_array(elbg->num_cb, sizeof(int)); -cell *list_buffer = av_malloc_array(numpoints, sizeof(cell)); -cell *free_cells; +int *const size_part = elbg->size_part; +int i, j, steps = 0; int best_idx = 0; int64_t last_error; elbg->error = INT64_MAX; -elbg->cells = av_malloc_array(elbg->num_cb, sizeof(cell *)); -elbg->utility = av_malloc_array(elbg->num_cb, sizeof(*elbg->utility)); elbg->points = points; -elbg->utility_inc = av_malloc_array(elbg->num_cb, sizeof(*elbg->utility_inc)); -elbg->scratchbuf = av_malloc_array(5 * elbg->dim, sizeof(int)); - -if (!size_part || !list_buffer || !elbg->cells || -!elbg->utility || !elbg->utility_inc || !elbg->scratchbuf) { -ret = AVERROR(ENOMEM); -goto out; -} - do { -free_cells = list_buffer; +cell *free_cells = elbg->cell_buffer; last_error = elbg->error; steps++; memset(elbg->utility, 0, elbg->num_cb * sizeof(*elbg->utility)); @@ -408,15 +407,6 @@ static int do_elbg(ELBGContext *elbg, int *points, int numpoints, } while(((last_error - elbg->error) > DELTA_ERR_MAX*elbg->error) && (steps < max_steps)); - -out: -av_free(size_part); -av_free(elbg->utility); -av_free(list_buffer); -av_free(elbg->cells); -av_free(elbg->utility_inc); -av_free(elbg->scratchbuf); -return ret; } #define BIG_PRIME 433494437LL @@ -450,13 +440,13 @@ static int init_elbg(ELBGContext *elbg, int *points, int numpoints, av_freep(&temp_points); return ret; } -ret = do_elbg(elbg, temp_points, numpoints / 8, 2 * max_steps); +do_elbg(elbg, temp_points, numpoints / 8, 2 * max_steps); av_free(temp_points); } else // If not, initialize the codebook with random positions for (int i = 0; i < elbg->num_cb; i++) memcpy(elbg->codebook + i * dim, points + ((i*BIG_PRIME)%numpoints)*dim, dim * sizeof(*elbg->codebook)); -return ret; +return 0; } int avpriv_elbg_do(ELBGContext **elbgp, int *points, int dim, int numpoints, @@ -476,13 +466,47 @@ int avpriv_elbg_do(ELBGContext **elbgp, int *points, int dim, int numpoints, elbg->num_cb = num_cb; elbg->dim= dim; +#define ALLOCATE_IF_NECESSARY(field, new_elements, multiplicator)\ +if (elbg->field ## _allocated < new_elements) { \ +av_freep(&elbg->field); \ +elbg->field = av_malloc_array(new_elements, \ + multiplicator * sizeof(*elbg->field)); \ +if (!elbg->field) { \ +elbg->field ## _allocated = 0; \ +return AVERROR(ENOMEM); \ +}
[FFmpeg-devel] [PATCH 11/13] avcodec/msvideo1enc: Check all calls to avpriv_elbg_do()
Signed-off-by: Andreas Rheinhardt --- libavcodec/msvideo1enc.c | 20 +--- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/libavcodec/msvideo1enc.c b/libavcodec/msvideo1enc.c index d43013ba5f..79810ec8c3 100644 --- a/libavcodec/msvideo1enc.c +++ b/libavcodec/msvideo1enc.c @@ -118,8 +118,10 @@ static int encode_frame(AVCodecContext *avctx, AVPacket *pkt, } // try to find optimal value to fill whole 4x4 block score = 0; -avpriv_elbg_do(&c->elbg, c->block, 3, 16, c->avg, - 1, 1, c->output, &c->rnd); +ret = avpriv_elbg_do(&c->elbg, c->block, 3, 16, c->avg, + 1, 1, c->output, &c->rnd); +if (ret < 0) +return ret; if(c->avg[0] == 1) // red component = 1 will be written as skip code c->avg[0] = 0; for(j = 0; j < 4; j++){ @@ -138,8 +140,10 @@ static int encode_frame(AVCodecContext *avctx, AVPacket *pkt, } // search for optimal filling of 2-color block score = 0; -avpriv_elbg_do(&c->elbg, c->block, 3, 16, c->codebook, - 2, 1, c->output, &c->rnd); +ret = avpriv_elbg_do(&c->elbg, c->block, 3, 16, c->codebook, + 2, 1, c->output, &c->rnd); +if (ret < 0) +return ret; // last output value should be always 1, swap codebooks if needed if(!c->output[15]){ for(i = 0; i < 3; i++) @@ -164,9 +168,11 @@ static int encode_frame(AVCodecContext *avctx, AVPacket *pkt, // search for optimal filling of 2-color 2x2 subblocks score = 0; for(i = 0; i < 4; i++){ -avpriv_elbg_do(&c->elbg, c->block2 + i * 4 * 3, 3, 4, - c->codebook2 + i * 2 * 3, 2, 1, - c->output2 + i*4, &c->rnd); +ret = avpriv_elbg_do(&c->elbg, c->block2 + i * 4 * 3, 3, 4, + c->codebook2 + i * 2 * 3, 2, 1, + c->output2 + i * 4, &c->rnd); +if (ret < 0) +return ret; } // last value should be always 1, swap codebooks if needed if(!c->output2[15]){ -- 2.30.2 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 13/13] avcodec/elbg: Add flags to avpriv_elbg_do()
This is currently unused and it is only added to enable changes while maintaining ABI compatibility. The type is uintptr_t in order to potentially accept a pointer argument. Signed-off-by: Andreas Rheinhardt --- libavcodec/a64multienc.c | 2 +- libavcodec/cinepakenc.c | 2 +- libavcodec/elbg.c| 2 +- libavcodec/elbg.h| 4 +++- libavcodec/msvideo1enc.c | 6 +++--- libavcodec/roqvideoenc.c | 2 +- libavfilter/vf_elbg.c| 2 +- 7 files changed, 11 insertions(+), 9 deletions(-) diff --git a/libavcodec/a64multienc.c b/libavcodec/a64multienc.c index 71a620b4ff..c368ac218a 100644 --- a/libavcodec/a64multienc.c +++ b/libavcodec/a64multienc.c @@ -338,7 +338,7 @@ static int a64multi_encode_frame(AVCodecContext *avctx, AVPacket *pkt, /* calc optimal new charset + charmaps */ ret = avpriv_elbg_do(&c->elbg, meta, 32, 1000 * c->mc_lifetime, - best_cb, CHARSET_CHARS, 50, charmap, &c->randctx); + best_cb, CHARSET_CHARS, 50, charmap, &c->randctx, 0); if (ret < 0) return ret; diff --git a/libavcodec/cinepakenc.c b/libavcodec/cinepakenc.c index d1bcf2b2d5..edb553f0db 100644 --- a/libavcodec/cinepakenc.c +++ b/libavcodec/cinepakenc.c @@ -764,7 +764,7 @@ static int quantize(CinepakEncContext *s, int h, uint8_t *data[4], size = i; ret = avpriv_elbg_do(&s->elbg, s->codebook_input, entry_size, i, codebook, - size, 1, s->codebook_closest, &s->randctx); + size, 1, s->codebook_closest, &s->randctx, 0); if (ret < 0) return ret; diff --git a/libavcodec/elbg.c b/libavcodec/elbg.c index 2bacf5b773..712c125a58 100644 --- a/libavcodec/elbg.c +++ b/libavcodec/elbg.c @@ -445,7 +445,7 @@ static void init_elbg(ELBGContext *elbg, int *points, int *temp_points, int avpriv_elbg_do(ELBGContext **elbgp, int *points, int dim, int numpoints, int *codebook, int num_cb, int max_steps, - int *closest_cb, AVLFG *rand_state) + int *closest_cb, AVLFG *rand_state, uintptr_t flags) { ELBGContext *const elbg = *elbgp ? *elbgp : av_mallocz(sizeof(*elbg)); diff --git a/libavcodec/elbg.h b/libavcodec/elbg.h index abeeb4ff44..34d96846b1 100644 --- a/libavcodec/elbg.h +++ b/libavcodec/elbg.h @@ -21,6 +21,7 @@ #ifndef AVCODEC_ELBG_H #define AVCODEC_ELBG_H +#include #include "libavutil/lfg.h" struct ELBGContext; @@ -41,11 +42,12 @@ struct ELBGContext; * @param num_steps The maximum number of steps. One step is already a good compromise between time and quality. * @param closest_cb Return the closest codebook to each point. Must be allocated. * @param rand_state A random number generator state. Should be already initialized by av_lfg_init(). + * @param flags Currently unused; must be set to 0. * @return < 0 in case of error, 0 otherwise */ int avpriv_elbg_do(struct ELBGContext **ctx, int *points, int dim, int numpoints, int *codebook, int num_cb, int num_steps, - int *closest_cb, AVLFG *rand_state); + int *closest_cb, AVLFG *rand_state, uintptr_t flags); /** * Free an ELBGContext and reset the pointer to it. diff --git a/libavcodec/msvideo1enc.c b/libavcodec/msvideo1enc.c index 79810ec8c3..a8761bdd2e 100644 --- a/libavcodec/msvideo1enc.c +++ b/libavcodec/msvideo1enc.c @@ -119,7 +119,7 @@ static int encode_frame(AVCodecContext *avctx, AVPacket *pkt, // try to find optimal value to fill whole 4x4 block score = 0; ret = avpriv_elbg_do(&c->elbg, c->block, 3, 16, c->avg, - 1, 1, c->output, &c->rnd); + 1, 1, c->output, &c->rnd, 0); if (ret < 0) return ret; if(c->avg[0] == 1) // red component = 1 will be written as skip code @@ -141,7 +141,7 @@ static int encode_frame(AVCodecContext *avctx, AVPacket *pkt, // search for optimal filling of 2-color block score = 0; ret = avpriv_elbg_do(&c->elbg, c->block, 3, 16, c->codebook, - 2, 1, c->output, &c->rnd); + 2, 1, c->output, &c->rnd, 0); if (ret < 0) return ret; // last output value should be always 1, swap codebooks if needed @@ -170,7 +170,7 @@ static int encode_frame(AVCodecContext *avctx, AVPacket *pkt, for(i = 0; i < 4; i++){ ret = avpriv_elbg_do(&c->elbg, c->block2 + i * 4 * 3, 3, 4, c->codebook2 + i * 2 * 3, 2, 1, - c->output2 + i * 4, &c->rnd); + c->output2 + i * 4, &c->rnd, 0); if (ret < 0) return ret; } diff --git a/libavcodec/roqvideoenc.c b/libavcodec/roqvideoenc
[FFmpeg-devel] [PATCH 07/13] avcodec/elbg: Move arguments to the context early if possible
This affects all the arguments that don't change during a call to avpriv_elbg_do(); doing so makes it easily recognizable which arguments change upon recursive calls. Signed-off-by: Andreas Rheinhardt --- libavcodec/elbg.c | 73 +++ 1 file changed, 35 insertions(+), 38 deletions(-) diff --git a/libavcodec/elbg.c b/libavcodec/elbg.c index 9eac802688..24c6f06f54 100644 --- a/libavcodec/elbg.c +++ b/libavcodec/elbg.c @@ -332,30 +332,22 @@ static void do_shiftings(ELBGContext *elbg) } } -static int do_elbg(int *points, int dim, int numpoints, int *codebook, - int num_cb, int max_steps, int *closest_cb, -AVLFG *rand_state) +static int do_elbg(ELBGContext *elbg, int *points, int numpoints, + int max_steps) { -int dist; -ELBGContext elbg_d; -ELBGContext *elbg = &elbg_d; int i, j, steps = 0, ret = 0; -int *size_part = av_malloc_array(num_cb, sizeof(int)); +int *size_part = av_malloc_array(elbg->num_cb, sizeof(int)); cell *list_buffer = av_malloc_array(numpoints, sizeof(cell)); cell *free_cells; -int best_dist, best_idx = 0; +int best_idx = 0; int64_t last_error; elbg->error = INT64_MAX; -elbg->dim = dim; -elbg->num_cb = num_cb; -elbg->codebook = codebook; -elbg->cells = av_malloc_array(num_cb, sizeof(cell *)); -elbg->utility = av_malloc_array(num_cb, sizeof(*elbg->utility)); -elbg->nearest_cb = closest_cb; +elbg->cells = av_malloc_array(elbg->num_cb, sizeof(cell *)); +elbg->utility = av_malloc_array(elbg->num_cb, sizeof(*elbg->utility)); elbg->points = points; -elbg->utility_inc = av_malloc_array(num_cb, sizeof(*elbg->utility_inc)); -elbg->scratchbuf = av_malloc_array(5*dim, sizeof(int)); +elbg->utility_inc = av_malloc_array(elbg->num_cb, sizeof(*elbg->utility_inc)); +elbg->scratchbuf = av_malloc_array(5 * elbg->dim, sizeof(int)); if (!size_part || !list_buffer || !elbg->cells || !elbg->utility || !elbg->utility_inc || !elbg->scratchbuf) { @@ -363,23 +355,26 @@ static int do_elbg(int *points, int dim, int numpoints, int *codebook, goto out; } -elbg->rand_state = rand_state; do { free_cells = list_buffer; last_error = elbg->error; steps++; -memset(elbg->utility, 0, num_cb * sizeof(*elbg->utility)); -memset(elbg->cells, 0, num_cb * sizeof(*elbg->cells)); +memset(elbg->utility, 0, elbg->num_cb * sizeof(*elbg->utility)); +memset(elbg->cells, 0, elbg->num_cb * sizeof(*elbg->cells)); elbg->error = 0; /* This loop evaluate the actual Voronoi partition. It is the most costly part of the algorithm. */ for (i=0; i < numpoints; i++) { -best_dist = distance_limited(elbg->points + i*elbg->dim, elbg->codebook + best_idx*elbg->dim, dim, INT_MAX); +int best_dist = distance_limited(elbg->points + i * elbg->dim, + elbg->codebook + best_idx * elbg->dim, + elbg->dim, INT_MAX); for (int k = 0; k < elbg->num_cb; k++) { -dist = distance_limited(elbg->points + i*elbg->dim, elbg->codebook + k*elbg->dim, dim, best_dist); +int dist = distance_limited(elbg->points + i * elbg->dim, +elbg->codebook + k * elbg->dim, +elbg->dim, best_dist); if (dist < best_dist) { best_dist = dist; best_idx = k; @@ -396,9 +391,9 @@ static int do_elbg(int *points, int dim, int numpoints, int *codebook, do_shiftings(elbg); -memset(size_part, 0, num_cb * sizeof(*size_part)); +memset(size_part, 0, elbg->num_cb * sizeof(*size_part)); -memset(elbg->codebook, 0, elbg->num_cb * dim * sizeof(*elbg->codebook)); +memset(elbg->codebook, 0, elbg->num_cb * elbg->dim * sizeof(*elbg->codebook)); for (i=0; i < numpoints; i++) { size_part[elbg->nearest_cb[i]]++; @@ -433,13 +428,13 @@ out: * points. * @return < 0 in case of error, 0 otherwise */ -static int init_elbg(int *points, int dim, int numpoints, int *codebook, - int num_cb, int max_steps, int *closest_cb, - AVLFG *rand_state) +static int init_elbg(ELBGContext *elbg, int *points, int numpoints, + int max_steps) { +int dim = elbg->dim; int ret = 0; -if (numpoints > 24LL * num_cb) { +if (numpoints > 24LL * elbg->num_cb) { /* ELBG is very costly for a big number of points. So if we have a lot of them, get a good initial codebook to save on iterations */ int *temp_points = av_malloc_array(dim, (numpoints/8)*sizeof(*temp_points)); @@ -450,19 +445,17 @@ static
[FFmpeg-devel] [PATCH 09/13] avcodec/elbg: Also allocate buffers for recursion only once
This is possible because the number of elements needed in each recursion step decreases geometrically, so the geometric series provides an upper bound for the sum of number of elements of the needed buffers. Signed-off-by: Andreas Rheinhardt --- libavcodec/elbg.c | 38 -- 1 file changed, 20 insertions(+), 18 deletions(-) diff --git a/libavcodec/elbg.c b/libavcodec/elbg.c index 4397bff1ef..2bacf5b773 100644 --- a/libavcodec/elbg.c +++ b/libavcodec/elbg.c @@ -53,6 +53,7 @@ typedef struct ELBGContext { int64_t *utility_inc; int *nearest_cb; int *points; +int *temp_points; int *size_part; AVLFG *rand_state; int *scratchbuf; @@ -67,6 +68,7 @@ typedef struct ELBGContext { unsigned cells_allocated; unsigned scratchbuf_allocated; unsigned cell_buffer_allocated; +unsigned temp_points_allocated; } ELBGContext; static inline int distance_limited(int *a, int *b, int dim, int limit) @@ -416,37 +418,29 @@ static void do_elbg(ELBGContext *elbg, int *points, int numpoints, * If numpoints <= 24 * num_cb this function fills codebook with random numbers. * If not, it calls do_elbg for a (smaller) random sample of the points in * points. - * @return < 0 in case of error, 0 otherwise */ -static int init_elbg(ELBGContext *elbg, int *points, int numpoints, - int max_steps) +static void init_elbg(ELBGContext *elbg, int *points, int *temp_points, + int numpoints, int max_steps) { int dim = elbg->dim; -int ret = 0; if (numpoints > 24LL * elbg->num_cb) { /* ELBG is very costly for a big number of points. So if we have a lot of them, get a good initial codebook to save on iterations */ -int *temp_points = av_malloc_array(dim, (numpoints/8)*sizeof(*temp_points)); -if (!temp_points) -return AVERROR(ENOMEM); for (int i = 0; i < numpoints / 8; i++) { int k = (i*BIG_PRIME) % numpoints; memcpy(temp_points + i*dim, points + k*dim, dim * sizeof(*temp_points)); } -ret = init_elbg(elbg, temp_points, numpoints / 8, 2 * max_steps); -if (ret < 0) { -av_freep(&temp_points); -return ret; -} +/* If anything is changed in the recursion parameters, + * the allocated size of temp_points will also need to be updated. */ +init_elbg(elbg, temp_points, temp_points + numpoints / 8 * dim, + numpoints / 8, 2 * max_steps); do_elbg(elbg, temp_points, numpoints / 8, 2 * max_steps); -av_free(temp_points); } else // If not, initialize the codebook with random positions for (int i = 0; i < elbg->num_cb; i++) memcpy(elbg->codebook + i * dim, points + ((i*BIG_PRIME)%numpoints)*dim, dim * sizeof(*elbg->codebook)); -return 0; } int avpriv_elbg_do(ELBGContext **elbgp, int *points, int dim, int numpoints, @@ -454,7 +448,6 @@ int avpriv_elbg_do(ELBGContext **elbgp, int *points, int dim, int numpoints, int *closest_cb, AVLFG *rand_state) { ELBGContext *const elbg = *elbgp ? *elbgp : av_mallocz(sizeof(*elbg)); -int ret; if (!elbg) return AVERROR(ENOMEM); @@ -487,10 +480,18 @@ int avpriv_elbg_do(ELBGContext **elbgp, int *points, int dim, int numpoints, ALLOCATE_IF_NECESSARY(size_part, num_cb,1) ALLOCATE_IF_NECESSARY(cell_buffer, numpoints, 1) ALLOCATE_IF_NECESSARY(scratchbuf, dim, 5) +if (numpoints > 24LL * elbg->num_cb) { +/* The first step in the recursion in init_elbg() needs a buffer with +* (numpoints / 8) * dim elements; the next step needs numpoints / 8 / 8 +* * dim elements etc. The geometric series leads to an upper bound of +* numpoints / 8 * 8 / 7 * dim elements. */ +uint64_t prod = dim * (uint64_t)(numpoints / 7U); +if (prod > INT_MAX) +return AVERROR(ERANGE); +ALLOCATE_IF_NECESSARY(temp_points, prod, 1) +} -ret = init_elbg(elbg, points, numpoints, max_steps); -if (ret < 0) -return ret; +init_elbg(elbg, points, elbg->temp_points, numpoints, max_steps); do_elbg (elbg, points, numpoints, max_steps); return 0; } @@ -507,6 +508,7 @@ av_cold void avpriv_elbg_free(ELBGContext **elbgp) av_freep(&elbg->cells); av_freep(&elbg->utility_inc); av_freep(&elbg->scratchbuf); +av_freep(&elbg->temp_points); av_freep(elbgp); } -- 2.30.2 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 10/13] avcodec/cinepakenc: Check all calls to avpriv_elbg_do()
Signed-off-by: Andreas Rheinhardt --- libavcodec/cinepakenc.c | 28 ++-- 1 file changed, 22 insertions(+), 6 deletions(-) diff --git a/libavcodec/cinepakenc.c b/libavcodec/cinepakenc.c index 2984b93de3..d1bcf2b2d5 100644 --- a/libavcodec/cinepakenc.c +++ b/libavcodec/cinepakenc.c @@ -709,6 +709,7 @@ static int quantize(CinepakEncContext *s, int h, uint8_t *data[4], uint8_t vq_pict_buf[(MB_AREA * 3) / 2]; uint8_t *sub_data[4], *vq_data[4]; int sub_linesize[4], vq_linesize[4]; +int ret; for (mbn = i = y = 0; y < h; y += MB_SIZE) { for (x = 0; x < s->w; x += MB_SIZE, ++mbn) { @@ -762,8 +763,10 @@ static int quantize(CinepakEncContext *s, int h, uint8_t *data[4], if (i < size) size = i; -avpriv_elbg_do(&s->elbg, s->codebook_input, entry_size, i, codebook, - size, 1, s->codebook_closest, &s->randctx); +ret = avpriv_elbg_do(&s->elbg, s->codebook_input, entry_size, i, codebook, + size, 1, s->codebook_closest, &s->randctx); +if (ret < 0) +return ret; // set up vq_data, which contains a single MB vq_data[0] = vq_pict_buf; @@ -888,8 +891,10 @@ static int rd_strip(CinepakEncContext *s, int y, int h, int keyframe, if (mode == MODE_V1_ONLY) { info.v1_size = v1_size; // the size may shrink even before optimizations if the input is short: -info.v1_size = quantize(s, h, data, linesize, 1, -&info, ENC_UNCERTAIN); +if ((new_v1_size = quantize(s, h, data, linesize, 1, +&info, ENC_UNCERTAIN)) < 0) +return new_v1_size; +info.v1_size = new_v1_size; if (info.v1_size < v1_size) // too few eligible blocks, no sense in trying bigger sizes v1enough = 1; @@ -902,8 +907,11 @@ static int rd_strip(CinepakEncContext *s, int y, int h, int keyframe, if (mode == MODE_V1_V4) { info.v4_size = v4_size; -info.v4_size = quantize(s, h, data, linesize, 0, -&info, ENC_UNCERTAIN); +new_v4_size = quantize(s, h, data, linesize, 0, + &info, ENC_UNCERTAIN); +if (new_v4_size < 0) +return new_v4_size; +info.v4_size = new_v4_size; if (info.v4_size < v4_size) // too few eligible blocks, no sense in trying bigger sizes v4enough = 1; @@ -921,11 +929,15 @@ static int rd_strip(CinepakEncContext *s, int y, int h, int keyframe, // we assume we _may_ come here with more blocks to encode than before info.v1_size = v1_size; new_v1_size = quantize(s, h, data, linesize, 1, &info, ENC_V1); +if (new_v1_size < 0) +return new_v1_size; if (new_v1_size < info.v1_size) info.v1_size = new_v1_size; // we assume we _may_ come here with more blocks to encode than before info.v4_size = v4_size; new_v4_size = quantize(s, h, data, linesize, 0, &info, ENC_V4); +if (new_v4_size < 0) +return new_v4_size; if (new_v4_size < info.v4_size) info.v4_size = new_v4_size; // calculate the resulting score @@ -942,12 +954,16 @@ static int rd_strip(CinepakEncContext *s, int y, int h, int keyframe, if (v1shrunk) { info.v1_size = v1_size; new_v1_size = quantize(s, h, data, linesize, 1, &info, ENC_V1); +if (new_v1_size < 0) +return new_v1_size; if (new_v1_size < info.v1_size) info.v1_size = new_v1_size; } if (v4shrunk) { info.v4_size = v4_size; new_v4_size = quantize(s, h, data, linesize, 0, &info, ENC_V4); +if (new_v4_size < 0) +return new_v4_size; if (new_v4_size < info.v4_size) info.v4_size = new_v4_size; } -- 2.30.2 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or
[FFmpeg-devel] [PATCH 12/13] avfilter/vf_elbg: Check call to avpriv_elbg_do()
Signed-off-by: Andreas Rheinhardt --- libavfilter/vf_elbg.c | 13 + 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/libavfilter/vf_elbg.c b/libavfilter/vf_elbg.c index 2c9c861d02..fac3b6f7fe 100644 --- a/libavfilter/vf_elbg.c +++ b/libavfilter/vf_elbg.c @@ -142,7 +142,7 @@ static int config_input(AVFilterLink *inlink) static int filter_frame(AVFilterLink *inlink, AVFrame *frame) { ELBGFilterContext *const elbg = inlink->dst->priv; -int i, j, k; +int i, j, k, ret; uint8_t *p, *p0; const uint8_t r_idx = elbg->rgba_map[R]; @@ -164,9 +164,14 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *frame) } /* compute the codebook */ -avpriv_elbg_do(&elbg->ctx, elbg->codeword, NB_COMPONENTS, elbg->codeword_length, - elbg->codebook, elbg->codebook_length, elbg->max_steps_nb, - elbg->codeword_closest_codebook_idxs, &elbg->lfg); +ret = avpriv_elbg_do(&elbg->ctx, elbg->codeword, NB_COMPONENTS, + elbg->codeword_length, elbg->codebook, + elbg->codebook_length, elbg->max_steps_nb, + elbg->codeword_closest_codebook_idxs, &elbg->lfg); +if (ret < 0) { +av_frame_free(&frame); +return ret; +} if (elbg->pal8) { AVFilterLink *outlink = inlink->dst->outputs[0]; -- 2.30.2 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v1 1/1] avformat/amr: Return PATCHWELCOME on stereo files
Ping. Anyone could review this patch? Thx. 在 2021年9月16日 +0800 11:24,sunzhenliang ,写道: > Signed-off-by: sunzhenliang > --- > libavformat/amr.c | 22 ++ > 1 file changed, 18 insertions(+), 4 deletions(-) > > diff --git a/libavformat/amr.c b/libavformat/amr.c > index 836b276fd5..2762010ebe 100644 > --- a/libavformat/amr.c > +++ b/libavformat/amr.c > @@ -36,8 +36,10 @@ typedef struct { > uint64_t block_count; > } AMRContext; > > -static const char AMR_header[] = "#!AMR\n"; > -static const char AMRWB_header[] = "#!AMR-WB\n"; > +static const char AMR_header[] = "#!AMR\n"; > +static const char AMR_MC_header[] = "#!AMR_MC1.0\n"; > +static const char AMRWB_header[] = "#!AMR-WB\n"; > +static const char AMRWB_MC_header[] = "#!AMR-WB_MC1.0\n"; > > static const uint8_t amrnb_packed_size[16] = { > 13, 14, 16, 18, 20, 21, 27, 32, 6, 1, 1, 1, 1, 1, 1, 1 > @@ -82,7 +84,7 @@ static int amr_read_header(AVFormatContext *s) > { > AVIOContext *pb = s->pb; > AVStream *st; > - uint8_t header[9]; > + uint8_t header[15]; > > if (avio_read(pb, header, 6) != 6) > return AVERROR_INVALIDDATA; > @@ -94,7 +96,19 @@ static int amr_read_header(AVFormatContext *s) > if (avio_read(pb, header + 6, 3) != 3) > return AVERROR_INVALIDDATA; > if (memcmp(header, AMRWB_header, 9)) { > - return -1; > + if (avio_read(pb, header + 6 + 3, 3) != 3) > + return AVERROR_INVALIDDATA; > + if (memcmp(header, AMR_MC_header, 12)) { > + if (avio_read(pb, header + 6 + 3 + 3, 3) != 3) > + return AVERROR_INVALIDDATA; > + if (memcmp(header, AMRWB_MC_header, 15)) { > + return -1; > + } > + avpriv_report_missing_feature(s, "multi-channel AMRWB"); > + return AVERROR_PATCHWELCOME; > + } > + avpriv_report_missing_feature(s, "multi-channel AMR"); > + return AVERROR_PATCHWELCOME; > } > > st->codecpar->codec_tag = MKTAG('s', 'a', 'w', 'b'); > -- > 2.25.1 > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH] avfilter/vf_zscale: fix output color_range discrepancy
This filter chain was supposed to convert from narrow range to full range yuv444p, but didn't: buffer=width=1280:height=720:pix_fmt=yuv444p:frame_rate=25/1:\ time_base=1/25:sar=1/1,zscale=min=709:rin=limited:pin=709:\ tin=709:t=linear,format=gbrpf32le,zscale=tin=linear:p=709:m=709:\ r=full:t=709,format=pix_fmts=yuv444p,buffersink --- libavfilter/vf_zscale.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavfilter/vf_zscale.c b/libavfilter/vf_zscale.c index dfea00f9eb..06a025e6e6 100644 --- a/libavfilter/vf_zscale.c +++ b/libavfilter/vf_zscale.c @@ -676,7 +676,7 @@ static int filter_frame(AVFilterLink *link, AVFrame *in) out->color_primaries = (int)s->dst_format.color_primaries; if (s->range != -1) -out->color_range = (int)s->dst_format.pixel_range; +out->color_range = (int)s->dst_format.pixel_range + 1; if (s->trc != -1) out->color_trc = (int)s->dst_format.transfer_characteristics; -- 2.26.2 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] avutil/buffer: Avoid allocation of AVBuffer when using buffer pool
Andreas Rheinhardt: > Do this by putting an AVBuffer structure into BufferPoolEntry and > reuse it for all subsequent uses of said BufferPoolEntry. > > Signed-off-by: Andreas Rheinhardt > --- > libavutil/buffer.c | 44 + > libavutil/buffer_internal.h | 11 ++ > 2 files changed, 41 insertions(+), 14 deletions(-) > > diff --git a/libavutil/buffer.c b/libavutil/buffer.c > index b13eeadffb..4d9ccf74b7 100644 > --- a/libavutil/buffer.c > +++ b/libavutil/buffer.c > @@ -26,16 +26,11 @@ > #include "mem.h" > #include "thread.h" > > -AVBufferRef *av_buffer_create(uint8_t *data, size_t size, > - void (*free)(void *opaque, uint8_t *data), > - void *opaque, int flags) > +static AVBufferRef *buffer_create(AVBuffer *buf, uint8_t *data, size_t size, > + void (*free)(void *opaque, uint8_t *data), > + void *opaque, int flags) > { > AVBufferRef *ref = NULL; > -AVBuffer*buf = NULL; > - > -buf = av_mallocz(sizeof(*buf)); > -if (!buf) > -return NULL; > > buf->data = data; > buf->size = size; > @@ -47,10 +42,8 @@ AVBufferRef *av_buffer_create(uint8_t *data, size_t size, > buf->flags = flags; > > ref = av_mallocz(sizeof(*ref)); > -if (!ref) { > -av_freep(&buf); > +if (!ref) > return NULL; > -} > > ref->buffer = buf; > ref->data = data; > @@ -59,6 +52,23 @@ AVBufferRef *av_buffer_create(uint8_t *data, size_t size, > return ref; > } > > +AVBufferRef *av_buffer_create(uint8_t *data, size_t size, > + void (*free)(void *opaque, uint8_t *data), > + void *opaque, int flags) > +{ > +AVBufferRef *ret; > +AVBuffer *buf = av_mallocz(sizeof(*buf)); > +if (!buf) > +return NULL; > + > +ret = buffer_create(buf, data, size, free, opaque, flags); > +if (!ret) { > +av_free(buf); > +return NULL; > +} > +return ret; > +} > + > void av_buffer_default_free(void *opaque, uint8_t *data) > { > av_free(data); > @@ -117,8 +127,12 @@ static void buffer_replace(AVBufferRef **dst, > AVBufferRef **src) > av_freep(dst); > > if (atomic_fetch_sub_explicit(&b->refcount, 1, memory_order_acq_rel) == > 1) { > +/* b->free below might already free the structure containing *b, > + * so we have to read the flag now to avoid use-after-free. */ > +int free_avbuffer = !(b->flags_internal & BUFFER_FLAG_NO_FREE); > b->free(b->opaque, b->data); > -av_freep(&b); > +if (free_avbuffer) > +av_free(b); > } > } > > @@ -378,11 +392,13 @@ AVBufferRef *av_buffer_pool_get(AVBufferPool *pool) > ff_mutex_lock(&pool->mutex); > buf = pool->pool; > if (buf) { > -ret = av_buffer_create(buf->data, pool->size, pool_release_buffer, > - buf, 0); > +memset(&buf->buffer, 0, sizeof(buf->buffer)); > +ret = buffer_create(&buf->buffer, buf->data, pool->size, > +pool_release_buffer, buf, 0); > if (ret) { > pool->pool = buf->next; > buf->next = NULL; > +buf->buffer.flags_internal |= BUFFER_FLAG_NO_FREE; > } > } else { > ret = pool_alloc_buffer(pool); > diff --git a/libavutil/buffer_internal.h b/libavutil/buffer_internal.h > index 839dc05f8f..bdff1b5b32 100644 > --- a/libavutil/buffer_internal.h > +++ b/libavutil/buffer_internal.h > @@ -30,6 +30,11 @@ > * The buffer was av_realloc()ed, so it is reallocatable. > */ > #define BUFFER_FLAG_REALLOCATABLE (1 << 0) > +/** > + * The AVBuffer structure is part of a larger structure > + * and should not be freed. > + */ > +#define BUFFER_FLAG_NO_FREE (1 << 1) > > struct AVBuffer { > uint8_t *data; /**< data described by this buffer */ > @@ -73,6 +78,12 @@ typedef struct BufferPoolEntry { > > AVBufferPool *pool; > struct BufferPoolEntry *next; > + > +/* > + * An AVBuffer structure to (re)use as AVBuffer for subsequent uses > + * of this BufferPoolEntry. > + */ > +AVBuffer buffer; > } BufferPoolEntry; > > struct AVBufferPool { > Will apply this tomorrow unless there are objections. - Andreas ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/12] Subtitle Filtering
> -Original Message- > From: ffmpeg-devel On Behalf Of > Soft Works > Sent: Thursday, 16 September 2021 19:46 > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/12] Subtitle Filtering > [..] > > - Sparseness. Subtitles streams have gaps, and synchronization with > > other streams requires a next frame, that can be minutes away or > > never > > come. This needs to be solved in a way compatible with > processing. > > I have kept the heartbeat logic from your sub2video implementation. > It makes sense, is required and I can't think of any better way to > handle this. It's just renamed to subtitle_heartbeat and used for > all subtitle formats. I'd like to add a few more details about that subject. The known and retained sub2video mechanism is used for feeding subtitle frames into the graph: it sends an initial empty frame (unless there's any actual one) and it repeats the most recent subtitle frame (or an empty subtitle frame) at a certain minimum interval into the graph. What's happening inside the graph is a different story and depends on each individual filter's implementation. For an example, let's look at the new textsub2video filter which renders ass subtitles (input) on transparent video frames (output). On the input side, the filter receives subtitle frames containing ass lines ("ass events"), but with libass, there's no relation between the time when its getting fed event lines and the output. These are totally independent: at the input side, you could feed the whole set of lines at once, while at the output side, you are specifying the ecact point in time for which you would like the subtitle images to get rendered. As such, the textsub2video filter cannot work in a way like producing one video frame at the output side for each subtitle text frame at the input side. That could be either too many or also too few frames, depending on the available compute resources, the source material and the desired results: Normally, subtitle text is changing at a rather low frequency, e.g. no more often than once or twice per second. That means that a filter like textsub2video would need to generate a new overlay image no more than once or twice per second. But wait - ass subtitles can also have animations; in that case, every single frame rendered by libass will be different. That will And animations require a much higher frame-rate for smooth appearance. Essentially, this leads to a similar problem like the heartbeat topic. > > > > > - Part3, avfilter support for subtitles in AVFrames. At this > point > > we > > > have a defined structure to store subtitles in AVFrames, and > actual > > > code that can generate or consume them. When approaching this, > the > > > same rules apply as before, existing subtitle functionality, as > > crude > > > as it may be, has to remain functional as exposed to the user. > > Check. > > > We need to decide which aspects of the subtitles formats are > > negotiated. > > > > At least, obviously, the text or bitmap aspect will be, with a > > conversion filter inserted automatically where needed. > > I'm inserting the graphicsub2video filter for keeping compatibility > with sub2video command lines, but I'm not a fan of any other > automatic filter insertion. > Let's talk about this in a separate conversation. > > > > But depending on the answers to the questions in part 1, we may > need > > to > > negotiate the pixel format and colorspace too. > > At current, bitmap subtitles are always PAL8 and that should be > (remain to be) the meaning if SUBTITLE_BITMAP. > > It will be easy to add additional subtitle formats which could > use a different kind of bitmap format. > > > > Unfortunately, the current negotiation code is messy and fragile. > We > > cannot afford to pile new code on top of it. However good the new > > code > > may be, adding it on top of messy code would only make it harder to > > clean up and maintain later. I absolutely oppose that. > > The situation may be messy for audio and video, but for subtitles > format negotiation is really simple (SUBTITLE_BITMAP, SUBTITLE_ASS > or SUBTITLE_TEXT). > > The improvements you are planning can easily be done afterwards > as the subtitle format negotiation really doesn't add any significant > technical debt to the situation. > > > Kind regards, > softworkz > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/12] Subtitle Filtering
> -Original Message- > From: ffmpeg-devel On Behalf Of Soft Works > Sent: Friday, 17 September 2021 06:35 > To: FFmpeg development discussions and patches > Subject: Re: [FFmpeg-devel] [PATCH v5 00/12] Subtitle Filtering > > > > > -Original Message- > > From: ffmpeg-devel On Behalf Of > > Soft Works > > Sent: Thursday, 16 September 2021 19:46 > > To: FFmpeg development discussions and patches > de...@ffmpeg.org> > > Subject: Re: [FFmpeg-devel] [PATCH v5 00/12] Subtitle Filtering > > > > [..] > > > > - Sparseness. Subtitles streams have gaps, and synchronization with > > > other streams requires a next frame, that can be minutes away or > > > never > > > come. This needs to be solved in a way compatible with > > processing. > > > > I have kept the heartbeat logic from your sub2video implementation. > > It makes sense, is required and I can't think of any better way to > > handle this. It's just renamed to subtitle_heartbeat and used for > > all subtitle formats. > > I'd like to add a few more details about that subject. > > The known and retained sub2video mechanism is used for feeding > subtitle frames into the graph: it sends an initial empty frame > (unless there's any actual one) and it repeats the most recent > subtitle frame (or an empty subtitle frame) at a certain minimum > interval into the graph. > > What's happening inside the graph is a different story and depends > on each individual filter's implementation. For an example, let's > look at the new textsub2video filter which renders ass subtitles > (input) on transparent video frames (output). > > On the input side, the filter receives subtitle frames containing > ass lines ("ass events"), but with libass, there's no relation > between the time when its getting fed event lines and the output. > These are totally independent: at the input side, you could > feed the whole set of lines at once, while at the output side, > you are specifying the ecact point in time for which you would > like the subtitle images to get rendered. > > As such, the textsub2video filter cannot work in a way like > producing one video frame at the output side for each subtitle > text frame at the input side. > That could be either too many or also too few frames, depending > on the available compute resources, the source material and > the desired results: > > Normally, subtitle text is changing at a rather low frequency, > e.g. no more often than once or twice per second. That means > that a filter like textsub2video would need to generate a new > overlay image no more than once or twice per second. > > But wait - ass subtitles can also have animations; in that case, > every single frame rendered by libass will be different. > That will And animations > require a much higher frame-rate for smooth appearance. > Sorry - accidentally sent before completion... The new textsub2video filter has a 'framerate' parameter. This allows to control the frequency at which an overlay frame is recreated. For example: ffmpeg -i INPUT -filter_complex "[0:2]textsub2video=r=2[sub];[0:1][sub]overlay=repeatlast=0" output.ts will provide better performance due to overlay frames getting recreated only twice per second while ffmpeg -i INPUT -filter_complex "[0:2]textsub2video=r=25[sub];[0:1][sub]overlay=repeatlast=0" output.ts will provide smooth animation due to frequent recreation of overlay frames. There's no epic conclusion - just another example for the wide range of new possibilities. Regards, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] avfilter/vf_zscale: fix output color_range discrepancy
I doubt this is correct, something is fishy here. On Fri, Sep 17, 2021 at 4:17 AM Pavel Koshevoy wrote: > This filter chain was supposed to convert from narrow range > to full range yuv444p, but didn't: > > buffer=width=1280:height=720:pix_fmt=yuv444p:frame_rate=25/1:\ > time_base=1/25:sar=1/1,zscale=min=709:rin=limited:pin=709:\ > tin=709:t=linear,format=gbrpf32le,zscale=tin=linear:p=709:m=709:\ > r=full:t=709,format=pix_fmts=yuv444p,buffersink > --- > libavfilter/vf_zscale.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/libavfilter/vf_zscale.c b/libavfilter/vf_zscale.c > index dfea00f9eb..06a025e6e6 100644 > --- a/libavfilter/vf_zscale.c > +++ b/libavfilter/vf_zscale.c > @@ -676,7 +676,7 @@ static int filter_frame(AVFilterLink *link, AVFrame > *in) > out->color_primaries = (int)s->dst_format.color_primaries; > > if (s->range != -1) > -out->color_range = (int)s->dst_format.pixel_range; > +out->color_range = (int)s->dst_format.pixel_range + 1; > > if (s->trc != -1) > out->color_trc = (int)s->dst_format.transfer_characteristics; > -- > 2.26.2 > > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 2/2] avcodec/wmaprodec: Check that the EOF frame was allocated before decoding into it
lgtm ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 11/13] avcodec/msvideo1enc: Check all calls to avpriv_elbg_do()
lgtm ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 01/13] avcodec/elbg: Remove avoidable buffer
lgtm ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v1 1/1] avformat/amr: Return PATCHWELCOME on stereo files
Are stereo files available somewhere? Known way to encode? On Fri, Sep 17, 2021 at 4:14 AM Sun Zhenliang wrote: > Ping. > > Anyone could review this patch? > > Thx. > 在 2021年9月16日 +0800 11:24,sunzhenliang ,写道: > > Signed-off-by: sunzhenliang > > --- > > libavformat/amr.c | 22 ++ > > 1 file changed, 18 insertions(+), 4 deletions(-) > > > > diff --git a/libavformat/amr.c b/libavformat/amr.c > > index 836b276fd5..2762010ebe 100644 > > --- a/libavformat/amr.c > > +++ b/libavformat/amr.c > > @@ -36,8 +36,10 @@ typedef struct { > > uint64_t block_count; > > } AMRContext; > > > > -static const char AMR_header[] = "#!AMR\n"; > > -static const char AMRWB_header[] = "#!AMR-WB\n"; > > +static const char AMR_header[] = "#!AMR\n"; > > +static const char AMR_MC_header[] = "#!AMR_MC1.0\n"; > > +static const char AMRWB_header[] = "#!AMR-WB\n"; > > +static const char AMRWB_MC_header[] = "#!AMR-WB_MC1.0\n"; > > > > static const uint8_t amrnb_packed_size[16] = { > > 13, 14, 16, 18, 20, 21, 27, 32, 6, 1, 1, 1, 1, 1, 1, 1 > > @@ -82,7 +84,7 @@ static int amr_read_header(AVFormatContext *s) > > { > > AVIOContext *pb = s->pb; > > AVStream *st; > > - uint8_t header[9]; > > + uint8_t header[15]; > > > > if (avio_read(pb, header, 6) != 6) > > return AVERROR_INVALIDDATA; > > @@ -94,7 +96,19 @@ static int amr_read_header(AVFormatContext *s) > > if (avio_read(pb, header + 6, 3) != 3) > > return AVERROR_INVALIDDATA; > > if (memcmp(header, AMRWB_header, 9)) { > > - return -1; > > + if (avio_read(pb, header + 6 + 3, 3) != 3) > > + return AVERROR_INVALIDDATA; > > + if (memcmp(header, AMR_MC_header, 12)) { > > + if (avio_read(pb, header + 6 + 3 + 3, 3) != 3) > > + return AVERROR_INVALIDDATA; > > + if (memcmp(header, AMRWB_MC_header, 15)) { > > + return -1; > > + } > > + avpriv_report_missing_feature(s, "multi-channel AMRWB"); > > + return AVERROR_PATCHWELCOME; > > + } > > + avpriv_report_missing_feature(s, "multi-channel AMR"); > > + return AVERROR_PATCHWELCOME; > > } > > > > st->codecpar->codec_tag = MKTAG('s', 'a', 'w', 'b'); > > -- > > 2.25.1 > > > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 06/13] avcodec/elbg: Add persistent ELBGContext
lgtm ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 04/13] avfilter/vf_elbg: Rename ELBGContext->ELBGFilterContext
lgtm ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 02/13] avcodec/elbg: Move avpriv_init_elbg() down
lgtm ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 10/13] avcodec/cinepakenc: Check all calls to avpriv_elbg_do()
lgtm ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 05/13] avcodec/elbg: Rename elbg_data to ELBGContext
lgtm ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 12/13] avfilter/vf_elbg: Check call to avpriv_elbg_do()
lgtm ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 09/13] avcodec/elbg: Also allocate buffers for recursion only once
probably fine ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 01/13] avcodec/elbg: Remove avoidable buffer
fre 2021-09-17 klockan 04:02 +0200 skrev Andreas Rheinhardt: > Signed-off-by: Andreas Rheinhardt > --- > libavcodec/elbg.c | 9 +++-- > 1 file changed, 3 insertions(+), 6 deletions(-) > > diff --git a/libavcodec/elbg.c b/libavcodec/elbg.c > index d012d9a384..795fc83f16 100644 > --- a/libavcodec/elbg.c > +++ b/libavcodec/elbg.c > @@ -376,7 +376,6 @@ int avpriv_do_elbg(int *points, int dim, int > numpoints, int *codebook, > elbg_data elbg_d; > elbg_data *elbg = &elbg_d; > int i, j, k, steps = 0, ret = 0; > - int *dist_cb = av_malloc_array(numpoints, sizeof(int)); > int *size_part = av_malloc_array(numCB, sizeof(int)); > cell *list_buffer = av_malloc_array(numpoints, sizeof(cell)); > cell *free_cells; > @@ -394,7 +393,7 @@ int avpriv_do_elbg(int *points, int dim, int > numpoints, int *codebook, > elbg->utility_inc = av_malloc_array(numCB, sizeof(*elbg- > >utility_inc)); > elbg->scratchbuf = av_malloc_array(5*dim, sizeof(int)); > > - if (!dist_cb || !size_part || !list_buffer || !elbg->cells || > + if (!size_part || !list_buffer || !elbg->cells || > !elbg->utility || !elbg->utility_inc || !elbg->scratchbuf) { > ret = AVERROR(ENOMEM); > goto out; > @@ -423,9 +422,8 @@ int avpriv_do_elbg(int *points, int dim, int > numpoints, int *codebook, > } > } > elbg->nearest_cb[i] = best_idx; > - dist_cb[i] = best_dist; > - elbg->error += dist_cb[i]; > - elbg->utility[elbg->nearest_cb[i]] += dist_cb[i]; > + elbg->error += best_dist; > + elbg->utility[elbg->nearest_cb[i]] += best_dist; > free_cells->index = i; > free_cells->next = elbg->cells[elbg->nearest_cb[i]]; > elbg->cells[elbg->nearest_cb[i]] = free_cells; > @@ -453,7 +451,6 @@ int avpriv_do_elbg(int *points, int dim, int > numpoints, int *codebook, > (steps < max_steps)); > > out: > - av_free(dist_cb); > av_free(size_part); > av_free(elbg->utility); > av_free(list_buffer); Nice catch. Looks good to me /Tomas ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 13/13] avcodec/elbg: Add flags to avpriv_elbg_do()
lgtm ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 08/13] avcodec/elbg: Keep buffers to avoid allocations and frees
lgtm ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 07/13] avcodec/elbg: Move arguments to the context early if possible
probably fine ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".