Re: [FFmpeg-devel] IMF demuxer ping
5 Dec 2021, 02:33 by p...@sandflow.com:
> Hi all,
>
> Quick ping re: libavformat/imf demuxer patch set.
>
> All outstanding feedback (thanks!) has been addressed as far as I know.
>
> What are the next steps?
>
> It would be good to make progress while it is fresh in peoples' minds.
>
> Latest patchset at [1] and tracking PR at [2] (including memory leak
> checking):
>
> [1] http://ffmpeg.org/pipermail/ffmpeg-devel/2021-November/288173.html
> [2] https://github.com/sandflow/ffmpeg-imf/pull/74
>
> Looking forward to (finally) adding support for IMF demuxing to ffmpeg.
>
There are still many issues.
Put your custom header beneath the standard ffmpeg header. Or
remove it altogether if you're allowed to.
You're not following our code style at all:
All internal functions (except intra-file functions) must have
an ff_ prefix. If it's a struct, the prefix is FF.
We do not put brackets on one-line statements, unless it's
as a part of an if {} else { } group where one statement has
more than 2 lines.
We're lenient towards 80-char line limits, where if it looks bad
and hampers readability, we let it go on, but the patches aren't
even trying.
It's all here:
https://ffmpeg.org/developer.html#Coding-Rules-1
"rsrc->edit_rate = av_make_q(0, 0);" <- this means that if
something goes wrong, there will be a division by 0. Timebases
must be initialized by av_make_q(0, 1);
Could you try to reduce the total number of allocations per-packet
by allocating all that you can during init-time, even if it's potentially
unused, and use av_fast_realloc for really necessary allocations during
packet parsing? That way, decoding would be quicker.
Squash the first 3 patches (header, processor and demuxer) patches
together, and move the build stuff from the final patch in with the
first, only keeping the Makefile code for building the tests into the 4th.
Otherwise, you can't build them.
Remove the "@author" and "@file" or other special documentation
commands, doxygen doesn't even use them. Use the copyright
field on top.
Since IMF Is meant for intermediate work, it does allow for weird
non-linear streams with loops, skips and such. We do (or attempt
to) do linearization inside the demuxer, due to bad choices, unless
an option is set, like with mp4/mov. So this demuxer should do
that as well.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 4/6] ffmpeg: deprecate passing numbers to -vsync
LGTM ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2] fftools/cmdutils: Avoid crash when opts is empty
Yu Yang:
> Opts is assigned by setup_find_stream_info_opts(). Could not get opts when
> nb_streams == 0.
> It should not return NULL but print AV_LOG_ERROR. when no alloc memory for
> stream options,
> it also need return an error to avoid crash when free. In total,
> setup_find_stream_info_opts()
> should not return NULL. It print AV_LOG_ERROR or correct value.
>
> coredump backtrace info:
> ==6235==ERROR: AddressSanitizer: SEGV on unknown address 0x (pc
> 0x06ba9c2f bp 0x7ffc3d5baa30 sp 0x7ffc3d5ba9a0 T0)
> ==6235==The signal is caused by a READ memory access.
> ==6235==Hint: address points to the zero page.
> #0 0x6ba9c2f in av_dict_free
> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libavutil/dict.c:205:23
> #1 0x4ce5ac in open_input_file
> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/fftools/ffmpeg_opt.c:1199:13
> #2 0x4c9dc0 in open_files
> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/fftools/ffmpeg_opt.c:3338:15
> #3 0x4c9295 in ffmpeg_parse_options
> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/fftools/ffmpeg_opt.c:3378:11
> #4 0x58f241 in main
> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/fftools/ffmpeg.c:4988:11
> #5 0x7fe35197f0b2 in __libc_start_main
> /build/glibc-eX1tMB/glibc-2.31/csu/../csu/libc-start.c:308:16
> #6 0x42033d in _start (/home/r1/ffmpeg/ffmpeg_4.4.1+0x42033d)
>
> Reported-by: TOTE Robot
> Signed-off-by: Yu Yang
> ---
> fftools/cmdutils.c | 6 --
> libavformat/demux.c | 12 +---
> 2 files changed, 9 insertions(+), 9 deletions(-)
>
> diff --git a/fftools/cmdutils.c b/fftools/cmdutils.c
> index 45322f8c71..f4333d8b65 100644
> --- a/fftools/cmdutils.c
> +++ b/fftools/cmdutils.c
> @@ -2182,12 +2182,14 @@ AVDictionary
> **setup_find_stream_info_opts(AVFormatContext *s,
> AVDictionary **opts;
>
> if (!s->nb_streams)
> -return NULL;
> +av_log(NULL, AV_LOG_ERROR,
> + "No stream exists, Could not get stream options.\n");
> +exit_program(1);
This is completely wrong: You are unconditionally exiting the program
(the indentation here is now misleading). Had you run FATE (see
https://ffmpeg.org/fate.html), you would have noticed this.
Exiting the program in case of no streams is wrong, too.
> opts = av_calloc(s->nb_streams, sizeof(*opts));
> if (!opts) {
> av_log(NULL, AV_LOG_ERROR,
> "Could not alloc memory for stream options.\n");
> -return NULL;
> +exit_program(1);
> }
> for (i = 0; i < s->nb_streams; i++)
> opts[i] = filter_codec_opts(codec_opts,
> s->streams[i]->codecpar->codec_id,
> diff --git a/libavformat/demux.c b/libavformat/demux.c
> index 745dc8687c..0738ef2e73 100644
> --- a/libavformat/demux.c
> +++ b/libavformat/demux.c
> @@ -2434,7 +2434,7 @@ int avformat_find_stream_info(AVFormatContext *ic,
> AVDictionary **options)
>
> for (unsigned i = 0; i < ic->nb_streams; i++) {
> const AVCodec *codec;
> -AVDictionary *thread_opt = NULL;
> +
> AVStream *const st = ic->streams[i];
> FFStream *const sti = ffstream(st);
> AVCodecContext *const avctx = sti->avctx;
> @@ -2474,26 +2474,24 @@ int avformat_find_stream_info(AVFormatContext *ic,
> AVDictionary **options)
>
> /* Force thread count to 1 since the H.264 decoder will not extract
> * SPS and PPS to extradata during multi-threaded decoding. */
> -av_dict_set(options ? &options[i] : &thread_opt, "threads", "1", 0);
> +av_dict_set(&options[i], "threads", "1", 0);
> /* Force lowres to 0. The decoder might reduce the video size by the
> * lowres factor, and we don't want that propagated to the stream's
> * codecpar */
> -av_dict_set(options ? &options[i] : &thread_opt, "lowres", "0", 0);
> +av_dict_set(&options[i], "lowres", "0", 0);
>
> if (ic->codec_whitelist)
> -av_dict_set(options ? &options[i] : &thread_opt,
> "codec_whitelist", ic->codec_whitelist, 0);
> +av_dict_set(&options[i], "codec_whitelist", ic->codec_whitelist,
> 0);
>
> // Try to just open decoders, in case this is enough to get
> parameters.
> // Also ensure that subtitle_header is properly set.
> if (!has_codec_parameters(st, NULL) && sti->request_probe <= 0 ||
> st->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE) {
> if (codec && !avctx->codec)
> -if (avcodec_open2(avctx, codec, options ? &options[i] :
> &thread_opt) < 0)
> +if (avcodec_open2(avctx, codec, &options[i]) < 0)
> av_log(ic, AV_LOG_WARNING,
> "Failed to open codec in %s\n",__FUNCTION__);
> }
> -if (!options)
> -av_dict_free(&thread_opt);
> }
>
> read_size = 0;
>
These changes to libavformat are even worse: Even if all the users in
fftools always set options does not mean that all users do; it is n
Re: [FFmpeg-devel] [PATCH 3/3] lavf/sdp: add more thorough error handling
On Sat, Dec 04, 2021 at 06:33:00PM +0100, Anton Khirnov wrote:
> Return error codes when constructing a stream config fails, rather than
> just disregarding the failure and continuing.
> Propagate the error codes from av_sdp_create().
> ---
> libavformat/internal.h | 7 +-
> libavformat/sdp.c | 189 +
> 2 files changed, 120 insertions(+), 76 deletions(-)
>
> diff --git a/libavformat/internal.h b/libavformat/internal.h
> index 528ff7e017..db1d83be17 100644
> --- a/libavformat/internal.h
> +++ b/libavformat/internal.h
> @@ -545,10 +545,11 @@ uint64_t ff_parse_ntp_time(uint64_t ntp_ts);
> * @param ttl the time to live of the stream, 0 if not multicast
> * @param fmt the AVFormatContext, which might contain options modifying
> *the generated SDP
> + * @return 0 on success, a negative error code on failure
> */
> -void ff_sdp_write_media(char *buff, int size, const AVStream *st, int idx,
> -const char *dest_addr, const char *dest_type,
> -int port, int ttl, AVFormatContext *fmt);
> +int ff_sdp_write_media(char *buff, int size, const AVStream *st, int idx,
> + const char *dest_addr, const char *dest_type,
> + int port, int ttl, AVFormatContext *fmt);
>
ff_sdp_write_media() is used by movenc.c also, maybe it's better to add the
error check there also.
> /**
> * Write a packet to another muxer than the one the user originally
> diff --git a/libavformat/sdp.c b/libavformat/sdp.c
> index 1cdd21c97b..a5aba8a80c 100644
> --- a/libavformat/sdp.c
> +++ b/libavformat/sdp.c
> @@ -151,7 +151,8 @@ static int sdp_get_address(char *dest_addr, int size, int
> *ttl, const char *url)
> }
>
> #define MAX_PSET_SIZE 1024
> -static char *extradata2psets(AVFormatContext *s, const AVCodecParameters
> *par)
> +static int extradata2psets(AVFormatContext *s, const AVCodecParameters *par,
> + char **out)
> {
> char *psets, *p;
> const uint8_t *r;
> @@ -162,15 +163,18 @@ static char *extradata2psets(AVFormatContext *s, const
> AVCodecParameters *par)
> uint8_t *tmpbuf = NULL;
> const uint8_t *sps = NULL, *sps_end;
>
> +*out = NULL;
> +
> if (par->extradata_size > MAX_EXTRADATA_SIZE) {
> av_log(s, AV_LOG_ERROR, "Too much extradata!\n");
> -
> -return NULL;
> +return AVERROR_INVALIDDATA;
> }
> if (par->extradata[0] == 1) {
> -if (ff_avc_write_annexb_extradata(par->extradata, &extradata,
> - &extradata_size))
> -return NULL;
> +int ret = ff_avc_write_annexb_extradata(par->extradata, &extradata,
> +&extradata_size);
> +if (ret < 0)
> +return ret;
> +
> tmpbuf = extradata;
> }
>
> @@ -178,7 +182,7 @@ static char *extradata2psets(AVFormatContext *s, const
> AVCodecParameters *par)
> if (!psets) {
> av_log(s, AV_LOG_ERROR, "Cannot allocate memory for the parameter
> sets.\n");
> av_free(tmpbuf);
> -return NULL;
> +return AVERROR(ENOMEM);
> }
> memcpy(psets, pset_string, strlen(pset_string));
> p = psets + strlen(pset_string);
> @@ -203,11 +207,12 @@ static char *extradata2psets(AVFormatContext *s, const
> AVCodecParameters *par)
> sps_end = r1;
> }
> if (!av_base64_encode(p, MAX_PSET_SIZE - (p - psets), r, r1 - r)) {
> -av_log(s, AV_LOG_ERROR, "Cannot Base64-encode
> %"PTRDIFF_SPECIFIER" %"PTRDIFF_SPECIFIER"!\n", MAX_PSET_SIZE - (p - psets),
> r1 - r);
> +av_log(s, AV_LOG_ERROR, "Cannot Base64-encode
> %"PTRDIFF_SPECIFIER" %"PTRDIFF_SPECIFIER"!\n",
> + MAX_PSET_SIZE - (p - psets), r1 - r);
> av_free(psets);
> av_free(tmpbuf);
>
> -return NULL;
> +return AVERROR_INVALIDDATA;
> }
> p += strlen(p);
> r = r1;
> @@ -220,10 +225,11 @@ static char *extradata2psets(AVFormatContext *s, const
> AVCodecParameters *par)
> }
> av_free(tmpbuf);
>
> -return psets;
> +*out = psets;
> +return 0;
> }
>
> -static char *extradata2psets_hevc(const AVCodecParameters *par)
> +static int extradata2psets_hevc(const AVCodecParameters *par, char **out)
> {
> char *psets;
> uint8_t *extradata = par->extradata;
> @@ -232,7 +238,9 @@ static char *extradata2psets_hevc(const AVCodecParameters
> *par)
> int ps_pos[3] = { 0 };
> static const char * const ps_names[3] = { "vps", "sps", "pps" };
> int num_arrays, num_nalus;
> -int pos, i, j;
> +int pos, i, j, ret;
> +
> +*out = NULL;
>
> // Convert to hvcc format. Since we need to group multiple NALUs of
> // the same type, and we might need to convert from one format to the
> @@ -240,9 +248,13 @@ static char *extradata2psets_hevc(co
Re: [FFmpeg-devel] [PATCH] libavcodec/hevc_mp4toannexb_bsf: update the extradata in codec par if change detected
Linjie Fu:
> From: Linjie Fu
>
> Container may support multiple sample descriptions in a single
> bitstream, like multiple stsd in mov, which introduces different
> sequence header(e.g.profile/bit_depth) in the middle of the bitstream.
>
> Update the extradata field in context parameter once packet with
> different extradata is got.
>
> Signed-off-by: Linjie Fu
> ---
> libavcodec/hevc_mp4toannexb_bsf.c | 23 +++
> 1 file changed, 23 insertions(+)
>
> diff --git a/libavcodec/hevc_mp4toannexb_bsf.c
> b/libavcodec/hevc_mp4toannexb_bsf.c
> index 790dfb0394..36a83d0c95 100644
> --- a/libavcodec/hevc_mp4toannexb_bsf.c
> +++ b/libavcodec/hevc_mp4toannexb_bsf.c
> @@ -125,6 +125,9 @@ static int hevc_mp4toannexb_filter(AVBSFContext *ctx,
> AVPacket *out)
> int got_irap = 0;
> int i, ret = 0;
>
> +uint8_t *extradata;
> +size_t extradata_size = 0;
> +
> ret = ff_bsf_get_packet(ctx, &in);
> if (ret < 0)
> return ret;
> @@ -135,6 +138,26 @@ static int hevc_mp4toannexb_filter(AVBSFContext *ctx,
> AVPacket *out)
> return 0;
> }
>
> +extradata = av_packet_get_side_data(in, AV_PKT_DATA_NEW_EXTRADATA,
> &extradata_size);
> +if (extradata && extradata_size &&
> +(ctx->par_in->extradata_size != extradata_size ||
> + memcmp(ctx->par_in->extradata, extradata, extradata_size))) {
> +av_log(ctx, AV_LOG_VERBOSE, "Update extradata, size = %zu\n",
> extradata_size);
We use SIZE_SPECIFIER instead of zu due to MSVC compatibility. I don't
know if we actually support MSVC versions that old any more, but it is
still used.
> +/* Update extradata */
> +av_freep(&ctx->par_in->extradata);
> +ctx->par_in->extradata_size = extradata_size;
> +ctx->par_in->extradata = av_mallocz(extradata_size +
> AV_INPUT_BUFFER_PADDING_SIZE);
> +if (!ctx->par_in->extradata)
> +return AVERROR(ENOMEM);
> +memcpy(ctx->par_in->extradata, extradata, extradata_size);
> +/* Reinit */
> +ret = hevc_extradata_to_annexb(ctx);
> +if (ret < 0)
> +return ret;
> +s->length_size = ret;
> +s->extradata_parsed = 1;
> +}
> +
> bytestream2_init(&gb, in->data, in->size);
>
> while (bytestream2_get_bytes_left(&gb)) {
>
1. This is an API violation: par_in is set by the user before
initializing the BSF, par_out is set by the BSF in init. After init,
both are immutable.
2. Instead you should make hevc_extradata_to_annexb() accept a buffer
with length field (or maybe even factor everything that
hevc_mp4toannexb_init() does out of it and make that function accept a
buffer and a length field; and another field to return the reformatted
extradata and its size).
3. Notice that you need to actually modify the packet's
AV_PKT_DATA_NEW_EXTRADATA side-data to make it Annex B. Otherwise the
output will be incorrect.
4. And in case this BSF actually inserts the extradata in-band, it would
need to retain an internal copy of this new extradata.
(Andriy Gelman once sent a good patch that never got applied that added
the correct extradata in-band even when there are in-band updates to the
extradata. I suggest you look at this patch if you want to tackle this
problem.)
- Andreas
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/4] fftools/cmdutils: Atomically add elements to list of pointers, fix crash
Andreas Rheinhardt:
> Currently, adding a (separately allocated) element to a list of pointers
> works by first reallocating the array of pointers and (on success)
> incrementing its size and only then allocating the new element.
> If the latter allocation fails, the size is inconsistent, i.e.
> array[nb_array_elems - 1] is NULL. Our cleanup code crashes in such
> scenarios.
>
> Fix this by adding an auxiliary function that atomically allocates
> and adds a new element to a list of pointers.
>
> Signed-off-by: Andreas Rheinhardt
> ---
> fftools/cmdutils.c | 16
> fftools/cmdutils.h | 17 +
> fftools/ffmpeg_filter.c | 17 -
> fftools/ffmpeg_opt.c| 22 ++
> 4 files changed, 43 insertions(+), 29 deletions(-)
>
> diff --git a/fftools/cmdutils.c b/fftools/cmdutils.c
> index 45322f8c71..1464b122df 100644
> --- a/fftools/cmdutils.c
> +++ b/fftools/cmdutils.c
> @@ -2214,6 +2214,22 @@ void *grow_array(void *array, int elem_size, int
> *size, int new_size)
> return array;
> }
>
> +void *allocate_array_elem(void *ptr, size_t elem_size, int *nb_elems)
> +{
> +void *new_elem, **array = (void**)ptr;
> +
> +if (*nb_elems == INT_MAX) {
> +av_log(NULL, AV_LOG_ERROR, "Array too big.\n");
> +exit_program(1);
> +}
> +new_elem = av_mallocz(elem_size);
> +if (!new_elem)
> +exit_program(1);
> +GROW_ARRAY(array, *nb_elems);
> +array[*nb_elems - 1] = new_elem;
> +return array;
> +}
> +
> double get_rotation(int32_t *displaymatrix)
> {
> double theta = 0;
> diff --git a/fftools/cmdutils.h b/fftools/cmdutils.h
> index 64dd7266bc..ae78e60f4c 100644
> --- a/fftools/cmdutils.h
> +++ b/fftools/cmdutils.h
> @@ -628,11 +628,28 @@ FILE *get_preset_file(char *filename, size_t
> filename_size,
> */
> void *grow_array(void *array, int elem_size, int *size, int new_size);
>
> +/**
> + * Atomically add a new element to an array of pointers, i.e. allocate
> + * a new entry, reallocate the array of pointers and make the new last
> + * member of this array point to the newly allocated buffer.
> + * Calls exit() on failure.
> + *
> + * @param array array of pointers to reallocate
> + * @param elem_size size of the new element to allocate
> + * @param nb_elems pointer to the number of elements of the array array;
> + * *nb_elems will be incremented by one by this function.
> + * @return reallocated array
> + */
> +void *allocate_array_elem(void *array, size_t elem_size, int *nb_elems);
> +
> #define media_type_string av_get_media_type_string
>
> #define GROW_ARRAY(array, nb_elems)\
> array = grow_array(array, sizeof(*array), &nb_elems, nb_elems + 1)
>
> +#define ALLOC_ARRAY_ELEM(array, nb_elems)\
> +array = allocate_array_elem(array, sizeof(*array[0]), &nb_elems)
> +
> #define GET_PIX_FMT_NAME(pix_fmt)\
> const char *name = av_get_pix_fmt_name(pix_fmt);
>
> diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c
> index dab0f28819..75194fe66e 100644
> --- a/fftools/ffmpeg_filter.c
> +++ b/fftools/ffmpeg_filter.c
> @@ -164,18 +164,14 @@ int init_simple_filtergraph(InputStream *ist,
> OutputStream *ost)
> exit_program(1);
> fg->index = nb_filtergraphs;
>
> -GROW_ARRAY(fg->outputs, fg->nb_outputs);
> -if (!(fg->outputs[0] = av_mallocz(sizeof(*fg->outputs[0]
> -exit_program(1);
> +ALLOC_ARRAY_ELEM(fg->outputs, fg->nb_outputs);
> fg->outputs[0]->ost = ost;
> fg->outputs[0]->graph = fg;
> fg->outputs[0]->format = -1;
>
> ost->filter = fg->outputs[0];
>
> -GROW_ARRAY(fg->inputs, fg->nb_inputs);
> -if (!(fg->inputs[0] = av_mallocz(sizeof(*fg->inputs[0]
> -exit_program(1);
> +ALLOC_ARRAY_ELEM(fg->inputs, fg->nb_inputs);
> fg->inputs[0]->ist = ist;
> fg->inputs[0]->graph = fg;
> fg->inputs[0]->format = -1;
> @@ -280,9 +276,7 @@ static void init_input_filter(FilterGraph *fg,
> AVFilterInOut *in)
> ist->decoding_needed |= DECODING_FOR_FILTER;
> ist->st->discard = AVDISCARD_NONE;
>
> -GROW_ARRAY(fg->inputs, fg->nb_inputs);
> -if (!(fg->inputs[fg->nb_inputs - 1] =
> av_mallocz(sizeof(*fg->inputs[0]
> -exit_program(1);
> +ALLOC_ARRAY_ELEM(fg->inputs, fg->nb_inputs);
> fg->inputs[fg->nb_inputs - 1]->ist = ist;
> fg->inputs[fg->nb_inputs - 1]->graph = fg;
> fg->inputs[fg->nb_inputs - 1]->format = -1;
> @@ -318,10 +312,7 @@ int init_complex_filtergraph(FilterGraph *fg)
> init_input_filter(fg, cur);
>
> for (cur = outputs; cur;) {
> -GROW_ARRAY(fg->outputs, fg->nb_outputs);
> -fg->outputs[fg->nb_outputs - 1] =
> av_mallocz(sizeof(*fg->outputs[0]));
> -if (!fg->outputs[fg->nb_outputs - 1])
> -exit_program(1);
> +ALLOC_ARRAY_ELEM(fg->outputs, fg->nb_outputs);
>
> fg->outputs[fg->nb_outputs - 1]->graph = fg;
> fg->o
Re: [FFmpeg-devel] [PATCH] avformat/smoothstreamingenc: Move buffers to the end of structs
Andreas Rheinhardt:
> This reduces codesize because the offsets of commonly used elements
> are now smaller and thus need less bytes to encode in ptr+offset
> addressing modes (with GCC 11.2 on x64: 0x1b8b -> 0x1a7b).
>
> Signed-off-by: Andreas Rheinhardt
> ---
> libavformat/smoothstreamingenc.c | 8
> 1 file changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/libavformat/smoothstreamingenc.c
> b/libavformat/smoothstreamingenc.c
> index 27b59c299c..ab0920c16f 100644
> --- a/libavformat/smoothstreamingenc.c
> +++ b/libavformat/smoothstreamingenc.c
> @@ -40,17 +40,15 @@
> #include "libavutil/intreadwrite.h"
>
> typedef struct Fragment {
> -char file[1024];
> -char infofile[1024];
> int64_t start_time, duration;
> int n;
> int64_t start_pos, size;
> +char file[1024];
> +char infofile[1024];
> } Fragment;
>
> typedef struct OutputStream {
> AVFormatContext *ctx;
> -char dirname[1024];
> -uint8_t iobuf[32768];
> URLContext *out; // Current output stream where all output is written
> URLContext *out2; // Auxiliary output stream where all output is also
> written
> URLContext *tail_out; // The actual main output stream, if we're
> currently seeked back to write elsewhere
> @@ -64,6 +62,8 @@ typedef struct OutputStream {
> char *private_str;
> int packet_size;
> int audio_tag;
> +char dirname[1024];
> +uint8_t iobuf[32768];
> } OutputStream;
>
> typedef struct SmoothStreamingContext {
>
Will apply this patchset tonight unless there are objections.
- Andreas
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] avformat/takdec: Simplify data->hex conversion
Andreas Rheinhardt:
> Signed-off-by: Andreas Rheinhardt
> ---
> libavformat/takdec.c | 9 -
> 1 file changed, 4 insertions(+), 5 deletions(-)
>
> diff --git a/libavformat/takdec.c b/libavformat/takdec.c
> index 839e3cc781..6bb26683b4 100644
> --- a/libavformat/takdec.c
> +++ b/libavformat/takdec.c
> @@ -109,7 +109,7 @@ static int tak_read_header(AVFormatContext *s)
> break;
> case TAK_METADATA_MD5: {
> uint8_t md5[16];
> -int i;
> +char md5_hex[2 * sizeof(md5) + 1];
>
> if (size != 19)
> return AVERROR_INVALIDDATA;
> @@ -121,10 +121,9 @@ static int tak_read_header(AVFormatContext *s)
> return AVERROR_INVALIDDATA;
> }
>
> -av_log(s, AV_LOG_VERBOSE, "MD5=");
> -for (i = 0; i < 16; i++)
> -av_log(s, AV_LOG_VERBOSE, "%02x", md5[i]);
> -av_log(s, AV_LOG_VERBOSE, "\n");
> +ff_data_to_hex(md5_hex, md5, sizeof(md5), 1);
> +md5_hex[2 * sizeof(md5)] = '\0';
> +av_log(s, AV_LOG_VERBOSE, "MD5=%s\n", md5_hex);
> break;
> }
> case TAK_METADATA_END: {
>
Will apply tonight unless there are objections.
- Andreas
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2] fftools/cmdutils: Avoid crash when opts is empty
> 2021年12月5日 下午6:56，Andreas Rheinhardt 写道：
>
> Yu Yang:
>> Opts is assigned by setup_find_stream_info_opts(). Could not get opts when
>> nb_streams == 0.
>> It should not return NULL but print AV_LOG_ERROR. when no alloc memory for
>> stream options,
>> it also need return an error to avoid crash when free. In total,
>> setup_find_stream_info_opts()
>> should not return NULL. It print AV_LOG_ERROR or correct value.
>>
>> coredump backtrace info:
>> ==6235==ERROR: AddressSanitizer: SEGV on unknown address 0x (pc
>> 0x06ba9c2f bp 0x7ffc3d5baa30 sp 0x7ffc3d5ba9a0 T0)
>> ==6235==The signal is caused by a READ memory access.
>> ==6235==Hint: address points to the zero page.
>>#0 0x6ba9c2f in av_dict_free
>> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libavutil/dict.c:205:23
>>#1 0x4ce5ac in open_input_file
>> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/fftools/ffmpeg_opt.c:1199:13
>>#2 0x4c9dc0 in open_files
>> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/fftools/ffmpeg_opt.c:3338:15
>>#3 0x4c9295 in ffmpeg_parse_options
>> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/fftools/ffmpeg_opt.c:3378:11
>>#4 0x58f241 in main
>> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/fftools/ffmpeg.c:4988:11
>>#5 0x7fe35197f0b2 in __libc_start_main
>> /build/glibc-eX1tMB/glibc-2.31/csu/../csu/libc-start.c:308:16
>>#6 0x42033d in _start (/home/r1/ffmpeg/ffmpeg_4.4.1+0x42033d)
>>
>> Reported-by: TOTE Robot
>> Signed-off-by: Yu Yang
>> ---
>> fftools/cmdutils.c | 6 --
>> libavformat/demux.c | 12 +---
>> 2 files changed, 9 insertions(+), 9 deletions(-)
>>
>> diff --git a/fftools/cmdutils.c b/fftools/cmdutils.c
>> index 45322f8c71..f4333d8b65 100644
>> --- a/fftools/cmdutils.c
>> +++ b/fftools/cmdutils.c
>> @@ -2182,12 +2182,14 @@ AVDictionary
>> **setup_find_stream_info_opts(AVFormatContext *s,
>> AVDictionary **opts;
>>
>> if (!s->nb_streams)
>> -return NULL;
>> +av_log(NULL, AV_LOG_ERROR,
>> + "No stream exists, Could not get stream options.\n");
>> +exit_program(1);
>
> This is completely wrong: You are unconditionally exiting the program
> (the indentation here is now misleading). Had you run FATE (see
> https://ffmpeg.org/fate.html), you would have noticed this.
I forgot to run 'make fate'.
> Exiting the program in case of no streams is wrong, too.
OK, got it.
>
>> opts = av_calloc(s->nb_streams, sizeof(*opts));
>> if (!opts) {
>> av_log(NULL, AV_LOG_ERROR,
>>"Could not alloc memory for stream options.\n");
>> -return NULL;
>> +exit_program(1);
>> }
>> for (i = 0; i < s->nb_streams; i++)
>> opts[i] = filter_codec_opts(codec_opts,
>> s->streams[i]->codecpar->codec_id,
>> diff --git a/libavformat/demux.c b/libavformat/demux.c
>> index 745dc8687c..0738ef2e73 100644
>> --- a/libavformat/demux.c
>> +++ b/libavformat/demux.c
>> @@ -2434,7 +2434,7 @@ int avformat_find_stream_info(AVFormatContext *ic,
>> AVDictionary **options)
>>
>> for (unsigned i = 0; i < ic->nb_streams; i++) {
>> const AVCodec *codec;
>> -AVDictionary *thread_opt = NULL;
>> +
>> AVStream *const st = ic->streams[i];
>> FFStream *const sti = ffstream(st);
>> AVCodecContext *const avctx = sti->avctx;
>> @@ -2474,26 +2474,24 @@ int avformat_find_stream_info(AVFormatContext *ic,
>> AVDictionary **options)
>>
>> /* Force thread count to 1 since the H.264 decoder will not extract
>> * SPS and PPS to extradata during multi-threaded decoding. */
>> -av_dict_set(options ? &options[i] : &thread_opt, "threads", "1", 0);
>> +av_dict_set(&options[i], "threads", "1", 0);
>> /* Force lowres to 0. The decoder might reduce the video size by the
>> * lowres factor, and we don't want that propagated to the stream's
>> * codecpar */
>> -av_dict_set(options ? &options[i] : &thread_opt, "lowres", "0", 0);
>> +av_dict_set(&options[i], "lowres", "0", 0);
>>
>> if (ic->codec_whitelist)
>> -av_dict_set(options ? &options[i] : &thread_opt,
>> "codec_whitelist", ic->codec_whitelist, 0);
>> +av_dict_set(&options[i], "codec_whitelist",
>> ic->codec_whitelist, 0);
>>
>> // Try to just open decoders, in case this is enough to get
>> parameters.
>> // Also ensure that subtitle_header is properly set.
>> if (!has_codec_parameters(st, NULL) && sti->request_probe <= 0 ||
>> st->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE) {
>> if (codec && !avctx->codec)
>> -if (avcodec_open2(avctx, codec, options ? &options[i] :
>> &thread_opt) < 0)
>> +if (avcodec_open2(avctx, codec, &options[i]) < 0)
>> av_log(ic, AV_LOG_WARNING,
>>"Failed to open codec in %s\n",__FUNCTION__);
>> }
>> -if (!options)
>> -av_dict_free(&thread_opt);
>> }
>
[FFmpeg-devel] [PATCH 1/6] avformat/rtsp: remove redundant assignment
From: Limin Wang Signed-off-by: Limin Wang --- libavformat/rtsp.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libavformat/rtsp.c b/libavformat/rtsp.c index deaed34..47120fd 100644 --- a/libavformat/rtsp.c +++ b/libavformat/rtsp.c @@ -1178,8 +1178,8 @@ int ff_rtsp_read_reply(AVFormatContext *s, RTSPMessageHeader *reply, char buf[MAX_URL_SIZE], buf1[MAX_URL_SIZE], *q; unsigned char ch; const char *p; -int ret, content_length, line_count = 0, request = 0; -unsigned char *content = NULL; +int ret, content_length, line_count, request; +unsigned char *content; start: line_count = 0; -- 1.8.3.1 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 2/6] avformat/rtsp: free the alloc memory if failed
From: Limin Wang
Signed-off-by: Limin Wang
---
libavformat/rtsp.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/libavformat/rtsp.c b/libavformat/rtsp.c
index 47120fd..5cffe0b 100644
--- a/libavformat/rtsp.c
+++ b/libavformat/rtsp.c
@@ -1245,8 +1245,10 @@ start:
content = av_malloc(content_length + 1);
if (!content)
return AVERROR(ENOMEM);
-if (ffurl_read_complete(rt->rtsp_hd, content, content_length) !=
content_length)
+if (ffurl_read_complete(rt->rtsp_hd, content, content_length) !=
content_length) {
+av_freep(&content);
return AVERROR(EIO);
+}
content[content_length] = '\0';
}
if (content_ptr)
--
1.8.3.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 3/6] avformat/rtsp: prefer to return EOF for incomplete read
From: Limin Wang
Signed-off-by: Limin Wang
---
libavformat/rtsp.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/libavformat/rtsp.c b/libavformat/rtsp.c
index 5cffe0b..2ee2463 100644
--- a/libavformat/rtsp.c
+++ b/libavformat/rtsp.c
@@ -1247,7 +1247,7 @@ start:
return AVERROR(ENOMEM);
if (ffurl_read_complete(rt->rtsp_hd, content, content_length) !=
content_length) {
av_freep(&content);
-return AVERROR(EIO);
+return AVERROR_EOF;
}
content[content_length] = '\0';
}
--
1.8.3.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 4/6] avformat/rtsp: check content_ptr before memory allocate
From: Limin Wang
Signed-off-by: Limin Wang
---
libavformat/rtsp.c | 7 ++-
1 file changed, 2 insertions(+), 5 deletions(-)
diff --git a/libavformat/rtsp.c b/libavformat/rtsp.c
index 2ee2463..233ed16 100644
--- a/libavformat/rtsp.c
+++ b/libavformat/rtsp.c
@@ -1240,7 +1240,7 @@ start:
av_strlcpy(rt->session_id, reply->session_id, sizeof(rt->session_id));
content_length = reply->content_length;
-if (content_length > 0) {
+if (content_ptr && content_length > 0) {
/* leave some room for a trailing '\0' (useful for simple parsing) */
content = av_malloc(content_length + 1);
if (!content)
@@ -1250,11 +1250,8 @@ start:
return AVERROR_EOF;
}
content[content_length] = '\0';
-}
-if (content_ptr)
*content_ptr = content;
-else
-av_freep(&content);
+}
if (request) {
char buf[MAX_URL_SIZE];
--
1.8.3.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 5/6] avformat/rtsp: use MAX_URL_SIZE for one line parse
From: Limin Wang
The buf is used for one line of sdp parsing, so it's ok to use MAX_URL_SIZE
Signed-off-by: Limin Wang
---
libavformat/rtsp.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/libavformat/rtsp.c b/libavformat/rtsp.c
index 233ed16..a8d079e 100644
--- a/libavformat/rtsp.c
+++ b/libavformat/rtsp.c
@@ -709,7 +709,7 @@ int ff_sdp_parse(AVFormatContext *s, const char *content)
{
const char *p;
int letter, i;
-char buf[SDP_MAX_SIZE], *q;
+char buf[MAX_URL_SIZE], *q;
SDPParseState sdp_parse_state = { { 0 } }, *s1 = &sdp_parse_state;
p = content;
--
1.8.3.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 6/6] avformat/rtsp: add error code handling for ff_rtsp_skip_packet()
From: Limin Wang
Signed-off-by: Limin Wang
---
libavformat/rtsp.c| 15 ++-
libavformat/rtsp.h| 4 +++-
libavformat/rtspenc.c | 7 +--
3 files changed, 18 insertions(+), 8 deletions(-)
diff --git a/libavformat/rtsp.c b/libavformat/rtsp.c
index a8d079e..6442e5b 100644
--- a/libavformat/rtsp.c
+++ b/libavformat/rtsp.c
@@ -1145,7 +1145,7 @@ void ff_rtsp_parse_line(AVFormatContext *s,
}
/* skip a RTP/TCP interleaved packet */
-void ff_rtsp_skip_packet(AVFormatContext *s)
+int ff_rtsp_skip_packet(AVFormatContext *s)
{
RTSPState *rt = s->priv_data;
int ret, len, len1;
@@ -1153,7 +1153,7 @@ void ff_rtsp_skip_packet(AVFormatContext *s)
ret = ffurl_read_complete(rt->rtsp_hd, buf, 3);
if (ret != 3)
-return;
+return AVERROR_EOF;
len = AV_RB16(buf + 1);
av_log(s, AV_LOG_TRACE, "skipping RTP packet len=%d\n", len);
@@ -1165,9 +1165,11 @@ void ff_rtsp_skip_packet(AVFormatContext *s)
len1 = sizeof(buf);
ret = ffurl_read_complete(rt->rtsp_hd, buf, len1);
if (ret != len1)
-return;
+return AVERROR_EOF;
len -= len1;
}
+
+return 0;
}
int ff_rtsp_read_reply(AVFormatContext *s, RTSPMessageHeader *reply,
@@ -1201,8 +1203,11 @@ start:
if (ch == '$' && q == buf) {
if (return_on_interleaved_data) {
return 1;
-} else
-ff_rtsp_skip_packet(s);
+} else {
+ret = ff_rtsp_skip_packet(s);
+if (ret < 0)
+return ret;
+}
} else if (ch != '\r') {
if ((q - buf) < sizeof(buf) - 1)
*q++ = ch;
diff --git a/libavformat/rtsp.h b/libavformat/rtsp.h
index d6fdfe0..4a413f0 100644
--- a/libavformat/rtsp.h
+++ b/libavformat/rtsp.h
@@ -560,8 +560,10 @@ int ff_rtsp_read_reply(AVFormatContext *s,
RTSPMessageHeader *reply,
/**
* Skip a RTP/TCP interleaved packet.
+ *
+ * @return 0 on success, < 0 on failure.
*/
-void ff_rtsp_skip_packet(AVFormatContext *s);
+int ff_rtsp_skip_packet(AVFormatContext *s);
/**
* Connect to the RTSP server and set up the individual media streams.
diff --git a/libavformat/rtspenc.c b/libavformat/rtspenc.c
index e0fed1e..2a00b3e 100644
--- a/libavformat/rtspenc.c
+++ b/libavformat/rtspenc.c
@@ -200,8 +200,11 @@ static int rtsp_write_packet(AVFormatContext *s, AVPacket
*pkt)
ret = ff_rtsp_read_reply(s, &reply, NULL, 1, NULL);
if (ret < 0)
return AVERROR(EPIPE);
-if (ret == 1)
-ff_rtsp_skip_packet(s);
+if (ret == 1) {
+ret = ff_rtsp_skip_packet(s);
+if (ret < 0)
+return ret;
+}
/* XXX: parse message */
if (rt->state != RTSP_STATE_STREAMING)
return AVERROR(EPIPE);
--
1.8.3.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v8 4/5] avformat/movenc: Refactor mov_write_dvcc_dvvc_tag to use ff_isom_put_dvcc_dvvc
quietvoid:
> On Fri, Dec 3, 2021 at 8:40 PM wrote:
>
>> On Fri, Dec 03, 2021 at 08:27:06PM -0500, quietvoid wrote:
>>> On Fri, Dec 3, 2021 at 8:19 PM wrote:
>>>
On Sat, Dec 04, 2021 at 02:09:04AM +0100, quietvoid wrote:
> From: quietvoid
>
> Improves code legibility by not using bit shifts.
> Also avoids duplicating the dvcC/dvvC ISOM box writing code.
>
> Signed-off-by: quietvoid
> ---
> libavformat/movenc.c | 26 +-
> 1 file changed, 9 insertions(+), 17 deletions(-)
>
> diff --git a/libavformat/movenc.c b/libavformat/movenc.c
> index 38ff90833a..79f7d70747 100644
> --- a/libavformat/movenc.c
> +++ b/libavformat/movenc.c
> @@ -27,6 +27,7 @@
> #include "movenc.h"
> #include "avformat.h"
> #include "avio_internal.h"
> +#include "dovi_isom.h"
> #include "riff.h"
> #include "avio.h"
> #include "isom.h"
> @@ -1911,6 +1912,9 @@ static int mov_write_sv3d_tag(AVFormatContext
>> *s,
AVIOContext *pb, AVSphericalMa
>
> static int mov_write_dvcc_dvvc_tag(AVFormatContext *s, AVIOContext
>> *pb,
AVDOVIDecoderConfigurationRecord *dovi)
> {
> +uint8_t buf[ISOM_DVCC_DVVC_SIZE];
> +int size;
> +
> avio_wb32(pb, 32); /* size = 8 + 24 */
> if (dovi->dv_profile > 10)
> ffio_wfourcc(pb, "dvwC");
> @@ -1918,23 +1922,11 @@ static int
mov_write_dvcc_dvvc_tag(AVFormatContext *s, AVIOContext *pb, AVDOVIDe
> ffio_wfourcc(pb, "dvvC");
> else
> ffio_wfourcc(pb, "dvcC");
> -avio_w8(pb, dovi->dv_version_major);
> -avio_w8(pb, dovi->dv_version_minor);
> -avio_wb16(pb, (dovi->dv_profile << 9) | (dovi->dv_level << 3) |
> - (dovi->rpu_present_flag << 2) |
>> (dovi->el_present_flag <<
1) |
> - dovi->bl_present_flag);
> -avio_wb32(pb, (dovi->dv_bl_signal_compatibility_id << 28) | 0);
> -
> -ffio_fill(pb, 0, 4 * 4); /* reserved */
> -av_log(s, AV_LOG_DEBUG, "DOVI in %s box, version: %d.%d,
>> profile:
%d, level: %d, "
> - "rpu flag: %d, el flag: %d, bl flag: %d, compatibility
>> id:
%d\n",
> - dovi->dv_profile > 10 ? "dvwC" : (dovi->dv_profile > 7 ?
"dvvC" : "dvcC"),
> - dovi->dv_version_major, dovi->dv_version_minor,
> - dovi->dv_profile, dovi->dv_level,
> - dovi->rpu_present_flag,
> - dovi->el_present_flag,
> - dovi->bl_present_flag,
> - dovi->dv_bl_signal_compatibility_id);
> +
> +size = ff_isom_put_dvcc_dvvc(s, buf, sizeof(buf), dovi);
I think you need check the return of ff_isom_put_dvcc_dvvc().
>>> Wouldn't it necessarily return the constant ISOM_DVCC_DVVC_SIZE in this
>>> case?
>>
>> yes, now it'll return negative error code in one error case. Why the out
>> parameter is
>> using array instead of pointer, as the size of buffer is one of parameters
>> also.
>> So if it's array, why it's necessary to check the size?
>>
>
> External users of the function can pass either an array or pointer.
This function is libavformat-only; there are no external users.
> The size addition was suggested here:
> https://ffmpeg.org/pipermail/ffmpeg-devel/2021-September/285877.html
I actually suggested this to you so that you can remove the check from
the function.
>
> So in the case a pointer is used, the size is necessary.
>
>> Since the only error condition is if the passed size is lower than that,
>>> but the buffer is constant sized.
>>>
>>> And the PutBitContext, being flushed, would return the same value.
>>>
>>>
> +
> +avio_write(pb, buf, size);
> +
> return 32; /* 8 + 24 */
> }
>
> --
> 2.34.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v19 01/20] avcodec, avutil: Move enum AVSubtitleType to avutil, add new and deprecate old values
On Fri, Dec 03, 2021 at 07:31:06PM +, Soft Works wrote: > Signed-off-by: softworkz > --- > doc/APIchanges | 6 > libavcodec/avcodec.h | 19 + > libavutil/Makefile | 1 + > libavutil/subfmt.h | 66 > libavutil/version.h | 1 + > 5 files changed, 75 insertions(+), 18 deletions(-) > create mode 100644 libavutil/subfmt.h Some of the patches in this patchset do not apply cleanly to master anymore Applying: avcodec, avutil: Move enum AVSubtitleType to avutil, add new and deprecate old values Using index info to reconstruct a base tree... M doc/APIchanges M libavutil/version.h Falling back to patching base and 3-way merge... Auto-merging libavutil/version.h Auto-merging doc/APIchanges CONFLICT (content): Merge conflict in doc/APIchanges error: Failed to merge in the changes. Patch failed at 0001 avcodec, avutil: Move enum AVSubtitleType to avutil, add new and deprecate old values Use 'git am --show-current-patch' to see the failed patch When you have resolved this problem, run "git am --continue". If you prefer to skip this patch, run "git am --skip" instead. To restore the original branch and stop patching, run "git am --abort". [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB In fact, the RIAA has been known to suggest that students drop out of college or go to community college in order to be able to afford settlements. -- The RIAA signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 1/2] fftools/ffmpeg_opt: Improve checks for truncation/alloc error
Do this by switching from the dynamic buffer API to the AVBPrint API;
the former has no defined way to check for errors.
This also avoids allocating an AVIOContext.
Signed-off-by: Andreas Rheinhardt
---
fftools/ffmpeg_opt.c | 22 ++
1 file changed, 10 insertions(+), 12 deletions(-)
diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c
index 6c2eb53290..78b5574a3d 100644
--- a/fftools/ffmpeg_opt.c
+++ b/fftools/ffmpeg_opt.c
@@ -34,6 +34,7 @@
#include "libavutil/avassert.h"
#include "libavutil/avstring.h"
#include "libavutil/avutil.h"
+#include "libavutil/bprint.h"
#include "libavutil/channel_layout.h"
#include "libavutil/intreadwrite.h"
#include "libavutil/fifo.h"
@@ -1649,29 +1650,26 @@ static void parse_matrix_coeffs(uint16_t *dest, const
char *str)
}
/* read file contents into a string */
-static uint8_t *read_file(const char *filename)
+static char *read_file(const char *filename)
{
AVIOContext *pb = NULL;
-AVIOContext *dyn_buf = NULL;
int ret = avio_open(&pb, filename, AVIO_FLAG_READ);
-uint8_t buf[1024], *str;
+AVBPrint bprint;
+char *str;
if (ret < 0) {
av_log(NULL, AV_LOG_ERROR, "Error opening file %s.\n", filename);
return NULL;
}
-ret = avio_open_dyn_buf(&dyn_buf);
+av_bprint_init(&bprint, 0, AV_BPRINT_SIZE_UNLIMITED);
+ret = avio_read_to_bprint(pb, &bprint, SIZE_MAX);
+avio_closep(&pb);
if (ret < 0) {
-avio_closep(&pb);
+av_bprint_finalize(&bprint, NULL);
return NULL;
}
-while ((ret = avio_read(pb, buf, sizeof(buf))) > 0)
-avio_write(dyn_buf, buf, ret);
-avio_w8(dyn_buf, 0);
-avio_closep(&pb);
-
-ret = avio_close_dyn_buf(dyn_buf, &str);
+ret = av_bprint_finalize(&bprint, &str);
if (ret < 0)
return NULL;
return str;
@@ -3279,7 +3277,7 @@ static int opt_filter_complex(void *optctx, const char
*opt, const char *arg)
static int opt_filter_complex_script(void *optctx, const char *opt, const char
*arg)
{
-uint8_t *graph_desc = read_file(arg);
+char *graph_desc = read_file(arg);
if (!graph_desc)
return AVERROR(EINVAL);
--
2.32.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 2/2] fftools/ffmpeg_opt: Improve alloc/truncation checks when reading lines
Do this by switching from the dynamic buffer API to the AVBPrint API;
the former has no defined way to check for errors.
This also avoids allocating an AVIOContext.
Signed-off-by: Andreas Rheinhardt
---
fftools/ffmpeg_opt.c | 28
1 file changed, 12 insertions(+), 16 deletions(-)
diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c
index 78b5574a3d..66e5afdf80 100644
--- a/fftools/ffmpeg_opt.c
+++ b/fftools/ffmpeg_opt.c
@@ -1356,23 +1356,18 @@ static int open_input_file(OptionsContext *o, const
char *filename)
return 0;
}
-static uint8_t *get_line(AVIOContext *s)
+static char *get_line(AVIOContext *s, AVBPrint *bprint)
{
-AVIOContext *line;
-uint8_t *buf;
char c;
-if (avio_open_dyn_buf(&line) < 0) {
+while ((c = avio_r8(s)) && c != '\n')
+av_bprint_chars(bprint, c, 1);
+
+if (!av_bprint_is_complete(bprint)) {
av_log(NULL, AV_LOG_FATAL, "Could not alloc buffer for reading
preset.\n");
exit_program(1);
}
-
-while ((c = avio_r8(s)) && c != '\n')
-avio_w8(line, c);
-avio_w8(line, 0);
-avio_close_dyn_buf(line, &buf);
-
-return buf;
+return bprint->str;
}
static int get_preset_file_2(const char *preset_name, const char *codec_name,
AVIOContext **s)
@@ -1503,20 +1498,21 @@ static OutputStream *new_output_stream(OptionsContext
*o, AVFormatContext *oc, e
ost->autoscale = 1;
MATCH_PER_STREAM_OPT(autoscale, i, ost->autoscale, oc, st);
if (preset && (!(ret = get_preset_file_2(preset, ost->enc->name,
&s {
+AVBPrint bprint;
+av_bprint_init(&bprint, 0, AV_BPRINT_SIZE_UNLIMITED);
do {
-buf = get_line(s);
-if (!buf[0] || buf[0] == '#') {
-av_free(buf);
+av_bprint_clear(&bprint);
+buf = get_line(s, &bprint);
+if (!buf[0] || buf[0] == '#')
continue;
-}
if (!(arg = strchr(buf, '='))) {
av_log(NULL, AV_LOG_FATAL, "Invalid line found in the
preset file.\n");
exit_program(1);
}
*arg++ = 0;
av_dict_set(&ost->encoder_opts, buf, arg,
AV_DICT_DONT_OVERWRITE);
-av_free(buf);
} while (!s->eof_reached);
+av_bprint_finalize(&bprint, NULL);
avio_closep(&s);
}
if (ret) {
--
2.32.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v20 00/20] Subtitle Filtering
New in V20 - Rebased. V19 didn't apply cleanly anymore New in V19 - Document API changes in all relevant commits - Move enum AVSubtitleType: Improved commit message - Put deprecated enum values under removal guards - Document AV_SUBTITLE_FMT_NB - Fixed all trailing whitespace - splitcc: Add scatter_realtime_output option - splitcc: Emit empty subtitle frames instead of repeating - New commit: Replace deprecated enum values New in V18 - rebased to latest head (to fix merge conflict in allfilters.c) - textmod: Removed two trailing blanks from sf_textmod.c - xsubdec: fix xsubdec regression found by Michael; packet size estimation as flawed - splitcc: Fix uninitialzed locals - splitcc: Do not propagate hwcontext on secondary output - testsub2video: fix output timeing, re-add mutex locking and implement render_latest_only parameter New in V17 - Applied almost all suggestions that were made (thanks everybody!) - Split the initial commit, no longer moving the legacy structs (AVSubtitle..) - Use the regular encoding API for subtitles - Updated subtitle encoders to handle packet allocation - Removed compatibility layer for accessing legacy encoders - Updated the compatibility implementation for the legacy subtitle encode api - Reordered commits and merged ffmpeg.c changes into a single commit New in V16 - Added missing reference to subfmt.h iun libzvbi-teletextdec.c - Fixed Fate error in patch 15/16 - Removed all trsiling whitespace in tilers.texi New in V15 - Rebased to upstream changes - avcodec/subtitles: Migrate subtitle encoders to frame-based API and provide compatibility shim for legacy api - fftools/ffmpeg: Use new frame-based subtitle encoding API - AVSubtitleArea: copy flags field, make params const - graphicsubs2text: Don't emit duplicate frames - graphicsubs2text: Combined OCR output into a single AVSubtitleArea (I have a prototype for detecting text colors and positions, but it's not ready at this point) - splitcc: cleanup local subtitle_header ref - stripstyles: add parameter for ass layer selection - avcodec/subtitles: deferred loading of ass header for text subtitle encoders - verified all example command lines in the docs are working, added somre more Kind regards, softworkz softworkz (20): avcodec,avutil: Move enum AVSubtitleType to avutil, add new and deprecate old values avutil/frame: Prepare AVFrame for subtitle handling avcodec/subtitles: Introduce new frame-based subtitle decoding API avfilter/subtitles: Update vf_subtitles to use new decoding api avcodec,avutil: Move ass helper functions to avutil as avpriv_ and extend ass dialog parsing avcodec/subtitles: Migrate subtitle encoders to frame-based API and provide a compatibility shim for the legacy api avcodec/subtitles: Replace deprecated enum values fftools/play,probe: Adjust for subtitle changes avfilter/subtitles: Add subtitles.c for subtitle frame allocation avfilter/avfilter: Handle subtitle frames avfilter/sbuffer: Add sbuffersrc and sbuffersink filters avfilter/overlaygraphicsubs: Add overlaygraphicsubs and graphicsub2video filters fftools/ffmpeg: Replace sub2video with subtitle frame filtering and use new frame-based subtitle encoding API avfilter/avfilter: Fix hardcoded input index avfilter/overlaytextsubs: Add overlaytextsubs and textsubs2video filters avfilter/textmod: Add textmod, censor and show_speaker filters avfilter/stripstyles: Add stripstyles filter avfilter/splitcc: Add splitcc filter for closed caption handling avfilter/graphicsub2text: Add new graphicsub2text filter (OCR) avfilter/subscale: Add filter for scaling and/or re-arranging graphical subtitles configure | 7 +- doc/APIchanges| 29 + doc/filters.texi | 756 +++ fftools/ffmpeg.c | 584 ++-- fftools/ffmpeg.h | 15 +- fftools/ffmpeg_filter.c | 217 +++-- fftools/ffmpeg_hw.c | 2 +- fftools/ffmpeg_opt.c | 3 +- fftools/ffplay.c | 102 +- fftools/ffprobe.c | 48 +- libavcodec/Makefile | 56 +- libavcodec/ass.h | 147 +-- libavcodec/assdec.c | 4 +- libavcodec/assenc.c | 92 +- libavcodec/avcodec.h | 32 +- libavcodec/ccaption_dec.c | 19 +- libavcodec/codec_desc.c | 11 + libavcodec/codec_desc.h | 8 + libavcodec/decode.c | 54 +- libavcodec/dvbsubdec.c| 2 +- libavcodec/dvbsubenc.c| 96 +- libavcodec/dvdsubdec.c| 2 +- libavcodec/dvdsubenc.c
[FFmpeg-devel] [PATCH v20 01/20] avcodec, avutil: Move enum AVSubtitleType to avutil, add new and deprecate old values
Signed-off-by: softworkz
---
doc/APIchanges | 6
libavcodec/avcodec.h | 19 +
libavutil/Makefile | 1 +
libavutil/subfmt.h | 66
libavutil/version.h | 1 +
5 files changed, 75 insertions(+), 18 deletions(-)
create mode 100644 libavutil/subfmt.h
diff --git a/doc/APIchanges b/doc/APIchanges
index 2914ad6734..a466a2ec22 100644
--- a/doc/APIchanges
+++ b/doc/APIchanges
@@ -14,6 +14,12 @@ libavutil: 2021-04-27
API changes, most recent first:
+2021-12-05 - xx - lavu 58.1.100 - subfmt.h
+ Add enum AVSubtitleType (moved from lavc), add new values, deprecate existing
+
+2021-12-05 - xx - lavc 60.1.100 - avcodec.h
+ Remove enum AVSubtitleType (moved to lavu)
+
2021-11-xx - xx - lavfi 8.19.100 - avfilter.h
Add AVFILTER_FLAG_METADATA_ONLY.
diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 7ee8bc2b7c..b05c12e47e 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -35,6 +35,7 @@
#include "libavutil/frame.h"
#include "libavutil/log.h"
#include "libavutil/pixfmt.h"
+#include "libavutil/subfmt.h"
#include "libavutil/rational.h"
#include "codec.h"
@@ -2238,24 +2239,6 @@ typedef struct AVHWAccel {
* @}
*/
-enum AVSubtitleType {
-SUBTITLE_NONE,
-
-SUBTITLE_BITMAP,///< A bitmap, pict will be set
-
-/**
- * Plain text, the text field must be set by the decoder and is
- * authoritative. ass and pict fields may contain approximations.
- */
-SUBTITLE_TEXT,
-
-/**
- * Formatted text, the ass field must be set by the decoder and is
- * authoritative. pict and text fields may contain approximations.
- */
-SUBTITLE_ASS,
-};
-
#define AV_SUBTITLE_FLAG_FORCED 0x0001
typedef struct AVSubtitleRect {
diff --git a/libavutil/Makefile b/libavutil/Makefile
index 529046dbc8..c7843db1e4 100644
--- a/libavutil/Makefile
+++ b/libavutil/Makefile
@@ -74,6 +74,7 @@ HEADERS = adler32.h
\
sha512.h \
spherical.h \
stereo3d.h\
+ subfmt.h \
threadmessage.h \
time.h\
timecode.h\
diff --git a/libavutil/subfmt.h b/libavutil/subfmt.h
new file mode 100644
index 00..88078a0d14
--- /dev/null
+++ b/libavutil/subfmt.h
@@ -0,0 +1,66 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#ifndef AVUTIL_SUBFMT_H
+#define AVUTIL_SUBFMT_H
+
+enum AVSubtitleType {
+
+/**
+ * Subtitle format unknown.
+ */
+AV_SUBTITLE_FMT_NONE = -1,
+
+/**
+ * Subtitle format unknown.
+ */
+AV_SUBTITLE_FMT_UNKNOWN = 0,
+#ifdef FF_API_OLD_SUBTITLES
+SUBTITLE_NONE = 0, ///< Deprecated, use AV_SUBTITLE_FMT_NONE
instead.
+#endif
+
+/**
+ * Bitmap area in AVSubtitleRect.data, pixfmt AV_PIX_FMT_PAL8.
+ */
+AV_SUBTITLE_FMT_BITMAP = 1,
+#ifdef FF_API_OLD_SUBTITLES
+SUBTITLE_BITMAP = 1,///< Deprecated, use AV_SUBTITLE_FMT_BITMAP
instead.
+#endif
+
+/**
+ * Plain text in AVSubtitleRect.text.
+ */
+AV_SUBTITLE_FMT_TEXT = 2,
+#ifdef FF_API_OLD_SUBTITLES
+SUBTITLE_TEXT = 2, ///< Deprecated, use AV_SUBTITLE_FMT_TEXT
instead.
+#endif
+
+/**
+ * Text Formatted as per ASS specification, contained AVSubtitleRect.ass.
+ */
+AV_SUBTITLE_FMT_ASS = 3,
+#ifdef FF_API_OLD_SUBTITLES
+SUBTITLE_ASS = 3, ///< Deprecated, use AV_SUBTITLE_FMT_ASS
instead.
+#endif
+
+AV_SUBTITLE_FMT_NB, ///< number of subtitle formats, DO NOT USE
THIS if you want to link with shared libav* because the number of formats might
differ between versions.
+};
+
+#endif /* AVUTIL_SUBFMT_H */
diff --git a/libavutil/version.h b/libavutil/version.h
index 017fc277a6..c6cd0b0d79 100644
--- a/libavutil/version.h
+++ b/libavutil/ve
[FFmpeg-devel] [PATCH v20 03/20] avcodec/subtitles: Introduce new frame-based subtitle decoding API
- Add avcodec_decode_subtitle3 which takes subtitle frames,
serving as compatibility shim to legacy subtitle decoding
- Add additional methods for conversion between old and new API
Signed-off-by: softworkz
---
doc/APIchanges | 7 ++
libavcodec/avcodec.h| 8 +-
libavcodec/codec_desc.c | 11 +++
libavcodec/codec_desc.h | 8 ++
libavcodec/decode.c | 54 ++--
libavcodec/internal.h | 16
libavcodec/utils.c | 182
7 files changed, 277 insertions(+), 9 deletions(-)
diff --git a/doc/APIchanges b/doc/APIchanges
index 8c63ea0311..49f1a28f71 100644
--- a/doc/APIchanges
+++ b/doc/APIchanges
@@ -14,6 +14,13 @@ libavutil: 2021-04-27
API changes, most recent first:
+2021-12-05 - xx - lavc 60.1.100 - codec_desc.h
+ Add avcodec_descriptor_get_subtitle_format()
+
+2021-12-05 - xx - lavc 60.1.100 - avcodec.h
+ Deprecate avsubtitle_free()
+ Deprecate avcodec_decode_subtitle2(), use regular decode api now
+
2021-12-05 - xx - lavu 58.1.100 - frame.h
Add AVMediaType field to AVFrame
Add Fields for carrying subtitle data to AVFrame
diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index b05c12e47e..3e734d3003 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -1675,7 +1675,7 @@ typedef struct AVCodecContext {
/**
* Header containing style information for text subtitles.
- * For SUBTITLE_ASS subtitle type, it should contain the whole ASS
+ * For AV_SUBTITLE_FMT_ASS subtitle type, it should contain the whole ASS
* [Script Info] and [V4+ Styles] section, plus the [Events] line and
* the Format line following. It shouldn't include any Dialogue line.
* - encoding: Set/allocated/freed by user (before avcodec_open2())
@@ -2417,7 +2417,10 @@ int avcodec_close(AVCodecContext *avctx);
* Free all allocated data in the given subtitle struct.
*
* @param sub AVSubtitle to free.
+ *
+ * @deprecated Use the regular frame based encode and decode APIs instead.
*/
+attribute_deprecated
void avsubtitle_free(AVSubtitle *sub);
/**
@@ -2510,7 +2513,10 @@ enum AVChromaLocation avcodec_chroma_pos_to_enum(int
xpos, int ypos);
* must be freed with avsubtitle_free if *got_sub_ptr is set.
* @param[in,out] got_sub_ptr Zero if no subtitle could be decompressed,
otherwise, it is nonzero.
* @param[in] avpkt The input AVPacket containing the input buffer.
+ *
+ * @deprecated Use the new decode API (avcodec_send_packet,
avcodec_receive_frame) instead.
*/
+attribute_deprecated
int avcodec_decode_subtitle2(AVCodecContext *avctx, AVSubtitle *sub,
int *got_sub_ptr,
AVPacket *avpkt);
diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c
index 0974ee03de..e48e4532ba 100644
--- a/libavcodec/codec_desc.c
+++ b/libavcodec/codec_desc.c
@@ -3548,3 +3548,14 @@ enum AVMediaType avcodec_get_type(enum AVCodecID
codec_id)
const AVCodecDescriptor *desc = avcodec_descriptor_get(codec_id);
return desc ? desc->type : AVMEDIA_TYPE_UNKNOWN;
}
+
+enum AVSubtitleType avcodec_descriptor_get_subtitle_format(const
AVCodecDescriptor *codec_descriptor)
+{
+if(codec_descriptor->props & AV_CODEC_PROP_BITMAP_SUB)
+return AV_SUBTITLE_FMT_BITMAP;
+
+if(codec_descriptor->props & AV_CODEC_PROP_TEXT_SUB)
+return AV_SUBTITLE_FMT_ASS;
+
+return AV_SUBTITLE_FMT_UNKNOWN;
+}
diff --git a/libavcodec/codec_desc.h b/libavcodec/codec_desc.h
index 126b52df47..ba68d24e0e 100644
--- a/libavcodec/codec_desc.h
+++ b/libavcodec/codec_desc.h
@@ -121,6 +121,14 @@ const AVCodecDescriptor *avcodec_descriptor_next(const
AVCodecDescriptor *prev);
*/
const AVCodecDescriptor *avcodec_descriptor_get_by_name(const char *name);
+/**
+ * Return subtitle format from a codec descriptor
+ *
+ * @param codec_descriptor codec descriptor
+ * @return the subtitle type (e.g. bitmap, text)
+ */
+enum AVSubtitleType avcodec_descriptor_get_subtitle_format(const
AVCodecDescriptor *codec_descriptor);
+
/**
* @}
*/
diff --git a/libavcodec/decode.c b/libavcodec/decode.c
index 52bf5dcd33..ac267f0df6 100644
--- a/libavcodec/decode.c
+++ b/libavcodec/decode.c
@@ -576,6 +576,37 @@ static int decode_receive_frame_internal(AVCodecContext
*avctx, AVFrame *frame)
return ret;
}
+static int decode_subtitle2_priv(AVCodecContext *avctx, AVSubtitle *sub,
+ int *got_sub_ptr, AVPacket *avpkt);
+
+static int decode_subtitle_shim(AVCodecContext *avctx, AVFrame *frame,
AVPacket *avpkt)
+{
+int ret, got_sub_ptr = 0;
+AVSubtitle subtitle = { 0 };
+
+if (frame->buf[0])
+return AVERROR(EAGAIN);
+
+av_frame_unref(frame);
+
+ret = decode_subtitle2_priv(avctx, &subtitle, &got_sub_ptr, avpkt);
+
+if (ret >= 0 && got_sub_ptr) {
+frame->type = AVMEDIA_TYPE_SUBTITLE;
+frame->format = subtitle.format;
+ret = a
[FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame for subtitle handling
Root commit for adding subtitle filtering capabilities.
In detail:
- Add type (AVMediaType) field to AVFrame
Replaces previous way of distinction which was based on checking
width and height to determine whether a frame is audio or video
- Add subtitle fields to AVFrame
- Add new struct AVSubtitleArea, similar to AVSubtitleRect, but different
allocation logic. Cannot and must not be used interchangeably, hence
the new struct
Signed-off-by: softworkz
---
doc/APIchanges | 10 +++
libavutil/Makefile | 1 +
libavutil/frame.c | 211 -
libavutil/frame.h | 77 -
libavutil/subfmt.c | 50 +++
libavutil/subfmt.h | 48 +++
6 files changed, 373 insertions(+), 24 deletions(-)
create mode 100644 libavutil/subfmt.c
diff --git a/doc/APIchanges b/doc/APIchanges
index a466a2ec22..8c63ea0311 100644
--- a/doc/APIchanges
+++ b/doc/APIchanges
@@ -14,6 +14,16 @@ libavutil: 2021-04-27
API changes, most recent first:
+2021-12-05 - xx - lavu 58.1.100 - frame.h
+ Add AVMediaType field to AVFrame
+ Add Fields for carrying subtitle data to AVFrame
+ (subtitle_areas, subtitle_header, subtitle_pts, start/end time, etc.)
+ Add av_frame_get_buffer2() and deprecate av_frame_get_buffer()
+
+2021-12-05 - xx - lavu 58.1.100 - subfmt.h
+ Add struct AVSubtitleArea (replaces AVSubtitle)
+ Add av_get_subtitle_fmt_name() and av_get_subtitle_fmt()
+
2021-12-05 - xx - lavu 58.1.100 - subfmt.h
Add enum AVSubtitleType (moved from lavc), add new values, deprecate existing
diff --git a/libavutil/Makefile b/libavutil/Makefile
index c7843db1e4..7e79936876 100644
--- a/libavutil/Makefile
+++ b/libavutil/Makefile
@@ -160,6 +160,7 @@ OBJS = adler32.o
\
slicethread.o\
spherical.o \
stereo3d.o \
+ subfmt.o \
threadmessage.o \
time.o \
timecode.o \
diff --git a/libavutil/frame.c b/libavutil/frame.c
index 64eed5b0dd..5da082adda 100644
--- a/libavutil/frame.c
+++ b/libavutil/frame.c
@@ -26,6 +26,7 @@
#include "imgutils.h"
#include "mem.h"
#include "samplefmt.h"
+#include "subfmt.h"
#include "hwcontext.h"
#define CHECK_CHANNELS_CONSISTENCY(frame) \
@@ -50,6 +51,9 @@ const char *av_get_colorspace_name(enum AVColorSpace val)
return name[val];
}
#endif
+
+static int frame_copy_subtitles(AVFrame *dst, const AVFrame *src, int
copy_data);
+
static void get_frame_defaults(AVFrame *frame)
{
if (frame->extended_data != frame->data)
@@ -73,7 +77,12 @@ static void get_frame_defaults(AVFrame *frame)
frame->colorspace = AVCOL_SPC_UNSPECIFIED;
frame->color_range = AVCOL_RANGE_UNSPECIFIED;
frame->chroma_location = AVCHROMA_LOC_UNSPECIFIED;
-frame->flags = 0;
+frame->subtitle_start_time = 0;
+frame->subtitle_end_time = 0;
+frame->num_subtitle_areas = 0;
+frame->subtitle_areas = NULL;
+frame->subtitle_pts= 0;
+frame->subtitle_header = NULL;
}
static void free_side_data(AVFrameSideData **ptr_sd)
@@ -244,23 +253,55 @@ static int get_audio_buffer(AVFrame *frame, int align)
}
+static int get_subtitle_buffer(AVFrame *frame)
+{
+// Buffers in AVFrame->buf[] are not used in case of subtitle frames.
+// To accomodate with existing code, checking ->buf[0] to determine
+// whether a frame is ref-counted or has data, we're adding a 1-byte
+// buffer here, which marks the subtitle frame to contain data.
+frame->buf[0] = av_buffer_alloc(1);
+if (!frame->buf[0]) {
+av_frame_unref(frame);
+return AVERROR(ENOMEM);
+}
+
+frame->extended_data = frame->data;
+
+return 0;
+}
+
int av_frame_get_buffer(AVFrame *frame, int align)
+{
+if (frame->width > 0 && frame->height > 0)
+frame->type = AVMEDIA_TYPE_VIDEO;
+else if (frame->nb_samples > 0 && (frame->channel_layout ||
frame->channels > 0))
+frame->type = AVMEDIA_TYPE_AUDIO;
+
+return av_frame_get_buffer2(frame, align);
+}
+
+int av_frame_get_buffer2(AVFrame *frame, int align)
{
if (frame->format < 0)
return AVERROR(EINVAL);
-if (frame->width > 0 && frame->height > 0)
+switch(frame->type) {
+case AVMEDIA_TYPE_VIDEO:
return get_video_buffer(frame, align);
-else if (frame->nb_samples > 0 && (frame->channel_layout ||
frame->channels > 0))
+case AVMEDIA_TYPE_AUDIO:
return get_audio_buffer(frame, align);
-
-return AVERROR(EINVAL);
+case AVMEDIA_TYPE_SUBTITLE:
+
[FFmpeg-devel] [PATCH v20 04/20] avfilter/subtitles: Update vf_subtitles to use new decoding api
Signed-off-by: softworkz
---
libavfilter/vf_subtitles.c | 54 +-
1 file changed, 42 insertions(+), 12 deletions(-)
diff --git a/libavfilter/vf_subtitles.c b/libavfilter/vf_subtitles.c
index 377160c72b..a240cbd415 100644
--- a/libavfilter/vf_subtitles.c
+++ b/libavfilter/vf_subtitles.c
@@ -35,14 +35,12 @@
# include "libavformat/avformat.h"
#endif
#include "libavutil/avstring.h"
-#include "libavutil/imgutils.h"
#include "libavutil/opt.h"
#include "libavutil/parseutils.h"
#include "drawutils.h"
#include "avfilter.h"
#include "internal.h"
#include "formats.h"
-#include "video.h"
typedef struct AssContext {
const AVClass *class;
@@ -292,6 +290,29 @@ static int attachment_is_font(AVStream * st)
return 0;
}
+static int decode(AVCodecContext *avctx, AVFrame *frame, int *got_frame,
AVPacket *pkt)
+{
+int ret;
+
+*got_frame = 0;
+
+if (pkt) {
+ret = avcodec_send_packet(avctx, pkt);
+// In particular, we don't expect AVERROR(EAGAIN), because we read all
+// decoded frames with avcodec_receive_frame() until done.
+if (ret < 0 && ret != AVERROR_EOF)
+return ret;
+}
+
+ret = avcodec_receive_frame(avctx, frame);
+if (ret < 0 && ret != AVERROR(EAGAIN))
+return ret;
+if (ret >= 0)
+*got_frame = 1;
+
+return 0;
+}
+
AVFILTER_DEFINE_CLASS(subtitles);
static av_cold int init_subtitles(AVFilterContext *ctx)
@@ -306,6 +327,7 @@ static av_cold int init_subtitles(AVFilterContext *ctx)
AVStream *st;
AVPacket pkt;
AssContext *ass = ctx->priv;
+enum AVSubtitleType subtitle_format;
/* Init libass */
ret = init(ctx);
@@ -386,13 +408,17 @@ static av_cold int init_subtitles(AVFilterContext *ctx)
ret = AVERROR_DECODER_NOT_FOUND;
goto end;
}
+
dec_desc = avcodec_descriptor_get(st->codecpar->codec_id);
-if (dec_desc && !(dec_desc->props & AV_CODEC_PROP_TEXT_SUB)) {
+subtitle_format = avcodec_descriptor_get_subtitle_format(dec_desc);
+
+if (subtitle_format != AV_SUBTITLE_FMT_ASS) {
av_log(ctx, AV_LOG_ERROR,
- "Only text based subtitles are currently supported\n");
-ret = AVERROR_PATCHWELCOME;
+ "Only text based subtitles are supported by this filter\n");
+ret = AVERROR_INVALIDDATA;
goto end;
}
+
if (ass->charenc)
av_dict_set(&codec_opts, "sub_charenc", ass->charenc, 0);
@@ -448,18 +474,22 @@ static av_cold int init_subtitles(AVFilterContext *ctx)
dec_ctx->subtitle_header_size);
while (av_read_frame(fmt, &pkt) >= 0) {
int i, got_subtitle;
-AVSubtitle sub = {0};
+AVFrame *sub = av_frame_alloc();
+if (!sub) {
+ret = AVERROR(ENOMEM);
+goto end;
+}
if (pkt.stream_index == sid) {
-ret = avcodec_decode_subtitle2(dec_ctx, &sub, &got_subtitle, &pkt);
+ret = decode(dec_ctx, sub, &got_subtitle, &pkt);
if (ret < 0) {
av_log(ctx, AV_LOG_WARNING, "Error decoding: %s (ignored)\n",
av_err2str(ret));
} else if (got_subtitle) {
-const int64_t start_time = av_rescale_q(sub.pts,
AV_TIME_BASE_Q, av_make_q(1, 1000));
-const int64_t duration = sub.end_display_time;
-for (i = 0; i < sub.num_rects; i++) {
-char *ass_line = sub.rects[i]->ass;
+const int64_t start_time = av_rescale_q(sub->subtitle_pts,
AV_TIME_BASE_Q, av_make_q(1, 1000));
+const int64_t duration = sub->subtitle_end_time;
+for (i = 0; i < sub->num_subtitle_areas; i++) {
+char *ass_line = sub->subtitle_areas[i]->ass;
if (!ass_line)
break;
ass_process_chunk(ass->track, ass_line, strlen(ass_line),
@@ -468,7 +498,7 @@ static av_cold int init_subtitles(AVFilterContext *ctx)
}
}
av_packet_unref(&pkt);
-avsubtitle_free(&sub);
+av_frame_free(&sub);
}
end:
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v20 05/20] avcodec, avutil: Move ass helper functions to avutil as avpriv_ and extend ass dialog parsing
Signed-off-by: softworkz
---
libavcodec/Makefile | 56 +++
libavcodec/ass.h | 147 ++
libavcodec/assdec.c | 2 +-
libavcodec/assenc.c | 2 +-
libavcodec/ccaption_dec.c | 19 +--
libavcodec/jacosubdec.c | 2 +-
libavcodec/libaribb24.c | 2 +-
libavcodec/libzvbi-teletextdec.c | 14 +-
libavcodec/microdvddec.c | 7 +-
libavcodec/movtextdec.c | 3 +-
libavcodec/movtextenc.c | 20 +--
libavcodec/mpl2dec.c | 2 +-
libavcodec/realtextdec.c | 2 +-
libavcodec/samidec.c | 2 +-
libavcodec/srtdec.c | 2 +-
libavcodec/srtenc.c | 16 +-
libavcodec/subviewerdec.c | 2 +-
libavcodec/textdec.c | 4 +-
libavcodec/ttmlenc.c | 15 +-
libavcodec/webvttdec.c| 2 +-
libavcodec/webvttenc.c| 16 +-
libavutil/Makefile| 2 +
{libavcodec => libavutil}/ass.c | 87 ---
libavutil/ass_internal.h | 133
{libavcodec => libavutil}/ass_split.c | 30 ++--
.../ass_split_internal.h | 24 +--
26 files changed, 339 insertions(+), 274 deletions(-)
rename {libavcodec => libavutil}/ass.c (65%)
create mode 100644 libavutil/ass_internal.h
rename {libavcodec => libavutil}/ass_split.c (94%)
rename libavcodec/ass_split.h => libavutil/ass_split_internal.h (89%)
diff --git a/libavcodec/Makefile b/libavcodec/Makefile
index 4122a9b144..b12638da5e 100644
--- a/libavcodec/Makefile
+++ b/libavcodec/Makefile
@@ -209,10 +209,10 @@ OBJS-$(CONFIG_APNG_DECODER)+= png.o pngdec.o
pngdsp.o
OBJS-$(CONFIG_APNG_ENCODER)+= png.o pngenc.o
OBJS-$(CONFIG_ARBC_DECODER)+= arbc.o
OBJS-$(CONFIG_ARGO_DECODER)+= argo.o
-OBJS-$(CONFIG_SSA_DECODER) += assdec.o ass.o
-OBJS-$(CONFIG_SSA_ENCODER) += assenc.o ass.o
-OBJS-$(CONFIG_ASS_DECODER) += assdec.o ass.o
-OBJS-$(CONFIG_ASS_ENCODER) += assenc.o ass.o
+OBJS-$(CONFIG_SSA_DECODER) += assdec.o
+OBJS-$(CONFIG_SSA_ENCODER) += assenc.o
+OBJS-$(CONFIG_ASS_DECODER) += assdec.o
+OBJS-$(CONFIG_ASS_ENCODER) += assenc.o
OBJS-$(CONFIG_ASV1_DECODER)+= asvdec.o asv.o mpeg12data.o
OBJS-$(CONFIG_ASV1_ENCODER)+= asvenc.o asv.o mpeg12data.o
OBJS-$(CONFIG_ASV2_DECODER)+= asvdec.o asv.o mpeg12data.o
@@ -253,7 +253,7 @@ OBJS-$(CONFIG_BRENDER_PIX_DECODER) += brenderpix.o
OBJS-$(CONFIG_C93_DECODER) += c93.o
OBJS-$(CONFIG_CAVS_DECODER)+= cavs.o cavsdec.o cavsdsp.o \
cavsdata.o
-OBJS-$(CONFIG_CCAPTION_DECODER)+= ccaption_dec.o ass.o
+OBJS-$(CONFIG_CCAPTION_DECODER)+= ccaption_dec.o
OBJS-$(CONFIG_CDGRAPHICS_DECODER) += cdgraphics.o
OBJS-$(CONFIG_CDTOONS_DECODER) += cdtoons.o
OBJS-$(CONFIG_CDXL_DECODER)+= cdxl.o
@@ -425,7 +425,7 @@ OBJS-$(CONFIG_INTERPLAY_ACM_DECODER) += interplayacm.o
OBJS-$(CONFIG_INTERPLAY_DPCM_DECODER) += dpcm.o
OBJS-$(CONFIG_INTERPLAY_VIDEO_DECODER) += interplayvideo.o
OBJS-$(CONFIG_IPU_DECODER) += mpeg12dec.o mpeg12.o mpeg12data.o
-OBJS-$(CONFIG_JACOSUB_DECODER) += jacosubdec.o ass.o
+OBJS-$(CONFIG_JACOSUB_DECODER) += jacosubdec.o
OBJS-$(CONFIG_JPEG2000_ENCODER)+= j2kenc.o mqcenc.o mqc.o jpeg2000.o \
jpeg2000dwt.o
OBJS-$(CONFIG_JPEG2000_DECODER)+= jpeg2000dec.o jpeg2000.o
jpeg2000dsp.o \
@@ -447,7 +447,7 @@ OBJS-$(CONFIG_MAGICYUV_ENCODER)+= magicyuvenc.o
OBJS-$(CONFIG_MDEC_DECODER)+= mdec.o mpeg12.o mpeg12data.o
OBJS-$(CONFIG_METASOUND_DECODER) += metasound.o metasound_data.o \
twinvq.o
-OBJS-$(CONFIG_MICRODVD_DECODER)+= microdvddec.o ass.o
+OBJS-$(CONFIG_MICRODVD_DECODER)+= microdvddec.o
OBJS-$(CONFIG_MIMIC_DECODER) += mimic.o
OBJS-$(CONFIG_MJPEG_DECODER) += mjpegdec.o mjpegdec_common.o
OBJS-$(CONFIG_MJPEG_QSV_DECODER) += qsvdec.o
@@ -462,8 +462,8 @@ OBJS-$(CONFIG_MLP_ENCODER) += mlpenc.o mlp.o
OBJS-$(CONFIG_MMVIDEO_DECODER) += mmvideo.o
OBJS-$(CONFIG_MOBICLIP_DECODER)+= mobiclip.o
OBJS-$(CONFIG_MOTIONPIXELS_DECODER)+= motionpixels.o
-OBJS-$(CONFIG_MOVTEXT_DECODER) += movtextdec.o ass.o
-OBJS-$(CONFIG_MOVTEXT_ENCODER) += movtextenc.o ass_split.o
+OBJS-$(CONFIG_MOVTEXT_DECODER) += movtextdec.o
+OBJS-$(CONFIG_MOVTE
[FFmpeg-devel] [PATCH v20 06/20] avcodec/subtitles: Migrate subtitle encoders to frame-based API and provide a compatibility shim for the legacy api
Signed-off-by: softworkz
---
doc/APIchanges | 3 +
libavcodec/assenc.c| 90 +
libavcodec/avcodec.h | 5 +-
libavcodec/dvbsubenc.c | 96 +--
libavcodec/dvdsubenc.c | 100 +++--
libavcodec/encode.c| 63 -
libavcodec/movtextenc.c| 112 +++--
libavcodec/srtenc.c| 104 ++
libavcodec/tests/avcodec.c | 2 -
libavcodec/ttmlenc.c | 98
libavcodec/webvttenc.c | 82 +++
libavcodec/xsubenc.c | 87 +---
12 files changed, 587 insertions(+), 255 deletions(-)
diff --git a/doc/APIchanges b/doc/APIchanges
index 49f1a28f71..83ac64c3f1 100644
--- a/doc/APIchanges
+++ b/doc/APIchanges
@@ -14,6 +14,9 @@ libavutil: 2021-04-27
API changes, most recent first:
+2021-12-05 - xx - lavc 60.1.100 - avcodec.h
+ Deprecate avcodec_encode_subtitle(), use regular encode api now
+
2021-12-05 - xx - lavc 60.1.100 - codec_desc.h
Add avcodec_descriptor_get_subtitle_format()
diff --git a/libavcodec/assenc.c b/libavcodec/assenc.c
index b0e475834b..94601bba68 100644
--- a/libavcodec/assenc.c
+++ b/libavcodec/assenc.c
@@ -22,48 +22,92 @@
#include
#include "avcodec.h"
+#include "encode.h"
#include "libavutil/ass_internal.h"
#include "internal.h"
#include "libavutil/avstring.h"
#include "libavutil/internal.h"
#include "libavutil/mem.h"
+static void check_write_header(AVCodecContext* avctx, const AVFrame* frame)
+{
+if (avctx->extradata_size)
+return;
+
+if (frame->subtitle_header && frame->subtitle_header->size > 0) {
+const char* subtitle_header = (char*)frame->subtitle_header->data;
+avctx->extradata_size = strlen(subtitle_header);
+avctx->extradata = av_mallocz(frame->subtitle_header->size + 1);
+memcpy(avctx->extradata, subtitle_header, avctx->extradata_size);
+avctx->extradata[avctx->extradata_size] = 0;
+}
+
+if (!avctx->extradata_size) {
+const char* subtitle_header =
avpriv_ass_get_subtitle_header_default(0);
+if (!subtitle_header)
+return;
+
+avctx->extradata_size = strlen(subtitle_header);
+avctx->extradata = av_mallocz(avctx->extradata_size + 1);
+memcpy(avctx->extradata, subtitle_header, avctx->extradata_size);
+avctx->extradata[avctx->extradata_size] = 0;
+av_freep(&subtitle_header);
+}
+}
+
static av_cold int ass_encode_init(AVCodecContext *avctx)
{
-avctx->extradata = av_malloc(avctx->subtitle_header_size + 1);
-if (!avctx->extradata)
-return AVERROR(ENOMEM);
-memcpy(avctx->extradata, avctx->subtitle_header,
avctx->subtitle_header_size);
-avctx->extradata_size = avctx->subtitle_header_size;
-avctx->extradata[avctx->extradata_size] = 0;
+if (avctx->subtitle_header_size) {
+avctx->extradata = av_malloc(avctx->subtitle_header_size + 1);
+if (!avctx->extradata)
+return AVERROR(ENOMEM);
+memcpy(avctx->extradata, avctx->subtitle_header,
avctx->subtitle_header_size);
+avctx->extradata_size = avctx->subtitle_header_size;
+avctx->extradata[avctx->extradata_size] = 0;
+}
+
return 0;
}
-static int ass_encode_frame(AVCodecContext *avctx,
-unsigned char *buf, int bufsize,
-const AVSubtitle *sub)
+static int ass_encode_frame(AVCodecContext* avctx, AVPacket* avpkt,
+const AVFrame* frame, int* got_packet)
{
-int i, len, total_len = 0;
+int ret;
+size_t req_len = 0, total_len = 0;
+
+check_write_header(avctx, frame);
-for (i=0; inum_rects; i++) {
-const char *ass = sub->rects[i]->ass;
+for (unsigned i = 0; i < frame->num_subtitle_areas; i++) {
+const char *ass = frame->subtitle_areas[i]->ass;
-if (sub->rects[i]->type != SUBTITLE_ASS) {
-av_log(avctx, AV_LOG_ERROR, "Only SUBTITLE_ASS type supported.\n");
+if (frame->subtitle_areas[i]->type != AV_SUBTITLE_FMT_ASS) {
+av_log(avctx, AV_LOG_ERROR, "Only AV_SUBTITLE_FMT_ASS type
supported.\n");
return AVERROR(EINVAL);
}
-len = av_strlcpy(buf+total_len, ass, bufsize-total_len);
+if (ass)
+req_len += strlen(ass);
+}
-if (len > bufsize-total_len-1) {
-av_log(avctx, AV_LOG_ERROR, "Buffer too small for ASS event.\n");
-return AVERROR_BUFFER_TOO_SMALL;
-}
+ret = ff_get_encode_buffer(avctx, avpkt, req_len + 1, 0);
+if (ret < 0) {
+av_log(avctx, AV_LOG_ERROR, "Error getting output packet.\n");
+return ret;
+}
-total_len += len;
+for (unsigned i = 0; i < frame->num_subtitle_areas; i++) {
+
[FFmpeg-devel] [PATCH v20 07/20] avcodec/subtitles: Replace deprecated enum values
Signed-off-by: softworkz
---
libavcodec/ass.h | 2 +-
libavcodec/assdec.c| 2 +-
libavcodec/dvbsubdec.c | 2 +-
libavcodec/dvdsubdec.c | 2 +-
libavcodec/dvdsubenc.c | 2 +-
libavcodec/pgssubdec.c | 2 +-
libavcodec/xsubdec.c | 2 +-
7 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/libavcodec/ass.h b/libavcodec/ass.h
index 9c9cb01c8a..8e885b3ae2 100644
--- a/libavcodec/ass.h
+++ b/libavcodec/ass.h
@@ -82,7 +82,7 @@ static inline int avpriv_ass_add_rect(AVSubtitle *sub, const
char *dialog,
rects[sub->num_rects] = av_mallocz(sizeof(*rects[0]));
if (!rects[sub->num_rects])
return AVERROR(ENOMEM);
-rects[sub->num_rects]->type = SUBTITLE_ASS;
+rects[sub->num_rects]->type = AV_SUBTITLE_FMT_ASS;
ass_str = avpriv_ass_get_dialog(readorder, layer, style, speaker, dialog);
if (!ass_str)
return AVERROR(ENOMEM);
diff --git a/libavcodec/assdec.c b/libavcodec/assdec.c
index 7802a44e71..fd321e7004 100644
--- a/libavcodec/assdec.c
+++ b/libavcodec/assdec.c
@@ -54,7 +54,7 @@ static int ass_decode_frame(AVCodecContext *avctx, void
*data, int *got_sub_ptr,
if (!sub->rects[0])
return AVERROR(ENOMEM);
sub->num_rects = 1;
-sub->rects[0]->type = SUBTITLE_ASS;
+sub->rects[0]->type = AV_SUBTITLE_FMT_ASS;
sub->rects[0]->ass = av_strdup(avpkt->data);
if (!sub->rects[0]->ass)
return AVERROR(ENOMEM);
diff --git a/libavcodec/dvbsubdec.c b/libavcodec/dvbsubdec.c
index 81ccaf4c57..b13244c803 100644
--- a/libavcodec/dvbsubdec.c
+++ b/libavcodec/dvbsubdec.c
@@ -795,7 +795,7 @@ static int save_subtitle_set(AVCodecContext *avctx,
AVSubtitle *sub, int *got_ou
rect->w = region->width;
rect->h = region->height;
rect->nb_colors = (1 << region->depth);
-rect->type = SUBTITLE_BITMAP;
+rect->type = AV_SUBTITLE_FMT_BITMAP;
rect->linesize[0] = region->width;
clut = get_clut(ctx, region->clut);
diff --git a/libavcodec/dvdsubdec.c b/libavcodec/dvdsubdec.c
index 52259f0730..b39b3d1838 100644
--- a/libavcodec/dvdsubdec.c
+++ b/libavcodec/dvdsubdec.c
@@ -406,7 +406,7 @@ static int decode_dvd_subtitles(DVDSubContext *ctx,
AVSubtitle *sub_header,
sub_header->rects[0]->y = y1;
sub_header->rects[0]->w = w;
sub_header->rects[0]->h = h;
-sub_header->rects[0]->type = SUBTITLE_BITMAP;
+sub_header->rects[0]->type = AV_SUBTITLE_FMT_BITMAP;
sub_header->rects[0]->linesize[0] = w;
sub_header->rects[0]->flags = is_menu ?
AV_SUBTITLE_FLAG_FORCED : 0;
}
diff --git a/libavcodec/dvdsubenc.c b/libavcodec/dvdsubenc.c
index 4916410fc8..fbab88a992 100644
--- a/libavcodec/dvdsubenc.c
+++ b/libavcodec/dvdsubenc.c
@@ -273,7 +273,7 @@ static int encode_dvd_subtitles(AVCodecContext* avctx,
AVPacket* avpkt,
return AVERROR(EINVAL);
for (i = 0; i < rects; i++)
-if (frame->subtitle_areas[i]->type != SUBTITLE_BITMAP) {
+if (frame->subtitle_areas[i]->type != AV_SUBTITLE_FMT_BITMAP) {
av_log(avctx, AV_LOG_ERROR, "Bitmap subtitle required\n");
return AVERROR(EINVAL);
}
diff --git a/libavcodec/pgssubdec.c b/libavcodec/pgssubdec.c
index 388639a110..4829babb7c 100644
--- a/libavcodec/pgssubdec.c
+++ b/libavcodec/pgssubdec.c
@@ -539,7 +539,7 @@ static int display_end_segment(AVCodecContext *avctx, void
*data,
return AVERROR(ENOMEM);
}
sub->num_rects++;
-sub->rects[i]->type = SUBTITLE_BITMAP;
+sub->rects[i]->type = AV_SUBTITLE_FMT_BITMAP;
/* Process bitmap */
object = find_object(ctx->presentation.objects[i].id, &ctx->objects);
diff --git a/libavcodec/xsubdec.c b/libavcodec/xsubdec.c
index 979399bae6..5c9e65f737 100644
--- a/libavcodec/xsubdec.c
+++ b/libavcodec/xsubdec.c
@@ -107,7 +107,7 @@ static int decode_frame(AVCodecContext *avctx, void *data,
int *got_sub_ptr,
}
sub->rects[0]->x = x; sub->rects[0]->y = y;
sub->rects[0]->w = w; sub->rects[0]->h = h;
-sub->rects[0]->type = SUBTITLE_BITMAP;
+sub->rects[0]->type = AV_SUBTITLE_FMT_BITMAP;
sub->rects[0]->linesize[0] = w;
sub->rects[0]->data[0] = av_malloc(w * h);
sub->rects[0]->nb_colors = 4;
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v20 08/20] fftools/play, probe: Adjust for subtitle changes
Signed-off-by: softworkz
---
fftools/ffplay.c | 102 +-
fftools/ffprobe.c | 48 ++
2 files changed, 78 insertions(+), 72 deletions(-)
diff --git a/fftools/ffplay.c b/fftools/ffplay.c
index e7b20be76b..0af32888da 100644
--- a/fftools/ffplay.c
+++ b/fftools/ffplay.c
@@ -152,7 +152,6 @@ typedef struct Clock {
/* Common struct for handling all types of decoded data and allocated render
buffers. */
typedef struct Frame {
AVFrame *frame;
-AVSubtitle sub;
int serial;
double pts; /* presentation timestamp for the frame */
double duration; /* estimated duration of the frame */
@@ -586,7 +585,7 @@ static int decoder_init(Decoder *d, AVCodecContext *avctx,
PacketQueue *queue, S
return 0;
}
-static int decoder_decode_frame(Decoder *d, AVFrame *frame, AVSubtitle *sub) {
+static int decoder_decode_frame(Decoder *d, AVFrame *frame) {
int ret = AVERROR(EAGAIN);
for (;;) {
@@ -620,6 +619,9 @@ static int decoder_decode_frame(Decoder *d, AVFrame *frame,
AVSubtitle *sub) {
}
}
break;
+case AVMEDIA_TYPE_SUBTITLE:
+ret = avcodec_receive_frame(d->avctx, frame);
+break;
}
if (ret == AVERROR_EOF) {
d->finished = d->pkt_serial;
@@ -652,25 +654,11 @@ static int decoder_decode_frame(Decoder *d, AVFrame
*frame, AVSubtitle *sub) {
av_packet_unref(d->pkt);
} while (1);
-if (d->avctx->codec_type == AVMEDIA_TYPE_SUBTITLE) {
-int got_frame = 0;
-ret = avcodec_decode_subtitle2(d->avctx, sub, &got_frame, d->pkt);
-if (ret < 0) {
-ret = AVERROR(EAGAIN);
-} else {
-if (got_frame && !d->pkt->data) {
-d->packet_pending = 1;
-}
-ret = got_frame ? 0 : (d->pkt->data ? AVERROR(EAGAIN) :
AVERROR_EOF);
-}
-av_packet_unref(d->pkt);
+if (avcodec_send_packet(d->avctx, d->pkt) == AVERROR(EAGAIN)) {
+av_log(d->avctx, AV_LOG_ERROR, "Receive_frame and send_packet both
returned EAGAIN, which is an API violation.\n");
+d->packet_pending = 1;
} else {
-if (avcodec_send_packet(d->avctx, d->pkt) == AVERROR(EAGAIN)) {
-av_log(d->avctx, AV_LOG_ERROR, "Receive_frame and send_packet
both returned EAGAIN, which is an API violation.\n");
-d->packet_pending = 1;
-} else {
-av_packet_unref(d->pkt);
-}
+av_packet_unref(d->pkt);
}
}
}
@@ -683,7 +671,6 @@ static void decoder_destroy(Decoder *d) {
static void frame_queue_unref_item(Frame *vp)
{
av_frame_unref(vp->frame);
-avsubtitle_free(&vp->sub);
}
static int frame_queue_init(FrameQueue *f, PacketQueue *pktq, int max_size,
int keep_last)
@@ -981,7 +968,7 @@ static void video_image_display(VideoState *is)
if (frame_queue_nb_remaining(&is->subpq) > 0) {
sp = frame_queue_peek(&is->subpq);
-if (vp->pts >= sp->pts + ((float) sp->sub.start_display_time /
1000)) {
+if (vp->pts >= sp->pts + ((float) sp->frame->subtitle_start_time /
1000)) {
if (!sp->uploaded) {
uint8_t* pixels[4];
int pitch[4];
@@ -993,25 +980,27 @@ static void video_image_display(VideoState *is)
if (realloc_texture(&is->sub_texture,
SDL_PIXELFORMAT_ARGB, sp->width, sp->height, SDL_BLENDMODE_BLEND, 1) < 0)
return;
-for (i = 0; i < sp->sub.num_rects; i++) {
-AVSubtitleRect *sub_rect = sp->sub.rects[i];
+for (i = 0; i < sp->frame->num_subtitle_areas; i++) {
+AVSubtitleArea *area = sp->frame->subtitle_areas[i];
+SDL_Rect sdl_rect = { .x = area->x, .y = area->y, .w =
area->w, .h = area->h };
-sub_rect->x = av_clip(sub_rect->x, 0, sp->width );
-sub_rect->y = av_clip(sub_rect->y, 0, sp->height);
-sub_rect->w = av_clip(sub_rect->w, 0, sp->width -
sub_rect->x);
-sub_rect->h = av_clip(sub_rect->h, 0, sp->height -
sub_rect->y);
+area->x = av_clip(area->x, 0, sp->width );
+area->y = av_clip(area->y, 0, sp->height);
+area->w = av_clip(area->w, 0, sp->width - area->x);
+area->h = av_clip(area->h, 0, sp->height - area->y);
is->sub_convert_ctx =
sws_getCachedContext(is->sub_convert_ctx,
-sub_rect->w, sub_rect->h, AV_PIX_FMT_PAL8,
-sub_rect->w, sub_rec
[FFmpeg-devel] [PATCH v20 09/20] avfilter/subtitles: Add subtitles.c for subtitle frame allocation
Analog to avfilter/video.c and avfilter/audio.c
Signed-off-by: softworkz
---
libavfilter/Makefile| 1 +
libavfilter/avfilter.c | 4 +++
libavfilter/internal.h | 1 +
libavfilter/subtitles.c | 63 +
libavfilter/subtitles.h | 44
5 files changed, 113 insertions(+)
create mode 100644 libavfilter/subtitles.c
create mode 100644 libavfilter/subtitles.h
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index c8082c4a2f..664566a18a 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -19,6 +19,7 @@ OBJS = allfilters.o
\
framequeue.o \
graphdump.o \
graphparser.o\
+ subtitles.o \
video.o \
OBJS-$(HAVE_THREADS) += pthread.o
diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
index 7362bcdab5..df5b8f483c 100644
--- a/libavfilter/avfilter.c
+++ b/libavfilter/avfilter.c
@@ -43,6 +43,7 @@
#include "formats.h"
#include "framepool.h"
#include "internal.h"
+#include "subtitles.h"
#include "libavutil/ffversion.h"
const char av_filter_ffversion[] = "FFmpeg version " FFMPEG_VERSION;
@@ -1475,6 +1476,9 @@ int ff_inlink_make_frame_writable(AVFilterLink *link,
AVFrame **rframe)
case AVMEDIA_TYPE_AUDIO:
out = ff_get_audio_buffer(link, frame->nb_samples);
break;
+case AVMEDIA_TYPE_SUBTITLE:
+out = ff_get_subtitles_buffer(link, link->format);
+break;
default:
return AVERROR(EINVAL);
}
diff --git a/libavfilter/internal.h b/libavfilter/internal.h
index 1099b82b4b..fc09ef574c 100644
--- a/libavfilter/internal.h
+++ b/libavfilter/internal.h
@@ -90,6 +90,7 @@ struct AVFilterPad {
union {
AVFrame *(*video)(AVFilterLink *link, int w, int h);
AVFrame *(*audio)(AVFilterLink *link, int nb_samples);
+AVFrame *(*subtitle)(AVFilterLink *link, int format);
} get_buffer;
/**
diff --git a/libavfilter/subtitles.c b/libavfilter/subtitles.c
new file mode 100644
index 00..951bfd612c
--- /dev/null
+++ b/libavfilter/subtitles.c
@@ -0,0 +1,63 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "libavutil/common.h"
+
+#include "subtitles.h"
+#include "avfilter.h"
+#include "internal.h"
+
+
+AVFrame *ff_null_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+return ff_get_subtitles_buffer(link->dst->outputs[0], format);
+}
+
+AVFrame *ff_default_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+AVFrame *frame;
+
+frame = av_frame_alloc();
+if (!frame)
+return NULL;
+
+frame->format = format;
+frame->type = AVMEDIA_TYPE_SUBTITLE;
+
+if (av_frame_get_buffer2(frame, 0) < 0) {
+av_frame_free(&frame);
+return NULL;
+}
+
+return frame;
+}
+
+AVFrame *ff_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+AVFrame *ret = NULL;
+
+if (link->dstpad->get_buffer.subtitle)
+ret = link->dstpad->get_buffer.subtitle(link, format);
+
+if (!ret)
+ret = ff_default_get_subtitles_buffer(link, format);
+
+return ret;
+}
diff --git a/libavfilter/subtitles.h b/libavfilter/subtitles.h
new file mode 100644
index 00..4a9115126e
--- /dev/null
+++ b/libavfilter/subtitles.h
@@ -0,0 +1,44 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have recei
[FFmpeg-devel] [PATCH v20 10/20] avfilter/avfilter: Handle subtitle frames
Signed-off-by: softworkz
---
doc/APIchanges | 3 +++
libavfilter/avfilter.c | 8 +---
libavfilter/avfilter.h | 11 +++
libavfilter/avfiltergraph.c | 5 +
libavfilter/formats.c | 22 ++
libavfilter/formats.h | 3 +++
libavfilter/internal.h | 18 +++---
7 files changed, 64 insertions(+), 6 deletions(-)
diff --git a/doc/APIchanges b/doc/APIchanges
index 83ac64c3f1..f695efe9ab 100644
--- a/doc/APIchanges
+++ b/doc/APIchanges
@@ -14,6 +14,9 @@ libavutil: 2021-04-27
API changes, most recent first:
+2021-12-05 - xx - lavc 9.1.100 - avfilter.h
+ Add fields subs_list and sub_fmt to AVFilter struct
+
2021-12-05 - xx - lavc 60.1.100 - avcodec.h
Deprecate avcodec_encode_subtitle(), use regular encode api now
diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
index df5b8f483c..75d5e86539 100644
--- a/libavfilter/avfilter.c
+++ b/libavfilter/avfilter.c
@@ -56,7 +56,8 @@ static void tlog_ref(void *ctx, AVFrame *ref, int end)
ref->linesize[0], ref->linesize[1], ref->linesize[2],
ref->linesize[3],
ref->pts, ref->pkt_pos);
-if (ref->width) {
+switch(ref->type) {
+case AVMEDIA_TYPE_VIDEO:
ff_tlog(ctx, " a:%d/%d s:%dx%d i:%c iskey:%d type:%c",
ref->sample_aspect_ratio.num, ref->sample_aspect_ratio.den,
ref->width, ref->height,
@@ -64,12 +65,13 @@ static void tlog_ref(void *ctx, AVFrame *ref, int end)
ref->top_field_first ? 'T' : 'B',/* Top / Bottom */
ref->key_frame,
av_get_picture_type_char(ref->pict_type));
-}
-if (ref->nb_samples) {
+break;
+case AVMEDIA_TYPE_AUDIO:
ff_tlog(ctx, " cl:%"PRId64"d n:%d r:%d",
ref->channel_layout,
ref->nb_samples,
ref->sample_rate);
+break;
}
ff_tlog(ctx, "]%s", end ? "\n" : "");
diff --git a/libavfilter/avfilter.h b/libavfilter/avfilter.h
index b105dc3159..9f917deb41 100644
--- a/libavfilter/avfilter.h
+++ b/libavfilter/avfilter.h
@@ -45,6 +45,7 @@
#include "libavutil/log.h"
#include "libavutil/samplefmt.h"
#include "libavutil/pixfmt.h"
+#include "libavutil/subfmt.h"
#include "libavutil/rational.h"
#include "libavfilter/version.h"
@@ -343,6 +344,12 @@ typedef struct AVFilter {
* and outputs use the same sample rate and channel count/layout.
*/
const enum AVSampleFormat *samples_list;
+/**
+ * Analogous to pixels, but delimited by AV_SUBTITLE_FMT_NONE
+ * and restricted to filters that only have AVMEDIA_TYPE_SUBTITLE
+ * inputs and outputs.
+ */
+const enum AVSubtitleType *subs_list;
/**
* Equivalent to { pix_fmt, AV_PIX_FMT_NONE } as pixels_list.
*/
@@ -351,6 +358,10 @@ typedef struct AVFilter {
* Equivalent to { sample_fmt, AV_SAMPLE_FMT_NONE } as samples_list.
*/
enum AVSampleFormat sample_fmt;
+/**
+ * Equivalent to { sub_fmt, AV_SUBTITLE_FMT_NONE } as subs_list.
+ */
+enum AVSubtitleType sub_fmt;
} formats;
int priv_size; ///< size of private data to allocate for the filter
diff --git a/libavfilter/avfiltergraph.c b/libavfilter/avfiltergraph.c
index b8b432e98b..f4987654af 100644
--- a/libavfilter/avfiltergraph.c
+++ b/libavfilter/avfiltergraph.c
@@ -311,6 +311,8 @@ static int filter_link_check_formats(void *log,
AVFilterLink *link, AVFilterForm
return ret;
break;
+case AVMEDIA_TYPE_SUBTITLE:
+return 0;
default:
av_assert0(!"reached");
}
@@ -441,6 +443,9 @@ static int query_formats(AVFilterGraph *graph, void
*log_ctx)
if (!link)
continue;
+if (link->type == AVMEDIA_TYPE_SUBTITLE)
+continue;
+
neg = ff_filter_get_negotiation(link);
av_assert0(neg);
for (neg_step = 1; neg_step < neg->nb_mergers; neg_step++) {
diff --git a/libavfilter/formats.c b/libavfilter/formats.c
index ba62f73248..46dbbd2975 100644
--- a/libavfilter/formats.c
+++ b/libavfilter/formats.c
@@ -19,6 +19,7 @@
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
+#include "libavcodec/avcodec.h"
#include "libavutil/avassert.h"
#include "libavutil/channel_layout.h"
#include "libavutil/common.h"
@@ -431,6 +432,12 @@ int ff_add_channel_layout(AVFilterChannelLayouts **l,
uint64_t channel_layout)
return 0;
}
+int ff_add_subtitle_type(AVFilterFormats **avff, int64_t fmt)
+{
+ADD_FORMAT(avff, fmt, ff_formats_unref, int, formats, nb_formats);
+return 0;
+}
+
AVFilterFormats *ff_make_formats_list_singleton(int fmt)
{
int fmts[2] = { fmt, -1 };
@@ -450,6 +457,13 @@ AVFilterFormats *ff_all_formats(enum AVMediaType type)
return NULL;
fmt++;
[FFmpeg-devel] [PATCH v20 11/20] avfilter/sbuffer: Add sbuffersrc and sbuffersink filters
Signed-off-by: softworkz
---
configure| 2 +-
libavfilter/allfilters.c | 2 ++
libavfilter/buffersink.c | 54 ++
libavfilter/buffersink.h | 7
libavfilter/buffersrc.c | 72
libavfilter/buffersrc.h | 1 +
6 files changed, 137 insertions(+), 1 deletion(-)
diff --git a/configure b/configure
index a98a18abaa..ea4c722a94 100755
--- a/configure
+++ b/configure
@@ -7808,7 +7808,7 @@ print_enabled_components(){
fi
done
if [ "$name" = "filter_list" ]; then
-for c in asrc_abuffer vsrc_buffer asink_abuffer vsink_buffer; do
+for c in asrc_abuffer vsrc_buffer ssrc_sbuffer asink_abuffer
vsink_buffer ssink_sbuffer; do
printf "&ff_%s,\n" $c >> $TMPH
done
fi
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index b1af2cbcc8..e134ac8059 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -553,8 +553,10 @@ extern const AVFilter ff_avsrc_movie;
* being the same while having different 'types'). */
extern const AVFilter ff_asrc_abuffer;
extern const AVFilter ff_vsrc_buffer;
+extern const AVFilter ff_ssrc_sbuffer;
extern const AVFilter ff_asink_abuffer;
extern const AVFilter ff_vsink_buffer;
+extern const AVFilter ff_ssink_sbuffer;
extern const AVFilter ff_af_afifo;
extern const AVFilter ff_vf_fifo;
diff --git a/libavfilter/buffersink.c b/libavfilter/buffersink.c
index c0215669e7..0b268c2fa4 100644
--- a/libavfilter/buffersink.c
+++ b/libavfilter/buffersink.c
@@ -29,6 +29,8 @@
#include "libavutil/internal.h"
#include "libavutil/opt.h"
+#include "libavcodec/avcodec.h"
+
#define FF_INTERNAL_FIELDS 1
#include "framequeue.h"
@@ -57,6 +59,10 @@ typedef struct BufferSinkContext {
int *sample_rates; ///< list of accepted sample rates
int sample_rates_size;
+/* only used for subtitles */
+enum AVSubtitleType *subtitle_types; ///< list of accepted subtitle
types, must be terminated with -1
+int subtitle_types_size;
+
AVFrame *peeked_frame;
} BufferSinkContext;
@@ -305,6 +311,28 @@ static int asink_query_formats(AVFilterContext *ctx)
return 0;
}
+static int ssink_query_formats(AVFilterContext *ctx)
+{
+BufferSinkContext *buf = ctx->priv;
+AVFilterFormats *formats = NULL;
+unsigned i;
+int ret;
+
+CHECK_LIST_SIZE(subtitle_types)
+if (buf->subtitle_types_size) {
+for (i = 0; i < NB_ITEMS(buf->subtitle_types); i++)
+if ((ret = ff_add_subtitle_type(&formats, buf->subtitle_types[i]))
< 0)
+return ret;
+if ((ret = ff_set_common_formats(ctx, formats)) < 0)
+return ret;
+} else {
+if ((ret = ff_default_query_formats(ctx)) < 0)
+return ret;
+}
+
+return 0;
+}
+
#define OFFSET(x) offsetof(BufferSinkContext, x)
#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM
static const AVOption buffersink_options[] = {
@@ -322,9 +350,16 @@ static const AVOption abuffersink_options[] = {
{ NULL },
};
#undef FLAGS
+#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_SUBTITLE_PARAM
+static const AVOption sbuffersink_options[] = {
+{ "subtitle_types", "set the supported subtitle formats",
OFFSET(subtitle_types), AV_OPT_TYPE_BINARY, .flags = FLAGS },
+{ NULL },
+};
+#undef FLAGS
AVFILTER_DEFINE_CLASS(buffersink);
AVFILTER_DEFINE_CLASS(abuffersink);
+AVFILTER_DEFINE_CLASS(sbuffersink);
static const AVFilterPad avfilter_vsink_buffer_inputs[] = {
{
@@ -363,3 +398,22 @@ const AVFilter ff_asink_abuffer = {
.outputs = NULL,
FILTER_QUERY_FUNC(asink_query_formats),
};
+
+static const AVFilterPad avfilter_ssink_sbuffer_inputs[] = {
+{
+.name = "default",
+.type = AVMEDIA_TYPE_SUBTITLE,
+},
+};
+
+const AVFilter ff_ssink_sbuffer = {
+.name = "sbuffersink",
+.description = NULL_IF_CONFIG_SMALL("Buffer subtitle frames, and make
them available to the end of the filter graph."),
+.priv_class= &sbuffersink_class,
+.priv_size = sizeof(BufferSinkContext),
+.init = common_init,
+.activate = activate,
+FILTER_INPUTS(avfilter_ssink_sbuffer_inputs),
+.outputs = NULL,
+FILTER_QUERY_FUNC(ssink_query_formats),
+};
diff --git a/libavfilter/buffersink.h b/libavfilter/buffersink.h
index 69ed0f29a8..11905abdc5 100644
--- a/libavfilter/buffersink.h
+++ b/libavfilter/buffersink.h
@@ -129,6 +129,13 @@ typedef struct AVABufferSinkParams {
*/
attribute_deprecated
AVABufferSinkParams *av_abuffersink_params_alloc(void);
+
+/**
+ * Deprecated and unused struct to use for initializing an sbuffersink context.
+ */
+typedef struct AVSBufferSinkParams {
+const int *subtitle_type;
+} AVSBufferSinkParams;
#endif
/**
diff --git a/libavfilter/buffersrc.c b/libavfilter/buffersrc.c
index b0611872f1..d2362999a2 100644
--- a/libavfilter/buffersrc.
[FFmpeg-devel] [PATCH v20 12/20] avfilter/overlaygraphicsubs: Add overlaygraphicsubs and graphicsub2video filters
- overlaygraphicsubs (VS -> V)
Overlay graphic subtitles onto a video stream
- graphicsub2video {S -> V)
Converts graphic subtitles to video frames (with alpha)
Gets auto-inserted for retaining compatibility with
sub2video command lines
Signed-off-by: softworkz
---
doc/filters.texi| 118 +
libavfilter/Makefile| 2 +
libavfilter/allfilters.c| 2 +
libavfilter/vf_overlaygraphicsubs.c | 737
4 files changed, 859 insertions(+)
create mode 100644 libavfilter/vf_overlaygraphicsubs.c
diff --git a/doc/filters.texi b/doc/filters.texi
index 3edf3f50b0..669b9f5365 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25635,6 +25635,124 @@ tools.
@c man end VIDEO SINKS
+@chapter Subtitle Filters
+@c man begin SUBTITLE FILTERS
+
+When you configure your FFmpeg build, you can disable any of the
+existing filters using @code{--disable-filters}.
+
+Below is a description of the currently available subtitle filters.
+
+@section graphicsub2video
+
+Renders graphic subtitles as video frames.
+
+This filter replaces the previous "sub2video" hack which did the conversion
implicitly and up-front as subtitle filtering wasn't possible at that time.
+To retain compatibility with earlier sub2video command lines, this filter is
being auto-inserted in those cases.
+
+For overlaying graphicsal subtitles it is recommended to use the
'overlay_graphicsubs' filter which is more efficient and takes less processing
resources.
+
+This filter is still useful in cases where the overlay is done with hardware
acceleration (e.g. overlay_qsv, overlay_vaapi, overlay_cuda) for preparing the
overlay frames.
+
+Inputs:
+@itemize
+@item 0: Subtitles [BITMAP]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video [RGB32]
+@end itemize
+
+
+It accepts the following parameters:
+
+@table @option
+@item size, s
+Set the size of the output video frame.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Overlay PGS subtitles
+(not recommended - better use overlay_graphicsubs)
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:1]graphicsub2video[subs];[0:0][subs]overlay" output.mp4
+@end example
+
+@item
+Overlay PGS subtitles implicitly
+The graphicsub2video is inserted automatically for compatibility with legacy
command lines.
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:0][0:1]overlay" output.mp4
+@end example
+@end itemize
+
+@section overlaygraphicsubs
+
+Overlay graphic subtitles onto a video stream.
+
+This filter can blend graphical subtitles on a video stream directly, i.e.
without creating full-size alpha images first.
+The blending operation is limited to the area of the subtitle rectangles,
which also means that no processing is done at times where no subtitles are to
be displayed.
+
+Inputs:
+@itemize
+@item 0: Video [YUV420P, YUV422P, YUV444P, ARGB, RGBA, ABGR, BGRA, RGB24,
BGR24]
+@item 1: Subtitles [BITMAP]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video (same as input)
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+@item x
+@item y
+Set the expression for the x and y coordinates of the overlaid video
+on the main video. Default value is "0" for both expressions. In case
+the expression is invalid, it is set to a huge value (meaning that the
+overlay will not be displayed within the output visible area).
+
+@item eof_action
+See @ref{framesync}.
+
+@item eval
+Set when the expressions for @option{x}, and @option{y} are evaluated.
+
+It accepts the following values:
+@table @samp
+@item init
+only evaluate expressions once during the filter initialization or
+when a command is processed
+
+@item frame
+evaluate expressions for each incoming frame
+@end table
+
+Default value is @samp{frame}.
+
+@item shortest
+See @ref{framesync}.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Overlay PGS subtitles
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:0][0:1]overlaygraphicsubs" output.mp4
+@end example
+@end itemize
+@c man end SUBTITLE FILTERS
+
@chapter Multimedia Filters
@c man begin MULTIMEDIA FILTERS
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 664566a18a..525b3a6e3c 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -295,6 +295,7 @@ OBJS-$(CONFIG_GBLUR_FILTER) += vf_gblur.o
OBJS-$(CONFIG_GBLUR_VULKAN_FILTER) += vf_gblur_vulkan.o vulkan.o
vulkan_filter.o
OBJS-$(CONFIG_GEQ_FILTER)+= vf_geq.o
OBJS-$(CONFIG_GRADFUN_FILTER)+= vf_gradfun.o
+OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlaygraphicsubs.o
framesync.o
OBJS-$(CONFIG_GRAPHMONITOR_FILTER) += f_graphmonitor.o
OBJS-$(CONFIG_GRAYWORLD_FILTER)
[FFmpeg-devel] [PATCH v20 13/20] fftools/ffmpeg: Replace sub2video with subtitle frame filtering and use new frame-based subtitle encoding API
This commit actually enables subtitle filtering in ffmpeg by
sending and receiving subtitle frames to and from a filtergraph.
The heartbeat functionality from the previous sub2video implementation
is retained and applied to all subtitle frames (bitmap, text, ..).
The other part of sub2video functionality is retained by
auto-insertion of the new graphicsub2video filter.
Justification for changed test refs:
- sub2video
The new results are identical excepting the last frame which
is due to the implementation changes
- sub2video_basic
The previous results had some incorrect output because multiple
frames had the same dts
The non-empty content frames are visually identical, the different
CRC is due to the different blending algorithm that is being used.
- sub2video_time_limited
The third frame in the previous ref was a repetition, which doesn't
happen anymore with the new subtitle filtering.
- sub-dvb
Running ffprobe -show_frames on the source file shows that there
are 7 subtitle frames with 0 rects in the source at the start
and 2 at the end. This translates to the 14 and 4 additional
entries in the new test results.
- filter-overlay-dvdsub-2397
Overlay results have slightly different CRCs due to different
blending implementation
Signed-off-by: softworkz
---
fftools/ffmpeg.c | 584 +++---
fftools/ffmpeg.h | 15 +-
fftools/ffmpeg_filter.c | 217 +---
fftools/ffmpeg_hw.c | 2 +-
fftools/ffmpeg_opt.c | 3 +-
tests/ref/fate/filter-overlay-dvdsub-2397 | 181 ---
tests/ref/fate/sub-dvb| 162 +++---
tests/ref/fate/sub2video | 116 ++---
tests/ref/fate/sub2video_basic| 135 ++---
tests/ref/fate/sub2video_time_limited | 4 +-
10 files changed, 725 insertions(+), 694 deletions(-)
diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c
index 47e6a8683e..acdaf62811 100644
--- a/fftools/ffmpeg.c
+++ b/fftools/ffmpeg.c
@@ -143,8 +143,6 @@ static int want_sdp = 1;
static BenchmarkTimeStamps current_time;
AVIOContext *progress_avio = NULL;
-static uint8_t *subtitle_out;
-
InputStream **input_streams = NULL;
intnb_input_streams = 0;
InputFile **input_files = NULL;
@@ -169,163 +167,6 @@ static int restore_tty;
static void free_input_threads(void);
#endif
-/* sub2video hack:
- Convert subtitles to video with alpha to insert them in filter graphs.
- This is a temporary solution until libavfilter gets real subtitles support.
- */
-
-static int sub2video_get_blank_frame(InputStream *ist)
-{
-int ret;
-AVFrame *frame = ist->sub2video.frame;
-
-av_frame_unref(frame);
-ist->sub2video.frame->width = ist->dec_ctx->width ? ist->dec_ctx->width
: ist->sub2video.w;
-ist->sub2video.frame->height = ist->dec_ctx->height ? ist->dec_ctx->height
: ist->sub2video.h;
-ist->sub2video.frame->format = AV_PIX_FMT_RGB32;
-if ((ret = av_frame_get_buffer(frame, 0)) < 0)
-return ret;
-memset(frame->data[0], 0, frame->height * frame->linesize[0]);
-return 0;
-}
-
-static void sub2video_copy_rect(uint8_t *dst, int dst_linesize, int w, int h,
-AVSubtitleRect *r)
-{
-uint32_t *pal, *dst2;
-uint8_t *src, *src2;
-int x, y;
-
-if (r->type != SUBTITLE_BITMAP) {
-av_log(NULL, AV_LOG_WARNING, "sub2video: non-bitmap subtitle\n");
-return;
-}
-if (r->x < 0 || r->x + r->w > w || r->y < 0 || r->y + r->h > h) {
-av_log(NULL, AV_LOG_WARNING, "sub2video: rectangle (%d %d %d %d)
overflowing %d %d\n",
-r->x, r->y, r->w, r->h, w, h
-);
-return;
-}
-
-dst += r->y * dst_linesize + r->x * 4;
-src = r->data[0];
-pal = (uint32_t *)r->data[1];
-for (y = 0; y < r->h; y++) {
-dst2 = (uint32_t *)dst;
-src2 = src;
-for (x = 0; x < r->w; x++)
-*(dst2++) = pal[*(src2++)];
-dst += dst_linesize;
-src += r->linesize[0];
-}
-}
-
-static void sub2video_push_ref(InputStream *ist, int64_t pts)
-{
-AVFrame *frame = ist->sub2video.frame;
-int i;
-int ret;
-
-av_assert1(frame->data[0]);
-ist->sub2video.last_pts = frame->pts = pts;
-for (i = 0; i < ist->nb_filters; i++) {
-ret = av_buffersrc_add_frame_flags(ist->filters[i]->filter, frame,
- AV_BUFFERSRC_FLAG_KEEP_REF |
- AV_BUFFERSRC_FLAG_PUSH);
-if (ret != AVERROR_EOF && ret < 0)
-av_log(NULL, AV_LOG_WARNING, "Error while add the frame to buffer
source(%s).\n",
- av_err2str(ret));
-}
-}
-
-void sub2video_update(InputStream *ist, int64_t heartbeat_pts, AVSubtitle *sub)
-{
-AVFrame *frame = ist->sub2video.frame;
-int8_t *dst;
-int dst_linesize;
-int num_rects, i;
-int64_t pts, end_pts;
-
-
[FFmpeg-devel] [PATCH v20 14/20] avfilter/avfilter: Fix hardcoded input index
This fix targets (rare) cases where multiple input pads have a
.filter_frame function. ff_request_frame_to_filter needs
to call ff_request_frame with the correct input pad
instead of the hardcoded first one.
Signed-off-by: softworkz
---
libavfilter/avfilter.c | 18 +-
1 file changed, 13 insertions(+), 5 deletions(-)
diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
index 75d5e86539..aa9aa71f53 100644
--- a/libavfilter/avfilter.c
+++ b/libavfilter/avfilter.c
@@ -463,7 +463,7 @@ static int64_t guess_status_pts(AVFilterContext *ctx, int
status, AVRational lin
return AV_NOPTS_VALUE;
}
-static int ff_request_frame_to_filter(AVFilterLink *link)
+static int ff_request_frame_to_filter(AVFilterLink *link, int input_index)
{
int ret = -1;
@@ -472,8 +472,8 @@ static int ff_request_frame_to_filter(AVFilterLink *link)
link->frame_blocked_in = 1;
if (link->srcpad->request_frame)
ret = link->srcpad->request_frame(link);
-else if (link->src->inputs[0])
-ret = ff_request_frame(link->src->inputs[0]);
+else if (link->src->inputs[input_index])
+ret = ff_request_frame(link->src->inputs[input_index]);
if (ret < 0) {
if (ret != AVERROR(EAGAIN) && ret != link->status_in)
ff_avfilter_link_set_in_status(link, ret,
guess_status_pts(link->src, ret, link->time_base));
@@ -1172,6 +1172,14 @@ static int forward_status_change(AVFilterContext
*filter, AVFilterLink *in)
{
unsigned out = 0, progress = 0;
int ret;
+int input_index = 0;
+
+for (int i = 0; i < in->dst->nb_inputs; i++) {
+if (&in->dst->input_pads[i] == in->dstpad) {
+input_index = i;
+break;
+}
+}
av_assert0(!in->status_out);
if (!filter->nb_outputs) {
@@ -1181,7 +1189,7 @@ static int forward_status_change(AVFilterContext *filter,
AVFilterLink *in)
while (!in->status_out) {
if (!filter->outputs[out]->status_in) {
progress++;
-ret = ff_request_frame_to_filter(filter->outputs[out]);
+ret = ff_request_frame_to_filter(filter->outputs[out],
input_index);
if (ret < 0)
return ret;
}
@@ -1218,7 +1226,7 @@ static int ff_filter_activate_default(AVFilterContext
*filter)
for (i = 0; i < filter->nb_outputs; i++) {
if (filter->outputs[i]->frame_wanted_out &&
!filter->outputs[i]->frame_blocked_in) {
-return ff_request_frame_to_filter(filter->outputs[i]);
+return ff_request_frame_to_filter(filter->outputs[i], 0);
}
}
return FFERROR_NOT_READY;
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v20 15/20] avfilter/overlaytextsubs: Add overlaytextsubs and textsubs2video filters
- overlaytextsubs {VS -> V)
Overlay text subtitles onto a video stream.
- textsubs2video {S -> V)
Converts text subtitles to video frames
Signed-off-by: softworkz
---
configure| 2 +
doc/filters.texi | 113 ++
libavfilter/Makefile | 2 +
libavfilter/allfilters.c | 4 +-
libavfilter/vf_overlaytextsubs.c | 646 +++
5 files changed, 766 insertions(+), 1 deletion(-)
create mode 100644 libavfilter/vf_overlaytextsubs.c
diff --git a/configure b/configure
index ea4c722a94..549382a61f 100755
--- a/configure
+++ b/configure
@@ -3666,6 +3666,7 @@ overlay_opencl_filter_deps="opencl"
overlay_qsv_filter_deps="libmfx"
overlay_qsv_filter_select="qsvvpp"
overlay_vulkan_filter_deps="vulkan spirv_compiler"
+overlaytextsubs_filter_deps="avcodec libass"
owdenoise_filter_deps="gpl"
pad_opencl_filter_deps="opencl"
pan_filter_deps="swresample"
@@ -3710,6 +3711,7 @@ superequalizer_filter_deps="avcodec"
superequalizer_filter_select="rdft"
surround_filter_deps="avcodec"
surround_filter_select="rdft"
+textsub2video_filter_deps="avcodec libass"
tinterlace_filter_deps="gpl"
tinterlace_merge_test_deps="tinterlace_filter"
tinterlace_pad_test_deps="tinterlace_filter"
diff --git a/doc/filters.texi b/doc/filters.texi
index 669b9f5365..2898ee0140 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25751,6 +25751,119 @@ Overlay PGS subtitles
ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:0][0:1]overlaygraphicsubs" output.mp4
@end example
@end itemize
+
+@section overlaytextsubs
+
+Overlay text subtitles onto a video stream.
+
+This filter supersedes the classic @ref{subtitles} filter opposed to which it
does no longer require to open and access the source stream separately, which
is often causing problems or doesn't even work for non-local or slow sources.
+
+Inputs:
+@itemize
+@item 0: Video [YUV420P, YUV422P, YUV444P, ARGB, RGBA, ABGR, BGRA, RGB24,
BGR24]
+@item 1: Subtitles [TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video (same as input)
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+
+@item alpha
+Process alpha channel, by default alpha channel is untouched.
+
+@item fonts_dir
+Set a directory path containing fonts that can be used by the filter.
+These fonts will be used in addition to whatever the font provider uses.
+
+@item default_font_path
+Path to a font file to be used as the default font.
+
+@item font_size
+Set the default font size.
+
+@item fontconfig_file
+Path to ASS fontconfig configuration file.
+
+@item force_style
+Override default style or script info parameters of the subtitles. It accepts a
+string containing ASS style format @code{KEY=VALUE} couples separated by ",".
+
+@item margin
+Set the rendering margin in pixels.
+
+@item render_latest_only
+For rendering, alway use the latest event only, which is covering the given
point in time
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Overlay ASS subtitles with animations:
+@example
+ffmpeg -i
"http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv";
-filter_complex "[0:v]overlaytextsubs" -map 0 -y out.mkv
+@end example
+@end itemize
+
+@section textsub2video
+
+Converts text subtitles to video frames.
+
+For overlaying text subtitles onto video frames it is recommended to use the
overlay_textsubs filter.
+The textsub2video is useful for for creating transparent text-frames when
overlay is done via hw acceleration
+
+Inputs:
+@itemize
+@item 0: Subtitles [TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video [RGB32]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+
+@item rate, r
+Set the framerate for updating overlay frames.
+Normally, overlay frames will only be updated each time when the subtitles to
display are changing.
+In cases where subtitles include advanced features (like animation), this
parameter determines the frequency by which the overlay frames should be
updated.
+
+@item size, s
+Set the output frame size.
+Allows to override the size of output video frames.
+
+@item fonts_dir
+Set a directory path containing fonts that can be used by the filter.
+These fonts will be used in addition to whatever the font provider uses.
+
+@item default_font_path
+Path to a font file to be used as the default font.
+
+@item font_size
+Set the default font size.
+
+@item fontconfig_file
+Path to ASS fontconfig configuration file.
+
+@item force_style
+Override default style or script info parameters of the subtitles. It accepts a
+string containing ASS style format @code{KEY=VALUE} couples separated by ",".
+
+@item margin
+Set the rendering margin in pixels.
+
+@item render_latest_only
+For rendering, alway use the latest event only, which is covering the given
point in time.
+@end table
+
@c man end SUBTITLE FILTERS
@chapter Multimedia Filters
diff --git a/libavf
[FFmpeg-devel] [PATCH v20 16/20] avfilter/textmod: Add textmod, censor and show_speaker filters
- textmod {S -> S)
Modify subtitle text in a number of ways
- censor {S -> S)
Censor subtitles using a word list
- show_speaker {S -> S)
Prepend speaker names from ASS subtitles to the visible text lines
Signed-off-by: softworkz
---
doc/filters.texi | 206
libavfilter/Makefile | 5 +
libavfilter/allfilters.c | 3 +
libavfilter/sf_textmod.c | 697 +++
4 files changed, 911 insertions(+)
create mode 100644 libavfilter/sf_textmod.c
diff --git a/doc/filters.texi b/doc/filters.texi
index 2898ee0140..c53bf0c60d 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25643,6 +25643,145 @@ existing filters using @code{--disable-filters}.
Below is a description of the currently available subtitle filters.
+
+@section censor
+
+Censor selected words in text subtitles.
+
+Inputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+@item mode
+The censoring mode to apply.
+
+Supported censoring modes are:
+
+@table @var
+@item 0, keep_first_last
+Replace all characters with the 'censor_char' except the first and the last
character of a word.
+For words with less than 4 characters, the last character will be replaced as
well.
+For words with less than 3 characters, the first character will be replaced as
well.
+@item 1, keep_first
+Replace all characters with the 'censor_char' except the first character of a
word.
+For words with less than 3 characters, the first character will be replaced as
well.
+@item 2, all
+Replace all characters with the 'censor_char'.
+@end table
+
+@item words
+A list of words to censor, separated by 'separator'.
+
+@item words_file
+Specify a file from which to load the contents for the 'words' parameter.
+
+@item censor_char
+Single character used as replacement for censoring.
+
+@item separator
+Delimiter character for words. Used with replace_words and remove_words- Must
be a single character.
+The default is '.'.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Censor a few given words with a pound character.
+@example
+ffmpeg -i
"http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv";
-filter_complex
"[0:1]censor=words='diss,louder,hope,beam,word':censor_char='#'" -map 0 -y
output.mkv
+@end example
+@end itemize
+
+
+@section textmod
+
+Modify subtitle text in a number of ways.
+
+Inputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+@item mode
+The kind of text modification to apply
+
+Supported operation modes are:
+
+@table @var
+@item 0, leet
+Convert subtitle text to 'leet speak'. It's primarily useful for testing as
the modification will be visible with almost all text lines.
+@item 1, to_upper
+Change all text to upper case. Might improve readability.
+@item 2, to_lower
+Change all text to lower case.
+@item 3, replace_chars
+Replace one or more characters. Requires the find and replace parameters to be
specified.
+Both need to be equal in length.
+The first char in find is replaced by the first char in replace, same for all
subsequent chars.
+@item 4, remove_chars
+Remove certain characters. Requires the find parameter to be specified.
+All chars in the find parameter string will be removed from all subtitle text.
+@item 5, replace_words
+Replace one or more words. Requires the find and replace parameters to be
specified. Multiple words must be separated by the delimiter char specified vie
the separator parameter (default: ',').
+The number of words in the find and replace parameters needs to be equal.
+The first word in find is replaced by the first word in replace, same for all
subsequent words
+@item 6, remove_words
+Remove certain words. Requires the find parameter to be specified. Multiple
words must be separated by the delimiter char specified vie the separator
parameter (default: ',').
+All words in the find parameter string will be removed from all subtitle text.
+@end table
+
+@item find
+Required for replace_chars, remove_chars, replace_words and remove_words.
+
+@item find_file
+Specify a file from which to load the contents for the 'find' parameter.
+
+@item replace
+Required for replace_chars and replace_words.
+
+@item replace_file
+Specify a file from which to load the contents for the 'replace' parameter.
+
+@item separator
+Delimiter character for words. Used with replace_words and remove_words- Must
be a single character.
+The default is '.'.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Change all characters to upper case while keeping all styles and animations:
+@example
+ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv";
-filter_complex "[0:s]textmod=mode=to_upper" -map 0 -y out.mkv
+@end example
+@item
+Remove a set of symbol characters for am improved and smooth
[FFmpeg-devel] [PATCH v20 17/20] avfilter/stripstyles: Add stripstyles filter
- stripstyles {S -> S)
Remove all inline styles from subtitle events
Signed-off-by: softworkz
---
doc/filters.texi | 37 +++
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/sf_stripstyles.c | 196 +++
4 files changed, 235 insertions(+)
create mode 100644 libavfilter/sf_stripstyles.c
diff --git a/doc/filters.texi b/doc/filters.texi
index c53bf0c60d..d4c564b3f0 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25703,6 +25703,43 @@ ffmpeg -i
"http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.
@end example
@end itemize
+@section stripstyles
+
+Remove all inline styles from subtitle events.
+
+Inputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+@item remove_animated
+Also remove text which is subject to animation (default: true)
+Usually, animated text elements are used used in addition to static subtitle
lines for creating effects, so in most cases it is safe to remove the animation
content.
+If subtitle text is missing, try setting this to false.
+
+@item select_layer
+Process only ASS subtitle events from a specific layer. This allows to filter
out certain effects where an ASS author duplicates the text onto multiple
layers.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Remove styles and animations from ASS subtitles and output events from ass
layer 0 only. Then convert asn save as SRT stream:
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv";
-filter_complex "[0:1]stripstyles=select_layer=0" -map 0 -c:s srt output.mkv
+@end example
+@end itemize
+
@section textmod
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index d2995fbec6..7d7da0c59a 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -557,6 +557,7 @@ OBJS-$(CONFIG_NULLSINK_FILTER) +=
vsink_nullsink.o
OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o
OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o
OBJS-$(CONFIG_TEXTMOD_FILTER)+= sf_textmod.o
+OBJS-$(CONFIG_STRIPSTYLES_FILTER)+= sf_stripstyles.o
# multimedia filters
OBJS-$(CONFIG_ABITSCOPE_FILTER) += avf_abitscope.o
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 673f9fb839..27680a2f00 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -546,6 +546,7 @@ extern const AVFilter ff_avf_showwavespic;
extern const AVFilter ff_vaf_spectrumsynth;
extern const AVFilter ff_sf_censor;
extern const AVFilter ff_sf_showspeaker;
+extern const AVFilter ff_sf_stripstyles;
extern const AVFilter ff_sf_textmod;
extern const AVFilter ff_svf_graphicsub2video;
extern const AVFilter ff_svf_textsub2video;
diff --git a/libavfilter/sf_stripstyles.c b/libavfilter/sf_stripstyles.c
new file mode 100644
index 00..82cb9c7647
--- /dev/null
+++ b/libavfilter/sf_stripstyles.c
@@ -0,0 +1,196 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/**
+ * @file
+ * text subtitle filter which removes inline-styles from subtitles
+ */
+
+#include "libavutil/opt.h"
+#include "internal.h"
+#include "libavutil/ass_split_internal.h"
+#include "libavutil/bprint.h"
+
+typedef struct StripStylesContext {
+const AVClass *class;
+enum AVSubtitleType format;
+int remove_animated;
+int select_layer;
+} StripStylesContext;
+
+typedef struct DialogContext {
+StripStylesContext* ss_ctx;
+AVBPrint buffer;
+int drawing_scale;
+int is_animated;
+} DialogContext;
+
+static void dialog_text_cb(void *priv, const char *text, int len)
+{
+DialogContext *s = priv;
+
+av_log(s->ss_ctx, AV_LOG_DEBUG, "dialog_text_cb: %s\n", text);
+
+if (!s->drawing_scale && (!s->is_animated || !s->ss_ctx->remove_animated))
+av_bprint_append_data(&s->buffer, text, len);
+}
+
+static void dialog_new_line_cb(void *priv, int forced)
+{
+DialogContext *s = priv;
+if (!s->drawing_scale && !s->is_animated)
+av_bprint_append_data(&s->buffer, forced ? "\\N" : "\\n", 2);
+}
+
+static void di
[FFmpeg-devel] [PATCH v20 18/20] avfilter/splitcc: Add splitcc filter for closed caption handling
- splitcc {V -> VS)
Extract closed-caption (A53) data from video
frames as subtitle Frames
ffmpeg -y -loglevel verbose -i
"https://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts";
-filter_complex
"[0:v]splitcc[vid1],textmod=mode=remove_chars:find='@',[vid1]overlay_textsubs"
output.mkv
Signed-off-by: softworkz
---
configure| 1 +
doc/filters.texi | 63 +++
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/sf_splitcc.c | 378 +++
5 files changed, 444 insertions(+)
create mode 100644 libavfilter/sf_splitcc.c
diff --git a/configure b/configure
index 549382a61f..2160521951 100755
--- a/configure
+++ b/configure
@@ -3704,6 +3704,7 @@ spp_filter_select="fft idctdsp fdctdsp me_cmp pixblockdsp"
sr_filter_deps="avformat swscale"
sr_filter_select="dnn"
stereo3d_filter_deps="gpl"
+splitcc_filter_deps="avcodec"
subtitles_filter_deps="avformat avcodec libass"
super2xsai_filter_deps="gpl"
pixfmts_super2xsai_test_deps="super2xsai_filter"
diff --git a/doc/filters.texi b/doc/filters.texi
index d4c564b3f0..5c1432311a 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -26053,6 +26053,69 @@ ffmpeg -i INPUT -filter_complex
"showspeaker=format=colon:style='@{\\c&HDD&\
@end example
@end itemize
+
+@section splitcc
+
+Split-out closed-caption/A53 subtitles from video frame side data.
+
+This filter provides an input and an output for video frames, which are just
passed through without modification.
+The second out provides subtitle frames which are extracted from video frame
side data.
+
+Inputs:
+@itemize
+@item 0: Video [ALL]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video (same as input)
+@item 1: Subtitles [TEXT]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+
+@item use_cc_styles
+Emit closed caption style header.
+This will make closed captions appear in white font with a black rectangle
background.
+
+@item real_time
+Emit subtitle events as they are decoded for real-time display.
+
+@item real_time_latency_msec
+Minimum elapsed time between emitting real-time subtitle events.
+Only applies to real_time mode.
+
+@item data_field
+Select data field. Possible values:
+
+@table @samp
+@item auto
+Pick first one that appears.
+@item first
+@item second
+@end table
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Extract closed captions as text subtitle stream and overlay it onto the video
in cc style (black bar background):
+@example
+ffmpeg -i "https://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts";
-filter_complex
"[0:v:0]splitcc=use_cc_styles=1[vid1][sub1];[vid1][sub1]overlaytextsubs"
output.mkv
+@end example
+
+@item
+A nicer variant, using realtime output from cc_dec and rendering it with the
render_latest_only parameter from overlaytextsubs to avoid ghosting by timely
overlap.
+@example
+ffmpeg -i "https://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts";
-filter_complex
"[0:v:0]splitcc=real_time=1:real_time_latency_msec=200[vid1][sub1];[vid1][sub1]overlaytextsubs=render_latest_only=1"
output.mkv
+@end example
+@end itemize
+
+
@section textsub2video
Converts text subtitles to video frames.
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 7d7da0c59a..2224e5fe5f 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -557,6 +557,7 @@ OBJS-$(CONFIG_NULLSINK_FILTER) +=
vsink_nullsink.o
OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o
OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o
OBJS-$(CONFIG_TEXTMOD_FILTER)+= sf_textmod.o
+OBJS-$(CONFIG_SPLITCC_FILTER)+= sf_splitcc.o
OBJS-$(CONFIG_STRIPSTYLES_FILTER)+= sf_stripstyles.o
# multimedia filters
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 27680a2f00..6adde2b9f6 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -546,6 +546,7 @@ extern const AVFilter ff_avf_showwavespic;
extern const AVFilter ff_vaf_spectrumsynth;
extern const AVFilter ff_sf_censor;
extern const AVFilter ff_sf_showspeaker;
+extern const AVFilter ff_sf_splitcc;
extern const AVFilter ff_sf_stripstyles;
extern const AVFilter ff_sf_textmod;
extern const AVFilter ff_svf_graphicsub2video;
diff --git a/libavfilter/sf_splitcc.c b/libavfilter/sf_splitcc.c
new file mode 100644
index 00..3556d084d9
--- /dev/null
+++ b/libavfilter/sf_splitcc.c
@@ -0,0 +1,378 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABIL
[FFmpeg-devel] [PATCH v20 19/20] avfilter/graphicsub2text: Add new graphicsub2text filter (OCR)
Signed-off-by: softworkz --- configure| 1 + doc/filters.texi | 55 + libavfilter/Makefile | 2 + libavfilter/allfilters.c | 1 + libavfilter/sf_graphicsub2text.c | 354 +++ 5 files changed, 413 insertions(+) create mode 100644 libavfilter/sf_graphicsub2text.c diff --git a/configure b/configure index 2160521951..1110c8fd51 100755 --- a/configure +++ b/configure @@ -3640,6 +3640,7 @@ frei0r_filter_deps="frei0r" frei0r_src_filter_deps="frei0r" fspp_filter_deps="gpl" gblur_vulkan_filter_deps="vulkan spirv_compiler" +graphicsub2text_filter_deps="libtesseract" hflip_vulkan_filter_deps="vulkan spirv_compiler" histeq_filter_deps="gpl" hqdn3d_filter_deps="gpl" diff --git a/doc/filters.texi b/doc/filters.texi index 5c1432311a..ea056be66b 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -25819,6 +25819,61 @@ ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv"; -filter_comple @end example @end itemize +@section graphicsub2text + +Converts graphic subtitles to text subtitles by performing OCR. + +For this filter to be available, ffmpeg needs to be compiled with libtesseract (see https://github.com/tesseract-ocr/tesseract). +Language models need to be downloaded from https://github.com/tesseract-ocr/tessdata and put into as subfolder named 'tessdata' or into a folder specified via the environment variable 'TESSDATA_PREFIX'. +The path can also be specified via filter option (see below). + +Note: These models are including the data for both OCR modes. + +Inputs: +- 0: Subtitles [bitmap] + +Outputs: +- 0: Subtitles [text] + +It accepts the following parameters: + +@table @option +@item ocr_mode +The character recognition mode to use. + +Supported OCR modes are: + +@table @var +@item 0, tesseract +This is the classic libtesseract operation mode. It is fast but less accurate than LSTM. +@item 1, lstm +Newer OCR implementation based on ML models. Provides usually better results, requires more processing resources. +@item 2, both +Use a combination of both modes. +@end table + +@item tessdata_path +The path to a folder containing the language models to be used. + +@item language +The recognition language. It needs to match the first three characters of a language model file in the tessdata path. + +@end table + + +@subsection Examples + +@itemize +@item +Convert DVB graphic subtitles to ASS (text) subtitles + +Note: For this to work, you need to have the data file 'eng.traineddata' in a 'tessdata' subfolder (see above). +@example +ffmpeg ffmpeg -loglevel verbose -i "https://streams.videolan.org/streams/ts/video_subs_ttxt%2Bdvbsub.ts"; -filter_complex "[0:13]graphicsub2text=ocr_mode=both" -c:s ass -y output.mkv +@end example +@end itemize + + @section graphicsub2video Renders graphic subtitles as video frames. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 2224e5fe5f..3b972e134b 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -296,6 +296,8 @@ OBJS-$(CONFIG_GBLUR_VULKAN_FILTER) += vf_gblur_vulkan.o vulkan.o vulka OBJS-$(CONFIG_GEQ_FILTER)+= vf_geq.o OBJS-$(CONFIG_GRADFUN_FILTER)+= vf_gradfun.o OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlaygraphicsubs.o framesync.o +OBJS-$(CONFIG_GRAPHICSUB2TEXT_FILTER)+= sf_graphicsub2text.o +OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlaygraphicsubs.o framesync.o OBJS-$(CONFIG_GRAPHMONITOR_FILTER) += f_graphmonitor.o OBJS-$(CONFIG_GRAYWORLD_FILTER) += vf_grayworld.o OBJS-$(CONFIG_GREYEDGE_FILTER) += vf_colorconstancy.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 6adde2b9f6..f70f08dc5a 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -545,6 +545,7 @@ extern const AVFilter ff_avf_showwaves; extern const AVFilter ff_avf_showwavespic; extern const AVFilter ff_vaf_spectrumsynth; extern const AVFilter ff_sf_censor; +extern const AVFilter ff_sf_graphicsub2text; extern const AVFilter ff_sf_showspeaker; extern const AVFilter ff_sf_splitcc; extern const AVFilter ff_sf_stripstyles; diff --git a/libavfilter/sf_graphicsub2text.c b/libavfilter/sf_graphicsub2text.c new file mode 100644 index 00..ef10d60efd --- /dev/null +++ b/libavfilter/sf_graphicsub2text.c @@ -0,0 +1,354 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for mor
[FFmpeg-devel] [PATCH v20 20/20] avfilter/subscale: Add filter for scaling and/or re-arranging graphical subtitles
Signed-off-by: softworkz
---
configure | 1 +
doc/filters.texi | 164 +++
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/sf_subscale.c | 883 ++
5 files changed, 1050 insertions(+)
create mode 100644 libavfilter/sf_subscale.c
diff --git a/configure b/configure
index 1110c8fd51..5188c3c75d 100755
--- a/configure
+++ b/configure
@@ -3706,6 +3706,7 @@ sr_filter_deps="avformat swscale"
sr_filter_select="dnn"
stereo3d_filter_deps="gpl"
splitcc_filter_deps="avcodec"
+subscale_filter_deps="swscale avcodec"
subtitles_filter_deps="avformat avcodec libass"
super2xsai_filter_deps="gpl"
pixfmts_super2xsai_test_deps="super2xsai_filter"
diff --git a/doc/filters.texi b/doc/filters.texi
index ea056be66b..a61093b9fb 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -26225,6 +26225,170 @@ Set the rendering margin in pixels.
For rendering, alway use the latest event only, which is covering the given
point in time.
@end table
+@section subscale
+
+Provides high-quality scaling and rearranging functionality for graphical
subtitles.
+
+The subscale filter provides multiple approaches for manipulating
+the size and position of graphical subtitle rectangles wich can
+be combined or used separately.
+Scaling is performed by converting the palettized subtitle bitmaps
+to RGBA and re-quantization to palette colors afterwards via elbg algorithm.
+
+The two major operations are 'scale' and 're-arrange' with the
+latter being separated as 'arrange_h' and 'arrange_v'.
+
+
+Inputs:
+- 0: Subtitles [bitmap]
+
+Outputs:
+- 0: Subtitles [bitmap]
+
+It accepts the following parameters:
+
+@table @option
+
+@item w, width
+Set the width of the output.
+Width and height in case of graphical subtitles are just indicating
+a virtual size for which the output (consisting of 0-n bitmap rectangles)
+is intended to be displayed on.
+
+@item h, height
+Set the height of the output.
+
+@item margin_h
+Sets a horizontal margin to be preserverved when using any
+of the arrange modes.
+
+@item margin_v
+Sets a vertical margin to be preserverved when using any
+of the arrange modes.
+
+@item force_original_aspect_ratio
+Enable decreasing or increasing output video width or height if necessary to
+keep the original aspect ratio. Possible values:
+
+@table @samp
+@item disable
+Scale the video as specified and disable this feature.
+
+@item decrease
+The output video dimensions will automatically be decreased if needed.
+
+@item increase
+The output video dimensions will automatically be increased if needed.
+
+@end table
+
+
+@item scale_mode
+Specifies how subtitle bitmaps should be scaled.
+The scale factor is determined by the the factor between input
+and output size.
+
+@table @samp
+@item none
+Do not apply any common scaling.
+
+@item uniform
+Uniformly scale all subtitle bitmaps including their positions.
+
+@item uniform_no_reposition
+Uniformly scale all subtitle bitmaps without changing positions.
+
+@end table
+
+
+@item arrange_h
+Specifies how subtitle bitmaps should be arranged horizontally.
+
+@item arrange_v
+Specifies how subtitle bitmaps should be arranged vertically.
+
+
+@table @samp
+@item none
+Do not rearrange subtitle bitmaps.
+
+@item margin_no_scale
+Move subtitle bitmaps to be positioned inside the specified
+margin (margin_h or margin_v) when possible and without scaling.
+
+@item margin_and_scale
+Move subtitle bitmaps to be positioned inside the specified
+margin (margin_h or margin_v) and scale in case it doesn't fit.
+
+@item snapalign_no_scale
+Categorize subtitle bitmap positions as one of left/center/right
+or top/bottom/middle based on original positioning and apply
+these alignments for the target positioning.
+No scaling will be applied.
+
+@item snapalign_and_scale
+Categorize subtitle bitmap positions as one of left/center/right
+or top/bottom/middle based on original positioning and apply
+these alignments for the target positioning.
+Bitmaps that do not fit inside the margins borders are
+scaled to fit.
+@end table
+
+@item eval
+Set evaluation mode for the expressions (@option{width}, @option{height}).
+
+It accepts the following values:
+@table @samp
+@item init
+Evaluate expressions only once during the filter initialization.
+
+@item frame
+Evaluate expressions for each incoming frame. This is way slower than the
+@samp{init} mode since it requires all the scalers to be re-computed, but it
+allows advanced dynamic expressions.
+@end table
+
+Default value is @samp{init}.
+
+
+@item num_colors
+Set the number of palette colors for output images.
+Choose the maximum (256) when further processing is done (e.g.
+overlaying on a video).
+When subtitles will be encoded as bitmap subtitles (e.g. dvbsub),
+a smaller number of palette colors (e.g. 4-16) might need to be used, depending
+on the target format and codec.
+
+@item bitmap_width_align
+@item bitmap_height_align
+Make sure that subtitle
Re: [FFmpeg-devel] [PATCH v19 01/20] avcodec, avutil: Move enum AVSubtitleType to avutil, add new and deprecate old values
> -Original Message- > From: ffmpeg-devel On Behalf Of Michael > Niedermayer > Sent: Sunday, December 5, 2021 4:53 PM > To: FFmpeg development discussions and patches > Subject: Re: [FFmpeg-devel] [PATCH v19 01/20] avcodec, avutil: Move enum > AVSubtitleType to avutil, add new and deprecate old values > > On Fri, Dec 03, 2021 at 07:31:06PM +, Soft Works wrote: > > Signed-off-by: softworkz > > --- > > doc/APIchanges | 6 > > libavcodec/avcodec.h | 19 + > > libavutil/Makefile | 1 + > > libavutil/subfmt.h | 66 > > libavutil/version.h | 1 + > > 5 files changed, 75 insertions(+), 18 deletions(-) > > create mode 100644 libavutil/subfmt.h > > Some of the patches in this patchset do not apply cleanly to master anymore Posted a new version. Thanks, sw ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame for subtitle handling
5 Dec 2021, 17:23 by softwo...@hotmail.com:
> @@ -491,6 +499,39 @@ typedef struct AVFrame {
> */
> uint64_t channel_layout;
>
> +/**
> + * Display start time, relative to packet pts, in ms.
> + */
> +uint32_t subtitle_start_time;
> +
> +/**
> + * Display end time, relative to packet pts, in ms.
> + */
> +uint32_t subtitle_end_time;
>
Milliseconds? Our entire API's based around timestamps
with time bases. Plus, we all know what happened when
Matroska settled onto milliseconds and ruined a perfectly
complex but good container.
Make this relative to the PTS field, with the same timebase
as the PTS field.
There's even a new AVFrame->time_base field for you to
set so you wouldn't forget it.
> +/**
> + * Number of items in the @ref subtitle_areas array.
> + */
> +unsigned num_subtitle_areas;
> +
> +/**
> + * Array of subtitle areas, may be empty.
> + */
> +AVSubtitleArea **subtitle_areas;
>
There's no reason why this cannot be handled using the buffer
and data fields. If you need more space there, you're free to bump
the maximum number of pointers, plus this removes the horrid
malloc(1) hack. We've discussed this, and I couldn't follow why
this was bad in the email discussion you linked.
> +/**
> + * Usually the same as packet pts, in AV_TIME_BASE.
> + *
> + * @deprecated This is kept for compatibility reasons and corresponds to
> + * AVSubtitle->pts. Might be removed in the future.
> + */
> +int64_t subtitle_pts;
>
I'm not going to accept a field which is instantly deprecated.
As we've discussed multiple times, please merge this into
the regular frame PTS field. We already have _2_ necessary
stard/end fields.
> +/**
> + * Header containing style information for text subtitles.
> + */
> +AVBufferRef *subtitle_header;
>
Good, I like this.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame for subtitle handling
On Sun, Dec 5, 2021 at 5:40 PM Lynne wrote: > > +/** > > + * Usually the same as packet pts, in AV_TIME_BASE. > > + * > > + * @deprecated This is kept for compatibility reasons and corresponds > > to > > + * AVSubtitle->pts. Might be removed in the future. > > + */ > > +int64_t subtitle_pts; > > > > I'm not going to accept a field which is instantly deprecated. > As we've discussed multiple times, please merge this into > the regular frame PTS field. We already have _2_ necessary > stard/end fields. > I agree with this entirely. Even ignoring the fact that adding a new field thats deprecated is instantly a disqualification, AVSubtitle had one pts field, AVFrame already has one pts field - both are even documented to have the same semantic. They should just contain the exact same data, thats how you achieve compatibility, not by claiming you need a new field for compatibility reasons. - Hendrik ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v9 0/5] Add support for Matroska BlockAdditionMapping elements
This patch set adds support for reading/writing the Matroska
BlockAdditionMapping
elements, as well as for reading/writing dvcC/dvvC blocks in Matroska.
Created utility functions to read/write Dolby Vision boxes for ISOM.
This was done to avoid duplicating the code, as the Matroska blocks and MOV
boxes
follow the same specification, defined by Dolby.
Refactored the reading/writing in mov/movenc to use the new dovi_isom functions.
v8: https://ffmpeg.org/pipermail/ffmpeg-devel/2021-December/288819.html
Changes since v8:
- Removed size argument from ff_isom_put_dvcc_dvvc, and changed the return
type to void.
The function always writes a constant number of bytes.
quietvoid (5):
avformat/dovi_isom: Implement Dolby Vision configuration
parsing/writing
avformat/matroska{dec, enc}: Parse BlockAdditionMapping elements
avformat/mov: Refactor mov_read_dvcc_dvvc to use
ff_isom_parse_dvcc_dvvc
avformat/movenc: Refactor mov_write_dvcc_dvvc_tag to use
ff_isom_put_dvcc_dvvc
fate/matroska: Add tests for reading/writing BlockAdditionMapping
elements
libavformat/Makefile | 4 +-
libavformat/dovi_isom.c | 118 ++
libavformat/dovi_isom.h | 35 +++
libavformat/matroska.h | 9 +
libavformat/matroskadec.c| 58 -
libavformat/matroskaenc.c| 40
libavformat/mov.c| 50 +
libavformat/movenc.c | 24 +-
tests/fate/matroska.mak | 9 +
tests/ref/fate/matroska-dovi-config-profile5 | 13 ++
tests/ref/fate/matroska-dovi-write-config| 223 +++
11 files changed, 521 insertions(+), 62 deletions(-)
create mode 100644 libavformat/dovi_isom.c
create mode 100644 libavformat/dovi_isom.h
create mode 100644 tests/ref/fate/matroska-dovi-config-profile5
create mode 100644 tests/ref/fate/matroska-dovi-write-config
--
2.34.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v9 1/5] avformat/dovi_isom: Implement Dolby Vision configuration parsing/writing
Both parse/write implementations are based on mov/movenc.
This only adds support for the "Dolby Vision configuration box".
Other configuration boxes, such as
"Dolby Vision enhancement layer configuration box" are not supported.
The new functions will be used to implement parsing/writing the DOVI config
for Matroska, as well as to refactor both mov/movenc to use dovi_isom functions.
Signed-off-by: quietvoid
---
libavformat/dovi_isom.c | 118
libavformat/dovi_isom.h | 35
2 files changed, 153 insertions(+)
create mode 100644 libavformat/dovi_isom.c
create mode 100644 libavformat/dovi_isom.h
diff --git a/libavformat/dovi_isom.c b/libavformat/dovi_isom.c
new file mode 100644
index 00..262d7e37ba
--- /dev/null
+++ b/libavformat/dovi_isom.c
@@ -0,0 +1,118 @@
+/*
+ * DOVI ISO Media common code
+ *
+ * Copyright (c) 2020 Vacing Fang
+ * Copyright (c) 2021 quietvoid
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "libavutil/dovi_meta.h"
+
+#include "libavcodec/put_bits.h"
+
+#include "avformat.h"
+#include "dovi_isom.h"
+
+int ff_isom_parse_dvcc_dvvc(AVFormatContext *s, AVStream *st, const uint8_t
*buf_ptr, uint64_t size)
+{
+uint32_t buf;
+AVDOVIDecoderConfigurationRecord *dovi;
+size_t dovi_size;
+int ret;
+
+if (size > (1 << 30) || size < 4)
+return AVERROR_INVALIDDATA;
+
+dovi = av_dovi_alloc(&dovi_size);
+if (!dovi)
+return AVERROR(ENOMEM);
+
+dovi->dv_version_major = *buf_ptr++;// 8 bits
+dovi->dv_version_minor = *buf_ptr++;// 8 bits
+
+buf = *buf_ptr++ << 8;
+buf |= *buf_ptr++;
+
+dovi->dv_profile= (buf >> 9) & 0x7f;// 7 bits
+dovi->dv_level = (buf >> 3) & 0x3f;// 6 bits
+dovi->rpu_present_flag = (buf >> 2) & 0x01;// 1 bit
+dovi->el_present_flag = (buf >> 1) & 0x01;// 1 bit
+dovi->bl_present_flag = buf & 0x01;// 1 bit
+
+// Has enough remaining data
+if (size >= 5) {
+dovi->dv_bl_signal_compatibility_id = ((*buf_ptr++) >> 4) & 0x0f; // 4
bits
+} else {
+// 0 stands for None
+// Dolby Vision V1.2.93 profiles and levels
+dovi->dv_bl_signal_compatibility_id = 0;
+}
+
+ret = av_stream_add_side_data(st, AV_PKT_DATA_DOVI_CONF,
+ (uint8_t *)dovi, dovi_size);
+if (ret < 0) {
+av_free(dovi);
+return ret;
+}
+
+av_log(s, AV_LOG_TRACE, "DOVI in dvcC/dvvC/dvwC box, version: %d.%d,
profile: %d, level: %d, "
+ "rpu flag: %d, el flag: %d, bl flag: %d, compatibility id: %d\n",
+ dovi->dv_version_major, dovi->dv_version_minor,
+ dovi->dv_profile, dovi->dv_level,
+ dovi->rpu_present_flag,
+ dovi->el_present_flag,
+ dovi->bl_present_flag,
+ dovi->dv_bl_signal_compatibility_id);
+
+return 0;
+}
+
+void ff_isom_put_dvcc_dvvc(AVFormatContext *s, uint8_t
out[ISOM_DVCC_DVVC_SIZE],
+ AVDOVIDecoderConfigurationRecord *dovi)
+{
+PutBitContext pb;
+
+init_put_bits(&pb, out, ISOM_DVCC_DVVC_SIZE);
+
+put_bits(&pb, 8, dovi->dv_version_major);
+put_bits(&pb, 8, dovi->dv_version_minor);
+put_bits(&pb, 7, dovi->dv_profile);
+put_bits(&pb, 6, dovi->dv_level);
+put_bits(&pb, 1, dovi->rpu_present_flag);
+put_bits(&pb, 1, dovi->el_present_flag);
+put_bits(&pb, 1, dovi->bl_present_flag);
+put_bits(&pb, 4, dovi->dv_bl_signal_compatibility_id);
+
+put_bits(&pb, 28, 0); /* reserved */
+put_bits32(&pb, 0); /* reserved */
+put_bits32(&pb, 0); /* reserved */
+put_bits32(&pb, 0); /* reserved */
+put_bits32(&pb, 0); /* reserved */
+
+flush_put_bits(&pb);
+
+av_log(s, AV_LOG_DEBUG, "DOVI in %s box, version: %d.%d, profile: %d,
level: %d, "
+ "rpu flag: %d, el flag: %d, bl flag: %d, compatibility id: %d\n",
+ dovi->dv_profile > 10 ? "dvwC" : (dovi->dv_profile > 7 ? "dvvC" :
"dvcC"),
+ dovi->dv_version_major, dovi->dv_version_minor,
+ dovi->dv_profile, dovi->dv_level,
+ dovi->rpu_present_flag,
+ dovi->el_present_flag,
+ dovi->bl_present_flag,
+ dovi->dv_bl_signal
[FFmpeg-devel] [PATCH v9 2/5] avformat/matroska{dec, enc}: Parse BlockAdditionMapping elements
Adds handling of dvcC/dvvC block addition mappings.
The parsing creates AVDOVIDecoderConfigurationRecord side data.
The configuration block is written when muxing into Matroska,
if DOVI side data is present for the track.
Most of the Matroska element parsing is based on Plex's FFmpeg source code.
Signed-off-by: quietvoid
---
libavformat/Makefile | 4 +--
libavformat/matroska.h| 9 ++
libavformat/matroskadec.c | 58 +--
libavformat/matroskaenc.c | 40 +++
4 files changed, 107 insertions(+), 4 deletions(-)
diff --git a/libavformat/Makefile b/libavformat/Makefile
index 2b5caf9d33..bd12562a24 100644
--- a/libavformat/Makefile
+++ b/libavformat/Makefile
@@ -314,11 +314,11 @@ OBJS-$(CONFIG_M4V_MUXER) += rawenc.o
OBJS-$(CONFIG_MATROSKA_DEMUXER) += matroskadec.o matroska.o \
flac_picture.o isom_tags.o
rmsipr.o \
oggparsevorbis.o vorbiscomment.o \
-qtpalette.o replaygain.o
+qtpalette.o replaygain.o
dovi_isom.o
OBJS-$(CONFIG_MATROSKA_MUXER)+= matroskaenc.o matroska.o \
av1.o avc.o hevc.o isom_tags.o \
flacenc_header.o avlanguage.o \
-vorbiscomment.o wv.o
+vorbiscomment.o wv.o dovi_isom.o
OBJS-$(CONFIG_MCA_DEMUXER) += mca.o
OBJS-$(CONFIG_MCC_DEMUXER) += mccdec.o subtitles.o
OBJS-$(CONFIG_MD5_MUXER) += hashenc.o
diff --git a/libavformat/matroska.h b/libavformat/matroska.h
index 2d04a6838b..16491aae22 100644
--- a/libavformat/matroska.h
+++ b/libavformat/matroska.h
@@ -111,6 +111,7 @@
#define MATROSKA_ID_TRACKCONTENTENCODING 0x6240
#define MATROSKA_ID_TRACKTIMECODESCALE 0x23314F
#define MATROSKA_ID_TRACKMAXBLKADDID 0x55EE
+#define MATROSKA_ID_TRACKBLKADDMAPPING 0x41E4
/* IDs in the trackvideo master */
#define MATROSKA_ID_VIDEOFRAMERATE 0x2383E3
@@ -189,6 +190,12 @@
#define MATROSKA_ID_ENCODINGSIGKEYID 0x47E4
#define MATROSKA_ID_ENCODINGSIGNATURE 0x47E3
+/* IDs in the block addition mapping master */
+#define MATROSKA_ID_BLKADDIDVALUE 0x41F0
+#define MATROSKA_ID_BLKADDIDNAME 0x41A4
+#define MATROSKA_ID_BLKADDIDTYPE 0x41E7
+#define MATROSKA_ID_BLKADDIDEXTRADATA 0x41ED
+
/* ID in the cues master */
#define MATROSKA_ID_POINTENTRY 0xBB
@@ -385,4 +392,6 @@ extern const char * const
ff_matroska_video_stereo_plane[MATROSKA_VIDEO_STEREO_P
int ff_mkv_stereo3d_conv(AVStream *st, MatroskaVideoStereoModeType
stereo_mode);
+#define DVCC_DVVC_BLOCK_TYPE_NAME "Dolby Vision configuration"
+
#endif /* AVFORMAT_MATROSKA_H */
diff --git a/libavformat/matroskadec.c b/libavformat/matroskadec.c
index a4bbbe954e..6ce553205d 100644
--- a/libavformat/matroskadec.c
+++ b/libavformat/matroskadec.c
@@ -53,6 +53,7 @@
#include "avformat.h"
#include "avio_internal.h"
+#include "dovi_isom.h"
#include "internal.h"
#include "isom.h"
#include "matroska.h"
@@ -239,6 +240,13 @@ typedef struct MatroskaTrackOperation {
EbmlList combine_planes;
} MatroskaTrackOperation;
+typedef struct MatroskaBlockAdditionMapping {
+uint64_t value;
+char *name;
+uint64_t type;
+EbmlBin extradata;
+} MatroskaBlockAdditionMapping;
+
typedef struct MatroskaTrack {
uint64_t num;
uint64_t uid;
@@ -269,6 +277,7 @@ typedef struct MatroskaTrack {
int ms_compat;
int needs_decoding;
uint64_t max_block_additional_id;
+EbmlList block_addition_mappings;
uint32_t palette[AVPALETTE_COUNT];
int has_palette;
@@ -419,8 +428,8 @@ typedef struct MatroskaDemuxContext {
// incomplete type (6.7.2 in C90, 6.9.2 in C99).
// Removing the sizes breaks MSVC.
static EbmlSyntax ebml_syntax[3], matroska_segment[9],
matroska_track_video_color[15], matroska_track_video[19],
- matroska_track[32], matroska_track_encoding[6],
matroska_track_encodings[2],
- matroska_track_combine_planes[2],
matroska_track_operation[2], matroska_tracks[2],
+ matroska_track[33], matroska_track_encoding[6],
matroska_track_encodings[2],
+ matroska_track_combine_planes[2],
matroska_track_operation[2], matroska_block_addition_mapping[5],
matroska_tracks[2],
matroska_attachments[2], matroska_chapter_entry[9],
matroska_chapter[6], matroska_chapters[2],
matroska_index_entry[3], matroska_index[2], matroska_tag[3],
matroska_tags[2], matroska_seekhead[2],
matroska_blockadditions[2], matroska_blockgroup[8],
matroska_cluster_parsing[8];
@@ -570,6 +579,14 @@ static EbmlSyntax matroska_track_operation[] = {
CHILD_OF(matroska_track)
};
+static EbmlSyntax matroska_block_addition_mapping[] = {
[FFmpeg-devel] [PATCH v9 3/5] avformat/mov: Refactor mov_read_dvcc_dvvc to use ff_isom_parse_dvcc_dvvc
To avoid duplicating code. The implementation in dovi_isom is identical.
Signed-off-by: quietvoid
---
libavformat/mov.c | 50 +--
1 file changed, 9 insertions(+), 41 deletions(-)
diff --git a/libavformat/mov.c b/libavformat/mov.c
index 5c74d099da..b5f974fe05 100644
--- a/libavformat/mov.c
+++ b/libavformat/mov.c
@@ -55,6 +55,7 @@
#include "avformat.h"
#include "internal.h"
#include "avio_internal.h"
+#include "dovi_isom.h"
#include "riff.h"
#include "isom.h"
#include "libavcodec/get_bits.h"
@@ -7051,58 +7052,25 @@ static int mov_read_dmlp(MOVContext *c, AVIOContext
*pb, MOVAtom atom)
static int mov_read_dvcc_dvvc(MOVContext *c, AVIOContext *pb, MOVAtom atom)
{
AVStream *st;
-uint32_t buf;
-AVDOVIDecoderConfigurationRecord *dovi;
-size_t dovi_size;
+uint8_t buf[ISOM_DVCC_DVVC_SIZE + AV_INPUT_BUFFER_PADDING_SIZE];
int ret;
+int64_t read_size = atom.size;
if (c->fc->nb_streams < 1)
return 0;
st = c->fc->streams[c->fc->nb_streams-1];
-if ((uint64_t)atom.size > (1<<30) || atom.size < 4)
-return AVERROR_INVALIDDATA;
-
-dovi = av_dovi_alloc(&dovi_size);
-if (!dovi)
-return AVERROR(ENOMEM);
-
-dovi->dv_version_major = avio_r8(pb);
-dovi->dv_version_minor = avio_r8(pb);
-
-buf = avio_rb16(pb);
-dovi->dv_profile= (buf >> 9) & 0x7f;// 7 bits
-dovi->dv_level = (buf >> 3) & 0x3f;// 6 bits
-dovi->rpu_present_flag = (buf >> 2) & 0x01;// 1 bit
-dovi->el_present_flag = (buf >> 1) & 0x01;// 1 bit
-dovi->bl_present_flag = buf & 0x01;// 1 bit
-if (atom.size >= 24) { // 4 + 4 + 4 * 4
-buf = avio_r8(pb);
-dovi->dv_bl_signal_compatibility_id = (buf >> 4) & 0x0f; // 4 bits
-} else {
-// 0 stands for None
-// Dolby Vision V1.2.93 profiles and levels
-dovi->dv_bl_signal_compatibility_id = 0;
+// At most 24 bytes
+if (read_size > ISOM_DVCC_DVVC_SIZE) {
+read_size = ISOM_DVCC_DVVC_SIZE;
}
-ret = av_stream_add_side_data(st, AV_PKT_DATA_DOVI_CONF,
- (uint8_t *)dovi, dovi_size);
-if (ret < 0) {
-av_free(dovi);
+if ((ret = avio_read(pb, buf, read_size)) < 0)
return ret;
-}
-av_log(c, AV_LOG_TRACE, "DOVI in dvcC/dvvC/dvwC box, version: %d.%d,
profile: %d, level: %d, "
- "rpu flag: %d, el flag: %d, bl flag: %d, compatibility id: %d\n",
- dovi->dv_version_major, dovi->dv_version_minor,
- dovi->dv_profile, dovi->dv_level,
- dovi->rpu_present_flag,
- dovi->el_present_flag,
- dovi->bl_present_flag,
- dovi->dv_bl_signal_compatibility_id
-);
+read_size = ret;
-return 0;
+return ff_isom_parse_dvcc_dvvc(c->fc, st, buf, read_size);
}
static int mov_read_kind(MOVContext *c, AVIOContext *pb, MOVAtom atom)
--
2.34.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v9 4/5] avformat/movenc: Refactor mov_write_dvcc_dvvc_tag to use ff_isom_put_dvcc_dvvc
Improves code legibility by not using bit shifts.
Also avoids duplicating the dvcC/dvvC ISOM box writing code.
Signed-off-by: quietvoid
---
libavformat/movenc.c | 24 +++-
1 file changed, 7 insertions(+), 17 deletions(-)
diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index 38ff90833a..208378fc11 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -27,6 +27,7 @@
#include "movenc.h"
#include "avformat.h"
#include "avio_internal.h"
+#include "dovi_isom.h"
#include "riff.h"
#include "avio.h"
#include "isom.h"
@@ -1911,6 +1912,8 @@ static int mov_write_sv3d_tag(AVFormatContext *s,
AVIOContext *pb, AVSphericalMa
static int mov_write_dvcc_dvvc_tag(AVFormatContext *s, AVIOContext *pb,
AVDOVIDecoderConfigurationRecord *dovi)
{
+uint8_t buf[ISOM_DVCC_DVVC_SIZE];
+
avio_wb32(pb, 32); /* size = 8 + 24 */
if (dovi->dv_profile > 10)
ffio_wfourcc(pb, "dvwC");
@@ -1918,23 +1921,10 @@ static int mov_write_dvcc_dvvc_tag(AVFormatContext *s,
AVIOContext *pb, AVDOVIDe
ffio_wfourcc(pb, "dvvC");
else
ffio_wfourcc(pb, "dvcC");
-avio_w8(pb, dovi->dv_version_major);
-avio_w8(pb, dovi->dv_version_minor);
-avio_wb16(pb, (dovi->dv_profile << 9) | (dovi->dv_level << 3) |
- (dovi->rpu_present_flag << 2) | (dovi->el_present_flag << 1) |
- dovi->bl_present_flag);
-avio_wb32(pb, (dovi->dv_bl_signal_compatibility_id << 28) | 0);
-
-ffio_fill(pb, 0, 4 * 4); /* reserved */
-av_log(s, AV_LOG_DEBUG, "DOVI in %s box, version: %d.%d, profile: %d,
level: %d, "
- "rpu flag: %d, el flag: %d, bl flag: %d, compatibility id: %d\n",
- dovi->dv_profile > 10 ? "dvwC" : (dovi->dv_profile > 7 ? "dvvC" :
"dvcC"),
- dovi->dv_version_major, dovi->dv_version_minor,
- dovi->dv_profile, dovi->dv_level,
- dovi->rpu_present_flag,
- dovi->el_present_flag,
- dovi->bl_present_flag,
- dovi->dv_bl_signal_compatibility_id);
+
+ff_isom_put_dvcc_dvvc(s, buf, dovi);
+avio_write(pb, buf, sizeof(buf));
+
return 32; /* 8 + 24 */
}
--
2.34.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v9 5/5] fate/matroska: Add tests for reading/writing BlockAdditionMapping elements
Tests the parsing and writing of AVDOVIDecoderConfigurationRecord, when it is present as a Dolby Vision configuration block addition mapping. Signed-off-by: quietvoid --- The required regression test file is available here: https://0x0.st/-hWK.mkv Should be moved to fate-suite/mkv/dovi-p5.mkv It is a blank frame encoded with x265. --- tests/fate/matroska.mak | 9 + tests/ref/fate/matroska-dovi-config-profile5 | 13 ++ tests/ref/fate/matroska-dovi-write-config| 223 +++ 3 files changed, 245 insertions(+) create mode 100644 tests/ref/fate/matroska-dovi-config-profile5 create mode 100644 tests/ref/fate/matroska-dovi-write-config diff --git a/tests/fate/matroska.mak b/tests/fate/matroska.mak index e117a0f6a6..ec2a0607b2 100644 --- a/tests/fate/matroska.mak +++ b/tests/fate/matroska.mak @@ -138,6 +138,15 @@ FATE_MATROSKA_FFMPEG_FFPROBE-$(call ALLYES, FILE_PROTOCOL WEBVTT_DEMUXER \ += fate-webm-webvtt-remux fate-webm-webvtt-remux: CMD = transcode webvtt $(TARGET_SAMPLES)/sub/WebVTT_capability_tester.vtt webm "-map 0 -map 0 -map 0 -map 0 -c:s copy -disposition:0 original+descriptions+hearing_impaired -disposition:1 lyrics+default+metadata -disposition:2 comment+forced -disposition:3 karaoke+captions+dub" "-map 0:0 -map 0:1 -c copy" "" "-show_entries stream_disposition:stream=index,codec_name:packet=stream_index,pts:packet_side_data_list -show_data_hash CRC32" +FATE_MATROSKA_FFMPEG_FFPROBE-$(call ALLYES, MATROSKA_DEMUXER) += fate-matroska-dovi-config-profile5 +fate-matroska-dovi-config-profile5: CMD = run ffprobe$(PROGSSUF)$(EXESUF) -show_entries stream_side_data_list -select_streams v -v 0 $(TARGET_SAMPLES)/mkv/dovi-p5.mkv + +FATE_MATROSKA_FFMPEG_FFPROBE-$(call ALLYES, FILE_PROTOCOL PIPE_PROTOCOL \ +MOV_DEMUXER MATROSKA_MUXER \ +FRAMECRC_MUXER) \ + += fate-matroska-dovi-write-config +fate-matroska-dovi-write-config: CMD = transcode mov $(TARGET_SAMPLES)/hevc/dv84.mov matroska "-c:v copy" "-map 0 -c copy" "" "-show_entries stream_side_data_list -select_streams v -v 0" + FATE_SAMPLES_AVCONV += $(FATE_MATROSKA-yes) FATE_SAMPLES_FFPROBE += $(FATE_MATROSKA_FFPROBE-yes) FATE_SAMPLES_FFMPEG_FFPROBE += $(FATE_MATROSKA_FFMPEG_FFPROBE-yes) diff --git a/tests/ref/fate/matroska-dovi-config-profile5 b/tests/ref/fate/matroska-dovi-config-profile5 new file mode 100644 index 00..a27976b71a --- /dev/null +++ b/tests/ref/fate/matroska-dovi-config-profile5 @@ -0,0 +1,13 @@ +[STREAM] +[SIDE_DATA] +side_data_type=DOVI configuration record +dv_version_major=1 +dv_version_minor=0 +dv_profile=5 +dv_level=4 +rpu_present_flag=1 +el_present_flag=0 +bl_present_flag=1 +dv_bl_signal_compatibility_id=0 +[/SIDE_DATA] +[/STREAM] diff --git a/tests/ref/fate/matroska-dovi-write-config b/tests/ref/fate/matroska-dovi-write-config new file mode 100644 index 00..a253db9f00 --- /dev/null +++ b/tests/ref/fate/matroska-dovi-write-config @@ -0,0 +1,223 @@ +47d2c151ff02720fff7bd37b3028097e *tests/data/fate/matroska-dovi-write-config.matroska +3618445 tests/data/fate/matroska-dovi-write-config.matroska +#extradata 0: 551, 0xa18acf66 +#tb 0: 1/1000 +#media_type 0: video +#codec_id 0: hevc +#dimensions 0: 1920x1080 +#sar 0: 0/1 +#tb 1: 1/1000 +#media_type 1: audio +#codec_id 1: ac3 +#sample_rate 1: 44100 +#channel_layout 1: 3 +#channel_layout_name 1: stereo +0,-67, 0, 33,63375, 0xc76606ab, S=1,8 +0,-34,133, 33,46706, 0x0e08a7e5, F=0x0 +0, 0, 73, 33,29766, 0x753c031a, F=0x0 +1, 0, 0, 34, 834, 0x6740ac04 +1, 35, 35, 34, 836, 0xe29a9a24 +0, 39, 39, 33,19409, 0x4b948b6c, F=0x0 +1, 70, 70, 34, 836, 0xf7329e5f +0, 73,106, 33,21086, 0x1b9412ce, F=0x0 +1,105,105, 34, 836, 0x9622a243 +0,106,273, 33,62043, 0xc2356b56, F=0x0 +0,133,206, 33,36175, 0x0a7df38c, F=0x0 +1,140,140, 34, 836, 0xb2d497c5 +0,173,173, 33,16028, 0xa57fcbe9, F=0x0 +1,174,174, 34, 836, 0x17c8980e +0,206,239, 33,15428, 0x9a91f357, F=0x0 +1,209,209, 34, 836, 0xfe288a7d +0,239,406, 33,66072, 0xa542b6d7, F=0x0 +1,244,244, 34, 836, 0x539e82b1 +0,273,339, 33,34985, 0xbfd8ff45, F=0x0 +1,279,279, 34, 836, 0x166291cb +0,306,306, 33,16036, 0xfc39c6ea, F=0x0 +1,314,314, 34, 836, 0x30127c33 +0,339,373, 33,19893, 0x7e746f4e, F=0x0 +1,348,348,
Re: [FFmpeg-devel] [PATCH v20 10/20] avfilter/avfilter: Handle subtitle frames
Soft Works:
> Signed-off-by: softworkz
> ---
> doc/APIchanges | 3 +++
> libavfilter/avfilter.c | 8 +---
> libavfilter/avfilter.h | 11 +++
> libavfilter/avfiltergraph.c | 5 +
> libavfilter/formats.c | 22 ++
> libavfilter/formats.h | 3 +++
> libavfilter/internal.h | 18 +++---
> 7 files changed, 64 insertions(+), 6 deletions(-)
>
> diff --git a/doc/APIchanges b/doc/APIchanges
> index 83ac64c3f1..f695efe9ab 100644
> --- a/doc/APIchanges
> +++ b/doc/APIchanges
> @@ -14,6 +14,9 @@ libavutil: 2021-04-27
>
> API changes, most recent first:
>
> +2021-12-05 - xx - lavc 9.1.100 - avfilter.h
> + Add fields subs_list and sub_fmt to AVFilter struct
> +
This is not public API, so don't make an APIchanges entry.
> 2021-12-05 - xx - lavc 60.1.100 - avcodec.h
>Deprecate avcodec_encode_subtitle(), use regular encode api now
>
> diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
> index df5b8f483c..75d5e86539 100644
> --- a/libavfilter/avfilter.c
> +++ b/libavfilter/avfilter.c
> @@ -56,7 +56,8 @@ static void tlog_ref(void *ctx, AVFrame *ref, int end)
> ref->linesize[0], ref->linesize[1], ref->linesize[2],
> ref->linesize[3],
> ref->pts, ref->pkt_pos);
>
> -if (ref->width) {
> +switch(ref->type) {
> +case AVMEDIA_TYPE_VIDEO:
> ff_tlog(ctx, " a:%d/%d s:%dx%d i:%c iskey:%d type:%c",
> ref->sample_aspect_ratio.num, ref->sample_aspect_ratio.den,
> ref->width, ref->height,
> @@ -64,12 +65,13 @@ static void tlog_ref(void *ctx, AVFrame *ref, int end)
> ref->top_field_first ? 'T' : 'B',/* Top / Bottom */
> ref->key_frame,
> av_get_picture_type_char(ref->pict_type));
> -}
> -if (ref->nb_samples) {
> +break;
> +case AVMEDIA_TYPE_AUDIO:
> ff_tlog(ctx, " cl:%"PRId64"d n:%d r:%d",
> ref->channel_layout,
> ref->nb_samples,
> ref->sample_rate);
> +break;
> }
>
> ff_tlog(ctx, "]%s", end ? "\n" : "");
> diff --git a/libavfilter/avfilter.h b/libavfilter/avfilter.h
> index b105dc3159..9f917deb41 100644
> --- a/libavfilter/avfilter.h
> +++ b/libavfilter/avfilter.h
> @@ -45,6 +45,7 @@
> #include "libavutil/log.h"
> #include "libavutil/samplefmt.h"
> #include "libavutil/pixfmt.h"
> +#include "libavutil/subfmt.h"
> #include "libavutil/rational.h"
>
> #include "libavfilter/version.h"
> @@ -343,6 +344,12 @@ typedef struct AVFilter {
> * and outputs use the same sample rate and channel count/layout.
> */
> const enum AVSampleFormat *samples_list;
> +/**
> + * Analogous to pixels, but delimited by AV_SUBTITLE_FMT_NONE
> + * and restricted to filters that only have AVMEDIA_TYPE_SUBTITLE
> + * inputs and outputs.
> + */
> +const enum AVSubtitleType *subs_list;
> /**
> * Equivalent to { pix_fmt, AV_PIX_FMT_NONE } as pixels_list.
> */
> @@ -351,6 +358,10 @@ typedef struct AVFilter {
> * Equivalent to { sample_fmt, AV_SAMPLE_FMT_NONE } as samples_list.
> */
> enum AVSampleFormat sample_fmt;
> +/**
> + * Equivalent to { sub_fmt, AV_SUBTITLE_FMT_NONE } as subs_list.
> + */
> +enum AVSubtitleType sub_fmt;
> } formats;
>
> int priv_size; ///< size of private data to allocate for the filter
> diff --git a/libavfilter/avfiltergraph.c b/libavfilter/avfiltergraph.c
> index b8b432e98b..f4987654af 100644
> --- a/libavfilter/avfiltergraph.c
> +++ b/libavfilter/avfiltergraph.c
> @@ -311,6 +311,8 @@ static int filter_link_check_formats(void *log,
> AVFilterLink *link, AVFilterForm
> return ret;
> break;
>
> +case AVMEDIA_TYPE_SUBTITLE:
> +return 0;
> default:
> av_assert0(!"reached");
> }
> @@ -441,6 +443,9 @@ static int query_formats(AVFilterGraph *graph, void
> *log_ctx)
> if (!link)
> continue;
>
> +if (link->type == AVMEDIA_TYPE_SUBTITLE)
> +continue;
> +
> neg = ff_filter_get_negotiation(link);
> av_assert0(neg);
> for (neg_step = 1; neg_step < neg->nb_mergers; neg_step++) {
> diff --git a/libavfilter/formats.c b/libavfilter/formats.c
> index ba62f73248..46dbbd2975 100644
> --- a/libavfilter/formats.c
> +++ b/libavfilter/formats.c
> @@ -19,6 +19,7 @@
> * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301
> USA
> */
>
> +#include "libavcodec/avcodec.h"
libavutil/subfmt.h, I presume?
> #include "libavutil/avassert.h"
> #include "libavutil/channel_layout.h"
> #include "libavutil/common.h"
> @@ -431,6 +432,12 @@ int ff_add_channel_layout(AVFilterChannelLayouts **l,
> uint64_t channel_layout)
> return 0;
> }
>
Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame for subtitle handling
> -Original Message-
> From: ffmpeg-devel On Behalf Of Lynne
> Sent: Sunday, December 5, 2021 5:40 PM
> To: FFmpeg development discussions and patches
> Subject: Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame
> for subtitle handling
>
> 5 Dec 2021, 17:23 by softwo...@hotmail.com:
>
> > @@ -491,6 +499,39 @@ typedef struct AVFrame {
> > */
> > uint64_t channel_layout;
> >
> > +/**
> > + * Display start time, relative to packet pts, in ms.
> > + */
> > +uint32_t subtitle_start_time;
> > +
> > +/**
> > + * Display end time, relative to packet pts, in ms.
> > + */
> > +uint32_t subtitle_end_time;
> >
>
> Milliseconds? Our entire API's based around timestamps
> with time bases. Plus, we all know what happened when
> Matroska settled onto milliseconds and ruined a perfectly
> complex but good container.
> Make this relative to the PTS field, with the same timebase
> as the PTS field.
> There's even a new AVFrame->time_base field for you to
> set so you wouldn't forget it.
The internal format for text subtitles is ASS, and this is
using a timebase of milliseconds.
All existing decoders and encoders are using this and I'm
afraid, but I will not go and change them all.
> > +/**
> > + * Number of items in the @ref subtitle_areas array.
> > + */
> > +unsigned num_subtitle_areas;
> > +
> > +/**
> > + * Array of subtitle areas, may be empty.
> > + */
> > +AVSubtitleArea **subtitle_areas;
> >
>
> There's no reason why this cannot be handled using the buffer
> and data fields. If you need more space there, you're free to bump
> the maximum number of pointers, plus this removes the horrid
> malloc(1) hack. We've discussed this, and I couldn't follow why
> this was bad in the email discussion you linked.
There are reasons. Some of those had I mentioned in an earlier
discussion with Hendrik.
The effort alone to relate the buffers to subtitle areas (which area
'owns' which buffer) is not reasonable. Too many cases to consider,
what's when there are 3 areas and the second area doesn't have any
buffer? The convention is that the buffer should be used contiguously.
Managing those relations is error-prone and would require a lot of code.
> > +/**
> > + * Usually the same as packet pts, in AV_TIME_BASE.
> > + *
> > + * @deprecated This is kept for compatibility reasons and corresponds
> to
> > + * AVSubtitle->pts. Might be removed in the future.
> > + */
> > +int64_t subtitle_pts;
> >
>
> I'm not going to accept a field which is instantly deprecated.
> As we've discussed multiple times, please merge this into
> the regular frame PTS field. We already have _2_ necessary
> stard/end fields.
--
> I agree with this entirely. Even ignoring the fact that adding a new
> field thats deprecated is instantly a disqualification, AVSubtitle had
> one pts field, AVFrame already has one pts field - both are even
> documented to have the same semantic. They should just contain the
> exact same data, thats how you achieve compatibility, not by claiming
> you need a new field for compatibility reasons.
>
> - Hendrik
I think the mistake is to declare subtitle_pts as deprecated. I had
added the deprecation at a very early point in time where I had still
thought that it can be eliminated.
Even though we are driving subtitle data through the graph attached
to AVFrame, the behavior involved is very different from audio and
video frames. Actually there's not one but many different ways how
subtitle data can appear in a source and travel through a filtergraph:
- Sometimes, subtitle events are muxed into a stream many seconds
ahead of display time. In this case, AVFrame.pts is the mux position
and AVFrame.subtitle_pts is the actual presentation time.
When filtering subtitles to modify something, it would be still desired
to retain the offset between mux time and display start
- Sometimes, subtitle events are occurring in the mux "live" - right in
the moment when they are meant to be shown. An example for this are
closed captions, and when extracting those via the new splitcc filter,
the subtitle_pts is equal to the frame.pts value.
But CC events do not come regularly, while downstream filters might
expect exactly that in order to proceed. That's why the splitcc filter
can emit subtitle frames at a certain framerate. It does so by re-
sending the most recent subtitle frame - basically not much different
than the main heatbeat mechanism (which cannot be used here because
the stream has it's origin inside the filtegraph (secondary output
of splitcc).
Now when splitcc sends such a repeated frame, it needs to adjust the
frame's pts in order to match the configured output framerate.
But that's for keeping the graph running, the actual display start
time (subtitle_pts) must not be changed.
It depends then on the downstream filters how to handle those frames.
For example, a f
Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame for subtitle handling
> -Original Message- > From: ffmpeg-devel On Behalf Of Hendrik > Leppkes > Sent: Sunday, December 5, 2021 5:45 PM > To: FFmpeg development discussions and patches > Subject: Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame > for subtitle handling > > On Sun, Dec 5, 2021 at 5:40 PM Lynne wrote: > > > +/** > > > + * Usually the same as packet pts, in AV_TIME_BASE. > > > + * > > > + * @deprecated This is kept for compatibility reasons and > corresponds to > > > + * AVSubtitle->pts. Might be removed in the future. > > > + */ > > > +int64_t subtitle_pts; > > > > > > > I'm not going to accept a field which is instantly deprecated. > > As we've discussed multiple times, please merge this into > > the regular frame PTS field. We already have _2_ necessary > > stard/end fields. > > > > I agree with this entirely. Even ignoring the fact that adding a new > field thats deprecated is instantly a disqualification, AVSubtitle had > one pts field, AVFrame already has one pts field The short answer to why it has worked before with a single pts field is: Because there was no filtering. Kind regards, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 3/6] avformat/rtsp: prefer to return EOF for incomplete read
On Sun, 5 Dec 2021, lance.lmw...@gmail.com wrote:
From: Limin Wang
Signed-off-by: Limin Wang
---
libavformat/rtsp.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/libavformat/rtsp.c b/libavformat/rtsp.c
index 5cffe0b..2ee2463 100644
--- a/libavformat/rtsp.c
+++ b/libavformat/rtsp.c
@@ -1247,7 +1247,7 @@ start:
return AVERROR(ENOMEM);
if (ffurl_read_complete(rt->rtsp_hd, content, content_length) !=
content_length) {
av_freep(&content);
-return AVERROR(EIO);
+return AVERROR_EOF;
This seems wrong, ffurl_read_complete can return error as well and you are
transforming it to AVERROR_EOF. Also if request content_length is not
honored, then it is surely an error, so AVERROR_EOF is not the correct
return value, because the protocol was violated.
Regards,
Marton
}
content[content_length] = '\0';
}
--
1.8.3.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame for subtitle handling
> -Original Message-
> From: ffmpeg-devel On Behalf Of Lynne
> Sent: Sunday, December 5, 2021 5:40 PM
> To: FFmpeg development discussions and patches
> Subject: Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame
> for subtitle handling
>
> 5 Dec 2021, 17:23 by softwo...@hotmail.com:
>
> > @@ -491,6 +499,39 @@ typedef struct AVFrame {
> > */
> > uint64_t channel_layout;
> >
> > +/**
> > + * Display start time, relative to packet pts, in ms.
> > + */
> > +uint32_t subtitle_start_time;
> > +
> > +/**
> > + * Display end time, relative to packet pts, in ms.
> > + */
> > +uint32_t subtitle_end_time;
> >
>
> Milliseconds? Our entire API's based around timestamps
> with time bases. Plus, we all know what happened when
> Matroska settled onto milliseconds and ruined a perfectly
> complex but good container.
> Make this relative to the PTS field, with the same timebase
> as the PTS field.
> There's even a new AVFrame->time_base field for you to
> set so you wouldn't forget it.
>
>
> > +/**
> > + * Number of items in the @ref subtitle_areas array.
> > + */
> > +unsigned num_subtitle_areas;
> > +
> > +/**
> > + * Array of subtitle areas, may be empty.
> > + */
> > +AVSubtitleArea **subtitle_areas;
> >
>
> There's no reason why this cannot be handled using the buffer
> and data fields. If you need more space there, you're free to bump
> the maximum number of pointers, plus this removes the horrid
> malloc(1) hack. We've discussed this, and I couldn't follow why
> this was bad in the email discussion you linked.
Hi Lynne,
let me add another note on these two topics.
One of the reasons why this subtitle filtering patchset has proceeded
rather smoothly so far, causing only a small amount of regressions
that were easy to find and fix, is that I have kept as much as possible
from the current AVSubtitle implementation, projecting it to the
new frame-based subtitle implementation, while keeping it mostly
compatible with the current logic.
Interestingly, an earlier attempt towards subtitle filtering has
gotten stuck right at the point about storing subtitle data in
AVFrame buffers. I decided to focus on functionality rather than
academic kind of details. What has been in place and working
for such a long time can't be so wrong that it can't be accepted.
We also need to consider that it's not my patchset that is deliberately
deciding to store data in pointers associate with rects/area and
to store start- and end-times in milliseconds: it's all the decoders
and encoders that are doing this. I'm just adapting to the status
quo.
These two things - changing the timebase of the ms-fields and
changing the location of the buffer pointers - is a gigantic task
that would require to change everything that this patchset covers
and implements - plus even more code that this patchset hasn't even
touched. It would require changes to everything everywhere where
subtitles are concerned.
It is a task that is risky as hell, will cause plenty of
regressions under guarantee, require extensive testing and might even
drive this whole endeavor into a dead end.
As I said before, please understand that I'm not ready to enter that
path.
Thanks,
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v21 00/20] Subtitle Filtering
New in V21 - Rebased. Patchset was broken again. - Don't declare AVFrame.subtitle_pts as deprecated and clarify doc text - Remove AVFilter.subs_list and AVFilter.sub_fmt from APIchanges - Change include in formats.c New in V20 - Rebased. V19 didn't apply cleanly anymore New in V19 - Document API changes in all relevant commits - Move enum AVSubtitleType: Improved commit message - Put deprecated enum values under removal guards - Document AV_SUBTITLE_FMT_NB - Fixed all trailing whitespace - splitcc: Add scatter_realtime_output option - splitcc: Emit empty subtitle frames instead of repeating - New commit: Replace deprecated enum values New in V18 - rebased to latest head (to fix merge conflict in allfilters.c) - textmod: Removed two trailing blanks from sf_textmod.c - xsubdec: fix xsubdec regression found by Michael; packet size estimation as flawed - splitcc: Fix uninitialzed locals - splitcc: Do not propagate hwcontext on secondary output - testsub2video: fix output timeing, re-add mutex locking and implement render_latest_only parameter New in V17 - Applied almost all suggestions that were made (thanks everybody!) - Split the initial commit, no longer moving the legacy structs (AVSubtitle..) - Use the regular encoding API for subtitles - Updated subtitle encoders to handle packet allocation - Removed compatibility layer for accessing legacy encoders - Updated the compatibility implementation for the legacy subtitle encode api - Reordered commits and merged ffmpeg.c changes into a single commit New in V16 - Added missing reference to subfmt.h iun libzvbi-teletextdec.c - Fixed Fate error in patch 15/16 - Removed all trsiling whitespace in tilers.texi New in V15 - Rebased to upstream changes - avcodec/subtitles: Migrate subtitle encoders to frame-based API and provide compatibility shim for legacy api - fftools/ffmpeg: Use new frame-based subtitle encoding API - AVSubtitleArea: copy flags field, make params const - graphicsubs2text: Don't emit duplicate frames - graphicsubs2text: Combined OCR output into a single AVSubtitleArea (I have a prototype for detecting text colors and positions, but it's not ready at this point) - splitcc: cleanup local subtitle_header ref - stripstyles: add parameter for ass layer selection - avcodec/subtitles: deferred loading of ass header for text subtitle encoders - verified all example command lines in the docs are working, added somre more Kind regards, softworkz softworkz (20): avcodec,avutil: Move enum AVSubtitleType to avutil, add new and deprecate old values avutil/frame: Prepare AVFrame for subtitle handling avcodec/subtitles: Introduce new frame-based subtitle decoding API avfilter/subtitles: Update vf_subtitles to use new decoding api avcodec,avutil: Move ass helper functions to avutil as avpriv_ and extend ass dialog parsing avcodec/subtitles: Migrate subtitle encoders to frame-based API and provide a compatibility shim for the legacy api avcodec/subtitles: Replace deprecated enum values fftools/play,probe: Adjust for subtitle changes avfilter/subtitles: Add subtitles.c for subtitle frame allocation avfilter/avfilter: Handle subtitle frames avfilter/sbuffer: Add sbuffersrc and sbuffersink filters avfilter/overlaygraphicsubs: Add overlaygraphicsubs and graphicsub2video filters fftools/ffmpeg: Replace sub2video with subtitle frame filtering and use new frame-based subtitle encoding API avfilter/avfilter: Fix hardcoded input index avfilter/overlaytextsubs: Add overlaytextsubs and textsubs2video filters avfilter/textmod: Add textmod, censor and show_speaker filters avfilter/stripstyles: Add stripstyles filter avfilter/splitcc: Add splitcc filter for closed caption handling avfilter/graphicsub2text: Add new graphicsub2text filter (OCR) avfilter/subscale: Add filter for scaling and/or re-arranging graphical subtitles configure | 7 +- doc/APIchanges| 26 + doc/filters.texi | 756 +++ fftools/ffmpeg.c | 584 ++-- fftools/ffmpeg.h | 15 +- fftools/ffmpeg_filter.c | 217 +++-- fftools/ffmpeg_hw.c | 2 +- fftools/ffmpeg_opt.c | 3 +- fftools/ffplay.c | 102 +- fftools/ffprobe.c | 48 +- libavcodec/Makefile | 56 +- libavcodec/ass.h | 147 +-- libavcodec/assdec.c | 4 +- libavcodec/assenc.c | 92 +- libavcodec/avcodec.h | 32 +- libavcodec/ccaption_dec.c | 19 +- libavcodec/codec_desc.c | 11 + libavcodec/codec_desc.h | 8 + libavcodec/decode.c |
[FFmpeg-devel] [PATCH v21 01/20] avcodec, avutil: Move enum AVSubtitleType to avutil, add new and deprecate old values
Signed-off-by: softworkz
---
doc/APIchanges | 6
libavcodec/avcodec.h | 19 +
libavutil/Makefile | 1 +
libavutil/subfmt.h | 66
libavutil/version.h | 1 +
5 files changed, 75 insertions(+), 18 deletions(-)
create mode 100644 libavutil/subfmt.h
diff --git a/doc/APIchanges b/doc/APIchanges
index 2914ad6734..a466a2ec22 100644
--- a/doc/APIchanges
+++ b/doc/APIchanges
@@ -14,6 +14,12 @@ libavutil: 2021-04-27
API changes, most recent first:
+2021-12-05 - xx - lavu 58.1.100 - subfmt.h
+ Add enum AVSubtitleType (moved from lavc), add new values, deprecate existing
+
+2021-12-05 - xx - lavc 60.1.100 - avcodec.h
+ Remove enum AVSubtitleType (moved to lavu)
+
2021-11-xx - xx - lavfi 8.19.100 - avfilter.h
Add AVFILTER_FLAG_METADATA_ONLY.
diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 7ee8bc2b7c..b05c12e47e 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -35,6 +35,7 @@
#include "libavutil/frame.h"
#include "libavutil/log.h"
#include "libavutil/pixfmt.h"
+#include "libavutil/subfmt.h"
#include "libavutil/rational.h"
#include "codec.h"
@@ -2238,24 +2239,6 @@ typedef struct AVHWAccel {
* @}
*/
-enum AVSubtitleType {
-SUBTITLE_NONE,
-
-SUBTITLE_BITMAP,///< A bitmap, pict will be set
-
-/**
- * Plain text, the text field must be set by the decoder and is
- * authoritative. ass and pict fields may contain approximations.
- */
-SUBTITLE_TEXT,
-
-/**
- * Formatted text, the ass field must be set by the decoder and is
- * authoritative. pict and text fields may contain approximations.
- */
-SUBTITLE_ASS,
-};
-
#define AV_SUBTITLE_FLAG_FORCED 0x0001
typedef struct AVSubtitleRect {
diff --git a/libavutil/Makefile b/libavutil/Makefile
index 529046dbc8..c7843db1e4 100644
--- a/libavutil/Makefile
+++ b/libavutil/Makefile
@@ -74,6 +74,7 @@ HEADERS = adler32.h
\
sha512.h \
spherical.h \
stereo3d.h\
+ subfmt.h \
threadmessage.h \
time.h\
timecode.h\
diff --git a/libavutil/subfmt.h b/libavutil/subfmt.h
new file mode 100644
index 00..88078a0d14
--- /dev/null
+++ b/libavutil/subfmt.h
@@ -0,0 +1,66 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#ifndef AVUTIL_SUBFMT_H
+#define AVUTIL_SUBFMT_H
+
+enum AVSubtitleType {
+
+/**
+ * Subtitle format unknown.
+ */
+AV_SUBTITLE_FMT_NONE = -1,
+
+/**
+ * Subtitle format unknown.
+ */
+AV_SUBTITLE_FMT_UNKNOWN = 0,
+#ifdef FF_API_OLD_SUBTITLES
+SUBTITLE_NONE = 0, ///< Deprecated, use AV_SUBTITLE_FMT_NONE
instead.
+#endif
+
+/**
+ * Bitmap area in AVSubtitleRect.data, pixfmt AV_PIX_FMT_PAL8.
+ */
+AV_SUBTITLE_FMT_BITMAP = 1,
+#ifdef FF_API_OLD_SUBTITLES
+SUBTITLE_BITMAP = 1,///< Deprecated, use AV_SUBTITLE_FMT_BITMAP
instead.
+#endif
+
+/**
+ * Plain text in AVSubtitleRect.text.
+ */
+AV_SUBTITLE_FMT_TEXT = 2,
+#ifdef FF_API_OLD_SUBTITLES
+SUBTITLE_TEXT = 2, ///< Deprecated, use AV_SUBTITLE_FMT_TEXT
instead.
+#endif
+
+/**
+ * Text Formatted as per ASS specification, contained AVSubtitleRect.ass.
+ */
+AV_SUBTITLE_FMT_ASS = 3,
+#ifdef FF_API_OLD_SUBTITLES
+SUBTITLE_ASS = 3, ///< Deprecated, use AV_SUBTITLE_FMT_ASS
instead.
+#endif
+
+AV_SUBTITLE_FMT_NB, ///< number of subtitle formats, DO NOT USE
THIS if you want to link with shared libav* because the number of formats might
differ between versions.
+};
+
+#endif /* AVUTIL_SUBFMT_H */
diff --git a/libavutil/version.h b/libavutil/version.h
index 017fc277a6..c6cd0b0d79 100644
--- a/libavutil/version.h
+++ b/libavutil/ve
[FFmpeg-devel] [PATCH v21 02/20] avutil/frame: Prepare AVFrame for subtitle handling
Root commit for adding subtitle filtering capabilities.
In detail:
- Add type (AVMediaType) field to AVFrame
Replaces previous way of distinction which was based on checking
width and height to determine whether a frame is audio or video
- Add subtitle fields to AVFrame
- Add new struct AVSubtitleArea, similar to AVSubtitleRect, but different
allocation logic. Cannot and must not be used interchangeably, hence
the new struct
Signed-off-by: softworkz
---
doc/APIchanges | 10 +++
libavutil/Makefile | 1 +
libavutil/frame.c | 211 -
libavutil/frame.h | 77 -
libavutil/subfmt.c | 50 +++
libavutil/subfmt.h | 48 +++
6 files changed, 373 insertions(+), 24 deletions(-)
create mode 100644 libavutil/subfmt.c
diff --git a/doc/APIchanges b/doc/APIchanges
index a466a2ec22..8c63ea0311 100644
--- a/doc/APIchanges
+++ b/doc/APIchanges
@@ -14,6 +14,16 @@ libavutil: 2021-04-27
API changes, most recent first:
+2021-12-05 - xx - lavu 58.1.100 - frame.h
+ Add AVMediaType field to AVFrame
+ Add Fields for carrying subtitle data to AVFrame
+ (subtitle_areas, subtitle_header, subtitle_pts, start/end time, etc.)
+ Add av_frame_get_buffer2() and deprecate av_frame_get_buffer()
+
+2021-12-05 - xx - lavu 58.1.100 - subfmt.h
+ Add struct AVSubtitleArea (replaces AVSubtitle)
+ Add av_get_subtitle_fmt_name() and av_get_subtitle_fmt()
+
2021-12-05 - xx - lavu 58.1.100 - subfmt.h
Add enum AVSubtitleType (moved from lavc), add new values, deprecate existing
diff --git a/libavutil/Makefile b/libavutil/Makefile
index c7843db1e4..7e79936876 100644
--- a/libavutil/Makefile
+++ b/libavutil/Makefile
@@ -160,6 +160,7 @@ OBJS = adler32.o
\
slicethread.o\
spherical.o \
stereo3d.o \
+ subfmt.o \
threadmessage.o \
time.o \
timecode.o \
diff --git a/libavutil/frame.c b/libavutil/frame.c
index 0912ad9131..62d29ae7d6 100644
--- a/libavutil/frame.c
+++ b/libavutil/frame.c
@@ -26,6 +26,7 @@
#include "imgutils.h"
#include "mem.h"
#include "samplefmt.h"
+#include "subfmt.h"
#include "hwcontext.h"
#define CHECK_CHANNELS_CONSISTENCY(frame) \
@@ -50,6 +51,9 @@ const char *av_get_colorspace_name(enum AVColorSpace val)
return name[val];
}
#endif
+
+static int frame_copy_subtitles(AVFrame *dst, const AVFrame *src, int
copy_data);
+
static void get_frame_defaults(AVFrame *frame)
{
memset(frame, 0, sizeof(*frame));
@@ -70,7 +74,12 @@ static void get_frame_defaults(AVFrame *frame)
frame->colorspace = AVCOL_SPC_UNSPECIFIED;
frame->color_range = AVCOL_RANGE_UNSPECIFIED;
frame->chroma_location = AVCHROMA_LOC_UNSPECIFIED;
-frame->flags = 0;
+frame->subtitle_start_time = 0;
+frame->subtitle_end_time = 0;
+frame->num_subtitle_areas = 0;
+frame->subtitle_areas = NULL;
+frame->subtitle_pts= 0;
+frame->subtitle_header = NULL;
}
static void free_side_data(AVFrameSideData **ptr_sd)
@@ -240,23 +249,55 @@ static int get_audio_buffer(AVFrame *frame, int align)
}
+static int get_subtitle_buffer(AVFrame *frame)
+{
+// Buffers in AVFrame->buf[] are not used in case of subtitle frames.
+// To accomodate with existing code, checking ->buf[0] to determine
+// whether a frame is ref-counted or has data, we're adding a 1-byte
+// buffer here, which marks the subtitle frame to contain data.
+frame->buf[0] = av_buffer_alloc(1);
+if (!frame->buf[0]) {
+av_frame_unref(frame);
+return AVERROR(ENOMEM);
+}
+
+frame->extended_data = frame->data;
+
+return 0;
+}
+
int av_frame_get_buffer(AVFrame *frame, int align)
+{
+if (frame->width > 0 && frame->height > 0)
+frame->type = AVMEDIA_TYPE_VIDEO;
+else if (frame->nb_samples > 0 && (frame->channel_layout ||
frame->channels > 0))
+frame->type = AVMEDIA_TYPE_AUDIO;
+
+return av_frame_get_buffer2(frame, align);
+}
+
+int av_frame_get_buffer2(AVFrame *frame, int align)
{
if (frame->format < 0)
return AVERROR(EINVAL);
-if (frame->width > 0 && frame->height > 0)
+switch(frame->type) {
+case AVMEDIA_TYPE_VIDEO:
return get_video_buffer(frame, align);
-else if (frame->nb_samples > 0 && (frame->channel_layout ||
frame->channels > 0))
+case AVMEDIA_TYPE_AUDIO:
return get_audio_buffer(frame, align);
-
-return AVERROR(EINVAL);
+case AVMEDIA_TYPE_SUBTITLE:
+retur
[FFmpeg-devel] [PATCH v21 05/20] avcodec, avutil: Move ass helper functions to avutil as avpriv_ and extend ass dialog parsing
Signed-off-by: softworkz
---
libavcodec/Makefile | 56 +++
libavcodec/ass.h | 147 ++
libavcodec/assdec.c | 2 +-
libavcodec/assenc.c | 2 +-
libavcodec/ccaption_dec.c | 19 +--
libavcodec/jacosubdec.c | 2 +-
libavcodec/libaribb24.c | 2 +-
libavcodec/libzvbi-teletextdec.c | 14 +-
libavcodec/microdvddec.c | 7 +-
libavcodec/movtextdec.c | 3 +-
libavcodec/movtextenc.c | 20 +--
libavcodec/mpl2dec.c | 2 +-
libavcodec/realtextdec.c | 2 +-
libavcodec/samidec.c | 2 +-
libavcodec/srtdec.c | 2 +-
libavcodec/srtenc.c | 16 +-
libavcodec/subviewerdec.c | 2 +-
libavcodec/textdec.c | 4 +-
libavcodec/ttmlenc.c | 15 +-
libavcodec/webvttdec.c| 2 +-
libavcodec/webvttenc.c| 16 +-
libavutil/Makefile| 2 +
{libavcodec => libavutil}/ass.c | 87 ---
libavutil/ass_internal.h | 133
{libavcodec => libavutil}/ass_split.c | 30 ++--
.../ass_split_internal.h | 24 +--
26 files changed, 339 insertions(+), 274 deletions(-)
rename {libavcodec => libavutil}/ass.c (65%)
create mode 100644 libavutil/ass_internal.h
rename {libavcodec => libavutil}/ass_split.c (94%)
rename libavcodec/ass_split.h => libavutil/ass_split_internal.h (89%)
diff --git a/libavcodec/Makefile b/libavcodec/Makefile
index 4122a9b144..b12638da5e 100644
--- a/libavcodec/Makefile
+++ b/libavcodec/Makefile
@@ -209,10 +209,10 @@ OBJS-$(CONFIG_APNG_DECODER)+= png.o pngdec.o
pngdsp.o
OBJS-$(CONFIG_APNG_ENCODER)+= png.o pngenc.o
OBJS-$(CONFIG_ARBC_DECODER)+= arbc.o
OBJS-$(CONFIG_ARGO_DECODER)+= argo.o
-OBJS-$(CONFIG_SSA_DECODER) += assdec.o ass.o
-OBJS-$(CONFIG_SSA_ENCODER) += assenc.o ass.o
-OBJS-$(CONFIG_ASS_DECODER) += assdec.o ass.o
-OBJS-$(CONFIG_ASS_ENCODER) += assenc.o ass.o
+OBJS-$(CONFIG_SSA_DECODER) += assdec.o
+OBJS-$(CONFIG_SSA_ENCODER) += assenc.o
+OBJS-$(CONFIG_ASS_DECODER) += assdec.o
+OBJS-$(CONFIG_ASS_ENCODER) += assenc.o
OBJS-$(CONFIG_ASV1_DECODER)+= asvdec.o asv.o mpeg12data.o
OBJS-$(CONFIG_ASV1_ENCODER)+= asvenc.o asv.o mpeg12data.o
OBJS-$(CONFIG_ASV2_DECODER)+= asvdec.o asv.o mpeg12data.o
@@ -253,7 +253,7 @@ OBJS-$(CONFIG_BRENDER_PIX_DECODER) += brenderpix.o
OBJS-$(CONFIG_C93_DECODER) += c93.o
OBJS-$(CONFIG_CAVS_DECODER)+= cavs.o cavsdec.o cavsdsp.o \
cavsdata.o
-OBJS-$(CONFIG_CCAPTION_DECODER)+= ccaption_dec.o ass.o
+OBJS-$(CONFIG_CCAPTION_DECODER)+= ccaption_dec.o
OBJS-$(CONFIG_CDGRAPHICS_DECODER) += cdgraphics.o
OBJS-$(CONFIG_CDTOONS_DECODER) += cdtoons.o
OBJS-$(CONFIG_CDXL_DECODER)+= cdxl.o
@@ -425,7 +425,7 @@ OBJS-$(CONFIG_INTERPLAY_ACM_DECODER) += interplayacm.o
OBJS-$(CONFIG_INTERPLAY_DPCM_DECODER) += dpcm.o
OBJS-$(CONFIG_INTERPLAY_VIDEO_DECODER) += interplayvideo.o
OBJS-$(CONFIG_IPU_DECODER) += mpeg12dec.o mpeg12.o mpeg12data.o
-OBJS-$(CONFIG_JACOSUB_DECODER) += jacosubdec.o ass.o
+OBJS-$(CONFIG_JACOSUB_DECODER) += jacosubdec.o
OBJS-$(CONFIG_JPEG2000_ENCODER)+= j2kenc.o mqcenc.o mqc.o jpeg2000.o \
jpeg2000dwt.o
OBJS-$(CONFIG_JPEG2000_DECODER)+= jpeg2000dec.o jpeg2000.o
jpeg2000dsp.o \
@@ -447,7 +447,7 @@ OBJS-$(CONFIG_MAGICYUV_ENCODER)+= magicyuvenc.o
OBJS-$(CONFIG_MDEC_DECODER)+= mdec.o mpeg12.o mpeg12data.o
OBJS-$(CONFIG_METASOUND_DECODER) += metasound.o metasound_data.o \
twinvq.o
-OBJS-$(CONFIG_MICRODVD_DECODER)+= microdvddec.o ass.o
+OBJS-$(CONFIG_MICRODVD_DECODER)+= microdvddec.o
OBJS-$(CONFIG_MIMIC_DECODER) += mimic.o
OBJS-$(CONFIG_MJPEG_DECODER) += mjpegdec.o mjpegdec_common.o
OBJS-$(CONFIG_MJPEG_QSV_DECODER) += qsvdec.o
@@ -462,8 +462,8 @@ OBJS-$(CONFIG_MLP_ENCODER) += mlpenc.o mlp.o
OBJS-$(CONFIG_MMVIDEO_DECODER) += mmvideo.o
OBJS-$(CONFIG_MOBICLIP_DECODER)+= mobiclip.o
OBJS-$(CONFIG_MOTIONPIXELS_DECODER)+= motionpixels.o
-OBJS-$(CONFIG_MOVTEXT_DECODER) += movtextdec.o ass.o
-OBJS-$(CONFIG_MOVTEXT_ENCODER) += movtextenc.o ass_split.o
+OBJS-$(CONFIG_MOVTEXT_DECODER) += movtextdec.o
+OBJS-$(CONFIG_MOVTE
[FFmpeg-devel] [PATCH v21 06/20] avcodec/subtitles: Migrate subtitle encoders to frame-based API and provide a compatibility shim for the legacy api
Signed-off-by: softworkz
---
doc/APIchanges | 3 +
libavcodec/assenc.c| 90 +
libavcodec/avcodec.h | 5 +-
libavcodec/dvbsubenc.c | 96 +--
libavcodec/dvdsubenc.c | 100 +++--
libavcodec/encode.c| 63 -
libavcodec/movtextenc.c| 112 +++--
libavcodec/srtenc.c| 104 ++
libavcodec/tests/avcodec.c | 2 -
libavcodec/ttmlenc.c | 98
libavcodec/webvttenc.c | 82 +++
libavcodec/xsubenc.c | 87 +---
12 files changed, 587 insertions(+), 255 deletions(-)
diff --git a/doc/APIchanges b/doc/APIchanges
index 49f1a28f71..83ac64c3f1 100644
--- a/doc/APIchanges
+++ b/doc/APIchanges
@@ -14,6 +14,9 @@ libavutil: 2021-04-27
API changes, most recent first:
+2021-12-05 - xx - lavc 60.1.100 - avcodec.h
+ Deprecate avcodec_encode_subtitle(), use regular encode api now
+
2021-12-05 - xx - lavc 60.1.100 - codec_desc.h
Add avcodec_descriptor_get_subtitle_format()
diff --git a/libavcodec/assenc.c b/libavcodec/assenc.c
index b0e475834b..94601bba68 100644
--- a/libavcodec/assenc.c
+++ b/libavcodec/assenc.c
@@ -22,48 +22,92 @@
#include
#include "avcodec.h"
+#include "encode.h"
#include "libavutil/ass_internal.h"
#include "internal.h"
#include "libavutil/avstring.h"
#include "libavutil/internal.h"
#include "libavutil/mem.h"
+static void check_write_header(AVCodecContext* avctx, const AVFrame* frame)
+{
+if (avctx->extradata_size)
+return;
+
+if (frame->subtitle_header && frame->subtitle_header->size > 0) {
+const char* subtitle_header = (char*)frame->subtitle_header->data;
+avctx->extradata_size = strlen(subtitle_header);
+avctx->extradata = av_mallocz(frame->subtitle_header->size + 1);
+memcpy(avctx->extradata, subtitle_header, avctx->extradata_size);
+avctx->extradata[avctx->extradata_size] = 0;
+}
+
+if (!avctx->extradata_size) {
+const char* subtitle_header =
avpriv_ass_get_subtitle_header_default(0);
+if (!subtitle_header)
+return;
+
+avctx->extradata_size = strlen(subtitle_header);
+avctx->extradata = av_mallocz(avctx->extradata_size + 1);
+memcpy(avctx->extradata, subtitle_header, avctx->extradata_size);
+avctx->extradata[avctx->extradata_size] = 0;
+av_freep(&subtitle_header);
+}
+}
+
static av_cold int ass_encode_init(AVCodecContext *avctx)
{
-avctx->extradata = av_malloc(avctx->subtitle_header_size + 1);
-if (!avctx->extradata)
-return AVERROR(ENOMEM);
-memcpy(avctx->extradata, avctx->subtitle_header,
avctx->subtitle_header_size);
-avctx->extradata_size = avctx->subtitle_header_size;
-avctx->extradata[avctx->extradata_size] = 0;
+if (avctx->subtitle_header_size) {
+avctx->extradata = av_malloc(avctx->subtitle_header_size + 1);
+if (!avctx->extradata)
+return AVERROR(ENOMEM);
+memcpy(avctx->extradata, avctx->subtitle_header,
avctx->subtitle_header_size);
+avctx->extradata_size = avctx->subtitle_header_size;
+avctx->extradata[avctx->extradata_size] = 0;
+}
+
return 0;
}
-static int ass_encode_frame(AVCodecContext *avctx,
-unsigned char *buf, int bufsize,
-const AVSubtitle *sub)
+static int ass_encode_frame(AVCodecContext* avctx, AVPacket* avpkt,
+const AVFrame* frame, int* got_packet)
{
-int i, len, total_len = 0;
+int ret;
+size_t req_len = 0, total_len = 0;
+
+check_write_header(avctx, frame);
-for (i=0; inum_rects; i++) {
-const char *ass = sub->rects[i]->ass;
+for (unsigned i = 0; i < frame->num_subtitle_areas; i++) {
+const char *ass = frame->subtitle_areas[i]->ass;
-if (sub->rects[i]->type != SUBTITLE_ASS) {
-av_log(avctx, AV_LOG_ERROR, "Only SUBTITLE_ASS type supported.\n");
+if (frame->subtitle_areas[i]->type != AV_SUBTITLE_FMT_ASS) {
+av_log(avctx, AV_LOG_ERROR, "Only AV_SUBTITLE_FMT_ASS type
supported.\n");
return AVERROR(EINVAL);
}
-len = av_strlcpy(buf+total_len, ass, bufsize-total_len);
+if (ass)
+req_len += strlen(ass);
+}
-if (len > bufsize-total_len-1) {
-av_log(avctx, AV_LOG_ERROR, "Buffer too small for ASS event.\n");
-return AVERROR_BUFFER_TOO_SMALL;
-}
+ret = ff_get_encode_buffer(avctx, avpkt, req_len + 1, 0);
+if (ret < 0) {
+av_log(avctx, AV_LOG_ERROR, "Error getting output packet.\n");
+return ret;
+}
-total_len += len;
+for (unsigned i = 0; i < frame->num_subtitle_areas; i++) {
+
[FFmpeg-devel] [PATCH v21 07/20] avcodec/subtitles: Replace deprecated enum values
Signed-off-by: softworkz
---
libavcodec/ass.h | 2 +-
libavcodec/assdec.c| 2 +-
libavcodec/dvbsubdec.c | 2 +-
libavcodec/dvdsubdec.c | 2 +-
libavcodec/dvdsubenc.c | 2 +-
libavcodec/pgssubdec.c | 2 +-
libavcodec/xsubdec.c | 2 +-
7 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/libavcodec/ass.h b/libavcodec/ass.h
index 9c9cb01c8a..8e885b3ae2 100644
--- a/libavcodec/ass.h
+++ b/libavcodec/ass.h
@@ -82,7 +82,7 @@ static inline int avpriv_ass_add_rect(AVSubtitle *sub, const
char *dialog,
rects[sub->num_rects] = av_mallocz(sizeof(*rects[0]));
if (!rects[sub->num_rects])
return AVERROR(ENOMEM);
-rects[sub->num_rects]->type = SUBTITLE_ASS;
+rects[sub->num_rects]->type = AV_SUBTITLE_FMT_ASS;
ass_str = avpriv_ass_get_dialog(readorder, layer, style, speaker, dialog);
if (!ass_str)
return AVERROR(ENOMEM);
diff --git a/libavcodec/assdec.c b/libavcodec/assdec.c
index 7802a44e71..fd321e7004 100644
--- a/libavcodec/assdec.c
+++ b/libavcodec/assdec.c
@@ -54,7 +54,7 @@ static int ass_decode_frame(AVCodecContext *avctx, void
*data, int *got_sub_ptr,
if (!sub->rects[0])
return AVERROR(ENOMEM);
sub->num_rects = 1;
-sub->rects[0]->type = SUBTITLE_ASS;
+sub->rects[0]->type = AV_SUBTITLE_FMT_ASS;
sub->rects[0]->ass = av_strdup(avpkt->data);
if (!sub->rects[0]->ass)
return AVERROR(ENOMEM);
diff --git a/libavcodec/dvbsubdec.c b/libavcodec/dvbsubdec.c
index 81ccaf4c57..b13244c803 100644
--- a/libavcodec/dvbsubdec.c
+++ b/libavcodec/dvbsubdec.c
@@ -795,7 +795,7 @@ static int save_subtitle_set(AVCodecContext *avctx,
AVSubtitle *sub, int *got_ou
rect->w = region->width;
rect->h = region->height;
rect->nb_colors = (1 << region->depth);
-rect->type = SUBTITLE_BITMAP;
+rect->type = AV_SUBTITLE_FMT_BITMAP;
rect->linesize[0] = region->width;
clut = get_clut(ctx, region->clut);
diff --git a/libavcodec/dvdsubdec.c b/libavcodec/dvdsubdec.c
index 52259f0730..b39b3d1838 100644
--- a/libavcodec/dvdsubdec.c
+++ b/libavcodec/dvdsubdec.c
@@ -406,7 +406,7 @@ static int decode_dvd_subtitles(DVDSubContext *ctx,
AVSubtitle *sub_header,
sub_header->rects[0]->y = y1;
sub_header->rects[0]->w = w;
sub_header->rects[0]->h = h;
-sub_header->rects[0]->type = SUBTITLE_BITMAP;
+sub_header->rects[0]->type = AV_SUBTITLE_FMT_BITMAP;
sub_header->rects[0]->linesize[0] = w;
sub_header->rects[0]->flags = is_menu ?
AV_SUBTITLE_FLAG_FORCED : 0;
}
diff --git a/libavcodec/dvdsubenc.c b/libavcodec/dvdsubenc.c
index 4916410fc8..fbab88a992 100644
--- a/libavcodec/dvdsubenc.c
+++ b/libavcodec/dvdsubenc.c
@@ -273,7 +273,7 @@ static int encode_dvd_subtitles(AVCodecContext* avctx,
AVPacket* avpkt,
return AVERROR(EINVAL);
for (i = 0; i < rects; i++)
-if (frame->subtitle_areas[i]->type != SUBTITLE_BITMAP) {
+if (frame->subtitle_areas[i]->type != AV_SUBTITLE_FMT_BITMAP) {
av_log(avctx, AV_LOG_ERROR, "Bitmap subtitle required\n");
return AVERROR(EINVAL);
}
diff --git a/libavcodec/pgssubdec.c b/libavcodec/pgssubdec.c
index 388639a110..4829babb7c 100644
--- a/libavcodec/pgssubdec.c
+++ b/libavcodec/pgssubdec.c
@@ -539,7 +539,7 @@ static int display_end_segment(AVCodecContext *avctx, void
*data,
return AVERROR(ENOMEM);
}
sub->num_rects++;
-sub->rects[i]->type = SUBTITLE_BITMAP;
+sub->rects[i]->type = AV_SUBTITLE_FMT_BITMAP;
/* Process bitmap */
object = find_object(ctx->presentation.objects[i].id, &ctx->objects);
diff --git a/libavcodec/xsubdec.c b/libavcodec/xsubdec.c
index 979399bae6..5c9e65f737 100644
--- a/libavcodec/xsubdec.c
+++ b/libavcodec/xsubdec.c
@@ -107,7 +107,7 @@ static int decode_frame(AVCodecContext *avctx, void *data,
int *got_sub_ptr,
}
sub->rects[0]->x = x; sub->rects[0]->y = y;
sub->rects[0]->w = w; sub->rects[0]->h = h;
-sub->rects[0]->type = SUBTITLE_BITMAP;
+sub->rects[0]->type = AV_SUBTITLE_FMT_BITMAP;
sub->rects[0]->linesize[0] = w;
sub->rects[0]->data[0] = av_malloc(w * h);
sub->rects[0]->nb_colors = 4;
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v21 08/20] fftools/play, probe: Adjust for subtitle changes
Signed-off-by: softworkz
---
fftools/ffplay.c | 102 +-
fftools/ffprobe.c | 48 ++
2 files changed, 78 insertions(+), 72 deletions(-)
diff --git a/fftools/ffplay.c b/fftools/ffplay.c
index e7b20be76b..0af32888da 100644
--- a/fftools/ffplay.c
+++ b/fftools/ffplay.c
@@ -152,7 +152,6 @@ typedef struct Clock {
/* Common struct for handling all types of decoded data and allocated render
buffers. */
typedef struct Frame {
AVFrame *frame;
-AVSubtitle sub;
int serial;
double pts; /* presentation timestamp for the frame */
double duration; /* estimated duration of the frame */
@@ -586,7 +585,7 @@ static int decoder_init(Decoder *d, AVCodecContext *avctx,
PacketQueue *queue, S
return 0;
}
-static int decoder_decode_frame(Decoder *d, AVFrame *frame, AVSubtitle *sub) {
+static int decoder_decode_frame(Decoder *d, AVFrame *frame) {
int ret = AVERROR(EAGAIN);
for (;;) {
@@ -620,6 +619,9 @@ static int decoder_decode_frame(Decoder *d, AVFrame *frame,
AVSubtitle *sub) {
}
}
break;
+case AVMEDIA_TYPE_SUBTITLE:
+ret = avcodec_receive_frame(d->avctx, frame);
+break;
}
if (ret == AVERROR_EOF) {
d->finished = d->pkt_serial;
@@ -652,25 +654,11 @@ static int decoder_decode_frame(Decoder *d, AVFrame
*frame, AVSubtitle *sub) {
av_packet_unref(d->pkt);
} while (1);
-if (d->avctx->codec_type == AVMEDIA_TYPE_SUBTITLE) {
-int got_frame = 0;
-ret = avcodec_decode_subtitle2(d->avctx, sub, &got_frame, d->pkt);
-if (ret < 0) {
-ret = AVERROR(EAGAIN);
-} else {
-if (got_frame && !d->pkt->data) {
-d->packet_pending = 1;
-}
-ret = got_frame ? 0 : (d->pkt->data ? AVERROR(EAGAIN) :
AVERROR_EOF);
-}
-av_packet_unref(d->pkt);
+if (avcodec_send_packet(d->avctx, d->pkt) == AVERROR(EAGAIN)) {
+av_log(d->avctx, AV_LOG_ERROR, "Receive_frame and send_packet both
returned EAGAIN, which is an API violation.\n");
+d->packet_pending = 1;
} else {
-if (avcodec_send_packet(d->avctx, d->pkt) == AVERROR(EAGAIN)) {
-av_log(d->avctx, AV_LOG_ERROR, "Receive_frame and send_packet
both returned EAGAIN, which is an API violation.\n");
-d->packet_pending = 1;
-} else {
-av_packet_unref(d->pkt);
-}
+av_packet_unref(d->pkt);
}
}
}
@@ -683,7 +671,6 @@ static void decoder_destroy(Decoder *d) {
static void frame_queue_unref_item(Frame *vp)
{
av_frame_unref(vp->frame);
-avsubtitle_free(&vp->sub);
}
static int frame_queue_init(FrameQueue *f, PacketQueue *pktq, int max_size,
int keep_last)
@@ -981,7 +968,7 @@ static void video_image_display(VideoState *is)
if (frame_queue_nb_remaining(&is->subpq) > 0) {
sp = frame_queue_peek(&is->subpq);
-if (vp->pts >= sp->pts + ((float) sp->sub.start_display_time /
1000)) {
+if (vp->pts >= sp->pts + ((float) sp->frame->subtitle_start_time /
1000)) {
if (!sp->uploaded) {
uint8_t* pixels[4];
int pitch[4];
@@ -993,25 +980,27 @@ static void video_image_display(VideoState *is)
if (realloc_texture(&is->sub_texture,
SDL_PIXELFORMAT_ARGB, sp->width, sp->height, SDL_BLENDMODE_BLEND, 1) < 0)
return;
-for (i = 0; i < sp->sub.num_rects; i++) {
-AVSubtitleRect *sub_rect = sp->sub.rects[i];
+for (i = 0; i < sp->frame->num_subtitle_areas; i++) {
+AVSubtitleArea *area = sp->frame->subtitle_areas[i];
+SDL_Rect sdl_rect = { .x = area->x, .y = area->y, .w =
area->w, .h = area->h };
-sub_rect->x = av_clip(sub_rect->x, 0, sp->width );
-sub_rect->y = av_clip(sub_rect->y, 0, sp->height);
-sub_rect->w = av_clip(sub_rect->w, 0, sp->width -
sub_rect->x);
-sub_rect->h = av_clip(sub_rect->h, 0, sp->height -
sub_rect->y);
+area->x = av_clip(area->x, 0, sp->width );
+area->y = av_clip(area->y, 0, sp->height);
+area->w = av_clip(area->w, 0, sp->width - area->x);
+area->h = av_clip(area->h, 0, sp->height - area->y);
is->sub_convert_ctx =
sws_getCachedContext(is->sub_convert_ctx,
-sub_rect->w, sub_rect->h, AV_PIX_FMT_PAL8,
-sub_rect->w, sub_rec
[FFmpeg-devel] [PATCH v21 03/20] avcodec/subtitles: Introduce new frame-based subtitle decoding API
- Add avcodec_decode_subtitle3 which takes subtitle frames,
serving as compatibility shim to legacy subtitle decoding
- Add additional methods for conversion between old and new API
Signed-off-by: softworkz
---
doc/APIchanges | 7 ++
libavcodec/avcodec.h| 8 +-
libavcodec/codec_desc.c | 11 +++
libavcodec/codec_desc.h | 8 ++
libavcodec/decode.c | 54 ++--
libavcodec/internal.h | 16
libavcodec/utils.c | 182
7 files changed, 277 insertions(+), 9 deletions(-)
diff --git a/doc/APIchanges b/doc/APIchanges
index 8c63ea0311..49f1a28f71 100644
--- a/doc/APIchanges
+++ b/doc/APIchanges
@@ -14,6 +14,13 @@ libavutil: 2021-04-27
API changes, most recent first:
+2021-12-05 - xx - lavc 60.1.100 - codec_desc.h
+ Add avcodec_descriptor_get_subtitle_format()
+
+2021-12-05 - xx - lavc 60.1.100 - avcodec.h
+ Deprecate avsubtitle_free()
+ Deprecate avcodec_decode_subtitle2(), use regular decode api now
+
2021-12-05 - xx - lavu 58.1.100 - frame.h
Add AVMediaType field to AVFrame
Add Fields for carrying subtitle data to AVFrame
diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index b05c12e47e..3e734d3003 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -1675,7 +1675,7 @@ typedef struct AVCodecContext {
/**
* Header containing style information for text subtitles.
- * For SUBTITLE_ASS subtitle type, it should contain the whole ASS
+ * For AV_SUBTITLE_FMT_ASS subtitle type, it should contain the whole ASS
* [Script Info] and [V4+ Styles] section, plus the [Events] line and
* the Format line following. It shouldn't include any Dialogue line.
* - encoding: Set/allocated/freed by user (before avcodec_open2())
@@ -2417,7 +2417,10 @@ int avcodec_close(AVCodecContext *avctx);
* Free all allocated data in the given subtitle struct.
*
* @param sub AVSubtitle to free.
+ *
+ * @deprecated Use the regular frame based encode and decode APIs instead.
*/
+attribute_deprecated
void avsubtitle_free(AVSubtitle *sub);
/**
@@ -2510,7 +2513,10 @@ enum AVChromaLocation avcodec_chroma_pos_to_enum(int
xpos, int ypos);
* must be freed with avsubtitle_free if *got_sub_ptr is set.
* @param[in,out] got_sub_ptr Zero if no subtitle could be decompressed,
otherwise, it is nonzero.
* @param[in] avpkt The input AVPacket containing the input buffer.
+ *
+ * @deprecated Use the new decode API (avcodec_send_packet,
avcodec_receive_frame) instead.
*/
+attribute_deprecated
int avcodec_decode_subtitle2(AVCodecContext *avctx, AVSubtitle *sub,
int *got_sub_ptr,
AVPacket *avpkt);
diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c
index 0974ee03de..e48e4532ba 100644
--- a/libavcodec/codec_desc.c
+++ b/libavcodec/codec_desc.c
@@ -3548,3 +3548,14 @@ enum AVMediaType avcodec_get_type(enum AVCodecID
codec_id)
const AVCodecDescriptor *desc = avcodec_descriptor_get(codec_id);
return desc ? desc->type : AVMEDIA_TYPE_UNKNOWN;
}
+
+enum AVSubtitleType avcodec_descriptor_get_subtitle_format(const
AVCodecDescriptor *codec_descriptor)
+{
+if(codec_descriptor->props & AV_CODEC_PROP_BITMAP_SUB)
+return AV_SUBTITLE_FMT_BITMAP;
+
+if(codec_descriptor->props & AV_CODEC_PROP_TEXT_SUB)
+return AV_SUBTITLE_FMT_ASS;
+
+return AV_SUBTITLE_FMT_UNKNOWN;
+}
diff --git a/libavcodec/codec_desc.h b/libavcodec/codec_desc.h
index 126b52df47..ba68d24e0e 100644
--- a/libavcodec/codec_desc.h
+++ b/libavcodec/codec_desc.h
@@ -121,6 +121,14 @@ const AVCodecDescriptor *avcodec_descriptor_next(const
AVCodecDescriptor *prev);
*/
const AVCodecDescriptor *avcodec_descriptor_get_by_name(const char *name);
+/**
+ * Return subtitle format from a codec descriptor
+ *
+ * @param codec_descriptor codec descriptor
+ * @return the subtitle type (e.g. bitmap, text)
+ */
+enum AVSubtitleType avcodec_descriptor_get_subtitle_format(const
AVCodecDescriptor *codec_descriptor);
+
/**
* @}
*/
diff --git a/libavcodec/decode.c b/libavcodec/decode.c
index 52bf5dcd33..ac267f0df6 100644
--- a/libavcodec/decode.c
+++ b/libavcodec/decode.c
@@ -576,6 +576,37 @@ static int decode_receive_frame_internal(AVCodecContext
*avctx, AVFrame *frame)
return ret;
}
+static int decode_subtitle2_priv(AVCodecContext *avctx, AVSubtitle *sub,
+ int *got_sub_ptr, AVPacket *avpkt);
+
+static int decode_subtitle_shim(AVCodecContext *avctx, AVFrame *frame,
AVPacket *avpkt)
+{
+int ret, got_sub_ptr = 0;
+AVSubtitle subtitle = { 0 };
+
+if (frame->buf[0])
+return AVERROR(EAGAIN);
+
+av_frame_unref(frame);
+
+ret = decode_subtitle2_priv(avctx, &subtitle, &got_sub_ptr, avpkt);
+
+if (ret >= 0 && got_sub_ptr) {
+frame->type = AVMEDIA_TYPE_SUBTITLE;
+frame->format = subtitle.format;
+ret = a
[FFmpeg-devel] [PATCH v21 09/20] avfilter/subtitles: Add subtitles.c for subtitle frame allocation
Analog to avfilter/video.c and avfilter/audio.c
Signed-off-by: softworkz
---
libavfilter/Makefile| 1 +
libavfilter/avfilter.c | 4 +++
libavfilter/internal.h | 1 +
libavfilter/subtitles.c | 63 +
libavfilter/subtitles.h | 44
5 files changed, 113 insertions(+)
create mode 100644 libavfilter/subtitles.c
create mode 100644 libavfilter/subtitles.h
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index c8082c4a2f..664566a18a 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -19,6 +19,7 @@ OBJS = allfilters.o
\
framequeue.o \
graphdump.o \
graphparser.o\
+ subtitles.o \
video.o \
OBJS-$(HAVE_THREADS) += pthread.o
diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
index 7362bcdab5..df5b8f483c 100644
--- a/libavfilter/avfilter.c
+++ b/libavfilter/avfilter.c
@@ -43,6 +43,7 @@
#include "formats.h"
#include "framepool.h"
#include "internal.h"
+#include "subtitles.h"
#include "libavutil/ffversion.h"
const char av_filter_ffversion[] = "FFmpeg version " FFMPEG_VERSION;
@@ -1475,6 +1476,9 @@ int ff_inlink_make_frame_writable(AVFilterLink *link,
AVFrame **rframe)
case AVMEDIA_TYPE_AUDIO:
out = ff_get_audio_buffer(link, frame->nb_samples);
break;
+case AVMEDIA_TYPE_SUBTITLE:
+out = ff_get_subtitles_buffer(link, link->format);
+break;
default:
return AVERROR(EINVAL);
}
diff --git a/libavfilter/internal.h b/libavfilter/internal.h
index 1099b82b4b..fc09ef574c 100644
--- a/libavfilter/internal.h
+++ b/libavfilter/internal.h
@@ -90,6 +90,7 @@ struct AVFilterPad {
union {
AVFrame *(*video)(AVFilterLink *link, int w, int h);
AVFrame *(*audio)(AVFilterLink *link, int nb_samples);
+AVFrame *(*subtitle)(AVFilterLink *link, int format);
} get_buffer;
/**
diff --git a/libavfilter/subtitles.c b/libavfilter/subtitles.c
new file mode 100644
index 00..951bfd612c
--- /dev/null
+++ b/libavfilter/subtitles.c
@@ -0,0 +1,63 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "libavutil/common.h"
+
+#include "subtitles.h"
+#include "avfilter.h"
+#include "internal.h"
+
+
+AVFrame *ff_null_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+return ff_get_subtitles_buffer(link->dst->outputs[0], format);
+}
+
+AVFrame *ff_default_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+AVFrame *frame;
+
+frame = av_frame_alloc();
+if (!frame)
+return NULL;
+
+frame->format = format;
+frame->type = AVMEDIA_TYPE_SUBTITLE;
+
+if (av_frame_get_buffer2(frame, 0) < 0) {
+av_frame_free(&frame);
+return NULL;
+}
+
+return frame;
+}
+
+AVFrame *ff_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+AVFrame *ret = NULL;
+
+if (link->dstpad->get_buffer.subtitle)
+ret = link->dstpad->get_buffer.subtitle(link, format);
+
+if (!ret)
+ret = ff_default_get_subtitles_buffer(link, format);
+
+return ret;
+}
diff --git a/libavfilter/subtitles.h b/libavfilter/subtitles.h
new file mode 100644
index 00..4a9115126e
--- /dev/null
+++ b/libavfilter/subtitles.h
@@ -0,0 +1,44 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have recei
[FFmpeg-devel] [PATCH v21 10/20] avfilter/avfilter: Handle subtitle frames
Signed-off-by: softworkz
---
libavfilter/avfilter.c | 8 +---
libavfilter/avfilter.h | 11 +++
libavfilter/avfiltergraph.c | 5 +
libavfilter/formats.c | 22 ++
libavfilter/formats.h | 3 +++
libavfilter/internal.h | 18 +++---
6 files changed, 61 insertions(+), 6 deletions(-)
diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
index df5b8f483c..75d5e86539 100644
--- a/libavfilter/avfilter.c
+++ b/libavfilter/avfilter.c
@@ -56,7 +56,8 @@ static void tlog_ref(void *ctx, AVFrame *ref, int end)
ref->linesize[0], ref->linesize[1], ref->linesize[2],
ref->linesize[3],
ref->pts, ref->pkt_pos);
-if (ref->width) {
+switch(ref->type) {
+case AVMEDIA_TYPE_VIDEO:
ff_tlog(ctx, " a:%d/%d s:%dx%d i:%c iskey:%d type:%c",
ref->sample_aspect_ratio.num, ref->sample_aspect_ratio.den,
ref->width, ref->height,
@@ -64,12 +65,13 @@ static void tlog_ref(void *ctx, AVFrame *ref, int end)
ref->top_field_first ? 'T' : 'B',/* Top / Bottom */
ref->key_frame,
av_get_picture_type_char(ref->pict_type));
-}
-if (ref->nb_samples) {
+break;
+case AVMEDIA_TYPE_AUDIO:
ff_tlog(ctx, " cl:%"PRId64"d n:%d r:%d",
ref->channel_layout,
ref->nb_samples,
ref->sample_rate);
+break;
}
ff_tlog(ctx, "]%s", end ? "\n" : "");
diff --git a/libavfilter/avfilter.h b/libavfilter/avfilter.h
index b105dc3159..9f917deb41 100644
--- a/libavfilter/avfilter.h
+++ b/libavfilter/avfilter.h
@@ -45,6 +45,7 @@
#include "libavutil/log.h"
#include "libavutil/samplefmt.h"
#include "libavutil/pixfmt.h"
+#include "libavutil/subfmt.h"
#include "libavutil/rational.h"
#include "libavfilter/version.h"
@@ -343,6 +344,12 @@ typedef struct AVFilter {
* and outputs use the same sample rate and channel count/layout.
*/
const enum AVSampleFormat *samples_list;
+/**
+ * Analogous to pixels, but delimited by AV_SUBTITLE_FMT_NONE
+ * and restricted to filters that only have AVMEDIA_TYPE_SUBTITLE
+ * inputs and outputs.
+ */
+const enum AVSubtitleType *subs_list;
/**
* Equivalent to { pix_fmt, AV_PIX_FMT_NONE } as pixels_list.
*/
@@ -351,6 +358,10 @@ typedef struct AVFilter {
* Equivalent to { sample_fmt, AV_SAMPLE_FMT_NONE } as samples_list.
*/
enum AVSampleFormat sample_fmt;
+/**
+ * Equivalent to { sub_fmt, AV_SUBTITLE_FMT_NONE } as subs_list.
+ */
+enum AVSubtitleType sub_fmt;
} formats;
int priv_size; ///< size of private data to allocate for the filter
diff --git a/libavfilter/avfiltergraph.c b/libavfilter/avfiltergraph.c
index b8b432e98b..f4987654af 100644
--- a/libavfilter/avfiltergraph.c
+++ b/libavfilter/avfiltergraph.c
@@ -311,6 +311,8 @@ static int filter_link_check_formats(void *log,
AVFilterLink *link, AVFilterForm
return ret;
break;
+case AVMEDIA_TYPE_SUBTITLE:
+return 0;
default:
av_assert0(!"reached");
}
@@ -441,6 +443,9 @@ static int query_formats(AVFilterGraph *graph, void
*log_ctx)
if (!link)
continue;
+if (link->type == AVMEDIA_TYPE_SUBTITLE)
+continue;
+
neg = ff_filter_get_negotiation(link);
av_assert0(neg);
for (neg_step = 1; neg_step < neg->nb_mergers; neg_step++) {
diff --git a/libavfilter/formats.c b/libavfilter/formats.c
index ba62f73248..5c972bb183 100644
--- a/libavfilter/formats.c
+++ b/libavfilter/formats.c
@@ -19,6 +19,7 @@
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
+#include "libavutil/subfmt.h"
#include "libavutil/avassert.h"
#include "libavutil/channel_layout.h"
#include "libavutil/common.h"
@@ -431,6 +432,12 @@ int ff_add_channel_layout(AVFilterChannelLayouts **l,
uint64_t channel_layout)
return 0;
}
+int ff_add_subtitle_type(AVFilterFormats **avff, int64_t fmt)
+{
+ADD_FORMAT(avff, fmt, ff_formats_unref, int, formats, nb_formats);
+return 0;
+}
+
AVFilterFormats *ff_make_formats_list_singleton(int fmt)
{
int fmts[2] = { fmt, -1 };
@@ -450,6 +457,13 @@ AVFilterFormats *ff_all_formats(enum AVMediaType type)
return NULL;
fmt++;
}
+} else if (type == AVMEDIA_TYPE_SUBTITLE) {
+if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_BITMAP) < 0)
+return NULL;
+if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_ASS) < 0)
+return NULL;
+if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_TEXT) < 0)
+return NULL;
}
return ret;
@@ -724,6 +738,10 @@ int ff_default_query_formats(AVFilterContext *ctx)
type= AVMEDIA_TYPE_AUDIO
[FFmpeg-devel] [PATCH v21 04/20] avfilter/subtitles: Update vf_subtitles to use new decoding api
Signed-off-by: softworkz
---
libavfilter/vf_subtitles.c | 54 +-
1 file changed, 42 insertions(+), 12 deletions(-)
diff --git a/libavfilter/vf_subtitles.c b/libavfilter/vf_subtitles.c
index 377160c72b..a240cbd415 100644
--- a/libavfilter/vf_subtitles.c
+++ b/libavfilter/vf_subtitles.c
@@ -35,14 +35,12 @@
# include "libavformat/avformat.h"
#endif
#include "libavutil/avstring.h"
-#include "libavutil/imgutils.h"
#include "libavutil/opt.h"
#include "libavutil/parseutils.h"
#include "drawutils.h"
#include "avfilter.h"
#include "internal.h"
#include "formats.h"
-#include "video.h"
typedef struct AssContext {
const AVClass *class;
@@ -292,6 +290,29 @@ static int attachment_is_font(AVStream * st)
return 0;
}
+static int decode(AVCodecContext *avctx, AVFrame *frame, int *got_frame,
AVPacket *pkt)
+{
+int ret;
+
+*got_frame = 0;
+
+if (pkt) {
+ret = avcodec_send_packet(avctx, pkt);
+// In particular, we don't expect AVERROR(EAGAIN), because we read all
+// decoded frames with avcodec_receive_frame() until done.
+if (ret < 0 && ret != AVERROR_EOF)
+return ret;
+}
+
+ret = avcodec_receive_frame(avctx, frame);
+if (ret < 0 && ret != AVERROR(EAGAIN))
+return ret;
+if (ret >= 0)
+*got_frame = 1;
+
+return 0;
+}
+
AVFILTER_DEFINE_CLASS(subtitles);
static av_cold int init_subtitles(AVFilterContext *ctx)
@@ -306,6 +327,7 @@ static av_cold int init_subtitles(AVFilterContext *ctx)
AVStream *st;
AVPacket pkt;
AssContext *ass = ctx->priv;
+enum AVSubtitleType subtitle_format;
/* Init libass */
ret = init(ctx);
@@ -386,13 +408,17 @@ static av_cold int init_subtitles(AVFilterContext *ctx)
ret = AVERROR_DECODER_NOT_FOUND;
goto end;
}
+
dec_desc = avcodec_descriptor_get(st->codecpar->codec_id);
-if (dec_desc && !(dec_desc->props & AV_CODEC_PROP_TEXT_SUB)) {
+subtitle_format = avcodec_descriptor_get_subtitle_format(dec_desc);
+
+if (subtitle_format != AV_SUBTITLE_FMT_ASS) {
av_log(ctx, AV_LOG_ERROR,
- "Only text based subtitles are currently supported\n");
-ret = AVERROR_PATCHWELCOME;
+ "Only text based subtitles are supported by this filter\n");
+ret = AVERROR_INVALIDDATA;
goto end;
}
+
if (ass->charenc)
av_dict_set(&codec_opts, "sub_charenc", ass->charenc, 0);
@@ -448,18 +474,22 @@ static av_cold int init_subtitles(AVFilterContext *ctx)
dec_ctx->subtitle_header_size);
while (av_read_frame(fmt, &pkt) >= 0) {
int i, got_subtitle;
-AVSubtitle sub = {0};
+AVFrame *sub = av_frame_alloc();
+if (!sub) {
+ret = AVERROR(ENOMEM);
+goto end;
+}
if (pkt.stream_index == sid) {
-ret = avcodec_decode_subtitle2(dec_ctx, &sub, &got_subtitle, &pkt);
+ret = decode(dec_ctx, sub, &got_subtitle, &pkt);
if (ret < 0) {
av_log(ctx, AV_LOG_WARNING, "Error decoding: %s (ignored)\n",
av_err2str(ret));
} else if (got_subtitle) {
-const int64_t start_time = av_rescale_q(sub.pts,
AV_TIME_BASE_Q, av_make_q(1, 1000));
-const int64_t duration = sub.end_display_time;
-for (i = 0; i < sub.num_rects; i++) {
-char *ass_line = sub.rects[i]->ass;
+const int64_t start_time = av_rescale_q(sub->subtitle_pts,
AV_TIME_BASE_Q, av_make_q(1, 1000));
+const int64_t duration = sub->subtitle_end_time;
+for (i = 0; i < sub->num_subtitle_areas; i++) {
+char *ass_line = sub->subtitle_areas[i]->ass;
if (!ass_line)
break;
ass_process_chunk(ass->track, ass_line, strlen(ass_line),
@@ -468,7 +498,7 @@ static av_cold int init_subtitles(AVFilterContext *ctx)
}
}
av_packet_unref(&pkt);
-avsubtitle_free(&sub);
+av_frame_free(&sub);
}
end:
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v21 11/20] avfilter/sbuffer: Add sbuffersrc and sbuffersink filters
Signed-off-by: softworkz
---
configure| 2 +-
libavfilter/allfilters.c | 2 ++
libavfilter/buffersink.c | 54 ++
libavfilter/buffersink.h | 7
libavfilter/buffersrc.c | 72
libavfilter/buffersrc.h | 1 +
6 files changed, 137 insertions(+), 1 deletion(-)
diff --git a/configure b/configure
index a98a18abaa..ea4c722a94 100755
--- a/configure
+++ b/configure
@@ -7808,7 +7808,7 @@ print_enabled_components(){
fi
done
if [ "$name" = "filter_list" ]; then
-for c in asrc_abuffer vsrc_buffer asink_abuffer vsink_buffer; do
+for c in asrc_abuffer vsrc_buffer ssrc_sbuffer asink_abuffer
vsink_buffer ssink_sbuffer; do
printf "&ff_%s,\n" $c >> $TMPH
done
fi
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index b1af2cbcc8..e134ac8059 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -553,8 +553,10 @@ extern const AVFilter ff_avsrc_movie;
* being the same while having different 'types'). */
extern const AVFilter ff_asrc_abuffer;
extern const AVFilter ff_vsrc_buffer;
+extern const AVFilter ff_ssrc_sbuffer;
extern const AVFilter ff_asink_abuffer;
extern const AVFilter ff_vsink_buffer;
+extern const AVFilter ff_ssink_sbuffer;
extern const AVFilter ff_af_afifo;
extern const AVFilter ff_vf_fifo;
diff --git a/libavfilter/buffersink.c b/libavfilter/buffersink.c
index c0215669e7..0b268c2fa4 100644
--- a/libavfilter/buffersink.c
+++ b/libavfilter/buffersink.c
@@ -29,6 +29,8 @@
#include "libavutil/internal.h"
#include "libavutil/opt.h"
+#include "libavcodec/avcodec.h"
+
#define FF_INTERNAL_FIELDS 1
#include "framequeue.h"
@@ -57,6 +59,10 @@ typedef struct BufferSinkContext {
int *sample_rates; ///< list of accepted sample rates
int sample_rates_size;
+/* only used for subtitles */
+enum AVSubtitleType *subtitle_types; ///< list of accepted subtitle
types, must be terminated with -1
+int subtitle_types_size;
+
AVFrame *peeked_frame;
} BufferSinkContext;
@@ -305,6 +311,28 @@ static int asink_query_formats(AVFilterContext *ctx)
return 0;
}
+static int ssink_query_formats(AVFilterContext *ctx)
+{
+BufferSinkContext *buf = ctx->priv;
+AVFilterFormats *formats = NULL;
+unsigned i;
+int ret;
+
+CHECK_LIST_SIZE(subtitle_types)
+if (buf->subtitle_types_size) {
+for (i = 0; i < NB_ITEMS(buf->subtitle_types); i++)
+if ((ret = ff_add_subtitle_type(&formats, buf->subtitle_types[i]))
< 0)
+return ret;
+if ((ret = ff_set_common_formats(ctx, formats)) < 0)
+return ret;
+} else {
+if ((ret = ff_default_query_formats(ctx)) < 0)
+return ret;
+}
+
+return 0;
+}
+
#define OFFSET(x) offsetof(BufferSinkContext, x)
#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM
static const AVOption buffersink_options[] = {
@@ -322,9 +350,16 @@ static const AVOption abuffersink_options[] = {
{ NULL },
};
#undef FLAGS
+#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_SUBTITLE_PARAM
+static const AVOption sbuffersink_options[] = {
+{ "subtitle_types", "set the supported subtitle formats",
OFFSET(subtitle_types), AV_OPT_TYPE_BINARY, .flags = FLAGS },
+{ NULL },
+};
+#undef FLAGS
AVFILTER_DEFINE_CLASS(buffersink);
AVFILTER_DEFINE_CLASS(abuffersink);
+AVFILTER_DEFINE_CLASS(sbuffersink);
static const AVFilterPad avfilter_vsink_buffer_inputs[] = {
{
@@ -363,3 +398,22 @@ const AVFilter ff_asink_abuffer = {
.outputs = NULL,
FILTER_QUERY_FUNC(asink_query_formats),
};
+
+static const AVFilterPad avfilter_ssink_sbuffer_inputs[] = {
+{
+.name = "default",
+.type = AVMEDIA_TYPE_SUBTITLE,
+},
+};
+
+const AVFilter ff_ssink_sbuffer = {
+.name = "sbuffersink",
+.description = NULL_IF_CONFIG_SMALL("Buffer subtitle frames, and make
them available to the end of the filter graph."),
+.priv_class= &sbuffersink_class,
+.priv_size = sizeof(BufferSinkContext),
+.init = common_init,
+.activate = activate,
+FILTER_INPUTS(avfilter_ssink_sbuffer_inputs),
+.outputs = NULL,
+FILTER_QUERY_FUNC(ssink_query_formats),
+};
diff --git a/libavfilter/buffersink.h b/libavfilter/buffersink.h
index 69ed0f29a8..11905abdc5 100644
--- a/libavfilter/buffersink.h
+++ b/libavfilter/buffersink.h
@@ -129,6 +129,13 @@ typedef struct AVABufferSinkParams {
*/
attribute_deprecated
AVABufferSinkParams *av_abuffersink_params_alloc(void);
+
+/**
+ * Deprecated and unused struct to use for initializing an sbuffersink context.
+ */
+typedef struct AVSBufferSinkParams {
+const int *subtitle_type;
+} AVSBufferSinkParams;
#endif
/**
diff --git a/libavfilter/buffersrc.c b/libavfilter/buffersrc.c
index b0611872f1..d2362999a2 100644
--- a/libavfilter/buffersrc.
[FFmpeg-devel] [PATCH v21 12/20] avfilter/overlaygraphicsubs: Add overlaygraphicsubs and graphicsub2video filters
- overlaygraphicsubs (VS -> V)
Overlay graphic subtitles onto a video stream
- graphicsub2video {S -> V)
Converts graphic subtitles to video frames (with alpha)
Gets auto-inserted for retaining compatibility with
sub2video command lines
Signed-off-by: softworkz
---
doc/filters.texi| 118 +
libavfilter/Makefile| 2 +
libavfilter/allfilters.c| 2 +
libavfilter/vf_overlaygraphicsubs.c | 737
4 files changed, 859 insertions(+)
create mode 100644 libavfilter/vf_overlaygraphicsubs.c
diff --git a/doc/filters.texi b/doc/filters.texi
index 3edf3f50b0..669b9f5365 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25635,6 +25635,124 @@ tools.
@c man end VIDEO SINKS
+@chapter Subtitle Filters
+@c man begin SUBTITLE FILTERS
+
+When you configure your FFmpeg build, you can disable any of the
+existing filters using @code{--disable-filters}.
+
+Below is a description of the currently available subtitle filters.
+
+@section graphicsub2video
+
+Renders graphic subtitles as video frames.
+
+This filter replaces the previous "sub2video" hack which did the conversion
implicitly and up-front as subtitle filtering wasn't possible at that time.
+To retain compatibility with earlier sub2video command lines, this filter is
being auto-inserted in those cases.
+
+For overlaying graphicsal subtitles it is recommended to use the
'overlay_graphicsubs' filter which is more efficient and takes less processing
resources.
+
+This filter is still useful in cases where the overlay is done with hardware
acceleration (e.g. overlay_qsv, overlay_vaapi, overlay_cuda) for preparing the
overlay frames.
+
+Inputs:
+@itemize
+@item 0: Subtitles [BITMAP]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video [RGB32]
+@end itemize
+
+
+It accepts the following parameters:
+
+@table @option
+@item size, s
+Set the size of the output video frame.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Overlay PGS subtitles
+(not recommended - better use overlay_graphicsubs)
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:1]graphicsub2video[subs];[0:0][subs]overlay" output.mp4
+@end example
+
+@item
+Overlay PGS subtitles implicitly
+The graphicsub2video is inserted automatically for compatibility with legacy
command lines.
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:0][0:1]overlay" output.mp4
+@end example
+@end itemize
+
+@section overlaygraphicsubs
+
+Overlay graphic subtitles onto a video stream.
+
+This filter can blend graphical subtitles on a video stream directly, i.e.
without creating full-size alpha images first.
+The blending operation is limited to the area of the subtitle rectangles,
which also means that no processing is done at times where no subtitles are to
be displayed.
+
+Inputs:
+@itemize
+@item 0: Video [YUV420P, YUV422P, YUV444P, ARGB, RGBA, ABGR, BGRA, RGB24,
BGR24]
+@item 1: Subtitles [BITMAP]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video (same as input)
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+@item x
+@item y
+Set the expression for the x and y coordinates of the overlaid video
+on the main video. Default value is "0" for both expressions. In case
+the expression is invalid, it is set to a huge value (meaning that the
+overlay will not be displayed within the output visible area).
+
+@item eof_action
+See @ref{framesync}.
+
+@item eval
+Set when the expressions for @option{x}, and @option{y} are evaluated.
+
+It accepts the following values:
+@table @samp
+@item init
+only evaluate expressions once during the filter initialization or
+when a command is processed
+
+@item frame
+evaluate expressions for each incoming frame
+@end table
+
+Default value is @samp{frame}.
+
+@item shortest
+See @ref{framesync}.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Overlay PGS subtitles
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:0][0:1]overlaygraphicsubs" output.mp4
+@end example
+@end itemize
+@c man end SUBTITLE FILTERS
+
@chapter Multimedia Filters
@c man begin MULTIMEDIA FILTERS
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 664566a18a..525b3a6e3c 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -295,6 +295,7 @@ OBJS-$(CONFIG_GBLUR_FILTER) += vf_gblur.o
OBJS-$(CONFIG_GBLUR_VULKAN_FILTER) += vf_gblur_vulkan.o vulkan.o
vulkan_filter.o
OBJS-$(CONFIG_GEQ_FILTER)+= vf_geq.o
OBJS-$(CONFIG_GRADFUN_FILTER)+= vf_gradfun.o
+OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlaygraphicsubs.o
framesync.o
OBJS-$(CONFIG_GRAPHMONITOR_FILTER) += f_graphmonitor.o
OBJS-$(CONFIG_GRAYWORLD_FILTER)
[FFmpeg-devel] [PATCH v21 13/20] fftools/ffmpeg: Replace sub2video with subtitle frame filtering and use new frame-based subtitle encoding API
This commit actually enables subtitle filtering in ffmpeg by
sending and receiving subtitle frames to and from a filtergraph.
The heartbeat functionality from the previous sub2video implementation
is retained and applied to all subtitle frames (bitmap, text, ..).
The other part of sub2video functionality is retained by
auto-insertion of the new graphicsub2video filter.
Justification for changed test refs:
- sub2video
The new results are identical excepting the last frame which
is due to the implementation changes
- sub2video_basic
The previous results had some incorrect output because multiple
frames had the same dts
The non-empty content frames are visually identical, the different
CRC is due to the different blending algorithm that is being used.
- sub2video_time_limited
The third frame in the previous ref was a repetition, which doesn't
happen anymore with the new subtitle filtering.
- sub-dvb
Running ffprobe -show_frames on the source file shows that there
are 7 subtitle frames with 0 rects in the source at the start
and 2 at the end. This translates to the 14 and 4 additional
entries in the new test results.
- filter-overlay-dvdsub-2397
Overlay results have slightly different CRCs due to different
blending implementation
Signed-off-by: softworkz
---
fftools/ffmpeg.c | 584 +++---
fftools/ffmpeg.h | 15 +-
fftools/ffmpeg_filter.c | 217 +---
fftools/ffmpeg_hw.c | 2 +-
fftools/ffmpeg_opt.c | 3 +-
tests/ref/fate/filter-overlay-dvdsub-2397 | 181 ---
tests/ref/fate/sub-dvb| 162 +++---
tests/ref/fate/sub2video | 116 ++---
tests/ref/fate/sub2video_basic| 135 ++---
tests/ref/fate/sub2video_time_limited | 4 +-
10 files changed, 725 insertions(+), 694 deletions(-)
diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c
index 855db934bf..c424a09d00 100644
--- a/fftools/ffmpeg.c
+++ b/fftools/ffmpeg.c
@@ -143,8 +143,6 @@ static int want_sdp = 1;
static BenchmarkTimeStamps current_time;
AVIOContext *progress_avio = NULL;
-static uint8_t *subtitle_out;
-
InputStream **input_streams = NULL;
intnb_input_streams = 0;
InputFile **input_files = NULL;
@@ -169,163 +167,6 @@ static int restore_tty;
static void free_input_threads(void);
#endif
-/* sub2video hack:
- Convert subtitles to video with alpha to insert them in filter graphs.
- This is a temporary solution until libavfilter gets real subtitles support.
- */
-
-static int sub2video_get_blank_frame(InputStream *ist)
-{
-int ret;
-AVFrame *frame = ist->sub2video.frame;
-
-av_frame_unref(frame);
-ist->sub2video.frame->width = ist->dec_ctx->width ? ist->dec_ctx->width
: ist->sub2video.w;
-ist->sub2video.frame->height = ist->dec_ctx->height ? ist->dec_ctx->height
: ist->sub2video.h;
-ist->sub2video.frame->format = AV_PIX_FMT_RGB32;
-if ((ret = av_frame_get_buffer(frame, 0)) < 0)
-return ret;
-memset(frame->data[0], 0, frame->height * frame->linesize[0]);
-return 0;
-}
-
-static void sub2video_copy_rect(uint8_t *dst, int dst_linesize, int w, int h,
-AVSubtitleRect *r)
-{
-uint32_t *pal, *dst2;
-uint8_t *src, *src2;
-int x, y;
-
-if (r->type != SUBTITLE_BITMAP) {
-av_log(NULL, AV_LOG_WARNING, "sub2video: non-bitmap subtitle\n");
-return;
-}
-if (r->x < 0 || r->x + r->w > w || r->y < 0 || r->y + r->h > h) {
-av_log(NULL, AV_LOG_WARNING, "sub2video: rectangle (%d %d %d %d)
overflowing %d %d\n",
-r->x, r->y, r->w, r->h, w, h
-);
-return;
-}
-
-dst += r->y * dst_linesize + r->x * 4;
-src = r->data[0];
-pal = (uint32_t *)r->data[1];
-for (y = 0; y < r->h; y++) {
-dst2 = (uint32_t *)dst;
-src2 = src;
-for (x = 0; x < r->w; x++)
-*(dst2++) = pal[*(src2++)];
-dst += dst_linesize;
-src += r->linesize[0];
-}
-}
-
-static void sub2video_push_ref(InputStream *ist, int64_t pts)
-{
-AVFrame *frame = ist->sub2video.frame;
-int i;
-int ret;
-
-av_assert1(frame->data[0]);
-ist->sub2video.last_pts = frame->pts = pts;
-for (i = 0; i < ist->nb_filters; i++) {
-ret = av_buffersrc_add_frame_flags(ist->filters[i]->filter, frame,
- AV_BUFFERSRC_FLAG_KEEP_REF |
- AV_BUFFERSRC_FLAG_PUSH);
-if (ret != AVERROR_EOF && ret < 0)
-av_log(NULL, AV_LOG_WARNING, "Error while add the frame to buffer
source(%s).\n",
- av_err2str(ret));
-}
-}
-
-void sub2video_update(InputStream *ist, int64_t heartbeat_pts, AVSubtitle *sub)
-{
-AVFrame *frame = ist->sub2video.frame;
-int8_t *dst;
-int dst_linesize;
-int num_rects, i;
-int64_t pts, end_pts;
-
-
[FFmpeg-devel] [PATCH v21 14/20] avfilter/avfilter: Fix hardcoded input index
This fix targets (rare) cases where multiple input pads have a
.filter_frame function. ff_request_frame_to_filter needs
to call ff_request_frame with the correct input pad
instead of the hardcoded first one.
Signed-off-by: softworkz
---
libavfilter/avfilter.c | 18 +-
1 file changed, 13 insertions(+), 5 deletions(-)
diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
index 75d5e86539..aa9aa71f53 100644
--- a/libavfilter/avfilter.c
+++ b/libavfilter/avfilter.c
@@ -463,7 +463,7 @@ static int64_t guess_status_pts(AVFilterContext *ctx, int
status, AVRational lin
return AV_NOPTS_VALUE;
}
-static int ff_request_frame_to_filter(AVFilterLink *link)
+static int ff_request_frame_to_filter(AVFilterLink *link, int input_index)
{
int ret = -1;
@@ -472,8 +472,8 @@ static int ff_request_frame_to_filter(AVFilterLink *link)
link->frame_blocked_in = 1;
if (link->srcpad->request_frame)
ret = link->srcpad->request_frame(link);
-else if (link->src->inputs[0])
-ret = ff_request_frame(link->src->inputs[0]);
+else if (link->src->inputs[input_index])
+ret = ff_request_frame(link->src->inputs[input_index]);
if (ret < 0) {
if (ret != AVERROR(EAGAIN) && ret != link->status_in)
ff_avfilter_link_set_in_status(link, ret,
guess_status_pts(link->src, ret, link->time_base));
@@ -1172,6 +1172,14 @@ static int forward_status_change(AVFilterContext
*filter, AVFilterLink *in)
{
unsigned out = 0, progress = 0;
int ret;
+int input_index = 0;
+
+for (int i = 0; i < in->dst->nb_inputs; i++) {
+if (&in->dst->input_pads[i] == in->dstpad) {
+input_index = i;
+break;
+}
+}
av_assert0(!in->status_out);
if (!filter->nb_outputs) {
@@ -1181,7 +1189,7 @@ static int forward_status_change(AVFilterContext *filter,
AVFilterLink *in)
while (!in->status_out) {
if (!filter->outputs[out]->status_in) {
progress++;
-ret = ff_request_frame_to_filter(filter->outputs[out]);
+ret = ff_request_frame_to_filter(filter->outputs[out],
input_index);
if (ret < 0)
return ret;
}
@@ -1218,7 +1226,7 @@ static int ff_filter_activate_default(AVFilterContext
*filter)
for (i = 0; i < filter->nb_outputs; i++) {
if (filter->outputs[i]->frame_wanted_out &&
!filter->outputs[i]->frame_blocked_in) {
-return ff_request_frame_to_filter(filter->outputs[i]);
+return ff_request_frame_to_filter(filter->outputs[i], 0);
}
}
return FFERROR_NOT_READY;
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v21 15/20] avfilter/overlaytextsubs: Add overlaytextsubs and textsubs2video filters
- overlaytextsubs {VS -> V)
Overlay text subtitles onto a video stream.
- textsubs2video {S -> V)
Converts text subtitles to video frames
Signed-off-by: softworkz
---
configure| 2 +
doc/filters.texi | 113 ++
libavfilter/Makefile | 2 +
libavfilter/allfilters.c | 4 +-
libavfilter/vf_overlaytextsubs.c | 646 +++
5 files changed, 766 insertions(+), 1 deletion(-)
create mode 100644 libavfilter/vf_overlaytextsubs.c
diff --git a/configure b/configure
index ea4c722a94..549382a61f 100755
--- a/configure
+++ b/configure
@@ -3666,6 +3666,7 @@ overlay_opencl_filter_deps="opencl"
overlay_qsv_filter_deps="libmfx"
overlay_qsv_filter_select="qsvvpp"
overlay_vulkan_filter_deps="vulkan spirv_compiler"
+overlaytextsubs_filter_deps="avcodec libass"
owdenoise_filter_deps="gpl"
pad_opencl_filter_deps="opencl"
pan_filter_deps="swresample"
@@ -3710,6 +3711,7 @@ superequalizer_filter_deps="avcodec"
superequalizer_filter_select="rdft"
surround_filter_deps="avcodec"
surround_filter_select="rdft"
+textsub2video_filter_deps="avcodec libass"
tinterlace_filter_deps="gpl"
tinterlace_merge_test_deps="tinterlace_filter"
tinterlace_pad_test_deps="tinterlace_filter"
diff --git a/doc/filters.texi b/doc/filters.texi
index 669b9f5365..2898ee0140 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25751,6 +25751,119 @@ Overlay PGS subtitles
ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:0][0:1]overlaygraphicsubs" output.mp4
@end example
@end itemize
+
+@section overlaytextsubs
+
+Overlay text subtitles onto a video stream.
+
+This filter supersedes the classic @ref{subtitles} filter opposed to which it
does no longer require to open and access the source stream separately, which
is often causing problems or doesn't even work for non-local or slow sources.
+
+Inputs:
+@itemize
+@item 0: Video [YUV420P, YUV422P, YUV444P, ARGB, RGBA, ABGR, BGRA, RGB24,
BGR24]
+@item 1: Subtitles [TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video (same as input)
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+
+@item alpha
+Process alpha channel, by default alpha channel is untouched.
+
+@item fonts_dir
+Set a directory path containing fonts that can be used by the filter.
+These fonts will be used in addition to whatever the font provider uses.
+
+@item default_font_path
+Path to a font file to be used as the default font.
+
+@item font_size
+Set the default font size.
+
+@item fontconfig_file
+Path to ASS fontconfig configuration file.
+
+@item force_style
+Override default style or script info parameters of the subtitles. It accepts a
+string containing ASS style format @code{KEY=VALUE} couples separated by ",".
+
+@item margin
+Set the rendering margin in pixels.
+
+@item render_latest_only
+For rendering, alway use the latest event only, which is covering the given
point in time
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Overlay ASS subtitles with animations:
+@example
+ffmpeg -i
"http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv";
-filter_complex "[0:v]overlaytextsubs" -map 0 -y out.mkv
+@end example
+@end itemize
+
+@section textsub2video
+
+Converts text subtitles to video frames.
+
+For overlaying text subtitles onto video frames it is recommended to use the
overlay_textsubs filter.
+The textsub2video is useful for for creating transparent text-frames when
overlay is done via hw acceleration
+
+Inputs:
+@itemize
+@item 0: Subtitles [TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video [RGB32]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+
+@item rate, r
+Set the framerate for updating overlay frames.
+Normally, overlay frames will only be updated each time when the subtitles to
display are changing.
+In cases where subtitles include advanced features (like animation), this
parameter determines the frequency by which the overlay frames should be
updated.
+
+@item size, s
+Set the output frame size.
+Allows to override the size of output video frames.
+
+@item fonts_dir
+Set a directory path containing fonts that can be used by the filter.
+These fonts will be used in addition to whatever the font provider uses.
+
+@item default_font_path
+Path to a font file to be used as the default font.
+
+@item font_size
+Set the default font size.
+
+@item fontconfig_file
+Path to ASS fontconfig configuration file.
+
+@item force_style
+Override default style or script info parameters of the subtitles. It accepts a
+string containing ASS style format @code{KEY=VALUE} couples separated by ",".
+
+@item margin
+Set the rendering margin in pixels.
+
+@item render_latest_only
+For rendering, alway use the latest event only, which is covering the given
point in time.
+@end table
+
@c man end SUBTITLE FILTERS
@chapter Multimedia Filters
diff --git a/libavf
[FFmpeg-devel] [PATCH v21 16/20] avfilter/textmod: Add textmod, censor and show_speaker filters
- textmod {S -> S)
Modify subtitle text in a number of ways
- censor {S -> S)
Censor subtitles using a word list
- show_speaker {S -> S)
Prepend speaker names from ASS subtitles to the visible text lines
Signed-off-by: softworkz
---
doc/filters.texi | 206
libavfilter/Makefile | 5 +
libavfilter/allfilters.c | 3 +
libavfilter/sf_textmod.c | 697 +++
4 files changed, 911 insertions(+)
create mode 100644 libavfilter/sf_textmod.c
diff --git a/doc/filters.texi b/doc/filters.texi
index 2898ee0140..c53bf0c60d 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25643,6 +25643,145 @@ existing filters using @code{--disable-filters}.
Below is a description of the currently available subtitle filters.
+
+@section censor
+
+Censor selected words in text subtitles.
+
+Inputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+@item mode
+The censoring mode to apply.
+
+Supported censoring modes are:
+
+@table @var
+@item 0, keep_first_last
+Replace all characters with the 'censor_char' except the first and the last
character of a word.
+For words with less than 4 characters, the last character will be replaced as
well.
+For words with less than 3 characters, the first character will be replaced as
well.
+@item 1, keep_first
+Replace all characters with the 'censor_char' except the first character of a
word.
+For words with less than 3 characters, the first character will be replaced as
well.
+@item 2, all
+Replace all characters with the 'censor_char'.
+@end table
+
+@item words
+A list of words to censor, separated by 'separator'.
+
+@item words_file
+Specify a file from which to load the contents for the 'words' parameter.
+
+@item censor_char
+Single character used as replacement for censoring.
+
+@item separator
+Delimiter character for words. Used with replace_words and remove_words- Must
be a single character.
+The default is '.'.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Censor a few given words with a pound character.
+@example
+ffmpeg -i
"http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv";
-filter_complex
"[0:1]censor=words='diss,louder,hope,beam,word':censor_char='#'" -map 0 -y
output.mkv
+@end example
+@end itemize
+
+
+@section textmod
+
+Modify subtitle text in a number of ways.
+
+Inputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+@item mode
+The kind of text modification to apply
+
+Supported operation modes are:
+
+@table @var
+@item 0, leet
+Convert subtitle text to 'leet speak'. It's primarily useful for testing as
the modification will be visible with almost all text lines.
+@item 1, to_upper
+Change all text to upper case. Might improve readability.
+@item 2, to_lower
+Change all text to lower case.
+@item 3, replace_chars
+Replace one or more characters. Requires the find and replace parameters to be
specified.
+Both need to be equal in length.
+The first char in find is replaced by the first char in replace, same for all
subsequent chars.
+@item 4, remove_chars
+Remove certain characters. Requires the find parameter to be specified.
+All chars in the find parameter string will be removed from all subtitle text.
+@item 5, replace_words
+Replace one or more words. Requires the find and replace parameters to be
specified. Multiple words must be separated by the delimiter char specified vie
the separator parameter (default: ',').
+The number of words in the find and replace parameters needs to be equal.
+The first word in find is replaced by the first word in replace, same for all
subsequent words
+@item 6, remove_words
+Remove certain words. Requires the find parameter to be specified. Multiple
words must be separated by the delimiter char specified vie the separator
parameter (default: ',').
+All words in the find parameter string will be removed from all subtitle text.
+@end table
+
+@item find
+Required for replace_chars, remove_chars, replace_words and remove_words.
+
+@item find_file
+Specify a file from which to load the contents for the 'find' parameter.
+
+@item replace
+Required for replace_chars and replace_words.
+
+@item replace_file
+Specify a file from which to load the contents for the 'replace' parameter.
+
+@item separator
+Delimiter character for words. Used with replace_words and remove_words- Must
be a single character.
+The default is '.'.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Change all characters to upper case while keeping all styles and animations:
+@example
+ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv";
-filter_complex "[0:s]textmod=mode=to_upper" -map 0 -y out.mkv
+@end example
+@item
+Remove a set of symbol characters for am improved and smooth
[FFmpeg-devel] [PATCH v21 17/20] avfilter/stripstyles: Add stripstyles filter
- stripstyles {S -> S)
Remove all inline styles from subtitle events
Signed-off-by: softworkz
---
doc/filters.texi | 37 +++
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/sf_stripstyles.c | 196 +++
4 files changed, 235 insertions(+)
create mode 100644 libavfilter/sf_stripstyles.c
diff --git a/doc/filters.texi b/doc/filters.texi
index c53bf0c60d..d4c564b3f0 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25703,6 +25703,43 @@ ffmpeg -i
"http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.
@end example
@end itemize
+@section stripstyles
+
+Remove all inline styles from subtitle events.
+
+Inputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+@item remove_animated
+Also remove text which is subject to animation (default: true)
+Usually, animated text elements are used used in addition to static subtitle
lines for creating effects, so in most cases it is safe to remove the animation
content.
+If subtitle text is missing, try setting this to false.
+
+@item select_layer
+Process only ASS subtitle events from a specific layer. This allows to filter
out certain effects where an ASS author duplicates the text onto multiple
layers.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Remove styles and animations from ASS subtitles and output events from ass
layer 0 only. Then convert asn save as SRT stream:
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv";
-filter_complex "[0:1]stripstyles=select_layer=0" -map 0 -c:s srt output.mkv
+@end example
+@end itemize
+
@section textmod
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index d2995fbec6..7d7da0c59a 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -557,6 +557,7 @@ OBJS-$(CONFIG_NULLSINK_FILTER) +=
vsink_nullsink.o
OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o
OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o
OBJS-$(CONFIG_TEXTMOD_FILTER)+= sf_textmod.o
+OBJS-$(CONFIG_STRIPSTYLES_FILTER)+= sf_stripstyles.o
# multimedia filters
OBJS-$(CONFIG_ABITSCOPE_FILTER) += avf_abitscope.o
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 673f9fb839..27680a2f00 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -546,6 +546,7 @@ extern const AVFilter ff_avf_showwavespic;
extern const AVFilter ff_vaf_spectrumsynth;
extern const AVFilter ff_sf_censor;
extern const AVFilter ff_sf_showspeaker;
+extern const AVFilter ff_sf_stripstyles;
extern const AVFilter ff_sf_textmod;
extern const AVFilter ff_svf_graphicsub2video;
extern const AVFilter ff_svf_textsub2video;
diff --git a/libavfilter/sf_stripstyles.c b/libavfilter/sf_stripstyles.c
new file mode 100644
index 00..82cb9c7647
--- /dev/null
+++ b/libavfilter/sf_stripstyles.c
@@ -0,0 +1,196 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/**
+ * @file
+ * text subtitle filter which removes inline-styles from subtitles
+ */
+
+#include "libavutil/opt.h"
+#include "internal.h"
+#include "libavutil/ass_split_internal.h"
+#include "libavutil/bprint.h"
+
+typedef struct StripStylesContext {
+const AVClass *class;
+enum AVSubtitleType format;
+int remove_animated;
+int select_layer;
+} StripStylesContext;
+
+typedef struct DialogContext {
+StripStylesContext* ss_ctx;
+AVBPrint buffer;
+int drawing_scale;
+int is_animated;
+} DialogContext;
+
+static void dialog_text_cb(void *priv, const char *text, int len)
+{
+DialogContext *s = priv;
+
+av_log(s->ss_ctx, AV_LOG_DEBUG, "dialog_text_cb: %s\n", text);
+
+if (!s->drawing_scale && (!s->is_animated || !s->ss_ctx->remove_animated))
+av_bprint_append_data(&s->buffer, text, len);
+}
+
+static void dialog_new_line_cb(void *priv, int forced)
+{
+DialogContext *s = priv;
+if (!s->drawing_scale && !s->is_animated)
+av_bprint_append_data(&s->buffer, forced ? "\\N" : "\\n", 2);
+}
+
+static void di
[FFmpeg-devel] [PATCH v21 18/20] avfilter/splitcc: Add splitcc filter for closed caption handling
- splitcc {V -> VS)
Extract closed-caption (A53) data from video
frames as subtitle Frames
ffmpeg -y -loglevel verbose -i
"https://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts";
-filter_complex
"[0:v]splitcc[vid1],textmod=mode=remove_chars:find='@',[vid1]overlay_textsubs"
output.mkv
Signed-off-by: softworkz
---
configure| 1 +
doc/filters.texi | 63 +++
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/sf_splitcc.c | 378 +++
5 files changed, 444 insertions(+)
create mode 100644 libavfilter/sf_splitcc.c
diff --git a/configure b/configure
index 549382a61f..2160521951 100755
--- a/configure
+++ b/configure
@@ -3704,6 +3704,7 @@ spp_filter_select="fft idctdsp fdctdsp me_cmp pixblockdsp"
sr_filter_deps="avformat swscale"
sr_filter_select="dnn"
stereo3d_filter_deps="gpl"
+splitcc_filter_deps="avcodec"
subtitles_filter_deps="avformat avcodec libass"
super2xsai_filter_deps="gpl"
pixfmts_super2xsai_test_deps="super2xsai_filter"
diff --git a/doc/filters.texi b/doc/filters.texi
index d4c564b3f0..5c1432311a 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -26053,6 +26053,69 @@ ffmpeg -i INPUT -filter_complex
"showspeaker=format=colon:style='@{\\c&HDD&\
@end example
@end itemize
+
+@section splitcc
+
+Split-out closed-caption/A53 subtitles from video frame side data.
+
+This filter provides an input and an output for video frames, which are just
passed through without modification.
+The second out provides subtitle frames which are extracted from video frame
side data.
+
+Inputs:
+@itemize
+@item 0: Video [ALL]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video (same as input)
+@item 1: Subtitles [TEXT]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+
+@item use_cc_styles
+Emit closed caption style header.
+This will make closed captions appear in white font with a black rectangle
background.
+
+@item real_time
+Emit subtitle events as they are decoded for real-time display.
+
+@item real_time_latency_msec
+Minimum elapsed time between emitting real-time subtitle events.
+Only applies to real_time mode.
+
+@item data_field
+Select data field. Possible values:
+
+@table @samp
+@item auto
+Pick first one that appears.
+@item first
+@item second
+@end table
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Extract closed captions as text subtitle stream and overlay it onto the video
in cc style (black bar background):
+@example
+ffmpeg -i "https://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts";
-filter_complex
"[0:v:0]splitcc=use_cc_styles=1[vid1][sub1];[vid1][sub1]overlaytextsubs"
output.mkv
+@end example
+
+@item
+A nicer variant, using realtime output from cc_dec and rendering it with the
render_latest_only parameter from overlaytextsubs to avoid ghosting by timely
overlap.
+@example
+ffmpeg -i "https://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts";
-filter_complex
"[0:v:0]splitcc=real_time=1:real_time_latency_msec=200[vid1][sub1];[vid1][sub1]overlaytextsubs=render_latest_only=1"
output.mkv
+@end example
+@end itemize
+
+
@section textsub2video
Converts text subtitles to video frames.
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 7d7da0c59a..2224e5fe5f 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -557,6 +557,7 @@ OBJS-$(CONFIG_NULLSINK_FILTER) +=
vsink_nullsink.o
OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o
OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o
OBJS-$(CONFIG_TEXTMOD_FILTER)+= sf_textmod.o
+OBJS-$(CONFIG_SPLITCC_FILTER)+= sf_splitcc.o
OBJS-$(CONFIG_STRIPSTYLES_FILTER)+= sf_stripstyles.o
# multimedia filters
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 27680a2f00..6adde2b9f6 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -546,6 +546,7 @@ extern const AVFilter ff_avf_showwavespic;
extern const AVFilter ff_vaf_spectrumsynth;
extern const AVFilter ff_sf_censor;
extern const AVFilter ff_sf_showspeaker;
+extern const AVFilter ff_sf_splitcc;
extern const AVFilter ff_sf_stripstyles;
extern const AVFilter ff_sf_textmod;
extern const AVFilter ff_svf_graphicsub2video;
diff --git a/libavfilter/sf_splitcc.c b/libavfilter/sf_splitcc.c
new file mode 100644
index 00..3556d084d9
--- /dev/null
+++ b/libavfilter/sf_splitcc.c
@@ -0,0 +1,378 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABIL
[FFmpeg-devel] [PATCH v21 19/20] avfilter/graphicsub2text: Add new graphicsub2text filter (OCR)
Signed-off-by: softworkz --- configure| 1 + doc/filters.texi | 55 + libavfilter/Makefile | 2 + libavfilter/allfilters.c | 1 + libavfilter/sf_graphicsub2text.c | 354 +++ 5 files changed, 413 insertions(+) create mode 100644 libavfilter/sf_graphicsub2text.c diff --git a/configure b/configure index 2160521951..1110c8fd51 100755 --- a/configure +++ b/configure @@ -3640,6 +3640,7 @@ frei0r_filter_deps="frei0r" frei0r_src_filter_deps="frei0r" fspp_filter_deps="gpl" gblur_vulkan_filter_deps="vulkan spirv_compiler" +graphicsub2text_filter_deps="libtesseract" hflip_vulkan_filter_deps="vulkan spirv_compiler" histeq_filter_deps="gpl" hqdn3d_filter_deps="gpl" diff --git a/doc/filters.texi b/doc/filters.texi index 5c1432311a..ea056be66b 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -25819,6 +25819,61 @@ ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv"; -filter_comple @end example @end itemize +@section graphicsub2text + +Converts graphic subtitles to text subtitles by performing OCR. + +For this filter to be available, ffmpeg needs to be compiled with libtesseract (see https://github.com/tesseract-ocr/tesseract). +Language models need to be downloaded from https://github.com/tesseract-ocr/tessdata and put into as subfolder named 'tessdata' or into a folder specified via the environment variable 'TESSDATA_PREFIX'. +The path can also be specified via filter option (see below). + +Note: These models are including the data for both OCR modes. + +Inputs: +- 0: Subtitles [bitmap] + +Outputs: +- 0: Subtitles [text] + +It accepts the following parameters: + +@table @option +@item ocr_mode +The character recognition mode to use. + +Supported OCR modes are: + +@table @var +@item 0, tesseract +This is the classic libtesseract operation mode. It is fast but less accurate than LSTM. +@item 1, lstm +Newer OCR implementation based on ML models. Provides usually better results, requires more processing resources. +@item 2, both +Use a combination of both modes. +@end table + +@item tessdata_path +The path to a folder containing the language models to be used. + +@item language +The recognition language. It needs to match the first three characters of a language model file in the tessdata path. + +@end table + + +@subsection Examples + +@itemize +@item +Convert DVB graphic subtitles to ASS (text) subtitles + +Note: For this to work, you need to have the data file 'eng.traineddata' in a 'tessdata' subfolder (see above). +@example +ffmpeg ffmpeg -loglevel verbose -i "https://streams.videolan.org/streams/ts/video_subs_ttxt%2Bdvbsub.ts"; -filter_complex "[0:13]graphicsub2text=ocr_mode=both" -c:s ass -y output.mkv +@end example +@end itemize + + @section graphicsub2video Renders graphic subtitles as video frames. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 2224e5fe5f..3b972e134b 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -296,6 +296,8 @@ OBJS-$(CONFIG_GBLUR_VULKAN_FILTER) += vf_gblur_vulkan.o vulkan.o vulka OBJS-$(CONFIG_GEQ_FILTER)+= vf_geq.o OBJS-$(CONFIG_GRADFUN_FILTER)+= vf_gradfun.o OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlaygraphicsubs.o framesync.o +OBJS-$(CONFIG_GRAPHICSUB2TEXT_FILTER)+= sf_graphicsub2text.o +OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlaygraphicsubs.o framesync.o OBJS-$(CONFIG_GRAPHMONITOR_FILTER) += f_graphmonitor.o OBJS-$(CONFIG_GRAYWORLD_FILTER) += vf_grayworld.o OBJS-$(CONFIG_GREYEDGE_FILTER) += vf_colorconstancy.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 6adde2b9f6..f70f08dc5a 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -545,6 +545,7 @@ extern const AVFilter ff_avf_showwaves; extern const AVFilter ff_avf_showwavespic; extern const AVFilter ff_vaf_spectrumsynth; extern const AVFilter ff_sf_censor; +extern const AVFilter ff_sf_graphicsub2text; extern const AVFilter ff_sf_showspeaker; extern const AVFilter ff_sf_splitcc; extern const AVFilter ff_sf_stripstyles; diff --git a/libavfilter/sf_graphicsub2text.c b/libavfilter/sf_graphicsub2text.c new file mode 100644 index 00..ef10d60efd --- /dev/null +++ b/libavfilter/sf_graphicsub2text.c @@ -0,0 +1,354 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for mor
[FFmpeg-devel] [PATCH v21 20/20] avfilter/subscale: Add filter for scaling and/or re-arranging graphical subtitles
Signed-off-by: softworkz
---
configure | 1 +
doc/filters.texi | 164 +++
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/sf_subscale.c | 883 ++
5 files changed, 1050 insertions(+)
create mode 100644 libavfilter/sf_subscale.c
diff --git a/configure b/configure
index 1110c8fd51..5188c3c75d 100755
--- a/configure
+++ b/configure
@@ -3706,6 +3706,7 @@ sr_filter_deps="avformat swscale"
sr_filter_select="dnn"
stereo3d_filter_deps="gpl"
splitcc_filter_deps="avcodec"
+subscale_filter_deps="swscale avcodec"
subtitles_filter_deps="avformat avcodec libass"
super2xsai_filter_deps="gpl"
pixfmts_super2xsai_test_deps="super2xsai_filter"
diff --git a/doc/filters.texi b/doc/filters.texi
index ea056be66b..a61093b9fb 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -26225,6 +26225,170 @@ Set the rendering margin in pixels.
For rendering, alway use the latest event only, which is covering the given
point in time.
@end table
+@section subscale
+
+Provides high-quality scaling and rearranging functionality for graphical
subtitles.
+
+The subscale filter provides multiple approaches for manipulating
+the size and position of graphical subtitle rectangles wich can
+be combined or used separately.
+Scaling is performed by converting the palettized subtitle bitmaps
+to RGBA and re-quantization to palette colors afterwards via elbg algorithm.
+
+The two major operations are 'scale' and 're-arrange' with the
+latter being separated as 'arrange_h' and 'arrange_v'.
+
+
+Inputs:
+- 0: Subtitles [bitmap]
+
+Outputs:
+- 0: Subtitles [bitmap]
+
+It accepts the following parameters:
+
+@table @option
+
+@item w, width
+Set the width of the output.
+Width and height in case of graphical subtitles are just indicating
+a virtual size for which the output (consisting of 0-n bitmap rectangles)
+is intended to be displayed on.
+
+@item h, height
+Set the height of the output.
+
+@item margin_h
+Sets a horizontal margin to be preserverved when using any
+of the arrange modes.
+
+@item margin_v
+Sets a vertical margin to be preserverved when using any
+of the arrange modes.
+
+@item force_original_aspect_ratio
+Enable decreasing or increasing output video width or height if necessary to
+keep the original aspect ratio. Possible values:
+
+@table @samp
+@item disable
+Scale the video as specified and disable this feature.
+
+@item decrease
+The output video dimensions will automatically be decreased if needed.
+
+@item increase
+The output video dimensions will automatically be increased if needed.
+
+@end table
+
+
+@item scale_mode
+Specifies how subtitle bitmaps should be scaled.
+The scale factor is determined by the the factor between input
+and output size.
+
+@table @samp
+@item none
+Do not apply any common scaling.
+
+@item uniform
+Uniformly scale all subtitle bitmaps including their positions.
+
+@item uniform_no_reposition
+Uniformly scale all subtitle bitmaps without changing positions.
+
+@end table
+
+
+@item arrange_h
+Specifies how subtitle bitmaps should be arranged horizontally.
+
+@item arrange_v
+Specifies how subtitle bitmaps should be arranged vertically.
+
+
+@table @samp
+@item none
+Do not rearrange subtitle bitmaps.
+
+@item margin_no_scale
+Move subtitle bitmaps to be positioned inside the specified
+margin (margin_h or margin_v) when possible and without scaling.
+
+@item margin_and_scale
+Move subtitle bitmaps to be positioned inside the specified
+margin (margin_h or margin_v) and scale in case it doesn't fit.
+
+@item snapalign_no_scale
+Categorize subtitle bitmap positions as one of left/center/right
+or top/bottom/middle based on original positioning and apply
+these alignments for the target positioning.
+No scaling will be applied.
+
+@item snapalign_and_scale
+Categorize subtitle bitmap positions as one of left/center/right
+or top/bottom/middle based on original positioning and apply
+these alignments for the target positioning.
+Bitmaps that do not fit inside the margins borders are
+scaled to fit.
+@end table
+
+@item eval
+Set evaluation mode for the expressions (@option{width}, @option{height}).
+
+It accepts the following values:
+@table @samp
+@item init
+Evaluate expressions only once during the filter initialization.
+
+@item frame
+Evaluate expressions for each incoming frame. This is way slower than the
+@samp{init} mode since it requires all the scalers to be re-computed, but it
+allows advanced dynamic expressions.
+@end table
+
+Default value is @samp{init}.
+
+
+@item num_colors
+Set the number of palette colors for output images.
+Choose the maximum (256) when further processing is done (e.g.
+overlaying on a video).
+When subtitles will be encoded as bitmap subtitles (e.g. dvbsub),
+a smaller number of palette colors (e.g. 4-16) might need to be used, depending
+on the target format and codec.
+
+@item bitmap_width_align
+@item bitmap_height_align
+Make sure that subtitle
[FFmpeg-devel] [PATCH 1/7] avformat/vivo: Do not use the general expression evaluator for parsing a floating point value
Fixes: Timeout
Fixes:
41564/clusterfuzz-testcase-minimized-ffmpeg_dem_VIVO_fuzzer-6309014024093696
Found-by: continuous fuzzing process
https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer
---
libavformat/vivo.c | 7 ---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/libavformat/vivo.c b/libavformat/vivo.c
index b2904cd25a7..8e819d910b7 100644
--- a/libavformat/vivo.c
+++ b/libavformat/vivo.c
@@ -206,11 +206,12 @@ static int vivo_read_header(AVFormatContext *s)
return AVERROR_INVALIDDATA;
value_used = 1;
} else if (!strcmp(key, "FPS")) {
-AVRational tmp;
+double d;
+if (sscanf(value, "%f", &d) != 1)
+return AVERROR_INVALIDDATA;
value_used = 1;
-if (!av_parse_ratio(&tmp, value, 1, AV_LOG_WARNING, s))
-fps = av_inv_q(tmp);
+fps = av_d2q(1/d, 1);
}
if (!value_used)
--
2.17.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 4/7] avformat/4xm: Consider max_streams on reallocating tracks array
Fixes: OOM
Fixes:
41595/clusterfuzz-testcase-minimized-ffmpeg_dem_FOURXM_fuzzer-6355979363549184
Found-by: continuous fuzzing process
https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer
---
libavformat/4xm.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/libavformat/4xm.c b/libavformat/4xm.c
index f918b1fc572..5cbae7053d8 100644
--- a/libavformat/4xm.c
+++ b/libavformat/4xm.c
@@ -137,7 +137,8 @@ static int parse_strk(AVFormatContext *s,
return AVERROR_INVALIDDATA;
track = AV_RL32(buf + 8);
-if ((unsigned)track >= UINT_MAX / sizeof(AudioTrack) - 1) {
+if ((unsigned)track >= UINT_MAX / sizeof(AudioTrack) - 1 ||
+s->max_streams && track >= s->max_streams) {
av_log(s, AV_LOG_ERROR, "current_track too large\n");
return AVERROR_INVALIDDATA;
}
--
2.17.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 5/7] avformat/mxfdec: Check for duplicate mxf_read_index_entry_array()
Fixes: memleak
Fixes:
41596/clusterfuzz-testcase-minimized-ffmpeg_dem_MXF_fuzzer-6439060204290048
Found-by: continuous fuzzing process
https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer
---
libavformat/mxfdec.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/libavformat/mxfdec.c b/libavformat/mxfdec.c
index c231c944c01..1d501982793 100644
--- a/libavformat/mxfdec.c
+++ b/libavformat/mxfdec.c
@@ -,6 +,9 @@ static int mxf_read_index_entry_array(AVIOContext *pb,
MXFIndexTableSegment *seg
{
int i, length;
+if (segment->temporal_offset_entries)
+return AVERROR_INVALIDDATA;
+
segment->nb_index_entries = avio_rb32(pb);
length = avio_rb32(pb);
--
2.17.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 3/7] tools/target_dem_fuzzer: Force interrupt for HLS
Fixes: Timeout
Fixes:
41580/clusterfuzz-testcase-minimized-ffmpeg_dem_HLS_fuzzer-5059099224571904
Found-by: continuous fuzzing process
https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer
---
tools/target_dem_fuzzer.c | 4
1 file changed, 4 insertions(+)
diff --git a/tools/target_dem_fuzzer.c b/tools/target_dem_fuzzer.c
index 6ee793a28ba..687989ccc89 100644
--- a/tools/target_dem_fuzzer.c
+++ b/tools/target_dem_fuzzer.c
@@ -172,6 +172,10 @@ int LLVMFuzzerTestOneInput(const uint8_t *data, size_t
size) {
avfmt->interrupt_callback.callback = interrupt_cb;
}
+// HLS uses a loop with sleep, we thus must breakout or we timeout
+if (!strcmp(fmt->name, "hls"))
+interrupt_counter &= 31;
+
if (!io_buffer_size || size / io_buffer_size > maxblocks)
io_buffer_size = size;
--
2.17.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 2/7] tools/target_dem_fuzzer: Test interrupt callback
Signed-off-by: Michael Niedermayer
---
tools/target_dem_fuzzer.c | 10 ++
1 file changed, 10 insertions(+)
diff --git a/tools/target_dem_fuzzer.c b/tools/target_dem_fuzzer.c
index 3c03c8d17c0..6ee793a28ba 100644
--- a/tools/target_dem_fuzzer.c
+++ b/tools/target_dem_fuzzer.c
@@ -34,6 +34,13 @@ typedef struct IOContext {
int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size);
+int64_t interrupt_counter;
+static int interrupt_cb(void *ctx)
+{
+interrupt_counter --;
+return interrupt_counter < 0;
+}
+
static void error(const char *err)
{
fprintf(stderr, "%s", err);
@@ -160,6 +167,9 @@ int LLVMFuzzerTestOneInput(const uint8_t *data, size_t
size) {
*strchr(extension, ',') = 0;
av_strlcatf(filename, sizeof(filename), ".%s", extension);
}
+
+interrupt_counter = bytestream2_get_le32(&gbc);
+avfmt->interrupt_callback.callback = interrupt_cb;
}
if (!io_buffer_size || size / io_buffer_size > maxblocks)
--
2.17.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 6/7] tools/target_dec_fuzzer: adjust threshold for gem
Fixes: Timeout
Fixes:
42035/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_GEM_fuzzer-5033604191748096
Found-by: continuous fuzzing process
https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer
---
tools/target_dec_fuzzer.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/tools/target_dec_fuzzer.c b/tools/target_dec_fuzzer.c
index b862d7a3a41..0c59242e3e4 100644
--- a/tools/target_dec_fuzzer.c
+++ b/tools/target_dec_fuzzer.c
@@ -166,6 +166,7 @@ int LLVMFuzzerTestOneInput(const uint8_t *data, size_t
size) {
case AV_CODEC_ID_FLAC:maxsamples /= 1024; break;
case AV_CODEC_ID_FLV1:maxpixels /= 1024; break;
case AV_CODEC_ID_G2M: maxpixels /= 1024; break;
+case AV_CODEC_ID_GEM: maxpixels /= 512; break;
case AV_CODEC_ID_GDV: maxpixels /= 512; break;
case AV_CODEC_ID_GIF: maxpixels /= 16;break;
case AV_CODEC_ID_H264:maxpixels /= 256; break;
--
2.17.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 7/7] avcodec/gemdec: Move all support checks before before image allocation
Signed-off-by: Michael Niedermayer
---
libavcodec/gemdec.c | 32 +---
1 file changed, 21 insertions(+), 11 deletions(-)
diff --git a/libavcodec/gemdec.c b/libavcodec/gemdec.c
index eee21a50d4b..fd14b22390c 100644
--- a/libavcodec/gemdec.c
+++ b/libavcodec/gemdec.c
@@ -157,10 +157,22 @@ static int gem_decode_frame(AVCodecContext *avctx,
if (header_size >= 11)
tag = bytestream2_peek_be32(&gb);
-if (tag == AV_RB32("STTT") || tag == AV_RB32("TIMG") || tag ==
AV_RB32("XIMG") ||
-planes == 1 || planes == 2 || planes == 3 || planes == 4 ||
-planes == 8 || planes == 16 || planes == 24) {
-} else {
+if (tag == AV_RB32("STTT")) {
+if (planes != 4) {
+avpriv_request_sample(avctx, "STTT planes=%d", planes);
+return AVERROR_PATCHWELCOME;
+}
+} else if (tag == AV_RB32("TIMG")) {
+if (planes != 15) {
+avpriv_request_sample(avctx, "TIMG planes=%d", planes);
+return AVERROR_PATCHWELCOME;
+}
+} else if (tag == AV_RB32("XIMG")) {
+if (planes != 1 && planes != 2 && planes != 4 && planes != 8 && planes
!= 16 && planes != 24 && planes != 32) {
+avpriv_request_sample(avctx, "XIMG planes=%d", planes);
+return AVERROR_PATCHWELCOME;
+}
+} else if (planes != 1 && planes != 2 && planes != 3 && planes != 4 &&
planes != 8 && planes != 16 && planes != 24) {
avpriv_request_sample(avctx, "planes=%d", planes);
return AVERROR_PATCHWELCOME;
}
@@ -184,14 +196,12 @@ static int gem_decode_frame(AVCodecContext *avctx,
palette[i] = 0xFF00 | r << 16 | g << 8 | b;
}
} else {
-avpriv_request_sample(avctx, "STTT planes=%d", planes);
-return AVERROR_PATCHWELCOME;
+av_assert0(0);
}
} else if (tag == AV_RB32("TIMG")) {
bytestream2_skip(&gb, 4);
if (planes != 15) {
-avpriv_request_sample(avctx, "TIMG planes=%d", planes);
-return AVERROR_PATCHWELCOME;
+av_assert0(0);
}
} else if (tag == AV_RB32("XIMG")) {
bytestream2_skip(&gb, 6);
@@ -215,8 +225,7 @@ static int gem_decode_frame(AVCodecContext *avctx,
row_width = avctx->width * pixel_size;
put_lines = put_lines_bytes;
} else {
-avpriv_request_sample(avctx, "XIMG planes=%d", planes);
-return AVERROR_PATCHWELCOME;
+av_assert0(0);
}
} else if (planes == 1) {
palette[0] = 0x;
@@ -244,7 +253,8 @@ static int gem_decode_frame(AVCodecContext *avctx,
planes = 1;
row_width = avctx->width * pixel_size;
put_lines = put_lines_bytes;
-}
+} else
+av_assert0(0);
ret = av_reallocp_array(&avctx->priv_data, planes, row_width);
if (ret < 0)
--
2.17.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v21 16/20] avfilter/textmod: Add textmod, censor and show_speaker filters
On Sun, Dec 05, 2021 at 07:41:56PM +, Soft Works wrote:
> - textmod {S -> S)
> Modify subtitle text in a number of ways
>
> - censor {S -> S)
> Censor subtitles using a word list
>
> - show_speaker {S -> S)
> Prepend speaker names from ASS subtitles to the visible text lines
>
> Signed-off-by: softworkz
> ---
> doc/filters.texi | 206
> libavfilter/Makefile | 5 +
> libavfilter/allfilters.c | 3 +
> libavfilter/sf_textmod.c | 697 +++
> 4 files changed, 911 insertions(+)
> create mode 100644 libavfilter/sf_textmod.c
doesnt apply
Applying: avfilter/textmod: Add textmod, censor and show_speaker filters
.git/rebase-apply/patch:269: trailing whitespace.
/*
.git/rebase-apply/patch:270: trailing whitespace.
* Copyright (c) 2021 softworkz
.git/rebase-apply/patch:271: trailing whitespace.
*
.git/rebase-apply/patch:272: trailing whitespace.
* This file is part of FFmpeg.
.git/rebase-apply/patch:273: trailing whitespace.
*
warning: squelched 692 whitespace errors
warning: 697 lines add whitespace errors.
Using index info to reconstruct a base tree...
.git/rebase-apply/patch:269: trailing whitespace.
/*
.git/rebase-apply/patch:270: trailing whitespace.
* Copyright (c) 2021 softworkz
.git/rebase-apply/patch:271: trailing whitespace.
*
.git/rebase-apply/patch:272: trailing whitespace.
* This file is part of FFmpeg.
.git/rebase-apply/patch:273: trailing whitespace.
*
error: patch failed: doc/filters.texi:25643
error: doc/filters.texi: patch does not apply
error: patch failed: libavfilter/Makefile:553
error: libavfilter/Makefile: patch does not apply
error: patch failed: libavfilter/allfilters.c:544
error: libavfilter/allfilters.c: patch does not apply
error: Did you hand edit your patch?
It does not apply to blobs recorded in its index.
Patch failed at 0001 avfilter/textmod: Add textmod, censor and show_speaker
filters
Use 'git am --show-current-patch' to see the failed patch
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".
[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
When you are offended at any man's fault, turn to yourself and study your
own failings. Then you will forget your anger. -- Epictetus
signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame for subtitle handling
5 Dec 2021, 20:21 by softwo...@hotmail.com:
>
>
>> -Original Message-
>> From: ffmpeg-devel On Behalf Of Lynne
>> Sent: Sunday, December 5, 2021 5:40 PM
>> To: FFmpeg development discussions and patches
>> Subject: Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame
>> for subtitle handling
>>
>> 5 Dec 2021, 17:23 by softwo...@hotmail.com:
>>
>> > @@ -491,6 +499,39 @@ typedef struct AVFrame {
>> > */
>> > uint64_t channel_layout;
>> >
>> > +/**
>> > + * Display start time, relative to packet pts, in ms.
>> > + */
>> > +uint32_t subtitle_start_time;
>> > +
>> > +/**
>> > + * Display end time, relative to packet pts, in ms.
>> > + */
>> > +uint32_t subtitle_end_time;
>> >
>>
>> Milliseconds? Our entire API's based around timestamps
>> with time bases. Plus, we all know what happened when
>> Matroska settled onto milliseconds and ruined a perfectly
>> complex but good container.
>> Make this relative to the PTS field, with the same timebase
>> as the PTS field.
>> There's even a new AVFrame->time_base field for you to
>> set so you wouldn't forget it.
>>
>>
>> > +/**
>> > + * Number of items in the @ref subtitle_areas array.
>> > + */
>> > +unsigned num_subtitle_areas;
>> > +
>> > +/**
>> > + * Array of subtitle areas, may be empty.
>> > + */
>> > +AVSubtitleArea **subtitle_areas;
>> >
>>
>> There's no reason why this cannot be handled using the buffer
>> and data fields. If you need more space there, you're free to bump
>> the maximum number of pointers, plus this removes the horrid
>> malloc(1) hack. We've discussed this, and I couldn't follow why
>> this was bad in the email discussion you linked.
>>
>
>
> Hi Lynne,
>
> let me add another note on these two topics.
>
> One of the reasons why this subtitle filtering patchset has proceeded
> rather smoothly so far, causing only a small amount of regressions
> that were easy to find and fix, is that I have kept as much as possible
> from the current AVSubtitle implementation, projecting it to the
> new frame-based subtitle implementation, while keeping it mostly
> compatible with the current logic.
> Interestingly, an earlier attempt towards subtitle filtering has
> gotten stuck right at the point about storing subtitle data in
> AVFrame buffers. I decided to focus on functionality rather than
> academic kind of details. What has been in place and working
> for such a long time can't be so wrong that it can't be accepted.
>
> We also need to consider that it's not my patchset that is deliberately
> deciding to store data in pointers associate with rects/area and
> to store start- and end-times in milliseconds: it's all the decoders
> and encoders that are doing this. I'm just adapting to the status
> quo.
>
> These two things - changing the timebase of the ms-fields and
> changing the location of the buffer pointers - is a gigantic task
> that would require to change everything that this patchset covers
> and implements - plus even more code that this patchset hasn't even
> touched. It would require changes to everything everywhere where
> subtitles are concerned.
> It is a task that is risky as hell, will cause plenty of
> regressions under guarantee, require extensive testing and might even
> drive this whole endeavor into a dead end.
>
> As I said before, please understand that I'm not ready to enter that
> path.
>
You don't have to break anything to make the start/end times proper
timestamps. We have timebases for that! If you'd like your timestamps
to have millisecond precision, you use a millisecond timebase!
So what if we use ASS internally which rounds timestamps? We have
bitmap formats too, and those are generally following the video
timestamps. You absolutely don't want the timestamps being rounded
in that case, or you risk having your subtitles be a frame late or
a frame early.
You could add a separate timebase for the subtitle start/end fields,
in case the packet's timebase doesn't match the subtitle format's
native timebase. I won't object to that.
You also don't have to risk that much by removing the subtitle_pts
field. Once the start/end times are converted to proper timestamps,
then the AVFrame->pts field can do whatever it wants to, whether
it mirrors the demuxer's timestamp, be constant, non-existent or even
go backwards.
If you just want to have an empty subtitle rect, you can simply
zero the subtitle struct's data and buffer fields.
Calling these issues too academic when the frame API is so very
straightforward and cut-and-paste is like calling computers
too academic when you have several thousand large numbers to add.
I'd really rather have a subtitle API that's usable, maintainable and easy
to extend, even if it's momentarily broken for certain corner cases
and weird files.
We can easily fix broken files one by one. We've got plenty of heuristics,
and have mostly abstracted and documented them well. We can add more.
We cannot re
Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame for subtitle handling
On Sun, Dec 5, 2021 at 6:58 PM Soft Works wrote:
>
>
>
> > -Original Message-
> > From: ffmpeg-devel On Behalf Of Lynne
> > Sent: Sunday, December 5, 2021 5:40 PM
> > To: FFmpeg development discussions and patches
> > Subject: Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame
> > for subtitle handling
> >
> > 5 Dec 2021, 17:23 by softwo...@hotmail.com:
> >
> > > @@ -491,6 +499,39 @@ typedef struct AVFrame {
> > > */
> > > uint64_t channel_layout;
> > >
> > > +/**
> > > + * Display start time, relative to packet pts, in ms.
> > > + */
> > > +uint32_t subtitle_start_time;
> > > +
> > > +/**
> > > + * Display end time, relative to packet pts, in ms.
> > > + */
> > > +uint32_t subtitle_end_time;
> > >
> >
> > Milliseconds? Our entire API's based around timestamps
> > with time bases. Plus, we all know what happened when
> > Matroska settled onto milliseconds and ruined a perfectly
> > complex but good container.
> > Make this relative to the PTS field, with the same timebase
> > as the PTS field.
> > There's even a new AVFrame->time_base field for you to
> > set so you wouldn't forget it.
>
> The internal format for text subtitles is ASS, and this is
> using a timebase of milliseconds.
>
> All existing decoders and encoders are using this and I'm
> afraid, but I will not go and change them all.
>
> > > +/**
> > > + * Number of items in the @ref subtitle_areas array.
> > > + */
> > > +unsigned num_subtitle_areas;
> > > +
> > > +/**
> > > + * Array of subtitle areas, may be empty.
> > > + */
> > > +AVSubtitleArea **subtitle_areas;
> > >
> >
> > There's no reason why this cannot be handled using the buffer
> > and data fields. If you need more space there, you're free to bump
> > the maximum number of pointers, plus this removes the horrid
> > malloc(1) hack. We've discussed this, and I couldn't follow why
> > this was bad in the email discussion you linked.
>
> There are reasons. Some of those had I mentioned in an earlier
> discussion with Hendrik.
> The effort alone to relate the buffers to subtitle areas (which area
> 'owns' which buffer) is not reasonable. Too many cases to consider,
> what's when there are 3 areas and the second area doesn't have any
> buffer? The convention is that the buffer should be used contiguously.
> Managing those relations is error-prone and would require a lot of code.
>
> > > +/**
> > > + * Usually the same as packet pts, in AV_TIME_BASE.
> > > + *
> > > + * @deprecated This is kept for compatibility reasons and corresponds
> > to
> > > + * AVSubtitle->pts. Might be removed in the future.
> > > + */
> > > +int64_t subtitle_pts;
> > >
> >
> > I'm not going to accept a field which is instantly deprecated.
> > As we've discussed multiple times, please merge this into
> > the regular frame PTS field. We already have _2_ necessary
> > stard/end fields.
>
> --
>
> > I agree with this entirely. Even ignoring the fact that adding a new
> > field thats deprecated is instantly a disqualification, AVSubtitle had
> > one pts field, AVFrame already has one pts field - both are even
> > documented to have the same semantic. They should just contain the
> > exact same data, thats how you achieve compatibility, not by claiming
> > you need a new field for compatibility reasons.
> >
> > - Hendrik
>
> I think the mistake is to declare subtitle_pts as deprecated. I had
> added the deprecation at a very early point in time where I had still
> thought that it can be eliminated.
>
> Even though we are driving subtitle data through the graph attached
> to AVFrame, the behavior involved is very different from audio and
> video frames. Actually there's not one but many different ways how
> subtitle data can appear in a source and travel through a filtergraph:
>
> - Sometimes, subtitle events are muxed into a stream many seconds
> ahead of display time. In this case, AVFrame.pts is the mux position
> and AVFrame.subtitle_pts is the actual presentation time.
> When filtering subtitles to modify something, it would be still desired
> to retain the offset between mux time and display start
>
This information was impossible to convey in AVSubtitle, as it had
exactly one pts field, so where does it come from now?
Also, isn't that kind of offset what subtitle_start_time is for, a
delta on top of the pts?
Sounds like its just duplicating logic once again.
pts is also by definition and name the "presentation time stamp", if a
timestamp of another meaning can convey additional information, we do
have a dts field in AVFrame, for the decoding timestamp.
> Now when splitcc sends such a repeated frame, it needs to adjust the
> frame's pts in order to match the configured output framerate.
> But that's for keeping the graph running, the actual display start
> time (subtitle_pts) must not be changed.
This sounds like an implementation detail, and not something we should
have in a
Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame for subtitle handling
On Sun, Dec 5, 2021 at 11:38 PM Hendrik Leppkes wrote:
>
> On Sun, Dec 5, 2021 at 6:58 PM Soft Works wrote:
> >
> >
> >
> > > -Original Message-
> > > From: ffmpeg-devel On Behalf Of Lynne
> > > Sent: Sunday, December 5, 2021 5:40 PM
> > > To: FFmpeg development discussions and patches
> > > Subject: Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare
> > > AVFrame
> > > for subtitle handling
> > >
> > > 5 Dec 2021, 17:23 by softwo...@hotmail.com:
> > >
> > > > @@ -491,6 +499,39 @@ typedef struct AVFrame {
> > > > */
> > > > uint64_t channel_layout;
> > > >
> > > > +/**
> > > > + * Display start time, relative to packet pts, in ms.
> > > > + */
> > > > +uint32_t subtitle_start_time;
> > > > +
> > > > +/**
> > > > + * Display end time, relative to packet pts, in ms.
> > > > + */
> > > > +uint32_t subtitle_end_time;
> > > >
> > >
> > > Milliseconds? Our entire API's based around timestamps
> > > with time bases. Plus, we all know what happened when
> > > Matroska settled onto milliseconds and ruined a perfectly
> > > complex but good container.
> > > Make this relative to the PTS field, with the same timebase
> > > as the PTS field.
> > > There's even a new AVFrame->time_base field for you to
> > > set so you wouldn't forget it.
> >
> > The internal format for text subtitles is ASS, and this is
> > using a timebase of milliseconds.
> >
> > All existing decoders and encoders are using this and I'm
> > afraid, but I will not go and change them all.
> >
> > > > +/**
> > > > + * Number of items in the @ref subtitle_areas array.
> > > > + */
> > > > +unsigned num_subtitle_areas;
> > > > +
> > > > +/**
> > > > + * Array of subtitle areas, may be empty.
> > > > + */
> > > > +AVSubtitleArea **subtitle_areas;
> > > >
> > >
> > > There's no reason why this cannot be handled using the buffer
> > > and data fields. If you need more space there, you're free to bump
> > > the maximum number of pointers, plus this removes the horrid
> > > malloc(1) hack. We've discussed this, and I couldn't follow why
> > > this was bad in the email discussion you linked.
> >
> > There are reasons. Some of those had I mentioned in an earlier
> > discussion with Hendrik.
> > The effort alone to relate the buffers to subtitle areas (which area
> > 'owns' which buffer) is not reasonable. Too many cases to consider,
> > what's when there are 3 areas and the second area doesn't have any
> > buffer? The convention is that the buffer should be used contiguously.
> > Managing those relations is error-prone and would require a lot of code.
> >
> > > > +/**
> > > > + * Usually the same as packet pts, in AV_TIME_BASE.
> > > > + *
> > > > + * @deprecated This is kept for compatibility reasons and
> > > > corresponds
> > > to
> > > > + * AVSubtitle->pts. Might be removed in the future.
> > > > + */
> > > > +int64_t subtitle_pts;
> > > >
> > >
> > > I'm not going to accept a field which is instantly deprecated.
> > > As we've discussed multiple times, please merge this into
> > > the regular frame PTS field. We already have _2_ necessary
> > > stard/end fields.
> >
> > --
> >
> > > I agree with this entirely. Even ignoring the fact that adding a new
> > > field thats deprecated is instantly a disqualification, AVSubtitle had
> > > one pts field, AVFrame already has one pts field - both are even
> > > documented to have the same semantic. They should just contain the
> > > exact same data, thats how you achieve compatibility, not by claiming
> > > you need a new field for compatibility reasons.
> > >
> > > - Hendrik
> >
> > I think the mistake is to declare subtitle_pts as deprecated. I had
> > added the deprecation at a very early point in time where I had still
> > thought that it can be eliminated.
> >
> > Even though we are driving subtitle data through the graph attached
> > to AVFrame, the behavior involved is very different from audio and
> > video frames. Actually there's not one but many different ways how
> > subtitle data can appear in a source and travel through a filtergraph:
> >
> > - Sometimes, subtitle events are muxed into a stream many seconds
> > ahead of display time. In this case, AVFrame.pts is the mux position
> > and AVFrame.subtitle_pts is the actual presentation time.
> > When filtering subtitles to modify something, it would be still desired
> > to retain the offset between mux time and display start
> >
>
> This information was impossible to convey in AVSubtitle, as it had
> exactly one pts field, so where does it come from now?
> Also, isn't that kind of offset what subtitle_start_time is for, a
> delta on top of the pts?
>
> Sounds like its just duplicating logic once again.
>
> pts is also by definition and name the "presentation time stamp", if a
> timestamp of another meaning can convey additional information, we do
> have a dts field in AVFrame, for the decoding timestamp.
To illustrate this, lets j
Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame for subtitle handling
> -Original Message-
> From: ffmpeg-devel On Behalf Of Soft Works
> Sent: Sunday, December 5, 2021 6:58 PM
> To: FFmpeg development discussions and patches
> Subject: Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame
> for subtitle handling
>
>
>
> > -Original Message-
> > From: ffmpeg-devel On Behalf Of Lynne
> > Sent: Sunday, December 5, 2021 5:40 PM
> > To: FFmpeg development discussions and patches
> > Subject: Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame
> > for subtitle handling
> >
> > 5 Dec 2021, 17:23 by softwo...@hotmail.com:
> >
> > > @@ -491,6 +499,39 @@ typedef struct AVFrame {
> > > */
> > > uint64_t channel_layout;
> > >
> > > +/**
> > > + * Display start time, relative to packet pts, in ms.
> > > + */
> > > +uint32_t subtitle_start_time;
> > > +
> > > +/**
> > > + * Display end time, relative to packet pts, in ms.
> > > + */
> > > +uint32_t subtitle_end_time;
> > >
> >
> > Milliseconds? Our entire API's based around timestamps
> > with time bases. Plus, we all know what happened when
> > Matroska settled onto milliseconds and ruined a perfectly
> > complex but good container.
> > Make this relative to the PTS field, with the same timebase
> > as the PTS field.
> > There's even a new AVFrame->time_base field for you to
> > set so you wouldn't forget it.
>
> The internal format for text subtitles is ASS, and this is
> using a timebase of milliseconds.
>
> All existing decoders and encoders are using this and I'm
> afraid, but I will not go and change them all.
>
> > > +/**
> > > + * Number of items in the @ref subtitle_areas array.
> > > + */
> > > +unsigned num_subtitle_areas;
> > > +
> > > +/**
> > > + * Array of subtitle areas, may be empty.
> > > + */
> > > +AVSubtitleArea **subtitle_areas;
> > >
> >
> > There's no reason why this cannot be handled using the buffer
> > and data fields. If you need more space there, you're free to bump
> > the maximum number of pointers, plus this removes the horrid
> > malloc(1) hack. We've discussed this, and I couldn't follow why
> > this was bad in the email discussion you linked.
>
> There are reasons. Some of those had I mentioned in an earlier
> discussion with Hendrik.
> The effort alone to relate the buffers to subtitle areas (which area
> 'owns' which buffer) is not reasonable. Too many cases to consider,
> what's when there are 3 areas and the second area doesn't have any
> buffer? The convention is that the buffer should be used contiguously.
> Managing those relations is error-prone and would require a lot of code.
>
> > > +/**
> > > + * Usually the same as packet pts, in AV_TIME_BASE.
> > > + *
> > > + * @deprecated This is kept for compatibility reasons and
> corresponds
> > to
> > > + * AVSubtitle->pts. Might be removed in the future.
> > > + */
> > > +int64_t subtitle_pts;
> > >
> >
> > I'm not going to accept a field which is instantly deprecated.
> > As we've discussed multiple times, please merge this into
> > the regular frame PTS field. We already have _2_ necessary
> > stard/end fields.
>
> --
>
> > I agree with this entirely. Even ignoring the fact that adding a new
> > field thats deprecated is instantly a disqualification, AVSubtitle had
> > one pts field, AVFrame already has one pts field - both are even
> > documented to have the same semantic. They should just contain the
> > exact same data, thats how you achieve compatibility, not by claiming
> > you need a new field for compatibility reasons.
> >
> > - Hendrik
>
> I think the mistake is to declare subtitle_pts as deprecated. I had
> added the deprecation at a very early point in time where I had still
> thought that it can be eliminated.
>
> Even though we are driving subtitle data through the graph attached
> to AVFrame, the behavior involved is very different from audio and
> video frames. Actually there's not one but many different ways how
> subtitle data can appear in a source and travel through a filtergraph:
>
> - Sometimes, subtitle events are muxed into a stream many seconds
> ahead of display time. In this case, AVFrame.pts is the mux position
> and AVFrame.subtitle_pts is the actual presentation time.
> When filtering subtitles to modify something, it would be still desired
> to retain the offset between mux time and display start
>
> - Sometimes, subtitle events are occurring in the mux "live" - right in
> the moment when they are meant to be shown. An example for this are
> closed captions, and when extracting those via the new splitcc filter,
> the subtitle_pts is equal to the frame.pts value.
> But CC events do not come regularly, while downstream filters might
> expect exactly that in order to proceed. That's why the splitcc filter
> can emit subtitle frames at a certain framerate. It does so by re-
> sending the most recent subtitle frame - basically not much different
>
[FFmpeg-devel] [PATCH v6 1/2] avformat/imf: Demuxer
From: Pierre-Anthony Lemieux
Signed-off-by: Pierre-Anthony Lemieux
---
Notes:
The IMF demuxer accepts as input an IMF CPL. The assets referenced by the
CPL can be
contained in multiple deliveries, each defined by an ASSETMAP file:
ffmpeg -assetmaps ,,... -i
If -assetmaps is not specified, FFMPEG looks for a file called ASSETMAP.xml
in the same directory as the CPL.
EXAMPLE:
ffmpeg -i
http://ffmpeg-imf-samples-public.s3-website-us-west-1.amazonaws.com/countdown/CPL_f5095caa-f204-4e1c-8a84-7af48c7ae16b.xml
out.mp4
The Interoperable Master Format (IMF) is a file-based media format for the
delivery and storage of professional audio-visual masters.
An IMF Composition consists of an XML playlist (the Composition Playlist)
and a collection of MXF files (the Track Files). The Composition Playlist
(CPL)
assembles the Track Files onto a timeline, which consists of multiple
tracks.
The location of the Track Files referenced by the Composition Playlist is
stored
in one or more XML documents called Asset Maps. More details at
https://www.imfug.com/explainer.
The IMF standard was first introduced in 2013 and is managed by the SMPTE.
CHANGE NOTES:
- reduced line width
- use ff_ and FF prefixes for non-local functions and structures
- modified copyright header
- fixed rational initialization
- removed extraneous call to xmlCleanupParser()
- fix if/for single line braces
- replace av_realloc_f with av_fast_realloc when allocating CPL Resources
MAINTAINERS | 1 +
configure| 3 +-
doc/demuxers.texi| 19 +
libavformat/Makefile | 1 +
libavformat/allformats.c | 1 +
libavformat/imf.h| 206 +
libavformat/imf_cpl.c| 769
libavformat/imfdec.c | 923 +++
8 files changed, 1922 insertions(+), 1 deletion(-)
create mode 100644 libavformat/imf.h
create mode 100644 libavformat/imf_cpl.c
create mode 100644 libavformat/imfdec.c
diff --git a/MAINTAINERS b/MAINTAINERS
index dcac46003e..7a6972fe1a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -433,6 +433,7 @@ Muxers/Demuxers:
idroqdec.cMike Melanson
iff.c Jaikrishnan Menon
img2*.c Michael Niedermayer
+ imf*.cMarc-Antoine Arnaud, Pierre-Anthony
Lemieux, Valentin Noël
ipmovie.c Mike Melanson
ircam*Paul B Mahol
iss.c Stefan Gehrer
diff --git a/configure b/configure
index a98a18abaa..022f17767b 100755
--- a/configure
+++ b/configure
@@ -298,7 +298,7 @@ External library support:
--enable-libxvid enable Xvid encoding via xvidcore,
native MPEG-4/Xvid encoder exists [no]
--enable-libxml2 enable XML parsing using the C library libxml2,
needed
- for dash demuxing support [no]
+ for dash and imf demuxing support [no]
--enable-libzimg enable z.lib, needed for zscale filter [no]
--enable-libzmq enable message passing via libzmq [no]
--enable-libzvbi enable teletext support via libzvbi [no]
@@ -3400,6 +3400,7 @@ hls_muxer_select="mpegts_muxer"
hls_muxer_suggest="gcrypt openssl"
image2_alias_pix_demuxer_select="image2_demuxer"
image2_brender_pix_demuxer_select="image2_demuxer"
+imf_demuxer_deps="libxml2"
ipod_muxer_select="mov_muxer"
ismv_muxer_select="mov_muxer"
ivf_muxer_select="av1_metadata_bsf vp9_superframe_bsf"
diff --git a/doc/demuxers.texi b/doc/demuxers.texi
index cab8a7072c..614f2e754a 100644
--- a/doc/demuxers.texi
+++ b/doc/demuxers.texi
@@ -267,6 +267,12 @@ which streams to actually receive.
Each stream mirrors the @code{id} and @code{bandwidth} properties from the
@code{} as metadata keys named "id" and "variant_bitrate"
respectively.
+@section imf
+
+Interoperable Master Format demuxer.
+
+This demuxer presents audio and video streams found in an IMF Composition.
+
@section flv, live_flv, kux
Adobe Flash Video Format demuxer.
@@ -770,6 +776,19 @@ MJPEG stream. Turning this option on by setting it to 1
will result in a stricte
of the boundary value.
@end table
+@section mxf
+
+MXF demuxer.
+
+@table @option
+
+@item -eia608_extract @var{bool}
+Extract EIA-608 captions from SMPTE 436M track
+
+@item -skip_audio_reordering @var{bool}
+This option will disable the audio reordering based on Multi-Channel Audio
(MCA) labelling (SMPTE ST-377-4).
+@end table
+
@section rawvideo
Raw video demuxer.
diff --git a/libavformat/Makefile b/libavformat/Makefile
index 2b5caf9d33..7f058f3ea0 100644
--- a/libavformat/Makefile
+++ b/libavformat/Makefile
@@ -285,6 +285,7 @@ OBJS-$(CONFIG_IMAGE_WEBP_PIPE_DEMUXER)+= img2dec.o
img2.o
OBJS-$(CO
[FFmpeg-devel] [PATCH v6 2/2] avformat/imf: Tests
From: Pierre-Anthony Lemieux Signed-off-by: Pierre-Anthony Lemieux --- Notes: Tests for the IMF demuxer. libavformat/Makefile| 1 + libavformat/tests/imf.c | 508 2 files changed, 509 insertions(+) create mode 100644 libavformat/tests/imf.c diff --git a/libavformat/Makefile b/libavformat/Makefile index 7f058f3ea0..533bb67f31 100644 --- a/libavformat/Makefile +++ b/libavformat/Makefile @@ -697,6 +697,7 @@ TESTPROGS-$(CONFIG_FFRTMPCRYPT_PROTOCOL) += rtmpdh TESTPROGS-$(CONFIG_MOV_MUXER)+= movenc TESTPROGS-$(CONFIG_NETWORK) += noproxy TESTPROGS-$(CONFIG_SRTP) += srtp +TESTPROGS-$(CONFIG_IMF_DEMUXER) += imf TOOLS = aviocat \ ismindex\ diff --git a/libavformat/tests/imf.c b/libavformat/tests/imf.c new file mode 100644 index 00..0e0f2146f1 --- /dev/null +++ b/libavformat/tests/imf.c @@ -0,0 +1,508 @@ +/* + * This file is part of FFmpeg. + * + * Copyright (c) Sandflow Consulting LLC + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are met: + * + * * Redistributions of source code must retain the above copyright notice, this + * list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright notice, + * this list of conditions and the following disclaimer in the documentation + * and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" + * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE + * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS + * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN + * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGE. + */ + +/** + * Tests for IMF CPL and ASSETMAP processing + * + * @author Valentin Noel + * @author Pierre-Anthony Lemieux + * @file + * @ingroup lavu_imf + */ + +#include "libavformat/imf_cpl.c" +#include "libavformat/imfdec.c" +#include "libavformat/mxf.h" + +#include + +const char *cpl_doc = +"http://www.smpte-ra.org/schemas/2067-3/2016\""; +" xmlns:cc=\"http://www.smpte-ra.org/schemas/2067-2/2016\""; +" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\";>" +"urn:uuid:8713c020-2489-45f5-a9f7-87be539e20b5" +"2021-07-13T17:06:22Z" +"FFMPEG" +"FFMPEG sample content" +"" +" " +"urn:uuid:8e097bb0-cff7-4969-a692-bad47bfb528f" +" " +"" +"24000 1001" +"" +"" +"urn:uuid:81fed4e5-9722-400a-b9d1-7f2bd21df4b6" +"" +"" +"urn:uuid:16327185-9205-47ef-a17b-ee28df251db7" +"urn:uuid:461f5424-8f6e-48a9-a385-5eda46fda381" +"" +"" +"urn:uuid:ea3d0f23-55d6-4e03-86ec-cfe0666f0e6a" +"24" +"" +"LFOA" +"5" +"" +"" +"" +"" +"" +"urn:uuid:6ae100b0-92d1-41be-9321-85e0933dfc42" +"urn:uuid:e8ef9653-565c-479c-8039-82d4547973c5" +"" +"" +"urn:uuid:7d418acb-07a3-4e57-984c-b8ea2f7de4ec" +"24" + "urn:uuid:f00e49a8-0dec-4e6c-95e7-078df988b751" +"urn:uuid:6f768ca4-c89e-4dac-9056-a29425d40ba1" +"" +"" +"" +"" +"urn:uuid:754dae53-c25f-4f3c-97e4-2bfe5463f83b" +"urn:uuid:68e3fae5-d94b-44d2-92a6-b94877fbcdb5" +"" +"" +"urn:uuid:61ce2a70-10a2-4521-850b-4218755ff3c9" +"24" + "urn:uuid:f00e49a8-0dec-4e6c-95e7-078df988b751" +"urn:uuid:381dadd2-061e-46cc-a63a-e3d58ce7f488" +"" +"" +"" +"" +"urn:uuid:d29b3884-6633-4dad-9c67-7154af342bc6" +"urn:uuid:6978c106-95bc-424b-a17c-628206a5892d" +"" +"" +"urn:uuid:001ea472-f5da-436c-86de-acaa68c1a7e4" +"24" + "urn:uuid:f00e49a8-0dec-4e6c-95e7-078df988b751" +"urn:uuid:381dadd2-061e-46cc-a63a-e3d58ce7f488" +"" +"" +"" +"" +"urn:uuid:02af22bf-f776-488a-b001-eb6e16953119" +"urn:uuid:19ff6da1-be79-4235-8d04-42201ad06e65" +"" +"" +"urn:uuid:dfa84292-0609-4097-824c-8e2e15e2ce4d" +"24" + "urn:uuid:f00e49a8-0dec-4e6c-95e7-078df988b751" +"urn:uuid:bd6272b6-511e-47c1-93bc-d56ebd314a70" +"" +"" +"" +"" +"" +"" +"urn:uuid:a94be493-cd55-4bf7-b496-ea87bfe38632" +"" +"" +"urn:uuid:20c6020b-1fc0-4080-bcf7-89f09f95bea8" +"urn:uuid:461f5424-8f6e-48a9-a385-5eda46fda381"
Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame for subtitle handling
On Mon, Dec 6, 2021 at 12:23 AM Soft Works wrote: > > It can only work like that. > You literally designed subtitle filtering. You defined how it works. All your explanations come down to some fancy form of "because I made it required". All I'm hearing is that some internal mechanisms in avfilter need it. Why is it a public field in one of the most key structures in our entire codebase? You understand this concern, right? Any public field should first (and only) answer the question, what information does this convey to the user? Does the user need this information? I'm not seeing any of that in your explanations. All I'm seeing is internal avfilter workings - cemented in public API. You are not integrating into an existing infrastructure, you are literally trying to add the whole thing, so you can change it to not need this redundant field. - Hendrik ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame for subtitle handling
> -Original Message- > From: ffmpeg-devel On Behalf Of Hendrik > Leppkes > Sent: Monday, December 6, 2021 12:37 AM > To: FFmpeg development discussions and patches > Subject: Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame > for subtitle handling > > On Mon, Dec 6, 2021 at 12:23 AM Soft Works wrote: > > > > It can only work like that. > > > > You literally designed subtitle filtering. You defined how it works. > All your explanations come down to some fancy form of "because I made > it required". The origin was in the context of keeping sub2video and the existing heartbeat functionality working, And in the end it's still about heartbeat and filter flow. Removing the field and replacing with the main pts field makes 18 fate tests fail. Those tests are not testing the new filtering. I guess that shouldn't happen when the field would be redundant. Thanks, sw ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame for subtitle handling
> -Original Message- > From: ffmpeg-devel On Behalf Of Hendrik > Leppkes > Sent: Monday, December 6, 2021 12:37 AM > To: FFmpeg development discussions and patches > Subject: Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame > for subtitle handling > > On Mon, Dec 6, 2021 at 12:23 AM Soft Works wrote: > > > > It can only work like that. > > > > You literally designed subtitle filtering. You defined how it works. > All your explanations come down to some fancy form of "because I made > it required". > > All I'm hearing is that some internal mechanisms in avfilter need it. > Why is it a public field in one of the most key structures in our > entire codebase? You understand this concern, right? Any public field > should first (and only) answer the question, what information does > this convey to the user? Does the user need this information? I'm not > seeing any of that in your explanations. All I'm seeing is internal > avfilter workings - cemented in public API. > > You are not integrating into an existing infrastructure, you are > literally trying to add the whole thing, so you can change it to not > need this redundant field. > > - Hendrik OK, maybe this is better to understand. Very simple burn-down: I send a subtitle frame into a filtergraph. pts = 10 subtitle_pts = 10 Now I need to re-send a duplicate of that frame into the graph (=> heartbeat - to keep the graph flowing) Now, it's not possible (by lavf design, not mine) to send another frame without increasing the pts of the frame. So we send it like this: pts = 11 subtitle_pts = 10 The pts needs to be increased. But the start-display time of the subtitle event must not be changed - it's still the same frame like before, just a duplicate. softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame for subtitle handling
On Mon, Dec 6, 2021 at 1:24 AM Soft Works wrote: > > > > > -Original Message- > > From: ffmpeg-devel On Behalf Of Hendrik > > Leppkes > > Sent: Monday, December 6, 2021 12:37 AM > > To: FFmpeg development discussions and patches > > Subject: Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame > > for subtitle handling > > > > On Mon, Dec 6, 2021 at 12:23 AM Soft Works wrote: > > > > > > It can only work like that. > > > > > > > You literally designed subtitle filtering. You defined how it works. > > All your explanations come down to some fancy form of "because I made > > it required". > > > > All I'm hearing is that some internal mechanisms in avfilter need it. > > Why is it a public field in one of the most key structures in our > > entire codebase? You understand this concern, right? Any public field > > should first (and only) answer the question, what information does > > this convey to the user? Does the user need this information? I'm not > > seeing any of that in your explanations. All I'm seeing is internal > > avfilter workings - cemented in public API. > > > > You are not integrating into an existing infrastructure, you are > > literally trying to add the whole thing, so you can change it to not > > need this redundant field. > > > > - Hendrik > > OK, maybe this is better to understand. Very simple burn-down: > > I send a subtitle frame into a filtergraph. > > pts = 10 > subtitle_pts = 10 > > Now I need to re-send a duplicate of that frame into the graph > (=> heartbeat - to keep the graph flowing) > > Now, it's not possible (by lavf design, not mine) to send another frame > without increasing the pts of the frame. So we send it like this: > > pts = 11 > subtitle_pts = 10 > > The pts needs to be increased. But the start-display time of the > subtitle event must not be changed - it's still the same frame like > before, just a duplicate. > You just keep re-iterating that avfilter internals need it. And I'll just keep saying that avfilter internals shall not define public API design. I have not heard anything that justifies yet another field that has no benefit to the external public API - to the contrary even, a user would assume pts is pts, and he would be wrong. - Hendrik ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame for subtitle handling
> -Original Message- > From: ffmpeg-devel On Behalf Of Hendrik > Leppkes > Sent: Monday, December 6, 2021 1:37 AM > To: FFmpeg development discussions and patches > Subject: Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame > for subtitle handling > > On Mon, Dec 6, 2021 at 1:24 AM Soft Works wrote: > > > > > > > > > -Original Message- > > > From: ffmpeg-devel On Behalf Of Hendrik > > > Leppkes > > > Sent: Monday, December 6, 2021 12:37 AM > > > To: FFmpeg development discussions and patches > > > Subject: Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare > AVFrame > > > for subtitle handling > > > > > > On Mon, Dec 6, 2021 at 12:23 AM Soft Works wrote: > > > > > > > > It can only work like that. > > > > > > > > > > You literally designed subtitle filtering. You defined how it works. > > > All your explanations come down to some fancy form of "because I made > > > it required". > > > > > > All I'm hearing is that some internal mechanisms in avfilter need it. > > > Why is it a public field in one of the most key structures in our > > > entire codebase? You understand this concern, right? Any public field > > > should first (and only) answer the question, what information does > > > this convey to the user? Does the user need this information? I'm not > > > seeing any of that in your explanations. All I'm seeing is internal > > > avfilter workings - cemented in public API. > > > > > > You are not integrating into an existing infrastructure, you are > > > literally trying to add the whole thing, so you can change it to not > > > need this redundant field. > > > > > > - Hendrik > > > > OK, maybe this is better to understand. Very simple burn-down: > > > > I send a subtitle frame into a filtergraph. > > > > pts = 10 > > subtitle_pts = 10 > > > > Now I need to re-send a duplicate of that frame into the graph > > (=> heartbeat - to keep the graph flowing) > > > > Now, it's not possible (by lavf design, not mine) to send another frame > > without increasing the pts of the frame. So we send it like this: > > > > pts = 11 > > subtitle_pts = 10 > > > > The pts needs to be increased. But the start-display time of the > > subtitle event must not be changed - it's still the same frame like > > before, just a duplicate. > > > > You just keep re-iterating that avfilter internals need it. And I'll > just keep saying that avfilter internals shall not define public API > design. I have not heard anything that justifies yet another field > that has no benefit to the external public API - to the contrary even, > a user would assume pts is pts, and he would be wrong. So, what solution would you suggest, then? sw ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 01/10] avformat/utils: Make ff_data_to_hex() zero-terminate the string
Most callers want it that way anyway.
Signed-off-by: Andreas Rheinhardt
---
libavformat/hls.c | 2 --
libavformat/hlsenc.c| 1 -
libavformat/httpauth.c | 5 -
libavformat/internal.h | 11 +++
libavformat/movenc.c| 1 -
libavformat/omadec.c| 1 -
libavformat/rtmpproto.c | 2 --
libavformat/sdp.c | 3 ---
libavformat/takdec.c| 1 -
libavformat/utils.c | 1 +
10 files changed, 12 insertions(+), 16 deletions(-)
diff --git a/libavformat/hls.c b/libavformat/hls.c
index 52a031ed54..557faf8e8d 100644
--- a/libavformat/hls.c
+++ b/libavformat/hls.c
@@ -1284,7 +1284,6 @@ static int open_input(HLSContext *c, struct playlist
*pls, struct segment *seg,
char iv[33], key[33], url[MAX_URL_SIZE];
ff_data_to_hex(iv, seg->iv, sizeof(seg->iv), 0);
ff_data_to_hex(key, pls->key, sizeof(pls->key), 0);
-iv[32] = key[32] = '\0';
if (strstr(seg->url, "://"))
snprintf(url, sizeof(url), "crypto+%s", seg->url);
else
@@ -2074,7 +2073,6 @@ static int hls_read_header(AVFormatContext *s)
if (strstr(in_fmt->name, "mov")) {
char key[33];
ff_data_to_hex(key, pls->key, sizeof(pls->key), 0);
-key[32] = '\0';
av_dict_set(&options, "decryption_key", key,
AV_OPT_FLAG_DECODING_PARAM);
} else if (!c->crypto_ctx.aes_ctx) {
c->crypto_ctx.aes_ctx = av_aes_alloc();
diff --git a/libavformat/hlsenc.c b/libavformat/hlsenc.c
index cfd0c036d1..1c2a556375 100644
--- a/libavformat/hlsenc.c
+++ b/libavformat/hlsenc.c
@@ -745,7 +745,6 @@ static int do_encrypt(AVFormatContext *s, VariantStream *vs)
memcpy(iv, hls->iv, sizeof(iv));
}
ff_data_to_hex(buf, iv, sizeof(iv), 0);
-buf[32] = '\0';
memcpy(hls->iv_string, buf, sizeof(hls->iv_string));
}
diff --git a/libavformat/httpauth.c b/libavformat/httpauth.c
index 4f79c78edc..0a98ff80a5 100644
--- a/libavformat/httpauth.c
+++ b/libavformat/httpauth.c
@@ -156,7 +156,6 @@ static char *make_digest_auth(HTTPAuthState *state, const
char *username,
for (i = 0; i < 2; i++)
cnonce_buf[i] = av_get_random_seed();
ff_data_to_hex(cnonce, (const uint8_t*) cnonce_buf, sizeof(cnonce_buf), 1);
-cnonce[2*sizeof(cnonce_buf)] = 0;
md5ctx = av_md5_alloc();
if (!md5ctx)
@@ -166,7 +165,6 @@ static char *make_digest_auth(HTTPAuthState *state, const
char *username,
update_md5_strings(md5ctx, username, ":", state->realm, ":", password,
NULL);
av_md5_final(md5ctx, hash);
ff_data_to_hex(A1hash, hash, 16, 1);
-A1hash[32] = 0;
if (!strcmp(digest->algorithm, "") || !strcmp(digest->algorithm, "MD5")) {
} else if (!strcmp(digest->algorithm, "MD5-sess")) {
@@ -174,7 +172,6 @@ static char *make_digest_auth(HTTPAuthState *state, const
char *username,
update_md5_strings(md5ctx, A1hash, ":", digest->nonce, ":", cnonce,
NULL);
av_md5_final(md5ctx, hash);
ff_data_to_hex(A1hash, hash, 16, 1);
-A1hash[32] = 0;
} else {
/* Unsupported algorithm */
av_free(md5ctx);
@@ -185,7 +182,6 @@ static char *make_digest_auth(HTTPAuthState *state, const
char *username,
update_md5_strings(md5ctx, method, ":", uri, NULL);
av_md5_final(md5ctx, hash);
ff_data_to_hex(A2hash, hash, 16, 1);
-A2hash[32] = 0;
av_md5_init(md5ctx);
update_md5_strings(md5ctx, A1hash, ":", digest->nonce, NULL);
@@ -195,7 +191,6 @@ static char *make_digest_auth(HTTPAuthState *state, const
char *username,
update_md5_strings(md5ctx, ":", A2hash, NULL);
av_md5_final(md5ctx, hash);
ff_data_to_hex(response, hash, 16, 1);
-response[32] = 0;
av_free(md5ctx);
diff --git a/libavformat/internal.h b/libavformat/internal.h
index 20e93d9267..f43e408548 100644
--- a/libavformat/internal.h
+++ b/libavformat/internal.h
@@ -484,6 +484,17 @@ void ff_flush_packet_queue(AVFormatContext *s);
*/
int ff_mkdir_p(const char *path);
+/**
+ * Write hexadecimal string corresponding to given binary data. The string
+ * is zero-terminated.
+ *
+ * @param buf the output string is written here;
+ * needs to be at least 2 * size + 1 bytes long.
+ * @param src the input data to be transformed.
+ * @param size the size (in byte) of src.
+ * @param lowercase determines whether to use the range [0-9a-f] or [0-9A-F].
+ * @return buf.
+ */
char *ff_data_to_hex(char *buf, const uint8_t *src, int size, int lowercase);
/**
diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index 38ff90833a..0f912dd012 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -4290,7 +4290,6 @@ static void param_write_hex(AVIOContext *pb, const char
*name, const uint8_t *va
char buf[150];
len = FFMIN(sizeof(buf) / 2 - 1, len);
ff_data_to_hex(buf, value, len, 0);
-buf[2 * len] = '\0';
avio_printf(pb, "\n",
nam
[FFmpeg-devel] [PATCH 02/10] avformat/aadec: Don't use the same loop counter in inner and outer loop
Due to this bush.aa (from the FATE suite) exported garbage metadata
with key "_040930".
Signed-off-by: Andreas Rheinhardt
---
libavformat/aadec.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/libavformat/aadec.c b/libavformat/aadec.c
index 72f4b6f7cc..a3a7664469 100644
--- a/libavformat/aadec.c
+++ b/libavformat/aadec.c
@@ -132,8 +132,8 @@ static int aa_read_header(AVFormatContext *s)
AV_WB32(&header_key[idx * 4], header_key_part[idx]); //
convert each part to BE!
}
av_log(s, AV_LOG_DEBUG, "Processed HeaderKey is ");
-for (i = 0; i < 16; i++)
-av_log(s, AV_LOG_DEBUG, "%02x", header_key[i]);
+for (int j = 0; j < 16; j++)
+av_log(s, AV_LOG_DEBUG, "%02x", header_key[j]);
av_log(s, AV_LOG_DEBUG, "\n");
} else {
av_dict_set(&s->metadata, key, val, 0);
--
2.32.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 03/10] avformat/aadec: Avoid copying data around
Up until now, the packets have been read in blocks of at most
eight bytes at a time; these blocks have then been decrypted
and then copied into a buffer on the stack (that was double
the size needed...). From there they have been copied to a packet.
This commit changes this: The data is read in one go; and
the decryption avoids temporary buffers, too, by making
use of the fact that src and dst of av_tea_crypt() can coincide.
Signed-off-by: Andreas Rheinhardt
---
libavformat/aadec.c | 42 +-
1 file changed, 13 insertions(+), 29 deletions(-)
diff --git a/libavformat/aadec.c b/libavformat/aadec.c
index a3a7664469..7e97120070 100644
--- a/libavformat/aadec.c
+++ b/libavformat/aadec.c
@@ -32,7 +32,6 @@
#include "libavutil/opt.h"
#define AA_MAGIC 1469084982 /* this identifies an audible .aa file */
-#define MAX_CODEC_SECOND_SIZE 3982
#define MAX_TOC_ENTRIES 16
#define MAX_DICTIONARY_ENTRIES 128
#define TEA_BLOCK_SIZE 8
@@ -247,13 +246,9 @@ static int aa_read_header(AVFormatContext *s)
static int aa_read_packet(AVFormatContext *s, AVPacket *pkt)
{
-uint8_t dst[TEA_BLOCK_SIZE];
-uint8_t src[TEA_BLOCK_SIZE];
int i;
-int trailing_bytes;
int blocks;
-uint8_t buf[MAX_CODEC_SECOND_SIZE * 2];
-int written = 0;
+uint8_t *buf;
int ret;
AADemuxContext *c = s->priv_data;
uint64_t pos = avio_tell(s->pb);
@@ -272,7 +267,6 @@ static int aa_read_packet(AVFormatContext *s, AVPacket *pkt)
av_log(s, AV_LOG_DEBUG, "Chapter %d (%" PRId64 " bytes)\n",
c->chapter_idx, c->current_chapter_size);
c->chapter_idx = c->chapter_idx + 1;
avio_skip(s->pb, 4); // data start offset
-pos += 8;
c->current_codec_second_size = c->codec_second_size;
}
@@ -281,24 +275,18 @@ static int aa_read_packet(AVFormatContext *s, AVPacket
*pkt)
c->current_codec_second_size = c->current_chapter_size %
c->current_codec_second_size;
}
+ret = av_get_packet(s->pb, pkt, c->current_codec_second_size);
+if (ret != c->current_codec_second_size)
+return AVERROR_EOF;
+
+buf = pkt->data;
// decrypt c->current_codec_second_size bytes
+// trailing bytes are left unencrypted!
blocks = c->current_codec_second_size / TEA_BLOCK_SIZE;
for (i = 0; i < blocks; i++) {
-ret = avio_read(s->pb, src, TEA_BLOCK_SIZE);
-if (ret != TEA_BLOCK_SIZE)
-return (ret < 0) ? ret : AVERROR_EOF;
av_tea_init(c->tea_ctx, c->file_key, 16);
-av_tea_crypt(c->tea_ctx, dst, src, 1, NULL, 1);
-memcpy(buf + written, dst, TEA_BLOCK_SIZE);
-written = written + TEA_BLOCK_SIZE;
-}
-trailing_bytes = c->current_codec_second_size % TEA_BLOCK_SIZE;
-if (trailing_bytes != 0) { // trailing bytes are left unencrypted!
-ret = avio_read(s->pb, src, trailing_bytes);
-if (ret != trailing_bytes)
-return (ret < 0) ? ret : AVERROR_EOF;
-memcpy(buf + written, src, trailing_bytes);
-written = written + trailing_bytes;
+av_tea_crypt(c->tea_ctx, buf, buf, 1, NULL, 1);
+buf += TEA_BLOCK_SIZE;
}
// update state
@@ -306,16 +294,12 @@ static int aa_read_packet(AVFormatContext *s, AVPacket
*pkt)
if (c->current_chapter_size <= 0)
c->current_chapter_size = 0;
-if (c->seek_offset > written)
+if (c->seek_offset > c->current_codec_second_size)
c->seek_offset = 0; // ignore wrong estimate
-
-ret = av_new_packet(pkt, written - c->seek_offset);
-if (ret < 0)
-return ret;
-memcpy(pkt->data, buf + c->seek_offset, written - c->seek_offset);
-pkt->pos = pos;
-
+pkt->data += c->seek_offset;
+pkt->size -= c->seek_offset;
c->seek_offset = 0;
+
return 0;
}
--
2.32.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 04/10] avformat/aadec: Don't unnecessarily reinitialize AVTEA context
We use ECB, not CBC mode here, so one does not need to reinitialize
the context; for the same reason, one can also just let av_tea_crypt()
loop over the blocks, avoiding a loop here.
Signed-off-by: Andreas Rheinhardt
---
libavformat/aadec.c | 15 ---
1 file changed, 4 insertions(+), 11 deletions(-)
diff --git a/libavformat/aadec.c b/libavformat/aadec.c
index 7e97120070..24116b1f70 100644
--- a/libavformat/aadec.c
+++ b/libavformat/aadec.c
@@ -173,6 +173,7 @@ static int aa_read_header(AVFormatContext *s)
for (i = 0; i < 16; i++)
av_log(s, AV_LOG_DEBUG, "%02x", c->file_key[i]);
av_log(s, AV_LOG_DEBUG, "\n");
+av_tea_init(c->tea_ctx, c->file_key, 16);
/* decoder setup */
st = avformat_new_stream(s, NULL);
@@ -246,9 +247,6 @@ static int aa_read_header(AVFormatContext *s)
static int aa_read_packet(AVFormatContext *s, AVPacket *pkt)
{
-int i;
-int blocks;
-uint8_t *buf;
int ret;
AADemuxContext *c = s->priv_data;
uint64_t pos = avio_tell(s->pb);
@@ -279,15 +277,10 @@ static int aa_read_packet(AVFormatContext *s, AVPacket
*pkt)
if (ret != c->current_codec_second_size)
return AVERROR_EOF;
-buf = pkt->data;
-// decrypt c->current_codec_second_size bytes
+// decrypt c->current_codec_second_size bytes in blocks of TEA_BLOCK_SIZE
// trailing bytes are left unencrypted!
-blocks = c->current_codec_second_size / TEA_BLOCK_SIZE;
-for (i = 0; i < blocks; i++) {
-av_tea_init(c->tea_ctx, c->file_key, 16);
-av_tea_crypt(c->tea_ctx, buf, buf, 1, NULL, 1);
-buf += TEA_BLOCK_SIZE;
-}
+av_tea_crypt(c->tea_ctx, pkt->data, pkt->data,
+ c->current_codec_second_size / TEA_BLOCK_SIZE, NULL, 1);
// update state
c->current_chapter_size = c->current_chapter_size -
c->current_codec_second_size;
--
2.32.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 05/10] avformat/aadec: Simplify deriving file key
Don't use different src and dst in av_tea_crypt(); use in-place
modifications instead. Also let av_tea_crypt() encrypt all three
blocks in one call.
Signed-off-by: Andreas Rheinhardt
---
libavformat/aadec.c | 22 +++---
1 file changed, 7 insertions(+), 15 deletions(-)
diff --git a/libavformat/aadec.c b/libavformat/aadec.c
index 24116b1f70..5f49a543e4 100644
--- a/libavformat/aadec.c
+++ b/libavformat/aadec.c
@@ -71,10 +71,10 @@ static int get_second_size(char *codec_name)
static int aa_read_header(AVFormatContext *s)
{
-int i, j, idx, largest_idx = -1;
+int i, idx, largest_idx = -1;
uint32_t toc_size, npairs, header_seed = 0, start;
char codec_name[64] = {0};
-uint8_t output[24], dst[8], src[8];
+uint8_t buf[24];
int64_t largest_size = -1, current_size = -1, chapter_pos;
struct toc_entry {
uint32_t offset;
@@ -156,19 +156,11 @@ static int aa_read_header(AVFormatContext *s)
if (!c->tea_ctx)
return AVERROR(ENOMEM);
av_tea_init(c->tea_ctx, c->aa_fixed_key, 16);
-output[0] = output[1] = 0; // purely for padding purposes
-memcpy(output + 2, header_key, 16);
-idx = 0;
-for (i = 0; i < 3; i++) { // TEA CBC with weird mixed endianness
-AV_WB32(src, header_seed);
-AV_WB32(src + 4, header_seed + 1);
-header_seed += 2;
-av_tea_crypt(c->tea_ctx, dst, src, 1, NULL, 0); // TEA ECB encrypt
-for (j = 0; j < TEA_BLOCK_SIZE && idx < 18; j+=1, idx+=1) {
-output[idx] = output[idx] ^ dst[j];
-}
-}
-memcpy(c->file_key, output + 2, 16); // skip first 2 bytes of output
+for (int i = 0; i < 6; i++)
+AV_WB32(buf + 4 * i, header_seed + i);
+av_tea_crypt(c->tea_ctx, buf, buf, 3, NULL, 0);
+AV_WN64(c->file_key, AV_RN64(buf + 2) ^ AV_RN64(header_key));
+AV_WN64(c->file_key + 8, AV_RN64(buf + 10) ^ AV_RN64(header_key + 8));
av_log(s, AV_LOG_DEBUG, "File key is ");
for (i = 0; i < 16; i++)
av_log(s, AV_LOG_DEBUG, "%02x", c->file_key[i]);
--
2.32.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 06/10] avformat/aadec: Simplify data->hex conversion
Signed-off-by: Andreas Rheinhardt
---
libavformat/aadec.c | 13 +
1 file changed, 5 insertions(+), 8 deletions(-)
diff --git a/libavformat/aadec.c b/libavformat/aadec.c
index 5f49a543e4..840a9968c6 100644
--- a/libavformat/aadec.c
+++ b/libavformat/aadec.c
@@ -83,6 +83,7 @@ static int aa_read_header(AVFormatContext *s)
uint32_t header_key_part[4];
uint8_t header_key[16] = {0};
AADemuxContext *c = s->priv_data;
+char file_key[2 * sizeof(c->file_key) + 1];
AVIOContext *pb = s->pb;
AVStream *st;
FFStream *sti;
@@ -130,10 +131,8 @@ static int aa_read_header(AVFormatContext *s)
for (idx = 0; idx < 4; idx++) {
AV_WB32(&header_key[idx * 4], header_key_part[idx]); //
convert each part to BE!
}
-av_log(s, AV_LOG_DEBUG, "Processed HeaderKey is ");
-for (int j = 0; j < 16; j++)
-av_log(s, AV_LOG_DEBUG, "%02x", header_key[j]);
-av_log(s, AV_LOG_DEBUG, "\n");
+ff_data_to_hex(key, header_key, sizeof(header_key), 1);
+av_log(s, AV_LOG_DEBUG, "Processed HeaderKey is %s\n", key);
} else {
av_dict_set(&s->metadata, key, val, 0);
}
@@ -161,10 +160,8 @@ static int aa_read_header(AVFormatContext *s)
av_tea_crypt(c->tea_ctx, buf, buf, 3, NULL, 0);
AV_WN64(c->file_key, AV_RN64(buf + 2) ^ AV_RN64(header_key));
AV_WN64(c->file_key + 8, AV_RN64(buf + 10) ^ AV_RN64(header_key + 8));
-av_log(s, AV_LOG_DEBUG, "File key is ");
-for (i = 0; i < 16; i++)
-av_log(s, AV_LOG_DEBUG, "%02x", c->file_key[i]);
-av_log(s, AV_LOG_DEBUG, "\n");
+ff_data_to_hex(file_key, c->file_key, sizeof(c->file_key), 1);
+av_log(s, AV_LOG_DEBUG, "File key is %s\n", file_key);
av_tea_init(c->tea_ctx, c->file_key, 16);
/* decoder setup */
--
2.32.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
