[FFmpeg-devel] [PATCH] configure: link to libatomic when it's present
C11 atomics in some configurations (e.g. 64bit operations on ppc64 with GCC) require linking to libatomic. --- Testing welcome, especially in configurations where * libatomic is not present * libatomic is actually needed --- configure | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/configure b/configure index 1413122d87..1ff5dbee5b 100755 --- a/configure +++ b/configure @@ -6324,7 +6324,14 @@ check_headers asm/types.h # it seems there are versions of clang in some distros that try to use the # gcc headers, which explodes for stdatomic # so we also check that atomics actually work here -check_builtin stdatomic stdatomic.h "atomic_int foo, bar = ATOMIC_VAR_INIT(-1); atomic_store(&foo, 0); foo += bar" +# +# some configurations also require linking to libatomic, so try +# both with -latomic and without +for LATOMIC in "-latomic" ""; do +check_builtin stdatomic stdatomic.h \ +"atomic_int foo, bar = ATOMIC_VAR_INIT(-1); atomic_store(&foo, 0); foo += bar" \ +$LATOMIC && add_extralibs $LATOMIC && break +done check_lib advapi32 "windows.h"RegCloseKey -ladvapi32 check_lib bcrypt "windows.h bcrypt.h" BCryptGenRandom -lbcrypt && -- 2.33.0 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] configure: link to libatomic when it's present
On 1/19/2022 8:54 AM, Anton Khirnov wrote: C11 atomics in some configurations (e.g. 64bit operations on ppc64 with GCC) require linking to libatomic. --- Testing welcome, especially in configurations where * libatomic is not present * libatomic is actually needed --- configure | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/configure b/configure index 1413122d87..1ff5dbee5b 100755 --- a/configure +++ b/configure @@ -6324,7 +6324,14 @@ check_headers asm/types.h # it seems there are versions of clang in some distros that try to use the # gcc headers, which explodes for stdatomic # so we also check that atomics actually work here -check_builtin stdatomic stdatomic.h "atomic_int foo, bar = ATOMIC_VAR_INIT(-1); atomic_store(&foo, 0); foo += bar" +# +# some configurations also require linking to libatomic, so try +# both with -latomic and without +for LATOMIC in "-latomic" ""; do Shouldn't you try without it first? On my toolchain libatomic is present, but libraries compile without linking to it just fine. That changes after this patch, where it starts linking to it explicitly. +check_builtin stdatomic stdatomic.h \ +"atomic_int foo, bar = ATOMIC_VAR_INIT(-1); atomic_store(&foo, 0); foo += bar" \ +$LATOMIC && add_extralibs $LATOMIC && break You should probably add it to the required libraries' extralibs only. Just replace the add_extralibs part with setting stdatomic_extralibs to $LATOMIC, and then add stdatomic to all the libraries' _suggest lists, same as we do for libm. +done check_lib advapi32 "windows.h"RegCloseKey -ladvapi32 check_lib bcrypt "windows.h bcrypt.h" BCryptGenRandom -lbcrypt && ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] configure: link to libatomic when it's present
James Almer: > > > On 1/19/2022 8:54 AM, Anton Khirnov wrote: >> C11 atomics in some configurations (e.g. 64bit operations on ppc64 with >> GCC) require linking to libatomic. >> --- >> Testing welcome, especially in configurations where >> * libatomic is not present >> * libatomic is actually needed >> --- >> configure | 9 - >> 1 file changed, 8 insertions(+), 1 deletion(-) >> >> diff --git a/configure b/configure >> index 1413122d87..1ff5dbee5b 100755 >> --- a/configure >> +++ b/configure >> @@ -6324,7 +6324,14 @@ check_headers asm/types.h >> # it seems there are versions of clang in some distros that try to >> use the >> # gcc headers, which explodes for stdatomic >> # so we also check that atomics actually work here >> -check_builtin stdatomic stdatomic.h "atomic_int foo, bar = >> ATOMIC_VAR_INIT(-1); atomic_store(&foo, 0); foo += bar" >> +# >> +# some configurations also require linking to libatomic, so try >> +# both with -latomic and without >> +for LATOMIC in "-latomic" ""; do > > Shouldn't you try without it first? On my toolchain libatomic is > present, but libraries compile without linking to it just fine. That > changes after this patch, where it starts linking to it explicitly. > This would work if this test checked for all the atomic operations that might be needed; but it doesn't: It checks just for atomic increment and only for atomic_int. What is if atomic_increment can be done without recourse to libatomic because of hardware support whereas another atomic operation needs libatomic? >> + check_builtin stdatomic >> stdatomic.h \ >> + "atomic_int foo, bar = ATOMIC_VAR_INIT(-1); >> atomic_store(&foo, 0); foo += bar" \ >> + $LATOMIC && add_extralibs $LATOMIC && break > > You should probably add it to the required libraries' extralibs only. > Just replace the add_extralibs part with setting stdatomic_extralibs to > $LATOMIC, and then add stdatomic to all the libraries' _suggest lists, > same as we do for libm. > >> +done >> check_lib advapi32 "windows.h" RegCloseKey >> -ladvapi32 >> check_lib bcrypt "windows.h bcrypt.h" BCryptGenRandom >> -lbcrypt && ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] configure: link to libatomic when it's present
Quoting James Almer (2022-01-19 13:34:20) > > > On 1/19/2022 8:54 AM, Anton Khirnov wrote: > > C11 atomics in some configurations (e.g. 64bit operations on ppc64 with > > GCC) require linking to libatomic. > > --- > > Testing welcome, especially in configurations where > > * libatomic is not present > > * libatomic is actually needed > > --- > > configure | 9 - > > 1 file changed, 8 insertions(+), 1 deletion(-) > > > > diff --git a/configure b/configure > > index 1413122d87..1ff5dbee5b 100755 > > --- a/configure > > +++ b/configure > > @@ -6324,7 +6324,14 @@ check_headers asm/types.h > > # it seems there are versions of clang in some distros that try to use the > > # gcc headers, which explodes for stdatomic > > # so we also check that atomics actually work here > > -check_builtin stdatomic stdatomic.h "atomic_int foo, bar = > > ATOMIC_VAR_INIT(-1); atomic_store(&foo, 0); foo += bar" > > +# > > +# some configurations also require linking to libatomic, so try > > +# both with -latomic and without > > +for LATOMIC in "-latomic" ""; do > > Shouldn't you try without it first? On my toolchain libatomic is > present, but libraries compile without linking to it just fine. That > changes after this patch, where it starts linking to it explicitly. No, because it may only be needed for some atomic sizes and operations, which the test in configure doesn't necessarily catch. And because we pass as-needed to the linker, the built libraries shouldn't actually require libatomic unless it's really needed. > > > +check_builtin stdatomic stdatomic.h > > \ > > +"atomic_int foo, bar = ATOMIC_VAR_INIT(-1); atomic_store(&foo, 0); > > foo += bar" \ > > +$LATOMIC && add_extralibs $LATOMIC && break > > You should probably add it to the required libraries' extralibs only. > Just replace the add_extralibs part with setting stdatomic_extralibs to > $LATOMIC, and then add stdatomic to all the libraries' _suggest lists, > same as we do for libm. Then we need to actively track which libraries actually use atomics, which also depends on which features are enabled. Given that this only fails on less-common arches, this sounds like a recipe for obscure build failures. Given that it's only really linked when needed, it seems better to just add it unconditionally. -- Anton Khirnov ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] configure: link to libatomic when it's present
On 1/19/2022 9:40 AM, Anton Khirnov wrote: Quoting James Almer (2022-01-19 13:34:20) On 1/19/2022 8:54 AM, Anton Khirnov wrote: C11 atomics in some configurations (e.g. 64bit operations on ppc64 with GCC) require linking to libatomic. --- Testing welcome, especially in configurations where * libatomic is not present * libatomic is actually needed --- configure | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/configure b/configure index 1413122d87..1ff5dbee5b 100755 --- a/configure +++ b/configure @@ -6324,7 +6324,14 @@ check_headers asm/types.h # it seems there are versions of clang in some distros that try to use the # gcc headers, which explodes for stdatomic # so we also check that atomics actually work here -check_builtin stdatomic stdatomic.h "atomic_int foo, bar = ATOMIC_VAR_INIT(-1); atomic_store(&foo, 0); foo += bar" +# +# some configurations also require linking to libatomic, so try +# both with -latomic and without +for LATOMIC in "-latomic" ""; do Shouldn't you try without it first? On my toolchain libatomic is present, but libraries compile without linking to it just fine. That changes after this patch, where it starts linking to it explicitly. No, because it may only be needed for some atomic sizes and operations, which the test in configure doesn't necessarily catch. And because we pass as-needed to the linker, the built libraries shouldn't actually require libatomic unless it's really needed. Ah, didn't consider --as-needed. +check_builtin stdatomic stdatomic.h \ +"atomic_int foo, bar = ATOMIC_VAR_INIT(-1); atomic_store(&foo, 0); foo += bar" \ +$LATOMIC && add_extralibs $LATOMIC && break You should probably add it to the required libraries' extralibs only. Just replace the add_extralibs part with setting stdatomic_extralibs to $LATOMIC, and then add stdatomic to all the libraries' _suggest lists, same as we do for libm. Then we need to actively track which libraries actually use atomics, which also depends on which features are enabled. Given that this only fails on less-common arches, this sounds like a recipe for obscure build failures. Given that it's only really linked when needed, it seems better to just add it unconditionally. Then just add it to all of them, like we do for libm. What i want to avoid is having it in EXTRALIBS. There was a huge configure rework long ago that removed everything from that variable (leaving it as the place where user defined --extra-libs arguments are dumped), and fine tuned ld arguments in a per library/module basis. I'd like to not go back to start dumping everything in EXTRALIBS. Like this, on top of this patch: diff --git a/configure b/configure index 1ff5dbee5b..43713a7679 100755 --- a/configure +++ b/configure @@ -3794,20 +3794,20 @@ cws2fws_extralibs="zlib_extralibs" # libraries, in any order avcodec_deps="avutil" -avcodec_suggest="libm" +avcodec_suggest="libm stdatomic" avdevice_deps="avformat avcodec avutil" -avdevice_suggest="libm" +avdevice_suggest="libm stdatomic" avfilter_deps="avutil" -avfilter_suggest="libm" +avfilter_suggest="libm stdatomic" avformat_deps="avcodec avutil" -avformat_suggest="libm network zlib" -avutil_suggest="clock_gettime ffnvcodec libm libdrm libmfx opencl user32 vaapi vulkan videotoolbox corefoundation corevideo coremedia bcrypt" +avformat_suggest="libm network stdatomic zlib" +avutil_suggest="clock_gettime ffnvcodec libm libdrm libmfx opencl stdatomic user32 vaapi vulkan videotoolbox corefoundation corevideo coremedia bcrypt" postproc_deps="avutil gpl" -postproc_suggest="libm" +postproc_suggest="libm stdatomic" swresample_deps="avutil" -swresample_suggest="libm libsoxr" +swresample_suggest="libm libsoxr stdatomic" swscale_deps="avutil" -swscale_suggest="libm" +swscale_suggest="libm stdatomic" avcodec_extralibs="pthreads_extralibs iconv_extralibs dxva2_extralibs" avfilter_extralibs="pthreads_extralibs" @@ -6330,7 +6330,7 @@ check_headers asm/types.h for LATOMIC in "-latomic" ""; do check_builtin stdatomic stdatomic.h \ "atomic_int foo, bar = ATOMIC_VAR_INIT(-1); atomic_store(&foo, 0); foo += bar" \ -$LATOMIC && add_extralibs $LATOMIC && break +$LATOMIC && eval stdatomic_extralibs="\$LATOMIC" && break done check_lib advapi32 "windows.h"RegCloseKey -ladvapi32 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 1/3] lavu/hwcontext_opencl: clear dangling pointers on map failure
--- untested --- libavutil/hwcontext_opencl.c | 5 + 1 file changed, 5 insertions(+) diff --git a/libavutil/hwcontext_opencl.c b/libavutil/hwcontext_opencl.c index 26a3a24593..40ee611943 100644 --- a/libavutil/hwcontext_opencl.c +++ b/libavutil/hwcontext_opencl.c @@ -2146,6 +2146,7 @@ fail: clReleaseMemObject(mapping->frame.planes[p]); } av_free(mapping); +memset(dst->data, 0, sizeof(dst->data)); return err; } @@ -2317,6 +2318,7 @@ fail: if (desc->planes[p]) clReleaseMemObject(desc->planes[p]); av_freep(&desc); +memset(dst->data, 0, sizeof(dst->data)); return err; } @@ -2407,6 +2409,7 @@ fail: 0, NULL, &event); if (cle == CL_SUCCESS) opencl_wait_events(dst_fc, &event, 1); +memset(dst->data, 0, sizeof(dst->data)); return err; } @@ -2562,6 +2565,7 @@ fail: 0, NULL, &event); if (cle == CL_SUCCESS) opencl_wait_events(dst_fc, &event, 1); +memset(dst->data, 0, sizeof(dst->data)); return err; } @@ -2793,6 +2797,7 @@ fail: clReleaseMemObject(mapping->object_buffers[i]); } av_free(mapping); +memset(dst->data, 0, sizeof(dst->data)); return err; } -- 2.33.0 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 2/3] lavu/hwcontext_vulkan: clear dangling pointers on map failure
--- untested --- libavutil/hwcontext_vulkan.c | 1 + 1 file changed, 1 insertion(+) diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c index 60a6cf6a91..ae19fc2ab6 100644 --- a/libavutil/hwcontext_vulkan.c +++ b/libavutil/hwcontext_vulkan.c @@ -2927,6 +2927,7 @@ static int vulkan_map_from_drm(AVHWFramesContext *hwfc, AVFrame *dst, fail: vulkan_frame_free(hwfc->device_ctx->hwctx, (uint8_t *)f); +dst->data[0] = NULL; return err; } -- 2.33.0 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 3/3] lavu/hwcontext: clarify behavior on av_hwframe_map() failure
Clear anything that av_hwframe_map() might have done to the destination
frame, but leave caller-provided fields unchanged.
---
libavutil/hwcontext.c | 23 +--
libavutil/hwcontext.h | 4
2 files changed, 25 insertions(+), 2 deletions(-)
diff --git a/libavutil/hwcontext.c b/libavutil/hwcontext.c
index 31c7840dba..ae33da1262 100644
--- a/libavutil/hwcontext.c
+++ b/libavutil/hwcontext.c
@@ -18,6 +18,7 @@
#include "config.h"
+#include "avassert.h"
#include "buffer.h"
#include "common.h"
#include "hwcontext.h"
@@ -788,6 +789,8 @@ fail:
int av_hwframe_map(AVFrame *dst, const AVFrame *src, int flags)
{
+AVBufferRef*orig_dst_frames = dst->hw_frames_ctx;
+enum AVPixelFormat orig_dst_fmt = dst->format;
AVHWFramesContext *src_frames, *dst_frames;
HWMapDescriptor *hwmap;
int ret;
@@ -825,7 +828,7 @@ int av_hwframe_map(AVFrame *dst, const AVFrame *src, int
flags)
ret = src_frames->internal->hw_type->map_from(src_frames,
dst, src, flags);
if (ret != AVERROR(ENOSYS))
-return ret;
+goto fail;
}
}
@@ -837,11 +840,27 @@ int av_hwframe_map(AVFrame *dst, const AVFrame *src, int
flags)
ret = dst_frames->internal->hw_type->map_to(dst_frames,
dst, src, flags);
if (ret != AVERROR(ENOSYS))
-return ret;
+goto fail;
}
}
return AVERROR(ENOSYS);
+
+fail:
+// if the caller provided dst frames context, it should be preserved
+// by this function
+av_assert0(orig_dst_frames == NULL ||
+ orig_dst_frames == dst->hw_frames_ctx);
+
+// preserve user-provided dst frame fields, but clean
+// anything we might have set
+dst->hw_frames_ctx = NULL;
+av_frame_unref(dst);
+
+dst->hw_frames_ctx = orig_dst_frames;
+dst->format= orig_dst_fmt;
+
+return ret;
}
int av_hwframe_ctx_create_derived(AVBufferRef **derived_frame_ctx,
diff --git a/libavutil/hwcontext.h b/libavutil/hwcontext.h
index 04d19d89c2..c18b7e1e8b 100644
--- a/libavutil/hwcontext.h
+++ b/libavutil/hwcontext.h
@@ -571,6 +571,10 @@ enum {
* possible with the given arguments and hwframe setup, while other return
* values indicate that it failed somehow.
*
+ * On failure, the destination frame will be left blank, except for the
+ * hw_frames_ctx/format fields thay may have been set by the caller - those
will
+ * be preserved as they were.
+ *
* @param dst Destination frame, to contain the mapping.
* @param src Source frame, to be mapped.
* @param flags Some combination of AV_HWFRAME_MAP_* flags.
--
2.33.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 3/3] lavu/hwcontext: clarify behavior on av_hwframe_map() failure
On 1/19/2022 10:40 AM, Anton Khirnov wrote:
Clear anything that av_hwframe_map() might have done to the destination
frame, but leave caller-provided fields unchanged.
---
libavutil/hwcontext.c | 23 +--
libavutil/hwcontext.h | 4
2 files changed, 25 insertions(+), 2 deletions(-)
diff --git a/libavutil/hwcontext.c b/libavutil/hwcontext.c
index 31c7840dba..ae33da1262 100644
--- a/libavutil/hwcontext.c
+++ b/libavutil/hwcontext.c
@@ -18,6 +18,7 @@
#include "config.h"
+#include "avassert.h"
#include "buffer.h"
#include "common.h"
#include "hwcontext.h"
@@ -788,6 +789,8 @@ fail:
int av_hwframe_map(AVFrame *dst, const AVFrame *src, int flags)
{
+AVBufferRef*orig_dst_frames = dst->hw_frames_ctx;
+enum AVPixelFormat orig_dst_fmt = dst->format;
AVHWFramesContext *src_frames, *dst_frames;
HWMapDescriptor *hwmap;
int ret;
@@ -825,7 +828,7 @@ int av_hwframe_map(AVFrame *dst, const AVFrame *src, int
flags)
ret = src_frames->internal->hw_type->map_from(src_frames,
dst, src, flags);
if (ret != AVERROR(ENOSYS))
-return ret;
+goto fail;
What if ret is 0? Is the stuff in the fail label meant to run on success?
}
}
@@ -837,11 +840,27 @@ int av_hwframe_map(AVFrame *dst, const AVFrame *src, int flags)
ret = dst_frames->internal->hw_type->map_to(dst_frames,
dst, src, flags);
if (ret != AVERROR(ENOSYS))
-return ret;
+goto fail;
}
}
return AVERROR(ENOSYS);
+
+fail:
+// if the caller provided dst frames context, it should be preserved
+// by this function
+av_assert0(orig_dst_frames == NULL ||
+ orig_dst_frames == dst->hw_frames_ctx);
+
+// preserve user-provided dst frame fields, but clean
+// anything we might have set
+dst->hw_frames_ctx = NULL;
+av_frame_unref(dst);
+
+dst->hw_frames_ctx = orig_dst_frames;
+dst->format= orig_dst_fmt;
+
+return ret;
}
int av_hwframe_ctx_create_derived(AVBufferRef **derived_frame_ctx,
diff --git a/libavutil/hwcontext.h b/libavutil/hwcontext.h
index 04d19d89c2..c18b7e1e8b 100644
--- a/libavutil/hwcontext.h
+++ b/libavutil/hwcontext.h
@@ -571,6 +571,10 @@ enum {
* possible with the given arguments and hwframe setup, while other return
* values indicate that it failed somehow.
*
+ * On failure, the destination frame will be left blank, except for the
+ * hw_frames_ctx/format fields thay may have been set by the caller - those
will
+ * be preserved as they were.
+ *
* @param dst Destination frame, to contain the mapping.
* @param src Source frame, to be mapped.
* @param flags Some combination of AV_HWFRAME_MAP_* flags.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 3/3] lavu/hwcontext: clarify behavior on av_hwframe_map() failure
Quoting James Almer (2022-01-19 14:45:12)
>
>
> On 1/19/2022 10:40 AM, Anton Khirnov wrote:
> > Clear anything that av_hwframe_map() might have done to the destination
> > frame, but leave caller-provided fields unchanged.
> > ---
> > libavutil/hwcontext.c | 23 +--
> > libavutil/hwcontext.h | 4
> > 2 files changed, 25 insertions(+), 2 deletions(-)
> >
> > diff --git a/libavutil/hwcontext.c b/libavutil/hwcontext.c
> > index 31c7840dba..ae33da1262 100644
> > --- a/libavutil/hwcontext.c
> > +++ b/libavutil/hwcontext.c
> > @@ -18,6 +18,7 @@
> >
> > #include "config.h"
> >
> > +#include "avassert.h"
> > #include "buffer.h"
> > #include "common.h"
> > #include "hwcontext.h"
> > @@ -788,6 +789,8 @@ fail:
> >
> > int av_hwframe_map(AVFrame *dst, const AVFrame *src, int flags)
> > {
> > +AVBufferRef*orig_dst_frames = dst->hw_frames_ctx;
> > +enum AVPixelFormat orig_dst_fmt = dst->format;
> > AVHWFramesContext *src_frames, *dst_frames;
> > HWMapDescriptor *hwmap;
> > int ret;
> > @@ -825,7 +828,7 @@ int av_hwframe_map(AVFrame *dst, const AVFrame *src,
> > int flags)
> > ret = src_frames->internal->hw_type->map_from(src_frames,
> > dst, src,
> > flags);
> > if (ret != AVERROR(ENOSYS))
> > -return ret;
> > +goto fail;
>
> What if ret is 0? Is the stuff in the fail label meant to run on success?
Critical brain failure. Will fix.
--
Anton Khirnov
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH] configure: link to libatomic when it's present
C11 atomics in some configurations (e.g. 64bit operations on ppc64 with GCC) require linking to libatomic. Fixes #9275 --- configure | 25 - 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/configure b/configure index 1413122d87..3059c154df 100755 --- a/configure +++ b/configure @@ -3794,20 +3794,20 @@ cws2fws_extralibs="zlib_extralibs" # libraries, in any order avcodec_deps="avutil" -avcodec_suggest="libm" +avcodec_suggest="libm stdatomic" avdevice_deps="avformat avcodec avutil" -avdevice_suggest="libm" +avdevice_suggest="libm stdatomic" avfilter_deps="avutil" -avfilter_suggest="libm" +avfilter_suggest="libm stdatomic" avformat_deps="avcodec avutil" -avformat_suggest="libm network zlib" -avutil_suggest="clock_gettime ffnvcodec libm libdrm libmfx opencl user32 vaapi vulkan videotoolbox corefoundation corevideo coremedia bcrypt" +avformat_suggest="libm network zlib stdatomic" +avutil_suggest="clock_gettime ffnvcodec libm libdrm libmfx opencl user32 vaapi vulkan videotoolbox corefoundation corevideo coremedia bcrypt stdatomic" postproc_deps="avutil gpl" -postproc_suggest="libm" +postproc_suggest="libm stdatomic" swresample_deps="avutil" -swresample_suggest="libm libsoxr" +swresample_suggest="libm libsoxr stdatomic" swscale_deps="avutil" -swscale_suggest="libm" +swscale_suggest="libm stdatomic" avcodec_extralibs="pthreads_extralibs iconv_extralibs dxva2_extralibs" avfilter_extralibs="pthreads_extralibs" @@ -6324,7 +6324,14 @@ check_headers asm/types.h # it seems there are versions of clang in some distros that try to use the # gcc headers, which explodes for stdatomic # so we also check that atomics actually work here -check_builtin stdatomic stdatomic.h "atomic_int foo, bar = ATOMIC_VAR_INIT(-1); atomic_store(&foo, 0); foo += bar" +# +# some configurations also require linking to libatomic, so try +# both with -latomic and without +for LATOMIC in "-latomic" ""; do +check_builtin stdatomic stdatomic.h \ +"atomic_int foo, bar = ATOMIC_VAR_INIT(-1); atomic_store(&foo, 0); foo += bar" \ +$LATOMIC && eval stdatomic_extralibs="\$LATOMIC" && break +done check_lib advapi32 "windows.h"RegCloseKey -ladvapi32 check_lib bcrypt "windows.h bcrypt.h" BCryptGenRandom -lbcrypt && -- 2.33.0 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH] lavu/hwcontext: clarify behavior on av_hwframe_map() failure
Clear anything that av_hwframe_map() might have done to the destination
frame, but leave caller-provided fields unchanged.
---
Interdiff:
diff --git a/libavutil/hwcontext.c b/libavutil/hwcontext.c
index ae33da1262..ab9ad3703e 100644
--- a/libavutil/hwcontext.c
+++ b/libavutil/hwcontext.c
@@ -827,7 +827,9 @@ int av_hwframe_map(AVFrame *dst, const AVFrame *src, int
flags)
src_frames->internal->hw_type->map_from) {
ret = src_frames->internal->hw_type->map_from(src_frames,
dst, src, flags);
-if (ret != AVERROR(ENOSYS))
+if (ret >= 0)
+return ret;
+else if (ret != AVERROR(ENOSYS))
goto fail;
}
}
@@ -839,7 +841,9 @@ int av_hwframe_map(AVFrame *dst, const AVFrame *src, int
flags)
dst_frames->internal->hw_type->map_to) {
ret = dst_frames->internal->hw_type->map_to(dst_frames,
dst, src, flags);
-if (ret != AVERROR(ENOSYS))
+if (ret >= 0)
+return ret;
+else if (ret != AVERROR(ENOSYS))
goto fail;
}
}
libavutil/hwcontext.c | 27 +--
libavutil/hwcontext.h | 4
2 files changed, 29 insertions(+), 2 deletions(-)
diff --git a/libavutil/hwcontext.c b/libavutil/hwcontext.c
index 31c7840dba..ab9ad3703e 100644
--- a/libavutil/hwcontext.c
+++ b/libavutil/hwcontext.c
@@ -18,6 +18,7 @@
#include "config.h"
+#include "avassert.h"
#include "buffer.h"
#include "common.h"
#include "hwcontext.h"
@@ -788,6 +789,8 @@ fail:
int av_hwframe_map(AVFrame *dst, const AVFrame *src, int flags)
{
+AVBufferRef*orig_dst_frames = dst->hw_frames_ctx;
+enum AVPixelFormat orig_dst_fmt = dst->format;
AVHWFramesContext *src_frames, *dst_frames;
HWMapDescriptor *hwmap;
int ret;
@@ -824,8 +827,10 @@ int av_hwframe_map(AVFrame *dst, const AVFrame *src, int
flags)
src_frames->internal->hw_type->map_from) {
ret = src_frames->internal->hw_type->map_from(src_frames,
dst, src, flags);
-if (ret != AVERROR(ENOSYS))
+if (ret >= 0)
return ret;
+else if (ret != AVERROR(ENOSYS))
+goto fail;
}
}
@@ -836,12 +841,30 @@ int av_hwframe_map(AVFrame *dst, const AVFrame *src, int
flags)
dst_frames->internal->hw_type->map_to) {
ret = dst_frames->internal->hw_type->map_to(dst_frames,
dst, src, flags);
-if (ret != AVERROR(ENOSYS))
+if (ret >= 0)
return ret;
+else if (ret != AVERROR(ENOSYS))
+goto fail;
}
}
return AVERROR(ENOSYS);
+
+fail:
+// if the caller provided dst frames context, it should be preserved
+// by this function
+av_assert0(orig_dst_frames == NULL ||
+ orig_dst_frames == dst->hw_frames_ctx);
+
+// preserve user-provided dst frame fields, but clean
+// anything we might have set
+dst->hw_frames_ctx = NULL;
+av_frame_unref(dst);
+
+dst->hw_frames_ctx = orig_dst_frames;
+dst->format= orig_dst_fmt;
+
+return ret;
}
int av_hwframe_ctx_create_derived(AVBufferRef **derived_frame_ctx,
diff --git a/libavutil/hwcontext.h b/libavutil/hwcontext.h
index 04d19d89c2..c18b7e1e8b 100644
--- a/libavutil/hwcontext.h
+++ b/libavutil/hwcontext.h
@@ -571,6 +571,10 @@ enum {
* possible with the given arguments and hwframe setup, while other return
* values indicate that it failed somehow.
*
+ * On failure, the destination frame will be left blank, except for the
+ * hw_frames_ctx/format fields thay may have been set by the caller - those
will
+ * be preserved as they were.
+ *
* @param dst Destination frame, to contain the mapping.
* @param src Source frame, to be mapped.
* @param flags Some combination of AV_HWFRAME_MAP_* flags.
--
2.33.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 0/5] macos avdevice fixes and improvements
This is a follow-up from a previous series of patches that fix, enhance and cleanup support for audio and video input on macos in libavdevice. Due to some important recent refactoring and addition, version is reset. Patches: libavdevice/avfoundation.m: use setAudioSettings, extend supported formats libavdevice/avfoundation.m: Replace mutex-based concurrency handling in avfoundation.m by a thread-safe fifo queue with maximum length libavdevice/avfoundation.m: Allow to select devices by unique ID Use appropriate method for device discovery. Add AudioToolbox audio input device. configure| 5 ++ doc/indevs.texi | 50 +++- libavdevice/Makefile | 1 + libavdevice/alldevices.c | 1 + libavdevice/audiotoolbox_dec.m (new) | 466 +++ libavdevice/avfoundation.m | 548 - 6 files changed, 781 insertions(+), 290 deletions(-) create mode 100644 libavdevice/audiotoolbox_dec.m ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 1/5] libavdevice/avfoundation.m: use setAudioSettings, extend supported formats
This patch switches the logic around audio settings to let the caller drive the
format.
After experimenting with the AudioConverter, we realized that, even when
adhering to a strict implementation of the documented API, we were still
getting errors during conversions. The input device would randomly change from
e.g. s32le to s24le between restarts and error out on conversion (using a
freshly initialized converter).
Using setAudioSettings allow the OS to drive audio conversion internally and
pick whatever appropriate settings for the audio device. This has been working
very well and is also the way AVFoundation audio input is setup in videolan.
--- Begin Message ---
From fd30f651bdaafe812b5cdc022ef3e4ebd74b6727 Mon Sep 17 00:00:00 2001
From: Romain Beauxis
Date: Mon, 29 Nov 2021 08:46:05 -0600
Subject: [PATCH] libavdevice/avfoundation.m: use setAudioSettings, extend
supported formats
X-Unsent: 1
To: ffmpeg-devel@ffmpeg.org
This fixes: https://trac.ffmpeg.org/ticket/9502
Signed-off-by: Romain Beauxis
---
libavdevice/avfoundation.m | 206 -
1 file changed, 63 insertions(+), 143 deletions(-)
diff --git a/libavdevice/avfoundation.m b/libavdevice/avfoundation.m
index 0cd6e646d5..77c6e68763 100644
--- a/libavdevice/avfoundation.m
+++ b/libavdevice/avfoundation.m
@@ -93,6 +93,11 @@
AVRational framerate;
int width, height;
+int channels;
+int big_endian;
+int sample_rate;
+enum AVSampleFormat sample_format;
+
int capture_cursor;
int capture_mouse_clicks;
int capture_raw_data;
@@ -111,17 +116,6 @@
int num_video_devices;
-int audio_channels;
-int audio_bits_per_sample;
-int audio_float;
-int audio_be;
-int audio_signed_integer;
-int audio_packed;
-int audio_non_interleaved;
-
-int32_t *audio_buffer;
-int audio_buffer_size;
-
enum AVPixelFormat pixel_format;
AVCaptureSession *capture_session;
@@ -298,14 +292,6 @@ static void destroy_context(AVFContext* ctx)
ctx->audio_output= NULL;
ctx->avf_delegate= NULL;
ctx->avf_audio_delegate = NULL;
-
-av_freep(&ctx->audio_buffer);
-
-pthread_mutex_destroy(&ctx->frame_lock);
-
-if (ctx->current_frame) {
-CFRelease(ctx->current_frame);
-}
}
static void parse_device_name(AVFormatContext *s)
@@ -671,88 +657,62 @@ static int get_video_config(AVFormatContext *s)
static int get_audio_config(AVFormatContext *s)
{
AVFContext *ctx = (AVFContext*)s->priv_data;
-CMFormatDescriptionRef format_desc;
-AVStream* stream = avformat_new_stream(s, NULL);
+AVStream* stream;
+int bits_per_sample, is_float;
-if (!stream) {
-return 1;
-}
+enum AVCodecID codec_id = av_get_pcm_codec(ctx->sample_format,
ctx->big_endian);
-// Take stream info from the first frame.
-while (ctx->audio_frames_captured < 1) {
-CFRunLoopRunInMode(kCFRunLoopDefaultMode, 0.1, YES);
+if (codec_id == AV_CODEC_ID_NONE) {
+ av_log(ctx, AV_LOG_ERROR, "Error: invalid sample format!\n");
+ return AVERROR(EINVAL);
}
-lock_frames(ctx);
-
-ctx->audio_stream_index = stream->index;
-
-avpriv_set_pts_info(stream, 64, 1, avf_time_base);
-
-format_desc = CMSampleBufferGetFormatDescription(ctx->current_audio_frame);
-const AudioStreamBasicDescription *basic_desc =
CMAudioFormatDescriptionGetStreamBasicDescription(format_desc);
+switch (ctx->sample_format) {
+case AV_SAMPLE_FMT_S16:
+bits_per_sample = 16;
+is_float = 0;
+break;
+case AV_SAMPLE_FMT_S32:
+bits_per_sample = 32;
+is_float = 0;
+break;
+case AV_SAMPLE_FMT_FLT:
+bits_per_sample = 32;
+is_float = 1;
+break;
+default:
+av_log(ctx, AV_LOG_ERROR, "Error: invalid sample format!\n");
+unlock_frames(ctx);
+return AVERROR(EINVAL);
+}
-if (!basic_desc) {
+[ctx->audio_output setAudioSettings:@{
+AVFormatIDKey: @(kAudioFormatLinearPCM),
+AVLinearPCMBitDepthKey: @(bits_per_sample),
+AVLinearPCMIsFloatKey: @(is_float),
+AVLinearPCMIsBigEndianKey: @(ctx->big_endian),
+AVNumberOfChannelsKey: @(ctx->channels),
+AVLinearPCMIsNonInterleaved: @NO,
+AVSampleRateKey: @(ctx->sample_rate)
+}];
+
+stream = avformat_new_stream(s, NULL);
+if (!stream) {
unlock_frames(ctx);
-av_log(s, AV_LOG_ERROR, "audio format not available\n");
-return 1;
+return -1;
}
+avpriv_set_pts_info(stream, 64, 1, avf_time_base);
+
stream->codecpar->codec_type = AVMED
Re: [FFmpeg-devel] [PATCH 1/5] libavdevice/avfoundation.m: use setAudioSettings, extend supported formats
On 19 Jan 2022, at 15:23, Romain Beauxis wrote: This patch switches the logic around audio settings to let the caller drive the format. Hi, just a quick technical feedback, with the patch sent as attachment it now finally applies fine for me, thanks! After experimenting with the AudioConverter, we realized that, even when adhering to a strict implementation of the documented API, we were still getting errors during conversions. The input device would randomly change from e.g. s32le to s24le between restarts and error out on conversion (using a freshly initialized converter). Using setAudioSettings allow the OS to drive audio conversion internally and pick whatever appropriate settings for the audio device. This has been working very well and is also the way AVFoundation audio input is setup in videolan. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/5] libavdevice/avfoundation.m: use setAudioSettings, extend supported formats
Romain Beauxis: > This patch switches the logic around audio settings to let the caller drive > the > format. > > After experimenting with the AudioConverter, we realized that, even when > adhering to a strict implementation of the documented API, we were still > getting > errors during conversions. The input device would randomly change from e.g. > s32le to s24le between restarts and error out on conversion (using a freshly > initialized converter). > > Using setAudioSettings allow the OS to drive audio conversion internally and > pick whatever appropriate settings for the audio device. This has been > working > very well and is also the way AVFoundation audio input is setup in videolan. > Your mail is broken: we don't want patches as eml; instead you should attach what git format-patch produces. Or you can use git send-email directly. - Andreas ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/5] libavdevice/avfoundation.m: use setAudioSettings, extend supported formats
On 2022-01-19 07:53 pm, Romain Beauxis wrote: This patch switches the logic around audio settings to let the caller drive the format. After experimenting with the AudioConverter, we realized that, even when adhering to a strict implementation of the documented API, we were still getting errors during conversions. The input device would randomly change from e.g. s32le to s24le between restarts and error out on conversion (using a freshly initialized converter). At present, the code uses the first frame to set attributes. If you wait for a few frames and then probe, the attributes are stable. Regards, Gyan ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 2/5] libavdevice/avfoundation.m: Replace mutex-based concurrency by a thread-safe fifo queue with maximum length
The existing implementation of avdevice input has issues in its concurrent
model as it only allows for one shared frame between writing and reading
threads. This means that, if reading thread gets late, frames get dropped,
resulting in corrupted input.
This patch changes the concurrency logic to use a single shared queue for both
video and audio frames. Previous version of the patch used separate queues for
audio and video but this could cause synchronization issues.
In order to avoid dropping initial audio frames, the video configuration logic
is also changed to assume height/width as configured when opening the input
device so as to not depend on the first video frame for it.
--- Begin Message ---
From d1a4c6e74ff589d9e59e1310a9afc9bc185382a1 Mon Sep 17 00:00:00 2001
From: Romain Beauxis
Date: Sun, 12 Dec 2021 17:29:27 -0600
Subject: [PATCH] libavdevice/avfoundation.m: Replace mutex-based concurrency
handling in avfoundation.m by a thread-safe fifo queue with maximum length
X-Unsent: 1
To: ffmpeg-devel@ffmpeg.org
* Use a shared CMSimpleQueueEnqueue with maximum length to queue and process
incoming audio and video frames.
* Simplify video configuration to avoid consuming first frame.
* Log avfoundation errors.
* Use AVERROR_EXTERNAL instead of AVERROR(EIO) in avfoundation errors.
Signed-off-by: Romain Beauxis
---
libavdevice/avfoundation.m | 227 +
1 file changed, 101 insertions(+), 126 deletions(-)
diff --git a/libavdevice/avfoundation.m b/libavdevice/avfoundation.m
index 77c6e68763..e6f64b35b8 100644
--- a/libavdevice/avfoundation.m
+++ b/libavdevice/avfoundation.m
@@ -26,7 +26,7 @@
*/
#import
-#include
+#import
#include "libavutil/channel_layout.h"
#include "libavutil/pixdesc.h"
@@ -39,6 +39,13 @@
#include "libavutil/imgutils.h"
#include "avdevice.h"
+static void av_log_avfoundation(void *s, int lvl, const char *str, OSStatus
err) {
+NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
+av_log(s, lvl, "AVFoundation: %s, %s\n", str,
+[[[NSError errorWithDomain:NSOSStatusErrorDomain code:err
userInfo:nil] localizedDescription] UTF8String]);
+[pool release];
+}
+
static const int avf_time_base = 100;
static const AVRational avf_time_base_q = {
@@ -84,9 +91,6 @@
{
AVClass*class;
-int frames_captured;
-int audio_frames_captured;
-pthread_mutex_t frame_lock;
id avf_delegate;
id avf_audio_delegate;
@@ -121,8 +125,9 @@
AVCaptureSession *capture_session;
AVCaptureVideoDataOutput *video_output;
AVCaptureAudioDataOutput *audio_output;
-CMSampleBufferRef current_frame;
-CMSampleBufferRef current_audio_frame;
+
+CMSimpleQueueRef frames_queue;
+int max_frames;
AVCaptureDevice *observed_device;
#if !TARGET_OS_IPHONE && __MAC_OS_X_VERSION_MIN_REQUIRED >= 1070
@@ -131,16 +136,6 @@
int observed_quit;
} AVFContext;
-static void lock_frames(AVFContext* ctx)
-{
-pthread_mutex_lock(&ctx->frame_lock);
-}
-
-static void unlock_frames(AVFContext* ctx)
-{
-pthread_mutex_unlock(&ctx->frame_lock);
-}
-
/** FrameReciever class - delegate for AVCaptureSession
*/
@interface AVFFrameReceiver : NSObject
@@ -218,17 +213,13 @@ - (void) captureOutput:(AVCaptureOutput *)captureOutput
didOutputSampleBuffer:(CMSampleBufferRef)videoFrame
fromConnection:(AVCaptureConnection *)connection
{
-lock_frames(_context);
+OSStatus ret = CMSimpleQueueEnqueue(_context->frames_queue, videoFrame);
-if (_context->current_frame != nil) {
-CFRelease(_context->current_frame);
+if (ret != noErr) {
+ av_log_avfoundation(_context, AV_LOG_DEBUG, "Error while queueing video
frame", ret);
}
-_context->current_frame = (CMSampleBufferRef)CFRetain(videoFrame);
-
-unlock_frames(_context);
-
-++_context->frames_captured;
+CFRetain(videoFrame);
}
@end
@@ -262,17 +253,13 @@ - (void) captureOutput:(AVCaptureOutput *)captureOutput
didOutputSampleBuffer:(CMSampleBufferRef)audioFrame
fromConnection:(AVCaptureConnection *)connection
{
-lock_frames(_context);
+OSStatus ret = CMSimpleQueueEnqueue(_context->frames_queue, audioFrame);
-if (_context->current_audio_frame != nil) {
-CFRelease(_context->current_audio_frame);
+if (ret != noErr) {
+ av_log_avfoundation(_context, AV_LOG_DEBUG, "Error while queueing audio
frame", ret);
}
-_context->current_audio_frame = (CMSampleBufferRef)CFRetain(audioFrame);
-
-unlock_frames(_context);
-
-++_context->audio_frames_captured;
+CFRetain(audioFrame);
}
@end
@@ -287,6 +274,19 @@ static void destroy_context(AVFContext* ctx)
[ctx->avf_delegaterelease];
[ctx->avf_audio_delegate release];
+CMSampleBufferRef frame;
+
+if (ctx->frames_queue) {
+
[FFmpeg-devel] [PATCH 3/5] libavdevice/avfoundation.m: Allow to select devices by unique ID
This patch changes the logic around device selection to allow to use a static,
machine-readable unique ID when selecting devices. Device names depends on
locale settings and device index can change when plugging/unplugging devices.
--- Begin Message ---
From bf72d48c846f5116866ec588fc0ee54a2c354e87 Mon Sep 17 00:00:00 2001
From: Romain Beauxis
Date: Mon, 13 Dec 2021 09:14:50 -0600
Subject: [PATCH] libavdevice/avfoundation.m: Allow to select devices by unique
ID
X-Unsent: 1
To: ffmpeg-devel@ffmpeg.org
Signed-off-by: Romain Beauxis
---
doc/indevs.texi| 6 ++--
libavdevice/avfoundation.m | 72 +-
2 files changed, 60 insertions(+), 18 deletions(-)
diff --git a/doc/indevs.texi b/doc/indevs.texi
index 9d8020311a..858c0fa4e4 100644
--- a/doc/indevs.texi
+++ b/doc/indevs.texi
@@ -114,7 +114,7 @@ The input filename has to be given in the following syntax:
-i "[[VIDEO]:[AUDIO]]"
@end example
The first entry selects the video input while the latter selects the audio
input.
-The stream has to be specified by the device name or the device index as shown
by the device list.
+The stream has to be specified by the device name, index or ID as shown by the
device list.
Alternatively, the video and/or audio input device can be chosen by index
using the
@option{
-video_device_index
@@ -127,7 +127,9 @@ and/or
device name or index given in the input filename.
All available devices can be enumerated by using @option{-list_devices true},
listing
-all device names and corresponding indices.
+all device names, corresponding indices and IDs, when available. Device name
can be
+tricky to use when localized and device index can change when devices are
plugged or unplugged. A device
+hash, when available, uniquely identifies a device and should not change over
time.
There are two device name aliases:
@table @code
diff --git a/libavdevice/avfoundation.m b/libavdevice/avfoundation.m
index e6f64b35b8..5d013cc0eb 100644
--- a/libavdevice/avfoundation.m
+++ b/libavdevice/avfoundation.m
@@ -39,6 +39,8 @@
#include "libavutil/imgutils.h"
#include "avdevice.h"
+#define CLEANUP_DEVICE_ID(s) [[s stringByReplacingOccurrencesOfString:@":"
withString:@"."] UTF8String]
+
static void av_log_avfoundation(void *s, int lvl, const char *str, OSStatus
err) {
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
av_log(s, lvl, "AVFoundation: %s, %s\n", str,
@@ -710,21 +712,23 @@ static int avf_read_header(AVFormatContext *s)
int index = 0;
av_log(ctx, AV_LOG_INFO, "AVFoundation video devices:\n");
for (AVCaptureDevice *device in devices) {
-const char *name = [[device localizedName] UTF8String];
-index= [devices indexOfObject:device];
-av_log(ctx, AV_LOG_INFO, "[%d] %s\n", index, name);
+const char *name = [[device localizedName] UTF8String];
+const char *uniqueId = CLEANUP_DEVICE_ID([device uniqueID]);
+index= [devices indexOfObject:device];
+av_log(ctx, AV_LOG_INFO, "[%d] %s (ID: %s)\n", index, name,
uniqueId);
}
for (AVCaptureDevice *device in devices_muxed) {
-const char *name = [[device localizedName] UTF8String];
-index= [devices count] + [devices_muxed
indexOfObject:device];
-av_log(ctx, AV_LOG_INFO, "[%d] %s\n", index, name);
+const char *name = [[device localizedName] UTF8String];
+const char *uniqueId = CLEANUP_DEVICE_ID([device uniqueID]);
+index= [devices count] + [devices_muxed
indexOfObject:device];
+av_log(ctx, AV_LOG_INFO, "[%d] %s (ID: %s)\n", index, name,
uniqueId);
}
#if !TARGET_OS_IPHONE && __MAC_OS_X_VERSION_MIN_REQUIRED >= 1070
if (num_screens > 0) {
CGDirectDisplayID screens[num_screens];
CGGetActiveDisplayList(num_screens, screens, &num_screens);
for (int i = 0; i < num_screens; i++) {
-av_log(ctx, AV_LOG_INFO, "[%d] Capture screen %d\n",
ctx->num_video_devices + i, i);
+av_log(ctx, AV_LOG_INFO, "[%d] Capture screen %d (ID:
AvfilterAvfoundationCaptureScreen%d)\n", ctx->num_video_devices + i, i,
screens[i]);
}
}
#endif
@@ -732,9 +736,10 @@ static int avf_read_header(AVFormatContext *s)
av_log(ctx, AV_LOG_INFO, "AVFoundation audio devices:\n");
devices = [AVCaptureDevice devicesWithMediaType:AVMediaTypeAudio];
for (AVCaptureDevice *device in devices) {
-const char *name = [[device localizedName] UTF8String];
-int index = [devices indexOfObject:device];
-av_log(ctx, AV_LOG_INFO, "[%d] %s\n", index, name);
+const char *name = [[device localizedName] UTF8String];
+const char *uniqueId = CLEANUP_DEVICE_ID([device uniqueID]);
+int index
[FFmpeg-devel] [PATCH 4/5] Use appropriate method for device discovery.
This adds support for the non-deprecated methods for selecting devices, when
available.
--- Begin Message ---
From c42612b455289622edb638436b1892e43279d8ac Mon Sep 17 00:00:00 2001
From: Romain Beauxis
Date: Fri, 14 Jan 2022 10:06:08 -0600
Subject: [PATCH] Use appropriate method for device discovery.
X-Unsent: 1
To: ffmpeg-devel@ffmpeg.org
Signed-off-by: Romain Beauxis
---
libavdevice/avfoundation.m | 77 --
1 file changed, 57 insertions(+), 20 deletions(-)
diff --git a/libavdevice/avfoundation.m b/libavdevice/avfoundation.m
index 5d013cc0eb..03ec49df86 100644
--- a/libavdevice/avfoundation.m
+++ b/libavdevice/avfoundation.m
@@ -27,6 +27,7 @@
#import
#import
+#include
#include "libavutil/channel_layout.h"
#include "libavutil/pixdesc.h"
@@ -698,8 +699,34 @@ static int avf_read_header(AVFormatContext *s)
AVCaptureDevice *video_device = nil;
AVCaptureDevice *audio_device = nil;
// Find capture device
-NSArray *devices = [AVCaptureDevice devicesWithMediaType:AVMediaTypeVideo];
-NSArray *devices_muxed = [AVCaptureDevice
devicesWithMediaType:AVMediaTypeMuxed];
+#if defined(__MAC_10_15) || (TARGET_OS_IPHONE && defined(__IPHONE_10_0))
+ AVCaptureDeviceDiscoverySession *discoverySession =
+[AVCaptureDeviceDiscoverySession discoverySessionWithDeviceTypes:@[
+#if TARGET_OS_IPHONE
+
AVCaptureDeviceTypeBuiltInDualCamera,
+
AVCaptureDeviceTypeBuiltInDualWideCamera,
+
AVCaptureDeviceTypeBuiltInUltraWideCamera,
+
AVCaptureDeviceTypeBuiltInTrueDepthCamera,
+
AVCaptureDeviceTypeBuiltInTelephotoCamera,
+#endif
+
AVCaptureDeviceTypeBuiltInWideAngleCamera,
+
AVCaptureDeviceTypeExternalUnknown
+ ]
+ mediaType:NULL
+
position:AVCaptureDevicePositionUnspecified];
+
+ NSMutableArray *devices = [NSMutableArray array];
+ NSMutableArray *devices_muxed = [NSMutableArray array];
+ for (AVCaptureDevice *device in [discoverySession devices]) {
+ if ([device hasMediaType:AVMediaTypeVideo])
+ [devices addObject:device];
+ else if ([device hasMediaType:AVMediaTypeMuxed])
+ [devices_muxed addObject:device];
+ }
+#else
+ NSArray *devices = [AVCaptureDevice
devicesWithMediaType:AVMediaTypeVideo];
+ NSArray *devices_muxed = [AVCaptureDevice
devicesWithMediaType:AVMediaTypeMuxed];
+#endif
ctx->num_video_devices = [devices count] + [devices_muxed count];
@@ -707,6 +734,21 @@ static int avf_read_header(AVFormatContext *s)
CGGetActiveDisplayList(0, NULL, &num_screens);
#endif
+NSArray *audio_devices;
+#if defined(__MAC_10_15) || (TARGET_OS_IPHONE && defined(__IPHONE_10_0))
+discoverySession =
+[AVCaptureDeviceDiscoverySession discoverySessionWithDeviceTypes:@[
+
AVCaptureDeviceTypeBuiltInMicrophone,
+ AVCaptureDeviceTypeExternalUnknown
+ ]
+ mediaType:AVMediaTypeAudio
+
position:AVCaptureDevicePositionUnspecified];
+
+audio_devices = [discoverySession devices];
+#else
+audio_devices = [AVCaptureDevice devicesWithMediaType:AVMediaTypeAudio];
+#endif
+
// List devices if requested
if (ctx->list_devices) {
int index = 0;
@@ -734,8 +776,7 @@ static int avf_read_header(AVFormatContext *s)
#endif
av_log(ctx, AV_LOG_INFO, "AVFoundation audio devices:\n");
-devices = [AVCaptureDevice devicesWithMediaType:AVMediaTypeAudio];
-for (AVCaptureDevice *device in devices) {
+for (AVCaptureDevice *device in audio_devices) {
const char *name = [[device localizedName] UTF8String];
const char *uniqueId = CLEANUP_DEVICE_ID([device uniqueID]);
int index= [devices indexOfObject:device];
@@ -885,9 +926,7 @@ static int avf_read_header(AVFormatContext *s)
// get audio device
if (ctx->audio_device_index >= 0) {
-NSArray *devices = [AVCaptureDevice
devicesWithMediaType:AVMediaTypeAudio];
-
-if (ctx->audio_device_index >= [devices count]) {
+if (ctx->audio_device_index >= [audio_devices count]) {
av_log(ctx, AV_LOG_ERROR, "Invalid audio device index\n");
goto fail;
}
@@ -898,22 +937,20 @@ static int avf_read_header(AVFormatContext *s)
if (!strncmp(ctx->audio_filename, "default", 7)) {
[FFmpeg-devel] [PATCH 5/5] Add AudioToolbox audio input device.
This patch adds support for a new, audio-specific input device using the
documented and battle-tested AUHAL input. This provides a pendant to the
AudioToolbox audio-only output.
A couple of advantages for this:
* It avoids a lot of the complexity of supporting audio and video in a single
input
* The AUHAL API seems tested, documented and robust
* This implementation hopefully gives good control over audio latency and also
minimizes data copy
--- Begin Message ---
From 803e1aa52018bdac0a448be255f645ca447273e9 Mon Sep 17 00:00:00 2001
From: Romain Beauxis
Date: Tue, 18 Jan 2022 16:29:59 -0600
Subject: [PATCH] Add AudioToolbox audio input device.
X-Unsent: 1
To: ffmpeg-devel@ffmpeg.org
Signed-off-by: Romain Beauxis
---
configure | 5 +
doc/indevs.texi| 44
libavdevice/Makefile | 1 +
libavdevice/alldevices.c | 1 +
libavdevice/audiotoolbox_dec.m | 466 +
5 files changed, 517 insertions(+)
create mode 100644 libavdevice/audiotoolbox_dec.m
diff --git a/configure b/configure
index 1413122d87..80e39aae44 100755
--- a/configure
+++ b/configure
@@ -204,6 +204,7 @@ External library support:
--disable-avfoundation disable Apple AVFoundation framework [autodetect]
--enable-avisynthenable reading of AviSynth script files [no]
--disable-bzlib disable bzlib [autodetect]
+ --disable-coremedia disable Apple CoreMedia framework [autodetect]
--disable-coreimage disable Apple CoreImage framework [autodetect]
--enable-chromaprint enable audio fingerprinting with chromaprint [no]
--enable-frei0r enable frei0r video filtering [no]
@@ -1750,6 +1751,7 @@ EXTERNAL_AUTODETECT_LIBRARY_LIST="
appkit
avfoundation
bzlib
+coremedia
coreimage
iconv
libxcb
@@ -3493,6 +3495,8 @@ alsa_outdev_deps="alsa"
avfoundation_indev_deps="avfoundation corevideo coremedia pthreads"
avfoundation_indev_suggest="coregraphics applicationservices"
avfoundation_indev_extralibs="-framework Foundation"
+audiotoolbox_indev_deps="coremedia audiotoolbox"
+audiotoolbox_indev_extralibs="-framework CoreMedia -framework AudioToolbox"
audiotoolbox_outdev_deps="audiotoolbox pthreads"
audiotoolbox_outdev_extralibs="-framework AudioToolbox -framework CoreAudio"
bktr_indev_deps_any="dev_bktr_ioctl_bt848_h machine_ioctl_bt848_h
dev_video_bktr_ioctl_bt848_h dev_ic_bt8xx_h"
@@ -6340,6 +6344,7 @@ check_lib camera2ndk "stdbool.h stdint.h
camera/NdkCameraManager.h" ACameraManag
enabled appkit && check_apple_framework AppKit
enabled audiotoolbox && check_apple_framework AudioToolbox
enabled avfoundation && check_apple_framework AVFoundation
+enabled coremedia&& check_apple_framework CoreMedia
enabled coreimage&& check_apple_framework CoreImage
enabled metal&& check_apple_framework Metal
enabled videotoolbox && check_apple_framework VideoToolbox
diff --git a/doc/indevs.texi b/doc/indevs.texi
index 858c0fa4e4..30a91d304f 100644
--- a/doc/indevs.texi
+++ b/doc/indevs.texi
@@ -103,6 +103,50 @@ Set the maximum number of frames to buffer. Default is 5.
@end table
+@section AudioToolbox
+
+AudioToolbox input device.
+
+Allows native input from CoreAudio devices on OSX.
+
+All available devices can be enumerated by using @option{-list_devices true},
listing
+all device names, and corresponding unique ID.
+
+@subsection Options
+
+AudioToolbox supports the following options:
+
+@table @option
+
+@item channels
+Set the number of channels. Default is device's default.
+
+@item frames_queue_length
+Maximum of buffers in the input queue
+
+@item buffer_frame_size
+Buffer frame size, gouverning internal latency
+
+@item big_endian
+Return big endian samples
+
+@item sample_format
+Sample format
+
+@end table
+
+@subsection Examples
+
+@itemize
+
+@item
+Print the list of supported devices
+@example
+$ ffmpeg -f audiotoolbox -list_devices true -i ""
+@end example
+
+@end itemize
+
@section avfoundation
AVFoundation input device.
diff --git a/libavdevice/Makefile b/libavdevice/Makefile
index 53efda0514..0c73255a21 100644
--- a/libavdevice/Makefile
+++ b/libavdevice/Makefile
@@ -14,6 +14,7 @@ OBJS-$(HAVE_LIBC_MSVCRT) += file_open.o
OBJS-$(CONFIG_ALSA_INDEV)+= alsa_dec.o alsa.o timefilter.o
OBJS-$(CONFIG_ALSA_OUTDEV) += alsa_enc.o alsa.o
OBJS-$(CONFIG_ANDROID_CAMERA_INDEV) += android_camera.o
+OBJS-$(CONFIG_AUDIOTOOLBOX_INDEV)+= audiotoolbox_dec.o
OBJS-$(CONFIG_AUDIOTOOLBOX_OUTDEV) += audiotoolbox.o
OBJS-$(CONFIG_AVFOUNDATION_INDEV)+= avfoundation.o
OBJS-$(CONFIG_BKTR_INDEV)+= bktr.o
diff --git a/libavdevice/alldevices.c b/libavdevice/alldevices.c
index 22323a0a44..fbecdbb0b2 100644
--- a/libavdevice/alldevices.c
+++ b/libavdevice/alldevices.c
@@ -26,6 +26,7 @@ extern const AVInputFormat ff_alsa_demuxer;
extern const AVOutputFormat ff_alsa_muxer;
extern c
Re: [FFmpeg-devel] [PATCH 1/5] libavdevice/avfoundation.m: use setAudioSettings, extend supported formats
Le mer. 19 janv. 2022 à 08:31, Gyan Doshi a écrit : > On 2022-01-19 07:53 pm, Romain Beauxis wrote: > > This patch switches the logic around audio settings to let the caller drive > > the format. > > > > After experimenting with the AudioConverter, we realized that, even when > > adhering to a strict implementation of the documented API, we were still > > getting errors during conversions. The input device would randomly change > > from e.g. s32le to s24le between restarts and error out on conversion > > (using a freshly initialized converter). > At present, the code uses the first frame to set attributes. If you > wait for a few frames and then probe, the attributes are stable. How is that supposed to work to get a full A/V stream? Discarding initial audio frames results in data loss in audio-only input and corrupted initial audio in A/V inputs. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/5] libavdevice/avfoundation.m: use setAudioSettings, extend supported formats
On 2022-01-19 08:44 pm, Romain Beauxis wrote: Le mer. 19 janv. 2022 à 08:31, Gyan Doshi a écrit : On 2022-01-19 07:53 pm, Romain Beauxis wrote: This patch switches the logic around audio settings to let the caller drive the format. After experimenting with the AudioConverter, we realized that, even when adhering to a strict implementation of the documented API, we were still getting errors during conversions. The input device would randomly change from e.g. s32le to s24le between restarts and error out on conversion (using a freshly initialized converter). At present, the code uses the first frame to set attributes. If you wait for a few frames and then probe, the attributes are stable. How is that supposed to work to get a full A/V stream? Discarding initial audio frames results in data loss in audio-only input and corrupted initial audio in A/V inputs. We're talking about around 5-6 packets, so ~100 ms. The streaming scenarios I worked on weren't sensitive to that amount of initial loss. YMMV. Regards Gyan ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/5] libavdevice/avfoundation.m: use setAudioSettings, extend supported formats
Le mer. 19 janv. 2022 à 09:19, Gyan Doshi a écrit : > On 2022-01-19 08:44 pm, Romain Beauxis wrote: > > Le mer. 19 janv. 2022 à 08:31, Gyan Doshi a écrit : > >> On 2022-01-19 07:53 pm, Romain Beauxis wrote: > >>> This patch switches the logic around audio settings to let the caller > >>> drive the format. > >>> > >>> After experimenting with the AudioConverter, we realized that, even when > >>> adhering to a strict implementation of the documented API, we were still > >>> getting errors during conversions. The input device would randomly change > >>> from e.g. s32le to s24le between restarts and error out on conversion > >>> (using a freshly initialized converter). > >>At present, the code uses the first frame to set attributes. If you > >> wait for a few frames and then probe, the attributes are stable. > > How is that supposed to work to get a full A/V stream? Discarding > > initial audio frames results in data loss in audio-only input and > > corrupted initial audio in A/V inputs. > > We're talking about around 5-6 packets, so ~100 ms. The streaming > scenarios I worked on weren't sensitive to that amount of initial loss. > YMMV. I see thanks. And what advantages does this method provide aside from supporting 24 bit sample formats which are currently excluded from these changes? ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] configure: link to libatomic when it's present
On 1/19/2022 10:48 AM, Anton Khirnov wrote: C11 atomics in some configurations (e.g. 64bit operations on ppc64 with GCC) require linking to libatomic. Fixes #9275 --- configure | 25 - 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/configure b/configure index 1413122d87..3059c154df 100755 --- a/configure +++ b/configure @@ -3794,20 +3794,20 @@ cws2fws_extralibs="zlib_extralibs" # libraries, in any order avcodec_deps="avutil" -avcodec_suggest="libm" +avcodec_suggest="libm stdatomic" avdevice_deps="avformat avcodec avutil" -avdevice_suggest="libm" +avdevice_suggest="libm stdatomic" avfilter_deps="avutil" -avfilter_suggest="libm" +avfilter_suggest="libm stdatomic" avformat_deps="avcodec avutil" -avformat_suggest="libm network zlib" -avutil_suggest="clock_gettime ffnvcodec libm libdrm libmfx opencl user32 vaapi vulkan videotoolbox corefoundation corevideo coremedia bcrypt" +avformat_suggest="libm network zlib stdatomic" +avutil_suggest="clock_gettime ffnvcodec libm libdrm libmfx opencl user32 vaapi vulkan videotoolbox corefoundation corevideo coremedia bcrypt stdatomic" postproc_deps="avutil gpl" -postproc_suggest="libm" +postproc_suggest="libm stdatomic" swresample_deps="avutil" -swresample_suggest="libm libsoxr" +swresample_suggest="libm libsoxr stdatomic" swscale_deps="avutil" -swscale_suggest="libm" +swscale_suggest="libm stdatomic" avcodec_extralibs="pthreads_extralibs iconv_extralibs dxva2_extralibs" avfilter_extralibs="pthreads_extralibs" @@ -6324,7 +6324,14 @@ check_headers asm/types.h # it seems there are versions of clang in some distros that try to use the # gcc headers, which explodes for stdatomic # so we also check that atomics actually work here -check_builtin stdatomic stdatomic.h "atomic_int foo, bar = ATOMIC_VAR_INIT(-1); atomic_store(&foo, 0); foo += bar" +# +# some configurations also require linking to libatomic, so try +# both with -latomic and without +for LATOMIC in "-latomic" ""; do +check_builtin stdatomic stdatomic.h \ +"atomic_int foo, bar = ATOMIC_VAR_INIT(-1); atomic_store(&foo, 0); foo += bar" \ +$LATOMIC && eval stdatomic_extralibs="\$LATOMIC" && break +done LGTM now, thanks. check_lib advapi32 "windows.h"RegCloseKey -ladvapi32 check_lib bcrypt "windows.h bcrypt.h" BCryptGenRandom -lbcrypt && ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/5] libavdevice/avfoundation.m: use setAudioSettings, extend supported formats
On 2022-01-19 08:51 pm, Romain Beauxis wrote: Le mer. 19 janv. 2022 à 09:19, Gyan Doshi a écrit : On 2022-01-19 08:44 pm, Romain Beauxis wrote: Le mer. 19 janv. 2022 à 08:31, Gyan Doshi a écrit : On 2022-01-19 07:53 pm, Romain Beauxis wrote: This patch switches the logic around audio settings to let the caller drive the format. After experimenting with the AudioConverter, we realized that, even when adhering to a strict implementation of the documented API, we were still getting errors during conversions. The input device would randomly change from e.g. s32le to s24le between restarts and error out on conversion (using a freshly initialized converter). At present, the code uses the first frame to set attributes. If you wait for a few frames and then probe, the attributes are stable. How is that supposed to work to get a full A/V stream? Discarding initial audio frames results in data loss in audio-only input and corrupted initial audio in A/V inputs. We're talking about around 5-6 packets, so ~100 ms. The streaming scenarios I worked on weren't sensitive to that amount of initial loss. YMMV. I see thanks. And what advantages does this method provide aside from supporting 24 bit sample formats which are currently excluded from these changes? I was remarking on a way to avoid format changes post-initialization. Not a comment on your patches. Gyan ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/5] libavdevice/avfoundation.m: use setAudioSettings, extend supported formats
Le mer. 19 janv. 2022 à 09:45, Gyan Doshi a écrit : > On 2022-01-19 08:51 pm, Romain Beauxis wrote: > > Le mer. 19 janv. 2022 à 09:19, Gyan Doshi a écrit : > >> On 2022-01-19 08:44 pm, Romain Beauxis wrote: > >>> Le mer. 19 janv. 2022 à 08:31, Gyan Doshi a écrit : > On 2022-01-19 07:53 pm, Romain Beauxis wrote: > > This patch switches the logic around audio settings to let the caller > > drive the format. > > > > After experimenting with the AudioConverter, we realized that, even > > when adhering to a strict implementation of the documented API, we were > > still getting errors during conversions. The input device would > > randomly change from e.g. s32le to s24le between restarts and error out > > on conversion (using a freshly initialized converter). > At present, the code uses the first frame to set attributes. If you > wait for a few frames and then probe, the attributes are stable. > >>> How is that supposed to work to get a full A/V stream? Discarding > >>> initial audio frames results in data loss in audio-only input and > >>> corrupted initial audio in A/V inputs. > >> We're talking about around 5-6 packets, so ~100 ms. The streaming > >> scenarios I worked on weren't sensitive to that amount of initial loss. > >> YMMV. > > I see thanks. And what advantages does this method provide aside from > > supporting 24 bit sample formats which are currently excluded from > > these changes? > > I was remarking on a way to avoid format changes post-initialization. > Not a comment on your patches. For sure, and I appreciate the feedback. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/5] libavdevice/avfoundation.m: use setAudioSettings, extend supported formats
Am 19.01.22 um 15:28 schrieb Andreas Rheinhardt: Romain Beauxis: This patch switches the logic around audio settings to let the caller drive the format. After experimenting with the AudioConverter, we realized that, even when adhering to a strict implementation of the documented API, we were still getting errors during conversions. The input device would randomly change from e.g. s32le to s24le between restarts and error out on conversion (using a freshly initialized converter). Using setAudioSettings allow the OS to drive audio conversion internally and pick whatever appropriate settings for the audio device. This has been working very well and is also the way AVFoundation audio input is setup in videolan. Your mail is broken: we don't want patches as eml; instead you should attach what git format-patch produces. Or you can use git send-email directly. +1, please. -Thilo ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 5/5] Add AudioToolbox audio input device.
On 19 Jan 2022, at 15:42, Romain Beauxis wrote:
Hi, thanks for the patch. I've not done a full code review yet, just a
few
initial remarks below:
This patch adds support for a new, audio-specific input device using
the documented and battle-tested AUHAL input. This provides a pendant
to the AudioToolbox audio-only output.
A couple of advantages for this:
* It avoids a lot of the complexity of supporting audio and video in a
single input
* The AUHAL API seems tested, documented and robust
* This implementation hopefully gives good control over audio latency
and also minimizes data copy
From: Romain Beauxis
To: ffmpeg-devel@ffmpeg.org
Subject: [PATCH] Add AudioToolbox audio input device.
Date: 18. January 2022 at 23:29
Signed-off-by: Romain Beauxis
---
configure | 5 +
doc/indevs.texi| 44
libavdevice/Makefile | 1 +
libavdevice/alldevices.c | 1 +
libavdevice/audiotoolbox_dec.m | 466
+
5 files changed, 517 insertions(+)
create mode 100644 libavdevice/audiotoolbox_dec.m
diff --git a/configure b/configure
index 1413122d87..80e39aae44 100755
--- a/configure
+++ b/configure
@@ -204,6 +204,7 @@ External library support:
--disable-avfoundation disable Apple AVFoundation framework
[autodetect]
--enable-avisynthenable reading of AviSynth script files
[no]
--disable-bzlib disable bzlib [autodetect]
+ --disable-coremedia disable Apple CoreMedia framework
[autodetect]
--disable-coreimage disable Apple CoreImage framework
[autodetect]
--enable-chromaprint enable audio fingerprinting with
chromaprint [no]
--enable-frei0r enable frei0r video filtering [no]
@@ -1750,6 +1751,7 @@ EXTERNAL_AUTODETECT_LIBRARY_LIST="
appkit
avfoundation
bzlib
+coremedia
coreimage
iconv
libxcb
@@ -3493,6 +3495,8 @@ alsa_outdev_deps="alsa"
avfoundation_indev_deps="avfoundation corevideo coremedia pthreads"
avfoundation_indev_suggest="coregraphics applicationservices"
avfoundation_indev_extralibs="-framework Foundation"
+audiotoolbox_indev_deps="coremedia audiotoolbox"
+audiotoolbox_indev_extralibs="-framework CoreMedia -framework
AudioToolbox"
audiotoolbox_outdev_deps="audiotoolbox pthreads"
audiotoolbox_outdev_extralibs="-framework AudioToolbox -framework
CoreAudio"
bktr_indev_deps_any="dev_bktr_ioctl_bt848_h machine_ioctl_bt848_h
dev_video_bktr_ioctl_bt848_h dev_ic_bt8xx_h"
@@ -6340,6 +6344,7 @@ check_lib camera2ndk "stdbool.h stdint.h
camera/NdkCameraManager.h" ACameraManag
enabled appkit && check_apple_framework AppKit
enabled audiotoolbox && check_apple_framework AudioToolbox
enabled avfoundation && check_apple_framework AVFoundation
+enabled coremedia&& check_apple_framework CoreMedia
enabled coreimage&& check_apple_framework CoreImage
enabled metal&& check_apple_framework Metal
enabled videotoolbox && check_apple_framework VideoToolbox
diff --git a/doc/indevs.texi b/doc/indevs.texi
index 858c0fa4e4..30a91d304f 100644
--- a/doc/indevs.texi
+++ b/doc/indevs.texi
@@ -103,6 +103,50 @@ Set the maximum number of frames to buffer.
Default is 5.
@end table
+@section AudioToolbox
+
+AudioToolbox input device.
+
+Allows native input from CoreAudio devices on OSX.
Nit: Nowadays it's macOS instead of OSX
+
+All available devices can be enumerated by using
@option{-list_devices true}, listing
+all device names, and corresponding unique ID.
Instead of adding another device that uses a custom list-devices option,
could you
instead implement the .get_device_list callback? (See alsa or pulse
modules for an
example) Then listing would work properly with the `ffmpeg -sources`
command and
devices could be iterated over using the avdevice APIs as well.
(Ideally we would have that for AVFoundation too but its really hard now
to do that
in a backwards compatible manner)
+
+@subsection Options
+
+AudioToolbox supports the following options:
+
+@table @option
+
+@item channels
+Set the number of channels. Default is device's default.
+
+@item frames_queue_length
+Maximum of buffers in the input queue
+
+@item buffer_frame_size
+Buffer frame size, gouverning internal latency
+
+@item big_endian
+Return big endian samples
+
+@item sample_format
+Sample format
+
+@end table
+
+@subsection Examples
+
+@itemize
+
+@item
+Print the list of supported devices
+@example
+$ ffmpeg -f audiotoolbox -list_devices true -i ""
+@end example
+
+@end itemize
+
@section avfoundation
[…]
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 274/281] swresample: convert to new channel layout API
James Almer:
> Signed-off-by: James Almer
> ---
> libswresample/options.c | 33 +++-
> libswresample/rematrix.c| 237 ++--
> libswresample/rematrix_template.c | 7 +-
> libswresample/swresample.c | 152 +++---
> libswresample/swresample.h | 63
This header still contains many references to swr_alloc_set_opts().
> libswresample/swresample_frame.c| 65 +++-
> libswresample/swresample_internal.h | 10 +-
> 7 files changed, 445 insertions(+), 122 deletions(-)
>
> diff --git a/libswresample/options.c b/libswresample/options.c
> index 6911709157..ffa27c590d 100644
> --- a/libswresample/options.c
> +++ b/libswresample/options.c
> @@ -34,12 +34,19 @@
>
> #define OFFSET(x) offsetof(SwrContext,x)
> #define PARAM AV_OPT_FLAG_AUDIO_PARAM
> +#define DEPREC AV_OPT_FLAG_DEPRECATED
>
> static const AVOption options[]={
> -{"ich" , "set input channel count" ,
> OFFSET(user_in_ch_count ), AV_OPT_TYPE_INT, {.i64=0}, 0
> , SWR_CH_MAX, PARAM},
> -{"in_channel_count" , "set input channel count" ,
> OFFSET(user_in_ch_count ), AV_OPT_TYPE_INT, {.i64=0}, 0
> , SWR_CH_MAX, PARAM},
> -{"och" , "set output channel count",
> OFFSET(user_out_ch_count ), AV_OPT_TYPE_INT, {.i64=0}, 0
> , SWR_CH_MAX, PARAM},
> -{"out_channel_count", "set output channel count",
> OFFSET(user_out_ch_count ), AV_OPT_TYPE_INT, {.i64=0}, 0
> , SWR_CH_MAX, PARAM},
> +#if FF_API_OLD_CHANNEL_LAYOUT
> +{"ich" , "set input channel count (Deprecated, use ichl)",
> +
> OFFSET(user_in_ch_count ), AV_OPT_TYPE_INT, {.i64=0}, 0
> , SWR_CH_MAX, PARAM|DEPREC},
> +{"in_channel_count" , "set input channel count (Deprecated, use
> in_chlayout)",
> +
> OFFSET(user_in_ch_count ), AV_OPT_TYPE_INT, {.i64=0}, 0
> , SWR_CH_MAX, PARAM|DEPREC},
> +{"och" , "set output channel count (Deprecated, use ochl)",
> +
> OFFSET(user_out_ch_count ), AV_OPT_TYPE_INT, {.i64=0}, 0
> , SWR_CH_MAX, PARAM|DEPREC},
> +{"out_channel_count", "set output channel count (Deprecated, use
> out_chlayout)",
> +
> OFFSET(user_out_ch_count ), AV_OPT_TYPE_INT, {.i64=0}, 0
> , SWR_CH_MAX, PARAM|DEPREC},
> +#endif
> {"uch" , "set used channel count" ,
> OFFSET(user_used_ch_count), AV_OPT_TYPE_INT, {.i64=0}, 0
> , SWR_CH_MAX, PARAM},
> {"used_channel_count" , "set used channel count" ,
> OFFSET(user_used_ch_count), AV_OPT_TYPE_INT, {.i64=0}, 0
> , SWR_CH_MAX, PARAM},
> {"isr" , "set input sample rate" , OFFSET(
> in_sample_rate), AV_OPT_TYPE_INT , {.i64=0 }, 0 ,
> INT_MAX , PARAM},
> @@ -52,10 +59,20 @@ static const AVOption options[]={
> {"out_sample_fmt" , "set output sample format",
> OFFSET(out_sample_fmt ), AV_OPT_TYPE_SAMPLE_FMT , {.i64=AV_SAMPLE_FMT_NONE},
> -1 , INT_MAX, PARAM},
> {"tsf" , "set internal sample format" ,
> OFFSET(user_int_sample_fmt), AV_OPT_TYPE_SAMPLE_FMT ,
> {.i64=AV_SAMPLE_FMT_NONE}, -1 , INT_MAX, PARAM},
> {"internal_sample_fmt" , "set internal sample format" ,
> OFFSET(user_int_sample_fmt), AV_OPT_TYPE_SAMPLE_FMT ,
> {.i64=AV_SAMPLE_FMT_NONE}, -1 , INT_MAX, PARAM},
> -{"icl" , "set input channel layout",
> OFFSET(user_in_ch_layout ), AV_OPT_TYPE_CHANNEL_LAYOUT, {.i64=0 },
> INT64_MIN, INT64_MAX , PARAM, "channel_layout"},
> -{"in_channel_layout", "set input channel layout",
> OFFSET(user_in_ch_layout ), AV_OPT_TYPE_CHANNEL_LAYOUT, {.i64=0 },
> INT64_MIN, INT64_MAX , PARAM, "channel_layout"},
> -{"ocl" , "set output channel layout" ,
> OFFSET(user_out_ch_layout), AV_OPT_TYPE_CHANNEL_LAYOUT, {.i64=0 },
> INT64_MIN, INT64_MAX , PARAM, "channel_layout"},
> -{"out_channel_layout" , "set output channel layout" ,
> OFFSET(user_out_ch_layout), AV_OPT_TYPE_CHANNEL_LAYOUT, {.i64=0 },
> INT64_MIN, INT64_MAX , PARAM, "channel_layout"},
> +#if FF_API_OLD_CHANNEL_LAYOUT
> +{"icl" , "set input channel layout (Deprecated, use ichl)",
> +
> OFFSET(user_in_ch_layout), AV_OPT_TYPE_CHANNEL_LAYOUT, {.i64=0 },
> INT64_MIN, INT64_MAX, PARAM|DEPREC, "channel_layout"},
> +{"in_channel_layout", "set input channel layout (Deprecated, use
> in_chlayout)",
> +
Re: [FFmpeg-devel] [PATCH 274/281] swresample: convert to new channel layout API
On 1/19/2022 2:20 PM, Andreas Rheinhardt wrote:
James Almer:
Signed-off-by: James Almer
---
libswresample/options.c | 33 +++-
libswresample/rematrix.c| 237 ++--
libswresample/rematrix_template.c | 7 +-
libswresample/swresample.c | 152 +++---
libswresample/swresample.h | 63
This header still contains many references to swr_alloc_set_opts().
Will change.
libswresample/swresample_frame.c| 65 +++-
libswresample/swresample_internal.h | 10 +-
7 files changed, 445 insertions(+), 122 deletions(-)
diff --git a/libswresample/options.c b/libswresample/options.c
index 6911709157..ffa27c590d 100644
--- a/libswresample/options.c
+++ b/libswresample/options.c
@@ -34,12 +34,19 @@
#define OFFSET(x) offsetof(SwrContext,x)
#define PARAM AV_OPT_FLAG_AUDIO_PARAM
+#define DEPREC AV_OPT_FLAG_DEPRECATED
static const AVOption options[]={
-{"ich" , "set input channel count" ,
OFFSET(user_in_ch_count ), AV_OPT_TYPE_INT, {.i64=0}, 0 , SWR_CH_MAX,
PARAM},
-{"in_channel_count" , "set input channel count" ,
OFFSET(user_in_ch_count ), AV_OPT_TYPE_INT, {.i64=0}, 0 , SWR_CH_MAX,
PARAM},
-{"och" , "set output channel count",
OFFSET(user_out_ch_count ), AV_OPT_TYPE_INT, {.i64=0}, 0 , SWR_CH_MAX,
PARAM},
-{"out_channel_count", "set output channel count",
OFFSET(user_out_ch_count ), AV_OPT_TYPE_INT, {.i64=0}, 0 , SWR_CH_MAX,
PARAM},
+#if FF_API_OLD_CHANNEL_LAYOUT
+{"ich" , "set input channel count (Deprecated, use ichl)",
+
OFFSET(user_in_ch_count ), AV_OPT_TYPE_INT, {.i64=0}, 0
, SWR_CH_MAX, PARAM|DEPREC},
+{"in_channel_count" , "set input channel count (Deprecated, use
in_chlayout)",
+
OFFSET(user_in_ch_count ), AV_OPT_TYPE_INT, {.i64=0}, 0
, SWR_CH_MAX, PARAM|DEPREC},
+{"och" , "set output channel count (Deprecated, use ochl)",
+
OFFSET(user_out_ch_count ), AV_OPT_TYPE_INT, {.i64=0}, 0
, SWR_CH_MAX, PARAM|DEPREC},
+{"out_channel_count", "set output channel count (Deprecated, use
out_chlayout)",
+
OFFSET(user_out_ch_count ), AV_OPT_TYPE_INT, {.i64=0}, 0
, SWR_CH_MAX, PARAM|DEPREC},
+#endif
{"uch" , "set used channel count" ,
OFFSET(user_used_ch_count), AV_OPT_TYPE_INT, {.i64=0}, 0 , SWR_CH_MAX,
PARAM},
{"used_channel_count" , "set used channel count" ,
OFFSET(user_used_ch_count), AV_OPT_TYPE_INT, {.i64=0}, 0 , SWR_CH_MAX,
PARAM},
{"isr" , "set input sample rate" , OFFSET(
in_sample_rate), AV_OPT_TYPE_INT , {.i64=0 }, 0 , INT_MAX , PARAM},
@@ -52,10 +59,20 @@ static const AVOption options[]={
{"out_sample_fmt" , "set output sample format",
OFFSET(out_sample_fmt ), AV_OPT_TYPE_SAMPLE_FMT , {.i64=AV_SAMPLE_FMT_NONE}, -1 , INT_MAX, PARAM},
{"tsf" , "set internal sample format" ,
OFFSET(user_int_sample_fmt), AV_OPT_TYPE_SAMPLE_FMT , {.i64=AV_SAMPLE_FMT_NONE}, -1 , INT_MAX,
PARAM},
{"internal_sample_fmt" , "set internal sample format" ,
OFFSET(user_int_sample_fmt), AV_OPT_TYPE_SAMPLE_FMT , {.i64=AV_SAMPLE_FMT_NONE}, -1 , INT_MAX,
PARAM},
-{"icl" , "set input channel layout", OFFSET(user_in_ch_layout ),
AV_OPT_TYPE_CHANNEL_LAYOUT, {.i64=0 }, INT64_MIN, INT64_MAX , PARAM, "channel_layout"},
-{"in_channel_layout", "set input channel layout", OFFSET(user_in_ch_layout ),
AV_OPT_TYPE_CHANNEL_LAYOUT, {.i64=0 }, INT64_MIN, INT64_MAX , PARAM, "channel_layout"},
-{"ocl" , "set output channel layout" , OFFSET(user_out_ch_layout),
AV_OPT_TYPE_CHANNEL_LAYOUT, {.i64=0 }, INT64_MIN, INT64_MAX , PARAM, "channel_layout"},
-{"out_channel_layout" , "set output channel layout" , OFFSET(user_out_ch_layout),
AV_OPT_TYPE_CHANNEL_LAYOUT, {.i64=0 }, INT64_MIN, INT64_MAX , PARAM, "channel_layout"},
+#if FF_API_OLD_CHANNEL_LAYOUT
+{"icl" , "set input channel layout (Deprecated, use ichl)",
+ OFFSET(user_in_ch_layout),
AV_OPT_TYPE_CHANNEL_LAYOUT, {.i64=0 }, INT64_MIN, INT64_MAX, PARAM|DEPREC,
"channel_layout"},
+{"in_channel_layout", "set input channel layout (Deprecated, use
in_chlayout)",
+ OFFSET(user_in_ch_layout),
AV_OPT_TYPE_CHANNEL_LAYOUT, {.i64=0 }, IN
Re: [FFmpeg-devel] [PATCH 001/289 v6] Add a new channel layout API
James Almer: > From: Anton Khirnov > > The new API is more extensible and allows for custom layouts. > More accurate information is exported, eg for decoders that do not > set a channel layout, lavc will not make one up for them. > > Deprecate the old API working with just uint64_t bitmasks. > > Expanded and completed by Vittorio Giovara > and James Almer . > Signed-off-by: Vittorio Giovara > Signed-off-by: James Almer > --- > Changes since last version: > > *av_channel_layout_from_string() and av_channel_layout_describe() now support > a "designation@name" syntax, effectively making both functions reciprocal > when there are custom names in some or all channels. > It's the syntax suggested by Marton and is both extensible if required and > not too ulgy in human readable output if the string is printed verbatim. > > *av_channel_layout_index_from_string() and > av_channel_layout_channel_from_string() also support this syntax now. > > I plan to push this version soon. Any extension to the syntax supported by > these > helpers can be written by whoever needs it. You forgot the examples and the tools; and ffplay. And do you really intend for the deprecation warnings in ffmpeg_opt.c to stay? - Andreas ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] avfilter/vf_libvmaf: update filter for libvmaf v2.0.0
Hi, On Thu, Jan 13, 2022 at 6:38 PM Kyle Swanson wrote: > > Hi, > > On Mon, Jan 10, 2022 at 10:22 AM Andreas Rheinhardt > wrote: > > 1. FFmpeg uses the ancient C90 rule that only allows variable > > declarations at the beginning of a block (before all statements). You > > should actually have received a compiler warning because of the above > > code (unless you use Clang, which claims to support this type of > > warning, yet doesn't). > > 2. You jump over the initializion of str_copy if the dict allocation > > fails and consequently you free an uninitialized string. > > New patch attached. Fixes all of the GCC -Wdeclaration-after-statement > warnings. You're right, I didn't notice because I was building with > Clang. Andreas, I will just wait for your LGTM before merging. > > Thanks, > Kyle Updated patch attached. Thank you to mkver for the mini review on IRC. The `delimited_dict_parse` multi-delimiter corner case should be sorted now. Thanks, Kyle 0001-avfilter-vf_libvmaf-update-filter-for-libvmaf-v2.0.0.patch Description: Binary data ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH] avfilter/adelay: Add command support
Adds command 'delays' to the adelay filter.
This command accepts same values as option with one difference, to apply
delay to all channels prefix 'all:' to the arguments is accepted.
Signed-off-by: David Lacko
---
libavfilter/af_adelay.c | 183 ++--
1 file changed, 157 insertions(+), 26 deletions(-)
diff --git a/libavfilter/af_adelay.c b/libavfilter/af_adelay.c
index ed8a8ae739..1e13cf7fb0 100644
--- a/libavfilter/af_adelay.c
+++ b/libavfilter/af_adelay.c
@@ -31,6 +31,7 @@ typedef struct ChanDelay {
int64_t delay;
size_t delay_index;
size_t index;
+unsigned int samples_size;
uint8_t *samples;
} ChanDelay;
@@ -48,13 +49,14 @@ typedef struct AudioDelayContext {
void (*delay_channel)(ChanDelay *d, int nb_samples,
const uint8_t *src, uint8_t *dst);
+int (*resize_channel_samples)(ChanDelay *d, int64_t new_delay);
} AudioDelayContext;
#define OFFSET(x) offsetof(AudioDelayContext, x)
#define A AV_OPT_FLAG_AUDIO_PARAM|AV_OPT_FLAG_FILTERING_PARAM
static const AVOption adelay_options[] = {
-{ "delays", "set list of delays for each channel", OFFSET(delays),
AV_OPT_TYPE_STRING, {.str=NULL}, 0, 0, A },
+{ "delays", "set list of delays for each channel", OFFSET(delays),
AV_OPT_TYPE_STRING, {.str=NULL}, 0, 0, A | AV_OPT_FLAG_RUNTIME_PARAM },
{ "all","use last available delay for remained channels", OFFSET(all),
AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, A },
{ NULL }
};
@@ -96,11 +98,92 @@ DELAY(s32, int32_t, 0)
DELAY(flt, float, 0)
DELAY(dbl, double, 0)
+#define CHANGE_DELAY(name, type, fill)
\
+static int resize_samples_## name ##p(ChanDelay *d, int64_t new_delay)
\
+{
\
+type *samples = (type *)d->samples;
\
+
\
+if (new_delay == d->delay) {
\
+return 0;
\
+}
\
+
\
+if (new_delay == 0) {
\
+av_freep(&d->samples);
\
+d->samples_size = 0;
\
+d->delay = 0;
\
+d->index = 0;
\
+return 0;
\
+}
\
+
\
+d->samples = av_fast_realloc(d->samples, &d->samples_size, new_delay *
sizeof(type)); \
+if (!d->samples) {
\
+av_freep(samples);
\
+return AVERROR(ENOMEM);
\
+}
\
+samples = (type *)d->samples;
\
+if (new_delay < d->delay) {
\
+if (d->index > new_delay) {
\
+d->index -= new_delay;
\
+memmove(samples, &samples[new_delay], d->index * sizeof(type));
\
+} else if (d->delay_index > d->index) {
\
+memmove(&samples[d->index],
&samples[d->index+(d->delay-new_delay)],\
+(new_delay - d->index) * sizeof(type));
\
+}
\
+d->delay_index = new_delay;
Re: [FFmpeg-devel] [PATCH 109/281] mxf: convert to new channel layout API
ons 2022-01-12 klockan 22:56 -0300 skrev James Almer: > From: Vittorio Giovara > > Signed-off-by: Vittorio Giovara > Signed-off-by: Anton Khirnov > Signed-off-by: James Almer > --- > libavformat/mxfdec.c | 34 +- > libavformat/mxfenc.c | 20 +++- > 2 files changed, 32 insertions(+), 22 deletions(-) Looks OK with the reservation that I haven't looked at the new API /Tomas ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] avfilter/adelay: Add command support
David Lacko:
> Adds command 'delays' to the adelay filter.
> This command accepts same values as option with one difference, to apply
> delay to all channels prefix 'all:' to the arguments is accepted.
>
> Signed-off-by: David Lacko
> ---
> libavfilter/af_adelay.c | 183 ++--
> 1 file changed, 157 insertions(+), 26 deletions(-)
>
> diff --git a/libavfilter/af_adelay.c b/libavfilter/af_adelay.c
> index ed8a8ae739..1e13cf7fb0 100644
> --- a/libavfilter/af_adelay.c
> +++ b/libavfilter/af_adelay.c
> @@ -31,6 +31,7 @@ typedef struct ChanDelay {
> int64_t delay;
> size_t delay_index;
> size_t index;
> +unsigned int samples_size;
> uint8_t *samples;
> } ChanDelay;
>
> @@ -48,13 +49,14 @@ typedef struct AudioDelayContext {
>
> void (*delay_channel)(ChanDelay *d, int nb_samples,
>const uint8_t *src, uint8_t *dst);
> +int (*resize_channel_samples)(ChanDelay *d, int64_t new_delay);
> } AudioDelayContext;
>
> #define OFFSET(x) offsetof(AudioDelayContext, x)
> #define A AV_OPT_FLAG_AUDIO_PARAM|AV_OPT_FLAG_FILTERING_PARAM
>
> static const AVOption adelay_options[] = {
> -{ "delays", "set list of delays for each channel", OFFSET(delays),
> AV_OPT_TYPE_STRING, {.str=NULL}, 0, 0, A },
> +{ "delays", "set list of delays for each channel", OFFSET(delays),
> AV_OPT_TYPE_STRING, {.str=NULL}, 0, 0, A | AV_OPT_FLAG_RUNTIME_PARAM },
> { "all","use last available delay for remained channels",
> OFFSET(all), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, A },
> { NULL }
> };
> @@ -96,11 +98,92 @@ DELAY(s32, int32_t, 0)
> DELAY(flt, float, 0)
> DELAY(dbl, double, 0)
>
> +#define CHANGE_DELAY(name, type, fill)
>\
> +static int resize_samples_## name ##p(ChanDelay *d, int64_t new_delay)
>\
> +{
>\
> +type *samples = (type *)d->samples;
>\
> +
>\
> +if (new_delay == d->delay) {
>\
> +return 0;
>\
> +}
>\
> +
>\
> +if (new_delay == 0) {
>\
> +av_freep(&d->samples);
>\
> +d->samples_size = 0;
>\
> +d->delay = 0;
>\
> +d->index = 0;
>\
> +return 0;
>\
> +}
>\
> +
>\
> +d->samples = av_fast_realloc(d->samples, &d->samples_size, new_delay *
> sizeof(type)); \
> +if (!d->samples) {
>\
> +av_freep(samples);
>\
av_free(samples) or av_freep(&samples), but not av_freep(samples).
The typical way to write this is btw tmp = av_fast_realloc(buf,...) (in
your case samples = av_fast_realloc(d->samples, ...) with
av_freep(&d->samples); in the error branch and d->samples = samples in
the non-error-case.
> +return AVERROR(ENOMEM);
>\
> +}
>\
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 1/2] libfdk-aacdec: Add an option for setting the decoder's DRC album mode
---
libavcodec/libfdk-aacdec.c | 14 ++
1 file changed, 14 insertions(+)
diff --git a/libavcodec/libfdk-aacdec.c b/libavcodec/libfdk-aacdec.c
index ffa1fdcce3..93b52023b0 100644
--- a/libavcodec/libfdk-aacdec.c
+++ b/libavcodec/libfdk-aacdec.c
@@ -56,6 +56,7 @@ typedef struct FDKAACDecContext {
int drc_heavy;
int drc_effect;
int drc_cut;
+int album_mode;
int level_limit;
int output_delay;
} FDKAACDecContext;
@@ -87,6 +88,10 @@ static const AVOption fdk_aac_dec_options[] = {
#if FDKDEC_VER_AT_LEAST(3, 0) // 3.0.0
{ "drc_effect","Dynamic Range Control: effect type, where e.g. [0] is none
and [6] is general",
OFFSET(drc_effect), AV_OPT_TYPE_INT, { .i64 = -1},
-1, 8, AD, NULL},
+#endif
+#if FDKDEC_VER_AT_LEAST(3, 1) // 3.1.0
+{ "album_mode","Dynamic Range Control: album mode, where [0] is off and
[1] is on",
+ OFFSET(album_mode), AV_OPT_TYPE_INT, { .i64 = -1},
-1, 1, AD, NULL},
#endif
{ NULL }
};
@@ -335,6 +340,15 @@ static av_cold int fdk_aac_decode_init(AVCodecContext
*avctx)
}
#endif
+#if FDKDEC_VER_AT_LEAST(3, 1) // 3.1.0
+if (s->album_mode != -1) {
+if (aacDecoder_SetParam(s->handle, AAC_UNIDRC_ALBUM_MODE,
s->album_mode) != AAC_DEC_OK) {
+av_log(avctx, AV_LOG_ERROR, "Unable to set album mode in the
decoder\n");
+return AVERROR_UNKNOWN;
+}
+}
+#endif
+
avctx->sample_fmt = AV_SAMPLE_FMT_S16;
s->decoder_buffer_size = DECODER_BUFFSIZE * DECODER_MAX_CHANNELS;
--
2.32.0 (Apple Git-132)
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 2/2] libfdk-aacdec: Flush delayed samples at the end
Also trim off delay samples at the start instead of adjusting pts
to compensate for them; this avoids unwanted offsets if working
with raw samples without considering their pts.
---
libavcodec/libfdk-aacdec.c | 80 +++---
1 file changed, 65 insertions(+), 15 deletions(-)
diff --git a/libavcodec/libfdk-aacdec.c b/libavcodec/libfdk-aacdec.c
index 93b52023b0..d560e313ca 100644
--- a/libavcodec/libfdk-aacdec.c
+++ b/libavcodec/libfdk-aacdec.c
@@ -58,7 +58,11 @@ typedef struct FDKAACDecContext {
int drc_cut;
int album_mode;
int level_limit;
-int output_delay;
+#if FDKDEC_VER_AT_LEAST(2, 5) // 2.5.10
+int output_delay_set;
+int flush_samples;
+int delay_samples;
+#endif
} FDKAACDecContext;
@@ -123,7 +127,12 @@ static int get_stream_info(AVCodecContext *avctx)
avctx->sample_rate = info->sampleRate;
avctx->frame_size = info->frameSize;
#if FDKDEC_VER_AT_LEAST(2, 5) // 2.5.10
-s->output_delay= info->outputDelay;
+if (!s->output_delay_set && info->outputDelay) {
+// Set this only once.
+s->flush_samples= info->outputDelay;
+s->delay_samples= info->outputDelay;
+s->output_delay_set = 1;
+}
#endif
for (i = 0; i < info->numChannels; i++) {
@@ -367,14 +376,31 @@ static int fdk_aac_decode_frame(AVCodecContext *avctx,
void *data,
int ret;
AAC_DECODER_ERROR err;
UINT valid = avpkt->size;
+UINT flags = 0;
+int input_offset = 0;
-err = aacDecoder_Fill(s->handle, &avpkt->data, &avpkt->size, &valid);
-if (err != AAC_DEC_OK) {
-av_log(avctx, AV_LOG_ERROR, "aacDecoder_Fill() failed: %x\n", err);
-return AVERROR_INVALIDDATA;
+if (avpkt->size) {
+err = aacDecoder_Fill(s->handle, &avpkt->data, &avpkt->size, &valid);
+if (err != AAC_DEC_OK) {
+av_log(avctx, AV_LOG_ERROR, "aacDecoder_Fill() failed: %x\n", err);
+return AVERROR_INVALIDDATA;
+}
+} else {
+#if FDKDEC_VER_AT_LEAST(2, 5) // 2.5.10
+/* Handle decoder draining */
+if (s->flush_samples > 0) {
+flags |= AACDEC_FLUSH;
+} else {
+return AVERROR_EOF;
+}
+#else
+return AVERROR_EOF;
+#endif
}
-err = aacDecoder_DecodeFrame(s->handle, (INT_PCM *) s->decoder_buffer,
s->decoder_buffer_size / sizeof(INT_PCM), 0);
+err = aacDecoder_DecodeFrame(s->handle, (INT_PCM *) s->decoder_buffer,
+ s->decoder_buffer_size / sizeof(INT_PCM),
+ flags);
if (err == AAC_DEC_NOT_ENOUGH_BITS) {
ret = avpkt->size - valid;
goto end;
@@ -390,16 +416,36 @@ static int fdk_aac_decode_frame(AVCodecContext *avctx,
void *data,
goto end;
frame->nb_samples = avctx->frame_size;
+#if FDKDEC_VER_AT_LEAST(2, 5) // 2.5.10
+if (flags & AACDEC_FLUSH) {
+// Only return the right amount of samples at the end; if calling the
+// decoder with AACDEC_FLUSH, it will keep returning frames
indefinitely.
+frame->nb_samples = FFMIN(s->flush_samples, frame->nb_samples);
+av_log(s, AV_LOG_DEBUG, "Returning %d/%d delayed samples.\n",
+frame->nb_samples, s->flush_samples);
+s->flush_samples -= frame->nb_samples;
+} else {
+// Trim off samples from the start to compensate for extra decoder
+// delay. We could also just adjust the pts, but this avoids
+// including the extra samples in the output altogether.
+if (s->delay_samples) {
+int drop_samples = FFMIN(s->delay_samples, frame->nb_samples);
+av_log(s, AV_LOG_DEBUG, "Dropping %d/%d delayed samples.\n",
+drop_samples, s->delay_samples);
+s->delay_samples -= drop_samples;
+frame->nb_samples -= drop_samples;
+input_offset = drop_samples * avctx->channels;
+if (frame->nb_samples <= 0)
+return 0;
+}
+}
+#endif
+
if ((ret = ff_get_buffer(avctx, frame, 0)) < 0)
goto end;
-if (frame->pts != AV_NOPTS_VALUE)
-frame->pts -= av_rescale_q(s->output_delay,
- (AVRational){1, avctx->sample_rate},
- avctx->time_base);
-
-memcpy(frame->extended_data[0], s->decoder_buffer,
- avctx->channels * avctx->frame_size *
+memcpy(frame->extended_data[0], s->decoder_buffer + input_offset,
+ avctx->channels * frame->nb_samples *
av_get_bytes_per_sample(avctx->sample_fmt));
*got_frame_ptr = 1;
@@ -432,7 +478,11 @@ const AVCodec ff_libfdk_aac_decoder = {
.decode = fdk_aac_decode_frame,
.close = fdk_aac_decode_close,
.flush = fdk_aac_decode_flush,
-.capabilities = AV_CODEC_CAP_DR1 | AV_CODEC_CAP_CHANNEL_CONF,
+.capabilities = AV_CODEC_CAP_DR1 | AV_CODEC_
Re: [FFmpeg-devel] [PATCH 001/289 v6] Add a new channel layout API
On Tue, 18 Jan 2022, James Almer wrote: From: Anton Khirnov The new API is more extensible and allows for custom layouts. More accurate information is exported, eg for decoders that do not set a channel layout, lavc will not make one up for them. Deprecate the old API working with just uint64_t bitmasks. Expanded and completed by Vittorio Giovara and James Almer . Signed-off-by: Vittorio Giovara Signed-off-by: James Almer --- Changes since last version: *av_channel_layout_from_string() and av_channel_layout_describe() now support a "designation@name" syntax, effectively making both functions reciprocal when there are custom names in some or all channels. It's the syntax suggested by Marton and is both extensible if required and not too ulgy in human readable output if the string is printed verbatim. *av_channel_layout_index_from_string() and av_channel_layout_channel_from_string() also support this syntax now. I have two suggestions. None of them is actually blocking, but since they concern public syntax and API it might be better to decide now. av_channel_layout_index_from_string() without "@" in the string matches both builtin names (designations) and custom names. I suggest for it to only match builtin names. If a user wants to match by custom name, "@name" may be specified without a designation. This keeps the two namespaces separate. av_channel_layout_channel_from_string() is an API which is not used in the codebase. I am not enterely sure about its purpose, so maybe it should be removed for now, and only added later, if some code actually needs it. Thanks, Marton ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2 1/6] avformat/mux: Remove assert based on faulty assumptions
This assert is based upon the wrong assumption that
the noninterleaved codepath is never used; if it is used,
max_interleave_delta is irrelevant. It furthermore
ignores audio_preload.
Signed-off-by: Andreas Rheinhardt
---
libavformat/mux.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/libavformat/mux.c b/libavformat/mux.c
index c387f8ec6e..e34fd88f05 100644
--- a/libavformat/mux.c
+++ b/libavformat/mux.c
@@ -697,7 +697,6 @@ static int write_packet(AVFormatContext *s, AVPacket *pkt)
);
}
} else {
-av_assert2(pkt->dts == AV_NOPTS_VALUE || pkt->dts >= 0 ||
s->max_interleave_delta > 0);
if (pkt->dts != AV_NOPTS_VALUE && pkt->dts < 0) {
av_log(s, AV_LOG_WARNING,
"Packets poorly interleaved, failed to avoid negative "
--
2.32.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2 2/6] fate/matroska: Add test for avoiding negative timestamps
This tests the issue from tickets #4536, #5784; the output of this test is currently broken. Signed-off-by: Andreas Rheinhardt --- Now only performing codec copy. tests/fate/matroska.mak | 17 + tests/ref/fate/matroska-avoid-negative-ts | 42 +++ 2 files changed, 59 insertions(+) create mode 100644 tests/ref/fate/matroska-avoid-negative-ts diff --git a/tests/fate/matroska.mak b/tests/fate/matroska.mak index 2fcac34d62..b65a76411b 100644 --- a/tests/fate/matroska.mak +++ b/tests/fate/matroska.mak @@ -90,6 +90,23 @@ FATE_MATROSKA_FFMPEG_FFPROBE-$(call ALLYES, FILE_PROTOCOL MOV_DEMUXER \ += fate-matroska-dovi-write-config7 fate-matroska-dovi-write-config7: CMD = transcode mov $(TARGET_SAMPLES)/mov/dovi-p7.mp4 matroska "-map 0 -c copy -cues_to_front yes -reserve_index_space 40 -metadata_header_padding 64339" "-map 0 -c copy" "" "-show_entries stream_side_data_list" +# This tests the scenario like tickets #4536, #5784 where +# the first packet (with the overall lowest dts) is a video packet, +# whereas an audio packet to be muxed later has the overall lowest pts +# which happens to be negative and therefore needs to be shifted. +# This is currently buggy (the timestamps of the video frames muxed +# before the first audio frame are not shifted). +# (-ss 1.09 ensures that a video frame has the lowest dts of all packets; +# yet there is an audio packet with the overall lowest pts. output_ts_offset +# makes the pts of the audio packet, but not the leading video packet negative +# so that we run into the above issue.) +FATE_MATROSKA-$(call ALLYES, FILE_PROTOCOL MPEGTS_DEMUXER MPEGVIDEO_PARSER \ + MPEG2VIDEO_DECODER EXTRACT_EXTRADATA_BSF \ + MP3FLOAT_DECODER MATROSKA_MUXER\ + MATROSKA_DEMUXER FRAMECRC_MUXER PIPE_PROTOCOL) \ ++= fate-matroska-avoid-negative-ts +fate-matroska-avoid-negative-ts: CMD = transcode mpegts $(TARGET_SAMPLES)/mpeg2/t.mpg matroska "-c copy -ss 1.09 -output_ts_offset -60ms" "-c copy -t 0.4" + # This tests writing the MS-compatibility modes V_MS/VFW/FOURCC and A_MS/ACM. # It furthermore tests writing the Cues at the front if the cues_to_front # option is set and more than enough space has been reserved in advance. diff --git a/tests/ref/fate/matroska-avoid-negative-ts b/tests/ref/fate/matroska-avoid-negative-ts new file mode 100644 index 00..a687c8f63c --- /dev/null +++ b/tests/ref/fate/matroska-avoid-negative-ts @@ -0,0 +1,42 @@ +3349536550047c5c553215003ba2acb7 *tests/data/fate/matroska-avoid-negative-ts.matroska +973070 tests/data/fate/matroska-avoid-negative-ts.matroska +#extradata 0: 22, 0x2885037c +#tb 0: 1/1000 +#media_type 0: video +#codec_id 0: mpeg2video +#dimensions 0: 352x288 +#sar 0: 12/11 +#tb 1: 1/1000 +#media_type 1: audio +#codec_id 1: mp3 +#sample_rate 1: 44100 +#channel_layout 1: 4 +#channel_layout_name 1: mono +0,-37, 24, 40, 9156, 0xe5bd034a, S=1, 40 +1, 0, 0, 26, 417, 0x7198c15e +0, 3, 3, 40, 1740, 0x29ac4480, F=0x0 +0, 24,123, 40, 3672, 0x98652013, F=0x0 +1, 26, 26, 26, 417, 0x3c67c32d +1, 52, 52, 26, 417, 0x8c24b1ca +1, 78, 78, 26, 417, 0x6ee576b7 +0, 83, 83, 40, 2532, 0xa2c42769, F=0x0 +1,104,104, 26, 417, 0x407603db +0,123,203, 40, 1728, 0xae823d3b, F=0x0 +1,130,130, 26, 417, 0xcf2804d2 +1,156,156, 26, 417, 0xcf2804d2 +0,163,163, 40, 1028, 0x286ac52a, F=0x0 +1,182,182, 26, 417, 0xcf2804d2 +0,203,283, 40, 1916, 0xd378899e, F=0x0 +1,208,208, 26, 417, 0xcf2804d2 +1,235,235, 26, 417, 0xcf2804d2 +0,243,243, 40, 1168, 0x424e12cf, F=0x0 +1,261,261, 26, 417, 0xcf2804d2 +0,283,363, 40, 1660, 0x5cec156c, F=0x0 +1,287,287, 26, 417, 0xcf2804d2 +1,313,313, 26, 417, 0xef163d04 +0,323,323, 40, 1004, 0xac0dce29, F=0x0 +1,339,339, 26, 417, 0x2a009b3a +0,363,443, 40, 3008, 0x0fc798bf, F=0x0 +1,365,365, 26, 417, 0xbedccb9d +1,365,365, 26, 417, 0x2214be3f +1,391,391, 26, 417, 0x8953b878 -- 2.32.0 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg
[FFmpeg-devel] [PATCH v2 3/6] avformat/avformat: Add AVFMT_AVOID_NEG_TS_DISABLED
And also don't use explicit constants in the movenc test.
Signed-off-by: Andreas Rheinhardt
---
libavformat/avformat.h | 1 +
libavformat/mux.c | 2 +-
libavformat/options_table.h | 2 +-
libavformat/tests/movenc.c | 4 ++--
libavformat/webm_chunk.c| 2 +-
5 files changed, 6 insertions(+), 5 deletions(-)
diff --git a/libavformat/avformat.h b/libavformat/avformat.h
index 6ce367e854..cd253fb28e 100644
--- a/libavformat/avformat.h
+++ b/libavformat/avformat.h
@@ -1533,6 +1533,7 @@ typedef struct AVFormatContext {
*/
int avoid_negative_ts;
#define AVFMT_AVOID_NEG_TS_AUTO -1 ///< Enabled when required by
target format
+#define AVFMT_AVOID_NEG_TS_DISABLED 0 ///< Do not shift timestamps
even when they are negative.
#define AVFMT_AVOID_NEG_TS_MAKE_NON_NEGATIVE 1 ///< Shift timestamps so they
are non negative
#define AVFMT_AVOID_NEG_TS_MAKE_ZERO 2 ///< Shift timestamps so that
they start at 0
diff --git a/libavformat/mux.c b/libavformat/mux.c
index e34fd88f05..a1917878a5 100644
--- a/libavformat/mux.c
+++ b/libavformat/mux.c
@@ -421,7 +421,7 @@ static int init_pts(AVFormatContext *s)
if (s->avoid_negative_ts < 0) {
av_assert2(s->avoid_negative_ts == AVFMT_AVOID_NEG_TS_AUTO);
if (s->oformat->flags & (AVFMT_TS_NEGATIVE | AVFMT_NOTIMESTAMPS)) {
-s->avoid_negative_ts = 0;
+s->avoid_negative_ts = AVFMT_AVOID_NEG_TS_DISABLED;
} else
s->avoid_negative_ts = AVFMT_AVOID_NEG_TS_MAKE_NON_NEGATIVE;
}
diff --git a/libavformat/options_table.h b/libavformat/options_table.h
index 62c5bb40a3..86d836cfeb 100644
--- a/libavformat/options_table.h
+++ b/libavformat/options_table.h
@@ -95,7 +95,7 @@ static const AVOption avformat_options[] = {
{"max_ts_probe", "maximum number of packets to read while waiting for the
first timestamp", OFFSET(max_ts_probe), AV_OPT_TYPE_INT, { .i64 = 50 }, 0,
INT_MAX, D },
{"avoid_negative_ts", "shift timestamps so they start at 0",
OFFSET(avoid_negative_ts), AV_OPT_TYPE_INT, {.i64 = -1}, -1, 2, E,
"avoid_negative_ts"},
{"auto", "enabled when required by target format",0,
AV_OPT_TYPE_CONST, {.i64 = AVFMT_AVOID_NEG_TS_AUTO }, INT_MIN,
INT_MAX, E, "avoid_negative_ts"},
-{"disabled", "do not change timestamps", 0,
AV_OPT_TYPE_CONST, {.i64 = 0 },INT_MIN,
INT_MAX, E, "avoid_negative_ts"},
+{"disabled", "do not change timestamps", 0,
AV_OPT_TYPE_CONST, {.i64 = AVFMT_AVOID_NEG_TS_DISABLED }, INT_MIN,
INT_MAX, E, "avoid_negative_ts"},
{"make_non_negative", "shift timestamps so they are non negative", 0,
AV_OPT_TYPE_CONST, {.i64 = AVFMT_AVOID_NEG_TS_MAKE_NON_NEGATIVE }, INT_MIN,
INT_MAX, E, "avoid_negative_ts"},
{"make_zero", "shift timestamps so they start at 0", 0,
AV_OPT_TYPE_CONST, {.i64 = AVFMT_AVOID_NEG_TS_MAKE_ZERO }, INT_MIN,
INT_MAX, E, "avoid_negative_ts"},
{"dump_separator", "set information dump field separator",
OFFSET(dump_separator), AV_OPT_TYPE_STRING, {.str = ", "}, 0, 0, D|E},
diff --git a/libavformat/tests/movenc.c b/libavformat/tests/movenc.c
index 2af72f11c7..ddcb053bf2 100644
--- a/libavformat/tests/movenc.c
+++ b/libavformat/tests/movenc.c
@@ -455,7 +455,7 @@ int main(int argc, char **argv)
init_count_warnings();
init_out("empty-moov-no-elst-no-adjust");
av_dict_set(&opts, "movflags", "frag_keyframe+empty_moov", 0);
-av_dict_set(&opts, "avoid_negative_ts", "0", 0);
+av_dict_set(&opts, "avoid_negative_ts", "disabled", 0);
init(1, 0);
mux_gops(2);
finish();
@@ -578,7 +578,7 @@ int main(int argc, char **argv)
// one before.
av_dict_set(&opts, "movflags", "frag_custom+empty_moov+dash+frag_discont",
0);
av_dict_set(&opts, "fragment_index", "2", 0);
-av_dict_set(&opts, "avoid_negative_ts", "0", 0);
+av_dict_set(&opts, "avoid_negative_ts", "disabled", 0);
av_dict_set(&opts, "use_editlist", "0", 0);
init(0, 0);
skip_gops(1);
diff --git a/libavformat/webm_chunk.c b/libavformat/webm_chunk.c
index 24390e8e74..9348e6680a 100644
--- a/libavformat/webm_chunk.c
+++ b/libavformat/webm_chunk.c
@@ -127,7 +127,7 @@ fail:
s->avoid_negative_ts = oc->avoid_negative_ts;
ffformatcontext(s)->avoid_negative_ts_use_pts =
ffformatcontext(oc)->avoid_negative_ts_use_pts;
-oc->avoid_negative_ts = 0;
+oc->avoid_negative_ts = AVFMT_AVOID_NEG_TS_DISABLED;
return 0;
}
--
2.32.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2 4/6] avformat/mux: Preserve sync even if later packet has negative ts
write_packet() has code to shift the packets timestamps
to make them nonnegative or even make them start at ts zero;
this code inspects every packet that is written and if a packet
with negative timestamp (whether this is dts or pts depends upon
another flag; basically: Matroska uses pts, everyone else dts)
is encountered, this is offset to make the timestamp zero.
All further packets will be offset accordingly (with the offset
converted according to the streams' timebases).
This is based around an assumption, namely that the timestamps
are indeed non-decreasing, so that the first packet with negative
timestamps is the first packet with timestamps. This assumption
is often fulfilled given that the default interleavement function
by default interleaves per dts; yet there are scenarios in which
it may not be fulfilled:
a) av_write_frame() instead of av_interleaved_write_frame() is used.
b) The audio_preload option is used.
c) When the timestamps that are made nonnegative/zero are pts
(i.e. with Matroska), because the packet with the smallest dts
is not necessarily the packet with the smallest pts.
d) Possibly with custom interleavement functions.
In these cases the relative sync of the first few packet(s) is offset
relative to the later packets. This contradicts the documentation
("When shifting is enabled, all output timestamps are shifted by
the same amount").
Therefore this commit changes this: As soon as the first packet
with valid timestamps is output, it is checked and recorded whether
the timestamps need to be shifted. Further packets are no longer
checked for needing to be offset; instead they are simply offset.
In the cases above this leads to packets with negative timestamps
(and the appropriate warnings) instead of desync. This will mostly
be fixed in the next commit.
This commit also factors handling the avoid_negative_ts stuff out
of write_packet() in order to be able to return immediately.
Tickets #4536 and #5784 as well as the matroska-avoid-negative-ts-test
are examples of c); as has been said, some timestamps are now negative,
yet the ref file update does not show it because ffmpeg.c sanitizes
the timestamps (-copyts disables it; ffprobe and mkvinfo also show
the original timestamps).
Signed-off-by: Andreas Rheinhardt
---
libavformat/internal.h| 23 +++--
libavformat/mux.c | 111 +-
libavformat/options.c | 1 -
tests/fate/matroska.mak | 4 +-
tests/ref/fate/matroska-avoid-negative-ts | 6 +-
5 files changed, 81 insertions(+), 64 deletions(-)
diff --git a/libavformat/internal.h b/libavformat/internal.h
index bffb8e66ff..f24c68703f 100644
--- a/libavformat/internal.h
+++ b/libavformat/internal.h
@@ -82,6 +82,17 @@ typedef struct FFFormatContext {
*/
int nb_interleaved_streams;
+/**
+ * Whether the timestamp shift offset has already been determined.
+ * -1: disabled, 0: not yet determined, 1: determined.
+ */
+enum {
+AVOID_NEGATIVE_TS_DISABLED = -1,
+AVOID_NEGATIVE_TS_UNKNOWN = 0,
+AVOID_NEGATIVE_TS_KNOWN= 1,
+} avoid_negative_ts_status;
+#define AVOID_NEGATIVE_TS_ENABLED(status) ((status) >= 0)
+
/**
* The interleavement function in use. Always set for muxers.
*/
@@ -135,18 +146,6 @@ typedef struct FFFormatContext {
*/
int raw_packet_buffer_size;
-/**
- * Offset to remap timestamps to be non-negative.
- * Expressed in timebase units.
- * @see AVStream.mux_ts_offset
- */
-int64_t offset;
-
-/**
- * Timebase for the timestamp offset.
- */
-AVRational offset_timebase;
-
#if FF_API_COMPUTE_PKT_FIELDS2
int missing_ts_warning;
#endif
diff --git a/libavformat/mux.c b/libavformat/mux.c
index a1917878a5..0810b674a7 100644
--- a/libavformat/mux.c
+++ b/libavformat/mux.c
@@ -388,6 +388,8 @@ fail:
static int init_pts(AVFormatContext *s)
{
+FFFormatContext *const si = ffformatcontext(s);
+
/* init PTS generation */
for (unsigned i = 0; i < s->nb_streams; i++) {
AVStream *const st = s->streams[i];
@@ -418,13 +420,16 @@ static int init_pts(AVFormatContext *s)
}
}
+si->avoid_negative_ts_status = AVOID_NEGATIVE_TS_UNKNOWN;
if (s->avoid_negative_ts < 0) {
av_assert2(s->avoid_negative_ts == AVFMT_AVOID_NEG_TS_AUTO);
if (s->oformat->flags & (AVFMT_TS_NEGATIVE | AVFMT_NOTIMESTAMPS)) {
s->avoid_negative_ts = AVFMT_AVOID_NEG_TS_DISABLED;
+si->avoid_negative_ts_status = AVOID_NEGATIVE_TS_DISABLED;
} else
s->avoid_negative_ts = AVFMT_AVOID_NEG_TS_MAKE_NON_NEGATIVE;
-}
+} else if (s->avoid_negative_ts == AVFMT_AVOID_NEG_TS_DISABLED)
+si->avoid_negative_ts_status = AVOID_NEGATIVE_TS_DISABLED;
return 0;
}
@@ -638,6 +643,64 @@ static void guess_pkt_duration(AVFormatContext *s,
AVStream *st, AVPacket *pkt)
}
}
+static v
[FFmpeg-devel] [PATCH v2 5/6] avformat/mux: Peek into the muxing queue for avoid_negative_ts
Peeking into the muxing queue can improve the estimate of
the lowest timestamp needed for avoid_negative_ts in case
the lowest timestamp is in a packet other than the first packet
to be muxed.
This fixes tickets #4536 and #5784 as well as the output from
the matroska-avoid-negative-ts FATE-test.
Signed-off-by: Andreas Rheinhardt
---
libavformat/avformat.h| 2 +-
libavformat/mux.c | 21 +++--
tests/fate/matroska.mak | 2 --
tests/ref/fate/matroska-avoid-negative-ts | 2 +-
4 files changed, 21 insertions(+), 6 deletions(-)
diff --git a/libavformat/avformat.h b/libavformat/avformat.h
index cd253fb28e..b4b8075ae6 100644
--- a/libavformat/avformat.h
+++ b/libavformat/avformat.h
@@ -1527,7 +1527,7 @@ typedef struct AVFormatContext {
/**
* Avoid negative timestamps during muxing.
* Any value of the AVFMT_AVOID_NEG_TS_* constants.
- * Note, this only works when using av_interleaved_write_frame.
(interleave_packet_per_dts is in use)
+ * Note, this works better when using av_interleaved_write_frame().
* - muxing: Set by user
* - demuxing: unused
*/
diff --git a/libavformat/mux.c b/libavformat/mux.c
index 0810b674a7..53eb56f0af 100644
--- a/libavformat/mux.c
+++ b/libavformat/mux.c
@@ -655,16 +655,33 @@ static void handle_avoid_negative_ts(FFFormatContext *si,
FFStream *sti,
if (si->avoid_negative_ts_status == AVOID_NEGATIVE_TS_UNKNOWN) {
int use_pts = si->avoid_negative_ts_use_pts;
int64_t ts = use_pts ? pkt->pts : pkt->dts;
+AVRational tb = sti->pub.time_base;
if (ts == AV_NOPTS_VALUE)
return;
+
+/* Peek into the muxing queue to improve our estimate
+ * of the lowest timestamp if av_interleaved_write_frame() is used. */
+for (const PacketListEntry *pktl = si->packet_buffer.head;
+ pktl; pktl = pktl->next) {
+AVRational cmp_tb = s->streams[pktl->pkt.stream_index]->time_base;
+int64_t cmp_ts = use_pts ? pktl->pkt.pts : pktl->pkt.dts;
+if (cmp_ts == AV_NOPTS_VALUE)
+continue;
+if (s->output_ts_offset)
+cmp_ts += av_rescale_q(s->output_ts_offset, AV_TIME_BASE_Q,
cmp_tb);
+if (av_compare_ts(cmp_ts, cmp_tb, ts, tb) < 0) {
+ts = cmp_ts;
+tb = cmp_tb;
+}
+}
+
if (ts < 0 ||
ts > 0 && s->avoid_negative_ts == AVFMT_AVOID_NEG_TS_MAKE_ZERO) {
for (unsigned i = 0; i < s->nb_streams; i++) {
AVStream *const st2 = s->streams[i];
FFStream *const sti2 = ffstream(st2);
-sti2->mux_ts_offset = av_rescale_q_rnd(-ts,
- sti->pub.time_base,
+sti2->mux_ts_offset = av_rescale_q_rnd(-ts, tb,
st2->time_base,
AV_ROUND_UP);
}
diff --git a/tests/fate/matroska.mak b/tests/fate/matroska.mak
index da1fdbd5ea..784b55f6e0 100644
--- a/tests/fate/matroska.mak
+++ b/tests/fate/matroska.mak
@@ -94,8 +94,6 @@ fate-matroska-dovi-write-config7: CMD = transcode mov
$(TARGET_SAMPLES)/mov/dovi
# the first packet (with the overall lowest dts) is a video packet,
# whereas an audio packet to be muxed later has the overall lowest pts
# which happens to be negative and therefore needs to be shifted.
-# This is currently buggy (the timestamps are not shifted properly:
-# the first audio packet has negative pts).
# (-ss 1.09 ensures that a video frame has the lowest dts of all packets;
# yet there is an audio packet with the overall lowest pts. output_ts_offset
# makes the pts of the audio packet, but not the leading video packet negative
diff --git a/tests/ref/fate/matroska-avoid-negative-ts
b/tests/ref/fate/matroska-avoid-negative-ts
index 1b9b2f2786..02790a3985 100644
--- a/tests/ref/fate/matroska-avoid-negative-ts
+++ b/tests/ref/fate/matroska-avoid-negative-ts
@@ -1,4 +1,4 @@
-90cf5a330659140d47ec11208f525908
*tests/data/fate/matroska-avoid-negative-ts.matroska
+804842437b2be0a1604ce33c6b08c800
*tests/data/fate/matroska-avoid-negative-ts.matroska
973070 tests/data/fate/matroska-avoid-negative-ts.matroska
#extradata 0: 22, 0x2885037c
#tb 0: 1/1000
--
2.32.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2 6/6] avformat/hls: Remove redundant cast
Signed-off-by: Andreas Rheinhardt
---
libavformat/hls.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/libavformat/hls.c b/libavformat/hls.c
index 8c526f748f..4568e72cb2 100644
--- a/libavformat/hls.c
+++ b/libavformat/hls.c
@@ -2024,7 +2024,7 @@ static int hls_read_header(AVFormatContext *s)
if (seg && seg->key_type == KEY_SAMPLE_AES && pls->is_id3_timestamped
&&
pls->audio_setup_info.codec_id != AV_CODEC_ID_NONE) {
void *iter = NULL;
-while ((in_fmt = (const AVInputFormat *)av_demuxer_iterate(&iter)))
+while ((in_fmt = av_demuxer_iterate(&iter)))
if (in_fmt->raw_codec_id == pls->audio_setup_info.codec_id)
break;
} else {
--
2.32.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 001/289 v6] Add a new channel layout API
On 1/19/2022 5:51 PM, Marton Balint wrote: On Tue, 18 Jan 2022, James Almer wrote: From: Anton Khirnov The new API is more extensible and allows for custom layouts. More accurate information is exported, eg for decoders that do not set a channel layout, lavc will not make one up for them. Deprecate the old API working with just uint64_t bitmasks. Expanded and completed by Vittorio Giovara and James Almer . Signed-off-by: Vittorio Giovara Signed-off-by: James Almer --- Changes since last version: *av_channel_layout_from_string() and av_channel_layout_describe() now support a "designation@name" syntax, effectively making both functions reciprocal when there are custom names in some or all channels. It's the syntax suggested by Marton and is both extensible if required and not too ulgy in human readable output if the string is printed verbatim. *av_channel_layout_index_from_string() and av_channel_layout_channel_from_string() also support this syntax now. I have two suggestions. None of them is actually blocking, but since they concern public syntax and API it might be better to decide now. av_channel_layout_index_from_string() without "@" in the string matches both builtin names (designations) and custom names. I suggest for it to only match builtin names. If a user wants to match by custom name, "@name" may be specified without a designation. This keeps the two namespaces separate. Ok, look into implementing this. av_channel_layout_channel_from_string() is an API which is not used in the codebase. I am not enterely sure about its purpose, so maybe it should be removed for now, and only added later, if some code actually needs it. Since you can set custom names and query them without the designation, it can be useful as a shortcut for "idx = av_channel_layout_index_from_string(str) && av_channel_layout_channel_from_index(idx)". If there were no custom names then yeah, there would be no point to it. Thanks, Marton ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 001/289 v6] Add a new channel layout API
On 1/19/2022 3:07 PM, Andreas Rheinhardt wrote: James Almer: From: Anton Khirnov The new API is more extensible and allows for custom layouts. More accurate information is exported, eg for decoders that do not set a channel layout, lavc will not make one up for them. Deprecate the old API working with just uint64_t bitmasks. Expanded and completed by Vittorio Giovara and James Almer . Signed-off-by: Vittorio Giovara Signed-off-by: James Almer --- Changes since last version: *av_channel_layout_from_string() and av_channel_layout_describe() now support a "designation@name" syntax, effectively making both functions reciprocal when there are custom names in some or all channels. It's the syntax suggested by Marton and is both extensible if required and not too ulgy in human readable output if the string is printed verbatim. *av_channel_layout_index_from_string() and av_channel_layout_channel_from_string() also support this syntax now. I plan to push this version soon. Any extension to the syntax supported by these helpers can be written by whoever needs it. You forgot the examples and the tools; and ffplay. And do you really Ported ffplay. Will port the rest later. intend for the deprecation warnings in ffmpeg_opt.c to stay? Ok, I'll silence them. - Andreas ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2 00/26] Subtitle Filtering 2022
Subtitle Filtering 2022
===
This is a substantial update to the earlier subtitle filtering patch series.
A primary goal has been to address others' concerns as much as possible on
one side and to provide more clarity and control over the way things are
working. Clarity is is specifically important to allow for a better
understanding of the need for a subtitle start pts value that can be
different from the frame's pts value. This is done by refactoring the
subtitle timing fields in AVFrame, adding a frame field to indicate repeated
subtitle frames, and finally the full removal of the heartbeat
functionality, replaced by a new 'subfeed' filter that provides different
modes for arbitrating subtitle frames in a filter graph. Finally, each
subtitle filter's documentation has been amended by a section describing the
filter's timeline behavior (in v3 update).
The update also includes major improvements to graphicsub2text and lots of
other details.
Versioning is restarting at v1 due to the new submission procedure.
Changes in v2
=
* added .gitattributes file to enforce binary diffs for the test refs that
cannot be applied when being sent via e-mail
* perform filter graph re-init due to subtitle "frame size" change only
when the size was unknown before and not set via -canvas_size
* overlaytextsubs: Make sure to request frames on the subtitle input
* avfilter/splitcc: Start parsing cc data on key frames only
* avcodec/webvttenc: Don't encode ass drawing codes and empty lines
* stripstyles: fix mem leak
* gs2t: improve color detection
* gs2t: empty frames must not be skipped
* subfeed: fix name
* textmod: preserve margins
* added .gitattributes file to enforce binary diffs for the test refs that
cannot be applied when being sent via e-mail
* perform filter graph re-init due to subtitle "frame size" change only
when the size was unknown before and not set via -canvas_size
* avcodec/dvbsubdec: Fix conditions for fallback to default resolution
* Made changes suggested by Andreas
* Fixed failing command line reported by Michael
Changes from previous version v24:
AVFrame
===
* Removed sub_start_time The start time is now added to the subtitle
start_pts during decoding The sub_end_time field is adjusted accordingly
* Renamed sub_end_time to duration which it is effectively after removing
the start_time
* Added a sub-struct 'subtitle_timing' to av frame Contains subtitle_pts
renamed to 'subtitle_timing.start_pts' and 'subtitle_timing.duration'
* Change both fields to (fixed) time_base AV_TIMEBASE
* add repeat_sub field provides a clear indication whether a subtitle frame
is an actual subtitle event or a repeated subtitle frame in a filter
graph
Heartbeat Removal
=
* completely removed the earlier heartbeat implementation
* filtering arbitration is now implemented in a new filter: 'subfeed'
* subfeed will be auto-inserted for compatiblity with sub2video command
lines
* the new behavior is not exactly identical to the earlier behavior, but it
basically allows to achieve the same results
New 'subfeed' Filter
* a versatile filter for solving all kinds of problems with subtile frame
flow in filter graphs
* Can be inserted at any position in a graph
* Auto-inserted for sub2video command lines (in repeat-mode)
* Allows duration fixup delay input frames with unknown duration and infer
duration from start of subsequent frame
* Provides multiple modes of operation:
* repeat mode (default) Queues input frames Outputs frames at a fixed
(configurable) rate Either sends a matching input frame (repeatedly) or
empty frames otherwise
* scatter mode similar to repeat mode, but splits input frames by
duration into small segments with same content
* forward mode No fixed output rate Useful in combination with duration
fixup or overlap fixup
ffmpeg Tool Changes
===
* delay subtitle output stream initialization (like for audio and video)
This is needed for example when a format header depends on having
received an initial frame to derive certain header values from
* decoding: set subtitle frame size from decoding context
* re-init graph when subtitle size changes
* always insert subscale filter for sub2video command lines (to ensure
correct scaling)
Subtitle Encoding
=
* ignore repeated frames for encoding based on repeat_sub field in AVFrame
* support multi-area encoding for text subtitles Subtitle OCR can create
multiple areas at different positions. Previously, the texts were always
squashed into a single area ('subtitle rect'), which was not ideal.
Multiple text areas are now generally supported:
* ASS Encoder Changed to use the 'receive_packet' encoding API A single
frame with multiple text areas will create multiple packets now
* All other text subtitle encoders A newline is inserte
[FFmpeg-devel] [PATCH v2 01/26] avcodec, avutil: Move enum AVSubtitleType to avutil, add new and deprecate old values
From: softworkz
Signed-off-by: softworkz
---
libavcodec/avcodec.h | 19 +
libavutil/Makefile | 1 +
libavutil/subfmt.h | 68
libavutil/version.h | 1 +
4 files changed, 71 insertions(+), 18 deletions(-)
create mode 100644 libavutil/subfmt.h
diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index ec1a0566a4..fe5a83cf85 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -35,6 +35,7 @@
#include "libavutil/frame.h"
#include "libavutil/log.h"
#include "libavutil/pixfmt.h"
+#include "libavutil/subfmt.h"
#include "libavutil/rational.h"
#include "codec.h"
@@ -2238,24 +2239,6 @@ typedef struct AVHWAccel {
* @}
*/
-enum AVSubtitleType {
-SUBTITLE_NONE,
-
-SUBTITLE_BITMAP,///< A bitmap, pict will be set
-
-/**
- * Plain text, the text field must be set by the decoder and is
- * authoritative. ass and pict fields may contain approximations.
- */
-SUBTITLE_TEXT,
-
-/**
- * Formatted text, the ass field must be set by the decoder and is
- * authoritative. pict and text fields may contain approximations.
- */
-SUBTITLE_ASS,
-};
-
#define AV_SUBTITLE_FLAG_FORCED 0x0001
typedef struct AVSubtitleRect {
diff --git a/libavutil/Makefile b/libavutil/Makefile
index d17876df1a..ce644f4d48 100644
--- a/libavutil/Makefile
+++ b/libavutil/Makefile
@@ -74,6 +74,7 @@ HEADERS = adler32.h
\
sha512.h \
spherical.h \
stereo3d.h\
+ subfmt.h \
threadmessage.h \
time.h\
timecode.h\
diff --git a/libavutil/subfmt.h b/libavutil/subfmt.h
new file mode 100644
index 00..791b45519f
--- /dev/null
+++ b/libavutil/subfmt.h
@@ -0,0 +1,68 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#ifndef AVUTIL_SUBFMT_H
+#define AVUTIL_SUBFMT_H
+
+#include "version.h"
+
+enum AVSubtitleType {
+
+/**
+ * Subtitle format unknown.
+ */
+AV_SUBTITLE_FMT_NONE = -1,
+
+/**
+ * Subtitle format unknown.
+ */
+AV_SUBTITLE_FMT_UNKNOWN = 0,
+#if FF_API_OLD_SUBTITLES
+SUBTITLE_NONE = 0, ///< Deprecated, use AV_SUBTITLE_FMT_NONE
instead.
+#endif
+
+/**
+ * Bitmap area in AVSubtitleRect.data, pixfmt AV_PIX_FMT_PAL8.
+ */
+AV_SUBTITLE_FMT_BITMAP = 1,
+#if FF_API_OLD_SUBTITLES
+SUBTITLE_BITMAP = 1,///< Deprecated, use AV_SUBTITLE_FMT_BITMAP
instead.
+#endif
+
+/**
+ * Plain text in AVSubtitleRect.text.
+ */
+AV_SUBTITLE_FMT_TEXT = 2,
+#if FF_API_OLD_SUBTITLES
+SUBTITLE_TEXT = 2, ///< Deprecated, use AV_SUBTITLE_FMT_TEXT
instead.
+#endif
+
+/**
+ * Text Formatted as per ASS specification, contained AVSubtitleRect.ass.
+ */
+AV_SUBTITLE_FMT_ASS = 3,
+#if FF_API_OLD_SUBTITLES
+SUBTITLE_ASS = 3, ///< Deprecated, use AV_SUBTITLE_FMT_ASS
instead.
+#endif
+
+AV_SUBTITLE_FMT_NB, ///< number of subtitle formats, DO NOT USE
THIS if you want to link with shared libav* because the number of formats might
differ between versions.
+};
+
+#endif /* AVUTIL_SUBFMT_H */
diff --git a/libavutil/version.h b/libavutil/version.h
index 953aac9d94..5bf48f6304 100644
--- a/libavutil/version.h
+++ b/libavutil/version.h
@@ -110,6 +110,7 @@
#define FF_API_COLORSPACE_NAME (LIBAVUTIL_VERSION_MAJOR < 58)
#define FF_API_AV_MALLOCZ_ARRAY (LIBAVUTIL_VERSION_MAJOR < 58)
#define FF_API_FIFO_PEEK2 (LIBAVUTIL_VERSION_MAJOR < 58)
+#define FF_API_OLD_SUBTITLES(LIBAVUTIL_VERSION_MAJOR < 58)
/**
* @}
--
ffmpeg-codebot
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or ema
[FFmpeg-devel] [PATCH v2 02/26] avutil/frame: Prepare AVFrame for subtitle handling
From: softworkz
Root commit for adding subtitle filtering capabilities.
In detail:
- Add type (AVMediaType) field to AVFrame
Replaces previous way of distinction which was based on checking
width and height to determine whether a frame is audio or video
- Add subtitle fields to AVFrame
- Add new struct AVSubtitleArea, similar to AVSubtitleRect, but
different allocation logic. Cannot and must not be used
interchangeably, hence the new struct
Signed-off-by: softworkz
---
libavutil/Makefile | 1 +
libavutil/frame.c | 211 -
libavutil/frame.h | 85 +-
libavutil/subfmt.c | 45 ++
libavutil/subfmt.h | 47 ++
5 files changed, 364 insertions(+), 25 deletions(-)
create mode 100644 libavutil/subfmt.c
diff --git a/libavutil/Makefile b/libavutil/Makefile
index ce644f4d48..8bc0a14942 100644
--- a/libavutil/Makefile
+++ b/libavutil/Makefile
@@ -160,6 +160,7 @@ OBJS = adler32.o
\
slicethread.o\
spherical.o \
stereo3d.o \
+ subfmt.o \
threadmessage.o \
time.o \
timecode.o \
diff --git a/libavutil/frame.c b/libavutil/frame.c
index 8997c85e35..2b95830b6f 100644
--- a/libavutil/frame.c
+++ b/libavutil/frame.c
@@ -26,6 +26,7 @@
#include "imgutils.h"
#include "mem.h"
#include "samplefmt.h"
+#include "subfmt.h"
#include "hwcontext.h"
#define CHECK_CHANNELS_CONSISTENCY(frame) \
@@ -50,6 +51,9 @@ const char *av_get_colorspace_name(enum AVColorSpace val)
return name[val];
}
#endif
+
+static int frame_copy_subtitles(AVFrame *dst, const AVFrame *src, int
copy_data);
+
static void get_frame_defaults(AVFrame *frame)
{
memset(frame, 0, sizeof(*frame));
@@ -70,7 +74,12 @@ static void get_frame_defaults(AVFrame *frame)
frame->colorspace = AVCOL_SPC_UNSPECIFIED;
frame->color_range = AVCOL_RANGE_UNSPECIFIED;
frame->chroma_location = AVCHROMA_LOC_UNSPECIFIED;
-frame->flags = 0;
+frame->num_subtitle_areas = 0;
+frame->subtitle_areas = NULL;
+frame->subtitle_header = NULL;
+frame->repeat_sub = 0;
+frame->subtitle_timing.start_pts = 0;
+frame->subtitle_timing.duration = 0;
}
static void free_side_data(AVFrameSideData **ptr_sd)
@@ -240,23 +249,55 @@ static int get_audio_buffer(AVFrame *frame, int align)
}
+static int get_subtitle_buffer(AVFrame *frame)
+{
+// Buffers in AVFrame->buf[] are not used in case of subtitle frames.
+// To accomodate with existing code, checking ->buf[0] to determine
+// whether a frame is ref-counted or has data, we're adding a 1-byte
+// buffer here, which marks the subtitle frame to contain data.
+frame->buf[0] = av_buffer_alloc(1);
+if (!frame->buf[0]) {
+av_frame_unref(frame);
+return AVERROR(ENOMEM);
+}
+
+frame->extended_data = frame->data;
+
+return 0;
+}
+
int av_frame_get_buffer(AVFrame *frame, int align)
+{
+if (frame->width > 0 && frame->height > 0)
+frame->type = AVMEDIA_TYPE_VIDEO;
+else if (frame->nb_samples > 0 && (frame->channel_layout ||
frame->channels > 0))
+frame->type = AVMEDIA_TYPE_AUDIO;
+
+return av_frame_get_buffer2(frame, align);
+}
+
+int av_frame_get_buffer2(AVFrame *frame, int align)
{
if (frame->format < 0)
return AVERROR(EINVAL);
-if (frame->width > 0 && frame->height > 0)
+switch(frame->type) {
+case AVMEDIA_TYPE_VIDEO:
return get_video_buffer(frame, align);
-else if (frame->nb_samples > 0 && (frame->channel_layout ||
frame->channels > 0))
+case AVMEDIA_TYPE_AUDIO:
return get_audio_buffer(frame, align);
-
-return AVERROR(EINVAL);
+case AVMEDIA_TYPE_SUBTITLE:
+return get_subtitle_buffer(frame);
+default:
+return AVERROR(EINVAL);
+}
}
static int frame_copy_props(AVFrame *dst, const AVFrame *src, int force_copy)
{
int ret, i;
+dst->type = src->type;
dst->key_frame = src->key_frame;
dst->pict_type = src->pict_type;
dst->sample_aspect_ratio= src->sample_aspect_ratio;
@@ -288,6 +329,12 @@ static int frame_copy_props(AVFrame *dst, const AVFrame
*src, int force_copy)
dst->colorspace = src->colorspace;
dst->color_range= src->color_range;
dst->chroma_location= src->chroma_location;
+dst->repeat_sub = src->repeat_sub;
+dst->subtitle_timing.start_pts = src->subtitle_timing.start_pts;
+
[FFmpeg-devel] [PATCH v2 03/26] avcodec/subtitles: Introduce new frame-based subtitle decoding API
From: softworkz
- Modify avcodec_send_packet() to support subtitles via the regular
frame based decoding API
- Add decode_subtitle_shim() which takes subtitle frames,
and serves as a compatibility shim to the legacy subtitle decoding
API until all subtitle decoders are migrated to the frame-based API
- Add additional methods for conversion between old and new API
Signed-off-by: softworkz
---
libavcodec/avcodec.h| 8 +-
libavcodec/codec_desc.c | 11 +++
libavcodec/codec_desc.h | 8 ++
libavcodec/decode.c | 60 +++--
libavcodec/internal.h | 16
libavcodec/utils.c | 184
6 files changed, 278 insertions(+), 9 deletions(-)
diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index fe5a83cf85..9d59f6e840 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -1675,7 +1675,7 @@ typedef struct AVCodecContext {
/**
* Header containing style information for text subtitles.
- * For SUBTITLE_ASS subtitle type, it should contain the whole ASS
+ * For AV_SUBTITLE_FMT_ASS subtitle type, it should contain the whole ASS
* [Script Info] and [V4+ Styles] section, plus the [Events] line and
* the Format line following. It shouldn't include any Dialogue line.
* - encoding: Set/allocated/freed by user (before avcodec_open2())
@@ -2415,7 +2415,10 @@ int avcodec_close(AVCodecContext *avctx);
* Free all allocated data in the given subtitle struct.
*
* @param sub AVSubtitle to free.
+ *
+ * @deprecated Use the regular frame based encode and decode APIs instead.
*/
+attribute_deprecated
void avsubtitle_free(AVSubtitle *sub);
/**
@@ -2508,7 +2511,10 @@ enum AVChromaLocation avcodec_chroma_pos_to_enum(int
xpos, int ypos);
* must be freed with avsubtitle_free if *got_sub_ptr is set.
* @param[in,out] got_sub_ptr Zero if no subtitle could be decompressed,
otherwise, it is nonzero.
* @param[in] avpkt The input AVPacket containing the input buffer.
+ *
+ * @deprecated Use the new decode API (avcodec_send_packet,
avcodec_receive_frame) instead.
*/
+attribute_deprecated
int avcodec_decode_subtitle2(AVCodecContext *avctx, AVSubtitle *sub,
int *got_sub_ptr,
AVPacket *avpkt);
diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c
index 0974ee03de..e48e4532ba 100644
--- a/libavcodec/codec_desc.c
+++ b/libavcodec/codec_desc.c
@@ -3548,3 +3548,14 @@ enum AVMediaType avcodec_get_type(enum AVCodecID
codec_id)
const AVCodecDescriptor *desc = avcodec_descriptor_get(codec_id);
return desc ? desc->type : AVMEDIA_TYPE_UNKNOWN;
}
+
+enum AVSubtitleType avcodec_descriptor_get_subtitle_format(const
AVCodecDescriptor *codec_descriptor)
+{
+if(codec_descriptor->props & AV_CODEC_PROP_BITMAP_SUB)
+return AV_SUBTITLE_FMT_BITMAP;
+
+if(codec_descriptor->props & AV_CODEC_PROP_TEXT_SUB)
+return AV_SUBTITLE_FMT_ASS;
+
+return AV_SUBTITLE_FMT_UNKNOWN;
+}
diff --git a/libavcodec/codec_desc.h b/libavcodec/codec_desc.h
index 126b52df47..ba68d24e0e 100644
--- a/libavcodec/codec_desc.h
+++ b/libavcodec/codec_desc.h
@@ -121,6 +121,14 @@ const AVCodecDescriptor *avcodec_descriptor_next(const
AVCodecDescriptor *prev);
*/
const AVCodecDescriptor *avcodec_descriptor_get_by_name(const char *name);
+/**
+ * Return subtitle format from a codec descriptor
+ *
+ * @param codec_descriptor codec descriptor
+ * @return the subtitle type (e.g. bitmap, text)
+ */
+enum AVSubtitleType avcodec_descriptor_get_subtitle_format(const
AVCodecDescriptor *codec_descriptor);
+
/**
* @}
*/
diff --git a/libavcodec/decode.c b/libavcodec/decode.c
index 0912f86a14..e3cf1cfa3d 100644
--- a/libavcodec/decode.c
+++ b/libavcodec/decode.c
@@ -576,6 +576,39 @@ static int decode_receive_frame_internal(AVCodecContext
*avctx, AVFrame *frame)
return ret;
}
+static int decode_subtitle2_priv(AVCodecContext *avctx, AVSubtitle *sub,
+ int *got_sub_ptr, AVPacket *avpkt);
+
+static int decode_subtitle_shim(AVCodecContext *avctx, AVFrame *frame,
AVPacket *avpkt)
+{
+int ret, got_sub_ptr = 0;
+AVSubtitle subtitle = { 0 };
+
+if (frame->buf[0])
+return AVERROR(EAGAIN);
+
+av_frame_unref(frame);
+
+ret = decode_subtitle2_priv(avctx, &subtitle, &got_sub_ptr, avpkt);
+
+if (ret >= 0 && got_sub_ptr) {
+frame->type = AVMEDIA_TYPE_SUBTITLE;
+frame->format = subtitle.format;
+ret = av_frame_get_buffer2(frame, 0);
+
+if (ret >= 0)
+ret = ff_frame_put_subtitle(frame, &subtitle);
+
+frame->width = avctx->width;
+frame->height = avctx->height;
+frame->pkt_dts = avpkt->dts;
+}
+
+avsubtitle_free(&subtitle);
+
+return ret;
+}
+
int attribute_align_arg avcodec_send_packet(AVCodecContext *avctx, const
AVPacket *avpkt)
{
AVCodecInternal *avci = avctx->internal;
@@ -59
[FFmpeg-devel] [PATCH v2 04/26] avfilter/subtitles: Update vf_subtitles to use new decoding api
From: softworkz
Signed-off-by: softworkz
---
libavfilter/vf_subtitles.c | 56 +-
1 file changed, 43 insertions(+), 13 deletions(-)
diff --git a/libavfilter/vf_subtitles.c b/libavfilter/vf_subtitles.c
index 3fc4eeb63d..25e217e845 100644
--- a/libavfilter/vf_subtitles.c
+++ b/libavfilter/vf_subtitles.c
@@ -35,14 +35,12 @@
# include "libavformat/avformat.h"
#endif
#include "libavutil/avstring.h"
-#include "libavutil/imgutils.h"
#include "libavutil/opt.h"
#include "libavutil/parseutils.h"
#include "drawutils.h"
#include "avfilter.h"
#include "internal.h"
#include "formats.h"
-#include "video.h"
typedef struct AssContext {
const AVClass *class;
@@ -292,6 +290,29 @@ static int attachment_is_font(AVStream * st)
return 0;
}
+static int decode(AVCodecContext *avctx, AVFrame *frame, int *got_frame,
AVPacket *pkt)
+{
+int ret;
+
+*got_frame = 0;
+
+if (pkt) {
+ret = avcodec_send_packet(avctx, pkt);
+// In particular, we don't expect AVERROR(EAGAIN), because we read all
+// decoded frames with avcodec_receive_frame() until done.
+if (ret < 0 && ret != AVERROR_EOF)
+return ret;
+}
+
+ret = avcodec_receive_frame(avctx, frame);
+if (ret < 0 && ret != AVERROR(EAGAIN))
+return ret;
+if (ret >= 0)
+*got_frame = 1;
+
+return 0;
+}
+
AVFILTER_DEFINE_CLASS(subtitles);
static av_cold int init_subtitles(AVFilterContext *ctx)
@@ -306,6 +327,7 @@ static av_cold int init_subtitles(AVFilterContext *ctx)
AVStream *st;
AVPacket pkt;
AssContext *ass = ctx->priv;
+enum AVSubtitleType subtitle_format;
/* Init libass */
ret = init(ctx);
@@ -386,13 +408,17 @@ static av_cold int init_subtitles(AVFilterContext *ctx)
ret = AVERROR_DECODER_NOT_FOUND;
goto end;
}
+
dec_desc = avcodec_descriptor_get(st->codecpar->codec_id);
-if (dec_desc && !(dec_desc->props & AV_CODEC_PROP_TEXT_SUB)) {
+subtitle_format = avcodec_descriptor_get_subtitle_format(dec_desc);
+
+if (subtitle_format != AV_SUBTITLE_FMT_ASS) {
av_log(ctx, AV_LOG_ERROR,
- "Only text based subtitles are currently supported\n");
-ret = AVERROR_PATCHWELCOME;
+ "Only text based subtitles are supported by this filter\n");
+ret = AVERROR_INVALIDDATA;
goto end;
}
+
if (ass->charenc)
av_dict_set(&codec_opts, "sub_charenc", ass->charenc, 0);
@@ -448,27 +474,31 @@ static av_cold int init_subtitles(AVFilterContext *ctx)
dec_ctx->subtitle_header_size);
while (av_read_frame(fmt, &pkt) >= 0) {
int i, got_subtitle;
-AVSubtitle sub = {0};
+AVFrame *sub = av_frame_alloc();
+if (!sub) {
+ret = AVERROR(ENOMEM);
+goto end;
+}
if (pkt.stream_index == sid) {
-ret = avcodec_decode_subtitle2(dec_ctx, &sub, &got_subtitle, &pkt);
+ret = decode(dec_ctx, sub, &got_subtitle, &pkt);
if (ret < 0) {
av_log(ctx, AV_LOG_WARNING, "Error decoding: %s (ignored)\n",
av_err2str(ret));
} else if (got_subtitle) {
-const int64_t start_time = av_rescale_q(sub.pts,
AV_TIME_BASE_Q, av_make_q(1, 1000));
-const int64_t duration = sub.end_display_time;
-for (i = 0; i < sub.num_rects; i++) {
-char *ass_line = sub.rects[i]->ass;
+const int64_t start_time =
av_rescale_q(sub->subtitle_timing.start_pts, AV_TIME_BASE_Q, av_make_q(1,
1000));
+const int64_t duration =
av_rescale_q(sub->subtitle_timing.duration, AV_TIME_BASE_Q, av_make_q(1, 1000));
+for (i = 0; i < sub->num_subtitle_areas; i++) {
+char *ass_line = sub->subtitle_areas[i]->ass;
if (!ass_line)
-break;
+continue;
ass_process_chunk(ass->track, ass_line, strlen(ass_line),
start_time, duration);
}
}
}
av_packet_unref(&pkt);
-avsubtitle_free(&sub);
+av_frame_free(&sub);
}
end:
--
ffmpeg-codebot
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2 10/26] avfilter/avfilter: Fix hardcoded input index
From: softworkz
This fix targets (rare) cases where multiple input pads have a
.filter_frame function. ff_request_frame_to_filter needs
to call ff_request_frame with the correct input pad
instead of the hardcoded first one.
Signed-off-by: softworkz
---
libavfilter/avfilter.c | 18 +-
1 file changed, 13 insertions(+), 5 deletions(-)
diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
index 75d5e86539..aa9aa71f53 100644
--- a/libavfilter/avfilter.c
+++ b/libavfilter/avfilter.c
@@ -463,7 +463,7 @@ static int64_t guess_status_pts(AVFilterContext *ctx, int
status, AVRational lin
return AV_NOPTS_VALUE;
}
-static int ff_request_frame_to_filter(AVFilterLink *link)
+static int ff_request_frame_to_filter(AVFilterLink *link, int input_index)
{
int ret = -1;
@@ -472,8 +472,8 @@ static int ff_request_frame_to_filter(AVFilterLink *link)
link->frame_blocked_in = 1;
if (link->srcpad->request_frame)
ret = link->srcpad->request_frame(link);
-else if (link->src->inputs[0])
-ret = ff_request_frame(link->src->inputs[0]);
+else if (link->src->inputs[input_index])
+ret = ff_request_frame(link->src->inputs[input_index]);
if (ret < 0) {
if (ret != AVERROR(EAGAIN) && ret != link->status_in)
ff_avfilter_link_set_in_status(link, ret,
guess_status_pts(link->src, ret, link->time_base));
@@ -1172,6 +1172,14 @@ static int forward_status_change(AVFilterContext
*filter, AVFilterLink *in)
{
unsigned out = 0, progress = 0;
int ret;
+int input_index = 0;
+
+for (int i = 0; i < in->dst->nb_inputs; i++) {
+if (&in->dst->input_pads[i] == in->dstpad) {
+input_index = i;
+break;
+}
+}
av_assert0(!in->status_out);
if (!filter->nb_outputs) {
@@ -1181,7 +1189,7 @@ static int forward_status_change(AVFilterContext *filter,
AVFilterLink *in)
while (!in->status_out) {
if (!filter->outputs[out]->status_in) {
progress++;
-ret = ff_request_frame_to_filter(filter->outputs[out]);
+ret = ff_request_frame_to_filter(filter->outputs[out],
input_index);
if (ret < 0)
return ret;
}
@@ -1218,7 +1226,7 @@ static int ff_filter_activate_default(AVFilterContext
*filter)
for (i = 0; i < filter->nb_outputs; i++) {
if (filter->outputs[i]->frame_wanted_out &&
!filter->outputs[i]->frame_blocked_in) {
-return ff_request_frame_to_filter(filter->outputs[i]);
+return ff_request_frame_to_filter(filter->outputs[i], 0);
}
}
return FFERROR_NOT_READY;
--
ffmpeg-codebot
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2 06/26] avcodec/subtitles: Replace deprecated enum values
From: softworkz
Signed-off-by: softworkz
---
libavcodec/ass.h | 2 +-
libavcodec/assdec.c| 2 +-
libavcodec/dvbsubdec.c | 2 +-
libavcodec/dvdsubdec.c | 2 +-
libavcodec/dvdsubenc.c | 2 +-
libavcodec/pgssubdec.c | 2 +-
libavcodec/xsubdec.c | 2 +-
7 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/libavcodec/ass.h b/libavcodec/ass.h
index 8bc13d7ab8..43c5ad651a 100644
--- a/libavcodec/ass.h
+++ b/libavcodec/ass.h
@@ -83,7 +83,7 @@ static inline int avpriv_ass_add_rect(AVSubtitle *sub, const
char *dialog,
rects[sub->num_rects] = av_mallocz(sizeof(*rects[0]));
if (!rects[sub->num_rects])
return AVERROR(ENOMEM);
-rects[sub->num_rects]->type = SUBTITLE_ASS;
+rects[sub->num_rects]->type = AV_SUBTITLE_FMT_ASS;
ass_str = avpriv_ass_get_dialog(readorder, layer, style, speaker, dialog);
if (!ass_str)
return AVERROR(ENOMEM);
diff --git a/libavcodec/assdec.c b/libavcodec/assdec.c
index 7802a44e71..fd321e7004 100644
--- a/libavcodec/assdec.c
+++ b/libavcodec/assdec.c
@@ -54,7 +54,7 @@ static int ass_decode_frame(AVCodecContext *avctx, void
*data, int *got_sub_ptr,
if (!sub->rects[0])
return AVERROR(ENOMEM);
sub->num_rects = 1;
-sub->rects[0]->type = SUBTITLE_ASS;
+sub->rects[0]->type = AV_SUBTITLE_FMT_ASS;
sub->rects[0]->ass = av_strdup(avpkt->data);
if (!sub->rects[0]->ass)
return AVERROR(ENOMEM);
diff --git a/libavcodec/dvbsubdec.c b/libavcodec/dvbsubdec.c
index bc741a1de6..0d64c6e71c 100644
--- a/libavcodec/dvbsubdec.c
+++ b/libavcodec/dvbsubdec.c
@@ -795,7 +795,7 @@ static int save_subtitle_set(AVCodecContext *avctx,
AVSubtitle *sub, int *got_ou
rect->w = region->width;
rect->h = region->height;
rect->nb_colors = (1 << region->depth);
-rect->type = SUBTITLE_BITMAP;
+rect->type = AV_SUBTITLE_FMT_BITMAP;
rect->linesize[0] = region->width;
clut = get_clut(ctx, region->clut);
diff --git a/libavcodec/dvdsubdec.c b/libavcodec/dvdsubdec.c
index 52259f0730..b39b3d1838 100644
--- a/libavcodec/dvdsubdec.c
+++ b/libavcodec/dvdsubdec.c
@@ -406,7 +406,7 @@ static int decode_dvd_subtitles(DVDSubContext *ctx,
AVSubtitle *sub_header,
sub_header->rects[0]->y = y1;
sub_header->rects[0]->w = w;
sub_header->rects[0]->h = h;
-sub_header->rects[0]->type = SUBTITLE_BITMAP;
+sub_header->rects[0]->type = AV_SUBTITLE_FMT_BITMAP;
sub_header->rects[0]->linesize[0] = w;
sub_header->rects[0]->flags = is_menu ?
AV_SUBTITLE_FLAG_FORCED : 0;
}
diff --git a/libavcodec/dvdsubenc.c b/libavcodec/dvdsubenc.c
index ff4fbed39d..943a7466d9 100644
--- a/libavcodec/dvdsubenc.c
+++ b/libavcodec/dvdsubenc.c
@@ -268,7 +268,7 @@ static int encode_dvd_subtitles(AVCodecContext *avctx,
if (rects == 0 || !h->rects)
return AVERROR(EINVAL);
for (i = 0; i < rects; i++)
-if (h->rects[i]->type != SUBTITLE_BITMAP) {
+if (h->rects[i]->type != AV_SUBTITLE_FMT_BITMAP) {
av_log(avctx, AV_LOG_ERROR, "Bitmap subtitle required\n");
return AVERROR(EINVAL);
}
diff --git a/libavcodec/pgssubdec.c b/libavcodec/pgssubdec.c
index bdd20c914b..22b6616f9b 100644
--- a/libavcodec/pgssubdec.c
+++ b/libavcodec/pgssubdec.c
@@ -535,7 +535,7 @@ static int display_end_segment(AVCodecContext *avctx, void
*data,
if (!rect)
return AVERROR(ENOMEM);
sub->rects[sub->num_rects++] = rect;
-rect->type = SUBTITLE_BITMAP;
+rect->type = AV_SUBTITLE_FMT_BITMAP;
/* Process bitmap */
object = find_object(ctx->presentation.objects[i].id, &ctx->objects);
diff --git a/libavcodec/xsubdec.c b/libavcodec/xsubdec.c
index 85cd7d1c20..a4be18a1d8 100644
--- a/libavcodec/xsubdec.c
+++ b/libavcodec/xsubdec.c
@@ -107,7 +107,7 @@ static int decode_frame(AVCodecContext *avctx, void *data,
int *got_sub_ptr,
sub->num_rects = 1;
rect->x = x; rect->y = y;
rect->w = w; rect->h = h;
-rect->type = SUBTITLE_BITMAP;
+rect->type = AV_SUBTITLE_FMT_BITMAP;
rect->linesize[0] = w;
rect->data[0] = av_malloc(w * h);
rect->nb_colors = 4;
--
ffmpeg-codebot
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2 05/26] avcodec, avutil: Move ass helper functions to avutil as avpriv_ and extend ass dialog parsing
From: softworkz
Also add
- hard_space callback (for upcoming fix)
- extensible callback (for future extension)
Signed-off-by: softworkz
---
libavcodec/Makefile | 56 +++
libavcodec/ass.h | 144 ++
libavcodec/assdec.c | 2 +-
libavcodec/assenc.c | 2 +-
libavcodec/ccaption_dec.c | 19 +--
libavcodec/jacosubdec.c | 2 +-
libavcodec/libaribb24.c | 2 +-
libavcodec/libzvbi-teletextdec.c | 14 +-
libavcodec/microdvddec.c | 7 +-
libavcodec/movtextdec.c | 3 +-
libavcodec/movtextenc.c | 20 +--
libavcodec/mpl2dec.c | 2 +-
libavcodec/realtextdec.c | 2 +-
libavcodec/samidec.c | 2 +-
libavcodec/srtdec.c | 2 +-
libavcodec/srtenc.c | 16 +-
libavcodec/subviewerdec.c | 2 +-
libavcodec/textdec.c | 4 +-
libavcodec/ttmlenc.c | 15 +-
libavcodec/webvttdec.c| 2 +-
libavcodec/webvttenc.c| 16 +-
libavutil/Makefile| 2 +
{libavcodec => libavutil}/ass.c | 91 +--
libavutil/ass_internal.h | 135
{libavcodec => libavutil}/ass_split.c | 30 ++--
.../ass_split_internal.h | 32 ++--
26 files changed, 355 insertions(+), 269 deletions(-)
rename {libavcodec => libavutil}/ass.c (65%)
create mode 100644 libavutil/ass_internal.h
rename {libavcodec => libavutil}/ass_split.c (94%)
rename libavcodec/ass_split.h => libavutil/ass_split_internal.h (86%)
diff --git a/libavcodec/Makefile b/libavcodec/Makefile
index cfc70a3eaf..80bf8ff2d2 100644
--- a/libavcodec/Makefile
+++ b/libavcodec/Makefile
@@ -215,10 +215,10 @@ OBJS-$(CONFIG_APNG_DECODER)+= png.o pngdec.o
pngdsp.o
OBJS-$(CONFIG_APNG_ENCODER)+= png.o pngenc.o
OBJS-$(CONFIG_ARBC_DECODER)+= arbc.o
OBJS-$(CONFIG_ARGO_DECODER)+= argo.o
-OBJS-$(CONFIG_SSA_DECODER) += assdec.o ass.o
-OBJS-$(CONFIG_SSA_ENCODER) += assenc.o ass.o
-OBJS-$(CONFIG_ASS_DECODER) += assdec.o ass.o
-OBJS-$(CONFIG_ASS_ENCODER) += assenc.o ass.o
+OBJS-$(CONFIG_SSA_DECODER) += assdec.o
+OBJS-$(CONFIG_SSA_ENCODER) += assenc.o
+OBJS-$(CONFIG_ASS_DECODER) += assdec.o
+OBJS-$(CONFIG_ASS_ENCODER) += assenc.o
OBJS-$(CONFIG_ASV1_DECODER)+= asvdec.o asv.o mpeg12data.o
OBJS-$(CONFIG_ASV1_ENCODER)+= asvenc.o asv.o mpeg12data.o
OBJS-$(CONFIG_ASV2_DECODER)+= asvdec.o asv.o mpeg12data.o
@@ -259,7 +259,7 @@ OBJS-$(CONFIG_BRENDER_PIX_DECODER) += brenderpix.o
OBJS-$(CONFIG_C93_DECODER) += c93.o
OBJS-$(CONFIG_CAVS_DECODER)+= cavs.o cavsdec.o cavsdsp.o \
cavsdata.o
-OBJS-$(CONFIG_CCAPTION_DECODER)+= ccaption_dec.o ass.o
+OBJS-$(CONFIG_CCAPTION_DECODER)+= ccaption_dec.o
OBJS-$(CONFIG_CDGRAPHICS_DECODER) += cdgraphics.o
OBJS-$(CONFIG_CDTOONS_DECODER) += cdtoons.o
OBJS-$(CONFIG_CDXL_DECODER)+= cdxl.o
@@ -434,7 +434,7 @@ OBJS-$(CONFIG_INTERPLAY_ACM_DECODER) += interplayacm.o
OBJS-$(CONFIG_INTERPLAY_DPCM_DECODER) += dpcm.o
OBJS-$(CONFIG_INTERPLAY_VIDEO_DECODER) += interplayvideo.o
OBJS-$(CONFIG_IPU_DECODER) += mpeg12dec.o mpeg12.o mpeg12data.o
-OBJS-$(CONFIG_JACOSUB_DECODER) += jacosubdec.o ass.o
+OBJS-$(CONFIG_JACOSUB_DECODER) += jacosubdec.o
OBJS-$(CONFIG_JPEG2000_ENCODER)+= j2kenc.o mqcenc.o mqc.o jpeg2000.o \
jpeg2000dwt.o
OBJS-$(CONFIG_JPEG2000_DECODER)+= jpeg2000dec.o jpeg2000.o
jpeg2000dsp.o \
@@ -456,7 +456,7 @@ OBJS-$(CONFIG_MAGICYUV_ENCODER)+= magicyuvenc.o
OBJS-$(CONFIG_MDEC_DECODER)+= mdec.o mpeg12.o mpeg12data.o
OBJS-$(CONFIG_METASOUND_DECODER) += metasound.o metasound_data.o \
twinvq.o
-OBJS-$(CONFIG_MICRODVD_DECODER)+= microdvddec.o ass.o
+OBJS-$(CONFIG_MICRODVD_DECODER)+= microdvddec.o
OBJS-$(CONFIG_MIMIC_DECODER) += mimic.o
OBJS-$(CONFIG_MJPEG_DECODER) += mjpegdec.o mjpegdec_common.o
OBJS-$(CONFIG_MJPEG_QSV_DECODER) += qsvdec.o
@@ -471,8 +471,8 @@ OBJS-$(CONFIG_MLP_ENCODER) += mlpenc.o mlp.o
OBJS-$(CONFIG_MMVIDEO_DECODER) += mmvideo.o
OBJS-$(CONFIG_MOBICLIP_DECODER)+= mobiclip.o
OBJS-$(CONFIG_MOTIONPIXELS_DECODER)+= motionpixels.o
-OBJS-$(CONFIG_MOVTEXT_DECODER) += movtextdec.o ass.o
-OBJS-$(CONFIG_MOVTEXT_ENCOD
[FFmpeg-devel] [PATCH v2 11/26] avfilter/sbuffer: Add sbuffersrc and sbuffersink filters
From: softworkz
Signed-off-by: softworkz
---
configure| 2 +-
libavfilter/allfilters.c | 2 ++
libavfilter/buffersink.c | 54 ++
libavfilter/buffersink.h | 7
libavfilter/buffersrc.c | 72
libavfilter/buffersrc.h | 1 +
6 files changed, 137 insertions(+), 1 deletion(-)
diff --git a/configure b/configure
index 1413122d87..7c2931bd50 100755
--- a/configure
+++ b/configure
@@ -7854,7 +7854,7 @@ print_enabled_components(){
fi
done
if [ "$name" = "filter_list" ]; then
-for c in asrc_abuffer vsrc_buffer asink_abuffer vsink_buffer; do
+for c in asrc_abuffer vsrc_buffer ssrc_sbuffer asink_abuffer
vsink_buffer ssink_sbuffer; do
printf "&ff_%s,\n" $c >> $TMPH
done
fi
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 4325a3e557..8a3bd03924 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -556,8 +556,10 @@ extern const AVFilter ff_avsrc_movie;
* being the same while having different 'types'). */
extern const AVFilter ff_asrc_abuffer;
extern const AVFilter ff_vsrc_buffer;
+extern const AVFilter ff_ssrc_sbuffer;
extern const AVFilter ff_asink_abuffer;
extern const AVFilter ff_vsink_buffer;
+extern const AVFilter ff_ssink_sbuffer;
extern const AVFilter ff_af_afifo;
extern const AVFilter ff_vf_fifo;
diff --git a/libavfilter/buffersink.c b/libavfilter/buffersink.c
index c0215669e7..0b268c2fa4 100644
--- a/libavfilter/buffersink.c
+++ b/libavfilter/buffersink.c
@@ -29,6 +29,8 @@
#include "libavutil/internal.h"
#include "libavutil/opt.h"
+#include "libavcodec/avcodec.h"
+
#define FF_INTERNAL_FIELDS 1
#include "framequeue.h"
@@ -57,6 +59,10 @@ typedef struct BufferSinkContext {
int *sample_rates; ///< list of accepted sample rates
int sample_rates_size;
+/* only used for subtitles */
+enum AVSubtitleType *subtitle_types; ///< list of accepted subtitle
types, must be terminated with -1
+int subtitle_types_size;
+
AVFrame *peeked_frame;
} BufferSinkContext;
@@ -305,6 +311,28 @@ static int asink_query_formats(AVFilterContext *ctx)
return 0;
}
+static int ssink_query_formats(AVFilterContext *ctx)
+{
+BufferSinkContext *buf = ctx->priv;
+AVFilterFormats *formats = NULL;
+unsigned i;
+int ret;
+
+CHECK_LIST_SIZE(subtitle_types)
+if (buf->subtitle_types_size) {
+for (i = 0; i < NB_ITEMS(buf->subtitle_types); i++)
+if ((ret = ff_add_subtitle_type(&formats, buf->subtitle_types[i]))
< 0)
+return ret;
+if ((ret = ff_set_common_formats(ctx, formats)) < 0)
+return ret;
+} else {
+if ((ret = ff_default_query_formats(ctx)) < 0)
+return ret;
+}
+
+return 0;
+}
+
#define OFFSET(x) offsetof(BufferSinkContext, x)
#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM
static const AVOption buffersink_options[] = {
@@ -322,9 +350,16 @@ static const AVOption abuffersink_options[] = {
{ NULL },
};
#undef FLAGS
+#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_SUBTITLE_PARAM
+static const AVOption sbuffersink_options[] = {
+{ "subtitle_types", "set the supported subtitle formats",
OFFSET(subtitle_types), AV_OPT_TYPE_BINARY, .flags = FLAGS },
+{ NULL },
+};
+#undef FLAGS
AVFILTER_DEFINE_CLASS(buffersink);
AVFILTER_DEFINE_CLASS(abuffersink);
+AVFILTER_DEFINE_CLASS(sbuffersink);
static const AVFilterPad avfilter_vsink_buffer_inputs[] = {
{
@@ -363,3 +398,22 @@ const AVFilter ff_asink_abuffer = {
.outputs = NULL,
FILTER_QUERY_FUNC(asink_query_formats),
};
+
+static const AVFilterPad avfilter_ssink_sbuffer_inputs[] = {
+{
+.name = "default",
+.type = AVMEDIA_TYPE_SUBTITLE,
+},
+};
+
+const AVFilter ff_ssink_sbuffer = {
+.name = "sbuffersink",
+.description = NULL_IF_CONFIG_SMALL("Buffer subtitle frames, and make
them available to the end of the filter graph."),
+.priv_class= &sbuffersink_class,
+.priv_size = sizeof(BufferSinkContext),
+.init = common_init,
+.activate = activate,
+FILTER_INPUTS(avfilter_ssink_sbuffer_inputs),
+.outputs = NULL,
+FILTER_QUERY_FUNC(ssink_query_formats),
+};
diff --git a/libavfilter/buffersink.h b/libavfilter/buffersink.h
index 69ed0f29a8..11905abdc5 100644
--- a/libavfilter/buffersink.h
+++ b/libavfilter/buffersink.h
@@ -129,6 +129,13 @@ typedef struct AVABufferSinkParams {
*/
attribute_deprecated
AVABufferSinkParams *av_abuffersink_params_alloc(void);
+
+/**
+ * Deprecated and unused struct to use for initializing an sbuffersink context.
+ */
+typedef struct AVSBufferSinkParams {
+const int *subtitle_type;
+} AVSBufferSinkParams;
#endif
/**
diff --git a/libavfilter/buffersrc.c b/libavfilter/buffersrc.c
index b0611872f1..d2362999a2 100644
--- a/liba
[FFmpeg-devel] [PATCH v2 07/26] fftools/play, probe: Adjust for subtitle changes
From: softworkz
Signed-off-by: softworkz
---
fftools/ffplay.c | 102 +-
fftools/ffprobe.c | 47 +
2 files changed, 77 insertions(+), 72 deletions(-)
diff --git a/fftools/ffplay.c b/fftools/ffplay.c
index e7b20be76b..94286eb678 100644
--- a/fftools/ffplay.c
+++ b/fftools/ffplay.c
@@ -152,7 +152,6 @@ typedef struct Clock {
/* Common struct for handling all types of decoded data and allocated render
buffers. */
typedef struct Frame {
AVFrame *frame;
-AVSubtitle sub;
int serial;
double pts; /* presentation timestamp for the frame */
double duration; /* estimated duration of the frame */
@@ -586,7 +585,7 @@ static int decoder_init(Decoder *d, AVCodecContext *avctx,
PacketQueue *queue, S
return 0;
}
-static int decoder_decode_frame(Decoder *d, AVFrame *frame, AVSubtitle *sub) {
+static int decoder_decode_frame(Decoder *d, AVFrame *frame) {
int ret = AVERROR(EAGAIN);
for (;;) {
@@ -620,6 +619,9 @@ static int decoder_decode_frame(Decoder *d, AVFrame *frame,
AVSubtitle *sub) {
}
}
break;
+case AVMEDIA_TYPE_SUBTITLE:
+ret = avcodec_receive_frame(d->avctx, frame);
+break;
}
if (ret == AVERROR_EOF) {
d->finished = d->pkt_serial;
@@ -652,25 +654,11 @@ static int decoder_decode_frame(Decoder *d, AVFrame
*frame, AVSubtitle *sub) {
av_packet_unref(d->pkt);
} while (1);
-if (d->avctx->codec_type == AVMEDIA_TYPE_SUBTITLE) {
-int got_frame = 0;
-ret = avcodec_decode_subtitle2(d->avctx, sub, &got_frame, d->pkt);
-if (ret < 0) {
-ret = AVERROR(EAGAIN);
-} else {
-if (got_frame && !d->pkt->data) {
-d->packet_pending = 1;
-}
-ret = got_frame ? 0 : (d->pkt->data ? AVERROR(EAGAIN) :
AVERROR_EOF);
-}
-av_packet_unref(d->pkt);
+if (avcodec_send_packet(d->avctx, d->pkt) == AVERROR(EAGAIN)) {
+av_log(d->avctx, AV_LOG_ERROR, "Receive_frame and send_packet both
returned EAGAIN, which is an API violation.\n");
+d->packet_pending = 1;
} else {
-if (avcodec_send_packet(d->avctx, d->pkt) == AVERROR(EAGAIN)) {
-av_log(d->avctx, AV_LOG_ERROR, "Receive_frame and send_packet
both returned EAGAIN, which is an API violation.\n");
-d->packet_pending = 1;
-} else {
-av_packet_unref(d->pkt);
-}
+av_packet_unref(d->pkt);
}
}
}
@@ -683,7 +671,6 @@ static void decoder_destroy(Decoder *d) {
static void frame_queue_unref_item(Frame *vp)
{
av_frame_unref(vp->frame);
-avsubtitle_free(&vp->sub);
}
static int frame_queue_init(FrameQueue *f, PacketQueue *pktq, int max_size,
int keep_last)
@@ -981,7 +968,7 @@ static void video_image_display(VideoState *is)
if (frame_queue_nb_remaining(&is->subpq) > 0) {
sp = frame_queue_peek(&is->subpq);
-if (vp->pts >= sp->pts + ((float) sp->sub.start_display_time /
1000)) {
+if (vp->pts >= sp->pts) {
if (!sp->uploaded) {
uint8_t* pixels[4];
int pitch[4];
@@ -993,25 +980,27 @@ static void video_image_display(VideoState *is)
if (realloc_texture(&is->sub_texture,
SDL_PIXELFORMAT_ARGB, sp->width, sp->height, SDL_BLENDMODE_BLEND, 1) < 0)
return;
-for (i = 0; i < sp->sub.num_rects; i++) {
-AVSubtitleRect *sub_rect = sp->sub.rects[i];
+for (i = 0; i < sp->frame->num_subtitle_areas; i++) {
+AVSubtitleArea *area = sp->frame->subtitle_areas[i];
+SDL_Rect sdl_rect = { .x = area->x, .y = area->y, .w =
area->w, .h = area->h };
-sub_rect->x = av_clip(sub_rect->x, 0, sp->width );
-sub_rect->y = av_clip(sub_rect->y, 0, sp->height);
-sub_rect->w = av_clip(sub_rect->w, 0, sp->width -
sub_rect->x);
-sub_rect->h = av_clip(sub_rect->h, 0, sp->height -
sub_rect->y);
+area->x = av_clip(area->x, 0, sp->width );
+area->y = av_clip(area->y, 0, sp->height);
+area->w = av_clip(area->w, 0, sp->width - area->x);
+area->h = av_clip(area->h, 0, sp->height - area->y);
is->sub_convert_ctx =
sws_getCachedContext(is->sub_convert_ctx,
-sub_rect->w, sub_rect->h, AV_PIX_FMT_PAL8,
-sub_rect->w, sub_rect->h, AV_PIX_FMT_BGRA,
+
[FFmpeg-devel] [PATCH v2 12/26] avfilter/overlaygraphicsubs: Add overlaygraphicsubs and graphicsub2video filters
From: softworkz
- overlaygraphicsubs (VS -> V)
Overlay graphic subtitles onto a video stream
- graphicsub2video {S -> V)
Converts graphic subtitles to video frames (with alpha)
Gets auto-inserted for retaining compatibility with
sub2video command lines
Signed-off-by: softworkz
---
doc/filters.texi| 118 +
libavfilter/Makefile| 2 +
libavfilter/allfilters.c| 2 +
libavfilter/vf_overlaygraphicsubs.c | 765
4 files changed, 887 insertions(+)
create mode 100644 libavfilter/vf_overlaygraphicsubs.c
diff --git a/doc/filters.texi b/doc/filters.texi
index 05d4b1a56e..4fc4a57dbb 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25713,6 +25713,124 @@ tools.
@c man end VIDEO SINKS
+@chapter Subtitle Filters
+@c man begin SUBTITLE FILTERS
+
+When you configure your FFmpeg build, you can disable any of the
+existing filters using @code{--disable-filters}.
+
+Below is a description of the currently available subtitle filters.
+
+@section graphicsub2video
+
+Renders graphic subtitles as video frames.
+
+This filter replaces the previous "sub2video" hack which did the conversion
implicitly and up-front as subtitle filtering wasn't possible at that time.
+To retain compatibility with earlier sub2video command lines, this filter is
being auto-inserted in those cases.
+
+For overlaying graphicsal subtitles it is recommended to use the
'overlay_graphicsubs' filter which is more efficient and takes less processing
resources.
+
+This filter is still useful in cases where the overlay is done with hardware
acceleration (e.g. overlay_qsv, overlay_vaapi, overlay_cuda) for preparing the
overlay frames.
+
+Inputs:
+@itemize
+@item 0: Subtitles [BITMAP]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video [RGB32]
+@end itemize
+
+
+It accepts the following parameters:
+
+@table @option
+@item size, s
+Set the size of the output video frame.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Overlay PGS subtitles
+(not recommended - better use overlay_graphicsubs)
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:1]graphicsub2video[subs];[0:0][subs]overlay" output.mp4
+@end example
+
+@item
+Overlay PGS subtitles implicitly
+The graphicsub2video is inserted automatically for compatibility with legacy
command lines.
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:0][0:1]overlay" output.mp4
+@end example
+@end itemize
+
+@section overlaygraphicsubs
+
+Overlay graphic subtitles onto a video stream.
+
+This filter can blend graphical subtitles on a video stream directly, i.e.
without creating full-size alpha images first.
+The blending operation is limited to the area of the subtitle rectangles,
which also means that no processing is done at times where no subtitles are to
be displayed.
+
+Inputs:
+@itemize
+@item 0: Video [YUV420P, YUV422P, YUV444P, ARGB, RGBA, ABGR, BGRA, RGB24,
BGR24]
+@item 1: Subtitles [BITMAP]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video (same as input)
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+@item x
+@item y
+Set the expression for the x and y coordinates of the overlaid video
+on the main video. Default value is "0" for both expressions. In case
+the expression is invalid, it is set to a huge value (meaning that the
+overlay will not be displayed within the output visible area).
+
+@item eof_action
+See @ref{framesync}.
+
+@item eval
+Set when the expressions for @option{x}, and @option{y} are evaluated.
+
+It accepts the following values:
+@table @samp
+@item init
+only evaluate expressions once during the filter initialization or
+when a command is processed
+
+@item frame
+evaluate expressions for each incoming frame
+@end table
+
+Default value is @samp{frame}.
+
+@item shortest
+See @ref{framesync}.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Overlay PGS subtitles
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:0][0:1]overlaygraphicsubs" output.mp4
+@end example
+@end itemize
+@c man end SUBTITLE FILTERS
+
@chapter Multimedia Filters
@c man begin MULTIMEDIA FILTERS
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 283dd436cd..a372effc12 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -298,6 +298,7 @@ OBJS-$(CONFIG_GBLUR_FILTER) += vf_gblur.o
OBJS-$(CONFIG_GBLUR_VULKAN_FILTER) += vf_gblur_vulkan.o vulkan.o
vulkan_filter.o
OBJS-$(CONFIG_GEQ_FILTER)+= vf_geq.o
OBJS-$(CONFIG_GRADFUN_FILTER)+= vf_gradfun.o
+OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlaygraphicsubs.o
framesync.o
OBJS-$(CONFIG_GRAPHMONITOR_FILTER) += f_graphmonitor.o
OBJS-$(CONFIG_GRAYWORLD_FIL
[FFmpeg-devel] [PATCH v2 08/26] avfilter/subtitles: Add subtitles.c for subtitle frame allocation
From: softworkz
Analog to avfilter/video.c and avfilter/audio.c
Signed-off-by: softworkz
---
libavfilter/Makefile| 1 +
libavfilter/avfilter.c | 4 +++
libavfilter/internal.h | 1 +
libavfilter/subtitles.c | 63 +
libavfilter/subtitles.h | 44
5 files changed, 113 insertions(+)
create mode 100644 libavfilter/subtitles.c
create mode 100644 libavfilter/subtitles.h
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 1adbea75bd..283dd436cd 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -19,6 +19,7 @@ OBJS = allfilters.o
\
framequeue.o \
graphdump.o \
graphparser.o\
+ subtitles.o \
video.o \
OBJS-$(HAVE_THREADS) += pthread.o
diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
index 7362bcdab5..df5b8f483c 100644
--- a/libavfilter/avfilter.c
+++ b/libavfilter/avfilter.c
@@ -43,6 +43,7 @@
#include "formats.h"
#include "framepool.h"
#include "internal.h"
+#include "subtitles.h"
#include "libavutil/ffversion.h"
const char av_filter_ffversion[] = "FFmpeg version " FFMPEG_VERSION;
@@ -1475,6 +1476,9 @@ int ff_inlink_make_frame_writable(AVFilterLink *link,
AVFrame **rframe)
case AVMEDIA_TYPE_AUDIO:
out = ff_get_audio_buffer(link, frame->nb_samples);
break;
+case AVMEDIA_TYPE_SUBTITLE:
+out = ff_get_subtitles_buffer(link, link->format);
+break;
default:
return AVERROR(EINVAL);
}
diff --git a/libavfilter/internal.h b/libavfilter/internal.h
index 1099b82b4b..fc09ef574c 100644
--- a/libavfilter/internal.h
+++ b/libavfilter/internal.h
@@ -90,6 +90,7 @@ struct AVFilterPad {
union {
AVFrame *(*video)(AVFilterLink *link, int w, int h);
AVFrame *(*audio)(AVFilterLink *link, int nb_samples);
+AVFrame *(*subtitle)(AVFilterLink *link, int format);
} get_buffer;
/**
diff --git a/libavfilter/subtitles.c b/libavfilter/subtitles.c
new file mode 100644
index 00..951bfd612c
--- /dev/null
+++ b/libavfilter/subtitles.c
@@ -0,0 +1,63 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "libavutil/common.h"
+
+#include "subtitles.h"
+#include "avfilter.h"
+#include "internal.h"
+
+
+AVFrame *ff_null_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+return ff_get_subtitles_buffer(link->dst->outputs[0], format);
+}
+
+AVFrame *ff_default_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+AVFrame *frame;
+
+frame = av_frame_alloc();
+if (!frame)
+return NULL;
+
+frame->format = format;
+frame->type = AVMEDIA_TYPE_SUBTITLE;
+
+if (av_frame_get_buffer2(frame, 0) < 0) {
+av_frame_free(&frame);
+return NULL;
+}
+
+return frame;
+}
+
+AVFrame *ff_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+AVFrame *ret = NULL;
+
+if (link->dstpad->get_buffer.subtitle)
+ret = link->dstpad->get_buffer.subtitle(link, format);
+
+if (!ret)
+ret = ff_default_get_subtitles_buffer(link, format);
+
+return ret;
+}
diff --git a/libavfilter/subtitles.h b/libavfilter/subtitles.h
new file mode 100644
index 00..4a9115126e
--- /dev/null
+++ b/libavfilter/subtitles.h
@@ -0,0 +1,44 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You
[FFmpeg-devel] [PATCH v2 13/26] avfilter/overlaytextsubs: Add overlaytextsubs and textsubs2video filters
From: softworkz
- overlaytextsubs {VS -> V)
Overlay text subtitles onto a video stream.
- textsubs2video {S -> V)
Converts text subtitles to video frames
Signed-off-by: softworkz
---
configure| 2 +
doc/filters.texi | 113 ++
libavfilter/Makefile | 2 +
libavfilter/allfilters.c | 4 +-
libavfilter/vf_overlaytextsubs.c | 678 +++
5 files changed, 798 insertions(+), 1 deletion(-)
create mode 100644 libavfilter/vf_overlaytextsubs.c
diff --git a/configure b/configure
index 7c2931bd50..340a198fa6 100755
--- a/configure
+++ b/configure
@@ -3691,6 +3691,7 @@ overlay_opencl_filter_deps="opencl"
overlay_qsv_filter_deps="libmfx"
overlay_qsv_filter_select="qsvvpp"
overlay_vulkan_filter_deps="vulkan spirv_compiler"
+overlaytextsubs_filter_deps="avcodec libass"
owdenoise_filter_deps="gpl"
pad_opencl_filter_deps="opencl"
pan_filter_deps="swresample"
@@ -3735,6 +3736,7 @@ superequalizer_filter_deps="avcodec"
superequalizer_filter_select="rdft"
surround_filter_deps="avcodec"
surround_filter_select="rdft"
+textsub2video_filter_deps="avcodec libass"
tinterlace_filter_deps="gpl"
tinterlace_merge_test_deps="tinterlace_filter"
tinterlace_pad_test_deps="tinterlace_filter"
diff --git a/doc/filters.texi b/doc/filters.texi
index 4fc4a57dbb..51771f88e4 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25829,6 +25829,119 @@ Overlay PGS subtitles
ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:0][0:1]overlaygraphicsubs" output.mp4
@end example
@end itemize
+
+@section overlaytextsubs
+
+Overlay text subtitles onto a video stream.
+
+This filter supersedes the classic @ref{subtitles} filter opposed to which it
does no longer require to open and access the source stream separately, which
is often causing problems or doesn't even work for non-local or slow sources.
+
+Inputs:
+@itemize
+@item 0: Video [YUV420P, YUV422P, YUV444P, ARGB, RGBA, ABGR, BGRA, RGB24,
BGR24]
+@item 1: Subtitles [TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video (same as input)
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+
+@item alpha
+Process alpha channel, by default alpha channel is untouched.
+
+@item fonts_dir
+Set a directory path containing fonts that can be used by the filter.
+These fonts will be used in addition to whatever the font provider uses.
+
+@item default_font_path
+Path to a font file to be used as the default font.
+
+@item font_size
+Set the default font size.
+
+@item fontconfig_file
+Path to ASS fontconfig configuration file.
+
+@item force_style
+Override default style or script info parameters of the subtitles. It accepts a
+string containing ASS style format @code{KEY=VALUE} couples separated by ",".
+
+@item margin
+Set the rendering margin in pixels.
+
+@item render_latest_only
+For rendering, alway use the latest event only, which is covering the given
point in time
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Overlay ASS subtitles with animations:
+@example
+ffmpeg -i
"http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv";
-filter_complex "[0:v]overlaytextsubs" -map 0 -y out.mkv
+@end example
+@end itemize
+
+@section textsub2video
+
+Converts text subtitles to video frames.
+
+For overlaying text subtitles onto video frames it is recommended to use the
overlay_textsubs filter.
+The textsub2video is useful for for creating transparent text-frames when
overlay is done via hw acceleration
+
+Inputs:
+@itemize
+@item 0: Subtitles [TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video [RGB32]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+
+@item rate, r
+Set the framerate for updating overlay frames.
+Normally, overlay frames will only be updated each time when the subtitles to
display are changing.
+In cases where subtitles include advanced features (like animation), this
parameter determines the frequency by which the overlay frames should be
updated.
+
+@item size, s
+Set the output frame size.
+Allows to override the size of output video frames.
+
+@item fonts_dir
+Set a directory path containing fonts that can be used by the filter.
+These fonts will be used in addition to whatever the font provider uses.
+
+@item default_font_path
+Path to a font file to be used as the default font.
+
+@item font_size
+Set the default font size.
+
+@item fontconfig_file
+Path to ASS fontconfig configuration file.
+
+@item force_style
+Override default style or script info parameters of the subtitles. It accepts a
+string containing ASS style format @code{KEY=VALUE} couples separated by ",".
+
+@item margin
+Set the rendering margin in pixels.
+
+@item render_latest_only
+For rendering, alway use the latest event only, which is covering the given
point in time.
+@end table
+
@c man end SUBTITLE FILTERS
@chapter Multimedia Filters
d
[FFmpeg-devel] [PATCH v2 09/26] avfilter/avfilter: Handle subtitle frames
From: softworkz
Signed-off-by: softworkz
---
libavfilter/avfilter.c | 8 +---
libavfilter/avfilter.h | 11 +++
libavfilter/avfiltergraph.c | 5 +
libavfilter/formats.c | 22 ++
libavfilter/formats.h | 3 +++
libavfilter/internal.h | 18 +++---
6 files changed, 61 insertions(+), 6 deletions(-)
diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
index df5b8f483c..75d5e86539 100644
--- a/libavfilter/avfilter.c
+++ b/libavfilter/avfilter.c
@@ -56,7 +56,8 @@ static void tlog_ref(void *ctx, AVFrame *ref, int end)
ref->linesize[0], ref->linesize[1], ref->linesize[2],
ref->linesize[3],
ref->pts, ref->pkt_pos);
-if (ref->width) {
+switch(ref->type) {
+case AVMEDIA_TYPE_VIDEO:
ff_tlog(ctx, " a:%d/%d s:%dx%d i:%c iskey:%d type:%c",
ref->sample_aspect_ratio.num, ref->sample_aspect_ratio.den,
ref->width, ref->height,
@@ -64,12 +65,13 @@ static void tlog_ref(void *ctx, AVFrame *ref, int end)
ref->top_field_first ? 'T' : 'B',/* Top / Bottom */
ref->key_frame,
av_get_picture_type_char(ref->pict_type));
-}
-if (ref->nb_samples) {
+break;
+case AVMEDIA_TYPE_AUDIO:
ff_tlog(ctx, " cl:%"PRId64"d n:%d r:%d",
ref->channel_layout,
ref->nb_samples,
ref->sample_rate);
+break;
}
ff_tlog(ctx, "]%s", end ? "\n" : "");
diff --git a/libavfilter/avfilter.h b/libavfilter/avfilter.h
index b105dc3159..9f917deb41 100644
--- a/libavfilter/avfilter.h
+++ b/libavfilter/avfilter.h
@@ -45,6 +45,7 @@
#include "libavutil/log.h"
#include "libavutil/samplefmt.h"
#include "libavutil/pixfmt.h"
+#include "libavutil/subfmt.h"
#include "libavutil/rational.h"
#include "libavfilter/version.h"
@@ -343,6 +344,12 @@ typedef struct AVFilter {
* and outputs use the same sample rate and channel count/layout.
*/
const enum AVSampleFormat *samples_list;
+/**
+ * Analogous to pixels, but delimited by AV_SUBTITLE_FMT_NONE
+ * and restricted to filters that only have AVMEDIA_TYPE_SUBTITLE
+ * inputs and outputs.
+ */
+const enum AVSubtitleType *subs_list;
/**
* Equivalent to { pix_fmt, AV_PIX_FMT_NONE } as pixels_list.
*/
@@ -351,6 +358,10 @@ typedef struct AVFilter {
* Equivalent to { sample_fmt, AV_SAMPLE_FMT_NONE } as samples_list.
*/
enum AVSampleFormat sample_fmt;
+/**
+ * Equivalent to { sub_fmt, AV_SUBTITLE_FMT_NONE } as subs_list.
+ */
+enum AVSubtitleType sub_fmt;
} formats;
int priv_size; ///< size of private data to allocate for the filter
diff --git a/libavfilter/avfiltergraph.c b/libavfilter/avfiltergraph.c
index b8b432e98b..f4987654af 100644
--- a/libavfilter/avfiltergraph.c
+++ b/libavfilter/avfiltergraph.c
@@ -311,6 +311,8 @@ static int filter_link_check_formats(void *log,
AVFilterLink *link, AVFilterForm
return ret;
break;
+case AVMEDIA_TYPE_SUBTITLE:
+return 0;
default:
av_assert0(!"reached");
}
@@ -441,6 +443,9 @@ static int query_formats(AVFilterGraph *graph, void
*log_ctx)
if (!link)
continue;
+if (link->type == AVMEDIA_TYPE_SUBTITLE)
+continue;
+
neg = ff_filter_get_negotiation(link);
av_assert0(neg);
for (neg_step = 1; neg_step < neg->nb_mergers; neg_step++) {
diff --git a/libavfilter/formats.c b/libavfilter/formats.c
index ba62f73248..5c972bb183 100644
--- a/libavfilter/formats.c
+++ b/libavfilter/formats.c
@@ -19,6 +19,7 @@
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
+#include "libavutil/subfmt.h"
#include "libavutil/avassert.h"
#include "libavutil/channel_layout.h"
#include "libavutil/common.h"
@@ -431,6 +432,12 @@ int ff_add_channel_layout(AVFilterChannelLayouts **l,
uint64_t channel_layout)
return 0;
}
+int ff_add_subtitle_type(AVFilterFormats **avff, int64_t fmt)
+{
+ADD_FORMAT(avff, fmt, ff_formats_unref, int, formats, nb_formats);
+return 0;
+}
+
AVFilterFormats *ff_make_formats_list_singleton(int fmt)
{
int fmts[2] = { fmt, -1 };
@@ -450,6 +457,13 @@ AVFilterFormats *ff_all_formats(enum AVMediaType type)
return NULL;
fmt++;
}
+} else if (type == AVMEDIA_TYPE_SUBTITLE) {
+if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_BITMAP) < 0)
+return NULL;
+if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_ASS) < 0)
+return NULL;
+if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_TEXT) < 0)
+return NULL;
}
return ret;
@@ -724,6 +738,10 @@ int ff_default_query_formats(AVFilterContext *ctx)
type=
[FFmpeg-devel] [PATCH v2 14/26] avfilter/textmod: Add textmod, censor and show_speaker filters
From: softworkz
- textmod {S -> S)
Modify subtitle text in a number of ways
- censor {S -> S)
Censor subtitles using a word list
- show_speaker {S -> S)
Prepend speaker names from ASS subtitles to the visible text lines
Signed-off-by: softworkz
---
doc/filters.texi | 206
libavfilter/Makefile | 5 +
libavfilter/allfilters.c | 3 +
libavfilter/sf_textmod.c | 710 +++
4 files changed, 924 insertions(+)
create mode 100644 libavfilter/sf_textmod.c
diff --git a/doc/filters.texi b/doc/filters.texi
index 51771f88e4..494ee6f062 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25721,6 +25721,145 @@ existing filters using @code{--disable-filters}.
Below is a description of the currently available subtitle filters.
+
+@section censor
+
+Censor selected words in text subtitles.
+
+Inputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+@item mode
+The censoring mode to apply.
+
+Supported censoring modes are:
+
+@table @var
+@item 0, keep_first_last
+Replace all characters with the 'censor_char' except the first and the last
character of a word.
+For words with less than 4 characters, the last character will be replaced as
well.
+For words with less than 3 characters, the first character will be replaced as
well.
+@item 1, keep_first
+Replace all characters with the 'censor_char' except the first character of a
word.
+For words with less than 3 characters, the first character will be replaced as
well.
+@item 2, all
+Replace all characters with the 'censor_char'.
+@end table
+
+@item words
+A list of words to censor, separated by 'separator'.
+
+@item words_file
+Specify a file from which to load the contents for the 'words' parameter.
+
+@item censor_char
+Single character used as replacement for censoring.
+
+@item separator
+Delimiter character for words. Used with replace_words and remove_words- Must
be a single character.
+The default is '.'.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Censor a few given words with a pound character.
+@example
+ffmpeg -i
"http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv";
-filter_complex
"[0:1]censor=words='diss,louder,hope,beam,word':censor_char='#'" -map 0 -y
output.mkv
+@end example
+@end itemize
+
+
+@section textmod
+
+Modify subtitle text in a number of ways.
+
+Inputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+@item mode
+The kind of text modification to apply
+
+Supported operation modes are:
+
+@table @var
+@item 0, leet
+Convert subtitle text to 'leet speak'. It's primarily useful for testing as
the modification will be visible with almost all text lines.
+@item 1, to_upper
+Change all text to upper case. Might improve readability.
+@item 2, to_lower
+Change all text to lower case.
+@item 3, replace_chars
+Replace one or more characters. Requires the find and replace parameters to be
specified.
+Both need to be equal in length.
+The first char in find is replaced by the first char in replace, same for all
subsequent chars.
+@item 4, remove_chars
+Remove certain characters. Requires the find parameter to be specified.
+All chars in the find parameter string will be removed from all subtitle text.
+@item 5, replace_words
+Replace one or more words. Requires the find and replace parameters to be
specified. Multiple words must be separated by the delimiter char specified vie
the separator parameter (default: ',').
+The number of words in the find and replace parameters needs to be equal.
+The first word in find is replaced by the first word in replace, same for all
subsequent words
+@item 6, remove_words
+Remove certain words. Requires the find parameter to be specified. Multiple
words must be separated by the delimiter char specified vie the separator
parameter (default: ',').
+All words in the find parameter string will be removed from all subtitle text.
+@end table
+
+@item find
+Required for replace_chars, remove_chars, replace_words and remove_words.
+
+@item find_file
+Specify a file from which to load the contents for the 'find' parameter.
+
+@item replace
+Required for replace_chars and replace_words.
+
+@item replace_file
+Specify a file from which to load the contents for the 'replace' parameter.
+
+@item separator
+Delimiter character for words. Used with replace_words and remove_words- Must
be a single character.
+The default is '.'.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Change all characters to upper case while keeping all styles and animations:
+@example
+ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv";
-filter_complex "[0:s]textmod=mode=to_upper" -map 0 -y out.mkv
+@end example
+@item
+Remove a set of symbol characters for am i
[FFmpeg-devel] [PATCH v2 15/26] avfilter/stripstyles: Add stripstyles filter
From: softworkz
- stripstyles {S -> S)
Remove all inline styles from subtitle events
Signed-off-by: softworkz
---
doc/filters.texi | 37 ++
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/sf_stripstyles.c | 211 +++
4 files changed, 250 insertions(+)
create mode 100644 libavfilter/sf_stripstyles.c
diff --git a/doc/filters.texi b/doc/filters.texi
index 494ee6f062..c0f0fe13e7 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25781,6 +25781,43 @@ ffmpeg -i
"http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.
@end example
@end itemize
+@section stripstyles
+
+Remove all inline styles from subtitle events.
+
+Inputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+@item remove_animated
+Also remove text which is subject to animation (default: true)
+Usually, animated text elements are used used in addition to static subtitle
lines for creating effects, so in most cases it is safe to remove the animation
content.
+If subtitle text is missing, try setting this to false.
+
+@item select_layer
+Process only ASS subtitle events from a specific layer. This allows to filter
out certain effects where an ASS author duplicates the text onto multiple
layers.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Remove styles and animations from ASS subtitles and output events from ass
layer 0 only. Then convert asn save as SRT stream:
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv";
-filter_complex "[0:1]stripstyles=select_layer=0" -map 0 -c:s srt output.mkv
+@end example
+@end itemize
+
@section textmod
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 1af4f4b9bc..d330020f67 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -565,6 +565,7 @@ OBJS-$(CONFIG_NULLSINK_FILTER) +=
vsink_nullsink.o
OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o
OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o
OBJS-$(CONFIG_TEXTMOD_FILTER)+= sf_textmod.o
+OBJS-$(CONFIG_STRIPSTYLES_FILTER)+= sf_stripstyles.o
# multimedia filters
OBJS-$(CONFIG_ABITSCOPE_FILTER) += avf_abitscope.o
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index ac7d71547b..b0c12595af 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -549,6 +549,7 @@ extern const AVFilter ff_avf_showwavespic;
extern const AVFilter ff_vaf_spectrumsynth;
extern const AVFilter ff_sf_censor;
extern const AVFilter ff_sf_showspeaker;
+extern const AVFilter ff_sf_stripstyles;
extern const AVFilter ff_sf_textmod;
extern const AVFilter ff_svf_graphicsub2video;
extern const AVFilter ff_svf_textsub2video;
diff --git a/libavfilter/sf_stripstyles.c b/libavfilter/sf_stripstyles.c
new file mode 100644
index 00..bc3c5d1441
--- /dev/null
+++ b/libavfilter/sf_stripstyles.c
@@ -0,0 +1,211 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/**
+ * @file
+ * text subtitle filter which removes inline-styles from subtitles
+ */
+
+#include "libavutil/opt.h"
+#include "internal.h"
+#include "libavutil/ass_split_internal.h"
+#include "libavutil/bprint.h"
+
+typedef struct StripStylesContext {
+const AVClass *class;
+enum AVSubtitleType format;
+int remove_animated;
+int select_layer;
+} StripStylesContext;
+
+typedef struct DialogContext {
+StripStylesContext* ss_ctx;
+AVBPrint buffer;
+int drawing_scale;
+int is_animated;
+} DialogContext;
+
+static void dialog_text_cb(void *priv, const char *text, int len)
+{
+DialogContext *s = priv;
+
+av_log(s->ss_ctx, AV_LOG_DEBUG, "dialog_text_cb: %s\n", text);
+
+if (!s->drawing_scale && (!s->is_animated || !s->ss_ctx->remove_animated))
+av_bprint_append_data(&s->buffer, text, len);
+}
+
+static void dialog_new_line_cb(void *priv, int forced)
+{
+DialogContext *s = priv;
+if (!s->drawing_scale && !s->is_animated)
+av_bprint_append_data(&s->buffer, forced ? "\\N" : "\\n", 2);
+}
[FFmpeg-devel] [PATCH v2 16/26] avfilter/splitcc: Add splitcc filter for closed caption handling
From: softworkz
- splitcc {V -> VS)
Extract closed-caption (A53) data from video
frames as subtitle Frames
ffmpeg -y -loglevel verbose -i "https://streams.videolan.org/streams
/ts/CC/NewsStream-608-ac3.ts" -filter_complex "[0:v]splitcc[vid1],
textmod=mode=remove_chars:find='@',[vid1]overlay_textsubs" output.mkv
Signed-off-by: softworkz
---
configure| 1 +
doc/filters.texi | 63 +++
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/sf_splitcc.c | 395 +++
5 files changed, 461 insertions(+)
create mode 100644 libavfilter/sf_splitcc.c
diff --git a/configure b/configure
index 340a198fa6..c1d2bc41c2 100755
--- a/configure
+++ b/configure
@@ -3729,6 +3729,7 @@ spp_filter_select="fft idctdsp fdctdsp me_cmp pixblockdsp"
sr_filter_deps="avformat swscale"
sr_filter_select="dnn"
stereo3d_filter_deps="gpl"
+splitcc_filter_deps="avcodec"
subtitles_filter_deps="avformat avcodec libass"
super2xsai_filter_deps="gpl"
pixfmts_super2xsai_test_deps="super2xsai_filter"
diff --git a/doc/filters.texi b/doc/filters.texi
index c0f0fe13e7..265a267e9d 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -26131,6 +26131,69 @@ ffmpeg -i INPUT -filter_complex
"showspeaker=format=colon:style='@{\\c&HDD&\
@end example
@end itemize
+
+@section splitcc
+
+Split-out closed-caption/A53 subtitles from video frame side data.
+
+This filter provides an input and an output for video frames, which are just
passed through without modification.
+The second out provides subtitle frames which are extracted from video frame
side data.
+
+Inputs:
+@itemize
+@item 0: Video [ALL]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video (same as input)
+@item 1: Subtitles [TEXT]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+
+@item use_cc_styles
+Emit closed caption style header.
+This will make closed captions appear in white font with a black rectangle
background.
+
+@item real_time
+Emit subtitle events as they are decoded for real-time display.
+
+@item real_time_latency_msec
+Minimum elapsed time between emitting real-time subtitle events.
+Only applies to real_time mode.
+
+@item data_field
+Select data field. Possible values:
+
+@table @samp
+@item auto
+Pick first one that appears.
+@item first
+@item second
+@end table
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Extract closed captions as text subtitle stream and overlay it onto the video
in cc style (black bar background):
+@example
+ffmpeg -i "https://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts";
-filter_complex
"[0:v:0]splitcc=use_cc_styles=1[vid1][sub1];[vid1][sub1]overlaytextsubs"
output.mkv
+@end example
+
+@item
+A nicer variant, using realtime output from cc_dec and rendering it with the
render_latest_only parameter from overlaytextsubs to avoid ghosting by timely
overlap.
+@example
+ffmpeg -i "https://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts";
-filter_complex
"[0:v:0]splitcc=real_time=1:real_time_latency_msec=200[vid1][sub1];[vid1][sub1]overlaytextsubs=render_latest_only=1"
output.mkv
+@end example
+@end itemize
+
+
@section textsub2video
Converts text subtitles to video frames.
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index d330020f67..c6a4a4f5ae 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -565,6 +565,7 @@ OBJS-$(CONFIG_NULLSINK_FILTER) +=
vsink_nullsink.o
OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o
OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o
OBJS-$(CONFIG_TEXTMOD_FILTER)+= sf_textmod.o
+OBJS-$(CONFIG_SPLITCC_FILTER)+= sf_splitcc.o
OBJS-$(CONFIG_STRIPSTYLES_FILTER)+= sf_stripstyles.o
# multimedia filters
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index b0c12595af..50498e8ec4 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -549,6 +549,7 @@ extern const AVFilter ff_avf_showwavespic;
extern const AVFilter ff_vaf_spectrumsynth;
extern const AVFilter ff_sf_censor;
extern const AVFilter ff_sf_showspeaker;
+extern const AVFilter ff_sf_splitcc;
extern const AVFilter ff_sf_stripstyles;
extern const AVFilter ff_sf_textmod;
extern const AVFilter ff_svf_graphicsub2video;
diff --git a/libavfilter/sf_splitcc.c b/libavfilter/sf_splitcc.c
new file mode 100644
index 00..14235e822c
--- /dev/null
+++ b/libavfilter/sf_splitcc.c
@@ -0,0 +1,395 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+
[FFmpeg-devel] [PATCH v2 17/26] avfilter/graphicsub2text: Add new graphicsub2text filter (OCR)
From: softworkz Signed-off-by: softworkz --- configure|1 + doc/filters.texi | 55 ++ libavfilter/Makefile |2 + libavfilter/allfilters.c |1 + libavfilter/sf_graphicsub2text.c | 1132 ++ 5 files changed, 1191 insertions(+) create mode 100644 libavfilter/sf_graphicsub2text.c diff --git a/configure b/configure index c1d2bc41c2..ee7afffb05 100755 --- a/configure +++ b/configure @@ -3665,6 +3665,7 @@ frei0r_filter_deps="frei0r" frei0r_src_filter_deps="frei0r" fspp_filter_deps="gpl" gblur_vulkan_filter_deps="vulkan spirv_compiler" +graphicsub2text_filter_deps="libtesseract" hflip_vulkan_filter_deps="vulkan spirv_compiler" histeq_filter_deps="gpl" hqdn3d_filter_deps="gpl" diff --git a/doc/filters.texi b/doc/filters.texi index 265a267e9d..9311714f82 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -25897,6 +25897,61 @@ ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv"; -filter_comple @end example @end itemize +@section graphicsub2text + +Converts graphic subtitles to text subtitles by performing OCR. + +For this filter to be available, ffmpeg needs to be compiled with libtesseract (see https://github.com/tesseract-ocr/tesseract). +Language models need to be downloaded from https://github.com/tesseract-ocr/tessdata and put into as subfolder named 'tessdata' or into a folder specified via the environment variable 'TESSDATA_PREFIX'. +The path can also be specified via filter option (see below). + +Note: These models are including the data for both OCR modes. + +Inputs: +- 0: Subtitles [bitmap] + +Outputs: +- 0: Subtitles [text] + +It accepts the following parameters: + +@table @option +@item ocr_mode +The character recognition mode to use. + +Supported OCR modes are: + +@table @var +@item 0, tesseract +This is the classic libtesseract operation mode. It is fast but less accurate than LSTM. +@item 1, lstm +Newer OCR implementation based on ML models. Provides usually better results, requires more processing resources. +@item 2, both +Use a combination of both modes. +@end table + +@item tessdata_path +The path to a folder containing the language models to be used. + +@item language +The recognition language. It needs to match the first three characters of a language model file in the tessdata path. + +@end table + + +@subsection Examples + +@itemize +@item +Convert DVB graphic subtitles to ASS (text) subtitles + +Note: For this to work, you need to have the data file 'eng.traineddata' in a 'tessdata' subfolder (see above). +@example +ffmpeg -loglevel verbose -i "https://streams.videolan.org/streams/ts/video_subs_ttxt%2Bdvbsub.ts"; -filter_complex "[0:13]graphicsub2text=delay_when_no_duration=1" -c:s ass -y output.mkv +@end example +@end itemize + + @section graphicsub2video Renders graphic subtitles as video frames. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index c6a4a4f5ae..ead3e38507 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -299,6 +299,8 @@ OBJS-$(CONFIG_GBLUR_VULKAN_FILTER) += vf_gblur_vulkan.o vulkan.o vulka OBJS-$(CONFIG_GEQ_FILTER)+= vf_geq.o OBJS-$(CONFIG_GRADFUN_FILTER)+= vf_gradfun.o OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlaygraphicsubs.o framesync.o +OBJS-$(CONFIG_GRAPHICSUB2TEXT_FILTER)+= sf_graphicsub2text.o +OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlaygraphicsubs.o framesync.o OBJS-$(CONFIG_GRAPHMONITOR_FILTER) += f_graphmonitor.o OBJS-$(CONFIG_GRAYWORLD_FILTER) += vf_grayworld.o OBJS-$(CONFIG_GREYEDGE_FILTER) += vf_colorconstancy.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 50498e8ec4..34576016ce 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -548,6 +548,7 @@ extern const AVFilter ff_avf_showwaves; extern const AVFilter ff_avf_showwavespic; extern const AVFilter ff_vaf_spectrumsynth; extern const AVFilter ff_sf_censor; +extern const AVFilter ff_sf_graphicsub2text; extern const AVFilter ff_sf_showspeaker; extern const AVFilter ff_sf_splitcc; extern const AVFilter ff_sf_stripstyles; diff --git a/libavfilter/sf_graphicsub2text.c b/libavfilter/sf_graphicsub2text.c new file mode 100644 index 00..9b413d314e --- /dev/null +++ b/libavfilter/sf_graphicsub2text.c @@ -0,0 +1,1132 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser Gene
[FFmpeg-devel] [PATCH v2 18/26] avfilter/subscale: Add filter for scaling and/or re-arranging graphical subtitles
From: softworkz
Signed-off-by: softworkz
---
configure | 1 +
doc/filters.texi | 164 +++
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/sf_subscale.c | 884 ++
5 files changed, 1051 insertions(+)
create mode 100644 libavfilter/sf_subscale.c
diff --git a/configure b/configure
index ee7afffb05..a8b7ce8a26 100755
--- a/configure
+++ b/configure
@@ -3731,6 +3731,7 @@ sr_filter_deps="avformat swscale"
sr_filter_select="dnn"
stereo3d_filter_deps="gpl"
splitcc_filter_deps="avcodec"
+subscale_filter_deps="swscale avcodec"
subtitles_filter_deps="avformat avcodec libass"
super2xsai_filter_deps="gpl"
pixfmts_super2xsai_test_deps="super2xsai_filter"
diff --git a/doc/filters.texi b/doc/filters.texi
index 9311714f82..b3d11c0ac0 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -26303,6 +26303,170 @@ Set the rendering margin in pixels.
For rendering, alway use the latest event only, which is covering the given
point in time.
@end table
+@section subscale
+
+Provides high-quality scaling and rearranging functionality for graphical
subtitles.
+
+The subscale filter provides multiple approaches for manipulating
+the size and position of graphical subtitle rectangles wich can
+be combined or used separately.
+Scaling is performed by converting the palettized subtitle bitmaps
+to RGBA and re-quantization to palette colors afterwards via elbg algorithm.
+
+The two major operations are 'scale' and 're-arrange' with the
+latter being separated as 'arrange_h' and 'arrange_v'.
+
+
+Inputs:
+- 0: Subtitles [bitmap]
+
+Outputs:
+- 0: Subtitles [bitmap]
+
+It accepts the following parameters:
+
+@table @option
+
+@item w, width
+Set the width of the output.
+Width and height in case of graphical subtitles are just indicating
+a virtual size for which the output (consisting of 0-n bitmap rectangles)
+is intended to be displayed on.
+
+@item h, height
+Set the height of the output.
+
+@item margin_h
+Sets a horizontal margin to be preserverved when using any
+of the arrange modes.
+
+@item margin_v
+Sets a vertical margin to be preserverved when using any
+of the arrange modes.
+
+@item force_original_aspect_ratio
+Enable decreasing or increasing output video width or height if necessary to
+keep the original aspect ratio. Possible values:
+
+@table @samp
+@item disable
+Scale the video as specified and disable this feature.
+
+@item decrease
+The output video dimensions will automatically be decreased if needed.
+
+@item increase
+The output video dimensions will automatically be increased if needed.
+
+@end table
+
+
+@item scale_mode
+Specifies how subtitle bitmaps should be scaled.
+The scale factor is determined by the the factor between input
+and output size.
+
+@table @samp
+@item none
+Do not apply any common scaling.
+
+@item uniform
+Uniformly scale all subtitle bitmaps including their positions.
+
+@item uniform_no_reposition
+Uniformly scale all subtitle bitmaps without changing positions.
+
+@end table
+
+
+@item arrange_h
+Specifies how subtitle bitmaps should be arranged horizontally.
+
+@item arrange_v
+Specifies how subtitle bitmaps should be arranged vertically.
+
+
+@table @samp
+@item none
+Do not rearrange subtitle bitmaps.
+
+@item margin_no_scale
+Move subtitle bitmaps to be positioned inside the specified
+margin (margin_h or margin_v) when possible and without scaling.
+
+@item margin_and_scale
+Move subtitle bitmaps to be positioned inside the specified
+margin (margin_h or margin_v) and scale in case it doesn't fit.
+
+@item snapalign_no_scale
+Categorize subtitle bitmap positions as one of left/center/right
+or top/bottom/middle based on original positioning and apply
+these alignments for the target positioning.
+No scaling will be applied.
+
+@item snapalign_and_scale
+Categorize subtitle bitmap positions as one of left/center/right
+or top/bottom/middle based on original positioning and apply
+these alignments for the target positioning.
+Bitmaps that do not fit inside the margins borders are
+scaled to fit.
+@end table
+
+@item eval
+Set evaluation mode for the expressions (@option{width}, @option{height}).
+
+It accepts the following values:
+@table @samp
+@item init
+Evaluate expressions only once during the filter initialization.
+
+@item frame
+Evaluate expressions for each incoming frame. This is way slower than the
+@samp{init} mode since it requires all the scalers to be re-computed, but it
+allows advanced dynamic expressions.
+@end table
+
+Default value is @samp{init}.
+
+
+@item num_colors
+Set the number of palette colors for output images.
+Choose the maximum (256) when further processing is done (e.g.
+overlaying on a video).
+When subtitles will be encoded as bitmap subtitles (e.g. dvbsub),
+a smaller number of palette colors (e.g. 4-16) might need to be used, depending
+on the target format and codec.
+
+@item bitmap_width_align
+@item bitmap_height_align
+Make s
[FFmpeg-devel] [PATCH v2 19/26] avfilter/subfeed: add subtitle feed filter
From: softworkz
Signed-off-by: softworkz
---
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/sf_subfeed.c | 366 +++
3 files changed, 368 insertions(+)
create mode 100644 libavfilter/sf_subfeed.c
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 0e3e48613e..5711c7770b 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -570,6 +570,7 @@ OBJS-$(CONFIG_TEXTMOD_FILTER)+= sf_textmod.o
OBJS-$(CONFIG_SPLITCC_FILTER)+= sf_splitcc.o
OBJS-$(CONFIG_STRIPSTYLES_FILTER)+= sf_stripstyles.o
OBJS-$(CONFIG_SUBSCALE_FILTER) += sf_subscale.o
+OBJS-$(CONFIG_SUBFEED_FILTER)+= sf_subfeed.o
# multimedia filters
OBJS-$(CONFIG_ABITSCOPE_FILTER) += avf_abitscope.o
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 8e4f2feca3..10b14b7b8e 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -553,6 +553,7 @@ extern const AVFilter ff_sf_showspeaker;
extern const AVFilter ff_sf_splitcc;
extern const AVFilter ff_sf_stripstyles;
extern const AVFilter ff_sf_subscale;
+extern const AVFilter ff_sf_subfeed;
extern const AVFilter ff_sf_textmod;
extern const AVFilter ff_svf_graphicsub2video;
extern const AVFilter ff_svf_textsub2video;
diff --git a/libavfilter/sf_subfeed.c b/libavfilter/sf_subfeed.c
new file mode 100644
index 00..7df6641f6a
--- /dev/null
+++ b/libavfilter/sf_subfeed.c
@@ -0,0 +1,366 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/**
+ * @file
+ * subtitle filter for feeding subtitle frames into a filtergraph in a
contiguous way
+ *
+ *
+ * also supports
+ * - duration fixup
+ * delaying a subtitle event with unknown duration and infer duration from
the
+ * start time of the subsequent subtitle
+ * - scattering
+ * splitting a subtitle event with unknown duration into multiple ones with
+ * a short and fixed duration
+ *
+ */
+
+#include "filters.h"
+#include "libavutil/opt.h"
+#include "subtitles.h"
+#include "libavutil/avassert.h"
+
+enum SubFeedMode {
+FM_REPEAT,
+FM_SCATTER,
+FM_FORWARD,
+};
+
+typedef struct SubFeedContext {
+const AVClass *class;
+enum AVSubtitleType format;
+enum SubFeedMode mode;
+
+AVRational frame_rate;
+int fix_durations;
+int fix_overlap;
+
+int current_frame_isnew;
+int eof;
+int got_first_input;
+int need_frame;
+int64_t next_pts_offset;
+int64_t recent_subtitle_pts;
+
+int64_t counter;
+
+/**
+ * Queue of frames waiting to be filtered.
+ */
+FFFrameQueue fifo;
+
+} SubFeedContext;
+
+static int64_t ms_to_avtb(int64_t ms)
+{
+return av_rescale_q(ms, (AVRational){ 1, 1000 }, AV_TIME_BASE_Q);
+}
+
+static int64_t avtb_to_ms(int64_t avtb)
+{
+return av_rescale_q(avtb, AV_TIME_BASE_Q, (AVRational){ 1, 1000 });
+}
+
+static int init(AVFilterContext *ctx)
+{
+SubFeedContext *s = ctx->priv;
+
+ff_framequeue_init(&s->fifo, NULL);
+
+return 0;
+}
+
+static void uninit(AVFilterContext *ctx)
+{
+SubFeedContext *s = ctx->priv;
+ff_framequeue_free(&s->fifo);
+}
+
+static int config_input(AVFilterLink *link)
+{
+const subfeedContext *context = link->dst->priv;
+
+return 0;
+}
+
+static int query_formats(AVFilterContext *ctx)
+{
+AVFilterFormats *formats;
+AVFilterLink *inlink0 = ctx->inputs[0];
+AVFilterLink *outlink0 = ctx->outputs[0];
+static const enum AVSubtitleType subtitle_fmts[] = {
AV_SUBTITLE_FMT_BITMAP, AV_SUBTITLE_FMT_ASS, AV_SUBTITLE_FMT_NB };
+int ret;
+
+formats = ff_make_format_list(subtitle_fmts);
+
+if ((ret = ff_formats_ref(formats, &inlink0->outcfg.formats)) < 0)
+return ret;
+
+if ((ret = ff_formats_ref(formats, &outlink0->incfg.formats)) < 0)
+return ret;
+
+return 0;
+}
+
+static int config_output(AVFilterLink *outlink)
+{
+SubFeedContext *s = outlink->src->priv;
+const AVFilterLink *inlink = outlink->src->inputs[0];
+
+outlink->time_base = AV_TIME_BASE_Q;
+outlink->format = inlink->format;
+outlink->w = inlink->w;
+outlink->h = inlink->h;
+
+if (s->mode == FM_FORWARD)
+
[FFmpeg-devel] [PATCH v2 20/26] avcodec/subtitles: Migrate subtitle encoders to frame-based API
From: softworkz
and provide a compatibility shim for the legacy api
Signed-off-by: softworkz
---
libavcodec/assenc.c| 189 ++---
libavcodec/avcodec.h | 5 +-
libavcodec/dvbsubenc.c | 96 ++-
libavcodec/dvdsubenc.c | 102
libavcodec/encode.c| 61 +++-
libavcodec/movtextenc.c| 114 --
libavcodec/srtenc.c| 108 ++---
libavcodec/tests/avcodec.c | 2 -
libavcodec/ttmlenc.c | 101 +++-
libavcodec/webvttenc.c | 86 -
libavcodec/xsubenc.c | 88 ++---
11 files changed, 687 insertions(+), 265 deletions(-)
diff --git a/libavcodec/assenc.c b/libavcodec/assenc.c
index b0e475834b..e1401b1ac5 100644
--- a/libavcodec/assenc.c
+++ b/libavcodec/assenc.c
@@ -22,70 +22,195 @@
#include
#include "avcodec.h"
+#include "encode.h"
#include "libavutil/ass_internal.h"
#include "internal.h"
#include "libavutil/avstring.h"
#include "libavutil/internal.h"
#include "libavutil/mem.h"
+typedef struct {
+AVCodecContext *avctx;
+AVFrame* current_frame;
+int have_frame;
+int current_area;
+} AssEncContext;
+
+static void check_write_header(AVCodecContext* avctx, const AVFrame* frame)
+{
+if (avctx->extradata_size)
+return;
+
+if (frame->subtitle_header && frame->subtitle_header->size > 0) {
+const char* subtitle_header = (char*)frame->subtitle_header->data;
+avctx->extradata_size = strlen(subtitle_header);
+avctx->extradata = av_mallocz(frame->subtitle_header->size + 1);
+memcpy(avctx->extradata, subtitle_header, avctx->extradata_size);
+avctx->extradata[avctx->extradata_size] = 0;
+}
+
+if (!avctx->extradata_size) {
+const char* subtitle_header =
avpriv_ass_get_subtitle_header_default(0);
+if (!subtitle_header)
+return;
+
+avctx->extradata_size = strlen(subtitle_header);
+avctx->extradata = av_mallocz(avctx->extradata_size + 1);
+memcpy(avctx->extradata, subtitle_header, avctx->extradata_size);
+avctx->extradata[avctx->extradata_size] = 0;
+av_freep(&subtitle_header);
+}
+}
+
static av_cold int ass_encode_init(AVCodecContext *avctx)
{
-avctx->extradata = av_malloc(avctx->subtitle_header_size + 1);
-if (!avctx->extradata)
-return AVERROR(ENOMEM);
-memcpy(avctx->extradata, avctx->subtitle_header,
avctx->subtitle_header_size);
-avctx->extradata_size = avctx->subtitle_header_size;
-avctx->extradata[avctx->extradata_size] = 0;
+AssEncContext *s = avctx->priv_data;
+
+if (avctx->subtitle_header_size) {
+avctx->extradata = av_malloc(avctx->subtitle_header_size + 1);
+if (!avctx->extradata)
+return AVERROR(ENOMEM);
+memcpy(avctx->extradata, avctx->subtitle_header,
avctx->subtitle_header_size);
+avctx->extradata_size = avctx->subtitle_header_size;
+avctx->extradata[avctx->extradata_size] = 0;
+}
+
+s->current_frame = av_frame_alloc();
+return 0;
+}
+
+static av_cold int ass_encode_close(AVCodecContext *avctx)
+{
+AssEncContext *s = avctx->priv_data;
+av_frame_free(&s->current_frame);
return 0;
}
-static int ass_encode_frame(AVCodecContext *avctx,
-unsigned char *buf, int bufsize,
-const AVSubtitle *sub)
+static int ass_encode_frame(AVCodecContext* avctx, AVPacket* avpkt,
+const AVFrame* frame, int* got_packet)
+{
+int ret;
+size_t req_len = 0, total_len = 0;
+
+check_write_header(avctx, frame);
+
+for (unsigned i = 0; i < frame->num_subtitle_areas; i++) {
+const char *ass = frame->subtitle_areas[i]->ass;
+
+if (frame->subtitle_areas[i]->type != AV_SUBTITLE_FMT_ASS) {
+av_log(avctx, AV_LOG_ERROR, "Only AV_SUBTITLE_FMT_ASS type
supported.\n");
+return AVERROR(EINVAL);
+}
+
+if (ass)
+req_len += strlen(ass);
+}
+
+ret = ff_get_encode_buffer(avctx, avpkt, req_len + 1, 0);
+if (ret < 0) {
+av_log(avctx, AV_LOG_ERROR, "Error getting output packet.\n");
+return ret;
+}
+
+for (unsigned i = 0; i < frame->num_subtitle_areas; i++) {
+const char *ass = frame->subtitle_areas[i]->ass;
+
+if (ass) {
+size_t len = av_strlcpy((char *)avpkt->data + total_len, ass,
avpkt->size - total_len);
+total_len += len;
+}
+}
+
+avpkt->size = total_len;
+*got_packet = total_len > 0;
+
+return 0;
+}
+
+static int ass_receive_packet(AVCodecContext *avctx, AVPacket *avpkt)
{
-int i, len, total_len = 0;
+AssEncContext *s = avctx-
[FFmpeg-devel] [PATCH v2 22/26] avutil/ass_split: Add parsing of hard-space tags (\h)
From: softworkz
The \h tag in ASS/SSA is indicating a non-breaking space. See
https://github.com/Aegisub/aegisite/blob/master/source/docs/3.2/
ASS_Tags.html.md
The ass_split implementation is used by almost all text subtitle
encoders and it didn't handle this tag. Interestingly, several
tests are testing for \h parsing and had incorrect reference data
for those tests.
The \h tag is specific to ASS and doesn't have any meaning outside
of ASS.
Still, the reference data for ttmlenc, textenc and webvttenc were
full of \h tags even though this tag doesn't have a meaning there.
Signed-off-by: softworkz
---
libavutil/ass_split.c| 7 +++
tests/ref/fate/.gitattributes| 3 +++
tests/ref/fate/mov-mp4-ttml-dfxp | 8
tests/ref/fate/mov-mp4-ttml-stpp | 8
tests/ref/fate/sub-textenc | 10 +-
tests/ref/fate/sub-ttmlenc | 8
tests/ref/fate/sub-webvttenc | 10 +-
7 files changed, 32 insertions(+), 22 deletions(-)
create mode 100644 tests/ref/fate/.gitattributes
diff --git a/libavutil/ass_split.c b/libavutil/ass_split.c
index c5963351fc..30512dfc74 100644
--- a/libavutil/ass_split.c
+++ b/libavutil/ass_split.c
@@ -484,6 +484,7 @@ int avpriv_ass_split_override_codes(const ASSCodesCallbacks
*callbacks, void *pr
while (buf && *buf) {
if (text && callbacks->text &&
(sscanf(buf, "\\%1[nN]", new_line) == 1 ||
+ sscanf(buf, "\\%1[hH]", new_line) == 1 ||
!strncmp(buf, "{\\", 2))) {
callbacks->text(priv, text, text_len);
text = NULL;
@@ -492,6 +493,12 @@ int avpriv_ass_split_override_codes(const
ASSCodesCallbacks *callbacks, void *pr
if (callbacks->new_line)
callbacks->new_line(priv, new_line[0] == 'N');
buf += 2;
+} else if (sscanf(buf, "\\%1[hH]", new_line) == 1) {
+if (callbacks->hard_space)
+callbacks->hard_space(priv);
+else if (callbacks->text)
+callbacks->text(priv, " ", 1);
+buf += 2;
} else if (!strncmp(buf, "{\\", 2)) {
buf++;
while (*buf == '\\') {
diff --git a/tests/ref/fate/.gitattributes b/tests/ref/fate/.gitattributes
new file mode 100644
index 00..19be64d085
--- /dev/null
+++ b/tests/ref/fate/.gitattributes
@@ -0,0 +1,3 @@
+sub-textenc -diff
+sub-ttmlenc -diff
+sub-webvttenc -diff
diff --git a/tests/ref/fate/mov-mp4-ttml-dfxp b/tests/ref/fate/mov-mp4-ttml-dfxp
index e24b5d618b..e565ffa1f6 100644
--- a/tests/ref/fate/mov-mp4-ttml-dfxp
+++ b/tests/ref/fate/mov-mp4-ttml-dfxp
@@ -1,9 +1,9 @@
-2e7e01c821c111466e7a2844826b7f6d *tests/data/fate/mov-mp4-ttml-dfxp.mp4
-8519 tests/data/fate/mov-mp4-ttml-dfxp.mp4
+658884e1b789e75c454b25bdf71283c9 *tests/data/fate/mov-mp4-ttml-dfxp.mp4
+8486 tests/data/fate/mov-mp4-ttml-dfxp.mp4
#tb 0: 1/1000
#media_type 0: data
#codec_id 0: none
-0, 0, 0,68500, 7866, 0x456c36b7
+0, 0, 0,68500, 7833, 0x31b22193
{
"packets": [
{
@@ -15,7 +15,7 @@
"dts_time": "0.00",
"duration": 68500,
"duration_time": "68.50",
-"size": "7866",
+"size": "7833",
"pos": "44",
"flags": "K_"
}
diff --git a/tests/ref/fate/mov-mp4-ttml-stpp b/tests/ref/fate/mov-mp4-ttml-stpp
index 77bd23b7bf..f25b5b2d28 100644
--- a/tests/ref/fate/mov-mp4-ttml-stpp
+++ b/tests/ref/fate/mov-mp4-ttml-stpp
@@ -1,9 +1,9 @@
-cbd2c7ff864a663b0d893deac5a0caec *tests/data/fate/mov-mp4-ttml-stpp.mp4
-8547 tests/data/fate/mov-mp4-ttml-stpp.mp4
+c9570de0ccebc858b0c662a7e449582c *tests/data/fate/mov-mp4-ttml-stpp.mp4
+8514 tests/data/fate/mov-mp4-ttml-stpp.mp4
#tb 0: 1/1000
#media_type 0: data
#codec_id 0: none
-0, 0, 0,68500, 7866, 0x456c36b7
+0, 0, 0,68500, 7833, 0x31b22193
{
"packets": [
{
@@ -15,7 +15,7 @@ cbd2c7ff864a663b0d893deac5a0caec
*tests/data/fate/mov-mp4-ttml-stpp.mp4
"dts_time": "0.00",
"duration": 68500,
"duration_time": "68.50",
-"size": "7866",
+"size": "7833",
"pos": "44",
"flags": "K_"
}
diff --git a/tests/ref/fate/sub-textenc b/tests/ref/fate/sub-textenc
index 3ea56b38f0..910ca3d6e3 100644
--- a/tests/ref/fate/sub-textenc
+++ b/tests/ref/fate/sub-textenc
@@ -160,18 +160,18 @@ but show this: {normal text}
\ N is a forced line break
\ h is a hard space
Normal spaces at the start and at the end of the line are trimmed while hard
spaces are not trimmed.
-The\hline\hwill\hnever\hbreak\hautomatically\hright\hbefore\hor\hafter\ha\hhard\hspace.\h:-D
+The line will never break automatically right before or after a hard space. :-D
31
00:00:54,501 --> 00:00:56,500
-\h\h\h\h\hA (05 hard spaces followed by a letter)
+ A (05 hard spaces followed by a let
[FFmpeg-devel] [PATCH v2 23/26] avcodec/webvttenc: convert hard-space tags to
From: softworkz
Signed-off-by: softworkz
---
libavcodec/webvttenc.c | 6 ++
tests/ref/fate/sub-webvttenc | 10 +-
2 files changed, 11 insertions(+), 5 deletions(-)
diff --git a/libavcodec/webvttenc.c b/libavcodec/webvttenc.c
index c0436f5739..48945dcb8e 100644
--- a/libavcodec/webvttenc.c
+++ b/libavcodec/webvttenc.c
@@ -123,6 +123,11 @@ static void webvtt_new_line_cb(void *priv, int forced)
webvtt_print(priv, "\n");
}
+static void webvtt_hard_space_cb(void *priv)
+{
+webvtt_print(priv, " ");
+}
+
static void webvtt_style_cb(void *priv, char style, int close)
{
if (style == 's') // strikethrough unsupported
@@ -147,6 +152,7 @@ static void webvtt_end_cb(void *priv)
static const ASSCodesCallbacks webvtt_callbacks = {
.text = webvtt_text_cb,
.new_line = webvtt_new_line_cb,
+.hard_space = webvtt_hard_space_cb,
.style= webvtt_style_cb,
.color= NULL,
.font_name= NULL,
diff --git a/tests/ref/fate/sub-webvttenc b/tests/ref/fate/sub-webvttenc
index f4172dcc84..ee9de2859e 100644
--- a/tests/ref/fate/sub-webvttenc
+++ b/tests/ref/fate/sub-webvttenc
@@ -132,26 +132,26 @@ but show this: {normal text}
\ N is a forced line break
\ h is a hard space
Normal spaces at the start and at the end of the line are trimmed while hard
spaces are not trimmed.
-The line will never break automatically right before or after a hard space. :-D
+The line will never break automatically right before or after a hard space. :-D
00:54.501 --> 00:56.500
- A (05 hard spaces followed by a letter)
+ A (05 hard spaces followed by a letter)
A (Normal spaces followed by a letter)
A (No hard spaces followed by a letter)
00:56.501 --> 00:58.500
- A (05 hard spaces followed by a letter)
+ A (05 hard spaces followed by a letter)
A (Normal spaces followed by a letter)
A (No hard spaces followed by a letter)
Show this: \TEST and this: \-)
00:58.501 --> 01:00.500
-A letter followed by 05 hard spaces: A
+A letter followed by 05 hard spaces: A
A letter followed by normal spaces: A
A letter followed by no hard spaces: A
-05 hard spaces between letters: A A
+05 hard spaces between letters: A A
5 normal spaces between letters: A A
^--Forced line break
--
ffmpeg-codebot
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2 24/26] doc/APIchanges: update for subtitle filtering changes
From: softworkz Signed-off-by: softworkz --- doc/APIchanges | 24 libavcodec/version.h | 2 +- libavutil/version.h | 2 +- 3 files changed, 26 insertions(+), 2 deletions(-) diff --git a/doc/APIchanges b/doc/APIchanges index 8df0364e4c..c8238fb008 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -14,6 +14,30 @@ libavutil: 2021-04-27 API changes, most recent first: +2021-12-05 - xx - lavc 59.15.100 - avcodec.h + Deprecate avcodec_encode_subtitle(), use regular encode api now + +2021-12-05 - xx - lavc 59.15.100 - codec_desc.h + Add avcodec_descriptor_get_subtitle_format() + +2021-12-05 - xx - lavc 59.15.100 - avcodec.h + Deprecate avsubtitle_free() + Deprecate avcodec_decode_subtitle2(), use regular decode api now + +2021-12-05 - xx - lavu 57.11.100 - frame.h + Add AVMediaType field to AVFrame + Add Fields for carrying subtitle data to AVFrame + (subtitle_areas, subtitle_header, subtitle_pts, start/end time, etc.) + Add av_frame_get_buffer2() and deprecate av_frame_get_buffer() + +2021-12-05 - xx - lavu 57.11.100 - subfmt.h + Add struct AVSubtitleArea (replaces AVSubtitle) + Add av_get_subtitle_fmt_name() and av_get_subtitle_fmt() + +2021-12-05 - xx - lavu 57.11.100 - subfmt.h + Add enum AVSubtitleType (moved from lavc), add new values, deprecate existing + +2021-11-xx - xx - lavfi 8.19.100 - avfilter.h 2022-01-04 - 78dc21b123e - lavu 57.16.100 - frame.h Add AV_FRAME_DATA_DOVI_METADATA. diff --git a/libavcodec/version.h b/libavcodec/version.h index a46fb05f1a..b5867ad041 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -28,7 +28,7 @@ #include "libavutil/version.h" #define LIBAVCODEC_VERSION_MAJOR 59 -#define LIBAVCODEC_VERSION_MINOR 20 +#define LIBAVCODEC_VERSION_MINOR 21 #define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ diff --git a/libavutil/version.h b/libavutil/version.h index 5bf48f6304..168e24f410 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,7 +79,7 @@ */ #define LIBAVUTIL_VERSION_MAJOR 57 -#define LIBAVUTIL_VERSION_MINOR 18 +#define LIBAVUTIL_VERSION_MINOR 19 #define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ -- ffmpeg-codebot ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2 25/26] avcodec/webvttenc: Don't encode drawing codes and empty lines
From: softworkz
Signed-off-by: softworkz
---
libavcodec/webvttenc.c | 31 +++
1 file changed, 27 insertions(+), 4 deletions(-)
diff --git a/libavcodec/webvttenc.c b/libavcodec/webvttenc.c
index 48945dcb8e..62c4aa7ffd 100644
--- a/libavcodec/webvttenc.c
+++ b/libavcodec/webvttenc.c
@@ -39,6 +39,8 @@ typedef struct {
int count;
char stack[WEBVTT_STACK_SIZE];
int stack_ptr;
+int has_text;
+int drawing_scale;
} WebVTTContext;
#ifdef __GNUC__
@@ -115,17 +117,24 @@ static void webvtt_style_apply(WebVTTContext *s, const
char *style)
static void webvtt_text_cb(void *priv, const char *text, int len)
{
WebVTTContext *s = priv;
-av_bprint_append_data(&s->buffer, text, len);
+if (!s->drawing_scale) {
+av_bprint_append_data(&s->buffer, text, len);
+s->has_text = 1;
+}
}
static void webvtt_new_line_cb(void *priv, int forced)
{
-webvtt_print(priv, "\n");
+WebVTTContext *s = priv;
+if (!s->drawing_scale)
+webvtt_print(priv, "\n");
}
static void webvtt_hard_space_cb(void *priv)
{
-webvtt_print(priv, " ");
+WebVTTContext *s = priv;
+if (!s->drawing_scale)
+webvtt_print(priv, " ");
}
static void webvtt_style_cb(void *priv, char style, int close)
@@ -149,6 +158,12 @@ static void webvtt_end_cb(void *priv)
webvtt_stack_push_pop(priv, 0, 1);
}
+static void dialog_drawing_mode_cb(void *priv, int scale)
+{
+WebVTTContext *s = priv;
+s->drawing_scale = scale;
+}
+
static const ASSCodesCallbacks webvtt_callbacks = {
.text = webvtt_text_cb,
.new_line = webvtt_new_line_cb,
@@ -161,6 +176,7 @@ static const ASSCodesCallbacks webvtt_callbacks = {
.cancel_overrides = webvtt_cancel_overrides_cb,
.move = NULL,
.end = webvtt_end_cb,
+.drawing_mode = dialog_drawing_mode_cb,
};
static void ensure_ass_context(WebVTTContext* s, const AVFrame* frame)
@@ -211,16 +227,23 @@ static int webvtt_encode_frame(AVCodecContext* avctx,
AVPacket* avpkt,
}
if (ass) {
+const unsigned saved_len = s->buffer.len;
-if (i > 0)
+if (i > 0 && s->buffer.len > 0)
webvtt_new_line_cb(s, 0);
+s->drawing_scale = 0;
+s->has_text = 0;
+
dialog = avpriv_ass_split_dialog(s->ass_ctx, ass);
if (!dialog)
return AVERROR(ENOMEM);
webvtt_style_apply(s, dialog->style);
avpriv_ass_split_override_codes(&webvtt_callbacks, s,
dialog->text);
avpriv_ass_free_dialog(&dialog);
+
+if (!s->has_text)
+s->buffer.len = saved_len;
}
}
--
ffmpeg-codebot
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2 26/26] avcodec/dvbsubdec: Fix conditions for fallback to default resolution
From: softworkz
The previous code expected a segment of type CLUT definition to exist
in order to accept a set of segments to be complete.
This was an incorrect assumption as the presence of a CLUT segment
is not mandatory.
(version 1.6.1 of the spec is probably a bit more clear about this
than earlier versions: https://www.etsi.org/deliver/etsi_en/
300700_300799/300743/01.06.01_20/en_300743v010601a.pdf)
The flawed condition prevented proper fallback to using the default
resolution for the decoding context.
Signed-off-by: softworkz
---
libavcodec/dvbsubdec.c | 51 +-
1 file changed, 30 insertions(+), 21 deletions(-)
diff --git a/libavcodec/dvbsubdec.c b/libavcodec/dvbsubdec.c
index 0d64c6e71c..3a6259101c 100644
--- a/libavcodec/dvbsubdec.c
+++ b/libavcodec/dvbsubdec.c
@@ -33,7 +33,7 @@
#define DVBSUB_CLUT_SEGMENT 0x12
#define DVBSUB_OBJECT_SEGMENT 0x13
#define DVBSUB_DISPLAYDEFINITION_SEGMENT 0x14
-#define DVBSUB_DISPLAY_SEGMENT 0x80
+#define DVBSUB_END_DISPLAY_SEGMENT 0x80
#define cm (ff_crop_tab + MAX_NEG_CROP)
@@ -1620,8 +1620,12 @@ static int dvbsub_decode(AVCodecContext *avctx,
int segment_length;
int i;
int ret = 0;
-int got_segment = 0;
-int got_dds = 0;
+//int got_segment = 0;
+int got_page = 0;
+int got_region = 0;
+int got_object = 0;
+int got_end_display = 0;
+int got_displaydef = 0;
ff_dlog(avctx, "DVB sub packet:\n");
@@ -1666,34 +1670,28 @@ static int dvbsub_decode(AVCodecContext *avctx,
switch (segment_type) {
case DVBSUB_PAGE_SEGMENT:
ret = dvbsub_parse_page_segment(avctx, p, segment_length, sub,
got_sub_ptr);
-got_segment |= 1;
+got_page = 1;
break;
case DVBSUB_REGION_SEGMENT:
ret = dvbsub_parse_region_segment(avctx, p, segment_length);
-got_segment |= 2;
+got_region = 1;
break;
case DVBSUB_CLUT_SEGMENT:
ret = dvbsub_parse_clut_segment(avctx, p, segment_length);
if (ret < 0) goto end;
-got_segment |= 4;
break;
case DVBSUB_OBJECT_SEGMENT:
ret = dvbsub_parse_object_segment(avctx, p, segment_length);
-got_segment |= 8;
+got_object = 1;
break;
case DVBSUB_DISPLAYDEFINITION_SEGMENT:
ret = dvbsub_parse_display_definition_segment(avctx, p,
segment_length);
-got_dds = 1;
+got_displaydef = 1;
break;
-case DVBSUB_DISPLAY_SEGMENT:
+case DVBSUB_END_DISPLAY_SEGMENT:
ret = dvbsub_display_end_segment(avctx, p, segment_length,
sub, got_sub_ptr);
-if (got_segment == 15 && !got_dds && !avctx->width &&
!avctx->height) {
-// Default from ETSI EN 300 743 V1.3.1 (7.2.1)
-avctx->width = 720;
-avctx->height = 576;
-}
-got_segment |= 16;
+got_end_display = 1;
break;
default:
ff_dlog(avctx, "Subtitling segment type 0x%x, page id %d,
length %d\n",
@@ -1706,13 +1704,24 @@ static int dvbsub_decode(AVCodecContext *avctx,
p += segment_length;
}
-// Some streams do not send a display segment but if we have all the other
-// segments then we need no further data.
-if (got_segment == 15) {
-av_log(avctx, AV_LOG_DEBUG, "Missing display_end_segment,
emulating\n");
-dvbsub_display_end_segment(avctx, p, 0, sub, got_sub_ptr);
-}
+// Even though not mandated by the spec, we're imposing a minimum
requirement
+// for a useful packet to have at least one page, region and object
segment.
+if (got_page && got_region && got_object && got_end_display) {
+
+if (!got_displaydef && !avctx->width && !avctx->height) {
+// Default from ETSI EN 300 743 V1.3.1 (7.2.1)
+avctx->width = 720;
+avctx->height = 576;
+}
+
+// Some streams do not send an end-of-display segment but if we have
all the other
+// segments then we need no further data.
+if (!got_end_display) {
+av_log(avctx, AV_LOG_DEBUG, "Missing display_end_segment,
emulating\n");
+dvbsub_display_end_segment(avctx, p, 0, sub, got_sub_ptr);
+}
+}
end:
if (ret < 0) {
return ret;
--
ffmpeg-codebot
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v3 00/26] Subtitle Filtering 2022
Subtitle Filtering 2022
===
This is a substantial update to the earlier subtitle filtering patch series.
A primary goal has been to address others' concerns as much as possible on
one side and to provide more clarity and control over the way things are
working. Clarity is is specifically important to allow for a better
understanding of the need for a subtitle start pts value that can be
different from the frame's pts value. This is done by refactoring the
subtitle timing fields in AVFrame, adding a frame field to indicate repeated
subtitle frames, and finally the full removal of the heartbeat
functionality, replaced by a new 'subfeed' filter that provides different
modes for arbitrating subtitle frames in a filter graph. Finally, each
subtitle filter's documentation has been amended by a section describing the
filter's timeline behavior (in v3 update).
The update also includes major improvements to graphicsub2text and lots of
other details.
Versioning is restarting at v1 due to the new submission procedure.
v3 - Rebase
===
due to merge conflicts - apologies.
Changes in v2
=
* added .gitattributes file to enforce binary diffs for the test refs that
cannot be applied when being sent via e-mail
* perform filter graph re-init due to subtitle "frame size" change only
when the size was unknown before and not set via -canvas_size
* overlaytextsubs: Make sure to request frames on the subtitle input
* avfilter/splitcc: Start parsing cc data on key frames only
* avcodec/webvttenc: Don't encode ass drawing codes and empty lines
* stripstyles: fix mem leak
* gs2t: improve color detection
* gs2t: empty frames must not be skipped
* subfeed: fix name
* textmod: preserve margins
* added .gitattributes file to enforce binary diffs for the test refs that
cannot be applied when being sent via e-mail
* perform filter graph re-init due to subtitle "frame size" change only
when the size was unknown before and not set via -canvas_size
* avcodec/dvbsubdec: Fix conditions for fallback to default resolution
* Made changes suggested by Andreas
* Fixed failing command line reported by Michael
Changes from previous version v24:
AVFrame
===
* Removed sub_start_time The start time is now added to the subtitle
start_pts during decoding The sub_end_time field is adjusted accordingly
* Renamed sub_end_time to duration which it is effectively after removing
the start_time
* Added a sub-struct 'subtitle_timing' to av frame Contains subtitle_pts
renamed to 'subtitle_timing.start_pts' and 'subtitle_timing.duration'
* Change both fields to (fixed) time_base AV_TIMEBASE
* add repeat_sub field provides a clear indication whether a subtitle frame
is an actual subtitle event or a repeated subtitle frame in a filter
graph
Heartbeat Removal
=
* completely removed the earlier heartbeat implementation
* filtering arbitration is now implemented in a new filter: 'subfeed'
* subfeed will be auto-inserted for compatiblity with sub2video command
lines
* the new behavior is not exactly identical to the earlier behavior, but it
basically allows to achieve the same results
New 'subfeed' Filter
* a versatile filter for solving all kinds of problems with subtile frame
flow in filter graphs
* Can be inserted at any position in a graph
* Auto-inserted for sub2video command lines (in repeat-mode)
* Allows duration fixup delay input frames with unknown duration and infer
duration from start of subsequent frame
* Provides multiple modes of operation:
* repeat mode (default) Queues input frames Outputs frames at a fixed
(configurable) rate Either sends a matching input frame (repeatedly) or
empty frames otherwise
* scatter mode similar to repeat mode, but splits input frames by
duration into small segments with same content
* forward mode No fixed output rate Useful in combination with duration
fixup or overlap fixup
ffmpeg Tool Changes
===
* delay subtitle output stream initialization (like for audio and video)
This is needed for example when a format header depends on having
received an initial frame to derive certain header values from
* decoding: set subtitle frame size from decoding context
* re-init graph when subtitle size changes
* always insert subscale filter for sub2video command lines (to ensure
correct scaling)
Subtitle Encoding
=
* ignore repeated frames for encoding based on repeat_sub field in AVFrame
* support multi-area encoding for text subtitles Subtitle OCR can create
multiple areas at different positions. Previously, the texts were always
squashed into a single area ('subtitle rect'), which was not ideal.
Multiple text areas are now generally supported:
* ASS Encoder Changed to use the 'receive_packet' encoding API A single
frame with multiple text areas will create multiple packets
[FFmpeg-devel] [PATCH v3 01/26] avcodec, avutil: Move enum AVSubtitleType to avutil, add new and deprecate old values
From: softworkz
Signed-off-by: softworkz
---
libavcodec/avcodec.h | 19 +
libavutil/Makefile | 1 +
libavutil/subfmt.h | 68
libavutil/version.h | 1 +
4 files changed, 71 insertions(+), 18 deletions(-)
create mode 100644 libavutil/subfmt.h
diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index ec1a0566a4..fe5a83cf85 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -35,6 +35,7 @@
#include "libavutil/frame.h"
#include "libavutil/log.h"
#include "libavutil/pixfmt.h"
+#include "libavutil/subfmt.h"
#include "libavutil/rational.h"
#include "codec.h"
@@ -2238,24 +2239,6 @@ typedef struct AVHWAccel {
* @}
*/
-enum AVSubtitleType {
-SUBTITLE_NONE,
-
-SUBTITLE_BITMAP,///< A bitmap, pict will be set
-
-/**
- * Plain text, the text field must be set by the decoder and is
- * authoritative. ass and pict fields may contain approximations.
- */
-SUBTITLE_TEXT,
-
-/**
- * Formatted text, the ass field must be set by the decoder and is
- * authoritative. pict and text fields may contain approximations.
- */
-SUBTITLE_ASS,
-};
-
#define AV_SUBTITLE_FLAG_FORCED 0x0001
typedef struct AVSubtitleRect {
diff --git a/libavutil/Makefile b/libavutil/Makefile
index d17876df1a..ce644f4d48 100644
--- a/libavutil/Makefile
+++ b/libavutil/Makefile
@@ -74,6 +74,7 @@ HEADERS = adler32.h
\
sha512.h \
spherical.h \
stereo3d.h\
+ subfmt.h \
threadmessage.h \
time.h\
timecode.h\
diff --git a/libavutil/subfmt.h b/libavutil/subfmt.h
new file mode 100644
index 00..791b45519f
--- /dev/null
+++ b/libavutil/subfmt.h
@@ -0,0 +1,68 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#ifndef AVUTIL_SUBFMT_H
+#define AVUTIL_SUBFMT_H
+
+#include "version.h"
+
+enum AVSubtitleType {
+
+/**
+ * Subtitle format unknown.
+ */
+AV_SUBTITLE_FMT_NONE = -1,
+
+/**
+ * Subtitle format unknown.
+ */
+AV_SUBTITLE_FMT_UNKNOWN = 0,
+#if FF_API_OLD_SUBTITLES
+SUBTITLE_NONE = 0, ///< Deprecated, use AV_SUBTITLE_FMT_NONE
instead.
+#endif
+
+/**
+ * Bitmap area in AVSubtitleRect.data, pixfmt AV_PIX_FMT_PAL8.
+ */
+AV_SUBTITLE_FMT_BITMAP = 1,
+#if FF_API_OLD_SUBTITLES
+SUBTITLE_BITMAP = 1,///< Deprecated, use AV_SUBTITLE_FMT_BITMAP
instead.
+#endif
+
+/**
+ * Plain text in AVSubtitleRect.text.
+ */
+AV_SUBTITLE_FMT_TEXT = 2,
+#if FF_API_OLD_SUBTITLES
+SUBTITLE_TEXT = 2, ///< Deprecated, use AV_SUBTITLE_FMT_TEXT
instead.
+#endif
+
+/**
+ * Text Formatted as per ASS specification, contained AVSubtitleRect.ass.
+ */
+AV_SUBTITLE_FMT_ASS = 3,
+#if FF_API_OLD_SUBTITLES
+SUBTITLE_ASS = 3, ///< Deprecated, use AV_SUBTITLE_FMT_ASS
instead.
+#endif
+
+AV_SUBTITLE_FMT_NB, ///< number of subtitle formats, DO NOT USE
THIS if you want to link with shared libav* because the number of formats might
differ between versions.
+};
+
+#endif /* AVUTIL_SUBFMT_H */
diff --git a/libavutil/version.h b/libavutil/version.h
index 953aac9d94..5bf48f6304 100644
--- a/libavutil/version.h
+++ b/libavutil/version.h
@@ -110,6 +110,7 @@
#define FF_API_COLORSPACE_NAME (LIBAVUTIL_VERSION_MAJOR < 58)
#define FF_API_AV_MALLOCZ_ARRAY (LIBAVUTIL_VERSION_MAJOR < 58)
#define FF_API_FIFO_PEEK2 (LIBAVUTIL_VERSION_MAJOR < 58)
+#define FF_API_OLD_SUBTITLES(LIBAVUTIL_VERSION_MAJOR < 58)
/**
* @}
--
ffmpeg-codebot
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or ema
[FFmpeg-devel] [PATCH v3 02/26] avutil/frame: Prepare AVFrame for subtitle handling
From: softworkz
Root commit for adding subtitle filtering capabilities.
In detail:
- Add type (AVMediaType) field to AVFrame
Replaces previous way of distinction which was based on checking
width and height to determine whether a frame is audio or video
- Add subtitle fields to AVFrame
- Add new struct AVSubtitleArea, similar to AVSubtitleRect, but
different allocation logic. Cannot and must not be used
interchangeably, hence the new struct
Signed-off-by: softworkz
---
libavutil/Makefile | 1 +
libavutil/frame.c | 211 -
libavutil/frame.h | 85 +-
libavutil/subfmt.c | 45 ++
libavutil/subfmt.h | 47 ++
5 files changed, 364 insertions(+), 25 deletions(-)
create mode 100644 libavutil/subfmt.c
diff --git a/libavutil/Makefile b/libavutil/Makefile
index ce644f4d48..8bc0a14942 100644
--- a/libavutil/Makefile
+++ b/libavutil/Makefile
@@ -160,6 +160,7 @@ OBJS = adler32.o
\
slicethread.o\
spherical.o \
stereo3d.o \
+ subfmt.o \
threadmessage.o \
time.o \
timecode.o \
diff --git a/libavutil/frame.c b/libavutil/frame.c
index 8997c85e35..2b95830b6f 100644
--- a/libavutil/frame.c
+++ b/libavutil/frame.c
@@ -26,6 +26,7 @@
#include "imgutils.h"
#include "mem.h"
#include "samplefmt.h"
+#include "subfmt.h"
#include "hwcontext.h"
#define CHECK_CHANNELS_CONSISTENCY(frame) \
@@ -50,6 +51,9 @@ const char *av_get_colorspace_name(enum AVColorSpace val)
return name[val];
}
#endif
+
+static int frame_copy_subtitles(AVFrame *dst, const AVFrame *src, int
copy_data);
+
static void get_frame_defaults(AVFrame *frame)
{
memset(frame, 0, sizeof(*frame));
@@ -70,7 +74,12 @@ static void get_frame_defaults(AVFrame *frame)
frame->colorspace = AVCOL_SPC_UNSPECIFIED;
frame->color_range = AVCOL_RANGE_UNSPECIFIED;
frame->chroma_location = AVCHROMA_LOC_UNSPECIFIED;
-frame->flags = 0;
+frame->num_subtitle_areas = 0;
+frame->subtitle_areas = NULL;
+frame->subtitle_header = NULL;
+frame->repeat_sub = 0;
+frame->subtitle_timing.start_pts = 0;
+frame->subtitle_timing.duration = 0;
}
static void free_side_data(AVFrameSideData **ptr_sd)
@@ -240,23 +249,55 @@ static int get_audio_buffer(AVFrame *frame, int align)
}
+static int get_subtitle_buffer(AVFrame *frame)
+{
+// Buffers in AVFrame->buf[] are not used in case of subtitle frames.
+// To accomodate with existing code, checking ->buf[0] to determine
+// whether a frame is ref-counted or has data, we're adding a 1-byte
+// buffer here, which marks the subtitle frame to contain data.
+frame->buf[0] = av_buffer_alloc(1);
+if (!frame->buf[0]) {
+av_frame_unref(frame);
+return AVERROR(ENOMEM);
+}
+
+frame->extended_data = frame->data;
+
+return 0;
+}
+
int av_frame_get_buffer(AVFrame *frame, int align)
+{
+if (frame->width > 0 && frame->height > 0)
+frame->type = AVMEDIA_TYPE_VIDEO;
+else if (frame->nb_samples > 0 && (frame->channel_layout ||
frame->channels > 0))
+frame->type = AVMEDIA_TYPE_AUDIO;
+
+return av_frame_get_buffer2(frame, align);
+}
+
+int av_frame_get_buffer2(AVFrame *frame, int align)
{
if (frame->format < 0)
return AVERROR(EINVAL);
-if (frame->width > 0 && frame->height > 0)
+switch(frame->type) {
+case AVMEDIA_TYPE_VIDEO:
return get_video_buffer(frame, align);
-else if (frame->nb_samples > 0 && (frame->channel_layout ||
frame->channels > 0))
+case AVMEDIA_TYPE_AUDIO:
return get_audio_buffer(frame, align);
-
-return AVERROR(EINVAL);
+case AVMEDIA_TYPE_SUBTITLE:
+return get_subtitle_buffer(frame);
+default:
+return AVERROR(EINVAL);
+}
}
static int frame_copy_props(AVFrame *dst, const AVFrame *src, int force_copy)
{
int ret, i;
+dst->type = src->type;
dst->key_frame = src->key_frame;
dst->pict_type = src->pict_type;
dst->sample_aspect_ratio= src->sample_aspect_ratio;
@@ -288,6 +329,12 @@ static int frame_copy_props(AVFrame *dst, const AVFrame
*src, int force_copy)
dst->colorspace = src->colorspace;
dst->color_range= src->color_range;
dst->chroma_location= src->chroma_location;
+dst->repeat_sub = src->repeat_sub;
+dst->subtitle_timing.start_pts = src->subtitle_timing.start_pts;
+
[FFmpeg-devel] [PATCH v3 03/26] avcodec/subtitles: Introduce new frame-based subtitle decoding API
From: softworkz
- Modify avcodec_send_packet() to support subtitles via the regular
frame based decoding API
- Add decode_subtitle_shim() which takes subtitle frames,
and serves as a compatibility shim to the legacy subtitle decoding
API until all subtitle decoders are migrated to the frame-based API
- Add additional methods for conversion between old and new API
Signed-off-by: softworkz
---
libavcodec/avcodec.h| 8 +-
libavcodec/codec_desc.c | 11 +++
libavcodec/codec_desc.h | 8 ++
libavcodec/decode.c | 60 +++--
libavcodec/internal.h | 16
libavcodec/utils.c | 184
6 files changed, 278 insertions(+), 9 deletions(-)
diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index fe5a83cf85..9d59f6e840 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -1675,7 +1675,7 @@ typedef struct AVCodecContext {
/**
* Header containing style information for text subtitles.
- * For SUBTITLE_ASS subtitle type, it should contain the whole ASS
+ * For AV_SUBTITLE_FMT_ASS subtitle type, it should contain the whole ASS
* [Script Info] and [V4+ Styles] section, plus the [Events] line and
* the Format line following. It shouldn't include any Dialogue line.
* - encoding: Set/allocated/freed by user (before avcodec_open2())
@@ -2415,7 +2415,10 @@ int avcodec_close(AVCodecContext *avctx);
* Free all allocated data in the given subtitle struct.
*
* @param sub AVSubtitle to free.
+ *
+ * @deprecated Use the regular frame based encode and decode APIs instead.
*/
+attribute_deprecated
void avsubtitle_free(AVSubtitle *sub);
/**
@@ -2508,7 +2511,10 @@ enum AVChromaLocation avcodec_chroma_pos_to_enum(int
xpos, int ypos);
* must be freed with avsubtitle_free if *got_sub_ptr is set.
* @param[in,out] got_sub_ptr Zero if no subtitle could be decompressed,
otherwise, it is nonzero.
* @param[in] avpkt The input AVPacket containing the input buffer.
+ *
+ * @deprecated Use the new decode API (avcodec_send_packet,
avcodec_receive_frame) instead.
*/
+attribute_deprecated
int avcodec_decode_subtitle2(AVCodecContext *avctx, AVSubtitle *sub,
int *got_sub_ptr,
AVPacket *avpkt);
diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c
index 0974ee03de..e48e4532ba 100644
--- a/libavcodec/codec_desc.c
+++ b/libavcodec/codec_desc.c
@@ -3548,3 +3548,14 @@ enum AVMediaType avcodec_get_type(enum AVCodecID
codec_id)
const AVCodecDescriptor *desc = avcodec_descriptor_get(codec_id);
return desc ? desc->type : AVMEDIA_TYPE_UNKNOWN;
}
+
+enum AVSubtitleType avcodec_descriptor_get_subtitle_format(const
AVCodecDescriptor *codec_descriptor)
+{
+if(codec_descriptor->props & AV_CODEC_PROP_BITMAP_SUB)
+return AV_SUBTITLE_FMT_BITMAP;
+
+if(codec_descriptor->props & AV_CODEC_PROP_TEXT_SUB)
+return AV_SUBTITLE_FMT_ASS;
+
+return AV_SUBTITLE_FMT_UNKNOWN;
+}
diff --git a/libavcodec/codec_desc.h b/libavcodec/codec_desc.h
index 126b52df47..ba68d24e0e 100644
--- a/libavcodec/codec_desc.h
+++ b/libavcodec/codec_desc.h
@@ -121,6 +121,14 @@ const AVCodecDescriptor *avcodec_descriptor_next(const
AVCodecDescriptor *prev);
*/
const AVCodecDescriptor *avcodec_descriptor_get_by_name(const char *name);
+/**
+ * Return subtitle format from a codec descriptor
+ *
+ * @param codec_descriptor codec descriptor
+ * @return the subtitle type (e.g. bitmap, text)
+ */
+enum AVSubtitleType avcodec_descriptor_get_subtitle_format(const
AVCodecDescriptor *codec_descriptor);
+
/**
* @}
*/
diff --git a/libavcodec/decode.c b/libavcodec/decode.c
index 0912f86a14..e3cf1cfa3d 100644
--- a/libavcodec/decode.c
+++ b/libavcodec/decode.c
@@ -576,6 +576,39 @@ static int decode_receive_frame_internal(AVCodecContext
*avctx, AVFrame *frame)
return ret;
}
+static int decode_subtitle2_priv(AVCodecContext *avctx, AVSubtitle *sub,
+ int *got_sub_ptr, AVPacket *avpkt);
+
+static int decode_subtitle_shim(AVCodecContext *avctx, AVFrame *frame,
AVPacket *avpkt)
+{
+int ret, got_sub_ptr = 0;
+AVSubtitle subtitle = { 0 };
+
+if (frame->buf[0])
+return AVERROR(EAGAIN);
+
+av_frame_unref(frame);
+
+ret = decode_subtitle2_priv(avctx, &subtitle, &got_sub_ptr, avpkt);
+
+if (ret >= 0 && got_sub_ptr) {
+frame->type = AVMEDIA_TYPE_SUBTITLE;
+frame->format = subtitle.format;
+ret = av_frame_get_buffer2(frame, 0);
+
+if (ret >= 0)
+ret = ff_frame_put_subtitle(frame, &subtitle);
+
+frame->width = avctx->width;
+frame->height = avctx->height;
+frame->pkt_dts = avpkt->dts;
+}
+
+avsubtitle_free(&subtitle);
+
+return ret;
+}
+
int attribute_align_arg avcodec_send_packet(AVCodecContext *avctx, const
AVPacket *avpkt)
{
AVCodecInternal *avci = avctx->internal;
@@ -59
[FFmpeg-devel] [PATCH v3 09/26] avfilter/avfilter: Handle subtitle frames
From: softworkz
Signed-off-by: softworkz
---
libavfilter/avfilter.c | 8 +---
libavfilter/avfilter.h | 11 +++
libavfilter/avfiltergraph.c | 5 +
libavfilter/formats.c | 22 ++
libavfilter/formats.h | 3 +++
libavfilter/internal.h | 18 +++---
6 files changed, 61 insertions(+), 6 deletions(-)
diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
index df5b8f483c..75d5e86539 100644
--- a/libavfilter/avfilter.c
+++ b/libavfilter/avfilter.c
@@ -56,7 +56,8 @@ static void tlog_ref(void *ctx, AVFrame *ref, int end)
ref->linesize[0], ref->linesize[1], ref->linesize[2],
ref->linesize[3],
ref->pts, ref->pkt_pos);
-if (ref->width) {
+switch(ref->type) {
+case AVMEDIA_TYPE_VIDEO:
ff_tlog(ctx, " a:%d/%d s:%dx%d i:%c iskey:%d type:%c",
ref->sample_aspect_ratio.num, ref->sample_aspect_ratio.den,
ref->width, ref->height,
@@ -64,12 +65,13 @@ static void tlog_ref(void *ctx, AVFrame *ref, int end)
ref->top_field_first ? 'T' : 'B',/* Top / Bottom */
ref->key_frame,
av_get_picture_type_char(ref->pict_type));
-}
-if (ref->nb_samples) {
+break;
+case AVMEDIA_TYPE_AUDIO:
ff_tlog(ctx, " cl:%"PRId64"d n:%d r:%d",
ref->channel_layout,
ref->nb_samples,
ref->sample_rate);
+break;
}
ff_tlog(ctx, "]%s", end ? "\n" : "");
diff --git a/libavfilter/avfilter.h b/libavfilter/avfilter.h
index b105dc3159..9f917deb41 100644
--- a/libavfilter/avfilter.h
+++ b/libavfilter/avfilter.h
@@ -45,6 +45,7 @@
#include "libavutil/log.h"
#include "libavutil/samplefmt.h"
#include "libavutil/pixfmt.h"
+#include "libavutil/subfmt.h"
#include "libavutil/rational.h"
#include "libavfilter/version.h"
@@ -343,6 +344,12 @@ typedef struct AVFilter {
* and outputs use the same sample rate and channel count/layout.
*/
const enum AVSampleFormat *samples_list;
+/**
+ * Analogous to pixels, but delimited by AV_SUBTITLE_FMT_NONE
+ * and restricted to filters that only have AVMEDIA_TYPE_SUBTITLE
+ * inputs and outputs.
+ */
+const enum AVSubtitleType *subs_list;
/**
* Equivalent to { pix_fmt, AV_PIX_FMT_NONE } as pixels_list.
*/
@@ -351,6 +358,10 @@ typedef struct AVFilter {
* Equivalent to { sample_fmt, AV_SAMPLE_FMT_NONE } as samples_list.
*/
enum AVSampleFormat sample_fmt;
+/**
+ * Equivalent to { sub_fmt, AV_SUBTITLE_FMT_NONE } as subs_list.
+ */
+enum AVSubtitleType sub_fmt;
} formats;
int priv_size; ///< size of private data to allocate for the filter
diff --git a/libavfilter/avfiltergraph.c b/libavfilter/avfiltergraph.c
index b8b432e98b..f4987654af 100644
--- a/libavfilter/avfiltergraph.c
+++ b/libavfilter/avfiltergraph.c
@@ -311,6 +311,8 @@ static int filter_link_check_formats(void *log,
AVFilterLink *link, AVFilterForm
return ret;
break;
+case AVMEDIA_TYPE_SUBTITLE:
+return 0;
default:
av_assert0(!"reached");
}
@@ -441,6 +443,9 @@ static int query_formats(AVFilterGraph *graph, void
*log_ctx)
if (!link)
continue;
+if (link->type == AVMEDIA_TYPE_SUBTITLE)
+continue;
+
neg = ff_filter_get_negotiation(link);
av_assert0(neg);
for (neg_step = 1; neg_step < neg->nb_mergers; neg_step++) {
diff --git a/libavfilter/formats.c b/libavfilter/formats.c
index ba62f73248..5c972bb183 100644
--- a/libavfilter/formats.c
+++ b/libavfilter/formats.c
@@ -19,6 +19,7 @@
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
+#include "libavutil/subfmt.h"
#include "libavutil/avassert.h"
#include "libavutil/channel_layout.h"
#include "libavutil/common.h"
@@ -431,6 +432,12 @@ int ff_add_channel_layout(AVFilterChannelLayouts **l,
uint64_t channel_layout)
return 0;
}
+int ff_add_subtitle_type(AVFilterFormats **avff, int64_t fmt)
+{
+ADD_FORMAT(avff, fmt, ff_formats_unref, int, formats, nb_formats);
+return 0;
+}
+
AVFilterFormats *ff_make_formats_list_singleton(int fmt)
{
int fmts[2] = { fmt, -1 };
@@ -450,6 +457,13 @@ AVFilterFormats *ff_all_formats(enum AVMediaType type)
return NULL;
fmt++;
}
+} else if (type == AVMEDIA_TYPE_SUBTITLE) {
+if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_BITMAP) < 0)
+return NULL;
+if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_ASS) < 0)
+return NULL;
+if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_TEXT) < 0)
+return NULL;
}
return ret;
@@ -724,6 +738,10 @@ int ff_default_query_formats(AVFilterContext *ctx)
type=
[FFmpeg-devel] [PATCH v3 04/26] avfilter/subtitles: Update vf_subtitles to use new decoding api
From: softworkz
Signed-off-by: softworkz
---
libavfilter/vf_subtitles.c | 56 +-
1 file changed, 43 insertions(+), 13 deletions(-)
diff --git a/libavfilter/vf_subtitles.c b/libavfilter/vf_subtitles.c
index 3fc4eeb63d..25e217e845 100644
--- a/libavfilter/vf_subtitles.c
+++ b/libavfilter/vf_subtitles.c
@@ -35,14 +35,12 @@
# include "libavformat/avformat.h"
#endif
#include "libavutil/avstring.h"
-#include "libavutil/imgutils.h"
#include "libavutil/opt.h"
#include "libavutil/parseutils.h"
#include "drawutils.h"
#include "avfilter.h"
#include "internal.h"
#include "formats.h"
-#include "video.h"
typedef struct AssContext {
const AVClass *class;
@@ -292,6 +290,29 @@ static int attachment_is_font(AVStream * st)
return 0;
}
+static int decode(AVCodecContext *avctx, AVFrame *frame, int *got_frame,
AVPacket *pkt)
+{
+int ret;
+
+*got_frame = 0;
+
+if (pkt) {
+ret = avcodec_send_packet(avctx, pkt);
+// In particular, we don't expect AVERROR(EAGAIN), because we read all
+// decoded frames with avcodec_receive_frame() until done.
+if (ret < 0 && ret != AVERROR_EOF)
+return ret;
+}
+
+ret = avcodec_receive_frame(avctx, frame);
+if (ret < 0 && ret != AVERROR(EAGAIN))
+return ret;
+if (ret >= 0)
+*got_frame = 1;
+
+return 0;
+}
+
AVFILTER_DEFINE_CLASS(subtitles);
static av_cold int init_subtitles(AVFilterContext *ctx)
@@ -306,6 +327,7 @@ static av_cold int init_subtitles(AVFilterContext *ctx)
AVStream *st;
AVPacket pkt;
AssContext *ass = ctx->priv;
+enum AVSubtitleType subtitle_format;
/* Init libass */
ret = init(ctx);
@@ -386,13 +408,17 @@ static av_cold int init_subtitles(AVFilterContext *ctx)
ret = AVERROR_DECODER_NOT_FOUND;
goto end;
}
+
dec_desc = avcodec_descriptor_get(st->codecpar->codec_id);
-if (dec_desc && !(dec_desc->props & AV_CODEC_PROP_TEXT_SUB)) {
+subtitle_format = avcodec_descriptor_get_subtitle_format(dec_desc);
+
+if (subtitle_format != AV_SUBTITLE_FMT_ASS) {
av_log(ctx, AV_LOG_ERROR,
- "Only text based subtitles are currently supported\n");
-ret = AVERROR_PATCHWELCOME;
+ "Only text based subtitles are supported by this filter\n");
+ret = AVERROR_INVALIDDATA;
goto end;
}
+
if (ass->charenc)
av_dict_set(&codec_opts, "sub_charenc", ass->charenc, 0);
@@ -448,27 +474,31 @@ static av_cold int init_subtitles(AVFilterContext *ctx)
dec_ctx->subtitle_header_size);
while (av_read_frame(fmt, &pkt) >= 0) {
int i, got_subtitle;
-AVSubtitle sub = {0};
+AVFrame *sub = av_frame_alloc();
+if (!sub) {
+ret = AVERROR(ENOMEM);
+goto end;
+}
if (pkt.stream_index == sid) {
-ret = avcodec_decode_subtitle2(dec_ctx, &sub, &got_subtitle, &pkt);
+ret = decode(dec_ctx, sub, &got_subtitle, &pkt);
if (ret < 0) {
av_log(ctx, AV_LOG_WARNING, "Error decoding: %s (ignored)\n",
av_err2str(ret));
} else if (got_subtitle) {
-const int64_t start_time = av_rescale_q(sub.pts,
AV_TIME_BASE_Q, av_make_q(1, 1000));
-const int64_t duration = sub.end_display_time;
-for (i = 0; i < sub.num_rects; i++) {
-char *ass_line = sub.rects[i]->ass;
+const int64_t start_time =
av_rescale_q(sub->subtitle_timing.start_pts, AV_TIME_BASE_Q, av_make_q(1,
1000));
+const int64_t duration =
av_rescale_q(sub->subtitle_timing.duration, AV_TIME_BASE_Q, av_make_q(1, 1000));
+for (i = 0; i < sub->num_subtitle_areas; i++) {
+char *ass_line = sub->subtitle_areas[i]->ass;
if (!ass_line)
-break;
+continue;
ass_process_chunk(ass->track, ass_line, strlen(ass_line),
start_time, duration);
}
}
}
av_packet_unref(&pkt);
-avsubtitle_free(&sub);
+av_frame_free(&sub);
}
end:
--
ffmpeg-codebot
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v3 10/26] avfilter/avfilter: Fix hardcoded input index
From: softworkz
This fix targets (rare) cases where multiple input pads have a
.filter_frame function. ff_request_frame_to_filter needs
to call ff_request_frame with the correct input pad
instead of the hardcoded first one.
Signed-off-by: softworkz
---
libavfilter/avfilter.c | 18 +-
1 file changed, 13 insertions(+), 5 deletions(-)
diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
index 75d5e86539..aa9aa71f53 100644
--- a/libavfilter/avfilter.c
+++ b/libavfilter/avfilter.c
@@ -463,7 +463,7 @@ static int64_t guess_status_pts(AVFilterContext *ctx, int
status, AVRational lin
return AV_NOPTS_VALUE;
}
-static int ff_request_frame_to_filter(AVFilterLink *link)
+static int ff_request_frame_to_filter(AVFilterLink *link, int input_index)
{
int ret = -1;
@@ -472,8 +472,8 @@ static int ff_request_frame_to_filter(AVFilterLink *link)
link->frame_blocked_in = 1;
if (link->srcpad->request_frame)
ret = link->srcpad->request_frame(link);
-else if (link->src->inputs[0])
-ret = ff_request_frame(link->src->inputs[0]);
+else if (link->src->inputs[input_index])
+ret = ff_request_frame(link->src->inputs[input_index]);
if (ret < 0) {
if (ret != AVERROR(EAGAIN) && ret != link->status_in)
ff_avfilter_link_set_in_status(link, ret,
guess_status_pts(link->src, ret, link->time_base));
@@ -1172,6 +1172,14 @@ static int forward_status_change(AVFilterContext
*filter, AVFilterLink *in)
{
unsigned out = 0, progress = 0;
int ret;
+int input_index = 0;
+
+for (int i = 0; i < in->dst->nb_inputs; i++) {
+if (&in->dst->input_pads[i] == in->dstpad) {
+input_index = i;
+break;
+}
+}
av_assert0(!in->status_out);
if (!filter->nb_outputs) {
@@ -1181,7 +1189,7 @@ static int forward_status_change(AVFilterContext *filter,
AVFilterLink *in)
while (!in->status_out) {
if (!filter->outputs[out]->status_in) {
progress++;
-ret = ff_request_frame_to_filter(filter->outputs[out]);
+ret = ff_request_frame_to_filter(filter->outputs[out],
input_index);
if (ret < 0)
return ret;
}
@@ -1218,7 +1226,7 @@ static int ff_filter_activate_default(AVFilterContext
*filter)
for (i = 0; i < filter->nb_outputs; i++) {
if (filter->outputs[i]->frame_wanted_out &&
!filter->outputs[i]->frame_blocked_in) {
-return ff_request_frame_to_filter(filter->outputs[i]);
+return ff_request_frame_to_filter(filter->outputs[i], 0);
}
}
return FFERROR_NOT_READY;
--
ffmpeg-codebot
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v3 05/26] avcodec, avutil: Move ass helper functions to avutil as avpriv_ and extend ass dialog parsing
From: softworkz
Also add
- hard_space callback (for upcoming fix)
- extensible callback (for future extension)
Signed-off-by: softworkz
---
libavcodec/Makefile | 56 +++
libavcodec/ass.h | 144 ++
libavcodec/assdec.c | 2 +-
libavcodec/assenc.c | 2 +-
libavcodec/ccaption_dec.c | 19 +--
libavcodec/jacosubdec.c | 2 +-
libavcodec/libaribb24.c | 2 +-
libavcodec/libzvbi-teletextdec.c | 14 +-
libavcodec/microdvddec.c | 7 +-
libavcodec/movtextdec.c | 3 +-
libavcodec/movtextenc.c | 20 +--
libavcodec/mpl2dec.c | 2 +-
libavcodec/realtextdec.c | 2 +-
libavcodec/samidec.c | 2 +-
libavcodec/srtdec.c | 2 +-
libavcodec/srtenc.c | 16 +-
libavcodec/subviewerdec.c | 2 +-
libavcodec/textdec.c | 4 +-
libavcodec/ttmlenc.c | 15 +-
libavcodec/webvttdec.c| 2 +-
libavcodec/webvttenc.c| 16 +-
libavutil/Makefile| 2 +
{libavcodec => libavutil}/ass.c | 91 +--
libavutil/ass_internal.h | 135
{libavcodec => libavutil}/ass_split.c | 30 ++--
.../ass_split_internal.h | 32 ++--
26 files changed, 355 insertions(+), 269 deletions(-)
rename {libavcodec => libavutil}/ass.c (65%)
create mode 100644 libavutil/ass_internal.h
rename {libavcodec => libavutil}/ass_split.c (94%)
rename libavcodec/ass_split.h => libavutil/ass_split_internal.h (86%)
diff --git a/libavcodec/Makefile b/libavcodec/Makefile
index cfc70a3eaf..80bf8ff2d2 100644
--- a/libavcodec/Makefile
+++ b/libavcodec/Makefile
@@ -215,10 +215,10 @@ OBJS-$(CONFIG_APNG_DECODER)+= png.o pngdec.o
pngdsp.o
OBJS-$(CONFIG_APNG_ENCODER)+= png.o pngenc.o
OBJS-$(CONFIG_ARBC_DECODER)+= arbc.o
OBJS-$(CONFIG_ARGO_DECODER)+= argo.o
-OBJS-$(CONFIG_SSA_DECODER) += assdec.o ass.o
-OBJS-$(CONFIG_SSA_ENCODER) += assenc.o ass.o
-OBJS-$(CONFIG_ASS_DECODER) += assdec.o ass.o
-OBJS-$(CONFIG_ASS_ENCODER) += assenc.o ass.o
+OBJS-$(CONFIG_SSA_DECODER) += assdec.o
+OBJS-$(CONFIG_SSA_ENCODER) += assenc.o
+OBJS-$(CONFIG_ASS_DECODER) += assdec.o
+OBJS-$(CONFIG_ASS_ENCODER) += assenc.o
OBJS-$(CONFIG_ASV1_DECODER)+= asvdec.o asv.o mpeg12data.o
OBJS-$(CONFIG_ASV1_ENCODER)+= asvenc.o asv.o mpeg12data.o
OBJS-$(CONFIG_ASV2_DECODER)+= asvdec.o asv.o mpeg12data.o
@@ -259,7 +259,7 @@ OBJS-$(CONFIG_BRENDER_PIX_DECODER) += brenderpix.o
OBJS-$(CONFIG_C93_DECODER) += c93.o
OBJS-$(CONFIG_CAVS_DECODER)+= cavs.o cavsdec.o cavsdsp.o \
cavsdata.o
-OBJS-$(CONFIG_CCAPTION_DECODER)+= ccaption_dec.o ass.o
+OBJS-$(CONFIG_CCAPTION_DECODER)+= ccaption_dec.o
OBJS-$(CONFIG_CDGRAPHICS_DECODER) += cdgraphics.o
OBJS-$(CONFIG_CDTOONS_DECODER) += cdtoons.o
OBJS-$(CONFIG_CDXL_DECODER)+= cdxl.o
@@ -434,7 +434,7 @@ OBJS-$(CONFIG_INTERPLAY_ACM_DECODER) += interplayacm.o
OBJS-$(CONFIG_INTERPLAY_DPCM_DECODER) += dpcm.o
OBJS-$(CONFIG_INTERPLAY_VIDEO_DECODER) += interplayvideo.o
OBJS-$(CONFIG_IPU_DECODER) += mpeg12dec.o mpeg12.o mpeg12data.o
-OBJS-$(CONFIG_JACOSUB_DECODER) += jacosubdec.o ass.o
+OBJS-$(CONFIG_JACOSUB_DECODER) += jacosubdec.o
OBJS-$(CONFIG_JPEG2000_ENCODER)+= j2kenc.o mqcenc.o mqc.o jpeg2000.o \
jpeg2000dwt.o
OBJS-$(CONFIG_JPEG2000_DECODER)+= jpeg2000dec.o jpeg2000.o
jpeg2000dsp.o \
@@ -456,7 +456,7 @@ OBJS-$(CONFIG_MAGICYUV_ENCODER)+= magicyuvenc.o
OBJS-$(CONFIG_MDEC_DECODER)+= mdec.o mpeg12.o mpeg12data.o
OBJS-$(CONFIG_METASOUND_DECODER) += metasound.o metasound_data.o \
twinvq.o
-OBJS-$(CONFIG_MICRODVD_DECODER)+= microdvddec.o ass.o
+OBJS-$(CONFIG_MICRODVD_DECODER)+= microdvddec.o
OBJS-$(CONFIG_MIMIC_DECODER) += mimic.o
OBJS-$(CONFIG_MJPEG_DECODER) += mjpegdec.o mjpegdec_common.o
OBJS-$(CONFIG_MJPEG_QSV_DECODER) += qsvdec.o
@@ -471,8 +471,8 @@ OBJS-$(CONFIG_MLP_ENCODER) += mlpenc.o mlp.o
OBJS-$(CONFIG_MMVIDEO_DECODER) += mmvideo.o
OBJS-$(CONFIG_MOBICLIP_DECODER)+= mobiclip.o
OBJS-$(CONFIG_MOTIONPIXELS_DECODER)+= motionpixels.o
-OBJS-$(CONFIG_MOVTEXT_DECODER) += movtextdec.o ass.o
-OBJS-$(CONFIG_MOVTEXT_ENCOD
[FFmpeg-devel] [PATCH v3 06/26] avcodec/subtitles: Replace deprecated enum values
From: softworkz
Signed-off-by: softworkz
---
libavcodec/ass.h | 2 +-
libavcodec/assdec.c| 2 +-
libavcodec/dvbsubdec.c | 2 +-
libavcodec/dvdsubdec.c | 2 +-
libavcodec/dvdsubenc.c | 2 +-
libavcodec/pgssubdec.c | 2 +-
libavcodec/xsubdec.c | 2 +-
7 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/libavcodec/ass.h b/libavcodec/ass.h
index 8bc13d7ab8..43c5ad651a 100644
--- a/libavcodec/ass.h
+++ b/libavcodec/ass.h
@@ -83,7 +83,7 @@ static inline int avpriv_ass_add_rect(AVSubtitle *sub, const
char *dialog,
rects[sub->num_rects] = av_mallocz(sizeof(*rects[0]));
if (!rects[sub->num_rects])
return AVERROR(ENOMEM);
-rects[sub->num_rects]->type = SUBTITLE_ASS;
+rects[sub->num_rects]->type = AV_SUBTITLE_FMT_ASS;
ass_str = avpriv_ass_get_dialog(readorder, layer, style, speaker, dialog);
if (!ass_str)
return AVERROR(ENOMEM);
diff --git a/libavcodec/assdec.c b/libavcodec/assdec.c
index 7802a44e71..fd321e7004 100644
--- a/libavcodec/assdec.c
+++ b/libavcodec/assdec.c
@@ -54,7 +54,7 @@ static int ass_decode_frame(AVCodecContext *avctx, void
*data, int *got_sub_ptr,
if (!sub->rects[0])
return AVERROR(ENOMEM);
sub->num_rects = 1;
-sub->rects[0]->type = SUBTITLE_ASS;
+sub->rects[0]->type = AV_SUBTITLE_FMT_ASS;
sub->rects[0]->ass = av_strdup(avpkt->data);
if (!sub->rects[0]->ass)
return AVERROR(ENOMEM);
diff --git a/libavcodec/dvbsubdec.c b/libavcodec/dvbsubdec.c
index bc741a1de6..0d64c6e71c 100644
--- a/libavcodec/dvbsubdec.c
+++ b/libavcodec/dvbsubdec.c
@@ -795,7 +795,7 @@ static int save_subtitle_set(AVCodecContext *avctx,
AVSubtitle *sub, int *got_ou
rect->w = region->width;
rect->h = region->height;
rect->nb_colors = (1 << region->depth);
-rect->type = SUBTITLE_BITMAP;
+rect->type = AV_SUBTITLE_FMT_BITMAP;
rect->linesize[0] = region->width;
clut = get_clut(ctx, region->clut);
diff --git a/libavcodec/dvdsubdec.c b/libavcodec/dvdsubdec.c
index 52259f0730..b39b3d1838 100644
--- a/libavcodec/dvdsubdec.c
+++ b/libavcodec/dvdsubdec.c
@@ -406,7 +406,7 @@ static int decode_dvd_subtitles(DVDSubContext *ctx,
AVSubtitle *sub_header,
sub_header->rects[0]->y = y1;
sub_header->rects[0]->w = w;
sub_header->rects[0]->h = h;
-sub_header->rects[0]->type = SUBTITLE_BITMAP;
+sub_header->rects[0]->type = AV_SUBTITLE_FMT_BITMAP;
sub_header->rects[0]->linesize[0] = w;
sub_header->rects[0]->flags = is_menu ?
AV_SUBTITLE_FLAG_FORCED : 0;
}
diff --git a/libavcodec/dvdsubenc.c b/libavcodec/dvdsubenc.c
index ff4fbed39d..943a7466d9 100644
--- a/libavcodec/dvdsubenc.c
+++ b/libavcodec/dvdsubenc.c
@@ -268,7 +268,7 @@ static int encode_dvd_subtitles(AVCodecContext *avctx,
if (rects == 0 || !h->rects)
return AVERROR(EINVAL);
for (i = 0; i < rects; i++)
-if (h->rects[i]->type != SUBTITLE_BITMAP) {
+if (h->rects[i]->type != AV_SUBTITLE_FMT_BITMAP) {
av_log(avctx, AV_LOG_ERROR, "Bitmap subtitle required\n");
return AVERROR(EINVAL);
}
diff --git a/libavcodec/pgssubdec.c b/libavcodec/pgssubdec.c
index bdd20c914b..22b6616f9b 100644
--- a/libavcodec/pgssubdec.c
+++ b/libavcodec/pgssubdec.c
@@ -535,7 +535,7 @@ static int display_end_segment(AVCodecContext *avctx, void
*data,
if (!rect)
return AVERROR(ENOMEM);
sub->rects[sub->num_rects++] = rect;
-rect->type = SUBTITLE_BITMAP;
+rect->type = AV_SUBTITLE_FMT_BITMAP;
/* Process bitmap */
object = find_object(ctx->presentation.objects[i].id, &ctx->objects);
diff --git a/libavcodec/xsubdec.c b/libavcodec/xsubdec.c
index 85cd7d1c20..a4be18a1d8 100644
--- a/libavcodec/xsubdec.c
+++ b/libavcodec/xsubdec.c
@@ -107,7 +107,7 @@ static int decode_frame(AVCodecContext *avctx, void *data,
int *got_sub_ptr,
sub->num_rects = 1;
rect->x = x; rect->y = y;
rect->w = w; rect->h = h;
-rect->type = SUBTITLE_BITMAP;
+rect->type = AV_SUBTITLE_FMT_BITMAP;
rect->linesize[0] = w;
rect->data[0] = av_malloc(w * h);
rect->nb_colors = 4;
--
ffmpeg-codebot
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v3 11/26] avfilter/sbuffer: Add sbuffersrc and sbuffersink filters
From: softworkz
Signed-off-by: softworkz
---
configure| 2 +-
libavfilter/allfilters.c | 2 ++
libavfilter/buffersink.c | 54 ++
libavfilter/buffersink.h | 7
libavfilter/buffersrc.c | 72
libavfilter/buffersrc.h | 1 +
6 files changed, 137 insertions(+), 1 deletion(-)
diff --git a/configure b/configure
index 52ef214b73..1928a77344 100755
--- a/configure
+++ b/configure
@@ -7856,7 +7856,7 @@ print_enabled_components(){
fi
done
if [ "$name" = "filter_list" ]; then
-for c in asrc_abuffer vsrc_buffer asink_abuffer vsink_buffer; do
+for c in asrc_abuffer vsrc_buffer ssrc_sbuffer asink_abuffer
vsink_buffer ssink_sbuffer; do
printf "&ff_%s,\n" $c >> $TMPH
done
fi
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 714468afce..ebb2e374bd 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -557,8 +557,10 @@ extern const AVFilter ff_avsrc_movie;
* being the same while having different 'types'). */
extern const AVFilter ff_asrc_abuffer;
extern const AVFilter ff_vsrc_buffer;
+extern const AVFilter ff_ssrc_sbuffer;
extern const AVFilter ff_asink_abuffer;
extern const AVFilter ff_vsink_buffer;
+extern const AVFilter ff_ssink_sbuffer;
extern const AVFilter ff_af_afifo;
extern const AVFilter ff_vf_fifo;
diff --git a/libavfilter/buffersink.c b/libavfilter/buffersink.c
index c0215669e7..0b268c2fa4 100644
--- a/libavfilter/buffersink.c
+++ b/libavfilter/buffersink.c
@@ -29,6 +29,8 @@
#include "libavutil/internal.h"
#include "libavutil/opt.h"
+#include "libavcodec/avcodec.h"
+
#define FF_INTERNAL_FIELDS 1
#include "framequeue.h"
@@ -57,6 +59,10 @@ typedef struct BufferSinkContext {
int *sample_rates; ///< list of accepted sample rates
int sample_rates_size;
+/* only used for subtitles */
+enum AVSubtitleType *subtitle_types; ///< list of accepted subtitle
types, must be terminated with -1
+int subtitle_types_size;
+
AVFrame *peeked_frame;
} BufferSinkContext;
@@ -305,6 +311,28 @@ static int asink_query_formats(AVFilterContext *ctx)
return 0;
}
+static int ssink_query_formats(AVFilterContext *ctx)
+{
+BufferSinkContext *buf = ctx->priv;
+AVFilterFormats *formats = NULL;
+unsigned i;
+int ret;
+
+CHECK_LIST_SIZE(subtitle_types)
+if (buf->subtitle_types_size) {
+for (i = 0; i < NB_ITEMS(buf->subtitle_types); i++)
+if ((ret = ff_add_subtitle_type(&formats, buf->subtitle_types[i]))
< 0)
+return ret;
+if ((ret = ff_set_common_formats(ctx, formats)) < 0)
+return ret;
+} else {
+if ((ret = ff_default_query_formats(ctx)) < 0)
+return ret;
+}
+
+return 0;
+}
+
#define OFFSET(x) offsetof(BufferSinkContext, x)
#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM
static const AVOption buffersink_options[] = {
@@ -322,9 +350,16 @@ static const AVOption abuffersink_options[] = {
{ NULL },
};
#undef FLAGS
+#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_SUBTITLE_PARAM
+static const AVOption sbuffersink_options[] = {
+{ "subtitle_types", "set the supported subtitle formats",
OFFSET(subtitle_types), AV_OPT_TYPE_BINARY, .flags = FLAGS },
+{ NULL },
+};
+#undef FLAGS
AVFILTER_DEFINE_CLASS(buffersink);
AVFILTER_DEFINE_CLASS(abuffersink);
+AVFILTER_DEFINE_CLASS(sbuffersink);
static const AVFilterPad avfilter_vsink_buffer_inputs[] = {
{
@@ -363,3 +398,22 @@ const AVFilter ff_asink_abuffer = {
.outputs = NULL,
FILTER_QUERY_FUNC(asink_query_formats),
};
+
+static const AVFilterPad avfilter_ssink_sbuffer_inputs[] = {
+{
+.name = "default",
+.type = AVMEDIA_TYPE_SUBTITLE,
+},
+};
+
+const AVFilter ff_ssink_sbuffer = {
+.name = "sbuffersink",
+.description = NULL_IF_CONFIG_SMALL("Buffer subtitle frames, and make
them available to the end of the filter graph."),
+.priv_class= &sbuffersink_class,
+.priv_size = sizeof(BufferSinkContext),
+.init = common_init,
+.activate = activate,
+FILTER_INPUTS(avfilter_ssink_sbuffer_inputs),
+.outputs = NULL,
+FILTER_QUERY_FUNC(ssink_query_formats),
+};
diff --git a/libavfilter/buffersink.h b/libavfilter/buffersink.h
index 69ed0f29a8..11905abdc5 100644
--- a/libavfilter/buffersink.h
+++ b/libavfilter/buffersink.h
@@ -129,6 +129,13 @@ typedef struct AVABufferSinkParams {
*/
attribute_deprecated
AVABufferSinkParams *av_abuffersink_params_alloc(void);
+
+/**
+ * Deprecated and unused struct to use for initializing an sbuffersink context.
+ */
+typedef struct AVSBufferSinkParams {
+const int *subtitle_type;
+} AVSBufferSinkParams;
#endif
/**
diff --git a/libavfilter/buffersrc.c b/libavfilter/buffersrc.c
index b0611872f1..d2362999a2 100644
--- a/liba
[FFmpeg-devel] [PATCH v3 07/26] fftools/play, probe: Adjust for subtitle changes
From: softworkz
Signed-off-by: softworkz
---
fftools/ffplay.c | 102 +-
fftools/ffprobe.c | 47 +
2 files changed, 77 insertions(+), 72 deletions(-)
diff --git a/fftools/ffplay.c b/fftools/ffplay.c
index e7b20be76b..94286eb678 100644
--- a/fftools/ffplay.c
+++ b/fftools/ffplay.c
@@ -152,7 +152,6 @@ typedef struct Clock {
/* Common struct for handling all types of decoded data and allocated render
buffers. */
typedef struct Frame {
AVFrame *frame;
-AVSubtitle sub;
int serial;
double pts; /* presentation timestamp for the frame */
double duration; /* estimated duration of the frame */
@@ -586,7 +585,7 @@ static int decoder_init(Decoder *d, AVCodecContext *avctx,
PacketQueue *queue, S
return 0;
}
-static int decoder_decode_frame(Decoder *d, AVFrame *frame, AVSubtitle *sub) {
+static int decoder_decode_frame(Decoder *d, AVFrame *frame) {
int ret = AVERROR(EAGAIN);
for (;;) {
@@ -620,6 +619,9 @@ static int decoder_decode_frame(Decoder *d, AVFrame *frame,
AVSubtitle *sub) {
}
}
break;
+case AVMEDIA_TYPE_SUBTITLE:
+ret = avcodec_receive_frame(d->avctx, frame);
+break;
}
if (ret == AVERROR_EOF) {
d->finished = d->pkt_serial;
@@ -652,25 +654,11 @@ static int decoder_decode_frame(Decoder *d, AVFrame
*frame, AVSubtitle *sub) {
av_packet_unref(d->pkt);
} while (1);
-if (d->avctx->codec_type == AVMEDIA_TYPE_SUBTITLE) {
-int got_frame = 0;
-ret = avcodec_decode_subtitle2(d->avctx, sub, &got_frame, d->pkt);
-if (ret < 0) {
-ret = AVERROR(EAGAIN);
-} else {
-if (got_frame && !d->pkt->data) {
-d->packet_pending = 1;
-}
-ret = got_frame ? 0 : (d->pkt->data ? AVERROR(EAGAIN) :
AVERROR_EOF);
-}
-av_packet_unref(d->pkt);
+if (avcodec_send_packet(d->avctx, d->pkt) == AVERROR(EAGAIN)) {
+av_log(d->avctx, AV_LOG_ERROR, "Receive_frame and send_packet both
returned EAGAIN, which is an API violation.\n");
+d->packet_pending = 1;
} else {
-if (avcodec_send_packet(d->avctx, d->pkt) == AVERROR(EAGAIN)) {
-av_log(d->avctx, AV_LOG_ERROR, "Receive_frame and send_packet
both returned EAGAIN, which is an API violation.\n");
-d->packet_pending = 1;
-} else {
-av_packet_unref(d->pkt);
-}
+av_packet_unref(d->pkt);
}
}
}
@@ -683,7 +671,6 @@ static void decoder_destroy(Decoder *d) {
static void frame_queue_unref_item(Frame *vp)
{
av_frame_unref(vp->frame);
-avsubtitle_free(&vp->sub);
}
static int frame_queue_init(FrameQueue *f, PacketQueue *pktq, int max_size,
int keep_last)
@@ -981,7 +968,7 @@ static void video_image_display(VideoState *is)
if (frame_queue_nb_remaining(&is->subpq) > 0) {
sp = frame_queue_peek(&is->subpq);
-if (vp->pts >= sp->pts + ((float) sp->sub.start_display_time /
1000)) {
+if (vp->pts >= sp->pts) {
if (!sp->uploaded) {
uint8_t* pixels[4];
int pitch[4];
@@ -993,25 +980,27 @@ static void video_image_display(VideoState *is)
if (realloc_texture(&is->sub_texture,
SDL_PIXELFORMAT_ARGB, sp->width, sp->height, SDL_BLENDMODE_BLEND, 1) < 0)
return;
-for (i = 0; i < sp->sub.num_rects; i++) {
-AVSubtitleRect *sub_rect = sp->sub.rects[i];
+for (i = 0; i < sp->frame->num_subtitle_areas; i++) {
+AVSubtitleArea *area = sp->frame->subtitle_areas[i];
+SDL_Rect sdl_rect = { .x = area->x, .y = area->y, .w =
area->w, .h = area->h };
-sub_rect->x = av_clip(sub_rect->x, 0, sp->width );
-sub_rect->y = av_clip(sub_rect->y, 0, sp->height);
-sub_rect->w = av_clip(sub_rect->w, 0, sp->width -
sub_rect->x);
-sub_rect->h = av_clip(sub_rect->h, 0, sp->height -
sub_rect->y);
+area->x = av_clip(area->x, 0, sp->width );
+area->y = av_clip(area->y, 0, sp->height);
+area->w = av_clip(area->w, 0, sp->width - area->x);
+area->h = av_clip(area->h, 0, sp->height - area->y);
is->sub_convert_ctx =
sws_getCachedContext(is->sub_convert_ctx,
-sub_rect->w, sub_rect->h, AV_PIX_FMT_PAL8,
-sub_rect->w, sub_rect->h, AV_PIX_FMT_BGRA,
+
[FFmpeg-devel] [PATCH v3 12/26] avfilter/overlaygraphicsubs: Add overlaygraphicsubs and graphicsub2video filters
From: softworkz
- overlaygraphicsubs (VS -> V)
Overlay graphic subtitles onto a video stream
- graphicsub2video {S -> V)
Converts graphic subtitles to video frames (with alpha)
Gets auto-inserted for retaining compatibility with
sub2video command lines
Signed-off-by: softworkz
---
doc/filters.texi| 118 +
libavfilter/Makefile| 2 +
libavfilter/allfilters.c| 2 +
libavfilter/vf_overlaygraphicsubs.c | 765
4 files changed, 887 insertions(+)
create mode 100644 libavfilter/vf_overlaygraphicsubs.c
diff --git a/doc/filters.texi b/doc/filters.texi
index 248c09caf8..2bd3737cf5 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25764,6 +25764,124 @@ tools.
@c man end VIDEO SINKS
+@chapter Subtitle Filters
+@c man begin SUBTITLE FILTERS
+
+When you configure your FFmpeg build, you can disable any of the
+existing filters using @code{--disable-filters}.
+
+Below is a description of the currently available subtitle filters.
+
+@section graphicsub2video
+
+Renders graphic subtitles as video frames.
+
+This filter replaces the previous "sub2video" hack which did the conversion
implicitly and up-front as subtitle filtering wasn't possible at that time.
+To retain compatibility with earlier sub2video command lines, this filter is
being auto-inserted in those cases.
+
+For overlaying graphicsal subtitles it is recommended to use the
'overlay_graphicsubs' filter which is more efficient and takes less processing
resources.
+
+This filter is still useful in cases where the overlay is done with hardware
acceleration (e.g. overlay_qsv, overlay_vaapi, overlay_cuda) for preparing the
overlay frames.
+
+Inputs:
+@itemize
+@item 0: Subtitles [BITMAP]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video [RGB32]
+@end itemize
+
+
+It accepts the following parameters:
+
+@table @option
+@item size, s
+Set the size of the output video frame.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Overlay PGS subtitles
+(not recommended - better use overlay_graphicsubs)
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:1]graphicsub2video[subs];[0:0][subs]overlay" output.mp4
+@end example
+
+@item
+Overlay PGS subtitles implicitly
+The graphicsub2video is inserted automatically for compatibility with legacy
command lines.
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:0][0:1]overlay" output.mp4
+@end example
+@end itemize
+
+@section overlaygraphicsubs
+
+Overlay graphic subtitles onto a video stream.
+
+This filter can blend graphical subtitles on a video stream directly, i.e.
without creating full-size alpha images first.
+The blending operation is limited to the area of the subtitle rectangles,
which also means that no processing is done at times where no subtitles are to
be displayed.
+
+Inputs:
+@itemize
+@item 0: Video [YUV420P, YUV422P, YUV444P, ARGB, RGBA, ABGR, BGRA, RGB24,
BGR24]
+@item 1: Subtitles [BITMAP]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video (same as input)
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+@item x
+@item y
+Set the expression for the x and y coordinates of the overlaid video
+on the main video. Default value is "0" for both expressions. In case
+the expression is invalid, it is set to a huge value (meaning that the
+overlay will not be displayed within the output visible area).
+
+@item eof_action
+See @ref{framesync}.
+
+@item eval
+Set when the expressions for @option{x}, and @option{y} are evaluated.
+
+It accepts the following values:
+@table @samp
+@item init
+only evaluate expressions once during the filter initialization or
+when a command is processed
+
+@item frame
+evaluate expressions for each incoming frame
+@end table
+
+Default value is @samp{frame}.
+
+@item shortest
+See @ref{framesync}.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Overlay PGS subtitles
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:0][0:1]overlaygraphicsubs" output.mp4
+@end example
+@end itemize
+@c man end SUBTITLE FILTERS
+
@chapter Multimedia Filters
@c man begin MULTIMEDIA FILTERS
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index f1176644c9..c326b8d32b 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -298,6 +298,7 @@ OBJS-$(CONFIG_GBLUR_FILTER) += vf_gblur.o
OBJS-$(CONFIG_GBLUR_VULKAN_FILTER) += vf_gblur_vulkan.o vulkan.o
vulkan_filter.o
OBJS-$(CONFIG_GEQ_FILTER)+= vf_geq.o
OBJS-$(CONFIG_GRADFUN_FILTER)+= vf_gradfun.o
+OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlaygraphicsubs.o
framesync.o
OBJS-$(CONFIG_GRAPHMONITOR_FILTER) += f_graphmonitor.o
OBJS-$(CONFIG_GRAYWORLD_FIL
[FFmpeg-devel] [PATCH v3 08/26] avfilter/subtitles: Add subtitles.c for subtitle frame allocation
From: softworkz
Analog to avfilter/video.c and avfilter/audio.c
Signed-off-by: softworkz
---
libavfilter/Makefile| 1 +
libavfilter/avfilter.c | 4 +++
libavfilter/internal.h | 1 +
libavfilter/subtitles.c | 63 +
libavfilter/subtitles.h | 44
5 files changed, 113 insertions(+)
create mode 100644 libavfilter/subtitles.c
create mode 100644 libavfilter/subtitles.h
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 282967144b..f1176644c9 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -19,6 +19,7 @@ OBJS = allfilters.o
\
framequeue.o \
graphdump.o \
graphparser.o\
+ subtitles.o \
video.o \
OBJS-$(HAVE_THREADS) += pthread.o
diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
index 7362bcdab5..df5b8f483c 100644
--- a/libavfilter/avfilter.c
+++ b/libavfilter/avfilter.c
@@ -43,6 +43,7 @@
#include "formats.h"
#include "framepool.h"
#include "internal.h"
+#include "subtitles.h"
#include "libavutil/ffversion.h"
const char av_filter_ffversion[] = "FFmpeg version " FFMPEG_VERSION;
@@ -1475,6 +1476,9 @@ int ff_inlink_make_frame_writable(AVFilterLink *link,
AVFrame **rframe)
case AVMEDIA_TYPE_AUDIO:
out = ff_get_audio_buffer(link, frame->nb_samples);
break;
+case AVMEDIA_TYPE_SUBTITLE:
+out = ff_get_subtitles_buffer(link, link->format);
+break;
default:
return AVERROR(EINVAL);
}
diff --git a/libavfilter/internal.h b/libavfilter/internal.h
index 1099b82b4b..fc09ef574c 100644
--- a/libavfilter/internal.h
+++ b/libavfilter/internal.h
@@ -90,6 +90,7 @@ struct AVFilterPad {
union {
AVFrame *(*video)(AVFilterLink *link, int w, int h);
AVFrame *(*audio)(AVFilterLink *link, int nb_samples);
+AVFrame *(*subtitle)(AVFilterLink *link, int format);
} get_buffer;
/**
diff --git a/libavfilter/subtitles.c b/libavfilter/subtitles.c
new file mode 100644
index 00..951bfd612c
--- /dev/null
+++ b/libavfilter/subtitles.c
@@ -0,0 +1,63 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "libavutil/common.h"
+
+#include "subtitles.h"
+#include "avfilter.h"
+#include "internal.h"
+
+
+AVFrame *ff_null_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+return ff_get_subtitles_buffer(link->dst->outputs[0], format);
+}
+
+AVFrame *ff_default_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+AVFrame *frame;
+
+frame = av_frame_alloc();
+if (!frame)
+return NULL;
+
+frame->format = format;
+frame->type = AVMEDIA_TYPE_SUBTITLE;
+
+if (av_frame_get_buffer2(frame, 0) < 0) {
+av_frame_free(&frame);
+return NULL;
+}
+
+return frame;
+}
+
+AVFrame *ff_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+AVFrame *ret = NULL;
+
+if (link->dstpad->get_buffer.subtitle)
+ret = link->dstpad->get_buffer.subtitle(link, format);
+
+if (!ret)
+ret = ff_default_get_subtitles_buffer(link, format);
+
+return ret;
+}
diff --git a/libavfilter/subtitles.h b/libavfilter/subtitles.h
new file mode 100644
index 00..4a9115126e
--- /dev/null
+++ b/libavfilter/subtitles.h
@@ -0,0 +1,44 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You
[FFmpeg-devel] [PATCH v3 13/26] avfilter/overlaytextsubs: Add overlaytextsubs and textsubs2video filters
From: softworkz
- overlaytextsubs {VS -> V)
Overlay text subtitles onto a video stream.
- textsubs2video {S -> V)
Converts text subtitles to video frames
Signed-off-by: softworkz
---
configure| 2 +
doc/filters.texi | 113 ++
libavfilter/Makefile | 2 +
libavfilter/allfilters.c | 6 +-
libavfilter/vf_overlaytextsubs.c | 678 +++
5 files changed, 799 insertions(+), 2 deletions(-)
create mode 100644 libavfilter/vf_overlaytextsubs.c
diff --git a/configure b/configure
index 1928a77344..7b66e315f2 100755
--- a/configure
+++ b/configure
@@ -3692,6 +3692,7 @@ overlay_qsv_filter_deps="libmfx"
overlay_qsv_filter_select="qsvvpp"
overlay_vaapi_filter_deps="vaapi VAProcPipelineCaps_blend_flags"
overlay_vulkan_filter_deps="vulkan spirv_compiler"
+overlaytextsubs_filter_deps="avcodec libass"
owdenoise_filter_deps="gpl"
pad_opencl_filter_deps="opencl"
pan_filter_deps="swresample"
@@ -3736,6 +3737,7 @@ superequalizer_filter_deps="avcodec"
superequalizer_filter_select="rdft"
surround_filter_deps="avcodec"
surround_filter_select="rdft"
+textsub2video_filter_deps="avcodec libass"
tinterlace_filter_deps="gpl"
tinterlace_merge_test_deps="tinterlace_filter"
tinterlace_pad_test_deps="tinterlace_filter"
diff --git a/doc/filters.texi b/doc/filters.texi
index 2bd3737cf5..02a854851c 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25880,6 +25880,119 @@ Overlay PGS subtitles
ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:0][0:1]overlaygraphicsubs" output.mp4
@end example
@end itemize
+
+@section overlaytextsubs
+
+Overlay text subtitles onto a video stream.
+
+This filter supersedes the classic @ref{subtitles} filter opposed to which it
does no longer require to open and access the source stream separately, which
is often causing problems or doesn't even work for non-local or slow sources.
+
+Inputs:
+@itemize
+@item 0: Video [YUV420P, YUV422P, YUV444P, ARGB, RGBA, ABGR, BGRA, RGB24,
BGR24]
+@item 1: Subtitles [TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video (same as input)
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+
+@item alpha
+Process alpha channel, by default alpha channel is untouched.
+
+@item fonts_dir
+Set a directory path containing fonts that can be used by the filter.
+These fonts will be used in addition to whatever the font provider uses.
+
+@item default_font_path
+Path to a font file to be used as the default font.
+
+@item font_size
+Set the default font size.
+
+@item fontconfig_file
+Path to ASS fontconfig configuration file.
+
+@item force_style
+Override default style or script info parameters of the subtitles. It accepts a
+string containing ASS style format @code{KEY=VALUE} couples separated by ",".
+
+@item margin
+Set the rendering margin in pixels.
+
+@item render_latest_only
+For rendering, alway use the latest event only, which is covering the given
point in time
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Overlay ASS subtitles with animations:
+@example
+ffmpeg -i
"http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv";
-filter_complex "[0:v]overlaytextsubs" -map 0 -y out.mkv
+@end example
+@end itemize
+
+@section textsub2video
+
+Converts text subtitles to video frames.
+
+For overlaying text subtitles onto video frames it is recommended to use the
overlay_textsubs filter.
+The textsub2video is useful for for creating transparent text-frames when
overlay is done via hw acceleration
+
+Inputs:
+@itemize
+@item 0: Subtitles [TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video [RGB32]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+
+@item rate, r
+Set the framerate for updating overlay frames.
+Normally, overlay frames will only be updated each time when the subtitles to
display are changing.
+In cases where subtitles include advanced features (like animation), this
parameter determines the frequency by which the overlay frames should be
updated.
+
+@item size, s
+Set the output frame size.
+Allows to override the size of output video frames.
+
+@item fonts_dir
+Set a directory path containing fonts that can be used by the filter.
+These fonts will be used in addition to whatever the font provider uses.
+
+@item default_font_path
+Path to a font file to be used as the default font.
+
+@item font_size
+Set the default font size.
+
+@item fontconfig_file
+Path to ASS fontconfig configuration file.
+
+@item force_style
+Override default style or script info parameters of the subtitles. It accepts a
+string containing ASS style format @code{KEY=VALUE} couples separated by ",".
+
+@item margin
+Set the rendering margin in pixels.
+
+@item render_latest_only
+For rendering, alway use the latest event only, which is covering the given
point in time.
+@end table
+
@c man end SUBTITLE FILTERS
[FFmpeg-devel] [PATCH v3 14/26] avfilter/textmod: Add textmod, censor and show_speaker filters
From: softworkz
- textmod {S -> S)
Modify subtitle text in a number of ways
- censor {S -> S)
Censor subtitles using a word list
- show_speaker {S -> S)
Prepend speaker names from ASS subtitles to the visible text lines
Signed-off-by: softworkz
---
doc/filters.texi | 206
libavfilter/Makefile | 5 +
libavfilter/allfilters.c | 3 +
libavfilter/sf_textmod.c | 710 +++
4 files changed, 924 insertions(+)
create mode 100644 libavfilter/sf_textmod.c
diff --git a/doc/filters.texi b/doc/filters.texi
index 02a854851c..6559f9d101 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25772,6 +25772,145 @@ existing filters using @code{--disable-filters}.
Below is a description of the currently available subtitle filters.
+
+@section censor
+
+Censor selected words in text subtitles.
+
+Inputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+@item mode
+The censoring mode to apply.
+
+Supported censoring modes are:
+
+@table @var
+@item 0, keep_first_last
+Replace all characters with the 'censor_char' except the first and the last
character of a word.
+For words with less than 4 characters, the last character will be replaced as
well.
+For words with less than 3 characters, the first character will be replaced as
well.
+@item 1, keep_first
+Replace all characters with the 'censor_char' except the first character of a
word.
+For words with less than 3 characters, the first character will be replaced as
well.
+@item 2, all
+Replace all characters with the 'censor_char'.
+@end table
+
+@item words
+A list of words to censor, separated by 'separator'.
+
+@item words_file
+Specify a file from which to load the contents for the 'words' parameter.
+
+@item censor_char
+Single character used as replacement for censoring.
+
+@item separator
+Delimiter character for words. Used with replace_words and remove_words- Must
be a single character.
+The default is '.'.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Censor a few given words with a pound character.
+@example
+ffmpeg -i
"http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv";
-filter_complex
"[0:1]censor=words='diss,louder,hope,beam,word':censor_char='#'" -map 0 -y
output.mkv
+@end example
+@end itemize
+
+
+@section textmod
+
+Modify subtitle text in a number of ways.
+
+Inputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+@item mode
+The kind of text modification to apply
+
+Supported operation modes are:
+
+@table @var
+@item 0, leet
+Convert subtitle text to 'leet speak'. It's primarily useful for testing as
the modification will be visible with almost all text lines.
+@item 1, to_upper
+Change all text to upper case. Might improve readability.
+@item 2, to_lower
+Change all text to lower case.
+@item 3, replace_chars
+Replace one or more characters. Requires the find and replace parameters to be
specified.
+Both need to be equal in length.
+The first char in find is replaced by the first char in replace, same for all
subsequent chars.
+@item 4, remove_chars
+Remove certain characters. Requires the find parameter to be specified.
+All chars in the find parameter string will be removed from all subtitle text.
+@item 5, replace_words
+Replace one or more words. Requires the find and replace parameters to be
specified. Multiple words must be separated by the delimiter char specified vie
the separator parameter (default: ',').
+The number of words in the find and replace parameters needs to be equal.
+The first word in find is replaced by the first word in replace, same for all
subsequent words
+@item 6, remove_words
+Remove certain words. Requires the find parameter to be specified. Multiple
words must be separated by the delimiter char specified vie the separator
parameter (default: ',').
+All words in the find parameter string will be removed from all subtitle text.
+@end table
+
+@item find
+Required for replace_chars, remove_chars, replace_words and remove_words.
+
+@item find_file
+Specify a file from which to load the contents for the 'find' parameter.
+
+@item replace
+Required for replace_chars and replace_words.
+
+@item replace_file
+Specify a file from which to load the contents for the 'replace' parameter.
+
+@item separator
+Delimiter character for words. Used with replace_words and remove_words- Must
be a single character.
+The default is '.'.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Change all characters to upper case while keeping all styles and animations:
+@example
+ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv";
-filter_complex "[0:s]textmod=mode=to_upper" -map 0 -y out.mkv
+@end example
+@item
+Remove a set of symbol characters for am i
[FFmpeg-devel] [PATCH v3 15/26] avfilter/stripstyles: Add stripstyles filter
From: softworkz
- stripstyles {S -> S)
Remove all inline styles from subtitle events
Signed-off-by: softworkz
---
doc/filters.texi | 37 ++
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/sf_stripstyles.c | 211 +++
4 files changed, 250 insertions(+)
create mode 100644 libavfilter/sf_stripstyles.c
diff --git a/doc/filters.texi b/doc/filters.texi
index 6559f9d101..2f0e560ca0 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25832,6 +25832,43 @@ ffmpeg -i
"http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.
@end example
@end itemize
+@section stripstyles
+
+Remove all inline styles from subtitle events.
+
+Inputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+@item remove_animated
+Also remove text which is subject to animation (default: true)
+Usually, animated text elements are used used in addition to static subtitle
lines for creating effects, so in most cases it is safe to remove the animation
content.
+If subtitle text is missing, try setting this to false.
+
+@item select_layer
+Process only ASS subtitle events from a specific layer. This allows to filter
out certain effects where an ASS author duplicates the text onto multiple
layers.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Remove styles and animations from ASS subtitles and output events from ass
layer 0 only. Then convert asn save as SRT stream:
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv";
-filter_complex "[0:1]stripstyles=select_layer=0" -map 0 -c:s srt output.mkv
+@end example
+@end itemize
+
@section textmod
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 4579b7c8c0..9c512e3527 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -566,6 +566,7 @@ OBJS-$(CONFIG_NULLSINK_FILTER) +=
vsink_nullsink.o
OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o
OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o
OBJS-$(CONFIG_TEXTMOD_FILTER)+= sf_textmod.o
+OBJS-$(CONFIG_STRIPSTYLES_FILTER)+= sf_stripstyles.o
# multimedia filters
OBJS-$(CONFIG_ABITSCOPE_FILTER) += avf_abitscope.o
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 9c489fdc66..23f65533a1 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -550,6 +550,7 @@ extern const AVFilter ff_avf_showwavespic;
extern const AVFilter ff_vaf_spectrumsynth;
extern const AVFilter ff_sf_censor;
extern const AVFilter ff_sf_showspeaker;
+extern const AVFilter ff_sf_stripstyles;
extern const AVFilter ff_sf_textmod;
extern const AVFilter ff_svf_graphicsub2video;
extern const AVFilter ff_svf_textsub2video;
diff --git a/libavfilter/sf_stripstyles.c b/libavfilter/sf_stripstyles.c
new file mode 100644
index 00..bc3c5d1441
--- /dev/null
+++ b/libavfilter/sf_stripstyles.c
@@ -0,0 +1,211 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/**
+ * @file
+ * text subtitle filter which removes inline-styles from subtitles
+ */
+
+#include "libavutil/opt.h"
+#include "internal.h"
+#include "libavutil/ass_split_internal.h"
+#include "libavutil/bprint.h"
+
+typedef struct StripStylesContext {
+const AVClass *class;
+enum AVSubtitleType format;
+int remove_animated;
+int select_layer;
+} StripStylesContext;
+
+typedef struct DialogContext {
+StripStylesContext* ss_ctx;
+AVBPrint buffer;
+int drawing_scale;
+int is_animated;
+} DialogContext;
+
+static void dialog_text_cb(void *priv, const char *text, int len)
+{
+DialogContext *s = priv;
+
+av_log(s->ss_ctx, AV_LOG_DEBUG, "dialog_text_cb: %s\n", text);
+
+if (!s->drawing_scale && (!s->is_animated || !s->ss_ctx->remove_animated))
+av_bprint_append_data(&s->buffer, text, len);
+}
+
+static void dialog_new_line_cb(void *priv, int forced)
+{
+DialogContext *s = priv;
+if (!s->drawing_scale && !s->is_animated)
+av_bprint_append_data(&s->buffer, forced ? "\\N" : "\\n", 2);
+}
[FFmpeg-devel] [PATCH v3 16/26] avfilter/splitcc: Add splitcc filter for closed caption handling
From: softworkz
- splitcc {V -> VS)
Extract closed-caption (A53) data from video
frames as subtitle Frames
ffmpeg -y -loglevel verbose -i "https://streams.videolan.org/streams
/ts/CC/NewsStream-608-ac3.ts" -filter_complex "[0:v]splitcc[vid1],
textmod=mode=remove_chars:find='@',[vid1]overlay_textsubs" output.mkv
Signed-off-by: softworkz
---
configure| 1 +
doc/filters.texi | 63 +++
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/sf_splitcc.c | 395 +++
5 files changed, 461 insertions(+)
create mode 100644 libavfilter/sf_splitcc.c
diff --git a/configure b/configure
index 7b66e315f2..2a561b6f75 100755
--- a/configure
+++ b/configure
@@ -3730,6 +3730,7 @@ spp_filter_select="fft idctdsp fdctdsp me_cmp pixblockdsp"
sr_filter_deps="avformat swscale"
sr_filter_select="dnn"
stereo3d_filter_deps="gpl"
+splitcc_filter_deps="avcodec"
subtitles_filter_deps="avformat avcodec libass"
super2xsai_filter_deps="gpl"
pixfmts_super2xsai_test_deps="super2xsai_filter"
diff --git a/doc/filters.texi b/doc/filters.texi
index 2f0e560ca0..c7a11a0896 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -26182,6 +26182,69 @@ ffmpeg -i INPUT -filter_complex
"showspeaker=format=colon:style='@{\\c&HDD&\
@end example
@end itemize
+
+@section splitcc
+
+Split-out closed-caption/A53 subtitles from video frame side data.
+
+This filter provides an input and an output for video frames, which are just
passed through without modification.
+The second out provides subtitle frames which are extracted from video frame
side data.
+
+Inputs:
+@itemize
+@item 0: Video [ALL]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video (same as input)
+@item 1: Subtitles [TEXT]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+
+@item use_cc_styles
+Emit closed caption style header.
+This will make closed captions appear in white font with a black rectangle
background.
+
+@item real_time
+Emit subtitle events as they are decoded for real-time display.
+
+@item real_time_latency_msec
+Minimum elapsed time between emitting real-time subtitle events.
+Only applies to real_time mode.
+
+@item data_field
+Select data field. Possible values:
+
+@table @samp
+@item auto
+Pick first one that appears.
+@item first
+@item second
+@end table
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Extract closed captions as text subtitle stream and overlay it onto the video
in cc style (black bar background):
+@example
+ffmpeg -i "https://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts";
-filter_complex
"[0:v:0]splitcc=use_cc_styles=1[vid1][sub1];[vid1][sub1]overlaytextsubs"
output.mkv
+@end example
+
+@item
+A nicer variant, using realtime output from cc_dec and rendering it with the
render_latest_only parameter from overlaytextsubs to avoid ghosting by timely
overlap.
+@example
+ffmpeg -i "https://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts";
-filter_complex
"[0:v:0]splitcc=real_time=1:real_time_latency_msec=200[vid1][sub1];[vid1][sub1]overlaytextsubs=render_latest_only=1"
output.mkv
+@end example
+@end itemize
+
+
@section textsub2video
Converts text subtitles to video frames.
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 9c512e3527..b2e715cf8a 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -566,6 +566,7 @@ OBJS-$(CONFIG_NULLSINK_FILTER) +=
vsink_nullsink.o
OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o
OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o
OBJS-$(CONFIG_TEXTMOD_FILTER)+= sf_textmod.o
+OBJS-$(CONFIG_SPLITCC_FILTER)+= sf_splitcc.o
OBJS-$(CONFIG_STRIPSTYLES_FILTER)+= sf_stripstyles.o
# multimedia filters
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 23f65533a1..4996d3d27a 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -550,6 +550,7 @@ extern const AVFilter ff_avf_showwavespic;
extern const AVFilter ff_vaf_spectrumsynth;
extern const AVFilter ff_sf_censor;
extern const AVFilter ff_sf_showspeaker;
+extern const AVFilter ff_sf_splitcc;
extern const AVFilter ff_sf_stripstyles;
extern const AVFilter ff_sf_textmod;
extern const AVFilter ff_svf_graphicsub2video;
diff --git a/libavfilter/sf_splitcc.c b/libavfilter/sf_splitcc.c
new file mode 100644
index 00..14235e822c
--- /dev/null
+++ b/libavfilter/sf_splitcc.c
@@ -0,0 +1,395 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+
[FFmpeg-devel] [PATCH v3 17/26] avfilter/graphicsub2text: Add new graphicsub2text filter (OCR)
From: softworkz Signed-off-by: softworkz --- configure|1 + doc/filters.texi | 55 ++ libavfilter/Makefile |2 + libavfilter/allfilters.c |1 + libavfilter/sf_graphicsub2text.c | 1132 ++ 5 files changed, 1191 insertions(+) create mode 100644 libavfilter/sf_graphicsub2text.c diff --git a/configure b/configure index 2a561b6f75..59d3bb86f0 100755 --- a/configure +++ b/configure @@ -3665,6 +3665,7 @@ frei0r_filter_deps="frei0r" frei0r_src_filter_deps="frei0r" fspp_filter_deps="gpl" gblur_vulkan_filter_deps="vulkan spirv_compiler" +graphicsub2text_filter_deps="libtesseract" hflip_vulkan_filter_deps="vulkan spirv_compiler" histeq_filter_deps="gpl" hqdn3d_filter_deps="gpl" diff --git a/doc/filters.texi b/doc/filters.texi index c7a11a0896..1c0a00c7ae 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -25948,6 +25948,61 @@ ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv"; -filter_comple @end example @end itemize +@section graphicsub2text + +Converts graphic subtitles to text subtitles by performing OCR. + +For this filter to be available, ffmpeg needs to be compiled with libtesseract (see https://github.com/tesseract-ocr/tesseract). +Language models need to be downloaded from https://github.com/tesseract-ocr/tessdata and put into as subfolder named 'tessdata' or into a folder specified via the environment variable 'TESSDATA_PREFIX'. +The path can also be specified via filter option (see below). + +Note: These models are including the data for both OCR modes. + +Inputs: +- 0: Subtitles [bitmap] + +Outputs: +- 0: Subtitles [text] + +It accepts the following parameters: + +@table @option +@item ocr_mode +The character recognition mode to use. + +Supported OCR modes are: + +@table @var +@item 0, tesseract +This is the classic libtesseract operation mode. It is fast but less accurate than LSTM. +@item 1, lstm +Newer OCR implementation based on ML models. Provides usually better results, requires more processing resources. +@item 2, both +Use a combination of both modes. +@end table + +@item tessdata_path +The path to a folder containing the language models to be used. + +@item language +The recognition language. It needs to match the first three characters of a language model file in the tessdata path. + +@end table + + +@subsection Examples + +@itemize +@item +Convert DVB graphic subtitles to ASS (text) subtitles + +Note: For this to work, you need to have the data file 'eng.traineddata' in a 'tessdata' subfolder (see above). +@example +ffmpeg -loglevel verbose -i "https://streams.videolan.org/streams/ts/video_subs_ttxt%2Bdvbsub.ts"; -filter_complex "[0:13]graphicsub2text=delay_when_no_duration=1" -c:s ass -y output.mkv +@end example +@end itemize + + @section graphicsub2video Renders graphic subtitles as video frames. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index b2e715cf8a..1397351373 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -299,6 +299,8 @@ OBJS-$(CONFIG_GBLUR_VULKAN_FILTER) += vf_gblur_vulkan.o vulkan.o vulka OBJS-$(CONFIG_GEQ_FILTER)+= vf_geq.o OBJS-$(CONFIG_GRADFUN_FILTER)+= vf_gradfun.o OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlaygraphicsubs.o framesync.o +OBJS-$(CONFIG_GRAPHICSUB2TEXT_FILTER)+= sf_graphicsub2text.o +OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlaygraphicsubs.o framesync.o OBJS-$(CONFIG_GRAPHMONITOR_FILTER) += f_graphmonitor.o OBJS-$(CONFIG_GRAYWORLD_FILTER) += vf_grayworld.o OBJS-$(CONFIG_GREYEDGE_FILTER) += vf_colorconstancy.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 4996d3d27a..544eca616a 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -549,6 +549,7 @@ extern const AVFilter ff_avf_showwaves; extern const AVFilter ff_avf_showwavespic; extern const AVFilter ff_vaf_spectrumsynth; extern const AVFilter ff_sf_censor; +extern const AVFilter ff_sf_graphicsub2text; extern const AVFilter ff_sf_showspeaker; extern const AVFilter ff_sf_splitcc; extern const AVFilter ff_sf_stripstyles; diff --git a/libavfilter/sf_graphicsub2text.c b/libavfilter/sf_graphicsub2text.c new file mode 100644 index 00..9b413d314e --- /dev/null +++ b/libavfilter/sf_graphicsub2text.c @@ -0,0 +1,1132 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser Gene
[FFmpeg-devel] [PATCH v3 18/26] avfilter/subscale: Add filter for scaling and/or re-arranging graphical subtitles
From: softworkz
Signed-off-by: softworkz
---
configure | 1 +
doc/filters.texi | 164 +++
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/sf_subscale.c | 884 ++
5 files changed, 1051 insertions(+)
create mode 100644 libavfilter/sf_subscale.c
diff --git a/configure b/configure
index 59d3bb86f0..acb788edfc 100755
--- a/configure
+++ b/configure
@@ -3732,6 +3732,7 @@ sr_filter_deps="avformat swscale"
sr_filter_select="dnn"
stereo3d_filter_deps="gpl"
splitcc_filter_deps="avcodec"
+subscale_filter_deps="swscale avcodec"
subtitles_filter_deps="avformat avcodec libass"
super2xsai_filter_deps="gpl"
pixfmts_super2xsai_test_deps="super2xsai_filter"
diff --git a/doc/filters.texi b/doc/filters.texi
index 1c0a00c7ae..c9a1616d84 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -26354,6 +26354,170 @@ Set the rendering margin in pixels.
For rendering, alway use the latest event only, which is covering the given
point in time.
@end table
+@section subscale
+
+Provides high-quality scaling and rearranging functionality for graphical
subtitles.
+
+The subscale filter provides multiple approaches for manipulating
+the size and position of graphical subtitle rectangles wich can
+be combined or used separately.
+Scaling is performed by converting the palettized subtitle bitmaps
+to RGBA and re-quantization to palette colors afterwards via elbg algorithm.
+
+The two major operations are 'scale' and 're-arrange' with the
+latter being separated as 'arrange_h' and 'arrange_v'.
+
+
+Inputs:
+- 0: Subtitles [bitmap]
+
+Outputs:
+- 0: Subtitles [bitmap]
+
+It accepts the following parameters:
+
+@table @option
+
+@item w, width
+Set the width of the output.
+Width and height in case of graphical subtitles are just indicating
+a virtual size for which the output (consisting of 0-n bitmap rectangles)
+is intended to be displayed on.
+
+@item h, height
+Set the height of the output.
+
+@item margin_h
+Sets a horizontal margin to be preserverved when using any
+of the arrange modes.
+
+@item margin_v
+Sets a vertical margin to be preserverved when using any
+of the arrange modes.
+
+@item force_original_aspect_ratio
+Enable decreasing or increasing output video width or height if necessary to
+keep the original aspect ratio. Possible values:
+
+@table @samp
+@item disable
+Scale the video as specified and disable this feature.
+
+@item decrease
+The output video dimensions will automatically be decreased if needed.
+
+@item increase
+The output video dimensions will automatically be increased if needed.
+
+@end table
+
+
+@item scale_mode
+Specifies how subtitle bitmaps should be scaled.
+The scale factor is determined by the the factor between input
+and output size.
+
+@table @samp
+@item none
+Do not apply any common scaling.
+
+@item uniform
+Uniformly scale all subtitle bitmaps including their positions.
+
+@item uniform_no_reposition
+Uniformly scale all subtitle bitmaps without changing positions.
+
+@end table
+
+
+@item arrange_h
+Specifies how subtitle bitmaps should be arranged horizontally.
+
+@item arrange_v
+Specifies how subtitle bitmaps should be arranged vertically.
+
+
+@table @samp
+@item none
+Do not rearrange subtitle bitmaps.
+
+@item margin_no_scale
+Move subtitle bitmaps to be positioned inside the specified
+margin (margin_h or margin_v) when possible and without scaling.
+
+@item margin_and_scale
+Move subtitle bitmaps to be positioned inside the specified
+margin (margin_h or margin_v) and scale in case it doesn't fit.
+
+@item snapalign_no_scale
+Categorize subtitle bitmap positions as one of left/center/right
+or top/bottom/middle based on original positioning and apply
+these alignments for the target positioning.
+No scaling will be applied.
+
+@item snapalign_and_scale
+Categorize subtitle bitmap positions as one of left/center/right
+or top/bottom/middle based on original positioning and apply
+these alignments for the target positioning.
+Bitmaps that do not fit inside the margins borders are
+scaled to fit.
+@end table
+
+@item eval
+Set evaluation mode for the expressions (@option{width}, @option{height}).
+
+It accepts the following values:
+@table @samp
+@item init
+Evaluate expressions only once during the filter initialization.
+
+@item frame
+Evaluate expressions for each incoming frame. This is way slower than the
+@samp{init} mode since it requires all the scalers to be re-computed, but it
+allows advanced dynamic expressions.
+@end table
+
+Default value is @samp{init}.
+
+
+@item num_colors
+Set the number of palette colors for output images.
+Choose the maximum (256) when further processing is done (e.g.
+overlaying on a video).
+When subtitles will be encoded as bitmap subtitles (e.g. dvbsub),
+a smaller number of palette colors (e.g. 4-16) might need to be used, depending
+on the target format and codec.
+
+@item bitmap_width_align
+@item bitmap_height_align
+Make s
[FFmpeg-devel] [PATCH v3 19/26] avfilter/subfeed: add subtitle feed filter
From: softworkz
Signed-off-by: softworkz
---
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/sf_subfeed.c | 366 +++
3 files changed, 368 insertions(+)
create mode 100644 libavfilter/sf_subfeed.c
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 94160d1429..ad8a5a9d18 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -571,6 +571,7 @@ OBJS-$(CONFIG_TEXTMOD_FILTER)+= sf_textmod.o
OBJS-$(CONFIG_SPLITCC_FILTER)+= sf_splitcc.o
OBJS-$(CONFIG_STRIPSTYLES_FILTER)+= sf_stripstyles.o
OBJS-$(CONFIG_SUBSCALE_FILTER) += sf_subscale.o
+OBJS-$(CONFIG_SUBFEED_FILTER)+= sf_subfeed.o
# multimedia filters
OBJS-$(CONFIG_ABITSCOPE_FILTER) += avf_abitscope.o
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 487e2bd6a8..fe1aec093b 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -554,6 +554,7 @@ extern const AVFilter ff_sf_showspeaker;
extern const AVFilter ff_sf_splitcc;
extern const AVFilter ff_sf_stripstyles;
extern const AVFilter ff_sf_subscale;
+extern const AVFilter ff_sf_subfeed;
extern const AVFilter ff_sf_textmod;
extern const AVFilter ff_svf_graphicsub2video;
extern const AVFilter ff_svf_textsub2video;
diff --git a/libavfilter/sf_subfeed.c b/libavfilter/sf_subfeed.c
new file mode 100644
index 00..7df6641f6a
--- /dev/null
+++ b/libavfilter/sf_subfeed.c
@@ -0,0 +1,366 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/**
+ * @file
+ * subtitle filter for feeding subtitle frames into a filtergraph in a
contiguous way
+ *
+ *
+ * also supports
+ * - duration fixup
+ * delaying a subtitle event with unknown duration and infer duration from
the
+ * start time of the subsequent subtitle
+ * - scattering
+ * splitting a subtitle event with unknown duration into multiple ones with
+ * a short and fixed duration
+ *
+ */
+
+#include "filters.h"
+#include "libavutil/opt.h"
+#include "subtitles.h"
+#include "libavutil/avassert.h"
+
+enum SubFeedMode {
+FM_REPEAT,
+FM_SCATTER,
+FM_FORWARD,
+};
+
+typedef struct SubFeedContext {
+const AVClass *class;
+enum AVSubtitleType format;
+enum SubFeedMode mode;
+
+AVRational frame_rate;
+int fix_durations;
+int fix_overlap;
+
+int current_frame_isnew;
+int eof;
+int got_first_input;
+int need_frame;
+int64_t next_pts_offset;
+int64_t recent_subtitle_pts;
+
+int64_t counter;
+
+/**
+ * Queue of frames waiting to be filtered.
+ */
+FFFrameQueue fifo;
+
+} SubFeedContext;
+
+static int64_t ms_to_avtb(int64_t ms)
+{
+return av_rescale_q(ms, (AVRational){ 1, 1000 }, AV_TIME_BASE_Q);
+}
+
+static int64_t avtb_to_ms(int64_t avtb)
+{
+return av_rescale_q(avtb, AV_TIME_BASE_Q, (AVRational){ 1, 1000 });
+}
+
+static int init(AVFilterContext *ctx)
+{
+SubFeedContext *s = ctx->priv;
+
+ff_framequeue_init(&s->fifo, NULL);
+
+return 0;
+}
+
+static void uninit(AVFilterContext *ctx)
+{
+SubFeedContext *s = ctx->priv;
+ff_framequeue_free(&s->fifo);
+}
+
+static int config_input(AVFilterLink *link)
+{
+const subfeedContext *context = link->dst->priv;
+
+return 0;
+}
+
+static int query_formats(AVFilterContext *ctx)
+{
+AVFilterFormats *formats;
+AVFilterLink *inlink0 = ctx->inputs[0];
+AVFilterLink *outlink0 = ctx->outputs[0];
+static const enum AVSubtitleType subtitle_fmts[] = {
AV_SUBTITLE_FMT_BITMAP, AV_SUBTITLE_FMT_ASS, AV_SUBTITLE_FMT_NB };
+int ret;
+
+formats = ff_make_format_list(subtitle_fmts);
+
+if ((ret = ff_formats_ref(formats, &inlink0->outcfg.formats)) < 0)
+return ret;
+
+if ((ret = ff_formats_ref(formats, &outlink0->incfg.formats)) < 0)
+return ret;
+
+return 0;
+}
+
+static int config_output(AVFilterLink *outlink)
+{
+SubFeedContext *s = outlink->src->priv;
+const AVFilterLink *inlink = outlink->src->inputs[0];
+
+outlink->time_base = AV_TIME_BASE_Q;
+outlink->format = inlink->format;
+outlink->w = inlink->w;
+outlink->h = inlink->h;
+
+if (s->mode == FM_FORWARD)
+
[FFmpeg-devel] [PATCH v3 20/26] avcodec/subtitles: Migrate subtitle encoders to frame-based API
From: softworkz
and provide a compatibility shim for the legacy api
Signed-off-by: softworkz
---
libavcodec/assenc.c| 189 ++---
libavcodec/avcodec.h | 5 +-
libavcodec/dvbsubenc.c | 96 ++-
libavcodec/dvdsubenc.c | 102
libavcodec/encode.c| 61 +++-
libavcodec/movtextenc.c| 114 --
libavcodec/srtenc.c| 108 ++---
libavcodec/tests/avcodec.c | 2 -
libavcodec/ttmlenc.c | 101 +++-
libavcodec/webvttenc.c | 86 -
libavcodec/xsubenc.c | 88 ++---
11 files changed, 687 insertions(+), 265 deletions(-)
diff --git a/libavcodec/assenc.c b/libavcodec/assenc.c
index b0e475834b..e1401b1ac5 100644
--- a/libavcodec/assenc.c
+++ b/libavcodec/assenc.c
@@ -22,70 +22,195 @@
#include
#include "avcodec.h"
+#include "encode.h"
#include "libavutil/ass_internal.h"
#include "internal.h"
#include "libavutil/avstring.h"
#include "libavutil/internal.h"
#include "libavutil/mem.h"
+typedef struct {
+AVCodecContext *avctx;
+AVFrame* current_frame;
+int have_frame;
+int current_area;
+} AssEncContext;
+
+static void check_write_header(AVCodecContext* avctx, const AVFrame* frame)
+{
+if (avctx->extradata_size)
+return;
+
+if (frame->subtitle_header && frame->subtitle_header->size > 0) {
+const char* subtitle_header = (char*)frame->subtitle_header->data;
+avctx->extradata_size = strlen(subtitle_header);
+avctx->extradata = av_mallocz(frame->subtitle_header->size + 1);
+memcpy(avctx->extradata, subtitle_header, avctx->extradata_size);
+avctx->extradata[avctx->extradata_size] = 0;
+}
+
+if (!avctx->extradata_size) {
+const char* subtitle_header =
avpriv_ass_get_subtitle_header_default(0);
+if (!subtitle_header)
+return;
+
+avctx->extradata_size = strlen(subtitle_header);
+avctx->extradata = av_mallocz(avctx->extradata_size + 1);
+memcpy(avctx->extradata, subtitle_header, avctx->extradata_size);
+avctx->extradata[avctx->extradata_size] = 0;
+av_freep(&subtitle_header);
+}
+}
+
static av_cold int ass_encode_init(AVCodecContext *avctx)
{
-avctx->extradata = av_malloc(avctx->subtitle_header_size + 1);
-if (!avctx->extradata)
-return AVERROR(ENOMEM);
-memcpy(avctx->extradata, avctx->subtitle_header,
avctx->subtitle_header_size);
-avctx->extradata_size = avctx->subtitle_header_size;
-avctx->extradata[avctx->extradata_size] = 0;
+AssEncContext *s = avctx->priv_data;
+
+if (avctx->subtitle_header_size) {
+avctx->extradata = av_malloc(avctx->subtitle_header_size + 1);
+if (!avctx->extradata)
+return AVERROR(ENOMEM);
+memcpy(avctx->extradata, avctx->subtitle_header,
avctx->subtitle_header_size);
+avctx->extradata_size = avctx->subtitle_header_size;
+avctx->extradata[avctx->extradata_size] = 0;
+}
+
+s->current_frame = av_frame_alloc();
+return 0;
+}
+
+static av_cold int ass_encode_close(AVCodecContext *avctx)
+{
+AssEncContext *s = avctx->priv_data;
+av_frame_free(&s->current_frame);
return 0;
}
-static int ass_encode_frame(AVCodecContext *avctx,
-unsigned char *buf, int bufsize,
-const AVSubtitle *sub)
+static int ass_encode_frame(AVCodecContext* avctx, AVPacket* avpkt,
+const AVFrame* frame, int* got_packet)
+{
+int ret;
+size_t req_len = 0, total_len = 0;
+
+check_write_header(avctx, frame);
+
+for (unsigned i = 0; i < frame->num_subtitle_areas; i++) {
+const char *ass = frame->subtitle_areas[i]->ass;
+
+if (frame->subtitle_areas[i]->type != AV_SUBTITLE_FMT_ASS) {
+av_log(avctx, AV_LOG_ERROR, "Only AV_SUBTITLE_FMT_ASS type
supported.\n");
+return AVERROR(EINVAL);
+}
+
+if (ass)
+req_len += strlen(ass);
+}
+
+ret = ff_get_encode_buffer(avctx, avpkt, req_len + 1, 0);
+if (ret < 0) {
+av_log(avctx, AV_LOG_ERROR, "Error getting output packet.\n");
+return ret;
+}
+
+for (unsigned i = 0; i < frame->num_subtitle_areas; i++) {
+const char *ass = frame->subtitle_areas[i]->ass;
+
+if (ass) {
+size_t len = av_strlcpy((char *)avpkt->data + total_len, ass,
avpkt->size - total_len);
+total_len += len;
+}
+}
+
+avpkt->size = total_len;
+*got_packet = total_len > 0;
+
+return 0;
+}
+
+static int ass_receive_packet(AVCodecContext *avctx, AVPacket *avpkt)
{
-int i, len, total_len = 0;
+AssEncContext *s = avctx-
[FFmpeg-devel] [PATCH v3 22/26] avutil/ass_split: Add parsing of hard-space tags (\h)
From: softworkz
The \h tag in ASS/SSA is indicating a non-breaking space. See
https://github.com/Aegisub/aegisite/blob/master/source/docs/3.2/
ASS_Tags.html.md
The ass_split implementation is used by almost all text subtitle
encoders and it didn't handle this tag. Interestingly, several
tests are testing for \h parsing and had incorrect reference data
for those tests.
The \h tag is specific to ASS and doesn't have any meaning outside
of ASS.
Still, the reference data for ttmlenc, textenc and webvttenc were
full of \h tags even though this tag doesn't have a meaning there.
Signed-off-by: softworkz
---
libavutil/ass_split.c| 7 +++
tests/ref/fate/.gitattributes| 3 +++
tests/ref/fate/mov-mp4-ttml-dfxp | 8
tests/ref/fate/mov-mp4-ttml-stpp | 8
tests/ref/fate/sub-textenc | 10 +-
tests/ref/fate/sub-ttmlenc | 8
tests/ref/fate/sub-webvttenc | 10 +-
7 files changed, 32 insertions(+), 22 deletions(-)
create mode 100644 tests/ref/fate/.gitattributes
diff --git a/libavutil/ass_split.c b/libavutil/ass_split.c
index c5963351fc..30512dfc74 100644
--- a/libavutil/ass_split.c
+++ b/libavutil/ass_split.c
@@ -484,6 +484,7 @@ int avpriv_ass_split_override_codes(const ASSCodesCallbacks
*callbacks, void *pr
while (buf && *buf) {
if (text && callbacks->text &&
(sscanf(buf, "\\%1[nN]", new_line) == 1 ||
+ sscanf(buf, "\\%1[hH]", new_line) == 1 ||
!strncmp(buf, "{\\", 2))) {
callbacks->text(priv, text, text_len);
text = NULL;
@@ -492,6 +493,12 @@ int avpriv_ass_split_override_codes(const
ASSCodesCallbacks *callbacks, void *pr
if (callbacks->new_line)
callbacks->new_line(priv, new_line[0] == 'N');
buf += 2;
+} else if (sscanf(buf, "\\%1[hH]", new_line) == 1) {
+if (callbacks->hard_space)
+callbacks->hard_space(priv);
+else if (callbacks->text)
+callbacks->text(priv, " ", 1);
+buf += 2;
} else if (!strncmp(buf, "{\\", 2)) {
buf++;
while (*buf == '\\') {
diff --git a/tests/ref/fate/.gitattributes b/tests/ref/fate/.gitattributes
new file mode 100644
index 00..19be64d085
--- /dev/null
+++ b/tests/ref/fate/.gitattributes
@@ -0,0 +1,3 @@
+sub-textenc -diff
+sub-ttmlenc -diff
+sub-webvttenc -diff
diff --git a/tests/ref/fate/mov-mp4-ttml-dfxp b/tests/ref/fate/mov-mp4-ttml-dfxp
index e24b5d618b..e565ffa1f6 100644
--- a/tests/ref/fate/mov-mp4-ttml-dfxp
+++ b/tests/ref/fate/mov-mp4-ttml-dfxp
@@ -1,9 +1,9 @@
-2e7e01c821c111466e7a2844826b7f6d *tests/data/fate/mov-mp4-ttml-dfxp.mp4
-8519 tests/data/fate/mov-mp4-ttml-dfxp.mp4
+658884e1b789e75c454b25bdf71283c9 *tests/data/fate/mov-mp4-ttml-dfxp.mp4
+8486 tests/data/fate/mov-mp4-ttml-dfxp.mp4
#tb 0: 1/1000
#media_type 0: data
#codec_id 0: none
-0, 0, 0,68500, 7866, 0x456c36b7
+0, 0, 0,68500, 7833, 0x31b22193
{
"packets": [
{
@@ -15,7 +15,7 @@
"dts_time": "0.00",
"duration": 68500,
"duration_time": "68.50",
-"size": "7866",
+"size": "7833",
"pos": "44",
"flags": "K_"
}
diff --git a/tests/ref/fate/mov-mp4-ttml-stpp b/tests/ref/fate/mov-mp4-ttml-stpp
index 77bd23b7bf..f25b5b2d28 100644
--- a/tests/ref/fate/mov-mp4-ttml-stpp
+++ b/tests/ref/fate/mov-mp4-ttml-stpp
@@ -1,9 +1,9 @@
-cbd2c7ff864a663b0d893deac5a0caec *tests/data/fate/mov-mp4-ttml-stpp.mp4
-8547 tests/data/fate/mov-mp4-ttml-stpp.mp4
+c9570de0ccebc858b0c662a7e449582c *tests/data/fate/mov-mp4-ttml-stpp.mp4
+8514 tests/data/fate/mov-mp4-ttml-stpp.mp4
#tb 0: 1/1000
#media_type 0: data
#codec_id 0: none
-0, 0, 0,68500, 7866, 0x456c36b7
+0, 0, 0,68500, 7833, 0x31b22193
{
"packets": [
{
@@ -15,7 +15,7 @@ cbd2c7ff864a663b0d893deac5a0caec
*tests/data/fate/mov-mp4-ttml-stpp.mp4
"dts_time": "0.00",
"duration": 68500,
"duration_time": "68.50",
-"size": "7866",
+"size": "7833",
"pos": "44",
"flags": "K_"
}
diff --git a/tests/ref/fate/sub-textenc b/tests/ref/fate/sub-textenc
index 3ea56b38f0..910ca3d6e3 100644
--- a/tests/ref/fate/sub-textenc
+++ b/tests/ref/fate/sub-textenc
@@ -160,18 +160,18 @@ but show this: {normal text}
\ N is a forced line break
\ h is a hard space
Normal spaces at the start and at the end of the line are trimmed while hard
spaces are not trimmed.
-The\hline\hwill\hnever\hbreak\hautomatically\hright\hbefore\hor\hafter\ha\hhard\hspace.\h:-D
+The line will never break automatically right before or after a hard space. :-D
31
00:00:54,501 --> 00:00:56,500
-\h\h\h\h\hA (05 hard spaces followed by a letter)
+ A (05 hard spaces followed by a let
[FFmpeg-devel] [PATCH v3 23/26] avcodec/webvttenc: convert hard-space tags to
From: softworkz
Signed-off-by: softworkz
---
libavcodec/webvttenc.c | 6 ++
tests/ref/fate/sub-webvttenc | 10 +-
2 files changed, 11 insertions(+), 5 deletions(-)
diff --git a/libavcodec/webvttenc.c b/libavcodec/webvttenc.c
index c0436f5739..48945dcb8e 100644
--- a/libavcodec/webvttenc.c
+++ b/libavcodec/webvttenc.c
@@ -123,6 +123,11 @@ static void webvtt_new_line_cb(void *priv, int forced)
webvtt_print(priv, "\n");
}
+static void webvtt_hard_space_cb(void *priv)
+{
+webvtt_print(priv, " ");
+}
+
static void webvtt_style_cb(void *priv, char style, int close)
{
if (style == 's') // strikethrough unsupported
@@ -147,6 +152,7 @@ static void webvtt_end_cb(void *priv)
static const ASSCodesCallbacks webvtt_callbacks = {
.text = webvtt_text_cb,
.new_line = webvtt_new_line_cb,
+.hard_space = webvtt_hard_space_cb,
.style= webvtt_style_cb,
.color= NULL,
.font_name= NULL,
diff --git a/tests/ref/fate/sub-webvttenc b/tests/ref/fate/sub-webvttenc
index f4172dcc84..ee9de2859e 100644
--- a/tests/ref/fate/sub-webvttenc
+++ b/tests/ref/fate/sub-webvttenc
@@ -132,26 +132,26 @@ but show this: {normal text}
\ N is a forced line break
\ h is a hard space
Normal spaces at the start and at the end of the line are trimmed while hard
spaces are not trimmed.
-The line will never break automatically right before or after a hard space. :-D
+The line will never break automatically right before or after a hard space. :-D
00:54.501 --> 00:56.500
- A (05 hard spaces followed by a letter)
+ A (05 hard spaces followed by a letter)
A (Normal spaces followed by a letter)
A (No hard spaces followed by a letter)
00:56.501 --> 00:58.500
- A (05 hard spaces followed by a letter)
+ A (05 hard spaces followed by a letter)
A (Normal spaces followed by a letter)
A (No hard spaces followed by a letter)
Show this: \TEST and this: \-)
00:58.501 --> 01:00.500
-A letter followed by 05 hard spaces: A
+A letter followed by 05 hard spaces: A
A letter followed by normal spaces: A
A letter followed by no hard spaces: A
-05 hard spaces between letters: A A
+05 hard spaces between letters: A A
5 normal spaces between letters: A A
^--Forced line break
--
ffmpeg-codebot
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v3 24/26] doc/APIchanges: update for subtitle filtering changes
From: softworkz Signed-off-by: softworkz --- doc/APIchanges | 24 libavcodec/version.h | 2 +- libavutil/version.h | 2 +- 3 files changed, 26 insertions(+), 2 deletions(-) diff --git a/doc/APIchanges b/doc/APIchanges index 8df0364e4c..c8238fb008 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -14,6 +14,30 @@ libavutil: 2021-04-27 API changes, most recent first: +2021-12-05 - xx - lavc 59.15.100 - avcodec.h + Deprecate avcodec_encode_subtitle(), use regular encode api now + +2021-12-05 - xx - lavc 59.15.100 - codec_desc.h + Add avcodec_descriptor_get_subtitle_format() + +2021-12-05 - xx - lavc 59.15.100 - avcodec.h + Deprecate avsubtitle_free() + Deprecate avcodec_decode_subtitle2(), use regular decode api now + +2021-12-05 - xx - lavu 57.11.100 - frame.h + Add AVMediaType field to AVFrame + Add Fields for carrying subtitle data to AVFrame + (subtitle_areas, subtitle_header, subtitle_pts, start/end time, etc.) + Add av_frame_get_buffer2() and deprecate av_frame_get_buffer() + +2021-12-05 - xx - lavu 57.11.100 - subfmt.h + Add struct AVSubtitleArea (replaces AVSubtitle) + Add av_get_subtitle_fmt_name() and av_get_subtitle_fmt() + +2021-12-05 - xx - lavu 57.11.100 - subfmt.h + Add enum AVSubtitleType (moved from lavc), add new values, deprecate existing + +2021-11-xx - xx - lavfi 8.19.100 - avfilter.h 2022-01-04 - 78dc21b123e - lavu 57.16.100 - frame.h Add AV_FRAME_DATA_DOVI_METADATA. diff --git a/libavcodec/version.h b/libavcodec/version.h index a46fb05f1a..b5867ad041 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -28,7 +28,7 @@ #include "libavutil/version.h" #define LIBAVCODEC_VERSION_MAJOR 59 -#define LIBAVCODEC_VERSION_MINOR 20 +#define LIBAVCODEC_VERSION_MINOR 21 #define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ diff --git a/libavutil/version.h b/libavutil/version.h index 5bf48f6304..168e24f410 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,7 +79,7 @@ */ #define LIBAVUTIL_VERSION_MAJOR 57 -#define LIBAVUTIL_VERSION_MINOR 18 +#define LIBAVUTIL_VERSION_MINOR 19 #define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ -- ffmpeg-codebot ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v3 25/26] avcodec/webvttenc: Don't encode drawing codes and empty lines
From: softworkz
Signed-off-by: softworkz
---
libavcodec/webvttenc.c | 31 +++
1 file changed, 27 insertions(+), 4 deletions(-)
diff --git a/libavcodec/webvttenc.c b/libavcodec/webvttenc.c
index 48945dcb8e..62c4aa7ffd 100644
--- a/libavcodec/webvttenc.c
+++ b/libavcodec/webvttenc.c
@@ -39,6 +39,8 @@ typedef struct {
int count;
char stack[WEBVTT_STACK_SIZE];
int stack_ptr;
+int has_text;
+int drawing_scale;
} WebVTTContext;
#ifdef __GNUC__
@@ -115,17 +117,24 @@ static void webvtt_style_apply(WebVTTContext *s, const
char *style)
static void webvtt_text_cb(void *priv, const char *text, int len)
{
WebVTTContext *s = priv;
-av_bprint_append_data(&s->buffer, text, len);
+if (!s->drawing_scale) {
+av_bprint_append_data(&s->buffer, text, len);
+s->has_text = 1;
+}
}
static void webvtt_new_line_cb(void *priv, int forced)
{
-webvtt_print(priv, "\n");
+WebVTTContext *s = priv;
+if (!s->drawing_scale)
+webvtt_print(priv, "\n");
}
static void webvtt_hard_space_cb(void *priv)
{
-webvtt_print(priv, " ");
+WebVTTContext *s = priv;
+if (!s->drawing_scale)
+webvtt_print(priv, " ");
}
static void webvtt_style_cb(void *priv, char style, int close)
@@ -149,6 +158,12 @@ static void webvtt_end_cb(void *priv)
webvtt_stack_push_pop(priv, 0, 1);
}
+static void dialog_drawing_mode_cb(void *priv, int scale)
+{
+WebVTTContext *s = priv;
+s->drawing_scale = scale;
+}
+
static const ASSCodesCallbacks webvtt_callbacks = {
.text = webvtt_text_cb,
.new_line = webvtt_new_line_cb,
@@ -161,6 +176,7 @@ static const ASSCodesCallbacks webvtt_callbacks = {
.cancel_overrides = webvtt_cancel_overrides_cb,
.move = NULL,
.end = webvtt_end_cb,
+.drawing_mode = dialog_drawing_mode_cb,
};
static void ensure_ass_context(WebVTTContext* s, const AVFrame* frame)
@@ -211,16 +227,23 @@ static int webvtt_encode_frame(AVCodecContext* avctx,
AVPacket* avpkt,
}
if (ass) {
+const unsigned saved_len = s->buffer.len;
-if (i > 0)
+if (i > 0 && s->buffer.len > 0)
webvtt_new_line_cb(s, 0);
+s->drawing_scale = 0;
+s->has_text = 0;
+
dialog = avpriv_ass_split_dialog(s->ass_ctx, ass);
if (!dialog)
return AVERROR(ENOMEM);
webvtt_style_apply(s, dialog->style);
avpriv_ass_split_override_codes(&webvtt_callbacks, s,
dialog->text);
avpriv_ass_free_dialog(&dialog);
+
+if (!s->has_text)
+s->buffer.len = saved_len;
}
}
--
ffmpeg-codebot
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v3 26/26] avcodec/dvbsubdec: Fix conditions for fallback to default resolution
From: softworkz
The previous code expected a segment of type CLUT definition to exist
in order to accept a set of segments to be complete.
This was an incorrect assumption as the presence of a CLUT segment
is not mandatory.
(version 1.6.1 of the spec is probably a bit more clear about this
than earlier versions: https://www.etsi.org/deliver/etsi_en/
300700_300799/300743/01.06.01_20/en_300743v010601a.pdf)
The flawed condition prevented proper fallback to using the default
resolution for the decoding context.
Signed-off-by: softworkz
---
libavcodec/dvbsubdec.c | 51 +-
1 file changed, 30 insertions(+), 21 deletions(-)
diff --git a/libavcodec/dvbsubdec.c b/libavcodec/dvbsubdec.c
index 0d64c6e71c..3a6259101c 100644
--- a/libavcodec/dvbsubdec.c
+++ b/libavcodec/dvbsubdec.c
@@ -33,7 +33,7 @@
#define DVBSUB_CLUT_SEGMENT 0x12
#define DVBSUB_OBJECT_SEGMENT 0x13
#define DVBSUB_DISPLAYDEFINITION_SEGMENT 0x14
-#define DVBSUB_DISPLAY_SEGMENT 0x80
+#define DVBSUB_END_DISPLAY_SEGMENT 0x80
#define cm (ff_crop_tab + MAX_NEG_CROP)
@@ -1620,8 +1620,12 @@ static int dvbsub_decode(AVCodecContext *avctx,
int segment_length;
int i;
int ret = 0;
-int got_segment = 0;
-int got_dds = 0;
+//int got_segment = 0;
+int got_page = 0;
+int got_region = 0;
+int got_object = 0;
+int got_end_display = 0;
+int got_displaydef = 0;
ff_dlog(avctx, "DVB sub packet:\n");
@@ -1666,34 +1670,28 @@ static int dvbsub_decode(AVCodecContext *avctx,
switch (segment_type) {
case DVBSUB_PAGE_SEGMENT:
ret = dvbsub_parse_page_segment(avctx, p, segment_length, sub,
got_sub_ptr);
-got_segment |= 1;
+got_page = 1;
break;
case DVBSUB_REGION_SEGMENT:
ret = dvbsub_parse_region_segment(avctx, p, segment_length);
-got_segment |= 2;
+got_region = 1;
break;
case DVBSUB_CLUT_SEGMENT:
ret = dvbsub_parse_clut_segment(avctx, p, segment_length);
if (ret < 0) goto end;
-got_segment |= 4;
break;
case DVBSUB_OBJECT_SEGMENT:
ret = dvbsub_parse_object_segment(avctx, p, segment_length);
-got_segment |= 8;
+got_object = 1;
break;
case DVBSUB_DISPLAYDEFINITION_SEGMENT:
ret = dvbsub_parse_display_definition_segment(avctx, p,
segment_length);
-got_dds = 1;
+got_displaydef = 1;
break;
-case DVBSUB_DISPLAY_SEGMENT:
+case DVBSUB_END_DISPLAY_SEGMENT:
ret = dvbsub_display_end_segment(avctx, p, segment_length,
sub, got_sub_ptr);
-if (got_segment == 15 && !got_dds && !avctx->width &&
!avctx->height) {
-// Default from ETSI EN 300 743 V1.3.1 (7.2.1)
-avctx->width = 720;
-avctx->height = 576;
-}
-got_segment |= 16;
+got_end_display = 1;
break;
default:
ff_dlog(avctx, "Subtitling segment type 0x%x, page id %d,
length %d\n",
@@ -1706,13 +1704,24 @@ static int dvbsub_decode(AVCodecContext *avctx,
p += segment_length;
}
-// Some streams do not send a display segment but if we have all the other
-// segments then we need no further data.
-if (got_segment == 15) {
-av_log(avctx, AV_LOG_DEBUG, "Missing display_end_segment,
emulating\n");
-dvbsub_display_end_segment(avctx, p, 0, sub, got_sub_ptr);
-}
+// Even though not mandated by the spec, we're imposing a minimum
requirement
+// for a useful packet to have at least one page, region and object
segment.
+if (got_page && got_region && got_object && got_end_display) {
+
+if (!got_displaydef && !avctx->width && !avctx->height) {
+// Default from ETSI EN 300 743 V1.3.1 (7.2.1)
+avctx->width = 720;
+avctx->height = 576;
+}
+
+// Some streams do not send an end-of-display segment but if we have
all the other
+// segments then we need no further data.
+if (!got_end_display) {
+av_log(avctx, AV_LOG_DEBUG, "Missing display_end_segment,
emulating\n");
+dvbsub_display_end_segment(avctx, p, 0, sub, got_sub_ptr);
+}
+}
end:
if (ret < 0) {
return ret;
--
ffmpeg-codebot
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH] libavcodec/qsvenc: fix a memory leak problem
"qf->frame" ref to input frame but it isn't released. av_frame_unref()
is added before refering qf->frame to new frame to make sure the previous
reference is released.
Signed-off-by: Wenbin Chen
---
libavcodec/qsvenc.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c
index 269386624d..fd5c460d68 100644
--- a/libavcodec/qsvenc.c
+++ b/libavcodec/qsvenc.c
@@ -1521,6 +1521,7 @@ static int submit_frame(QSVEncContext *q, const AVFrame
*frame,
return ret;
}
} else {
+av_frame_unref(qf->frame);
ret = av_frame_ref(qf->frame, frame);
if (ret < 0)
return ret;
--
2.25.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
