Re: [FFmpeg-devel] Scaling PAL8 images with alpha
> -Original Message- > From: ffmpeg-devel On Behalf Of Tomas > Härdin > Sent: Friday, 24 September 2021 17:34 > To: FFmpeg development discussions and patches > Subject: Re: [FFmpeg-devel] Scaling PAL8 images with alpha > > fre 2021-09-24 klockan 10:30 + skrev Soft Works: > > Hi, > > > > for a new filter, I want to rescale PAL8 subtitle bitmaps where the > > palette includes > > colors with alpha. > > Isn't this typically done at the player level? What's your use case? Ok, the full story about this is also one out of several reasons for going forward with subtitle filtering. One thing that I often needed to tell users doing live transcoding with burn-in of graphical subtitles on system with weak CPUs was like: "This is an expensive processing which requires a significant amount of CPU resources and may sometimes lead to transcoding speeds below realtime (1.0x)" But the truth is, that it's not that expensive at all - it's just expensive due to the way how ffmpeg is doing it - with sub2video. Let's assume there's a 4k video with graphical subtitles that need to be overlaid on top of the video. For this purpose (and due to the lack of subtitle filtering), sub2video is generating a full frame RGBA (4k) with transparency onto which it renders the graphical subtitles or just an empty 4k frame when no subtitles are to be shown. These RGBA frames are sent into a filtergraph where they can be used with the overlay filter to blend over the video frames. A rather small problem is the required conversion when overlaying over e.g. yuv420p (happening automatically) The bigger one is that this creates a situation where every single 4k video frame needs to be blended with a 4k overlay frame, iterating over every single pixel and even in periods when no subtitles are visible at all. When looking at bitmap subtitles and how they work, we can see that these are actually covering just one or few small rect regions of PAL8 images, making up just a fraction of the frame area. While my patchset includes a 'graphicsub2video' filter which replicates the sub2video behavior (still useful when uploading to hw for example), the new way to go is the 'overlay_graphicsubs' filter, which has a video and a subtitle input and a video output. It receives the subtitle rects directly instead of a pre-rendered full-size frame. When there are no subtitle rects at a time, it just does - nothing and passes the input video unchanged to the output. In case there are bitmap subtitles to display, it iterates over them and only blends the region of each subtitle rect with the main image. When blending over a yuv video, there's no need to convert a full-size RGBA frame to yuv; not even the subtitle region. I just convert the PAL8 RGBA palette to yuv - and use it for lookup when blending. Now that the "why" is clear, there's one problem with it: How to handle cases where the video needs to be scaled? Previously, we've simply been scaling that transparent full frame in the same way like the main video. Needless to say that this produces great results, having a full byte per color per pixel. But this suffers from the same problem as before. We need to stick to handling individual subtitle bitmaps. Also, it's not guaranteed that these will end up being used for overlay - they could also be used for re-encoding as graphical subtitle stream. For those purposes I'm working on a sub_scale filter, allowing to scale-down or scale-up those PAL8 subtitle bitmaps. Due to the previous(current) implementation using RGBA images for blending and scaling, this endeavor is practically "doomed" to deliver results of at least similar quality, but with PAL8 instead of RGBA32 images. Before concluding that this would not be achievable, there's one important point to consider: these bitmaps are not like photos. It's about text having a primary color, possibly a border color and possibly a shadow color. Sometimes plus intermediate colors for smooth blending at the edges. It doesn't require a large number of colors in such cases to achieve equal results than with full RGBA, but it does require a good algorithm to do the color quantization (palettization) after scaling (with temporary RGBA conversion) or a scaling that can work with PAL8 images directly and is able to produce an adaptive palette for the output. I'm not sure whether that adds much to the actual problem, but you got the full story at least :-) Thanks again for all suggestion I got already! softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] Scaling PAL8 images with alpha
> -Original Message- > From: ffmpeg-devel On Behalf Of Michael > Niedermayer > Sent: Friday, 24 September 2021 17:40 > To: FFmpeg development discussions and patches > Subject: Re: [FFmpeg-devel] Scaling PAL8 images with alpha > > On Fri, Sep 24, 2021 at 10:30:31AM +, Soft Works wrote: > > Hi, > > > > for a new filter, I want to rescale PAL8 subtitle bitmaps where the palette > includes > > colors with alpha. > > > > From what I’ve seen, swscale doesn’t support PAL8-to-PAL8, only PAL8-to- > BGR8 > > which doesn’t support alpha and the palette is fixed with 256 entries > defined by > > convention, while I would ideally like to be able to allow the following: > > > > - constrain the output to use the same palette as the input > > - adaptively quantize it to a palette with a configurable number of colors > > > > Thus it's about the palette quantization (with or without dithering) after > > scaling in RGBA. (or some cool algorithmic trick I'm not aware of) > > > > Is there any existing code that I could reuse? The closest I could find > > is pngenc, but I'm wondering whether there's something else/better > > somewhere in the ffmpeg libs that I haven't seen? > > There are some non linear scaling filters which may make sense when you do > not have the full linear space available see: > hqx, epx, xbr > you could also try some neural net stuff Photoshop can perfectly do what is needed, but their algorithms are not public. > or the insane solution, OCR, scale text, render font into RGBA and then back > to PAL8 LOL, for a moment I had thought the same after the great OCR results I had seen recently... softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] Scaling PAL8 images with alpha
> -Original Message- > From: ffmpeg-devel On Behalf Of > Soft Works > Sent: Friday, 24 September 2021 19:03 > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] Scaling PAL8 images with alpha > > > > > -Original Message- > > From: ffmpeg-devel On Behalf Of > Michael > > Niedermayer > > Sent: Friday, 24 September 2021 17:40 > > To: FFmpeg development discussions and patches de...@ffmpeg.org> > > Subject: Re: [FFmpeg-devel] Scaling PAL8 images with alpha > > > > On Fri, Sep 24, 2021 at 10:30:31AM +, Soft Works wrote: > > > Hi, > > > > > > for a new filter, I want to rescale PAL8 subtitle bitmaps where > the palette > > includes > > > colors with alpha. > > > > > > From what I’ve seen, swscale doesn’t support PAL8-to-PAL8, only > PAL8-to- > > BGR8 > > > which doesn’t support alpha and the palette is fixed with 256 > entries > > defined by > > > convention, while I would ideally like to be able to allow the > following: > > > > > > - constrain the output to use the same palette as the input > > > - adaptively quantize it to a palette with a configurable number > of colors > > > > > > Thus it's about the palette quantization (with or without > dithering) after > > > scaling in RGBA. (or some cool algorithmic trick I'm not aware > of) > > > > > > Is there any existing code that I could reuse? The closest I > could find > > > is pngenc, but I'm wondering whether there's something > else/better > > > somewhere in the ffmpeg libs that I haven't seen? > > > > There are some non linear scaling filters which may make sense when > you do > > not have the full linear space available see: > > hqx, epx, xbr > > you could also try some neural net stuff > > Photoshop can perfectly do what is needed, but their algorithms are > not > public. I've put a few thing together to illustrate what I'm talking about: https://gist.github.com/softworkz/deef5c2a43d3d629c3e17f9e21544a8f Meanwhile I found what I need: https://github.com/ImageOptim/libimagequant Interestingly, they are comparing their lib specifically to the Photoshop feature that I mentioned: https://pngquant.org/vsphotoshop.html In the 2000's, the PS implementation for image quantization, optimization and compression (profanely named "Save for Web") had remained unparalleled for years, that's why it was one of my first thoughts. I hadn't heard about libimagequant before, but it seems to do exactly what is needed for rescaling PAL8 to PAL8 at a high quality. Before spending any time on it, I wanted ask whether that library would be acceptable for adding it as a (GPLv3) reference to the project and as a dependency to my filter? Kind regards, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] Scaling PAL8 images with alpha
> -Original Message- > From: ffmpeg-devel On Behalf Of > Michael Niedermayer > Sent: Saturday, 25 September 2021 16:30 > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] Scaling PAL8 images with alpha > > On Sat, Sep 25, 2021 at 10:23:56AM +0200, Hendrik Leppkes wrote: > > On Sat, Sep 25, 2021 at 5:00 AM Soft Works > wrote: > > > > > > > > > > > > > -----Original Message- > > > > From: ffmpeg-devel On Behalf > Of > > > > Soft Works > > > > Sent: Friday, 24 September 2021 19:03 > > > > To: FFmpeg development discussions and patches > > > de...@ffmpeg.org> > > > > Subject: Re: [FFmpeg-devel] Scaling PAL8 images with alpha > > > > > > > > > > > > > > > > > -Original Message- > > > > > From: ffmpeg-devel On > Behalf Of > > > > Michael > > > > > Niedermayer > > > > > Sent: Friday, 24 September 2021 17:40 > > > > > To: FFmpeg development discussions and patches > > > de...@ffmpeg.org> > > > > > Subject: Re: [FFmpeg-devel] Scaling PAL8 images with alpha > > > > > > > > > > On Fri, Sep 24, 2021 at 10:30:31AM +, Soft Works wrote: > > > > > > Hi, > > > > > > > > > > > > for a new filter, I want to rescale PAL8 subtitle bitmaps > where > > > > the palette > > > > > includes > > > > > > colors with alpha. > > > > > > > > > > > > From what I’ve seen, swscale doesn’t support PAL8-to-PAL8, > only > > > > PAL8-to- > > > > > BGR8 > > > > > > which doesn’t support alpha and the palette is fixed with > 256 > > > > entries > > > > > defined by > > > > > > convention, while I would ideally like to be able to allow > the > > > > following: > > > > > > > > > > > > - constrain the output to use the same palette as the input > > > > > > - adaptively quantize it to a palette with a configurable > number > > > > of colors > > > > > > > > > > > > Thus it's about the palette quantization (with or without > > > > dithering) after > > > > > > scaling in RGBA. (or some cool algorithmic trick I'm not > aware > > > > of) > > > > > > > > > > > > Is there any existing code that I could reuse? The closest > I > > > > could find > > > > > > is pngenc, but I'm wondering whether there's something > > > > else/better > > > > > > somewhere in the ffmpeg libs that I haven't seen? > > > > > > > > > > There are some non linear scaling filters which may make > sense when > > > > you do > > > > > not have the full linear space available see: > > > > > hqx, epx, xbr > > > > > you could also try some neural net stuff > > > > > > > > Photoshop can perfectly do what is needed, but their algorithms > are > > > > not > > > > public. > > > > > > I've put a few thing together to illustrate what I'm talking > about: > > > > > > > https://gist.github.com/softworkz/deef5c2a43d3d629c3e17f9e21544a8f > > > > > > > > > Meanwhile I found what I need: > https://github.com/ImageOptim/libimagequant > > > > > > Interestingly, they are comparing their lib specifically to the > > > Photoshop feature that I mentioned: > https://pngquant.org/vsphotoshop.html > > > In the 2000's, the PS implementation for image quantization, > optimization > > > and compression (profanely named "Save for Web") had remained > > > unparalleled for years, that's why it was one of my first > thoughts. > > > > > > I hadn't heard about libimagequant before, but it seems to do > exactly > > > what is needed for rescaling PAL8 to PAL8 at a high quality. > > > > > > Before spending any time on it, I wanted ask whether that library > > > would be acceptable for adding it as a (GPLv3) reference to the > > > project and as a dependency to my filter? > > > > > > > External dependencies for what is supposed to be a rather base > feature > > are not ideal. > > +1 > > i see no reason why this would need an external dependancy > we have filters implementing much more complex things than scaling a > pal8 > image I dislike GPL3 as well and needing a library for this might seem to be a bit too much for solving the problem. I'm just not sure whether it can be solved easily. I mean it can easily be solved in a basic way, but I'm not sure whether it would be possible to achieve the same quality in results. I've put up here some examples and also a comparison of results from palettegen+paletteuse and Photoshop(which I assume to be at the same level of quality like libimagequant). https://gist.github.com/softworkz/deef5c2a43d3d629c3e17f9e21544a8f Earlier versions of the library were published under a BSD-like license. Assuming it would do the job, would it be possible to include that code? (I'm not an expert in these things) Kind regards, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] Scaling PAL8 images with alpha
> -Original Message- > From: ffmpeg-devel On Behalf Of > Paul B Mahol > Sent: Saturday, 25 September 2021 17:51 > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] Scaling PAL8 images with alpha > > On Sat, Sep 25, 2021 at 5:46 PM Soft Works > wrote: > > > > > > > > -Original Message- > > > From: ffmpeg-devel On Behalf Of > > > Michael Niedermayer > > > Sent: Saturday, 25 September 2021 16:30 > > > To: FFmpeg development discussions and patches > > de...@ffmpeg.org> > > > Subject: Re: [FFmpeg-devel] Scaling PAL8 images with alpha > > > > > > On Sat, Sep 25, 2021 at 10:23:56AM +0200, Hendrik Leppkes wrote: > > > > On Sat, Sep 25, 2021 at 5:00 AM Soft Works > > > > wrote: > > > > > > > > > > > > > > > > > > > > > -Original Message- > > > > > > From: ffmpeg-devel On > Behalf > > > Of > > > > > > Soft Works > > > > > > Sent: Friday, 24 September 2021 19:03 > > > > > > To: FFmpeg development discussions and patches > > > > > de...@ffmpeg.org> > > > > > > Subject: Re: [FFmpeg-devel] Scaling PAL8 images with alpha > > > > > > > > > > > > > > > > > > > > > > > > > -Original Message- > > > > > > > From: ffmpeg-devel On > > > Behalf Of > > > > > > Michael > > > > > > > Niedermayer > > > > > > > Sent: Friday, 24 September 2021 17:40 > > > > > > > To: FFmpeg development discussions and patches > > > > > de...@ffmpeg.org> > > > > > > > Subject: Re: [FFmpeg-devel] Scaling PAL8 images with > alpha > > > > > > > > > > > > > > On Fri, Sep 24, 2021 at 10:30:31AM +, Soft Works > wrote: > > > > > > > > Hi, > > > > > > > > > > > > > > > > for a new filter, I want to rescale PAL8 subtitle > bitmaps > > > where > > > > > > the palette > > > > > > > includes > > > > > > > > colors with alpha. > > > > > > > > > > > > > > > > From what I’ve seen, swscale doesn’t support PAL8-to- > PAL8, > > > only > > > > > > PAL8-to- > > > > > > > BGR8 > > > > > > > > which doesn’t support alpha and the palette is fixed > with > > > 256 > > > > > > entries > > > > > > > defined by > > > > > > > > convention, while I would ideally like to be able to > allow > > > the > > > > > > following: > > > > > > > > > > > > > > > > - constrain the output to use the same palette as the > input > > > > > > > > - adaptively quantize it to a palette with a > configurable > > > number > > > > > > of colors > > > > > > > > > > > > > > > > Thus it's about the palette quantization (with or > without > > > > > > dithering) after > > > > > > > > scaling in RGBA. (or some cool algorithmic trick I'm > not > > > aware > > > > > > of) > > > > > > > > > > > > > > > > Is there any existing code that I could reuse? The > closest > > > I > > > > > > could find > > > > > > > > is pngenc, but I'm wondering whether there's something > > > > > > else/better > > > > > > > > somewhere in the ffmpeg libs that I haven't seen? > > > > > > > > > > > > > > There are some non linear scaling filters which may make > > > sense when > > > > > > you do > > > > > > > not have the full linear space available see: > > > > > > > hqx, epx, xbr > > > > > > > you could also try some neural net stuff > > > > > > > > > > > > Photoshop can perfectly do what is needed, but their > algorithms > > > are > > > > > > not > > > > > > public. > > > > > > > > > > I've put a few thing together to illustrate what
Re: [FFmpeg-devel] Scaling PAL8 images with alpha
> -Original Message- > From: ffmpeg-devel On Behalf Of > Michael Niedermayer > Sent: Saturday, 25 September 2021 22:47 > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] Scaling PAL8 images with alpha > > On Sat, Sep 25, 2021 at 10:34:32PM +0200, Michael Niedermayer wrote: > > On Sat, Sep 25, 2021 at 03:46:25PM +, Soft Works wrote: > > > > > > > > > > -Original Message- > > > > From: ffmpeg-devel On Behalf > Of > > > > Michael Niedermayer > > > > Sent: Saturday, 25 September 2021 16:30 > > > > To: FFmpeg development discussions and patches > > > de...@ffmpeg.org> > > > > Subject: Re: [FFmpeg-devel] Scaling PAL8 images with alpha > > > > > > > > On Sat, Sep 25, 2021 at 10:23:56AM +0200, Hendrik Leppkes > wrote: > > > > > On Sat, Sep 25, 2021 at 5:00 AM Soft Works > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > -Original Message- > > > > > > > From: ffmpeg-devel On > Behalf > > > > Of > > > > > > > Soft Works > > > > > > > Sent: Friday, 24 September 2021 19:03 > > > > > > > To: FFmpeg development discussions and patches > > > > > > de...@ffmpeg.org> > > > > > > > Subject: Re: [FFmpeg-devel] Scaling PAL8 images with > alpha > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -Original Message- > > > > > > > > From: ffmpeg-devel On > > > > Behalf Of > > > > > > > Michael > > > > > > > > Niedermayer > > > > > > > > Sent: Friday, 24 September 2021 17:40 > > > > > > > > To: FFmpeg development discussions and patches > > > > > > de...@ffmpeg.org> > > > > > > > > Subject: Re: [FFmpeg-devel] Scaling PAL8 images with > alpha > > > > > > > > > > > > > > > > On Fri, Sep 24, 2021 at 10:30:31AM +, Soft Works > wrote: > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > for a new filter, I want to rescale PAL8 subtitle > bitmaps > > > > where > > > > > > > the palette > > > > > > > > includes > > > > > > > > > colors with alpha. > > > > > > > > > > > > > > > > > > From what I’ve seen, swscale doesn’t support PAL8-to- > PAL8, > > > > only > > > > > > > PAL8-to- > > > > > > > > BGR8 > > > > > > > > > which doesn’t support alpha and the palette is fixed > with > > > > 256 > > > > > > > entries > > > > > > > > defined by > > > > > > > > > convention, while I would ideally like to be able to > allow > > > > the > > > > > > > following: > > > > > > > > > > > > > > > > > > - constrain the output to use the same palette as the > input > > > > > > > > > - adaptively quantize it to a palette with a > configurable > > > > number > > > > > > > of colors > > > > > > > > > > > > > > > > > > Thus it's about the palette quantization (with or > without > > > > > > > dithering) after > > > > > > > > > scaling in RGBA. (or some cool algorithmic trick I'm > not > > > > aware > > > > > > > of) > > > > > > > > > > > > > > > > > > Is there any existing code that I could reuse? The > closest > > > > I > > > > > > > could find > > > > > > > > > is pngenc, but I'm wondering whether there's > something > > > > > > > else/better > > > > > > > > > somewhere in the ffmpeg libs that I haven't seen? > > > > > > > > > > > > > > > > There are some non linear scaling filters which may > make > > > > sense when > > > > > > > you do > > > > > > >
[FFmpeg-devel] [PATCH] avfilter/palettegen, paletteuse: Extend the palette conversion filters to support palettes with alpha
Usage example:
ffmpeg -y -loglevel verbose -i "..\fate-suite\apng\o_sample.png"
-filter_complex
"split[split1][split2];[split1]palettegen=max_colors=254:use_alpha=1[pal1];[split2][pal1]paletteuse=use_alpha=1"
-frames:v 1 out.png
Signed-off-by: softworkz
---
doc/filters.texi| 8 ++
libavfilter/vf_palettegen.c | 140 ++
libavfilter/vf_paletteuse.c | 225 +---
3 files changed, 233 insertions(+), 140 deletions(-)
diff --git a/doc/filters.texi b/doc/filters.texi
index 36113e5c4b..7e4806235c 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -16454,6 +16454,9 @@ Compute new histogram for each frame.
@end table
Default value is @var{full}.
+@item use_alpha
+Create a palette of colors with alpha components.
+Setting this, will automatically disable 'reserve_transparent'.
@end table
The filter also exports the frame metadata @code{lavfi.color_quant_ratio}
@@ -16532,6 +16535,11 @@ will be treated as completely opaque, and values below
this threshold will be
treated as completely transparent.
The option must be an integer value in the range [0,255]. Default is @var{128}.
+
+@item use_alpha
+Apply the palette by taking alpha values into account. Only useful with
+palettes that are containing multiple colors with alpha components.
+Setting this will automatically disable 'alpha_treshold'.
@end table
@subsection Examples
diff --git a/libavfilter/vf_palettegen.c b/libavfilter/vf_palettegen.c
index 4c2fbd36d7..7a74a3752f 100644
--- a/libavfilter/vf_palettegen.c
+++ b/libavfilter/vf_palettegen.c
@@ -59,7 +59,7 @@ enum {
};
#define NBITS 5
-#define HIST_SIZE (1<<(3*NBITS))
+#define HIST_SIZE (1<<(4*NBITS))
typedef struct PaletteGenContext {
const AVClass *class;
@@ -67,6 +67,7 @@ typedef struct PaletteGenContext {
int max_colors;
int reserve_transparent;
int stats_mode;
+int use_alpha;
AVFrame *prev_frame;// previous frame used for the
diff stats_mode
struct hist_node histogram[HIST_SIZE]; // histogram/hashtable of the
colors
@@ -88,6 +89,7 @@ static const AVOption palettegen_options[] = {
{ "full", "compute full frame histograms", 0, AV_OPT_TYPE_CONST,
{.i64=STATS_MODE_ALL_FRAMES}, INT_MIN, INT_MAX, FLAGS, "mode" },
{ "diff", "compute histograms only for the part that differs from
previous frame", 0, AV_OPT_TYPE_CONST, {.i64=STATS_MODE_DIFF_FRAMES}, INT_MIN,
INT_MAX, FLAGS, "mode" },
{ "single", "compute new histogram for each frame", 0,
AV_OPT_TYPE_CONST, {.i64=STATS_MODE_SINGLE_FRAMES}, INT_MIN, INT_MAX, FLAGS,
"mode" },
+{ "use_alpha", "create a palette including alpha values",
OFFSET(use_alpha), AV_OPT_TYPE_BOOL, {.i64 = 0}, 0, 1, FLAGS },
{ NULL }
};
@@ -113,15 +115,16 @@ static int cmp_##name(const void *pa, const void *pb) \
{ \
const struct color_ref * const *a = pa; \
const struct color_ref * const *b = pb; \
-return (int)((*a)->color >> (8 * (2 - (pos))) & 0xff) \
- - (int)((*b)->color >> (8 * (2 - (pos))) & 0xff); \
+return (int)((*a)->color >> (8 * (3 - (pos))) & 0xff) \
+ - (int)((*b)->color >> (8 * (3 - (pos))) & 0xff); \
}
-DECLARE_CMP_FUNC(r, 0)
-DECLARE_CMP_FUNC(g, 1)
-DECLARE_CMP_FUNC(b, 2)
+DECLARE_CMP_FUNC(a, 0)
+DECLARE_CMP_FUNC(r, 1)
+DECLARE_CMP_FUNC(g, 2)
+DECLARE_CMP_FUNC(b, 3)
-static const cmp_func cmp_funcs[] = {cmp_r, cmp_g, cmp_b};
+static const cmp_func cmp_funcs[] = {cmp_a, cmp_r, cmp_g, cmp_b};
/**
* Simple color comparison for sorting the final palette
@@ -143,6 +146,17 @@ static av_always_inline int diff(const uint32_t a, const
uint32_t b)
return dr*dr + dg*dg + db*db;
}
+static av_always_inline int diff_alpha(const uint32_t a, const uint32_t b)
+{
+const uint8_t c1[] = {a >> 24 & 0xff, a >> 16 & 0xff, a >> 8 & 0xff, a &
0xff};
+const uint8_t c2[] = {b >> 24 & 0xff, b >> 16 & 0xff, b >> 8 & 0xff, b &
0xff};
+const int da = c1[0] - c2[0];
+const int dr = c1[1] - c2[1];
+const int dg = c1[2] - c2[2];
+const int db = c1[3] - c2[3];
+return da*da + dr*dr + dg*dg + db*db;
+}
+
/**
* Find the next box to split: pick the one with the highest variance
*/
@@ -164,7 +178,10 @@ static int get_next_box_id_to_split(PaletteGenContext *s)
for (i = 0; i < box->len; i++) {
const struct color_ref *ref = s->refs[box->start + i];
-variance += diff(ref->color, box->color) * ref->count;
+if (s->use_alpha)
+variance += (int64_t)diff_alpha(ref->color,
box->color) * ref->count;
+else
+variance += (int64_t)diff(ref->color, box->color) *
ref->count;
}
box->variance = variance;
}
@@ -184,24 +201,31 @@ static int get_next_box_id_to_split(Palett
[FFmpeg-devel] [PATCH] avfilter/elbg: Extend filter to include alpha values in the quantization procedure
Usage example:
ffmpeg -y -loglevel verbose -i "..\fate-suite\apng\o_sample.png"
-filter_complex "elbg=pal8=1:use_alpha=1" -frames:v 1 out.png
Signed-off-by: softworkz
---
doc/filters.texi | 4
libavfilter/vf_elbg.c | 25 -
2 files changed, 20 insertions(+), 9 deletions(-)
diff --git a/doc/filters.texi b/doc/filters.texi
index 7e4806235c..fa450b3eb4 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -11396,6 +11396,10 @@ will try to use a good random seed on a best effort
basis.
@item pal8
Set pal8 output pixel format. This option does not work with codebook
length greater than 256. Default is disabled.
+
+@item use_alpha
+Include alpha values in the quantization calculation. Allows creating
+palettized output images (e.g. PNG8) with multiple alpha smooth blending.
@end table
@section entropy
diff --git a/libavfilter/vf_elbg.c b/libavfilter/vf_elbg.c
index 0bebcdbda5..7f40be6092 100644
--- a/libavfilter/vf_elbg.c
+++ b/libavfilter/vf_elbg.c
@@ -46,6 +46,7 @@ typedef struct ELBGFilterContext {
int codebook_length;
const AVPixFmtDescriptor *pix_desc;
uint8_t rgba_map[4];
+int use_alpha;
int pal8;
} ELBGFilterContext;
@@ -60,6 +61,7 @@ static const AVOption elbg_options[] = {
{ "seed", "set the random seed", OFFSET(lfg_seed), AV_OPT_TYPE_INT64,
{.i64 = -1}, -1, UINT32_MAX, FLAGS },
{ "s","set the random seed", OFFSET(lfg_seed), AV_OPT_TYPE_INT64, {
.i64 = -1 }, -1, UINT32_MAX, FLAGS },
{ "pal8", "set the pal8 output", OFFSET(pal8), AV_OPT_TYPE_BOOL, { .i64 =
0 }, 0, 1, FLAGS },
+{ "use_alpha", "use alpha channel for mapping", OFFSET(use_alpha),
AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, FLAGS },
{ NULL }
};
@@ -105,7 +107,7 @@ static int query_formats(AVFilterContext *ctx)
return 0;
}
-#define NB_COMPONENTS 3
+#define NB_COMPONENTS 4
static int config_input(AVFilterLink *inlink)
{
@@ -138,6 +140,7 @@ static int config_input(AVFilterLink *inlink)
#define R 0
#define G 1
#define B 2
+#define A 3
static int filter_frame(AVFilterLink *inlink, AVFrame *frame)
{
@@ -148,6 +151,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame
*frame)
const uint8_t r_idx = elbg->rgba_map[R];
const uint8_t g_idx = elbg->rgba_map[G];
const uint8_t b_idx = elbg->rgba_map[B];
+const uint8_t a_idx = elbg->rgba_map[A];
/* build the codeword */
p0 = frame->data[0];
@@ -155,9 +159,10 @@ static int filter_frame(AVFilterLink *inlink, AVFrame
*frame)
for (i = 0; i < inlink->h; i++) {
p = p0;
for (j = 0; j < inlink->w; j++) {
-elbg->codeword[k++] = p[r_idx];
-elbg->codeword[k++] = p[g_idx];
elbg->codeword[k++] = p[b_idx];
+elbg->codeword[k++] = p[g_idx];
+elbg->codeword[k++] = p[r_idx];
+elbg->codeword[k++] = elbg->use_alpha ? p[a_idx] : 0xff;
p += elbg->pix_desc->nb_components;
}
p0 += frame->linesize[0];
@@ -188,10 +193,11 @@ static int filter_frame(AVFilterLink *inlink, AVFrame
*frame)
p0 = (uint8_t *)out->data[0];
for (i = 0; i < elbg->codebook_length; i++) {
-pal[i] = 0xFFU << 24 |
- (elbg->codebook[i*3 ] << 16) |
- (elbg->codebook[i*3+1] << 8) |
- elbg->codebook[i*3+2];
+const int al = elbg->use_alpha ? elbg->codebook[i*4+3] : 0xff;
+pal[i] = al<< 24 |
+ (elbg->codebook[i*4+2] << 16) |
+ (elbg->codebook[i*4+1] << 8) |
+ elbg->codebook[i*4 ];
}
k = 0;
@@ -214,9 +220,10 @@ static int filter_frame(AVFilterLink *inlink, AVFrame
*frame)
p = p0;
for (j = 0; j < inlink->w; j++) {
int cb_idx = NB_COMPONENTS *
elbg->codeword_closest_codebook_idxs[k++];
-p[r_idx] = elbg->codebook[cb_idx];
+p[b_idx] = elbg->codebook[cb_idx];
p[g_idx] = elbg->codebook[cb_idx+1];
-p[b_idx] = elbg->codebook[cb_idx+2];
+p[r_idx] = elbg->codebook[cb_idx+2];
+p[a_idx] = elbg->use_alpha ? elbg->codebook[cb_idx+3] : 0xFFu;
p += elbg->pix_desc->nb_components;
}
p0 += frame->linesize[0];
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] Scaling PAL8 images with alpha
> -Original Message- > From: ffmpeg-devel On Behalf Of > Soft Works > Sent: Sunday, 26 September 2021 01:00 > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] Scaling PAL8 images with alpha > > > > > -Original Message- > > From: ffmpeg-devel On Behalf Of > > Michael Niedermayer > > Sent: Saturday, 25 September 2021 22:47 > > To: FFmpeg development discussions and patches > de...@ffmpeg.org> > > Subject: Re: [FFmpeg-devel] Scaling PAL8 images with alpha > > > > On Sat, Sep 25, 2021 at 10:34:32PM +0200, Michael Niedermayer > wrote: > > > On Sat, Sep 25, 2021 at 03:46:25PM +, Soft Works wrote: > > > > > > > > > > > > > -Original Message- > > > > > From: ffmpeg-devel On > Behalf > > Of > > > > > Michael Niedermayer > > > > > Sent: Saturday, 25 September 2021 16:30 > > > > > To: FFmpeg development discussions and patches > > > > de...@ffmpeg.org> > > > > > Subject: Re: [FFmpeg-devel] Scaling PAL8 images with alpha > > > > > > > > > > On Sat, Sep 25, 2021 at 10:23:56AM +0200, Hendrik Leppkes > > wrote: > > > > > > On Sat, Sep 25, 2021 at 5:00 AM Soft Works > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -Original Message- > > > > > > > > From: ffmpeg-devel On > > Behalf > > > > > Of > > > > > > > > Soft Works > > > > > > > > Sent: Friday, 24 September 2021 19:03 > > > > > > > > To: FFmpeg development discussions and patches > > > > > > > de...@ffmpeg.org> > > > > > > > > Subject: Re: [FFmpeg-devel] Scaling PAL8 images with > > alpha > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -Original Message- > > > > > > > > > From: ffmpeg-devel > On > > > > > Behalf Of > > > > > > > > Michael > > > > > > > > > Niedermayer > > > > > > > > > Sent: Friday, 24 September 2021 17:40 > > > > > > > > > To: FFmpeg development discussions and patches > > > > > > > > de...@ffmpeg.org> > > > > > > > > > Subject: Re: [FFmpeg-devel] Scaling PAL8 images with > > alpha > > > > > > > > > > > > > > > > > > On Fri, Sep 24, 2021 at 10:30:31AM +, Soft Works > > wrote: > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > for a new filter, I want to rescale PAL8 subtitle > > bitmaps > > > > > where > > > > > > > > the palette > > > > > > > > > includes > > > > > > > > > > colors with alpha. > > > > > > > > > > > > > > > > > > > > From what I’ve seen, swscale doesn’t support PAL8- > to- > > PAL8, > > > > > only > > > > > > > > PAL8-to- > > > > > > > > > BGR8 > > > > > > > > > > which doesn’t support alpha and the palette is > fixed > > with > > > > > 256 > > > > > > > > entries > > > > > > > > > defined by > > > > > > > > > > convention, while I would ideally like to be able > to > > allow > > > > > the > > > > > > > > following: > > > > > > > > > > > > > > > > > > > > - constrain the output to use the same palette as > the > > input > > > > > > > > > > - adaptively quantize it to a palette with a > > configurable > > > > > number > > > > > > > > of colors > > > > > > > > > > > > > > > > > > > > Thus it's about the palette quantization (with or > > without > > > > > > > > dithering) after > > > > > > > > > > scaling in RGBA. (or some cool algorithmic trick > I'm > > not > >
Re: [FFmpeg-devel] [PATCH] avfilter/palettegen, paletteuse: Extend the palette conversion filters to support palettes with alpha
> -Original Message- > From: ffmpeg-devel On Behalf Of > Michael Niedermayer > Sent: Sunday, 26 September 2021 19:01 > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH] avfilter/palettegen, paletteuse: > Extend the palette conversion filters to support palettes with alpha > > > [...] > > -const int s = kd->split; > > +const int split = kd->split; > > also unrelated to the alpha change This change is required due to the newly added parameter 's'. The context is named 's' in all other functions, that's why was required to name the previous local 's' variable. Thanks, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2 1/2] avfilter/palettegen, paletteuse: Extend the palette conversion filters to support palettes with alpha
Usage example:
ffmpeg -y -loglevel verbose -i "..\fate-suite\apng\o_sample.png"
-filter_complex
"split[split1][split2];[split1]palettegen=max_colors=254:use_alpha=1[pal1];[split2][pal1]paletteuse=use_alpha=1"
-frames:v 1 out.png
Signed-off-by: softworkz
---
doc/filters.texi| 8 ++
libavfilter/vf_palettegen.c | 136 +++---
libavfilter/vf_paletteuse.c | 225 +---
3 files changed, 231 insertions(+), 138 deletions(-)
diff --git a/doc/filters.texi b/doc/filters.texi
index 36113e5c4b..7e4806235c 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -16454,6 +16454,9 @@ Compute new histogram for each frame.
@end table
Default value is @var{full}.
+@item use_alpha
+Create a palette of colors with alpha components.
+Setting this, will automatically disable 'reserve_transparent'.
@end table
The filter also exports the frame metadata @code{lavfi.color_quant_ratio}
@@ -16532,6 +16535,11 @@ will be treated as completely opaque, and values below
this threshold will be
treated as completely transparent.
The option must be an integer value in the range [0,255]. Default is @var{128}.
+
+@item use_alpha
+Apply the palette by taking alpha values into account. Only useful with
+palettes that are containing multiple colors with alpha components.
+Setting this will automatically disable 'alpha_treshold'.
@end table
@subsection Examples
diff --git a/libavfilter/vf_palettegen.c b/libavfilter/vf_palettegen.c
index 4c2fbd36d7..98dff46fe0 100644
--- a/libavfilter/vf_palettegen.c
+++ b/libavfilter/vf_palettegen.c
@@ -59,7 +59,7 @@ enum {
};
#define NBITS 5
-#define HIST_SIZE (1<<(3*NBITS))
+#define HIST_SIZE (1<<(4*NBITS))
typedef struct PaletteGenContext {
const AVClass *class;
@@ -67,6 +67,7 @@ typedef struct PaletteGenContext {
int max_colors;
int reserve_transparent;
int stats_mode;
+int use_alpha;
AVFrame *prev_frame;// previous frame used for the
diff stats_mode
struct hist_node histogram[HIST_SIZE]; // histogram/hashtable of the
colors
@@ -88,6 +89,7 @@ static const AVOption palettegen_options[] = {
{ "full", "compute full frame histograms", 0, AV_OPT_TYPE_CONST,
{.i64=STATS_MODE_ALL_FRAMES}, INT_MIN, INT_MAX, FLAGS, "mode" },
{ "diff", "compute histograms only for the part that differs from
previous frame", 0, AV_OPT_TYPE_CONST, {.i64=STATS_MODE_DIFF_FRAMES}, INT_MIN,
INT_MAX, FLAGS, "mode" },
{ "single", "compute new histogram for each frame", 0,
AV_OPT_TYPE_CONST, {.i64=STATS_MODE_SINGLE_FRAMES}, INT_MIN, INT_MAX, FLAGS,
"mode" },
+{ "use_alpha", "create a palette including alpha values",
OFFSET(use_alpha), AV_OPT_TYPE_BOOL, {.i64 = 0}, 0, 1, FLAGS },
{ NULL }
};
@@ -113,15 +115,16 @@ static int cmp_##name(const void *pa, const void *pb) \
{ \
const struct color_ref * const *a = pa; \
const struct color_ref * const *b = pb; \
-return (int)((*a)->color >> (8 * (2 - (pos))) & 0xff) \
- - (int)((*b)->color >> (8 * (2 - (pos))) & 0xff); \
+return (int)((*a)->color >> (8 * (3 - (pos))) & 0xff) \
+ - (int)((*b)->color >> (8 * (3 - (pos))) & 0xff); \
}
-DECLARE_CMP_FUNC(r, 0)
-DECLARE_CMP_FUNC(g, 1)
-DECLARE_CMP_FUNC(b, 2)
+DECLARE_CMP_FUNC(a, 0)
+DECLARE_CMP_FUNC(r, 1)
+DECLARE_CMP_FUNC(g, 2)
+DECLARE_CMP_FUNC(b, 3)
-static const cmp_func cmp_funcs[] = {cmp_r, cmp_g, cmp_b};
+static const cmp_func cmp_funcs[] = {cmp_a, cmp_r, cmp_g, cmp_b};
/**
* Simple color comparison for sorting the final palette
@@ -143,6 +146,17 @@ static av_always_inline int diff(const uint32_t a, const
uint32_t b)
return dr*dr + dg*dg + db*db;
}
+static av_always_inline int diff_alpha(const uint32_t a, const uint32_t b)
+{
+const uint8_t c1[] = {a >> 24 & 0xff, a >> 16 & 0xff, a >> 8 & 0xff, a &
0xff};
+const uint8_t c2[] = {b >> 24 & 0xff, b >> 16 & 0xff, b >> 8 & 0xff, b &
0xff};
+const int da = c1[0] - c2[0];
+const int dr = c1[1] - c2[1];
+const int dg = c1[2] - c2[2];
+const int db = c1[3] - c2[3];
+return da*da + dr*dr + dg*dg + db*db;
+}
+
/**
* Find the next box to split: pick the one with the highest variance
*/
@@ -164,7 +178,10 @@ static int get_next_box_id_to_split(PaletteGenContext *s)
for (i = 0; i < box->len; i++) {
const struct color_ref *ref = s->refs[box->start + i];
-variance += diff(ref->color, box->color) * ref->count;
+if (s->use_alpha)
+variance += (int64_t)diff_alpha(ref->color,
box->color) * ref->count;
+else
+variance += (int64_t)diff(ref->color, box->color) *
ref->count;
}
box->variance = variance;
}
@@ -184,24 +201,31 @@ static int get_next_box_id_to_split(Palett
[FFmpeg-devel] [PATCH v2 2/2] avfilter/palettegen: Cosmetic changes
Signed-off-by: softworkz
---
libavfilter/vf_palettegen.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/libavfilter/vf_palettegen.c b/libavfilter/vf_palettegen.c
index 98dff46fe0..7a74a3752f 100644
--- a/libavfilter/vf_palettegen.c
+++ b/libavfilter/vf_palettegen.c
@@ -266,7 +266,7 @@ static void write_palette(AVFilterContext *ctx, AVFrame
*out)
if (box_id < s->nb_boxes) {
pal[x] = s->boxes[box_id++].color;
if ((x || y) && pal[x] == last_color)
-av_log(ctx, AV_LOG_WARNING, "Dupped color: %08"PRIX32"\n",
pal[x]);
+av_log(ctx, AV_LOG_WARNING, "Duped color: %08"PRIX32"\n",
pal[x]);
last_color = pal[x];
} else {
pal[x] = last_color; // pad with last color
@@ -438,7 +438,7 @@ static inline unsigned color_hash(uint32_t color, int
use_alpha)
return a << (NBITS * 3) | r << (NBITS * 2) | g << NBITS | b;
}
-return r<<(NBITS*2) | g
Re: [FFmpeg-devel] [PATCH] avfilter/palettegen, paletteuse: Extend the palette conversion filters to support palettes with alpha
> -Original Message- > From: ffmpeg-devel On Behalf Of > Michael Niedermayer > Sent: Sunday, 26 September 2021 19:01 > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH] avfilter/palettegen, paletteuse: > Extend the palette conversion filters to support palettes with alpha > A general coding question on the use of the comma operator: if (dx <= 0) nearer_kd_id = kd->left_id, further_kd_id = kd->right_id; else nearer_kd_id = kd->right_id, further_kd_id = kd->left_id; I was tempted to change this, but I'm not sure how it is considered by the developers here? softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] avfilter/elbg: Extend filter to include alpha values in the quantization procedure
> -Original Message- > From: ffmpeg-devel On Behalf Of > Michael Niedermayer > Sent: Sunday, 26 September 2021 18:52 > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH] avfilter/elbg: Extend filter to > include alpha values in the quantization procedure > > On Sun, Sep 26, 2021 at 04:57:25AM +, Soft Works wrote: > > Usage example: > > > > ffmpeg -y -loglevel verbose -i "..\fate-suite\apng\o_sample.png" - > filter_complex "elbg=pal8=1:use_alpha=1" -frames:v 1 out.png > > > > Signed-off-by: softworkz > > --- > > doc/filters.texi | 4 > > libavfilter/vf_elbg.c | 25 - > > 2 files changed, 20 insertions(+), 9 deletions(-) > > will apply Thank you. Please allow me another note: I think the naming of this filter is terrible. I've gone through the list of all ffmpeg filters many more times than the typical user, but without the specific pointers I had gotten from you and a few other devs, I would have never identified the "elbg" filter as something that would do what I'm looking for. Given the great results that this algorithm can provide, it's a bit sad when this filter would remain to be a widely unknown hidden gem in the future. No matter whether, 'posterize', 'color_quantize', 'elbg_palettize' - everything would be better than 'elbg'... Kind regards, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH] avfilter/frames: Ensure frames are writable when processing in-place
With the introduction of subtitle filtering, subsequent video frames
which are sharing the same data won't be as rare as this happened
to occur yet.
Ensuring per-frame uniqueness when data is modified is not only important
to avoid issues in the future, but a common API requirement anyway.
Signed-off-by: softworkz
---
libavfilter/vf_chromakey.c| 5 +
libavfilter/vf_codecview.c| 6 ++
libavfilter/vf_colorcontrast.c| 5 +
libavfilter/vf_colorcorrect.c | 6 ++
libavfilter/vf_colorize.c | 6 ++
libavfilter/vf_colorkey.c | 5 +
libavfilter/vf_colortemperature.c | 6 ++
libavfilter/vf_cover_rect.c | 7 +--
libavfilter/vf_datascope.c| 7 ++-
libavfilter/vf_despill.c | 5 +
libavfilter/vf_exposure.c | 6 ++
libavfilter/vf_fade.c | 6 ++
libavfilter/vf_fillborders.c | 6 ++
libavfilter/vf_floodfill.c| 5 +
libavfilter/vf_hsvkey.c | 5 +
libavfilter/vf_lumakey.c | 5 +
libavfilter/vf_subtitles.c| 7 ++-
libavfilter/vf_swaprect.c | 5 +
libavfilter/vf_vflip.c| 7 ++-
libavfilter/vf_vibrance.c | 5 +
20 files changed, 110 insertions(+), 5 deletions(-)
diff --git a/libavfilter/vf_chromakey.c b/libavfilter/vf_chromakey.c
index 304cb9ee6c..cf28cbadcb 100644
--- a/libavfilter/vf_chromakey.c
+++ b/libavfilter/vf_chromakey.c
@@ -260,6 +260,11 @@ static int filter_frame(AVFilterLink *link, AVFrame *frame)
ChromakeyContext *ctx = avctx->priv;
int res;
+if ((res = av_frame_make_writable(frame)) < 0) {
+av_frame_free(&frame);
+return res;
+}
+
if (res = ff_filter_execute(avctx, ctx->do_slice, frame, NULL,
FFMIN(frame->height,
ff_filter_get_nb_threads(avctx
return res;
diff --git a/libavfilter/vf_codecview.c b/libavfilter/vf_codecview.c
index dc3d3acd82..61416ae733 100644
--- a/libavfilter/vf_codecview.c
+++ b/libavfilter/vf_codecview.c
@@ -215,7 +215,13 @@ static int filter_frame(AVFilterLink *inlink, AVFrame
*frame)
AVFilterContext *ctx = inlink->dst;
CodecViewContext *s = ctx->priv;
AVFilterLink *outlink = ctx->outputs[0];
+int ret;
+if ((ret = av_frame_make_writable(frame)) < 0) {
+av_frame_free(&frame);
+return ret;
+}
+
if (s->qp) {
int qstride, qp_type, ret;
int8_t *qp_table;
diff --git a/libavfilter/vf_colorcontrast.c b/libavfilter/vf_colorcontrast.c
index e89b3e7af4..410a51cba7 100644
--- a/libavfilter/vf_colorcontrast.c
+++ b/libavfilter/vf_colorcontrast.c
@@ -303,6 +303,11 @@ static int filter_frame(AVFilterLink *link, AVFrame *frame)
ColorContrastContext *s = ctx->priv;
int res;
+if ((res = av_frame_make_writable(frame)) < 0) {
+av_frame_free(&frame);
+return res;
+}
+
if (res = ff_filter_execute(ctx, s->do_slice, frame, NULL,
FFMIN(frame->height,
ff_filter_get_nb_threads(ctx
return res;
diff --git a/libavfilter/vf_colorcorrect.c b/libavfilter/vf_colorcorrect.c
index 2570bb105c..ce18413697 100644
--- a/libavfilter/vf_colorcorrect.c
+++ b/libavfilter/vf_colorcorrect.c
@@ -398,6 +398,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame
*frame)
AVFilterContext *ctx = inlink->dst;
ColorCorrectContext *s = ctx->priv;
const int nb_threads = s->analyze == MEDIAN ? 1 : FFMIN(s->planeheight[1],
ff_filter_get_nb_threads(ctx));
+int ret;
if (s->analyze) {
const int nb_athreads = s->analyze == MEDIAN ? 1 : nb_threads;
@@ -423,6 +424,11 @@ static int filter_frame(AVFilterLink *inlink, AVFrame
*frame)
s->rh = -rh;
}
+if ((ret = av_frame_make_writable(frame)) < 0) {
+av_frame_free(&frame);
+return ret;
+}
+
ff_filter_execute(ctx, s->do_slice, frame, NULL, nb_threads);
return ff_filter_frame(ctx->outputs[0], frame);
diff --git a/libavfilter/vf_colorize.c b/libavfilter/vf_colorize.c
index 4b57998a22..3f14547c00 100644
--- a/libavfilter/vf_colorize.c
+++ b/libavfilter/vf_colorize.c
@@ -199,6 +199,12 @@ static int filter_frame(AVFilterLink *inlink, AVFrame
*frame)
AVFilterContext *ctx = inlink->dst;
ColorizeContext *s = ctx->priv;
float c[3];
+int ret;
+
+if ((ret = av_frame_make_writable(frame)) < 0) {
+av_frame_free(&frame);
+return ret;
+}
hsl2rgb(s->hue, s->saturation, s->lightness, &c[0], &c[1], &c[2]);
rgb2yuv(c[0], c[1], c[2], &s->c[0], &s->c[1], &s->c[2], s->depth);
diff --git a/libavfilter/vf_colorkey.c b/libavfilter/vf_colorkey.c
index bef3e37414..45a61df104 100644
--- a/libavfilter/vf_colorkey.c
+++ b/libavfilter/vf_colorkey.c
@@ -135,6 +135,11 @@ static int filter_frame(AVFilterLink *link, AVFrame *frame)
ColorkeyContext *ctx = avctx->priv;
int res;
+if ((res = av_frame_make_writable(
Re: [FFmpeg-devel] [PATCH] avfilter/frames: Ensure frames are writable when processing in-place
> -Original Message- > From: ffmpeg-devel On Behalf Of > Soft Works > Sent: Monday, 27 September 2021 14:07 > To: ffmpeg-devel@ffmpeg.org > Subject: [FFmpeg-devel] [PATCH] avfilter/frames: Ensure frames are > writable when processing in-place > > With the introduction of subtitle filtering, subsequent video frames > which are sharing the same data won't be as rare as this happened > to occur yet. > Ensuring per-frame uniqueness when data is modified is not only > important > to avoid issues in the future, but a common API requirement anyway. IIUC ;-) ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] avfilter/frames: Ensure frames are writable when processing in-place
> -Original Message- > From: ffmpeg-devel On Behalf Of > Nicolas George > Sent: Monday, 27 September 2021 14:10 > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH] avfilter/frames: Ensure frames > are writable when processing in-place > > Soft Works (12021-09-27): > > With the introduction of subtitle filtering, subsequent video > frames > > which are sharing the same data won't be as rare as this happened > > to occur yet. > > Ensuring per-frame uniqueness when data is modified is not only > important > > to avoid issues in the future, but a common API requirement anyway. > > > > Signed-off-by: softworkz > > --- > > NAK, redundant. Why? ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] avfilter/frames: Ensure frames are writable when processing in-place
> -Original Message- > From: ffmpeg-devel On Behalf Of > Soft Works > Sent: Monday, 27 September 2021 14:12 > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH] avfilter/frames: Ensure frames > are writable when processing in-place > > > > > -Original Message- > > From: ffmpeg-devel On Behalf Of > > Nicolas George > > Sent: Monday, 27 September 2021 14:10 > > To: FFmpeg development discussions and patches > de...@ffmpeg.org> > > Subject: Re: [FFmpeg-devel] [PATCH] avfilter/frames: Ensure frames > > are writable when processing in-place > > > > Soft Works (12021-09-27): > > > With the introduction of subtitle filtering, subsequent video > > frames > > > which are sharing the same data won't be as rare as this happened > > > to occur yet. > > > Ensuring per-frame uniqueness when data is modified is not only > > important > > > to avoid issues in the future, but a common API requirement > anyway. > > > > > > Signed-off-by: softworkz > > > --- > > > > NAK, redundant. > > Why? You mean because of AVFILTERPAD_FLAG_NEEDS_WRITABLE? It should be deprecated IMO, it's not a good API design. softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] avfilter/elbg: Extend filter to include alpha values in the quantization procedure
> -Original Message- > From: ffmpeg-devel On Behalf Of > Paul B Mahol > Sent: Monday, 27 September 2021 18:37 > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH] avfilter/elbg: Extend filter to > include alpha values in the quantization procedure > > On Mon, Sep 27, 2021 at 5:30 PM Michael Niedermayer > > wrote: > > > On Sun, Sep 26, 2021 at 05:56:36PM +, Soft Works wrote: > > > > > > > > > > -Original Message- > > > > From: ffmpeg-devel On Behalf > Of > > > > Michael Niedermayer > > > > Sent: Sunday, 26 September 2021 18:52 > > > > To: FFmpeg development discussions and patches > > > de...@ffmpeg.org> > > > > Subject: Re: [FFmpeg-devel] [PATCH] avfilter/elbg: Extend > filter to > > > > include alpha values in the quantization procedure > > > > > > > > On Sun, Sep 26, 2021 at 04:57:25AM +, Soft Works wrote: > > > > > Usage example: > > > > > > > > > > ffmpeg -y -loglevel verbose -i "..\fate- > suite\apng\o_sample.png" - > > > > filter_complex "elbg=pal8=1:use_alpha=1" -frames:v 1 out.png > > > > > > > > > > Signed-off-by: softworkz > > > > > --- > > > > > doc/filters.texi | 4 > > > > > libavfilter/vf_elbg.c | 25 - > > > > > 2 files changed, 20 insertions(+), 9 deletions(-) > > > > > > > > will apply > > > > > > Thank you. > > > > > > Please allow me another note: I think the naming of this filter > > > is terrible. I've gone through the list of all ffmpeg filters > > > many more times than the typical user, but without the specific > > > pointers I had gotten from you and a few other devs, I would > > > have never identified the "elbg" filter as something that > > > would do what I'm looking for. > > > > i would tend to agree, yes, in fact i think > > finding filters based on what they do is hard > > and iam not sure their names alone should be how they need to be > found > > The names can't be it alone, but good naming would be a helpful element. > Every filter have short description. So next time use grep or search > feature. You have added a large range of audio filters and I'm sure many of them are very useful, but I'm afraid that often I do not have the slightest clue about what they are doing, not even when reading the description text ;-) softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] avfilter/elbg: Extend filter to include alpha values in the quantization procedure
> -Original Message- > From: ffmpeg-devel On Behalf Of > Paul B Mahol > Sent: Tuesday, 28 September 2021 18:05 > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH] avfilter/elbg: Extend filter to > include alpha values in the quantization procedure > > On Tue, Sep 28, 2021 at 5:43 PM Soft Works > wrote: > > > > > > > > -Original Message- > > > From: ffmpeg-devel On Behalf Of > > > Paul B Mahol > > > Sent: Monday, 27 September 2021 18:37 > > > To: FFmpeg development discussions and patches > > de...@ffmpeg.org> > > > Subject: Re: [FFmpeg-devel] [PATCH] avfilter/elbg: Extend filter > to > > > include alpha values in the quantization procedure > > > > > > On Mon, Sep 27, 2021 at 5:30 PM Michael Niedermayer > > > > > > wrote: > > > > > > > On Sun, Sep 26, 2021 at 05:56:36PM +, Soft Works wrote: > > > > > > > > > > > > > > > > -Original Message- > > > > > > From: ffmpeg-devel On > Behalf > > > Of > > > > > > Michael Niedermayer > > > > > > Sent: Sunday, 26 September 2021 18:52 > > > > > > To: FFmpeg development discussions and patches > > > > > de...@ffmpeg.org> > > > > > > Subject: Re: [FFmpeg-devel] [PATCH] avfilter/elbg: Extend > > > filter to > > > > > > include alpha values in the quantization procedure > > > > > > > > > > > > On Sun, Sep 26, 2021 at 04:57:25AM +, Soft Works wrote: > > > > > > > Usage example: > > > > > > > > > > > > > > ffmpeg -y -loglevel verbose -i "..\fate- > > > suite\apng\o_sample.png" - > > > > > > filter_complex "elbg=pal8=1:use_alpha=1" -frames:v 1 > out.png > > > > > > > > > > > > > > Signed-off-by: softworkz > > > > > > > --- > > > > > > > doc/filters.texi | 4 > > > > > > > libavfilter/vf_elbg.c | 25 - > > > > > > > 2 files changed, 20 insertions(+), 9 deletions(-) > > > > > > > > > > > > will apply > > > > > > > > > > Thank you. > > > > > > > > > > Please allow me another note: I think the naming of this > filter > > > > > is terrible. I've gone through the list of all ffmpeg filters > > > > > many more times than the typical user, but without the > specific > > > > > pointers I had gotten from you and a few other devs, I would > > > > > have never identified the "elbg" filter as something that > > > > > would do what I'm looking for. > > > > > > > > i would tend to agree, yes, in fact i think > > > > finding filters based on what they do is hard > > > > and iam not sure their names alone should be how they need to > be > > > found > > > > > > > > The names can't be it alone, but good naming would be a helpful > > element. > > > > > > > Every filter have short description. So next time use grep or > search > > > feature. > > > > You have added a large range of audio filters and I'm sure many of > > them are very useful, but I'm afraid that often I do not have the > > slightest clue about what they are doing, not even when reading the > > description text ;-) > > > > Are you audio engineer? Follow documentation. Accidentally, I'm quite familiar with audio production and studio technology, that's why I'm sure that even for audio engineers it's not always clear what the filters are doing. The documentation is often too minimal and it would be nice when it would include example use cases and example command lines. Same as for the elbg filter, I don't mean it in a negative way. What I mean is that it's somewhat unfortunate to put so much effort in developing those filters, when their benefits remain to be invisible for the majority of users. softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] avfilter/elbg: Extend filter to include alpha values in the quantization procedure
> -Original Message- > From: ffmpeg-devel On Behalf Of > Paul B Mahol > Sent: Tuesday, 28 September 2021 19:35 > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH] avfilter/elbg: Extend filter to > include alpha values in the quantization procedure > > On Tue, Sep 28, 2021 at 7:16 PM Soft Works > wrote: > > > > > > > > -Original Message- > > > From: ffmpeg-devel On Behalf Of > > > Paul B Mahol > > > Sent: Tuesday, 28 September 2021 18:05 > > > To: FFmpeg development discussions and patches > > de...@ffmpeg.org> > > > Subject: Re: [FFmpeg-devel] [PATCH] avfilter/elbg: Extend filter > to > > > include alpha values in the quantization procedure > > > [..] > > > Are you audio engineer? Follow documentation. > > > > Accidentally, I'm quite familiar with audio production and studio > > technology, that's why I'm sure that even for audio engineers it's > > not always clear what the filters are doing. The documentation is > > often too minimal and it would be nice when it would include > > example use cases and example command lines. > > Same as for the elbg filter, I don't mean it in a negative way. > > What I mean is that it's somewhat unfortunate to put so much effort > > in developing those filters, when their benefits remain to be > invisible > > for the majority of users. > > > > Give examples, propose improvements, elbg is not only filter. It's not my current focus, but I think it would be a good idea in general when the .texi docs would include at least one working example for each filter that demonstrates the provided capability at its best. By 'working example', I mean a command line that is not just using placeholders (like '-I INPUT') but specific public streams like from https://streams.videolan.org/ffmpeg/ I've done that for the new subtitle filters and I think it would be very helpful for users to better understand a filter's benefits when there's an example that can be run and allows to look at the results. Kind regards, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2] DXVA2: Add ARGB format
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Soft Works
> Sent: Thursday, 26 August 2021 21:43
> To: FFmpeg development discussions and patches de...@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v2] DXVA2: Add ARGB format
>
>
>
> > -Original Message-
> > From: ffmpeg-devel On Behalf Of
> > Soft Works
> > Sent: Saturday, 7 August 2021 07:02
> > To: ffmpeg-devel@ffmpeg.org
> > Subject: [FFmpeg-devel] [PATCH v2] DXVA2: Add ARGB format
> >
> > Required for uploading frames with alpha for qsv_overlay
> > (v2: remove tab indent)
> >
> > Signed-off-by: softworkz
> > ---
> > libavutil/hwcontext_dxva2.c | 1 +
> > 1 file changed, 1 insertion(+)
> >
> > diff --git a/libavutil/hwcontext_dxva2.c
> > b/libavutil/hwcontext_dxva2.c
> > index 63b037da4a..9987cdc578 100644
> > --- a/libavutil/hwcontext_dxva2.c
> > +++ b/libavutil/hwcontext_dxva2.c
> > @@ -83,6 +83,7 @@ static const struct {
> > { MKTAG('N', 'V', '1', '2'), AV_PIX_FMT_NV12 },
> > { MKTAG('P', '0', '1', '0'), AV_PIX_FMT_P010 },
> > { D3DFMT_P8, AV_PIX_FMT_PAL8 },
> > +{ D3DFMT_A8R8G8B8, AV_PIX_FMT_BGRA },
> > };
> >
> > DEFINE_GUID(video_decoder_service, 0xfc51a551, 0xd5e7, 0x11d9,
> > 0xaf, 0x55, 0x00, 0x05, 0x4e, 0x43, 0xff, 0x02);
> > --
>
> Could somebody push this please?
>
> Thanks,
> softworkz
Friendly ping.
sw
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure frames are writable when processing in-place
Signed-off-by: softworkz
---
v2: Reduced to cases without AVFILTERPAD_FLAG_NEEDS_WRITABLE
libavfilter/vf_cover_rect.c | 7 +--
libavfilter/vf_floodfill.c | 5 +
libavfilter/vf_vflip.c | 7 ++-
3 files changed, 16 insertions(+), 3 deletions(-)
diff --git a/libavfilter/vf_cover_rect.c b/libavfilter/vf_cover_rect.c
index 0a8c10e06d..2367afb4b3 100644
--- a/libavfilter/vf_cover_rect.c
+++ b/libavfilter/vf_cover_rect.c
@@ -136,7 +136,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in)
AVFilterContext *ctx = inlink->dst;
CoverContext *cover = ctx->priv;
AVDictionaryEntry *ex, *ey, *ew, *eh;
-int x = -1, y = -1, w = -1, h = -1;
+int x = -1, y = -1, w = -1, h = -1, ret;
char *xendptr = NULL, *yendptr = NULL, *wendptr = NULL, *hendptr = NULL;
ex = av_dict_get(in->metadata, "lavfi.rect.x", NULL, AV_DICT_MATCH_CASE);
@@ -181,7 +181,10 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in)
x = av_clip(x, 0, in->width - w);
y = av_clip(y, 0, in->height - h);
-av_frame_make_writable(in);
+if ((ret = av_frame_make_writable(in)) < 0) {
+av_frame_free(&in);
+return ret;
+}
if (cover->mode == MODE_BLUR) {
blur (cover, in, x, y);
diff --git a/libavfilter/vf_floodfill.c b/libavfilter/vf_floodfill.c
index 21741cdb4f..292b27505e 100644
--- a/libavfilter/vf_floodfill.c
+++ b/libavfilter/vf_floodfill.c
@@ -294,6 +294,11 @@ static int filter_frame(AVFilterLink *link, AVFrame *frame)
const int h = frame->height;
int i, ret;
+if ((ret = av_frame_make_writable(frame)) < 0) {
+av_frame_free(&frame);
+return ret;
+}
+
if (is_inside(s->x, s->y, w, h)) {
s->pick_pixel(frame, s->x, s->y, &s0, &s1, &s2, &s3);
diff --git a/libavfilter/vf_vflip.c b/libavfilter/vf_vflip.c
index 0d624512f9..622bd46db3 100644
--- a/libavfilter/vf_vflip.c
+++ b/libavfilter/vf_vflip.c
@@ -108,11 +108,16 @@ static int flip_bayer(AVFilterLink *link, AVFrame *in)
static int filter_frame(AVFilterLink *link, AVFrame *frame)
{
FlipContext *flip = link->dst->priv;
-int i;
+int i, ret;
if (flip->bayer)
return flip_bayer(link, frame);
+if ((ret = av_frame_make_writable(frame)) < 0) {
+av_frame_free(&frame);
+return ret;
+}
+
for (i = 0; i < 4; i ++) {
int vsub = i == 1 || i == 2 ? flip->vsub : 0;
int height = AV_CEIL_RSHIFT(link->h, vsub);
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v3 1/1] libavformat/asfdec: A collection of related fixes for asfdec
Fix 1: Commit c8140fe7324f264faacf7395b27e12531d1f13f7 had introduced a check
for value_len > UINT16_MAX.
As a consequence, attached images of sizes larger than UINT16_MAX could no
longer be read.
Fix 2: The value_len is an uint32 not an int32 per spec. That value must not be
truncated, neither by casting to int, nor by any conditional checks, because at
the end of get_tag, this value is needed to move forward in parsing. When the
len value gets modified, the parsing may break.
Fix 3: get_value had a return type of int, which means that reading QWORDS was
broken due to truncation.
Fix 4: Parsing of GUID values wasn't implemented
Fix 5: In get_tag, the code was adding 22 bytes (in order to allow it to hold
64bit numbers as string) to the value len for creating creating a buffer. This
was unnecessarily imposing a size-constraint on the value_len parameter.
Fix 6: The code in get_tag, was limiting the maximum value_len to half the size
of INT32. This was applied for all value types, even though it is required only
in case of ASF_UNICODE, not for any other ones (like ASCII).
Fix 7: get_tag was always allocating a buffer regardless of the datatype, even
though this isn't required in case of ASF_BYTE_ARRAY
Fix 8: The spec allows attachment sizes of up to UINT32_MAX while we can handle
only sizes up to INT32_MAX. The debug.assert didn't really address this, and
truncating the value_len in calling methods cannot be used as the length value
is required in order to continue parsing. I have added a check with log message
in ff_asf_handle_byte_array to handle this.
Signed-off-by: softworkz
---
v3: Rebased to latest & ping
libavformat/asf.c | 12 +++--
libavformat/asf.h | 2 +-
libavformat/asfdec_f.c | 115 +
3 files changed, 81 insertions(+), 48 deletions(-)
diff --git a/libavformat/asf.c b/libavformat/asf.c
index 1ac8b5f078..179b66a2b4 100644
--- a/libavformat/asf.c
+++ b/libavformat/asf.c
@@ -267,12 +267,18 @@ static int get_id3_tag(AVFormatContext *s, int len)
}
int ff_asf_handle_byte_array(AVFormatContext *s, const char *name,
- int val_len)
+ uint32_t val_len)
{
+if (val_len > INT32_MAX) {
+av_log(s, AV_LOG_VERBOSE, "Unable to handle byte arrays > INT32_MAX
in tag %s.\n", name);
+return 1;
+}
+
if (!strcmp(name, "WM/Picture")) // handle cover art
-return asf_read_picture(s, val_len);
+return asf_read_picture(s, (int)val_len);
else if (!strcmp(name, "ID3")) // handle ID3 tag
-return get_id3_tag(s, val_len);
+return get_id3_tag(s, (int)val_len);
+av_log(s, AV_LOG_VERBOSE, "Unsupported byte array in tag %s.\n", name);
return 1;
}
diff --git a/libavformat/asf.h b/libavformat/asf.h
index 01cc4f7a46..4d28560f56 100644
--- a/libavformat/asf.h
+++ b/libavformat/asf.h
@@ -111,7 +111,7 @@ extern const AVMetadataConv ff_asf_metadata_conv[];
* is unsupported by this function and 0 otherwise.
*/
int ff_asf_handle_byte_array(AVFormatContext *s, const char *name,
- int val_len);
+ uint32_t val_len);
#define ASF_PACKET_FLAG_ERROR_CORRECTION_PRESENT 0x80 //1000
diff --git a/libavformat/asfdec_f.c b/libavformat/asfdec_f.c
index 72ba8b32a0..f0940b5b08 100644
--- a/libavformat/asfdec_f.c
+++ b/libavformat/asfdec_f.c
@@ -149,7 +149,7 @@ static const ff_asf_guid asf_audio_conceal_none = {
};
#define PRINT_IF_GUID(g, cmp) \
-if (!ff_guidcmp(g, &cmp)) \
+if (!ff_guidcmp(g, (&cmp))) \
av_log(NULL, AV_LOG_TRACE, "(GUID: %s) ", # cmp)
static void print_guid(ff_asf_guid *g)
@@ -202,71 +202,100 @@ static int asf_probe(const AVProbeData *pd)
/* size of type 2 (BOOL) is 32bit for "Extended Content Description Object"
* but 16 bit for "Metadata Object" and "Metadata Library Object" */
-static int get_value(AVIOContext *pb, int type, int type2_size)
+static uint64_t get_value(AVIOContext *pb, int type, int type2_size)
{
switch (type) {
-case 2:
+case ASF_BOOL:
return (type2_size == 32) ? avio_rl32(pb) : avio_rl16(pb);
-case 3:
+case ASF_DWORD:
return avio_rl32(pb);
-case 4:
+case ASF_QWORD:
return avio_rl64(pb);
-case 5:
+case ASF_WORD:
return avio_rl16(pb);
default:
return INT_MIN;
}
}
-static void get_tag(AVFormatContext *s, const char *key, int type, int len,
int type2_size)
+static void get_tag(AVFormatContext *s, const char *key, int type, uint32_t
len, int type2_size)
{
ASFContext *asf = s->priv_data;
-char *value = NULL;
int64_t off = avio_tell(s->pb);
-#define LEN 22
-
-av_assert0((unsigned)len < (INT_MAX - LEN) / 2);
+char *buffer = NULL;
+uint64_t required_bufferlen;
+int buffer_len;
if (!asf->export_xmp && !strncmp(key, "xmp", 3))
goto finish;
-value = av_malloc(2
[FFmpeg-devel] [PATCH v4 1/1] libavformat/asfdec: A collection of related fixes for asfdec
Fix 1: Commit c8140fe7324f264faacf7395b27e12531d1f13f7 had
introduced a check for value_len > UINT16_MAX.
As a consequence, attached images of sizes larger than
UINT16_MAX could no longer be read.
Fix 2: The value_len is an uint32 not an int32 per spec. That
value must not be truncated, neither by casting to int, nor by any
conditional checks, because at the end of get_tag, this value is
needed to move forward in parsing. When the len value gets
modified, the parsing may break.
Fix 3: get_value had a return type of int, which means that reading
QWORDS was broken due to truncation.
Fix 4: Parsing of GUID values wasn't implemented
Fix 5: In get_tag, the code was adding 22 bytes (in order to allow
it to hold 64bit numbers as string) to the value len for creating
creating a buffer. This was unnecessarily imposing a
size-constraint on the value_len parameter.
Fix 6: The code in get_tag, was limiting the maximum value_len to
half the size of INT32. This was applied for all value types, even
though it is required only in case of ASF_UNICODE, not for any
other ones (like ASCII).
Fix 7: get_tag was always allocating a buffer regardless of the
datatype, even though this isn't required in case of ASF_BYTE_ARRAY
Fix 8: The spec allows attachment sizes of up to UINT32_MAX while
we can handle only sizes up to INT32_MAX. The debug.assert didn't
really address this, and truncating the value_len in calling
methods cannot be used as the length value is required in order to
continue parsing. I have added a check with log message in
ff_asf_handle_byte_array to handle this.
Signed-off-by: softworkz
---
v4: Added line-breaks to commit message
libavformat/asf.c | 12 +++--
libavformat/asf.h | 2 +-
libavformat/asfdec_f.c | 115 +
3 files changed, 81 insertions(+), 48 deletions(-)
diff --git a/libavformat/asf.c b/libavformat/asf.c
index 1ac8b5f078..179b66a2b4 100644
--- a/libavformat/asf.c
+++ b/libavformat/asf.c
@@ -267,12 +267,18 @@ static int get_id3_tag(AVFormatContext *s, int len)
}
int ff_asf_handle_byte_array(AVFormatContext *s, const char *name,
- int val_len)
+ uint32_t val_len)
{
+if (val_len > INT32_MAX) {
+av_log(s, AV_LOG_VERBOSE, "Unable to handle byte arrays > INT32_MAX
in tag %s.\n", name);
+return 1;
+}
+
if (!strcmp(name, "WM/Picture")) // handle cover art
-return asf_read_picture(s, val_len);
+return asf_read_picture(s, (int)val_len);
else if (!strcmp(name, "ID3")) // handle ID3 tag
-return get_id3_tag(s, val_len);
+return get_id3_tag(s, (int)val_len);
+av_log(s, AV_LOG_VERBOSE, "Unsupported byte array in tag %s.\n", name);
return 1;
}
diff --git a/libavformat/asf.h b/libavformat/asf.h
index 01cc4f7a46..4d28560f56 100644
--- a/libavformat/asf.h
+++ b/libavformat/asf.h
@@ -111,7 +111,7 @@ extern const AVMetadataConv ff_asf_metadata_conv[];
* is unsupported by this function and 0 otherwise.
*/
int ff_asf_handle_byte_array(AVFormatContext *s, const char *name,
- int val_len);
+ uint32_t val_len);
#define ASF_PACKET_FLAG_ERROR_CORRECTION_PRESENT 0x80 //1000
diff --git a/libavformat/asfdec_f.c b/libavformat/asfdec_f.c
index 72ba8b32a0..f0940b5b08 100644
--- a/libavformat/asfdec_f.c
+++ b/libavformat/asfdec_f.c
@@ -149,7 +149,7 @@ static const ff_asf_guid asf_audio_conceal_none = {
};
#define PRINT_IF_GUID(g, cmp) \
-if (!ff_guidcmp(g, &cmp)) \
+if (!ff_guidcmp(g, (&cmp))) \
av_log(NULL, AV_LOG_TRACE, "(GUID: %s) ", # cmp)
static void print_guid(ff_asf_guid *g)
@@ -202,71 +202,100 @@ static int asf_probe(const AVProbeData *pd)
/* size of type 2 (BOOL) is 32bit for "Extended Content Description Object"
* but 16 bit for "Metadata Object" and "Metadata Library Object" */
-static int get_value(AVIOContext *pb, int type, int type2_size)
+static uint64_t get_value(AVIOContext *pb, int type, int type2_size)
{
switch (type) {
-case 2:
+case ASF_BOOL:
return (type2_size == 32) ? avio_rl32(pb) : avio_rl16(pb);
-case 3:
+case ASF_DWORD:
return avio_rl32(pb);
-case 4:
+case ASF_QWORD:
return avio_rl64(pb);
-case 5:
+case ASF_WORD:
return avio_rl16(pb);
default:
return INT_MIN;
}
}
-static void get_tag(AVFormatContext *s, const char *key, int type, int len,
int type2_size)
+static void get_tag(AVFormatContext *s, const char *key, int type, uint32_t
len, int type2_size)
{
ASFContext *asf = s->priv_data;
-char *value = NULL;
int64_t off = avio_tell(s->pb);
-#define LEN 22
-
-av_assert0((unsigned)len < (INT_MAX - LEN) / 2);
+char *buffer = NULL;
+uint64_t required_bufferlen;
+int buffer_len;
if (!asf->export_xmp && !strncmp(key, "xmp", 3))
goto finish;
-value = av_malloc(2 * l
Re: [FFmpeg-devel] [PATCH v4 1/1] libavformat/asfdec: A collection of related fixes for asfdec
> -Original Message- > From: ffmpeg-devel On Behalf Of > Michael Niedermayer > Sent: Tuesday, 28 September 2021 22:18 > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v4 1/1] libavformat/asfdec: A > collection of related fixes for asfdec > > On Tue, Sep 28, 2021 at 08:10:02PM +, Soft Works wrote: > > Fix 1: Commit c8140fe7324f264faacf7395b27e12531d1f13f7 had > > introduced a check for value_len > UINT16_MAX. > > As a consequence, attached images of sizes larger than > > UINT16_MAX could no longer be read. > > > > Fix 2: The value_len is an uint32 not an int32 per spec. That > > value must not be truncated, neither by casting to int, nor by any > > conditional checks, because at the end of get_tag, this value is > > needed to move forward in parsing. When the len value gets > > modified, the parsing may break. > > > > Fix 3: get_value had a return type of int, which means that reading > > QWORDS was broken due to truncation. > > > > Fix 4: Parsing of GUID values wasn't implemented > > > > Fix 5: In get_tag, the code was adding 22 bytes (in order to allow > > it to hold 64bit numbers as string) to the value len for creating > > creating a buffer. This was unnecessarily imposing a > > size-constraint on the value_len parameter. > > > > Fix 6: The code in get_tag, was limiting the maximum value_len to > > half the size of INT32. This was applied for all value types, even > > though it is required only in case of ASF_UNICODE, not for any > > other ones (like ASCII). > > > > Fix 7: get_tag was always allocating a buffer regardless of the > > datatype, even though this isn't required in case of ASF_BYTE_ARRAY > > > > Fix 8: The spec allows attachment sizes of up to UINT32_MAX while > > we can handle only sizes up to INT32_MAX. The debug.assert didn't > > really address this, and truncating the value_len in calling > > methods cannot be used as the length value is required in order to > > continue parsing. I have added a check with log message in > > ff_asf_handle_byte_array to handle this. > > This probably should be split into multiple self contained patches I'll happily do when I would get some rough feedback on it :-) Then I'd know what would be desired or undesired and how's the best way to split these.. Thanks, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure frames are writable when processing in-place
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> James Almer
> Sent: Tuesday, 28 September 2021 22:08
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure
> frames are writable when processing in-place
>
> On 9/28/2021 4:54 PM, Soft Works wrote:
> > Signed-off-by: softworkz
> > ---
> > v2: Reduced to cases without AVFILTERPAD_FLAG_NEEDS_WRITABLE
>
> Can't this flag used in these filters?
Not in vf_vflip, because in case of flip_bayer, it's using a
custom allocation (.get_buffer.video)
> > libavfilter/vf_cover_rect.c | 7 +--
> > libavfilter/vf_floodfill.c | 5 +
> > libavfilter/vf_vflip.c | 7 ++-
> > 3 files changed, 16 insertions(+), 3 deletions(-)
> >
> > diff --git a/libavfilter/vf_cover_rect.c
> b/libavfilter/vf_cover_rect.c
> > index 0a8c10e06d..2367afb4b3 100644
> > --- a/libavfilter/vf_cover_rect.c
> > +++ b/libavfilter/vf_cover_rect.c
> > @@ -136,7 +136,7 @@ static int filter_frame(AVFilterLink *inlink,
> AVFrame *in)
> > AVFilterContext *ctx = inlink->dst;
> > CoverContext *cover = ctx->priv;
> > AVDictionaryEntry *ex, *ey, *ew, *eh;
> > -int x = -1, y = -1, w = -1, h = -1;
> > +int x = -1, y = -1, w = -1, h = -1, ret;
> > char *xendptr = NULL, *yendptr = NULL, *wendptr = NULL,
> *hendptr = NULL;
> >
> > ex = av_dict_get(in->metadata, "lavfi.rect.x", NULL,
> AV_DICT_MATCH_CASE);
> > @@ -181,7 +181,10 @@ static int filter_frame(AVFilterLink *inlink,
> AVFrame *in)
> > x = av_clip(x, 0, in->width - w);
> > y = av_clip(y, 0, in->height - h);
> >
> > -av_frame_make_writable(in);
> > +if ((ret = av_frame_make_writable(in)) < 0) {
> > +av_frame_free(&in);
> > +return ret;
> > +}
> >
> > if (cover->mode == MODE_BLUR) {
> > blur (cover, in, x, y);
> > diff --git a/libavfilter/vf_floodfill.c
> b/libavfilter/vf_floodfill.c
> > index 21741cdb4f..292b27505e 100644
> > --- a/libavfilter/vf_floodfill.c
> > +++ b/libavfilter/vf_floodfill.c
> > @@ -294,6 +294,11 @@ static int filter_frame(AVFilterLink *link,
> AVFrame *frame)
> > const int h = frame->height;
> > int i, ret;
> >
> > +if ((ret = av_frame_make_writable(frame)) < 0) {
> > +av_frame_free(&frame);
> > +return ret;
> > +}
> > +
> > if (is_inside(s->x, s->y, w, h)) {
> > s->pick_pixel(frame, s->x, s->y, &s0, &s1, &s2, &s3);
> >
> > diff --git a/libavfilter/vf_vflip.c b/libavfilter/vf_vflip.c
> > index 0d624512f9..622bd46db3 100644
> > --- a/libavfilter/vf_vflip.c
> > +++ b/libavfilter/vf_vflip.c
> > @@ -108,11 +108,16 @@ static int flip_bayer(AVFilterLink *link,
> AVFrame *in)
> > static int filter_frame(AVFilterLink *link, AVFrame *frame)
> > {
> > FlipContext *flip = link->dst->priv;
> > -int i;
> > +int i, ret;
> >
> > if (flip->bayer)
> > return flip_bayer(link, frame);
> >
> > +if ((ret = av_frame_make_writable(frame)) < 0) {
>
> vf_vflip defines a custom get_buffer.video() function, so you can't
> use
> av_frame_make_writable() as the buffer it will return in case it
> needs
> to allocate a writable one may not be suitable. You need to use
> ff_inlink_make_frame_writable() instead, and in all three filters
> while
> at it.
.get_buffer.video is used for the case of flip_bayer, but not otherwise.
Otherwise it was currently writing to the incoming frame directly,
and that's what this patch fixes.
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure frames are writable when processing in-place
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> James Almer
> Sent: Tuesday, 28 September 2021 23:19
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure
> frames are writable when processing in-place
>
> On 9/28/2021 6:00 PM, Soft Works wrote:
> >
> >
> >> -Original Message-
> >> From: ffmpeg-devel On Behalf Of
> >> James Almer
> >> Sent: Tuesday, 28 September 2021 22:08
> >> To: ffmpeg-devel@ffmpeg.org
> >> Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure
> >> frames are writable when processing in-place
> >>
> >> On 9/28/2021 4:54 PM, Soft Works wrote:
> >>> Signed-off-by: softworkz
> >>> ---
> >>> v2: Reduced to cases without AVFILTERPAD_FLAG_NEEDS_WRITABLE
> >>
> >> Can't this flag used in these filters?
> >
> > Not in vf_vflip, because in case of flip_bayer, it's using a
> > custom allocation (.get_buffer.video)
> >
> >>>libavfilter/vf_cover_rect.c | 7 +--
> >>>libavfilter/vf_floodfill.c | 5 +
> >>>libavfilter/vf_vflip.c | 7 ++-
> >>>3 files changed, 16 insertions(+), 3 deletions(-)
> >>>
> >>> diff --git a/libavfilter/vf_cover_rect.c
> >> b/libavfilter/vf_cover_rect.c
> >>> index 0a8c10e06d..2367afb4b3 100644
> >>> --- a/libavfilter/vf_cover_rect.c
> >>> +++ b/libavfilter/vf_cover_rect.c
> >>> @@ -136,7 +136,7 @@ static int filter_frame(AVFilterLink *inlink,
> >> AVFrame *in)
> >>>AVFilterContext *ctx = inlink->dst;
> >>>CoverContext *cover = ctx->priv;
> >>>AVDictionaryEntry *ex, *ey, *ew, *eh;
> >>> -int x = -1, y = -1, w = -1, h = -1;
> >>> +int x = -1, y = -1, w = -1, h = -1, ret;
> >>>char *xendptr = NULL, *yendptr = NULL, *wendptr = NULL,
> >> *hendptr = NULL;
> >>>
> >>>ex = av_dict_get(in->metadata, "lavfi.rect.x", NULL,
> >> AV_DICT_MATCH_CASE);
> >>> @@ -181,7 +181,10 @@ static int filter_frame(AVFilterLink
> *inlink,
> >> AVFrame *in)
> >>>x = av_clip(x, 0, in->width - w);
> >>>y = av_clip(y, 0, in->height - h);
> >>>
> >>> -av_frame_make_writable(in);
> >>> +if ((ret = av_frame_make_writable(in)) < 0) {
> >>> +av_frame_free(&in);
> >>> +return ret;
> >>> +}
> >>>
> >>>if (cover->mode == MODE_BLUR) {
> >>>blur (cover, in, x, y);
> >>> diff --git a/libavfilter/vf_floodfill.c
> >> b/libavfilter/vf_floodfill.c
> >>> index 21741cdb4f..292b27505e 100644
> >>> --- a/libavfilter/vf_floodfill.c
> >>> +++ b/libavfilter/vf_floodfill.c
> >>> @@ -294,6 +294,11 @@ static int filter_frame(AVFilterLink *link,
> >> AVFrame *frame)
> >>>const int h = frame->height;
> >>>int i, ret;
> >>>
> >>> +if ((ret = av_frame_make_writable(frame)) < 0) {
> >>> +av_frame_free(&frame);
> >>> +return ret;
> >>> +}
> >>> +
> >>>if (is_inside(s->x, s->y, w, h)) {
> >>>s->pick_pixel(frame, s->x, s->y, &s0, &s1, &s2, &s3);
> >>>
> >>> diff --git a/libavfilter/vf_vflip.c b/libavfilter/vf_vflip.c
> >>> index 0d624512f9..622bd46db3 100644
> >>> --- a/libavfilter/vf_vflip.c
> >>> +++ b/libavfilter/vf_vflip.c
> >>> @@ -108,11 +108,16 @@ static int flip_bayer(AVFilterLink *link,
> >> AVFrame *in)
> >>>static int filter_frame(AVFilterLink *link, AVFrame *frame)
> >>>{
> >>>FlipContext *flip = link->dst->priv;
> >>> -int i;
> >>> +int i, ret;
> >>>
> >>>if (flip->bayer)
> >>>return flip_bayer(link, frame);
> >>>
> >>> +if ((ret = av_frame_make_writable(frame)) < 0) {
> >>
> >> vf_vflip defines a custom get_buffer.video() function, so you
> can't
> >> use
> >> av_frame_make_writable() as the buffer it will return in case it
> >> needs
> >> to allocate a writable one may not be suitable. You need to use
> >> ff_inlink_make_frame_writable() instead, and in all three filters
> >> while
> >> at it.
> >
> > .get_buffer.video is used for the case of flip_bayer, but not
> otherwise.
> > Otherwise it was currently writing to the incoming frame directly,
> > and that's what this patch fixes.
>
> Ok, but i insist you should use ff_inlink_make_frame_writable(). If
> it
> wont use the vflip custom get_buffer.video function, it will instead
> use
> the default allocator, which makes use of a lavfi frame pool.
I've gone through very single video filter's filter_frame function
and I've seen not a single one making use of it.
That might be a task for a different day, I suppose...?
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure frames are writable when processing in-place
> -Original Message- > From: ffmpeg-devel On Behalf Of > James Almer > Sent: Tuesday, 28 September 2021 22:08 > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure > frames are writable when processing in-place > > On 9/28/2021 4:54 PM, Soft Works wrote: > > Signed-off-by: softworkz > > --- > > v2: Reduced to cases without AVFILTERPAD_FLAG_NEEDS_WRITABLE > > Can't this flag used in these filters? I mentioned before that I don’t consider this as a good API design. It had even caused a review of my subtitle filters to assume that my filter code would be incorrect, even though I had been using that flag. It provides no other benefit than that it saves you from writing two or three lines of code. Minus the one required to specify the flag makes just one or two lines left. Much higher than the benefit is the cost that is involved: Filtering code (filter_frame) is no longer 1:1 exchangeable between filters (in case of only one having that flag). In order not to make mistakes, a developer needs to be aware of that little specific detail, and even when he is, he might still forget about it at times. The ffmpeg filtering implementation is not overly complex, especially when comparing to a similar (but much more capable) framework like DirectShow. But it's those little unexpected specifics like AVFILTERPAD_FLAG_NEEDS_WRITABLE which are making it less easily accessible than it could be. Kind regards, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure frames are writable when processing in-place
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> James Almer
> Sent: Tuesday, 28 September 2021 23:50
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure
> frames are writable when processing in-place
>
> On 9/28/2021 6:22 PM, Soft Works wrote:
> >
> >
> >> -Original Message-
> >> From: ffmpeg-devel On Behalf Of
> >> James Almer
> >> Sent: Tuesday, 28 September 2021 23:19
> >> To: ffmpeg-devel@ffmpeg.org
> >> Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure
> >> frames are writable when processing in-place
> >>
> >> On 9/28/2021 6:00 PM, Soft Works wrote:
> >>>
> >>>
> >>>> -Original Message-
> >>>> From: ffmpeg-devel On Behalf
> Of
> >>>> James Almer
> >>>> Sent: Tuesday, 28 September 2021 22:08
> >>>> To: ffmpeg-devel@ffmpeg.org
> >>>> Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames:
> Ensure
> >>>> frames are writable when processing in-place
> >>>>
> >>>> On 9/28/2021 4:54 PM, Soft Works wrote:
> >>>>> Signed-off-by: softworkz
> >>>>> ---
> >>>>> v2: Reduced to cases without AVFILTERPAD_FLAG_NEEDS_WRITABLE
> >>>>
> >>>> Can't this flag used in these filters?
> >>>
> >>> Not in vf_vflip, because in case of flip_bayer, it's using a
> >>> custom allocation (.get_buffer.video)
> >>>
> >>>>> libavfilter/vf_cover_rect.c | 7 +--
> >>>>> libavfilter/vf_floodfill.c | 5 +
> >>>>> libavfilter/vf_vflip.c | 7 ++-
> >>>>> 3 files changed, 16 insertions(+), 3 deletions(-)
> >>>>>
> >>>>> diff --git a/libavfilter/vf_cover_rect.c
> >>>> b/libavfilter/vf_cover_rect.c
> >>>>> index 0a8c10e06d..2367afb4b3 100644
> >>>>> --- a/libavfilter/vf_cover_rect.c
> >>>>> +++ b/libavfilter/vf_cover_rect.c
> >>>>> @@ -136,7 +136,7 @@ static int filter_frame(AVFilterLink
> *inlink,
> >>>> AVFrame *in)
> >>>>> AVFilterContext *ctx = inlink->dst;
> >>>>> CoverContext *cover = ctx->priv;
> >>>>> AVDictionaryEntry *ex, *ey, *ew, *eh;
> >>>>> -int x = -1, y = -1, w = -1, h = -1;
> >>>>> +int x = -1, y = -1, w = -1, h = -1, ret;
> >>>>> char *xendptr = NULL, *yendptr = NULL, *wendptr = NULL,
> >>>> *hendptr = NULL;
> >>>>>
> >>>>> ex = av_dict_get(in->metadata, "lavfi.rect.x", NULL,
> >>>> AV_DICT_MATCH_CASE);
> >>>>> @@ -181,7 +181,10 @@ static int filter_frame(AVFilterLink
> >> *inlink,
> >>>> AVFrame *in)
> >>>>> x = av_clip(x, 0, in->width - w);
> >>>>> y = av_clip(y, 0, in->height - h);
> >>>>>
> >>>>> -av_frame_make_writable(in);
> >>>>> +if ((ret = av_frame_make_writable(in)) < 0) {
> >>>>> +av_frame_free(&in);
> >>>>> +return ret;
> >>>>> +}
> >>>>>
> >>>>> if (cover->mode == MODE_BLUR) {
> >>>>> blur (cover, in, x, y);
> >>>>> diff --git a/libavfilter/vf_floodfill.c
> >>>> b/libavfilter/vf_floodfill.c
> >>>>> index 21741cdb4f..292b27505e 100644
> >>>>> --- a/libavfilter/vf_floodfill.c
> >>>>> +++ b/libavfilter/vf_floodfill.c
> >>>>> @@ -294,6 +294,11 @@ static int filter_frame(AVFilterLink
> *link,
> >>>> AVFrame *frame)
> >>>>> const int h = frame->height;
> >>>>> int i, ret;
> >>>>>
> >>>>> +if ((ret = av_frame_make_writable(frame)) < 0) {
> >>>>> +av_frame_free(&frame);
> >>>>> +return ret;
> >>>>> +}
> >>>>> +
> >>>>> if (is_inside(s->x, s->y, w, h)) {
> >>>>> s->pick_pixel(frame, s->x, s->y, &s0, &s1, &s2,
> &s3);
> >>>>>
> >>>>> diff --git a/libavfilter/vf_vflip.
Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure frames are writable when processing in-place
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Soft Works
> Sent: Wednesday, 29 September 2021 00:01
> To: FFmpeg development discussions and patches de...@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure
> frames are writable when processing in-place
>
>
>
> > -Original Message-
> > From: ffmpeg-devel On Behalf Of
> > James Almer
> > Sent: Tuesday, 28 September 2021 23:50
> > To: ffmpeg-devel@ffmpeg.org
> > Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure
> > frames are writable when processing in-place
> >
> > On 9/28/2021 6:22 PM, Soft Works wrote:
> > >
> > >
> > >> -Original Message-
> > >> From: ffmpeg-devel On Behalf
> Of
> > >> James Almer
> > >> Sent: Tuesday, 28 September 2021 23:19
> > >> To: ffmpeg-devel@ffmpeg.org
> > >> Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames:
> Ensure
> > >> frames are writable when processing in-place
> > >>
> > >> On 9/28/2021 6:00 PM, Soft Works wrote:
> > >>>
> > >>>
> > >>>> -Original Message-
> > >>>> From: ffmpeg-devel On Behalf
> > Of
> > >>>> James Almer
> > >>>> Sent: Tuesday, 28 September 2021 22:08
> > >>>> To: ffmpeg-devel@ffmpeg.org
> > >>>> Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames:
> > Ensure
> > >>>> frames are writable when processing in-place
> > >>>>
> > >>>> On 9/28/2021 4:54 PM, Soft Works wrote:
> > >>>>> Signed-off-by: softworkz
> > >>>>> ---
> > >>>>> v2: Reduced to cases without AVFILTERPAD_FLAG_NEEDS_WRITABLE
> > >>>>
> > >>>> Can't this flag used in these filters?
> > >>>
> > >>> Not in vf_vflip, because in case of flip_bayer, it's using a
> > >>> custom allocation (.get_buffer.video)
> > >>>
> > >>>>> libavfilter/vf_cover_rect.c | 7 +--
> > >>>>> libavfilter/vf_floodfill.c | 5 +
> > >>>>> libavfilter/vf_vflip.c | 7 ++-
> > >>>>> 3 files changed, 16 insertions(+), 3 deletions(-)
> > >>>>>
> > >>>>> diff --git a/libavfilter/vf_cover_rect.c
> > >>>> b/libavfilter/vf_cover_rect.c
> > >>>>> index 0a8c10e06d..2367afb4b3 100644
> > >>>>> --- a/libavfilter/vf_cover_rect.c
> > >>>>> +++ b/libavfilter/vf_cover_rect.c
> > >>>>> @@ -136,7 +136,7 @@ static int filter_frame(AVFilterLink
> > *inlink,
> > >>>> AVFrame *in)
> > >>>>> AVFilterContext *ctx = inlink->dst;
> > >>>>> CoverContext *cover = ctx->priv;
> > >>>>> AVDictionaryEntry *ex, *ey, *ew, *eh;
> > >>>>> -int x = -1, y = -1, w = -1, h = -1;
> > >>>>> +int x = -1, y = -1, w = -1, h = -1, ret;
> > >>>>> char *xendptr = NULL, *yendptr = NULL, *wendptr =
> NULL,
> > >>>> *hendptr = NULL;
> > >>>>>
> > >>>>> ex = av_dict_get(in->metadata, "lavfi.rect.x", NULL,
> > >>>> AV_DICT_MATCH_CASE);
> > >>>>> @@ -181,7 +181,10 @@ static int filter_frame(AVFilterLink
> > >> *inlink,
> > >>>> AVFrame *in)
> > >>>>> x = av_clip(x, 0, in->width - w);
> > >>>>> y = av_clip(y, 0, in->height - h);
> > >>>>>
> > >>>>> -av_frame_make_writable(in);
> > >>>>> +if ((ret = av_frame_make_writable(in)) < 0) {
> > >>>>> +av_frame_free(&in);
> > >>>>> +return ret;
> > >>>>> +}
> > >>>>>
> > >>>>> if (cover->mode == MODE_BLUR) {
> > >>>>> blur (cover, in, x, y);
> > >>>>> diff --git a/libavfilter/vf_floodfill.c
> > >>>> b/libavfilter/vf_floodfill.c
> > >>>>> index 21741cdb4f..292b27505e 100644
> > >>>>> --- a/libavfilter/vf_floodfill.c
> > >>>>> +++ b/libavfilter/vf_floodfill.c
> > >>>>>
Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure frames are writable when processing in-place
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> James Almer
> Sent: Wednesday, 29 September 2021 00:27
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure
> frames are writable when processing in-place
>
> On 9/28/2021 4:54 PM, Soft Works wrote:
> > Signed-off-by: softworkz
> > ---
> > v2: Reduced to cases without AVFILTERPAD_FLAG_NEEDS_WRITABLE
> >
> > libavfilter/vf_cover_rect.c | 7 +--
> > libavfilter/vf_floodfill.c | 5 +
> > libavfilter/vf_vflip.c | 7 ++-
> > 3 files changed, 16 insertions(+), 3 deletions(-)
> >
> > diff --git a/libavfilter/vf_cover_rect.c
> b/libavfilter/vf_cover_rect.c
> > index 0a8c10e06d..2367afb4b3 100644
> > --- a/libavfilter/vf_cover_rect.c
> > +++ b/libavfilter/vf_cover_rect.c
> > @@ -136,7 +136,7 @@ static int filter_frame(AVFilterLink *inlink,
> AVFrame *in)
> > AVFilterContext *ctx = inlink->dst;
> > CoverContext *cover = ctx->priv;
> > AVDictionaryEntry *ex, *ey, *ew, *eh;
> > -int x = -1, y = -1, w = -1, h = -1;
> > +int x = -1, y = -1, w = -1, h = -1, ret;
> > char *xendptr = NULL, *yendptr = NULL, *wendptr = NULL,
> *hendptr = NULL;
> >
> > ex = av_dict_get(in->metadata, "lavfi.rect.x", NULL,
> AV_DICT_MATCH_CASE);
> > @@ -181,7 +181,10 @@ static int filter_frame(AVFilterLink *inlink,
> AVFrame *in)
> > x = av_clip(x, 0, in->width - w);
> > y = av_clip(y, 0, in->height - h);
> >
> > -av_frame_make_writable(in);
> > +if ((ret = av_frame_make_writable(in)) < 0) {
> > +av_frame_free(&in);
> > +return ret;
> > +}
> >
> > if (cover->mode == MODE_BLUR) {
> > blur (cover, in, x, y);
> > diff --git a/libavfilter/vf_floodfill.c
> b/libavfilter/vf_floodfill.c
> > index 21741cdb4f..292b27505e 100644
> > --- a/libavfilter/vf_floodfill.c
> > +++ b/libavfilter/vf_floodfill.c
> > @@ -294,6 +294,11 @@ static int filter_frame(AVFilterLink *link,
> AVFrame *frame)
> > const int h = frame->height;
> > int i, ret;
> >
> > +if ((ret = av_frame_make_writable(frame)) < 0) {
>
> Is this one even needed? s->pick_pixel() does not change the frame,
> and
> an av_frame_make_writable() call is already in place before any
> s->set_pixel() call in this same function.
Yes, you're right in this case. In fact I missed the existing
av_frame_make_writable() call.
Thanks,
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure frames are writable when processing in-place
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> James Almer
> Sent: Wednesday, 29 September 2021 00:30
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure
> frames are writable when processing in-place
>
> On 9/28/2021 7:00 PM, Soft Works wrote:
> >
> >
> >> -Original Message-
> >> From: ffmpeg-devel On Behalf Of
> >> James Almer
> >> Sent: Tuesday, 28 September 2021 23:50
> >> To: ffmpeg-devel@ffmpeg.org
> >> Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure
> >> frames are writable when processing in-place
> >>
> >> On 9/28/2021 6:22 PM, Soft Works wrote:
> >>>
> >>>
> >>>> -Original Message-
> >>>> From: ffmpeg-devel On Behalf
> Of
> >>>> James Almer
> >>>> Sent: Tuesday, 28 September 2021 23:19
> >>>> To: ffmpeg-devel@ffmpeg.org
> >>>> Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames:
> Ensure
> >>>> frames are writable when processing in-place
> >>>>
> >>>> On 9/28/2021 6:00 PM, Soft Works wrote:
> >>>>>
> >>>>>
> >>>>>> -Original Message-----
> >>>>>> From: ffmpeg-devel On Behalf
> >> Of
> >>>>>> James Almer
> >>>>>> Sent: Tuesday, 28 September 2021 22:08
> >>>>>> To: ffmpeg-devel@ffmpeg.org
> >>>>>> Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames:
> >> Ensure
> >>>>>> frames are writable when processing in-place
> >>>>>>
> >>>>>> On 9/28/2021 4:54 PM, Soft Works wrote:
> >>>>>>> Signed-off-by: softworkz
> >>>>>>> ---
> >>>>>>> v2: Reduced to cases without AVFILTERPAD_FLAG_NEEDS_WRITABLE
> >>>>>>
> >>>>>> Can't this flag used in these filters?
> >>>>>
> >>>>> Not in vf_vflip, because in case of flip_bayer, it's using a
> >>>>> custom allocation (.get_buffer.video)
> >>>>>
> >>>>>>> libavfilter/vf_cover_rect.c | 7 +--
> >>>>>>> libavfilter/vf_floodfill.c | 5 +
> >>>>>>> libavfilter/vf_vflip.c | 7 ++-
> >>>>>>> 3 files changed, 16 insertions(+), 3 deletions(-)
> >>>>>>>
> >>>>>>> diff --git a/libavfilter/vf_cover_rect.c
> >>>>>> b/libavfilter/vf_cover_rect.c
> >>>>>>> index 0a8c10e06d..2367afb4b3 100644
> >>>>>>> --- a/libavfilter/vf_cover_rect.c
> >>>>>>> +++ b/libavfilter/vf_cover_rect.c
> >>>>>>> @@ -136,7 +136,7 @@ static int filter_frame(AVFilterLink
> >> *inlink,
> >>>>>> AVFrame *in)
> >>>>>>> AVFilterContext *ctx = inlink->dst;
> >>>>>>> CoverContext *cover = ctx->priv;
> >>>>>>> AVDictionaryEntry *ex, *ey, *ew, *eh;
> >>>>>>> -int x = -1, y = -1, w = -1, h = -1;
> >>>>>>> +int x = -1, y = -1, w = -1, h = -1, ret;
> >>>>>>> char *xendptr = NULL, *yendptr = NULL, *wendptr =
> NULL,
> >>>>>> *hendptr = NULL;
> >>>>>>>
> >>>>>>> ex = av_dict_get(in->metadata, "lavfi.rect.x", NULL,
> >>>>>> AV_DICT_MATCH_CASE);
> >>>>>>> @@ -181,7 +181,10 @@ static int filter_frame(AVFilterLink
> >>>> *inlink,
> >>>>>> AVFrame *in)
> >>>>>>> x = av_clip(x, 0, in->width - w);
> >>>>>>> y = av_clip(y, 0, in->height - h);
> >>>>>>>
> >>>>>>> -av_frame_make_writable(in);
> >>>>>>> +if ((ret = av_frame_make_writable(in)) < 0) {
> >>>>>>> +av_frame_free(&in);
> >>>>>>> +return ret;
> >>>>>>> +}
> >>>>>>>
> >>>>>>> if (cover->mode == MODE_BLUR) {
> >>>>>>> blur (cover, in, x, y);
> >>>>>>> diff --git a/libavf
Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure frames are writable when processing in-place
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Andreas Rheinhardt
> Sent: Wednesday, 29 September 2021 01:42
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure
> frames are writable when processing in-place
>
> Soft Works:
> >
> >
> >> -Original Message-
> >> From: ffmpeg-devel On Behalf Of
> >> James Almer
> >> Sent: Tuesday, 28 September 2021 22:08
> >> To: ffmpeg-devel@ffmpeg.org
> >> Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure
> >> frames are writable when processing in-place
> >>
> >> On 9/28/2021 4:54 PM, Soft Works wrote:
> >>> Signed-off-by: softworkz
> >>> ---
> >>> v2: Reduced to cases without AVFILTERPAD_FLAG_NEEDS_WRITABLE
> >>
> >> Can't this flag used in these filters?
> >
> > Not in vf_vflip, because in case of flip_bayer, it's using a
> > custom allocation (.get_buffer.video)
> >
> >>> libavfilter/vf_cover_rect.c | 7 +--
> >>> libavfilter/vf_floodfill.c | 5 +
> >>> libavfilter/vf_vflip.c | 7 ++-
> >>> 3 files changed, 16 insertions(+), 3 deletions(-)
> >>>
> >>> diff --git a/libavfilter/vf_cover_rect.c
> >> b/libavfilter/vf_cover_rect.c
> >>> index 0a8c10e06d..2367afb4b3 100644
> >>> --- a/libavfilter/vf_cover_rect.c
> >>> +++ b/libavfilter/vf_cover_rect.c
> >>> @@ -136,7 +136,7 @@ static int filter_frame(AVFilterLink *inlink,
> >> AVFrame *in)
> >>> AVFilterContext *ctx = inlink->dst;
> >>> CoverContext *cover = ctx->priv;
> >>> AVDictionaryEntry *ex, *ey, *ew, *eh;
> >>> -int x = -1, y = -1, w = -1, h = -1;
> >>> +int x = -1, y = -1, w = -1, h = -1, ret;
> >>> char *xendptr = NULL, *yendptr = NULL, *wendptr = NULL,
> >> *hendptr = NULL;
> >>>
> >>> ex = av_dict_get(in->metadata, "lavfi.rect.x", NULL,
> >> AV_DICT_MATCH_CASE);
> >>> @@ -181,7 +181,10 @@ static int filter_frame(AVFilterLink
> *inlink,
> >> AVFrame *in)
> >>> x = av_clip(x, 0, in->width - w);
> >>> y = av_clip(y, 0, in->height - h);
> >>>
> >>> -av_frame_make_writable(in);
> >>> +if ((ret = av_frame_make_writable(in)) < 0) {
> >>> +av_frame_free(&in);
> >>> +return ret;
> >>> +}
> >>>
> >>> if (cover->mode == MODE_BLUR) {
> >>> blur (cover, in, x, y);
> >>> diff --git a/libavfilter/vf_floodfill.c
> >> b/libavfilter/vf_floodfill.c
> >>> index 21741cdb4f..292b27505e 100644
> >>> --- a/libavfilter/vf_floodfill.c
> >>> +++ b/libavfilter/vf_floodfill.c
> >>> @@ -294,6 +294,11 @@ static int filter_frame(AVFilterLink *link,
> >> AVFrame *frame)
> >>> const int h = frame->height;
> >>> int i, ret;
> >>>
> >>> +if ((ret = av_frame_make_writable(frame)) < 0) {
> >>> +av_frame_free(&frame);
> >>> +return ret;
> >>> +}
> >>> +
> >>> if (is_inside(s->x, s->y, w, h)) {
> >>> s->pick_pixel(frame, s->x, s->y, &s0, &s1, &s2, &s3);
> >>>
> >>> diff --git a/libavfilter/vf_vflip.c b/libavfilter/vf_vflip.c
> >>> index 0d624512f9..622bd46db3 100644
> >>> --- a/libavfilter/vf_vflip.c
> >>> +++ b/libavfilter/vf_vflip.c
> >>> @@ -108,11 +108,16 @@ static int flip_bayer(AVFilterLink *link,
> >> AVFrame *in)
> >>> static int filter_frame(AVFilterLink *link, AVFrame *frame)
> >>> {
> >>> FlipContext *flip = link->dst->priv;
> >>> -int i;
> >>> +int i, ret;
> >>>
> >>> if (flip->bayer)
> >>> return flip_bayer(link, frame);
> >>>
> >>> +if ((ret = av_frame_make_writable(frame)) < 0) {
> >>
> >> vf_vflip defines a custom get_buffer.video() function, so you
> can't
> >> use
> >> av_frame_make_writable() as the buffer it will return in case it
> >> needs
> >> to allocate a writable one may not be suitable. You need to use
> >> ff_inlink_make_frame_writable() instead, and in all three filters
Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure frames are writable when processing in-place
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Andreas Rheinhardt
> Sent: Wednesday, 29 September 2021 02:04
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure
> frames are writable when processing in-place
>
> Soft Works:
> >
> >
> >> -Original Message-
> >> From: ffmpeg-devel On Behalf Of
> >> Andreas Rheinhardt
> >> Sent: Wednesday, 29 September 2021 01:42
> >> To: ffmpeg-devel@ffmpeg.org
> >> Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure
> >> frames are writable when processing in-place
> >>
> >> Soft Works:
> >>>
> >>>
> >>>> -Original Message-
> >>>> From: ffmpeg-devel On Behalf
> Of
> >>>> James Almer
> >>>> Sent: Tuesday, 28 September 2021 22:08
> >>>> To: ffmpeg-devel@ffmpeg.org
> >>>> Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames:
> Ensure
> >>>> frames are writable when processing in-place
> >>>>
> >>>> On 9/28/2021 4:54 PM, Soft Works wrote:
> >>>>> Signed-off-by: softworkz
> >>>>> ---
> >>>>> v2: Reduced to cases without AVFILTERPAD_FLAG_NEEDS_WRITABLE
> >>>>
> >>>> Can't this flag used in these filters?
> >>>
> >>> Not in vf_vflip, because in case of flip_bayer, it's using a
> >>> custom allocation (.get_buffer.video)
> >>>
> >>>>> libavfilter/vf_cover_rect.c | 7 +--
> >>>>> libavfilter/vf_floodfill.c | 5 +
> >>>>> libavfilter/vf_vflip.c | 7 ++-
> >>>>> 3 files changed, 16 insertions(+), 3 deletions(-)
> >>>>>
> >>>>> diff --git a/libavfilter/vf_cover_rect.c
> >>>> b/libavfilter/vf_cover_rect.c
> >>>>> index 0a8c10e06d..2367afb4b3 100644
> >>>>> --- a/libavfilter/vf_cover_rect.c
> >>>>> +++ b/libavfilter/vf_cover_rect.c
> >>>>> @@ -136,7 +136,7 @@ static int filter_frame(AVFilterLink
> *inlink,
> >>>> AVFrame *in)
> >>>>> AVFilterContext *ctx = inlink->dst;
> >>>>> CoverContext *cover = ctx->priv;
> >>>>> AVDictionaryEntry *ex, *ey, *ew, *eh;
> >>>>> -int x = -1, y = -1, w = -1, h = -1;
> >>>>> +int x = -1, y = -1, w = -1, h = -1, ret;
> >>>>> char *xendptr = NULL, *yendptr = NULL, *wendptr = NULL,
> >>>> *hendptr = NULL;
> >>>>>
> >>>>> ex = av_dict_get(in->metadata, "lavfi.rect.x", NULL,
> >>>> AV_DICT_MATCH_CASE);
> >>>>> @@ -181,7 +181,10 @@ static int filter_frame(AVFilterLink
> >> *inlink,
> >>>> AVFrame *in)
> >>>>> x = av_clip(x, 0, in->width - w);
> >>>>> y = av_clip(y, 0, in->height - h);
> >>>>>
> >>>>> -av_frame_make_writable(in);
> >>>>> +if ((ret = av_frame_make_writable(in)) < 0) {
> >>>>> +av_frame_free(&in);
> >>>>> +return ret;
> >>>>> +}
> >>>>>
> >>>>> if (cover->mode == MODE_BLUR) {
> >>>>> blur (cover, in, x, y);
> >>>>> diff --git a/libavfilter/vf_floodfill.c
> >>>> b/libavfilter/vf_floodfill.c
> >>>>> index 21741cdb4f..292b27505e 100644
> >>>>> --- a/libavfilter/vf_floodfill.c
> >>>>> +++ b/libavfilter/vf_floodfill.c
> >>>>> @@ -294,6 +294,11 @@ static int filter_frame(AVFilterLink
> *link,
> >>>> AVFrame *frame)
> >>>>> const int h = frame->height;
> >>>>> int i, ret;
> >>>>>
> >>>>> +if ((ret = av_frame_make_writable(frame)) < 0) {
> >>>>> +av_frame_free(&frame);
> >>>>> +return ret;
> >>>>> +}
> >>>>> +
> >>>>> if (is_inside(s->x, s->y, w, h)) {
> >>>>> s->pick_pixel(frame, s->x, s->y, &s0, &s1, &s2, &s3);
> >>>>>
> >>>>> diff --git a/libavfilter/vf_vflip.c b/libavfilter/vf_vflip.c
> >>>>> index 0d624512f9..622bd4
Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure frames are writable when processing in-place
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Andreas Rheinhardt
> Sent: Wednesday, 29 September 2021 02:25
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure
> frames are writable when processing in-place
>
> Soft Works:
> >
> >
> >> -Original Message-
> >> From: ffmpeg-devel On Behalf Of
> >> Andreas Rheinhardt
> >> Sent: Wednesday, 29 September 2021 02:04
> >> To: ffmpeg-devel@ffmpeg.org
> >> Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure
> >> frames are writable when processing in-place
> >>
> >> Soft Works:
> >>>
> >>>
> >>>> -Original Message-
> >>>> From: ffmpeg-devel On Behalf
> Of
> >>>> Andreas Rheinhardt
> >>>> Sent: Wednesday, 29 September 2021 01:42
> >>>> To: ffmpeg-devel@ffmpeg.org
> >>>> Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames:
> Ensure
> >>>> frames are writable when processing in-place
> >>>>
> >>>> Soft Works:
> >>>>>
> >>>>>
> >>>>>> -Original Message-----
> >>>>>> From: ffmpeg-devel On Behalf
> >> Of
> >>>>>> James Almer
> >>>>>> Sent: Tuesday, 28 September 2021 22:08
> >>>>>> To: ffmpeg-devel@ffmpeg.org
> >>>>>> Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames:
> >> Ensure
> >>>>>> frames are writable when processing in-place
> >>>>>>
> >>>>>> On 9/28/2021 4:54 PM, Soft Works wrote:
> >>>>>>> Signed-off-by: softworkz
> >>>>>>> ---
> >>>>>>> v2: Reduced to cases without AVFILTERPAD_FLAG_NEEDS_WRITABLE
> >>>>>>
> >>>>>> Can't this flag used in these filters?
> >>>>>
> >>>>> Not in vf_vflip, because in case of flip_bayer, it's using a
> >>>>> custom allocation (.get_buffer.video)
> >>>>>
> >>>>>>> libavfilter/vf_cover_rect.c | 7 +--
> >>>>>>> libavfilter/vf_floodfill.c | 5 +
> >>>>>>> libavfilter/vf_vflip.c | 7 ++-
> >>>>>>> 3 files changed, 16 insertions(+), 3 deletions(-)
> >>>>>>>
> >>>>>>> diff --git a/libavfilter/vf_cover_rect.c
> >>>>>> b/libavfilter/vf_cover_rect.c
> >>>>>>> index 0a8c10e06d..2367afb4b3 100644
> >>>>>>> --- a/libavfilter/vf_cover_rect.c
> >>>>>>> +++ b/libavfilter/vf_cover_rect.c
> >>>>>>> @@ -136,7 +136,7 @@ static int filter_frame(AVFilterLink
> >> *inlink,
> >>>>>> AVFrame *in)
> >>>>>>> AVFilterContext *ctx = inlink->dst;
> >>>>>>> CoverContext *cover = ctx->priv;
> >>>>>>> AVDictionaryEntry *ex, *ey, *ew, *eh;
> >>>>>>> -int x = -1, y = -1, w = -1, h = -1;
> >>>>>>> +int x = -1, y = -1, w = -1, h = -1, ret;
> >>>>>>> char *xendptr = NULL, *yendptr = NULL, *wendptr = NULL,
> >>>>>> *hendptr = NULL;
> >>>>>>>
> >>>>>>> ex = av_dict_get(in->metadata, "lavfi.rect.x", NULL,
> >>>>>> AV_DICT_MATCH_CASE);
> >>>>>>> @@ -181,7 +181,10 @@ static int filter_frame(AVFilterLink
> >>>> *inlink,
> >>>>>> AVFrame *in)
> >>>>>>> x = av_clip(x, 0, in->width - w);
> >>>>>>> y = av_clip(y, 0, in->height - h);
> >>>>>>>
> >>>>>>> -av_frame_make_writable(in);
> >>>>>>> +if ((ret = av_frame_make_writable(in)) < 0) {
> >>>>>>> +av_frame_free(&in);
> >>>>>>> +return ret;
> >>>>>>> +}
> >>>>>>>
> >>>>>>> if (cover->mode == MODE_BLUR) {
> >>>>>>> blur (cover, in, x, y);
> >>>>>>> diff --git a/libavfilter/vf_floodfill.c
> >>>>>> b/libavfilter/vf_floodfill.c
> >>>>>>> index 2174
Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure frames are writable when processing in-place
> -Original Message- > From: ffmpeg-devel On Behalf Of > Soft Works > Sent: Wednesday, 29 September 2021 02:48 > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure > frames are writable when processing in-place > > > > > -Original Message- > > From: ffmpeg-devel On Behalf Of > > Andreas Rheinhardt > > Sent: Wednesday, 29 September 2021 02:25 > > To: ffmpeg-devel@ffmpeg.org > > Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure > > frames are writable when processing in-place > > > > Soft Works: > > > > > > > > >> -Original Message- > > >> From: ffmpeg-devel On Behalf > Of > > >> Andreas Rheinhardt > > >> Sent: Wednesday, 29 September 2021 02:04 > > >> To: ffmpeg-devel@ffmpeg.org > > >> Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: > Ensure > > >> frames are writable when processing in-place > > >> > > >> Soft Works: > > >>> > > >>> > > >>>> -Original Message- > > >>>> From: ffmpeg-devel On Behalf > > Of > > >>>> Andreas Rheinhardt > > >>>> Sent: Wednesday, 29 September 2021 01:42 > > >>>> To: ffmpeg-devel@ffmpeg.org > > >>>> Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: > > Ensure > > >>>> frames are writable when processing in-place > > >>>> > > >>>> Soft Works: > > >>>>> > > >>>>> > > >>>>>> -Original Message- > > >>>>>> From: ffmpeg-devel On > Behalf > > >> Of > > >>>>>> James Almer > > >>>>>> Sent: Tuesday, 28 September 2021 22:08 > > >>>>>> To: ffmpeg-devel@ffmpeg.org > > >>>>>> Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: > > >> Ensure > > >>>>>> frames are writable when processing in-place > > >>>>>> > > >>>>>> On 9/28/2021 4:54 PM, Soft Works wrote: > > >>>>>>> Signed-off-by: softworkz > > >>>>>>> --- > > >>>>>>> v2: Reduced to cases without > AVFILTERPAD_FLAG_NEEDS_WRITABLE > > >>>>>> > > >>>>>> Can't this flag used in these filters? > > >>>>> > > >>>>> Not in vf_vflip, because in case of flip_bayer, it's using a > > >>>>> custom allocation (.get_buffer.video) > > >>>>> > > >>>>>>> libavfilter/vf_cover_rect.c | 7 +-- > > >>>>>>> libavfilter/vf_floodfill.c | 5 + > > >>>>>>> libavfilter/vf_vflip.c | 7 ++- > > >>>>>>> 3 files changed, 16 insertions(+), 3 deletions(-) > > >>>>>>> > > >>>>>>> diff --git a/libavfilter/vf_cover_rect.c > > >>>>>> b/libavfilter/vf_cover_rect.c > > >>>>>>> index 0a8c10e06d..2367afb4b3 100644 > > >>>>>>> --- a/libavfilter/vf_cover_rect.c > > >>>>>>> +++ b/libavfilter/vf_cover_rect.c > > >>>>>>> @@ -136,7 +136,7 @@ static int filter_frame(AVFilterLink > > >> *inlink, > > >>>>>> AVFrame *in) > > >>>>>>> AVFilterContext *ctx = inlink->dst; > > >>>>>>> CoverContext *cover = ctx->priv; > > >>>>>>> AVDictionaryEntry *ex, *ey, *ew, *eh; > > >>>>>>> -int x = -1, y = -1, w = -1, h = -1; > > >>>>>>> +int x = -1, y = -1, w = -1, h = -1, ret; > > >>>>>>> char *xendptr = NULL, *yendptr = NULL, *wendptr = > NULL, > > >>>>>> *hendptr = NULL; > > >>>>>>> > > >>>>>>> ex = av_dict_get(in->metadata, "lavfi.rect.x", NULL, > > >>>>>> AV_DICT_MATCH_CASE); > > >>>>>>> @@ -181,7 +181,10 @@ static int filter_frame(AVFilterLink > > >>>> *inlink, > > >>>>>> AVFrame *in) > > >>>>>>> x = av_clip(x, 0, in-
Re: [FFmpeg-devel] [PATCH] avfilter/vf_morpho: Fix compilation with MSVC
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Andreas Rheinhardt
> Sent: Wednesday, 29 September 2021 07:10
> To: ffmpeg-devel@ffmpeg.org
> Cc: Andreas Rheinhardt
> Subject: [FFmpeg-devel] [PATCH] avfilter/vf_morpho: Fix compilation
> with MSVC
>
> MSVC (or rather the windef.h header) defines min and max as macros,
> breaking compilation here. Fix this by adding parentheses around min
> and max, so that they are no longer recognizable as function-like
> macros.
>
> Signed-off-by: Andreas Rheinhardt
> ---
> Untested as I don't have an MSVC setup.
>
> libavfilter/vf_morpho.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/libavfilter/vf_morpho.c b/libavfilter/vf_morpho.c
> index fd181ddaa5..b2a0a61543 100644
> --- a/libavfilter/vf_morpho.c
> +++ b/libavfilter/vf_morpho.c
> @@ -304,7 +304,7 @@ static void compute_min_row(IPlane *f, LUT *Ty,
> chord_set *SE, int r, int y)
> for (int i = 1; i < SE->Lnum; i++) {
> int d = SE->R[i] - SE->R[i - 1];
>
> -f->min(Ty->arr[r][i] - Ty->pre_pad_x * f->type_size,
> +(f->min)(Ty->arr[r][i] - Ty->pre_pad_x * f->type_size,
> Ty->arr[r][i - 1] - Ty->pre_pad_x * f->type_size,
> Ty->arr[r][i - 1] + (d - Ty->pre_pad_x) * f->type_size,
> Ty->X + Ty->pre_pad_x - d);
> @@ -358,7 +358,7 @@ static void compute_max_row(IPlane *f, LUT *Ty,
> chord_set *SE, int r, int y)
> for (int i = 1; i < SE->Lnum; i++) {
> int d = SE->R[i] - SE->R[i - 1];
>
> -f->max(Ty->arr[r][i] - Ty->pre_pad_x * f->type_size,
> +(f->max)(Ty->arr[r][i] - Ty->pre_pad_x * f->type_size,
> Ty->arr[r][i - 1] - Ty->pre_pad_x * f->type_size,
> Ty->arr[r][i - 1] + (d - Ty->pre_pad_x) * f->type_size,
> Ty->X + Ty->pre_pad_x - d);
> --
It compiles successfully with MSVC.
But I think you should add some comments explaining the parentheses
to prevent other developers from thinking they would be useless
and could be removed.
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure frames are writable when processing in-place
> -Original Message- > From: ffmpeg-devel On Behalf Of > Andreas Rheinhardt > Sent: Wednesday, 29 September 2021 10:28 > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure > frames are writable when processing in-place > > Soft Works: > > > > > >> -Original Message- > >> From: ffmpeg-devel On Behalf Of > >> James Almer > >> Sent: Tuesday, 28 September 2021 22:08 > >> To: ffmpeg-devel@ffmpeg.org > >> Subject: Re: [FFmpeg-devel] [PATCH v2 1/1] avfilter/frames: Ensure > >> frames are writable when processing in-place > >> > >> On 9/28/2021 4:54 PM, Soft Works wrote: > >>> Signed-off-by: softworkz > >>> --- > >>> v2: Reduced to cases without AVFILTERPAD_FLAG_NEEDS_WRITABLE > >> > >> Can't this flag used in these filters? > > > > I mentioned before that I don’t consider this as a good API > > design. It had even caused a review of my subtitle filters to > > assume that my filter code would be incorrect, even though I > > had been using that flag. > > Would you please provide a reference for this? Especially after what > you > said on IRC I have to presume that this refers to my review of your > textmod filter [1] contained in v5 of your patches. Yet you did not > use > said flag in this patch at all (you seem to have never used that flag > in > any of your patches); in fact, using that flag would not have worked > at > all, as it was only v6 which added support for making subtitle frames > writable in ff_inlink_make_frame_writable(). In v5, not even > av_frame_make_writable() would have worked, as I have explained to > you > in [2]. I apologize when I mixed up one or two details, that I had memorized differently. Though, it still illustrates why I'm saying that the existence of that flag is causing unnecessary confusion for minimal benefit. Kind regards, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v5 1/7] libavformat/asf: Fix handling of byte array length values
The spec allows attachment sizes of up to UINT32_MAX while
we can handle only sizes up to INT32_MAX (in downstream
code).
The debug.assert in get_tag didn't really address this,
and truncating the value_len in calling methods cannot
be used because the length value is required in order to
continue parsing. This adds a check with log message in
ff_asf_handle_byte_array to handle those (rare) cases.
Signed-off-by: softworkz
---
v5: Split into pieces as requested
libavformat/asf.c | 12 +---
libavformat/asf.h | 2 +-
2 files changed, 10 insertions(+), 4 deletions(-)
diff --git a/libavformat/asf.c b/libavformat/asf.c
index 1ac8b5f078..179b66a2b4 100644
--- a/libavformat/asf.c
+++ b/libavformat/asf.c
@@ -267,12 +267,18 @@ static int get_id3_tag(AVFormatContext *s, int len)
}
int ff_asf_handle_byte_array(AVFormatContext *s, const char *name,
- int val_len)
+ uint32_t val_len)
{
+if (val_len > INT32_MAX) {
+av_log(s, AV_LOG_VERBOSE, "Unable to handle byte arrays > INT32_MAX
in tag %s.\n", name);
+return 1;
+}
+
if (!strcmp(name, "WM/Picture")) // handle cover art
-return asf_read_picture(s, val_len);
+return asf_read_picture(s, (int)val_len);
else if (!strcmp(name, "ID3")) // handle ID3 tag
-return get_id3_tag(s, val_len);
+return get_id3_tag(s, (int)val_len);
+av_log(s, AV_LOG_VERBOSE, "Unsupported byte array in tag %s.\n", name);
return 1;
}
diff --git a/libavformat/asf.h b/libavformat/asf.h
index 01cc4f7a46..4d28560f56 100644
--- a/libavformat/asf.h
+++ b/libavformat/asf.h
@@ -111,7 +111,7 @@ extern const AVMetadataConv ff_asf_metadata_conv[];
* is unsupported by this function and 0 otherwise.
*/
int ff_asf_handle_byte_array(AVFormatContext *s, const char *name,
- int val_len);
+ uint32_t val_len);
#define ASF_PACKET_FLAG_ERROR_CORRECTION_PRESENT 0x80 //1000
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v5 2/7] libavformat/asfdec: Fix get_value return type
get_value had a return type of int, which means that reading
QWORDS (case 4) was broken due to truncation of the result from
avio_rl64().
Signed-off-by: softworkz
---
v5: Split into pieces as requested
libavformat/asfdec_f.c | 10 +-
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/libavformat/asfdec_f.c b/libavformat/asfdec_f.c
index 72ba8b32a0..076b5ab147 100644
--- a/libavformat/asfdec_f.c
+++ b/libavformat/asfdec_f.c
@@ -202,7 +202,7 @@ static int asf_probe(const AVProbeData *pd)
/* size of type 2 (BOOL) is 32bit for "Extended Content Description Object"
* but 16 bit for "Metadata Object" and "Metadata Library Object" */
-static int get_value(AVIOContext *pb, int type, int type2_size)
+static uint64_t get_value(AVIOContext *pb, int type, int type2_size)
{
switch (type) {
case 2:
@@ -568,9 +568,9 @@ static int asf_read_ext_content_desc(AVFormatContext *s,
int64_t size)
* ASF stream count starts at 1. I am using 0 to the container value
* since it's unused. */
if (!strcmp(name, "AspectRatioX"))
-asf->dar[0].num = get_value(s->pb, value_type, 32);
+asf->dar[0].num = (int)get_value(s->pb, value_type, 32);
else if (!strcmp(name, "AspectRatioY"))
-asf->dar[0].den = get_value(s->pb, value_type, 32);
+asf->dar[0].den = (int)get_value(s->pb, value_type, 32);
else
get_tag(s, name, value_type, value_len, 32);
}
@@ -630,11 +630,11 @@ static int asf_read_metadata(AVFormatContext *s, int64_t
size)
i, stream_num, name_len_utf16, value_type, value_len, name);
if (!strcmp(name, "AspectRatioX")){
-int aspect_x = get_value(s->pb, value_type, 16);
+int aspect_x = (int)get_value(s->pb, value_type, 16);
if(stream_num < 128)
asf->dar[stream_num].num = aspect_x;
} else if(!strcmp(name, "AspectRatioY")){
-int aspect_y = get_value(s->pb, value_type, 16);
+int aspect_y = (int)get_value(s->pb, value_type, 16);
if(stream_num < 128)
asf->dar[stream_num].den = aspect_y;
} else {
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v5 3/7] libavformat/asfdec: Fix type of value_len
The value_len is an uint32 not an int32 per spec. That
value must not be truncated, neither by casting to int, nor by any
conditional checks, because at the end of get_tag, this value is
needed to move forward in parsing. When the len value gets
modified, the parsing may break.
Signed-off-by: softworkz
---
v5: Split into pieces as requested
libavformat/asfdec_f.c | 24 +++-
1 file changed, 11 insertions(+), 13 deletions(-)
diff --git a/libavformat/asfdec_f.c b/libavformat/asfdec_f.c
index 076b5ab147..d017fae019 100644
--- a/libavformat/asfdec_f.c
+++ b/libavformat/asfdec_f.c
@@ -218,7 +218,7 @@ static uint64_t get_value(AVIOContext *pb, int type, int
type2_size)
}
}
-static void get_tag(AVFormatContext *s, const char *key, int type, int len,
int type2_size)
+static void get_tag(AVFormatContext *s, const char *key, int type, uint32_t
len, int type2_size)
{
ASFContext *asf = s->priv_data;
char *value = NULL;
@@ -528,7 +528,7 @@ static int asf_read_ext_stream_properties(AVFormatContext
*s, int64_t size)
static int asf_read_content_desc(AVFormatContext *s, int64_t size)
{
AVIOContext *pb = s->pb;
-int len1, len2, len3, len4, len5;
+uint32_t len1, len2, len3, len4, len5;
len1 = avio_rl16(pb);
len2 = avio_rl16(pb);
@@ -602,25 +602,23 @@ static int asf_read_metadata(AVFormatContext *s, int64_t
size)
{
AVIOContext *pb = s->pb;
ASFContext *asf = s->priv_data;
-int n, stream_num, name_len_utf16, name_len_utf8, value_len;
+int n, name_len_utf8;
+uint16_t stream_num, name_len_utf16, value_type;
+uint32_t value_len;
int ret, i;
n = avio_rl16(pb);
for (i = 0; i < n; i++) {
uint8_t *name;
-int value_type;
avio_rl16(pb); // lang_list_index
-stream_num = avio_rl16(pb);
-name_len_utf16 = avio_rl16(pb);
-value_type = avio_rl16(pb); /* value_type */
-value_len = avio_rl32(pb);
+stream_num = (uint16_t)avio_rl16(pb);
+name_len_utf16 = (uint16_t)avio_rl16(pb);
+value_type = (uint16_t)avio_rl16(pb); /* value_type */
+value_len = avio_rl32(pb);
-if (value_len < 0 || value_len > UINT16_MAX)
-return AVERROR_INVALIDDATA;
-
-name_len_utf8 = 2*name_len_utf16 + 1;
-name = av_malloc(name_len_utf8);
+name_len_utf8 = 2 * name_len_utf16 + 1;
+name = av_malloc(name_len_utf8);
if (!name)
return AVERROR(ENOMEM);
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v5 4/7] libavformat/asfdec: Fixing get_tag
These three are closely related and can't be separated easily:
In get_tag, the code was adding 22 bytes (in order to allow
it to hold 64bit numbers as string) to the value len for creating
creating a buffer. This was unnecessarily imposing a
size-constraint on the value_len parameter.
The code in get_tag, was limiting the maximum value_len to
half the size of INT32. This was applied for all value types, even
though it is required only in case of ASF_UNICODE, not for any
other ones (like ASCII).
get_tag was always allocating a buffer regardless of the
datatype, even though this isn't required in case of ASF_BYTE_ARRAY
The check for the return value from ff_asf_handle_byte_array()
being >0 is removed here because the log message is emitted
by the function itself now.
Signed-off-by: softworkz
---
v5: Split into pieces as requested
libavformat/asfdec_f.c | 50 --
1 file changed, 38 insertions(+), 12 deletions(-)
diff --git a/libavformat/asfdec_f.c b/libavformat/asfdec_f.c
index d017fae019..d35eec7082 100644
--- a/libavformat/asfdec_f.c
+++ b/libavformat/asfdec_f.c
@@ -221,37 +221,63 @@ static uint64_t get_value(AVIOContext *pb, int type, int
type2_size)
static void get_tag(AVFormatContext *s, const char *key, int type, uint32_t
len, int type2_size)
{
ASFContext *asf = s->priv_data;
-char *value = NULL;
int64_t off = avio_tell(s->pb);
-#define LEN 22
-
-av_assert0((unsigned)len < (INT_MAX - LEN) / 2);
+char *value = NULL;
+uint64_t required_bufferlen;
+int buffer_len;
if (!asf->export_xmp && !strncmp(key, "xmp", 3))
goto finish;
-value = av_malloc(2 * len + LEN);
+switch (type) {
+case ASF_UNICODE:
+required_bufferlen = (uint64_t)len * 2 + 1;
+break;
+case -1: // ASCII
+required_bufferlen = (uint64_t)len + 1;
+break;
+case ASF_BYTE_ARRAY:
+ff_asf_handle_byte_array(s, key, len);
+goto finish;
+case ASF_BOOL:
+case ASF_DWORD:
+case ASF_QWORD:
+case ASF_WORD:
+required_bufferlen = 22;
+break;
+case ASF_GUID:
+required_bufferlen = 33;
+break;
+default:
+required_bufferlen = len;
+break;
+}
+
+if (required_bufferlen > INT32_MAX) {
+av_log(s, AV_LOG_VERBOSE, "Unable to handle values > INT32_MAX in tag
%s.\n", key);
+goto finish;
+}
+
+buffer_len = (int)required_bufferlen;
+
+value = av_malloc(buffer_len);
if (!value)
goto finish;
switch (type) {
case ASF_UNICODE:
-avio_get_str16le(s->pb, len, value, 2 * len + 1);
+avio_get_str16le(s->pb, len, value, buffer_len);
break;
-case -1: // ASCI
+case -1: // ASCII
avio_read(s->pb, value, len);
value[len]=0;
break;
-case ASF_BYTE_ARRAY:
-if (ff_asf_handle_byte_array(s, key, len) > 0)
-av_log(s, AV_LOG_VERBOSE, "Unsupported byte array in tag %s.\n",
key);
-goto finish;
case ASF_BOOL:
case ASF_DWORD:
case ASF_QWORD:
case ASF_WORD: {
uint64_t num = get_value(s->pb, type, type2_size);
-snprintf(value, LEN, "%"PRIu64, num);
+snprintf(value, buffer_len, "%"PRIu64, num);
break;
}
case ASF_GUID:
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v5 5/7] libavformat/asfdec: Implement parsing of GUID values
Signed-off-by: softworkz
---
v5: Split into pieces as requested
libavformat/asfdec_f.c | 9 ++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/libavformat/asfdec_f.c b/libavformat/asfdec_f.c
index d35eec7082..7ac4e460a7 100644
--- a/libavformat/asfdec_f.c
+++ b/libavformat/asfdec_f.c
@@ -280,9 +280,12 @@ static void get_tag(AVFormatContext *s, const char *key,
int type, uint32_t len,
snprintf(value, buffer_len, "%"PRIu64, num);
break;
}
-case ASF_GUID:
-av_log(s, AV_LOG_DEBUG, "Unsupported GUID value in tag %s.\n", key);
-goto finish;
+case ASF_GUID: {
+ff_asf_guid g;
+ff_get_guid(s->pb, &g);
+snprintf(value, buffer_len, "%x", g[0]);
+break;
+}
default:
av_log(s, AV_LOG_DEBUG,
"Unsupported value type %d in tag %s.\n", type, key);
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v5 6/7] libavformat/asfdec: Use predefined constants
Signed-off-by: softworkz
---
v5: Split into pieces as requested
libavformat/asfdec_f.c | 8
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/libavformat/asfdec_f.c b/libavformat/asfdec_f.c
index 7ac4e460a7..c4bcf0904d 100644
--- a/libavformat/asfdec_f.c
+++ b/libavformat/asfdec_f.c
@@ -205,13 +205,13 @@ static int asf_probe(const AVProbeData *pd)
static uint64_t get_value(AVIOContext *pb, int type, int type2_size)
{
switch (type) {
-case 2:
+case ASF_BOOL:
return (type2_size == 32) ? avio_rl32(pb) : avio_rl16(pb);
-case 3:
+case ASF_DWORD:
return avio_rl32(pb);
-case 4:
+case ASF_QWORD:
return avio_rl64(pb);
-case 5:
+case ASF_WORD:
return avio_rl16(pb);
default:
return INT_MIN;
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v5 7/7] libavformat/asfdec: Add braces to define
Signed-off-by: softworkz
---
v5: Split into pieces as requested
libavformat/asfdec_f.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/libavformat/asfdec_f.c b/libavformat/asfdec_f.c
index c4bcf0904d..870d4bc7e6 100644
--- a/libavformat/asfdec_f.c
+++ b/libavformat/asfdec_f.c
@@ -149,7 +149,7 @@ static const ff_asf_guid asf_audio_conceal_none = {
};
#define PRINT_IF_GUID(g, cmp) \
-if (!ff_guidcmp(g, &cmp)) \
+if (!ff_guidcmp(g, &(cmp))) \
av_log(NULL, AV_LOG_TRACE, "(GUID: %s) ", # cmp)
static void print_guid(ff_asf_guid *g)
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 1/1] avutil/frame: Document the possibility of negative line sizes
Signed-off-by: softworkz
---
libavutil/frame.h | 13 +++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/libavutil/frame.h b/libavutil/frame.h
index ff2540a20f..f07e690410 100644
--- a/libavutil/frame.h
+++ b/libavutil/frame.h
@@ -304,7 +304,8 @@ typedef struct AVFrame {
#define AV_NUM_DATA_POINTERS 8
/**
* pointer to the picture/channel planes.
- * This might be different from the first allocated byte
+ * This might be different from the first allocated byte. For video,
+ * it could even point to the end of the image data.
*
* Some decoders access areas outside 0,0 - width,height, please
* see avcodec_align_dimensions2(). Some filters and swscale can read
@@ -313,11 +314,16 @@ typedef struct AVFrame {
*
* NOTE: Except for hwaccel formats, pointers not needed by the format
* MUST be set to NULL.
+ *
+ * @attention In case of video, the data[] pointers can point to the
+ * end of image data in order to reverse line order, when used in
+ * combination with negative values in the linesize[] array.
*/
uint8_t *data[AV_NUM_DATA_POINTERS];
/**
- * For video, size in bytes of each picture line.
+ * For video, a positive or negative value, the ABS() value of which is
indicating
+ * the size in bytes of each picture line.
* For audio, size in bytes of each plane.
*
* For audio, only linesize[0] may be set. For planar audio, each channel
@@ -330,6 +336,9 @@ typedef struct AVFrame {
*
* @note The linesize may be larger than the size of usable data -- there
* may be extra padding present for performance reasons.
+ *
+ * @attention In case of video, line size values can be negative to achieve
+ * a vertically inverted iteration over image lines.
*/
int linesize[AV_NUM_DATA_POINTERS];
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 3/7] libavformat/asfdec: Fix type of value_len
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> James Almer
> Sent: Thursday, 30 September 2021 05:53
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v5 3/7] libavformat/asfdec: Fix
> type of value_len
>
> On 9/29/2021 11:58 PM, Soft Works wrote:
> > The value_len is an uint32 not an int32 per spec. That
> > value must not be truncated, neither by casting to int, nor by any
> > conditional checks, because at the end of get_tag, this value is
> > needed to move forward in parsing. When the len value gets
> > modified, the parsing may break.
> >
> > Signed-off-by: softworkz
> > ---
> > v5: Split into pieces as requested
> >
> > libavformat/asfdec_f.c | 24 +++-
> > 1 file changed, 11 insertions(+), 13 deletions(-)
> >
> > diff --git a/libavformat/asfdec_f.c b/libavformat/asfdec_f.c
> > index 076b5ab147..d017fae019 100644
> > --- a/libavformat/asfdec_f.c
> > +++ b/libavformat/asfdec_f.c
> > @@ -218,7 +218,7 @@ static uint64_t get_value(AVIOContext *pb, int
> type, int type2_size)
> > }
> > }
> >
> > -static void get_tag(AVFormatContext *s, const char *key, int type,
> int len, int type2_size)
> > +static void get_tag(AVFormatContext *s, const char *key, int type,
> uint32_t len, int type2_size)
>
> There's an av_assert0() in this function to ensure len will never be
> bigger than (INT_MAX - 22) / 2. And len is used as argument for
> avio_read(), which takes an int, not an unsigned int. So this change
> is
> not ok.
You missed this part:
if (required_bufferlen > INT32_MAX) {
av_log(s, AV_LOG_VERBOSE, "Unable to handle values > INT32_MAX in tag
%s.\n", key);
goto finish;
}
which comes before avio_read.
goto finish doesn't mean to error out. It just means that value cannot
be processed (even though it's valid from the ASF spec).
But parsing continues now without erroring, because now we can handle
those values > INT32_MAX because we supply the correct value to
avio_seek() (the last statement).
Also:
The division by two applies to Unicode only, that's why I'm applying it
only to the Unicode case. The subtraction of '22' was not required, that's
why it's removed.
Kind regards,
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 3/7] libavformat/asfdec: Fix type of value_len
> -Original Message- > From: ffmpeg-devel On Behalf Of > James Almer > Sent: Thursday, 30 September 2021 05:53 > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH v5 3/7] libavformat/asfdec: Fix > type of value_len > > On 9/29/2021 11:58 PM, Soft Works wrote: > > The value_len is an uint32 not an int32 per spec. That > > value must not be truncated, neither by casting to int, nor by any > > conditional checks, because at the end of get_tag, this value is > > needed to move forward in parsing. When the len value gets > > modified, the parsing may break. > > > > Signed-off-by: softworkz > > --- > > v5: Split into pieces as requested > > > > libavformat/asfdec_f.c | 24 +++- > > 1 file changed, 11 insertions(+), 13 deletions(-) > > > > diff --git a/libavformat/asfdec_f.c b/libavformat/asfdec_f.c > > index 076b5ab147..d017fae019 100644 > > --- a/libavformat/asfdec_f.c > > +++ b/libavformat/asfdec_f.c > > @@ -218,7 +218,7 @@ static uint64_t get_value(AVIOContext *pb, int > type, int type2_size) > > } > > } > > > > -static void get_tag(AVFormatContext *s, const char *key, int type, > int len, int type2_size) > > +static void get_tag(AVFormatContext *s, const char *key, int type, > uint32_t len, int type2_size) > > There's an av_assert0() in this function to ensure len will never be > bigger than (INT_MAX - 22) / 2. And len is used as argument for > avio_read(), which takes an int, not an unsigned int. So this change > is > not ok. The general situation is this: The ASF spec allows sizes of various elements in its format up to a maximum of UINT32_MAX. As you have correctly noted: we can't process those sizes because some of the internal functions are taking int32 parameters rather than uint32. (e.g. with avio_read, get_id3_tag, asf_read_picture, ...) This patch can't fix that part. But what it fixes is that such values between INT32_MAX and UINT32_MAX won't break ASF parsing anymore. We emit a message that there's some metadata value that we can't read, but we skip over it and continue parsing. That's one part of what this patch does. Kind regards, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 3/7] libavformat/asfdec: Fix type of value_len
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Soft Works
> Sent: Thursday, 30 September 2021 04:59
> To: ffmpeg-devel@ffmpeg.org
> Subject: [FFmpeg-devel] [PATCH v5 3/7] libavformat/asfdec: Fix type
> of value_len
>
> The value_len is an uint32 not an int32 per spec. That
> value must not be truncated, neither by casting to int, nor by any
> conditional checks, because at the end of get_tag, this value is
> needed to move forward in parsing. When the len value gets
> modified, the parsing may break.
>
> Signed-off-by: softworkz
> ---
> v5: Split into pieces as requested
>
> libavformat/asfdec_f.c | 24 +++-
> 1 file changed, 11 insertions(+), 13 deletions(-)
>
> diff --git a/libavformat/asfdec_f.c b/libavformat/asfdec_f.c
> index 076b5ab147..d017fae019 100644
> --- a/libavformat/asfdec_f.c
> +++ b/libavformat/asfdec_f.c
> @@ -218,7 +218,7 @@ static uint64_t get_value(AVIOContext *pb, int
> type, int type2_size)
> }
> }
>
> -static void get_tag(AVFormatContext *s, const char *key, int type,
> int len, int type2_size)
> +static void get_tag(AVFormatContext *s, const char *key, int type,
> uint32_t len, int type2_size)
> {
> ASFContext *asf = s->priv_data;
> char *value = NULL;
> @@ -528,7 +528,7 @@ static int
> asf_read_ext_stream_properties(AVFormatContext *s, int64_t size)
> static int asf_read_content_desc(AVFormatContext *s, int64_t size)
> {
> AVIOContext *pb = s->pb;
> -int len1, len2, len3, len4, len5;
> +uint32_t len1, len2, len3, len4, len5;
>
> len1 = avio_rl16(pb);
> len2 = avio_rl16(pb);
> @@ -602,25 +602,23 @@ static int asf_read_metadata(AVFormatContext
> *s, int64_t size)
> {
> AVIOContext *pb = s->pb;
> ASFContext *asf = s->priv_data;
> -int n, stream_num, name_len_utf16, name_len_utf8, value_len;
> +int n, name_len_utf8;
> +uint16_t stream_num, name_len_utf16, value_type;
> +uint32_t value_len;
> int ret, i;
> n = avio_rl16(pb);
>
> for (i = 0; i < n; i++) {
> uint8_t *name;
> -int value_type;
>
> avio_rl16(pb); // lang_list_index
> -stream_num = avio_rl16(pb);
> -name_len_utf16 = avio_rl16(pb);
> -value_type = avio_rl16(pb); /* value_type */
> -value_len = avio_rl32(pb);
> +stream_num = (uint16_t)avio_rl16(pb);
> +name_len_utf16 = (uint16_t)avio_rl16(pb);
> +value_type = (uint16_t)avio_rl16(pb); /* value_type */
> +value_len = avio_rl32(pb);
>
> -if (value_len < 0 || value_len > UINT16_MAX)
> -return AVERROR_INVALIDDATA;
The previous two lines, which were added about a year ago, are actually
the origin of this patchset:
A while ago after rebasing, I noticed that album image extraction from
.wma didn't work anymore in certain cases, namely:
image attachments > 32kB
I found those two lines and wondered, checked against the ASF spec and
noticed the other flaws.
Regarding those two lines, I would suppose that they might have slipped in
accidentally - at least the check against UINT_16_MAX.
Kind regards,
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 2/7] libavformat/asfdec: Fix get_value return type
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Michael Niedermayer
> Sent: Thursday, 30 September 2021 14:04
> To: FFmpeg development discussions and patches de...@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v5 2/7] libavformat/asfdec: Fix
> get_value return type
>
> On Thu, Sep 30, 2021 at 01:45:46PM +0200, Michael Niedermayer wrote:
> > On Thu, Sep 30, 2021 at 02:58:30AM +, Soft Works wrote:
> > > get_value had a return type of int, which means that reading
> > > QWORDS (case 4) was broken due to truncation of the result from
> > > avio_rl64().
> > >
> > > Signed-off-by: softworkz
> > > ---
> > > v5: Split into pieces as requested
> > >
> > > libavformat/asfdec_f.c | 10 +-
> > > 1 file changed, 5 insertions(+), 5 deletions(-)
> > >
> > > diff --git a/libavformat/asfdec_f.c b/libavformat/asfdec_f.c
> > > index 72ba8b32a0..076b5ab147 100644
> > > --- a/libavformat/asfdec_f.c
> > > +++ b/libavformat/asfdec_f.c
> > > @@ -202,7 +202,7 @@ static int asf_probe(const AVProbeData *pd)
> > >
> > > /* size of type 2 (BOOL) is 32bit for "Extended Content
> Description Object"
> > > * but 16 bit for "Metadata Object" and "Metadata Library
> Object" */
> > > -static int get_value(AVIOContext *pb, int type, int type2_size)
> > > +static uint64_t get_value(AVIOContext *pb, int type, int
> type2_size)
> > > {
> > > switch (type) {
> > > case 2:
> > > @@ -568,9 +568,9 @@ static int
> asf_read_ext_content_desc(AVFormatContext *s, int64_t size)
> > > * ASF stream count starts at 1. I am using 0 to the
> container value
> > > * since it's unused. */
> > > if (!strcmp(name, "AspectRatioX"))
> > > -asf->dar[0].num = get_value(s->pb, value_type, 32);
> > > +asf->dar[0].num = (int)get_value(s->pb, value_type,
> 32);
> > > else if (!strcmp(name, "AspectRatioY"))
> > > -asf->dar[0].den = get_value(s->pb, value_type, 32);
> > > +asf->dar[0].den = (int)get_value(s->pb, value_type,
> 32);
> > > else
> > > get_tag(s, name, value_type, value_len, 32);
> > > }
> > > @@ -630,11 +630,11 @@ static int
> asf_read_metadata(AVFormatContext *s, int64_t size)
> > > i, stream_num, name_len_utf16, value_type,
> value_len, name);
> > >
> > > if (!strcmp(name, "AspectRatioX")){
> > > -int aspect_x = get_value(s->pb, value_type, 16);
> > > +int aspect_x = (int)get_value(s->pb, value_type,
> 16);
> > > if(stream_num < 128)
> > > asf->dar[stream_num].num = aspect_x;
> > > } else if(!strcmp(name, "AspectRatioY")){
> > > -int aspect_y = get_value(s->pb, value_type, 16);
> > > +int aspect_y = (int)get_value(s->pb, value_type,
> 16);
> > > if(stream_num < 128)
> > > asf->dar[stream_num].den = aspect_y;
> > > } else {
> >
> > a 64bit/64bit aspect does not work after this, it still just
> silently
> > truncates the value to 32bit, the truncation just happens later
>
> forgot: see av_reduce()
These lines are not about making any change. The get_value() calls
for AspectRatioX/Y are returning 16bit values, before and after.
I added the explicit cast to (int) because get_value() doesn't return
int anymore and as such, to indicate that the cast is intentional.
But the values in this case are limited to 16bit integers anyway.
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v10 00/14] Subtitle Filtering
v10 Update:
- FEATURE-COMPLETE!
No more features or filters will be added to this
patchset
- Added subscale filter
Provides high-quality scaling and rearranging
functionality for graphical subtitles.
This has been the last missing bit for making this
a useful and versatile toolset for subtitle manipulation
- The common ass utility functions remain to reside
in avutil, but in this version, they are marked as
avpriv_ and no longer public like in the previous
version.
- A number of detail fixes are included, but there
haven't been significant changes to the core
implementation in the past few versions.
I'm quite confident about the current state.
- As a final step I'll be going to create a wiki page
presenting a comprehensive collection of usage examples,
showing off the whole range of new capabilities
and also providing proof that it is working universally
in a wide range of use cases.
v9 Update:
- Addressed all review comments (thanks Andreas!)
- Move ass utility functions to avutil as public API
- Handle subtitle decoding via new decode API
(removed av_decode_subtitle3)
- Change 'subtitle_header' to use refcounted buffers
- Allow mapping subtitle streams to filter graph
and direct outputs simultaneously
- Added explanation and justification for test changes
to commit message
--
This patchset introduces filtering support for subtitles.
Besides the ground work, this patchset includes a range
of new filters for subtitle processing:
- subscale (S -> S)
Provides high-quality scaling and rearranging
functionality for graphical subtitles.
- graphicsubs2text (S -> S)
Convert graphic subtitles to text subtitles via OCR
- censor {S -> S)
Allows censoring word in subtitles by a configurable
word list
- show_speaker {S -> S)
Prepend the speaker names to text lines
(those are often available in ass subs)
- split_cc (V -> VS)
Splits out Closed-Caption data as a subtitle stream
for filtering and encoding
- overlay_graphicsubs (VS -> V)
Overlay graphic subtitles onto a video stream
- graphicsub2video {S -> V)
Converts graphic subtitles to (transparent) video frames
- overlay_textsubs {VS -> V)
Overlay text subtitles onto a video stream.
- textsubs2video {S -> V)
Converts text subtitles to video frames
- textmod {S -> S)
Modify subtitle text in a number of ways
- stripstyles {S -> S)
Remove all inline styles from subtitle events
Regards,
softworkz
softworkz (14):
fftools/play,probe: Adjust for subtitle changes
avfilter/subtitles: Add subtitles.c for subtitle frame allocation
avfilter/avfilter: Handle subtitle frames
avfilter/sbuffer: Add sbuffersrc and sbuffersink filters
avfilter/overlay_graphicsubs: Add overlay_graphicsubs and
graphicsub2video filters
fftools/ffmpeg: Replace sub2video with subtitle frame filtering
avfilter/overlay_textsubs: Add overlay_textsubs and textsubs2video
filters
avfilter/textmod: Add textmod, censor and show_speaker filters
avfilter/stripstyles: Add stripstyles filter
avfilter/split_cc: Add split_cc filter for closed caption handling
avfilter/graphicsub2text: Add new graphicsub2text filter (OCR)
avfilter/subscale: Add filter for scaling and/or re-arranging
graphical subtitles
avformat/utils: Add av_stream_get_codec_properties()
ffprobe: Fix incorrect display of closed_captions property
configure | 7 +-
doc/APIchanges| 3 +
doc/filters.texi | 662 +
fftools/ffmpeg.c | 507 ++---
fftools/ffmpeg.h | 14 +-
fftools/ffmpeg_filter.c | 209 --
fftools/ffmpeg_hw.c | 2 +-
fftools/ffmpeg_opt.c | 3 +-
fftools/ffplay.c | 102 ++-
fftools/ffprobe.c | 53 +-
libavfilter/Makefile | 14 +
libavfilter/allfilters.c | 13 +
libavfilter/avfilter.c| 30 +-
libavfilter/avfiltergraph.c | 5 +
libavfilter/buffersink.c | 63 ++
libavfilter/buffersink.h | 15 +
libavfilter/buffersrc.c | 72 ++
libavfilter/buffersrc.h | 1 +
libavfilter/formats.c | 14 +
libavfilter/formats.h | 3 +
libavfilter/internal.h| 1 +
libavfilter/sf_graphicsub2text.c | 323
libavfilter/sf_split_cc.c | 298
libavfilter/sf_stripstyles.c | 211 ++
libavfilter/sf_subscale.c | 867 ++
libavfilter/sf_textmod.c | 719 ++
libavfilter/subtitles.c | 63 ++
libavfilter/subtitles.h | 44
[FFmpeg-devel] [PATCH v10 01/14] fftools/play, probe: Adjust for subtitle changes
Signed-off-by: softworkz
---
fftools/ffplay.c | 102 +-
fftools/ffprobe.c | 48 ++
2 files changed, 79 insertions(+), 71 deletions(-)
diff --git a/fftools/ffplay.c b/fftools/ffplay.c
index ccea0e4578..e8d36485ae 100644
--- a/fftools/ffplay.c
+++ b/fftools/ffplay.c
@@ -152,7 +152,6 @@ typedef struct Clock {
/* Common struct for handling all types of decoded data and allocated render
buffers. */
typedef struct Frame {
AVFrame *frame;
-AVSubtitle sub;
int serial;
double pts; /* presentation timestamp for the frame */
double duration; /* estimated duration of the frame */
@@ -586,7 +585,7 @@ static int decoder_init(Decoder *d, AVCodecContext *avctx,
PacketQueue *queue, S
return 0;
}
-static int decoder_decode_frame(Decoder *d, AVFrame *frame, AVSubtitle *sub) {
+static int decoder_decode_frame(Decoder *d, AVFrame *frame) {
int ret = AVERROR(EAGAIN);
for (;;) {
@@ -620,6 +619,9 @@ static int decoder_decode_frame(Decoder *d, AVFrame *frame,
AVSubtitle *sub) {
}
}
break;
+case AVMEDIA_TYPE_SUBTITLE:
+ret = avcodec_receive_frame(d->avctx, frame);
+break;
}
if (ret == AVERROR_EOF) {
d->finished = d->pkt_serial;
@@ -652,25 +654,11 @@ static int decoder_decode_frame(Decoder *d, AVFrame
*frame, AVSubtitle *sub) {
av_packet_unref(d->pkt);
} while (1);
-if (d->avctx->codec_type == AVMEDIA_TYPE_SUBTITLE) {
-int got_frame = 0;
-ret = avcodec_decode_subtitle2(d->avctx, sub, &got_frame, d->pkt);
-if (ret < 0) {
-ret = AVERROR(EAGAIN);
-} else {
-if (got_frame && !d->pkt->data) {
-d->packet_pending = 1;
-}
-ret = got_frame ? 0 : (d->pkt->data ? AVERROR(EAGAIN) :
AVERROR_EOF);
-}
-av_packet_unref(d->pkt);
+if (avcodec_send_packet(d->avctx, d->pkt) == AVERROR(EAGAIN)) {
+av_log(d->avctx, AV_LOG_ERROR, "Receive_frame and send_packet both
returned EAGAIN, which is an API violation.\n");
+d->packet_pending = 1;
} else {
-if (avcodec_send_packet(d->avctx, d->pkt) == AVERROR(EAGAIN)) {
-av_log(d->avctx, AV_LOG_ERROR, "Receive_frame and send_packet
both returned EAGAIN, which is an API violation.\n");
-d->packet_pending = 1;
-} else {
-av_packet_unref(d->pkt);
-}
+av_packet_unref(d->pkt);
}
}
}
@@ -683,7 +671,6 @@ static void decoder_destroy(Decoder *d) {
static void frame_queue_unref_item(Frame *vp)
{
av_frame_unref(vp->frame);
-avsubtitle_free(&vp->sub);
}
static int frame_queue_init(FrameQueue *f, PacketQueue *pktq, int max_size,
int keep_last)
@@ -981,7 +968,7 @@ static void video_image_display(VideoState *is)
if (frame_queue_nb_remaining(&is->subpq) > 0) {
sp = frame_queue_peek(&is->subpq);
-if (vp->pts >= sp->pts + ((float) sp->sub.start_display_time /
1000)) {
+if (vp->pts >= sp->pts + ((float) sp->frame->subtitle_start_time /
1000)) {
if (!sp->uploaded) {
uint8_t* pixels[4];
int pitch[4];
@@ -993,25 +980,27 @@ static void video_image_display(VideoState *is)
if (realloc_texture(&is->sub_texture,
SDL_PIXELFORMAT_ARGB, sp->width, sp->height, SDL_BLENDMODE_BLEND, 1) < 0)
return;
-for (i = 0; i < sp->sub.num_rects; i++) {
-AVSubtitleRect *sub_rect = sp->sub.rects[i];
+for (i = 0; i < sp->frame->num_subtitle_areas; i++) {
+AVSubtitleArea *area = sp->frame->subtitle_areas[i];
+SDL_Rect sdl_rect = { .x = area->x, .y = area->y, .w =
area->w, .h = area->h };
-sub_rect->x = av_clip(sub_rect->x, 0, sp->width );
-sub_rect->y = av_clip(sub_rect->y, 0, sp->height);
-sub_rect->w = av_clip(sub_rect->w, 0, sp->width -
sub_rect->x);
-sub_rect->h = av_clip(sub_rect->h, 0, sp->height -
sub_rect->y);
+area->x = av_clip(area->x, 0, sp->width );
+area->y = av_clip(area->y, 0, sp->height);
+area->w = av_clip(area->w, 0, sp->width - area->x);
+area->h = av_clip(area->h, 0, sp->height - area->y);
is->sub_convert_ctx =
sws_getCachedContext(is->sub_convert_ctx,
-sub_rect->w, sub_rect->h, AV_PIX_FMT_PAL8,
-sub_rect->w, sub_rec
[FFmpeg-devel] [PATCH v10 02/14] avfilter/subtitles: Add subtitles.c for subtitle frame allocation
Analog to avfilter/video.c and avfilter/audio.c
Signed-off-by: softworkz
---
libavfilter/Makefile| 1 +
libavfilter/avfilter.c | 4 +++
libavfilter/internal.h | 1 +
libavfilter/subtitles.c | 63 +
libavfilter/subtitles.h | 44
5 files changed, 113 insertions(+)
create mode 100644 libavfilter/subtitles.c
create mode 100644 libavfilter/subtitles.h
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index fa2d18e45f..357926624e 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -19,6 +19,7 @@ OBJS = allfilters.o
\
framequeue.o \
graphdump.o \
graphparser.o\
+ subtitles.o \
video.o \
OBJS-$(HAVE_THREADS) += pthread.o
diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
index f325918021..ef349a67f9 100644
--- a/libavfilter/avfilter.c
+++ b/libavfilter/avfilter.c
@@ -43,6 +43,7 @@
#include "formats.h"
#include "framepool.h"
#include "internal.h"
+#include "subtitles.h"
#include "libavutil/ffversion.h"
const char av_filter_ffversion[] = "FFmpeg version " FFMPEG_VERSION;
@@ -1476,6 +1477,9 @@ int ff_inlink_make_frame_writable(AVFilterLink *link,
AVFrame **rframe)
case AVMEDIA_TYPE_AUDIO:
out = ff_get_audio_buffer(link, frame->nb_samples);
break;
+case AVMEDIA_TYPE_SUBTITLE:
+out = ff_get_subtitles_buffer(link, link->format);
+break;
default:
return AVERROR(EINVAL);
}
diff --git a/libavfilter/internal.h b/libavfilter/internal.h
index e7c154aff0..8977dda2b3 100644
--- a/libavfilter/internal.h
+++ b/libavfilter/internal.h
@@ -90,6 +90,7 @@ struct AVFilterPad {
union {
AVFrame *(*video)(AVFilterLink *link, int w, int h);
AVFrame *(*audio)(AVFilterLink *link, int nb_samples);
+AVFrame *(*subtitle)(AVFilterLink *link, int format);
} get_buffer;
/**
diff --git a/libavfilter/subtitles.c b/libavfilter/subtitles.c
new file mode 100644
index 00..951bfd612c
--- /dev/null
+++ b/libavfilter/subtitles.c
@@ -0,0 +1,63 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "libavutil/common.h"
+
+#include "subtitles.h"
+#include "avfilter.h"
+#include "internal.h"
+
+
+AVFrame *ff_null_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+return ff_get_subtitles_buffer(link->dst->outputs[0], format);
+}
+
+AVFrame *ff_default_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+AVFrame *frame;
+
+frame = av_frame_alloc();
+if (!frame)
+return NULL;
+
+frame->format = format;
+frame->type = AVMEDIA_TYPE_SUBTITLE;
+
+if (av_frame_get_buffer2(frame, 0) < 0) {
+av_frame_free(&frame);
+return NULL;
+}
+
+return frame;
+}
+
+AVFrame *ff_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+AVFrame *ret = NULL;
+
+if (link->dstpad->get_buffer.subtitle)
+ret = link->dstpad->get_buffer.subtitle(link, format);
+
+if (!ret)
+ret = ff_default_get_subtitles_buffer(link, format);
+
+return ret;
+}
diff --git a/libavfilter/subtitles.h b/libavfilter/subtitles.h
new file mode 100644
index 00..4a9115126e
--- /dev/null
+++ b/libavfilter/subtitles.h
@@ -0,0 +1,44 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have recei
[FFmpeg-devel] [PATCH v10 03/14] avfilter/avfilter: Handle subtitle frames
Signed-off-by: softworkz
---
libavfilter/avfilter.c | 8 +---
libavfilter/avfiltergraph.c | 5 +
libavfilter/formats.c | 14 ++
libavfilter/formats.h | 3 +++
4 files changed, 27 insertions(+), 3 deletions(-)
diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
index ef349a67f9..c567b7893b 100644
--- a/libavfilter/avfilter.c
+++ b/libavfilter/avfilter.c
@@ -57,7 +57,8 @@ void ff_tlog_ref(void *ctx, AVFrame *ref, int end)
ref->linesize[0], ref->linesize[1], ref->linesize[2],
ref->linesize[3],
ref->pts, ref->pkt_pos);
-if (ref->width) {
+switch(ref->type) {
+case AVMEDIA_TYPE_VIDEO:
ff_tlog(ctx, " a:%d/%d s:%dx%d i:%c iskey:%d type:%c",
ref->sample_aspect_ratio.num, ref->sample_aspect_ratio.den,
ref->width, ref->height,
@@ -65,12 +66,13 @@ void ff_tlog_ref(void *ctx, AVFrame *ref, int end)
ref->top_field_first ? 'T' : 'B',/* Top / Bottom */
ref->key_frame,
av_get_picture_type_char(ref->pict_type));
-}
-if (ref->nb_samples) {
+break;
+case AVMEDIA_TYPE_AUDIO:
ff_tlog(ctx, " cl:%"PRId64"d n:%d r:%d",
ref->channel_layout,
ref->nb_samples,
ref->sample_rate);
+break;
}
ff_tlog(ctx, "]%s", end ? "\n" : "");
diff --git a/libavfilter/avfiltergraph.c b/libavfilter/avfiltergraph.c
index 45b028cd9c..7a5a4ea419 100644
--- a/libavfilter/avfiltergraph.c
+++ b/libavfilter/avfiltergraph.c
@@ -314,6 +314,8 @@ static int filter_link_check_formats(void *log,
AVFilterLink *link, AVFilterForm
return ret;
break;
+case AVMEDIA_TYPE_SUBTITLE:
+return 0;
default:
av_assert0(!"reached");
}
@@ -444,6 +446,9 @@ static int query_formats(AVFilterGraph *graph, void
*log_ctx)
if (!link)
continue;
+if (link->type == AVMEDIA_TYPE_SUBTITLE)
+continue;
+
neg = ff_filter_get_negotiation(link);
av_assert0(neg);
for (neg_step = 1; neg_step < neg->nb_mergers; neg_step++) {
diff --git a/libavfilter/formats.c b/libavfilter/formats.c
index 1d2a51c0af..94c1891b78 100644
--- a/libavfilter/formats.c
+++ b/libavfilter/formats.c
@@ -19,6 +19,7 @@
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
+#include "libavcodec/avcodec.h"
#include "libavutil/avassert.h"
#include "libavutil/channel_layout.h"
#include "libavutil/common.h"
@@ -430,6 +431,12 @@ int ff_add_channel_layout(AVFilterChannelLayouts **l,
uint64_t channel_layout)
return 0;
}
+int ff_add_subtitle_type(AVFilterFormats **avff, int64_t fmt)
+{
+ADD_FORMAT(avff, fmt, ff_formats_unref, int, formats, nb_formats);
+return 0;
+}
+
AVFilterFormats *ff_all_formats(enum AVMediaType type)
{
AVFilterFormats *ret = NULL;
@@ -443,6 +450,13 @@ AVFilterFormats *ff_all_formats(enum AVMediaType type)
return NULL;
fmt++;
}
+} else if (type == AVMEDIA_TYPE_SUBTITLE) {
+if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_BITMAP) < 0)
+return NULL;
+if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_ASS) < 0)
+return NULL;
+if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_TEXT) < 0)
+return NULL;
}
return ret;
diff --git a/libavfilter/formats.h b/libavfilter/formats.h
index 42fe068765..7581fae01c 100644
--- a/libavfilter/formats.h
+++ b/libavfilter/formats.h
@@ -180,6 +180,9 @@ int ff_set_common_formats_from_list(AVFilterContext *ctx,
const int *fmts);
av_warn_unused_result
int ff_add_channel_layout(AVFilterChannelLayouts **l, uint64_t channel_layout);
+av_warn_unused_result
+int ff_add_subtitle_type(AVFilterFormats **avff, int64_t fmt);
+
/**
* Add *ref as a new reference to f.
*/
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v10 04/14] avfilter/sbuffer: Add sbuffersrc and sbuffersink filters
Signed-off-by: softworkz
---
configure| 2 +-
libavfilter/allfilters.c | 2 ++
libavfilter/buffersink.c | 63 +++
libavfilter/buffersink.h | 15 +
libavfilter/buffersrc.c | 72
libavfilter/buffersrc.h | 1 +
6 files changed, 154 insertions(+), 1 deletion(-)
diff --git a/configure b/configure
index 231d0398a8..d18bb73ed6 100755
--- a/configure
+++ b/configure
@@ -7720,7 +7720,7 @@ print_enabled_components(){
fi
done
if [ "$name" = "filter_list" ]; then
-for c in asrc_abuffer vsrc_buffer asink_abuffer vsink_buffer; do
+for c in asrc_abuffer vsrc_buffer ssrc_sbuffer asink_abuffer
vsink_buffer ssink_sbuffer; do
printf "&ff_%s,\n" $c >> $TMPH
done
fi
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 338561a446..51a1d09b47 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -536,8 +536,10 @@ extern const AVFilter ff_avsrc_movie;
* being the same while having different 'types'). */
extern const AVFilter ff_asrc_abuffer;
extern const AVFilter ff_vsrc_buffer;
+extern const AVFilter ff_ssrc_sbuffer;
extern const AVFilter ff_asink_abuffer;
extern const AVFilter ff_vsink_buffer;
+extern const AVFilter ff_ssink_sbuffer;
extern const AVFilter ff_af_afifo;
extern const AVFilter ff_vf_fifo;
diff --git a/libavfilter/buffersink.c b/libavfilter/buffersink.c
index 8b46dcb15e..cd038e1782 100644
--- a/libavfilter/buffersink.c
+++ b/libavfilter/buffersink.c
@@ -29,6 +29,8 @@
#include "libavutil/internal.h"
#include "libavutil/opt.h"
+#include "libavcodec/avcodec.h"
+
#define FF_INTERNAL_FIELDS 1
#include "framequeue.h"
@@ -57,6 +59,10 @@ typedef struct BufferSinkContext {
int *sample_rates; ///< list of accepted sample
rates, terminated by -1
int sample_rates_size;
+/* only used for subtitles */
+enum AVSubtitleType *subtitle_types; ///< list of accepted subtitle
types, must be terminated with -1
+int subtitle_types_size;
+
AVFrame *peeked_frame;
} BufferSinkContext;
@@ -168,6 +174,15 @@ AVABufferSinkParams *av_abuffersink_params_alloc(void)
return NULL;
return params;
}
+
+AVSBufferSinkParams *av_sbuffersink_params_alloc(void)
+{
+AVSBufferSinkParams *params = av_mallocz(sizeof(AVSBufferSinkParams));
+
+if (!params)
+return NULL;
+return params;
+}
#endif
static av_cold int common_init(AVFilterContext *ctx)
@@ -305,6 +320,28 @@ static int asink_query_formats(AVFilterContext *ctx)
return 0;
}
+static int ssink_query_formats(AVFilterContext *ctx)
+{
+BufferSinkContext *buf = ctx->priv;
+AVFilterFormats *formats = NULL;
+unsigned i;
+int ret;
+
+CHECK_LIST_SIZE(subtitle_types)
+if (buf->subtitle_types_size) {
+for (i = 0; i < NB_ITEMS(buf->subtitle_types); i++)
+if ((ret = ff_add_subtitle_type(&formats, buf->subtitle_types[i]))
< 0)
+return ret;
+if ((ret = ff_set_common_formats(ctx, formats)) < 0)
+return ret;
+} else {
+if ((ret = ff_default_query_formats(ctx)) < 0)
+return ret;
+}
+
+return 0;
+}
+
#define OFFSET(x) offsetof(BufferSinkContext, x)
#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM
static const AVOption buffersink_options[] = {
@@ -322,9 +359,16 @@ static const AVOption abuffersink_options[] = {
{ NULL },
};
#undef FLAGS
+#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_SUBTITLE_PARAM
+static const AVOption sbuffersink_options[] = {
+{ "subtitle_types", "set the supported subtitle formats",
OFFSET(subtitle_types), AV_OPT_TYPE_BINARY, .flags = FLAGS },
+{ NULL },
+};
+#undef FLAGS
AVFILTER_DEFINE_CLASS(buffersink);
AVFILTER_DEFINE_CLASS(abuffersink);
+AVFILTER_DEFINE_CLASS(sbuffersink);
static const AVFilterPad avfilter_vsink_buffer_inputs[] = {
{
@@ -363,3 +407,22 @@ const AVFilter ff_asink_abuffer = {
FILTER_INPUTS(avfilter_asink_abuffer_inputs),
.outputs = NULL,
};
+
+static const AVFilterPad avfilter_ssink_sbuffer_inputs[] = {
+{
+.name = "default",
+.type = AVMEDIA_TYPE_SUBTITLE,
+},
+};
+
+const AVFilter ff_ssink_sbuffer = {
+.name = "sbuffersink",
+.description = NULL_IF_CONFIG_SMALL("Buffer subtitle frames, and make
them available to the end of the filter graph."),
+.priv_class= &sbuffersink_class,
+.priv_size = sizeof(BufferSinkContext),
+.init = common_init,
+.query_formats = ssink_query_formats,
+.activate = activate,
+FILTER_INPUTS(avfilter_ssink_sbuffer_inputs),
+.outputs = NULL,
+};
diff --git a/libavfilter/buffersink.h b/libavfilter/buffersink.h
index 69ed0f29a8..b439b586c5 100644
--- a/libavfilter/buffersink.h
+++ b/libavfilter/buffersink.h
@@ -129,6 +129,21 @@ typedef struct AVABufferS
[FFmpeg-devel] [PATCH v10 05/14] avfilter/overlay_graphicsubs: Add overlay_graphicsubs and graphicsub2video filters
- overlay_graphicsubs (VS -> V)
Overlay graphic subtitles onto a video stream
- graphicsub2video {S -> V)
Converts graphic subtitles to video frames (with alpha)
Gets auto-inserted for retaining compatibility with
sub2video command lines
Signed-off-by: softworkz
---
doc/filters.texi | 104
libavfilter/Makefile | 2 +
libavfilter/allfilters.c | 2 +
libavfilter/vf_overlay_graphicsubs.c | 724 +++
4 files changed, 832 insertions(+)
create mode 100644 libavfilter/vf_overlay_graphicsubs.c
diff --git a/doc/filters.texi b/doc/filters.texi
index 6328a8c563..18b5273e6e 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25195,6 +25195,110 @@ tools.
@c man end VIDEO SINKS
+@chapter Subtitle Filters
+@c man begin SUBTITLE FILTERS
+
+When you configure your FFmpeg build, you can disable any of the
+existing filters using @code{--disable-filters}.
+
+Below is a description of the currently available subtitle filters.
+
+@section graphicsub2video
+
+Renders graphic subtitles as video frames.
+
+This filter replaces the previous "sub2video" hack which did the conversion
implicitly and up-front as subtitle filtering wasn't possible at that time.
+To retain compatibility with earlier sub2video command lines, this filter is
being auto-inserted in those cases.
+
+For overlaying graphicsal subtitles it is recommended to use the
'overlay_graphicsubs' filter which is more efficient and takes less processing
resources.
+
+This filter is still useful in cases where the overlay is done with hardware
acceleration (e.g. overlay_qsv, overlay_vaapi, overlay_cuda) for preparing the
overlay frames.
+
+It accepts the following parameters:
+
+@table @option
+@item size, s
+Set the size of the output video frame.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Overlay PGS subtitles
+(not recommended - better use overlay_graphicsubs)
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:1]graphicsub2video[subs];[0:0][subs]overlay" output.mp4
+@end example
+
+@item
+Overlay PGS subtitles implicitly
+The graphicsub2video is inserted automatically for compatibility with legacy
command lines.
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:0][0:1]overlay" output.mp4
+@end example
+@end itemize
+
+@section overlay_graphicsubs
+
+Overlay graphic subtitles onto a video stream.
+
+This filter can blend graphical subtitles on a video stream directly, i.e.
without creating full-size alpha images first.
+The blending operation is limited to the area of the subtitle rectangles,
which also means that no processing is done at times where no subtitles are to
be displayed.
+
+
+It accepts the following parameters:
+
+@table @option
+@item x
+@item y
+Set the expression for the x and y coordinates of the overlaid video
+on the main video. Default value is "0" for both expressions. In case
+the expression is invalid, it is set to a huge value (meaning that the
+overlay will not be displayed within the output visible area).
+
+@item eof_action
+See @ref{framesync}.
+
+@item eval
+Set when the expressions for @option{x}, and @option{y} are evaluated.
+
+It accepts the following values:
+@table @samp
+@item init
+only evaluate expressions once during the filter initialization or
+when a command is processed
+
+@item frame
+evaluate expressions for each incoming frame
+@end table
+
+Default value is @samp{frame}.
+
+@item shortest
+See @ref{framesync}.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Overlay PGS subtitles
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:1]graphicsub2video[subs];[0:0][subs]overlay" output.mp4
+@end example
+
+@item
+Overlay PGS subtitles implicitly
+The graphicsub2video is inserted automatically for compatibility with legacy
command lines.
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:0][0:1]overlay" output.mp4
+@end example
+@end itemize
+@c man end SUBTITLE FILTERS
+
@chapter Multimedia Filters
@c man begin MULTIMEDIA FILTERS
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 357926624e..01c3926bf8 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -290,6 +290,7 @@ OBJS-$(CONFIG_FSPP_FILTER) += vf_fspp.o
qp_table.o
OBJS-$(CONFIG_GBLUR_FILTER) += vf_gblur.o
OBJS-$(CONFIG_GEQ_FILTER)+= vf_geq.o
OBJS-$(CONFIG_GRADFUN_FILTER)+= vf_gradfun.o
+OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlay_graphicsubs.o
framesync.o
OBJS-$(CONFIG_GRAPHMONITOR_FILTER) += f_graphmonitor.o
OBJS-$(CONFIG_GRAYWORLD_FILTER) += vf_grayworld.o
[FFmpeg-devel] [PATCH v10 06/14] fftools/ffmpeg: Replace sub2video with subtitle frame filtering
This commit actually enables subtitle filtering in ffmpeg by
sending and receiving subtitle frames to and from a filtergraph.
The heartbeat functionality from the previous sub2video implementation
is retained and applied to all subtitle frames (bitmap, text, ..).
The other part of sub2video functionality is retained by
auto-insertion of the new graphicsub2video filter.
Justification for changed test refs:
- sub2video
The new results are identical excepting the last frame which
is due to the implementation changes
- sub2video_basic
The previous results had some incorrect output because multiple
frames had the same dts (even the first one was incorrect).
The non-empty content frames are visually identical, the different
CRC is due to the different blending algorithm that is being used.
- sub2video_time_limited
The previous results were incorrect. There was an initial empty frame
with the same dts generated. This is no longer happening.
The third frame was a repetition, which doesn't happen anymore with
the new subtitle filtering.
The source file contains a single subtitle event and the new and
correct result is a single output frame
- sub-dvb
Running ffprobe -show_frames on the source file shows that there
are 7 subtitle frames with 0 rects in the source at the start
and 2 at the end. This translates to the 14 and 4 additional
entries in the new test results.
- filter-overlay-dvdsub-2397
Overlay results have slightly different CRCs due to different
blending implementation
Signed-off-by: softworkz
---
fftools/ffmpeg.c | 507 ++
fftools/ffmpeg.h | 14 +-
fftools/ffmpeg_filter.c | 209 ++---
fftools/ffmpeg_hw.c | 2 +-
fftools/ffmpeg_opt.c | 3 +-
tests/ref/fate/filter-overlay-dvdsub-2397 | 181
tests/ref/fate/sub-dvb| 162 ---
tests/ref/fate/sub2video | 1 -
tests/ref/fate/sub2video_basic| 93 ++--
tests/ref/fate/sub2video_time_limited | 4 +-
10 files changed, 622 insertions(+), 554 deletions(-)
diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c
index 98c2421938..6e7f621d61 100644
--- a/fftools/ffmpeg.c
+++ b/fftools/ffmpeg.c
@@ -169,163 +169,6 @@ static int restore_tty;
static void free_input_threads(void);
#endif
-/* sub2video hack:
- Convert subtitles to video with alpha to insert them in filter graphs.
- This is a temporary solution until libavfilter gets real subtitles support.
- */
-
-static int sub2video_get_blank_frame(InputStream *ist)
-{
-int ret;
-AVFrame *frame = ist->sub2video.frame;
-
-av_frame_unref(frame);
-ist->sub2video.frame->width = ist->dec_ctx->width ? ist->dec_ctx->width
: ist->sub2video.w;
-ist->sub2video.frame->height = ist->dec_ctx->height ? ist->dec_ctx->height
: ist->sub2video.h;
-ist->sub2video.frame->format = AV_PIX_FMT_RGB32;
-if ((ret = av_frame_get_buffer(frame, 0)) < 0)
-return ret;
-memset(frame->data[0], 0, frame->height * frame->linesize[0]);
-return 0;
-}
-
-static void sub2video_copy_rect(uint8_t *dst, int dst_linesize, int w, int h,
-AVSubtitleRect *r)
-{
-uint32_t *pal, *dst2;
-uint8_t *src, *src2;
-int x, y;
-
-if (r->type != SUBTITLE_BITMAP) {
-av_log(NULL, AV_LOG_WARNING, "sub2video: non-bitmap subtitle\n");
-return;
-}
-if (r->x < 0 || r->x + r->w > w || r->y < 0 || r->y + r->h > h) {
-av_log(NULL, AV_LOG_WARNING, "sub2video: rectangle (%d %d %d %d)
overflowing %d %d\n",
-r->x, r->y, r->w, r->h, w, h
-);
-return;
-}
-
-dst += r->y * dst_linesize + r->x * 4;
-src = r->data[0];
-pal = (uint32_t *)r->data[1];
-for (y = 0; y < r->h; y++) {
-dst2 = (uint32_t *)dst;
-src2 = src;
-for (x = 0; x < r->w; x++)
-*(dst2++) = pal[*(src2++)];
-dst += dst_linesize;
-src += r->linesize[0];
-}
-}
-
-static void sub2video_push_ref(InputStream *ist, int64_t pts)
-{
-AVFrame *frame = ist->sub2video.frame;
-int i;
-int ret;
-
-av_assert1(frame->data[0]);
-ist->sub2video.last_pts = frame->pts = pts;
-for (i = 0; i < ist->nb_filters; i++) {
-ret = av_buffersrc_add_frame_flags(ist->filters[i]->filter, frame,
- AV_BUFFERSRC_FLAG_KEEP_REF |
- AV_BUFFERSRC_FLAG_PUSH);
-if (ret != AVERROR_EOF && ret < 0)
-av_log(NULL, AV_LOG_WARNING, "Error while add the frame to buffer
source(%s).\n",
- av_err2str(ret));
-}
-}
-
-void sub2video_update(InputStream *ist, int64_t heartbeat_pts, AVSubtitle *sub)
-{
-AVFrame *frame = ist->sub2video.frame;
-int8_t *dst;
-int dst_linesize;
-int num_rects, i;
-int64_t pts, end_pts;
-
-if (!fra
[FFmpeg-devel] [PATCH v10 07/14] avfilter/overlay_textsubs: Add overlay_textsubs and textsubs2video filters
- overlay_textsubs {VS -> V)
Overlay text subtitles onto a video stream.
- textsubs2video {S -> V)
Converts text subtitles to video frames
Signed-off-by: softworkz
---
configure | 2 +
doc/filters.texi | 69
libavfilter/Makefile | 2 +
libavfilter/allfilters.c | 2 +
libavfilter/avfilter.c| 18 +-
libavfilter/vf_overlay_textsubs.c | 609 ++
6 files changed, 697 insertions(+), 5 deletions(-)
create mode 100644 libavfilter/vf_overlay_textsubs.c
diff --git a/configure b/configure
index d18bb73ed6..37fc4c20e7 100755
--- a/configure
+++ b/configure
@@ -3624,6 +3624,7 @@ openclsrc_filter_deps="opencl"
overlay_opencl_filter_deps="opencl"
overlay_qsv_filter_deps="libmfx"
overlay_qsv_filter_select="qsvvpp"
+overlay_textsubs_filter_deps="avcodec libass"
overlay_vulkan_filter_deps="vulkan_lib libglslang"
owdenoise_filter_deps="gpl"
pad_opencl_filter_deps="opencl"
@@ -3669,6 +3670,7 @@ superequalizer_filter_deps="avcodec"
superequalizer_filter_select="rdft"
surround_filter_deps="avcodec"
surround_filter_select="rdft"
+textsub2video_filter_deps="avcodec libass"
tinterlace_filter_deps="gpl"
tinterlace_merge_test_deps="tinterlace_filter"
tinterlace_pad_test_deps="tinterlace_filter"
diff --git a/doc/filters.texi b/doc/filters.texi
index 18b5273e6e..2f81df6489 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25297,6 +25297,75 @@ The graphicsub2video is inserted automatically for
compatibility with legacy com
ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:0][0:1]overlay" output.mp4
@end example
@end itemize
+
+@section overlay_textsubs
+
+Overlay text subtitles onto a video stream.
+
+This filter supersedes the classic @ref{subtitles} filter opposed to which it
does no longer require to open and access the source stream separately, which
is often causing problems or doesn't even work for non-local or slow sources.
+
+Inputs:
+- 0: Video [YUV420P, YUV422P, YUV444P, ARGB, RGBA, ABGR, BGRA, RGB24, BGR24]
+- 1: Subtitles [text]
+
+Outputs:
+- 0: Video (same as input)
+
+It accepts the following parameters:
+
+@table @option
+
+@item alpha
+Process alpha channel, by default alpha channel is untouched.
+
+@item fonts_dir
+Set a directory path containing fonts that can be used by the filter.
+These fonts will be used in addition to whatever the font provider uses.
+
+@item force_style
+Override default style or script info parameters of the subtitles. It accepts a
+string containing ASS style format @code{KEY=VALUE} couples separated by ",".
+
+@end table
+
+@section textsub2video
+
+Converts text subtitles to video frames.
+
+For overlaying text subtitles onto video frames it is recommended to use the
overlay_textsubs filter.
+The textsub2video is useful for for creating transparent text-frames when
overlay is done via hw acceleration
+
+Inputs:
+- 0: Subtitles [text]
+
+Outputs:
+- 0: Video [ARGB, RGBA, ABGR, BGRA]
+
+It accepts the following parameters:
+
+@table @option
+
+@item rate, r
+Set the framerate for updating overlay frames.
+Normally, overlay frames will only be updated each time when the subtitles to
display are changing.
+In cases where subtitles include advanced features (like animation), this
parameter determines the frequency by which the overlay frames should be
updated.
+
+@item size, s
+Set the output frame size.
+Allows to override the size of output video frames.
+
+@item alpha
+Process alpha channel, by default alpha channel is untouched.
+
+@item fonts_dir
+Set a directory path containing fonts that can be used by the filter.
+These fonts will be used in addition to whatever the font provider uses.
+
+@item force_style
+Override default style or script info parameters of the subtitles. It accepts a
+string containing ASS style format @code{KEY=VALUE} couples separated by ",".
+
+@end table
@c man end SUBTITLE FILTERS
@chapter Multimedia Filters
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 01c3926bf8..584a97c9a3 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -366,6 +366,7 @@ OBJS-$(CONFIG_OVERLAY_OPENCL_FILTER) +=
vf_overlay_opencl.o opencl.o \
opencl/overlay.o framesync.o
OBJS-$(CONFIG_OVERLAY_QSV_FILTER)+= vf_overlay_qsv.o framesync.o
OBJS-$(CONFIG_OVERLAY_GRAPHICSUBS_FILTER)+= vf_overlay_graphicsubs.o
framesync.o
+OBJS-$(CONFIG_OVERLAY_TEXTSUBS_FILTER) += vf_overlay_textsubs.o
OBJS-$(CONFIG_OVERLAY_VULKAN_FILTER) += vf_overlay_vulkan.o vulkan.o
OBJS-$(CONFIG_OWDENOISE_FILTER) += vf_owdenoise.o
OBJS-$(CONFIG_PAD_FILTER)+= vf_pad.o
@@ -454,6 +455,7 @@ OBJS-$(CONFIG_SWAPRECT_FILTER) +=
vf_swaprect.o
OBJS-$(CONFIG_SWAPUV_FILTER) += vf_swapuv.o
OBJS-$(CONFIG_TBLEND_FILTER)
[FFmpeg-devel] [PATCH v10 08/14] avfilter/textmod: Add textmod, censor and show_speaker filters
- textmod {S -> S)
Modify subtitle text in a number of ways
- censor {S -> S)
Censor subtitles using a word list
- show_speaker {S -> S)
Prepend speaker names from ASS subtitles to the visible text lines
Signed-off-by: softworkz
---
doc/filters.texi | 229 +
libavfilter/Makefile | 5 +
libavfilter/allfilters.c | 3 +
libavfilter/sf_textmod.c | 719 +++
4 files changed, 956 insertions(+)
create mode 100644 libavfilter/sf_textmod.c
diff --git a/doc/filters.texi b/doc/filters.texi
index 2f81df6489..c751211017 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25195,6 +25195,7 @@ tools.
@c man end VIDEO SINKS
+
@chapter Subtitle Filters
@c man begin SUBTITLE FILTERS
@@ -25203,6 +25204,166 @@ existing filters using @code{--disable-filters}.
Below is a description of the currently available subtitle filters.
+
+@section censor
+
+Censor selected words in text subtitles.
+
+Inputs:
+- 0: Subtitles [text]
+
+Outputs:
+- 0: Subtitles [text]
+
+It accepts the following parameters:
+
+@table @option
+@item mode
+The censoring mode to apply.
+
+Supported censoring modes are:
+
+@table @var
+@item 0, keep_first_last
+Replace all characters with the 'censor_char' except the first and the last
character of a word.
+For words with less than 4 characters, the last character will be replaced as
well.
+For words with less than 3 characters, the first character will be replaced as
well.
+@item 1, keep_first
+Replace all characters with the 'censor_char' except the first character of a
word.
+For words with less than 3 characters, the first character will be replaced as
well.
+@item 2, all
+Replace all characters with the 'censor_char'.
+@end table
+
+@item words
+A list of words to censor, separated by 'separator'.
+
+@item words_file
+Specify a file from which to load the contents for the 'words' parameter.
+
+@item censor_char
+Single character used as replacement for censoring.
+
+@item separator
+Delimiter character for words. Used with replace_words and remove_words- Must
be a single character.
+The default is '.'.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Change all characters to upper case while keeping all styles and animations:
+@example
+ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv";
-filter_complex "[0:s]textmod=mode=to_upper" -map 0 -y out.mkv
+@end example
+@item
+Remove a set of symbol characters for am improved and smoother visual
apperance:
+@example
+ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv";
-filter_complex "[0:s]textmod=mode=remove_chars:find='$&#@*§'" -map 0 -y
out.mkv
+@end example
+@end itemize
+
+
+@section stripstyles
+
+Remove all inline styles from subtitle events.
+
+It accepts the following parameters:
+
+@table @option
+@item remove_animated
+Also remove text which is subject to animation (default: true)
+Usually, animated text elements are used used in addition to static subtitle
lines for creating effects, so in most cases it is safe to remove the animation
content.
+If subtitle text is missing, try setting this to false.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Remove all styles and animations from subtitles:
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv";
-filter_complex "[0:1]stripstyles" -map 0 output.mkv
+@end example
+@end itemize
+
+@section textmod
+
+Modify subtitle text in a number of ways.
+
+Inputs:
+- 0: Subtitles [text]
+
+Outputs:
+- 0: Subtitles [text]
+
+It accepts the following parameters:
+
+@table @option
+@item mode
+The kind of text modification to apply
+
+Supported operation modes are:
+
+@table @var
+@item 0, leet
+Convert subtitle text to 'leet speak'. It's primarily useful for testing as
the modification will be visible with almost all text lines.
+@item 1, to_upper
+Change all text to upper case. Might improve readability.
+@item 2, to_lower
+Change all text to lower case.
+@item 3, replace_chars
+Replace one or more characters. Requires the find and replace parameters to be
specified.
+Both need to be equal in length.
+The first char in find is replaced by the first char in replace, same for all
subsequent chars.
+@item 4, remove_chars
+Remove certain characters. Requires the find parameter to be specified.
+All chars in the find parameter string will be removed from all subtitle text.
+@item 5, replace_words
+Replace one or more words. Requires the find and replace parameters to be
specified. Multiple words must be separated by the delimiter char specified vie
the separator parameter (default: ',').
+The number of words in the find and replace parameters needs to be equal.
+The first word in find is replaced by the first word in replace, same for all
subsequent words
+@item 6, remove_words
+Remove certain words. Requires the find parameter to be specified. Multiple
words must be separated by the delimiter char specified v
[FFmpeg-devel] [PATCH v10 09/14] avfilter/stripstyles: Add stripstyles filter
- stripstyles {S -> S)
Remove all inline styles from subtitle events
Signed-off-by: softworkz
---
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/sf_stripstyles.c | 211 +++
3 files changed, 213 insertions(+)
create mode 100644 libavfilter/sf_stripstyles.c
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index df250ed07b..c991a525f0 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -541,6 +541,7 @@ OBJS-$(CONFIG_NULLSINK_FILTER) +=
vsink_nullsink.o
OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o
OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o
OBJS-$(CONFIG_TEXTMOD_FILTER)+= sf_textmod.o
+OBJS-$(CONFIG_STRIPSTYLES_FILTER)+= sf_stripstyles.o
# multimedia filters
OBJS-$(CONFIG_ABITSCOPE_FILTER) += avf_abitscope.o
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 232eac4ef6..bdf4c2ae52 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -529,6 +529,7 @@ extern const AVFilter ff_avf_showwavespic;
extern const AVFilter ff_vaf_spectrumsynth;
extern const AVFilter ff_sf_censor;
extern const AVFilter ff_sf_show_speaker;
+extern const AVFilter ff_sf_stripstyles;
extern const AVFilter ff_sf_textmod;
extern const AVFilter ff_svf_graphicsub2video;
extern const AVFilter ff_svf_textsub2video;
diff --git a/libavfilter/sf_stripstyles.c b/libavfilter/sf_stripstyles.c
new file mode 100644
index 00..bdde28577f
--- /dev/null
+++ b/libavfilter/sf_stripstyles.c
@@ -0,0 +1,211 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/**
+ * @file
+ * text subtitle filter which removes inline-styles from subtitles
+ */
+
+#include "libavutil/opt.h"
+#include "internal.h"
+#include "libavutil/ass_split_internal.h"
+#include "libavutil/bprint.h"
+
+typedef struct StripStylesContext {
+const AVClass *class;
+enum AVSubtitleType format;
+int remove_animated;
+} StripStylesContext;
+
+typedef struct DialogContext {
+StripStylesContext* ss_ctx;
+AVBPrint buffer;
+int drawing_scale;
+int is_animated;
+} DialogContext;
+
+static void dialog_text_cb(void *priv, const char *text, int len)
+{
+DialogContext *s = priv;
+
+av_log(s->ss_ctx, AV_LOG_DEBUG, "dialog_text_cb: %s\n", text);
+
+if (!s->drawing_scale && (!s->is_animated || !s->ss_ctx->remove_animated))
+av_bprint_append_data(&s->buffer, text, len);
+}
+
+static void dialog_new_line_cb(void *priv, int forced)
+{
+DialogContext *s = priv;
+if (!s->drawing_scale && !s->is_animated)
+av_bprint_append_data(&s->buffer, forced ? "\\N" : "\\n", 2);
+}
+
+static void dialog_drawing_mode_cb(void *priv, int scale)
+{
+DialogContext *s = priv;
+s->drawing_scale = scale;
+}
+
+static void dialog_animate_cb(void *priv, int t1, int t2, int accel, char
*style)
+{
+DialogContext *s = priv;
+s->is_animated = 1;
+}
+
+static void dialog_move_cb(void *priv, int x1, int y1, int x2, int y2, int t1,
int t2)
+{
+DialogContext *s = priv;
+if (t1 >= 0 || t2 >= 0)
+s->is_animated = 1;
+}
+
+static const ASSCodesCallbacks dialog_callbacks = {
+.text = dialog_text_cb,
+.new_line = dialog_new_line_cb,
+.drawing_mode = dialog_drawing_mode_cb,
+.animate = dialog_animate_cb,
+.move = dialog_move_cb,
+};
+
+static int query_formats(AVFilterContext *ctx)
+{
+AVFilterFormats *formats;
+AVFilterLink *inlink = ctx->inputs[0];
+AVFilterLink *outlink = ctx->outputs[0];
+static const enum AVSubtitleType subtitle_fmts[] = { AV_SUBTITLE_FMT_ASS,
AV_SUBTITLE_FMT_NONE };
+int ret;
+
+/* set input subtitle format */
+formats = ff_make_format_list(subtitle_fmts);
+if ((ret = ff_formats_ref(formats, &inlink->outcfg.formats)) < 0)
+return ret;
+
+/* set output video format */
+if ((ret = ff_formats_ref(formats, &outlink->incfg.formats)) < 0)
+return ret;
+
+return 0;
+}
+
+static char *ass_get_line(int readorder, int layer, const char *style,
+const char *speaker, const char *effect, const char
*t
[FFmpeg-devel] [PATCH v10 10/14] avfilter/split_cc: Add split_cc filter for closed caption handling
- split_cc {V -> VS)
Extract closed-caption (A53) data from video
frames as subtitle Frames
ffmpeg -y -loglevel verbose -i
"https://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts";
-filter_complex
"[0:v]split_cc[vid1],textmod=mode=remove_chars:find='@',[vid1]overlay_textsubs"
output.mkv
Signed-off-by: softworkz
---
configure | 1 +
doc/filters.texi | 45 ++
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/sf_split_cc.c | 298 ++
5 files changed, 346 insertions(+)
create mode 100644 libavfilter/sf_split_cc.c
diff --git a/configure b/configure
index 37fc4c20e7..6da059e6ee 100755
--- a/configure
+++ b/configure
@@ -3663,6 +3663,7 @@ spp_filter_select="fft idctdsp fdctdsp me_cmp pixblockdsp"
sr_filter_deps="avformat swscale"
sr_filter_select="dnn"
stereo3d_filter_deps="gpl"
+split_cc_filter_deps="cc_dec_decoder"
subtitles_filter_deps="avformat avcodec libass"
super2xsai_filter_deps="gpl"
pixfmts_super2xsai_test_deps="super2xsai_filter"
diff --git a/doc/filters.texi b/doc/filters.texi
index c751211017..001d2baa8f 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25494,6 +25494,16 @@ string containing ASS style format @code{KEY=VALUE}
couples separated by ",".
@end table
+@subsection Examples
+
+@itemize
+@item
+Overlay ASS subtitles with animations:
+@example
+ffmpeg -i
"http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv";
-filter_complex "[0:v]overlay_textsubs" -map 0 -y out.mkv
+@end example
+@end itemize
+
@section show_speaker
Prepend speaker names to subtitle lines (when available).
@@ -25557,6 +25567,41 @@ ffmpeg -i INPUT -filter_complex
"show_speaker=format=colon:style='@{\\c&HDD&
@end example
@end itemize
+
+@section split_cc
+
+Split-out closed-caption/A53 subtitles from video frame side data.
+
+This filter provides an input and an output for video frames, which are just
passed through without modification.
+The second out provides subtitle frames which are extracted from video frame
side data.
+
+Inputs:
+- 0: Video
+
+Outputs:
+- 0: Video (same as input)
+- 1: Subtitles [text]
+
+It accepts the following parameters:
+
+@table @option
+
+@item use_cc_styles
+Emit closed caption style header. This will make closed captions look like on
normal TV devices.
+(white font on black background rectangles)
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Extract closed captions as text subtitle stream and overlay it onto the video:
+@example
+ffmpeg -i "https://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts";
-filter_complex "[0:v:0]split_cc[vid1][sub1];[vid1][sub1]overlay_textsubs"
output.mkv
+@end example
+@end itemize
+
@section textsub2video
Converts text subtitles to video frames.
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index c991a525f0..b260e7e0a2 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -541,6 +541,7 @@ OBJS-$(CONFIG_NULLSINK_FILTER) +=
vsink_nullsink.o
OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o
OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o
OBJS-$(CONFIG_TEXTMOD_FILTER)+= sf_textmod.o
+OBJS-$(CONFIG_SPLIT_CC_FILTER) += sf_split_cc.o
OBJS-$(CONFIG_STRIPSTYLES_FILTER)+= sf_stripstyles.o
# multimedia filters
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index bdf4c2ae52..40bb3c229e 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -529,6 +529,7 @@ extern const AVFilter ff_avf_showwavespic;
extern const AVFilter ff_vaf_spectrumsynth;
extern const AVFilter ff_sf_censor;
extern const AVFilter ff_sf_show_speaker;
+extern const AVFilter ff_sf_split_cc;
extern const AVFilter ff_sf_stripstyles;
extern const AVFilter ff_sf_textmod;
extern const AVFilter ff_svf_graphicsub2video;
diff --git a/libavfilter/sf_split_cc.c b/libavfilter/sf_split_cc.c
new file mode 100644
index 00..d96912a90b
--- /dev/null
+++ b/libavfilter/sf_split_cc.c
@@ -0,0 +1,298 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/**
+ * @file
+ * subtitle filter for splitting out closed-caption/A
[FFmpeg-devel] [PATCH v10 11/14] avfilter/graphicsub2text: Add new graphicsub2text filter (OCR)
Signed-off-by: softworkz --- configure| 1 + doc/filters.texi | 55 ++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/sf_graphicsub2text.c | 323 +++ 5 files changed, 381 insertions(+) create mode 100644 libavfilter/sf_graphicsub2text.c diff --git a/configure b/configure index 6da059e6ee..9108742122 100755 --- a/configure +++ b/configure @@ -3601,6 +3601,7 @@ frei0r_deps_any="libdl LoadLibrary" frei0r_filter_deps="frei0r" frei0r_src_filter_deps="frei0r" fspp_filter_deps="gpl" +graphicsub2text_filter_deps="libtesseract" histeq_filter_deps="gpl" hqdn3d_filter_deps="gpl" interlace_filter_deps="gpl" diff --git a/doc/filters.texi b/doc/filters.texi index 001d2baa8f..e709fcbc03 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -25364,6 +25364,61 @@ ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv"; -filter_comple @end example @end itemize +@section graphicsub2text + +Converts graphic subtitles to text subtitles by performing OCR. + +For this filter to be available, ffmpeg needs to be compiled with libtesseract (see https://github.com/tesseract-ocr/tesseract). +Language models need to be downloaded from https://github.com/tesseract-ocr/tessdata and put into as subfolder named 'tessdata' or into a folder specified via the environment variable 'TESSDATA_PREFIX'. +The path can also be specified via filter option (see below). + +Note: These models are including the data for both OCR modes. + +Inputs: +- 0: Subtitles [bitmap] + +Outputs: +- 0: Subtitles [text] + +It accepts the following parameters: + +@table @option +@item ocr_mode +The character recognition mode to use. + +Supported OCR modes are: + +@table @var +@item 0, tesseract +This is the classic libtesseract operation mode. It is fast but less accurate than LSTM. +@item 1, lstm +Newer OCR implementation based on ML models. Provides usually better results, requires more processing resources. +@item 2, both +Use a combination of both modes. +@end table + +@item tessdata_path +The path to a folder containing the language models to be used. + +@item language +The recognition language. It needs to match the first three characters of a language model file in the tessdata path. + +@end table + + +@subsection Examples + +@itemize +@item +Convert DVB graphic subtitles to ASS (text) subtitles + +Note: For this to work, you need to have the data file 'eng.traineddata' in a 'tessdata' subfolder (see above). +@example +ffmpeg ffmpeg -loglevel verbose -i "https://streams.videolan.org/streams/ts/video_subs_ttxt%2Bdvbsub.ts"; -filter_complex "[0:13]graphicsub2text=ocr_mode=both" -c:s ass -y output.mkv +@end example +@end itemize + + @section graphicsub2video Renders graphic subtitles as video frames. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index b260e7e0a2..fc12664c56 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -290,6 +290,7 @@ OBJS-$(CONFIG_FSPP_FILTER) += vf_fspp.o qp_table.o OBJS-$(CONFIG_GBLUR_FILTER) += vf_gblur.o OBJS-$(CONFIG_GEQ_FILTER)+= vf_geq.o OBJS-$(CONFIG_GRADFUN_FILTER)+= vf_gradfun.o +OBJS-$(CONFIG_GRAPHICSUB2TEXT_FILTER)+= sf_graphicsub2text.o OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlay_graphicsubs.o framesync.o OBJS-$(CONFIG_GRAPHMONITOR_FILTER) += f_graphmonitor.o OBJS-$(CONFIG_GRAYWORLD_FILTER) += vf_grayworld.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 40bb3c229e..72ab388518 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -528,6 +528,7 @@ extern const AVFilter ff_avf_showwaves; extern const AVFilter ff_avf_showwavespic; extern const AVFilter ff_vaf_spectrumsynth; extern const AVFilter ff_sf_censor; +extern const AVFilter ff_sf_graphicsub2text; extern const AVFilter ff_sf_show_speaker; extern const AVFilter ff_sf_split_cc; extern const AVFilter ff_sf_stripstyles; diff --git a/libavfilter/sf_graphicsub2text.c b/libavfilter/sf_graphicsub2text.c new file mode 100644 index 00..6c5c3d14ce --- /dev/null +++ b/libavfilter/sf_graphicsub2text.c @@ -0,0 +1,323 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the
[FFmpeg-devel] [PATCH v10 12/14] avfilter/subscale: Add filter for scaling and/or re-arranging graphical subtitles
Signed-off-by: softworkz
---
configure | 1 +
doc/filters.texi | 160 +++
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/sf_subscale.c | 867 ++
5 files changed, 1030 insertions(+)
create mode 100644 libavfilter/sf_subscale.c
diff --git a/configure b/configure
index 9108742122..93b26da269 100755
--- a/configure
+++ b/configure
@@ -3665,6 +3665,7 @@ sr_filter_deps="avformat swscale"
sr_filter_select="dnn"
stereo3d_filter_deps="gpl"
split_cc_filter_deps="cc_dec_decoder"
+subscale_filter_deps="swscale avcodec"
subtitles_filter_deps="avformat avcodec libass"
super2xsai_filter_deps="gpl"
pixfmts_super2xsai_test_deps="super2xsai_filter"
diff --git a/doc/filters.texi b/doc/filters.texi
index e709fcbc03..9cc6b05835 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25695,6 +25695,166 @@ Override default style or script info parameters of
the subtitles. It accepts a
string containing ASS style format @code{KEY=VALUE} couples separated by ",".
@end table
+
+@section subscale
+
+Provides high-quality scaling and rearranging functionality for graphical
subtitles.
+
+The subscale filter provides multiple approaches for manipulating
+the size and position of graphical subtitle rectangles wich can
+be combined or used separately.
+Scaling is performed by converting the palettized subtitle bitmaps
+to RGBA and re-quantization to palette colors afterwards via elbg algorithm.
+
+The two major operations are 'scale' and 're-arrange' with the
+latter being separated as 'arrange_h' and 'arrange_v'.
+
+
+Inputs:
+- 0: Subtitles [bitmap]
+
+Outputs:
+- 0: Subtitles [bitmap]
+
+It accepts the following parameters:
+
+@table @option
+
+@item w, width
+Set the width of the output.
+Width and height in case of graphical subtitles are just indicating
+a virtual size for which the output (consisting of 0-n bitmap rectangles)
+is intended to be displayed on.
+
+@item h, height
+Set the height of the output.
+
+@item margin_h
+Sets a horizontal margin to be preserverved when using any
+of the arrange modes.
+
+@item margin_v
+Sets a vertical margin to be preserverved when using any
+of the arrange modes.
+
+@item force_original_aspect_ratio
+Enable decreasing or increasing output video width or height if necessary to
+keep the original aspect ratio. Possible values:
+
+@table @samp
+@item disable
+Scale the video as specified and disable this feature.
+
+@item decrease
+The output video dimensions will automatically be decreased if needed.
+
+@item increase
+The output video dimensions will automatically be increased if needed.
+
+@end table
+
+
+@item scale_mode
+Specifies how subtitle bitmaps should be scaled.
+The scale factor is determined by the the factor between input
+and output size.
+
+@table @samp
+@item none
+Do not apply any common scaling.
+
+@item uniform
+Uniformly scale all subtitle bitmaps including their positions.
+
+@item uniform_no_reposition
+Uniformly scale all subtitle bitmaps without changing positions.
+
+@end table
+
+
+@item arrange_h
+Specifies how subtitle bitmaps should be arranged horizontally.
+
+@item arrange_v
+Specifies how subtitle bitmaps should be arranged vertically.
+
+
+@table @samp
+@item none
+Do not rearrange subtitle bitmaps.
+
+@item margin_no_scale
+Move subtitle bitmaps to be positioned inside the specified
+margin (margin_h or margin_v) when possible and without scaling.
+
+@item margin_and_scale
+Move subtitle bitmaps to be positioned inside the specified
+margin (margin_h or margin_v) and scale in case it doesn't fit.
+
+@item snapalign_no_scale
+Categorize subtitle bitmap positions as one of left/center/right
+or top/bottom/middle based on original positioning and apply
+these alignments for the target positioning.
+No scaling will be applied.
+
+@item snapalign_and_scale
+Categorize subtitle bitmap positions as one of left/center/right
+or top/bottom/middle based on original positioning and apply
+these alignments for the target positioning.
+Bitmaps that do not fit inside the margins borders are
+scaled to fit.
+@end table
+
+@item eval
+Set evaluation mode for the expressions (@option{width}, @option{height}).
+
+It accepts the following values:
+@table @samp
+@item init
+Evaluate expressions only once during the filter initialization.
+
+@item frame
+Evaluate expressions for each incoming frame. This is way slower than the
+@samp{init} mode since it requires all the scalers to be re-computed, but it
+allows advanced dynamic expressions.
+@end table
+
+Default value is @samp{init}.
+
+
+@item num_colors
+Set the number of palette colors for output images.
+Choose the maximum (256) when further processing is done (e.g.
+overlaying on a video).
+When subtitles will be encoded as bitmap subtitles (e.g. dvbsub),
+a smaller number of palette colors (e.g. 4-16) might need to be used, depending
+on the target format and codec.
+
+@item bitmap_width_align
[FFmpeg-devel] [PATCH v10 13/14] avformat/utils: Add av_stream_get_codec_properties()
Signed-off-by: softworkz
---
doc/APIchanges | 3 +++
libavformat/avformat.h | 9 +
libavformat/utils.c| 5 +
libavformat/version.h | 2 +-
4 files changed, 18 insertions(+), 1 deletion(-)
diff --git a/doc/APIchanges b/doc/APIchanges
index 7b267a79ac..20848fe93f 100644
--- a/doc/APIchanges
+++ b/doc/APIchanges
@@ -14,6 +14,9 @@ libavutil: 2021-04-27
API changes, most recent first:
+2021-04-27 - xx - lavf 59.6.100 - avformat.h
+ Add av_stream_get_codec_properties()
+
2021-09-21 - xx - lavu 57.7.100 - pixfmt.h
Add AV_PIX_FMT_X2BGR10.
diff --git a/libavformat/avformat.h b/libavformat/avformat.h
index 3a5bc8a06d..89ed984d1d 100644
--- a/libavformat/avformat.h
+++ b/libavformat/avformat.h
@@ -2763,6 +2763,15 @@ int avformat_transfer_internal_stream_timing_info(const
AVOutputFormat *ofmt,
*/
AVRational av_stream_get_codec_timebase(const AVStream *st);
+/**
+ * Get the internal codec properties from a stream.
+ *
+ * See @ref AVCodecContext.properties.
+ *
+ * @param st input stream to extract the timebase from
+ */
+unsigned av_stream_get_codec_properties(const AVStream *st);
+
/**
* @}
*/
diff --git a/libavformat/utils.c b/libavformat/utils.c
index f1ec4c4c2f..64b90e785d 100644
--- a/libavformat/utils.c
+++ b/libavformat/utils.c
@@ -1945,6 +1945,11 @@ AVRational av_stream_get_codec_timebase(const AVStream
*st)
return cffstream(st)->avctx->time_base;
}
+unsigned av_stream_get_codec_properties(const AVStream *st)
+{
+return cffstream(st)->avctx->properties;
+}
+
void ff_format_set_url(AVFormatContext *s, char *url)
{
av_assert0(url);
diff --git a/libavformat/version.h b/libavformat/version.h
index 13df244d97..d5dd22059b 100644
--- a/libavformat/version.h
+++ b/libavformat/version.h
@@ -32,7 +32,7 @@
// Major bumping may affect Ticket5467, 5421, 5451(compatibility with Chromium)
// Also please add any ticket numbers that you believe might be affected here
#define LIBAVFORMAT_VERSION_MAJOR 59
-#define LIBAVFORMAT_VERSION_MINOR 5
+#define LIBAVFORMAT_VERSION_MINOR 6
#define LIBAVFORMAT_VERSION_MICRO 100
#define LIBAVFORMAT_VERSION_INT AV_VERSION_INT(LIBAVFORMAT_VERSION_MAJOR, \
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v10 14/14] ffprobe: Fix incorrect display of closed_captions property
Repro Example:
ffprobe -show_entries stream=closed_captions:disposition=:side_data=
"http://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts";
While the codec string includes "Closed Captions",
the stream data is showing: closed_captions=0
The test ref was incorrect as the test media file
actually does have cc.
Signed-off-by: softworkz
---
fftools/ffprobe.c | 5 +++--
tests/ref/fate/ts-demux | 2 +-
2 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/fftools/ffprobe.c b/fftools/ffprobe.c
index 4e08807706..201f02bee1 100644
--- a/fftools/ffprobe.c
+++ b/fftools/ffprobe.c
@@ -2669,10 +2669,11 @@ static int show_stream(WriterContext *w,
AVFormatContext *fmt_ctx, int stream_id
print_int("width",par->width);
print_int("height", par->height);
if (dec_ctx) {
+unsigned codec_properties =
av_stream_get_codec_properties(ist->st);
print_int("coded_width", dec_ctx->coded_width);
print_int("coded_height", dec_ctx->coded_height);
-print_int("closed_captions", !!(dec_ctx->properties &
FF_CODEC_PROPERTY_CLOSED_CAPTIONS));
-print_int("film_grain", !!(dec_ctx->properties &
FF_CODEC_PROPERTY_FILM_GRAIN));
+print_int("closed_captions", !!(codec_properties &
FF_CODEC_PROPERTY_CLOSED_CAPTIONS));
+print_int("film_grain", !!(codec_properties &
FF_CODEC_PROPERTY_FILM_GRAIN));
}
print_int("has_b_frames", par->video_delay);
sar = av_guess_sample_aspect_ratio(fmt_ctx, stream, NULL);
diff --git a/tests/ref/fate/ts-demux b/tests/ref/fate/ts-demux
index 8e7a81da41..1d1382cf37 100644
--- a/tests/ref/fate/ts-demux
+++ b/tests/ref/fate/ts-demux
@@ -41,7 +41,7 @@
packet|codec_type=audio|stream_index=2|pts=3912642700|pts_time=43473.807778|dts=
packet|codec_type=video|stream_index=0|pts=3912686363|pts_time=43474.292922|dts=3912686363|dts_time=43474.292922|duration=1501|duration_time=0.016678|size=4944|pos=506660|flags=__|data_hash=CRC32:54a86cbb
packet|codec_type=audio|stream_index=1|pts=3912644825|pts_time=43473.831389|dts=3912644825|dts_time=43473.831389|duration=2880|duration_time=0.032000|size=906|pos=474888|flags=K_|data_hash=CRC32:0893d398
packet|codec_type=audio|stream_index=2|pts=3912645580|pts_time=43473.839778|dts=3912645580|dts_time=43473.839778|duration=2880|duration_time=0.032000|size=354|pos=491808|flags=K_|data_hash=CRC32:f5963fa6
-stream|index=0|codec_name=mpeg2video|profile=4|codec_type=video|codec_tag_string=[2][0][0][0]|codec_tag=0x0002|width=1280|height=720|coded_width=0|coded_height=0|closed_captions=0|film_grain=0|has_b_frames=1|sample_aspect_ratio=1:1|display_aspect_ratio=16:9|pix_fmt=yuv420p|level=4|color_range=tv|color_space=unknown|color_transfer=unknown|color_primaries=unknown|chroma_location=left|field_order=progressive|refs=1|id=0x31|r_frame_rate=6/1001|avg_frame_rate=6/1001|time_base=1/9|start_pts=3912669846|start_time=43474.109400|duration_ts=19519|duration=0.216878|bit_rate=1500|max_bit_rate=N/A|bits_per_raw_sample=N/A|nb_frames=N/A|nb_read_frames=N/A|nb_read_packets=15|extradata_hash=CRC32:53134fa8|disposition:default=0|disposition:dub=0|disposition:original=0|disposition:comment=0|disposition:lyrics=0|disposition:karaoke=0|disposition:forced=0|disposition:hearing_impaired=0|disposition:visual_impaired=0|disposition:clean_effects=0|disposition:attached_pic=0|disposition:timed_
thumbnails=0|disposition:captions=0|disposition:descriptions=0|disposition:metadata=0|disposition:dependent=0|disposition:still_image=0
+stream|index=0|codec_name=mpeg2video|profile=4|codec_type=video|codec_tag_string=[2][0][0][0]|codec_tag=0x0002|width=1280|height=720|coded_width=0|coded_height=0|closed_captions=1|film_grain=0|has_b_frames=1|sample_aspect_ratio=1:1|display_aspect_ratio=16:9|pix_fmt=yuv420p|level=4|color_range=tv|color_space=unknown|color_transfer=unknown|color_primaries=unknown|chroma_location=left|field_order=progressive|refs=1|id=0x31|r_frame_rate=6/1001|avg_frame_rate=6/1001|time_base=1/9|start_pts=3912669846|start_time=43474.109400|duration_ts=19519|duration=0.216878|bit_rate=1500|max_bit_rate=N/A|bits_per_raw_sample=N/A|nb_frames=N/A|nb_read_frames=N/A|nb_read_packets=15|extradata_hash=CRC32:53134fa8|disposition:default=0|disposition:dub=0|disposition:original=0|disposition:comment=0|disposition:lyrics=0|disposition:karaoke=0|disposition:forced=0|disposition:hearing_impaired=0|disposition:visual_impaired=0|disposition:clean_effects=0|disposition:attached_pic=0|disposition:timed_
thumbnails=0|disposition:captions=0|disposition:descriptions=0|disposition:metadata=0|disposition:dependent=0|disposition:still_image=0
side_data|side_data_type=CPB
properties|max_bitrate=1500|min_bitrate=0|avg_bitrate=0|buffer_size=9781248|vbv_delay=-1
stream|index=1|codec_name=ac3|profile=unknown|codec_type=audio|codec_tag_string=[4][0][0][0]|codec_tag=0x0004|sample_fmt=fltp|sample_rate=48000|channe
Re: [FFmpeg-devel] [PATCH v10 00/14] Subtitle Filtering
I am very SORRY! This v10 patchset version has gotten all wrong.
Please ignore!
softworkz
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Soft Works
> Sent: Thursday, 30 September 2021 21:30
> To: ffmpeg-devel@ffmpeg.org
> Subject: [FFmpeg-devel] [PATCH v10 00/14] Subtitle Filtering
>
> v10 Update:
>
> - FEATURE-COMPLETE!
> No more features or filters will be added to this
> patchset
> - Added subscale filter
> Provides high-quality scaling and rearranging
> functionality for graphical subtitles.
> This has been the last missing bit for making this
> a useful and versatile toolset for subtitle manipulation
> - The common ass utility functions remain to reside
> in avutil, but in this version, they are marked as
> avpriv_ and no longer public like in the previous
> version.
> - A number of detail fixes are included, but there
> haven't been significant changes to the core
> implementation in the past few versions.
> I'm quite confident about the current state.
> - As a final step I'll be going to create a wiki page
> presenting a comprehensive collection of usage examples,
> showing off the whole range of new capabilities
> and also providing proof that it is working universally
> in a wide range of use cases.
>
>
> v9 Update:
>
> - Addressed all review comments (thanks Andreas!)
> - Move ass utility functions to avutil as public API
> - Handle subtitle decoding via new decode API
> (removed av_decode_subtitle3)
> - Change 'subtitle_header' to use refcounted buffers
> - Allow mapping subtitle streams to filter graph
> and direct outputs simultaneously
> - Added explanation and justification for test changes
> to commit message
>
> --
>
> This patchset introduces filtering support for subtitles.
>
> Besides the ground work, this patchset includes a range
> of new filters for subtitle processing:
>
> - subscale (S -> S)
> Provides high-quality scaling and rearranging
> functionality for graphical subtitles.
>
> - graphicsubs2text (S -> S)
> Convert graphic subtitles to text subtitles via OCR
>
> - censor {S -> S)
> Allows censoring word in subtitles by a configurable
> word list
>
> - show_speaker {S -> S)
> Prepend the speaker names to text lines
> (those are often available in ass subs)
>
> - split_cc (V -> VS)
> Splits out Closed-Caption data as a subtitle stream
> for filtering and encoding
>
> - overlay_graphicsubs (VS -> V)
> Overlay graphic subtitles onto a video stream
>
> - graphicsub2video {S -> V)
> Converts graphic subtitles to (transparent) video frames
>
> - overlay_textsubs {VS -> V)
> Overlay text subtitles onto a video stream.
>
> - textsubs2video {S -> V)
> Converts text subtitles to video frames
>
> - textmod {S -> S)
> Modify subtitle text in a number of ways
>
> - stripstyles {S -> S)
> Remove all inline styles from subtitle events
>
> Regards,
> softworkz
>
>
> softworkz (14):
> fftools/play,probe: Adjust for subtitle changes
> avfilter/subtitles: Add subtitles.c for subtitle frame allocation
> avfilter/avfilter: Handle subtitle frames
> avfilter/sbuffer: Add sbuffersrc and sbuffersink filters
> avfilter/overlay_graphicsubs: Add overlay_graphicsubs and
> graphicsub2video filters
> fftools/ffmpeg: Replace sub2video with subtitle frame filtering
> avfilter/overlay_textsubs: Add overlay_textsubs and textsubs2video
> filters
> avfilter/textmod: Add textmod, censor and show_speaker filters
> avfilter/stripstyles: Add stripstyles filter
> avfilter/split_cc: Add split_cc filter for closed caption handling
> avfilter/graphicsub2text: Add new graphicsub2text filter (OCR)
> avfilter/subscale: Add filter for scaling and/or re-arranging
> graphical subtitles
> avformat/utils: Add av_stream_get_codec_properties()
> ffprobe: Fix incorrect display of closed_captions property
>
> configure | 7 +-
> doc/APIchanges| 3 +
> doc/filters.texi | 662 +
> fftools/ffmpeg.c | 507 ++---
> fftools/ffmpeg.h | 14 +-
> fftools/ffmpeg_filter.c | 209 --
> fftools/ffmpeg_hw.c | 2 +-
> fftools/ffmpeg_opt.c | 3 +-
> fftools/ffplay.c | 102 ++-
> fftools/ffprobe.c | 53 +-
> libavfilter/Makefile
[FFmpeg-devel] [PATCH v11 00/14] Subtitle Filtering
v11 Update:
- FEATURE-COMPLETE!
No more features or filters will be added to this
patchset
- Added subscale filter
Provides high-quality scaling and rearranging
functionality for graphical subtitles.
This has been the last missing bit for making this
a useful and versatile toolset for subtitle manipulation
- The common ass utility functions remain to reside
in avutil, but in this version, they are marked as
avpriv_ and no longer public like in the previous
version.
- A number of detail fixes are included, but there
haven't been significant changes to the core
implementation in the past few versions.
I'm quite confident about the current state.
- As a final step I'll be going to create a wiki page
presenting a comprehensive collection of usage examples,
showing off the whole range of new capabilities
and also providing proof that it is working universally
in a wide range of use cases.
v10 Update: INVALID
- Incorrect patch sequence sent.
v9 Update:
- Addressed all review comments (thanks Andreas!)
- Move ass utility functions to avutil as public API
- Handle subtitle decoding via new decode API
(removed av_decode_subtitle3)
- Change 'subtitle_header' to use refcounted buffers
- Allow mapping subtitle streams to filter graph
and direct outputs simultaneously
- Added explanation and justification for test changes
to commit message
--
This patchset introduces filtering support for subtitles.
Besides the ground work, this patchset includes a range
of new filters for subtitle processing:
- subscale (S -> S)
Provides high-quality scaling and rearranging
functionality for graphical subtitles.
- graphicsubs2text (S -> S)
Convert graphic subtitles to text subtitles via OCR
- censor {S -> S)
Allows censoring word in subtitles by a configurable
word list
- show_speaker {S -> S)
Prepend the speaker names to text lines
(those are often available in ass subs)
- split_cc (V -> VS)
Splits out Closed-Caption data as a subtitle stream
for filtering and encoding
- overlay_graphicsubs (VS -> V)
Overlay graphic subtitles onto a video stream
- graphicsub2video {S -> V)
Converts graphic subtitles to (transparent) video frames
- overlay_textsubs {VS -> V)
Overlay text subtitles onto a video stream.
- textsubs2video {S -> V)
Converts text subtitles to video frames
- textmod {S -> S)
Modify subtitle text in a number of ways
- stripstyles {S -> S)
Remove all inline styles from subtitle events
Regards,
softworkz
softworkz (14):
global: Prepare AVFrame for subtitle handling
global: Move ass helper functions to avutil as avpriv_ and extend ass
dialog parsing
fftools/play,probe: Adjust for subtitle changes
avfilter/subtitles: Add subtitles.c for subtitle frame allocation
avfilter/avfilter: Handle subtitle frames
avfilter/sbuffer: Add sbuffersrc and sbuffersink filters
avfilter/overlay_graphicsubs: Add overlay_graphicsubs and
graphicsub2video filters
fftools/ffmpeg: Replace sub2video with subtitle frame filtering
avfilter/overlay_textsubs: Add overlay_textsubs and textsubs2video
filters
avfilter/textmod: Add textmod, censor and show_speaker filters
avfilter/stripstyles: Add stripstyles filter
avfilter/split_cc: Add split_cc filter for closed caption handling
avfilter/graphicsub2text: Add new graphicsub2text filter (OCR)
avfilter/subscale: Add filter for scaling and/or re-arranging
graphical subtitles
configure | 7 +-
doc/filters.texi | 662 +
fftools/ffmpeg.c | 507 +-
fftools/ffmpeg.h | 14 +-
fftools/ffmpeg_filter.c | 209 +++--
fftools/ffmpeg_hw.c | 2 +-
fftools/ffmpeg_opt.c | 3 +-
fftools/ffplay.c | 102 +--
fftools/ffprobe.c | 48 +-
libavcodec/Makefile | 56 +-
libavcodec/ass.h | 129 +--
libavcodec/assdec.c | 2 +-
libavcodec/assenc.c | 2 +-
libavcodec/avcodec.c | 19 -
libavcodec/avcodec.h | 75 +-
libavcodec/ccaption_dec.c | 19 +-
libavcodec/decode.c | 53 +-
libavcodec/jacosubdec.c | 2 +-
libavcodec/microdvddec.c | 7 +-
libavcodec/movtextdec.c | 3 +-
libavcodec/movtextenc.c | 20 +-
libavcodec/mpl2dec.c | 2 +-
libavcodec/pgssubdec.c| 1 +
libavcodec/realtextdec.c | 2 +-
libavcodec/samidec.c | 2 +-
libavcodec/srtdec.c
[FFmpeg-devel] [PATCH v11 01/14] global: Prepare AVFrame for subtitle handling
Root commit for adding subtitle filtering capabilities.
In detail:
- Add type (AVMediaType) field to AVFrame
Replaces previous way of distinction which was based on checking
width and height to determine whether a frame is audio or video
- Add subtitle fields to AVFrame
- Add new struct AVSubtitleArea, similar to AVSubtitleRect, but different
allocation logic. Cannot and must not be used interchangeably, hence
the new struct
- Move enum AVSubtitleType, AVSubtitle and AVSubtitleRect to avutil
- Add public-named members to enum AVSubtitleType (AV_SUBTITLE_FMT_)
- Add avcodec_decode_subtitle3 which takes subtitle frames,
serving as compatibility shim to legacy subtitle decoding
- Add additional methods for conversion between old and new API
Signed-off-by: softworkz
---
libavcodec/avcodec.c | 19 ---
libavcodec/avcodec.h | 75 ++--
libavcodec/decode.c| 53 ++--
libavcodec/pgssubdec.c | 1 +
libavcodec/utils.c | 11 ++
libavfilter/vf_subtitles.c | 50 ++--
libavformat/utils.c| 1 +
libavutil/Makefile | 2 +
libavutil/frame.c | 191 +
libavutil/frame.h | 93 +-
libavutil/subfmt.c | 240 +
libavutil/subfmt.h | 185
12 files changed, 796 insertions(+), 125 deletions(-)
create mode 100644 libavutil/subfmt.c
create mode 100644 libavutil/subfmt.h
diff --git a/libavcodec/avcodec.c b/libavcodec/avcodec.c
index ff3d73e237..52ce345d49 100644
--- a/libavcodec/avcodec.c
+++ b/libavcodec/avcodec.c
@@ -422,25 +422,6 @@ void avcodec_flush_buffers(AVCodecContext *avctx)
av_bsf_flush(avci->bsf);
}
-void avsubtitle_free(AVSubtitle *sub)
-{
-int i;
-
-for (i = 0; i < sub->num_rects; i++) {
-av_freep(&sub->rects[i]->data[0]);
-av_freep(&sub->rects[i]->data[1]);
-av_freep(&sub->rects[i]->data[2]);
-av_freep(&sub->rects[i]->data[3]);
-av_freep(&sub->rects[i]->text);
-av_freep(&sub->rects[i]->ass);
-av_freep(&sub->rects[i]);
-}
-
-av_freep(&sub->rects);
-
-memset(sub, 0, sizeof(*sub));
-}
-
av_cold int avcodec_close(AVCodecContext *avctx)
{
int i;
diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 7ee8bc2b7c..0c5819b116 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -35,6 +35,7 @@
#include "libavutil/frame.h"
#include "libavutil/log.h"
#include "libavutil/pixfmt.h"
+#include "libavutil/subfmt.h"
#include "libavutil/rational.h"
#include "codec.h"
@@ -1674,7 +1675,7 @@ typedef struct AVCodecContext {
/**
* Header containing style information for text subtitles.
- * For SUBTITLE_ASS subtitle type, it should contain the whole ASS
+ * For AV_SUBTITLE_FMT_ASS subtitle type, it should contain the whole ASS
* [Script Info] and [V4+ Styles] section, plus the [Events] line and
* the Format line following. It shouldn't include any Dialogue line.
* - encoding: Set/allocated/freed by user (before avcodec_open2())
@@ -2238,63 +2239,8 @@ typedef struct AVHWAccel {
* @}
*/
-enum AVSubtitleType {
-SUBTITLE_NONE,
-
-SUBTITLE_BITMAP,///< A bitmap, pict will be set
-
-/**
- * Plain text, the text field must be set by the decoder and is
- * authoritative. ass and pict fields may contain approximations.
- */
-SUBTITLE_TEXT,
-
-/**
- * Formatted text, the ass field must be set by the decoder and is
- * authoritative. pict and text fields may contain approximations.
- */
-SUBTITLE_ASS,
-};
-
#define AV_SUBTITLE_FLAG_FORCED 0x0001
-typedef struct AVSubtitleRect {
-int x; ///< top left corner of pict, undefined when pict is not
set
-int y; ///< top left corner of pict, undefined when pict is not
set
-int w; ///< widthof pict, undefined when pict is not
set
-int h; ///< height of pict, undefined when pict is not
set
-int nb_colors; ///< number of colors in pict, undefined when pict is not
set
-
-/**
- * data+linesize for the bitmap of this subtitle.
- * Can be set for text/ass as well once they are rendered.
- */
-uint8_t *data[4];
-int linesize[4];
-
-enum AVSubtitleType type;
-
-char *text; ///< 0 terminated plain UTF-8 text
-
-/**
- * 0 terminated ASS/SSA compatible event line.
- * The presentation of this is unaffected by the other values in this
- * struct.
- */
-char *ass;
-
-int flags;
-} AVSubtitleRect;
-
-typedef struct AVSubtitle {
-uint16_t format; /* 0 = graphics */
-uint32_t start_display_time; /* relative to packet pts, in ms */
-uint32_t end_display_time; /* relative to packet pts, in ms */
-unsigned num_rects;
-AVSubtitleRect **rects;
-int64_t pts;///< Same as packet pts, in AV_TIME_BASE
-
[FFmpeg-devel] [PATCH v11 02/14] global: Move ass helper functions to avutil as avpriv_ and extend ass dialog parsing
Signed-off-by: softworkz
---
libavcodec/Makefile | 56 +++
libavcodec/ass.h | 129
libavcodec/assdec.c | 2 +-
libavcodec/assenc.c | 2 +-
libavcodec/ccaption_dec.c | 19 +--
libavcodec/jacosubdec.c | 2 +-
libavcodec/microdvddec.c | 7 +-
libavcodec/movtextdec.c | 3 +-
libavcodec/movtextenc.c | 20 +--
libavcodec/mpl2dec.c | 2 +-
libavcodec/realtextdec.c | 2 +-
libavcodec/samidec.c | 2 +-
libavcodec/srtdec.c | 2 +-
libavcodec/srtenc.c | 16 +-
libavcodec/subviewerdec.c | 2 +-
libavcodec/textdec.c | 4 +-
libavcodec/ttmlenc.c | 15 +-
libavcodec/webvttdec.c| 2 +-
libavcodec/webvttenc.c| 16 +-
libavutil/Makefile| 4 +
{libavcodec => libavutil}/ass.c | 73 +
libavutil/ass_internal.h | 139 ++
{libavcodec => libavutil}/ass_split.c | 30 ++--
.../ass_split_internal.h | 24 +--
24 files changed, 325 insertions(+), 248 deletions(-)
rename {libavcodec => libavutil}/ass.c (73%)
create mode 100644 libavutil/ass_internal.h
rename {libavcodec => libavutil}/ass_split.c (94%)
rename libavcodec/ass_split.h => libavutil/ass_split_internal.h (89%)
diff --git a/libavcodec/Makefile b/libavcodec/Makefile
index 8012b69f61..3d4602d4d4 100644
--- a/libavcodec/Makefile
+++ b/libavcodec/Makefile
@@ -209,10 +209,10 @@ OBJS-$(CONFIG_APNG_DECODER)+= png.o pngdec.o
pngdsp.o
OBJS-$(CONFIG_APNG_ENCODER)+= png.o pngenc.o
OBJS-$(CONFIG_ARBC_DECODER)+= arbc.o
OBJS-$(CONFIG_ARGO_DECODER)+= argo.o
-OBJS-$(CONFIG_SSA_DECODER) += assdec.o ass.o
-OBJS-$(CONFIG_SSA_ENCODER) += assenc.o ass.o
-OBJS-$(CONFIG_ASS_DECODER) += assdec.o ass.o
-OBJS-$(CONFIG_ASS_ENCODER) += assenc.o ass.o
+OBJS-$(CONFIG_SSA_DECODER) += assdec.o
+OBJS-$(CONFIG_SSA_ENCODER) += assenc.o
+OBJS-$(CONFIG_ASS_DECODER) += assdec.o
+OBJS-$(CONFIG_ASS_ENCODER) += assenc.o
OBJS-$(CONFIG_ASV1_DECODER)+= asvdec.o asv.o mpeg12data.o
OBJS-$(CONFIG_ASV1_ENCODER)+= asvenc.o asv.o mpeg12data.o
OBJS-$(CONFIG_ASV2_DECODER)+= asvdec.o asv.o mpeg12data.o
@@ -252,7 +252,7 @@ OBJS-$(CONFIG_BRENDER_PIX_DECODER) += brenderpix.o
OBJS-$(CONFIG_C93_DECODER) += c93.o
OBJS-$(CONFIG_CAVS_DECODER)+= cavs.o cavsdec.o cavsdsp.o \
cavsdata.o
-OBJS-$(CONFIG_CCAPTION_DECODER)+= ccaption_dec.o ass.o
+OBJS-$(CONFIG_CCAPTION_DECODER)+= ccaption_dec.o
OBJS-$(CONFIG_CDGRAPHICS_DECODER) += cdgraphics.o
OBJS-$(CONFIG_CDTOONS_DECODER) += cdtoons.o
OBJS-$(CONFIG_CDXL_DECODER)+= cdxl.o
@@ -423,7 +423,7 @@ OBJS-$(CONFIG_INTERPLAY_ACM_DECODER) += interplayacm.o
OBJS-$(CONFIG_INTERPLAY_DPCM_DECODER) += dpcm.o
OBJS-$(CONFIG_INTERPLAY_VIDEO_DECODER) += interplayvideo.o
OBJS-$(CONFIG_IPU_DECODER) += mpeg12dec.o mpeg12.o mpeg12data.o
-OBJS-$(CONFIG_JACOSUB_DECODER) += jacosubdec.o ass.o
+OBJS-$(CONFIG_JACOSUB_DECODER) += jacosubdec.o
OBJS-$(CONFIG_JPEG2000_ENCODER)+= j2kenc.o mqcenc.o mqc.o jpeg2000.o \
jpeg2000dwt.o
OBJS-$(CONFIG_JPEG2000_DECODER)+= jpeg2000dec.o jpeg2000.o
jpeg2000dsp.o \
@@ -445,7 +445,7 @@ OBJS-$(CONFIG_MAGICYUV_ENCODER)+= magicyuvenc.o
OBJS-$(CONFIG_MDEC_DECODER)+= mdec.o mpeg12.o mpeg12data.o
OBJS-$(CONFIG_METASOUND_DECODER) += metasound.o metasound_data.o \
twinvq.o
-OBJS-$(CONFIG_MICRODVD_DECODER)+= microdvddec.o ass.o
+OBJS-$(CONFIG_MICRODVD_DECODER)+= microdvddec.o
OBJS-$(CONFIG_MIMIC_DECODER) += mimic.o
OBJS-$(CONFIG_MJPEG_DECODER) += mjpegdec.o mjpegdec_common.o
OBJS-$(CONFIG_MJPEG_QSV_DECODER) += qsvdec.o
@@ -460,8 +460,8 @@ OBJS-$(CONFIG_MLP_ENCODER) += mlpenc.o mlp.o
OBJS-$(CONFIG_MMVIDEO_DECODER) += mmvideo.o
OBJS-$(CONFIG_MOBICLIP_DECODER)+= mobiclip.o
OBJS-$(CONFIG_MOTIONPIXELS_DECODER)+= motionpixels.o
-OBJS-$(CONFIG_MOVTEXT_DECODER) += movtextdec.o ass.o
-OBJS-$(CONFIG_MOVTEXT_ENCODER) += movtextenc.o ass_split.o
+OBJS-$(CONFIG_MOVTEXT_DECODER) += movtextdec.o
+OBJS-$(CONFIG_MOVTEXT_ENCODER) += movtextenc.o
OBJS-$(CONFIG_MP1_DECODER) += mpegaudiodec_fixed.o
OBJS-$(CONFIG
[FFmpeg-devel] [PATCH v11 03/14] fftools/play, probe: Adjust for subtitle changes
Signed-off-by: softworkz
---
fftools/ffplay.c | 102 +-
fftools/ffprobe.c | 48 ++
2 files changed, 79 insertions(+), 71 deletions(-)
diff --git a/fftools/ffplay.c b/fftools/ffplay.c
index ccea0e4578..e8d36485ae 100644
--- a/fftools/ffplay.c
+++ b/fftools/ffplay.c
@@ -152,7 +152,6 @@ typedef struct Clock {
/* Common struct for handling all types of decoded data and allocated render
buffers. */
typedef struct Frame {
AVFrame *frame;
-AVSubtitle sub;
int serial;
double pts; /* presentation timestamp for the frame */
double duration; /* estimated duration of the frame */
@@ -586,7 +585,7 @@ static int decoder_init(Decoder *d, AVCodecContext *avctx,
PacketQueue *queue, S
return 0;
}
-static int decoder_decode_frame(Decoder *d, AVFrame *frame, AVSubtitle *sub) {
+static int decoder_decode_frame(Decoder *d, AVFrame *frame) {
int ret = AVERROR(EAGAIN);
for (;;) {
@@ -620,6 +619,9 @@ static int decoder_decode_frame(Decoder *d, AVFrame *frame,
AVSubtitle *sub) {
}
}
break;
+case AVMEDIA_TYPE_SUBTITLE:
+ret = avcodec_receive_frame(d->avctx, frame);
+break;
}
if (ret == AVERROR_EOF) {
d->finished = d->pkt_serial;
@@ -652,25 +654,11 @@ static int decoder_decode_frame(Decoder *d, AVFrame
*frame, AVSubtitle *sub) {
av_packet_unref(d->pkt);
} while (1);
-if (d->avctx->codec_type == AVMEDIA_TYPE_SUBTITLE) {
-int got_frame = 0;
-ret = avcodec_decode_subtitle2(d->avctx, sub, &got_frame, d->pkt);
-if (ret < 0) {
-ret = AVERROR(EAGAIN);
-} else {
-if (got_frame && !d->pkt->data) {
-d->packet_pending = 1;
-}
-ret = got_frame ? 0 : (d->pkt->data ? AVERROR(EAGAIN) :
AVERROR_EOF);
-}
-av_packet_unref(d->pkt);
+if (avcodec_send_packet(d->avctx, d->pkt) == AVERROR(EAGAIN)) {
+av_log(d->avctx, AV_LOG_ERROR, "Receive_frame and send_packet both
returned EAGAIN, which is an API violation.\n");
+d->packet_pending = 1;
} else {
-if (avcodec_send_packet(d->avctx, d->pkt) == AVERROR(EAGAIN)) {
-av_log(d->avctx, AV_LOG_ERROR, "Receive_frame and send_packet
both returned EAGAIN, which is an API violation.\n");
-d->packet_pending = 1;
-} else {
-av_packet_unref(d->pkt);
-}
+av_packet_unref(d->pkt);
}
}
}
@@ -683,7 +671,6 @@ static void decoder_destroy(Decoder *d) {
static void frame_queue_unref_item(Frame *vp)
{
av_frame_unref(vp->frame);
-avsubtitle_free(&vp->sub);
}
static int frame_queue_init(FrameQueue *f, PacketQueue *pktq, int max_size,
int keep_last)
@@ -981,7 +968,7 @@ static void video_image_display(VideoState *is)
if (frame_queue_nb_remaining(&is->subpq) > 0) {
sp = frame_queue_peek(&is->subpq);
-if (vp->pts >= sp->pts + ((float) sp->sub.start_display_time /
1000)) {
+if (vp->pts >= sp->pts + ((float) sp->frame->subtitle_start_time /
1000)) {
if (!sp->uploaded) {
uint8_t* pixels[4];
int pitch[4];
@@ -993,25 +980,27 @@ static void video_image_display(VideoState *is)
if (realloc_texture(&is->sub_texture,
SDL_PIXELFORMAT_ARGB, sp->width, sp->height, SDL_BLENDMODE_BLEND, 1) < 0)
return;
-for (i = 0; i < sp->sub.num_rects; i++) {
-AVSubtitleRect *sub_rect = sp->sub.rects[i];
+for (i = 0; i < sp->frame->num_subtitle_areas; i++) {
+AVSubtitleArea *area = sp->frame->subtitle_areas[i];
+SDL_Rect sdl_rect = { .x = area->x, .y = area->y, .w =
area->w, .h = area->h };
-sub_rect->x = av_clip(sub_rect->x, 0, sp->width );
-sub_rect->y = av_clip(sub_rect->y, 0, sp->height);
-sub_rect->w = av_clip(sub_rect->w, 0, sp->width -
sub_rect->x);
-sub_rect->h = av_clip(sub_rect->h, 0, sp->height -
sub_rect->y);
+area->x = av_clip(area->x, 0, sp->width );
+area->y = av_clip(area->y, 0, sp->height);
+area->w = av_clip(area->w, 0, sp->width - area->x);
+area->h = av_clip(area->h, 0, sp->height - area->y);
is->sub_convert_ctx =
sws_getCachedContext(is->sub_convert_ctx,
-sub_rect->w, sub_rect->h, AV_PIX_FMT_PAL8,
-sub_rect->w, sub_rec
[FFmpeg-devel] [PATCH v11 04/14] avfilter/subtitles: Add subtitles.c for subtitle frame allocation
Analog to avfilter/video.c and avfilter/audio.c
Signed-off-by: softworkz
---
libavfilter/Makefile| 1 +
libavfilter/avfilter.c | 4 +++
libavfilter/internal.h | 1 +
libavfilter/subtitles.c | 63 +
libavfilter/subtitles.h | 44
5 files changed, 113 insertions(+)
create mode 100644 libavfilter/subtitles.c
create mode 100644 libavfilter/subtitles.h
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index fa2d18e45f..357926624e 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -19,6 +19,7 @@ OBJS = allfilters.o
\
framequeue.o \
graphdump.o \
graphparser.o\
+ subtitles.o \
video.o \
OBJS-$(HAVE_THREADS) += pthread.o
diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
index f325918021..ef349a67f9 100644
--- a/libavfilter/avfilter.c
+++ b/libavfilter/avfilter.c
@@ -43,6 +43,7 @@
#include "formats.h"
#include "framepool.h"
#include "internal.h"
+#include "subtitles.h"
#include "libavutil/ffversion.h"
const char av_filter_ffversion[] = "FFmpeg version " FFMPEG_VERSION;
@@ -1476,6 +1477,9 @@ int ff_inlink_make_frame_writable(AVFilterLink *link,
AVFrame **rframe)
case AVMEDIA_TYPE_AUDIO:
out = ff_get_audio_buffer(link, frame->nb_samples);
break;
+case AVMEDIA_TYPE_SUBTITLE:
+out = ff_get_subtitles_buffer(link, link->format);
+break;
default:
return AVERROR(EINVAL);
}
diff --git a/libavfilter/internal.h b/libavfilter/internal.h
index e7c154aff0..8977dda2b3 100644
--- a/libavfilter/internal.h
+++ b/libavfilter/internal.h
@@ -90,6 +90,7 @@ struct AVFilterPad {
union {
AVFrame *(*video)(AVFilterLink *link, int w, int h);
AVFrame *(*audio)(AVFilterLink *link, int nb_samples);
+AVFrame *(*subtitle)(AVFilterLink *link, int format);
} get_buffer;
/**
diff --git a/libavfilter/subtitles.c b/libavfilter/subtitles.c
new file mode 100644
index 00..951bfd612c
--- /dev/null
+++ b/libavfilter/subtitles.c
@@ -0,0 +1,63 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "libavutil/common.h"
+
+#include "subtitles.h"
+#include "avfilter.h"
+#include "internal.h"
+
+
+AVFrame *ff_null_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+return ff_get_subtitles_buffer(link->dst->outputs[0], format);
+}
+
+AVFrame *ff_default_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+AVFrame *frame;
+
+frame = av_frame_alloc();
+if (!frame)
+return NULL;
+
+frame->format = format;
+frame->type = AVMEDIA_TYPE_SUBTITLE;
+
+if (av_frame_get_buffer2(frame, 0) < 0) {
+av_frame_free(&frame);
+return NULL;
+}
+
+return frame;
+}
+
+AVFrame *ff_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+AVFrame *ret = NULL;
+
+if (link->dstpad->get_buffer.subtitle)
+ret = link->dstpad->get_buffer.subtitle(link, format);
+
+if (!ret)
+ret = ff_default_get_subtitles_buffer(link, format);
+
+return ret;
+}
diff --git a/libavfilter/subtitles.h b/libavfilter/subtitles.h
new file mode 100644
index 00..4a9115126e
--- /dev/null
+++ b/libavfilter/subtitles.h
@@ -0,0 +1,44 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have recei
[FFmpeg-devel] [PATCH v11 05/14] avfilter/avfilter: Handle subtitle frames
Signed-off-by: softworkz
---
libavfilter/avfilter.c | 8 +---
libavfilter/avfiltergraph.c | 5 +
libavfilter/formats.c | 14 ++
libavfilter/formats.h | 3 +++
4 files changed, 27 insertions(+), 3 deletions(-)
diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
index ef349a67f9..c567b7893b 100644
--- a/libavfilter/avfilter.c
+++ b/libavfilter/avfilter.c
@@ -57,7 +57,8 @@ void ff_tlog_ref(void *ctx, AVFrame *ref, int end)
ref->linesize[0], ref->linesize[1], ref->linesize[2],
ref->linesize[3],
ref->pts, ref->pkt_pos);
-if (ref->width) {
+switch(ref->type) {
+case AVMEDIA_TYPE_VIDEO:
ff_tlog(ctx, " a:%d/%d s:%dx%d i:%c iskey:%d type:%c",
ref->sample_aspect_ratio.num, ref->sample_aspect_ratio.den,
ref->width, ref->height,
@@ -65,12 +66,13 @@ void ff_tlog_ref(void *ctx, AVFrame *ref, int end)
ref->top_field_first ? 'T' : 'B',/* Top / Bottom */
ref->key_frame,
av_get_picture_type_char(ref->pict_type));
-}
-if (ref->nb_samples) {
+break;
+case AVMEDIA_TYPE_AUDIO:
ff_tlog(ctx, " cl:%"PRId64"d n:%d r:%d",
ref->channel_layout,
ref->nb_samples,
ref->sample_rate);
+break;
}
ff_tlog(ctx, "]%s", end ? "\n" : "");
diff --git a/libavfilter/avfiltergraph.c b/libavfilter/avfiltergraph.c
index 45b028cd9c..7a5a4ea419 100644
--- a/libavfilter/avfiltergraph.c
+++ b/libavfilter/avfiltergraph.c
@@ -314,6 +314,8 @@ static int filter_link_check_formats(void *log,
AVFilterLink *link, AVFilterForm
return ret;
break;
+case AVMEDIA_TYPE_SUBTITLE:
+return 0;
default:
av_assert0(!"reached");
}
@@ -444,6 +446,9 @@ static int query_formats(AVFilterGraph *graph, void
*log_ctx)
if (!link)
continue;
+if (link->type == AVMEDIA_TYPE_SUBTITLE)
+continue;
+
neg = ff_filter_get_negotiation(link);
av_assert0(neg);
for (neg_step = 1; neg_step < neg->nb_mergers; neg_step++) {
diff --git a/libavfilter/formats.c b/libavfilter/formats.c
index 1d2a51c0af..94c1891b78 100644
--- a/libavfilter/formats.c
+++ b/libavfilter/formats.c
@@ -19,6 +19,7 @@
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
+#include "libavcodec/avcodec.h"
#include "libavutil/avassert.h"
#include "libavutil/channel_layout.h"
#include "libavutil/common.h"
@@ -430,6 +431,12 @@ int ff_add_channel_layout(AVFilterChannelLayouts **l,
uint64_t channel_layout)
return 0;
}
+int ff_add_subtitle_type(AVFilterFormats **avff, int64_t fmt)
+{
+ADD_FORMAT(avff, fmt, ff_formats_unref, int, formats, nb_formats);
+return 0;
+}
+
AVFilterFormats *ff_all_formats(enum AVMediaType type)
{
AVFilterFormats *ret = NULL;
@@ -443,6 +450,13 @@ AVFilterFormats *ff_all_formats(enum AVMediaType type)
return NULL;
fmt++;
}
+} else if (type == AVMEDIA_TYPE_SUBTITLE) {
+if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_BITMAP) < 0)
+return NULL;
+if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_ASS) < 0)
+return NULL;
+if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_TEXT) < 0)
+return NULL;
}
return ret;
diff --git a/libavfilter/formats.h b/libavfilter/formats.h
index 42fe068765..7581fae01c 100644
--- a/libavfilter/formats.h
+++ b/libavfilter/formats.h
@@ -180,6 +180,9 @@ int ff_set_common_formats_from_list(AVFilterContext *ctx,
const int *fmts);
av_warn_unused_result
int ff_add_channel_layout(AVFilterChannelLayouts **l, uint64_t channel_layout);
+av_warn_unused_result
+int ff_add_subtitle_type(AVFilterFormats **avff, int64_t fmt);
+
/**
* Add *ref as a new reference to f.
*/
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v11 06/14] avfilter/sbuffer: Add sbuffersrc and sbuffersink filters
Signed-off-by: softworkz
---
configure| 2 +-
libavfilter/allfilters.c | 2 ++
libavfilter/buffersink.c | 63 +++
libavfilter/buffersink.h | 15 +
libavfilter/buffersrc.c | 72
libavfilter/buffersrc.h | 1 +
6 files changed, 154 insertions(+), 1 deletion(-)
diff --git a/configure b/configure
index 231d0398a8..d18bb73ed6 100755
--- a/configure
+++ b/configure
@@ -7720,7 +7720,7 @@ print_enabled_components(){
fi
done
if [ "$name" = "filter_list" ]; then
-for c in asrc_abuffer vsrc_buffer asink_abuffer vsink_buffer; do
+for c in asrc_abuffer vsrc_buffer ssrc_sbuffer asink_abuffer
vsink_buffer ssink_sbuffer; do
printf "&ff_%s,\n" $c >> $TMPH
done
fi
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 338561a446..51a1d09b47 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -536,8 +536,10 @@ extern const AVFilter ff_avsrc_movie;
* being the same while having different 'types'). */
extern const AVFilter ff_asrc_abuffer;
extern const AVFilter ff_vsrc_buffer;
+extern const AVFilter ff_ssrc_sbuffer;
extern const AVFilter ff_asink_abuffer;
extern const AVFilter ff_vsink_buffer;
+extern const AVFilter ff_ssink_sbuffer;
extern const AVFilter ff_af_afifo;
extern const AVFilter ff_vf_fifo;
diff --git a/libavfilter/buffersink.c b/libavfilter/buffersink.c
index 8b46dcb15e..cd038e1782 100644
--- a/libavfilter/buffersink.c
+++ b/libavfilter/buffersink.c
@@ -29,6 +29,8 @@
#include "libavutil/internal.h"
#include "libavutil/opt.h"
+#include "libavcodec/avcodec.h"
+
#define FF_INTERNAL_FIELDS 1
#include "framequeue.h"
@@ -57,6 +59,10 @@ typedef struct BufferSinkContext {
int *sample_rates; ///< list of accepted sample
rates, terminated by -1
int sample_rates_size;
+/* only used for subtitles */
+enum AVSubtitleType *subtitle_types; ///< list of accepted subtitle
types, must be terminated with -1
+int subtitle_types_size;
+
AVFrame *peeked_frame;
} BufferSinkContext;
@@ -168,6 +174,15 @@ AVABufferSinkParams *av_abuffersink_params_alloc(void)
return NULL;
return params;
}
+
+AVSBufferSinkParams *av_sbuffersink_params_alloc(void)
+{
+AVSBufferSinkParams *params = av_mallocz(sizeof(AVSBufferSinkParams));
+
+if (!params)
+return NULL;
+return params;
+}
#endif
static av_cold int common_init(AVFilterContext *ctx)
@@ -305,6 +320,28 @@ static int asink_query_formats(AVFilterContext *ctx)
return 0;
}
+static int ssink_query_formats(AVFilterContext *ctx)
+{
+BufferSinkContext *buf = ctx->priv;
+AVFilterFormats *formats = NULL;
+unsigned i;
+int ret;
+
+CHECK_LIST_SIZE(subtitle_types)
+if (buf->subtitle_types_size) {
+for (i = 0; i < NB_ITEMS(buf->subtitle_types); i++)
+if ((ret = ff_add_subtitle_type(&formats, buf->subtitle_types[i]))
< 0)
+return ret;
+if ((ret = ff_set_common_formats(ctx, formats)) < 0)
+return ret;
+} else {
+if ((ret = ff_default_query_formats(ctx)) < 0)
+return ret;
+}
+
+return 0;
+}
+
#define OFFSET(x) offsetof(BufferSinkContext, x)
#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM
static const AVOption buffersink_options[] = {
@@ -322,9 +359,16 @@ static const AVOption abuffersink_options[] = {
{ NULL },
};
#undef FLAGS
+#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_SUBTITLE_PARAM
+static const AVOption sbuffersink_options[] = {
+{ "subtitle_types", "set the supported subtitle formats",
OFFSET(subtitle_types), AV_OPT_TYPE_BINARY, .flags = FLAGS },
+{ NULL },
+};
+#undef FLAGS
AVFILTER_DEFINE_CLASS(buffersink);
AVFILTER_DEFINE_CLASS(abuffersink);
+AVFILTER_DEFINE_CLASS(sbuffersink);
static const AVFilterPad avfilter_vsink_buffer_inputs[] = {
{
@@ -363,3 +407,22 @@ const AVFilter ff_asink_abuffer = {
FILTER_INPUTS(avfilter_asink_abuffer_inputs),
.outputs = NULL,
};
+
+static const AVFilterPad avfilter_ssink_sbuffer_inputs[] = {
+{
+.name = "default",
+.type = AVMEDIA_TYPE_SUBTITLE,
+},
+};
+
+const AVFilter ff_ssink_sbuffer = {
+.name = "sbuffersink",
+.description = NULL_IF_CONFIG_SMALL("Buffer subtitle frames, and make
them available to the end of the filter graph."),
+.priv_class= &sbuffersink_class,
+.priv_size = sizeof(BufferSinkContext),
+.init = common_init,
+.query_formats = ssink_query_formats,
+.activate = activate,
+FILTER_INPUTS(avfilter_ssink_sbuffer_inputs),
+.outputs = NULL,
+};
diff --git a/libavfilter/buffersink.h b/libavfilter/buffersink.h
index 69ed0f29a8..b439b586c5 100644
--- a/libavfilter/buffersink.h
+++ b/libavfilter/buffersink.h
@@ -129,6 +129,21 @@ typedef struct AVABufferS
[FFmpeg-devel] [PATCH v11 07/14] avfilter/overlay_graphicsubs: Add overlay_graphicsubs and graphicsub2video filters
- overlay_graphicsubs (VS -> V)
Overlay graphic subtitles onto a video stream
- graphicsub2video {S -> V)
Converts graphic subtitles to video frames (with alpha)
Gets auto-inserted for retaining compatibility with
sub2video command lines
Signed-off-by: softworkz
---
doc/filters.texi | 104
libavfilter/Makefile | 2 +
libavfilter/allfilters.c | 2 +
libavfilter/vf_overlay_graphicsubs.c | 724 +++
4 files changed, 832 insertions(+)
create mode 100644 libavfilter/vf_overlay_graphicsubs.c
diff --git a/doc/filters.texi b/doc/filters.texi
index 6328a8c563..18b5273e6e 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25195,6 +25195,110 @@ tools.
@c man end VIDEO SINKS
+@chapter Subtitle Filters
+@c man begin SUBTITLE FILTERS
+
+When you configure your FFmpeg build, you can disable any of the
+existing filters using @code{--disable-filters}.
+
+Below is a description of the currently available subtitle filters.
+
+@section graphicsub2video
+
+Renders graphic subtitles as video frames.
+
+This filter replaces the previous "sub2video" hack which did the conversion
implicitly and up-front as subtitle filtering wasn't possible at that time.
+To retain compatibility with earlier sub2video command lines, this filter is
being auto-inserted in those cases.
+
+For overlaying graphicsal subtitles it is recommended to use the
'overlay_graphicsubs' filter which is more efficient and takes less processing
resources.
+
+This filter is still useful in cases where the overlay is done with hardware
acceleration (e.g. overlay_qsv, overlay_vaapi, overlay_cuda) for preparing the
overlay frames.
+
+It accepts the following parameters:
+
+@table @option
+@item size, s
+Set the size of the output video frame.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Overlay PGS subtitles
+(not recommended - better use overlay_graphicsubs)
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:1]graphicsub2video[subs];[0:0][subs]overlay" output.mp4
+@end example
+
+@item
+Overlay PGS subtitles implicitly
+The graphicsub2video is inserted automatically for compatibility with legacy
command lines.
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:0][0:1]overlay" output.mp4
+@end example
+@end itemize
+
+@section overlay_graphicsubs
+
+Overlay graphic subtitles onto a video stream.
+
+This filter can blend graphical subtitles on a video stream directly, i.e.
without creating full-size alpha images first.
+The blending operation is limited to the area of the subtitle rectangles,
which also means that no processing is done at times where no subtitles are to
be displayed.
+
+
+It accepts the following parameters:
+
+@table @option
+@item x
+@item y
+Set the expression for the x and y coordinates of the overlaid video
+on the main video. Default value is "0" for both expressions. In case
+the expression is invalid, it is set to a huge value (meaning that the
+overlay will not be displayed within the output visible area).
+
+@item eof_action
+See @ref{framesync}.
+
+@item eval
+Set when the expressions for @option{x}, and @option{y} are evaluated.
+
+It accepts the following values:
+@table @samp
+@item init
+only evaluate expressions once during the filter initialization or
+when a command is processed
+
+@item frame
+evaluate expressions for each incoming frame
+@end table
+
+Default value is @samp{frame}.
+
+@item shortest
+See @ref{framesync}.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Overlay PGS subtitles
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:1]graphicsub2video[subs];[0:0][subs]overlay" output.mp4
+@end example
+
+@item
+Overlay PGS subtitles implicitly
+The graphicsub2video is inserted automatically for compatibility with legacy
command lines.
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:0][0:1]overlay" output.mp4
+@end example
+@end itemize
+@c man end SUBTITLE FILTERS
+
@chapter Multimedia Filters
@c man begin MULTIMEDIA FILTERS
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 357926624e..01c3926bf8 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -290,6 +290,7 @@ OBJS-$(CONFIG_FSPP_FILTER) += vf_fspp.o
qp_table.o
OBJS-$(CONFIG_GBLUR_FILTER) += vf_gblur.o
OBJS-$(CONFIG_GEQ_FILTER)+= vf_geq.o
OBJS-$(CONFIG_GRADFUN_FILTER)+= vf_gradfun.o
+OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlay_graphicsubs.o
framesync.o
OBJS-$(CONFIG_GRAPHMONITOR_FILTER) += f_graphmonitor.o
OBJS-$(CONFIG_GRAYWORLD_FILTER) += vf_grayworld.o
[FFmpeg-devel] [PATCH v11 08/14] fftools/ffmpeg: Replace sub2video with subtitle frame filtering
This commit actually enables subtitle filtering in ffmpeg by
sending and receiving subtitle frames to and from a filtergraph.
The heartbeat functionality from the previous sub2video implementation
is retained and applied to all subtitle frames (bitmap, text, ..).
The other part of sub2video functionality is retained by
auto-insertion of the new graphicsub2video filter.
Justification for changed test refs:
- sub2video
The new results are identical excepting the last frame which
is due to the implementation changes
- sub2video_basic
The previous results had some incorrect output because multiple
frames had the same dts (even the first one was incorrect).
The non-empty content frames are visually identical, the different
CRC is due to the different blending algorithm that is being used.
- sub2video_time_limited
The previous results were incorrect. There was an initial empty frame
with the same dts generated. This is no longer happening.
The third frame was a repetition, which doesn't happen anymore with
the new subtitle filtering.
The source file contains a single subtitle event and the new and
correct result is a single output frame
- sub-dvb
Running ffprobe -show_frames on the source file shows that there
are 7 subtitle frames with 0 rects in the source at the start
and 2 at the end. This translates to the 14 and 4 additional
entries in the new test results.
- filter-overlay-dvdsub-2397
Overlay results have slightly different CRCs due to different
blending implementation
Signed-off-by: softworkz
---
fftools/ffmpeg.c | 507 ++
fftools/ffmpeg.h | 14 +-
fftools/ffmpeg_filter.c | 209 ++---
fftools/ffmpeg_hw.c | 2 +-
fftools/ffmpeg_opt.c | 3 +-
tests/ref/fate/filter-overlay-dvdsub-2397 | 181
tests/ref/fate/sub-dvb| 162 ---
tests/ref/fate/sub2video | 1 -
tests/ref/fate/sub2video_basic| 93 ++--
tests/ref/fate/sub2video_time_limited | 4 +-
10 files changed, 622 insertions(+), 554 deletions(-)
diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c
index 98c2421938..6e7f621d61 100644
--- a/fftools/ffmpeg.c
+++ b/fftools/ffmpeg.c
@@ -169,163 +169,6 @@ static int restore_tty;
static void free_input_threads(void);
#endif
-/* sub2video hack:
- Convert subtitles to video with alpha to insert them in filter graphs.
- This is a temporary solution until libavfilter gets real subtitles support.
- */
-
-static int sub2video_get_blank_frame(InputStream *ist)
-{
-int ret;
-AVFrame *frame = ist->sub2video.frame;
-
-av_frame_unref(frame);
-ist->sub2video.frame->width = ist->dec_ctx->width ? ist->dec_ctx->width
: ist->sub2video.w;
-ist->sub2video.frame->height = ist->dec_ctx->height ? ist->dec_ctx->height
: ist->sub2video.h;
-ist->sub2video.frame->format = AV_PIX_FMT_RGB32;
-if ((ret = av_frame_get_buffer(frame, 0)) < 0)
-return ret;
-memset(frame->data[0], 0, frame->height * frame->linesize[0]);
-return 0;
-}
-
-static void sub2video_copy_rect(uint8_t *dst, int dst_linesize, int w, int h,
-AVSubtitleRect *r)
-{
-uint32_t *pal, *dst2;
-uint8_t *src, *src2;
-int x, y;
-
-if (r->type != SUBTITLE_BITMAP) {
-av_log(NULL, AV_LOG_WARNING, "sub2video: non-bitmap subtitle\n");
-return;
-}
-if (r->x < 0 || r->x + r->w > w || r->y < 0 || r->y + r->h > h) {
-av_log(NULL, AV_LOG_WARNING, "sub2video: rectangle (%d %d %d %d)
overflowing %d %d\n",
-r->x, r->y, r->w, r->h, w, h
-);
-return;
-}
-
-dst += r->y * dst_linesize + r->x * 4;
-src = r->data[0];
-pal = (uint32_t *)r->data[1];
-for (y = 0; y < r->h; y++) {
-dst2 = (uint32_t *)dst;
-src2 = src;
-for (x = 0; x < r->w; x++)
-*(dst2++) = pal[*(src2++)];
-dst += dst_linesize;
-src += r->linesize[0];
-}
-}
-
-static void sub2video_push_ref(InputStream *ist, int64_t pts)
-{
-AVFrame *frame = ist->sub2video.frame;
-int i;
-int ret;
-
-av_assert1(frame->data[0]);
-ist->sub2video.last_pts = frame->pts = pts;
-for (i = 0; i < ist->nb_filters; i++) {
-ret = av_buffersrc_add_frame_flags(ist->filters[i]->filter, frame,
- AV_BUFFERSRC_FLAG_KEEP_REF |
- AV_BUFFERSRC_FLAG_PUSH);
-if (ret != AVERROR_EOF && ret < 0)
-av_log(NULL, AV_LOG_WARNING, "Error while add the frame to buffer
source(%s).\n",
- av_err2str(ret));
-}
-}
-
-void sub2video_update(InputStream *ist, int64_t heartbeat_pts, AVSubtitle *sub)
-{
-AVFrame *frame = ist->sub2video.frame;
-int8_t *dst;
-int dst_linesize;
-int num_rects, i;
-int64_t pts, end_pts;
-
-if (!fra
[FFmpeg-devel] [PATCH v11 09/14] avfilter/overlay_textsubs: Add overlay_textsubs and textsubs2video filters
- overlay_textsubs {VS -> V)
Overlay text subtitles onto a video stream.
- textsubs2video {S -> V)
Converts text subtitles to video frames
Signed-off-by: softworkz
---
configure | 2 +
doc/filters.texi | 69
libavfilter/Makefile | 2 +
libavfilter/allfilters.c | 2 +
libavfilter/avfilter.c| 18 +-
libavfilter/vf_overlay_textsubs.c | 609 ++
6 files changed, 697 insertions(+), 5 deletions(-)
create mode 100644 libavfilter/vf_overlay_textsubs.c
diff --git a/configure b/configure
index d18bb73ed6..37fc4c20e7 100755
--- a/configure
+++ b/configure
@@ -3624,6 +3624,7 @@ openclsrc_filter_deps="opencl"
overlay_opencl_filter_deps="opencl"
overlay_qsv_filter_deps="libmfx"
overlay_qsv_filter_select="qsvvpp"
+overlay_textsubs_filter_deps="avcodec libass"
overlay_vulkan_filter_deps="vulkan_lib libglslang"
owdenoise_filter_deps="gpl"
pad_opencl_filter_deps="opencl"
@@ -3669,6 +3670,7 @@ superequalizer_filter_deps="avcodec"
superequalizer_filter_select="rdft"
surround_filter_deps="avcodec"
surround_filter_select="rdft"
+textsub2video_filter_deps="avcodec libass"
tinterlace_filter_deps="gpl"
tinterlace_merge_test_deps="tinterlace_filter"
tinterlace_pad_test_deps="tinterlace_filter"
diff --git a/doc/filters.texi b/doc/filters.texi
index 18b5273e6e..2f81df6489 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25297,6 +25297,75 @@ The graphicsub2video is inserted automatically for
compatibility with legacy com
ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:0][0:1]overlay" output.mp4
@end example
@end itemize
+
+@section overlay_textsubs
+
+Overlay text subtitles onto a video stream.
+
+This filter supersedes the classic @ref{subtitles} filter opposed to which it
does no longer require to open and access the source stream separately, which
is often causing problems or doesn't even work for non-local or slow sources.
+
+Inputs:
+- 0: Video [YUV420P, YUV422P, YUV444P, ARGB, RGBA, ABGR, BGRA, RGB24, BGR24]
+- 1: Subtitles [text]
+
+Outputs:
+- 0: Video (same as input)
+
+It accepts the following parameters:
+
+@table @option
+
+@item alpha
+Process alpha channel, by default alpha channel is untouched.
+
+@item fonts_dir
+Set a directory path containing fonts that can be used by the filter.
+These fonts will be used in addition to whatever the font provider uses.
+
+@item force_style
+Override default style or script info parameters of the subtitles. It accepts a
+string containing ASS style format @code{KEY=VALUE} couples separated by ",".
+
+@end table
+
+@section textsub2video
+
+Converts text subtitles to video frames.
+
+For overlaying text subtitles onto video frames it is recommended to use the
overlay_textsubs filter.
+The textsub2video is useful for for creating transparent text-frames when
overlay is done via hw acceleration
+
+Inputs:
+- 0: Subtitles [text]
+
+Outputs:
+- 0: Video [ARGB, RGBA, ABGR, BGRA]
+
+It accepts the following parameters:
+
+@table @option
+
+@item rate, r
+Set the framerate for updating overlay frames.
+Normally, overlay frames will only be updated each time when the subtitles to
display are changing.
+In cases where subtitles include advanced features (like animation), this
parameter determines the frequency by which the overlay frames should be
updated.
+
+@item size, s
+Set the output frame size.
+Allows to override the size of output video frames.
+
+@item alpha
+Process alpha channel, by default alpha channel is untouched.
+
+@item fonts_dir
+Set a directory path containing fonts that can be used by the filter.
+These fonts will be used in addition to whatever the font provider uses.
+
+@item force_style
+Override default style or script info parameters of the subtitles. It accepts a
+string containing ASS style format @code{KEY=VALUE} couples separated by ",".
+
+@end table
@c man end SUBTITLE FILTERS
@chapter Multimedia Filters
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 01c3926bf8..584a97c9a3 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -366,6 +366,7 @@ OBJS-$(CONFIG_OVERLAY_OPENCL_FILTER) +=
vf_overlay_opencl.o opencl.o \
opencl/overlay.o framesync.o
OBJS-$(CONFIG_OVERLAY_QSV_FILTER)+= vf_overlay_qsv.o framesync.o
OBJS-$(CONFIG_OVERLAY_GRAPHICSUBS_FILTER)+= vf_overlay_graphicsubs.o
framesync.o
+OBJS-$(CONFIG_OVERLAY_TEXTSUBS_FILTER) += vf_overlay_textsubs.o
OBJS-$(CONFIG_OVERLAY_VULKAN_FILTER) += vf_overlay_vulkan.o vulkan.o
OBJS-$(CONFIG_OWDENOISE_FILTER) += vf_owdenoise.o
OBJS-$(CONFIG_PAD_FILTER)+= vf_pad.o
@@ -454,6 +455,7 @@ OBJS-$(CONFIG_SWAPRECT_FILTER) +=
vf_swaprect.o
OBJS-$(CONFIG_SWAPUV_FILTER) += vf_swapuv.o
OBJS-$(CONFIG_TBLEND_FILTER)
[FFmpeg-devel] [PATCH v11 10/14] avfilter/textmod: Add textmod, censor and show_speaker filters
- textmod {S -> S)
Modify subtitle text in a number of ways
- censor {S -> S)
Censor subtitles using a word list
- show_speaker {S -> S)
Prepend speaker names from ASS subtitles to the visible text lines
Signed-off-by: softworkz
---
doc/filters.texi | 229 +
libavfilter/Makefile | 5 +
libavfilter/allfilters.c | 3 +
libavfilter/sf_textmod.c | 719 +++
4 files changed, 956 insertions(+)
create mode 100644 libavfilter/sf_textmod.c
diff --git a/doc/filters.texi b/doc/filters.texi
index 2f81df6489..c751211017 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25195,6 +25195,7 @@ tools.
@c man end VIDEO SINKS
+
@chapter Subtitle Filters
@c man begin SUBTITLE FILTERS
@@ -25203,6 +25204,166 @@ existing filters using @code{--disable-filters}.
Below is a description of the currently available subtitle filters.
+
+@section censor
+
+Censor selected words in text subtitles.
+
+Inputs:
+- 0: Subtitles [text]
+
+Outputs:
+- 0: Subtitles [text]
+
+It accepts the following parameters:
+
+@table @option
+@item mode
+The censoring mode to apply.
+
+Supported censoring modes are:
+
+@table @var
+@item 0, keep_first_last
+Replace all characters with the 'censor_char' except the first and the last
character of a word.
+For words with less than 4 characters, the last character will be replaced as
well.
+For words with less than 3 characters, the first character will be replaced as
well.
+@item 1, keep_first
+Replace all characters with the 'censor_char' except the first character of a
word.
+For words with less than 3 characters, the first character will be replaced as
well.
+@item 2, all
+Replace all characters with the 'censor_char'.
+@end table
+
+@item words
+A list of words to censor, separated by 'separator'.
+
+@item words_file
+Specify a file from which to load the contents for the 'words' parameter.
+
+@item censor_char
+Single character used as replacement for censoring.
+
+@item separator
+Delimiter character for words. Used with replace_words and remove_words- Must
be a single character.
+The default is '.'.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Change all characters to upper case while keeping all styles and animations:
+@example
+ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv";
-filter_complex "[0:s]textmod=mode=to_upper" -map 0 -y out.mkv
+@end example
+@item
+Remove a set of symbol characters for am improved and smoother visual
apperance:
+@example
+ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv";
-filter_complex "[0:s]textmod=mode=remove_chars:find='$&#@*§'" -map 0 -y
out.mkv
+@end example
+@end itemize
+
+
+@section stripstyles
+
+Remove all inline styles from subtitle events.
+
+It accepts the following parameters:
+
+@table @option
+@item remove_animated
+Also remove text which is subject to animation (default: true)
+Usually, animated text elements are used used in addition to static subtitle
lines for creating effects, so in most cases it is safe to remove the animation
content.
+If subtitle text is missing, try setting this to false.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Remove all styles and animations from subtitles:
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv";
-filter_complex "[0:1]stripstyles" -map 0 output.mkv
+@end example
+@end itemize
+
+@section textmod
+
+Modify subtitle text in a number of ways.
+
+Inputs:
+- 0: Subtitles [text]
+
+Outputs:
+- 0: Subtitles [text]
+
+It accepts the following parameters:
+
+@table @option
+@item mode
+The kind of text modification to apply
+
+Supported operation modes are:
+
+@table @var
+@item 0, leet
+Convert subtitle text to 'leet speak'. It's primarily useful for testing as
the modification will be visible with almost all text lines.
+@item 1, to_upper
+Change all text to upper case. Might improve readability.
+@item 2, to_lower
+Change all text to lower case.
+@item 3, replace_chars
+Replace one or more characters. Requires the find and replace parameters to be
specified.
+Both need to be equal in length.
+The first char in find is replaced by the first char in replace, same for all
subsequent chars.
+@item 4, remove_chars
+Remove certain characters. Requires the find parameter to be specified.
+All chars in the find parameter string will be removed from all subtitle text.
+@item 5, replace_words
+Replace one or more words. Requires the find and replace parameters to be
specified. Multiple words must be separated by the delimiter char specified vie
the separator parameter (default: ',').
+The number of words in the find and replace parameters needs to be equal.
+The first word in find is replaced by the first word in replace, same for all
subsequent words
+@item 6, remove_words
+Remove certain words. Requires the find parameter to be specified. Multiple
words must be separated by the delimiter char specified v
[FFmpeg-devel] [PATCH v11 11/14] avfilter/stripstyles: Add stripstyles filter
- stripstyles {S -> S)
Remove all inline styles from subtitle events
Signed-off-by: softworkz
---
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/sf_stripstyles.c | 211 +++
3 files changed, 213 insertions(+)
create mode 100644 libavfilter/sf_stripstyles.c
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index df250ed07b..c991a525f0 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -541,6 +541,7 @@ OBJS-$(CONFIG_NULLSINK_FILTER) +=
vsink_nullsink.o
OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o
OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o
OBJS-$(CONFIG_TEXTMOD_FILTER)+= sf_textmod.o
+OBJS-$(CONFIG_STRIPSTYLES_FILTER)+= sf_stripstyles.o
# multimedia filters
OBJS-$(CONFIG_ABITSCOPE_FILTER) += avf_abitscope.o
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 232eac4ef6..bdf4c2ae52 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -529,6 +529,7 @@ extern const AVFilter ff_avf_showwavespic;
extern const AVFilter ff_vaf_spectrumsynth;
extern const AVFilter ff_sf_censor;
extern const AVFilter ff_sf_show_speaker;
+extern const AVFilter ff_sf_stripstyles;
extern const AVFilter ff_sf_textmod;
extern const AVFilter ff_svf_graphicsub2video;
extern const AVFilter ff_svf_textsub2video;
diff --git a/libavfilter/sf_stripstyles.c b/libavfilter/sf_stripstyles.c
new file mode 100644
index 00..bdde28577f
--- /dev/null
+++ b/libavfilter/sf_stripstyles.c
@@ -0,0 +1,211 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/**
+ * @file
+ * text subtitle filter which removes inline-styles from subtitles
+ */
+
+#include "libavutil/opt.h"
+#include "internal.h"
+#include "libavutil/ass_split_internal.h"
+#include "libavutil/bprint.h"
+
+typedef struct StripStylesContext {
+const AVClass *class;
+enum AVSubtitleType format;
+int remove_animated;
+} StripStylesContext;
+
+typedef struct DialogContext {
+StripStylesContext* ss_ctx;
+AVBPrint buffer;
+int drawing_scale;
+int is_animated;
+} DialogContext;
+
+static void dialog_text_cb(void *priv, const char *text, int len)
+{
+DialogContext *s = priv;
+
+av_log(s->ss_ctx, AV_LOG_DEBUG, "dialog_text_cb: %s\n", text);
+
+if (!s->drawing_scale && (!s->is_animated || !s->ss_ctx->remove_animated))
+av_bprint_append_data(&s->buffer, text, len);
+}
+
+static void dialog_new_line_cb(void *priv, int forced)
+{
+DialogContext *s = priv;
+if (!s->drawing_scale && !s->is_animated)
+av_bprint_append_data(&s->buffer, forced ? "\\N" : "\\n", 2);
+}
+
+static void dialog_drawing_mode_cb(void *priv, int scale)
+{
+DialogContext *s = priv;
+s->drawing_scale = scale;
+}
+
+static void dialog_animate_cb(void *priv, int t1, int t2, int accel, char
*style)
+{
+DialogContext *s = priv;
+s->is_animated = 1;
+}
+
+static void dialog_move_cb(void *priv, int x1, int y1, int x2, int y2, int t1,
int t2)
+{
+DialogContext *s = priv;
+if (t1 >= 0 || t2 >= 0)
+s->is_animated = 1;
+}
+
+static const ASSCodesCallbacks dialog_callbacks = {
+.text = dialog_text_cb,
+.new_line = dialog_new_line_cb,
+.drawing_mode = dialog_drawing_mode_cb,
+.animate = dialog_animate_cb,
+.move = dialog_move_cb,
+};
+
+static int query_formats(AVFilterContext *ctx)
+{
+AVFilterFormats *formats;
+AVFilterLink *inlink = ctx->inputs[0];
+AVFilterLink *outlink = ctx->outputs[0];
+static const enum AVSubtitleType subtitle_fmts[] = { AV_SUBTITLE_FMT_ASS,
AV_SUBTITLE_FMT_NONE };
+int ret;
+
+/* set input subtitle format */
+formats = ff_make_format_list(subtitle_fmts);
+if ((ret = ff_formats_ref(formats, &inlink->outcfg.formats)) < 0)
+return ret;
+
+/* set output video format */
+if ((ret = ff_formats_ref(formats, &outlink->incfg.formats)) < 0)
+return ret;
+
+return 0;
+}
+
+static char *ass_get_line(int readorder, int layer, const char *style,
+const char *speaker, const char *effect, const char
*t
[FFmpeg-devel] [PATCH v11 12/14] avfilter/split_cc: Add split_cc filter for closed caption handling
- split_cc {V -> VS)
Extract closed-caption (A53) data from video
frames as subtitle Frames
ffmpeg -y -loglevel verbose -i
"https://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts";
-filter_complex
"[0:v]split_cc[vid1],textmod=mode=remove_chars:find='@',[vid1]overlay_textsubs"
output.mkv
Signed-off-by: softworkz
---
configure | 1 +
doc/filters.texi | 45 ++
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/sf_split_cc.c | 298 ++
5 files changed, 346 insertions(+)
create mode 100644 libavfilter/sf_split_cc.c
diff --git a/configure b/configure
index 37fc4c20e7..6da059e6ee 100755
--- a/configure
+++ b/configure
@@ -3663,6 +3663,7 @@ spp_filter_select="fft idctdsp fdctdsp me_cmp pixblockdsp"
sr_filter_deps="avformat swscale"
sr_filter_select="dnn"
stereo3d_filter_deps="gpl"
+split_cc_filter_deps="cc_dec_decoder"
subtitles_filter_deps="avformat avcodec libass"
super2xsai_filter_deps="gpl"
pixfmts_super2xsai_test_deps="super2xsai_filter"
diff --git a/doc/filters.texi b/doc/filters.texi
index c751211017..001d2baa8f 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25494,6 +25494,16 @@ string containing ASS style format @code{KEY=VALUE}
couples separated by ",".
@end table
+@subsection Examples
+
+@itemize
+@item
+Overlay ASS subtitles with animations:
+@example
+ffmpeg -i
"http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv";
-filter_complex "[0:v]overlay_textsubs" -map 0 -y out.mkv
+@end example
+@end itemize
+
@section show_speaker
Prepend speaker names to subtitle lines (when available).
@@ -25557,6 +25567,41 @@ ffmpeg -i INPUT -filter_complex
"show_speaker=format=colon:style='@{\\c&HDD&
@end example
@end itemize
+
+@section split_cc
+
+Split-out closed-caption/A53 subtitles from video frame side data.
+
+This filter provides an input and an output for video frames, which are just
passed through without modification.
+The second out provides subtitle frames which are extracted from video frame
side data.
+
+Inputs:
+- 0: Video
+
+Outputs:
+- 0: Video (same as input)
+- 1: Subtitles [text]
+
+It accepts the following parameters:
+
+@table @option
+
+@item use_cc_styles
+Emit closed caption style header. This will make closed captions look like on
normal TV devices.
+(white font on black background rectangles)
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Extract closed captions as text subtitle stream and overlay it onto the video:
+@example
+ffmpeg -i "https://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts";
-filter_complex "[0:v:0]split_cc[vid1][sub1];[vid1][sub1]overlay_textsubs"
output.mkv
+@end example
+@end itemize
+
@section textsub2video
Converts text subtitles to video frames.
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index c991a525f0..b260e7e0a2 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -541,6 +541,7 @@ OBJS-$(CONFIG_NULLSINK_FILTER) +=
vsink_nullsink.o
OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o
OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o
OBJS-$(CONFIG_TEXTMOD_FILTER)+= sf_textmod.o
+OBJS-$(CONFIG_SPLIT_CC_FILTER) += sf_split_cc.o
OBJS-$(CONFIG_STRIPSTYLES_FILTER)+= sf_stripstyles.o
# multimedia filters
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index bdf4c2ae52..40bb3c229e 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -529,6 +529,7 @@ extern const AVFilter ff_avf_showwavespic;
extern const AVFilter ff_vaf_spectrumsynth;
extern const AVFilter ff_sf_censor;
extern const AVFilter ff_sf_show_speaker;
+extern const AVFilter ff_sf_split_cc;
extern const AVFilter ff_sf_stripstyles;
extern const AVFilter ff_sf_textmod;
extern const AVFilter ff_svf_graphicsub2video;
diff --git a/libavfilter/sf_split_cc.c b/libavfilter/sf_split_cc.c
new file mode 100644
index 00..d96912a90b
--- /dev/null
+++ b/libavfilter/sf_split_cc.c
@@ -0,0 +1,298 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/**
+ * @file
+ * subtitle filter for splitting out closed-caption/A
[FFmpeg-devel] [PATCH v11 13/14] avfilter/graphicsub2text: Add new graphicsub2text filter (OCR)
Signed-off-by: softworkz --- configure| 1 + doc/filters.texi | 55 ++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/sf_graphicsub2text.c | 323 +++ 5 files changed, 381 insertions(+) create mode 100644 libavfilter/sf_graphicsub2text.c diff --git a/configure b/configure index 6da059e6ee..9108742122 100755 --- a/configure +++ b/configure @@ -3601,6 +3601,7 @@ frei0r_deps_any="libdl LoadLibrary" frei0r_filter_deps="frei0r" frei0r_src_filter_deps="frei0r" fspp_filter_deps="gpl" +graphicsub2text_filter_deps="libtesseract" histeq_filter_deps="gpl" hqdn3d_filter_deps="gpl" interlace_filter_deps="gpl" diff --git a/doc/filters.texi b/doc/filters.texi index 001d2baa8f..e709fcbc03 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -25364,6 +25364,61 @@ ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv"; -filter_comple @end example @end itemize +@section graphicsub2text + +Converts graphic subtitles to text subtitles by performing OCR. + +For this filter to be available, ffmpeg needs to be compiled with libtesseract (see https://github.com/tesseract-ocr/tesseract). +Language models need to be downloaded from https://github.com/tesseract-ocr/tessdata and put into as subfolder named 'tessdata' or into a folder specified via the environment variable 'TESSDATA_PREFIX'. +The path can also be specified via filter option (see below). + +Note: These models are including the data for both OCR modes. + +Inputs: +- 0: Subtitles [bitmap] + +Outputs: +- 0: Subtitles [text] + +It accepts the following parameters: + +@table @option +@item ocr_mode +The character recognition mode to use. + +Supported OCR modes are: + +@table @var +@item 0, tesseract +This is the classic libtesseract operation mode. It is fast but less accurate than LSTM. +@item 1, lstm +Newer OCR implementation based on ML models. Provides usually better results, requires more processing resources. +@item 2, both +Use a combination of both modes. +@end table + +@item tessdata_path +The path to a folder containing the language models to be used. + +@item language +The recognition language. It needs to match the first three characters of a language model file in the tessdata path. + +@end table + + +@subsection Examples + +@itemize +@item +Convert DVB graphic subtitles to ASS (text) subtitles + +Note: For this to work, you need to have the data file 'eng.traineddata' in a 'tessdata' subfolder (see above). +@example +ffmpeg ffmpeg -loglevel verbose -i "https://streams.videolan.org/streams/ts/video_subs_ttxt%2Bdvbsub.ts"; -filter_complex "[0:13]graphicsub2text=ocr_mode=both" -c:s ass -y output.mkv +@end example +@end itemize + + @section graphicsub2video Renders graphic subtitles as video frames. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index b260e7e0a2..fc12664c56 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -290,6 +290,7 @@ OBJS-$(CONFIG_FSPP_FILTER) += vf_fspp.o qp_table.o OBJS-$(CONFIG_GBLUR_FILTER) += vf_gblur.o OBJS-$(CONFIG_GEQ_FILTER)+= vf_geq.o OBJS-$(CONFIG_GRADFUN_FILTER)+= vf_gradfun.o +OBJS-$(CONFIG_GRAPHICSUB2TEXT_FILTER)+= sf_graphicsub2text.o OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlay_graphicsubs.o framesync.o OBJS-$(CONFIG_GRAPHMONITOR_FILTER) += f_graphmonitor.o OBJS-$(CONFIG_GRAYWORLD_FILTER) += vf_grayworld.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 40bb3c229e..72ab388518 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -528,6 +528,7 @@ extern const AVFilter ff_avf_showwaves; extern const AVFilter ff_avf_showwavespic; extern const AVFilter ff_vaf_spectrumsynth; extern const AVFilter ff_sf_censor; +extern const AVFilter ff_sf_graphicsub2text; extern const AVFilter ff_sf_show_speaker; extern const AVFilter ff_sf_split_cc; extern const AVFilter ff_sf_stripstyles; diff --git a/libavfilter/sf_graphicsub2text.c b/libavfilter/sf_graphicsub2text.c new file mode 100644 index 00..6c5c3d14ce --- /dev/null +++ b/libavfilter/sf_graphicsub2text.c @@ -0,0 +1,323 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the
[FFmpeg-devel] [PATCH v11 14/14] avfilter/subscale: Add filter for scaling and/or re-arranging graphical subtitles
Signed-off-by: softworkz
---
configure | 1 +
doc/filters.texi | 160 +++
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/sf_subscale.c | 867 ++
5 files changed, 1030 insertions(+)
create mode 100644 libavfilter/sf_subscale.c
diff --git a/configure b/configure
index 9108742122..93b26da269 100755
--- a/configure
+++ b/configure
@@ -3665,6 +3665,7 @@ sr_filter_deps="avformat swscale"
sr_filter_select="dnn"
stereo3d_filter_deps="gpl"
split_cc_filter_deps="cc_dec_decoder"
+subscale_filter_deps="swscale avcodec"
subtitles_filter_deps="avformat avcodec libass"
super2xsai_filter_deps="gpl"
pixfmts_super2xsai_test_deps="super2xsai_filter"
diff --git a/doc/filters.texi b/doc/filters.texi
index e709fcbc03..9cc6b05835 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25695,6 +25695,166 @@ Override default style or script info parameters of
the subtitles. It accepts a
string containing ASS style format @code{KEY=VALUE} couples separated by ",".
@end table
+
+@section subscale
+
+Provides high-quality scaling and rearranging functionality for graphical
subtitles.
+
+The subscale filter provides multiple approaches for manipulating
+the size and position of graphical subtitle rectangles wich can
+be combined or used separately.
+Scaling is performed by converting the palettized subtitle bitmaps
+to RGBA and re-quantization to palette colors afterwards via elbg algorithm.
+
+The two major operations are 'scale' and 're-arrange' with the
+latter being separated as 'arrange_h' and 'arrange_v'.
+
+
+Inputs:
+- 0: Subtitles [bitmap]
+
+Outputs:
+- 0: Subtitles [bitmap]
+
+It accepts the following parameters:
+
+@table @option
+
+@item w, width
+Set the width of the output.
+Width and height in case of graphical subtitles are just indicating
+a virtual size for which the output (consisting of 0-n bitmap rectangles)
+is intended to be displayed on.
+
+@item h, height
+Set the height of the output.
+
+@item margin_h
+Sets a horizontal margin to be preserverved when using any
+of the arrange modes.
+
+@item margin_v
+Sets a vertical margin to be preserverved when using any
+of the arrange modes.
+
+@item force_original_aspect_ratio
+Enable decreasing or increasing output video width or height if necessary to
+keep the original aspect ratio. Possible values:
+
+@table @samp
+@item disable
+Scale the video as specified and disable this feature.
+
+@item decrease
+The output video dimensions will automatically be decreased if needed.
+
+@item increase
+The output video dimensions will automatically be increased if needed.
+
+@end table
+
+
+@item scale_mode
+Specifies how subtitle bitmaps should be scaled.
+The scale factor is determined by the the factor between input
+and output size.
+
+@table @samp
+@item none
+Do not apply any common scaling.
+
+@item uniform
+Uniformly scale all subtitle bitmaps including their positions.
+
+@item uniform_no_reposition
+Uniformly scale all subtitle bitmaps without changing positions.
+
+@end table
+
+
+@item arrange_h
+Specifies how subtitle bitmaps should be arranged horizontally.
+
+@item arrange_v
+Specifies how subtitle bitmaps should be arranged vertically.
+
+
+@table @samp
+@item none
+Do not rearrange subtitle bitmaps.
+
+@item margin_no_scale
+Move subtitle bitmaps to be positioned inside the specified
+margin (margin_h or margin_v) when possible and without scaling.
+
+@item margin_and_scale
+Move subtitle bitmaps to be positioned inside the specified
+margin (margin_h or margin_v) and scale in case it doesn't fit.
+
+@item snapalign_no_scale
+Categorize subtitle bitmap positions as one of left/center/right
+or top/bottom/middle based on original positioning and apply
+these alignments for the target positioning.
+No scaling will be applied.
+
+@item snapalign_and_scale
+Categorize subtitle bitmap positions as one of left/center/right
+or top/bottom/middle based on original positioning and apply
+these alignments for the target positioning.
+Bitmaps that do not fit inside the margins borders are
+scaled to fit.
+@end table
+
+@item eval
+Set evaluation mode for the expressions (@option{width}, @option{height}).
+
+It accepts the following values:
+@table @samp
+@item init
+Evaluate expressions only once during the filter initialization.
+
+@item frame
+Evaluate expressions for each incoming frame. This is way slower than the
+@samp{init} mode since it requires all the scalers to be re-computed, but it
+allows advanced dynamic expressions.
+@end table
+
+Default value is @samp{init}.
+
+
+@item num_colors
+Set the number of palette colors for output images.
+Choose the maximum (256) when further processing is done (e.g.
+overlaying on a video).
+When subtitles will be encoded as bitmap subtitles (e.g. dvbsub),
+a smaller number of palette colors (e.g. 4-16) might need to be used, depending
+on the target format and codec.
+
+@item bitmap_width_align
Re: [FFmpeg-devel] [PATCH v11 10/14] avfilter/textmod: Add textmod, censor and show_speaker filters
> -Original Message- > From: ffmpeg-devel On Behalf Of > Paul B Mahol > Sent: Friday, 1 October 2021 10:57 > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v11 10/14] avfilter/textmod: Add > textmod, censor and show_speaker filters > > Please remove '_' from all your filters names. It is not needed Yes ok, will do. Should I also rename: overlay_graphicsubs => overlaygraphicsubs ? Any other suggestions about filter naming, do you think they are fine? Thanks, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 2/7] libavformat/asfdec: Fix get_value return type
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Michael Niedermayer
> Sent: Friday, 1 October 2021 19:56
> To: FFmpeg development discussions and patches de...@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v5 2/7] libavformat/asfdec: Fix
> get_value return type
>
> On Thu, Sep 30, 2021 at 12:11:13PM +, Soft Works wrote:
> >
> >
> > > -Original Message-
> > > From: ffmpeg-devel On Behalf Of
> > > Michael Niedermayer
> > > Sent: Thursday, 30 September 2021 14:04
> > > To: FFmpeg development discussions and patches > > de...@ffmpeg.org>
> > > Subject: Re: [FFmpeg-devel] [PATCH v5 2/7] libavformat/asfdec:
> Fix
> > > get_value return type
> > >
> > > On Thu, Sep 30, 2021 at 01:45:46PM +0200, Michael Niedermayer
> wrote:
> > > > On Thu, Sep 30, 2021 at 02:58:30AM +, Soft Works wrote:
> > > > > get_value had a return type of int, which means that reading
> > > > > QWORDS (case 4) was broken due to truncation of the result
> from
> > > > > avio_rl64().
> > > > >
> > > > > Signed-off-by: softworkz
> > > > > ---
> > > > > v5: Split into pieces as requested
> > > > >
> > > > > libavformat/asfdec_f.c | 10 +-
> > > > > 1 file changed, 5 insertions(+), 5 deletions(-)
> > > > >
> > > > > diff --git a/libavformat/asfdec_f.c b/libavformat/asfdec_f.c
> > > > > index 72ba8b32a0..076b5ab147 100644
> > > > > --- a/libavformat/asfdec_f.c
> > > > > +++ b/libavformat/asfdec_f.c
> > > > > @@ -202,7 +202,7 @@ static int asf_probe(const AVProbeData
> *pd)
> > > > >
> > > > > /* size of type 2 (BOOL) is 32bit for "Extended Content
> > > Description Object"
> > > > > * but 16 bit for "Metadata Object" and "Metadata Library
> > > Object" */
> > > > > -static int get_value(AVIOContext *pb, int type, int
> type2_size)
> > > > > +static uint64_t get_value(AVIOContext *pb, int type, int
> > > type2_size)
> > > > > {
> > > > > switch (type) {
> > > > > case 2:
> > > > > @@ -568,9 +568,9 @@ static int
> > > asf_read_ext_content_desc(AVFormatContext *s, int64_t size)
> > > > > * ASF stream count starts at 1. I am using 0 to the
> > > container value
> > > > > * since it's unused. */
> > > > > if (!strcmp(name, "AspectRatioX"))
> > > > > -asf->dar[0].num = get_value(s->pb, value_type,
> 32);
> > > > > +asf->dar[0].num = (int)get_value(s->pb,
> value_type,
> > > 32);
> > > > > else if (!strcmp(name, "AspectRatioY"))
> > > > > -asf->dar[0].den = get_value(s->pb, value_type,
> 32);
> > > > > +asf->dar[0].den = (int)get_value(s->pb,
> value_type,
> > > 32);
> > > > > else
> > > > > get_tag(s, name, value_type, value_len, 32);
> > > > > }
> > > > > @@ -630,11 +630,11 @@ static int
> > > asf_read_metadata(AVFormatContext *s, int64_t size)
> > > > > i, stream_num, name_len_utf16, value_type,
> > > value_len, name);
> > > > >
> > > > > if (!strcmp(name, "AspectRatioX")){
> > > > > -int aspect_x = get_value(s->pb, value_type, 16);
> > > > > +int aspect_x = (int)get_value(s->pb, value_type,
> > > 16);
> > > > > if(stream_num < 128)
> > > > > asf->dar[stream_num].num = aspect_x;
> > > > > } else if(!strcmp(name, "AspectRatioY")){
> > > > > -int aspect_y = get_value(s->pb, value_type, 16);
> > > > > +int aspect_y = (int)get_value(s->pb, value_type,
> > > 16);
> > > > > if(stream_num < 128)
> > > > > asf->dar[stream_num].den = aspect_y;
> > > > > } else {
> > > >
> > > > a 64bit/64bit aspect does not work after this, it still just
> > > silently
> > > > truncates the value to 32bit, the truncation just happens later
> > >
> > > forgot: se
Re: [FFmpeg-devel] [PATCH v11 10/14] avfilter/textmod: Add textmod, censor and show_speaker filters
> -Original Message- > From: ffmpeg-devel On Behalf Of > Paul B Mahol > Sent: Friday, 1 October 2021 22:32 > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v11 10/14] avfilter/textmod: Add > textmod, censor and show_speaker filters > > On Fri, Oct 1, 2021 at 9:43 PM Soft Works > wrote: > > > > > > > > -Original Message- > > > From: ffmpeg-devel On Behalf Of > > > Paul B Mahol > > > Sent: Friday, 1 October 2021 10:57 > > > To: FFmpeg development discussions and patches > > de...@ffmpeg.org> > > > Subject: Re: [FFmpeg-devel] [PATCH v11 10/14] avfilter/textmod: > Add > > > textmod, censor and show_speaker filters > > > > > > Please remove '_' from all your filters names. It is not needed > > > > Yes ok, will do. > > > > Should I also rename: overlay_graphicsubs => overlaygraphicsubs ? > > > > that is really very long, maybe overlayvsubs vs overlaytsubs What's the v in overlay_v_subs - video? I'm not sure about that, but I'll make another post about naming to gather some comments. Thanks, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [RFC] Naming of Subtitle Types
Hi, I wanted to gather some opinions about the naming of subtitle types. In my patched I have consistently used the terms "Graphic Subtitles" and "Text Subtitles" everywhere. I think "Text Subtitles" is out of question, but for the other one, there are two viable options: "Bitmap Subtitles" or "Graphc Subtitles" I tend to the latter, but if anybody had objections, now would be a good time to let us know :-) Thanks, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [RFC] Naming of Subtitle Types
> -Original Message- > From: ffmpeg-devel On Behalf Of > Soft Works > Sent: Friday, 1 October 2021 23:09 > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: [FFmpeg-devel] [RFC] Naming of Subtitle Types > > Hi, > > I wanted to gather some opinions about the naming of subtitle types. > In my patched I have consistently used the terms "Graphic Subtitles" > and "Text Subtitles" everywhere. > > I think "Text Subtitles" is out of question, but for the other one, > > there are two viable options: > > "Bitmap Subtitles" > > or > > "Graphc Subtitles" > > I tend to the latter, but if anybody had objections, now would be > a good time to let us know :-) Same goes for the individual filter names. Paul has suggested to remove the underscores, for those I'm showing the underscore-less name in square brackets: - subscale Provides high-quality scaling and rearranging functionality for graphical subtitles. - graphicsubs2text Convert graphic subtitles to text subtitles via OCR - censor Allows censoring word in subtitles by a configurable word list - show_speaker [showspeaker] Prepend the speaker names to text lines (those are often available in ass subs) - split_cc [splitcc] Splits out Closed-Caption data as a subtitle stream for filtering and encoding - overlay_graphicsubs [overlaygraphicsubs] Overlay graphic subtitles onto a video stream - graphicsub2video Converts graphic subtitles to (transparent) video frames - overlay_textsubs [overlaytextsubs] Overlay text subtitles onto a video stream. - textsubs2video Converts text subtitles to video frames - textmod Modify subtitle text in a number of ways - stripstyles Remove all inline styles from subtitle events Please let me know when there are objections regarding any of those names. Thanks, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 2/7] libavformat/asfdec: Fix get_value return type
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Michael Niedermayer
> Sent: Saturday, 2 October 2021 13:46
> To: FFmpeg development discussions and patches de...@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v5 2/7] libavformat/asfdec: Fix
> get_value return type
>
> On Fri, Oct 01, 2021 at 08:44:04PM +, Soft Works wrote:
> >
> >
> > > -Original Message-
> > > From: ffmpeg-devel On Behalf Of
> > > Michael Niedermayer
> > > Sent: Friday, 1 October 2021 19:56
> > > To: FFmpeg development discussions and patches > > de...@ffmpeg.org>
> > > Subject: Re: [FFmpeg-devel] [PATCH v5 2/7] libavformat/asfdec:
> Fix
> > > get_value return type
> > >
> > > On Thu, Sep 30, 2021 at 12:11:13PM +, Soft Works wrote:
> > > >
> > > >
> > > > > -Original Message-
> > > > > From: ffmpeg-devel On
> Behalf Of
> > > > > Michael Niedermayer
> > > > > Sent: Thursday, 30 September 2021 14:04
> > > > > To: FFmpeg development discussions and patches > > > > de...@ffmpeg.org>
> > > > > Subject: Re: [FFmpeg-devel] [PATCH v5 2/7]
> libavformat/asfdec:
> > > Fix
> > > > > get_value return type
> > > > >
> > > > > On Thu, Sep 30, 2021 at 01:45:46PM +0200, Michael Niedermayer
> > > wrote:
> > > > > > On Thu, Sep 30, 2021 at 02:58:30AM +, Soft Works wrote:
> > > > > > > get_value had a return type of int, which means that
> reading
> > > > > > > QWORDS (case 4) was broken due to truncation of the
> result
> > > from
> > > > > > > avio_rl64().
> > > > > > >
> > > > > > > Signed-off-by: softworkz
> > > > > > > ---
> > > > > > > v5: Split into pieces as requested
> > > > > > >
> > > > > > > libavformat/asfdec_f.c | 10 +-
> > > > > > > 1 file changed, 5 insertions(+), 5 deletions(-)
> > > > > > >
> > > > > > > diff --git a/libavformat/asfdec_f.c
> b/libavformat/asfdec_f.c
> > > > > > > index 72ba8b32a0..076b5ab147 100644
> > > > > > > --- a/libavformat/asfdec_f.c
> > > > > > > +++ b/libavformat/asfdec_f.c
> > > > > > > @@ -202,7 +202,7 @@ static int asf_probe(const
> AVProbeData
> > > *pd)
> > > > > > >
> > > > > > > /* size of type 2 (BOOL) is 32bit for "Extended Content
> > > > > Description Object"
> > > > > > > * but 16 bit for "Metadata Object" and "Metadata
> Library
> > > > > Object" */
> > > > > > > -static int get_value(AVIOContext *pb, int type, int
> > > type2_size)
> > > > > > > +static uint64_t get_value(AVIOContext *pb, int type, int
> > > > > type2_size)
> > > > > > > {
> > > > > > > switch (type) {
> > > > > > > case 2:
> > > > > > > @@ -568,9 +568,9 @@ static int
> > > > > asf_read_ext_content_desc(AVFormatContext *s, int64_t size)
> > > > > > > * ASF stream count starts at 1. I am using 0 to
> the
> > > > > container value
> > > > > > > * since it's unused. */
> > > > > > > if (!strcmp(name, "AspectRatioX"))
> > > > > > > -asf->dar[0].num = get_value(s->pb,
> value_type,
> > > 32);
> > > > > > > +asf->dar[0].num = (int)get_value(s->pb,
> > > value_type,
> > > > > 32);
> > > > > > > else if (!strcmp(name, "AspectRatioY"))
> > > > > > > -asf->dar[0].den = get_value(s->pb,
> value_type,
> > > 32);
> > > > > > > +asf->dar[0].den = (int)get_value(s->pb,
> > > value_type,
> > > > > 32);
> > > > > > > else
> > > > > > > get_tag(s, name, value_type, value_len, 32);
> > > > > > > }
> > > > > > > @@ -630,11 +630,11 @@ static int
> > > > > asf_read_metadata(AVFormatContext *s, int64_t size)
> > > > > > > i, stream_num
Re: [FFmpeg-devel] [PATCH v2 2/2] ffprobe: Fix incorrect display of closed_captions property
> -Original Message- > From: ffmpeg-devel On Behalf Of > Soft Works > Sent: Monday, 20 September 2021 16:58 > To: FFmpeg development discussions and patches de...@ffmpeg.org>; James Almer > Subject: Re: [FFmpeg-devel] [PATCH v2 2/2] ffprobe: Fix incorrect > display of closed_captions property > > > > > -Original Message- > > From: ffmpeg-devel On Behalf Of > Anton > > Khirnov > > Sent: Monday, 20 September 2021 13:07 > > To: James Almer ; ffmpeg-devel@ffmpeg.org > > Subject: Re: [FFmpeg-devel] [PATCH v2 2/2] ffprobe: Fix incorrect > display of > > closed_captions property Hi James, hi Anton, it seems we've got a bit stuck on this issue, so I would like to remind that I'm in no way insisting on a certain way for getting at the required information. This patch was just suggesting one possible way, but what I wouldn't find acceptable is when you'd want to say that there's no acceptable way at all. The requirements are simple and clear: - to get the results of the avformat_find_stream_info() run - without repeating or replicating this procedure in ffprobe and without performing additional decoding in ffprobe I'm pretty sure that when any of you had these requirements as part of your own work, you would figure out a way to do it while adhering to them. As such it would be nice when you would be able to make a suggestion for resolving this. Also, let's not forget that this is not a private requirement that I have. It's about ffprobe. And its output with regard to closed captions is currently wrong. Thanks and kind regards, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2 1/2] avcodec/codec_par: Add codec properties field to AVCodecParameters
Signed-off-by: softworkz
---
v2: include codec properties in AVCodecParameters instead of adding an API to
retrieve it
doc/APIchanges | 3 +++
libavcodec/codec_par.c | 2 ++
libavcodec/codec_par.h | 4
libavcodec/version.h | 2 +-
4 files changed, 10 insertions(+), 1 deletion(-)
diff --git a/doc/APIchanges b/doc/APIchanges
index 7b267a79ac..2be3303efa 100644
--- a/doc/APIchanges
+++ b/doc/APIchanges
@@ -14,6 +14,9 @@ libavutil: 2021-04-27
API changes, most recent first:
+2021-10-07 - xx - lavc 59.11.100 - codec_par.h
+ Add codec properties field to AVCodecParameters
+
2021-09-21 - xx - lavu 57.7.100 - pixfmt.h
Add AV_PIX_FMT_X2BGR10.
diff --git a/libavcodec/codec_par.c b/libavcodec/codec_par.c
index 1a5168a04b..f6c13f7d11 100644
--- a/libavcodec/codec_par.c
+++ b/libavcodec/codec_par.c
@@ -101,6 +101,7 @@ int avcodec_parameters_from_context(AVCodecParameters *par,
par->bits_per_raw_sample = codec->bits_per_raw_sample;
par->profile = codec->profile;
par->level = codec->level;
+par->properties= codec->properties;
switch (par->codec_type) {
case AVMEDIA_TYPE_VIDEO:
@@ -156,6 +157,7 @@ int avcodec_parameters_to_context(AVCodecContext *codec,
codec->bits_per_raw_sample = par->bits_per_raw_sample;
codec->profile = par->profile;
codec->level = par->level;
+codec->properties= par->properties;
switch (par->codec_type) {
case AVMEDIA_TYPE_VIDEO:
diff --git a/libavcodec/codec_par.h b/libavcodec/codec_par.h
index 10cf79dff1..42ed8deb13 100644
--- a/libavcodec/codec_par.h
+++ b/libavcodec/codec_par.h
@@ -198,6 +198,10 @@ typedef struct AVCodecParameters {
* Audio only. Number of samples to skip after a discontinuity.
*/
int seek_preroll;
+/**
+ * Codec properties of the stream that gets decoded
+ */
+unsigned properties;
} AVCodecParameters;
/**
diff --git a/libavcodec/version.h b/libavcodec/version.h
index 4bd22f7e93..1c28fd0be5 100644
--- a/libavcodec/version.h
+++ b/libavcodec/version.h
@@ -28,7 +28,7 @@
#include "libavutil/version.h"
#define LIBAVCODEC_VERSION_MAJOR 59
-#define LIBAVCODEC_VERSION_MINOR 10
+#define LIBAVCODEC_VERSION_MINOR 11
#define LIBAVCODEC_VERSION_MICRO 100
#define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2 2/2] ffprobe: Fix incorrect display of closed_captions property
Repro Example:
ffprobe -show_entries stream=closed_captions:disposition=:side_data=
"http://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts";
While the codec string includes "Closed Captions",
the stream data is showing: closed_captions=0
The test ref was incorrect as the test media file
actually does have cc.
Signed-off-by: softworkz
---
fftools/ffprobe.c | 4 ++--
tests/ref/fate/ts-demux | 2 +-
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/fftools/ffprobe.c b/fftools/ffprobe.c
index 90e895bbf9..f5c6335a13 100644
--- a/fftools/ffprobe.c
+++ b/fftools/ffprobe.c
@@ -2655,9 +2655,9 @@ static int show_stream(WriterContext *w, AVFormatContext
*fmt_ctx, int stream_id
if (dec_ctx) {
print_int("coded_width", dec_ctx->coded_width);
print_int("coded_height", dec_ctx->coded_height);
-print_int("closed_captions", !!(dec_ctx->properties &
FF_CODEC_PROPERTY_CLOSED_CAPTIONS));
-print_int("film_grain", !!(dec_ctx->properties &
FF_CODEC_PROPERTY_FILM_GRAIN));
}
+print_int("closed_captions", !!(par->properties &
FF_CODEC_PROPERTY_CLOSED_CAPTIONS));
+print_int("film_grain", !!(par->properties &
FF_CODEC_PROPERTY_FILM_GRAIN));
print_int("has_b_frames", par->video_delay);
sar = av_guess_sample_aspect_ratio(fmt_ctx, stream, NULL);
if (sar.num) {
diff --git a/tests/ref/fate/ts-demux b/tests/ref/fate/ts-demux
index 8e7a81da41..1d1382cf37 100644
--- a/tests/ref/fate/ts-demux
+++ b/tests/ref/fate/ts-demux
@@ -41,7 +41,7 @@
packet|codec_type=audio|stream_index=2|pts=3912642700|pts_time=43473.807778|dts=
packet|codec_type=video|stream_index=0|pts=3912686363|pts_time=43474.292922|dts=3912686363|dts_time=43474.292922|duration=1501|duration_time=0.016678|size=4944|pos=506660|flags=__|data_hash=CRC32:54a86cbb
packet|codec_type=audio|stream_index=1|pts=3912644825|pts_time=43473.831389|dts=3912644825|dts_time=43473.831389|duration=2880|duration_time=0.032000|size=906|pos=474888|flags=K_|data_hash=CRC32:0893d398
packet|codec_type=audio|stream_index=2|pts=3912645580|pts_time=43473.839778|dts=3912645580|dts_time=43473.839778|duration=2880|duration_time=0.032000|size=354|pos=491808|flags=K_|data_hash=CRC32:f5963fa6
-stream|index=0|codec_name=mpeg2video|profile=4|codec_type=video|codec_tag_string=[2][0][0][0]|codec_tag=0x0002|width=1280|height=720|coded_width=0|coded_height=0|closed_captions=0|film_grain=0|has_b_frames=1|sample_aspect_ratio=1:1|display_aspect_ratio=16:9|pix_fmt=yuv420p|level=4|color_range=tv|color_space=unknown|color_transfer=unknown|color_primaries=unknown|chroma_location=left|field_order=progressive|refs=1|id=0x31|r_frame_rate=6/1001|avg_frame_rate=6/1001|time_base=1/9|start_pts=3912669846|start_time=43474.109400|duration_ts=19519|duration=0.216878|bit_rate=1500|max_bit_rate=N/A|bits_per_raw_sample=N/A|nb_frames=N/A|nb_read_frames=N/A|nb_read_packets=15|extradata_hash=CRC32:53134fa8|disposition:default=0|disposition:dub=0|disposition:original=0|disposition:comment=0|disposition:lyrics=0|disposition:karaoke=0|disposition:forced=0|disposition:hearing_impaired=0|disposition:visual_impaired=0|disposition:clean_effects=0|disposition:attached_pic=0|disposition:timed_
thumbnails=0|disposition:captions=0|disposition:descriptions=0|disposition:metadata=0|disposition:dependent=0|disposition:still_image=0
+stream|index=0|codec_name=mpeg2video|profile=4|codec_type=video|codec_tag_string=[2][0][0][0]|codec_tag=0x0002|width=1280|height=720|coded_width=0|coded_height=0|closed_captions=1|film_grain=0|has_b_frames=1|sample_aspect_ratio=1:1|display_aspect_ratio=16:9|pix_fmt=yuv420p|level=4|color_range=tv|color_space=unknown|color_transfer=unknown|color_primaries=unknown|chroma_location=left|field_order=progressive|refs=1|id=0x31|r_frame_rate=6/1001|avg_frame_rate=6/1001|time_base=1/9|start_pts=3912669846|start_time=43474.109400|duration_ts=19519|duration=0.216878|bit_rate=1500|max_bit_rate=N/A|bits_per_raw_sample=N/A|nb_frames=N/A|nb_read_frames=N/A|nb_read_packets=15|extradata_hash=CRC32:53134fa8|disposition:default=0|disposition:dub=0|disposition:original=0|disposition:comment=0|disposition:lyrics=0|disposition:karaoke=0|disposition:forced=0|disposition:hearing_impaired=0|disposition:visual_impaired=0|disposition:clean_effects=0|disposition:attached_pic=0|disposition:timed_
thumbnails=0|disposition:captions=0|disposition:descriptions=0|disposition:metadata=0|disposition:dependent=0|disposition:still_image=0
side_data|side_data_type=CPB
properties|max_bitrate=1500|min_bitrate=0|avg_bitrate=0|buffer_size=9781248|vbv_delay=-1
stream|index=1|codec_name=ac3|profile=unknown|codec_type=audio|codec_tag_string=[4][0][0][0]|codec_tag=0x0004|sample_fmt=fltp|sample_rate=48000|channels=6|channel_layout=5.1(side)|bits_per_sample=0|id=0x34|r_frame_rate=0/0|avg_frame_rate=0/0|time_base=1/9|start_pts=3912633305|start_time=43473.703389|duration_ts=14
[FFmpeg-devel] [PATCH v6 01/11] libavformat/asf: Fix handling of byte array length values
The spec allows attachment sizes of up to UINT32_MAX while
we can handle only sizes up to INT32_MAX (in downstream
code).
The debug.assert in get_tag didn't really address this,
and truncating the value_len in calling methods cannot
be used because the length value is required in order to
continue parsing. This adds a check with log message in
ff_asf_handle_byte_array to handle those (rare) cases.
Signed-off-by: softworkz
---
libavformat/asf.c | 12 +---
libavformat/asf.h | 2 +-
2 files changed, 10 insertions(+), 4 deletions(-)
diff --git a/libavformat/asf.c b/libavformat/asf.c
index 1ac8b5f078..179b66a2b4 100644
--- a/libavformat/asf.c
+++ b/libavformat/asf.c
@@ -267,12 +267,18 @@ static int get_id3_tag(AVFormatContext *s, int len)
}
int ff_asf_handle_byte_array(AVFormatContext *s, const char *name,
- int val_len)
+ uint32_t val_len)
{
+if (val_len > INT32_MAX) {
+av_log(s, AV_LOG_VERBOSE, "Unable to handle byte arrays > INT32_MAX
in tag %s.\n", name);
+return 1;
+}
+
if (!strcmp(name, "WM/Picture")) // handle cover art
-return asf_read_picture(s, val_len);
+return asf_read_picture(s, (int)val_len);
else if (!strcmp(name, "ID3")) // handle ID3 tag
-return get_id3_tag(s, val_len);
+return get_id3_tag(s, (int)val_len);
+av_log(s, AV_LOG_VERBOSE, "Unsupported byte array in tag %s.\n", name);
return 1;
}
diff --git a/libavformat/asf.h b/libavformat/asf.h
index 01cc4f7a46..4d28560f56 100644
--- a/libavformat/asf.h
+++ b/libavformat/asf.h
@@ -111,7 +111,7 @@ extern const AVMetadataConv ff_asf_metadata_conv[];
* is unsupported by this function and 0 otherwise.
*/
int ff_asf_handle_byte_array(AVFormatContext *s, const char *name,
- int val_len);
+ uint32_t val_len);
#define ASF_PACKET_FLAG_ERROR_CORRECTION_PRESENT 0x80 //1000
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v6 02/11] libavformat/asfdec: Fix get_value return type and add checks for unsupported values
get_value had a return type of int, which means that reading
QWORDS (case 4) was broken due to truncation of the result from
avio_rl64().
Signed-off-by: softworkz
---
libavformat/asfdec_f.c | 38 +-
1 file changed, 29 insertions(+), 9 deletions(-)
diff --git a/libavformat/asfdec_f.c b/libavformat/asfdec_f.c
index a8f36ed286..d31e1d581d 100644
--- a/libavformat/asfdec_f.c
+++ b/libavformat/asfdec_f.c
@@ -202,7 +202,7 @@ static int asf_probe(const AVProbeData *pd)
/* size of type 2 (BOOL) is 32bit for "Extended Content Description Object"
* but 16 bit for "Metadata Object" and "Metadata Library Object" */
-static int get_value(AVIOContext *pb, int type, int type2_size)
+static uint64_t get_value(AVIOContext *pb, int type, int type2_size)
{
switch (type) {
case ASF_BOOL:
@@ -567,10 +567,22 @@ static int asf_read_ext_content_desc(AVFormatContext *s,
int64_t size)
/* My sample has that stream set to 0 maybe that mean the container.
* ASF stream count starts at 1. I am using 0 to the container value
* since it's unused. */
-if (!strcmp(name, "AspectRatioX"))
-asf->dar[0].num = get_value(s->pb, value_type, 32);
-else if (!strcmp(name, "AspectRatioY"))
-asf->dar[0].den = get_value(s->pb, value_type, 32);
+if (!strcmp(name, "AspectRatioX")) {
+const uint64_t value = get_value(s->pb, value_type, 32);
+if (value > INT32_MAX) {
+av_log(s, AV_LOG_DEBUG, "Unsupported AspectRatioX value:
%"PRIu64"\n", value);
+return AVERROR(ENOTSUP);
+}
+asf->dar[0].num = (int)value;
+}
+else if (!strcmp(name, "AspectRatioY")) {
+const uint64_t value = get_value(s->pb, value_type, 32);
+if (value > INT32_MAX) {
+av_log(s, AV_LOG_DEBUG, "Unsupported AspectRatioY value:
%"PRIu64"\n", value);
+return AVERROR(ENOTSUP);
+}
+asf->dar[0].den = (int)value;
+}
else
get_tag(s, name, value_type, value_len, 32);
}
@@ -630,13 +642,21 @@ static int asf_read_metadata(AVFormatContext *s, int64_t
size)
i, stream_num, name_len_utf16, value_type, value_len, name);
if (!strcmp(name, "AspectRatioX")){
-int aspect_x = get_value(s->pb, value_type, 16);
+const uint64_t aspect_x = get_value(s->pb, value_type, 16);
+if (aspect_x > INT32_MAX) {
+av_log(s, AV_LOG_DEBUG, "Unsupported AspectRatioX value:
%"PRIu64"\n", aspect_x);
+return AVERROR(ENOTSUP);
+}
if(stream_num < 128)
-asf->dar[stream_num].num = aspect_x;
+asf->dar[stream_num].num = (int)aspect_x;
} else if(!strcmp(name, "AspectRatioY")){
-int aspect_y = get_value(s->pb, value_type, 16);
+const uint64_t aspect_y = get_value(s->pb, value_type, 16);
+if (aspect_y > INT32_MAX) {
+av_log(s, AV_LOG_DEBUG, "Unsupported AspectRatioY value:
%"PRIu64"\n", aspect_y);
+return AVERROR(ENOTSUP);
+}
if(stream_num < 128)
-asf->dar[stream_num].den = aspect_y;
+asf->dar[stream_num].den = (int)aspect_y;
} else {
get_tag(s, name, value_type, value_len, 16);
}
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v6 03/11] libavformat/asfdec: Fix type of value_len
The value_len is an uint32 not an int32 per spec. That
value must not be truncated, neither by casting to int, nor by any
conditional checks, because at the end of get_tag, this value is
needed to move forward in parsing. When the len value gets
modified, the parsing may break.
Signed-off-by: softworkz
---
libavformat/asfdec_f.c | 24 +++-
1 file changed, 11 insertions(+), 13 deletions(-)
diff --git a/libavformat/asfdec_f.c b/libavformat/asfdec_f.c
index d31e1d581d..29b429fee9 100644
--- a/libavformat/asfdec_f.c
+++ b/libavformat/asfdec_f.c
@@ -218,7 +218,7 @@ static uint64_t get_value(AVIOContext *pb, int type, int
type2_size)
}
}
-static void get_tag(AVFormatContext *s, const char *key, int type, int len,
int type2_size)
+static void get_tag(AVFormatContext *s, const char *key, int type, uint32_t
len, int type2_size)
{
ASFContext *asf = s->priv_data;
char *value = NULL;
@@ -528,7 +528,7 @@ static int asf_read_ext_stream_properties(AVFormatContext
*s, int64_t size)
static int asf_read_content_desc(AVFormatContext *s, int64_t size)
{
AVIOContext *pb = s->pb;
-int len1, len2, len3, len4, len5;
+uint32_t len1, len2, len3, len4, len5;
len1 = avio_rl16(pb);
len2 = avio_rl16(pb);
@@ -614,25 +614,23 @@ static int asf_read_metadata(AVFormatContext *s, int64_t
size)
{
AVIOContext *pb = s->pb;
ASFContext *asf = s->priv_data;
-int n, stream_num, name_len_utf16, name_len_utf8, value_len;
+int n, name_len_utf8;
+uint16_t stream_num, name_len_utf16, value_type;
+uint32_t value_len;
int ret, i;
n = avio_rl16(pb);
for (i = 0; i < n; i++) {
uint8_t *name;
-int value_type;
avio_rl16(pb); // lang_list_index
-stream_num = avio_rl16(pb);
-name_len_utf16 = avio_rl16(pb);
-value_type = avio_rl16(pb); /* value_type */
-value_len = avio_rl32(pb);
+stream_num = (uint16_t)avio_rl16(pb);
+name_len_utf16 = (uint16_t)avio_rl16(pb);
+value_type = (uint16_t)avio_rl16(pb); /* value_type */
+value_len = avio_rl32(pb);
-if (value_len < 0 || value_len > UINT16_MAX)
-return AVERROR_INVALIDDATA;
-
-name_len_utf8 = 2*name_len_utf16 + 1;
-name = av_malloc(name_len_utf8);
+name_len_utf8 = 2 * name_len_utf16 + 1;
+name = av_malloc(name_len_utf8);
if (!name)
return AVERROR(ENOMEM);
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v6 04/11] libavformat/asfdec: Fixing get_tag
These three are closely related and can't be separated easily:
In get_tag, the code was adding 22 bytes (in order to allow
it to hold 64bit numbers as string) to the value len for creating
creating a buffer. This was unnecessarily imposing a
size-constraint on the value_len parameter.
The code in get_tag, was limiting the maximum value_len to
half the size of INT32. This was applied for all value types, even
though it is required only in case of ASF_UNICODE, not for any
other ones (like ASCII).
get_tag was always allocating a buffer regardless of the
datatype, even though this isn't required in case of ASF_BYTE_ARRAY
The check for the return value from ff_asf_handle_byte_array()
being >0 is removed here because the log message is emitted
by the function itself now.
Signed-off-by: softworkz
---
libavformat/asfdec_f.c | 54 +++---
1 file changed, 40 insertions(+), 14 deletions(-)
diff --git a/libavformat/asfdec_f.c b/libavformat/asfdec_f.c
index 29b429fee9..58c424b565 100644
--- a/libavformat/asfdec_f.c
+++ b/libavformat/asfdec_f.c
@@ -221,37 +221,63 @@ static uint64_t get_value(AVIOContext *pb, int type, int
type2_size)
static void get_tag(AVFormatContext *s, const char *key, int type, uint32_t
len, int type2_size)
{
ASFContext *asf = s->priv_data;
-char *value = NULL;
int64_t off = avio_tell(s->pb);
-#define LEN 22
-
-av_assert0((unsigned)len < (INT_MAX - LEN) / 2);
+char *value = NULL;
+uint64_t required_bufferlen;
+int buffer_len;
if (!asf->export_xmp && !strncmp(key, "xmp", 3))
goto finish;
-value = av_malloc(2 * len + LEN);
+switch (type) {
+case ASF_UNICODE:
+required_bufferlen = (uint64_t)len * 2 + 1;
+break;
+case -1: // ASCII
+required_bufferlen = (uint64_t)len + 1;
+break;
+case ASF_BYTE_ARRAY:
+ff_asf_handle_byte_array(s, key, len);
+goto finish;
+case ASF_BOOL:
+case ASF_DWORD:
+case ASF_QWORD:
+case ASF_WORD:
+required_bufferlen = 22;
+break;
+case ASF_GUID:
+required_bufferlen = 33;
+break;
+default:
+required_bufferlen = len;
+break;
+}
+
+if (required_bufferlen > INT32_MAX) {
+av_log(s, AV_LOG_VERBOSE, "Unable to handle values > INT32_MAX in tag
%s.\n", key);
+goto finish;
+}
+
+buffer_len = (int)required_bufferlen;
+
+value = av_malloc(buffer_len);
if (!value)
goto finish;
switch (type) {
case ASF_UNICODE:
-avio_get_str16le(s->pb, len, value, 2 * len + 1);
+avio_get_str16le(s->pb, len, value, buffer_len);
break;
-case -1: // ASCI
-avio_read(s->pb, value, len);
-value[len]=0;
+case -1: // ASCII
+avio_read(s->pb, value, buffer_len - 1);
+value[buffer_len - 1] = 0;
break;
-case ASF_BYTE_ARRAY:
-if (ff_asf_handle_byte_array(s, key, len) > 0)
-av_log(s, AV_LOG_VERBOSE, "Unsupported byte array in tag %s.\n",
key);
-goto finish;
case ASF_BOOL:
case ASF_DWORD:
case ASF_QWORD:
case ASF_WORD: {
uint64_t num = get_value(s->pb, type, type2_size);
-snprintf(value, LEN, "%"PRIu64, num);
+snprintf(value, buffer_len, "%"PRIu64, num);
break;
}
case ASF_GUID:
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v6 05/11] libavformat/asfdec: Implement parsing of GUID values
Signed-off-by: softworkz
---
libavformat/asfdec_f.c | 9 ++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/libavformat/asfdec_f.c b/libavformat/asfdec_f.c
index 58c424b565..4c898ab3f2 100644
--- a/libavformat/asfdec_f.c
+++ b/libavformat/asfdec_f.c
@@ -280,9 +280,12 @@ static void get_tag(AVFormatContext *s, const char *key,
int type, uint32_t len,
snprintf(value, buffer_len, "%"PRIu64, num);
break;
}
-case ASF_GUID:
-av_log(s, AV_LOG_DEBUG, "Unsupported GUID value in tag %s.\n", key);
-goto finish;
+case ASF_GUID: {
+ff_asf_guid g;
+ff_get_guid(s->pb, &g);
+snprintf(value, buffer_len, "%x", g[0]);
+break;
+}
default:
av_log(s, AV_LOG_DEBUG,
"Unsupported value type %d in tag %s.\n", type, key);
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v6 06/11] libavformat/asfdec: Remove unused parameters
Signed-off-by: softworkz
---
libavformat/asfdec_f.c | 30 +++---
1 file changed, 15 insertions(+), 15 deletions(-)
diff --git a/libavformat/asfdec_f.c b/libavformat/asfdec_f.c
index 4c898ab3f2..e87c78cd6c 100644
--- a/libavformat/asfdec_f.c
+++ b/libavformat/asfdec_f.c
@@ -299,7 +299,7 @@ finish:
avio_seek(s->pb, off + len, SEEK_SET);
}
-static int asf_read_file_properties(AVFormatContext *s, int64_t size)
+static int asf_read_file_properties(AVFormatContext *s)
{
ASFContext *asf = s->priv_data;
AVIOContext *pb = s->pb;
@@ -494,7 +494,7 @@ static int asf_read_stream_properties(AVFormatContext *s,
int64_t size)
return 0;
}
-static int asf_read_ext_stream_properties(AVFormatContext *s, int64_t size)
+static int asf_read_ext_stream_properties(AVFormatContext *s)
{
ASFContext *asf = s->priv_data;
AVIOContext *pb = s->pb;
@@ -554,7 +554,7 @@ static int asf_read_ext_stream_properties(AVFormatContext
*s, int64_t size)
return 0;
}
-static int asf_read_content_desc(AVFormatContext *s, int64_t size)
+static int asf_read_content_desc(AVFormatContext *s)
{
AVIOContext *pb = s->pb;
uint32_t len1, len2, len3, len4, len5;
@@ -573,7 +573,7 @@ static int asf_read_content_desc(AVFormatContext *s,
int64_t size)
return 0;
}
-static int asf_read_ext_content_desc(AVFormatContext *s, int64_t size)
+static int asf_read_ext_content_desc(AVFormatContext *s)
{
AVIOContext *pb = s->pb;
ASFContext *asf = s->priv_data;
@@ -619,7 +619,7 @@ static int asf_read_ext_content_desc(AVFormatContext *s,
int64_t size)
return 0;
}
-static int asf_read_language_list(AVFormatContext *s, int64_t size)
+static int asf_read_language_list(AVFormatContext *s)
{
AVIOContext *pb = s->pb;
ASFContext *asf = s->priv_data;
@@ -639,7 +639,7 @@ static int asf_read_language_list(AVFormatContext *s,
int64_t size)
return 0;
}
-static int asf_read_metadata(AVFormatContext *s, int64_t size)
+static int asf_read_metadata(AVFormatContext *s)
{
AVIOContext *pb = s->pb;
ASFContext *asf = s->priv_data;
@@ -693,7 +693,7 @@ static int asf_read_metadata(AVFormatContext *s, int64_t
size)
return 0;
}
-static int asf_read_marker(AVFormatContext *s, int64_t size)
+static int asf_read_marker(AVFormatContext *s)
{
AVIOContext *pb = s->pb;
ASFContext *asf = s->priv_data;
@@ -772,21 +772,21 @@ static int asf_read_header(AVFormatContext *s)
if (gsize < 24)
return AVERROR_INVALIDDATA;
if (!ff_guidcmp(&g, &ff_asf_file_header)) {
-ret = asf_read_file_properties(s, gsize);
+ret = asf_read_file_properties(s);
} else if (!ff_guidcmp(&g, &ff_asf_stream_header)) {
ret = asf_read_stream_properties(s, gsize);
} else if (!ff_guidcmp(&g, &ff_asf_comment_header)) {
-asf_read_content_desc(s, gsize);
+asf_read_content_desc(s);
} else if (!ff_guidcmp(&g, &ff_asf_language_guid)) {
-asf_read_language_list(s, gsize);
+asf_read_language_list(s);
} else if (!ff_guidcmp(&g, &ff_asf_extended_content_header)) {
-asf_read_ext_content_desc(s, gsize);
+asf_read_ext_content_desc(s);
} else if (!ff_guidcmp(&g, &ff_asf_metadata_header)) {
-asf_read_metadata(s, gsize);
+asf_read_metadata(s);
} else if (!ff_guidcmp(&g, &ff_asf_metadata_library_header)) {
-asf_read_metadata(s, gsize);
+asf_read_metadata(s);
} else if (!ff_guidcmp(&g, &ff_asf_ext_stream_header)) {
-asf_read_ext_stream_properties(s, gsize);
+asf_read_ext_stream_properties(s);
// there could be an optional stream properties object to follow
// if so the next iteration will pick it up
@@ -796,7 +796,7 @@ static int asf_read_header(AVFormatContext *s)
avio_skip(pb, 6);
continue;
} else if (!ff_guidcmp(&g, &ff_asf_marker_header)) {
-asf_read_marker(s, gsize);
+asf_read_marker(s);
} else if (avio_feof(pb)) {
return AVERROR_EOF;
} else {
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v6 07/11] libavformat/asfdec: Fix macro definition and use
Signed-off-by: softworkz
---
libavformat/asfdec_f.c | 24
1 file changed, 12 insertions(+), 12 deletions(-)
diff --git a/libavformat/asfdec_f.c b/libavformat/asfdec_f.c
index e87c78cd6c..a7b5ffe465 100644
--- a/libavformat/asfdec_f.c
+++ b/libavformat/asfdec_f.c
@@ -896,21 +896,21 @@ static int asf_read_header(AVFormatContext *s)
}
#define DO_2BITS(bits, var, defval) \
-switch (bits & 3) { \
+switch ((bits) & 3) { \
case 3: \
-var = avio_rl32(pb);\
+(var) = avio_rl32(pb); \
rsize += 4; \
break; \
case 2: \
-var = avio_rl16(pb);\
+(var) = avio_rl16(pb); \
rsize += 2; \
break; \
case 1: \
-var = avio_r8(pb); \
+(var) = avio_r8(pb);\
rsize++;\
break; \
default:\
-var = defval; \
+(var) = (defval); \
break; \
}
@@ -993,9 +993,9 @@ static int asf_get_packet(AVFormatContext *s, AVIOContext
*pb)
asf->packet_flags= c;
asf->packet_property = d;
-DO_2BITS(asf->packet_flags >> 5, packet_length, s->packet_size);
-DO_2BITS(asf->packet_flags >> 1, padsize, 0); // sequence ignored
-DO_2BITS(asf->packet_flags >> 3, padsize, 0); // padding length
+DO_2BITS(asf->packet_flags >> 5, packet_length, s->packet_size)
+DO_2BITS(asf->packet_flags >> 1, padsize, 0) // sequence ignored
+DO_2BITS(asf->packet_flags >> 3, padsize, 0) // padding length
// the following checks prevent overflows and infinite loops
if (!packet_length || packet_length >= (1U << 29)) {
@@ -1056,9 +1056,9 @@ static int asf_read_frame_header(AVFormatContext *s,
AVIOContext *pb)
asf->stream_index = asf->asfid2avid[num & 0x7f];
asfst = &asf->streams[num & 0x7f];
// sequence should be ignored!
-DO_2BITS(asf->packet_property >> 4, asf->packet_seq, 0);
-DO_2BITS(asf->packet_property >> 2, asf->packet_frag_offset, 0);
-DO_2BITS(asf->packet_property, asf->packet_replic_size, 0);
+DO_2BITS(asf->packet_property >> 4, asf->packet_seq, 0)
+DO_2BITS(asf->packet_property >> 2, asf->packet_frag_offset, 0)
+DO_2BITS(asf->packet_property, asf->packet_replic_size, 0)
av_log(asf, AV_LOG_TRACE, "key:%d stream:%d seq:%d offset:%d
replic_size:%d num:%X packet_property %X\n",
asf->packet_key_frame, asf->stream_index, asf->packet_seq,
asf->packet_frag_offset, asf->packet_replic_size, num,
asf->packet_property);
@@ -1134,7 +1134,7 @@ static int asf_read_frame_header(AVFormatContext *s,
AVIOContext *pb)
return AVERROR_INVALIDDATA;
}
if (asf->packet_flags & 0x01) {
-DO_2BITS(asf->packet_segsizetype >> 6, asf->packet_frag_size, 0); // 0
is illegal
+DO_2BITS(asf->packet_segsizetype >> 6, asf->packet_frag_size, 0) // 0
is illegal
if (rsize > asf->packet_size_left) {
av_log(s, AV_LOG_ERROR, "packet_replic_size is invalid\n");
return AVERROR_INVALIDDATA;
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v6 08/11] libavformat/asfdec: Remove variable redefinition in inner scope
Signed-off-by: softworkz
---
libavformat/asfdec_f.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/libavformat/asfdec_f.c b/libavformat/asfdec_f.c
index a7b5ffe465..8283f245ab 100644
--- a/libavformat/asfdec_f.c
+++ b/libavformat/asfdec_f.c
@@ -1181,7 +1181,7 @@ static int asf_parse_packet(AVFormatContext *s,
AVIOContext *pb, AVPacket *pkt)
return AVERROR_EOF;
if (asf->packet_size_left < FRAME_HEADER_SIZE ||
asf->packet_segments < 1 && asf->packet_time_start == 0) {
-int ret = asf->packet_size_left + asf->packet_padsize;
+ret = asf->packet_size_left + asf->packet_padsize;
if (asf->packet_size_left && asf->packet_size_left <
FRAME_HEADER_SIZE)
av_log(s, AV_LOG_WARNING, "Skip due to FRAME_HEADER_SIZE\n");
@@ -1250,7 +1250,6 @@ static int asf_parse_packet(AVFormatContext *s,
AVIOContext *pb, AVPacket *pkt)
if (asf_st->pkt.size != asf_st->packet_obj_size ||
// FIXME is this condition sufficient?
asf_st->frag_offset + asf->packet_frag_size > asf_st->pkt.size) {
-int ret;
if (asf_st->pkt.data) {
av_log(s, AV_LOG_INFO,
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v6 09/11] libavformat/asfdec: Ensure variables are initialized
Signed-off-by: softworkz
---
libavformat/asfdec_f.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/libavformat/asfdec_f.c b/libavformat/asfdec_f.c
index 8283f245ab..024d77903b 100644
--- a/libavformat/asfdec_f.c
+++ b/libavformat/asfdec_f.c
@@ -968,6 +968,7 @@ static int asf_get_packet(AVFormatContext *s, AVIOContext
*pb)
avio_seek(pb, -1, SEEK_CUR); // FIXME
}
} else {
+d = e = 0;
c = avio_r8(pb);
if (c & 0x80) {
rsize ++;
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
