Re: [FFmpeg-devel] [PATCH v3 1/2] avcodec/codec_par: Add codec properties field to AVCodecParameters
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Hendrik Leppkes
> Sent: Wednesday, October 6, 2021 8:57 AM
> To: FFmpeg development discussions and patches de...@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v3 1/2] avcodec/codec_par: Add
> codec properties field to AVCodecParameters
>
> On Wed, Oct 6, 2021 at 8:45 AM Soft Works
> wrote:
> > diff --git a/libavcodec/codec_par.h b/libavcodec/codec_par.h
> > index 10cf79dff1..42ed8deb13 100644
> > --- a/libavcodec/codec_par.h
> > +++ b/libavcodec/codec_par.h
> > @@ -198,6 +198,10 @@ typedef struct AVCodecParameters {
> > * Audio only. Number of samples to skip after a
> discontinuity.
> > */
> > int seek_preroll;
> > +/**
> > + * Codec properties of the stream that gets decoded
> > + */
> > +unsigned properties;
> > } AVCodecParameters;
> >
>
> This field is severly underspecified/underdocumented. I realize you
> just copied it, but if I'm looking at this without pre-existing
> knowledge, then I have absolutely no idea what it might be for, or
> what kind of values go in there. The old field had the defines of the
> possible bits right there for context - this doesn't have this, so it
> would definitely benefit from added documentation.
Hello Hendrik,
that's right, but what would you suggest? We can't duplicate the defines,
though I could add a line in the help text like
/**
* Codec properties of the stream that gets decoded.
* Corresponds to @ref AVCodecContext.properties "properties".
*/
unsigned properties;
Would that be OK?
Thanks,
sw
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v3 1/2] avcodec/codec_par: Add codec properties field to AVCodecParameters
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Soft Works
> Sent: Wednesday, October 6, 2021 8:45 AM
> To: ffmpeg-devel@ffmpeg.org
> Subject: [FFmpeg-devel] [PATCH v3 1/2] avcodec/codec_par: Add codec
> properties field to AVCodecParameters
>
> This fixes incorrect display of closed_captions property in ffprobe.
>
> Repro Example:
> ffprobe -show_entries stream=closed_captions:disposition=:side_data=
> "http://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts";
>
> While the codec string included "Closed Captions",
> the stream data is showed: closed_captions=0
>
> The test ref was incorrect as the test media file actually does
> have cc, which is fixed by this commit, so the test ref needs
> to be updated.
>
> Signed-off-by: softworkz
> ---
> v3: Moved test update to the right (this) commit
>
> doc/APIchanges | 3 +++
> libavcodec/codec_par.c | 2 ++
> libavcodec/codec_par.h | 4
> libavcodec/version.h| 2 +-
> tests/ref/fate/ts-demux | 2 +-
> 5 files changed, 11 insertions(+), 2 deletions(-)
>
> diff --git a/doc/APIchanges b/doc/APIchanges
> index 7b267a79ac..2be3303efa 100644
> --- a/doc/APIchanges
> +++ b/doc/APIchanges
> @@ -14,6 +14,9 @@ libavutil: 2021-04-27
>
> API changes, most recent first:
>
> +2021-10-07 - xx - lavc 59.11.100 - codec_par.h
> + Add codec properties field to AVCodecParameters
> +
> 2021-09-21 - xx - lavu 57.7.100 - pixfmt.h
>Add AV_PIX_FMT_X2BGR10.
>
> diff --git a/libavcodec/codec_par.c b/libavcodec/codec_par.c
> index 1a5168a04b..f6c13f7d11 100644
> --- a/libavcodec/codec_par.c
> +++ b/libavcodec/codec_par.c
> @@ -101,6 +101,7 @@ int
> avcodec_parameters_from_context(AVCodecParameters *par,
> par->bits_per_raw_sample = codec->bits_per_raw_sample;
> par->profile = codec->profile;
> par->level = codec->level;
> +par->properties= codec->properties;
>
> switch (par->codec_type) {
> case AVMEDIA_TYPE_VIDEO:
> @@ -156,6 +157,7 @@ int avcodec_parameters_to_context(AVCodecContext
> *codec,
> codec->bits_per_raw_sample = par->bits_per_raw_sample;
> codec->profile = par->profile;
> codec->level = par->level;
> +codec->properties= par->properties;
>
> switch (par->codec_type) {
> case AVMEDIA_TYPE_VIDEO:
> diff --git a/libavcodec/codec_par.h b/libavcodec/codec_par.h
> index 10cf79dff1..42ed8deb13 100644
> --- a/libavcodec/codec_par.h
> +++ b/libavcodec/codec_par.h
> @@ -198,6 +198,10 @@ typedef struct AVCodecParameters {
> * Audio only. Number of samples to skip after a discontinuity.
> */
> int seek_preroll;
> +/**
> + * Codec properties of the stream that gets decoded
> + */
> +unsigned properties;
> } AVCodecParameters;
>
> /**
> diff --git a/libavcodec/version.h b/libavcodec/version.h
> index 4bd22f7e93..1c28fd0be5 100644
> --- a/libavcodec/version.h
> +++ b/libavcodec/version.h
> @@ -28,7 +28,7 @@
> #include "libavutil/version.h"
>
> #define LIBAVCODEC_VERSION_MAJOR 59
> -#define LIBAVCODEC_VERSION_MINOR 10
> +#define LIBAVCODEC_VERSION_MINOR 11
> #define LIBAVCODEC_VERSION_MICRO 100
>
> #define LIBAVCODEC_VERSION_INT
> AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \
> diff --git a/tests/ref/fate/ts-demux b/tests/ref/fate/ts-demux
> index 8e7a81da41..1d1382cf37 100644
> --- a/tests/ref/fate/ts-demux
> +++ b/tests/ref/fate/ts-demux
> @@ -41,7 +41,7 @@
> packet|codec_type=audio|stream_index=2|pts=3912642700|pts_time=43473.
> 807778|dts=
>
> packet|codec_type=video|stream_index=0|pts=3912686363|pts_time=43474.
> 292922|dts=3912686363|dts_time=43474.292922|duration=1501|duration_ti
> me=0.016678|size=4944|pos=506660|flags=__|data_hash=CRC32:54a86cbb
>
> packet|codec_type=audio|stream_index=1|pts=3912644825|pts_time=43473.
> 831389|dts=3912644825|dts_time=43473.831389|duration=2880|duration_ti
> me=0.032000|size=906|pos=474888|flags=K_|data_hash=CRC32:0893d398
>
> packet|codec_type=audio|stream_index=2|pts=3912645580|pts_time=43473.
> 839778|dts=3912645580|dts_time=43473.839778|duration=2880|duration_ti
> me=0.032000|size=354|pos=491808|flags=K_|data_hash=CRC32:f5963fa6
> -
> stream|index=0|codec_name=mpeg2video|profile=4|codec_type=video|codec
> _tag_string=[2][0][0][0]|codec_tag=0x0002|width=1280|height=720|coded
> _width=0|coded_height=0|closed_captions=0|film_grain=0|has_b_frames=1
> |sample_aspect_ratio=1:1|display_aspect_ratio=16:9|pix_fmt=yuv420p|le
> vel=4|color_range=tv|color_space=unknown|color_transfer=unknown|color
> _primaries=unknown|chroma_location=left|field_order=progressive|refs=
> 1|id=0x31|r_frame_rate=6/1001|avg_frame_rate=6/1001|time_base
> =1/9|start_pts=3912669846|start_time=43474.109400|duration_ts=195
> 19|duration=0.216878|bit_rate=1500|max_bit_rate=N/A|bits_per_raw_
> sample=N/A|nb_frames=N/A|nb_read_frames=N/A|nb_read_packets=15|extrad
> ata_hash=CRC32:53134fa8|disposit
[FFmpeg-devel] [PATCH v12 00/14] Subtitle Filtering
v12 Update:
- FEATURE-COMPLETE
No more features or filters are needed for the introduction
of subtitle filtering
- Adapted patchset to the changes regarding .query_formats
in avfilter
- Removed underscores in filter names
- Expose all options from cc_dec from the splitcc filter
- Added 'render_latest_only' mode to overlaytextsubs
and textsubs2video filters (allows rendering closed captions
without black background rectangles)
v11 Update:
- FEATURE-COMPLETE
No more features or filters will be added to this
patchset
- Added subscale filter
Provides high-quality scaling and rearranging
functionality for graphical subtitles.
This has been the last missing bit for making this
a useful and versatile toolset for subtitle manipulation
- The common ass utility functions remain to reside
in avutil, but in this version, they are marked as
avpriv_ and no longer public like in the previous
version.
- A number of detail fixes are included, but there
haven't been significant changes to the core
implementation in the past few versions.
I'm quite confident about the current state.
- As a final step I'll be going to create a wiki page
presenting a comprehensive collection of usage examples,
showing off the whole range of new capabilities
and also providing proof that it is working universally
in a wide range of use cases.
--
This patchset introduces filtering support for subtitles.
Besides the ground work, this patchset includes a range
of new filters for subtitle processing:
- subscale (S -> S)
Provides high-quality scaling and rearranging
functionality for graphical subtitles.
- graphicsubs2text (S -> S)
Convert graphic subtitles to text subtitles via OCR
- censor {S -> S)
Allows censoring word in subtitles by a configurable
word list
- showspeaker {S -> S)
Prepend the speaker names to text lines
(those are often available in ass subs)
- splitcc (V -> VS)
Splits out Closed-Caption data as a subtitle stream
for filtering and encoding
- overlaygraphicsubs (VS -> V)
Overlay graphic subtitles onto a video stream
- graphicsub2video {S -> V)
Converts graphic subtitles to (transparent) video frames
- overlaytextsubs {VS -> V)
Overlay text subtitles onto a video stream.
- textsubs2video {S -> V)
Converts text subtitles to video frames
- textmod {S -> S)
Modify subtitle text in a number of ways
- stripstyles {S -> S)
Remove all inline styles from subtitle events
Regards,
softworkz
softworkz (14):
global: Prepare AVFrame for subtitle handling
global: Move ass helper functions to avutil as avpriv_ and extend ass
dialog parsing
fftools/play,probe: Adjust for subtitle changes
avfilter/subtitles: Add subtitles.c for subtitle frame allocation
avfilter/avfilter: Handle subtitle frames
avfilter/sbuffer: Add sbuffersrc and sbuffersink filters
avfilter/overlaygraphicsubs: Add overlaygraphicsubs and
graphicsub2video filters
fftools/ffmpeg: Replace sub2video with subtitle frame filtering
avfilter/overlaytextsubs: Add overlaytextsubs and textsubs2video
filters
avfilter/textmod: Add textmod, censor and show_speaker filters
avfilter/stripstyles: Add stripstyles filter
avfilter/splitcc: Add splitcc filter for closed caption handling
avfilter/graphicsub2text: Add new graphicsub2text filter (OCR)
avfilter/subscale: Add filter for scaling and/or re-arranging
graphical subtitles
configure | 7 +-
doc/filters.texi | 747 +++
fftools/ffmpeg.c | 509 +--
fftools/ffmpeg.h | 14 +-
fftools/ffmpeg_filter.c | 209 +++--
fftools/ffmpeg_hw.c | 2 +-
fftools/ffmpeg_opt.c | 3 +-
fftools/ffplay.c | 102 +--
fftools/ffprobe.c | 48 +-
libavcodec/Makefile | 56 +-
libavcodec/ass.h | 129 +--
libavcodec/assdec.c | 2 +-
libavcodec/assenc.c | 2 +-
libavcodec/avcodec.c | 19 -
libavcodec/avcodec.h | 75 +-
libavcodec/ccaption_dec.c | 19 +-
libavcodec/decode.c | 53 +-
libavcodec/jacosubdec.c | 2 +-
libavcodec/libaribb24.c | 2 +-
libavcodec/libzvbi-teletextdec.c | 14 +-
libavcodec/microdvddec.c | 7 +-
libavcodec/movtextdec.c | 3 +-
libavcodec/movtextenc.c | 20 +-
libavcodec/mpl2dec.c | 2 +-
libavcodec/pgssubdec.c| 1 +
libavcodec/realtextdec.c | 2 +-
lib
[FFmpeg-devel] [PATCH v12 01/14] global: Prepare AVFrame for subtitle handling
Root commit for adding subtitle filtering capabilities.
In detail:
- Add type (AVMediaType) field to AVFrame
Replaces previous way of distinction which was based on checking
width and height to determine whether a frame is audio or video
- Add subtitle fields to AVFrame
- Add new struct AVSubtitleArea, similar to AVSubtitleRect, but different
allocation logic. Cannot and must not be used interchangeably, hence
the new struct
- Move enum AVSubtitleType, AVSubtitle and AVSubtitleRect to avutil
- Add public-named members to enum AVSubtitleType (AV_SUBTITLE_FMT_)
- Add avcodec_decode_subtitle3 which takes subtitle frames,
serving as compatibility shim to legacy subtitle decoding
- Add additional methods for conversion between old and new API
Signed-off-by: softworkz
---
libavcodec/avcodec.c | 19 ---
libavcodec/avcodec.h | 75 ++--
libavcodec/decode.c| 53 ++--
libavcodec/pgssubdec.c | 1 +
libavcodec/utils.c | 11 ++
libavfilter/vf_subtitles.c | 50 ++--
libavformat/utils.c| 1 +
libavutil/Makefile | 2 +
libavutil/frame.c | 191 +
libavutil/frame.h | 93 +-
libavutil/subfmt.c | 240 +
libavutil/subfmt.h | 185
12 files changed, 796 insertions(+), 125 deletions(-)
create mode 100644 libavutil/subfmt.c
create mode 100644 libavutil/subfmt.h
diff --git a/libavcodec/avcodec.c b/libavcodec/avcodec.c
index ff3d73e237..52ce345d49 100644
--- a/libavcodec/avcodec.c
+++ b/libavcodec/avcodec.c
@@ -422,25 +422,6 @@ void avcodec_flush_buffers(AVCodecContext *avctx)
av_bsf_flush(avci->bsf);
}
-void avsubtitle_free(AVSubtitle *sub)
-{
-int i;
-
-for (i = 0; i < sub->num_rects; i++) {
-av_freep(&sub->rects[i]->data[0]);
-av_freep(&sub->rects[i]->data[1]);
-av_freep(&sub->rects[i]->data[2]);
-av_freep(&sub->rects[i]->data[3]);
-av_freep(&sub->rects[i]->text);
-av_freep(&sub->rects[i]->ass);
-av_freep(&sub->rects[i]);
-}
-
-av_freep(&sub->rects);
-
-memset(sub, 0, sizeof(*sub));
-}
-
av_cold int avcodec_close(AVCodecContext *avctx)
{
int i;
diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 7ee8bc2b7c..0c5819b116 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -35,6 +35,7 @@
#include "libavutil/frame.h"
#include "libavutil/log.h"
#include "libavutil/pixfmt.h"
+#include "libavutil/subfmt.h"
#include "libavutil/rational.h"
#include "codec.h"
@@ -1674,7 +1675,7 @@ typedef struct AVCodecContext {
/**
* Header containing style information for text subtitles.
- * For SUBTITLE_ASS subtitle type, it should contain the whole ASS
+ * For AV_SUBTITLE_FMT_ASS subtitle type, it should contain the whole ASS
* [Script Info] and [V4+ Styles] section, plus the [Events] line and
* the Format line following. It shouldn't include any Dialogue line.
* - encoding: Set/allocated/freed by user (before avcodec_open2())
@@ -2238,63 +2239,8 @@ typedef struct AVHWAccel {
* @}
*/
-enum AVSubtitleType {
-SUBTITLE_NONE,
-
-SUBTITLE_BITMAP,///< A bitmap, pict will be set
-
-/**
- * Plain text, the text field must be set by the decoder and is
- * authoritative. ass and pict fields may contain approximations.
- */
-SUBTITLE_TEXT,
-
-/**
- * Formatted text, the ass field must be set by the decoder and is
- * authoritative. pict and text fields may contain approximations.
- */
-SUBTITLE_ASS,
-};
-
#define AV_SUBTITLE_FLAG_FORCED 0x0001
-typedef struct AVSubtitleRect {
-int x; ///< top left corner of pict, undefined when pict is not
set
-int y; ///< top left corner of pict, undefined when pict is not
set
-int w; ///< widthof pict, undefined when pict is not
set
-int h; ///< height of pict, undefined when pict is not
set
-int nb_colors; ///< number of colors in pict, undefined when pict is not
set
-
-/**
- * data+linesize for the bitmap of this subtitle.
- * Can be set for text/ass as well once they are rendered.
- */
-uint8_t *data[4];
-int linesize[4];
-
-enum AVSubtitleType type;
-
-char *text; ///< 0 terminated plain UTF-8 text
-
-/**
- * 0 terminated ASS/SSA compatible event line.
- * The presentation of this is unaffected by the other values in this
- * struct.
- */
-char *ass;
-
-int flags;
-} AVSubtitleRect;
-
-typedef struct AVSubtitle {
-uint16_t format; /* 0 = graphics */
-uint32_t start_display_time; /* relative to packet pts, in ms */
-uint32_t end_display_time; /* relative to packet pts, in ms */
-unsigned num_rects;
-AVSubtitleRect **rects;
-int64_t pts;///< Same as packet pts, in AV_TIME_BASE
-
[FFmpeg-devel] [PATCH v12 02/14] global: Move ass helper functions to avutil as avpriv_ and extend ass dialog parsing
Signed-off-by: softworkz
---
libavcodec/Makefile | 56 +++
libavcodec/ass.h | 129
libavcodec/assdec.c | 2 +-
libavcodec/assenc.c | 2 +-
libavcodec/ccaption_dec.c | 19 +--
libavcodec/jacosubdec.c | 2 +-
libavcodec/libaribb24.c | 2 +-
libavcodec/libzvbi-teletextdec.c | 14 +-
libavcodec/microdvddec.c | 7 +-
libavcodec/movtextdec.c | 3 +-
libavcodec/movtextenc.c | 20 +--
libavcodec/mpl2dec.c | 2 +-
libavcodec/realtextdec.c | 2 +-
libavcodec/samidec.c | 2 +-
libavcodec/srtdec.c | 2 +-
libavcodec/srtenc.c | 16 +-
libavcodec/subviewerdec.c | 2 +-
libavcodec/textdec.c | 4 +-
libavcodec/ttmlenc.c | 15 +-
libavcodec/webvttdec.c| 2 +-
libavcodec/webvttenc.c| 16 +-
libavutil/Makefile| 2 +
{libavcodec => libavutil}/ass.c | 73 +
libavutil/ass_internal.h | 139 ++
{libavcodec => libavutil}/ass_split.c | 30 ++--
.../ass_split_internal.h | 24 +--
26 files changed, 331 insertions(+), 256 deletions(-)
rename {libavcodec => libavutil}/ass.c (73%)
create mode 100644 libavutil/ass_internal.h
rename {libavcodec => libavutil}/ass_split.c (94%)
rename libavcodec/ass_split.h => libavutil/ass_split_internal.h (89%)
diff --git a/libavcodec/Makefile b/libavcodec/Makefile
index ce928720e1..115611703d 100644
--- a/libavcodec/Makefile
+++ b/libavcodec/Makefile
@@ -209,10 +209,10 @@ OBJS-$(CONFIG_APNG_DECODER)+= png.o pngdec.o
pngdsp.o
OBJS-$(CONFIG_APNG_ENCODER)+= png.o pngenc.o
OBJS-$(CONFIG_ARBC_DECODER)+= arbc.o
OBJS-$(CONFIG_ARGO_DECODER)+= argo.o
-OBJS-$(CONFIG_SSA_DECODER) += assdec.o ass.o
-OBJS-$(CONFIG_SSA_ENCODER) += assenc.o ass.o
-OBJS-$(CONFIG_ASS_DECODER) += assdec.o ass.o
-OBJS-$(CONFIG_ASS_ENCODER) += assenc.o ass.o
+OBJS-$(CONFIG_SSA_DECODER) += assdec.o
+OBJS-$(CONFIG_SSA_ENCODER) += assenc.o
+OBJS-$(CONFIG_ASS_DECODER) += assdec.o
+OBJS-$(CONFIG_ASS_ENCODER) += assenc.o
OBJS-$(CONFIG_ASV1_DECODER)+= asvdec.o asv.o mpeg12data.o
OBJS-$(CONFIG_ASV1_ENCODER)+= asvenc.o asv.o mpeg12data.o
OBJS-$(CONFIG_ASV2_DECODER)+= asvdec.o asv.o mpeg12data.o
@@ -252,7 +252,7 @@ OBJS-$(CONFIG_BRENDER_PIX_DECODER) += brenderpix.o
OBJS-$(CONFIG_C93_DECODER) += c93.o
OBJS-$(CONFIG_CAVS_DECODER)+= cavs.o cavsdec.o cavsdsp.o \
cavsdata.o
-OBJS-$(CONFIG_CCAPTION_DECODER)+= ccaption_dec.o ass.o
+OBJS-$(CONFIG_CCAPTION_DECODER)+= ccaption_dec.o
OBJS-$(CONFIG_CDGRAPHICS_DECODER) += cdgraphics.o
OBJS-$(CONFIG_CDTOONS_DECODER) += cdtoons.o
OBJS-$(CONFIG_CDXL_DECODER)+= cdxl.o
@@ -423,7 +423,7 @@ OBJS-$(CONFIG_INTERPLAY_ACM_DECODER) += interplayacm.o
OBJS-$(CONFIG_INTERPLAY_DPCM_DECODER) += dpcm.o
OBJS-$(CONFIG_INTERPLAY_VIDEO_DECODER) += interplayvideo.o
OBJS-$(CONFIG_IPU_DECODER) += mpeg12dec.o mpeg12.o mpeg12data.o
-OBJS-$(CONFIG_JACOSUB_DECODER) += jacosubdec.o ass.o
+OBJS-$(CONFIG_JACOSUB_DECODER) += jacosubdec.o
OBJS-$(CONFIG_JPEG2000_ENCODER)+= j2kenc.o mqcenc.o mqc.o jpeg2000.o \
jpeg2000dwt.o
OBJS-$(CONFIG_JPEG2000_DECODER)+= jpeg2000dec.o jpeg2000.o
jpeg2000dsp.o \
@@ -445,7 +445,7 @@ OBJS-$(CONFIG_MAGICYUV_ENCODER)+= magicyuvenc.o
OBJS-$(CONFIG_MDEC_DECODER)+= mdec.o mpeg12.o mpeg12data.o
OBJS-$(CONFIG_METASOUND_DECODER) += metasound.o metasound_data.o \
twinvq.o
-OBJS-$(CONFIG_MICRODVD_DECODER)+= microdvddec.o ass.o
+OBJS-$(CONFIG_MICRODVD_DECODER)+= microdvddec.o
OBJS-$(CONFIG_MIMIC_DECODER) += mimic.o
OBJS-$(CONFIG_MJPEG_DECODER) += mjpegdec.o mjpegdec_common.o
OBJS-$(CONFIG_MJPEG_QSV_DECODER) += qsvdec.o
@@ -460,8 +460,8 @@ OBJS-$(CONFIG_MLP_ENCODER) += mlpenc.o mlp.o
OBJS-$(CONFIG_MMVIDEO_DECODER) += mmvideo.o
OBJS-$(CONFIG_MOBICLIP_DECODER)+= mobiclip.o
OBJS-$(CONFIG_MOTIONPIXELS_DECODER)+= motionpixels.o
-OBJS-$(CONFIG_MOVTEXT_DECODER) += movtextdec.o ass.o
-OBJS-$(CONFIG_MOVTEXT_ENCODER) += movtextenc.o ass_split.o
+OBJS-$(CONFIG_MOVTEXT_DECODER) += movtextdec.o
+OBJS-$(CONFIG_MOVTEXT
[FFmpeg-devel] [PATCH v12 03/14] fftools/play, probe: Adjust for subtitle changes
Signed-off-by: softworkz
---
fftools/ffplay.c | 102 +-
fftools/ffprobe.c | 48 ++
2 files changed, 78 insertions(+), 72 deletions(-)
diff --git a/fftools/ffplay.c b/fftools/ffplay.c
index ccea0e4578..e8d36485ae 100644
--- a/fftools/ffplay.c
+++ b/fftools/ffplay.c
@@ -152,7 +152,6 @@ typedef struct Clock {
/* Common struct for handling all types of decoded data and allocated render
buffers. */
typedef struct Frame {
AVFrame *frame;
-AVSubtitle sub;
int serial;
double pts; /* presentation timestamp for the frame */
double duration; /* estimated duration of the frame */
@@ -586,7 +585,7 @@ static int decoder_init(Decoder *d, AVCodecContext *avctx,
PacketQueue *queue, S
return 0;
}
-static int decoder_decode_frame(Decoder *d, AVFrame *frame, AVSubtitle *sub) {
+static int decoder_decode_frame(Decoder *d, AVFrame *frame) {
int ret = AVERROR(EAGAIN);
for (;;) {
@@ -620,6 +619,9 @@ static int decoder_decode_frame(Decoder *d, AVFrame *frame,
AVSubtitle *sub) {
}
}
break;
+case AVMEDIA_TYPE_SUBTITLE:
+ret = avcodec_receive_frame(d->avctx, frame);
+break;
}
if (ret == AVERROR_EOF) {
d->finished = d->pkt_serial;
@@ -652,25 +654,11 @@ static int decoder_decode_frame(Decoder *d, AVFrame
*frame, AVSubtitle *sub) {
av_packet_unref(d->pkt);
} while (1);
-if (d->avctx->codec_type == AVMEDIA_TYPE_SUBTITLE) {
-int got_frame = 0;
-ret = avcodec_decode_subtitle2(d->avctx, sub, &got_frame, d->pkt);
-if (ret < 0) {
-ret = AVERROR(EAGAIN);
-} else {
-if (got_frame && !d->pkt->data) {
-d->packet_pending = 1;
-}
-ret = got_frame ? 0 : (d->pkt->data ? AVERROR(EAGAIN) :
AVERROR_EOF);
-}
-av_packet_unref(d->pkt);
+if (avcodec_send_packet(d->avctx, d->pkt) == AVERROR(EAGAIN)) {
+av_log(d->avctx, AV_LOG_ERROR, "Receive_frame and send_packet both
returned EAGAIN, which is an API violation.\n");
+d->packet_pending = 1;
} else {
-if (avcodec_send_packet(d->avctx, d->pkt) == AVERROR(EAGAIN)) {
-av_log(d->avctx, AV_LOG_ERROR, "Receive_frame and send_packet
both returned EAGAIN, which is an API violation.\n");
-d->packet_pending = 1;
-} else {
-av_packet_unref(d->pkt);
-}
+av_packet_unref(d->pkt);
}
}
}
@@ -683,7 +671,6 @@ static void decoder_destroy(Decoder *d) {
static void frame_queue_unref_item(Frame *vp)
{
av_frame_unref(vp->frame);
-avsubtitle_free(&vp->sub);
}
static int frame_queue_init(FrameQueue *f, PacketQueue *pktq, int max_size,
int keep_last)
@@ -981,7 +968,7 @@ static void video_image_display(VideoState *is)
if (frame_queue_nb_remaining(&is->subpq) > 0) {
sp = frame_queue_peek(&is->subpq);
-if (vp->pts >= sp->pts + ((float) sp->sub.start_display_time /
1000)) {
+if (vp->pts >= sp->pts + ((float) sp->frame->subtitle_start_time /
1000)) {
if (!sp->uploaded) {
uint8_t* pixels[4];
int pitch[4];
@@ -993,25 +980,27 @@ static void video_image_display(VideoState *is)
if (realloc_texture(&is->sub_texture,
SDL_PIXELFORMAT_ARGB, sp->width, sp->height, SDL_BLENDMODE_BLEND, 1) < 0)
return;
-for (i = 0; i < sp->sub.num_rects; i++) {
-AVSubtitleRect *sub_rect = sp->sub.rects[i];
+for (i = 0; i < sp->frame->num_subtitle_areas; i++) {
+AVSubtitleArea *area = sp->frame->subtitle_areas[i];
+SDL_Rect sdl_rect = { .x = area->x, .y = area->y, .w =
area->w, .h = area->h };
-sub_rect->x = av_clip(sub_rect->x, 0, sp->width );
-sub_rect->y = av_clip(sub_rect->y, 0, sp->height);
-sub_rect->w = av_clip(sub_rect->w, 0, sp->width -
sub_rect->x);
-sub_rect->h = av_clip(sub_rect->h, 0, sp->height -
sub_rect->y);
+area->x = av_clip(area->x, 0, sp->width );
+area->y = av_clip(area->y, 0, sp->height);
+area->w = av_clip(area->w, 0, sp->width - area->x);
+area->h = av_clip(area->h, 0, sp->height - area->y);
is->sub_convert_ctx =
sws_getCachedContext(is->sub_convert_ctx,
-sub_rect->w, sub_rect->h, AV_PIX_FMT_PAL8,
-sub_rect->w, sub_rec
[FFmpeg-devel] [PATCH v12 04/14] avfilter/subtitles: Add subtitles.c for subtitle frame allocation
Analog to avfilter/video.c and avfilter/audio.c
Signed-off-by: softworkz
---
libavfilter/Makefile| 1 +
libavfilter/avfilter.c | 4 +++
libavfilter/internal.h | 1 +
libavfilter/subtitles.c | 63 +
libavfilter/subtitles.h | 44
5 files changed, 113 insertions(+)
create mode 100644 libavfilter/subtitles.c
create mode 100644 libavfilter/subtitles.h
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index dbd8685791..b1b3cbc349 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -19,6 +19,7 @@ OBJS = allfilters.o
\
framequeue.o \
graphdump.o \
graphparser.o\
+ subtitles.o \
video.o \
OBJS-$(HAVE_THREADS) += pthread.o
diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
index 7362bcdab5..df5b8f483c 100644
--- a/libavfilter/avfilter.c
+++ b/libavfilter/avfilter.c
@@ -43,6 +43,7 @@
#include "formats.h"
#include "framepool.h"
#include "internal.h"
+#include "subtitles.h"
#include "libavutil/ffversion.h"
const char av_filter_ffversion[] = "FFmpeg version " FFMPEG_VERSION;
@@ -1475,6 +1476,9 @@ int ff_inlink_make_frame_writable(AVFilterLink *link,
AVFrame **rframe)
case AVMEDIA_TYPE_AUDIO:
out = ff_get_audio_buffer(link, frame->nb_samples);
break;
+case AVMEDIA_TYPE_SUBTITLE:
+out = ff_get_subtitles_buffer(link, link->format);
+break;
default:
return AVERROR(EINVAL);
}
diff --git a/libavfilter/internal.h b/libavfilter/internal.h
index 1099b82b4b..fc09ef574c 100644
--- a/libavfilter/internal.h
+++ b/libavfilter/internal.h
@@ -90,6 +90,7 @@ struct AVFilterPad {
union {
AVFrame *(*video)(AVFilterLink *link, int w, int h);
AVFrame *(*audio)(AVFilterLink *link, int nb_samples);
+AVFrame *(*subtitle)(AVFilterLink *link, int format);
} get_buffer;
/**
diff --git a/libavfilter/subtitles.c b/libavfilter/subtitles.c
new file mode 100644
index 00..951bfd612c
--- /dev/null
+++ b/libavfilter/subtitles.c
@@ -0,0 +1,63 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "libavutil/common.h"
+
+#include "subtitles.h"
+#include "avfilter.h"
+#include "internal.h"
+
+
+AVFrame *ff_null_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+return ff_get_subtitles_buffer(link->dst->outputs[0], format);
+}
+
+AVFrame *ff_default_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+AVFrame *frame;
+
+frame = av_frame_alloc();
+if (!frame)
+return NULL;
+
+frame->format = format;
+frame->type = AVMEDIA_TYPE_SUBTITLE;
+
+if (av_frame_get_buffer2(frame, 0) < 0) {
+av_frame_free(&frame);
+return NULL;
+}
+
+return frame;
+}
+
+AVFrame *ff_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+AVFrame *ret = NULL;
+
+if (link->dstpad->get_buffer.subtitle)
+ret = link->dstpad->get_buffer.subtitle(link, format);
+
+if (!ret)
+ret = ff_default_get_subtitles_buffer(link, format);
+
+return ret;
+}
diff --git a/libavfilter/subtitles.h b/libavfilter/subtitles.h
new file mode 100644
index 00..4a9115126e
--- /dev/null
+++ b/libavfilter/subtitles.h
@@ -0,0 +1,44 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have recei
[FFmpeg-devel] [PATCH v12 05/14] avfilter/avfilter: Handle subtitle frames
Signed-off-by: softworkz
---
libavfilter/avfilter.c | 8 +---
libavfilter/avfilter.h | 11 +++
libavfilter/avfiltergraph.c | 5 +
libavfilter/formats.c | 14 ++
libavfilter/formats.h | 3 +++
libavfilter/internal.h | 18 +++---
6 files changed, 53 insertions(+), 6 deletions(-)
diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
index df5b8f483c..75d5e86539 100644
--- a/libavfilter/avfilter.c
+++ b/libavfilter/avfilter.c
@@ -56,7 +56,8 @@ static void tlog_ref(void *ctx, AVFrame *ref, int end)
ref->linesize[0], ref->linesize[1], ref->linesize[2],
ref->linesize[3],
ref->pts, ref->pkt_pos);
-if (ref->width) {
+switch(ref->type) {
+case AVMEDIA_TYPE_VIDEO:
ff_tlog(ctx, " a:%d/%d s:%dx%d i:%c iskey:%d type:%c",
ref->sample_aspect_ratio.num, ref->sample_aspect_ratio.den,
ref->width, ref->height,
@@ -64,12 +65,13 @@ static void tlog_ref(void *ctx, AVFrame *ref, int end)
ref->top_field_first ? 'T' : 'B',/* Top / Bottom */
ref->key_frame,
av_get_picture_type_char(ref->pict_type));
-}
-if (ref->nb_samples) {
+break;
+case AVMEDIA_TYPE_AUDIO:
ff_tlog(ctx, " cl:%"PRId64"d n:%d r:%d",
ref->channel_layout,
ref->nb_samples,
ref->sample_rate);
+break;
}
ff_tlog(ctx, "]%s", end ? "\n" : "");
diff --git a/libavfilter/avfilter.h b/libavfilter/avfilter.h
index f7208754a7..ee2f5b594d 100644
--- a/libavfilter/avfilter.h
+++ b/libavfilter/avfilter.h
@@ -45,6 +45,7 @@
#include "libavutil/log.h"
#include "libavutil/samplefmt.h"
#include "libavutil/pixfmt.h"
+#include "libavutil/subfmt.h"
#include "libavutil/rational.h"
#include "libavfilter/version.h"
@@ -327,6 +328,12 @@ typedef struct AVFilter {
* and outputs use the same sample rate and channel count/layout.
*/
const enum AVSampleFormat *samples_list;
+/**
+ * Analogous to pixels, but delimited by AV_SUBTITLE_FMT_NONE
+ * and restricted to filters that only have AVMEDIA_TYPE_SUBTITLE
+ * inputs and outputs.
+ */
+const enum AVSubtitleType *subs_list;
/**
* Equivalent to { pix_fmt, AV_PIX_FMT_NONE } as pixels_list.
*/
@@ -335,6 +342,10 @@ typedef struct AVFilter {
* Equivalent to { sample_fmt, AV_SAMPLE_FMT_NONE } as samples_list.
*/
enum AVSampleFormat sample_fmt;
+/**
+ * Equivalent to { sub_fmt, AV_SUBTITLE_FMT_NONE } as subs_list.
+ */
+enum AVSubtitleType sub_fmt;
} formats;
int priv_size; ///< size of private data to allocate for the filter
diff --git a/libavfilter/avfiltergraph.c b/libavfilter/avfiltergraph.c
index e536abef8e..5914ebe989 100644
--- a/libavfilter/avfiltergraph.c
+++ b/libavfilter/avfiltergraph.c
@@ -311,6 +311,8 @@ static int filter_link_check_formats(void *log,
AVFilterLink *link, AVFilterForm
return ret;
break;
+case AVMEDIA_TYPE_SUBTITLE:
+return 0;
default:
av_assert0(!"reached");
}
@@ -441,6 +443,9 @@ static int query_formats(AVFilterGraph *graph, void
*log_ctx)
if (!link)
continue;
+if (link->type == AVMEDIA_TYPE_SUBTITLE)
+continue;
+
neg = ff_filter_get_negotiation(link);
av_assert0(neg);
for (neg_step = 1; neg_step < neg->nb_mergers; neg_step++) {
diff --git a/libavfilter/formats.c b/libavfilter/formats.c
index ba62f73248..d77088ab0d 100644
--- a/libavfilter/formats.c
+++ b/libavfilter/formats.c
@@ -19,6 +19,7 @@
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
+#include "libavcodec/avcodec.h"
#include "libavutil/avassert.h"
#include "libavutil/channel_layout.h"
#include "libavutil/common.h"
@@ -431,6 +432,12 @@ int ff_add_channel_layout(AVFilterChannelLayouts **l,
uint64_t channel_layout)
return 0;
}
+int ff_add_subtitle_type(AVFilterFormats **avff, int64_t fmt)
+{
+ADD_FORMAT(avff, fmt, ff_formats_unref, int, formats, nb_formats);
+return 0;
+}
+
AVFilterFormats *ff_make_formats_list_singleton(int fmt)
{
int fmts[2] = { fmt, -1 };
@@ -450,6 +457,13 @@ AVFilterFormats *ff_all_formats(enum AVMediaType type)
return NULL;
fmt++;
}
+} else if (type == AVMEDIA_TYPE_SUBTITLE) {
+if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_BITMAP) < 0)
+return NULL;
+if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_ASS) < 0)
+return NULL;
+if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_TEXT) < 0)
+return NULL;
}
return ret;
diff --git a/libavfilter/formats.h b/libavfilter/formats.h
index a884d15213..94754ebd88 100644
---
[FFmpeg-devel] [PATCH v12 06/14] avfilter/sbuffer: Add sbuffersrc and sbuffersink filters
Signed-off-by: softworkz
---
configure| 2 +-
libavfilter/allfilters.c | 2 ++
libavfilter/buffersink.c | 63 +++
libavfilter/buffersink.h | 15 +
libavfilter/buffersrc.c | 72
libavfilter/buffersrc.h | 1 +
6 files changed, 154 insertions(+), 1 deletion(-)
diff --git a/configure b/configure
index 231d0398a8..d18bb73ed6 100755
--- a/configure
+++ b/configure
@@ -7720,7 +7720,7 @@ print_enabled_components(){
fi
done
if [ "$name" = "filter_list" ]; then
-for c in asrc_abuffer vsrc_buffer asink_abuffer vsink_buffer; do
+for c in asrc_abuffer vsrc_buffer ssrc_sbuffer asink_abuffer
vsink_buffer ssink_sbuffer; do
printf "&ff_%s,\n" $c >> $TMPH
done
fi
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 761c261005..7fb4851624 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -538,8 +538,10 @@ extern const AVFilter ff_avsrc_movie;
* being the same while having different 'types'). */
extern const AVFilter ff_asrc_abuffer;
extern const AVFilter ff_vsrc_buffer;
+extern const AVFilter ff_ssrc_sbuffer;
extern const AVFilter ff_asink_abuffer;
extern const AVFilter ff_vsink_buffer;
+extern const AVFilter ff_ssink_sbuffer;
extern const AVFilter ff_af_afifo;
extern const AVFilter ff_vf_fifo;
diff --git a/libavfilter/buffersink.c b/libavfilter/buffersink.c
index b8ddafec35..8306312acc 100644
--- a/libavfilter/buffersink.c
+++ b/libavfilter/buffersink.c
@@ -29,6 +29,8 @@
#include "libavutil/internal.h"
#include "libavutil/opt.h"
+#include "libavcodec/avcodec.h"
+
#define FF_INTERNAL_FIELDS 1
#include "framequeue.h"
@@ -57,6 +59,10 @@ typedef struct BufferSinkContext {
int *sample_rates; ///< list of accepted sample
rates, terminated by -1
int sample_rates_size;
+/* only used for subtitles */
+enum AVSubtitleType *subtitle_types; ///< list of accepted subtitle
types, must be terminated with -1
+int subtitle_types_size;
+
AVFrame *peeked_frame;
} BufferSinkContext;
@@ -168,6 +174,15 @@ AVABufferSinkParams *av_abuffersink_params_alloc(void)
return NULL;
return params;
}
+
+AVSBufferSinkParams *av_sbuffersink_params_alloc(void)
+{
+AVSBufferSinkParams *params = av_mallocz(sizeof(AVSBufferSinkParams));
+
+if (!params)
+return NULL;
+return params;
+}
#endif
static av_cold int common_init(AVFilterContext *ctx)
@@ -305,6 +320,28 @@ static int asink_query_formats(AVFilterContext *ctx)
return 0;
}
+static int ssink_query_formats(AVFilterContext *ctx)
+{
+BufferSinkContext *buf = ctx->priv;
+AVFilterFormats *formats = NULL;
+unsigned i;
+int ret;
+
+CHECK_LIST_SIZE(subtitle_types)
+if (buf->subtitle_types_size) {
+for (i = 0; i < NB_ITEMS(buf->subtitle_types); i++)
+if ((ret = ff_add_subtitle_type(&formats, buf->subtitle_types[i]))
< 0)
+return ret;
+if ((ret = ff_set_common_formats(ctx, formats)) < 0)
+return ret;
+} else {
+if ((ret = ff_default_query_formats(ctx)) < 0)
+return ret;
+}
+
+return 0;
+}
+
#define OFFSET(x) offsetof(BufferSinkContext, x)
#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM
static const AVOption buffersink_options[] = {
@@ -322,9 +359,16 @@ static const AVOption abuffersink_options[] = {
{ NULL },
};
#undef FLAGS
+#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_SUBTITLE_PARAM
+static const AVOption sbuffersink_options[] = {
+{ "subtitle_types", "set the supported subtitle formats",
OFFSET(subtitle_types), AV_OPT_TYPE_BINARY, .flags = FLAGS },
+{ NULL },
+};
+#undef FLAGS
AVFILTER_DEFINE_CLASS(buffersink);
AVFILTER_DEFINE_CLASS(abuffersink);
+AVFILTER_DEFINE_CLASS(sbuffersink);
static const AVFilterPad avfilter_vsink_buffer_inputs[] = {
{
@@ -363,3 +407,22 @@ const AVFilter ff_asink_abuffer = {
.outputs = NULL,
FILTER_QUERY_FUNC(asink_query_formats),
};
+
+static const AVFilterPad avfilter_ssink_sbuffer_inputs[] = {
+{
+.name = "default",
+.type = AVMEDIA_TYPE_SUBTITLE,
+},
+};
+
+const AVFilter ff_ssink_sbuffer = {
+.name = "sbuffersink",
+.description = NULL_IF_CONFIG_SMALL("Buffer subtitle frames, and make
them available to the end of the filter graph."),
+.priv_class= &sbuffersink_class,
+.priv_size = sizeof(BufferSinkContext),
+.init = common_init,
+.activate = activate,
+FILTER_INPUTS(avfilter_ssink_sbuffer_inputs),
+.outputs = NULL,
+FILTER_QUERY_FUNC(ssink_query_formats),
+};
diff --git a/libavfilter/buffersink.h b/libavfilter/buffersink.h
index 69ed0f29a8..b439b586c5 100644
--- a/libavfilter/buffersink.h
+++ b/libavfilter/buffersink.h
@@ -129,6 +129,21 @@ typedef struct AVABufferSinkP
[FFmpeg-devel] [PATCH v12 07/14] avfilter/overlaygraphicsubs: Add overlaygraphicsubs and graphicsub2video filters
- overlaygraphicsubs (VS -> V)
Overlay graphic subtitles onto a video stream
- graphicsub2video {S -> V)
Converts graphic subtitles to video frames (with alpha)
Gets auto-inserted for retaining compatibility with
sub2video command lines
Signed-off-by: softworkz
---
doc/filters.texi| 118 +
libavfilter/Makefile| 2 +
libavfilter/allfilters.c| 2 +
libavfilter/vf_overlaygraphicsubs.c | 724
4 files changed, 846 insertions(+)
create mode 100644 libavfilter/vf_overlaygraphicsubs.c
diff --git a/doc/filters.texi b/doc/filters.texi
index 1994467f46..b9daffca03 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25198,6 +25198,124 @@ tools.
@c man end VIDEO SINKS
+@chapter Subtitle Filters
+@c man begin SUBTITLE FILTERS
+
+When you configure your FFmpeg build, you can disable any of the
+existing filters using @code{--disable-filters}.
+
+Below is a description of the currently available subtitle filters.
+
+@section graphicsub2video
+
+Renders graphic subtitles as video frames.
+
+This filter replaces the previous "sub2video" hack which did the conversion
implicitly and up-front as subtitle filtering wasn't possible at that time.
+To retain compatibility with earlier sub2video command lines, this filter is
being auto-inserted in those cases.
+
+For overlaying graphicsal subtitles it is recommended to use the
'overlay_graphicsubs' filter which is more efficient and takes less processing
resources.
+
+This filter is still useful in cases where the overlay is done with hardware
acceleration (e.g. overlay_qsv, overlay_vaapi, overlay_cuda) for preparing the
overlay frames.
+
+Inputs:
+@itemize
+@item 0: Subtitles [BITMAP]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video [RGB32]
+@end itemize
+
+
+It accepts the following parameters:
+
+@table @option
+@item size, s
+Set the size of the output video frame.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Overlay PGS subtitles
+(not recommended - better use overlay_graphicsubs)
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:1]graphicsub2video[subs];[0:0][subs]overlay" output.mp4
+@end example
+
+@item
+Overlay PGS subtitles implicitly
+The graphicsub2video is inserted automatically for compatibility with legacy
command lines.
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:0][0:1]overlay" output.mp4
+@end example
+@end itemize
+
+@section overlaygraphicsubs
+
+Overlay graphic subtitles onto a video stream.
+
+This filter can blend graphical subtitles on a video stream directly, i.e.
without creating full-size alpha images first.
+The blending operation is limited to the area of the subtitle rectangles,
which also means that no processing is done at times where no subtitles are to
be displayed.
+
+Inputs:
+@itemize
+@item 0: Video [YUV420P, YUV422P, YUV444P, ARGB, RGBA, ABGR, BGRA, RGB24,
BGR24]
+@item 1: Subtitles [BITMAP]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video (same as input)
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+@item x
+@item y
+Set the expression for the x and y coordinates of the overlaid video
+on the main video. Default value is "0" for both expressions. In case
+the expression is invalid, it is set to a huge value (meaning that the
+overlay will not be displayed within the output visible area).
+
+@item eof_action
+See @ref{framesync}.
+
+@item eval
+Set when the expressions for @option{x}, and @option{y} are evaluated.
+
+It accepts the following values:
+@table @samp
+@item init
+only evaluate expressions once during the filter initialization or
+when a command is processed
+
+@item frame
+evaluate expressions for each incoming frame
+@end table
+
+Default value is @samp{frame}.
+
+@item shortest
+See @ref{framesync}.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Overlay PGS subtitles
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:9][0:1]overlaygraphicsubs" output.mp4
+@end example
+@end itemize
+@c man end SUBTITLE FILTERS
+
@chapter Multimedia Filters
@c man begin MULTIMEDIA FILTERS
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index b1b3cbc349..495a2d63f6 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -291,6 +291,7 @@ OBJS-$(CONFIG_FSPP_FILTER) += vf_fspp.o
qp_table.o
OBJS-$(CONFIG_GBLUR_FILTER) += vf_gblur.o
OBJS-$(CONFIG_GEQ_FILTER)+= vf_geq.o
OBJS-$(CONFIG_GRADFUN_FILTER)+= vf_gradfun.o
+OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlaygraphicsubs.o
framesync.o
OBJS-$(CONFIG_GRAPHMONITOR_FILTER) += f_graphmonitor.o
OBJS-$(CONFIG_GRAYWORLD_FILTER) += vf_grayworld.o
[FFmpeg-devel] [PATCH v12 09/14] avfilter/overlaytextsubs: Add overlaytextsubs and textsubs2video filters
- overlaytextsubs {VS -> V)
Overlay text subtitles onto a video stream.
- textsubs2video {S -> V)
Converts text subtitles to video frames
Signed-off-by: softworkz
---
configure| 2 +
doc/filters.texi | 113 ++
libavfilter/Makefile | 2 +
libavfilter/allfilters.c | 4 +-
libavfilter/avfilter.c | 18 +-
libavfilter/vf_overlaytextsubs.c | 630 +++
6 files changed, 763 insertions(+), 6 deletions(-)
create mode 100644 libavfilter/vf_overlaytextsubs.c
diff --git a/configure b/configure
index d18bb73ed6..30584b0fdf 100755
--- a/configure
+++ b/configure
@@ -3625,6 +3625,7 @@ overlay_opencl_filter_deps="opencl"
overlay_qsv_filter_deps="libmfx"
overlay_qsv_filter_select="qsvvpp"
overlay_vulkan_filter_deps="vulkan_lib libglslang"
+overlaytextsubs_filter_deps="avcodec libass"
owdenoise_filter_deps="gpl"
pad_opencl_filter_deps="opencl"
pan_filter_deps="swresample"
@@ -3669,6 +3670,7 @@ superequalizer_filter_deps="avcodec"
superequalizer_filter_select="rdft"
surround_filter_deps="avcodec"
surround_filter_select="rdft"
+textsub2video_filter_deps="avcodec libass"
tinterlace_filter_deps="gpl"
tinterlace_merge_test_deps="tinterlace_filter"
tinterlace_pad_test_deps="tinterlace_filter"
diff --git a/doc/filters.texi b/doc/filters.texi
index b9daffca03..992ab0ddf7 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25314,6 +25314,119 @@ Overlay PGS subtitles
ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:9][0:1]overlaygraphicsubs" output.mp4
@end example
@end itemize
+
+@section overlaytextsubs
+
+Overlay text subtitles onto a video stream.
+
+This filter supersedes the classic @ref{subtitles} filter opposed to which it
does no longer require to open and access the source stream separately, which
is often causing problems or doesn't even work for non-local or slow sources.
+
+Inputs:
+@itemize
+@item 0: Video [YUV420P, YUV422P, YUV444P, ARGB, RGBA, ABGR, BGRA, RGB24,
BGR24]
+@item 1: Subtitles [TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video (same as input)
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+
+@item alpha
+Process alpha channel, by default alpha channel is untouched.
+
+@item fonts_dir
+Set a directory path containing fonts that can be used by the filter.
+These fonts will be used in addition to whatever the font provider uses.
+
+@item default_font_path
+Path to a font file to be used as the default font.
+
+@item font_size
+Set the default font size.
+
+@item fontconfig_file
+Path to ASS fontconfig configuration file.
+
+@item force_style
+Override default style or script info parameters of the subtitles. It accepts a
+string containing ASS style format @code{KEY=VALUE} couples separated by ",".
+
+@item margin
+Set the rendering margin in pixels.
+
+@item render_latest_only
+For rendering, alway use the latest event only, which is covering the given
point in time
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Overlay ASS subtitles with animations:
+@example
+ffmpeg -i
"http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv";
-filter_complex "[0:v]overlaytextsubs" -map 0 -y out.mkv
+@end example
+@end itemize
+
+@section textsub2video
+
+Converts text subtitles to video frames.
+
+For overlaying text subtitles onto video frames it is recommended to use the
overlay_textsubs filter.
+The textsub2video is useful for for creating transparent text-frames when
overlay is done via hw acceleration
+
+Inputs:
+@itemize
+@item 0: Subtitles [TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video [RGB32]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+
+@item rate, r
+Set the framerate for updating overlay frames.
+Normally, overlay frames will only be updated each time when the subtitles to
display are changing.
+In cases where subtitles include advanced features (like animation), this
parameter determines the frequency by which the overlay frames should be
updated.
+
+@item size, s
+Set the output frame size.
+Allows to override the size of output video frames.
+
+@item fonts_dir
+Set a directory path containing fonts that can be used by the filter.
+These fonts will be used in addition to whatever the font provider uses.
+
+@item default_font_path
+Path to a font file to be used as the default font.
+
+@item font_size
+Set the default font size.
+
+@item fontconfig_file
+Path to ASS fontconfig configuration file.
+
+@item force_style
+Override default style or script info parameters of the subtitles. It accepts a
+string containing ASS style format @code{KEY=VALUE} couples separated by ",".
+
+@item margin
+Set the rendering margin in pixels.
+
+@item render_latest_only
+For rendering, alway use the latest event only, which is covering the given
point in time.
+@end table
+
@c man end SUBTITLE FILTERS
@
[FFmpeg-devel] [PATCH v12 10/14] avfilter/textmod: Add textmod, censor and show_speaker filters
- textmod {S -> S)
Modify subtitle text in a number of ways
- censor {S -> S)
Censor subtitles using a word list
- show_speaker {S -> S)
Prepend speaker names from ASS subtitles to the visible text lines
Signed-off-by: softworkz
---
doc/filters.texi | 211
libavfilter/Makefile | 5 +
libavfilter/allfilters.c | 3 +
libavfilter/sf_textmod.c | 697 +++
4 files changed, 916 insertions(+)
create mode 100644 libavfilter/sf_textmod.c
diff --git a/doc/filters.texi b/doc/filters.texi
index 992ab0ddf7..8266dc111e 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25206,6 +25206,150 @@ existing filters using @code{--disable-filters}.
Below is a description of the currently available subtitle filters.
+
+@section censor
+
+Censor selected words in text subtitles.
+
+Inputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+@item mode
+The censoring mode to apply.
+
+Supported censoring modes are:
+
+@table @var
+@item 0, keep_first_last
+Replace all characters with the 'censor_char' except the first and the last
character of a word.
+For words with less than 4 characters, the last character will be replaced as
well.
+For words with less than 3 characters, the first character will be replaced as
well.
+@item 1, keep_first
+Replace all characters with the 'censor_char' except the first character of a
word.
+For words with less than 3 characters, the first character will be replaced as
well.
+@item 2, all
+Replace all characters with the 'censor_char'.
+@end table
+
+@item words
+A list of words to censor, separated by 'separator'.
+
+@item words_file
+Specify a file from which to load the contents for the 'words' parameter.
+
+@item censor_char
+Single character used as replacement for censoring.
+
+@item separator
+Delimiter character for words. Used with replace_words and remove_words- Must
be a single character.
+The default is '.'.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Change all characters to upper case while keeping all styles and animations:
+@example
+ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv";
-filter_complex "[0:s]textmod=mode=to_upper" -map 0 -y out.mkv
+@end example
+@item
+Remove a set of symbol characters for am improved and smoother visual
apperance:
+@example
+ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv";
-filter_complex "[0:s]textmod=mode=remove_chars:find='$&#@*§'" -map 0 -y
out.mkv
+@end example
+@end itemize
+
+
+@section textmod
+
+Modify subtitle text in a number of ways.
+
+Inputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+@item mode
+The kind of text modification to apply
+
+Supported operation modes are:
+
+@table @var
+@item 0, leet
+Convert subtitle text to 'leet speak'. It's primarily useful for testing as
the modification will be visible with almost all text lines.
+@item 1, to_upper
+Change all text to upper case. Might improve readability.
+@item 2, to_lower
+Change all text to lower case.
+@item 3, replace_chars
+Replace one or more characters. Requires the find and replace parameters to be
specified.
+Both need to be equal in length.
+The first char in find is replaced by the first char in replace, same for all
subsequent chars.
+@item 4, remove_chars
+Remove certain characters. Requires the find parameter to be specified.
+All chars in the find parameter string will be removed from all subtitle text.
+@item 5, replace_words
+Replace one or more words. Requires the find and replace parameters to be
specified. Multiple words must be separated by the delimiter char specified vie
the separator parameter (default: ',').
+The number of words in the find and replace parameters needs to be equal.
+The first word in find is replaced by the first word in replace, same for all
subsequent words
+@item 6, remove_words
+Remove certain words. Requires the find parameter to be specified. Multiple
words must be separated by the delimiter char specified vie the separator
parameter (default: ',').
+All words in the find parameter string will be removed from all subtitle text.
+@end table
+
+@item find
+Required for replace_chars, remove_chars, replace_words and remove_words.
+
+@item find_file
+Specify a file from which to load the contents for the 'find' parameter.
+
+@item replace
+Required for replace_chars and replace_words.
+
+@item replace_file
+Specify a file from which to load the contents for the 'replace' parameter.
+
+@item separator
+Delimiter character for words. Used with replace_words and remove_words- Must
be a single character.
+The default is '.'.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Change all characters to upper case while keeping all styles and
[FFmpeg-devel] [PATCH v12 08/14] fftools/ffmpeg: Replace sub2video with subtitle frame filtering
This commit actually enables subtitle filtering in ffmpeg by
sending and receiving subtitle frames to and from a filtergraph.
The heartbeat functionality from the previous sub2video implementation
is retained and applied to all subtitle frames (bitmap, text, ..).
The other part of sub2video functionality is retained by
auto-insertion of the new graphicsub2video filter.
Justification for changed test refs:
- sub2video
The new results are identical excepting the last frame which
is due to the implementation changes
- sub2video_basic
The previous results had some incorrect output because multiple
frames had the same dts (even the first one was incorrect).
The non-empty content frames are visually identical, the different
CRC is due to the different blending algorithm that is being used.
- sub2video_time_limited
The previous results were incorrect. There was an initial empty frame
with the same dts generated. This is no longer happening.
The third frame was a repetition, which doesn't happen anymore with
the new subtitle filtering.
The source file contains a single subtitle event and the new and
correct result is a single output frame
- sub-dvb
Running ffprobe -show_frames on the source file shows that there
are 7 subtitle frames with 0 rects in the source at the start
and 2 at the end. This translates to the 14 and 4 additional
entries in the new test results.
- filter-overlay-dvdsub-2397
Overlay results have slightly different CRCs due to different
blending implementation
Signed-off-by: softworkz
---
fftools/ffmpeg.c | 509 ++
fftools/ffmpeg.h | 14 +-
fftools/ffmpeg_filter.c | 209 ++---
fftools/ffmpeg_hw.c | 2 +-
fftools/ffmpeg_opt.c | 3 +-
tests/ref/fate/filter-overlay-dvdsub-2397 | 181
tests/ref/fate/sub-dvb| 162 ---
tests/ref/fate/sub2video | 1 -
tests/ref/fate/sub2video_basic| 93 ++--
tests/ref/fate/sub2video_time_limited | 4 +-
10 files changed, 623 insertions(+), 555 deletions(-)
diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c
index bb29e3cec5..5d48dd118d 100644
--- a/fftools/ffmpeg.c
+++ b/fftools/ffmpeg.c
@@ -169,163 +169,6 @@ static int restore_tty;
static void free_input_threads(void);
#endif
-/* sub2video hack:
- Convert subtitles to video with alpha to insert them in filter graphs.
- This is a temporary solution until libavfilter gets real subtitles support.
- */
-
-static int sub2video_get_blank_frame(InputStream *ist)
-{
-int ret;
-AVFrame *frame = ist->sub2video.frame;
-
-av_frame_unref(frame);
-ist->sub2video.frame->width = ist->dec_ctx->width ? ist->dec_ctx->width
: ist->sub2video.w;
-ist->sub2video.frame->height = ist->dec_ctx->height ? ist->dec_ctx->height
: ist->sub2video.h;
-ist->sub2video.frame->format = AV_PIX_FMT_RGB32;
-if ((ret = av_frame_get_buffer(frame, 0)) < 0)
-return ret;
-memset(frame->data[0], 0, frame->height * frame->linesize[0]);
-return 0;
-}
-
-static void sub2video_copy_rect(uint8_t *dst, int dst_linesize, int w, int h,
-AVSubtitleRect *r)
-{
-uint32_t *pal, *dst2;
-uint8_t *src, *src2;
-int x, y;
-
-if (r->type != SUBTITLE_BITMAP) {
-av_log(NULL, AV_LOG_WARNING, "sub2video: non-bitmap subtitle\n");
-return;
-}
-if (r->x < 0 || r->x + r->w > w || r->y < 0 || r->y + r->h > h) {
-av_log(NULL, AV_LOG_WARNING, "sub2video: rectangle (%d %d %d %d)
overflowing %d %d\n",
-r->x, r->y, r->w, r->h, w, h
-);
-return;
-}
-
-dst += r->y * dst_linesize + r->x * 4;
-src = r->data[0];
-pal = (uint32_t *)r->data[1];
-for (y = 0; y < r->h; y++) {
-dst2 = (uint32_t *)dst;
-src2 = src;
-for (x = 0; x < r->w; x++)
-*(dst2++) = pal[*(src2++)];
-dst += dst_linesize;
-src += r->linesize[0];
-}
-}
-
-static void sub2video_push_ref(InputStream *ist, int64_t pts)
-{
-AVFrame *frame = ist->sub2video.frame;
-int i;
-int ret;
-
-av_assert1(frame->data[0]);
-ist->sub2video.last_pts = frame->pts = pts;
-for (i = 0; i < ist->nb_filters; i++) {
-ret = av_buffersrc_add_frame_flags(ist->filters[i]->filter, frame,
- AV_BUFFERSRC_FLAG_KEEP_REF |
- AV_BUFFERSRC_FLAG_PUSH);
-if (ret != AVERROR_EOF && ret < 0)
-av_log(NULL, AV_LOG_WARNING, "Error while add the frame to buffer
source(%s).\n",
- av_err2str(ret));
-}
-}
-
-void sub2video_update(InputStream *ist, int64_t heartbeat_pts, AVSubtitle *sub)
-{
-AVFrame *frame = ist->sub2video.frame;
-int8_t *dst;
-int dst_linesize;
-int num_rects, i;
-int64_t pts, end_pts;
-
-if (!fra
[FFmpeg-devel] [PATCH v12 11/14] avfilter/stripstyles: Add stripstyles filter
- stripstyles {S -> S)
Remove all inline styles from subtitle events
Signed-off-by: softworkz
---
doc/filters.texi | 34 +++
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/sf_stripstyles.c | 191 +++
4 files changed, 227 insertions(+)
create mode 100644 libavfilter/sf_stripstyles.c
diff --git a/doc/filters.texi b/doc/filters.texi
index 8266dc111e..25fab83800 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25271,6 +25271,40 @@ ffmpeg -i
"https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv"; -filter_comple
@end example
@end itemize
+@section stripstyles
+
+Remove all inline styles from subtitle events.
+
+Inputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+@item remove_animated
+Also remove text which is subject to animation (default: true)
+Usually, animated text elements are used used in addition to static subtitle
lines for creating effects, so in most cases it is safe to remove the animation
content.
+If subtitle text is missing, try setting this to false.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Remove all styles and animations from subtitles:
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv";
-filter_complex "[0:1]stripstyles" -map 0 output.mkv
+@end example
+@end itemize
+
@section textmod
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 5d5cef7f8e..4e5a3f6d1d 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -543,6 +543,7 @@ OBJS-$(CONFIG_NULLSINK_FILTER) +=
vsink_nullsink.o
OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o
OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o
OBJS-$(CONFIG_TEXTMOD_FILTER)+= sf_textmod.o
+OBJS-$(CONFIG_STRIPSTYLES_FILTER)+= sf_stripstyles.o
# multimedia filters
OBJS-$(CONFIG_ABITSCOPE_FILTER) += avf_abitscope.o
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index f8be4a53aa..68487dffd8 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -531,6 +531,7 @@ extern const AVFilter ff_avf_showwavespic;
extern const AVFilter ff_vaf_spectrumsynth;
extern const AVFilter ff_sf_censor;
extern const AVFilter ff_sf_showspeaker;
+extern const AVFilter ff_sf_stripstyles;
extern const AVFilter ff_sf_textmod;
extern const AVFilter ff_svf_graphicsub2video;
extern const AVFilter ff_svf_textsub2video;
diff --git a/libavfilter/sf_stripstyles.c b/libavfilter/sf_stripstyles.c
new file mode 100644
index 00..ed35c2afcc
--- /dev/null
+++ b/libavfilter/sf_stripstyles.c
@@ -0,0 +1,191 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/**
+ * @file
+ * text subtitle filter which removes inline-styles from subtitles
+ */
+
+#include "libavutil/opt.h"
+#include "internal.h"
+#include "libavutil/ass_split_internal.h"
+#include "libavutil/bprint.h"
+
+typedef struct StripStylesContext {
+const AVClass *class;
+enum AVSubtitleType format;
+int remove_animated;
+} StripStylesContext;
+
+typedef struct DialogContext {
+StripStylesContext* ss_ctx;
+AVBPrint buffer;
+int drawing_scale;
+int is_animated;
+} DialogContext;
+
+static void dialog_text_cb(void *priv, const char *text, int len)
+{
+DialogContext *s = priv;
+
+av_log(s->ss_ctx, AV_LOG_DEBUG, "dialog_text_cb: %s\n", text);
+
+if (!s->drawing_scale && (!s->is_animated || !s->ss_ctx->remove_animated))
+av_bprint_append_data(&s->buffer, text, len);
+}
+
+static void dialog_new_line_cb(void *priv, int forced)
+{
+DialogContext *s = priv;
+if (!s->drawing_scale && !s->is_animated)
+av_bprint_append_data(&s->buffer, forced ? "\\N" : "\\n", 2);
+}
+
+static void dialog_drawing_mode_cb(void *priv, int scale)
+{
+DialogContext *s = priv;
+s->drawing_scale = scale;
+}
+
+static void dialog_animate_cb(void *priv, int t1, int t2, int accel, char
*style)
+{
+DialogContext *s = priv;
+s->is_animated = 1;
+}
+
+static void dialog_move_cb(void *priv, int x1, i
[FFmpeg-devel] [PATCH v12 12/14] avfilter/splitcc: Add splitcc filter for closed caption handling
- splitcc {V -> VS)
Extract closed-caption (A53) data from video
frames as subtitle Frames
ffmpeg -y -loglevel verbose -i
"https://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts";
-filter_complex
"[0:v]splitcc[vid1],textmod=mode=remove_chars:find='@',[vid1]overlay_textsubs"
output.mkv
Signed-off-by: softworkz
---
configure| 1 +
doc/filters.texi | 57 +++
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/sf_splitcc.c | 315 +++
5 files changed, 375 insertions(+)
create mode 100644 libavfilter/sf_splitcc.c
diff --git a/configure b/configure
index 30584b0fdf..0ad739fa56 100755
--- a/configure
+++ b/configure
@@ -3663,6 +3663,7 @@ spp_filter_select="fft idctdsp fdctdsp me_cmp pixblockdsp"
sr_filter_deps="avformat swscale"
sr_filter_select="dnn"
stereo3d_filter_deps="gpl"
+splitcc_filter_deps="cc_dec_decoder"
subtitles_filter_deps="avformat avcodec libass"
super2xsai_filter_deps="gpl"
pixfmts_super2xsai_test_deps="super2xsai_filter"
diff --git a/doc/filters.texi b/doc/filters.texi
index 25fab83800..8d5763f057 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25618,6 +25618,63 @@ ffmpeg -i INPUT -filter_complex
"showspeaker=format=colon:style='@{\\c&HDD&\
@end example
@end itemize
+
+@section splitcc
+
+Split-out closed-caption/A53 subtitles from video frame side data.
+
+This filter provides an input and an output for video frames, which are just
passed through without modification.
+The second out provides subtitle frames which are extracted from video frame
side data.
+
+Inputs:
+@itemize
+@item 0: Video [ALL]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video (same as input)
+@item 1: Subtitles [TEXT]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+
+@item use_cc_styles
+Emit closed caption style header.
+This will make closed captions appear in white font with a black rectangle
background.
+
+@item real_time
+Emit subtitle events as they are decoded for real-time display.
+
+@item real_time_latency_msec
+Minimum elapsed time between emitting real-time subtitle events.
+Only applies to real_time mode.
+
+@item data_field
+Select data field. Possible values:
+
+@table @samp
+@item auto
+Pick first one that appears.
+@item first
+@item second
+@end table
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Extract closed captions as text subtitle stream and overlay it onto the video:
+@example
+ffmpeg -i "https://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts";
-filter_complex "[0:v:0]splitcc[vid1][sub1];[vid1][sub1]overlay_textsubs"
output.mkv
+@end example
+@end itemize
+
+
@section textsub2video
Converts text subtitles to video frames.
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 4e5a3f6d1d..84854bbda0 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -543,6 +543,7 @@ OBJS-$(CONFIG_NULLSINK_FILTER) +=
vsink_nullsink.o
OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o
OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o
OBJS-$(CONFIG_TEXTMOD_FILTER)+= sf_textmod.o
+OBJS-$(CONFIG_SPLITCC_FILTER)+= sf_splitcc.o
OBJS-$(CONFIG_STRIPSTYLES_FILTER)+= sf_stripstyles.o
# multimedia filters
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 68487dffd8..fdca1dac63 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -531,6 +531,7 @@ extern const AVFilter ff_avf_showwavespic;
extern const AVFilter ff_vaf_spectrumsynth;
extern const AVFilter ff_sf_censor;
extern const AVFilter ff_sf_showspeaker;
+extern const AVFilter ff_sf_splitcc;
extern const AVFilter ff_sf_stripstyles;
extern const AVFilter ff_sf_textmod;
extern const AVFilter ff_svf_graphicsub2video;
diff --git a/libavfilter/sf_splitcc.c b/libavfilter/sf_splitcc.c
new file mode 100644
index 00..b48dfd5fa6
--- /dev/null
+++ b/libavfilter/sf_splitcc.c
@@ -0,0 +1,315 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/**
+ * @file
+ * subtitle filter for splitting out closed-caption/A53 subtitles from video
frame side data
+ */
+
+#include "libavutil/
[FFmpeg-devel] [PATCH v12 13/14] avfilter/graphicsub2text: Add new graphicsub2text filter (OCR)
Signed-off-by: softworkz --- configure| 1 + doc/filters.texi | 55 ++ libavfilter/Makefile | 2 + libavfilter/allfilters.c | 1 + libavfilter/sf_graphicsub2text.c | 323 +++ 5 files changed, 382 insertions(+) create mode 100644 libavfilter/sf_graphicsub2text.c diff --git a/configure b/configure index 0ad739fa56..1b02b56ea1 100755 --- a/configure +++ b/configure @@ -3601,6 +3601,7 @@ frei0r_deps_any="libdl LoadLibrary" frei0r_filter_deps="frei0r" frei0r_src_filter_deps="frei0r" fspp_filter_deps="gpl" +graphicsub2text_filter_deps="libtesseract" histeq_filter_deps="gpl" hqdn3d_filter_deps="gpl" interlace_filter_deps="gpl" diff --git a/doc/filters.texi b/doc/filters.texi index 8d5763f057..fa27a4f946 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -25384,6 +25384,61 @@ ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv"; -filter_comple @end example @end itemize +@section graphicsub2text + +Converts graphic subtitles to text subtitles by performing OCR. + +For this filter to be available, ffmpeg needs to be compiled with libtesseract (see https://github.com/tesseract-ocr/tesseract). +Language models need to be downloaded from https://github.com/tesseract-ocr/tessdata and put into as subfolder named 'tessdata' or into a folder specified via the environment variable 'TESSDATA_PREFIX'. +The path can also be specified via filter option (see below). + +Note: These models are including the data for both OCR modes. + +Inputs: +- 0: Subtitles [bitmap] + +Outputs: +- 0: Subtitles [text] + +It accepts the following parameters: + +@table @option +@item ocr_mode +The character recognition mode to use. + +Supported OCR modes are: + +@table @var +@item 0, tesseract +This is the classic libtesseract operation mode. It is fast but less accurate than LSTM. +@item 1, lstm +Newer OCR implementation based on ML models. Provides usually better results, requires more processing resources. +@item 2, both +Use a combination of both modes. +@end table + +@item tessdata_path +The path to a folder containing the language models to be used. + +@item language +The recognition language. It needs to match the first three characters of a language model file in the tessdata path. + +@end table + + +@subsection Examples + +@itemize +@item +Convert DVB graphic subtitles to ASS (text) subtitles + +Note: For this to work, you need to have the data file 'eng.traineddata' in a 'tessdata' subfolder (see above). +@example +ffmpeg ffmpeg -loglevel verbose -i "https://streams.videolan.org/streams/ts/video_subs_ttxt%2Bdvbsub.ts"; -filter_complex "[0:13]graphicsub2text=ocr_mode=both" -c:s ass -y output.mkv +@end example +@end itemize + + @section graphicsub2video Renders graphic subtitles as video frames. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 84854bbda0..5a1e03a18a 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -292,6 +292,8 @@ OBJS-$(CONFIG_GBLUR_FILTER) += vf_gblur.o OBJS-$(CONFIG_GEQ_FILTER)+= vf_geq.o OBJS-$(CONFIG_GRADFUN_FILTER)+= vf_gradfun.o OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlaygraphicsubs.o framesync.o +OBJS-$(CONFIG_GRAPHICSUB2TEXT_FILTER)+= sf_graphicsub2text.o +OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlaygraphicsubs.o framesync.o OBJS-$(CONFIG_GRAPHMONITOR_FILTER) += f_graphmonitor.o OBJS-$(CONFIG_GRAYWORLD_FILTER) += vf_grayworld.o OBJS-$(CONFIG_GREYEDGE_FILTER) += vf_colorconstancy.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index fdca1dac63..e6161b84b6 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -530,6 +530,7 @@ extern const AVFilter ff_avf_showwaves; extern const AVFilter ff_avf_showwavespic; extern const AVFilter ff_vaf_spectrumsynth; extern const AVFilter ff_sf_censor; +extern const AVFilter ff_sf_graphicsub2text; extern const AVFilter ff_sf_showspeaker; extern const AVFilter ff_sf_splitcc; extern const AVFilter ff_sf_stripstyles; diff --git a/libavfilter/sf_graphicsub2text.c b/libavfilter/sf_graphicsub2text.c new file mode 100644 index 00..c963ba4384 --- /dev/null +++ b/libavfilter/sf_graphicsub2text.c @@ -0,0 +1,323 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of
[FFmpeg-devel] [PATCH v12 14/14] avfilter/subscale: Add filter for scaling and/or re-arranging graphical subtitles
Signed-off-by: softworkz
---
configure | 1 +
doc/filters.texi | 159 +++
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/sf_subscale.c | 847 ++
5 files changed, 1009 insertions(+)
create mode 100644 libavfilter/sf_subscale.c
diff --git a/configure b/configure
index 1b02b56ea1..69f754fca7 100755
--- a/configure
+++ b/configure
@@ -3665,6 +3665,7 @@ sr_filter_deps="avformat swscale"
sr_filter_select="dnn"
stereo3d_filter_deps="gpl"
splitcc_filter_deps="cc_dec_decoder"
+subscale_filter_deps="swscale avcodec"
subtitles_filter_deps="avformat avcodec libass"
super2xsai_filter_deps="gpl"
pixfmts_super2xsai_test_deps="super2xsai_filter"
diff --git a/doc/filters.texi b/doc/filters.texi
index fa27a4f946..40dadebbd9 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25784,6 +25784,165 @@ Set the rendering margin in pixels.
For rendering, alway use the latest event only, which is covering the given
point in time.
@end table
+@section subscale
+
+Provides high-quality scaling and rearranging functionality for graphical
subtitles.
+
+The subscale filter provides multiple approaches for manipulating
+the size and position of graphical subtitle rectangles wich can
+be combined or used separately.
+Scaling is performed by converting the palettized subtitle bitmaps
+to RGBA and re-quantization to palette colors afterwards via elbg algorithm.
+
+The two major operations are 'scale' and 're-arrange' with the
+latter being separated as 'arrange_h' and 'arrange_v'.
+
+
+Inputs:
+- 0: Subtitles [bitmap]
+
+Outputs:
+- 0: Subtitles [bitmap]
+
+It accepts the following parameters:
+
+@table @option
+
+@item w, width
+Set the width of the output.
+Width and height in case of graphical subtitles are just indicating
+a virtual size for which the output (consisting of 0-n bitmap rectangles)
+is intended to be displayed on.
+
+@item h, height
+Set the height of the output.
+
+@item margin_h
+Sets a horizontal margin to be preserverved when using any
+of the arrange modes.
+
+@item margin_v
+Sets a vertical margin to be preserverved when using any
+of the arrange modes.
+
+@item force_original_aspect_ratio
+Enable decreasing or increasing output video width or height if necessary to
+keep the original aspect ratio. Possible values:
+
+@table @samp
+@item disable
+Scale the video as specified and disable this feature.
+
+@item decrease
+The output video dimensions will automatically be decreased if needed.
+
+@item increase
+The output video dimensions will automatically be increased if needed.
+
+@end table
+
+
+@item scale_mode
+Specifies how subtitle bitmaps should be scaled.
+The scale factor is determined by the the factor between input
+and output size.
+
+@table @samp
+@item none
+Do not apply any common scaling.
+
+@item uniform
+Uniformly scale all subtitle bitmaps including their positions.
+
+@item uniform_no_reposition
+Uniformly scale all subtitle bitmaps without changing positions.
+
+@end table
+
+
+@item arrange_h
+Specifies how subtitle bitmaps should be arranged horizontally.
+
+@item arrange_v
+Specifies how subtitle bitmaps should be arranged vertically.
+
+
+@table @samp
+@item none
+Do not rearrange subtitle bitmaps.
+
+@item margin_no_scale
+Move subtitle bitmaps to be positioned inside the specified
+margin (margin_h or margin_v) when possible and without scaling.
+
+@item margin_and_scale
+Move subtitle bitmaps to be positioned inside the specified
+margin (margin_h or margin_v) and scale in case it doesn't fit.
+
+@item snapalign_no_scale
+Categorize subtitle bitmap positions as one of left/center/right
+or top/bottom/middle based on original positioning and apply
+these alignments for the target positioning.
+No scaling will be applied.
+
+@item snapalign_and_scale
+Categorize subtitle bitmap positions as one of left/center/right
+or top/bottom/middle based on original positioning and apply
+these alignments for the target positioning.
+Bitmaps that do not fit inside the margins borders are
+scaled to fit.
+@end table
+
+@item eval
+Set evaluation mode for the expressions (@option{width}, @option{height}).
+
+It accepts the following values:
+@table @samp
+@item init
+Evaluate expressions only once during the filter initialization.
+
+@item frame
+Evaluate expressions for each incoming frame. This is way slower than the
+@samp{init} mode since it requires all the scalers to be re-computed, but it
+allows advanced dynamic expressions.
+@end table
+
+Default value is @samp{init}.
+
+
+@item num_colors
+Set the number of palette colors for output images.
+Choose the maximum (256) when further processing is done (e.g.
+overlaying on a video).
+When subtitles will be encoded as bitmap subtitles (e.g. dvbsub),
+a smaller number of palette colors (e.g. 4-16) might need to be used, depending
+on the target format and codec.
+
+@item bitmap_width_align
+@item bitmap_height_align
+Make s
Re: [FFmpeg-devel] [PATCH v3 1/2] avcodec/codec_par: Add codec properties field to AVCodecParameters
Soft Works:
>
>
>> -Original Message-
>> From: ffmpeg-devel On Behalf Of
>> Hendrik Leppkes
>> Sent: Wednesday, October 6, 2021 8:57 AM
>> To: FFmpeg development discussions and patches > de...@ffmpeg.org>
>> Subject: Re: [FFmpeg-devel] [PATCH v3 1/2] avcodec/codec_par: Add
>> codec properties field to AVCodecParameters
>>
>> On Wed, Oct 6, 2021 at 8:45 AM Soft Works
>> wrote:
>>> diff --git a/libavcodec/codec_par.h b/libavcodec/codec_par.h
>>> index 10cf79dff1..42ed8deb13 100644
>>> --- a/libavcodec/codec_par.h
>>> +++ b/libavcodec/codec_par.h
>>> @@ -198,6 +198,10 @@ typedef struct AVCodecParameters {
>>> * Audio only. Number of samples to skip after a
>> discontinuity.
>>> */
>>> int seek_preroll;
>>> +/**
>>> + * Codec properties of the stream that gets decoded
>>> + */
>>> +unsigned properties;
>>> } AVCodecParameters;
>>>
>>
>> This field is severly underspecified/underdocumented. I realize you
>> just copied it, but if I'm looking at this without pre-existing
>> knowledge, then I have absolutely no idea what it might be for, or
>> what kind of values go in there. The old field had the defines of the
>> possible bits right there for context - this doesn't have this, so it
>> would definitely benefit from added documentation.
>
> Hello Hendrik,
>
> that's right, but what would you suggest? We can't duplicate the defines,
> though I could add a line in the help text like
>
> /**
> * Codec properties of the stream that gets decoded.
> * Corresponds to @ref AVCodecContext.properties "properties".
> */
> unsigned properties;
>
> Would that be OK?
>
The defines could be moved to defs.h (while just at it, we could also
add properly prefixed alternatives and deprecate the FF_ ones).
The copying that you are adding in avcodec_parameters_from_context is
problematic, as the relevant AVCodecContext field is marked as "set by
libavcodec" for decoding, which means that decoders expect it to be
blank initially. Imagine a scenario where the beginning (that gets read
in avformat_find_stream_info()) of an input stream has e.g. embedded cc
subtitles, but the part that lateron gets fed to the decoder does not.
If the decoder has been initialized from the AVStream's
AVCodecParameters, then said decoder would still indicate that there are
closed captions.* IIRC Anton already mentioned that this is actually a
frame/packet property, not a stream property.
- Andreas
*: It seems that the properties field is treated inconsistently:
LOSSLESS and CLOSED_CAPTIONS are only ever set, never unset, FILM_GRAIN
is also unset.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] point releases
Hi all I do plan to make releases from 4.4, 4.2, 4.1, 4.0, 3.4, 3.2, 2.8 branches why ? they are used by maintained distributions as listed on https://trac.ffmpeg.org/wiki/Downstreams (hint, please keep this page updated so release work is targeted to where its most usefull) why 4.1 again ? because debian security people asked about the newly accumulating fixes in the branch and the next release and IMHO it makes sense thx -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB "You are 36 times more likely to die in a bathtub than at the hands of a terrorist. Also, you are 2.5 times more likely to become a president and 2 times more likely to become an astronaut, than to die in a terrorist attack." -- Thoughty2 signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v3 1/2] avcodec/codec_par: Add codec properties field to AVCodecParameters
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Andreas Rheinhardt
> Sent: Wednesday, October 6, 2021 11:30 AM
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v3 1/2] avcodec/codec_par: Add
> codec properties field to AVCodecParameters
>
> Soft Works:
> >
> >
> >> -Original Message-
> >> From: ffmpeg-devel On Behalf Of
> >> Hendrik Leppkes
> >> Sent: Wednesday, October 6, 2021 8:57 AM
> >> To: FFmpeg development discussions and patches >> de...@ffmpeg.org>
> >> Subject: Re: [FFmpeg-devel] [PATCH v3 1/2] avcodec/codec_par: Add
> >> codec properties field to AVCodecParameters
> >>
> >> On Wed, Oct 6, 2021 at 8:45 AM Soft Works
> >> wrote:
> >>> diff --git a/libavcodec/codec_par.h b/libavcodec/codec_par.h
> >>> index 10cf79dff1..42ed8deb13 100644
> >>> --- a/libavcodec/codec_par.h
> >>> +++ b/libavcodec/codec_par.h
> >>> @@ -198,6 +198,10 @@ typedef struct AVCodecParameters {
> >>> * Audio only. Number of samples to skip after a
> >> discontinuity.
> >>> */
> >>> int seek_preroll;
> >>> +/**
> >>> + * Codec properties of the stream that gets decoded
> >>> + */
> >>> +unsigned properties;
> >>> } AVCodecParameters;
> >>>
> >>
> >> This field is severly underspecified/underdocumented. I realize
> you
> >> just copied it, but if I'm looking at this without pre-existing
> >> knowledge, then I have absolutely no idea what it might be for, or
> >> what kind of values go in there. The old field had the defines of
> the
> >> possible bits right there for context - this doesn't have this, so
> it
> >> would definitely benefit from added documentation.
> >
> > Hello Hendrik,
> >
> > that's right, but what would you suggest? We can't duplicate the
> defines,
> > though I could add a line in the help text like
> >
> > /**
> > * Codec properties of the stream that gets decoded.
> > * Corresponds to @ref AVCodecContext.properties "properties".
> > */
> > unsigned properties;
> >
> > Would that be OK?
> >
>
> The defines could be moved to defs.h (while just at it, we could also
> add properly prefixed alternatives and deprecate the FF_ ones).
> The copying that you are adding in avcodec_parameters_from_context
> is
> problematic, as the relevant AVCodecContext field is marked as "set
> by
> libavcodec" for decoding, which means that decoders expect it to be
> blank initially.
Why do you think that decoders would expect it to be blank initially?
What about the other fields then, like format, profile, level, width,
height, etc. - these are copied in the same way.
Are you saying that decoders expect all other fields to be filled but
the properties field to be zero?
Also, the docs say that it's SET by decoders, not READ by decoders,
which means that no decoder could rely on this field. There's also
no undefined-state defined (like -1), which means that a decoder
couldn't know whether a 0-value indicates a valid value or a not-yet
initialized value.
> Imagine a scenario where the beginning (that gets
> read
> in avformat_find_stream_info()) of an input stream has e.g. embedded
> cc
> subtitles, but the part that lateron gets fed to the decoder does
> not.
> If the decoder has been initialized from the AVStream's
> AVCodecParameters, then said decoder would still indicate that there
> are
> closed captions.* IIRC Anton already mentioned that this is actually
> a
> frame/packet property, not a stream property.
That statement was not thought through. Of course it IS a frame
property as well, but there's no difference to other properties like
width, height or aspect ratio:
- as a frame property, it is indicating whether a frame carries
closed caption data
- as a stream property, it indicates the summary result from the
probing process
Just the same like some other fields in this struct.
Kind regards,
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] next release
Hi Should the next release be called "LTS"? Should the next release be 4.5 or 5.0 ? Should it be made in december 2021 ? (as was suggested in jbs release mail) thx -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Does the universe only have a finite lifespan? No, its going to go on forever, its just that you wont like living in it. -- Hiranya Peiri signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 1/8] avcodec/bitpacked: check av_buffer_ref result
From: Limin Wang
Signed-off-by: Limin Wang
---
libavcodec/bitpacked.c | 4
1 file changed, 4 insertions(+)
diff --git a/libavcodec/bitpacked.c b/libavcodec/bitpacked.c
index 8143151..0440df9 100644
--- a/libavcodec/bitpacked.c
+++ b/libavcodec/bitpacked.c
@@ -45,6 +45,10 @@ static int bitpacked_decode_uyvy422(AVCodecContext *avctx,
AVFrame *frame,
/* there is no need to copy as the data already match
* a known pixel format */
frame->buf[0] = av_buffer_ref(avpkt->buf);
+if (!frame->buf[0]) {
+return AVERROR(ENOMEM);
+}
+
ret = av_image_fill_arrays(frame->data, frame->linesize, avpkt->data,
avctx->pix_fmt, avctx->width, avctx->height, 1);
if (ret < 0) {
--
1.8.3.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 2/8] avformat/rtpdec_rfc4175: use av_get_bits_per_pixel()
From: Limin Wang
Signed-off-by: Limin Wang
---
libavformat/rtpdec_rfc4175.c | 7 +++
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/libavformat/rtpdec_rfc4175.c b/libavformat/rtpdec_rfc4175.c
index 4daff4d..e1e1452 100644
--- a/libavformat/rtpdec_rfc4175.c
+++ b/libavformat/rtpdec_rfc4175.c
@@ -43,8 +43,8 @@ struct PayloadContext {
static int rfc4175_parse_format(AVStream *stream, PayloadContext *data)
{
enum AVPixelFormat pixfmt = AV_PIX_FMT_NONE;
-int bits_per_sample = 0;
int tag = 0;
+const AVPixFmtDescriptor *desc;
if (!strncmp(data->sampling, "YCbCr-4:2:2", 11)) {
tag = MKTAG('U', 'Y', 'V', 'Y');
@@ -52,11 +52,9 @@ static int rfc4175_parse_format(AVStream *stream,
PayloadContext *data)
if (data->depth == 8) {
data->pgroup = 4;
-bits_per_sample = 16;
pixfmt = AV_PIX_FMT_UYVY422;
} else if (data->depth == 10) {
data->pgroup = 5;
-bits_per_sample = 20;
pixfmt = AV_PIX_FMT_YUV422P10;
} else {
return AVERROR_INVALIDDATA;
@@ -65,9 +63,10 @@ static int rfc4175_parse_format(AVStream *stream,
PayloadContext *data)
return AVERROR_INVALIDDATA;
}
+desc = av_pix_fmt_desc_get(pixfmt);
stream->codecpar->format = pixfmt;
stream->codecpar->codec_tag = tag;
-stream->codecpar->bits_per_coded_sample = bits_per_sample;
+stream->codecpar->bits_per_coded_sample = av_get_bits_per_pixel(desc);
data->frame_size = data->width * data->height * data->pgroup / data->xinc;
return 0;
--
1.8.3.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 3/8] avformat/rtpdec_rfc4175: Remove redundant initialization
From: Limin Wang
Signed-off-by: Limin Wang
---
libavformat/rtpdec_rfc4175.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/libavformat/rtpdec_rfc4175.c b/libavformat/rtpdec_rfc4175.c
index e1e1452..97939b0 100644
--- a/libavformat/rtpdec_rfc4175.c
+++ b/libavformat/rtpdec_rfc4175.c
@@ -42,8 +42,8 @@ struct PayloadContext {
static int rfc4175_parse_format(AVStream *stream, PayloadContext *data)
{
-enum AVPixelFormat pixfmt = AV_PIX_FMT_NONE;
-int tag = 0;
+enum AVPixelFormat pixfmt;
+int tag;
const AVPixFmtDescriptor *desc;
if (!strncmp(data->sampling, "YCbCr-4:2:2", 11)) {
--
1.8.3.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 4/8] avformat/rtpdec_rfc4175: return the proper value
From: Limin Wang Signed-off-by: Limin Wang --- libavformat/rtpdec_rfc4175.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavformat/rtpdec_rfc4175.c b/libavformat/rtpdec_rfc4175.c index 97939b0..46d30ed 100644 --- a/libavformat/rtpdec_rfc4175.c +++ b/libavformat/rtpdec_rfc4175.c @@ -105,7 +105,7 @@ static int rfc4175_parse_sdp_line(AVFormatContext *s, int st_index, if (!data->sampling || !data->depth || !data->width || !data->height) -return -1; +return AVERROR(EINVAL); stream->codecpar->width = data->width; stream->codecpar->height = data->height; -- 1.8.3.1 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 5/8] avformat/rtpdec_rfc4175: add support for exactframerate
From: Limin Wang
Signed-off-by: Limin Wang
---
libavformat/rtpdec_rfc4175.c | 14 ++
1 file changed, 14 insertions(+)
diff --git a/libavformat/rtpdec_rfc4175.c b/libavformat/rtpdec_rfc4175.c
index 46d30ed..367567d 100644
--- a/libavformat/rtpdec_rfc4175.c
+++ b/libavformat/rtpdec_rfc4175.c
@@ -25,9 +25,11 @@
#include "rtpdec_formats.h"
#include "libavutil/avstring.h"
#include "libavutil/pixdesc.h"
+#include "libavutil/parseutils.h"
struct PayloadContext {
char *sampling;
+char *framerate;
int depth;
int width;
int height;
@@ -45,6 +47,7 @@ static int rfc4175_parse_format(AVStream *stream,
PayloadContext *data)
enum AVPixelFormat pixfmt;
int tag;
const AVPixFmtDescriptor *desc;
+AVRational framerate;
if (!strncmp(data->sampling, "YCbCr-4:2:2", 11)) {
tag = MKTAG('U', 'Y', 'V', 'Y');
@@ -69,6 +72,14 @@ static int rfc4175_parse_format(AVStream *stream,
PayloadContext *data)
stream->codecpar->bits_per_coded_sample = av_get_bits_per_pixel(desc);
data->frame_size = data->width * data->height * data->pgroup / data->xinc;
+if (data->framerate) {
+if (av_parse_video_rate(&framerate, data->framerate) < 0)
+return AVERROR(EINVAL);
+stream->avg_frame_rate = framerate;
+if (framerate.den)
+stream->codecpar->bit_rate = data->frame_size * av_q2d(framerate)
* 8;
+}
+
return 0;
}
@@ -84,6 +95,8 @@ static int rfc4175_parse_fmtp(AVFormatContext *s, AVStream
*stream,
data->sampling = av_strdup(value);
else if (!strncmp(attr, "depth", 5))
data->depth = atoi(value);
+else if (!strncmp(attr, "exactframerate", 14))
+data->framerate = av_strdup(value);
return 0;
}
@@ -112,6 +125,7 @@ static int rfc4175_parse_sdp_line(AVFormatContext *s, int
st_index,
ret = rfc4175_parse_format(stream, data);
av_freep(&data->sampling);
+av_freep(&data->framerate);
return ret;
}
--
1.8.3.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 6/8] avformat/rtpdec_rfc4175: add support for TCS
From: Limin Wang
Signed-off-by: Limin Wang
---
libavformat/rtpdec_rfc4175.c | 19 +++
1 file changed, 19 insertions(+)
diff --git a/libavformat/rtpdec_rfc4175.c b/libavformat/rtpdec_rfc4175.c
index 367567d..db6b9b8 100644
--- a/libavformat/rtpdec_rfc4175.c
+++ b/libavformat/rtpdec_rfc4175.c
@@ -30,6 +30,7 @@
struct PayloadContext {
char *sampling;
char *framerate;
+char *TCS;
int depth;
int width;
int height;
@@ -66,6 +67,21 @@ static int rfc4175_parse_format(AVStream *stream,
PayloadContext *data)
return AVERROR_INVALIDDATA;
}
+if (!data->TCS)
+stream->codecpar->color_trc = AVCOL_TRC_UNSPECIFIED;
+else if (!strncmp(data->TCS, "SDR", 3))
+stream->codecpar->color_trc = AVCOL_TRC_BT709;
+else if (!strncmp(data->TCS, "PQ", 2))
+stream->codecpar->color_trc = AVCOL_TRC_SMPTE2084;
+else if (!strncmp(data->TCS, "HLG", 3))
+stream->codecpar->color_trc = AVCOL_TRC_ARIB_STD_B67;
+else if (!strncmp(data->TCS, "LINEAR", 6))
+stream->codecpar->color_trc = AVCOL_TRC_LINEAR;
+else if (!strncmp(data->TCS, "ST428-1", 7))
+stream->codecpar->color_trc = AVCOL_TRC_SMPTEST428_1;
+else if (!strncmp(data->TCS, "UNSPECIFIED", 11))
+stream->codecpar->color_trc = AVCOL_TRC_UNSPECIFIED;
+
desc = av_pix_fmt_desc_get(pixfmt);
stream->codecpar->format = pixfmt;
stream->codecpar->codec_tag = tag;
@@ -97,6 +113,8 @@ static int rfc4175_parse_fmtp(AVFormatContext *s, AVStream
*stream,
data->depth = atoi(value);
else if (!strncmp(attr, "exactframerate", 14))
data->framerate = av_strdup(value);
+else if (!strncmp(attr, "TCS", 3))
+data->TCS = av_strdup(value);
return 0;
}
@@ -126,6 +144,7 @@ static int rfc4175_parse_sdp_line(AVFormatContext *s, int
st_index,
ret = rfc4175_parse_format(stream, data);
av_freep(&data->sampling);
av_freep(&data->framerate);
+av_freep(&data->TCS);
return ret;
}
--
1.8.3.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 7/8] avformat/rtpdec_rfc4175: add support for colorimetry
From: Limin Wang
Signed-off-by: Limin Wang
---
libavformat/rtpdec_rfc4175.c | 18 ++
1 file changed, 18 insertions(+)
diff --git a/libavformat/rtpdec_rfc4175.c b/libavformat/rtpdec_rfc4175.c
index db6b9b8..23a3f4f 100644
--- a/libavformat/rtpdec_rfc4175.c
+++ b/libavformat/rtpdec_rfc4175.c
@@ -31,6 +31,7 @@ struct PayloadContext {
char *sampling;
char *framerate;
char *TCS;
+char *colorimetry;
int depth;
int width;
int height;
@@ -82,6 +83,20 @@ static int rfc4175_parse_format(AVStream *stream,
PayloadContext *data)
else if (!strncmp(data->TCS, "UNSPECIFIED", 11))
stream->codecpar->color_trc = AVCOL_TRC_UNSPECIFIED;
+if (!data->colorimetry) {
+stream->codecpar->color_primaries = AVCOL_PRI_UNSPECIFIED;
+stream->codecpar->color_space = AVCOL_SPC_UNSPECIFIED;
+} else if (!strncmp(data->colorimetry, "BT601", 5)) {
+stream->codecpar->color_primaries = AVCOL_PRI_BT470BG;
+stream->codecpar->color_space = AVCOL_SPC_BT470BG;
+} else if (!strncmp(data->colorimetry, "BT709", 5)) {
+stream->codecpar->color_primaries = AVCOL_PRI_BT709;
+stream->codecpar->color_space = AVCOL_SPC_BT709;
+} else if (!strncmp(data->colorimetry, "BT2020", 6)) {
+stream->codecpar->color_primaries = AVCOL_PRI_BT2020;
+stream->codecpar->color_space = AVCOL_SPC_BT2020_NCL;
+}
+
desc = av_pix_fmt_desc_get(pixfmt);
stream->codecpar->format = pixfmt;
stream->codecpar->codec_tag = tag;
@@ -115,6 +130,8 @@ static int rfc4175_parse_fmtp(AVFormatContext *s, AVStream
*stream,
data->framerate = av_strdup(value);
else if (!strncmp(attr, "TCS", 3))
data->TCS = av_strdup(value);
+else if (!strncmp(attr, "colorimetry", 11))
+data->colorimetry = av_strdup(value);
return 0;
}
@@ -145,6 +162,7 @@ static int rfc4175_parse_sdp_line(AVFormatContext *s, int
st_index,
av_freep(&data->sampling);
av_freep(&data->framerate);
av_freep(&data->TCS);
+av_freep(&data->colorimetry);
return ret;
}
--
1.8.3.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 8/8] avformat/rtpdec_rfc4175: add support for RANGE
From: Limin Wang
Signed-off-by: Limin Wang
---
libavformat/rtpdec_rfc4175.c | 11 +++
1 file changed, 11 insertions(+)
diff --git a/libavformat/rtpdec_rfc4175.c b/libavformat/rtpdec_rfc4175.c
index 23a3f4f..86ffb38 100644
--- a/libavformat/rtpdec_rfc4175.c
+++ b/libavformat/rtpdec_rfc4175.c
@@ -32,6 +32,7 @@ struct PayloadContext {
char *framerate;
char *TCS;
char *colorimetry;
+char *range;
int depth;
int width;
int height;
@@ -97,6 +98,13 @@ static int rfc4175_parse_format(AVStream *stream,
PayloadContext *data)
stream->codecpar->color_space = AVCOL_SPC_BT2020_NCL;
}
+if (!data->range)
+stream->codecpar->color_range = AVCOL_RANGE_MPEG;
+else if (!strncmp(data->range, "NARROW", 6))
+stream->codecpar->color_range = AVCOL_RANGE_MPEG;
+else if (!strncmp(data->range, "FULL", 4))
+stream->codecpar->color_range = AVCOL_RANGE_JPEG;
+
desc = av_pix_fmt_desc_get(pixfmt);
stream->codecpar->format = pixfmt;
stream->codecpar->codec_tag = tag;
@@ -132,6 +140,8 @@ static int rfc4175_parse_fmtp(AVFormatContext *s, AVStream
*stream,
data->TCS = av_strdup(value);
else if (!strncmp(attr, "colorimetry", 11))
data->colorimetry = av_strdup(value);
+else if (!strncmp(attr, "RANGE", 5))
+data->range = av_strdup(value);
return 0;
}
@@ -163,6 +173,7 @@ static int rfc4175_parse_sdp_line(AVFormatContext *s, int
st_index,
av_freep(&data->framerate);
av_freep(&data->TCS);
av_freep(&data->colorimetry);
+av_freep(&data->range);
return ret;
}
--
1.8.3.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v3 1/2] avcodec/codec_par: Add codec properties field to AVCodecParameters
Soft Works:
>
>
>> -Original Message-
>> From: ffmpeg-devel On Behalf Of
>> Andreas Rheinhardt
>> Sent: Wednesday, October 6, 2021 11:30 AM
>> To: ffmpeg-devel@ffmpeg.org
>> Subject: Re: [FFmpeg-devel] [PATCH v3 1/2] avcodec/codec_par: Add
>> codec properties field to AVCodecParameters
>>
>> Soft Works:
>>>
>>>
-Original Message-
From: ffmpeg-devel On Behalf Of
Hendrik Leppkes
Sent: Wednesday, October 6, 2021 8:57 AM
To: FFmpeg development discussions and patches >>> de...@ffmpeg.org>
Subject: Re: [FFmpeg-devel] [PATCH v3 1/2] avcodec/codec_par: Add
codec properties field to AVCodecParameters
On Wed, Oct 6, 2021 at 8:45 AM Soft Works
wrote:
> diff --git a/libavcodec/codec_par.h b/libavcodec/codec_par.h
> index 10cf79dff1..42ed8deb13 100644
> --- a/libavcodec/codec_par.h
> +++ b/libavcodec/codec_par.h
> @@ -198,6 +198,10 @@ typedef struct AVCodecParameters {
> * Audio only. Number of samples to skip after a
discontinuity.
> */
> int seek_preroll;
> +/**
> + * Codec properties of the stream that gets decoded
> + */
> +unsigned properties;
> } AVCodecParameters;
>
This field is severly underspecified/underdocumented. I realize
>> you
just copied it, but if I'm looking at this without pre-existing
knowledge, then I have absolutely no idea what it might be for, or
what kind of values go in there. The old field had the defines of
>> the
possible bits right there for context - this doesn't have this, so
>> it
would definitely benefit from added documentation.
>>>
>>> Hello Hendrik,
>>>
>>> that's right, but what would you suggest? We can't duplicate the
>> defines,
>>> though I could add a line in the help text like
>>>
>>> /**
>>> * Codec properties of the stream that gets decoded.
>>> * Corresponds to @ref AVCodecContext.properties "properties".
>>> */
>>> unsigned properties;
>>>
>>> Would that be OK?
>>>
>>
>> The defines could be moved to defs.h (while just at it, we could also
>> add properly prefixed alternatives and deprecate the FF_ ones).
>> The copying that you are adding in avcodec_parameters_from_context
>> is
>> problematic, as the relevant AVCodecContext field is marked as "set
>> by
>> libavcodec" for decoding, which means that decoders expect it to be
>> blank initially.
>
> Why do you think that decoders would expect it to be blank initially?
>
> What about the other fields then, like format, profile, level, width,
> height, etc. - these are copied in the same way.
> Are you saying that decoders expect all other fields to be filled but
> the properties field to be zero?
>
Of course not. E.g. the documentation of the dimension explicitly says
so: "May be set by the user before opening the decoder if known e.g.
from the container. Some decoders will require the dimensions to be set
by the caller. During decoding, the decoder may overwrite those values
as required while parsing the data."
The semantics of profile and level are closer, yet the difference is
that if the decoder sets these fields, it overwrites any garbage it may
have had; this is not so with a bitfield like properties.
> Also, the docs say that it's SET by decoders, not READ by decoders,
> which means that no decoder could rely on this field. There's also
> no undefined-state defined (like -1), which means that a decoder
> couldn't know whether a 0-value indicates a valid value or a not-yet
> initialized value.
>
You completely misunderstood what my concern was about: In the scenario
described above an API user querying the properties field would get the
information that this stream contains CC data; even when it doesn't.
And your other claims are also wrong: A decoder may read anything from
an AVCodecContext or from the AVPackets and AVFrames with the exception
of data that is opaque to it. And if a field is described as
out-of-touch by the user, then the codec may rely on it not having
arbitrary values, but only the values it has set itself (or the default
value). Notice that if the user initializes the AVCodecContext via
avcodec_parameters_from_context(), then it still counts as user-set even
though said function lives in libavcodec, too.
>> Imagine a scenario where the beginning (that gets
>> read
>> in avformat_find_stream_info()) of an input stream has e.g. embedded
>> cc
>> subtitles, but the part that lateron gets fed to the decoder does
>> not.
>> If the decoder has been initialized from the AVStream's
>> AVCodecParameters, then said decoder would still indicate that there
>> are
>> closed captions.* IIRC Anton already mentioned that this is actually
>> a
>> frame/packet property, not a stream property.
>
> That statement was not thought through. Of course it IS a frame
> property as well, but there's no difference to other properties like
> width, height or aspect ratio:
>
> - as a frame
Re: [FFmpeg-devel] next release
Hello, On Wed, 6 Oct 2021, at 11:57, Michael Niedermayer wrote: > Should the next release be called "LTS"? Yes. > Should the next release be 4.5 or 5.0 ? 5.0 seeing the major break in API. > Should it be made in december 2021 ? (as was suggested in jbs release mail) I still think so. ( :D ) -- Jean-Baptiste Kempf - President +33 672 704 734 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 4/4] avcodec/hevcdec: set the film grain property if present
On Tue, Sep 28, 2021 at 09:36:38PM +0800, lance.lmw...@gmail.com wrote: > From: Limin Wang > > Signed-off-by: Limin Wang > --- > libavcodec/hevcdec.c | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/libavcodec/hevcdec.c b/libavcodec/hevcdec.c > index 57a6175..b4edb3b 100644 > --- a/libavcodec/hevcdec.c > +++ b/libavcodec/hevcdec.c > @@ -385,6 +385,9 @@ static int export_stream_params_from_sei(HEVCContext *s) > avctx->color_trc = > s->sei.alternative_transfer.preferred_transfer_characteristics; > } > > +if (s->sei.film_grain_characteristics.present) > +avctx->properties |= FF_CODEC_PROPERTY_FILM_GRAIN; > + > return 0; > } > > -- > 1.8.3.1 > will apply patch 2,3(update log message), 4 after two day unless there are any objections. -- Thanks, Limin Wang ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH] fate/image: Modify aliaspix tests to also cover the encoder
Signed-off-by: Andreas Rheinhardt --- Will apply this tonight unless there are objections. tests/fate/image.mak | 9 ++--- tests/ref/fate/aliaspix-bgr | 8 tests/ref/fate/aliaspix-gray | 8 3 files changed, 22 insertions(+), 3 deletions(-) diff --git a/tests/fate/image.mak b/tests/fate/image.mak index d1f9765eea..573d398915 100644 --- a/tests/fate/image.mak +++ b/tests/fate/image.mak @@ -1,10 +1,13 @@ FATE_ALIASPIX += fate-aliaspix-bgr -fate-aliaspix-bgr: CMD = framecrc -i $(TARGET_SAMPLES)/aliaspix/first.pix -pix_fmt bgr24 +fate-aliaspix-bgr: CMD = transcode alias_pix $(TARGET_SAMPLES)/aliaspix/first.pix image2 "-c alias_pix" "-map 0 -map 0 -pix_fmt:0 bgr24 -c:v:1 copy" FATE_ALIASPIX += fate-aliaspix-gray -fate-aliaspix-gray: CMD = framecrc -i $(TARGET_SAMPLES)/aliaspix/firstgray.pix -pix_fmt gray +fate-aliaspix-gray: CMD = transcode alias_pix $(TARGET_SAMPLES)/aliaspix/firstgray.pix image2 "-c alias_pix" "-map 0 -map 0 -pix_fmt:0 gray -c:v:1 copy" -FATE_ALIASPIX-$(call DEMDEC, IMAGE2, ALIAS_PIX) += $(FATE_ALIASPIX) +FATE_ALIASPIX-$(call ALLYES, FILE_PROTOCOL IMAGE2_ALIAS_PIX_DEMUXER \ + ALIAS_PIX_DECODER ALIAS_PIX_ENCODER\ + IMAGE2_MUXER RAWVIDEO_ENCODER \ + FRAMECRC_MUXER PIPE_PROTOCOL) += $(FATE_ALIASPIX) FATE_IMAGE += $(FATE_ALIASPIX-yes) fate-aliaspix: $(FATE_ALIASPIX-yes) diff --git a/tests/ref/fate/aliaspix-bgr b/tests/ref/fate/aliaspix-bgr index f5c6b7d475..4f55e0dcf6 100644 --- a/tests/ref/fate/aliaspix-bgr +++ b/tests/ref/fate/aliaspix-bgr @@ -1,6 +1,14 @@ +667513c8869eea09e1374a73eb21233c *tests/data/fate/aliaspix-bgr.image2 +63090 tests/data/fate/aliaspix-bgr.image2 #tb 0: 1/25 #media_type 0: video #codec_id 0: rawvideo #dimensions 0: 201x79 #sar 0: 0/1 +#tb 1: 1/25 +#media_type 1: video +#codec_id 1: alias_pix +#dimensions 1: 201x79 +#sar 1: 0/1 0, 0, 0,1,47637, 0xcb77368f +1, 0, 0,1,63090, 0xe49bba3b diff --git a/tests/ref/fate/aliaspix-gray b/tests/ref/fate/aliaspix-gray index aa7b4649b2..4bf9c43e0e 100644 --- a/tests/ref/fate/aliaspix-gray +++ b/tests/ref/fate/aliaspix-gray @@ -1,6 +1,14 @@ +2d7ae310710798257ebcff8a41c646ee *tests/data/fate/aliaspix-gray.image2 +28868 tests/data/fate/aliaspix-gray.image2 #tb 0: 1/25 #media_type 0: video #codec_id 0: rawvideo #dimensions 0: 199x81 #sar 0: 0/1 +#tb 1: 1/25 +#media_type 1: video +#codec_id 1: alias_pix +#dimensions 1: 199x81 +#sar 1: 0/1 0, 0, 0,1,16119, 0x3d0785f5 +1, 0, 0,1,28868, 0x02523ce2 -- 2.30.2 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v3 1/2] avcodec/codec_par: Add codec properties field to AVCodecParameters
> -Original Message- > From: ffmpeg-devel On Behalf Of > Andreas Rheinhardt > Sent: Wednesday, October 6, 2021 12:18 PM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH v3 1/2] avcodec/codec_par: Add > codec properties field to AVCodecParameters > > Soft Works: > > > > > >> -Original Message- > >> From: ffmpeg-devel On Behalf Of > >> Andreas Rheinhardt > >> Sent: Wednesday, October 6, 2021 11:30 AM > >> To: ffmpeg-devel@ffmpeg.org > >> Subject: Re: [FFmpeg-devel] [PATCH v3 1/2] avcodec/codec_par: Add > >> codec properties field to AVCodecParameters > >> > >> Soft Works: > >>> > >>> > -Original Message- > From: ffmpeg-devel On Behalf > Of > Hendrik Leppkes > Sent: Wednesday, October 6, 2021 8:57 AM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v3 1/2] avcodec/codec_par: > Add > codec properties field to AVCodecParameters > > On Wed, Oct 6, 2021 at 8:45 AM Soft Works [...] > >>> > >> > >> The defines could be moved to defs.h (while just at it, we could > also > >> add properly prefixed alternatives and deprecate the FF_ ones). > >> The copying that you are adding in > avcodec_parameters_from_context > >> is > >> problematic, as the relevant AVCodecContext field is marked as > "set > >> by > >> libavcodec" for decoding, which means that decoders expect it to > be > >> blank initially. > > > > Why do you think that decoders would expect it to be blank > initially? > > > > What about the other fields then, like format, profile, level, > width, > > height, etc. - these are copied in the same way. > > Are you saying that decoders expect all other fields to be filled > but > > the properties field to be zero? > > > > Of course not. E.g. the documentation of the dimension explicitly > says > so: "May be set by the user before opening the decoder if known e.g. > from the container. Some decoders will require the dimensions to be > set > by the caller. During decoding, the decoder may overwrite those > values > as required while parsing the data." The docs for properties don't say "May be set by the user". > You completely misunderstood what my concern was about: In the > scenario > described above an API user querying the properties field would get > the > information that this stream contains CC data; even when it doesn't. But where does that codec_par data come from? When it's set by the user he has no reason to be surprised when it doesn't match reality. When it comes from probing, then it DID have CC data during probing which means that even if it the CC data has stopped to be transmitted exactly within those few milliseconds between probing and decoding, it is still likely that it will have CC data again at a future point in time. (even though that's not a realistic scenario at all; practically tv streams either have always cc or never cc) Kind regards, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 2/2] avcodec/aliaspixenc: Remove redundant counter
Improves performance by 33.8% for BGR24 and by 26.4% for GRAY8.
Signed-off-by: Andreas Rheinhardt
---
libavcodec/aliaspixenc.c | 16
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/libavcodec/aliaspixenc.c b/libavcodec/aliaspixenc.c
index 01461c984b..fa273df9c2 100644
--- a/libavcodec/aliaspixenc.c
+++ b/libavcodec/aliaspixenc.c
@@ -31,8 +31,8 @@
static int encode_frame(AVCodecContext *avctx, AVPacket *pkt,
const AVFrame *frame, int *got_packet)
{
-int width, height, bits_pixel, i, j, length, ret;
-uint8_t *in_buf, *buf;
+int width, height, bits_pixel, length, ret;
+uint8_t *buf;
width = avctx->width;
height = avctx->height;
@@ -66,15 +66,16 @@ static int encode_frame(AVCodecContext *avctx, AVPacket
*pkt,
bytestream_put_be32(&buf, 0); /* X, Y offset */
bytestream_put_be16(&buf, bits_pixel);
-for (j = 0; j < height; j++) {
-in_buf = frame->data[0] + frame->linesize[0] * j;
-for (i = 0; i < width; ) {
+for (int j = 0, bytes_pixel = bits_pixel >> 3; j < height; j++) {
+const uint8_t *in_buf = frame->data[0] + frame->linesize[0] * j;
+const uint8_t *const line_end = in_buf + bytes_pixel * width;
+while (in_buf < line_end) {
int count = 0;
int pixel;
if (avctx->pix_fmt == AV_PIX_FMT_GRAY8) {
pixel = *in_buf;
-while (count < 255 && count + i < width && pixel == *in_buf) {
+while (count < 255 && in_buf < line_end && pixel == *in_buf) {
count++;
in_buf++;
}
@@ -82,7 +83,7 @@ static int encode_frame(AVCodecContext *avctx, AVPacket *pkt,
bytestream_put_byte(&buf, pixel);
} else { /* AV_PIX_FMT_BGR24 */
pixel = AV_RB24(in_buf);
-while (count < 255 && count + i < width &&
+while (count < 255 && in_buf < line_end &&
pixel == AV_RB24(in_buf)) {
count++;
in_buf += 3;
@@ -90,7 +91,6 @@ static int encode_frame(AVCodecContext *avctx, AVPacket *pkt,
bytestream_put_byte(&buf, count);
bytestream_put_be24(&buf, pixel);
}
-i += count;
}
}
--
2.30.2
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v3 1/2] avcodec/codec_par: Add codec properties field to AVCodecParameters
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Andreas Rheinhardt
> Sent: Wednesday, October 6, 2021 12:18 PM
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v3 1/2] avcodec/codec_par: Add
> codec properties field to AVCodecParameters
>
> Soft Works:
> >
> >
> >> -Original Message-
> >> From: ffmpeg-devel On Behalf Of
> >> Andreas Rheinhardt
> >> Sent: Wednesday, October 6, 2021 11:30 AM
> >> To: ffmpeg-devel@ffmpeg.org
> >> Subject: Re: [FFmpeg-devel] [PATCH v3 1/2] avcodec/codec_par: Add
> >> codec properties field to AVCodecParameters
> >>
> >> Soft Works:
> >>>
> >>>
> -Original Message-
> From: ffmpeg-devel On Behalf
> Of
> Hendrik Leppkes
> Sent: Wednesday, October 6, 2021 8:57 AM
> To: FFmpeg development discussions and patches de...@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v3 1/2] avcodec/codec_par:
> Add
> codec properties field to AVCodecParameters
>
> On Wed, Oct 6, 2021 at 8:45 AM Soft Works
>
> wrote:
> > diff --git a/libavcodec/codec_par.h b/libavcodec/codec_par.h
> > index 10cf79dff1..42ed8deb13 100644
> > --- a/libavcodec/codec_par.h
> > +++ b/libavcodec/codec_par.h
> > @@ -198,6 +198,10 @@ typedef struct AVCodecParameters {
> > * Audio only. Number of samples to skip after a
> discontinuity.
> > */
> > int seek_preroll;
> > +/**
> > + * Codec properties of the stream that gets decoded
> > + */
> > +unsigned properties;
> > } AVCodecParameters;
> >
>
> This field is severly underspecified/underdocumented. I realize
> >> you
> just copied it, but if I'm looking at this without pre-existing
> knowledge, then I have absolutely no idea what it might be for,
> or
> what kind of values go in there. The old field had the defines
> of
> >> the
> possible bits right there for context - this doesn't have this,
> so
> >> it
> would definitely benefit from added documentation.
> >>>
> >>> Hello Hendrik,
> >>>
> >>> that's right, but what would you suggest? We can't duplicate the
> >> defines,
> >>> though I could add a line in the help text like
> >>>
> >>> /**
> >>> * Codec properties of the stream that gets decoded.
> >>> * Corresponds to @ref AVCodecContext.properties
> "properties".
> >>> */
> >>> unsigned properties;
> >>>
> >>> Would that be OK?
> >>>
> >>
> >> The defines could be moved to defs.h (while just at it, we could
> also
> >> add properly prefixed alternatives and deprecate the FF_ ones).
> >> The copying that you are adding in
> avcodec_parameters_from_context
> >> is
> >> problematic, as the relevant AVCodecContext field is marked as
> "set
> >> by
> >> libavcodec" for decoding, which means that decoders expect it to
> be
> >> blank initially.
> >
> > Why do you think that decoders would expect it to be blank
> initially?
> >
> > What about the other fields then, like format, profile, level,
> width,
> > height, etc. - these are copied in the same way.
> > Are you saying that decoders expect all other fields to be filled
> but
> > the properties field to be zero?
> >
>
> Of course not. E.g. the documentation of the dimension explicitly
> says
> so: "May be set by the user before opening the decoder if known e.g.
> from the container. Some decoders will require the dimensions to be
> set
> by the caller. During decoding, the decoder may overwrite those
> values
> as required while parsing the data."
> The semantics of profile and level are closer, yet the difference is
> that if the decoder sets these fields, it overwrites any garbage it
> may
> have had; this is not so with a bitfield like properties.
>
> > Also, the docs say that it's SET by decoders, not READ by decoders,
> > which means that no decoder could rely on this field. There's also
> > no undefined-state defined (like -1), which means that a decoder
> > couldn't know whether a 0-value indicates a valid value or a not-
> yet
> > initialized value.
> >
>
> You completely misunderstood what my concern was about: In the
> scenario
> described above an API user querying the properties field would get
> the
> information that this stream contains CC data; even when it doesn't.
>
> And your other claims are also wrong: A decoder may read anything
> from
> an AVCodecContext or from the AVPackets and AVFrames with the
> exception
> of data that is opaque to it. And if a field is described as
> out-of-touch by the user, then the codec may rely on it not having
> arbitrary values, but only the values it has set itself (or the
> default
> value). Notice that if the user initializes the AVCodecContext via
> avcodec_parameters_from_context(), then it still counts as user-set
> even
> though said function lives in libavcodec, too.
When you think that this is the wrong way, what would you suggest?
Thanks,
sw
___
ffm
Re: [FFmpeg-devel] next release
On 10/6/2021 6:57 AM, Michael Niedermayer wrote: Hi Should the next release be called "LTS"? Should the next release be 4.5 or 5.0 ? 5.0. The major bump removed old core API, so it needs to be reflected in more ways than just soname. Should it be made in december 2021 ? (as was suggested in jbs release mail) Seeing it's a huge departure from 4.x, the chances for it to be included by Debian and Ubuntu LTS (Which comes out next April) depends on all packages that use libav* having migrated to the new APIs. I want to believe this already happened, considering the most important one is in place since like ffmpeg 3.0 from many years ago, but you never know. My point is, if this isn't the case by the time Debian and Ubuntu freeze, they will skip 5.0 for their upcoming releases, keep using 4.4, and that one will therefore become an LTS release by force, like it's currently the case with 4.3, 4.1, and 3.2. Also, wouldn't it be better to make the last release using a given soname the LTS, instead of the first? The codebase will be more mature by then, it could potentially include pre-bump deprecations (like av_packet_init in 4.4) to reduce their widespread use and ensure library users have an easier time migrating in the future, and it would probably simplify backporting fixes from master (Right now there are more conflicts backporting to 4.1 than to 4.4, for example). thx ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] next release
LTS releases would be a superb option for corporate FFMpeg users. От: ffmpeg-devel от имени Michael Niedermayer Отправлено: 6 октября 2021 г. 12:57 Кому: FFmpeg development discussions and patches Тема: [FFmpeg-devel] next release External email: Use caution opening links or attachments Hi Should the next release be called "LTS"? Should the next release be 4.5 or 5.0 ? Should it be made in december 2021 ? (as was suggested in jbs release mail) thx -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Does the universe only have a finite lifespan? No, its going to go on forever, its just that you wont like living in it. -- Hiranya Peiri ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2] DXVA2: Add ARGB format
On Tue, Sep 28, 2021 at 12:22 PM Soft Works wrote:
>
>
> > -Original Message-
> > From: ffmpeg-devel On Behalf Of
> > Soft Works
> > Sent: Thursday, 26 August 2021 21:43
> > To: FFmpeg development discussions and patches > de...@ffmpeg.org>
> > Subject: Re: [FFmpeg-devel] [PATCH v2] DXVA2: Add ARGB format
> >
> >
> >
> > > -Original Message-
> > > From: ffmpeg-devel On Behalf Of
> > > Soft Works
> > > Sent: Saturday, 7 August 2021 07:02
> > > To: ffmpeg-devel@ffmpeg.org
> > > Subject: [FFmpeg-devel] [PATCH v2] DXVA2: Add ARGB format
> > >
> > > Required for uploading frames with alpha for qsv_overlay
> > > (v2: remove tab indent)
> > >
> > > Signed-off-by: softworkz
> > > ---
> > > libavutil/hwcontext_dxva2.c | 1 +
> > > 1 file changed, 1 insertion(+)
> > >
> > > diff --git a/libavutil/hwcontext_dxva2.c
> > > b/libavutil/hwcontext_dxva2.c
> > > index 63b037da4a..9987cdc578 100644
> > > --- a/libavutil/hwcontext_dxva2.c
> > > +++ b/libavutil/hwcontext_dxva2.c
> > > @@ -83,6 +83,7 @@ static const struct {
> > > { MKTAG('N', 'V', '1', '2'), AV_PIX_FMT_NV12 },
> > > { MKTAG('P', '0', '1', '0'), AV_PIX_FMT_P010 },
> > > { D3DFMT_P8, AV_PIX_FMT_PAL8 },
> > > +{ D3DFMT_A8R8G8B8, AV_PIX_FMT_BGRA },
> > > };
>
LGTM. Will merge tomorrow if no one objects.
Aman
> > >
> > > DEFINE_GUID(video_decoder_service, 0xfc51a551, 0xd5e7, 0x11d9,
> > > 0xaf, 0x55, 0x00, 0x05, 0x4e, 0x43, 0xff, 0x02);
> > > --
> >
> > Could somebody push this please?
> >
> > Thanks,
> > softworkz
>
> Friendly ping.
>
> sw
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
>
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/3] avformat/matroskadec: Reset state also on failure in matroska_reset_status()
On Tue, Sep 14, 2021 at 06:13:43PM +0200, Michael Niedermayer wrote:
> On Mon, Sep 13, 2021 at 10:57:41PM +0200, Andreas Rheinhardt wrote:
> > Michael Niedermayer:
> > > On Mon, Jul 19, 2021 at 11:12:11PM +0200, Andreas Rheinhardt wrote:
> > >> Michael Niedermayer:
> > >>> On Fri, Jul 02, 2021 at 06:17:58PM +0200, Andreas Rheinhardt wrote:
> > Michael Niedermayer:
> > > The calling code does not handle failures and will fail with
> > > assertion failures later.
> > > Seeking can always fail even when the position was previously read.
> > >
> > > Fixes: Assertion failure
> > > Fixes:
> > > 35253/clusterfuzz-testcase-minimized-ffmpeg_dem_MATROSKA_fuzzer-4693059982983168
> > >
> > > Found-by: continuous fuzzing process
> > > https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> > > Signed-off-by: Michael Niedermayer
> > > ---
> > > libavformat/matroskadec.c | 19 ---
> > > 1 file changed, 12 insertions(+), 7 deletions(-)
> > >
> > > diff --git a/libavformat/matroskadec.c b/libavformat/matroskadec.c
> > > index 356a02339c..a0e6e0cf8b 100644
> > > --- a/libavformat/matroskadec.c
> > > +++ b/libavformat/matroskadec.c
> > > @@ -804,20 +804,22 @@ static int matroska_read_close(AVFormatContext
> > > *s);
> > > static int matroska_reset_status(MatroskaDemuxContext *matroska,
> > > uint32_t id, int64_t position)
> > > {
> > > +int64_t err = 0;
> > > if (position >= 0) {
> > > -int64_t err = avio_seek(matroska->ctx->pb, position,
> > > SEEK_SET);
> > > -if (err < 0)
> > > -return err;
> > > -}
> > > +err = avio_seek(matroska->ctx->pb, position, SEEK_SET);
> > > +if (err > 0)
> > > +err = 0;
> > > +} else
> > > +position = avio_tell(matroska->ctx->pb);
> > >
> > > matroska->current_id= id;
> > > matroska->num_levels= 1;
> > > matroska->unknown_count = 0;
> > > -matroska->resync_pos = avio_tell(matroska->ctx->pb);
> > > +matroska->resync_pos= position;
> > > if (id)
> > > matroska->resync_pos -= (av_log2(id) + 7) / 8;
> > >
> > > -return 0;
> > > +return err;
> >
> > The changes here will make the demuxer update its internal state as if
> > it had seeked to its target level-1-element, even though it didn't. Is
> > this really good?
> > >>>
> > >>> I dont know.
> > >>> Ive not seen this issue happen in reality just in a fuzzer
> > >>> environment.
> > >>>
> > >>
> > >> Can you send me this sample (with instructions how to reproduce it if
> > >> necessary)?
> > >
> > > This seems still "crashing" according to the tracker
> > > has this been fixed ?
> > >
> >
> > No, I have not found time to look at it. Proceed as you wish.
>
> well, i will look at other issues first then, maybe someone else
> will fix this ...
noone ? :(
if noone wants to look into this then ill apply the original patch
thx
[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
If you fake or manipulate statistics in a paper in physics you will never
get a job again.
If you fake or manipulate statistics in a paper in medicin you will get
a job for life at the pharma industry.
signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] point releases
Michael Niedermayer 于2021年10月6日周三 下午5:35写道： > > Hi all > > I do plan to make releases from > 4.4, 4.2, 4.1, 4.0, 3.4, 3.2, 2.8 branches Why not include 4.3 or have some reason? Thanks Steven ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
