Re: [FFmpeg-devel] [PATCH 1/2] docs/filters: add documentation to all existing OpenCL filters
On 31-07-2018 08:44 PM, Danil Iashchenko wrote:
This hinders overall progress on the documentation and filter implementation of
my GSoC project and there is not much time left. I suggest putting it on the
backburner for the moment and leaving it out until the next patch.
OK.
+@item -init_hw_device
@var{type}[=@var{name}][:@var{device}[,@var{key=value}...]]
s/type/opencl. This section is specifically opencl.
+@item planes
+Set which planes to filter. By default value @code{0xf}, all planes will be
processed.
--> "Set which planes to filter. Default value is @code{0xf}, by which
all planes are processed."
+@item sizeY
+Set vertical radius size, if zero it will be same as @code{sizeX}.
+Range is @code{[1, 1024]} and default value is @code{0}.
--> "Set vertical radius size. Range is @code{[1, 1024]} and default
value is @code{0}. If zero, @code{sizeX} value will be used."
+@section convolution_opencl
+@item 0mode
+@item 1mode
+@item 2mode
+@item 3mode
+Set matrix mode for each plane. Can be @var{square}, @var{row} or @var{column}.
+Default is @var{square}.
The opencl filter does not have mode selection options, software one
does. So I assume only square mode is used. Remove these entries and
change the docs for the others to remove references to modes.
+@section overlay_opencl
+@item x
+Set the x coordinate of the overlaid video on the main video.
+By default value is @code{0}.
--> "Default value is @code{0}."
+@item y
+Set the x coordinate of the overlaid video on the main video.
+By default value is @code{0}.
Same.
+Insert a PNG logo in the top-left corner of the input
What if it's a JPG logo? :) See below.
+@example
+-i INPUT -i LOGO -filter_complex "[0:v]hwupload[a], [1:v]format=yuva420p,
hwupload[b], [a][b]overlay_opencl, hwdownload" OUTPUT
So the reason, I suspect, you used PNG above is to indicate that pixel
format conversion may be required. In your earlier patch version, you
used JPG and no conversion.
So, for OpenCL, which all formats are acceptable without conversion?
This should be mentioned just below the start of the overall section,
after the device init part.
+@section prewitt_opencl
+@item planes
+Set which planes will be processed, unprocessed planes will be copied.
+By default value @code{0xf}, all planes will be processed.
Same as earlier.
+@section roberts_opencl
+@item planes
+Set which planes will be processed, unprocessed planes will be copied.
+By default value @code{0xf}, all planes will be processed.
Same as earlier.
+@section sobel_opencl
+@item planes
+Set which planes will be processed, unprocessed planes will be copied.
+By default value @code{0xf}, all planes will be processed.
Same as earlier.
+@section unsharp_opencl
+@item luma_msize_x, lx
+Set the luma matrix horizontal size. It must be an odd integer between
+@code{3} and @code{23}. The default value is @code{5}.
The source allows for value 1.
+@item luma_msize_y, ly
> +@item chroma_msize_x, cx
> +@item chroma_msize_y, cy
Same.
+@item luma_amount, la
+Set the luma effect strength. It must be a floating point number, reasonable
+values lay between @code{-1.5} and @code{1.5}.
Range missing.
+@item chroma_amount, ca
Same.
I still had 3 warnings. In your editor, check for trailing whitespaces
and remove those.
Regards,
Gyan
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] avformat/avs: add descriptive name and url
On 8/2/18, Gyan Doshi wrote:
> {no text body}
LGTM
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] avformat/avs: add descriptive name and url
Pushed as c9118d4d64a661fddd431a7437994c31eafb32c6 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] Why does ffmpeg h264 decoder stop decoding when certain type of packets are coming in the input stream?
Thank you Carl. Is this a known issue? Is there any known good version which does not have have this issue? On Wed, Aug 1, 2018 at 4:48 AM Carl Eugen Hoyos wrote: > 2018-07-28 14:57 GMT+02:00, Naveed Basha : > > Yes, please find the test stream here. I was using ffmpeg v3.4.2. > > > > https://drive.google.com/open?id=1rHCfG4csA3rB4LSgErEBn1F3WfI5nUVr > > This is now ticket #7340, thank you for the sample. > > Carl Eugen > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel > -- Cheers, Naveed ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] Mixture of homographies video stabilization
Hello! I want to write a more advanced video stabilizer for libavfilter (*), implementing the algorithm described here - https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/37744.pdf The focus of the paper is rolling shutter removal, but it builds on top of another algorithm that does full frame stabilization, and the new algorithm does that as well. This is the algorithm used in YouTube's stabilizing filter, and is state of the art. Adobe calls it Warp Stabilizer (it's the same thing as far as I can tell from public information anyways). 3 questions: 1. Is there a linear algebra library already in use? I didn't see anything in configure, but would be surprised if none of the existing filters work with matrices? 2. Is there anything to watch out for re. a high frame delay (say a few hundred frames)? Looking at the API, I don't see a callback to flush out remaining frames when input frames are finished? Is doing it in two passes the only option? 3. doc/writing_filters.txt says only slice threading is available. That's not really possible with this filter, but frame threading is. Can I just buffer frames internally (which I need to do anyways to smooth out motion), and do my own threading? * vid.stab is good for what it does, but it only does rotation and translation, and doesn't handle zoom, perspective distortion, or rolling shutter. This means it's limited when it comes to things like scuba diving videos, where the camera is filming the seabed at a small distance and at an angle. Thanks! Matthew ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] Mixture of homographies video stabilization
On 8/2/18, Matthew Lai wrote: > Hello! > > I want to write a more advanced video stabilizer for libavfilter (*), > implementing the algorithm described here - > https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/37744.pdf > The focus of the paper is rolling shutter removal, but it builds on top of > another algorithm that does full frame stabilization, and the new algorithm > does that as well. > > This is the algorithm used in YouTube's stabilizing filter, and is state of > the art. Adobe calls it Warp Stabilizer (it's the same thing as far as I > can tell from public information anyways). > > 3 questions: > 1. Is there a linear algebra library already in use? I didn't see anything > in configure, but would be surprised if none of the existing filters work > with matrices? There is no such library here used. There are indeed video/audio filters that work with matrices. > 2. Is there anything to watch out for re. a high frame delay (say a few > hundred frames)? Looking at the API, I don't see a callback to flush out > remaining frames when input frames are finished? Is doing it in two passes > the only option? It is handled internally, there are two internal APIs, activate one and legacy. With legacy you can flush frames when you receive last frame from input. With newer, activate API, its similar. > 3. doc/writing_filters.txt says only slice threading is available. That's > not really possible with this filter, but frame threading is. Can I just > buffer frames internally (which I need to do anyways to smooth out motion), > and do my own threading? You could do it. > > * vid.stab is good for what it does, but it only does rotation and > translation, and doesn't handle zoom, perspective distortion, or rolling > shutter. This means it's limited when it comes to things like scuba diving > videos, where the camera is filming the seabed at a small distance and at > an angle. > > Thanks! > Matthew > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] Mixture of homographies video stabilization
Ah ok thanks! I'm surprised no one has need a linear algebra library. I guess there's OpenCV and people use it to do the heavy lifting? Will look into the API more. Thanks matthew On Thu, Aug 2, 2018 at 3:31 PM Paul B Mahol wrote: > On 8/2/18, Matthew Lai wrote: > > Hello! > > > > I want to write a more advanced video stabilizer for libavfilter (*), > > implementing the algorithm described here - > > > https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/37744.pdf > > The focus of the paper is rolling shutter removal, but it builds on top > of > > another algorithm that does full frame stabilization, and the new > algorithm > > does that as well. > > > > This is the algorithm used in YouTube's stabilizing filter, and is state > of > > the art. Adobe calls it Warp Stabilizer (it's the same thing as far as I > > can tell from public information anyways). > > > > 3 questions: > > 1. Is there a linear algebra library already in use? I didn't see > anything > > in configure, but would be surprised if none of the existing filters work > > with matrices? > > There is no such library here used. There are indeed video/audio > filters that work with matrices. > > > 2. Is there anything to watch out for re. a high frame delay (say a few > > hundred frames)? Looking at the API, I don't see a callback to flush out > > remaining frames when input frames are finished? Is doing it in two > passes > > the only option? > > It is handled internally, there are two internal APIs, activate one and > legacy. > With legacy you can flush frames when you receive last frame from input. > With newer, activate API, its similar. > > > 3. doc/writing_filters.txt says only slice threading is available. That's > > not really possible with this filter, but frame threading is. Can I just > > buffer frames internally (which I need to do anyways to smooth out > motion), > > and do my own threading? > > You could do it. > > > > > * vid.stab is good for what it does, but it only does rotation and > > translation, and doesn't handle zoom, perspective distortion, or rolling > > shutter. This means it's limited when it comes to things like scuba > diving > > videos, where the camera is filming the seabed at a small distance and at > > an angle. > > > > Thanks! > > Matthew > > ___ > > ffmpeg-devel mailing list > > ffmpeg-devel@ffmpeg.org > > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 5/5] avformat/matroskaenc: update AV1 support
On 7/30/2018 12:13 PM, James Almer wrote: > On 7/30/2018 2:03 AM, Steve Lhomme wrote: >> On 26/07/2018 03:11, James Almer wrote: >>> Make sure to not write forbidden OBUs to CodecPrivate, and do the same >>> with >>> unnecessary OBUs for packets. >> >> Does this include reordering the OBUs ? The first one must be the >> Sequence Header OBU. Also it must be the only one of that kind. > > No, it doesn't. I can make that change Pushed the set plus the two patches that implement this behavior. I'll update things if anything comes out of the isobmff ticket (and is eventually adopted by Matroska) that differs from the current behavior. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 1/5] avcodec/dump_extradata_bsf: don't allocate a new AVPacket per filtered packet
On 7/27/2018 4:55 PM, Michael Niedermayer wrote: > On Wed, Jul 25, 2018 at 10:11:47PM -0300, James Almer wrote: >> ~4x speedup in dump_extradata() >> >> Signed-off-by: James Almer >> --- >> libavcodec/dump_extradata_bsf.c | 7 --- >> 1 file changed, 4 insertions(+), 3 deletions(-) > > should be ok > and nice speedup > > thx Patchset pushed, thanks. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [PATCH] avformat/tls_schannel: Fix use of uninitialized variable
Fixes: runtime error: passing uninitialized value to FreeContextBuffer
causes a crash
---
libavformat/tls_schannel.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/libavformat/tls_schannel.c b/libavformat/tls_schannel.c
index 065dccb..6953008 100644
--- a/libavformat/tls_schannel.c
+++ b/libavformat/tls_schannel.c
@@ -154,6 +154,8 @@ static int tls_client_handshake_loop(URLContext *h, int
initial)
SecBufferDesc inbuf_desc;
int i, ret = 0, read_data = initial;
+memset(outbuf, 0, sizeof(outbuf));
+
if (c->enc_buf == NULL) {
c->enc_buf_offset = 0;
ret = av_reallocp(&c->enc_buf, SCHANNEL_INITIAL_BUFFER_SIZE);
--
2.10.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [PATCH 7/7] libavfilter: Adds proper file descriptions to dnn_srcnn.h and dnn_espcn.h.
--- libavfilter/dnn_espcn.h | 3 ++- libavfilter/dnn_srcnn.h | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/libavfilter/dnn_espcn.h b/libavfilter/dnn_espcn.h index 9344aa90fe..e0013fe1dd 100644 --- a/libavfilter/dnn_espcn.h +++ b/libavfilter/dnn_espcn.h @@ -20,7 +20,8 @@ /** * @file - * Default cnn weights for x2 upscaling with espcn model. + * This file contains CNN weights for ESPCN model (https://arxiv.org/abs/1609.05158), + * auto generated by scripts provided in the repository: https://github.com/HighVoltageRocknRoll/sr.git. */ #ifndef AVFILTER_DNN_ESPCN_H diff --git a/libavfilter/dnn_srcnn.h b/libavfilter/dnn_srcnn.h index 4f5332ce18..8bf563bd62 100644 --- a/libavfilter/dnn_srcnn.h +++ b/libavfilter/dnn_srcnn.h @@ -20,7 +20,8 @@ /** * @file - * Default cnn weights for x2 upscaling with srcnn model. + * This file contains CNN weights for SRCNN model (https://arxiv.org/abs/1501.00092), + * auto generated by scripts provided in the repository: https://github.com/HighVoltageRocknRoll/sr.git. */ #ifndef AVFILTER_DNN_SRCNN_H -- 2.14.1 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [PATCH 5/7] libavfilter/dnn_backend_tf.c: Fixes ff_dnn_free_model_tf.
---
libavfilter/dnn_backend_tf.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/libavfilter/dnn_backend_tf.c b/libavfilter/dnn_backend_tf.c
index 7a4ad72d27..662a2a3c6e 100644
--- a/libavfilter/dnn_backend_tf.c
+++ b/libavfilter/dnn_backend_tf.c
@@ -570,7 +570,9 @@ void ff_dnn_free_model_tf(DNNModel **model)
if (tf_model->input_tensor){
TF_DeleteTensor(tf_model->input_tensor);
}
-av_freep(&tf_model->output_data->data);
+if (tf_model->output_data){
+av_freep(&(tf_model->output_data->data));
+}
av_freep(&tf_model);
av_freep(model);
}
--
2.14.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [PATCH 6/7] libavfilter/vf_sr.c: Removes uint8 -> float and float -> uint8 conversions.
This patch removes conversions, declared inside the sr filter, and uses
libswscale inside
the filter to perform them for only Y channel of input. The sr filter still has
uint
formats as input, as it does not use chroma channels in models and these
channels are
upscaled using libswscale, float formats for input would cause unnecessary
conversions
during scaling for these channels.
---
libavfilter/vf_sr.c | 134 +++-
1 file changed, 48 insertions(+), 86 deletions(-)
diff --git a/libavfilter/vf_sr.c b/libavfilter/vf_sr.c
index 944a0e28e7..5ad1baa4c0 100644
--- a/libavfilter/vf_sr.c
+++ b/libavfilter/vf_sr.c
@@ -45,8 +45,8 @@ typedef struct SRContext {
DNNModel *model;
DNNData input, output;
int scale_factor;
-struct SwsContext *sws_context;
-int sws_slice_h;
+struct SwsContext *sws_contexts[3];
+int sws_slice_h, sws_input_linesize, sws_output_linesize;
} SRContext;
#define OFFSET(x) offsetof(SRContext, x)
@@ -95,6 +95,10 @@ static av_cold int init(AVFilterContext *context)
return AVERROR(EIO);
}
+sr_context->sws_contexts[0] = NULL;
+sr_context->sws_contexts[1] = NULL;
+sr_context->sws_contexts[2] = NULL;
+
return 0;
}
@@ -110,6 +114,7 @@ static int query_formats(AVFilterContext *context)
av_log(context, AV_LOG_ERROR, "could not create formats list\n");
return AVERROR(ENOMEM);
}
+
return ff_set_common_formats(context, formats_list);
}
@@ -140,21 +145,31 @@ static int config_props(AVFilterLink *inlink)
else{
outlink->h = sr_context->output.height;
outlink->w = sr_context->output.width;
+sr_context->sws_contexts[1] = sws_getContext(sr_context->input.width,
sr_context->input.height, AV_PIX_FMT_GRAY8,
+ sr_context->input.width,
sr_context->input.height, AV_PIX_FMT_GRAYF32,
+ 0, NULL, NULL, NULL);
+sr_context->sws_input_linesize = sr_context->input.width << 2;
+sr_context->sws_contexts[2] = sws_getContext(sr_context->output.width,
sr_context->output.height, AV_PIX_FMT_GRAYF32,
+ sr_context->output.width,
sr_context->output.height, AV_PIX_FMT_GRAY8,
+ 0, NULL, NULL, NULL);
+sr_context->sws_output_linesize = sr_context->output.width << 2;
+if (!sr_context->sws_contexts[1] || !sr_context->sws_contexts[2]){
+av_log(context, AV_LOG_ERROR, "could not create SwsContext for
conversions\n");
+return AVERROR(ENOMEM);
+}
switch (sr_context->model_type){
case SRCNN:
-sr_context->sws_context = sws_getContext(inlink->w, inlink->h,
inlink->format,
- outlink->w, outlink->h,
outlink->format, SWS_BICUBIC, NULL, NULL, NULL);
-if (!sr_context->sws_context){
-av_log(context, AV_LOG_ERROR, "could not create SwsContext\n");
+sr_context->sws_contexts[0] = sws_getContext(inlink->w, inlink->h,
inlink->format,
+ outlink->w,
outlink->h, outlink->format,
+ SWS_BICUBIC, NULL,
NULL, NULL);
+if (!sr_context->sws_contexts[0]){
+av_log(context, AV_LOG_ERROR, "could not create SwsContext for
scaling\n");
return AVERROR(ENOMEM);
}
sr_context->sws_slice_h = inlink->h;
break;
case ESPCN:
-if (inlink->format == AV_PIX_FMT_GRAY8){
-sr_context->sws_context = NULL;
-}
-else{
+if (inlink->format != AV_PIX_FMT_GRAY8){
sws_src_h = sr_context->input.height;
sws_src_w = sr_context->input.width;
sws_dst_h = sr_context->output.height;
@@ -184,13 +199,14 @@ static int config_props(AVFilterLink *inlink)
sws_dst_w = AV_CEIL_RSHIFT(sws_dst_w, 2);
break;
default:
-av_log(context, AV_LOG_ERROR, "could not create SwsContext
for input pixel format");
+av_log(context, AV_LOG_ERROR, "could not create SwsContext
for scaling for given input pixel format");
return AVERROR(EIO);
}
-sr_context->sws_context = sws_getContext(sws_src_w, sws_src_h,
AV_PIX_FMT_GRAY8,
- sws_dst_w, sws_dst_h,
AV_PIX_FMT_GRAY8, SWS_BICUBIC, NULL, NULL, NULL);
-if (!sr_context->sws_context){
-av_log(context, AV_LOG_ERROR, "could not create
SwsContext\n");
+sr_context->sws_contexts[0] = sws_getContext(sws_src_w,
sws_src_h, AV_PIX_FMT_GRAY8,
+
[FFmpeg-devel] [PATCH 3/7] libavfilter: Fixes warnings for unused variables in dnn_srcnn.h, dnn_espcn.h, dnn_backend_tf.c.
---
libavfilter/dnn_backend_tf.c | 64 +++-
libavfilter/dnn_espcn.h | 37 -
libavfilter/dnn_srcnn.h | 35
3 files changed, 63 insertions(+), 73 deletions(-)
diff --git a/libavfilter/dnn_backend_tf.c b/libavfilter/dnn_backend_tf.c
index 6307c794a5..7a4ad72d27 100644
--- a/libavfilter/dnn_backend_tf.c
+++ b/libavfilter/dnn_backend_tf.c
@@ -374,9 +374,71 @@ DNNModel *ff_dnn_load_default_model_tf(DNNDefaultModel
model_type)
TFModel *tf_model = NULL;
TF_OperationDescription *op_desc;
TF_Operation *op;
-TF_Operation *const_ops_buffer[6];
TF_Output input;
int64_t input_shape[] = {1, -1, -1, 1};
+const char tanh[] = "Tanh";
+const char sigmoid[] = "Sigmoid";
+const char relu[] = "Relu";
+
+const float *srcnn_consts[] = {
+srcnn_conv1_kernel,
+srcnn_conv1_bias,
+srcnn_conv2_kernel,
+srcnn_conv2_bias,
+srcnn_conv3_kernel,
+srcnn_conv3_bias
+};
+const long int *srcnn_consts_dims[] = {
+srcnn_conv1_kernel_dims,
+srcnn_conv1_bias_dims,
+srcnn_conv2_kernel_dims,
+srcnn_conv2_bias_dims,
+srcnn_conv3_kernel_dims,
+srcnn_conv3_bias_dims
+};
+const int srcnn_consts_dims_len[] = {
+4,
+1,
+4,
+1,
+4,
+1
+};
+const char *srcnn_activations[] = {
+relu,
+relu,
+relu
+};
+
+const float *espcn_consts[] = {
+espcn_conv1_kernel,
+espcn_conv1_bias,
+espcn_conv2_kernel,
+espcn_conv2_bias,
+espcn_conv3_kernel,
+espcn_conv3_bias
+};
+const long int *espcn_consts_dims[] = {
+espcn_conv1_kernel_dims,
+espcn_conv1_bias_dims,
+espcn_conv2_kernel_dims,
+espcn_conv2_bias_dims,
+espcn_conv3_kernel_dims,
+espcn_conv3_bias_dims
+};
+const int espcn_consts_dims_len[] = {
+4,
+1,
+4,
+1,
+4,
+1
+};
+const char *espcn_activations[] = {
+tanh,
+tanh,
+sigmoid
+};
input.index = 0;
diff --git a/libavfilter/dnn_espcn.h b/libavfilter/dnn_espcn.h
index a0dd61cd0d..9344aa90fe 100644
--- a/libavfilter/dnn_espcn.h
+++ b/libavfilter/dnn_espcn.h
@@ -5398,41 +5398,4 @@ static const long int espcn_conv3_bias_dims[] = {
4
};
-static const float *espcn_consts[] = {
-espcn_conv1_kernel,
-espcn_conv1_bias,
-espcn_conv2_kernel,
-espcn_conv2_bias,
-espcn_conv3_kernel,
-espcn_conv3_bias
-};
-
-static const long int *espcn_consts_dims[] = {
-espcn_conv1_kernel_dims,
-espcn_conv1_bias_dims,
-espcn_conv2_kernel_dims,
-espcn_conv2_bias_dims,
-espcn_conv3_kernel_dims,
-espcn_conv3_bias_dims
-};
-
-static const int espcn_consts_dims_len[] = {
-4,
-1,
-4,
-1,
-4,
-1
-};
-
-static const char espcn_tanh[] = "Tanh";
-
-static const char espcn_sigmoid[] = "Sigmoid";
-
-static const char *espcn_activations[] = {
-espcn_tanh,
-espcn_tanh,
-espcn_sigmoid
-};
-
#endif
diff --git a/libavfilter/dnn_srcnn.h b/libavfilter/dnn_srcnn.h
index 26143654b8..4f5332ce18 100644
--- a/libavfilter/dnn_srcnn.h
+++ b/libavfilter/dnn_srcnn.h
@@ -2110,39 +2110,4 @@ static const long int srcnn_conv3_bias_dims[] = {
1
};
-static const float *srcnn_consts[] = {
-srcnn_conv1_kernel,
-srcnn_conv1_bias,
-srcnn_conv2_kernel,
-srcnn_conv2_bias,
-srcnn_conv3_kernel,
-srcnn_conv3_bias
-};
-
-static const long int *srcnn_consts_dims[] = {
-srcnn_conv1_kernel_dims,
-srcnn_conv1_bias_dims,
-srcnn_conv2_kernel_dims,
-srcnn_conv2_bias_dims,
-srcnn_conv3_kernel_dims,
-srcnn_conv3_bias_dims
-};
-
-static const int srcnn_consts_dims_len[] = {
-4,
-1,
-4,
-1,
-4,
-1
-};
-
-static const char srcnn_relu[] = "Relu";
-
-static const char *srcnn_activations[] = {
-srcnn_relu,
-srcnn_relu,
-srcnn_relu
-};
-
#endif
--
2.14.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [GSOC][PATCH 0/7] Improvements for sr filter and DNN module
Hello, These patches address several raised concerns regarding sr filter and DNN module. I included three patches, that I've already sent, but they still have not been reviewed properly. libavfilter: Adds on the fly generation of default DNN models for tensorflow backend instead of storing binary model. libavfilter: Code style fixes for pointers in DNN module and sr filter. libavfilter: Fixes warnings for unused variables in dnn_srcnn.h, dnn_espcn.h, dnn_backend_tf.c. Adds gray floating-point pixel formats. libavfilter/dnn_backend_tf.c: Fixes ff_dnn_free_model_tf. libavfilter/vf_sr.c: Removes uint8 -> float and float -> uint8 conversions. libavfilter: Adds proper file descriptions to dnn_srcnn.h and dnn_espcn.h. libavfilter/dnn_backend_native.c |96 +- libavfilter/dnn_backend_native.h | 8 +- libavfilter/dnn_backend_tf.c | 396 +- libavfilter/dnn_backend_tf.h | 8 +- libavfilter/dnn_espcn.h | 17947 +++-- libavfilter/dnn_interface.c | 4 +- libavfilter/dnn_interface.h |16 +- libavfilter/dnn_srcnn.h | 6979 +- libavfilter/vf_sr.c | 194 +- libavutil/pixdesc.c |22 + libavutil/pixfmt.h | 5 + libswscale/swscale_internal.h| 7 + libswscale/swscale_unscaled.c|54 +- libswscale/utils.c | 5 +- 14 files changed, 7983 insertions(+), 17758 deletions(-) -- 2.14.1 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [PATCH 2/7] libavfilter: Code style fixes for pointers in DNN module and sr filter.
---
libavfilter/dnn_backend_native.c | 84 +++---
libavfilter/dnn_backend_native.h | 8 +--
libavfilter/dnn_backend_tf.c | 108 +++
libavfilter/dnn_backend_tf.h | 8 +--
libavfilter/dnn_espcn.h | 6 +--
libavfilter/dnn_interface.c | 4 +-
libavfilter/dnn_interface.h | 16 +++---
libavfilter/dnn_srcnn.h | 6 +--
libavfilter/vf_sr.c | 60 +++---
9 files changed, 150 insertions(+), 150 deletions(-)
diff --git a/libavfilter/dnn_backend_native.c b/libavfilter/dnn_backend_native.c
index 3e6b86280d..baefea7fcb 100644
--- a/libavfilter/dnn_backend_native.c
+++ b/libavfilter/dnn_backend_native.c
@@ -34,15 +34,15 @@ typedef enum {RELU, TANH, SIGMOID} ActivationFunc;
typedef struct Layer{
LayerType type;
-float* output;
-void* params;
+float *output;
+void *params;
} Layer;
typedef struct ConvolutionalParams{
int32_t input_num, output_num, kernel_size;
ActivationFunc activation;
-float* kernel;
-float* biases;
+float *kernel;
+float *biases;
} ConvolutionalParams;
typedef struct InputParams{
@@ -55,16 +55,16 @@ typedef struct DepthToSpaceParams{
// Represents simple feed-forward convolutional network.
typedef struct ConvolutionalNetwork{
-Layer* layers;
+Layer *layers;
int32_t layers_num;
} ConvolutionalNetwork;
-static DNNReturnType set_input_output_native(void* model, DNNData* input,
DNNData* output)
+static DNNReturnType set_input_output_native(void *model, DNNData *input,
DNNData *output)
{
-ConvolutionalNetwork* network = (ConvolutionalNetwork*)model;
-InputParams* input_params;
-ConvolutionalParams* conv_params;
-DepthToSpaceParams* depth_to_space_params;
+ConvolutionalNetwork *network = (ConvolutionalNetwork *)model;
+InputParams *input_params;
+ConvolutionalParams *conv_params;
+DepthToSpaceParams *depth_to_space_params;
int cur_width, cur_height, cur_channels;
int32_t layer;
@@ -72,7 +72,7 @@ static DNNReturnType set_input_output_native(void* model,
DNNData* input, DNNDat
return DNN_ERROR;
}
else{
-input_params = (InputParams*)network->layers[0].params;
+input_params = (InputParams *)network->layers[0].params;
input_params->width = cur_width = input->width;
input_params->height = cur_height = input->height;
input_params->channels = cur_channels = input->channels;
@@ -88,14 +88,14 @@ static DNNReturnType set_input_output_native(void* model,
DNNData* input, DNNDat
for (layer = 1; layer < network->layers_num; ++layer){
switch (network->layers[layer].type){
case CONV:
-conv_params = (ConvolutionalParams*)network->layers[layer].params;
+conv_params = (ConvolutionalParams *)network->layers[layer].params;
if (conv_params->input_num != cur_channels){
return DNN_ERROR;
}
cur_channels = conv_params->output_num;
break;
case DEPTH_TO_SPACE:
-depth_to_space_params =
(DepthToSpaceParams*)network->layers[layer].params;
+depth_to_space_params = (DepthToSpaceParams
*)network->layers[layer].params;
if (cur_channels % (depth_to_space_params->block_size *
depth_to_space_params->block_size) != 0){
return DNN_ERROR;
}
@@ -127,16 +127,16 @@ static DNNReturnType set_input_output_native(void* model,
DNNData* input, DNNDat
// layers_num,layer_type,layer_parameterss,layer_type,layer_parameters...
// For CONV layer: activation_function, input_num, output_num, kernel_size,
kernel, biases
// For DEPTH_TO_SPACE layer: block_size
-DNNModel* ff_dnn_load_model_native(const char* model_filename)
+DNNModel *ff_dnn_load_model_native(const char *model_filename)
{
-DNNModel* model = NULL;
-ConvolutionalNetwork* network = NULL;
-AVIOContext* model_file_context;
+DNNModel *model = NULL;
+ConvolutionalNetwork *network = NULL;
+AVIOContext *model_file_context;
int file_size, dnn_size, kernel_size, i;
int32_t layer;
LayerType layer_type;
-ConvolutionalParams* conv_params;
-DepthToSpaceParams* depth_to_space_params;
+ConvolutionalParams *conv_params;
+DepthToSpaceParams *depth_to_space_params;
model = av_malloc(sizeof(DNNModel));
if (!model){
@@ -155,7 +155,7 @@ DNNModel* ff_dnn_load_model_native(const char*
model_filename)
av_freep(&model);
return NULL;
}
-model->model = (void*)network;
+model->model = (void *)network;
network->layers_num = 1 + (int32_t)avio_rl32(model_file_context);
dnn_size = 4;
@@ -251,10 +251,10 @@ DNNModel* ff_dnn_load_model_native(const char*
model_filename)
return model;
}
-static int set_up_conv_layer(Layer* layer, const float* kernel, const float*
biases, ActivationFunc activation,
+stat
[FFmpeg-devel] [PATCH 4/7] Adds gray floating-point pixel formats.
This patch adds two floating-point gray formats to use them in sr filter for
conversion with libswscale. I added conversion from uint gray to float and
backwards in swscale_unscaled.c, that is enough for sr filter. But for
proper format addition, should I add anything else?
---
libavutil/pixdesc.c | 22 ++
libavutil/pixfmt.h| 5
libswscale/swscale_internal.h | 7 ++
libswscale/swscale_unscaled.c | 54 +--
libswscale/utils.c| 5 +++-
5 files changed, 90 insertions(+), 3 deletions(-)
diff --git a/libavutil/pixdesc.c b/libavutil/pixdesc.c
index 96e079584a..7d307d9120 100644
--- a/libavutil/pixdesc.c
+++ b/libavutil/pixdesc.c
@@ -2198,6 +2198,28 @@ static const AVPixFmtDescriptor
av_pix_fmt_descriptors[AV_PIX_FMT_NB] = {
.flags = AV_PIX_FMT_FLAG_PLANAR | AV_PIX_FMT_FLAG_ALPHA |
AV_PIX_FMT_FLAG_RGB | AV_PIX_FMT_FLAG_FLOAT,
},
+[AV_PIX_FMT_GRAYF32BE] = {
+.name = "grayf32be",
+.nb_components = 1,
+.log2_chroma_w = 0,
+.log2_chroma_h = 0,
+.comp = {
+{ 0, 4, 0, 0, 32, 3, 31, 1 }, /* Y */
+},
+.flags = AV_PIX_FMT_FLAG_BE | AV_PIX_FMT_FLAG_FLOAT,
+.alias = "yf32be",
+},
+[AV_PIX_FMT_GRAYF32LE] = {
+.name = "grayf32le",
+.nb_components = 1,
+.log2_chroma_w = 0,
+.log2_chroma_h = 0,
+.comp = {
+{ 0, 4, 0, 0, 32, 3, 31, 1 }, /* Y */
+},
+.flags = AV_PIX_FMT_FLAG_FLOAT,
+.alias = "yf32le",
+},
[AV_PIX_FMT_DRM_PRIME] = {
.name = "drm_prime",
.flags = AV_PIX_FMT_FLAG_HWACCEL,
diff --git a/libavutil/pixfmt.h b/libavutil/pixfmt.h
index 2b3307845e..aa9a4f60c1 100644
--- a/libavutil/pixfmt.h
+++ b/libavutil/pixfmt.h
@@ -320,6 +320,9 @@ enum AVPixelFormat {
AV_PIX_FMT_GBRAPF32BE, ///< IEEE-754 single precision planar GBRA 4:4:4:4,
128bpp, big-endian
AV_PIX_FMT_GBRAPF32LE, ///< IEEE-754 single precision planar GBRA 4:4:4:4,
128bpp, little-endian
+AV_PIX_FMT_GRAYF32BE, ///< IEEE-754 single precision Y, 32bpp, big-endian
+AV_PIX_FMT_GRAYF32LE, ///< IEEE-754 single precision Y, 32bpp,
little-endian
+
/**
* DRM-managed buffers exposed through PRIME buffer sharing.
*
@@ -405,6 +408,8 @@ enum AVPixelFormat {
#define AV_PIX_FMT_GBRPF32AV_PIX_FMT_NE(GBRPF32BE, GBRPF32LE)
#define AV_PIX_FMT_GBRAPF32 AV_PIX_FMT_NE(GBRAPF32BE, GBRAPF32LE)
+#define AV_PIX_FMT_GRAYF32 AV_PIX_FMT_NE(GRAYF32BE, GRAYF32LE)
+
#define AV_PIX_FMT_YUVA420P9 AV_PIX_FMT_NE(YUVA420P9BE , YUVA420P9LE)
#define AV_PIX_FMT_YUVA422P9 AV_PIX_FMT_NE(YUVA422P9BE , YUVA422P9LE)
#define AV_PIX_FMT_YUVA444P9 AV_PIX_FMT_NE(YUVA444P9BE , YUVA444P9LE)
diff --git a/libswscale/swscale_internal.h b/libswscale/swscale_internal.h
index 1703856ab2..4a2cdfe658 100644
--- a/libswscale/swscale_internal.h
+++ b/libswscale/swscale_internal.h
@@ -764,6 +764,13 @@ static av_always_inline int isAnyRGB(enum AVPixelFormat
pix_fmt)
pix_fmt == AV_PIX_FMT_MONOBLACK || pix_fmt == AV_PIX_FMT_MONOWHITE;
}
+static av_always_inline int isFloat(enum AVPixelFormat pix_fmt)
+{
+const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(pix_fmt);
+av_assert0(desc);
+return desc->flags & AV_PIX_FMT_FLAG_FLOAT;
+}
+
static av_always_inline int isALPHA(enum AVPixelFormat pix_fmt)
{
const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(pix_fmt);
diff --git a/libswscale/swscale_unscaled.c b/libswscale/swscale_unscaled.c
index 6480070cbf..f5b4c9be9d 100644
--- a/libswscale/swscale_unscaled.c
+++ b/libswscale/swscale_unscaled.c
@@ -1467,6 +1467,46 @@ static int yvu9ToYv12Wrapper(SwsContext *c, const
uint8_t *src[],
return srcSliceH;
}
+static int uint_y_to_float_y_wrapper(SwsContext *c, const uint8_t *src[],
+ int srcStride[], int srcSliceY,
+ int srcSliceH, uint8_t *dst[], int
dstStride[])
+{
+int y, x;
+int dstStrideFloat = dstStride[0] >> 2;;
+const uint8_t *srcPtr = src[0];
+float *dstPtr = (float *)(dst[0] + dstStride[0] * srcSliceY);
+
+for (y = 0; y < srcSliceH; ++y){
+for (x = 0; x < c->srcW; ++x){
+dstPtr[x] = (float)srcPtr[x] / 255.0f;
+}
+srcPtr += srcStride[0];
+dstPtr += dstStrideFloat;
+}
+
+return srcSliceH;
+}
+
+static int float_y_to_uint_y_wrapper(SwsContext *c, const uint8_t* src[],
+ int srcStride[], int srcSliceY,
+ int srcSliceH, uint8_t* dst[], int
dstStride[])
+{
+int y, x;
+int srcStrideFloat = srcStride[0] >> 2;
+const float *srcPtr = (const float *)src[0];
+uint8_t *dstPtr = dst[0] + dstStride[0] * srcSliceY;
+
+for (y = 0; y < srcSliceH; ++y){
+for (x = 0; x < c->srcW; ++x){
+dstPtr[x] = (uint8_t)(255.0
[FFmpeg-devel] [PATCH] avformat/tls_schannel: Fix use of uninitialized variable
Fixes: runtime error: passing uninitialized value to FreeContextBuffer
causes a crash
Signed-off-by: Paweł Wegner
---
libavformat/tls_schannel.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/libavformat/tls_schannel.c b/libavformat/tls_schannel.c
index f41b007773..6b51c1f3dc 100644
--- a/libavformat/tls_schannel.c
+++ b/libavformat/tls_schannel.c
@@ -154,6 +154,8 @@ static int tls_client_handshake_loop(URLContext *h, int
initial)
SecBufferDesc inbuf_desc;
int i, ret = 0, read_data = initial;
+memset(outbuf, 0, sizeof(outbuf));
+
if (c->enc_buf == NULL) {
c->enc_buf_offset = 0;
ret = av_reallocp(&c->enc_buf, SCHANNEL_INITIAL_BUFFER_SIZE);
--
2.17.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 4/7] Adds gray floating-point pixel formats.
>
> +static int uint_y_to_float_y_wrapper(SwsContext *c, const uint8_t *src[],
> + int srcStride[], int srcSliceY,
> + int srcSliceH, uint8_t *dst[], int
> dstStride[])
> +{
> +int y, x;
> +int dstStrideFloat = dstStride[0] >> 2;;
> +const uint8_t *srcPtr = src[0];
> +float *dstPtr = (float *)(dst[0] + dstStride[0] * srcSliceY);
> +
> +for (y = 0; y < srcSliceH; ++y){
> +for (x = 0; x < c->srcW; ++x){
> +dstPtr[x] = (float)srcPtr[x] / 255.0f;
> +}
> +srcPtr += srcStride[0];
> +dstPtr += dstStrideFloat;
> +}
> +
> +return srcSliceH;
> +}
> +
> +static int float_y_to_uint_y_wrapper(SwsContext *c, const uint8_t* src[],
> + int srcStride[], int srcSliceY,
> + int srcSliceH, uint8_t* dst[], int
> dstStride[])
> +{
> +int y, x;
> +int srcStrideFloat = srcStride[0] >> 2;
> +const float *srcPtr = (const float *)src[0];
> +uint8_t *dstPtr = dst[0] + dstStride[0] * srcSliceY;
> +
> +for (y = 0; y < srcSliceH; ++y){
> +for (x = 0; x < c->srcW; ++x){
> +dstPtr[x] = (uint8_t)(255.0f * FFMIN(FFMAX(srcPtr[x], 0.0f),
> 1.0f));
> +}
> +srcPtr += srcStrideFloat;
> +dstPtr += dstStride[0];
> +}
> +
> +return srcSliceH;
> +}
>
Maybe you can avoid to use float for these conversions
like in libavcodec/exr.c, where there is a float to uint16 conversion
Martin
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 4/7] Adds gray floating-point pixel formats.
On 8/2/2018 4:22 PM, Martin Vignali wrote:
>>
>> +static int uint_y_to_float_y_wrapper(SwsContext *c, const uint8_t *src[],
>> + int srcStride[], int srcSliceY,
>> + int srcSliceH, uint8_t *dst[], int
>> dstStride[])
>> +{
>> +int y, x;
>> +int dstStrideFloat = dstStride[0] >> 2;;
>> +const uint8_t *srcPtr = src[0];
>> +float *dstPtr = (float *)(dst[0] + dstStride[0] * srcSliceY);
>> +
>> +for (y = 0; y < srcSliceH; ++y){
>> +for (x = 0; x < c->srcW; ++x){
>> +dstPtr[x] = (float)srcPtr[x] / 255.0f;
>> +}
>> +srcPtr += srcStride[0];
>> +dstPtr += dstStrideFloat;
>> +}
>> +
>> +return srcSliceH;
>> +}
>> +
>> +static int float_y_to_uint_y_wrapper(SwsContext *c, const uint8_t* src[],
>> + int srcStride[], int srcSliceY,
>> + int srcSliceH, uint8_t* dst[], int
>> dstStride[])
>> +{
>> +int y, x;
>> +int srcStrideFloat = srcStride[0] >> 2;
>> +const float *srcPtr = (const float *)src[0];
>> +uint8_t *dstPtr = dst[0] + dstStride[0] * srcSliceY;
>> +
>> +for (y = 0; y < srcSliceH; ++y){
>> +for (x = 0; x < c->srcW; ++x){
>> +dstPtr[x] = (uint8_t)(255.0f * FFMIN(FFMAX(srcPtr[x], 0.0f),
>> 1.0f));
>> +}
>> +srcPtr += srcStrideFloat;
>> +dstPtr += dstStride[0];
>> +}
>> +
>> +return srcSliceH;
>> +}
>>
>
>
> Maybe you can avoid to use float for these conversions
> like in libavcodec/exr.c, where there is a float to uint16 conversion
>
> Martin
Unless i'm missing something, I think the point is that he was asked to
not do format conversion within the filter itself.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] Why does ffmpeg h264 decoder stop decoding when certain type of packets are coming in the input stream?
2018-08-02 13:34 GMT+02:00, Naveed Basha : > Is this a known issue? I don't think it was known before you reported it. > Is there any known good version which does not > have have this issue? Not sure, I don't think so. Please do not top-post here and please move this discussion to the user mailing list where it belongs. Carl Eugen ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 4/7] Adds gray floating-point pixel formats.
2018-08-02 20:52 GMT+02:00, Sergey Lavrushkin : > This patch adds two floating-point gray formats to use them in sr filter for > conversion with libswscale. I added conversion from uint gray to float and > backwards in swscale_unscaled.c, that is enough for sr filter. But for > proper format addition, should I add anything else? I would have expected the conversion to be from and to GRAY16, is this simply a wrong assumption? Carl Eugen ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 1/2] avcodec/qtrle: Check remaining bytestream in qtrle_decode_XYbpp()
On Mon, Jul 30, 2018 at 03:33:29AM +0200, Michael Niedermayer wrote: > Fixes: Timeout > Fixes: > 9213/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_QTRLE_fuzzer-5649753332252672 > > Found-by: continuous fuzzing process > https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg > Signed-off-by: Michael Niedermayer > --- > libavcodec/qtrle.c | 10 ++ > 1 file changed, 10 insertions(+) will apply patchset [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB "You are 36 times more likely to die in a bathtub than at the hands of a terrorist. Also, you are 2.5 times more likely to become a president and 2 times more likely to become an astronaut, than to die in a terrorist attack." -- Thoughty2 signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] avformat/tls_schannel: Fix use of uninitialized variable
2018-08-02 20:46 GMT+02:00, Paweł Wegner : > Fixes: runtime error: passing uninitialized value to FreeContextBuffer > causes a crash > --- > libavformat/tls_schannel.c | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/libavformat/tls_schannel.c b/libavformat/tls_schannel.c > index 065dccb..6953008 100644 > --- a/libavformat/tls_schannel.c > +++ b/libavformat/tls_schannel.c > @@ -154,6 +154,8 @@ static int tls_client_handshake_loop(URLContext *h, int > initial) > SecBufferDesc inbuf_desc; > int i, ret = 0, read_data = initial; > > +memset(outbuf, 0, sizeof(outbuf)); Is the memset necessary or is an initialization of the variable sufficient? Carl Eugen ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 6/8] avcodec/h264_redundant_pps_bsf: implement a AVBSFContext.flush() callback
On Wed, Aug 01, 2018 at 11:20:46PM +0100, Mark Thompson wrote: > On 28/07/18 22:59, Michael Niedermayer wrote: > > On Fri, Jul 27, 2018 at 11:57:47AM -0300, James Almer wrote: > >> Signed-off-by: James Almer > >> --- > >> I'm not 100% sure this is correct. I also don't know if the CBS contexts > >> need > >> to be fully reinitialized or not in this scenario. Because if so, then > >> every > >> bsf using cbs will require a flush() callback as well. > >> > >> libavcodec/h264_redundant_pps_bsf.c | 9 + > >> 1 file changed, 9 insertions(+) > > > > Do we have a fate test for this ? > > No - I don't have a public file it applies to. (I can share an example > privately.) > > > Also from a quick look, isnt this filter lacking checks ? > > it removes PPS but i see no check that these PPS differ only in the qp > > related parameters. Also doesnt this filter miss that there can be multiple > > PPS with different id# > > The problematic streams repeatedly overwrite PPS id 0 with different > parameters, and that's what we're fixing up here - using a stream like that > in any case with global extradata fails on seeking or fragmenting. > > > Maybe iam missing something and this is of course not about the > > patch at hand but it looks like this is a quite specific filter that > > cannot saftely be applied to h264 in general and if so this is not > > obvious from the documentation. > > The documentation says the filter "applies a specific fixup to some Blu-ray > streams", which implies that it doesn't apply to anything else? Feel free to > add some additional text which clarifies it to your reading if you don't > agree. I think we interpret this differently. "applies F to B" to me does not imply anything about "Not B". And naivly i would have expected that it does not break "Not B". Being able to "just" apply this filter to all h264 without the need to manually first check each file if it is originating from blue-ray seems a realistic use case. I mean it could be a bit incovenient if one had a large amount of diverse input material. More so if not all is h264 ... Obviously this was not what others had in mind I can suggest a patch to clarify the docs if there is no interrest in making tha filter safe to "just" add thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Republics decline into democracies and democracies degenerate into despotisms. -- Aristotle signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] Mixture of homographies video stabilization
On Thu, Aug 02, 2018 at 05:24:08PM +0100, Matthew Lai wrote: > Ah ok thanks! I'm surprised no one has need a linear algebra library. I > guess there's OpenCV and people use it to do the heavy lifting? > > Will look into the API more. alot ot linear algebra we needed has been implemented directly. Some of it is shared and available from libavutil, like lls.* Can you elaboarte what exact linear algebra operations are needed? also what amounts of data (matrix types/sizes) and speed requirements this has Also IIUC (please correct me if iam wrong) this uses motion estimation at one step. IIRC ronald and others) want motion estimation to be factored and shared and not duplicated. The current motion estimation is in libavcodec/motion_est*. Thanks > > Thanks > matthew > > On Thu, Aug 2, 2018 at 3:31 PM Paul B Mahol wrote: > > > On 8/2/18, Matthew Lai wrote: > > > Hello! > > > > > > I want to write a more advanced video stabilizer for libavfilter (*), > > > implementing the algorithm described here - > > > > > https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/37744.pdf > > > The focus of the paper is rolling shutter removal, but it builds on top > > of > > > another algorithm that does full frame stabilization, and the new > > algorithm > > > does that as well. > > > > > > This is the algorithm used in YouTube's stabilizing filter, and is state > > of > > > the art. Adobe calls it Warp Stabilizer (it's the same thing as far as I > > > can tell from public information anyways). > > > > > > 3 questions: > > > 1. Is there a linear algebra library already in use? I didn't see > > anything > > > in configure, but would be surprised if none of the existing filters work > > > with matrices? > > > > There is no such library here used. There are indeed video/audio > > filters that work with matrices. > > > > > 2. Is there anything to watch out for re. a high frame delay (say a few > > > hundred frames)? Looking at the API, I don't see a callback to flush out > > > remaining frames when input frames are finished? Is doing it in two > > passes > > > the only option? > > > > It is handled internally, there are two internal APIs, activate one and > > legacy. > > With legacy you can flush frames when you receive last frame from input. > > With newer, activate API, its similar. > > > > > 3. doc/writing_filters.txt says only slice threading is available. That's > > > not really possible with this filter, but frame threading is. Can I just > > > buffer frames internally (which I need to do anyways to smooth out > > motion), > > > and do my own threading? > > > > You could do it. > > > > > > > > * vid.stab is good for what it does, but it only does rotation and > > > translation, and doesn't handle zoom, perspective distortion, or rolling > > > shutter. This means it's limited when it comes to things like scuba > > diving > > > videos, where the camera is filming the seabed at a small distance and at > > > an angle. > > > > > > Thanks! > > > Matthew > > > ___ > > > ffmpeg-devel mailing list > > > ffmpeg-devel@ffmpeg.org > > > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > > > ___ > > ffmpeg-devel mailing list > > ffmpeg-devel@ffmpeg.org > > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB It is dangerous to be right in matters on which the established authorities are wrong. -- Voltaire signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
