Re: [FFmpeg-devel] [PATCH] avfilter: add hsvkey and hsvhold video filters
will apply soon ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 1/1] avcodec/vc1dec: Return value check for init_get_bits
lgtm ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH] arm: hevc_qpel: Fix the assembly to work with non-multiple of 8 widths
This unbreaks the fate-checkasm-hevc_pel test on arm targets.
The assembly assumed that the width passed to the DSP functions is
a multiple of 8, while the checkasm test used other widths too.
This wasn't noticed before, because the hevc_pel checkasm tests
(that were added in 9c513edb7999a35ddcc6e3a8d984a96c8fb492a3 in
January) weren't run as part of fate until in
b492cacffd36ad4cb251ba1f13ac398318ee639a in August.
As this hasn't been an issue in practice with actual full decoding
tests, it seems like the actual decoder doesn't call these functions
with such widths. Therefore, we could alternatively fix the test
to only test things that the real decoder does, and this modification
could be reverted.
Signed-off-by: Martin Storsjö
---
libavcodec/arm/hevcdsp_qpel_neon.S | 18 +-
1 file changed, 9 insertions(+), 9 deletions(-)
diff --git a/libavcodec/arm/hevcdsp_qpel_neon.S
b/libavcodec/arm/hevcdsp_qpel_neon.S
index caa6efa766..f71bec05ed 100644
--- a/libavcodec/arm/hevcdsp_qpel_neon.S
+++ b/libavcodec/arm/hevcdsp_qpel_neon.S
@@ -237,7 +237,7 @@
vld1.8{d23}, [r2], r3
bne 8b
subs r5, #8
-beq 99f
+ble 99f
mov r4, r12
add r6, #16
mov r0, r6
@@ -280,7 +280,7 @@
vld1.8{d23}, [r2], r3
bne 8b
subs r5, #8
-beq 99f
+ble 99f
mov r4, r12
add r6, #8
mov r0, r6
@@ -310,7 +310,7 @@
vld1.8{d23}, [r2], r3
bne 8b
subs r5, #8
-beq 99f
+ble 99f
mov r4, r12
add r6, #8
mov r0, r6
@@ -377,7 +377,7 @@ endfunc
vst1.16 {q7}, [r0], r1
bne 8b
subs r5, #8
-beq 99f
+ble 99f
mov r4, r12
add r6, #16
mov r0, r6
@@ -417,7 +417,7 @@ endfunc
vst1.8d0, [r0], r1
bne 8b
subs r5, #8
-beq 99f
+ble 99f
mov r4, r12
add r6, #8
mov r0, r6
@@ -446,7 +446,7 @@ endfunc
vst1.8 d0, [r0], r1
bne 8b
subs r5, #8
-beq 99f
+ble 99f
mov r4, r12
add r6, #8
add r10, #16
@@ -533,7 +533,7 @@ endfunc
\filterh q7
bne 8b
subs r5, #8
-beq 99f
+ble 99f
mov r4, r12
add r6, #16
mov r0, r6
@@ -594,7 +594,7 @@ endfunc
\filterh q7
bne 8b
subs r5, #8
-beq 99f
+ble 99f
mov r4, r12
add r6, #8
mov r0, r6
@@ -641,7 +641,7 @@ endfunc
\filterh q7
bne 8b
subs r5, #8
-beq 99f
+ble 99f
mov r4, r12
add r6, #8
mov r0, r6
--
2.25.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 00/10] make QSV works with Intel's oneVPL
On Mon, 2021-08-23 at 05:06 +, Soft Works wrote: > > -Original Message- > > From: ffmpeg-devel On Behalf Of Xiang, > > Haihao > > Sent: Monday, 23 August 2021 06:29 > > To: ffmpeg-devel@ffmpeg.org > > Subject: Re: [FFmpeg-devel] [PATCH v2 00/10] make QSV works with Intel's > > oneVPL > > > > On Mon, 2021-08-16 at 15:33 +0800, Haihao Xiang wrote: > > > The oneAPI Video Processing Library (oneVPL) is a single interface for > > > encode, decode and video processing[1]. oneVPL is a successor to Intel's > > > > Media > > > SDK (MediaSDK), but removed obsolete features. MediaSDK lifetime comes to > > > > an > > > end now[2]. > > > > > > This patchset fixes compiler errors when building FFmpeg against oneVPL > > > and > > > uses > > > a new way to create MFX session for oneVPL. New features for oneVPL will > > > be > > > implemented in other patchset. > > > > > > The oneVPL source code: > > > https://github.com/oneapi-src/oneVPL > > > The oneVPL runtime for new Intel Gen platforms: > > > https://github.com/oneapi-src/oneVPL-intel-gpu > > > > > > [1] > > > > https://spec.oneapi.io/versions/latest/elements/oneVPL/source/index.html > > > [2] https://github.com/Intel-Media-SDK/MediaSDK/#media-sdk-support-matrix > > > > > > > Hi Zhong / Mark / Softworkz, > > > > Is there any comment to this new patchset version? This version should not > > have > > any impact to current user if --enable-libmfx option is used. We added -- > > enable- > > onevpl option for oneVPL usage but the suffix 'qsv' is still used in codec > > and > > filter, so user needn't change their app or command if they don't use the > > obsolete features. > > > Hi Haihao, > > I'm still struggling to understand this. You are doing a major version bump > to 2.x even there's not a single benefit but features are being removed > instead. > > At this time, I don't see any plausible reason to move away from libmfx. According to the links[1][2] below, MediaSDK is going to be supported in maintenance mode, new features will be added only to oneVPL, including the new features for future hardwares. The deleted features are less used in practice. oneVPL dispatcher may load multiple runtimes, user may use dispatcher APIs to find an available implementation[3], so it is possible to run FFmpeg with libvpl on different implementations. This patchset is not to replace libmfx with libvpl, libmfx is still used if running configure with --enable-libmfx option. [1]https://github.com/Intel-Media-SDK/MediaSDK/#media-sdk-support-matrix [2] https://software.intel.com/content/www/us/en/develop/articles/upgrading-from-msdk-to-onevpl.html [3] https://spec.oneapi.io/versions/latest/elements/oneVPL/source/API_ref/VPL_disp_api_func.html#dispatcher-api-function-reference Thanks Haiaho > > Kind regards, > softworkz > > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] configure: set IceLake-AVX512 as the minimum baseline
Ronald S. Bultje Wrote: > I'm OK with the current approach also. So I will not object to the current > patch anymore. > Hi Ronald, I am sorry I'm not familiar of the review flow. Does this proposal still need to be reviewed and approved by other maintainers? Or if it's acceptable, I added the runtime check in http://ffmpeg.org/pipermail/ffmpeg-devel/2021-August/283973.html. Could you help review the patches? Best regards, Jianhua ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v4 1/7] lavfi/dnn: Task-based Inference in Native Backend
This commit rearranges the code in Native Backend to use the TaskItem
for inference.
Signed-off-by: Shubhanshu Saxena
---
libavfilter/dnn/dnn_backend_native.c | 176 ++-
libavfilter/dnn/dnn_backend_native.h | 2 +
2 files changed, 121 insertions(+), 57 deletions(-)
diff --git a/libavfilter/dnn/dnn_backend_native.c
b/libavfilter/dnn/dnn_backend_native.c
index a6be27f1fd..3b2a3aa55d 100644
--- a/libavfilter/dnn/dnn_backend_native.c
+++ b/libavfilter/dnn/dnn_backend_native.c
@@ -45,9 +45,29 @@ static const AVClass dnn_native_class = {
.category = AV_CLASS_CATEGORY_FILTER,
};
-static DNNReturnType execute_model_native(const DNNModel *model, const char
*input_name, AVFrame *in_frame,
- const char **output_names, uint32_t
nb_output, AVFrame *out_frame,
- int do_ioproc);
+static DNNReturnType execute_model_native(Queue *inference_queue);
+
+static DNNReturnType extract_inference_from_task(TaskItem *task, Queue
*inference_queue)
+{
+NativeModel *native_model = task->model;
+NativeContext *ctx = &native_model->ctx;
+InferenceItem *inference = av_malloc(sizeof(*inference));
+
+if (!inference) {
+av_log(ctx, AV_LOG_ERROR, "Unable to allocate space for
InferenceItem\n");
+return DNN_ERROR;
+}
+task->inference_todo = 1;
+task->inference_done = 0;
+inference->task = task;
+
+if (ff_queue_push_back(inference_queue, inference) < 0) {
+av_log(ctx, AV_LOG_ERROR, "Failed to push back inference_queue.\n");
+av_freep(&inference);
+return DNN_ERROR;
+}
+return DNN_SUCCESS;
+}
static DNNReturnType get_input_native(void *model, DNNData *input, const char
*input_name)
{
@@ -78,34 +98,36 @@ static DNNReturnType get_input_native(void *model, DNNData
*input, const char *i
static DNNReturnType get_output_native(void *model, const char *input_name,
int input_width, int input_height,
const char *output_name, int
*output_width, int *output_height)
{
-DNNReturnType ret;
+DNNReturnType ret = 0;
NativeModel *native_model = model;
NativeContext *ctx = &native_model->ctx;
-AVFrame *in_frame = av_frame_alloc();
-AVFrame *out_frame = NULL;
-
-if (!in_frame) {
-av_log(ctx, AV_LOG_ERROR, "Could not allocate memory for input
frame\n");
-return DNN_ERROR;
+TaskItem task;
+DNNExecBaseParams exec_params = {
+.input_name = input_name,
+.output_names = &output_name,
+.nb_output = 1,
+.in_frame = NULL,
+.out_frame = NULL,
+};
+
+if (ff_dnn_fill_gettingoutput_task(&task, &exec_params, native_model,
input_height, input_width, ctx) != DNN_SUCCESS) {
+ret = DNN_ERROR;
+goto err;
}
-out_frame = av_frame_alloc();
-
-if (!out_frame) {
-av_log(ctx, AV_LOG_ERROR, "Could not allocate memory for output
frame\n");
-av_frame_free(&in_frame);
-return DNN_ERROR;
+if (extract_inference_from_task(&task, native_model->inference_queue) !=
DNN_SUCCESS) {
+av_log(ctx, AV_LOG_ERROR, "unable to extract inference from task.\n");
+ret = DNN_ERROR;
+goto err;
}
-in_frame->width = input_width;
-in_frame->height = input_height;
-
-ret = execute_model_native(native_model->model, input_name, in_frame,
&output_name, 1, out_frame, 0);
-*output_width = out_frame->width;
-*output_height = out_frame->height;
+ret = execute_model_native(native_model->inference_queue);
+*output_width = task.out_frame->width;
+*output_height = task.out_frame->height;
-av_frame_free(&out_frame);
-av_frame_free(&in_frame);
+err:
+av_frame_free(&task.out_frame);
+av_frame_free(&task.in_frame);
return ret;
}
@@ -190,6 +212,11 @@ DNNModel *ff_dnn_load_model_native(const char
*model_filename, DNNFunctionType f
goto fail;
}
+native_model->inference_queue = ff_queue_create();
+if (!native_model->inference_queue) {
+goto fail;
+}
+
for (layer = 0; layer < native_model->layers_num; ++layer){
layer_type = (int32_t)avio_rl32(model_file_context);
dnn_size += 4;
@@ -259,50 +286,66 @@ fail:
return NULL;
}
-static DNNReturnType execute_model_native(const DNNModel *model, const char
*input_name, AVFrame *in_frame,
- const char **output_names, uint32_t
nb_output, AVFrame *out_frame,
- int do_ioproc)
+static DNNReturnType execute_model_native(Queue *inference_queue)
{
-NativeModel *native_model = model->model;
-NativeContext *ctx = &native_model->ctx;
+NativeModel *native_model = NULL;
+NativeContext *ctx = NULL;
int32_t layer;
DNNData input, output;
DnnOperand *oprd = NULL;
+InferenceItem *inference = NULL;
+
[FFmpeg-devel] [PATCH v4 2/7] libavfilter: Unify Execution Modes in DNN Filters
This commit unifies the async and sync mode from the DNN filters'
perspective. As of this commit, the Native backend only supports
synchronous execution mode.
Now the user can switch between async and sync mode by using the
'async' option in the backend_configs. The values can be 1 for
async and 0 for sync mode of execution.
This commit affects the following filters:
1. vf_dnn_classify
2. vf_dnn_detect
3. vf_dnn_processing
4. vf_sr
5. vf_derain
Signed-off-by: Shubhanshu Saxena
---
libavfilter/dnn/dnn_backend_common.c | 2 +-
libavfilter/dnn/dnn_backend_common.h | 5 +-
libavfilter/dnn/dnn_backend_native.c | 59 +++-
libavfilter/dnn/dnn_backend_native.h | 6 ++
libavfilter/dnn/dnn_backend_openvino.c | 94 ++
libavfilter/dnn/dnn_backend_openvino.h | 3 +-
libavfilter/dnn/dnn_backend_tf.c | 35 ++
libavfilter/dnn/dnn_backend_tf.h | 3 +-
libavfilter/dnn/dnn_interface.c| 8 +--
libavfilter/dnn_filter_common.c| 23 +--
libavfilter/dnn_filter_common.h| 3 +-
libavfilter/dnn_interface.h| 4 +-
libavfilter/vf_derain.c| 7 ++
libavfilter/vf_dnn_classify.c | 4 +-
libavfilter/vf_dnn_detect.c| 10 +--
libavfilter/vf_dnn_processing.c| 10 +--
libavfilter/vf_sr.c| 8 +++
17 files changed, 142 insertions(+), 142 deletions(-)
diff --git a/libavfilter/dnn/dnn_backend_common.c
b/libavfilter/dnn/dnn_backend_common.c
index 426683b73d..d2bc016fef 100644
--- a/libavfilter/dnn/dnn_backend_common.c
+++ b/libavfilter/dnn/dnn_backend_common.c
@@ -138,7 +138,7 @@ DNNReturnType ff_dnn_start_inference_async(void *ctx,
DNNAsyncExecModule *async_
return DNN_SUCCESS;
}
-DNNAsyncStatusType ff_dnn_get_async_result_common(Queue *task_queue, AVFrame
**in, AVFrame **out)
+DNNAsyncStatusType ff_dnn_get_result_common(Queue *task_queue, AVFrame **in,
AVFrame **out)
{
TaskItem *task = ff_queue_peek_front(task_queue);
diff --git a/libavfilter/dnn/dnn_backend_common.h
b/libavfilter/dnn/dnn_backend_common.h
index 604c1d3bd7..78e62a94a2 100644
--- a/libavfilter/dnn/dnn_backend_common.h
+++ b/libavfilter/dnn/dnn_backend_common.h
@@ -29,7 +29,8 @@
#include "libavutil/thread.h"
#define DNN_BACKEND_COMMON_OPTIONS \
-{ "nireq", "number of request",
OFFSET(options.nireq), AV_OPT_TYPE_INT,{ .i64 = 0 }, 0,
INT_MAX, FLAGS },
+{ "nireq", "number of request",
OFFSET(options.nireq), AV_OPT_TYPE_INT,{ .i64 = 0 }, 0,
INT_MAX, FLAGS }, \
+{ "async", "use DNN async inference",
OFFSET(options.async), AV_OPT_TYPE_BOOL, { .i64 = 1 }, 0,
1, FLAGS },
// one task for one function call from dnn interface
typedef struct TaskItem {
@@ -135,7 +136,7 @@ DNNReturnType ff_dnn_start_inference_async(void *ctx,
DNNAsyncExecModule *async_
* @retval DAST_NOT_READY if inference not completed yet.
* @retval DAST_SUCCESS if result successfully extracted
*/
-DNNAsyncStatusType ff_dnn_get_async_result_common(Queue *task_queue, AVFrame
**in, AVFrame **out);
+DNNAsyncStatusType ff_dnn_get_result_common(Queue *task_queue, AVFrame **in,
AVFrame **out);
/**
* Allocate input and output frames and fill the Task
diff --git a/libavfilter/dnn/dnn_backend_native.c
b/libavfilter/dnn/dnn_backend_native.c
index 3b2a3aa55d..2d34b88f8a 100644
--- a/libavfilter/dnn/dnn_backend_native.c
+++ b/libavfilter/dnn/dnn_backend_native.c
@@ -34,6 +34,7 @@
#define FLAGS AV_OPT_FLAG_FILTERING_PARAM
static const AVOption dnn_native_options[] = {
{ "conv2d_threads", "threads num for conv2d layer",
OFFSET(options.conv2d_threads), AV_OPT_TYPE_INT, { .i64 = 0 }, INT_MIN,
INT_MAX, FLAGS },
+{ "async", "use DNN async inference", OFFSET(options.async),
AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, FLAGS },
{ NULL },
};
@@ -189,6 +190,11 @@ DNNModel *ff_dnn_load_model_native(const char
*model_filename, DNNFunctionType f
goto fail;
native_model->model = model;
+if (native_model->ctx.options.async) {
+av_log(&native_model->ctx, AV_LOG_WARNING, "Async not supported.
Rolling back to sync\n");
+native_model->ctx.options.async = 0;
+}
+
#if !HAVE_PTHREAD_CANCEL
if (native_model->ctx.options.conv2d_threads > 1){
av_log(&native_model->ctx, AV_LOG_WARNING, "'conv2d_threads' option
was set but it is not supported "
@@ -212,6 +218,11 @@ DNNModel *ff_dnn_load_model_native(const char
*model_filename, DNNFunctionType f
goto fail;
}
+native_model->task_queue = ff_queue_create();
+if (!native_model->task_queue) {
+goto fail;
+}
+
native_model->inference_queue = ff_queue_create();
if (!native_model->inference_queue) {
goto fail;
@@ -425,17 +436,30 @@ DNNReturnType ff_dnn_execute_model_native(const DNNModel
*model, DNNExe
[FFmpeg-devel] [PATCH v4 3/7] libavfilter: Remove synchronous functions from DNN filters
This commit removes the unused sync mode specific code from the DNN
filters since the sync and async mode are now unified from the
filters' perspective.
Signed-off-by: Shubhanshu Saxena
---
libavfilter/vf_dnn_detect.c | 71 +---
libavfilter/vf_dnn_processing.c | 84 +
2 files changed, 4 insertions(+), 151 deletions(-)
diff --git a/libavfilter/vf_dnn_detect.c b/libavfilter/vf_dnn_detect.c
index 11de376753..809d70b930 100644
--- a/libavfilter/vf_dnn_detect.c
+++ b/libavfilter/vf_dnn_detect.c
@@ -353,63 +353,6 @@ static int dnn_detect_query_formats(AVFilterContext
*context)
return ff_set_common_formats_from_list(context, pix_fmts);
}
-static int dnn_detect_filter_frame(AVFilterLink *inlink, AVFrame *in)
-{
-AVFilterContext *context = inlink->dst;
-AVFilterLink *outlink = context->outputs[0];
-DnnDetectContext *ctx = context->priv;
-DNNReturnType dnn_result;
-
-dnn_result = ff_dnn_execute_model(&ctx->dnnctx, in, in);
-if (dnn_result != DNN_SUCCESS){
-av_log(ctx, AV_LOG_ERROR, "failed to execute model\n");
-av_frame_free(&in);
-return AVERROR(EIO);
-}
-
-return ff_filter_frame(outlink, in);
-}
-
-static int dnn_detect_activate_sync(AVFilterContext *filter_ctx)
-{
-AVFilterLink *inlink = filter_ctx->inputs[0];
-AVFilterLink *outlink = filter_ctx->outputs[0];
-AVFrame *in = NULL;
-int64_t pts;
-int ret, status;
-int got_frame = 0;
-
-FF_FILTER_FORWARD_STATUS_BACK(outlink, inlink);
-
-do {
-// drain all input frames
-ret = ff_inlink_consume_frame(inlink, &in);
-if (ret < 0)
-return ret;
-if (ret > 0) {
-ret = dnn_detect_filter_frame(inlink, in);
-if (ret < 0)
-return ret;
-got_frame = 1;
-}
-} while (ret > 0);
-
-// if frame got, schedule to next filter
-if (got_frame)
-return 0;
-
-if (ff_inlink_acknowledge_status(inlink, &status, &pts)) {
-if (status == AVERROR_EOF) {
-ff_outlink_set_status(outlink, status, pts);
-return ret;
-}
-}
-
-FF_FILTER_FORWARD_WANTED(outlink, inlink);
-
-return FFERROR_NOT_READY;
-}
-
static int dnn_detect_flush_frame(AVFilterLink *outlink, int64_t pts, int64_t
*out_pts)
{
DnnDetectContext *ctx = outlink->src->priv;
@@ -439,7 +382,7 @@ static int dnn_detect_flush_frame(AVFilterLink *outlink,
int64_t pts, int64_t *o
return 0;
}
-static int dnn_detect_activate_async(AVFilterContext *filter_ctx)
+static int dnn_detect_activate(AVFilterContext *filter_ctx)
{
AVFilterLink *inlink = filter_ctx->inputs[0];
AVFilterLink *outlink = filter_ctx->outputs[0];
@@ -496,16 +439,6 @@ static int dnn_detect_activate_async(AVFilterContext
*filter_ctx)
return 0;
}
-static av_unused int dnn_detect_activate(AVFilterContext *filter_ctx)
-{
-DnnDetectContext *ctx = filter_ctx->priv;
-
-if (ctx->dnnctx.async)
-return dnn_detect_activate_async(filter_ctx);
-else
-return dnn_detect_activate_sync(filter_ctx);
-}
-
static av_cold void dnn_detect_uninit(AVFilterContext *context)
{
DnnDetectContext *ctx = context->priv;
@@ -537,5 +470,5 @@ const AVFilter ff_vf_dnn_detect = {
FILTER_INPUTS(dnn_detect_inputs),
FILTER_OUTPUTS(dnn_detect_outputs),
.priv_class= &dnn_detect_class,
-.activate = dnn_detect_activate_async,
+.activate = dnn_detect_activate,
};
diff --git a/libavfilter/vf_dnn_processing.c b/libavfilter/vf_dnn_processing.c
index 7435dd4959..55634efde5 100644
--- a/libavfilter/vf_dnn_processing.c
+++ b/libavfilter/vf_dnn_processing.c
@@ -244,76 +244,6 @@ static int copy_uv_planes(DnnProcessingContext *ctx,
AVFrame *out, const AVFrame
return 0;
}
-static int filter_frame(AVFilterLink *inlink, AVFrame *in)
-{
-AVFilterContext *context = inlink->dst;
-AVFilterLink *outlink = context->outputs[0];
-DnnProcessingContext *ctx = context->priv;
-DNNReturnType dnn_result;
-AVFrame *out;
-
-out = ff_get_video_buffer(outlink, outlink->w, outlink->h);
-if (!out) {
-av_frame_free(&in);
-return AVERROR(ENOMEM);
-}
-av_frame_copy_props(out, in);
-
-dnn_result = ff_dnn_execute_model(&ctx->dnnctx, in, out);
-if (dnn_result != DNN_SUCCESS){
-av_log(ctx, AV_LOG_ERROR, "failed to execute model\n");
-av_frame_free(&in);
-av_frame_free(&out);
-return AVERROR(EIO);
-}
-
-if (isPlanarYUV(in->format))
-copy_uv_planes(ctx, out, in);
-
-av_frame_free(&in);
-return ff_filter_frame(outlink, out);
-}
-
-static int activate_sync(AVFilterContext *filter_ctx)
-{
-AVFilterLink *inlink = filter_ctx->inputs[0];
-AVFilterLink *outlink = filter_ctx->outputs[0];
-AVFrame *in = NULL;
-int64_t pts;
-int ret, status;
-int got_frame = 0;
-
-FF_FILTER_FORWARD_STAT
[FFmpeg-devel] [PATCH v4 4/7] libavfilter: Remove Async Flag from DNN Filter Side
Remove async flag from filter's perspective after the unification
of async and sync modes in the DNN backend.
Signed-off-by: Shubhanshu Saxena
---
doc/filters.texi | 14 --
libavfilter/dnn/dnn_backend_tf.c | 7 +++
libavfilter/dnn_filter_common.c | 7 ---
libavfilter/dnn_filter_common.h | 2 +-
4 files changed, 12 insertions(+), 18 deletions(-)
diff --git a/doc/filters.texi b/doc/filters.texi
index b902aca12d..d99368e64b 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -10283,11 +10283,8 @@ and the second line is the name of label id 1, etc.
The label id is considered as name if the label file is not provided.
@item backend_configs
-Set the configs to be passed into backend
-
-@item async
-use DNN async execution if set (default: set),
-roll back to sync execution if the backend does not support async.
+Set the configs to be passed into backend. To use async execution, set async
(default: set).
+Roll back to sync execution if the backend does not support async.
@end table
@@ -10339,15 +10336,12 @@ Set the input name of the dnn network.
Set the output name of the dnn network.
@item backend_configs
-Set the configs to be passed into backend
+Set the configs to be passed into backend. To use async execution, set async
(default: set).
+Roll back to sync execution if the backend does not support async.
For tensorflow backend, you can set its configs with @option{sess_config}
options,
please use tools/python/tf_sess_config.py to get the configs of TensorFlow
backend for your system.
-@item async
-use DNN async execution if set (default: set),
-roll back to sync execution if the backend does not support async.
-
@end table
@subsection Examples
diff --git a/libavfilter/dnn/dnn_backend_tf.c b/libavfilter/dnn/dnn_backend_tf.c
index 4a0b561f29..906934d8c0 100644
--- a/libavfilter/dnn/dnn_backend_tf.c
+++ b/libavfilter/dnn/dnn_backend_tf.c
@@ -884,6 +884,13 @@ DNNModel *ff_dnn_load_model_tf(const char *model_filename,
DNNFunctionType func_
ctx->options.nireq = av_cpu_count() / 2 + 1;
}
+#if !HAVE_PTHREAD_CANCEL
+if (ctx->options.async) {
+ctx->options.async = 0;
+av_log(filter_ctx, AV_LOG_WARNING, "pthread is not supported, roll
back to sync.\n");
+}
+#endif
+
tf_model->request_queue = ff_safe_queue_create();
if (!tf_model->request_queue) {
goto err;
diff --git a/libavfilter/dnn_filter_common.c b/libavfilter/dnn_filter_common.c
index 455eaa37f4..3045ce0131 100644
--- a/libavfilter/dnn_filter_common.c
+++ b/libavfilter/dnn_filter_common.c
@@ -84,13 +84,6 @@ int ff_dnn_init(DnnContext *ctx, DNNFunctionType func_type,
AVFilterContext *fil
return AVERROR(EINVAL);
}
-#if !HAVE_PTHREAD_CANCEL
-if (ctx->async) {
-ctx->async = 0;
-av_log(filter_ctx, AV_LOG_WARNING, "pthread is not supported, roll
back to sync.\n");
-}
-#endif
-
return 0;
}
diff --git a/libavfilter/dnn_filter_common.h b/libavfilter/dnn_filter_common.h
index 4d92c1dc36..635ae631c1 100644
--- a/libavfilter/dnn_filter_common.h
+++ b/libavfilter/dnn_filter_common.h
@@ -46,7 +46,7 @@ typedef struct DnnContext {
{ "output", "output name of the model",
OFFSET(model_outputnames_string), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0,
FLAGS },\
{ "backend_configs","backend configs",
OFFSET(backend_options), AV_OPT_TYPE_STRING,{ .str = NULL }, 0, 0, FLAGS
},\
{ "options", "backend configs (deprecated, use backend_configs)",
OFFSET(backend_options), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, FLAGS |
AV_OPT_FLAG_DEPRECATED},\
-{ "async", "use DNN async inference",OFFSET(async),
AV_OPT_TYPE_BOOL, { .i64 = 1}, 0, 1, FLAGS},
+{ "async", "use DNN async inference (ignored, use
backend_configs='async=1')",OFFSET(async),AV_OPT_TYPE_BOOL,
{ .i64 = 1}, 0, 1, FLAGS},
int ff_dnn_init(DnnContext *ctx, DNNFunctionType func_type, AVFilterContext
*filter_ctx);
--
2.25.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v4 5/7] lavfi/dnn: Rename InferenceItem to LastLevelTaskItem
This patch renames the InferenceItem to LastLevelTaskItem in the
three backends to avoid confusion among the meanings of these structs.
The following are the renames done in this patch:
1. extract_inference_from_task -> extract_lltask_from_task
2. InferenceItem -> LastLevelTaskItem
3. inference_queue -> lltask_queue
4. inference -> lltask
5. inference_count -> lltask_count
Signed-off-by: Shubhanshu Saxena
---
libavfilter/dnn/dnn_backend_common.h | 4 +-
libavfilter/dnn/dnn_backend_native.c | 58 ++---
libavfilter/dnn/dnn_backend_native.h | 2 +-
libavfilter/dnn/dnn_backend_openvino.c | 110 -
libavfilter/dnn/dnn_backend_tf.c | 76 -
5 files changed, 125 insertions(+), 125 deletions(-)
diff --git a/libavfilter/dnn/dnn_backend_common.h
b/libavfilter/dnn/dnn_backend_common.h
index 78e62a94a2..6b6a5e21ae 100644
--- a/libavfilter/dnn/dnn_backend_common.h
+++ b/libavfilter/dnn/dnn_backend_common.h
@@ -47,10 +47,10 @@ typedef struct TaskItem {
} TaskItem;
// one task might have multiple inferences
-typedef struct InferenceItem {
+typedef struct LastLevelTaskItem {
TaskItem *task;
uint32_t bbox_index;
-} InferenceItem;
+} LastLevelTaskItem;
/**
* Common Async Execution Mechanism for the DNN Backends.
diff --git a/libavfilter/dnn/dnn_backend_native.c
b/libavfilter/dnn/dnn_backend_native.c
index 2d34b88f8a..13436c0484 100644
--- a/libavfilter/dnn/dnn_backend_native.c
+++ b/libavfilter/dnn/dnn_backend_native.c
@@ -46,25 +46,25 @@ static const AVClass dnn_native_class = {
.category = AV_CLASS_CATEGORY_FILTER,
};
-static DNNReturnType execute_model_native(Queue *inference_queue);
+static DNNReturnType execute_model_native(Queue *lltask_queue);
-static DNNReturnType extract_inference_from_task(TaskItem *task, Queue
*inference_queue)
+static DNNReturnType extract_lltask_from_task(TaskItem *task, Queue
*lltask_queue)
{
NativeModel *native_model = task->model;
NativeContext *ctx = &native_model->ctx;
-InferenceItem *inference = av_malloc(sizeof(*inference));
+LastLevelTaskItem *lltask = av_malloc(sizeof(*lltask));
-if (!inference) {
-av_log(ctx, AV_LOG_ERROR, "Unable to allocate space for
InferenceItem\n");
+if (!lltask) {
+av_log(ctx, AV_LOG_ERROR, "Unable to allocate space for
LastLevelTaskItem\n");
return DNN_ERROR;
}
task->inference_todo = 1;
task->inference_done = 0;
-inference->task = task;
+lltask->task = task;
-if (ff_queue_push_back(inference_queue, inference) < 0) {
-av_log(ctx, AV_LOG_ERROR, "Failed to push back inference_queue.\n");
-av_freep(&inference);
+if (ff_queue_push_back(lltask_queue, lltask) < 0) {
+av_log(ctx, AV_LOG_ERROR, "Failed to push back lltask_queue.\n");
+av_freep(&lltask);
return DNN_ERROR;
}
return DNN_SUCCESS;
@@ -116,13 +116,13 @@ static DNNReturnType get_output_native(void *model, const
char *input_name, int
goto err;
}
-if (extract_inference_from_task(&task, native_model->inference_queue) !=
DNN_SUCCESS) {
-av_log(ctx, AV_LOG_ERROR, "unable to extract inference from task.\n");
+if (extract_lltask_from_task(&task, native_model->lltask_queue) !=
DNN_SUCCESS) {
+av_log(ctx, AV_LOG_ERROR, "unable to extract last level task from
task.\n");
ret = DNN_ERROR;
goto err;
}
-ret = execute_model_native(native_model->inference_queue);
+ret = execute_model_native(native_model->lltask_queue);
*output_width = task.out_frame->width;
*output_height = task.out_frame->height;
@@ -223,8 +223,8 @@ DNNModel *ff_dnn_load_model_native(const char
*model_filename, DNNFunctionType f
goto fail;
}
-native_model->inference_queue = ff_queue_create();
-if (!native_model->inference_queue) {
+native_model->lltask_queue = ff_queue_create();
+if (!native_model->lltask_queue) {
goto fail;
}
@@ -297,24 +297,24 @@ fail:
return NULL;
}
-static DNNReturnType execute_model_native(Queue *inference_queue)
+static DNNReturnType execute_model_native(Queue *lltask_queue)
{
NativeModel *native_model = NULL;
NativeContext *ctx = NULL;
int32_t layer;
DNNData input, output;
DnnOperand *oprd = NULL;
-InferenceItem *inference = NULL;
+LastLevelTaskItem *lltask = NULL;
TaskItem *task = NULL;
DNNReturnType ret = 0;
-inference = ff_queue_pop_front(inference_queue);
-if (!inference) {
-av_log(NULL, AV_LOG_ERROR, "Failed to get inference item\n");
+lltask = ff_queue_pop_front(lltask_queue);
+if (!lltask) {
+av_log(NULL, AV_LOG_ERROR, "Failed to get LastLevelTaskItem\n");
ret = DNN_ERROR;
goto err;
}
-task = inference->task;
+task = lltask->task;
native_model = task->model;
ctx = &native_model->ctx;
@@ -428,7 +428,7 @@ static DNNReturnType e
[FFmpeg-devel] [PATCH v4 6/7] doc/filters.texi: Include dnn_processing in docs of sr and derain filter
Signed-off-by: Shubhanshu Saxena
---
doc/filters.texi | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/doc/filters.texi b/doc/filters.texi
index d99368e64b..112adc5d94 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -9982,7 +9982,7 @@ Note that different backends use different file formats.
TensorFlow and native
backend can load files for only its format.
@end table
-It can also be finished with @ref{dnn_processing} filter.
+To get full functionality (such as async execution), please use the
@ref{dnn_processing} filter.
@section deshake
@@ -19271,7 +19271,7 @@ Default value is @code{2}. Scale factor is necessary
for SRCNN model, because it
input upscaled using bicubic upscaling with proper scale factor.
@end table
-This feature can also be finished with @ref{dnn_processing} filter.
+To get full functionality (such as async execution), please use the
@ref{dnn_processing} filter.
@section ssim
--
2.25.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v4 7/7] libavfilter: Send only input frame for DNN Detect and Classify
This commit updates the following two filters to send only the
input frame and send NULL as output frame instead of input frame
to the DNN backends.
1. vf_dnn_detect
2. vf_dnn_classify
Signed-off-by: Shubhanshu Saxena
---
libavfilter/dnn/dnn_backend_common.c | 2 +-
libavfilter/dnn/dnn_backend_openvino.c | 5 +++--
libavfilter/dnn/dnn_backend_tf.c | 2 +-
libavfilter/vf_dnn_classify.c | 14 ++
libavfilter/vf_dnn_detect.c| 14 ++
5 files changed, 17 insertions(+), 20 deletions(-)
diff --git a/libavfilter/dnn/dnn_backend_common.c
b/libavfilter/dnn/dnn_backend_common.c
index d2bc016fef..6a9c4cc87f 100644
--- a/libavfilter/dnn/dnn_backend_common.c
+++ b/libavfilter/dnn/dnn_backend_common.c
@@ -38,7 +38,7 @@ int ff_check_exec_params(void *ctx, DNNBackendType backend,
DNNFunctionType func
return AVERROR(EINVAL);
}
-if (!exec_params->out_frame) {
+if (!exec_params->out_frame && func_type == DFT_PROCESS_FRAME) {
av_log(ctx, AV_LOG_ERROR, "out frame is NULL when execute model.\n");
return AVERROR(EINVAL);
}
diff --git a/libavfilter/dnn/dnn_backend_openvino.c
b/libavfilter/dnn/dnn_backend_openvino.c
index 76dc06c6d7..f5b1454d21 100644
--- a/libavfilter/dnn/dnn_backend_openvino.c
+++ b/libavfilter/dnn/dnn_backend_openvino.c
@@ -272,14 +272,14 @@ static void infer_completion_callback(void *args)
av_log(ctx, AV_LOG_ERROR, "detect filter needs to provide post
proc\n");
return;
}
-ov_model->model->detect_post_proc(task->out_frame, &output, 1,
ov_model->model->filter_ctx);
+ov_model->model->detect_post_proc(task->in_frame, &output, 1,
ov_model->model->filter_ctx);
break;
case DFT_ANALYTICS_CLASSIFY:
if (!ov_model->model->classify_post_proc) {
av_log(ctx, AV_LOG_ERROR, "classify filter needs to provide
post proc\n");
return;
}
-ov_model->model->classify_post_proc(task->out_frame, &output,
request->lltasks[i]->bbox_index, ov_model->model->filter_ctx);
+ov_model->model->classify_post_proc(task->in_frame, &output,
request->lltasks[i]->bbox_index, ov_model->model->filter_ctx);
break;
default:
av_assert0(!"should not reach here");
@@ -819,6 +819,7 @@ DNNReturnType ff_dnn_execute_model_ov(const DNNModel
*model, DNNExecBaseParams *
if (model->func_type == DFT_ANALYTICS_CLASSIFY) {
// Classification filter has not been completely
// tested with the sync mode. So, do not support now.
+avpriv_report_missing_feature(ctx, "classify for sync execution");
return DNN_ERROR;
}
diff --git a/libavfilter/dnn/dnn_backend_tf.c b/libavfilter/dnn/dnn_backend_tf.c
index dfac58b357..c95cad7944 100644
--- a/libavfilter/dnn/dnn_backend_tf.c
+++ b/libavfilter/dnn/dnn_backend_tf.c
@@ -1069,7 +1069,7 @@ static void infer_completion_callback(void *args) {
av_log(ctx, AV_LOG_ERROR, "Detect filter needs provide post
proc\n");
return;
}
-tf_model->model->detect_post_proc(task->out_frame, outputs,
task->nb_output, tf_model->model->filter_ctx);
+tf_model->model->detect_post_proc(task->in_frame, outputs,
task->nb_output, tf_model->model->filter_ctx);
break;
default:
av_log(ctx, AV_LOG_ERROR, "Tensorflow backend does not support this
kind of dnn filter now\n");
diff --git a/libavfilter/vf_dnn_classify.c b/libavfilter/vf_dnn_classify.c
index d5ee65d403..d1ba8dffbc 100644
--- a/libavfilter/vf_dnn_classify.c
+++ b/libavfilter/vf_dnn_classify.c
@@ -225,13 +225,12 @@ static int dnn_classify_flush_frame(AVFilterLink
*outlink, int64_t pts, int64_t
AVFrame *in_frame = NULL;
AVFrame *out_frame = NULL;
async_state = ff_dnn_get_result(&ctx->dnnctx, &in_frame, &out_frame);
-if (out_frame) {
-av_assert0(in_frame == out_frame);
-ret = ff_filter_frame(outlink, out_frame);
+if (async_state == DAST_SUCCESS) {
+ret = ff_filter_frame(outlink, in_frame);
if (ret < 0)
return ret;
if (out_pts)
-*out_pts = out_frame->pts + pts;
+*out_pts = in_frame->pts + pts;
}
av_usleep(5000);
} while (async_state >= DAST_NOT_READY);
@@ -258,7 +257,7 @@ static int dnn_classify_activate(AVFilterContext
*filter_ctx)
if (ret < 0)
return ret;
if (ret > 0) {
-if (ff_dnn_execute_model_classification(&ctx->dnnctx, in, in,
ctx->target) != DNN_SUCCESS) {
+if (ff_dnn_execute_model_classification(&ctx->dnnctx, in, NULL,
ctx->target) != DNN_SUCCESS) {
return AVERROR(EIO);
}
}
@@ -269,9 +268,8 @@ static int dnn_classify_activate(AVFilterContext
*filter_ctx)
AVF
[FFmpeg-devel] [PATCH] ffmpeg: fix loosing gaps between audio frame timestamps when filtering
Signed-off-by: Paul B Mahol
---
fftools/ffmpeg.c| 5 +
fftools/ffmpeg.h| 1 +
fftools/ffmpeg_opt.c| 1 +
tests/ref/fate/adpcm-ima-smjpeg | 658
tests/ref/fate/dcinema-encode | 26 +-
tests/ref/lavf/smjpeg | 2 +-
6 files changed, 350 insertions(+), 343 deletions(-)
diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c
index b0ce7c7c32..bde36dbb12 100644
--- a/fftools/ffmpeg.c
+++ b/fftools/ffmpeg.c
@@ -2382,6 +2382,11 @@ static int decode_audio(InputStream *ist, AVPacket *pkt,
int *got_output,
decoded_frame->pts = ist->dts;
decoded_frame_tb = AV_TIME_BASE_Q;
}
+if (pkt && pkt->duration && ist->prev_pkt_pts != AV_NOPTS_VALUE &&
+pkt->pts != AV_NOPTS_VALUE && pkt->pts - ist->prev_pkt_pts >
pkt->duration)
+ist->filter_in_rescale_delta_last = 0;
+if (pkt)
+ist->prev_pkt_pts = pkt->pts;
if (decoded_frame->pts != AV_NOPTS_VALUE)
decoded_frame->pts = av_rescale_delta(decoded_frame_tb,
decoded_frame->pts,
(AVRational){1,
avctx->sample_rate}, decoded_frame->nb_samples,
&ist->filter_in_rescale_delta_last,
diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h
index d2dd7ca092..6657773d32 100644
--- a/fftools/ffmpeg.h
+++ b/fftools/ffmpeg.h
@@ -313,6 +313,7 @@ typedef struct InputStream {
AVFrame *filter_frame; /* a ref of decoded_frame, to be sent to filters */
AVPacket *pkt;
+int64_t prev_pkt_pts;
int64_t start; /* time when read started */
/* predicted dts of the next packet read for this stream or (when there are
* several frames in a packet) of the next frame in current packet (in
AV_TIME_BASE units) */
diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c
index 428934a3d8..a7d35b1ee2 100644
--- a/fftools/ffmpeg_opt.c
+++ b/fftools/ffmpeg_opt.c
@@ -867,6 +867,7 @@ static void add_input_streams(OptionsContext *o,
AVFormatContext *ic)
}
ist->filter_in_rescale_delta_last = AV_NOPTS_VALUE;
+ist->prev_pkt_pts = AV_NOPTS_VALUE;
ist->dec_ctx = avcodec_alloc_context3(ist->dec);
if (!ist->dec_ctx) {
diff --git a/tests/ref/fate/adpcm-ima-smjpeg b/tests/ref/fate/adpcm-ima-smjpeg
index d56014ea01..a216e3ab5e 100644
--- a/tests/ref/fate/adpcm-ima-smjpeg
+++ b/tests/ref/fate/adpcm-ima-smjpeg
@@ -9,348 +9,348 @@
0, 1024, 1024, 512, 1024, 0xed2d3f6b
0, 1533, 1533, 512, 1024, 0x51f6ccb3
0, 2040, 2040, 512, 1024, 0x58bd75aa
-0, 2552, 2552, 512, 1024, 0xd857a310
-0, 3064, 3064, 512, 1024, 0xc483a5b8
-0, 3576, 3576, 512, 1024, 0x923ecf67
-0, 4088, 4088, 512, 1024, 0xf87dcd53
+0, 2558, 2558, 512, 1024, 0xd857a310
+0, 3070, 3070, 512, 1024, 0xc483a5b8
+0, 3582, 3582, 512, 1024, 0x923ecf67
+0, 4091, 4091, 512, 1024, 0xf87dcd53
0, 4598, 4598, 512, 1024, 0xdc32c002
-0, 5110, 5110, 512, 1024, 0xb760def1
-0, 5622, 5622, 512, 1024, 0x6838d2b2
-0, 6134, 6134, 512, 1024, 0xe45aca1e
-0, 6646, 6646, 512, 1024, 0xde1fb955
-0, 7158, 7158, 512, 1024, 0x9e23b949
-0, 7670, 7670, 512, 1024, 0x840cc000
-0, 8182, 8182, 512, 1024, 0x0a29cbfa
-0, 8694, 8694, 512, 1024, 0x9871d4c4
+0, 5116, 5116, 512, 1024, 0xb760def1
+0, 5628, 5628, 512, 1024, 0x6838d2b2
+0, 6140, 6140, 512, 1024, 0xe45aca1e
+0, 6649, 6649, 512, 1024, 0xde1fb955
+0, 7166, 7166, 512, 1024, 0x9e23b949
+0, 7678, 7678, 512, 1024, 0x840cc000
+0, 8190, 8190, 512, 1024, 0x0a29cbfa
+0, 8699, 8699, 512, 1024, 0x9871d4c4
0, 9206, 9206, 512, 1024, 0xb35dc9f2
-0, 9718, 9718, 512, 1024, 0xf37fda0a
-0, 10230, 10230, 512, 1024, 0xa640f990
-0, 10742, 10742, 512, 1024, 0x516fe6f5
-0, 11254, 11254, 512, 1024, 0xc78bc6a6
-0, 11766, 11766, 512, 1024, 0x700fd6ee
-0, 12278, 12278, 512, 1024, 0x5383d5ad
-0, 12790, 12790, 512, 1024, 0xbe01d091
-0, 13302, 13302, 512, 1024, 0x72dfcfc7
-0, 13814, 13814, 512, 1024, 0xd8fecea9
-0, 14326, 14326, 512, 1024, 0xa464d79b
-0, 14838, 14838, 512, 1024, 0xf394e2cb
-0, 15350, 15350, 512, 1024, 0xa301ec49
-0, 15862, 15862, 512, 1024, 0x5e09d60f
-0, 16374, 16374, 512, 1024, 0xd13edd6f
-0, 16886,
Re: [FFmpeg-devel] [PATCH 1/2] avformat hls fix to seek logic
lgtm ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] Compatibility of different library versions (was: avformat: Add internal flags for AV(In|Out)putFormat)
Top-posting on purpose: Several core developers stated to be in favor of locking lavd<->lavf together at the minor version, for instance since it solves a lot of the problems the intimate linking between the two libraries creates. Should this be turned into a patch somehow (i'm not sure how to do such linking)? If so, could someone send this patch or guide me through it, so that the discussion can be finished? Thanks! On Thu, Jun 17, 2021 at 12:45 AM James Almer wrote: > > On 6/16/2021 7:14 PM, Nicolas George wrote: > > James Almer (12021-06-16): > >> I'm not sure what you mean. I would not be against it, it's just that if we > >> were to merge lavf and lavd, this wouldn't even be something to consider. > > > > Have you not read the discussion? The benefits go way beyond the tiny > > lavf-lavd issues. > > > >>> and why you are > >>> against for other libraries. > >> Can you be more specific? > > > > When I say "I am for X" and you reply "I am not against Y", with Y⊂X and > > Y≠X, you are implicitly saying that you are against X∖Y. I proposed to > > restrict to matching versions on all libraries, you replied you were not > > against restricting for lavf-lavd, stating implicitly that you are > > against it for the libraries. > > Considering this discussion started about lavd and lavf, and at no point > in your email you said "all libraries" when suggesting version locking > anything (In fact, you mistakenly wrote "only versions from the same > version"), i figured it would be obvious i was commenting specifically > about lavd<->lavf. > > > > > So, I ask explicitly: are you against restricting to matching versions > > for all the libraries? If so, then why? > > Of course i am against doing so for all libraries. They are already > locked at the major soname level as required, and that's enough. > Lavd as is is locking certain lavf's public structs and making it > impossible to add new fields to them, which constricts development > considerably, so making their locking strict to at least the minor > soname level is a very good idea. But this isn't an issue at all for > other libraries. > I can add new fields to any public lavc struct, compile it and have a > lavf that was linked to an old lavc use it at runtime, and it will work > just fine. > I see no benefit to your suggestion that's worth removing such freedom > from the end user. > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 1/5] libavfilter/x86/vf_gblur: add ff_postscale_slice_avx512()
will apply shortly ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 1/5] libavfilter/x86/vf_gblur: add ff_postscale_slice_avx512()
On 8/19/2021 3:58 AM, Paul B Mahol wrote: On Mon, Aug 16, 2021 at 1:49 PM James Almer wrote: On 8/16/2021 6:27 AM, Wu, Jianhua wrote: Yes, but we don't want to use AVX512 on hardware that downclocks heavily. It's okay. Should I updated the configure file to let the IceLake-AVX512 to be the minimum baseline or add a new macro name AVX512ICL? Update the test in configure to look for an instruction available on Ice Lake (like VL), then update the check in cpu.c to make sure AVX2 is only enabled if AVX2-related flags Ice Lake and newer CPUs have in common are signaled. I think you are confused. AVX2 is doing fine. Meant to say AVX512, yes. What are you planning to push? Was a patch bumping the requirements for the current AVX512 flag as discussed submitted or not? Jianhua ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 1/5] libavfilter/x86/vf_gblur: add ff_postscale_slice_avx512()
On Wed, Aug 25, 2021 at 3:51 PM James Almer wrote: > On 8/19/2021 3:58 AM, Paul B Mahol wrote: > > On Mon, Aug 16, 2021 at 1:49 PM James Almer wrote: > > > >> On 8/16/2021 6:27 AM, Wu, Jianhua wrote: > > Yes, but we don't want to use AVX512 on hardware that downclocks > >> heavily. > > >>> > >>> It's okay. Should I updated the configure file to let the > IceLake-AVX512 > >> to be the minimum baseline or add a new macro name AVX512ICL? > >> > >> Update the test in configure to look for an instruction available on Ice > >> Lake (like VL), then update the check in cpu.c to make sure AVX2 is only > >> enabled if AVX2-related flags Ice Lake and newer CPUs have in common are > >> signaled. > >> > > > > I think you are confused. AVX2 is doing fine. > > Meant to say AVX512, yes. > > What are you planning to push? Was a patch bumping the requirements for > the current AVX512 flag as discussed submitted or not? > Everything. You can add additional commits later when you all finally decide what to actually do. > > > > >> > >>> > >>> Jianhua > >>> > >>> ___ > >>> ffmpeg-devel mailing list > >>> ffmpeg-devel@ffmpeg.org > >>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > >>> > >>> To unsubscribe, visit link above, or email > >>> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". > >>> > >> > >> ___ > >> ffmpeg-devel mailing list > >> ffmpeg-devel@ffmpeg.org > >> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > >> > >> To unsubscribe, visit link above, or email > >> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". > >> > > ___ > > ffmpeg-devel mailing list > > ffmpeg-devel@ffmpeg.org > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > > To unsubscribe, visit link above, or email > > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". > > > > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v1] avfilter/vf_guided: support enhanced guided filter
From: Xuewei Meng
Enhanced guided filter and fast enhanced guided filter are supported.
The enhanced guided filter can better preserve edges for denoising tasks.
And the box filter operation is optimized.
Signed-off-by: Xuewei Meng
---
doc/filters.texi| 14 +-
libavfilter/vf_guided.c | 331
2 files changed, 315 insertions(+), 30 deletions(-)
diff --git a/doc/filters.texi b/doc/filters.texi
index f8d99b7..c65a347 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -13093,13 +13093,13 @@ Set regularization parameter (with square).
Allowed range is 0 to 1. Default is 0.01.
@item mode
-Set filter mode. Can be @code{basic} or @code{fast}.
+Set filter mode. Can be @code{basic}, @code{fast basic}, @code{enhanced} or
@code{fast enhanced}.
Default is @code{basic}.
@item sub
-Set subsampling ratio for @code{fast} mode.
+Set subsampling ratio for @code{fast} or @code{fast enhanced} mode.
Range is 2 to 64. Default is 4.
-No subsampling occurs in @code{basic} mode.
+No subsampling occurs in @code{basic} and @code{enhanced} mode.
@item guidance
Set guidance mode. Can be @code{off} or @code{on}. Default is @code{off}.
@@ -13123,6 +13123,14 @@ ffmpeg -i in.png -vf guided out.png
@end example
@item
+Edge-preserving smoothing with enhanced guided filter.
+For the details of enhanced guided filter, refer to paper "Side window guided
filtering".
+See:
@url{https://www.sciencedirect.com/science/article/abs/pii/S0165168419302798}.
+@example
+ffmpeg -i in.png -vf guided=mode=2 out.png
+@end example
+
+@item
Dehazing, structure-transferring filtering, detail enhancement with guided
filter.
For the generation of guidance image, refer to paper "Guided Image Filtering".
See: @url{http://kaiminghe.com/publications/pami12guidedfilter.pdf}.
diff --git a/libavfilter/vf_guided.c b/libavfilter/vf_guided.c
index 4003b95..c030080 100644
--- a/libavfilter/vf_guided.c
+++ b/libavfilter/vf_guided.c
@@ -30,7 +30,9 @@
enum FilterModes {
BASIC,
-FAST,
+FAST_BASIC,
+ENHANCED,
+FAST_ENHANCED,
NB_MODES,
};
@@ -40,6 +42,18 @@ enum GuidanceModes {
NB_GUIDANCE_MODES,
};
+enum SideWindowModes {
+LEFT,
+RIGHT,
+UP,
+DOWN,
+NW, // North west
+NE, // North east
+SW, // South west
+SE, // South east
+NB_SIDEWINDOWMODES
+};
+
typedef struct GuidedContext {
const AVClass *class;
FFFrameSync fs;
@@ -60,6 +74,7 @@ typedef struct GuidedContext {
int planeheight[4];
int (*box_slice)(AVFilterContext *ctx, void *arg, int jobnr, int nb_jobs);
+int (*box_enhanced_slice)(AVFilterContext *ctx, void *arg, int jobnr, int
nb_jobs);
} GuidedContext;
#define OFFSET(x) offsetof(GuidedContext, x)
@@ -68,9 +83,12 @@ typedef struct GuidedContext {
static const AVOption guided_options[] = {
{ "radius", "set the box radius",
OFFSET(radius), AV_OPT_TYPE_INT, {.i64 = 3}, 1,
20, FLAGS },
{ "eps", "set the regularization parameter (with square)",
OFFSET(eps), AV_OPT_TYPE_FLOAT, {.dbl = 0.01 }, 0.0,
1, FLAGS },
-{ "mode", "set filtering mode (0: basic mode; 1: fast mode)",
OFFSET(mode), AV_OPT_TYPE_INT, {.i64 = BASIC}, BASIC, NB_MODES -
1, FLAGS, "mode" },
+{ "mode", "set filtering mode (0: basic mode; 1: fast basic mode;
3: enhanced mode; 4: fast enhanced mode)",
+
OFFSET(mode), AV_OPT_TYPE_INT, {.i64 = BASIC }, BASIC,
NB_MODES - 1, FLAGS, "mode" },
{ "basic","basic guided filter", 0,
AV_OPT_TYPE_CONST, {.i64 = BASIC}, 0, 0,
FLAGS, "mode" },
-{ "fast", "fast guided filter", 0,
AV_OPT_TYPE_CONST, {.i64 = FAST }, 0, 0,
FLAGS, "mode" },
+{ "basic fast","fast basic guided filter", 0,
AV_OPT_TYPE_CONST, {.i64 = FAST_BASIC}, 0,
0, FLAGS, "mode" },
+{ "enhanced", "enhanced guided filter", 0,
AV_OPT_TYPE_CONST, {.i64 = ENHANCED }, 0,
0, FLAGS, "mode" },
+{ "enhanced fast", "fast enhanced guided filter", 0,
AV_OPT_TYPE_CONST, {.i64 = FAST_ENHANCED }, 0,
0, FLAGS, "mode" },
{ "sub", "subsampling ratio for fast mode",
OFFSET(sub), AV_OPT_TYPE_INT, {.i64 = 4}, 2,
64, FLAGS },
{ "guidance", "set guidance mode (0: off mode; 1: on mode)",
OFFSET(guidance), AV_OPT_TYPE_INT, {.i64 = OFF }, OFF, NB_GUIDANCE_MODES -
1, FLAGS, "guidance" },
{ "off", "only one input is enabled",
[FFmpeg-devel] [PATCH v2] avfilter/vf_guided: support enhanced guided filter
From: Xuewei Meng
Check block algin
Signed-off-by: Xuewei Meng
---
libavfilter/vf_guided.c | 19 +--
1 file changed, 9 insertions(+), 10 deletions(-)
diff --git a/libavfilter/vf_guided.c b/libavfilter/vf_guided.c
index c030080..94318c0 100644
--- a/libavfilter/vf_guided.c
+++ b/libavfilter/vf_guided.c
@@ -81,19 +81,19 @@ typedef struct GuidedContext {
#define FLAGS
AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_RUNTIME_PARAM
static const AVOption guided_options[] = {
-{ "radius", "set the box radius",
OFFSET(radius), AV_OPT_TYPE_INT, {.i64 = 3}, 1,
20, FLAGS },
-{ "eps", "set the regularization parameter (with square)",
OFFSET(eps), AV_OPT_TYPE_FLOAT, {.dbl = 0.01 }, 0.0,
1, FLAGS },
+{ "radius","set the box radius",
OFFSET(radius), AV_OPT_TYPE_INT, {.i64 = 3 }, 1,
20, FLAGS },
+{ "eps", "set the regularization parameter (with square)",
OFFSET(eps), AV_OPT_TYPE_FLOAT, {.dbl = 0.01 }, 0.0,
1, FLAGS },
{ "mode", "set filtering mode (0: basic mode; 1: fast basic mode;
3: enhanced mode; 4: fast enhanced mode)",
OFFSET(mode), AV_OPT_TYPE_INT, {.i64 = BASIC }, BASIC,
NB_MODES - 1, FLAGS, "mode" },
-{ "basic","basic guided filter", 0,
AV_OPT_TYPE_CONST, {.i64 = BASIC}, 0, 0,
FLAGS, "mode" },
+{ "basic", "basic guided filter", 0,
AV_OPT_TYPE_CONST, {.i64 = BASIC }, 0,
0, FLAGS, "mode" },
{ "basic fast","fast basic guided filter", 0,
AV_OPT_TYPE_CONST, {.i64 = FAST_BASIC}, 0,
0, FLAGS, "mode" },
{ "enhanced", "enhanced guided filter", 0,
AV_OPT_TYPE_CONST, {.i64 = ENHANCED }, 0,
0, FLAGS, "mode" },
{ "enhanced fast", "fast enhanced guided filter", 0,
AV_OPT_TYPE_CONST, {.i64 = FAST_ENHANCED }, 0,
0, FLAGS, "mode" },
-{ "sub", "subsampling ratio for fast mode",
OFFSET(sub), AV_OPT_TYPE_INT, {.i64 = 4}, 2,
64, FLAGS },
-{ "guidance", "set guidance mode (0: off mode; 1: on mode)",
OFFSET(guidance), AV_OPT_TYPE_INT, {.i64 = OFF }, OFF, NB_GUIDANCE_MODES -
1, FLAGS, "guidance" },
-{ "off", "only one input is enabled",0,
AV_OPT_TYPE_CONST, {.i64 = OFF }, 0, 0,
FLAGS, "guidance" },
-{ "on", "two inputs are required", 0,
AV_OPT_TYPE_CONST, {.i64 = ON }, 0, 0,
FLAGS, "guidance" },
-{ "planes", "set planes to filter",
OFFSET(planes), AV_OPT_TYPE_INT, {.i64 = 1}, 0,
0xF, FLAGS },
+{ "sub", "subsampling ratio for fast mode",
OFFSET(sub), AV_OPT_TYPE_INT, {.i64 = 4 }, 2,
64, FLAGS },
+{ "guidance", "set guidance mode (0: off mode; 1: on mode)",
OFFSET(guidance), AV_OPT_TYPE_INT, {.i64 = OFF }, OFF,
NB_GUIDANCE_MODES - 1, FLAGS, "guidance" },
+{ "off", "only one input is enabled",0,
AV_OPT_TYPE_CONST, {.i64 = OFF }, 0,
0, FLAGS, "guidance" },
+{ "on","two inputs are required", 0,
AV_OPT_TYPE_CONST, {.i64 = ON}, 0,
0, FLAGS, "guidance" },
+{ "planes","set planes to filter",
OFFSET(planes), AV_OPT_TYPE_INT, {.i64 = 1 }, 0,
0xF, FLAGS },
{ NULL }
};
@@ -288,9 +288,8 @@ static int config_input(AVFilterLink *inlink)
} else if (s->mode == FAST_BASIC || s->mode == FAST_ENHANCED) {
if (s->radius >= s->sub)
s->radius = s->radius / s->sub;
-else {
+else
s->radius = 1;
-}
}
s->depth = desc->comp[0].depth;
--
1.9.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/8] avformat/vorbiscomment: Don't compute strlen twice
lgtm ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 2/2] avcodec/wma: handle run_level_decode error
lgtm ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 2/8] avformat/movenc: Avoid calling strlen multiple times
lgtm ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 3/3] avcodec/h264dec: apply H.274 film grain
> -Original Message-
> From: ffmpeg-devel On Behalf Of Niklas Haas
> Sent: Tuesday, August 17, 2021 12:26 PM
> To: ffmpeg-devel@ffmpeg.org
> Cc: Niklas Haas
> Subject: [FFmpeg-devel] [PATCH v2 3/3] avcodec/h264dec: apply H.274 film grain
>
> From: Niklas Haas
>
> Because we need access to ref frames without film grain applied, we have
> to add an extra AVFrame to H264Picture to avoid messing with the
> original. This requires some amount of overhead to make the reference
> moves work out, but it allows us to benefit from frame multithreading
> for film grain application "for free".
>
> Unfortunately, this approach requires twice as much RAM to be constantly
> allocated for ref frames, due to the need for an extra buffer per
> H264Picture. In theory, we could get away with freeing up this memory as
> soon as it's no longer needed (since ref frames do not need film grain
> buffers any longer), but trying to call ff_thread_release_buffer() from
> output_frame() conflicts with possible later accesses to that same frame
> and I'm not sure how to synchronize that well.
>
> Tested on all three cases of (no fg), (fg present but exported) and (fg
> present and not exported), with and without threading.
>
> Signed-off-by: Niklas Haas
> ---
> libavcodec/h264_picture.c | 35 +--
> libavcodec/h264_slice.c | 16 ++--
> libavcodec/h264dec.c | 39 +++
> libavcodec/h264dec.h | 6 ++
> 4 files changed, 80 insertions(+), 16 deletions(-)
>
> diff --git a/libavcodec/h264_picture.c b/libavcodec/h264_picture.c
> index ff30166b4d..5944798394 100644
> --- a/libavcodec/h264_picture.c
> +++ b/libavcodec/h264_picture.c
> @@ -43,13 +43,14 @@
>
> void ff_h264_unref_picture(H264Context *h, H264Picture *pic)
> {
> -int off = offsetof(H264Picture, tf) + sizeof(pic->tf);
> +int off = offsetof(H264Picture, tf_grain) + sizeof(pic->tf_grain);
> int i;
>
> if (!pic->f || !pic->f->buf[0])
> return;
>
> ff_thread_release_buffer(h->avctx, &pic->tf);
> +ff_thread_release_buffer(h->avctx, &pic->tf_grain);
> av_buffer_unref(&pic->hwaccel_priv_buf);
>
> av_buffer_unref(&pic->qscale_table_buf);
> @@ -93,6 +94,7 @@ static void h264_copy_picture_params(H264Picture *dst,
> const H264Picture *src)
> dst->mb_width = src->mb_width;
> dst->mb_height = src->mb_height;
> dst->mb_stride = src->mb_stride;
> +dst->needs_fg = src->needs_fg;
> }
>
> int ff_h264_ref_picture(H264Context *h, H264Picture *dst, H264Picture *src)
> @@ -108,6 +110,14 @@ int ff_h264_ref_picture(H264Context *h, H264Picture
> *dst, H264Picture *src)
> if (ret < 0)
> goto fail;
>
> +if (src->needs_fg) {
> +av_assert0(src->tf_grain.f == src->f_grain);
> +dst->tf_grain.f = dst->f_grain;
> +ret = ff_thread_ref_frame(&dst->tf_grain, &src->tf_grain);
> +if (ret < 0)
> +goto fail;
> +}
> +
> dst->qscale_table_buf = av_buffer_ref(src->qscale_table_buf);
> dst->mb_type_buf = av_buffer_ref(src->mb_type_buf);
> dst->pps_buf = av_buffer_ref(src->pps_buf);
> @@ -159,6 +169,15 @@ int ff_h264_replace_picture(H264Context *h, H264Picture
> *dst, const H264Picture
> if (ret < 0)
> goto fail;
>
> +if (src->needs_fg) {
> +av_assert0(src->tf_grain.f == src->f_grain);
> +dst->tf_grain.f = dst->f_grain;
> +ff_thread_release_buffer(h->avctx, &dst->tf_grain);
> +ret = ff_thread_ref_frame(&dst->tf_grain, &src->tf_grain);
> +if (ret < 0)
> +goto fail;
> +}
> +
> ret = av_buffer_replace(&dst->qscale_table_buf, src->qscale_table_buf);
> ret |= av_buffer_replace(&dst->mb_type_buf, src->mb_type_buf);
> ret |= av_buffer_replace(&dst->pps_buf, src->pps_buf);
> @@ -212,6 +231,7 @@ void ff_h264_set_erpic(ERPicture *dst, H264Picture *src)
> int ff_h264_field_end(H264Context *h, H264SliceContext *sl, int in_setup)
> {
> AVCodecContext *const avctx = h->avctx;
> +H264Picture *cur = h->cur_pic_ptr;
> int err = 0;
> h->mb_y = 0;
>
> @@ -230,10 +250,21 @@ int ff_h264_field_end(H264Context *h, H264SliceContext
> *sl, int in_setup)
> if (err < 0)
> av_log(avctx, AV_LOG_ERROR,
> "hardware accelerator failed to decode picture\n");
> +} else if (!in_setup && cur->needs_fg) {
> +AVFrameSideData *sd = av_frame_get_side_data(cur->f,
> AV_FRAME_DATA_FILM_GRAIN_PARAMS);
> +av_assert0(sd); // always present if `cur->needs_fg`
> +err = ff_h274_apply_film_grain(cur->f_grain, cur->f, &h->h274db,
> + (AVFilmGrainParams *) sd->data);
> +if (err < 0) {
> +av_log(h->avctx, AV_LOG_WARNING, "Failed synthesizing film "
> + "grain, ignoring: %s\n", av_err2str(err));
> +cur->needs_fg =
Re: [FFmpeg-devel] [PATCH v3] avcodec/h264dec: apply H.274 film grain
> -Original Message-
> From: ffmpeg-devel On Behalf Of Niklas Haas
> Sent: Tuesday, August 17, 2021 12:55 PM
> To: ffmpeg-devel@ffmpeg.org
> Cc: Niklas Haas
> Subject: [FFmpeg-devel] [PATCH v3] avcodec/h264dec: apply H.274 film grain
>
> From: Niklas Haas
>
> Because we need access to ref frames without film grain applied, we have
> to add an extra AVFrame to H264Picture to avoid messing with the
> original. This requires some amount of overhead to make the reference
> moves work out, but it allows us to benefit from frame multithreading
> for film grain application "for free".
>
> Unfortunately, this approach requires twice as much RAM to be constantly
> allocated for ref frames, due to the need for an extra buffer per
> H264Picture. In theory, we could get away with freeing up this memory as
> soon as it's no longer needed (since ref frames do not need film grain
> buffers any longer), but trying to call ff_thread_release_buffer() from
> output_frame() conflicts with possible later accesses to that same frame
> and I'm not sure how to synchronize that well.
>
> Tested on all three cases of (no fg), (fg present but exported) and (fg
> present and not exported), with and without threading.
>
> Signed-off-by: Niklas Haas
> ---
> libavcodec/h264_picture.c | 35 +++--
> libavcodec/h264_slice.c | 16 ++--
> libavcodec/h264dec.c | 55 ++-
> libavcodec/h264dec.h | 6 +
> 4 files changed, 90 insertions(+), 22 deletions(-)
>
> diff --git a/libavcodec/h264_picture.c b/libavcodec/h264_picture.c
> index ff30166b4d..5944798394 100644
> --- a/libavcodec/h264_picture.c
> +++ b/libavcodec/h264_picture.c
> @@ -43,13 +43,14 @@
>
> void ff_h264_unref_picture(H264Context *h, H264Picture *pic)
> {
> -int off = offsetof(H264Picture, tf) + sizeof(pic->tf);
> +int off = offsetof(H264Picture, tf_grain) + sizeof(pic->tf_grain);
> int i;
>
> if (!pic->f || !pic->f->buf[0])
> return;
>
> ff_thread_release_buffer(h->avctx, &pic->tf);
> +ff_thread_release_buffer(h->avctx, &pic->tf_grain);
> av_buffer_unref(&pic->hwaccel_priv_buf);
>
> av_buffer_unref(&pic->qscale_table_buf);
> @@ -93,6 +94,7 @@ static void h264_copy_picture_params(H264Picture *dst,
> const H264Picture *src)
> dst->mb_width = src->mb_width;
> dst->mb_height = src->mb_height;
> dst->mb_stride = src->mb_stride;
> +dst->needs_fg = src->needs_fg;
> }
>
> int ff_h264_ref_picture(H264Context *h, H264Picture *dst, H264Picture *src)
> @@ -108,6 +110,14 @@ int ff_h264_ref_picture(H264Context *h, H264Picture
> *dst, H264Picture *src)
> if (ret < 0)
> goto fail;
>
> +if (src->needs_fg) {
> +av_assert0(src->tf_grain.f == src->f_grain);
> +dst->tf_grain.f = dst->f_grain;
> +ret = ff_thread_ref_frame(&dst->tf_grain, &src->tf_grain);
> +if (ret < 0)
> +goto fail;
> +}
> +
> dst->qscale_table_buf = av_buffer_ref(src->qscale_table_buf);
> dst->mb_type_buf = av_buffer_ref(src->mb_type_buf);
> dst->pps_buf = av_buffer_ref(src->pps_buf);
> @@ -159,6 +169,15 @@ int ff_h264_replace_picture(H264Context *h, H264Picture
> *dst, const H264Picture
> if (ret < 0)
> goto fail;
>
> +if (src->needs_fg) {
> +av_assert0(src->tf_grain.f == src->f_grain);
> +dst->tf_grain.f = dst->f_grain;
> +ff_thread_release_buffer(h->avctx, &dst->tf_grain);
> +ret = ff_thread_ref_frame(&dst->tf_grain, &src->tf_grain);
> +if (ret < 0)
> +goto fail;
> +}
> +
> ret = av_buffer_replace(&dst->qscale_table_buf, src->qscale_table_buf);
> ret |= av_buffer_replace(&dst->mb_type_buf, src->mb_type_buf);
> ret |= av_buffer_replace(&dst->pps_buf, src->pps_buf);
> @@ -212,6 +231,7 @@ void ff_h264_set_erpic(ERPicture *dst, H264Picture *src)
> int ff_h264_field_end(H264Context *h, H264SliceContext *sl, int in_setup)
> {
> AVCodecContext *const avctx = h->avctx;
> +H264Picture *cur = h->cur_pic_ptr;
> int err = 0;
> h->mb_y = 0;
>
> @@ -230,10 +250,21 @@ int ff_h264_field_end(H264Context *h, H264SliceContext
> *sl, int in_setup)
> if (err < 0)
> av_log(avctx, AV_LOG_ERROR,
> "hardware accelerator failed to decode picture\n");
> +} else if (!in_setup && cur->needs_fg) {
> +AVFrameSideData *sd = av_frame_get_side_data(cur->f,
> AV_FRAME_DATA_FILM_GRAIN_PARAMS);
> +av_assert0(sd); // always present if `cur->needs_fg`
> +err = ff_h274_apply_film_grain(cur->f_grain, cur->f, &h->h274db,
> + (AVFilmGrainParams *) sd->data);
> +if (err < 0) {
> +av_log(h->avctx, AV_LOG_WARNING, "Failed synthesizing film "
> + "grain, ignoring: %s\n", av_err2str(err));
> +cur->needs_fg = 0;
> +
[FFmpeg-devel] Sending patch to FFMPEG
If I'm missing something, please let me know. 0001-Include-missing-documentation-for-yasm-in-INSTALL.md.patch Description: Binary data ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] Sending patch to FFMPEG
5 Aug 2021, 21:05 by janpoonthong...@gmail.com: > If I'm missing something, please let me know. > We've recommended nasm instead of yasm for years now. You don't need to have nasm installed to compile, you can disable assembly optimizations. configure even tells you how, and forces you to in case neither is found. I think it's fine as-is. Besides, you don't even need either on anything that isn't x86. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH] fate: Fix mp4 ttml tests on Windows
Override the fate-sub-* wide comparison function of CMP=rawdiff back to the default, CMP=diff, which ignores line ending differences (where the ffprobe output contains CRLF line endings). Signed-off-by: Martin Storsjö --- tests/fate/subtitles.mak | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/fate/subtitles.mak b/tests/fate/subtitles.mak index 880109f201..0c26f582e0 100644 --- a/tests/fate/subtitles.mak +++ b/tests/fate/subtitles.mak @@ -110,7 +110,9 @@ FATE_SUBTITLES-$(call ALLYES, FILE_PROTOCOL PIPE_PROTOCOL SRT_DEMUXER SUBRIP_DEC fate-sub-ttmlenc: CMD = fmtstdout ttml -i $(TARGET_SAMPLES)/sub/SubRip_capability_tester.srt FATE_SUBTITLES-$(call ALLYES, FILE_PROTOCOL SRT_DEMUXER MOV_DEMUXER SUBRIP_DECODER TTML_ENCODER TTML_MUXER MOV_MUXER) += fate-sub-ttml-mp4-stpp fate-sub-ttml-mp4-dfxp +fate-sub-ttml-mp4-stpp: CMP = diff fate-sub-ttml-mp4-stpp: CMD = transcode srt $(TARGET_SAMPLES)/sub/SubRip_capability_tester.srt mp4 "-map 0:s -c:s ttml -time_base:s 1:1000" "-map 0 -c copy" "" "-of json -show_entries packet:stream=index,codec_type,codec_tag_string,codec_tag,codec_name,time_base,start_time,duration_ts,duration,nb_frames,nb_read_packets:stream_tags" +fate-sub-ttml-mp4-dfxp: CMP = diff fate-sub-ttml-mp4-dfxp: CMD = transcode srt $(TARGET_SAMPLES)/sub/SubRip_capability_tester.srt mp4 "-map 0:s -c:s ttml -time_base:s 1:1000 -tag:s dfxp -strict unofficial" "-map 0 -c copy" "" "-of json -show_entries packet:stream=index,codec_type,codec_tag_string,codec_tag,codec_name,time_base,start_time,duration_ts,duration,nb_frames,nb_read_packets:stream_tags" FATE_SUBTITLES-$(call ENCMUX, ASS, ASS) += $(FATE_SUBTITLES_ASS-yes) -- 2.25.1 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH] h274: remove optimization pragma
This results in warnings on compilers which don't support it,
objections were raised during the review process about it but went unnoticed,
and the speed benefit is highly compiler and version specific, and
also not very critical.
We generally hand-write assembly to optimize loops like that, rather
than use compiler magic, and for 40% best case scenario, it's simply
not worth it.
Plus, tree vectorization is still problematic with GCC and disabled by default
for a good reason, so enabling it locally is sketchy.
Patch attached.
>From 6480a3c4079f9993139ae167019d95f9e9b22ea8 Mon Sep 17 00:00:00 2001
From: Lynne
Date: Wed, 25 Aug 2021 21:20:18 +0200
Subject: [PATCH] h274: remove optimization pragma
This results in warnings on compilers which don't support it,
objections were raised during the review process about it but went unnoticed,
and the speed benefit is highly compiler and version specific, and
also not very critical.
We generally hand-write assembly to optimize loops like that, rather
than use compiler magic, and for 40% best case scenario, it's simply
not worth it.
Plus, tree vectorization is still problematic with GCC and disabled by default
for a good reason, so enabling it locally is sketchy.
---
libavcodec/h274.c | 4
1 file changed, 4 deletions(-)
diff --git a/libavcodec/h274.c b/libavcodec/h274.c
index 0efc00ca1d..5e2cf150ea 100644
--- a/libavcodec/h274.c
+++ b/libavcodec/h274.c
@@ -30,10 +30,6 @@
#include "h274.h"
-// The code in this file has a lot of loops that vectorize very well, this is
-// about a 40% speedup for no obvious downside.
-#pragma GCC optimize("tree-vectorize")
-
static const int8_t Gaussian_LUT[2048+256];
static const uint32_t Seed_LUT[256];
static const int8_t R64T[64][64];
--
2.33.0.252.g9b09ab0cd71
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH] tests/fate: move TTML-in-MP4 tests from subtitles.mak to mov.mak
subtitles.mak utilizes a more strict comparator and - to be honest -
the tests feel more at home in this location.
---
tests/fate/mov.mak | 4
tests/fate/subtitles.mak| 4
tests/ref/fate/{sub-ttml-mp4-dfxp => mov-mp4-ttml-dfxp} | 4 ++--
tests/ref/fate/{sub-ttml-mp4-stpp => mov-mp4-ttml-stpp} | 4 ++--
4 files changed, 8 insertions(+), 8 deletions(-)
rename tests/ref/fate/{sub-ttml-mp4-dfxp => mov-mp4-ttml-dfxp} (90%)
rename tests/ref/fate/{sub-ttml-mp4-stpp => mov-mp4-ttml-stpp} (90%)
diff --git a/tests/fate/mov.mak b/tests/fate/mov.mak
index 4c1014be92..0c9177aa81 100644
--- a/tests/fate/mov.mak
+++ b/tests/fate/mov.mak
@@ -132,6 +132,10 @@ FATE_MOV_FFMPEG_FFPROBE-$(call ALLYES, FILE_PROTOCOL
MOV_DEMUXER MJPEG_DECODER \
+= fate-mov-cover-image
fate-mov-cover-image: CMD = transcode mov
$(TARGET_SAMPLES)/cover_art/Owner-iTunes_9.0.3.15.m4a mp4 "-map 0 -map 0:v -c:a
copy -c:v:0 copy -filter:v:1 scale -c:v:1 png" "-map 0 -t 0.1 -c copy" ""
"-show_entries stream_disposition=attached_pic:stream=index,codec_name"
+FATE_MOV_FFMPEG_FFPROBE-$(call ALLYES, FILE_PROTOCOL SRT_DEMUXER MOV_DEMUXER
SUBRIP_DECODER TTML_ENCODER TTML_MUXER MOV_MUXER) += fate-mov-mp4-ttml-stpp
fate-mov-mp4-ttml-dfxp
+fate-mov-mp4-ttml-stpp: CMD = transcode srt
$(TARGET_SAMPLES)/sub/SubRip_capability_tester.srt mp4 "-map 0:s -c:s ttml
-time_base:s 1:1000" "-map 0 -c copy" "" "-of json -show_entries
packet:stream=index,codec_type,codec_tag_string,codec_tag,codec_name,time_base,start_time,duration_ts,duration,nb_frames,nb_read_packets:stream_tags"
+fate-mov-mp4-ttml-dfxp: CMD = transcode srt
$(TARGET_SAMPLES)/sub/SubRip_capability_tester.srt mp4 "-map 0:s -c:s ttml
-time_base:s 1:1000 -tag:s dfxp -strict unofficial" "-map 0 -c copy" "" "-of
json -show_entries
packet:stream=index,codec_type,codec_tag_string,codec_tag,codec_name,time_base,start_time,duration_ts,duration,nb_frames,nb_read_packets:stream_tags"
+
FATE_SAMPLES_FFMPEG_FFPROBE += $(FATE_MOV_FFMPEG_FFPROBE-yes)
fate-mov: $(FATE_MOV) $(FATE_MOV_FFPROBE) $(FATE_MOV_FASTSTART)
$(FATE_MOV_FFMPEG_FFPROBE-yes)
diff --git a/tests/fate/subtitles.mak b/tests/fate/subtitles.mak
index 880109f201..ee65afe35b 100644
--- a/tests/fate/subtitles.mak
+++ b/tests/fate/subtitles.mak
@@ -109,10 +109,6 @@ fate-sub-dvb: CMD = framecrc -i
$(TARGET_SAMPLES)/sub/dvbsubtest_filter.ts -map
FATE_SUBTITLES-$(call ALLYES, FILE_PROTOCOL PIPE_PROTOCOL SRT_DEMUXER
SUBRIP_DECODER TTML_ENCODER TTML_MUXER) += fate-sub-ttmlenc
fate-sub-ttmlenc: CMD = fmtstdout ttml -i
$(TARGET_SAMPLES)/sub/SubRip_capability_tester.srt
-FATE_SUBTITLES-$(call ALLYES, FILE_PROTOCOL SRT_DEMUXER MOV_DEMUXER
SUBRIP_DECODER TTML_ENCODER TTML_MUXER MOV_MUXER) += fate-sub-ttml-mp4-stpp
fate-sub-ttml-mp4-dfxp
-fate-sub-ttml-mp4-stpp: CMD = transcode srt
$(TARGET_SAMPLES)/sub/SubRip_capability_tester.srt mp4 "-map 0:s -c:s ttml
-time_base:s 1:1000" "-map 0 -c copy" "" "-of json -show_entries
packet:stream=index,codec_type,codec_tag_string,codec_tag,codec_name,time_base,start_time,duration_ts,duration,nb_frames,nb_read_packets:stream_tags"
-fate-sub-ttml-mp4-dfxp: CMD = transcode srt
$(TARGET_SAMPLES)/sub/SubRip_capability_tester.srt mp4 "-map 0:s -c:s ttml
-time_base:s 1:1000 -tag:s dfxp -strict unofficial" "-map 0 -c copy" "" "-of
json -show_entries
packet:stream=index,codec_type,codec_tag_string,codec_tag,codec_name,time_base,start_time,duration_ts,duration,nb_frames,nb_read_packets:stream_tags"
-
FATE_SUBTITLES-$(call ENCMUX, ASS, ASS) += $(FATE_SUBTITLES_ASS-yes)
FATE_SUBTITLES += $(FATE_SUBTITLES-yes)
diff --git a/tests/ref/fate/sub-ttml-mp4-dfxp b/tests/ref/fate/mov-mp4-ttml-dfxp
similarity index 90%
rename from tests/ref/fate/sub-ttml-mp4-dfxp
rename to tests/ref/fate/mov-mp4-ttml-dfxp
index 0172e5b7e6..e24b5d618b 100644
--- a/tests/ref/fate/sub-ttml-mp4-dfxp
+++ b/tests/ref/fate/mov-mp4-ttml-dfxp
@@ -1,5 +1,5 @@
-2e7e01c821c111466e7a2844826b7f6d *tests/data/fate/sub-ttml-mp4-dfxp.mp4
-8519 tests/data/fate/sub-ttml-mp4-dfxp.mp4
+2e7e01c821c111466e7a2844826b7f6d *tests/data/fate/mov-mp4-ttml-dfxp.mp4
+8519 tests/data/fate/mov-mp4-ttml-dfxp.mp4
#tb 0: 1/1000
#media_type 0: data
#codec_id 0: none
diff --git a/tests/ref/fate/sub-ttml-mp4-stpp b/tests/ref/fate/mov-mp4-ttml-stpp
similarity index 90%
rename from tests/ref/fate/sub-ttml-mp4-stpp
rename to tests/ref/fate/mov-mp4-ttml-stpp
index a5165b568d..77bd23b7bf 100644
--- a/tests/ref/fate/sub-ttml-mp4-stpp
+++ b/tests/ref/fate/mov-mp4-ttml-stpp
@@ -1,5 +1,5 @@
-cbd2c7ff864a663b0d893deac5a0caec *tests/data/fate/sub-ttml-mp4-stpp.mp4
-8547 tests/data/fate/sub-ttml-mp4-stpp.mp4
+cbd2c7ff864a663b0d893deac5a0caec *tests/data/fate/mov-mp4-ttml-stpp.mp4
+8547 tests/data/fate/mov-mp4-ttml-stpp.mp4
#tb 0: 1/1000
#media_type 0: data
#codec_id 0: none
--
2.31.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffm
Re: [FFmpeg-devel] [PATCH] tests/fate: move TTML-in-MP4 tests from subtitles.mak to mov.mak
On Wed, 25 Aug 2021, Jan Ekström wrote: subtitles.mak utilizes a more strict comparator and - to be honest - the tests feel more at home in this location. --- LGTM, although your commit message forgets to point out the relevant effect of it - we're not just doing this for sorting things in the most matching bucket, but because those tests were broken on Windows. Please mention that and mention that it was due to fate-sub-% getting CMP=rawdiff. // Martin ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] tests/fate: move TTML-in-MP4 tests from subtitles.mak to mov.mak
On Wed, Aug 25, 2021 at 10:58 PM Martin Storsjö wrote: > > On Wed, 25 Aug 2021, Jan Ekström wrote: > > > subtitles.mak utilizes a more strict comparator and - to be honest - > > the tests feel more at home in this location. > > --- > > LGTM, although your commit message forgets to point out the relevant > effect of it - we're not just doing this for sorting things in the most > matching bucket, but because those tests were broken on Windows. Please > mention that and mention that it was due to fate-sub-% getting > CMP=rawdiff. > Thanks, applied with a better explanation part as d48232fb4cdb55b3add08cef89c222a60d0896a9 . Jan ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] arm: hevc_qpel: Fix the assembly to work with non-multiple of 8 widths
On Wed, 25 Aug 2021, Martin Storsjö wrote: This unbreaks the fate-checkasm-hevc_pel test on arm targets. The assembly assumed that the width passed to the DSP functions is a multiple of 8, while the checkasm test used other widths too. This wasn't noticed before, because the hevc_pel checkasm tests (that were added in 9c513edb7999a35ddcc6e3a8d984a96c8fb492a3 in January) weren't run as part of fate until in b492cacffd36ad4cb251ba1f13ac398318ee639a in August. As this hasn't been an issue in practice with actual full decoding tests, it seems like the actual decoder doesn't call these functions with such widths. Therefore, we could alternatively fix the test to only test things that the real decoder does, and this modification could be reverted. Signed-off-by: Martin Storsjö --- libavcodec/arm/hevcdsp_qpel_neon.S | 18 +- 1 file changed, 9 insertions(+), 9 deletions(-) Pushed this one to unbreak fate now, although I guess it'd be nice to align the checkasm test with what the decoder actually does. // Martin ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v3] avcodec/h264dec: apply H.274 film grain
On 8/25/2021 3:06 PM, Eoff, Ullysses A wrote:
-Original Message-
From: ffmpeg-devel On Behalf Of Niklas Haas
Sent: Tuesday, August 17, 2021 12:55 PM
To: ffmpeg-devel@ffmpeg.org
Cc: Niklas Haas
Subject: [FFmpeg-devel] [PATCH v3] avcodec/h264dec: apply H.274 film grain
From: Niklas Haas
Because we need access to ref frames without film grain applied, we have
to add an extra AVFrame to H264Picture to avoid messing with the
original. This requires some amount of overhead to make the reference
moves work out, but it allows us to benefit from frame multithreading
for film grain application "for free".
Unfortunately, this approach requires twice as much RAM to be constantly
allocated for ref frames, due to the need for an extra buffer per
H264Picture. In theory, we could get away with freeing up this memory as
soon as it's no longer needed (since ref frames do not need film grain
buffers any longer), but trying to call ff_thread_release_buffer() from
output_frame() conflicts with possible later accesses to that same frame
and I'm not sure how to synchronize that well.
Tested on all three cases of (no fg), (fg present but exported) and (fg
present and not exported), with and without threading.
Signed-off-by: Niklas Haas
---
libavcodec/h264_picture.c | 35 +++--
libavcodec/h264_slice.c | 16 ++--
libavcodec/h264dec.c | 55 ++-
libavcodec/h264dec.h | 6 +
4 files changed, 90 insertions(+), 22 deletions(-)
diff --git a/libavcodec/h264_picture.c b/libavcodec/h264_picture.c
index ff30166b4d..5944798394 100644
--- a/libavcodec/h264_picture.c
+++ b/libavcodec/h264_picture.c
@@ -43,13 +43,14 @@
void ff_h264_unref_picture(H264Context *h, H264Picture *pic)
{
-int off = offsetof(H264Picture, tf) + sizeof(pic->tf);
+int off = offsetof(H264Picture, tf_grain) + sizeof(pic->tf_grain);
int i;
if (!pic->f || !pic->f->buf[0])
return;
ff_thread_release_buffer(h->avctx, &pic->tf);
+ff_thread_release_buffer(h->avctx, &pic->tf_grain);
av_buffer_unref(&pic->hwaccel_priv_buf);
av_buffer_unref(&pic->qscale_table_buf);
@@ -93,6 +94,7 @@ static void h264_copy_picture_params(H264Picture *dst, const
H264Picture *src)
dst->mb_width = src->mb_width;
dst->mb_height = src->mb_height;
dst->mb_stride = src->mb_stride;
+dst->needs_fg = src->needs_fg;
}
int ff_h264_ref_picture(H264Context *h, H264Picture *dst, H264Picture *src)
@@ -108,6 +110,14 @@ int ff_h264_ref_picture(H264Context *h, H264Picture *dst,
H264Picture *src)
if (ret < 0)
goto fail;
+if (src->needs_fg) {
+av_assert0(src->tf_grain.f == src->f_grain);
+dst->tf_grain.f = dst->f_grain;
+ret = ff_thread_ref_frame(&dst->tf_grain, &src->tf_grain);
+if (ret < 0)
+goto fail;
+}
+
dst->qscale_table_buf = av_buffer_ref(src->qscale_table_buf);
dst->mb_type_buf = av_buffer_ref(src->mb_type_buf);
dst->pps_buf = av_buffer_ref(src->pps_buf);
@@ -159,6 +169,15 @@ int ff_h264_replace_picture(H264Context *h, H264Picture
*dst, const H264Picture
if (ret < 0)
goto fail;
+if (src->needs_fg) {
+av_assert0(src->tf_grain.f == src->f_grain);
+dst->tf_grain.f = dst->f_grain;
+ff_thread_release_buffer(h->avctx, &dst->tf_grain);
+ret = ff_thread_ref_frame(&dst->tf_grain, &src->tf_grain);
+if (ret < 0)
+goto fail;
+}
+
ret = av_buffer_replace(&dst->qscale_table_buf, src->qscale_table_buf);
ret |= av_buffer_replace(&dst->mb_type_buf, src->mb_type_buf);
ret |= av_buffer_replace(&dst->pps_buf, src->pps_buf);
@@ -212,6 +231,7 @@ void ff_h264_set_erpic(ERPicture *dst, H264Picture *src)
int ff_h264_field_end(H264Context *h, H264SliceContext *sl, int in_setup)
{
AVCodecContext *const avctx = h->avctx;
+H264Picture *cur = h->cur_pic_ptr;
int err = 0;
h->mb_y = 0;
@@ -230,10 +250,21 @@ int ff_h264_field_end(H264Context *h, H264SliceContext
*sl, int in_setup)
if (err < 0)
av_log(avctx, AV_LOG_ERROR,
"hardware accelerator failed to decode picture\n");
+} else if (!in_setup && cur->needs_fg) {
+AVFrameSideData *sd = av_frame_get_side_data(cur->f,
AV_FRAME_DATA_FILM_GRAIN_PARAMS);
+av_assert0(sd); // always present if `cur->needs_fg`
+err = ff_h274_apply_film_grain(cur->f_grain, cur->f, &h->h274db,
+ (AVFilmGrainParams *) sd->data);
+if (err < 0) {
+av_log(h->avctx, AV_LOG_WARNING, "Failed synthesizing film "
+ "grain, ignoring: %s\n", av_err2str(err));
+cur->needs_fg = 0;
+err = 0;
+}
}
if (!in_setup && !h->droppable)
-ff_thread_report_progress(&h->cur_pic_ptr->tf, INT_MAX,
+ff_thread_repor
[FFmpeg-devel] [PATCH v5 1/6] lavfi/dnn: Task-based Inference in Native Backend
This commit rearranges the code in Native Backend to use the TaskItem
for inference.
Signed-off-by: Shubhanshu Saxena
---
libavfilter/dnn/dnn_backend_native.c | 176 ++-
libavfilter/dnn/dnn_backend_native.h | 2 +
2 files changed, 121 insertions(+), 57 deletions(-)
diff --git a/libavfilter/dnn/dnn_backend_native.c
b/libavfilter/dnn/dnn_backend_native.c
index a6be27f1fd..3b2a3aa55d 100644
--- a/libavfilter/dnn/dnn_backend_native.c
+++ b/libavfilter/dnn/dnn_backend_native.c
@@ -45,9 +45,29 @@ static const AVClass dnn_native_class = {
.category = AV_CLASS_CATEGORY_FILTER,
};
-static DNNReturnType execute_model_native(const DNNModel *model, const char
*input_name, AVFrame *in_frame,
- const char **output_names, uint32_t
nb_output, AVFrame *out_frame,
- int do_ioproc);
+static DNNReturnType execute_model_native(Queue *inference_queue);
+
+static DNNReturnType extract_inference_from_task(TaskItem *task, Queue
*inference_queue)
+{
+NativeModel *native_model = task->model;
+NativeContext *ctx = &native_model->ctx;
+InferenceItem *inference = av_malloc(sizeof(*inference));
+
+if (!inference) {
+av_log(ctx, AV_LOG_ERROR, "Unable to allocate space for
InferenceItem\n");
+return DNN_ERROR;
+}
+task->inference_todo = 1;
+task->inference_done = 0;
+inference->task = task;
+
+if (ff_queue_push_back(inference_queue, inference) < 0) {
+av_log(ctx, AV_LOG_ERROR, "Failed to push back inference_queue.\n");
+av_freep(&inference);
+return DNN_ERROR;
+}
+return DNN_SUCCESS;
+}
static DNNReturnType get_input_native(void *model, DNNData *input, const char
*input_name)
{
@@ -78,34 +98,36 @@ static DNNReturnType get_input_native(void *model, DNNData
*input, const char *i
static DNNReturnType get_output_native(void *model, const char *input_name,
int input_width, int input_height,
const char *output_name, int
*output_width, int *output_height)
{
-DNNReturnType ret;
+DNNReturnType ret = 0;
NativeModel *native_model = model;
NativeContext *ctx = &native_model->ctx;
-AVFrame *in_frame = av_frame_alloc();
-AVFrame *out_frame = NULL;
-
-if (!in_frame) {
-av_log(ctx, AV_LOG_ERROR, "Could not allocate memory for input
frame\n");
-return DNN_ERROR;
+TaskItem task;
+DNNExecBaseParams exec_params = {
+.input_name = input_name,
+.output_names = &output_name,
+.nb_output = 1,
+.in_frame = NULL,
+.out_frame = NULL,
+};
+
+if (ff_dnn_fill_gettingoutput_task(&task, &exec_params, native_model,
input_height, input_width, ctx) != DNN_SUCCESS) {
+ret = DNN_ERROR;
+goto err;
}
-out_frame = av_frame_alloc();
-
-if (!out_frame) {
-av_log(ctx, AV_LOG_ERROR, "Could not allocate memory for output
frame\n");
-av_frame_free(&in_frame);
-return DNN_ERROR;
+if (extract_inference_from_task(&task, native_model->inference_queue) !=
DNN_SUCCESS) {
+av_log(ctx, AV_LOG_ERROR, "unable to extract inference from task.\n");
+ret = DNN_ERROR;
+goto err;
}
-in_frame->width = input_width;
-in_frame->height = input_height;
-
-ret = execute_model_native(native_model->model, input_name, in_frame,
&output_name, 1, out_frame, 0);
-*output_width = out_frame->width;
-*output_height = out_frame->height;
+ret = execute_model_native(native_model->inference_queue);
+*output_width = task.out_frame->width;
+*output_height = task.out_frame->height;
-av_frame_free(&out_frame);
-av_frame_free(&in_frame);
+err:
+av_frame_free(&task.out_frame);
+av_frame_free(&task.in_frame);
return ret;
}
@@ -190,6 +212,11 @@ DNNModel *ff_dnn_load_model_native(const char
*model_filename, DNNFunctionType f
goto fail;
}
+native_model->inference_queue = ff_queue_create();
+if (!native_model->inference_queue) {
+goto fail;
+}
+
for (layer = 0; layer < native_model->layers_num; ++layer){
layer_type = (int32_t)avio_rl32(model_file_context);
dnn_size += 4;
@@ -259,50 +286,66 @@ fail:
return NULL;
}
-static DNNReturnType execute_model_native(const DNNModel *model, const char
*input_name, AVFrame *in_frame,
- const char **output_names, uint32_t
nb_output, AVFrame *out_frame,
- int do_ioproc)
+static DNNReturnType execute_model_native(Queue *inference_queue)
{
-NativeModel *native_model = model->model;
-NativeContext *ctx = &native_model->ctx;
+NativeModel *native_model = NULL;
+NativeContext *ctx = NULL;
int32_t layer;
DNNData input, output;
DnnOperand *oprd = NULL;
+InferenceItem *inference = NULL;
+
[FFmpeg-devel] [PATCH v5 2/6] libavfilter: Unify Execution Modes in DNN Filters
This commit unifies the async and sync mode from the DNN filters'
perspective. As of this commit, the Native backend only supports
synchronous execution mode.
Now the user can switch between async and sync mode by using the
'async' option in the backend_configs. The values can be 1 for
async and 0 for sync mode of execution.
This commit affects the following filters:
1. vf_dnn_classify
2. vf_dnn_detect
3. vf_dnn_processing
4. vf_sr
5. vf_derain
This commit also updates the filters vf_dnn_detect and vf_dnn_classify
to send only the input frame and send NULL as output frame instead of
input frame to the DNN backends.
Signed-off-by: Shubhanshu Saxena
---
libavfilter/dnn/dnn_backend_common.c | 4 +-
libavfilter/dnn/dnn_backend_common.h | 5 +-
libavfilter/dnn/dnn_backend_native.c | 59 ++-
libavfilter/dnn/dnn_backend_native.h | 6 ++
libavfilter/dnn/dnn_backend_openvino.c | 99 ++
libavfilter/dnn/dnn_backend_openvino.h | 3 +-
libavfilter/dnn/dnn_backend_tf.c | 37 ++
libavfilter/dnn/dnn_backend_tf.h | 3 +-
libavfilter/dnn/dnn_interface.c| 8 +--
libavfilter/dnn_filter_common.c| 23 +-
libavfilter/dnn_filter_common.h| 3 +-
libavfilter/dnn_interface.h| 4 +-
libavfilter/vf_derain.c| 7 ++
libavfilter/vf_dnn_classify.c | 18 +++--
libavfilter/vf_dnn_detect.c| 22 +++---
libavfilter/vf_dnn_processing.c| 10 +--
libavfilter/vf_sr.c| 8 +++
17 files changed, 158 insertions(+), 161 deletions(-)
diff --git a/libavfilter/dnn/dnn_backend_common.c
b/libavfilter/dnn/dnn_backend_common.c
index 426683b73d..6a9c4cc87f 100644
--- a/libavfilter/dnn/dnn_backend_common.c
+++ b/libavfilter/dnn/dnn_backend_common.c
@@ -38,7 +38,7 @@ int ff_check_exec_params(void *ctx, DNNBackendType backend,
DNNFunctionType func
return AVERROR(EINVAL);
}
-if (!exec_params->out_frame) {
+if (!exec_params->out_frame && func_type == DFT_PROCESS_FRAME) {
av_log(ctx, AV_LOG_ERROR, "out frame is NULL when execute model.\n");
return AVERROR(EINVAL);
}
@@ -138,7 +138,7 @@ DNNReturnType ff_dnn_start_inference_async(void *ctx,
DNNAsyncExecModule *async_
return DNN_SUCCESS;
}
-DNNAsyncStatusType ff_dnn_get_async_result_common(Queue *task_queue, AVFrame
**in, AVFrame **out)
+DNNAsyncStatusType ff_dnn_get_result_common(Queue *task_queue, AVFrame **in,
AVFrame **out)
{
TaskItem *task = ff_queue_peek_front(task_queue);
diff --git a/libavfilter/dnn/dnn_backend_common.h
b/libavfilter/dnn/dnn_backend_common.h
index 604c1d3bd7..78e62a94a2 100644
--- a/libavfilter/dnn/dnn_backend_common.h
+++ b/libavfilter/dnn/dnn_backend_common.h
@@ -29,7 +29,8 @@
#include "libavutil/thread.h"
#define DNN_BACKEND_COMMON_OPTIONS \
-{ "nireq", "number of request",
OFFSET(options.nireq), AV_OPT_TYPE_INT,{ .i64 = 0 }, 0,
INT_MAX, FLAGS },
+{ "nireq", "number of request",
OFFSET(options.nireq), AV_OPT_TYPE_INT,{ .i64 = 0 }, 0,
INT_MAX, FLAGS }, \
+{ "async", "use DNN async inference",
OFFSET(options.async), AV_OPT_TYPE_BOOL, { .i64 = 1 }, 0,
1, FLAGS },
// one task for one function call from dnn interface
typedef struct TaskItem {
@@ -135,7 +136,7 @@ DNNReturnType ff_dnn_start_inference_async(void *ctx,
DNNAsyncExecModule *async_
* @retval DAST_NOT_READY if inference not completed yet.
* @retval DAST_SUCCESS if result successfully extracted
*/
-DNNAsyncStatusType ff_dnn_get_async_result_common(Queue *task_queue, AVFrame
**in, AVFrame **out);
+DNNAsyncStatusType ff_dnn_get_result_common(Queue *task_queue, AVFrame **in,
AVFrame **out);
/**
* Allocate input and output frames and fill the Task
diff --git a/libavfilter/dnn/dnn_backend_native.c
b/libavfilter/dnn/dnn_backend_native.c
index 3b2a3aa55d..2d34b88f8a 100644
--- a/libavfilter/dnn/dnn_backend_native.c
+++ b/libavfilter/dnn/dnn_backend_native.c
@@ -34,6 +34,7 @@
#define FLAGS AV_OPT_FLAG_FILTERING_PARAM
static const AVOption dnn_native_options[] = {
{ "conv2d_threads", "threads num for conv2d layer",
OFFSET(options.conv2d_threads), AV_OPT_TYPE_INT, { .i64 = 0 }, INT_MIN,
INT_MAX, FLAGS },
+{ "async", "use DNN async inference", OFFSET(options.async),
AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, FLAGS },
{ NULL },
};
@@ -189,6 +190,11 @@ DNNModel *ff_dnn_load_model_native(const char
*model_filename, DNNFunctionType f
goto fail;
native_model->model = model;
+if (native_model->ctx.options.async) {
+av_log(&native_model->ctx, AV_LOG_WARNING, "Async not supported.
Rolling back to sync\n");
+native_model->ctx.options.async = 0;
+}
+
#if !HAVE_PTHREAD_CANCEL
if (native_model->ctx.options.conv2d_threads > 1){
av_log(&native_model
Re: [FFmpeg-devel] [PATCH] avcodec/av1dec: Adjust incorrect error output
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Wang, Fei W
> Sent: Tuesday, 24 August 2021 07:32
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH] avcodec/av1dec: Adjust incorrect
> error output
>
> On Tue, 2021-08-24 at 02:45 +, Soft Works wrote:
> > > -Original Message-
> > > From: ffmpeg-devel On Behalf Of
> > > Ronald S. Bultje
> > > Sent: Tuesday, 24 August 2021 04:34
> > > To: FFmpeg development discussions and patches > > de...@ffmpeg.org>
> > > Subject: Re: [FFmpeg-devel] [PATCH] avcodec/av1dec: Adjust
> > > incorrect
> > > error output
> > >
> > > Hi,
> > >
> > > On Mon, Aug 23, 2021 at 9:39 PM Soft Works
>
> > > wrote:
> > >
> > > > The current message "Your platform doesn't suppport
> > > > hardware accelerated AV1 decoding." is inaccurate and
> > > > misleading. When a user doesn't specify a hwcaccel,
> > > > this doesn't tell anything about what is actually
> > > > supported on that platform.
> > > >
> > > > Signed-off-by: softworkz
> > > > ---
> > > > libavcodec/av1dec.c | 4 ++--
> > > > 1 file changed, 2 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/libavcodec/av1dec.c b/libavcodec/av1dec.c
> > > > index a75d6744d3..8dfcd26cb6 100644
> > > > --- a/libavcodec/av1dec.c
> > > > +++ b/libavcodec/av1dec.c
> > > > @@ -462,8 +462,8 @@ static int get_pixel_format(AVCodecContext
> > >
> > > *avctx)
> > > > * implemented in the future, need remove this check.
> > > > */
> > > > if (!avctx->hwaccel) {
> > > > -av_log(avctx, AV_LOG_ERROR, "Your platform doesn't
> > >
> > > suppport"
> > > > - " hardware accelerated AV1 decoding.\n");
> > > > +av_log(avctx, AV_LOG_ERROR, "AV1 decoding requires a
> hw
> > > > acceleration"
> > > > + " to be specified.\n");
> > > >
> > >
> > > This is misleading.
> > >
> > > This error message (for me) happens when I accidentally compile
> > > FFmpeg
> > > without aom/dav1d support, and try to play AV1 content.
> >
> > It also happens to me when I specify:
> >
> > ffmpeg -c:v av1 -i INPUT OUTPUT
> >
> > (e.g. like when I'd have forgotten to specify the hwaccel)
> >
> > > AV1 decoding
> > > does
> > > not require hw decoding - it simply requires an external library.
> > >
> > > Will the hwaccel be autodetected when AV1 hw decoding is
> supported
> > > by
> > > the
> > > platform?
> >
> > No.
> >
> > How about this: "The AV1 decoder requires a hw acceleration to be
> > specified." ?
>
> This will bring new misleading if user specify the hw acceleration
> but
> the hardware itself doesn't support av1 dec.
Such situations may happen, but this is not subject of this patch.
This patch is about the error that is printed when no hw acceleration
has been specified.
If the specified hw accel is not available or cannot be initialized,
then there will be an error reported at an earlier stage.
If the specified hw accel has been initialized successfully but the
hwaccel doesn't support AV1, this check won't catch it anyway.
Kind regards,
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v5 3/6] libavfilter: Remove synchronous functions from DNN filters
This commit removes the unused sync mode specific code from the DNN
filters since the sync and async mode are now unified from the
filters' perspective.
Signed-off-by: Shubhanshu Saxena
---
libavfilter/vf_dnn_detect.c | 71 +---
libavfilter/vf_dnn_processing.c | 84 +
2 files changed, 4 insertions(+), 151 deletions(-)
diff --git a/libavfilter/vf_dnn_detect.c b/libavfilter/vf_dnn_detect.c
index 675113d676..637874b2a1 100644
--- a/libavfilter/vf_dnn_detect.c
+++ b/libavfilter/vf_dnn_detect.c
@@ -353,63 +353,6 @@ static int dnn_detect_query_formats(AVFilterContext
*context)
return ff_set_common_formats_from_list(context, pix_fmts);
}
-static int dnn_detect_filter_frame(AVFilterLink *inlink, AVFrame *in)
-{
-AVFilterContext *context = inlink->dst;
-AVFilterLink *outlink = context->outputs[0];
-DnnDetectContext *ctx = context->priv;
-DNNReturnType dnn_result;
-
-dnn_result = ff_dnn_execute_model(&ctx->dnnctx, in, in);
-if (dnn_result != DNN_SUCCESS){
-av_log(ctx, AV_LOG_ERROR, "failed to execute model\n");
-av_frame_free(&in);
-return AVERROR(EIO);
-}
-
-return ff_filter_frame(outlink, in);
-}
-
-static int dnn_detect_activate_sync(AVFilterContext *filter_ctx)
-{
-AVFilterLink *inlink = filter_ctx->inputs[0];
-AVFilterLink *outlink = filter_ctx->outputs[0];
-AVFrame *in = NULL;
-int64_t pts;
-int ret, status;
-int got_frame = 0;
-
-FF_FILTER_FORWARD_STATUS_BACK(outlink, inlink);
-
-do {
-// drain all input frames
-ret = ff_inlink_consume_frame(inlink, &in);
-if (ret < 0)
-return ret;
-if (ret > 0) {
-ret = dnn_detect_filter_frame(inlink, in);
-if (ret < 0)
-return ret;
-got_frame = 1;
-}
-} while (ret > 0);
-
-// if frame got, schedule to next filter
-if (got_frame)
-return 0;
-
-if (ff_inlink_acknowledge_status(inlink, &status, &pts)) {
-if (status == AVERROR_EOF) {
-ff_outlink_set_status(outlink, status, pts);
-return ret;
-}
-}
-
-FF_FILTER_FORWARD_WANTED(outlink, inlink);
-
-return FFERROR_NOT_READY;
-}
-
static int dnn_detect_flush_frame(AVFilterLink *outlink, int64_t pts, int64_t
*out_pts)
{
DnnDetectContext *ctx = outlink->src->priv;
@@ -438,7 +381,7 @@ static int dnn_detect_flush_frame(AVFilterLink *outlink,
int64_t pts, int64_t *o
return 0;
}
-static int dnn_detect_activate_async(AVFilterContext *filter_ctx)
+static int dnn_detect_activate(AVFilterContext *filter_ctx)
{
AVFilterLink *inlink = filter_ctx->inputs[0];
AVFilterLink *outlink = filter_ctx->outputs[0];
@@ -494,16 +437,6 @@ static int dnn_detect_activate_async(AVFilterContext
*filter_ctx)
return 0;
}
-static av_unused int dnn_detect_activate(AVFilterContext *filter_ctx)
-{
-DnnDetectContext *ctx = filter_ctx->priv;
-
-if (ctx->dnnctx.async)
-return dnn_detect_activate_async(filter_ctx);
-else
-return dnn_detect_activate_sync(filter_ctx);
-}
-
static av_cold void dnn_detect_uninit(AVFilterContext *context)
{
DnnDetectContext *ctx = context->priv;
@@ -535,5 +468,5 @@ const AVFilter ff_vf_dnn_detect = {
FILTER_INPUTS(dnn_detect_inputs),
FILTER_OUTPUTS(dnn_detect_outputs),
.priv_class= &dnn_detect_class,
-.activate = dnn_detect_activate_async,
+.activate = dnn_detect_activate,
};
diff --git a/libavfilter/vf_dnn_processing.c b/libavfilter/vf_dnn_processing.c
index 7435dd4959..55634efde5 100644
--- a/libavfilter/vf_dnn_processing.c
+++ b/libavfilter/vf_dnn_processing.c
@@ -244,76 +244,6 @@ static int copy_uv_planes(DnnProcessingContext *ctx,
AVFrame *out, const AVFrame
return 0;
}
-static int filter_frame(AVFilterLink *inlink, AVFrame *in)
-{
-AVFilterContext *context = inlink->dst;
-AVFilterLink *outlink = context->outputs[0];
-DnnProcessingContext *ctx = context->priv;
-DNNReturnType dnn_result;
-AVFrame *out;
-
-out = ff_get_video_buffer(outlink, outlink->w, outlink->h);
-if (!out) {
-av_frame_free(&in);
-return AVERROR(ENOMEM);
-}
-av_frame_copy_props(out, in);
-
-dnn_result = ff_dnn_execute_model(&ctx->dnnctx, in, out);
-if (dnn_result != DNN_SUCCESS){
-av_log(ctx, AV_LOG_ERROR, "failed to execute model\n");
-av_frame_free(&in);
-av_frame_free(&out);
-return AVERROR(EIO);
-}
-
-if (isPlanarYUV(in->format))
-copy_uv_planes(ctx, out, in);
-
-av_frame_free(&in);
-return ff_filter_frame(outlink, out);
-}
-
-static int activate_sync(AVFilterContext *filter_ctx)
-{
-AVFilterLink *inlink = filter_ctx->inputs[0];
-AVFilterLink *outlink = filter_ctx->outputs[0];
-AVFrame *in = NULL;
-int64_t pts;
-int ret, status;
-int got_frame = 0;
-
-FF_FILTER_FORWARD_STAT
[FFmpeg-devel] [PATCH v5 4/6] libavfilter: Remove Async Flag from DNN Filter Side
Remove async flag from filter's perspective after the unification
of async and sync modes in the DNN backend.
Signed-off-by: Shubhanshu Saxena
---
doc/filters.texi | 14 --
libavfilter/dnn/dnn_backend_tf.c | 7 +++
libavfilter/dnn_filter_common.c | 7 ---
libavfilter/dnn_filter_common.h | 2 +-
4 files changed, 12 insertions(+), 18 deletions(-)
diff --git a/doc/filters.texi b/doc/filters.texi
index 47ceeb18c6..7aff920311 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -10283,11 +10283,8 @@ and the second line is the name of label id 1, etc.
The label id is considered as name if the label file is not provided.
@item backend_configs
-Set the configs to be passed into backend
-
-@item async
-use DNN async execution if set (default: set),
-roll back to sync execution if the backend does not support async.
+Set the configs to be passed into backend. To use async execution, set async
(default: set).
+Roll back to sync execution if the backend does not support async.
@end table
@@ -10339,15 +10336,12 @@ Set the input name of the dnn network.
Set the output name of the dnn network.
@item backend_configs
-Set the configs to be passed into backend
+Set the configs to be passed into backend. To use async execution, set async
(default: set).
+Roll back to sync execution if the backend does not support async.
For tensorflow backend, you can set its configs with @option{sess_config}
options,
please use tools/python/tf_sess_config.py to get the configs of TensorFlow
backend for your system.
-@item async
-use DNN async execution if set (default: set),
-roll back to sync execution if the backend does not support async.
-
@end table
@subsection Examples
diff --git a/libavfilter/dnn/dnn_backend_tf.c b/libavfilter/dnn/dnn_backend_tf.c
index 84952ece5a..6e41470da4 100644
--- a/libavfilter/dnn/dnn_backend_tf.c
+++ b/libavfilter/dnn/dnn_backend_tf.c
@@ -884,6 +884,13 @@ DNNModel *ff_dnn_load_model_tf(const char *model_filename,
DNNFunctionType func_
ctx->options.nireq = av_cpu_count() / 2 + 1;
}
+#if !HAVE_PTHREAD_CANCEL
+if (ctx->options.async) {
+ctx->options.async = 0;
+av_log(filter_ctx, AV_LOG_WARNING, "pthread is not supported, roll
back to sync.\n");
+}
+#endif
+
tf_model->request_queue = ff_safe_queue_create();
if (!tf_model->request_queue) {
goto err;
diff --git a/libavfilter/dnn_filter_common.c b/libavfilter/dnn_filter_common.c
index 455eaa37f4..3045ce0131 100644
--- a/libavfilter/dnn_filter_common.c
+++ b/libavfilter/dnn_filter_common.c
@@ -84,13 +84,6 @@ int ff_dnn_init(DnnContext *ctx, DNNFunctionType func_type,
AVFilterContext *fil
return AVERROR(EINVAL);
}
-#if !HAVE_PTHREAD_CANCEL
-if (ctx->async) {
-ctx->async = 0;
-av_log(filter_ctx, AV_LOG_WARNING, "pthread is not supported, roll
back to sync.\n");
-}
-#endif
-
return 0;
}
diff --git a/libavfilter/dnn_filter_common.h b/libavfilter/dnn_filter_common.h
index 4d92c1dc36..635ae631c1 100644
--- a/libavfilter/dnn_filter_common.h
+++ b/libavfilter/dnn_filter_common.h
@@ -46,7 +46,7 @@ typedef struct DnnContext {
{ "output", "output name of the model",
OFFSET(model_outputnames_string), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0,
FLAGS },\
{ "backend_configs","backend configs",
OFFSET(backend_options), AV_OPT_TYPE_STRING,{ .str = NULL }, 0, 0, FLAGS
},\
{ "options", "backend configs (deprecated, use backend_configs)",
OFFSET(backend_options), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, FLAGS |
AV_OPT_FLAG_DEPRECATED},\
-{ "async", "use DNN async inference",OFFSET(async),
AV_OPT_TYPE_BOOL, { .i64 = 1}, 0, 1, FLAGS},
+{ "async", "use DNN async inference (ignored, use
backend_configs='async=1')",OFFSET(async),AV_OPT_TYPE_BOOL,
{ .i64 = 1}, 0, 1, FLAGS},
int ff_dnn_init(DnnContext *ctx, DNNFunctionType func_type, AVFilterContext
*filter_ctx);
--
2.25.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v5 5/6] lavfi/dnn: Rename InferenceItem to LastLevelTaskItem
This patch renames the InferenceItem to LastLevelTaskItem in the
three backends to avoid confusion among the meanings of these structs.
The following are the renames done in this patch:
1. extract_inference_from_task -> extract_lltask_from_task
2. InferenceItem -> LastLevelTaskItem
3. inference_queue -> lltask_queue
4. inference -> lltask
5. inference_count -> lltask_count
Signed-off-by: Shubhanshu Saxena
---
libavfilter/dnn/dnn_backend_common.h | 4 +-
libavfilter/dnn/dnn_backend_native.c | 58 ++---
libavfilter/dnn/dnn_backend_native.h | 2 +-
libavfilter/dnn/dnn_backend_openvino.c | 110 -
libavfilter/dnn/dnn_backend_tf.c | 76 -
5 files changed, 125 insertions(+), 125 deletions(-)
diff --git a/libavfilter/dnn/dnn_backend_common.h
b/libavfilter/dnn/dnn_backend_common.h
index 78e62a94a2..6b6a5e21ae 100644
--- a/libavfilter/dnn/dnn_backend_common.h
+++ b/libavfilter/dnn/dnn_backend_common.h
@@ -47,10 +47,10 @@ typedef struct TaskItem {
} TaskItem;
// one task might have multiple inferences
-typedef struct InferenceItem {
+typedef struct LastLevelTaskItem {
TaskItem *task;
uint32_t bbox_index;
-} InferenceItem;
+} LastLevelTaskItem;
/**
* Common Async Execution Mechanism for the DNN Backends.
diff --git a/libavfilter/dnn/dnn_backend_native.c
b/libavfilter/dnn/dnn_backend_native.c
index 2d34b88f8a..13436c0484 100644
--- a/libavfilter/dnn/dnn_backend_native.c
+++ b/libavfilter/dnn/dnn_backend_native.c
@@ -46,25 +46,25 @@ static const AVClass dnn_native_class = {
.category = AV_CLASS_CATEGORY_FILTER,
};
-static DNNReturnType execute_model_native(Queue *inference_queue);
+static DNNReturnType execute_model_native(Queue *lltask_queue);
-static DNNReturnType extract_inference_from_task(TaskItem *task, Queue
*inference_queue)
+static DNNReturnType extract_lltask_from_task(TaskItem *task, Queue
*lltask_queue)
{
NativeModel *native_model = task->model;
NativeContext *ctx = &native_model->ctx;
-InferenceItem *inference = av_malloc(sizeof(*inference));
+LastLevelTaskItem *lltask = av_malloc(sizeof(*lltask));
-if (!inference) {
-av_log(ctx, AV_LOG_ERROR, "Unable to allocate space for
InferenceItem\n");
+if (!lltask) {
+av_log(ctx, AV_LOG_ERROR, "Unable to allocate space for
LastLevelTaskItem\n");
return DNN_ERROR;
}
task->inference_todo = 1;
task->inference_done = 0;
-inference->task = task;
+lltask->task = task;
-if (ff_queue_push_back(inference_queue, inference) < 0) {
-av_log(ctx, AV_LOG_ERROR, "Failed to push back inference_queue.\n");
-av_freep(&inference);
+if (ff_queue_push_back(lltask_queue, lltask) < 0) {
+av_log(ctx, AV_LOG_ERROR, "Failed to push back lltask_queue.\n");
+av_freep(&lltask);
return DNN_ERROR;
}
return DNN_SUCCESS;
@@ -116,13 +116,13 @@ static DNNReturnType get_output_native(void *model, const
char *input_name, int
goto err;
}
-if (extract_inference_from_task(&task, native_model->inference_queue) !=
DNN_SUCCESS) {
-av_log(ctx, AV_LOG_ERROR, "unable to extract inference from task.\n");
+if (extract_lltask_from_task(&task, native_model->lltask_queue) !=
DNN_SUCCESS) {
+av_log(ctx, AV_LOG_ERROR, "unable to extract last level task from
task.\n");
ret = DNN_ERROR;
goto err;
}
-ret = execute_model_native(native_model->inference_queue);
+ret = execute_model_native(native_model->lltask_queue);
*output_width = task.out_frame->width;
*output_height = task.out_frame->height;
@@ -223,8 +223,8 @@ DNNModel *ff_dnn_load_model_native(const char
*model_filename, DNNFunctionType f
goto fail;
}
-native_model->inference_queue = ff_queue_create();
-if (!native_model->inference_queue) {
+native_model->lltask_queue = ff_queue_create();
+if (!native_model->lltask_queue) {
goto fail;
}
@@ -297,24 +297,24 @@ fail:
return NULL;
}
-static DNNReturnType execute_model_native(Queue *inference_queue)
+static DNNReturnType execute_model_native(Queue *lltask_queue)
{
NativeModel *native_model = NULL;
NativeContext *ctx = NULL;
int32_t layer;
DNNData input, output;
DnnOperand *oprd = NULL;
-InferenceItem *inference = NULL;
+LastLevelTaskItem *lltask = NULL;
TaskItem *task = NULL;
DNNReturnType ret = 0;
-inference = ff_queue_pop_front(inference_queue);
-if (!inference) {
-av_log(NULL, AV_LOG_ERROR, "Failed to get inference item\n");
+lltask = ff_queue_pop_front(lltask_queue);
+if (!lltask) {
+av_log(NULL, AV_LOG_ERROR, "Failed to get LastLevelTaskItem\n");
ret = DNN_ERROR;
goto err;
}
-task = inference->task;
+task = lltask->task;
native_model = task->model;
ctx = &native_model->ctx;
@@ -428,7 +428,7 @@ static DNNReturnType e
[FFmpeg-devel] [PATCH v5 6/6] doc/filters.texi: Include dnn_processing in docs of sr and derain filter
Signed-off-by: Shubhanshu Saxena
---
doc/filters.texi | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/doc/filters.texi b/doc/filters.texi
index 7aff920311..4da1cdc2aa 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -9982,7 +9982,7 @@ Note that different backends use different file formats.
TensorFlow and native
backend can load files for only its format.
@end table
-It can also be finished with @ref{dnn_processing} filter.
+To get full functionality (such as async execution), please use the
@ref{dnn_processing} filter.
@section deshake
@@ -19347,7 +19347,7 @@ Default value is @code{2}. Scale factor is necessary
for SRCNN model, because it
input upscaled using bicubic upscaling with proper scale factor.
@end table
-This feature can also be finished with @ref{dnn_processing} filter.
+To get full functionality (such as async execution), please use the
@ref{dnn_processing} filter.
@section ssim
--
2.25.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] Sending patch to FFMPEG
Makes sense, but there is no mentioning of nasm is recommended . Is there any down side of --disable x86? On Thu, Aug 26, 2021 at 2:12 AM Lynne wrote: > 5 Aug 2021, 21:05 by janpoonthong...@gmail.com: > > > If I'm missing something, please let me know. > > > > We've recommended nasm instead of yasm for years now. > You don't need to have nasm installed to compile, you > can disable assembly optimizations. configure even > tells you how, and forces you to in case neither is found. > > I think it's fine as-is. Besides, you don't even need either > on anything that isn't x86. > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] h274: remove optimization pragma
I didn't really understand what you meant. So I should install nasm and run ./configure or just ./configure --disable-x86asm? On Thu, Aug 26, 2021 at 2:24 AM Lynne wrote: > This results in warnings on compilers which don't support it, > objections were raised during the review process about it but went > unnoticed, > and the speed benefit is highly compiler and version specific, and > also not very critical. > > We generally hand-write assembly to optimize loops like that, rather > than use compiler magic, and for 40% best case scenario, it's simply > not worth it. > > Plus, tree vectorization is still problematic with GCC and disabled by > default > for a good reason, so enabling it locally is sketchy. > > Patch attached. > > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] Missing sudo on make install in INSTALL.md
0001-Missing-sudo-on-make-install.patch Description: Binary data ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] Missing sudo on make install in INSTALL.md
> I tried try `make install` on my machine and I got: >install: cannot create regular file '/usr/local/share/man/man1/ffmpeg.1': > Permission denied >install: cannot create regular file '/usr/local/share/man/man1/ffprobe.1': > Permission denied >install: cannot create regular file > '/usr/local/share/man/man1/ffmpeg-all.1': Permission denied >install: cannot create regular file > '/usr/local/share/man/man1/ffprobe-all.1': Permission denied >install: cannot create regular file > '/usr/local/share/man/man1/ffmpeg-utils.1': Permission denied >install: cannot create regular file > '/usr/local/share/man/man1/ffmpeg-scaler.1': Permission denied >install: cannot create regular file > '/usr/local/share/man/man1/ffmpeg-resampler.1': Permission denied >install: cannot create regular file > '/usr/local/share/man/man1/ffmpeg-codecs.1': Permission denied >install: cannot create regular file > '/usr/local/share/man/man1/ffmpeg-bitstream-filters.1': Permission denied >install: cannot create regular file > '/usr/local/share/man/man1/ffmpeg-formats.1': Permission denied >install: cannot create regular file > '/usr/local/share/man/man1/ffmpeg-protocols.1': Permission denied >install: cannot create regular file > '/usr/local/share/man/man1/ffmpeg-devices.1': Permission denied >install: cannot create regular file > '/usr/local/share/man/man1/ffmpeg-filters.1': Permission denied >make: *** [doc/Makefile:126: install-man] Error 1 > > By doing `sudo make install` it fixs the problem. User are not on root all > the time. Nope. sudo is unnecessary or unavailable on some platform. You can install it to other directories without the need of root permission. Most importantly, sudo is a dangerous advice for novice. On the other hand, for anyone who already know what sudo is he/she doesn’t need such advice. > --- > INSTALL.md | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/INSTALL.md b/INSTALL.md > index 3b220bc6ff..7600fc2ce3 100644 > --- a/INSTALL.md > +++ b/INSTALL.md > @@ -9,7 +9,7 @@ path when launching `configure`, e.g. > `/ffmpegdir/ffmpeg/configure`. > > 2. Then type `make` to build FFmpeg. GNU Make 3.81 or later is required. > > -3. Type `make install` to install all binaries and libraries you built. > +3. Type `sudo make install` to install all binaries and libraries you built. > > NOTICE > -- > -- > 2.25.1 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 1/1] avcodec/vc1dec: Return value check for init_get_bits
Thanks, So when will you apply this? ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v3 1/1] avcodec/vp6: return value check for init_get_bits
Thanks, So when will you apply this? ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] Sending patch to FFMPEG
On Thu, Aug 26, 2021 at 4:58 AM Jan Poonthong wrote: > Makes sense, but there is no mentioning of nasm is recommended . Is there > any down side of --disable x86? > Yes, big one. > > On Thu, Aug 26, 2021 at 2:12 AM Lynne wrote: > > > 5 Aug 2021, 21:05 by janpoonthong...@gmail.com: > > > > > If I'm missing something, please let me know. > > > > > > > We've recommended nasm instead of yasm for years now. > > You don't need to have nasm installed to compile, you > > can disable assembly optimizations. configure even > > tells you how, and forces you to in case neither is found. > > > > I think it's fine as-is. Besides, you don't even need either > > on anything that isn't x86. > > ___ > > ffmpeg-devel mailing list > > ffmpeg-devel@ffmpeg.org > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > > To unsubscribe, visit link above, or email > > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". > > > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
