PR #21598 opened by TADANO Tokumei (aimoff) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21598 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21598.patch
This PR is another implementation of: https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=12899 This patch add a v360gopro filter to convert GoPro Max native .360 files to various 360 video formats. The GoPro Max .360 file contains separated two video streams. The v360gopro filter acts as preprocessing filter of v360. It combines two video streams into single video stream, convert to normalized EAC (Equi-Angular Cubemap) format, and pass it to v360 filter. Abstruct of GoPro Max .360 video file format is described in: https://gopro.com/news/max-tech-specs-stitching-resolution More information is in: https://www.trekview.org/blog/reverse-engineering-gopro-360-file-format-part-3/ The specification is little bit buggy. The format is based on EAC, and there are overlapped pixels at boundaries of front and rear cams. Probably, the desinger intended to add 2 x 32 (= total 64) ovelapped pixels. But actual format has 2 x 64 pixels overlapped area. Thus the width will be 2 x 32 pixels shorter than standard EAC format after blending overlapped area. >From c3021659fba44baa61899e787cedcf405fc4df37 Mon Sep 17 00:00:00 2001 From: TADANO Tokumei <[email protected]> Date: Tue, 27 Jan 2026 16:32:48 +0900 Subject: [PATCH] lavfi/vf_v360: add GoPro Max video filter Add a v360gopro filter to convert GoPro Max native .360 files to various 360 video formats. The GoPro Max .360 file contains separated two video streams. This filter combine two streams into single stream with standard format. --- doc/filters.texi | 49 +++++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/v360.h | 13 ++ libavfilter/vf_v360.c | 275 +++++++++++++++++++++++++++++++++++++++ 5 files changed, 339 insertions(+) diff --git a/doc/filters.texi b/doc/filters.texi index 0f64b4a3fa..473d5909db 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -24331,6 +24331,7 @@ Convert 360 videos between various formats. The filter accepts the following options: +@anchor{v360 options} @table @option @item input @@ -24739,6 +24740,54 @@ v360=eac:equirect:in_stereo=sbs:in_trans=1:ih_flip=1:out_stereo=tb This filter supports subset of above options as @ref{commands}. +@section v360gopro + +Convert GoPro Max .360 video to various formats. + +This filter is designed to use GoPro Max .360 files as inputs. +Native .360 files are sort of EAC files, in fact the front and rear lenses streams are the top and the bottom of the EAC projection. + +The .360 file contains two video streams. +Most of cases, one is stream #0:0, and the other is stream #0:5. +Please check actual stream number with @code{ffprobe} command. +This filter combine two streams to single stream. + +The .360 contains also 2x64 pixels of overlapped area. +The filter blends overlapped images in these two areas. + +The filter accepts most of @ref{v360 options}, but ignores @code{input} option. +There is a @code{v360gopro} specific option: + +@table @option + +@item overlap +Set number of overlapped pixels on input .360 video. + +No need to specify this option for native .360 video file. +This option is for rescaled video or future video format change. + +Default is @code{64}. + +@end table + +@subsection Example + +@itemize +@item +Convert .360 to Equirectangular projection. +@example +ffmpeg -i INPUT.360 -filter_complex '[0:0][0:5]v360gopro=output=e:w=4096:h=2048' -map 0:1 -map 0:3 -c:a copy -c:u copy OUTPUT.mp4 +@end example + +Two video streams (#0:0 and #0:5) are combined and converted to equirectangular projection with specified resolution. +Stream #0:1 (GoPro AAC) and stream #0:3 (GoPro MET) are copied with the video stream. + +@end itemize + +@subsection Commands + +This filter supports subset of @ref{v360 options} as @ref{commands}. + @section vaguedenoiser Apply a wavelet based denoiser. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 4e128ed109..a0e19a020e 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -557,6 +557,7 @@ OBJS-$(CONFIG_UNSHARP_OPENCL_FILTER) += vf_unsharp_opencl.o opencl.o \ OBJS-$(CONFIG_UNTILE_FILTER) += vf_untile.o OBJS-$(CONFIG_USPP_FILTER) += vf_uspp.o qp_table.o OBJS-$(CONFIG_V360_FILTER) += vf_v360.o +OBJS-$(CONFIG_V360GOPRO_FILTER) += vf_v360.o framesync.o OBJS-$(CONFIG_VAGUEDENOISER_FILTER) += vf_vaguedenoiser.o OBJS-$(CONFIG_VARBLUR_FILTER) += vf_varblur.o framesync.o OBJS-$(CONFIG_VECTORSCOPE_FILTER) += vf_vectorscope.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 33cd637706..58b6545a31 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -523,6 +523,7 @@ extern const FFFilter ff_vf_unsharp_opencl; extern const FFFilter ff_vf_untile; extern const FFFilter ff_vf_uspp; extern const FFFilter ff_vf_v360; +extern const FFFilter ff_vf_v360gopro; extern const FFFilter ff_vf_vaguedenoiser; extern const FFFilter ff_vf_varblur; extern const FFFilter ff_vf_vectorscope; diff --git a/libavfilter/v360.h b/libavfilter/v360.h index b3660c234c..02fa977d10 100644 --- a/libavfilter/v360.h +++ b/libavfilter/v360.h @@ -20,7 +20,10 @@ #ifndef AVFILTER_V360_H #define AVFILTER_V360_H +#include "config_components.h" + #include "avfilter.h" +#include "framesync.h" enum StereoFormats { STEREO_2D, @@ -178,6 +181,16 @@ typedef struct V360Context { SliceXYRemap *slice_remap; unsigned map[AV_VIDEO_MAX_PLANES]; +#if CONFIG_V360GOPRO_FILTER + FFFrameSync fs; + + int gopromax; + int overlap; + + int hsub[AV_VIDEO_MAX_PLANES], vsub[AV_VIDEO_MAX_PLANES]; + uint8_t **work; +#endif + int (*in_transform)(const struct V360Context *s, const float *vec, int width, int height, int16_t us[4][4], int16_t vs[4][4], float *du, float *dv); diff --git a/libavfilter/vf_v360.c b/libavfilter/vf_v360.c index a5297c81e9..0f5a39a920 100644 --- a/libavfilter/vf_v360.c +++ b/libavfilter/vf_v360.c @@ -167,6 +167,9 @@ static const AVOption v360_options[] = { { "v_offset", "output vertical off-axis offset", OFFSET(v_offset), AV_OPT_TYPE_FLOAT,{.dbl=0.f}, -1.f, 1.f,TFLAGS, .unit = "v_offset"}, {"alpha_mask", "build mask in alpha plane", OFFSET(alpha), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, FLAGS, .unit = "alpha"}, { "reset_rot", "reset rotation", OFFSET(reset_rot), AV_OPT_TYPE_BOOL, {.i64=0}, -1, 1,TFLAGS, .unit = "reset_rot"}, +#if CONFIG_V360GOPRO_FILTER + { "overlap", "overlapped pixels for GoPro Max", OFFSET(overlap), AV_OPT_TYPE_INT, {.i64=64}, 0, 128, FLAGS, .unit = "overlap"}, +#endif { NULL } }; @@ -4430,6 +4433,14 @@ static int config_output(AVFilterLink *outlink) av_assert0(0); } +#if CONFIG_V360GOPRO_FILTER + if (s->gopromax) { + s->in = EQUIANGULAR; + w = inlink->h * 3; + h = inlink->h * 2; + } +#endif + set_dimensions(s->inplanewidth, s->inplaneheight, w, h, desc); set_dimensions(s->in_offset_w, s->in_offset_h, in_offset_w, in_offset_h, desc); @@ -5008,3 +5019,267 @@ const FFFilter ff_vf_v360 = { FILTER_QUERY_FUNC2(query_formats), .process_command = process_command, }; + +#if CONFIG_V360GOPRO_FILTER +FRAMESYNC_DEFINE_CLASS_EXT(v360gopro, V360Context, fs, v360_options); + +typedef struct ThreadDataGopro { + AVFrame *in; + AVFrame *out; + int y; +} ThreadDataGopro; + +// Convert GoPro Max format to normalized EAC + +static void gopro_remap_cube_8bit_c(uint8_t *dst, const uint8_t *const src, uint8_t *buf, + const int cube_size, const int gp_cube_width, + const int cube_sub, const int overlap) +{ + unsigned cl, cr; + const uint8_t *p = src; + uint8_t *d = dst; + uint8_t *b = buf; + const int cs = gp_cube_width - overlap; + + // merge overlapped area + memcpy(b, p, cube_sub); + p += cube_sub; + b += cube_sub; + for (int i = 0; i < overlap; i++) { + cl = *p; + cr = *(p + overlap); + *b = (cl * (overlap - i) + cr * i) / overlap; + p++; + b++; + } + p += overlap; + memcpy(b, p, cube_sub); + + // rescale + for (int i = 0; i < cube_size; i++) { + int n = cs * i / cube_size; + int m = cs * i % cube_size; + b = buf + n; + + cl = *b; + cr = *(b + 1); + *d = (cl * (cube_size - m) + cr * m) / cube_size; + d++; + } +} + +static void gopro_remap_line_8bit_c(uint8_t *dst, const uint8_t *const src, uint8_t *buf, + int cube_size, const int gp_cube_width, + const int cube_sub, const int overlap) +{ + const uint8_t *p = src; + uint8_t *d = dst; + + gopro_remap_cube_8bit_c(d, p, buf, cube_size, + gp_cube_width, cube_sub, overlap); + p += gp_cube_width; + d += cube_size; + + memcpy(d, p, cube_size); + p += cube_size; + d += cube_size; + + gopro_remap_cube_8bit_c(d, p, buf, cube_size, + gp_cube_width, cube_sub, overlap); +} + +static int gopro_slice(AVFilterContext *ctx, void *arg, int jobnr, int nb_jobs) +{ + V360Context *s = ctx->priv; + ThreadDataGopro *td = arg; + AVFrame *in = td->in; + AVFrame *out = td->out; + uint8_t *buf = s->work[jobnr]; + + for (int plane = 0; plane < s->nb_planes && in->data[plane] && in->linesize[plane]; plane++) { + const int in_width = AV_CEIL_RSHIFT(in->width, s->hsub[plane]); + const int width = AV_CEIL_RSHIFT(out->width, s->hsub[plane]); + const int height = AV_CEIL_RSHIFT(in->height, s->vsub[plane]); + const int offset_h = AV_CEIL_RSHIFT(td->y, s->vsub[plane]); + const int cube_size = width / 3; + const int overlap = AV_CEIL_RSHIFT(s->overlap, s->hsub[plane]); + const int gp_cube_width = (in_width - cube_size) / 2; + const int gp_cube_sub = (gp_cube_width - overlap * 2) / 2; + const int start = (height * jobnr ) / nb_jobs; + const int end = (height * (jobnr+1)) / nb_jobs; + uint8_t *inrow, *outrow; + + inrow = in ->data[plane] + start * in->linesize[plane]; + outrow = out->data[plane] + (offset_h + start) * out->linesize[plane]; + + for (int y = start; y < end && y < height && in->linesize[plane]; y++) { + gopro_remap_line_8bit_c(outrow, inrow, buf, cube_size, + gp_cube_width, gp_cube_sub, overlap); + inrow += in ->linesize[plane]; + outrow += out->linesize[plane]; + } + } + + return 0; +} + +static int v360gopro_filter_frame(FFFrameSync *fs) +{ + AVFilterContext *ctx = fs->parent; + V360Context *s = ctx->priv; + AVFrame *front, *rear, *in; + ThreadDataGopro td; + int ret; + + if ((ret = ff_framesync_dualinput_get(fs, &front, &rear)) < 0) + return ret; + if (!rear) { + av_log(ctx, AV_LOG_ERROR, "Can't get 2nd video frame.\n"); + av_frame_free(&front); + return AVERROR(EINVAL); + } + + in = av_frame_alloc(); + if (!in) { + av_log(ctx, AV_LOG_ERROR, "Can't allocate work video frame.\n"); + av_frame_free(&front); + return AVERROR(ENOMEM); + } + in->width = front->height * 3; + in->height = front->height + rear->height; + in->format = ctx->inputs[0]->format; + + if ((ret = av_frame_get_buffer(in, 0)) < 0) { + av_log(ctx, AV_LOG_ERROR, "Can't allocate work video buffer.\n"); + av_frame_free(&in); + av_frame_free(&front); + return ret; + } + av_frame_copy_props(in, front); + + td.in = front; + td.out = in; + td.y = 0; + ff_filter_execute(ctx, gopro_slice, &td, NULL, s->nb_threads); + td.in = rear; + td.y = front->height; + ff_filter_execute(ctx, gopro_slice, &td, NULL, s->nb_threads); + + av_frame_free(&front); /* rear frame will be freed */ + + return filter_frame(ctx->inputs[0], in); +} + +static int v360gopro_config_output(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + AVFilterLink *inlink = ctx->inputs[0]; + V360Context *s = ctx->priv; + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(inlink->format); + const int cube_size = inlink->h; + int err; + + if ((err = ff_framesync_init_dualinput(&s->fs, ctx)) < 0) + return err; + + if ((inlink->w != ctx->inputs[1]->w) || + (inlink->h != ctx->inputs[1]->h) || + (inlink->format != ctx->inputs[1]->format) || + desc->comp[0].depth > 8) { + av_log(ctx, AV_LOG_ERROR, "Incompatible inputs for GoPro Max.\n"); + return AVERROR(EINVAL); + } + + if ((err = config_output(ctx->outputs[0])) < 0) + return err; + + s->hsub[0] = s->hsub[3] = 0; + s->hsub[1] = s->hsub[2] = desc->log2_chroma_w; + s->vsub[0] = s->vsub[3] = 0; + s->vsub[1] = s->vsub[2] = desc->log2_chroma_h; + + s->work = av_calloc(s->nb_threads, sizeof(uint8_t *)); + if (!s->work) + return AVERROR(ENOMEM); + for (int i = 0; i < s->nb_threads; i++) { + s->work[i] = av_calloc(cube_size, sizeof(uint8_t) * 2); + if (!s->work[i]) + return AVERROR(ENOMEM); + } + + outlink->time_base = inlink->time_base; + outlink->format = inlink->format; + outlink->sample_aspect_ratio = inlink->sample_aspect_ratio; + + err = ff_framesync_configure(&s->fs); + outlink->time_base = s->fs.time_base; + + return err; +} + +static int v360gopro_activate(AVFilterContext *ctx) +{ + V360Context *s = ctx->priv; + + return ff_framesync_activate(&s->fs); +} + +static av_cold void v360gopro_uninit(AVFilterContext *ctx) +{ + V360Context *s = ctx->priv; + + ff_framesync_uninit(&s->fs); + if (s->work) + for (int n = 0; n < s->nb_threads; n++) + av_freep(&s->work[n]); + av_freep(&s->work); + + uninit(ctx); +} + +static av_cold int v360gopro_init(AVFilterContext *ctx) +{ + V360Context *s = ctx->priv; + + s->gopromax = 1; + s->in = EQUIANGULAR; + s->fs.on_event = v360gopro_filter_frame; + + return init(ctx); +} + +static const AVFilterPad v360gopro_inputs[] = { + { + .name = "front", + .type = AVMEDIA_TYPE_VIDEO, + }, + { + .name = "rear", + .type = AVMEDIA_TYPE_VIDEO, + }, +}; + +static const AVFilterPad v360gopro_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = v360gopro_config_output, + }, +}; + +const FFFilter ff_vf_v360gopro = { + .p.name = "v360gopro", + .p.description = NULL_IF_CONFIG_SMALL("Convert GoPro Max 360 projection of video."), + .p.priv_class = &v360gopro_class, + .p.flags = AVFILTER_FLAG_SLICE_THREADS, + .priv_size = sizeof(V360Context), + .preinit = v360gopro_framesync_preinit, + .init = v360gopro_init, + .uninit = v360gopro_uninit, + .activate = v360gopro_activate, + FILTER_INPUTS(v360gopro_inputs), + FILTER_OUTPUTS(v360gopro_outputs), + FILTER_QUERY_FUNC2(query_formats), + .process_command = process_command, +}; +#endif -- 2.52.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
