> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-boun...@ffmpeg.org> On Behalf Of
> > Chen, Wenbin
> > Sent: Wednesday, November 10, 2021 4:03 AM
> > To: ffmpeg-devel@ffmpeg.org
> > Subject: Re: [FFmpeg-devel] [PATCH 2/4] libavutil/hwcontext_qsv: fix
> > a bug when malloc handle_pairs_internal
> >
> > > This commandline cause core dumped:
> > > ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 \
> > > -hwaccel_output_format vaapi -i input.264 \
> > > -vf "hwmap=derive_device=qsv,format=qsv" \
> > > -c:v h264_qsv output.264
> > >
> > > reason: We use nb_surfaces to assign surface to
> > handle_pairs_internal
> > > but handle_pairs_internal is alloced with the size of
> > init_pool_size.
> > > This lead to access to illegal address.
> > >
> > > Now change it to use nb_surfaces to allocate handle_pairs_internal
> > and the
>
> I'm not sure about whether this is right.
>
> When we look at the top of the qsv_frames_derive_to function that you
> are changing, there is this:
>
>
> if (src_ctx->initial_pool_size == 0) {
> av_log(dst_ctx, AV_LOG_ERROR, "Only fixed-size pools can be "
> "mapped to QSV frames.\n");
> return AVERROR(EINVAL);
> }
>
> It's because QSV doesn't support dynamic pool sizes.
>
> When we look at the vaapi_pool_alloc function in hwcontext_vaapi.c, we
> can see that:
>
> when initial_pool_size is > 0, the pool cannot grow beyond this value,
> so nb_surfaces cannot be > initial_pool_size
>
> So I'm wondering what could have caused the segfault? Which values did
> you have there for nb_surfaces and initial_pool_size?
>
>
> > > core dumped error is unseen. Also change D3D11VA to use nb_surfaces
> > > to align to VAAPI and DXVA2.
>
> Those changes are unrelated to fixing the issue with VAAPI.
> (besides that I don't think these are needed at all)
>
> Kind regards,
> softworkz
You are right. The real cause is that vaapi_decode_make_config() is called
twice.
The init_pool_size is changed on the second call. I will resubmit patch to fix
this
Thanks
Wenbin
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
