On Mon, Apr 05, 2021 at 03:44:33AM +0200, Andreas Rheinhardt wrote: > The RealVideo 3.0 and 4.0 decoders call ff_mpv_common_init() only during > their init function and not during decode_frame(); when the size of the > frame changes, they call ff_mpv_common_frame_size_change(). Yet upon > error, said function calls ff_mpv_common_end() which frees the whole > MpegEncContext and not only those parts that > ff_mpv_common_frame_size_change() reinits. As a result, the context will > never be usable again; worse, because decode_frame() contains no check > for whether the context is initialized or not, it is presumed that it is > initialized, leading to segfaults. Basically the same happens if > rv34_decoder_realloc() fails. > > This commit fixes this by only resetting the parts that > ff_mpv_common_frame_size_change() changes upon error and by actually > checking whether the context is in need of reinitialization in > ff_rv34_decode_frame(). > > Signed-off-by: Andreas Rheinhardt <andreas.rheinha...@outlook.com>
> --- > I actually don't like that we have two flags that indicate whether > a MpegEncContext is usable or not; how about we always call > ff_mpv_common_init() during init (and never lateron) and make it > unconditionally allocate the stuff that does not depend upon resolution > etc. and add a parameter to said function to also allocate the latter. > The decode_frame functions would then be modified to always use > ff_mpv_common_frame_size_change(). sure if that ends up being cleaner [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB I know you won't believe me, but the highest form of Human Excellence is to question oneself and others. -- Socrates
signature.asc
Description: PGP signature
_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
