On 7 May 2016 at 18:12, Rostislav Pehlivanov <atomnu...@gmail.com> wrote:
> The problem is that with particularly complex images and especially at
> high bit depths and 5-level transforms the coefficients would overflow,
> causing huge artifacts to appear. This was discovered thanks to the fate
> tests, which will have to be redone as this fixes a multitude of
> problems and increases PSNR.
>
> There is a slight performance drop associated with this change, making
> the encoder slower by 1.15 times, however this is necessary in order to
> avoid undefined behavior and overflows.
>
> It would be worth to template the transforms to keep the performance for
> 8 bit images as 32 bit coefficients are unnecessary for that case, but
> the primary use of the encoder is to encode video at 10 bits.
>
> Signed-off-by: Rostislav Pehlivanov <atomnu...@gmail.com>
> ---
> libavcodec/vc2enc_dwt.h | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/libavcodec/vc2enc_dwt.h b/libavcodec/vc2enc_dwt.h
> index af6fe3e..7fbbfbe 100644
> --- a/libavcodec/vc2enc_dwt.h
> +++ b/libavcodec/vc2enc_dwt.h
> @@ -25,7 +25,7 @@
> #include <stddef.h>
> #include <stdint.h>
>
> -typedef int16_t dwtcoef;
> +typedef int32_t dwtcoef;
>
> enum VC2TransformType {
> VC2_TRANSFORM_9_7 = 0, /* Deslauriers-Dubuc (9,7) */
> --
> 2.8.1.369.geae769a
>
>
Pushed
Thanks for the reviews
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
