Re: [FFmpeg-devel] [PATCH] fixed luma quantizing when q >= MAX_STORED_Q

2018-12-28 Thread Derek Buitenhuis
On 28/12/2018 17:04, Alex Mogurenko wrote: > problem occurs in slice quant estimation and slice encoding. > > if slice quant >= MAX_STORED_Q we dont use pre-calculated quant matrices > but generate new: > > qmat = ctx->custom_q; > > qmat_chroma = ctx->custom_q; > > for (i = 0; i

Re: [FFmpeg-devel] [PATCH] fixed luma quantizing when q >= MAX_STORED_Q

2018-12-28 Thread Alex Mogurenko
problem occurs in slice quant estimation and slice encoding. if slice quant >= MAX_STORED_Q we dont use pre-calculated quant matrices but generate new: qmat = ctx->custom_q; qmat_chroma = ctx->custom_q; for (i = 0; i < 64; i++) { qmat[i] = ctx->quant_mat[i] * quant;

Re: [FFmpeg-devel] [PATCH] fixed luma quantizing when q >= MAX_STORED_Q

2018-12-28 Thread Derek Buitenhuis
On 27/12/2018 19:28, Alex Mogurenko wrote: > --- > libavcodec/proresenc_kostya.c | 6 -- > 1 file changed, 4 insertions(+), 2 deletions(-) Can you give a little more detail about what's changed and why, in the commit message? It looks like custom_chroma_q is zero initialized and never set to

[FFmpeg-devel] [PATCH] fixed luma quantizing when q >= MAX_STORED_Q

2018-12-27 Thread Alex Mogurenko
--- libavcodec/proresenc_kostya.c | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/libavcodec/proresenc_kostya.c b/libavcodec/proresenc_kostya.c index 9a77d24fb6..e045a972f1 100644 --- a/libavcodec/proresenc_kostya.c +++ b/libavcodec/proresenc_kostya.c @@ -222,6 +222,7 @@