On Tue, Jul 2, 2024, at 21:55, Joel Jacobson wrote: > On Tue, Jul 2, 2024, at 20:53, Joel Jacobson wrote: >> Trying to wrap my head around what could cause this.
I found the bug in the case 3 code,
and it turns out the same type of bug also exists in the case 2 code:
case 2:
newdig = (int) var1digits[1] *
var2digits[res_ndigits - 4];
The problem here is that res_ndigits could become less than 4,
if rscale is low enough,
and then we would try to access a negative array index in var2digits.
Fix:
case 2:
if (res_ndigits - 4 >= 0 && res_ndigits - 4 <
var2ndigits)
newdig = (int) var1digits[1] *
var2digits[res_ndigits - 4];
else
newdig = 0;
Another problem in the case 2 code:
if (res_ndigits - 3 < var2ndigits)
newdig += (int) var1digits[0] *
var2digits[res_ndigits - 3];
Fix:
if (res_ndigits - 3 >= 0 && res_ndigits - 3 <
var2ndigits)
newdig += (int) var1digits[0] *
var2digits[res_ndigits - 3];
New version attached that fixes both the case 2 and case 3 code.
Regards,
Joel
v4-optimize-numeric-mul_var-small-var1-arbitrary-var2.patch
Description: Binary data
