On Wed, Jul 3, 2024, at 22:27, Joel Jacobson wrote:
> On Wed, Jul 3, 2024, at 20:57, Dean Rasheed wrote:
>> I wouldn't expect it to ever be off by more than 1, given that
>> MUL_GUARD_DIGITS = 2, which corresponds to 8 decimal digits, and the
>> number of digits in the smaller input (and hence the number of digit
>> products in each column) is limited to something like 16,000 NBASE
>> digits.
>
> OK, so then the cases I found where it was off by 2 for the mul_var_int()
> patch
> are unexpected?
Sorry, I meant off by 2 for the mul_var_small() patch, these cases that I found:
var1 | var2 | rscale_adjustment | result |
numeric_mul_patch_small
-------------------+----------------+-------------------+------------+-------------------------
8952.12658563 | 0.902315486665 | -16 | 8077.6425 |
8077.6423
0.881715409579 | 0.843165739371 | -16 | 0.74343223 |
0.74343221
0.905322758954 | 0.756905996850 | -16 | 0.68524423 |
0.68524421
8464.043170546608 | 0.518100129611 | -20 | 4385.2219 |
4385.2217
5253.006296984449 | 0.989308019355 | -20 | 5196.8413 |
5196.8411
(5 rows)
Regards,
Joel
