On Tue, 29 Jun 2021 at 12:08, Dean Rasheed <dean.a.rash...@gmail.com> wrote: > > Numeric x^y is supported for x < 0 if y is an integer, but this > currently fails if y is outside the range of an int32 >
I've been doing some more testing of this, and I spotted another problem with numeric_power(). This is what happens when raising 0.9999999999 to increasingly large powers, which should decrease to zero: exp 0.9999999999^exp 10000000000 0.3678794411530483 100000000000 0.[4 zeros]4539992973978489 1000000000000 0.[43 zeros]3720075957420456 10000000000000 0.[434 zeros]5075958643751518 20000000000000 0.[868 zeros]2576535615307575 21000000000000 0.[912 zeros]9584908195943232 22000000000000 0.[955 zeros]3565658653381070 23000000000000 0.[998 zeros]13 23100000000000 0.[1000 zeros] 23200000000000 0.[1000 zeros] 23300000000000 1.[1000 zeros] *** WRONG *** 30000000000000 1.[1000 zeros] *** WRONG *** 40000000000000 1.[1000 zeros] *** WRONG *** 50000000000000 1.[1000 zeros] *** WRONG *** 60000000000000 1.[1000 zeros] *** WRONG *** 70000000000000 ERROR: value overflows numeric format The cases where it returns 1 are a trivial logic bug in the local_rscale calculation in power_var() -- when it computes local_rscale from rscale and val, it needs to do so before clipping rscale to NUMERIC_MAX_DISPLAY_SCALE, otherwise it ends up setting local_rscale = 0, and loses all precision. I also don't think it should be throwing an overflow error here. Some code paths through numeric_power() catch cases that would underflow, and return zero instead, but not all cases are caught. There's a similar overflow error with numeric_exp() for large negative inputs (-5999 returns 0, but -6000 overflows). It's arguable though that numeric power() and exp() (and mul() for that matter) should never return 0 for finite non-zero inputs, but instead should throw underflow errors, which would make them compatible with their floating-point counterparts. I don't think that's useful though, and it's more likely to break people's code for no real benefit. No other numeric code throws underflow errors. So I think we should just attempt to avoid all such overflow errors, that are actually underflows, and return zero instead. Regards, Dean
