On 11/4/21 23:56, Tomas Vondra wrote:
Hi,
On 11/4/21 17:53, Justin Pryzby wrote:
On Thu, Nov 04, 2021 at 09:46:49AM +0100, Andreas Seltenreich wrote:
sqlsmith triggers the following assertion when testing REL_14_STABLE:
TRAP: FailedAssertion("a1 <= a2", File: "brin_minmax_multi.c",
Line: 1879, PID: 631814)
I can reproduce it with the following query on a fresh regression
database:
insert into public.brintest_multi (float4col) values (real 'nan');
The branch was at f6162c020c while testing, backtrace below.
I couldn't reproduce this, but it reminds me of this one, which we
also had
trouble reproducing.
I can reproduce that just fine - all I had to do was 'make installcheck'
and then connect to the regression db and run the insert.
It seems to be a simple case of confusion in handling of NaN values. We
do sort them correctly (by calling float4_lt), but given two values
arg1 = nan (0x400000)
arg2 = 0.0909090936
then a simple comparison does not give the expected result
(arg1 < arg2)
(arg1 == arg2)
(arg1 > arg2)
all evaluate to false, which is why the assert fails. So I guess the
distance function for float4 (and probably float8 too) need a couple
more lines checking NaN.
Here's a patch that should fix this. It simply handled NaN and returns
distance 0.0 for two NaN values and Infinity for NaN vs. something else.
I did check what happens with Infinity values, but I think those are
fine - we can actually calculate distance with them, and the assert
works for them too.
I'll improve the regression tests to include a couple NaN/Infinity cases
tomorrow, and then I'll get this pushed.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
