Re: [HACKERS] pgsql: Protect GIST logic that assumes penalty values can't be negative

2011-06-01 Thread Alexander Korotkov
On Wed, Jun 1, 2011 at 3:57 AM, Greg Stark wrote: > I guess it was obvious but that was "expect the *penalty* function to > obey the triangle inequality" > Actually, penalty function for boxes is even not commutative. Fox example: A = ((0,0)-(1,1)) B = ((0,0)-(2,2)) penalty(A,B) = 3 penalty(B,A

Re: [HACKERS] pgsql: Protect GIST logic that assumes penalty values can't be negative

2011-05-31 Thread Greg Stark
On Tue, May 31, 2011 at 4:11 PM, Tom Lane wrote: >> Do gistchoose et al expect the triangle function to obey the triangle >> inequality? > > Don't think so. > I guess it was obvious but that was "expect the *penalty* function to obey the triangle inequality" -- greg -- Sent via pgsql-hackers

Re: [HACKERS] pgsql: Protect GIST logic that assumes penalty values can't be negative

2011-05-31 Thread Tom Lane
Greg Stark writes: > On Tue, May 31, 2011 at 2:54 PM, Tom Lane wrote: >>  Prevent problems by clamping negative penalty values to >> zero.  (Just to be really sure, I also made it force NaNs to zero.) > Do gistchoose et al expect the triangle function to obey the triangle > inequality? Don't th

Re: [HACKERS] pgsql: Protect GIST logic that assumes penalty values can't be negative

2011-05-31 Thread Greg Stark
On Tue, May 31, 2011 at 2:54 PM, Tom Lane wrote: >  Prevent problems by clamping negative penalty values to > zero.  (Just to be really sure, I also made it force NaNs to zero.) Do gistchoose et al expect the triangle function to obey the triangle inequality? If so isn't it possible treating NaNs