Re: [HACKERS] GiST API Adancement

2017-06-15 Thread Andrew Borodin
Hi, Dong! 2017-06-15 21:19 GMT+05:00 Yuan Dong : > I'm going to hack on my own. With the help of Andrew Borodin, I want to > start the project with adding a third state to collision check. The third > state is that: > subtree is totally within the query. In this case, GiST scan can scan down > su

Re: [HACKERS] GiST support for UUIDs

2016-11-29 Thread Chris Bandy
On Tue, Nov 29, 2016 at 1:13 PM, Tom Lane wrote: > Pushed with that change and some other mostly-cosmetic tweaking. Thank you for addressing all those issues, Tom! I tested some exclusion constraints that are interesting to me, and everything seems to be working well. -- Chris -- Sent via pgs

Re: [HACKERS] GiST support for UUIDs

2016-11-29 Thread Tom Lane
I wrote: > I'm kind of inclined to change uuid_parts_distance to just convert > a given pg_uuid_t to "double" and then apply penalty_num(), as is > done in gbt_macad_penalty. Pushed with that change and some other mostly-cosmetic tweaking. One not too cosmetic fix was that gbt_uuid_union was decl

Re: [HACKERS] GiST support for UUIDs

2016-11-29 Thread Tom Lane
Chris Bandy writes: > [ btree_gist_uuid_8.patch ] Um ... is there a reason why the penalty logic in gbt_uuid_penalty() is completely unlike that for any other btree_gist module? As best I can tell from the (admittedly documentation-free) code elsewhere, the penalty ought to be proportional to th

Re: [HACKERS] GiST support for UUIDs

2016-11-28 Thread Chris Bandy
On Mon, Nov 28, 2016 at 4:04 PM, Tom Lane wrote: > > What I would suggest is that you forget the union hack and just use > memcmp in all the comparison functions. It's not likely to be worth > the trouble to try to get those calls to be safely aligned. The > only place where you need go to any t

Re: [HACKERS] GiST support for UUIDs

2016-11-28 Thread Tom Lane
Adam Brusselback writes: > [ btree_gist_uuid_7.patch ] I spent awhile looking at this. I have exactly no faith that it won't crash on alignment-picky hardware, because this declaration: union pg_uuid_t { unsigned char data[UUID_LEN]; uint64 v64[2]; }; is not the same as the exi

Re: [HACKERS] GiST support for UUIDs

2016-11-22 Thread Haribabu Kommi
On Wed, Nov 2, 2016 at 3:49 AM, Adam Brusselback wrote: > So I apologize in advance if I didn't follow the processes exactly, I was > going to attempt to review this to move it along, but ran into issues > applying the patch cleanly to master. I fixed the issues I was having > applying it, and c

Re: [HACKERS] GiST support for UUIDs

2016-11-01 Thread Adam Brusselback
So I apologize in advance if I didn't follow the processes exactly, I was going to attempt to review this to move it along, but ran into issues applying the patch cleanly to master. I fixed the issues I was having applying it, and created a new patch (attached). Managed to test it out after I got

Re: [HACKERS] GiST penalty functions [PoC]

2016-10-02 Thread Andrew Borodin
> This thread has basically died, so marking as returned with feedback. Well, not exactly died, but it's kind of in blind lead. I'll summarize a bit for future references: 1. cube extension for indexing lowered dimensionality data (2d in 3d) is broken. Queries effectively will degrade to full inde

Re: [HACKERS] GiST penalty functions [PoC]

2016-10-02 Thread Michael Paquier
On Thu, Sep 22, 2016 at 3:45 AM, Andrew Borodin wrote: > [blah] > > Practically, this algorithm cannot be implemented in current GiST API > (only if we provide non-penalty-based choose subtree function, > optional for GiST extension), but it certainly has scientific value. This thread has basical

Re: [HACKERS] GiST support for UUIDs

2016-10-02 Thread Michael Paquier
On Wed, Aug 19, 2015 at 1:25 PM, Paul A Jungwirth wrote: > This is my first patch, so my apologies if anything is missing. I went > the guidelines and I think I have everything covered. :-) I am moving this patch to next CF, removing Julien Rouhaud and Teodor Sigaev as reviewers because they have

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-21 Thread Andrew Borodin
Hi Heikki! > Got a link for a description of the RR*-tree? Would be good to reference it > in the patch comments, too. Well, as for now, the best way to reach the paper is DOI 10.1145/1559845.1559929 http://sci-hub.cc/ Authors in conversations clearly stated that they endorse (not sure in correct

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-21 Thread Heikki Linnakangas
On 09/14/2016 07:57 PM, Andrew Borodin wrote: Here is v6 of cube penalty patch. There was a warning about unused edge function under systems without __STDC_IEC_559__ defined, this patch fixes it. Thanks! Reading through this thread again, I think we got a bit side-tracked with this float bit

Re: [HACKERS] GiST: interpretation of NaN from penalty function

2016-09-15 Thread Andrew Borodin
> Certainly, "NaN means infinity", as your patch proposes, has no more basis to > it than "NaN means zero". You are absolutley right. Now I see that best interpretation is "NaN means NaN". Seems like we need only drop a check. Nodes with NaN penalties will be considered even worser than those with

Re: [HACKERS] GiST: interpretation of NaN from penalty function

2016-09-14 Thread Tom Lane
Andrew Borodin writes: > Currently GiST treats NaN penalty as zero penalty, in terms of > generalized tree this means "perfect fit". I think that this situation > should be considered "worst fit" instead. On what basis? It seems hard to me to make any principled argument here. Certainly, "NaN me

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-14 Thread Andrew Borodin
Here is v6 of cube penalty patch. There was a warning about unused edge function under systems without __STDC_IEC_559__ defined, this patch fixes it. Regards, Andrey Borodin. diff --git a/contrib/cube/cube.c b/contrib/cube/cube.c index 3feddef..ad868ac 100644 --- a/contrib/cube/cube.c +++ b/contr

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-13 Thread Andrew Borodin
Hi hackers! Here is gist cube penaly patch v5. Changes: 1. union version of pack_float() is used, without warings and with minimum asm commands emitted 2. test for __STDC_IEC_559__ is added, this is a standard way of C99 to determine float compliance with IEEE754. ANSI-only compiler+libs, along wi

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-10 Thread Andrew Borodin
>Personally I wouldn't be very happy about an IEEE754 assumption. Ok, so let's avoid autoconf man. There is no real explanations of the ground for this assumption, just a reference to paper of David Goldberg (and there is no metion about IEEE754 is absoulte everywhere). BTW, may be we can ask David

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-09 Thread Greg Stark
On Thu, Sep 8, 2016 at 9:29 AM, Andrew Borodin wrote: >>autoconf check for IEEE 754 floats > Autoconf man says folowing: >>it is safe to assume IEEE-754 in most portable code these days > https://www.gnu.org/software/autoconf/manual/autoconf.html#Floating-Point-Portability Personally I wouldn't

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-09 Thread Михаил Бахтерев
If you are still interested in. Here are 3 versions of pack-float. The union version of pack-float should run faster. The code is simpler, the dependencies are easier. But it may be less accurate or even wrong, as for signed integers (x>>2) and (x/4) are not the same. Consider x = -1. You may try

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-09 Thread Михаил Бахтерев
Yes. You are right, ANSI C allows only load-time initializers. Attached ANSI compatible version leads to the same assembly. And let me suggest a bit-twiddling version as well. It gives 12 instructions, instead of 13. 12 is better, as modern x86 CPU will fetch them at most in 3 cycles, one less tha

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-08 Thread Andrey Borodin
Thank you for your attention to details, Mikhail. pack_float_good() looks good. But I'm not sure inline strict init is allowed under ansi C. Converting to regular ancient form b.fp = v; won't change compile result, would it? Regards, Andrey Borodin. -- Sent via pgsql-hackers mailing list (pgs

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-08 Thread Михаил Бахтерев
Excuse me for intervention. It depends. For instance, i run PostgreSQL on the modern MIPS CPU, which does not have sqrt support. But you are right, it is supported in most cases. And if execution speed of this very fuction is of concern, sqrtf(x) should be used instead of sqrt(x). Despite this,

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-08 Thread Andrew Borodin
>BTW, would someone explain to me why using a float here will not fail >catastrophically for inputs outside the range of float? Indeed, it will. And that's one of the reason I'm considering advancing GiST API instead of advancing extensions. It won't crash, but will produce terrible index, not s

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-08 Thread Tom Lane
Heikki Linnakangas writes: > BTW, I would be OK with the bit-twiddling hack, if we had an autoconf > check for IEEE 754 floats, and a graceful fallback for other systems. I would still object to the version submitted, on the grounds of the compiler warnings it causes. Possibly those could be a

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-08 Thread Andrew Borodin
>autoconf check for IEEE 754 floats Autoconf man says folowing: >it is safe to assume IEEE-754 in most portable code these days https://www.gnu.org/software/autoconf/manual/autoconf.html#Floating-Point-Portability > A union might be more readable Here is union version of the patch. It's slower 10%

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-08 Thread Heikki Linnakangas
On 09/08/2016 09:39 AM, Михаил Бахтерев wrote: Excuse me for intervention. It depends. For instance, i run PostgreSQL on the modern MIPS CPU, which does not have sqrt support. But you are right, it is supported in most cases. And if execution speed of this very fuction is of concern, sqrtf(x) s

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-07 Thread Tom Lane
Heikki Linnakangas writes: > Unfortunately, sqrt(x) isn't very cheap. You'd be surprised: sqrt is built-in on most modern hardware. On my three-year-old workstation, sqrt(x) seems to take about 2.6ns. For comparison, the pack_float version posted in takes 3.9ns (and draws multiple compiler war

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-07 Thread Heikki Linnakangas
On 09/07/2016 09:20 PM, Andrew Borodin wrote: Well, arithmetics is too fragile. This version of float packing with arithmetical packaging static float pack_float(float actualValue, int realm) { double max,min; max = FLT_MAX / ( 8 >> realm ); min = FLT_MAX / ( 16 >> realm ); if( realm == 0 ) min

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-07 Thread Andrew Borodin
Well, arithmetics is too fragile. This version of float packing with arithmetical packaging static float pack_float(float actualValue, int realm) { double max,min; max = FLT_MAX / ( 8 >> realm ); min = FLT_MAX / ( 16 >> realm ); if( realm == 0 ) min = 0; /* squeeze the actual value between min and

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-07 Thread Andrew Borodin
Oh, sorry, made one more attemp and now I see your algorithm differently. So you propose to use oE and oV as a markers of borders for what I call Realm. But there may be very little significant bits in one of this ranges. pg_sphere and PostGiS extensions tried to use 1 as a marker, with alike form

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-06 Thread Heikki Linnakangas
On 09/07/2016 09:42 AM, Andrew Borodin wrote: 2. Your algorithm, among loosing some bits of precision (which is absolutely acceptable - we have like 31 of them and that’s a lot) rely on false assumption. We compare tuples on page not by MBR of inserted tuple, but by MBR of tuple on page, which is

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-06 Thread Andrew Borodin
Hi Heikki! Thank you for reviewing the code, it's always inspiring when a work is noted (: >Looking at the code, by "margin", you mean the sum of all "edges", i.e. of each dimension, of the cube. Indeed. As far as I remember, this is a terminology of old R*-tree paper. Now they use the word "perim

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-06 Thread Heikki Linnakangas
On 08/29/2016 09:04 AM, Andrew Borodin wrote: In this message I address only first problem. I want to construct a penalty function that will choose: 1.Entry with a zero volume and smallest margin, that can accommodate insertion tuple without extension, if one exists; 2.Entry with smallest

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-01 Thread Andrew Borodin
Here is new version of the patch. Now function pack_float is commented better. All inline keywords are removed. I haven't found any performance loss for that. Remove of static's showed 1%-7% performance loss in SELECT computation (3 test runs), so I left statics where they are. Actually, to avoid

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-01 Thread Andrew Borodin
Thank you for your coments, Tom. > Modern compilers are generally able to make their own decisions about it, and > trying to put your thumb on the scales this heavily is not likely to improve > the code. Well, I have tested that combination of "static inline" affets performance of index build on

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-01 Thread Tom Lane
Andrew Borodin writes: > Does every supported Postgres platform conforms to IEEE 754 floating > point specification? Possibly, but it is against project policy to allow code to assume so. That pack_float function is absolutely not acceptable IMV, and would still be if it were adequately documente

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-01 Thread Andrew Borodin
Hi hackers! Here is the new patch version. With the help of Mikhail Bakhterev I've optimized subroutines of the penalty function. Index build time now is roughly equivalent to build time before patch (test attached to thread start). Time of SELECT statement execution is reduced by 40%. Changes in

Re: [HACKERS] GiST index build versus NaN coordinates

2016-07-16 Thread Tom Lane
Andreas Seltenreich writes: > I wrote: >> Sounds like some fuzz testing with nan/infinity is in order. > related fallout: close_ps returns a NULL pointer with NaNs around: > select close_ps('(nan,nan)', '(nan,nan),(nan,nan)'); > -- TRAP: FailedAssertion("!(result != ((void *)0))", File: "geo_ops.

Re: [HACKERS] GiST index build versus NaN coordinates

2016-07-16 Thread Andreas Seltenreich
I wrote: > Sounds like some fuzz testing with nan/infinity is in order. related fallout: close_ps returns a NULL pointer with NaNs around: select close_ps('(nan,nan)', '(nan,nan),(nan,nan)'); -- TRAP: FailedAssertion("!(result != ((void *)0))", File: "geo_ops.c", Line: 2860) regards, Andreas

Re: [HACKERS] GiST index build versus NaN coordinates

2016-07-12 Thread Tom Lane
I wrote: > Andreas Seltenreich writes: >> The infinite loop from the bug report was triggered. Further, two >> previously unseen errors are logged: >> ERROR: timestamp cannot be NaN >> ERROR: getQuadrant: impossible case >> The first is porbably as boring as it gets, the second one is from the >

Re: [HACKERS] GiST index build versus NaN coordinates

2016-07-12 Thread Tom Lane
I wrote: > I looked into the problem reported in bug #14238, > https://www.postgresql.org/message-id/20160708151747.1426.60...@wrigleys.postgresql.org > I think that probably the most reasonable answer is to replace all the > raw "double" comparisons in this code with float8_cmp_internal() or > som

Re: [HACKERS] GiST index build versus NaN coordinates

2016-07-12 Thread Tom Lane
Andreas Seltenreich writes: > Tom Lane writes: >> More generally, this example makes me fearful that NaN coordinates in >> geometric values are likely to cause all sorts of issues. It's too late >> to disallow them, probably, but I wonder how can we identify other bugs >> of this ilk. > Sounds l

Re: [HACKERS] GiST index build versus NaN coordinates

2016-07-12 Thread Andreas Seltenreich
Tom Lane writes: > More generally, this example makes me fearful that NaN coordinates in > geometric values are likely to cause all sorts of issues. It's too late > to disallow them, probably, but I wonder how can we identify other bugs > of this ilk. Sounds like some fuzz testing with nan/infin

Re: [HACKERS] GiST index build versus NaN coordinates

2016-07-11 Thread Tom Lane
Emre Hasegeli writes: >> I think that probably the most reasonable answer is to replace all the >> raw "double" comparisons in this code with float8_cmp_internal() or >> something that's the moral equivalent of that. Does anyone want to >> propose something else? > We can probably get away by ch

Re: [HACKERS] GiST index build versus NaN coordinates

2016-07-11 Thread Emre Hasegeli
> I think that probably the most reasonable answer is to replace all the > raw "double" comparisons in this code with float8_cmp_internal() or > something that's the moral equivalent of that. Does anyone want to > propose something else? We can probably get away by changing the comparison on the

Re: [HACKERS] GiST support for UUIDs

2015-06-25 Thread Tom Lane
Paul A Jungwirth writes: > On Thu, Jun 25, 2015 at 8:06 AM, Tom Lane wrote: >> Paul A Jungwirth writes: >>> I'm interested in adding GiST support for the UUID column type >>> . . . . So I'm curious where this change would go? >> btree_gist, I'd think > Okay, thank you for your answer! I was wo

Re: [HACKERS] GiST support for UUIDs

2015-06-25 Thread Paul A Jungwirth
On Thu, Jun 25, 2015 at 8:06 AM, Tom Lane wrote: > Paul A Jungwirth writes: >> I'm interested in adding GiST support for the UUID column type >> . . . . So I'm curious where this change would go? > btree_gist, I'd think Okay, thank you for your answer! I was worried about the effects of having b

Re: [HACKERS] GiST support for UUIDs

2015-06-25 Thread Tom Lane
Paul A Jungwirth writes: > I'm interested in adding GiST support for the UUID column type from > the uuid-ossp extension. This has been requested and attempted before: > I've used Postgres for a long time, but I've only dabbled a bit in the > source code. So I'm curious where this change would go?

Re: [HACKERS] GiST KNN Crasher

2015-05-21 Thread Tom Lane
Paul Ramsey writes: > I'm implementing the recheck functionality for PostGIS so we can > support it when 9.5 comes out, and came across this fun little > crasher. Should be fixed as of git tip. Thanks for the report! regards, tom lane -- Sent via pgsql-hackers mailing

Re: [HACKERS] GiST KNN Crasher

2015-05-21 Thread Tom Lane
I wrote: > Heikki Linnakangas writes: >> I think that trying to find the equivalence member in >> create_index_scan() is too fragile. > I agree; will contemplate how to do this better. I think probably what we ought to do here is just use exprType() of the ORDER BY expression. There are opclas

Re: [HACKERS] GiST KNN Crasher

2015-05-21 Thread Tom Lane
Heikki Linnakangas writes: > The find_ec_member_for_expr() call in create_indexscan_plan() fails to > find the equivalence member for the path key. > match_clause_to_ordering_op() found the match by commuting the operator, > but code in create_indexscan_plan() doesn't take that into account. R

Re: [HACKERS] GiST KNN Crasher

2015-05-21 Thread Heikki Linnakangas
On 05/21/2015 11:28 PM, Tom Lane wrote: Paul Ramsey writes: I'm implementing the recheck functionality for PostGIS so we can support it when 9.5 comes out, and came across this fun little crasher. Hmm, works in 9.3 and 9.4, so it's been broken recently. It was clearly broken by knn-with-rec

Re: [HACKERS] GiST KNN Crasher

2015-05-21 Thread Tom Lane
Paul Ramsey writes: > I'm implementing the recheck functionality for PostGIS so we can > support it when 9.5 comes out, and came across this fun little > crasher. Hmm, works in 9.3 and 9.4, so it's been broken recently. Will look, thanks! regards, tom lane -- Sent via

Re: [HACKERS] GiST kNN search queue (Re: KNN-GiST with recheck)

2015-02-17 Thread Heikki Linnakangas
On 02/17/2015 02:56 PM, Alexander Korotkov wrote: Hi! On Mon, Dec 22, 2014 at 1:07 PM, Heikki Linnakangas wrote: Ok, thanks for the review! I have committed this, with some cleanup and more comments added. ISTM that checks in pairingheap_GISTSearchItem_cmp is incorrect. This function shoul

Re: [HACKERS] GiST kNN search queue (Re: KNN-GiST with recheck)

2015-02-17 Thread Alexander Korotkov
Hi! On Mon, Dec 22, 2014 at 1:07 PM, Heikki Linnakangas wrote: > Ok, thanks for the review! I have committed this, with some cleanup and > more comments added. > ISTM that checks in pairingheap_GISTSearchItem_cmp is incorrect. This function should perform inverse comparison. Thus, if item a sho

Re: [HACKERS] GiST kNN search queue (Re: KNN-GiST with recheck)

2014-12-22 Thread Heikki Linnakangas
On 12/20/2014 10:50 PM, Peter Geoghegan wrote: On Wed, Dec 17, 2014 at 6:07 AM, Heikki Linnakangas wrote: How about adding a src/backend/lib/README for that, per attached? I took a quick look at this. Some observations: * I like the idea of adding a .../lib README. However, the README fails

Re: [HACKERS] GiST kNN search queue (Re: KNN-GiST with recheck)

2014-12-20 Thread Peter Geoghegan
On Wed, Dec 17, 2014 at 6:07 AM, Heikki Linnakangas wrote: > How about adding a src/backend/lib/README for that, per attached? I took a quick look at this. Some observations: * I like the idea of adding a .../lib README. However, the README fails to note that ilist.c implements an *integrated* l

Re: [HACKERS] GiST kNN search queue (Re: KNN-GiST with recheck)

2014-12-17 Thread Heikki Linnakangas
On 12/15/2014 03:14 PM, Andres Freund wrote: If we add another heap implementation we probably should at least hint at the different advantages somewhere. How about adding a src/backend/lib/README for that, per attached? - Heikki diff --git a/src/backend/lib/Makefile b/src/backend/lib/Makefil

Re: [HACKERS] GiST kNN search queue (Re: KNN-GiST with recheck)

2014-12-16 Thread Heikki Linnakangas
On 12/15/2014 11:59 PM, Jeff Janes wrote: On Mon, Dec 15, 2014 at 5:08 AM, Heikki Linnakangas wrote: Here's a new version of the patch. It now uses the same pairing heap code that I posted in the other thread ("advance local xmin more aggressivley", http://www.postgresql.org/message-id/5488ac

Re: [HACKERS] GiST kNN search queue (Re: KNN-GiST with recheck)

2014-12-15 Thread Jeff Janes
On Mon, Dec 15, 2014 at 5:08 AM, Heikki Linnakangas wrote: Here's a new version of the patch. It now uses the same pairing heap code > that I posted in the other thread ("advance local xmin more aggressivley", > http://www.postgresql.org/message-id/5488acf0.8050...@vmware.com). The > pairingheap_

Re: [HACKERS] GiST kNN search queue (Re: KNN-GiST with recheck)

2014-12-15 Thread Andres Freund
On 2014-12-15 15:08:28 +0200, Heikki Linnakangas wrote: > +/*- > + * > + * pairingheap.c > + * A Pairing Heap implementation > + * > + * Portions Copyright (c) 2012-2014, PostgreSQL Global Development Group > + * > + * IDEN

Re: [HACKERS] GiST kNN search queue (Re: KNN-GiST with recheck)

2014-12-15 Thread Heikki Linnakangas
On 12/15/2014 03:49 AM, Michael Paquier wrote: On Thu, Dec 11, 2014 at 12:50 AM, Heikki Linnakangas wrote: On 01/28/2014 04:12 PM, Alexander Korotkov wrote: 3. A binary heap would be a better data structure to buffer the rechecked values. A Red-Black tree allows random insertions and deletio

Re: [HACKERS] GiST kNN search queue (Re: KNN-GiST with recheck)

2014-12-14 Thread Michael Paquier
On Thu, Dec 11, 2014 at 12:50 AM, Heikki Linnakangas wrote: > On 01/28/2014 04:12 PM, Alexander Korotkov wrote: >>> >>> >3. A binary heap would be a better data structure to buffer the >>> > rechecked >>> >values. A Red-Black tree allows random insertions and deletions, but in >>> >this case you n

Re: [HACKERS] GiST kNN search queue (Re: KNN-GiST with recheck)

2014-12-10 Thread Heikki Linnakangas
On 12/10/2014 10:59 PM, Arthur Silva wrote: It may be better to replace the lib/binaryheap altogether as it offers comparable/better performance. It's not always better. A binary heap is more memory-efficient, for starters. There are only two uses of lib/binaryheap: reorderbuffer.c and merge

Re: [HACKERS] GiST kNN search queue (Re: KNN-GiST with recheck)

2014-12-10 Thread Arthur Silva
On Wed, Dec 10, 2014 at 1:50 PM, Heikki Linnakangas wrote: > On 01/28/2014 04:12 PM, Alexander Korotkov wrote: > >> >3. A binary heap would be a better data structure to buffer the rechecked >>> >values. A Red-Black tree allows random insertions and deletions, but in >>> >this case you need to in

Re: [HACKERS] gist vacuum gist access

2014-12-07 Thread Michael Paquier
On Tue, Sep 9, 2014 at 12:47 AM, Heikki Linnakangas wrote: > On 09/08/2014 03:26 PM, Alexander Korotkov wrote: >> >> On Mon, Sep 8, 2014 at 12:51 PM, Heikki Linnakangas >> >> >>> wrote: >> >> >>> On 09/08/2014 11:19 AM, Alexander Korotkov wrote: >>> On Mon, Sep 8, 2014 at 12:08 PM, Alexander

Re: [HACKERS] GiST splitting on empty pages

2014-10-03 Thread Heikki Linnakangas
On 10/03/2014 05:03 AM, Andrew Gierth wrote: This is from Bug #11555, which is still in moderation as I type this (analysis was done via IRC). The GiST insertion code appears to have no length checks at all on the inserted entry. index_form_tuple checks for length <= 8191, with the default block

Re: [HACKERS] gist vacuum seaq access

2014-09-20 Thread Костя Кузнецов
Heikki.I have idea. when i begining vacuum i must create structure like hash table. And the first action i fill this hash table.in code this look like:for( blkno = ..; all pages; blkno++) {  if(! gistPageIsLeaf(blkno)) { for( all tuples in blkno ) { hash[ blkno that is referenced by the

Re: [HACKERS] gist vacuum seaq access

2014-09-12 Thread Heikki Linnakangas
On 09/11/2014 12:16 PM, Костя Кузнецов wrote: After discussion of gist seaq access in vaccum there are 2 issue: Heikki says : Vacuum needs to memorize the current NSN when it begins 1) how i may getting corect NSN. The "current NSN" is just current WAL insert location, so GetXLogInsertRecPtr()

Re: [HACKERS] gist vacuum gist access

2014-09-08 Thread Heikki Linnakangas
On 09/08/2014 03:26 PM, Alexander Korotkov wrote: On Mon, Sep 8, 2014 at 12:51 PM, Heikki Linnakangas wrote: On 09/08/2014 11:19 AM, Alexander Korotkov wrote: On Mon, Sep 8, 2014 at 12:08 PM, Alexander Korotkov wrote: On Mon, Sep 8, 2014 at 11:13 AM, Heikki Linnakangas < hlinnakan...

Re: [HACKERS] gist vacuum gist access

2014-09-08 Thread Alexander Korotkov
On Mon, Sep 8, 2014 at 12:51 PM, Heikki Linnakangas wrote: > On 09/08/2014 11:19 AM, Alexander Korotkov wrote: > >> On Mon, Sep 8, 2014 at 12:08 PM, Alexander Korotkov > > >> wrote: >> >> On Mon, Sep 8, 2014 at 11:13 AM, Heikki Linnakangas < >>> hlinnakan...@vmware.com> wrote: >>> >>> In the b-

Re: [HACKERS] gist vacuum gist access

2014-09-08 Thread Heikki Linnakangas
On 09/08/2014 11:19 AM, Alexander Korotkov wrote: On Mon, Sep 8, 2014 at 12:08 PM, Alexander Korotkov wrote: On Mon, Sep 8, 2014 at 11:13 AM, Heikki Linnakangas < hlinnakan...@vmware.com> wrote: In the b-tree code, we solved that problem back in 2006, so it can be done but requires a bit mor

Re: [HACKERS] gist vacuum gist access

2014-09-08 Thread Heikki Linnakangas
On 09/08/2014 11:08 AM, Alexander Korotkov wrote: On Mon, Sep 8, 2014 at 11:13 AM, Heikki Linnakangas wrote: On 09/07/2014 05:11 PM, Костя Кузнецов wrote: hello. i recode vacuum for gist index. all tests is ok. also i test vacuum on table size 2 million rows. all is ok. on my machine old va

Re: [HACKERS] gist vacuum gist access

2014-09-08 Thread Alexander Korotkov
On Mon, Sep 8, 2014 at 12:08 PM, Alexander Korotkov wrote: > On Mon, Sep 8, 2014 at 11:13 AM, Heikki Linnakangas < > hlinnakan...@vmware.com> wrote: > >> On 09/07/2014 05:11 PM, Костя Кузнецов wrote: >> >>> hello. >>> i recode vacuum for gist index. >>> all tests is ok. >>> also i test vacuum on

Re: [HACKERS] gist vacuum gist access

2014-09-08 Thread Alexander Korotkov
On Mon, Sep 8, 2014 at 11:13 AM, Heikki Linnakangas wrote: > On 09/07/2014 05:11 PM, Костя Кузнецов wrote: > >> hello. >> i recode vacuum for gist index. >> all tests is ok. >> also i test vacuum on table size 2 million rows. all is ok. >> on my machine old vaccum work about 9 second. this versio

Re: [HACKERS] gist vacuum gist access

2014-09-08 Thread Heikki Linnakangas
On 09/07/2014 05:11 PM, Костя Кузнецов wrote: hello. i recode vacuum for gist index. all tests is ok. also i test vacuum on table size 2 million rows. all is ok. on my machine old vaccum work about 9 second. this version work about 6-7 sec . review please. If I'm reading this correctly, the pat

Re: [HACKERS] GIST create index very very slow

2014-08-18 Thread Greg Stark
On Mon, Aug 18, 2014 at 5:47 PM, Robert Haas wrote: > Sounds pretty weird I recall GIST being really slow in the distant past in cases where the page split choices were really bad. Is timerange an interval? Or a Range?I wonder if the pagesplit function for some of the newish data types like range

Re: [HACKERS] GIST create index very very slow

2014-08-18 Thread Heikki Linnakangas
On 08/18/2014 07:47 PM, Robert Haas wrote: I think something's missing from your report, though, because if there is 3.5 Mb/s of write I/O and only 8 b/s of file growth, nearly all of the writes are doing something other than extending that file. Hmm. Sounds like it's churning through temporary

Re: [HACKERS] GIST create index very very slow

2014-08-18 Thread Robert Haas
On Fri, Aug 15, 2014 at 5:59 PM, worthy7 wrote: > CREATE INDEX USING GIST(timerange); > > On 1.3 million rows this took only 30 seconds. > on 70 million its already taken over a day. I swear it didn't take this long > on version 9.3 > > > Is there some kind of known bug with GIST? CPU is at 4% or

Re: [HACKERS] GiST support for inet datatypes

2014-04-09 Thread Emre Hasegeli
Tom Lane : > Committed with some additional documentation work. Thanks for > submitting this! Thank you for committing. I had not thought of using different structure for the index. It works faster with my test case, too. I am sending rebased version of the consistent operator names patch in cas

Re: [HACKERS] GiST support for inet datatypes

2014-04-08 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Tue, Apr 8, 2014 at 3:51 PM, Tom Lane wrote: > > I wrote: > >> [ inet-gist-v6.patch ] > > > > Committed with some additional documentation work. Thanks for > > submitting this! > > NICE. I'd like to tell you how excited I am about this part: >

Re: [HACKERS] GiST support for inet datatypes

2014-04-08 Thread Tom Lane
Robert Haas writes: > NICE. I'd like to tell you how excited I am about this part: > # It also handles a new network > # operator inet && inet (overlaps, a/k/a "is supernet or subnet of"), > # which is expected to be useful in exclusion constraints. > ...but I can't, because my mouth is too ful

Re: [HACKERS] GiST support for inet datatypes

2014-04-08 Thread Robert Haas
On Tue, Apr 8, 2014 at 3:51 PM, Tom Lane wrote: > I wrote: >> [ inet-gist-v6.patch ] > > Committed with some additional documentation work. Thanks for > submitting this! NICE. I'd like to tell you how excited I am about this part: # It also handles a new network # operator inet && inet (overla

Re: [HACKERS] GiST support for inet datatypes

2014-04-08 Thread Tom Lane
I wrote: > [ inet-gist-v6.patch ] Committed with some additional documentation work. Thanks for submitting this! regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mail

Re: [HACKERS] GiST support for inet datatypes

2014-04-04 Thread Andreas Karlsson
On 03/08/2014 10:40 PM, Emre Hasegeli wrote: Fourth version of the patch attached. It is rebased to the HEAD (8879fa0). Operator name formatting patch rebased on top of it. I will put the selectivity estimation patch to the next commit fest. This version of the patch does not touch to the btree_

Re: [HACKERS] GiST support for inet datatypes

2014-04-04 Thread Andres Freund
On 2014-04-04 16:50:36 +0200, Andreas Karlsson wrote: > On 04/04/2014 04:01 PM, Andres Freund wrote: > >This patch looks like it can be applied much more realistically, but it > >looks too late for 9.4. I suggest moving it to the next CF? > > If it does not change the default operator class I do n

Re: [HACKERS] GiST support for inet datatypes

2014-04-04 Thread Andreas Karlsson
On 04/04/2014 04:01 PM, Andres Freund wrote: This patch looks like it can be applied much more realistically, but it looks too late for 9.4. I suggest moving it to the next CF? If it does not change the default operator class I do not see anything preventing it from being applied to 9.4, as lo

Re: [HACKERS] GiST support for inet datatypes

2014-04-04 Thread Andres Freund
Hi, On 2014-03-08 23:40:31 +0200, Emre Hasegeli wrote: > Fourth version of the patch attached. It is rebased to the HEAD (8879fa0). > Operator name formatting patch rebased on top of it. I will put > the selectivity estimation patch to the next commit fest. > > This version of the patch does not

Re: [HACKERS] GiST support for inet datatypes

2014-03-03 Thread Tom Lane
Robert Haas writes: > On Thu, Feb 27, 2014 at 12:30 PM, Tom Lane wrote: >> The value in it is roughly the same as the reason we don't include a >> version number when dumping CREATE EXTENSION. If you had a default >> opclass in the source database, you probably want a default opclass >> in the d

Re: [HACKERS] GiST support for inet datatypes

2014-03-03 Thread Robert Haas
On Thu, Feb 27, 2014 at 12:30 PM, Tom Lane wrote: > Florian Pflug writes: >> On Feb27, 2014, at 17:56 , Tom Lane wrote: >>> That's not a bug, it's a feature, for much the same reasons that pg_dump >>> tries to minimize explicit schema-qualification. > >> I fail to see the value in this for opcla

Re: [HACKERS] GiST support for inet datatypes

2014-02-28 Thread Emre Hasegeli
2014-02-28 17:30, Florian Pflug : > Hm, what if we put the new opclass into an extension of its own, say > inet_gist, > instead of into core? It will work but I do not think it is better than adding it in core as not default. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] GiST support for inet datatypes

2014-02-28 Thread Florian Pflug
On Feb28, 2014, at 15:45 , Emre Hasegeli wrote: > The problem is that pg_dump --binary-upgrade dumps objects in the extension > on the old cluster, not just the CREATE EXTENSION statement. pg_upgrade > fails to restore them, if the new operator class already exists on the new > cluster as the defa

Re: [HACKERS] GiST support for inet datatypes

2014-02-28 Thread Emre Hasegeli
2014-02-27 18:15, Florian Pflug : >> It can be possible to update the new operator class in the new cluster >> as not default, before restore. After the restore, pg_upgrade can upgrade >> the btree_gist extension and reset the operator class as the default. >> pg_upgrade suggests to re-initdb the n

Re: [HACKERS] GiST support for inet datatypes

2014-02-28 Thread Greg Stark
On Thu, Feb 27, 2014 at 5:30 PM, Tom Lane wrote: > Indeed. The root of the problem here is that we've never really thought > about changing a type's default opclass before. I don't think that a > two-line change in pg_dump fixes all the issues this will bring up. I think we did actually do this

Re: [HACKERS] GiST support for inet datatypes

2014-02-27 Thread Tom Lane
Florian Pflug writes: > On Feb27, 2014, at 17:56 , Tom Lane wrote: >> That's not a bug, it's a feature, for much the same reasons that pg_dump >> tries to minimize explicit schema-qualification. > I fail to see the value in this for opclasses. It's certainly nice for > schema qualifications, bec

Re: [HACKERS] GiST support for inet datatypes

2014-02-27 Thread Florian Pflug
On Feb27, 2014, at 17:56 , Tom Lane wrote: > Florian Pflug writes: >> Maybe I'm missing something, but isn't the gist of the problem here that >> pg_dump won't explicitly state the operator class if it's the default? > > That's not a bug, it's a feature, for much the same reasons that pg_dump >

Re: [HACKERS] GiST support for inet datatypes

2014-02-27 Thread Tom Lane
Florian Pflug writes: > Maybe I'm missing something, but isn't the gist of the problem here that > pg_dump won't explicitly state the operator class if it's the default? That's not a bug, it's a feature, for much the same reasons that pg_dump tries to minimize explicit schema-qualification. At l

Re: [HACKERS] GiST support for inet datatypes

2014-02-27 Thread Florian Pflug
On Feb27, 2014, at 11:39 , Emre Hasegeli wrote: > 2014-02-24 17:55, Bruce Momjian : > >> pg_upgrade has _never_ modified the old cluster, and I am hesitant to do >> that now. Can we make the changes in the new cluster, or in pg_dump >> when in binary upgrade mode? > > It can be possible to upda

Re: [HACKERS] GiST support for inet datatypes

2014-02-27 Thread Emre Hasegeli
2014-02-24 17:55, Bruce Momjian : > pg_upgrade has _never_ modified the old cluster, and I am hesitant to do > that now. Can we make the changes in the new cluster, or in pg_dump > when in binary upgrade mode? It can be possible to update the new operator class in the new cluster as not default,

  1   2   3   4   >