Re: [HACKERS] prefix btree implementation

2005-10-07 Thread Bruce Momjian
OK, TODO updated: * Consider compressing indexes by storing key values duplicated in several rows as a single index entry This is difficult because it requires datatype-specific knowledge. --- Simon Riggs wrote: > On

Re: [HACKERS] prefix btree implementation

2005-10-07 Thread Simon Riggs
On Thu, 2005-10-06 at 22:43 -0400, Bruce Momjian wrote: > Jim C. Nasby wrote: > > On Wed, Oct 05, 2005 at 03:40:43PM -0700, Qingqing Zhou wrote: > > > We do the prefix sharing when we build up index only, never on the fly. > > > > So are you saying that inserts of new data wouldn't make any use of

Re: [HACKERS] prefix btree implementation

2005-10-06 Thread Qingqing Zhou
On Thu, 6 Oct 2005, Jim C. Nasby wrote: > On Wed, Oct 05, 2005 at 03:40:43PM -0700, Qingqing Zhou wrote: > > We do the prefix sharing when we build up index only, never on the fly. > > So are you saying that inserts of new data wouldn't make any use of > this? ISTM that greatly reduces the usefu

Re: [HACKERS] prefix btree implementation

2005-10-06 Thread Bruce Momjian
Jim C. Nasby wrote: > On Wed, Oct 05, 2005 at 03:40:43PM -0700, Qingqing Zhou wrote: > > We do the prefix sharing when we build up index only, never on the fly. > > So are you saying that inserts of new data wouldn't make any use of > this? ISTM that greatly reduces the usefulness, though I'm not

Re: [HACKERS] prefix btree implementation

2005-10-06 Thread Jim C. Nasby
On Wed, Oct 05, 2005 at 03:40:43PM -0700, Qingqing Zhou wrote: > We do the prefix sharing when we build up index only, never on the fly. So are you saying that inserts of new data wouldn't make any use of this? ISTM that greatly reduces the usefulness, though I'm not objecting because compression

Re: [HACKERS] prefix btree implementation

2005-10-06 Thread Simon Riggs
On Thu, 2005-10-06 at 09:38 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > It might be worth teaching the optimiser that if it has an index on an > > immutable function that if we have WHERE x = k and a functional index on > > f(x) then we can access the functional index with

Re: [HACKERS] prefix btree implementation

2005-10-06 Thread Martijn van Oosterhout
On Thu, Oct 06, 2005 at 08:53:25AM +0100, Simon Riggs wrote: > It would be possible to compress on similar values, since we know the > output of the comparison in the final stage of the sort of the index > build. That wouldn't need to rely upon anything to do with the datatype, > since "they are eq

Re: [HACKERS] prefix btree implementation

2005-10-06 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > It might be worth teaching the optimiser that if it has an index on an > immutable function that if we have WHERE x = k and a functional index on > f(x) then we can access the functional index with > f(x) = f(k), as long as we also reapply the original WHE

Re: [HACKERS] prefix btree implementation

2005-10-06 Thread Simon Riggs
On Wed, 2005-10-05 at 00:50 -0400, Tom Lane wrote: > Qingqing Zhou <[EMAIL PROTECTED]> writes: > > 1/ What types of prefix compression shall we support? > > Given the requirement of datatype independence, this idea seems a > complete nonstarter to me... It would be possible to compress on similar

Re: [HACKERS] prefix btree implementation

2005-10-05 Thread Junji TERAMOTO
Hello all, I also was examining a similar compression method just. Qingqing Zhou wrote: > We can find a way to handle the above case, but it is better to find a > general way to handle any data types(include UDT). "Each type optionally > provide the required routines" could be a way, more detai

Resultset duplicates (was Re: [HACKERS] prefix btree implementation)

2005-10-05 Thread Richard Huxton
Qingqing Zhou wrote: Oracle 9 uses the grammar like this: CREATE INDEX ... [ COMPRESS ] So it gives the flexibility of choosing optimal number of coulumns to the user. The script mentioned in the article guesses the optimal number by estimating the size of each choice. But I am thinking

Re: [HACKERS] prefix btree implementation

2005-10-05 Thread Qingqing Zhou
"Alvaro Herrera" <[EMAIL PROTECTED]> wrote > On Wed, Oct 05, 2005 at 12:50:41AM -0400, Tom Lane wrote: >> Qingqing Zhou <[EMAIL PROTECTED]> writes: >> > 1/ What types of prefix compression shall we support? >> >> Given the requirement of datatype independence, this idea seems a >> complete nonstar

Re: [HACKERS] prefix btree implementation

2005-10-05 Thread Qingqing Zhou
"Bricklen Anderson" <[EMAIL PROTECTED]> wrote > > Oracle implements something similar called index compression, but I > believe it > is only for common column values. I haven't checked in versions>9r1 so > maybe > there are other options implemented by now. > > Jonathan Lewis describes some pros

Re: [HACKERS] prefix btree implementation

2005-10-05 Thread Bricklen Anderson
Qingqing Zhou wrote: > I am not sure if this idea was mentioned before. > > The basic prefix btree idea is quite straightforward, i.e., try to > compress the key items within a data page by sharing the common prefix. > Thus the fanout of the page is increased and the benefits is obvious > theorect

Re: [HACKERS] prefix btree implementation

2005-10-05 Thread Alvaro Herrera
On Wed, Oct 05, 2005 at 12:50:41AM -0400, Tom Lane wrote: > Qingqing Zhou <[EMAIL PROTECTED]> writes: > > 1/ What types of prefix compression shall we support? > > Given the requirement of datatype independence, this idea seems a > complete nonstarter to me... How about having each type optionall

Re: [HACKERS] prefix btree implementation

2005-10-04 Thread Tom Lane
Qingqing Zhou <[EMAIL PROTECTED]> writes: > 1/ What types of prefix compression shall we support? Given the requirement of datatype independence, this idea seems a complete nonstarter to me... regards, tom lane ---(end of broadcast)

[HACKERS] prefix btree implementation

2005-10-04 Thread Qingqing Zhou
I am not sure if this idea was mentioned before. The basic prefix btree idea is quite straightforward, i.e., try to compress the key items within a data page by sharing the common prefix. Thus the fanout of the page is increased and the benefits is obvious theorectically. Consider the following