Re: [HACKERS] Index/Function organized table layout

2003-10-05 Thread Mike Mascari
Hannu Krosing wrote: > Greg Stark kirjutas P, 05.10.2003 kell 00:17: > >>I've never seen anyone use this feature, and I never seriously considered it >>myself. It sort of has the feel of an antiquated feature that traded too much >>flexibility and abstraction for raw performance on very slow disk

Re: [HACKERS] Index/Function organized table layout

2003-10-05 Thread Hannu Krosing
Greg Stark kirjutas P, 05.10.2003 kell 00:17: > I've never seen anyone use this feature, and I never seriously considered it > myself. It sort of has the feel of an antiquated feature that traded too much > flexibility and abstraction for raw performance on very slow disk hardware. Read "A Conve

Re: [PERFORM] COUNT(*) again (was Re: [HACKERS] Index/Function organized table layout)

2003-10-04 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> I think that's not happening, conditionally or otherwise. The atomicity >> problems alone are sufficient reason why not, even before you look at >> the performance issues. > What are the atomicity problems of adding a create/expire xi

Re: [HACKERS] Index/Function organized table layout

2003-10-04 Thread Tom Lane
Greg Stark <[EMAIL PROTECTED]> writes: > So I'm a bit confused about the term "Clustering". It seems Postgres uses it > to mean simply ordering the tuple storage within an otherwise normal table. > However in other databases it seems to mean something more complex. My take is that "clustering" me

Re: [HACKERS] Index/Function organized table layout

2003-10-04 Thread Greg Stark
James Rogers <[EMAIL PROTECTED]> writes: > On 10/2/03 11:34 PM, "Hannu Krosing" <[EMAIL PROTECTED]> wrote: > > > > What I actually thought I was describing is how CLUSTER should work in a > > postgres flavour of MVCC storage ;). Not the CLUSTER command, but the > > whole feature. > > > Yeah, I

Re: [PERFORM] COUNT(*) again (was Re: [HACKERS] Index/Function organized table layout)

2003-10-04 Thread Tom Lane
Hannu Krosing <[EMAIL PROTECTED]> writes: > The point I was trying to make was that faster count(*)'s is just a side > effect. If we could (conditionally) keep visibility info in indexes, I think that's not happening, conditionally or otherwise. The atomicity problems alone are sufficient reason

Re: [HACKERS] Index/Function organized table layout

2003-10-04 Thread James Rogers
On 10/2/03 11:34 PM, "Hannu Krosing" <[EMAIL PROTECTED]> wrote: > James Rogers kirjutas N, 02.10.2003 kell 23:44: >> Not exactly. What you are describing is more akin to partitioning or >> hash-organized tables i.e. sorting insert/update tuples to various pages >> according to some hash function. >

COUNT(*) again (was Re: [HACKERS] Index/Function organized table layout)

2003-10-04 Thread Tom Lane
Hannu Krosing <[EMAIL PROTECTED]> writes: > Christopher Browne kirjutas R, 03.10.2003 kell 00:57: >> A while back I outlined how this would have to be done, and for it to >> be done efficiently, it would be anything BUT simple. > Could this be made a TODO item, perhaps with your attack plan. I

Re: [HACKERS] Index/Function organized table layout (from Re:

2003-10-04 Thread Hannu Krosing
Christopher Browne kirjutas R, 03.10.2003 kell 00:57: > [EMAIL PROTECTED] (Jean-Luc Lachance) writes: > > That's one of the draw back of MVCC. > > I once suggested that the transaction number and other house keeping > > info be included in the index, but was told to forget it... > > It would solv

Re: [HACKERS] Index/Function organized table layout

2003-10-02 Thread Hannu Krosing
James Rogers kirjutas N, 02.10.2003 kell 23:44: > On Thu, 2003-10-02 at 12:09, Hannu Krosing wrote: > > So what you really need is the CLUSTER command to leave pages half-empty > > and the tuple placement logic on inserts/updates to place new tuples > > near the place where they would be placed by

Re: [HACKERS] Index/Function organized table layout

2003-10-02 Thread Alvaro Herrera Munoz
On Thu, Oct 02, 2003 at 10:09:12PM +0300, Hannu Krosing wrote: > So what you really need is the CLUSTER command to leave pages half-empty > and the tuple placement logic on inserts/updates to place new tuples > near the place where they would be placed by CLUSTER. I.e. the code that > does actual

Re: [HACKERS] Index/Function organized table layout

2003-10-02 Thread James Rogers
On Thu, 2003-10-02 at 12:09, Hannu Krosing wrote: > So what you really need is the CLUSTER command to leave pages half-empty > and the tuple placement logic on inserts/updates to place new tuples > near the place where they would be placed by CLUSTER. I.e. the code that > does actual inserting shou

Re: [HACKERS] Index/Function organized table layout

2003-10-02 Thread Hannu Krosing
James Rogers kirjutas N, 02.10.2003 kell 20:50: > To give a real world example, a standard query on one of our tables that > has not been CLUSTER-ed recently (i.e. within the last several days) > generates an average of ~2,000 cache misses. Recently CLUSTER-ed, it > generates ~0 cache misses on a

Re: [HACKERS] Index/Function organized table layout

2003-10-02 Thread James Rogers
On Wed, 2003-10-01 at 08:37, Tom Lane wrote: > I think you'd need to do some basic architectural work first. Right now > we have a clean API for index access methods, but there is no comparable > abstraction layer for heaps (tables). It'd probably be necessary to > create such a layer in order to

Re: [HACKERS] Index/Function organized table layout

2003-10-02 Thread James Rogers
On Wed, 2003-10-01 at 09:29, Alvaro Herrera wrote: > On Wed, Oct 01, 2003 at 11:37:38AM -0400, Tom Lane wrote: > > Hm, are you sure that smarter buffer management wouldn't serve the > > purpose? > > It doesn't help when there a lot of access locality in searching. In my > case I want to select so

Re: [HACKERS] Index/Function organized table layout

2003-10-01 Thread Alvaro Herrera
On Wed, Oct 01, 2003 at 11:37:38AM -0400, Tom Lane wrote: > James Rogers <[EMAIL PROTECTED]> writes: > > Both of these things really are attempts to address the same basic problem, > > which is optimizing the number of buffers a given query uses by making the > > tables layout reflect typical quer

Re: [HACKERS] Index/Function organized table layout

2003-10-01 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > As for other indexes, I'm not sure why you say this precludes the use of > other indexes. The only thing they have to do is keep pointers to index > elements, instead of heap elements. Doesn't sound impossible to me. However, btree feels free to move

Re: [HACKERS] Index/Function organized table layout

2003-10-01 Thread Tom Lane
James Rogers <[EMAIL PROTECTED]> writes: > Now, I've actually hacked commercial MVCC engines back in the day, and am > comfortable playing around in database internals. I have an "itch to > scratch" for improving the scalability of Really Large Tables by explicitly > allowing control of table layo

Re: [HACKERS] Index/Function organized table layout

2003-10-01 Thread Alvaro Herrera
On Tue, Sep 30, 2003 at 11:31:26PM -0700, James Rogers wrote: > The problem: My working set is typically several million rows (and growing) > at any one time, which has a tendency to thrash the buffers mercilessly. > Records are inserted in an order that does not reflect typical retrieval > such t