Re: [GENERAL] clustering without locking

2008-05-03 Thread Craig Ringer
Tom Lane wrote: Anyway I think the main practical problem would be with deadlocks against other transactions trying to update/delete tuples at the same times you need to move them. Dealing with uncommitted insertions would be tricky too --- I think you'd need to wait out the inserting transacti

Re: [GENERAL] clustering without locking

2008-05-03 Thread Tom Lane
Craig Ringer <[EMAIL PROTECTED]> writes: > Begin a transaction and free the first chunk (2 tuples in this case, but > obviously many more in a real case): > --- > ..473612058 > --- > Use that freed space to store the first ordered tuples: > --- > 014736.2.58 > --

Re: [GENERAL] clustering without locking

2008-05-03 Thread Craig Ringer
Tom Lane wrote: Craig Ringer <[EMAIL PROTECTED]> writes: Later on, though, less new space would have to be allocated because more and more of the space allocated earlier to hold moved tuples would be being freed up in useful chunks that could be reused. I don't see how that works. If the min

Re: [GENERAL] clustering without locking

2008-05-03 Thread Tom Lane
Craig Ringer <[EMAIL PROTECTED]> writes: > Later on, though, less new space would have to be allocated because more > and more of the space allocated earlier to hold moved tuples would be > being freed up in useful chunks that could be reused. I don't see how that works. If the minimum size of

Re: [GENERAL] clustering without locking

2008-05-02 Thread Craig Ringer
Tom Lane wrote: Craig Ringer <[EMAIL PROTECTED]> writes: So ... is this crazy? Concurrently clustering the table by moving each record *twice*, in batches, with pauses to allow old versions to cease being visible by any live transaction? Or can it actually work? It seems to me you'd have a pr

Re: [GENERAL] clustering without locking

2008-05-02 Thread Tom Lane
Craig Ringer <[EMAIL PROTECTED]> writes: > So ... is this crazy? Concurrently clustering the table by moving each > record *twice*, in batches, with pauses to allow old versions to cease > being visible by any live transaction? Or can it actually work? It seems to me you'd have a pretty horrid b

Re: [GENERAL] clustering without locking

2008-05-02 Thread Craig Ringer
Scott Ribe wrote: Wouldn't new / updated tuples just get put in the hole, fairly rapidly un-clustering the table again? How is that different than putting them in newly-allocated space at the end of the table? It isn't, I just wasn't thinking straight. This is probably a stupid idea

Re: [GENERAL] clustering without locking

2008-05-02 Thread Scott Ribe
> Huh? A plain vacuum wouldn't fix that; a vacuum full would close up the > hole, but (a) it'd not preserve the row ordering, and (b) it'd take an > exclusive lock. OK. -- Scott Ribe [EMAIL PROTECTED] http://www.killerbytes.com/ (303) 722-0567 voice -- Sent via pgsql-general mailing list (p

Re: [GENERAL] clustering without locking

2008-05-02 Thread Tom Lane
Scott Ribe <[EMAIL PROTECTED]> writes: >> Huh? If I'm understanding you correctly you'll end up with rows in >> order, but with a really big hole in the middle of the table. I'm not >> sure if that qualifies as "clusters". > That's why he said vacuum when done. Huh? A plain vacuum wouldn't fix t

Re: [GENERAL] clustering without locking

2008-05-02 Thread Scott Ribe
> Wouldn't new / updated tuples just get put in the hole, fairly rapidly > un-clustering the table again? How is that different than putting them in newly-allocated space at the end of the table? -- Scott Ribe [EMAIL PROTECTED] http://www.killerbytes.com/ (303) 722-0567 voice -- Sent via pgs

Re: [GENERAL] clustering without locking

2008-05-02 Thread Craig Ringer
Scott Ribe wrote: Huh? If I'm understanding you correctly you'll end up with rows in order, but with a really big hole in the middle of the table. I'm not sure if that qualifies as "clusters". That's why he said vacuum when done. Anyway, I'm not sure that a big *contiguous* hole in the mid

Re: [GENERAL] clustering without locking

2008-05-02 Thread Scott Ribe
> Huh? If I'm understanding you correctly you'll end up with rows in > order, but with a really big hole in the middle of the table. I'm not > sure if that qualifies as "clusters". That's why he said vacuum when done. Anyway, I'm not sure that a big *contiguous* hole in the middle of the table wou

Re: [GENERAL] clustering without locking

2008-05-02 Thread Fujii Masao
On Fri, May 2, 2008 at 9:12 AM, fschmidt <[EMAIL PROTECTED]> wrote: > > An implementation of clustering without locking would start by comparing the > index to the table from the beginning to find the first mismatch. Rows > before the mismatch are fine, and can be left alone. From here on, go

Re: [GENERAL] clustering without locking

2008-05-01 Thread Martijn van Oosterhout
On Thu, May 01, 2008 at 05:12:52PM -0700, fschmidt wrote: > > An implementation of clustering without locking would start by comparing the > index to the table from the beginning to find the first mismatch. Rows > before the mismatch are fine, and can be left alone. From here on, go > through th

[GENERAL] clustering without locking

2008-05-01 Thread fschmidt
An implementation of clustering without locking would start by comparing the index to the table from the beginning to find the first mismatch. Rows before the mismatch are fine, and can be left alone. From here on, go through the index and rewrite each row in order. This will put the rows at th