Re: [HACKERS] Subtransaction commits and Hot Standby

2008-10-20 Thread Simon Riggs
On Mon, 2008-10-20 at 16:23 -0300, Alvaro Herrera wrote: > Simon Riggs wrote: > > > Renamed to set_status_by_pages because we never use this on the whole > > tree. Added comments to say that. > > > > Overall, cleaner and more readable now. Thanks. > > Committed, thanks. Cheers. -- Simon Rig

Re: [HACKERS] Subtransaction commits and Hot Standby

2008-10-20 Thread Alvaro Herrera
Simon Riggs wrote: > Renamed to set_status_by_pages because we never use this on the whole > tree. Added comments to say that. > > Overall, cleaner and more readable now. Thanks. Committed, thanks. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replic

Re: [HACKERS] Subtransaction commits and Hot Standby

2008-10-07 Thread Simon Riggs
On Sun, 2008-10-05 at 14:51 -0400, Alvaro Herrera wrote: > Simon Riggs wrote: > > > OK, spent long time testing various batching scenarios for this using a > > custom test harness to simulate various spreads of xids in transaction > > trees. All looks fine now. > > I had a look and was mostly re

Re: [HACKERS] Subtransaction commits and Hot Standby

2008-10-06 Thread Simon Riggs
On Sun, 2008-10-05 at 14:51 -0400, Alvaro Herrera wrote: > Simon Riggs wrote: > > > OK, spent long time testing various batching scenarios for this using a > > custom test harness to simulate various spreads of xids in transaction > > trees. All looks fine now. > > I had a look and was mostly re

Re: [HACKERS] Subtransaction commits and Hot Standby

2008-10-05 Thread Alvaro Herrera
Simon Riggs wrote: > OK, spent long time testing various batching scenarios for this using a > custom test harness to simulate various spreads of xids in transaction > trees. All looks fine now. I had a look and was mostly rephrasing some comments and the README (hopefully I didn't make any of th

Re: [PATCHES] [HACKERS] Subtransaction commits and Hot Standby

2008-09-24 Thread Simon Riggs
On Wed, 2008-09-24 at 13:48 +0100, Simon Riggs wrote: > On Tue, 2008-09-23 at 22:47 +0100, Simon Riggs wrote: > > > I've tested this some more and am much happier with it now. > > The concept is fine, but I've found a coding bug in further testing. > Please wait now for new version before review

Re: [PATCHES] [HACKERS] Subtransaction commits and Hot Standby

2008-09-24 Thread Simon Riggs
On Tue, 2008-09-23 at 22:47 +0100, Simon Riggs wrote: > I've tested this some more and am much happier with it now. The concept is fine, but I've found a coding bug in further testing. Please wait now for new version before review. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Trai

Re: [HACKERS] Subtransaction commits and Hot Standby

2008-09-23 Thread Simon Riggs
On Thu, 2008-09-18 at 15:59 +0100, Simon Riggs wrote: > On Tue, 2008-09-16 at 10:11 -0400, Alvaro Herrera wrote: > > > I wonder if the improved clog API required to mark multiple > > transactions as committed at once would be also useful to > > TransactionIdCommitTree which is used in regular tra

Re: [HACKERS] Subtransaction commits and Hot Standby

2008-09-18 Thread Simon Riggs
On Tue, 2008-09-16 at 10:11 -0400, Alvaro Herrera wrote: > I wonder if the improved clog API required to mark multiple > transactions as committed at once would be also useful to > TransactionIdCommitTree which is used in regular transaction commit. I've hacked together this concept patch (WIP).

Re: [HACKERS] Subtransaction commits and Hot Standby

2008-09-16 Thread Simon Riggs
On Tue, 2008-09-16 at 11:08 -0400, Alvaro Herrera wrote: > Simon Riggs wrote: > > > Perhaps it is sufficient to throw an error if the subxid cache > > overflows? But I suspect that may not be acceptable... > > Certainly not. Yeh :-) ... it was just a rhetorical question. I'll try to avoid those

Re: [HACKERS] Subtransaction commits and Hot Standby

2008-09-16 Thread Simon Riggs
On Tue, 2008-09-16 at 15:38 +0100, Simon Riggs wrote: > On Tue, 2008-09-16 at 17:01 +0300, Heikki Linnakangas wrote: > > Simon Riggs wrote: > > > Subtransactions cause a couple of problems for Hot Standby: > > > > Do we need to treat subtransactions any differently from normal > > transactions?

Re: [HACKERS] Subtransaction commits and Hot Standby

2008-09-16 Thread Simon Riggs
On Tue, 2008-09-16 at 10:11 -0400, Alvaro Herrera wrote: > >> Right now we lock and unlock the clog for each committed subtransaction > >> at commit time, which is wasteful. A better scheme: > >> pre-scan the list of xids to derive list of pages > >> if we have just a single page to update >

Re: [HACKERS] Subtransaction commits and Hot Standby

2008-09-16 Thread Alvaro Herrera
Simon Riggs wrote: > Perhaps it is sufficient to throw an error if the subxid cache > overflows? But I suspect that may not be acceptable... Certainly not. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7

Re: [HACKERS] Subtransaction commits and Hot Standby

2008-09-16 Thread Simon Riggs
On Tue, 2008-09-16 at 10:11 -0400, Alvaro Herrera wrote: > Heikki Linnakangas wrote: > > Simon Riggs wrote: > >> Subtransactions cause a couple of problems for Hot Standby: > > > > Do we need to treat subtransactions any differently from normal > > transactions? Just treat all subtransactions as

Re: [HACKERS] Subtransaction commits and Hot Standby

2008-09-16 Thread Simon Riggs
On Tue, 2008-09-16 at 17:01 +0300, Heikki Linnakangas wrote: > Simon Riggs wrote: > > Subtransactions cause a couple of problems for Hot Standby: > > Do we need to treat subtransactions any differently from normal > transactions? Just treat all subtransactions as top-level transactions > until

Re: [HACKERS] Subtransaction commits and Hot Standby

2008-09-16 Thread Alvaro Herrera
Heikki Linnakangas wrote: > Simon Riggs wrote: >> Subtransactions cause a couple of problems for Hot Standby: > > Do we need to treat subtransactions any differently from normal > transactions? Just treat all subtransactions as top-level transactions > until commit, and mark them all as committ

Re: [HACKERS] Subtransaction commits and Hot Standby

2008-09-16 Thread Heikki Linnakangas
Simon Riggs wrote: Subtransactions cause a couple of problems for Hot Standby: Do we need to treat subtransactions any differently from normal transactions? Just treat all subtransactions as top-level transactions until commit, and mark them all as committed when you see the commit record fo

[HACKERS] Subtransaction commits and Hot Standby

2008-09-16 Thread Simon Riggs
Subtransactions cause a couple of problems for Hot Standby: * we don't record new subtransactionids in WAL, so we have no direct way to issue updates to subtrans while in recovery mode as would normally happen when we assign subtransaction ids (subxids) * we don't record subtransaction commit in WA