Re: [HACKERS] WALInsertLock tuning

2011-10-14 Thread Robert Haas
On Thu, Oct 13, 2011 at 9:35 PM, Bruce Momjian wrote: > > I assume this was addressed with this commit: > >        commit 465883b0a2b4236ba6b31b648a9eabef3b7cdddb >        Author: Simon Riggs >        Date:   Tue Jun 28 22:58:17 2011 +0100 > >            Introduce compact WAL record for the commo

Re: [HACKERS] WALInsertLock tuning

2011-10-13 Thread Bruce Momjian
I assume this was addressed with this commit: commit 465883b0a2b4236ba6b31b648a9eabef3b7cdddb Author: Simon Riggs Date: Tue Jun 28 22:58:17 2011 +0100 Introduce compact WAL record for the common case of commit (non-DDL). XLOG_XACT_COMMI

Re: [HACKERS] WALInsertLock tuning

2011-06-07 Thread Fujii Masao
On Tue, Jun 7, 2011 at 9:54 PM, Simon Riggs wrote: > On Tue, Jun 7, 2011 at 1:24 PM, Robert Haas wrote: > >> One other thought is that I think that this patch might cause a >> user-visible behavior change.  Right now, when you hit the end of >> recovery, you most typically get a message saying -

Re: [HACKERS] WALInsertLock tuning

2011-06-07 Thread Simon Riggs
On Tue, Jun 7, 2011 at 4:57 PM, Tom Lane wrote: > Heikki Linnakangas writes: >> On 07.06.2011 10:55, Simon Riggs wrote: >>> How would that help? > >> It doesn't matter whether the pages are zeroed while they sit in memory. >> And if you write a full page of WAL data, any wasted bytes at the end o

Re: [HACKERS] WALInsertLock tuning

2011-06-07 Thread Tom Lane
Heikki Linnakangas writes: > On 07.06.2011 10:55, Simon Riggs wrote: >> How would that help? > It doesn't matter whether the pages are zeroed while they sit in memory. > And if you write a full page of WAL data, any wasted bytes at the end of > the page don't matter, because they're ignored at

Re: [HACKERS] WALInsertLock tuning

2011-06-07 Thread Heikki Linnakangas
On 07.06.2011 10:55, Simon Riggs wrote: On Tue, Jun 7, 2011 at 8:27 AM, Heikki Linnakangas wrote: You would only need to do it just before you write out the WAL. I guess you'd need to grab WALInsertLock in XLogWrite() to prevent more WAL records from being inserted on the page until you're don

Re: [HACKERS] WALInsertLock tuning

2011-06-07 Thread Simon Riggs
On Tue, Jun 7, 2011 at 1:24 PM, Robert Haas wrote: > One other thought is that I think that this patch might cause a > user-visible behavior change.  Right now, when you hit the end of > recovery, you most typically get a message saying - record with zero > length.  Not always, but often.  If we

Re: [HACKERS] WALInsertLock tuning

2011-06-07 Thread Robert Haas
On Tue, Jun 7, 2011 at 3:21 AM, Simon Riggs wrote: >> It strikes me, though, that we >> could probably get nearly all of the benefit of this patch by being >> willing to zero the first sizeof(XLogRecord) bytes following a record, >> but not the rest of the buffer.  That would pretty much wipe out

Re: [HACKERS] WALInsertLock tuning

2011-06-07 Thread Simon Riggs
On Tue, Jun 7, 2011 at 8:27 AM, Heikki Linnakangas wrote: > On 07.06.2011 10:21, Simon Riggs wrote: >> >> On Mon, Jun 6, 2011 at 11:25 PM, Robert Haas >>  wrote: >>> >>> It strikes me, though, that we >>> could probably get nearly all of the benefit of this patch by being >>> willing to zero the f

Re: [HACKERS] WALInsertLock tuning

2011-06-07 Thread Heikki Linnakangas
On 07.06.2011 10:21, Simon Riggs wrote: On Mon, Jun 6, 2011 at 11:25 PM, Robert Haas wrote: It strikes me, though, that we could probably get nearly all of the benefit of this patch by being willing to zero the first sizeof(XLogRecord) bytes following a record, but not the rest of the buffer.

Re: [HACKERS] WALInsertLock tuning

2011-06-07 Thread Simon Riggs
On Mon, Jun 6, 2011 at 11:25 PM, Robert Haas wrote: > As to the question of whether it's safe, I think I'd agree that the > chances of this backfiring are pretty remote.  I think that with the > zeroing they are exactly zero, because (now that we start XLOG > positions at 0/1) there is now way th

Re: [HACKERS] WALInsertLock tuning

2011-06-06 Thread Robert Haas
On Mon, Jun 6, 2011 at 6:05 PM, Simon Riggs wrote: > On Mon, Jun 6, 2011 at 10:09 PM, Jeff Janes wrote: >> On Mon, Jun 6, 2011 at 11:27 AM, Simon Riggs wrote: >>> >>> But that even assumes we write the unzeroed data at the end of the >>> buffer. We don't. We only write data up to the end of the

Re: [HACKERS] WALInsertLock tuning

2011-06-06 Thread Simon Riggs
On Mon, Jun 6, 2011 at 10:09 PM, Jeff Janes wrote: > On Mon, Jun 6, 2011 at 11:27 AM, Simon Riggs wrote: >> >> But that even assumes we write the unzeroed data at the end of the >> buffer. We don't. We only write data up to the end of the WAL record >> on the current page, unless we do a continua

Re: [HACKERS] WALInsertLock tuning

2011-06-06 Thread Jeff Janes
On Mon, Jun 6, 2011 at 11:27 AM, Simon Riggs wrote: > > But that even assumes we write the unzeroed data at the end of the > buffer. We don't. We only write data up to the end of the WAL record > on the current page, unless we do a continuation record, I see no codepath in XLogWrite which writes

Re: [HACKERS] WALInsertLock tuning

2011-06-06 Thread Simon Riggs
On Mon, Jun 6, 2011 at 5:47 PM, Tom Lane wrote: > Simon Riggs writes: >> In earlier discussions of how to improve WALInsertLock contention, it >> was observed that we must zero each new page before we advance the WAL >> insertion point. >> http://postgresql.1045698.n5.nabble.com/Reworking-WAL-loc

Re: [HACKERS] WALInsertLock tuning

2011-06-06 Thread Tom Lane
Simon Riggs writes: > In earlier discussions of how to improve WALInsertLock contention, it > was observed that we must zero each new page before we advance the WAL > insertion point. > http://postgresql.1045698.n5.nabble.com/Reworking-WAL-locking-td1983647.html > IMHO the page zeroing is complet

[HACKERS] WALInsertLock tuning

2011-06-06 Thread Simon Riggs
In earlier discussions of how to improve WALInsertLock contention, it was observed that we must zero each new page before we advance the WAL insertion point. http://postgresql.1045698.n5.nabble.com/Reworking-WAL-locking-td1983647.html IMHO the page zeroing is completely unnecessary, and replicatio