Re: [HACKERS] Count backend self-sync calls

2010-11-15 Thread Greg Smith
Jeff Janes wrote: Do you know where this competition is happening? Is it on the platters, or is it in the hard drive write cache (I thought high-end hardware had tagged writes to avoid that), or in the kernel? Kernel. Linux systems with lots of memory will happily queue up gigabytes of me

Re: [HACKERS] Count backend self-sync calls

2010-11-15 Thread Jeff Janes
On Sun, Nov 14, 2010 at 2:07 PM, Greg Smith wrote: > The attached patch adds a new field to pg_stat_bgwriter, counting the number > of times backends execute their own fsync calls.  Normally, when a backend > needs to fsync data, it passes a request to the background writer, which > then absorbs t

Re: [HACKERS] Count backend self-sync calls

2010-11-15 Thread Robert Haas
On Sun, Nov 14, 2010 at 8:31 PM, Robert Haas wrote: > On Sun, Nov 14, 2010 at 7:19 PM, Greg Smith wrote: >>> But if this is generating a lot of log data or adding a lot of >>> overhead, then you have bigger problems anyway: >>> >>> +               elog(DEBUG1, "Unable to forward fsync request, ex

Re: [HACKERS] Count backend self-sync calls

2010-11-14 Thread Robert Haas
On Sun, Nov 14, 2010 at 7:27 PM, Tom Lane wrote: > Robert Haas writes: >> It might be even better to mention that the reason why we couldn't >> forward the fsync request is that the fsync request queue is full. >> I'm not sure exactly how to phrase that.  I thought about: > >> fsync request queue

Re: [HACKERS] Count backend self-sync calls

2010-11-14 Thread Robert Haas
On Sun, Nov 14, 2010 at 7:19 PM, Greg Smith wrote: >> But if this is generating a lot of log data or adding a lot of >> overhead, then you have bigger problems anyway: >> >> +               elog(DEBUG1, "Unable to forward fsync request, executing >> directly"); >> > > The argument against this log

Re: [HACKERS] Count backend self-sync calls

2010-11-14 Thread Tom Lane
Robert Haas writes: > It might be even better to mention that the reason why we couldn't > forward the fsync request is that the fsync request queue is full. > I'm not sure exactly how to phrase that. I thought about: > fsync request queue is full > But that seems not to answer the "so what" qu

Re: [HACKERS] Count backend self-sync calls

2010-11-14 Thread Greg Smith
Robert Haas wrote: I think this one could be removed: + if (n > 0) + elog(DEBUG1,"Absorbing %d fsync requests",n); Right; that one is just there to let you know the patch is working, and how much the background writer does for you per pass, mainly for the purpose of re

Re: [HACKERS] Count backend self-sync calls

2010-11-14 Thread Robert Haas
On Sun, Nov 14, 2010 at 6:10 PM, Tom Lane wrote: > Robert Haas writes: >> With those changes, I think this is committable, and will do so, >> barring objections. > > Obey message style guidelines, please, especially if you're going > to promote any of those to ereports. The only new message woul

Re: [HACKERS] Count backend self-sync calls

2010-11-14 Thread Tom Lane
Robert Haas writes: > With those changes, I think this is committable, and will do so, > barring objections. Obey message style guidelines, please, especially if you're going to promote any of those to ereports. regards, tom lane -- Sent via pgsql-hackers mailing list (

Re: [HACKERS] Count backend self-sync calls

2010-11-14 Thread Robert Haas
On Sun, Nov 14, 2010 at 5:07 PM, Greg Smith wrote: > The patch also adds some logging to the innards involved here, to help with > understanding problems in this area.  I don't think that should be in the > version committed as is.  May want to drop the logging level or make it > disabled in regul

[HACKERS] Count backend self-sync calls

2010-11-14 Thread Greg Smith
The attached patch adds a new field to pg_stat_bgwriter, counting the number of times backends execute their own fsync calls. Normally, when a backend needs to fsync data, it passes a request to the background writer, which then absorbs the call into its own queue of work to do. However, unde