Re: [HACKERS] Reduced power consumption in WAL Writer process

2011-07-18 Thread Tom Lane
Simon Riggs writes: > On Sun, Jul 17, 2011 at 10:41 PM, Tom Lane wrote: >> Yeah, I agree with putting the actual SetLatch call after we release the >> lock ... but doesn't the calculation need to be done while we're still >> holding it?  Otherwise it'd be using potentially-inconsistent values. >

Re: [HACKERS] Reduced power consumption in WAL Writer process

2011-07-18 Thread Simon Riggs
On Mon, Jul 18, 2011 at 1:48 AM, Robert Haas wrote: > Might be good to start a new thread for each auxilliary process, or we > may get confused. +1 --  Simon Riggs   http://www.2ndQuadrant.com/  PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hacke

Re: [HACKERS] Reduced power consumption in WAL Writer process

2011-07-17 Thread Simon Riggs
On Sun, Jul 17, 2011 at 10:41 PM, Tom Lane wrote: > Simon Riggs writes: >> On Fri, Jul 15, 2011 at 6:33 PM, Tom Lane wrote: >>> I'd say send the signal when wal buffers are more than X% full (maybe >>> half).  The suggestion to send it when wrapping around at the end of the >>> array is not quit

Re: [HACKERS] Reduced power consumption in WAL Writer process

2011-07-17 Thread Robert Haas
On Sun, Jul 17, 2011 at 8:20 PM, Peter Geoghegan wrote: > This is a bit of a detour, but probably a useful one. Attached is a > patch that replaces a tight PostmasterIsAlive() polling loop in the AV > launcher with a latch, making use of the new WL_POSTMASTER_DEATH > functionality. It's similar to

Re: [HACKERS] Reduced power consumption in WAL Writer process

2011-07-17 Thread Peter Geoghegan
This is a bit of a detour, but probably a useful one. Attached is a patch that replaces a tight PostmasterIsAlive() polling loop in the AV launcher with a latch, making use of the new WL_POSTMASTER_DEATH functionality. It's similar to what we've already done for the archiver. It is relatively strai

Re: [HACKERS] Reduced power consumption in WAL Writer process

2011-07-17 Thread Tom Lane
Simon Riggs writes: > On Fri, Jul 15, 2011 at 6:33 PM, Tom Lane wrote: >> I'd say send the signal when wal buffers are more than X% full (maybe >> half).  The suggestion to send it when wrapping around at the end of the >> array is not quite right, because that's an arbitrary condition that's >>

Re: [HACKERS] Reduced power consumption in WAL Writer process

2011-07-17 Thread Simon Riggs
On Fri, Jul 15, 2011 at 6:33 PM, Tom Lane wrote: > Robert Haas writes: >> On Jul 15, 2011, at 8:55 AM, Simon Riggs wrote: >>> The only difference is how bulk write operations are handled. As long >>> as we wake WALWriter before wal_buffers fills then we'll be good. >>> Wakeup once per wal buffer

Re: [HACKERS] Reduced power consumption in WAL Writer process

2011-07-15 Thread Tom Lane
Robert Haas writes: > On Jul 15, 2011, at 8:55 AM, Simon Riggs wrote: >> The only difference is how bulk write operations are handled. As long >> as we wake WALWriter before wal_buffers fills then we'll be good. >> Wakeup once per wal buffer is too much. I agree we should measure this >> to check

Re: [HACKERS] Reduced power consumption in WAL Writer process

2011-07-15 Thread Robert Haas
On Jul 15, 2011, at 8:55 AM, Simon Riggs wrote: > On Fri, Jul 15, 2011 at 2:29 PM, Robert Haas wrote: > >> If the primary goal here is to reduce power consumption, another option >> would be to keep the regular wake-ups most of the time but have some >> mechanism for putting the process to sleep

Re: [HACKERS] Reduced power consumption in WAL Writer process

2011-07-15 Thread Alvaro Herrera
Excerpts from Simon Riggs's message of vie jul 15 09:55:40 -0400 2011: > On Fri, Jul 15, 2011 at 2:29 PM, Robert Haas wrote: > > > If the primary goal here is to reduce power consumption, another option > > would be to keep the regular wake-ups most of the time but have some > > mechanism for put

Re: [HACKERS] Reduced power consumption in WAL Writer process

2011-07-15 Thread Simon Riggs
On Fri, Jul 15, 2011 at 2:29 PM, Robert Haas wrote: > If the primary goal here is to reduce power consumption, another option > would be to keep the regular wake-ups most of the time but have some > mechanism for putting the process to sleep until wakened when no activity > happens for a certain

Re: [HACKERS] Reduced power consumption in WAL Writer process

2011-07-15 Thread Peter Geoghegan
My instrumentation wasn't that good. I was using powertop 1.13, which apparently goes to great lengths to group processes by various criteria (including process group), but doesn't actually offer the option of seeing that instrumentation per process. I'm using version 1.98 beta 1 as of now, which

Re: [HACKERS] Reduced power consumption in WAL Writer process

2011-07-15 Thread Robert Haas
On Jul 14, 2011, at 4:42 AM, Simon Riggs wrote: > On Thu, Jul 14, 2011 at 9:57 AM, Fujii Masao wrote: > >> Currently walwriter might write out the WAL before a transaction commits. >> IOW, walwriter tries to write out the WAL in wal_buffers in every wakeups. >> This might be useful for long tran

Re: [HACKERS] Reduced power consumption in WAL Writer process

2011-07-14 Thread Simon Riggs
On Wed, Jul 13, 2011 at 10:56 PM, Peter Geoghegan wrote: > Attached is patch for the WAL writer that removes its tight polling > loop (which probably doesn't get hit often in practice, as we just > sleep if wal_writer_delay is under a second), and, at least > potentially, reduces power consumption

Re: [HACKERS] Reduced power consumption in WAL Writer process

2011-07-14 Thread Simon Riggs
On Thu, Jul 14, 2011 at 10:53 AM, Heikki Linnakangas wrote: > On 14.07.2011 12:42, Simon Riggs wrote: >> >> On Thu, Jul 14, 2011 at 9:57 AM, Fujii Masao >>  wrote: >> >>> Currently walwriter might write out the WAL before a transaction commits. >>> IOW, walwriter tries to write out the WAL in wal_

Re: [HACKERS] Reduced power consumption in WAL Writer process

2011-07-14 Thread Heikki Linnakangas
On 14.07.2011 12:42, Simon Riggs wrote: On Thu, Jul 14, 2011 at 9:57 AM, Fujii Masao wrote: Currently walwriter might write out the WAL before a transaction commits. IOW, walwriter tries to write out the WAL in wal_buffers in every wakeups. This might be useful for long transaction which gener

Re: [HACKERS] Reduced power consumption in WAL Writer process

2011-07-14 Thread Simon Riggs
On Thu, Jul 14, 2011 at 9:57 AM, Fujii Masao wrote: > Currently walwriter might write out the WAL before a transaction commits. > IOW, walwriter tries to write out the WAL in wal_buffers in every wakeups. > This might be useful for long transaction which generates lots of WAL > records before com

Re: [HACKERS] Reduced power consumption in WAL Writer process

2011-07-14 Thread Fujii Masao
On Thu, Jul 14, 2011 at 5:39 PM, Simon Riggs wrote: > On Wed, Jul 13, 2011 at 10:56 PM, Peter Geoghegan > wrote: >> Attached is patch for the WAL writer that removes its tight polling >> loop (which probably doesn't get hit often in practice, as we just >> sleep if wal_writer_delay is under a se

[HACKERS] Reduced power consumption in WAL Writer process

2011-07-13 Thread Peter Geoghegan
Attached is patch for the WAL writer that removes its tight polling loop (which probably doesn't get hit often in practice, as we just sleep if wal_writer_delay is under a second), and, at least potentially, reduces power consumption when idle by using a latch. I will break all remaining power con