Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-11 Thread Heikki Linnakangas
On 11.05.2011 13:34, Peter Geoghegan wrote: On 11 May 2011 09:54, Magnus Hagander wrote: If you're doing this Win32 specific, take a look at src/backend/port/win32/signal.c for an example. If you're not doing this win32-specific, I doubt we really want threads to be involved... Well, that s

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-11 Thread Peter Geoghegan
On 11 May 2011 09:54, Magnus Hagander wrote: > If you're doing this Win32 specific, take a look at > src/backend/port/win32/signal.c for an example. > > If you're not doing this win32-specific, I doubt we really want > threads to be involved... Well, that seems to be the traditional wisdom. It s

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-11 Thread Magnus Hagander
On Wed, May 11, 2011 at 10:52, Peter Geoghegan wrote: > On 9 May 2011 11:19, Heikki Linnakangas > wrote: > >> In the child, spawn a thread > > How exactly should I go about this? The one place in the code that I > knew to use multiple threads, pgbench, falls back on "emulation with > fork()" on s

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-11 Thread Peter Geoghegan
On 9 May 2011 11:19, Heikki Linnakangas wrote: > In the child, spawn a thread How exactly should I go about this? The one place in the code that I knew to use multiple threads, pgbench, falls back on "emulation with fork()" on some platforms. -- Peter Geoghegan       http://www.2ndQuadrant.com

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-10 Thread Peter Geoghegan
On 10 May 2011 17:43, Heikki Linnakangas wrote: > It should be an anonymous pipe that's inherited by the child process by > rather than a named pipe. Otherwise seems fine to me, as far as this proof > of concept program goes. Alright, thanks. I'll use an anonymous pipe in the patch itself. --

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-10 Thread Heikki Linnakangas
On 10.05.2011 14:39, Peter Geoghegan wrote: Attached is win32 implementation of the "named pipe trick". It consists of a Visual Studio 2008 solution that contains two projects, named_pipe_trick (which represents the postmaster) and auxiliary_backend (which represents each auxiliary process). I s

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-10 Thread Tom Lane
Simon Riggs writes: > I've got a feeling that things will go easier if we have a separate > connection for the feedback channel. > Yes, two connections, one in either direction. > That would make everything simple, nice one way connections. It would > also mean we could stream at higher data rat

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-10 Thread Simon Riggs
On Tue, May 10, 2011 at 12:45 PM, Robert Haas wrote: > On Tue, May 10, 2011 at 5:14 AM, Peter Geoghegan > wrote: >> On 10 May 2011 09:45, Heikki Linnakangas >> wrote: >> >>> I think we need to refactor the function into something like: >>> >>> #define WL_LATCH_SET    1 >>> #define WL_SOCKET_REA

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-10 Thread Robert Haas
On Tue, May 10, 2011 at 5:14 AM, Peter Geoghegan wrote: > On 10 May 2011 09:45, Heikki Linnakangas > wrote: > >> I think we need to refactor the function into something like: >> >> #define WL_LATCH_SET    1 >> #define WL_SOCKET_READABLE 2 >> #define WL_SOCKET_WRITEABLE 4 >> #define WL_TIMEOUT    

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-10 Thread Peter Geoghegan
On 10 May 2011 09:45, Heikki Linnakangas wrote: > I think we need to refactor the function into something like: > > #define WL_LATCH_SET    1 > #define WL_SOCKET_READABLE 2 > #define WL_SOCKET_WRITEABLE 4 > #define WL_TIMEOUT      8 > #define WL_POSTMASTER_DEATH 16 While I agree with the need to

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-10 Thread Heikki Linnakangas
On 10.05.2011 11:22, Peter Geoghegan wrote: On 10 May 2011 02:58, Fujii Masao wrote: Alright. I'm currently working on a proof-of-concept implementation of that. In the meantime, any thoughts on how this should meld with the existing latch implementation? How about making WaitLatch monitor th

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-10 Thread Peter Geoghegan
On 10 May 2011 02:58, Fujii Masao wrote: >> Alright. I'm currently working on a proof-of-concept implementation of >> that. In the meantime, any thoughts on how this should meld with the >> existing latch implementation? > > How about making WaitLatch monitor the file descriptor for the pipe > by

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-09 Thread Fujii Masao
On Mon, May 9, 2011 at 8:27 PM, Peter Geoghegan wrote: > On 9 May 2011 11:19, Heikki Linnakangas > wrote: > >> Can't we use the pipe trick on Windows? The API is different, but we use >> pipes on Windows for other things already. When a process is launched, open >> a pipe between postmaster and t

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-09 Thread Peter Geoghegan
On 9 May 2011 11:19, Heikki Linnakangas wrote: > Can't we use the pipe trick on Windows? The API is different, but we use > pipes on Windows for other things already. When a process is launched, open > a pipe between postmaster and the child process. In the child, spawn a > thread that just calls

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-09 Thread Heikki Linnakangas
On 09.05.2011 12:20, Peter Geoghegan wrote: I've taken a look into it, and I'm not optimistic about the likelihood of the way I've suggested that we can register a callback on process termination on windows being acceptable. It seems to be a kludge too far. It does work on Vista, just not very we

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-09 Thread Peter Geoghegan
I've taken a look into it, and I'm not optimistic about the likelihood of the way I've suggested that we can register a callback on process termination on windows being acceptable. It seems to be a kludge too far. It does work on Vista, just not very well. There is a considerable delay on closing t

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-07 Thread Peter Geoghegan
On 7 May 2011 18:07, Tom Lane wrote: > The aspect of this that *is* relevant is that if you haven't > deliberately defeated the interlock (and thereby put your data at risk), > you won't be able to start a new postmaster until all the old > shmem-attached children are gone.  And that's why having

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-07 Thread Tom Lane
Peter Geoghegan writes: > Perhaps I'm missing the point here, but I don't think that I have to > make an argument for why it might be acceptable to have two archivers > running at once, or two of any other auxiliary process. Let's assume > that it's completely unacceptable. It may still be worth w

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-07 Thread Peter Geoghegan
On 6 May 2011 15:00, Tom Lane wrote: > Peter Geoghegan writes: >> On 5 May 2011 21:05, Tom Lane wrote: >>> The major problem I'm aware of for getting rid of periodic wakeups is >>> the need for child processes to notice when the postmaster has died >>> unexpectedly. > >> Could you please expand

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-06 Thread Robert Haas
On Fri, May 6, 2011 at 10:13 AM, Robert Haas wrote: > On Fri, May 6, 2011 at 8:16 AM, Peter Geoghegan wrote: >> Could you please expand upon this? Why is it of any consequence if the >> archiver notices that the postmaster is dead after 60 seconds rather >> than after 1? So control in the archive

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-06 Thread Robert Haas
On Fri, May 6, 2011 at 8:16 AM, Peter Geoghegan wrote: > Could you please expand upon this? Why is it of any consequence if the > archiver notices that the postmaster is dead after 60 seconds rather > than after 1? So control in the archiver is going to stay in its event > loop for longer than it

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-06 Thread Tom Lane
Peter Geoghegan writes: > On 5 May 2011 21:05, Tom Lane wrote: >> The major problem I'm aware of for getting rid of periodic wakeups is >> the need for child processes to notice when the postmaster has died >> unexpectedly. > Could you please expand upon this? Why is it of any consequence if the

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-06 Thread Peter Geoghegan
On 5 May 2011 21:05, Tom Lane wrote: > The major problem I'm aware of for getting rid of periodic wakeups is > the need for child processes to notice when the postmaster has died > unexpectedly.  Your patch appears to degrade the archiver's response > time for that really significantly, like from

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-05 Thread Tom Lane
Peter Geoghegan writes: > On 5 May 2011 22:22, Tom Lane wrote: >> What that really means is that any WaitOnLatch call with a finite >> timeout ought to be viewed with a jaundiced eye.  Ideally, we want them >> all to be waiting for latch release and nothing else.  I'm concerned >> that we're goin

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-05 Thread Peter Geoghegan
On 5 May 2011 22:22, Tom Lane wrote: > Robert Haas writes: >> On Thu, May 5, 2011 at 4:05 PM, Tom Lane wrote: +              * The caveat about signals invalidating the timeout of +              * WaitLatch() on some platforms can be safely disregarded, > >>> Really? > >> I'm a bit con

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-05 Thread Tom Lane
Robert Haas writes: > On Thu, May 5, 2011 at 4:05 PM, Tom Lane wrote: >>> + * The caveat about signals invalidating the timeout of >>> + * WaitLatch() on some platforms can be safely disregarded, >> Really? > I'm a bit confused by the phrasing of this comment as well,

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-05 Thread Robert Haas
On Thu, May 5, 2011 at 4:05 PM, Tom Lane wrote: >> +              * The caveat about signals invalidating the timeout of >> +              * WaitLatch() on some platforms can be safely disregarded, > > Really? I'm a bit confused by the phrasing of this comment as well, but it does seem to me that

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-05 Thread A.M.
On May 5, 2011, at 4:08 PM, Alvaro Herrera wrote: > Excerpts from Peter Geoghegan's message of jue may 05 16:49:25 -0300 2011: > >> I'll need to take a look at statistics, autovacuum and Logger >> processes too, to see if they present more subtle opportunities for >> reduced idle power consumpti

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-05 Thread Alvaro Herrera
Excerpts from Peter Geoghegan's message of jue may 05 16:49:25 -0300 2011: > I'll need to take a look at statistics, autovacuum and Logger > processes too, to see if they present more subtle opportunities for > reduced idle power consumption. More subtle? Autovacuum wakes up once per second and

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-05 Thread Tom Lane
Peter Geoghegan writes: > Attached is the first in what I hope will become a series of patches > for reducing power consumption when idle. Cool. This has been on my personal to-do list for awhile, but it keeps on failing to get to the top, so I'm glad to see somebody else putting time into it.

[HACKERS] Process wakeups when idle and power consumption

2011-05-05 Thread Peter Geoghegan
There is a general need to have Postgres consume fewer CPU cycles and less power when idle. Until something is done about this, shared hosting providers, particularly those who want to deploy many VM instances with databases, will continue to choose MySQL out of hand. I have quantified the differe