Re: [HACKERS] WALWriter active during recovery

2015-07-02 Thread Simon Riggs
On 2 July 2015 at 14:38, Andres Freund wrote: > On 2015-07-02 14:34:48 +0100, Simon Riggs wrote: > > This was pushed back from last CF and I haven't worked on it at all, nor > > will I. > > > > Pushing back again. > > Let's "return with feedback", not " move", it then.. Moving a entries > along w

Re: [HACKERS] WALWriter active during recovery

2015-07-02 Thread Andres Freund
On 2015-07-02 14:34:48 +0100, Simon Riggs wrote: > This was pushed back from last CF and I haven't worked on it at all, nor > will I. > > Pushing back again. Let's "return with feedback", not " move", it then.. Moving a entries along which aren't expected to receive updates anytime soon isn't a g

Re: [HACKERS] WALWriter active during recovery

2015-07-02 Thread Simon Riggs
On 2 July 2015 at 14:31, Fujii Masao wrote: > On Thu, Mar 5, 2015 at 5:22 PM, Fujii Masao wrote: > > On Thu, Dec 18, 2014 at 6:43 PM, Fujii Masao > wrote: > >> On Tue, Dec 16, 2014 at 3:51 AM, Simon Riggs > wrote: > >>> Currently, WALReceiver writes and fsyncs data it receives. Clearly, > >>>

Re: [HACKERS] WALWriter active during recovery

2015-07-02 Thread Fujii Masao
On Thu, Mar 5, 2015 at 5:22 PM, Fujii Masao wrote: > On Thu, Dec 18, 2014 at 6:43 PM, Fujii Masao wrote: >> On Tue, Dec 16, 2014 at 3:51 AM, Simon Riggs wrote: >>> Currently, WALReceiver writes and fsyncs data it receives. Clearly, >>> while we are waiting for an fsync we aren't doing any other

Re: [HACKERS] WALWriter active during recovery

2015-03-05 Thread Fujii Masao
On Thu, Dec 18, 2014 at 6:43 PM, Fujii Masao wrote: > On Tue, Dec 16, 2014 at 3:51 AM, Simon Riggs wrote: >> Currently, WALReceiver writes and fsyncs data it receives. Clearly, >> while we are waiting for an fsync we aren't doing any other useful >> work. >> >> Following patch starts WALWriter du

Re: [HACKERS] WALWriter active during recovery

2015-02-12 Thread Michael Paquier
On Thu, Dec 18, 2014 at 6:43 PM, Fujii Masao wrote: > On Tue, Dec 16, 2014 at 3:51 AM, Simon Riggs > wrote: > > Currently, WALReceiver writes and fsyncs data it receives. Clearly, > > while we are waiting for an fsync we aren't doing any other useful > > work. > > > > Following patch starts WALW

Re: [HACKERS] WALWriter active during recovery

2014-12-18 Thread Fujii Masao
On Tue, Dec 16, 2014 at 3:51 AM, Simon Riggs wrote: > Currently, WALReceiver writes and fsyncs data it receives. Clearly, > while we are waiting for an fsync we aren't doing any other useful > work. > > Following patch starts WALWriter during recovery and makes it > responsible for fsyncing data,

Re: [HACKERS] WALWriter active during recovery

2014-12-17 Thread didier
Hi On Wed, Dec 17, 2014 at 2:39 PM, Alvaro Herrera wrote: > didier wrote: > >> On many Linux systems it may not do that much (2.6.32 and 3.2 are bad, >> 3.13 is better but still it slows the fsync). >> >> If there's a fsync in progress WALReceiver will: >> 1- slow the fsync because its writes to

Re: [HACKERS] WALWriter active during recovery

2014-12-17 Thread Alvaro Herrera
didier wrote: > On many Linux systems it may not do that much (2.6.32 and 3.2 are bad, > 3.13 is better but still it slows the fsync). > > If there's a fsync in progress WALReceiver will: > 1- slow the fsync because its writes to the same file are grabbed by the fsync > 2- stall until the end of

Re: [HACKERS] WALWriter active during recovery

2014-12-17 Thread Simon Riggs
On 17 December 2014 at 11:27, didier wrote: > If there's a fsync in progress WALReceiver will: > 1- slow the fsync because its writes to the same file are grabbed by the fsync > 2- stall until the end of fsync. PostgreSQL already fsyncs files while they are being written to. Are you saying we sh

Re: [HACKERS] WALWriter active during recovery

2014-12-17 Thread didier
Hi, On Tue, Dec 16, 2014 at 6:07 PM, Simon Riggs wrote: > On 16 December 2014 at 14:12, Heikki Linnakangas > wrote: >> On 12/15/2014 08:51 PM, Simon Riggs wrote: >>> >>> Currently, WALReceiver writes and fsyncs data it receives. Clearly, >>> while we are waiting for an fsync we aren't doing any

Re: [HACKERS] WALWriter active during recovery

2014-12-16 Thread Simon Riggs
On 16 December 2014 at 14:12, Heikki Linnakangas wrote: > On 12/15/2014 08:51 PM, Simon Riggs wrote: >> >> Currently, WALReceiver writes and fsyncs data it receives. Clearly, >> while we are waiting for an fsync we aren't doing any other useful >> work. >> >> Following patch starts WALWriter durin

Re: [HACKERS] WALWriter active during recovery

2014-12-16 Thread Andres Freund
On 2014-12-16 16:12:40 +0200, Heikki Linnakangas wrote: > On 12/15/2014 08:51 PM, Simon Riggs wrote: > >Currently, WALReceiver writes and fsyncs data it receives. Clearly, > >while we are waiting for an fsync we aren't doing any other useful > >work. > > > >Following patch starts WALWriter during r

Re: [HACKERS] WALWriter active during recovery

2014-12-16 Thread Heikki Linnakangas
On 12/15/2014 08:51 PM, Simon Riggs wrote: Currently, WALReceiver writes and fsyncs data it receives. Clearly, while we are waiting for an fsync we aren't doing any other useful work. Following patch starts WALWriter during recovery and makes it responsible for fsyncing data, allowing WALReceive

Re: [HACKERS] WALWriter active during recovery

2014-12-15 Thread Andres Freund
Hi, On 2014-12-15 18:51:44 +, Simon Riggs wrote: > Currently, WALReceiver writes and fsyncs data it receives. Clearly, > while we are waiting for an fsync we aren't doing any other useful > work. Well, it can still buffer data on the network level, but there's definitely limits to that. So I