Re: patch to allow disable of WAL recycling

2019-03-04 Thread Jerry Jelinek
g new blocks on every write. I'll send out a new draft of the patch once all of these changes are done. Thanks again, Jerry On Wed, Feb 27, 2019 at 4:12 PM Alvaro Herrera wrote: > On 2019-Feb-05, Jerry Jelinek wrote: > > > First, since last fall, we have found another pe

Re: patch to allow disable of WAL recycling

2019-03-05 Thread Jerry Jelinek
confusing wording around "allocating blocks". I ran a clean build and make check passes. The new patch is attached. Thanks, Jerry On Wed, Feb 27, 2019 at 4:12 PM Alvaro Herrera wrote: > On 2019-Feb-05, Jerry Jelinek wrote: > > > First, since last fall, we have found anoth

Re: patch to allow disable of WAL recycling

2019-03-06 Thread Jerry Jelinek
On Wed, Mar 6, 2019 at 11:02 AM Alvaro Herrera wrote: > On 2019-Mar-06, Robert Haas wrote: > > > On Wed, Mar 6, 2019 at 12:13 PM Alvaro Herrera > wrote: > > > I want your dictating software. > > > > I'm afraid this is just me and a keyboard, but sadly for me you're not > > the first person to ac

Re: patch to allow disable of WAL recycling

2019-03-07 Thread Jerry Jelinek
On Wed, Mar 6, 2019 at 4:14 PM Jerry Jelinek wrote: > > It sounds like everyone is in agreement that I should get rid of the > single COW GUC tunable and provide two different tunables instead. I will > update the patch to go back to the original name (wal_recycle) for the &g

Re: patch to allow disable of WAL recycling

2019-03-07 Thread Jerry Jelinek
Thomas, Responses in-line. On Thu, Mar 7, 2019 at 3:09 PM Thomas Munro wrote: > On Fri, Mar 8, 2019 at 10:13 AM Jerry Jelinek > wrote: > > I have attached a new version of the patch that implements the changes > we've discussed over the past couple of days. Let me k

Re: patch to allow disable of WAL recycling

2019-03-26 Thread Jerry Jelinek
On Thu, Mar 7, 2019 at 6:26 PM Thomas Munro wrote: > On Fri, Mar 8, 2019 at 12:35 PM Jerry Jelinek > wrote: > > On Thu, Mar 7, 2019 at 3:09 PM Thomas Munro > wrote: > >> My understanding is that it's not really the COW-ness that makes it > >> not n

Re: patch to allow disable of WAL recycling

2019-03-29 Thread Jerry Jelinek
On Thu, Mar 28, 2019 at 6:10 PM Thomas Munro wrote: > On Fri, Mar 29, 2019 at 10:47 AM Thomas Munro > wrote: > > On Fri, Mar 29, 2019 at 8:59 AM Robert Haas > wrote: > > > On Tue, Mar 26, 2019 at 3:24 PM Jerry Jelinek < > jerry.jeli...@joyent.com> wrote: &g

Re: patch to allow disable of WAL recycling

2019-04-02 Thread Jerry Jelinek
On Mon, Apr 1, 2019 at 7:48 PM Thomas Munro wrote: > On Sat, Mar 30, 2019 at 11:18 AM Jerry Jelinek > wrote: > > I went through your new version of the patch and it all looks great to > me. > > I moved the error handling logic around a bit so we'd capture errno > i

Re: patch to allow disable of WAL recycling

2019-02-05 Thread Jerry Jelinek
On Mon, Oct 1, 2018 at 7:16 PM Michael Paquier wrote: > On Thu, Sep 13, 2018 at 02:56:42PM -0600, Jerry Jelinek wrote: > > I'll take a look at that. I had been trying to keep the patch as minimal > as > > possible, but I'm happy to work through this. > > (

Re: patch to allow disable of WAL recycling

2018-07-05 Thread Jerry Jelinek
Peter, Thanks for taking a look a this. I have a few responses in line. I am not a PG expert, so if there is something here that I've misunderstood, please let me know. On Sun, Jul 1, 2018 at 6:54 AM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 26.06.18 15:35,

Re: patch to allow disable of WAL recycling

2018-07-06 Thread Jerry Jelinek
Thomas, We're using a zfs recordsize of 8k to match the PG blocksize of 8k, so what you're describing is not the issue here. Thanks, Jerry On Thu, Jul 5, 2018 at 3:44 PM, Thomas Munro wrote: > On Fri, Jul 6, 2018 at 3:37 AM, Jerry Jelinek > wrote: > >> If the pr

Re: patch to allow disable of WAL recycling

2018-07-10 Thread Jerry Jelinek
system specific, is there specific reasons why we can't provide a tunable to disable this behavior for filesystems which don't behave that way? Thanks again, Jerry On Tue, Jun 26, 2018 at 7:35 AM, Jerry Jelinek wrote: > Hello All, > > Attached is a patch to provide a

Re: patch to allow disable of WAL recycling

2018-07-11 Thread Jerry Jelinek
Alvaro, I'll perform several test runs with various combinations and post the results. Thanks, Jerry On Tue, Jul 10, 2018 at 2:34 PM, Alvaro Herrera wrote: > On 2018-Jul-10, Jerry Jelinek wrote: > > > 2) Disabling WAL recycling reduces reliability, even on COW filesystems

Re: patch to allow disable of WAL recycling

2018-07-12 Thread Jerry Jelinek
other information I could provide. Thanks, Jerry On Tue, Jun 26, 2018 at 7:35 AM, Jerry Jelinek wrote: > Hello All, > > Attached is a patch to provide an option to disable WAL recycling. We have > found that this can help performance by eliminating read-modify-write > behavior

Re: patch to allow disable of WAL recycling

2018-07-13 Thread Jerry Jelinek
Thanks to everyone who has taken the time to look at this patch and provide all of the feedback. I'm going to wait another day to see if there are any more comments. If not, then first thing next week, I will send out a revised patch with improvements to the man page change as requested. If anyone

Re: patch to allow disable of WAL recycling

2018-07-16 Thread Jerry Jelinek
There have been quite a few comments since last week, so at this point I am uncertain how to proceed with this change. I don't think I saw anything concrete in the recent emails that I can act upon. I would like to respond to the comment about trying to "self-tune" the behavior based on inferences

Re: patch to allow disable of WAL recycling

2018-07-18 Thread Jerry Jelinek
ussed last week. I have attached an updated patch which does incorporate man page changes, in case that is the blocker. However, if this patch is simply rejected, I'd appreciate it if I could get a definitive statement to that effect. Thanks, Jerry On Tue, Jun 26, 2018 at 7:35 AM, Jerry Jeli

Re: patch to allow disable of WAL recycling

2018-07-20 Thread Jerry Jelinek
l 18, 2018 at 1:43 PM, Robert Haas wrote: > On Wed, Jul 18, 2018 at 3:22 PM, Jerry Jelinek > wrote: > > I've gotten a wide variety of feedback on the proposed patch. The > comments > > range from rough approval through various discussion about alternative > > soluti

Re: patch to allow disable of WAL recycling

2018-07-20 Thread Jerry Jelinek
dquadrant.com> wrote: > On 17.07.18 00:04, Jerry Jelinek wrote: > > There have been quite a few comments since last week, so at this point I > > am uncertain how to proceed with this change. I don't think I saw > > anything concrete in the recent emails that I can

Re: patch to allow disable of WAL recycling

2018-07-20 Thread Jerry Jelinek
me know and I'll see what I can do on my own. Thanks again, Jerry On Tue, Jul 17, 2018 at 2:47 PM, Tomas Vondra wrote: > On 07/17/2018 09:12 PM, Peter Eisentraut wrote: > > On 17.07.18 00:04, Jerry Jelinek wrote: > >> There have been quite a few comments since last week,

Re: patch to allow disable of WAL recycling

2018-07-27 Thread Jerry Jelinek
ble to say these are essentially equal, but I can collect more data across more different times if necessary. I'm also happy to collect more data if people have suggestions for different parameters on the pgbench run. Thanks, Jerry On Fri, Jul 20, 2018 at 4:04 PM, Jerry Jelinek wrote: >

Re: patch to allow disable of WAL recycling

2018-08-03 Thread Jerry Jelinek
different parameters on the pgbench run. > > Thanks, > Jerry > > > On Fri, Jul 20, 2018 at 4:04 PM, Jerry Jelinek > wrote: > >> Thomas, >> >> Thanks for your offer to run some tests on different OSes and filesystems >> that you have. Anything you can

patch to allow disable of WAL recycling

2018-06-26 Thread Jerry Jelinek
Hello All, Attached is a patch to provide an option to disable WAL recycling. We have found that this can help performance by eliminating read-modify-write behavior on old WAL files that are no longer resident in the filesystem cache. The is a lot more detail on the background of the motivation fo

Re: patch to allow disable of WAL recycling

2018-08-21 Thread Jerry Jelinek
mas Vondra wrote: > > On 07/21/2018 12:04 AM, Jerry Jelinek wrote: > >> Thomas, > >> > >> Thanks for your offer to run some tests on different OSes and > >> filesystems that you have. Anything you can provide here would be much > >> appreciated. I

Re: patch to allow disable of WAL recycling

2018-08-24 Thread Jerry Jelinek
Alvaro, I have previously posted ZFS numbers for SmartOS and FreeBSD to this thread, although not with the exact same benchmark runs that Tomas did. I think the main purpose of running the benchmarks is to demonstrate that there is no significant performance regression with wal recycling disabled

Re: patch to allow disable of WAL recycling

2018-08-29 Thread Jerry Jelinek
Tomas, This is really interesting data, thanks a lot for collecting all of it and formatting the helpful graphs. Jerry On Sun, Aug 26, 2018 at 4:14 PM, Tomas Vondra wrote: > > > On 08/25/2018 12:11 AM, Jerry Jelinek wrote: > > Alvaro, > > > > I have previously pos

Re: patch to allow disable of WAL recycling

2018-09-10 Thread Jerry Jelinek
Tomas, Thank you again for running all of these tests on your various hardware configurations. I was not aware of the convention that the commented example in the config file is expected to match the default value, so I was actually trying to show what to use if you didn't want the default, but I

Re: patch to allow disable of WAL recycling

2018-09-13 Thread Jerry Jelinek
Hi Peter, I'll take a look at that. I had been trying to keep the patch as minimal as possible, but I'm happy to work through this. Thanks, Jerry On Tue, Sep 11, 2018 at 9:39 AM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 10/09/2018 16:10, Jerry Jelin