Re: Refactor pg_rewind code and make it work against a standby

2020-11-20 Thread Heikki Linnakangas
On 20/11/2020 19:14, Andres Freund wrote: Hi, On 2020-11-20 16:19:03 +0200, Heikki Linnakangas wrote: Pushed a fix similar to your patch, but I put the wait_for_catchup() before running pg_rewind. The point of inserting the 'in A, after C was promoted' row is that it's present in B when pg_rewi

Re: Refactor pg_rewind code and make it work against a standby

2020-11-20 Thread Andres Freund
Hi, On 2020-11-20 16:19:03 +0200, Heikki Linnakangas wrote: > Pushed a fix similar to your patch, but I put the wait_for_catchup() before > running pg_rewind. The point of inserting the 'in A, after C was promoted' > row is that it's present in B when pg_rewind runs. Hm - don't we possibly need *

Re: Refactor pg_rewind code and make it work against a standby

2020-11-20 Thread Heikki Linnakangas
On 20/11/2020 02:38, Andres Freund wrote: I locally, on a heavily modified branch (AIO support), started to get consistent failures in this test. I *suspect*, but am not sure, that it's the test's fault, not the fault of modifications. As far as I can tell, after the pg_rewind call, there's no g

Re: Refactor pg_rewind code and make it work against a standby

2020-11-19 Thread Andres Freund
Hi, On 2020-11-15 17:10:53 +0200, Heikki Linnakangas wrote: > Yep, quite right. Fixed that way, thanks for the debugging! I locally, on a heavily modified branch (AIO support), started to get consistent failures in this test. I *suspect*, but am not sure, that it's the test's fault, not the fault

Re: Refactor pg_rewind code and make it work against a standby

2020-11-15 Thread Heikki Linnakangas
On 15/11/2020 09:07, Tom Lane wrote: I wrote: Not sure if you noticed, but piculet has twice failed the 007_standby_source.pl test that was added by 9c4f5192f: ... Now, I'm not sure what to make of that, but I can't help noticing that piculet uses --disable-atomics while francolin uses --disable

Re: Refactor pg_rewind code and make it work against a standby

2020-11-14 Thread Tom Lane
I wrote: > Not sure if you noticed, but piculet has twice failed the > 007_standby_source.pl test that was added by 9c4f5192f: > ... > Now, I'm not sure what to make of that, but I can't help noticing that > piculet uses --disable-atomics while francolin uses --disable-spinlocks. > That leads the m

Re: Refactor pg_rewind code and make it work against a standby

2020-11-14 Thread Tom Lane
Not sure if you noticed, but piculet has twice failed the 007_standby_source.pl test that was added by 9c4f5192f: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=piculet&dt=2020-11-15%2006%3A00%3A11 https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=piculet&dt=2020-11-13%2011%3A20%3A1

Re: Refactor pg_rewind code and make it work against a standby

2020-11-12 Thread Heikki Linnakangas
On 04/11/2020 11:23, Heikki Linnakangas wrote: I read through the patches one more time, fixed a bunch of typos and such, and pushed patches 1-4. I'm going to spend some more time on testing the last patch. It allows using a standby server as the source, and we don't have any tests for that yet.

Re: Refactor pg_rewind code and make it work against a standby

2020-11-04 Thread Heikki Linnakangas
On 25/09/2020 02:56, Soumyadeep Chakraborty wrote: On Thu, Sep 24, 2020 at 10:27 AM Heikki Linnakangas wrote: 7. Please address the FIXME for the symlink case: /* FIXME: Check if it points to the same target? */ It's not a new issue. Would be nice to fix, of course. I'm not sure what the righ

Re: Refactor pg_rewind code and make it work against a standby

2020-09-24 Thread Soumyadeep Chakraborty
On Thu, Sep 24, 2020 at 10:27 AM Heikki Linnakangas wrote: > >> /* > >> * If this is a relation file, copy the modified blocks. > >> * > >> * This is in addition to any other changes. > >> */ > >> iter = datapagemap_iterate(&entry->target_modified_pages); > >> while (datapagemap_next(iter,

Re: Refactor pg_rewind code and make it work against a standby

2020-09-24 Thread Heikki Linnakangas
On 20/09/2020 23:44, Soumyadeep Chakraborty wrote: Before getting into the code review for the patch, I wanted to know why we don't use a Bitmapset for target_modified_pages? Bitmapset is not available in client code. Perhaps it could be moved to src/common with some changes, but doesn't seem

Re: Refactor pg_rewind code and make it work against a standby

2020-09-24 Thread Heikki Linnakangas
Thanks for the review! I'll post a new version shortly, with your comments incorporated. More detailed response to a few of them below: On 18/09/2020 10:41, Kyotaro Horiguchi wrote: I don't think filemap_finalize needs to iterate over filemap twice. True, but I thought it's more clear that wa

Re: Refactor pg_rewind code and make it work against a standby

2020-09-20 Thread Soumyadeep Chakraborty
Hey Heikki, Thanks for refactoring and making the code much easier to read! Before getting into the code review for the patch, I wanted to know why we don't use a Bitmapset for target_modified_pages? Code review: 1. We need to update the comments for process_source_file and process_target_file

Re: Refactor pg_rewind code and make it work against a standby

2020-09-18 Thread Kyotaro Horiguchi
Hello. It needed rebasing. (Attached) At Tue, 25 Aug 2020 16:32:02 +0300, Heikki Linnakangas wrote in > On 20/08/2020 11:32, Kyotaro Horiguchi wrote: > > 0002: Rewording that old->target and new->source makes the meaning far > > Good idea! I changed the patch that way. Looks Good. > > 0003:

Re: Refactor pg_rewind code and make it work against a standby

2020-09-16 Thread Michael Paquier
On Tue, Aug 25, 2020 at 04:32:02PM +0300, Heikki Linnakangas wrote: > On 20/08/2020 11:32, Kyotaro Horiguchi wrote: > > 0002: Rewording that old->target and new->source makes the meaning far > > clearer. Moving decisions core code into filemap_finalize is > > reasonable. > > > > By th

Re: Refactor pg_rewind code and make it work against a standby

2020-08-25 Thread Heikki Linnakangas
On 20/08/2020 11:32, Kyotaro Horiguchi wrote: 0002: Rewording that old->target and new->source makes the meaning far clearer. Moving decisions core code into filemap_finalize is reasonable. By the way, some of the rules are remaining in process_source/target_file. For example,

Re: Refactor pg_rewind code and make it work against a standby

2020-08-20 Thread Kyotaro Horiguchi
Hello. At Wed, 19 Aug 2020 15:50:16 +0300, Heikki Linnakangas wrote in > Hi, > > I started to hack on making pg_rewind crash-safe (see [1]), but I > quickly got side-tracked into refactoring and tidying up up the code > in general. I ended up with this series of patches: ^^; > The first four