Re: Adding a TAP test checking data consistency on standby with minRecoveryPoint

2019-04-18 Thread Michael Paquier
On Thu, Apr 18, 2019 at 03:16:17PM -0400, Robert Haas wrote: > On Tue, Apr 16, 2019 at 2:45 AM Michael Paquier wrote: >> I think that we have race conditions with checkpointing and shutdown >> on HEAD. > > Any idea whether it's something newly-introduced or of long standing? I would suggest to k

Re: Adding a TAP test checking data consistency on standby with minRecoveryPoint

2019-04-18 Thread Robert Haas
On Tue, Apr 16, 2019 at 2:45 AM Michael Paquier wrote: > I think that we have race conditions with checkpointing and shutdown > on HEAD. Any idea whether it's something newly-introduced or of long standing? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Compa

Re: Adding a TAP test checking data consistency on standby with minRecoveryPoint

2019-04-15 Thread Michael Paquier
On Sun, Mar 24, 2019 at 09:47:58PM +0900, Michael Paquier wrote: > The failure is a bit weird, as I would expect all those three actions > to be sequential. piculet is the only failure happening on the > buildfarm and it uses --disable-atomics, so I am wondering if that is > related and if 0dfe3d0

Re: Adding a TAP test checking data consistency on standby with minRecoveryPoint

2019-03-24 Thread Michael Paquier
On Sat, Mar 23, 2019 at 04:08:42PM -0700, Peter Geoghegan wrote: > Seems like there might be a problem either caused by or detected by > 016_min_consistency.pl on piculet: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=piculet&dt=2019-03-23%2022%3A28%3A59 Interesting. Based on what

Re: Adding a TAP test checking data consistency on standby with minRecoveryPoint

2019-03-23 Thread Peter Geoghegan
On Wed, Mar 13, 2019 at 12:22 AM Michael Paquier wrote: > On Tue, Mar 12, 2019 at 01:16:14PM +0300, Arthur Zakirov wrote: > > On my environment installcheck passes as well as check. Also I ran tests > > with reverted c186ba13, 016_min_consistency.pl doesn't pass as expected. Seems like there migh

Re: Adding a TAP test checking data consistency on standby with minRecoveryPoint

2019-03-13 Thread Michael Paquier
On Tue, Mar 12, 2019 at 01:16:14PM +0300, Arthur Zakirov wrote: > On my environment installcheck passes as well as check. Also I ran tests > with reverted c186ba13, 016_min_consistency.pl doesn't pass as expected. Thanks all for the feedback! I have done an extra pass on the new, tweaking some se

Re: Adding a TAP test checking data consistency on standby with minRecoveryPoint

2019-03-12 Thread Georgios Kokolatos
Hi, you are right in saying that my comment didn't offer much of a constructive explanation. Apologies for that. To the issue at hand. Tests were run in the same manner as in all other cases and the test in question was the only one to fail in the whole tree. By looking a bit closer to the err

Re: Adding a TAP test checking data consistency on standby with minRecoveryPoint

2019-03-12 Thread Arthur Zakirov
On 12.03.2019 08:19, Michael Paquier wrote: On Mon, Mar 11, 2019 at 02:35:49PM +, Georgios Kokolatos wrote: To be honest, I have not checked closely on the failure, still it is the only test failing which by itself should be worthwhile mentioning. This works for me, as well as a plain inst

Re: Adding a TAP test checking data consistency on standby with minRecoveryPoint

2019-03-11 Thread Michael Paquier
On Mon, Mar 11, 2019 at 02:35:49PM +, Georgios Kokolatos wrote: > To be honest, I have not checked closely on the failure, still it is > the only test failing which by itself should be worthwhile > mentioning. This works for me, as well as a plain installcheck and check from the subpath. Are

Re: Adding a TAP test checking data consistency on standby with minRecoveryPoint

2019-03-11 Thread Georgios Kokolatos
Hi, I applied the patch on current master and run the tests, but I am afraid that the newly introduced test failed on installcheck-world: ```t/016_min_consistency.pl . # Looks like your test exited with 29 before it could output anything. t/016_min_consistency.pl . Dubio

Re: Adding a TAP test checking data consistency on standby with minRecoveryPoint

2019-01-30 Thread Michael Paquier
On Mon, Nov 12, 2018 at 01:08:38PM +0900, Michael Paquier wrote: > Thanks! I am stealing that stuff, and I have added an offline check by > comparing the value of minRecoveryPoint in pg_controldata. Again, if > you revert c186ba13 and run the tests, both online and offline failures > are showing

Re: Adding a TAP test checking data consistency on standby with minRecoveryPoint

2018-11-11 Thread Michael Paquier
On Fri, Nov 09, 2018 at 12:42:27PM +, Andrew Gierth wrote: > Seems simple enough - the LSN on the page is actually two 4-byte values > with the most significant one first, regardless of platform byte order > (though each individual word is in native order), so > > my ($hi,$lo) = unpack("LL

Re: Adding a TAP test checking data consistency on standby with minRecoveryPoint

2018-11-09 Thread Andrew Gierth
> "Michael" == Michael Paquier writes: > On Thu, Nov 08, 2018 at 04:00:26PM +0900, Michael Paquier wrote: >> Yes, I don't disagree with you and I thought about it. Fetching the >> value from the control file is easy, doing the comparison between >> two LSNs is also simple by doing it dire

Re: Adding a TAP test checking data consistency on standby with minRecoveryPoint

2018-11-08 Thread Michael Paquier
On Thu, Nov 08, 2018 at 04:00:26PM +0900, Michael Paquier wrote: > Yes, I don't disagree with you and I thought about it. Fetching the > value from the control file is easy, doing the comparison between two > LSNs is also simple by doing it directly with pg_lsn in the database > (and I don't want

Re: Adding a TAP test checking data consistency on standby with minRecoveryPoint

2018-11-07 Thread Michael Paquier
On Thu, Nov 08, 2018 at 06:00:29AM +, Andrew Gierth wrote: > I think it would be nice to have a test that didn't rely on that, too. Yes, I don't disagree with you and I thought about it. Fetching the value from the control file is easy, doing the comparison between two LSNs is also simple by

Re: Adding a TAP test checking data consistency on standby with minRecoveryPoint

2018-11-07 Thread Andrew Gierth
> "Michael" == Michael Paquier writes: >> How? It's OK (and normal) for in-core pages to have newer LSNs than >> minRecoveryPoint, it's only on-disk pages that must not be more >> recent than that. And pageinspect will show the in-core page... Michael> If the standby is stopped cleanly o

Re: Adding a TAP test checking data consistency on standby with minRecoveryPoint

2018-11-07 Thread Michael Paquier
On Thu, Nov 08, 2018 at 05:04:35AM +, Andrew Gierth wrote: > How? It's OK (and normal) for in-core pages to have newer LSNs than > minRecoveryPoint, it's only on-disk pages that must not be more recent > than that. And pageinspect will show the in-core page... If the standby is stopped cleanly

Re: Adding a TAP test checking data consistency on standby with minRecoveryPoint

2018-11-07 Thread Andrew Gierth
> "Michael" == Michael Paquier writes: Michael> I have also been trying to shape that into a TAP test which Michael> can be added into the in-core recovery test suite, and it Michael> happens that the part which scans if all the pages of a Michael> relation are not newer than what minReco

Adding a TAP test checking data consistency on standby with minRecoveryPoint

2018-11-07 Thread Michael Paquier
Hi all, While working on a fix for c186ba13 which fixes the way minRecoveryPoint is updated for other processes than the startup process, I was struggling about being able to get that into a reproducible test case. I have been thinking what Andrew Gierth has mentioned yesterday, and roughly desig