Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-22 Thread Nathan Bossart
On Wed, Feb 22, 2023 at 12:30:20PM +0900, Michael Paquier wrote: > Okay, applied and backpatched with a minimal test set, then. I have > kept the tweaks I did to the tests with extra comments. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-21 Thread Michael Paquier
On Tue, Feb 21, 2023 at 10:00:11AM -0800, Nathan Bossart wrote: > What is the purpose of testing it without the checkpoint? Perhaps none, I was wondering whether it would be worth testing that with the flush phase, but perhaps that's just extra cycles wasted at this point. > Other than that > que

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-21 Thread Nathan Bossart
On Tue, Feb 21, 2023 at 03:13:10PM +0900, Michael Paquier wrote: > The test enforces a checkpoint after the table creation on the > template, so what about testing it also without a checkpoint, like the > extended version attached? I have tweaked a few things in the test, > while on it. What is t

RE: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-21 Thread Ryo Matsumura (Fujitsu)
> On Thu, Feb 16, 2023 at 10:24:13AM +0530, Dilip Kumar wrote: > > Yes, there is no reason to pass this as false, seems like this is > > passed false by mistake. And your patch fixes the issue. On Thu, Feb 16, 2023 at 02:26:55PM +0900, Michael Paquier wrote: > So, if I am understanding this stuff

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-20 Thread Michael Paquier
On Mon, Feb 20, 2023 at 04:43:22PM -0800, Nathan Bossart wrote: > On Mon, Feb 20, 2023 at 05:02:01PM +0900, Michael Paquier wrote: >> On Fri, Feb 17, 2023 at 02:35:30PM -0800, Nathan Bossart wrote: >>> I'm happy to move this new test to wherever folks think it should go. I'll >>> look around to se

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-20 Thread Nathan Bossart
On Mon, Feb 20, 2023 at 05:02:01PM +0900, Michael Paquier wrote: > On Fri, Feb 17, 2023 at 02:35:30PM -0800, Nathan Bossart wrote: >> I'm happy to move this new test to wherever folks think it should go. I'll >> look around to see if I can find a better place, too. > > I think that src/test/recov

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-20 Thread Michael Paquier
On Fri, Feb 17, 2023 at 02:35:30PM -0800, Nathan Bossart wrote: > I'm happy to move this new test to wherever folks think it should go. I'll > look around to see if I can find a better place, too. I think that src/test/recovery/ is the best fit, because this stresses a code path for WAL replay on

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-17 Thread Nathan Bossart
On Fri, Feb 17, 2023 at 03:13:32PM +0100, Peter Eisentraut wrote: > On 16.02.23 22:29, Andres Freund wrote: >> What's the story behind 100_bugs.pl? This name clearly is copied from >> src/test/subscription/t/100_bugs.pl - but I've never understood why that is >> outside of the normal numbering spac

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-17 Thread Peter Eisentraut
On 16.02.23 22:29, Andres Freund wrote: What's the story behind 100_bugs.pl? This name clearly is copied from src/test/subscription/t/100_bugs.pl - but I've never understood why that is outside of the normal numbering space. Mainly to avoid awkwardness for backpatching. The number of tests in

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-16 Thread Amit Kapila
On Fri, Feb 17, 2023 at 2:59 AM Andres Freund wrote: > > On 2023-02-16 12:37:57 +0530, Robert Haas wrote: > > The patch creates 100_bugs.pl > > What's the story behind 100_bugs.pl? This name clearly is copied from > src/test/subscription/t/100_bugs.pl - but I've never understood why that is > outs

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-16 Thread Andres Freund
On 2023-02-16 12:37:57 +0530, Robert Haas wrote: > The patch creates 100_bugs.pl What's the story behind 100_bugs.pl? This name clearly is copied from src/test/subscription/t/100_bugs.pl - but I've never understood why that is outside of the normal numbering space.

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-15 Thread Dilip Kumar
On Thu, Feb 16, 2023 at 12:38 PM Robert Haas wrote: > > On Thu, Feb 16, 2023 at 12:32 PM Dilip Kumar wrote: > > I think the below change is not relevant to this bug right? > > > > diff --git a/src/test/recovery/meson.build b/src/test/recovery/meson.build > > index 209118a639..6e9f8a7c7f 100644 >

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-15 Thread Michael Paquier
On Thu, Feb 16, 2023 at 12:37:57PM +0530, Robert Haas wrote: > Why not? The patch creates 100_bugs.pl so it also adds it to meson.build. It would not matter for REL_15_STABLE, but on HEAD all the new TAP test files have to be added in their respective meson.build. If you don't do that, the CFBot

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-15 Thread Robert Haas
On Thu, Feb 16, 2023 at 12:32 PM Dilip Kumar wrote: > I think the below change is not relevant to this bug right? > > diff --git a/src/test/recovery/meson.build b/src/test/recovery/meson.build > index 209118a639..6e9f8a7c7f 100644 > --- a/src/test/recovery/meson.build > +++ b/src/test/recovery/mes

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-15 Thread Dilip Kumar
On Thu, Feb 16, 2023 at 12:11 PM Nathan Bossart wrote: > > On Thu, Feb 16, 2023 at 02:26:55PM +0900, Michael Paquier wrote: > > So, if I am understanding this stuff right, this issue can create data > > corruption once a DDL updates any pages of pg_class stored in a > > template database that gets

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-15 Thread Nathan Bossart
On Thu, Feb 16, 2023 at 02:26:55PM +0900, Michael Paquier wrote: > So, if I am understanding this stuff right, this issue can create data > corruption once a DDL updates any pages of pg_class stored in a > template database that gets copied by this routine. In this case, the > patch sent makes sur

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-15 Thread Michael Paquier
On Thu, Feb 16, 2023 at 10:24:13AM +0530, Dilip Kumar wrote: > Yes, there is no reason to pass this as false, seems like this is > passed false by mistake. And your patch fixes the issue. So, if I am understanding this stuff right, this issue can create data corruption once a DDL updates any page

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-15 Thread Dilip Kumar
On Thu, Feb 16, 2023 at 5:37 AM Nathan Bossart wrote: > > On Wed, Feb 15, 2023 at 04:49:38AM +, Ryo Matsumura (Fujitsu) wrote: > > The above is occured by the following call. > > The argument 'permanent' of ReadBufferWithoutRelcache() is passed to > > BufferAlloc() as 'relpersistence'. > > > >

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-15 Thread Nathan Bossart
On Wed, Feb 15, 2023 at 04:49:38AM +, Ryo Matsumura (Fujitsu) wrote: > The above is occured by the following call. > The argument 'permanent' of ReadBufferWithoutRelcache() is passed to > BufferAlloc() as 'relpersistence'. > > [src/backend/commands/] > 298 buf = ReadBufferWithoutRelca

Re: DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-15 Thread Justin Pryzby
On Wed, Feb 15, 2023 at 04:49:38AM +, Ryo Matsumura (Fujitsu) wrote: > Hi, hackers. > > I found that CREATE DATABASE occurs lost of DDL result after crash server. > The direct cause is that the checkpoint skips sync for page of template1's > main fork > because buffer status is not marked as

DDL result is lost by CREATE DATABASE with WAL_LOG strategy

2023-02-14 Thread Ryo Matsumura (Fujitsu)
Hi, hackers. I found that CREATE DATABASE occurs lost of DDL result after crash server. The direct cause is that the checkpoint skips sync for page of template1's main fork because buffer status is not marked as BM_PERMANENT in BufferAlloc(). Have you any knowledge about it? Reproduction: 1) Do