Re: pg15b3: recovery fails with wal prefetch enabled

2022-09-08 Thread Thomas Munro
On Wed, Sep 7, 2022 at 1:56 AM Jonathan S. Katz wrote: > To close this loop, I added a section for "fixed before RC1" to Open > Items since this is presumably the next release. We can include it there > once committed. Done yesterday. To tie up a couple of loose ends from this thread: On Thu, S

Re: pg15b3: recovery fails with wal prefetch enabled

2022-09-06 Thread Jonathan S. Katz
On 9/5/22 10:03 PM, Thomas Munro wrote: On Tue, Sep 6, 2022 at 1:51 PM Tom Lane wrote: "Jonathan S. Katz" writes: On 9/5/22 7:18 PM, Thomas Munro wrote: Well I was about to commit this, but beta4 just got stamped (but not yet tagged). I see now that Jonathan (with RMT hat on, CC'd) meant co

Re: pg15b3: recovery fails with wal prefetch enabled

2022-09-05 Thread Thomas Munro
On Tue, Sep 6, 2022 at 1:51 PM Tom Lane wrote: > "Jonathan S. Katz" writes: > > On 9/5/22 7:18 PM, Thomas Munro wrote: > >> Well I was about to commit this, but beta4 just got stamped (but not > >> yet tagged). I see now that Jonathan (with RMT hat on, CC'd) meant > >> commits should be in by th

Re: pg15b3: recovery fails with wal prefetch enabled

2022-09-05 Thread Tom Lane
"Jonathan S. Katz" writes: > On 9/5/22 7:18 PM, Thomas Munro wrote: >> Well I was about to commit this, but beta4 just got stamped (but not >> yet tagged). I see now that Jonathan (with RMT hat on, CC'd) meant >> commits should be in by the *start* of the 5th AoE, not the end. So >> the procedur

Re: pg15b3: recovery fails with wal prefetch enabled

2022-09-05 Thread Jonathan S. Katz
On 9/5/22 7:18 PM, Thomas Munro wrote: On Mon, Sep 5, 2022 at 9:08 PM Thomas Munro wrote: At Mon, 05 Sep 2022 14:15:27 +0900 (JST), Kyotaro Horiguchi wrote in At Mon, 5 Sep 2022 16:54:07 +1200, Thomas Munro wrote in On reflection, it'd be better not to clobber any pre-existing error there,

Re: pg15b3: recovery fails with wal prefetch enabled

2022-09-05 Thread Kyotaro Horiguchi
At Mon, 5 Sep 2022 21:08:16 +1200, Thomas Munro wrote in > We also need the LSN that is past that record. > XLogReleasePreviousRecord() could return it (or we could use > reader->EndRecPtr I suppose). Thoughts on this version? (Catching the gap...) It is easier to read. Thanks! regards. --

Re: pg15b3: recovery fails with wal prefetch enabled

2022-09-05 Thread Thomas Munro
On Mon, Sep 5, 2022 at 9:08 PM Thomas Munro wrote: > > At Mon, 05 Sep 2022 14:15:27 +0900 (JST), Kyotaro Horiguchi > > wrote in > > At Mon, 5 Sep 2022 16:54:07 +1200, Thomas Munro > > wrote in > > > On reflection, it'd be better not to clobber any pre-existing error > > > there, but report one

Re: pg15b3: recovery fails with wal prefetch enabled

2022-09-05 Thread Thomas Munro
On Mon, Sep 5, 2022 at 5:34 PM Kyotaro Horiguchi wrote: > At Mon, 05 Sep 2022 14:15:27 +0900 (JST), Kyotaro Horiguchi > wrote in > me> +1 for showing any message for the failure, but I think we shouldn't > me> hide an existing message if any. > > At Mon, 5 Sep 2022 16:54:07 +1200, Thomas Munro

Re: pg15b3: recovery fails with wal prefetch enabled

2022-09-04 Thread Kyotaro Horiguchi
(the previous mail was crossing with yours..) At Mon, 05 Sep 2022 14:15:27 +0900 (JST), Kyotaro Horiguchi wrote in me> +1 for showing any message for the failure, but I think we shouldn't me> hide an existing message if any. At Mon, 5 Sep 2022 16:54:07 +1200, Thomas Munro wrote in > On refl

Re: pg15b3: recovery fails with wal prefetch enabled

2022-09-04 Thread Kyotaro Horiguchi
At Mon, 5 Sep 2022 13:28:12 +1200, Thomas Munro wrote in > I had this more or less figured out on Friday when I wrote last, but I > got stuck on a weird problem with 026_overwrite_contrecord.pl. I > think that failure case should report an error, no? I find it strange > that we end recovery in

Re: pg15b3: recovery fails with wal prefetch enabled

2022-09-04 Thread Thomas Munro
On Mon, Sep 5, 2022 at 1:28 PM Thomas Munro wrote: > I had this more or less figured out on Friday when I wrote last, but I > got stuck on a weird problem with 026_overwrite_contrecord.pl. I > think that failure case should report an error, no? I find it strange > that we end recovery in silence

Re: pg15b3: recovery fails with wal prefetch enabled

2022-09-04 Thread Thomas Munro
On Fri, Sep 2, 2022 at 6:20 PM Thomas Munro wrote: > ... The active ingredient here is a setting of > maintenance_io_concurency=0, which runs into a dumb accounting problem > of the fencepost variety and incorrectly concludes it's reached the > end early. Setting it to 3 or higher allows his syst

Re: pg15b3: recovery fails with wal prefetch enabled

2022-09-01 Thread Thomas Munro
On Thu, Sep 1, 2022 at 11:18 PM Thomas Munro wrote: > Ahh, problem repro'd here with WAL compression. More soon. I followed some false pistes for a while there, but I finally figured it out what's happening here after Justin kindly shared some files with me. The active ingredient here is a sett

Re: pg15b3: recovery fails with wal prefetch enabled

2022-09-01 Thread Thomas Munro
On Thu, Sep 1, 2022 at 5:52 PM Justin Pryzby wrote: > compression method: zstd Ahh, problem repro'd here with WAL compression. More soon.

Re: pg15b3: recovery fails with wal prefetch enabled

2022-08-31 Thread Justin Pryzby
On Thu, Sep 01, 2022 at 05:35:23PM +1200, Thomas Munro wrote: > So it *looks* like it finished early (and without the expected > error?). But it also looks like it replayed that record, according to > the page LSN. So which is it? Could you recompile with WAL_DEBUG > defined in pg_config_manual.

Re: pg15b3: recovery fails with wal prefetch enabled

2022-08-31 Thread Thomas Munro
On Thu, Sep 1, 2022 at 5:18 PM Kyotaro Horiguchi wrote: > At Wed, 31 Aug 2022 23:47:53 -0500, Justin Pryzby > wrote in > > On Thu, Sep 01, 2022 at 04:22:20PM +1200, Thomas Munro wrote: > > > Hmm. Justin, when you built from source, which commit were you at? > > > If it's REL_15_BETA3, > > > > N

Re: pg15b3: recovery fails with wal prefetch enabled

2022-08-31 Thread Kyotaro Horiguchi
At Wed, 31 Aug 2022 23:47:53 -0500, Justin Pryzby wrote in > On Thu, Sep 01, 2022 at 04:22:20PM +1200, Thomas Munro wrote: > > On Thu, Sep 1, 2022 at 3:08 PM Kyotaro Horiguchi > > wrote: > > > Just for information, there was a fixed bug about > > > overwrite-aborted-contrecord feature, which ca

Re: pg15b3: recovery fails with wal prefetch enabled

2022-08-31 Thread Justin Pryzby
On Thu, Sep 01, 2022 at 04:22:20PM +1200, Thomas Munro wrote: > On Thu, Sep 1, 2022 at 3:08 PM Kyotaro Horiguchi > wrote: > > At Thu, 1 Sep 2022 12:05:36 +1200, Thomas Munro > > wrote in > > > On Thu, Sep 1, 2022 at 2:01 AM Justin Pryzby wrote: > > > > < 2022-08-31 08:44:10.495 CDT >LOG: chec

Re: pg15b3: recovery fails with wal prefetch enabled

2022-08-31 Thread Thomas Munro
On Thu, Sep 1, 2022 at 3:08 PM Kyotaro Horiguchi wrote: > At Thu, 1 Sep 2022 12:05:36 +1200, Thomas Munro > wrote in > > On Thu, Sep 1, 2022 at 2:01 AM Justin Pryzby wrote: > > > < 2022-08-31 08:44:10.495 CDT >LOG: checkpoint starting: > > > end-of-recovery immediate wait > > > < 2022-08-31

Re: pg15b3: recovery fails with wal prefetch enabled

2022-08-31 Thread Kyotaro Horiguchi
At Thu, 1 Sep 2022 12:05:36 +1200, Thomas Munro wrote in > On Thu, Sep 1, 2022 at 2:01 AM Justin Pryzby wrote: > > < 2022-08-31 08:44:10.495 CDT >LOG: checkpoint starting: end-of-recovery > > immediate wait > > < 2022-08-31 08:44:10.609 CDT >LOG: request to flush past end of > > generated

Re: pg15b3: recovery fails with wal prefetch enabled

2022-08-31 Thread Justin Pryzby
Some more details, in case they're important: First: the server has wal_compression=zstd (I wonder if something doesn't allow/accomodate compressed FPI?) I thought to mention that after compiling pg15 locally and forgetting to use --with-zstd. I compiled it to enable your debug logging, which wr

Re: pg15b3: recovery fails with wal prefetch enabled

2022-08-31 Thread Thomas Munro
On Thu, Sep 1, 2022 at 12:53 PM Justin Pryzby wrote: > Yes, I have a copy that reproduces the issue: That's good news. So the last record touching that page was: > rmgr: Heap2 len (rec/tot): 59/59, tx: 0, lsn: > 1201/1CAF84B0, prev 1201/1CAF8478, desc: VISIBLE cutoff xid

Re: pg15b3: recovery fails with wal prefetch enabled

2022-08-31 Thread Justin Pryzby
On Thu, Sep 01, 2022 at 12:05:36PM +1200, Thomas Munro wrote: > On Thu, Sep 1, 2022 at 2:01 AM Justin Pryzby wrote: > > < 2022-08-31 08:44:10.495 CDT >LOG: checkpoint starting: end-of-recovery > > immediate wait > > < 2022-08-31 08:44:10.609 CDT >LOG: request to flush past end of > > generat

Re: pg15b3: recovery fails with wal prefetch enabled

2022-08-31 Thread Thomas Munro
On Thu, Sep 1, 2022 at 2:01 AM Justin Pryzby wrote: > < 2022-08-31 08:44:10.495 CDT >LOG: checkpoint starting: end-of-recovery > immediate wait > < 2022-08-31 08:44:10.609 CDT >LOG: request to flush past end of generated > WAL; request 1201/1CAF84F0, current position 1201/1CADB730 > < 2022-0

pg15b3: recovery fails with wal prefetch enabled

2022-08-31 Thread Justin Pryzby
An internal VM crashed last night due to OOM. When I tried to start postgres, it failed like: < 2022-08-31 08:44:10.495 CDT >LOG: checkpoint starting: end-of-recovery immediate wait < 2022-08-31 08:44:10.609 CDT >LOG: request to flush past end of generated WAL; request 1201/1CAF84F0, curren