Re: Race between KeepFileRestoredFromArchive() and restartpoint

2022-08-04 Thread Noah Misch
On Thu, Aug 04, 2022 at 06:32:38AM -0400, David Steele wrote: > On 8/4/22 04:06, Kyotaro Horiguchi wrote: > >At Wed, 3 Aug 2022 23:24:56 -0700, Noah Misch wrote in > I think in this case a HINT might be sufficient to at least keep people > from > wasting time tracking down a problem t

Re: Race between KeepFileRestoredFromArchive() and restartpoint

2022-08-04 Thread David Steele
On 8/4/22 04:06, Kyotaro Horiguchi wrote: At Wed, 3 Aug 2022 23:24:56 -0700, Noah Misch wrote in I think in this case a HINT might be sufficient to at least keep people from wasting time tracking down a problem that has already been fixed. Here's a patch to add that HINT. I figure it's bette

Re: Race between KeepFileRestoredFromArchive() and restartpoint

2022-08-04 Thread Kyotaro Horiguchi
At Wed, 3 Aug 2022 23:24:56 -0700, Noah Misch wrote in > > > I think in this case a HINT might be sufficient to at least keep people > > > from > > > wasting time tracking down a problem that has already been fixed. > > Here's a patch to add that HINT. I figure it's better to do this before ne

Re: Race between KeepFileRestoredFromArchive() and restartpoint

2022-08-03 Thread Noah Misch
On Tue, Aug 02, 2022 at 07:37:27AM -0700, Noah Misch wrote: > On Tue, Aug 02, 2022 at 10:14:22AM -0400, David Steele wrote: > > On 7/31/22 02:17, Noah Misch wrote: > > >On Tue, Jul 26, 2022 at 07:21:29AM -0400, David Steele wrote: > > >>On 6/19/21 16:39, Noah Misch wrote: > > >>>On Tue, Feb 02, 202

Re: Race between KeepFileRestoredFromArchive() and restartpoint

2022-08-03 Thread Kyotaro Horiguchi
At Wed, 3 Aug 2022 00:28:47 -0700, Noah Misch wrote in > On Wed, Aug 03, 2022 at 11:24:17AM +0900, Kyotaro Horiguchi wrote: > > At Tue, 2 Aug 2022 16:03:42 -0500, Don Seiler wrote in > > > could not link file “pg_wal/xlogtemp.18799" to > > > > “pg_wal/0001D4530010”: File exists > >

Re: Race between KeepFileRestoredFromArchive() and restartpoint

2022-08-03 Thread Noah Misch
On Wed, Aug 03, 2022 at 11:24:17AM +0900, Kyotaro Horiguchi wrote: > At Tue, 2 Aug 2022 16:03:42 -0500, Don Seiler wrote in > > could not link file “pg_wal/xlogtemp.18799" to > > > “pg_wal/0001D4530010”: File exists > Hmm. It seems like a race condition betwen StartupXLOG() and > Re

Re: Race between KeepFileRestoredFromArchive() and restartpoint

2022-08-02 Thread Kyotaro Horiguchi
At Tue, 2 Aug 2022 16:03:42 -0500, Don Seiler wrote in > On Tue, Aug 2, 2022 at 10:01 AM David Steele wrote: > > > > > > That makes sense. Each iteration of the restartpoint recycle loop has a > > 1/N > > > chance of failing. Recovery adds >N files between restartpoints. > > Hence, the > > >

Re: Race between KeepFileRestoredFromArchive() and restartpoint

2022-08-02 Thread Don Seiler
On Tue, Aug 2, 2022 at 10:01 AM David Steele wrote: > > > That makes sense. Each iteration of the restartpoint recycle loop has a > 1/N > > chance of failing. Recovery adds >N files between restartpoints. > Hence, the > > WAL directory grows without bound. Is that roughly the theory in mind? >

Re: Race between KeepFileRestoredFromArchive() and restartpoint

2022-08-02 Thread David Steele
On 8/2/22 10:37, Noah Misch wrote: On Tue, Aug 02, 2022 at 10:14:22AM -0400, David Steele wrote: On 7/31/22 02:17, Noah Misch wrote: On Tue, Jul 26, 2022 at 07:21:29AM -0400, David Steele wrote: On 6/19/21 16:39, Noah Misch wrote: On Tue, Feb 02, 2021 at 07:14:16AM -0800, Noah Misch wrote:

Re: Race between KeepFileRestoredFromArchive() and restartpoint

2022-08-02 Thread Noah Misch
On Tue, Aug 02, 2022 at 10:14:22AM -0400, David Steele wrote: > On 7/31/22 02:17, Noah Misch wrote: > >On Tue, Jul 26, 2022 at 07:21:29AM -0400, David Steele wrote: > >>On 6/19/21 16:39, Noah Misch wrote: > >>>On Tue, Feb 02, 2021 at 07:14:16AM -0800, Noah Misch wrote: > Recycling and prealloca

Re: Race between KeepFileRestoredFromArchive() and restartpoint

2022-08-02 Thread David Steele
On 7/31/22 02:17, Noah Misch wrote: On Tue, Jul 26, 2022 at 07:21:29AM -0400, David Steele wrote: On 6/19/21 16:39, Noah Misch wrote: On Tue, Feb 02, 2021 at 07:14:16AM -0800, Noah Misch wrote: Recycling and preallocation are wasteful during archive recovery, because KeepFileRestoredFromArchiv

Re: Race between KeepFileRestoredFromArchive() and restartpoint

2022-07-30 Thread Noah Misch
On Tue, Jul 26, 2022 at 07:21:29AM -0400, David Steele wrote: > On 6/19/21 16:39, Noah Misch wrote: > >On Tue, Feb 02, 2021 at 07:14:16AM -0800, Noah Misch wrote: > >>Recycling and preallocation are wasteful during archive recovery, because > >>KeepFileRestoredFromArchive() unlinks every entry in i

Re: Race between KeepFileRestoredFromArchive() and restartpoint

2022-07-26 Thread David Steele
Hi Noah, On 6/19/21 16:39, Noah Misch wrote: On Tue, Feb 02, 2021 at 07:14:16AM -0800, Noah Misch wrote: Recycling and preallocation are wasteful during archive recovery, because KeepFileRestoredFromArchive() unlinks every entry in its path. I propose to fix the race by adding an XLogCtl flag

Re: Race between KeepFileRestoredFromArchive() and restartpoint

2021-06-19 Thread Noah Misch
On Tue, Feb 02, 2021 at 07:14:16AM -0800, Noah Misch wrote: > Recycling and preallocation are wasteful during archive recovery, because > KeepFileRestoredFromArchive() unlinks every entry in its path. I propose to > fix the race by adding an XLogCtl flag indicating which regime currently owns > th