Re: Add timeline to partial WAL segments

2020-09-30 Thread Michael Paquier
On Sat, Jan 05, 2019 at 08:32:55AM +0200, David Steele wrote: > Excellent. This now requires work on my part so I have marked the patch > "Waiting on Author". As nothing has happened here, I have marked the patch as RwF. -- Michael signature.asc Description: PGP signature

Re: Add timeline to partial WAL segments

2019-01-04 Thread David Steele
On 1/5/19 3:31 AM, Michael Paquier wrote: On Mon, Dec 31, 2018 at 01:07:34PM +0200, David Steele wrote: In short, the *initial* name of the WAL file is set to what it should be if it doesn't complete so we don't need to run around and try to rename files on failure. Only on success do we need t

Re: Add timeline to partial WAL segments

2019-01-04 Thread Michael Paquier
On Mon, Dec 31, 2018 at 01:07:34PM +0200, David Steele wrote: > In short, the *initial* name of the WAL file is set to what it should be if > it doesn't complete so we don't need to run around and try to rename files > on failure. Only on success do we need to rename. > > Sound plausible? It doe

Re: Add timeline to partial WAL segments

2018-12-31 Thread David Steele
On 12/21/18 2:10 AM, Michael Paquier wrote: On Thu, Dec 20, 2018 at 02:13:01PM +0200, David Steele wrote: Or perhaps just always add the timeline to the .partial? That way it doesn't need to be renamed later. Also, there would be a consistent name, rather than sometimes .partial, sometimes ..p

Re: Add timeline to partial WAL segments

2018-12-20 Thread David Steele
On 12/20/18 10:56 PM, Robert Haas wrote: On Fri, Dec 14, 2018 at 6:05 PM David Steele wrote: The question in my mind: is it safe to back-patch? I cannot imagine it being a good idea to stick a behavioral change like this into a minor release. Yeah, it lets people get out from under this prob

Re: Add timeline to partial WAL segments

2018-12-20 Thread Michael Paquier
On Thu, Dec 20, 2018 at 02:13:01PM +0200, David Steele wrote: > Or perhaps just always add the timeline to the .partial? That way it > doesn't need to be renamed later. Also, there would be a consistent name, > rather than sometimes .partial, sometimes ..partial. Hm. A renaming still needs to h

Re: Add timeline to partial WAL segments

2018-12-20 Thread Robert Haas
On Fri, Dec 14, 2018 at 6:05 PM David Steele wrote: > The question in my mind: is it safe to back-patch? I cannot imagine it being a good idea to stick a behavioral change like this into a minor release. Yeah, it lets people get out from under this problem a lot sooner, but it potentially breaks

Re: Add timeline to partial WAL segments

2018-12-20 Thread David Steele
On 12/15/18 1:56 AM, Michael Paquier wrote: On Fri, Dec 14, 2018 at 06:05:18PM -0500, David Steele wrote: On 12/14/18 3:26 PM, Robert Haas wrote: The new TLI is the only thing that is guaranteed to be unique with each new promotion, and I would guess that it is therefore the right thing to use

Re: Add timeline to partial WAL segments

2018-12-14 Thread Michael Paquier
On Fri, Dec 14, 2018 at 06:05:18PM -0500, David Steele wrote: > On 12/14/18 3:26 PM, Robert Haas wrote: >> The new TLI is the only thing that is guaranteed to be unique with >> each new promotion, and I would guess that it is therefore the right >> thing to use to disambiguate them. > > This is th

Re: Add timeline to partial WAL segments

2018-12-14 Thread David Steele
On 12/14/18 3:26 PM, Robert Haas wrote: > On Thu, Dec 13, 2018 at 12:17 AM Michael Paquier wrote: >> On Wed, Dec 12, 2018 at 07:54:05AM -0500, David Steele wrote: >>> The LSN switch point is often the same even when servers are going to >>> different timelines. If the LSN is different enough then

Re: Add timeline to partial WAL segments

2018-12-14 Thread Robert Haas
On Thu, Dec 13, 2018 at 12:17 AM Michael Paquier wrote: > On Wed, Dec 12, 2018 at 07:54:05AM -0500, David Steele wrote: > > The LSN switch point is often the same even when servers are going to > > different timelines. If the LSN is different enough then the problem > > solves itself since the .p

Re: Add timeline to partial WAL segments

2018-12-13 Thread David Steele
On 12/13/18 8:35 AM, David Steele wrote: > On 12/12/18 7:17 PM, Michael Paquier wrote: >> On Wed, Dec 12, 2018 at 07:54:05AM -0500, David Steele wrote: > >>> But, we could at least use the . notation and end up with something like >>> 000100010001.0002.partial or perhaps >>> 00

Re: Add timeline to partial WAL segments

2018-12-13 Thread David Steele
On 12/12/18 7:17 PM, Michael Paquier wrote: > On Wed, Dec 12, 2018 at 07:54:05AM -0500, David Steele wrote: >> But, we could at least use the . notation and end up with something like >> 000100010001.0002.partial or perhaps >> 000100010001.T0002.partial? Maybe >> 0

Re: Add timeline to partial WAL segments

2018-12-12 Thread Michael Paquier
On Wed, Dec 12, 2018 at 07:54:05AM -0500, David Steele wrote: > The LSN switch point is often the same even when servers are going to > different timelines. If the LSN is different enough then the problem > solves itself since the .partial will be on an entirely different > segment. That would me

Re: Add timeline to partial WAL segments

2018-12-12 Thread David Steele
On 12/12/18 12:14 AM, Michael Paquier wrote: > On Tue, Dec 11, 2018 at 11:27:30PM -0500, David Steele wrote: >> And yeah, I'm not sure that I'm totally sold on this idea either, but I >> wanted to get a conversation going. > > Another idea which I think we could live with is to embed within the >

Re: Add timeline to partial WAL segments

2018-12-11 Thread Michael Paquier
On Tue, Dec 11, 2018 at 11:27:30PM -0500, David Steele wrote: > And yeah, I'm not sure that I'm totally sold on this idea either, but I > wanted to get a conversation going. Another idea which I think we could live with is to embed within the segment file name the LSN switch point, in a format con

Re: Add timeline to partial WAL segments

2018-12-11 Thread David Steele
On 12/11/18 11:14 PM, Robert Haas wrote: > On Wed, Dec 12, 2018 at 1:13 PM David Steele wrote: >> On 12/11/18 11:06 PM, Robert Haas wrote: >>> On Wed, Dec 12, 2018 at 11:29 AM Michael Paquier >>> wrote: I really don't think that it is a good idea to link a future timeline within a segm

Re: Add timeline to partial WAL segments

2018-12-11 Thread Robert Haas
On Wed, Dec 12, 2018 at 1:13 PM David Steele wrote: > On 12/11/18 11:06 PM, Robert Haas wrote: > > On Wed, Dec 12, 2018 at 11:29 AM Michael Paquier > > wrote: > >> I really don't think that it is a good idea to link a future timeline > >> within a segment which includes in its name a direct refe

Re: Add timeline to partial WAL segments

2018-12-11 Thread David Steele
On 12/11/18 11:06 PM, Robert Haas wrote: > On Wed, Dec 12, 2018 at 11:29 AM Michael Paquier wrote: >> I really don't think that it is a good idea to link a future timeline >> within a segment which includes in its name a direct reference to its >> current timeline. Also I don't buy much the argu

Re: Add timeline to partial WAL segments

2018-12-11 Thread Robert Haas
On Wed, Dec 12, 2018 at 11:29 AM Michael Paquier wrote: > I really don't think that it is a good idea to link a future timeline > within a segment which includes in its name a direct reference to its > current timeline. Also I don't buy much the argument that those > segments are a nuisance as w

Re: Add timeline to partial WAL segments

2018-12-11 Thread Michael Paquier
On Tue, Dec 11, 2018 at 09:15:16AM -0500, David Steele wrote: > It looks to me like the history file is written first, and then the > .partial. But because we process WAL alphabetically the .partial ends > up being pushed first. Yes, that's right. > The idea is to stake a claim to the timeline a

Re: Add timeline to partial WAL segments

2018-12-11 Thread David Steele
Hi Michael, On 12/10/18 8:43 PM, Michael Paquier wrote: > On Mon, Dec 10, 2018 at 10:21:23AM -0500, David Steele wrote: >> We recommend that archive commands not overwrite an existing segment. >> Some backup tools will compare the contents and succeed if they are >> equal, but in this case that wi

Re: Add timeline to partial WAL segments

2018-12-10 Thread Michael Paquier
On Mon, Dec 10, 2018 at 10:21:23AM -0500, David Steele wrote: > We recommend that archive commands not overwrite an existing segment. > Some backup tools will compare the contents and succeed if they are > equal, but in this case that will still often fail because recycled WAL > segments will have