On Mon, Mar 26, 2018 at 5:16 PM, Masahiko Sawada
wrote:
> On Mon, Mar 26, 2018 at 4:52 PM, Pavan Deolasee
>
> >>
> >
> > This one-liner patch fixes it for me.
> >
>
> Isn't this issue already fixed by commit
> d0c0c894533f906b13b79813f02b2982ac675074?
Ah, right. Thanks for pointing out and sorr
On Mon, Mar 26, 2018 at 4:52 PM, Pavan Deolasee
wrote:
>
>
> On Mon, Mar 26, 2018 at 1:03 PM, Pavan Deolasee
> wrote:
>>
>> On Fri, Mar 23, 2018 at 9:51 PM, David Steele wrote:
>>>
>>> On 3/23/18 12:14 PM, Teodor Sigaev wrote:
>>> >
>>> > Thank you, pushed
>>>
>>
>> Is it just me or the newly ad
On Mon, Mar 26, 2018 at 1:03 PM, Pavan Deolasee
wrote:
> On Fri, Mar 23, 2018 at 9:51 PM, David Steele wrote:
>
>> On 3/23/18 12:14 PM, Teodor Sigaev wrote:
>> >
>> > Thank you, pushed
>>
>>
> Is it just me or the newly added test in 010_pg_basebackup.pl failing for
> others too?
>
> # Failed
On Fri, Mar 23, 2018 at 9:51 PM, David Steele wrote:
> On 3/23/18 12:14 PM, Teodor Sigaev wrote:
> >
> > Thank you, pushed
>
>
Is it just me or the newly added test in 010_pg_basebackup.pl failing for
others too?
# Failed test 'unlogged main fork not in backup'
# at t/010_pg_basebackup.pl li
On 3/23/18 12:14 PM, Teodor Sigaev wrote:
>
> Thank you, pushed
Thank you, Teodor! I'll rebase the temp table exclusion patch and
provide an updated patch soon.
--
-David
da...@pgmasters.net
Thank you, pushed
David Steele wrote:
On 1/29/18 8:10 PM, Masahiko Sawada wrote:
On Tue, Jan 30, 2018 at 5:45 AM, Adam Brightwell
If it is agreed that the temp file exclusion should be submitted as a
separate patch, then I will mark 'ready for committer'.
Agreed, please mark this patch as
On 1/29/18 8:10 PM, Masahiko Sawada wrote:
> On Tue, Jan 30, 2018 at 5:45 AM, Adam Brightwell
>>
>> If it is agreed that the temp file exclusion should be submitted as a
>> separate patch, then I will mark 'ready for committer'.
>
> Agreed, please mark this patch as "Ready for Committer".
Attache
On 1/29/18 8:10 PM, Masahiko Sawada wrote:
> On Tue, Jan 30, 2018 at 5:45 AM, Adam Brightwell
> wrote:
>> On Mon, Jan 29, 2018 at 1:17 PM, David Steele wrote:
>>>
>>> Whoops, my bad. Temp relations are stored in the db directories with a
>>> "t" prefix. Looks like we can take care of those easi
On Tue, Jan 30, 2018 at 5:45 AM, Adam Brightwell
wrote:
> On Mon, Jan 29, 2018 at 1:17 PM, David Steele wrote:
>> On 1/29/18 9:13 AM, David Steele wrote:
>>> On 1/29/18 5:28 AM, Masahiko Sawada wrote:
But I
have a question; can we exclude temp tables as well? The pg_basebackup
incl
On Mon, Jan 29, 2018 at 1:17 PM, David Steele wrote:
> On 1/29/18 9:13 AM, David Steele wrote:
>> On 1/29/18 5:28 AM, Masahiko Sawada wrote:
>>> But I
>>> have a question; can we exclude temp tables as well? The pg_basebackup
>>> includes even temp tables. But I don't think that it's necessary for
On 1/29/18 9:13 AM, David Steele wrote:
> On 1/29/18 5:28 AM, Masahiko Sawada wrote:
>> But I
>> have a question; can we exclude temp tables as well? The pg_basebackup
>> includes even temp tables. But I don't think that it's necessary for
>> backups
> Thank you for having another look at the patch
On 1/29/18 5:28 AM, Masahiko Sawada wrote:
> On Fri, Jan 26, 2018 at 4:58 AM, David Steele wrote:
>>
>> Attached is a new patch that uses stat() to determine if the init fork
>> for a relation file exists. I decided not to build a hash table as it
>> could use considerable memory and I didn't thi
On Mon, Jan 29, 2018 at 07:28:22PM +0900, Masahiko Sawada wrote:
> Thank you for updating the patch! The patch looks good to me. But I
> have a question; can we exclude temp tables as well? The pg_basebackup
> includes even temp tables. But I don't think that it's necessary for
> backups.
They are
On Fri, Jan 26, 2018 at 4:58 AM, David Steele wrote:
> On 1/25/18 12:31 AM, Masahiko Sawada wrote:
>> On Thu, Jan 25, 2018 at 3:25 AM, David Steele wrote:
Here is the first review comments.
+ unloggedDelim = strrchr(path, '/');
I think it doesn't work fine on w
On Wed, Jan 24, 2018 at 1:25 PM, David Steele wrote:
> I think you mean DEBUG1? It's already at DEBUG2.
>
> I considered using DEBUG1 but decided against it. The other exclusions
> will produce a limited amount of output because there are only a few of
> them. In the case of unlogged tables the
On 1/25/18 12:31 AM, Masahiko Sawada wrote:
> On Thu, Jan 25, 2018 at 3:25 AM, David Steele wrote:
>>>
>>> Here is the first review comments.
>>>
>>> + unloggedDelim = strrchr(path, '/');
>>>
>>> I think it doesn't work fine on windows. How about using
>>> last_dir_separator() instead?
>>
>>
On Thu, Jan 25, 2018 at 3:25 AM, David Steele wrote:
> Hi Masahiko,
>
> Thanks for the review!
>
> On 1/22/18 3:14 AM, Masahiko Sawada wrote:
>> On Thu, Dec 14, 2017 at 11:58 PM, Robert Haas wrote:
>>>
>>> We would also have a problem if the missing file caused something in
>>> recovery to croak
On 1/24/18 4:02 PM, Adam Brightwell wrote:
>>> If a new unlogged relation is created after constructed the
>>> unloggedHash before sending file, we cannot exclude such relation. It
>>> would not be problem if the taking backup is not long because the new
>>> unlogged relation unlikely becomes so la
> I agree with #1 and feel the updated docs are reasonable and
> sufficient to address this case for now.
>
> I have retested these patches against master at d6ab720360.
>
> All test succeed.
>
> Marking "Ready for Committer".
Actually, marked it "Ready for Review" to wait for Masahiko to comment/
>> If a new unlogged relation is created after constructed the
>> unloggedHash before sending file, we cannot exclude such relation. It
>> would not be problem if the taking backup is not long because the new
>> unlogged relation unlikely becomes so large. However, if takeing a
>> backup takes a lo
Hi Masahiko,
Thanks for the review!
On 1/22/18 3:14 AM, Masahiko Sawada wrote:
> On Thu, Dec 14, 2017 at 11:58 PM, Robert Haas wrote:
>>
>> We would also have a problem if the missing file caused something in
>> recovery to croak on the grounds that the file was expected to be
>> there, but I do
On Thu, Dec 14, 2017 at 11:58 PM, Robert Haas wrote:
> On Tue, Dec 12, 2017 at 8:48 PM, Stephen Frost wrote:
>> If the persistence is changed then the table will be written into the
>> WAL, no? All of the WAL generated during a backup (which is what we're
>> talking about here) has to be replaye
All,
I have reviewed and tested these patches.
The patches applied cleanly in order against master at (90947674fc).
I ran the provided regression tests and a 'check-world'. All tests succeeded.
Marking ready for committer.
-Adam
On Tue, Dec 12, 2017 at 8:48 PM, Stephen Frost wrote:
> If the persistence is changed then the table will be written into the
> WAL, no? All of the WAL generated during a backup (which is what we're
> talking about here) has to be replayed after the restore is done and is
> before the database is
On 12/13/17 10:04 AM, Stephen Frost wrote:
Just to be clear- the new base backup code doesn't actually *do* the
non-init fork removal, it simply doesn't include the non-init fork in
the backup when there is an init fork, right?
It does *not* do the unlogged non-init fork removal. The code I
David,
* David Steele (da...@pgmasters.net) wrote:
> On 12/12/17 8:48 PM, Stephen Frost wrote:
> > I don't think there is, because, as David points out, the unlogged
> > tables are cleaned up first and then WAL replay happens during recovery,
> > so the init fork will cause the relation to be over
On 12/12/17 8:48 PM, Stephen Frost wrote:
> Andres,
>
> * Andres Freund (and...@anarazel.de) wrote:
>> On 2017-12-12 18:04:44 -0500, David Steele wrote:
>>> If the forks are written out of order (i.e. main before init), which is
>>> definitely possible, then I think worst case is some files will b
Andres,
* Andres Freund (and...@anarazel.de) wrote:
> On 2017-12-12 18:04:44 -0500, David Steele wrote:
> > If the forks are written out of order (i.e. main before init), which is
> > definitely possible, then I think worst case is some files will be backed up
> > that don't need to be. The main
On 12/12/17 6:33 PM, Andres Freund wrote:
On 2017-12-12 18:30:47 -0500, David Steele wrote:
If we had a way to prevent relfilenode reuse across multiple checkpoints
this'd be easier, although ALTER TABLE SET UNLOGGED still'd complicate.
Or error the backup if there is wraparound?
That seems
Hi,
On 2017-12-12 18:30:47 -0500, David Steele wrote:
> > If we had a way to prevent relfilenode reuse across multiple checkpoints
> > this'd be easier, although ALTER TABLE SET UNLOGGED still'd complicate.
>
> Or error the backup if there is wraparound?
That seems entirely unacceptable to me. O
On 12/12/17 6:21 PM, Andres Freund wrote:
On 2017-12-12 18:18:09 -0500, David Steele wrote:
On 12/12/17 6:07 PM, Andres Freund wrote:
It's quite different - in the recovery case there's no other write
activity going on. But on a normally running cluster the persistence of
existing tables can g
Hi Michael,
On 12/12/17 6:08 PM, Michael Paquier wrote:
If the forks are written out of order (i.e. main before init), which is
definitely possible, then I think worst case is some files will be backed up
that don't need to be. The main fork is unlikely to be very large at that
point so it do
On 2017-12-12 18:18:09 -0500, David Steele wrote:
> On 12/12/17 6:07 PM, Andres Freund wrote:
> > >
> > > I don't see this as any different than what happens during recovery. The
> > > unlogged forks are cleaned / re-inited before replay starts which is the
> > > same thing we are doing here.
> >
On 12/12/17 6:07 PM, Andres Freund wrote:
I don't see this as any different than what happens during recovery. The
unlogged forks are cleaned / re-inited before replay starts which is the
same thing we are doing here.
It's quite different - in the recovery case there's no other write
activity
On Wed, Dec 13, 2017 at 8:04 AM, David Steele wrote:
> On 12/12/17 5:52 PM, Andres Freund wrote:
>> On 2017-12-12 17:49:54 -0500, David Steele wrote:
>>>
>>> Including unlogged relations in base backups takes up space and is
>>> wasteful
>>> since they are truncated during backup recovery.
>>>
>>>
Hi,
On 2017-12-12 18:04:44 -0500, David Steele wrote:
> On 12/12/17 5:52 PM, Andres Freund wrote:
> > On 2017-12-12 17:49:54 -0500, David Steele wrote:
> > > Including unlogged relations in base backups takes up space and is
> > > wasteful
> > > since they are truncated during backup recovery.
>
Hi Andres,
On 12/12/17 5:52 PM, Andres Freund wrote:
On 2017-12-12 17:49:54 -0500, David Steele wrote:
Including unlogged relations in base backups takes up space and is wasteful
since they are truncated during backup recovery.
The attached patches exclude unlogged relations from base backups
Hi,
On 2017-12-12 17:49:54 -0500, David Steele wrote:
> Including unlogged relations in base backups takes up space and is wasteful
> since they are truncated during backup recovery.
>
> The attached patches exclude unlogged relations from base backups except for
> the init fork, which is require
Including unlogged relations in base backups takes up space and is
wasteful since they are truncated during backup recovery.
The attached patches exclude unlogged relations from base backups except
for the init fork, which is required to recreate the main fork during
recovery.
* exclude-unlo
39 matches
Mail list logo
