Brian Weaver writes:
> Here's a very minimal fix then, perhaps it will be more palatable.
I did some further work on this to improve comments and clean up the
pg_dump end of things, and have committed it.
> Even though I regret the effort I put into the first patch it's in my
> employer's best i
On Thu, Sep 27, 2012 at 6:43 PM, Tom Lane wrote:
> Brian Weaver writes:
>> OK, here is my attempt at patching and correcting the issue in this
>> thread. I have done my best to test to ensure that hot standby,
>> pg_basebackup, and pg_restore of older files work without issues. I
>> think this mi
Magnus Hagander writes:
> On Fri, Sep 28, 2012 at 12:55 AM, Tom Lane wrote:
>> I think it's clear that we should make all versions of pg_restore accept
>> either spelling of the magic string. It's less clear that we should
>> change the output of pg_dump in back branches though. I think the onl
On Fri, Sep 28, 2012 at 12:55 AM, Tom Lane wrote:
> Andrew Dunstan writes:
>> On 09/27/2012 06:30 PM, Tom Lane wrote:
>>> Having said all that, I don't think we have a lot of choices here.
>>> A "tar format" output option that isn't actually tar format has hardly
>>> any excuse to live at all.
>
Andrew Dunstan writes:
> On 09/27/2012 06:30 PM, Tom Lane wrote:
>> Having said all that, I don't think we have a lot of choices here.
>> A "tar format" output option that isn't actually tar format has hardly
>> any excuse to live at all.
> I agree, but it's possibly worth pointing out that GNU t
On Fri, Sep 28, 2012 at 12:45 AM, Tom Lane wrote:
> Magnus Hagander writes:
>> Ah, yeah, that should also work I guess. But you could also just leave
>> the the data in the temporary tarfile the whole time. IIRC, you can
>> just cat one tarfile to the end of another one, right?
>
> Not if they're
On 09/27/2012 06:30 PM, Tom Lane wrote:
Magnus Hagander writes:
On Tue, Sep 25, 2012 at 4:07 AM, Tom Lane wrote:
I'm a bit concerned about backwards compatibility issues. It looks to
me like existing versions of pg_restore will flat out reject files that
have a spec-compliant "ustar\0" MAGI
Magnus Hagander writes:
> Ah, yeah, that should also work I guess. But you could also just leave
> the the data in the temporary tarfile the whole time. IIRC, you can
> just cat one tarfile to the end of another one, right?
Not if they're written according to spec, I think. There is an EOF
marke
Brian Weaver writes:
> OK, here is my attempt at patching and correcting the issue in this
> thread. I have done my best to test to ensure that hot standby,
> pg_basebackup, and pg_restore of older files work without issues. I
> think this might be a larger patch that expected, I took some
> liber
On Fri, Sep 28, 2012 at 12:30 AM, Tom Lane wrote:
> Magnus Hagander writes:
>> On Tue, Sep 25, 2012 at 4:07 AM, Tom Lane wrote:
>>> I'm a bit concerned about backwards compatibility issues. It looks to
>>> me like existing versions of pg_restore will flat out reject files that
>>> have a spec-c
Magnus Hagander writes:
> On Tue, Sep 25, 2012 at 4:07 AM, Tom Lane wrote:
>> I'm a bit concerned about backwards compatibility issues. It looks to
>> me like existing versions of pg_restore will flat out reject files that
>> have a spec-compliant "ustar\0" MAGIC field. Is it going to be
>> suf
On Fri, Sep 28, 2012 at 12:12 AM, Brian Weaver wrote:
> Magnus,
>
> I probably just did a poor job of explaining what I wanted to try. I
> was going to have the base backup open two connections; one to stream
> the tar archive, the second to receive the wal files like
> pg_receivexlog.
This is wh
OK, here is my attempt at patching and correcting the issue in this
thread. I have done my best to test to ensure that hot standby,
pg_basebackup, and pg_restore of older files work without issues. I
think this might be a larger patch that expected, I took some
liberties of trying to clean up a bit
Magnus,
I probably just did a poor job of explaining what I wanted to try. I
was going to have the base backup open two connections; one to stream
the tar archive, the second to receive the wal files like
pg_receivexlog.
The wal files received on the second connection would be streamed to a
tempo
On Tue, Sep 25, 2012 at 4:07 AM, Tom Lane wrote:
> Brian Weaver writes:
>> While researching the way streaming replication works I was examining
>> the construction of the tar file header. By comparing documentation on
>> the tar header format from various sources I certain the following
>> patch
On Tue, Sep 25, 2012 at 5:08 PM, Brian Weaver wrote:
> Unless I misread the code, the tar format and streaming xlog are
> mutually exclusive. Considering my normal state of fatigue it's not
> unlikely. I don't want to have to set my wal_keep_segments
> artificially high just for the backup
Correc
Tom,
I'm fine with submitting highly focused patches first. I was just
explaining my end-goal. Still I will need time to patch, compile, and
test before submitting so you're not going to see any output from me
for a few days. That's all assuming my employer can leave me alone
long enough to focus
Unless I misread the code, the tar format and streaming xlog are
mutually exclusive. Considering my normal state of fatigue it's not
unlikely. I don't want to have to set my wal_keep_segments
artificially high just for the backup
On Tue, Sep 25, 2012 at 10:05 AM, Marko Tiikkaja wrote:
> On 9/25/1
Brian Weaver writes:
> If you're willing to wait a bit on me to code and test my extensions
> to pg_basebackup I will try to address some of the deficiencies as
> well add new features.
I think it's a mistake to try to handle these issues in the same patch
as feature extensions. If you want to s
On 9/25/12 3:38 PM, Brian Weaver wrote:
I want
to modify pg_basebackup to include the WAL files in the tar output.
Doesn't pg_basebackup -x do exactly that?
Regards,
Marko Tiikkaja
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
h
Tom,
I actually plan on doing a lot of work on the frontend pg_basebackup
for my employer. pg_basebackup is 90% of the way to a solution that I
need for doing backups of *large* databases while allowing the
database to continue to work. The problem is a lack of secondary disk
space to save a repli
Brian Weaver writes:
> Here are lines 321 through 329 of 'archive_read_support_format_tar.c'
> from libarchive
> 321 /* Recognize POSIX formats. */
> 322 if ((memcmp(header->magic, "ustar\0", 6) == 0)
> 323 && (memcmp(header->version, "00", 2) == 0))
> 324
Tom,
I'm still investigating and I have been looking at various sources. I
have checked lots of pages on the web and I was just looking at the
libarchive source from github. I found an interesting sequence in
libarchive that implies that the 'ustar00\0' marks the header as GNU
Tar format.
Here ar
Brian Weaver writes:
> While researching the way streaming replication works I was examining
> the construction of the tar file header. By comparing documentation on
> the tar header format from various sources I certain the following
> patch should be applied to so the group identifier is put int
Um I apologize for the third e-mail on the topic. It seems that my
C coding is a bit rusty from years of neglect. No sooner had I hit the
send button then I realized that trying to embed a null character in a
string might not work, especially when it's followed by two
consecutive zeros.
Here i
Actually I found one other issue while continuing my investigation.
The insertion of the 'ustar' and version '00' has the '00' version at
the wrong offset. The patch is attached.
-- Brian
On Mon, Sep 24, 2012 at 7:51 PM, Brian Weaver wrote:
> While researching the way streaming replication works
While researching the way streaming replication works I was examining
the construction of the tar file header. By comparing documentation on
the tar header format from various sources I certain the following
patch should be applied to so the group identifier is put into thee
header properly.
While
27 matches
Mail list logo
