Magnus Hagander writes:
>> Until we get integrated WAL streaming while the base backup is ongoing.
>> We don't know when that is (9.1 or future), but that's what we're aiming
>> to now, right?
>
> Yeah, it does sound like a plan. But to still allow both - streaming
> it in parallell will eat two c
On Mon, Jan 17, 2011 at 11:18, Dimitri Fontaine wrote:
> Magnus Hagander writes:
>> With pg_basebackup, you can set up streaming replication in what's
>> basically a single command (run the base backup, copy i na
>> recovery.conf file). In my first version I even had a switch that
>> would create
Magnus Hagander writes:
> With pg_basebackup, you can set up streaming replication in what's
> basically a single command (run the base backup, copy i na
> recovery.conf file). In my first version I even had a switch that
> would create the recovery.conf file for you - should we bring that
> back?
On Mon, Jan 17, 2011 at 03:32, Tatsuo Ishii wrote:
>>> Good point. I have been always wondering why we can't use exiting WAL
>>> transporting infrastructure for sending/receiving WAL archive
>>> segments in streaming replication.
>>> If my memory serves, Fujii has already proposed such an idea but
On Mon, Jan 17, 2011 at 11:32 AM, Tatsuo Ishii wrote:
>>> Good point. I have been always wondering why we can't use exiting WAL
>>> transporting infrastructure for sending/receiving WAL archive
>>> segments in streaming replication.
>>> If my memory serves, Fujii has already proposed such an idea
>> Good point. I have been always wondering why we can't use exiting WAL
>> transporting infrastructure for sending/receiving WAL archive
>> segments in streaming replication.
>> If my memory serves, Fujii has already proposed such an idea but was
>> rejected for some reason I don't understand.
>
On Sat, Jan 15, 2011 at 8:33 PM, Tatsuo Ishii wrote:
>> When do the standby launch its walreceiver? It would be extra-nice for
>> the base backup tool to optionally continue streaming WALs until the
>> standby starts doing it itself, so that wal_keep_segments is really
>> deprecated. No idea how
> When do the standby launch its walreceiver? It would be extra-nice for
> the base backup tool to optionally continue streaming WALs until the
> standby starts doing it itself, so that wal_keep_segments is really
> deprecated. No idea how feasible that is, though.
Good point. I have been always
On Sat, Jan 15, 2011 at 19:27, Tom Lane wrote:
> Magnus Hagander writes:
>> Something like this to fix? or is this going to put those "warnings by
>> stupid versions of gcc" back?
>
> Possibly. If so, I'll fix it --- I have an old gcc to test against
> here.
Ok, thanks, I'll commit tihs one the
Magnus Hagander writes:
> Something like this to fix? or is this going to put those "warnings by
> stupid versions of gcc" back?
Possibly. If so, I'll fix it --- I have an old gcc to test against
here.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hac
On Sat, Jan 15, 2011 at 16:54, Tom Lane wrote:
> Heikki Linnakangas writes:
>> On 15.01.2011 17:30, Tom Lane wrote:
>>> So what? The needed actions will be covered by WAL replay.
>
>> No, they won't, if pg_base_backup() is called *after* getting the list
>> of tablespaces.
>
> Ah. Then the fix
Heikki Linnakangas writes:
> On 15.01.2011 17:30, Tom Lane wrote:
>> So what? The needed actions will be covered by WAL replay.
> No, they won't, if pg_base_backup() is called *after* getting the list
> of tablespaces.
Ah. Then the fix is to change the order in which those things are done.
On 15.01.2011 17:30, Tom Lane wrote:
Heikki Linnakangas writes:
BTW, I just spotted a small race condition between creating a new table
space and base backup. We take a snapshot of all the tablespaces in
pg_tblspc before calling pg_start_backup(). If someone creates a new
tablespace and puts so
Heikki Linnakangas writes:
> BTW, I just spotted a small race condition between creating a new table
> space and base backup. We take a snapshot of all the tablespaces in
> pg_tblspc before calling pg_start_backup(). If someone creates a new
> tablespace and puts some data in it in the window b
On 14.01.2011 13:38, Magnus Hagander wrote:
On Fri, Jan 14, 2011 at 11:19, Heikki Linnakangas
wrote:
On 14.01.2011 08:45, Fujii Masao wrote:
1. Smart shutdown is requested while walsender is sending a backup.
2. Shutdown causes archiver to end.
(Though shutdown sends SIGUSR2 to walsende
On Fri, Jan 14, 2011 at 11:19, Heikki Linnakangas
wrote:
> On 14.01.2011 08:45, Fujii Masao wrote:
>>
>> On Fri, Jan 14, 2011 at 4:13 AM, Magnus Hagander
>> wrote:
At the end of the backup by walsender, it forces a switch to a new
WAL file and waits until the last WAL file has been
On 14.01.2011 08:45, Fujii Masao wrote:
On Fri, Jan 14, 2011 at 4:13 AM, Magnus Hagander wrote:
At the end of the backup by walsender, it forces a switch to a new
WAL file and waits until the last WAL file has been archived. So we
should change postmaster so that it doesn't cause the archiver t
On Fri, Jan 14, 2011 at 4:13 AM, Magnus Hagander wrote:
>> While walsender is sending a base backup, WalSndWakeup should
>> not send the signal to that walsender?
>
> True, it's not necessary. How bad does it actually hurt things though?
> Given that the walsender running the backup isn't actually
On Wed, Jan 12, 2011 at 10:39, Fujii Masao wrote:
> On Mon, Jan 10, 2011 at 11:09 PM, Magnus Hagander wrote:
>> I've committed the backend side of this, without that. Still working
>> on the client, and on cleaning up Heikki's patch for grammar/parser
>> support.
>
> Great work!
>
> I have some c
On Mon, Jan 10, 2011 at 11:09 PM, Magnus Hagander wrote:
> I've committed the backend side of this, without that. Still working
> on the client, and on cleaning up Heikki's patch for grammar/parser
> support.
Great work!
I have some comments:
While walsender is sending a base backup, WalSndWake
On Tue, Jan 11, 2011 at 12:45:02PM -0500, Tom Lane wrote:
> Florian Pflug writes:
> > On Jan11, 2011, at 18:09 , Garick Hamlin wrote:
> >> My gut was that direct io would likely work right on Linux
> >> and Solaris, at least.
>
> > Didn't we discover recently that O_DIRECT fails for ext4 on linux
Florian Pflug writes:
> On Jan11, 2011, at 18:09 , Garick Hamlin wrote:
>> My gut was that direct io would likely work right on Linux
>> and Solaris, at least.
> Didn't we discover recently that O_DIRECT fails for ext4 on linux
> if ordered=data, or something like that?
Quite. Blithe assertions
2011/1/11 Garick Hamlin :
> On Tue, Jan 11, 2011 at 11:39:20AM -0500, Cédric Villemain wrote:
>> 2011/1/11 Garick Hamlin :
>> > On Mon, Jan 10, 2011 at 09:09:28AM -0500, Magnus Hagander wrote:
>> >> On Sun, Jan 9, 2011 at 23:33, Cédric Villemain
>> >> wrote:
>> >> > 2011/1/7 Magnus Hagander :
>> >
On Jan11, 2011, at 18:09 , Garick Hamlin wrote:
> My gut was that direct io would likely work right on Linux
> and Solaris, at least.
Didn't we discover recently that O_DIRECT fails for ext4 on linux
if ordered=data, or something like that?
best regards,
Florian Pflug
--
Sent via pgsql-hacker
On Tue, Jan 11, 2011 at 11:39:20AM -0500, Cédric Villemain wrote:
> 2011/1/11 Garick Hamlin :
> > On Mon, Jan 10, 2011 at 09:09:28AM -0500, Magnus Hagander wrote:
> >> On Sun, Jan 9, 2011 at 23:33, Cédric Villemain
> >> wrote:
> >> > 2011/1/7 Magnus Hagander :
> >> >> On Fri, Jan 7, 2011 at 01:47,
2011/1/11 Garick Hamlin :
> On Mon, Jan 10, 2011 at 09:09:28AM -0500, Magnus Hagander wrote:
>> On Sun, Jan 9, 2011 at 23:33, Cédric Villemain
>> wrote:
>> > 2011/1/7 Magnus Hagander :
>> >> On Fri, Jan 7, 2011 at 01:47, Cédric Villemain
>> >> wrote:
>> >>> 2011/1/5 Magnus Hagander :
>> On W
On Mon, Jan 10, 2011 at 09:09:28AM -0500, Magnus Hagander wrote:
> On Sun, Jan 9, 2011 at 23:33, Cédric Villemain
> wrote:
> > 2011/1/7 Magnus Hagander :
> >> On Fri, Jan 7, 2011 at 01:47, Cédric Villemain
> >> wrote:
> >>> 2011/1/5 Magnus Hagander :
> On Wed, Jan 5, 2011 at 22:58, Dimitri F
On Tue, Jan 11, 2011 at 01:28, Cédric Villemain
wrote:
> 2011/1/10 Magnus Hagander :
>> On Sun, Jan 9, 2011 at 23:33, Cédric Villemain
>> wrote:
>> I've committed the backend side of this, without that. Still working
>> on the client, and on cleaning up Heikki's patch for grammar/parser
>> suppor
2011/1/10 Magnus Hagander :
> On Sun, Jan 9, 2011 at 23:33, Cédric Villemain
> wrote:
>> 2011/1/7 Magnus Hagander :
>>> On Fri, Jan 7, 2011 at 01:47, Cédric Villemain
>>> wrote:
2011/1/5 Magnus Hagander :
> On Wed, Jan 5, 2011 at 22:58, Dimitri Fontaine
> wrote:
>> Magnus Hagan
2011/1/10 Stefan Kaltenbrunner :
> On 01/10/2011 08:13 PM, Cédric Villemain wrote:
>>
>> 2011/1/10 Magnus Hagander:
>>>
>>> On Sun, Jan 9, 2011 at 23:33, Cédric Villemain
>>> wrote:
2011/1/7 Magnus Hagander:
>
> On Fri, Jan 7, 2011 at 01:47, Cédric Villemain
> wrote:
>
On 01/10/2011 08:13 PM, Cédric Villemain wrote:
2011/1/10 Magnus Hagander:
On Sun, Jan 9, 2011 at 23:33, Cédric Villemain
wrote:
2011/1/7 Magnus Hagander:
On Fri, Jan 7, 2011 at 01:47, Cédric Villemain
wrote:
2011/1/5 Magnus Hagander:
On Wed, Jan 5, 2011 at 22:58, Dimitri Fontaine wrote
2011/1/10 Magnus Hagander :
> On Sun, Jan 9, 2011 at 23:33, Cédric Villemain
> wrote:
>> 2011/1/7 Magnus Hagander :
>>> On Fri, Jan 7, 2011 at 01:47, Cédric Villemain
>>> wrote:
2011/1/5 Magnus Hagander :
> On Wed, Jan 5, 2011 at 22:58, Dimitri Fontaine
> wrote:
>> Magnus Hagan
On Sun, Jan 9, 2011 at 23:33, Cédric Villemain
wrote:
> 2011/1/7 Magnus Hagander :
>> On Fri, Jan 7, 2011 at 01:47, Cédric Villemain
>> wrote:
>>> 2011/1/5 Magnus Hagander :
On Wed, Jan 5, 2011 at 22:58, Dimitri Fontaine
wrote:
> Magnus Hagander writes:
>> * Stefan mentiond i
2011/1/7 Magnus Hagander :
> On Fri, Jan 7, 2011 at 01:47, Cédric Villemain
> wrote:
>> 2011/1/5 Magnus Hagander :
>>> On Wed, Jan 5, 2011 at 22:58, Dimitri Fontaine
>>> wrote:
Magnus Hagander writes:
> * Stefan mentiond it might be useful to put some
> posix_fadvise(POSIX_FADV_DON
2011/1/7 Garick Hamlin :
> On Thu, Jan 06, 2011 at 07:47:39PM -0500, Cédric Villemain wrote:
>> 2011/1/5 Magnus Hagander :
>> > On Wed, Jan 5, 2011 at 22:58, Dimitri Fontaine
>> > wrote:
>> >> Magnus Hagander writes:
>> >>> * Stefan mentiond it might be useful to put some
>> >>> posix_fadvise(PO
On Sun, Jan 9, 2011 at 12:05, Hannu Krosing wrote:
> On 9.1.2011 10:44, Magnus Hagander wrote:
>>
>> On Sun, Jan 9, 2011 at 09:55, Hannu Krosing wrote:
>>>
>>> On 7.1.2011 15:45, Magnus Hagander wrote:
On Fri, Jan 7, 2011 at 02:15, Simon Riggs
wrote:
> One very useful fea
On 9.1.2011 10:44, Magnus Hagander wrote:
On Sun, Jan 9, 2011 at 09:55, Hannu Krosing wrote:
On 7.1.2011 15:45, Magnus Hagander wrote:
On Fri, Jan 7, 2011 at 02:15, Simon Riggswrote:
One very useful feature will be some way of confirming the number and
size of files to transfer, so that
On Sun, Jan 9, 2011 at 09:55, Hannu Krosing wrote:
> On 7.1.2011 15:45, Magnus Hagander wrote:
>>
>> On Fri, Jan 7, 2011 at 02:15, Simon Riggs wrote:
>>
>>> One very useful feature will be some way of confirming the number and
>>> size of files to transfer, so that the base backup client can find
On 7.1.2011 15:45, Magnus Hagander wrote:
On Fri, Jan 7, 2011 at 02:15, Simon Riggs wrote:
One very useful feature will be some way of confirming the number and
size of files to transfer, so that the base backup client can find out
the progress.
The patch already does this. Or rather, as it's
On Thu, Jan 6, 2011 at 23:57, Heikki Linnakangas
wrote:
>
> Looks like pg_streamrecv creates the pg_xlog and pg_tblspc directories,
> because they're not included in the streamed tar. Wouldn't it be better to
> include them in the tar as empty directories at the server-side? Otherwise
> if you wri
On 05.01.2011 15:54, Magnus Hagander wrote:
I've implemented a frontend for this in pg_streamrecv, based on the assumption
that we wanted to include this in bin/ for 9.1 - and that it seems like a
reasonable place to put it. This can obviously be moved elsewhere if we want to.
That code needs a l
On 05.01.2011 15:54, Magnus Hagander wrote:
* Suggestion from Heikki: perhaps at some point we're going to need a full
bison grammar for walsender commands.
Here's a patch for this (Also available at
g...@github.com:hlinnaka/postgres.git, branch "streaming_base"). I
thought I know our biso
On Fri, Jan 07, 2011 at 10:26:29AM -0500, Garick Hamlin wrote:
> On Thu, Jan 06, 2011 at 07:47:39PM -0500, Cédric Villemain wrote:
> > 2011/1/5 Magnus Hagander :
> > > On Wed, Jan 5, 2011 at 22:58, Dimitri Fontaine
> > > wrote:
> > >> Magnus Hagander writes:
> > >>> * Stefan mentiond it might be
On Thu, Jan 06, 2011 at 07:47:39PM -0500, Cédric Villemain wrote:
> 2011/1/5 Magnus Hagander :
> > On Wed, Jan 5, 2011 at 22:58, Dimitri Fontaine
> > wrote:
> >> Magnus Hagander writes:
> >>> * Stefan mentiond it might be useful to put some
> >>> posix_fadvise(POSIX_FADV_DONTNEED)
> >>> in the
On Fri, Jan 7, 2011 at 01:47, Cédric Villemain
wrote:
> 2011/1/5 Magnus Hagander :
>> On Wed, Jan 5, 2011 at 22:58, Dimitri Fontaine
>> wrote:
>>> Magnus Hagander writes:
* Stefan mentiond it might be useful to put some
posix_fadvise(POSIX_FADV_DONTNEED)
in the process that str
On Fri, Jan 7, 2011 at 02:15, Simon Riggs wrote:
> On Wed, 2011-01-05 at 14:54 +0100, Magnus Hagander wrote:
>
>> The basic implementation is: Add a new command to the replication mode called
>> BASE_BACKUP, that will initiate a base backup, stream the contents (in tar
>> compatible format) of the
On Wed, 2011-01-05 at 14:54 +0100, Magnus Hagander wrote:
> The basic implementation is: Add a new command to the replication mode called
> BASE_BACKUP, that will initiate a base backup, stream the contents (in tar
> compatible format) of the data directory and all tablespaces, and then end
> the
2011/1/5 Magnus Hagander :
> On Wed, Jan 5, 2011 at 22:58, Dimitri Fontaine wrote:
>> Magnus Hagander writes:
>>> * Stefan mentiond it might be useful to put some
>>> posix_fadvise(POSIX_FADV_DONTNEED)
>>> in the process that streams all the files out. Seems useful, as long as
>>> that
>>> d
On Thu, Jan 6, 2011 at 23:57, Heikki Linnakangas
wrote:
> On 05.01.2011 15:54, Magnus Hagander wrote:
>>
>> Attached is an updated streaming base backup patch, based off the work
>> that Heikki started.
>> ...
>> I've implemented a frontend for this in pg_streamrecv, based on the
>> assumption
>>
On 05.01.2011 15:54, Magnus Hagander wrote:
Attached is an updated streaming base backup patch, based off the work
that Heikki started.
...
I've implemented a frontend for this in pg_streamrecv, based on the assumption
that we wanted to include this in bin/ for 9.1 - and that it seems like a
reas
On Wed, Jan 5, 2011 at 23:27, Dimitri Fontaine wrote:
> Magnus Hagander writes:
>>> Well, I would guess that if you're streaming the WAL files in parallel
>>> while the base backup is taken, then you're able to have it all without
>>> archiving setup, and the server could still recycling them.
>
On Wed, Jan 5, 2011 at 23:58, Dimitri Fontaine wrote:
>> * Stefan mentiond it might be useful to put some
>> posix_fadvise(POSIX_FADV_DONTNEED)
>> in the process that streams all the files out. Seems useful, as long as
>> that
>> doesn't kick them out of the cache *completely*, for other back
On 06.01.2011 00:27, Dimitri Fontaine wrote:
Magnus Hagander writes:
What about pg_streamrecv | gzip> …, which has the big advantage of
That's part of what I meant with "easier and more useful".
Well…
One thing to keep in mind is that if you do compression in libpq for the
transfer, and
Magnus Hagander writes:
>> Compression in libpq would be a nice way to solve it, later.
>
> Yeah, I'm pretty much set on postponing that one.
+1, in case it was not clear for whoever's counting the votes :)
>> What about pg_streamrecv | gzip > …, which has the big advantage of
>
> That's part of
On Wed, Jan 5, 2011 at 22:58, Dimitri Fontaine wrote:
> Magnus Hagander writes:
>> Attached is an updated streaming base backup patch, based off the work
>
> Thanks! :)
>
>> * Compression: Do we want to be able to compress the backups server-side? Or
>> defer that to whenever we get compression
Magnus Hagander writes:
> Attached is an updated streaming base backup patch, based off the work
Thanks! :)
> * Compression: Do we want to be able to compress the backups server-side? Or
> defer that to whenever we get compression in libpq? (you can still tunnel it
> through for example SSH
On 01/05/2011 02:54 PM, Magnus Hagander wrote:
[..]
Some remaining thoughts and must-dos:
* Compression: Do we want to be able to compress the backups server-side? Or
defer that to whenever we get compression in libpq? (you can still tunnel it
through for example SSH to get compression if
Attached is an updated streaming base backup patch, based off the work
that Heikki
started. It includes support for tablespaces, permissions, progress
reporting and
some actual documentation of the protocol changes (user interface
documentation is
going to be depending on exactly what the frontend
58 matches
Mail list logo
