On 29/04/15 22:12, David Rowley wrote:
On 25 April 2015 at 00:32, Andres Freund mailto:and...@anarazel.de>> wrote:
Attached is a patch that does this, and some more, renaming. That was
more work than I'd imagined. I've also made the internal naming in
origin.c more consistent/simpl
On 25 April 2015 at 00:32, Andres Freund wrote:
>
> Attached is a patch that does this, and some more, renaming. That was
> more work than I'd imagined. I've also made the internal naming in
> origin.c more consistent/simpler and did a bunch of other cleanup.
>
>
Hi,
It looks like bowerbird is
On Tue, Apr 28, 2015 at 10:00 PM, Peter Eisentraut wrote:
> On 4/24/15 4:29 PM, Andres Freund wrote:
>>> Shouldn't this be backed up by pg_dump(all?)?
>>
>> Given it deals with LSNs and is, quite fundamentally due to concurrency, non
>> transactional, I doubt it's worth it. The other side's slots
On 4/24/15 4:29 PM, Andres Freund wrote:
>> Shouldn't this be backed up by pg_dump(all?)?
>
> Given it deals with LSNs and is, quite fundamentally due to concurrency, non
> transactional, I doubt it's worth it. The other side's slots also aren't
> going to be backed up as pg dump obviously can't
On April 24, 2015 10:26:23 PM GMT+02:00, Peter Eisentraut
wrote:
>On 4/24/15 8:32 AM, Andres Freund wrote:
>> On 2015-04-20 11:26:29 +0300, Heikki Linnakangas wrote:
>>> On 04/17/2015 11:54 AM, Andres Freund wrote:
I've attached a rebased patch, that adds decision about origin
>logging
On 4/24/15 8:32 AM, Andres Freund wrote:
> On 2015-04-20 11:26:29 +0300, Heikki Linnakangas wrote:
>> On 04/17/2015 11:54 AM, Andres Freund wrote:
>>> I've attached a rebased patch, that adds decision about origin logging
>>> to the relevant XLogInsert() callsites for "external" 2 byte identifiers
On 24/04/15 14:32, Andres Freund wrote:
On 2015-04-20 11:26:29 +0300, Heikki Linnakangas wrote:
On 04/17/2015 11:54 AM, Andres Freund wrote:
I've attached a rebased patch, that adds decision about origin logging
to the relevant XLogInsert() callsites for "external" 2 byte identifiers
and remove
On 2015-04-20 11:26:29 +0300, Heikki Linnakangas wrote:
> On 04/17/2015 11:54 AM, Andres Freund wrote:
> >I've attached a rebased patch, that adds decision about origin logging
> >to the relevant XLogInsert() callsites for "external" 2 byte identifiers
> >and removes the pad-reusing version in the
On 2015-03-24 22:22:29 +0100, Petr Jelinek wrote:
> Perhaps we should have some Logical replication developer documentation
> section and put all those three as subsections of that?
So I just played around with this and it didn't find it
worthwhile. Primarily because there's lots of uses of logica
On 21/04/15 22:36, Andres Freund wrote:
On 2015-04-21 16:26:08 -0400, Robert Haas wrote:
On Tue, Apr 21, 2015 at 8:08 AM, Andres Freund wrote:
I've now named the functions:
* pg_replication_origin_create
* pg_replication_origin_drop
* pg_replication_origin_get (map from name to id)
* pg_repli
On 2015-04-21 16:26:08 -0400, Robert Haas wrote:
> On Tue, Apr 21, 2015 at 8:08 AM, Andres Freund wrote:
> > I've now named the functions:
> >
> > * pg_replication_origin_create
> > * pg_replication_origin_drop
> > * pg_replication_origin_get (map from name to id)
> > * pg_replication_progress_set
On Tue, Apr 21, 2015 at 8:08 AM, Andres Freund wrote:
> I've now named the functions:
>
> * pg_replication_origin_create
> * pg_replication_origin_drop
> * pg_replication_origin_get (map from name to id)
> * pg_replication_progress_setup_origin : configure session to replicate
> from a specific
On 2015-04-21 12:20:42 -0300, Alvaro Herrera wrote:
> Andres Freund wrote:
> > Catalog wise there's an actual table 'pg_replication_origin' that maps
> > between 'roident' and 'roname'. There's a pg_replication_progress view
> > (used to be named pg_replication_identifier_progress). I'm not sure if
Andres Freund wrote:
> I'm working on changing this (I've implemented the missing WAL
> bits). I'd like to discuss the new terms for a sec, before I go and
> revise the docs.
>
> I'm now calling the feature 'replication progress tracking'. There's
> "replication origins" and there's progress trac
On 2015-04-20 10:28:02 +0200, Andres Freund wrote:
> On 2015-04-20 11:26:29 +0300, Heikki Linnakangas wrote:
> > I just realized that it talks about "replication identifier" as the new
> > fundamental concept. The system table is called "pg_replication_identifier".
> > But that's like talking about
On 20 April 2015 at 09:28, Andres Freund wrote:
> On 2015-04-20 11:26:29 +0300, Heikki Linnakangas wrote:
> > I just realized that it talks about "replication identifier" as the new
> > fundamental concept. The system table is called
> "pg_replication_identifier".
> > But that's like talking abou
On 2015-04-20 11:26:29 +0300, Heikki Linnakangas wrote:
> I just realized that it talks about "replication identifier" as the new
> fundamental concept. The system table is called "pg_replication_identifier".
> But that's like talking about "index identifiers", instead of just indexes,
> and callin
On 04/17/2015 11:54 AM, Andres Freund wrote:
I've attached a rebased patch, that adds decision about origin logging
to the relevant XLogInsert() callsites for "external" 2 byte identifiers
and removes the pad-reusing version in the interest of moving forward.
Putting aside the 2 vs. 4 byte iden
On 2015-04-20 11:09:20 +0300, Heikki Linnakangas wrote:
> Can you name some of the bigger problems you'd have?
Several parts of the system are O(#max_replication_slots). Having 100k
outgoing logical replication slots is going to be expensive.
Nothing unsolvable, but the 65k 16 bit limit surely is
On 04/17/2015 11:45 PM, Petr Jelinek wrote:
>The argument to move to 4 bytes is a poor one. If it was reasonable in
>terms of code or cosmetic value then all values used in the backend
>would be 4 bytes. We wouldn't have any 2 byte values anywhere. But we
>don't do that.
>
>The change does nothin
On 04/17/2015 11:36 PM, Simon Riggs wrote:
On 17 April 2015 at 19:18, Heikki Linnakangas wrote:
To be honest, I'm not entirely sure what we're arguing over.
When arguing over something you consider small, it is customary to allow
the author precedence. We can't do things our own way all the t
On 17/04/15 22:36, Simon Riggs wrote:
I said that IMO the difference in WAL size is so small that we
should just use 4-byte OIDs for the replication identifiers, instead
of trying to make do with 2 bytes. Not because I find it too likely
that you'll run out of IDs (although it co
On 17 April 2015 at 19:18, Heikki Linnakangas wrote:
> To be honest, I'm not entirely sure what we're arguing over.
When arguing over something you consider small, it is customary to allow
the author precedence. We can't do things our own way all the time.
I didn't much like pg_rewind, but it
On 04/17/2015 08:36 PM, Simon Riggs wrote:
On 17 April 2015 at 18:12, Heikki Linnakangas wrote:
On 04/17/2015 12:04 PM, Simon Riggs wrote:
On 17 April 2015 at 09:54, Andres Freund wrote:
Hrmpf. Says the person that used a lot of padding, without much
discussion, for the WAL level infras
On 17 April 2015 at 18:12, Heikki Linnakangas wrote:
> On 04/17/2015 12:04 PM, Simon Riggs wrote:
>
>> On 17 April 2015 at 09:54, Andres Freund wrote:
>>
>> Hrmpf. Says the person that used a lot of padding, without much
>>> discussion, for the WAL level infrastructure making pg_rewind more
>>>
On 04/17/2015 12:04 PM, Simon Riggs wrote:
On 17 April 2015 at 09:54, Andres Freund wrote:
Hrmpf. Says the person that used a lot of padding, without much
discussion, for the WAL level infrastructure making pg_rewind more
maintainable.
Sounds bad. What padding are we talking about?
In the
On 17 April 2015 at 09:54, Andres Freund wrote:
> Hrmpf. Says the person that used a lot of padding, without much
> discussion, for the WAL level infrastructure making pg_rewind more
> maintainable.
Sounds bad. What padding are we talking about?
--
Simon Riggshttp://www.2ndQua
At 2015-04-17 10:54:51 +0200, and...@anarazel.de wrote:
>
> (The FPI percentage display above is arguably borked. Interesting.)
Sorry for the trouble. Patch attached.
-- Abhijit
>From 1e5c5d5948030e8ff6ccdd2309a97fb1e116d8e2 Mon Sep 17 00:00:00 2001
From: Abhijit Menon-Sen
Date: Fri, 17 Apr 2015
On 2015-04-12 22:02:38 +0300, Heikki Linnakangas wrote:
> This needs to be weighed against removing the padding bytes
> altogether.
Hrmpf. Says the person that used a lot of padding, without much
discussion, for the WAL level infrastructure making pg_rewind more
maintainable. And you deemed to be
On 04/12/2015 02:56 AM, Petr Jelinek wrote:
On 10/04/15 18:03, Andres Freund wrote:
On 2015-04-07 17:08:16 +0200, Andres Freund wrote:
I'm starting benchmarks now.
What I'm benchmarking here is the WAL overhead, since that's what we're
debating.
The test setup I used was a pgbench scale 10 i
On 10/04/15 18:03, Andres Freund wrote:
On 2015-04-07 17:08:16 +0200, Andres Freund wrote:
I'm starting benchmarks now.
What I'm benchmarking here is the WAL overhead, since that's what we're
debating.
The test setup I used was a pgbench scale 10 instance. I've run with
full_page_write=off to
On 08/04/15 14:22, Andres Freund wrote:
On 2015-04-08 14:17:04 +0200, Petr Jelinek wrote:
And you guys are not getting my point. What I proposed was to not reuse the
RI id immediately because that can make debugging issues with
replication/conflict handling harder when something happens after cl
On 2015-04-08 14:17:04 +0200, Petr Jelinek wrote:
> And you guys are not getting my point. What I proposed was to not reuse the
> RI id immediately because that can make debugging issues with
> replication/conflict handling harder when something happens after cluster
> configuration has changed.
I
On 08/04/15 06:59, Michael Paquier wrote:
On Tue, Apr 7, 2015 at 11:37 PM, Andres Freund wrote:
On 2015-04-07 16:30:25 +0200, Andres Freund wrote:
And with temp tables (or much more extremely WITH OID tables)
and such it's not that hard to reach that point.
Oh, and obviously toast data. A co
On Tue, Apr 7, 2015 at 11:37 PM, Andres Freund wrote:
> On 2015-04-07 16:30:25 +0200, Andres Freund wrote:
>> And with temp tables (or much more extremely WITH OID tables)
>> and such it's not that hard to reach that point.
>
> Oh, and obviously toast data. A couple tables with toasted columns is
On 4/7/15 10:58 AM, Andres Freund wrote:
Why not just create a sequence? I suspect it may not be as fast to assign as
an OID, but it's not like you'd be doing this all the time.
What does that have to do with the thread?
The original bit was...
And finally I have issue with how the new iden
> Why not just create a sequence? I suspect it may not be as fast to assign as
> an OID, but it's not like you'd be doing this all the time.
What does that have to do with the thread?
Greetings,
Andres Freund
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make change
On 4/7/15 9:30 AM, Andres Freund wrote:
On 2015-03-28 23:50:20 +0100, Petr Jelinek wrote:
And finally I have issue with how the new identifiers are allocated.
Currently, if you create identifier 'foo', remove identifier 'foo' and
create identifier 'bar', the identifier 'bar' will have same id as
On 2015-03-24 23:11:26 -0400, Robert Haas wrote:
> On Mon, Feb 16, 2015 at 4:46 AM, Andres Freund wrote:
> >> At a quick glance, this basic design seems workable. I would suggest
> >> expanding the replication IDs to regular 4 byte oids. Two extra bytes is a
> >> small price to pay, to make it wor
On 2015-04-07 16:30:25 +0200, Andres Freund wrote:
> And with temp tables (or much more extremely WITH OID tables)
> and such it's not that hard to reach that point.
Oh, and obviously toast data. A couple tables with toasted columns is
also a good way to rapidly consume oids.
Greetings,
Andres F
On 2015-03-28 23:50:20 +0100, Petr Jelinek wrote:
> And finally I have issue with how the new identifiers are allocated.
> Currently, if you create identifier 'foo', remove identifier 'foo' and
> create identifier 'bar', the identifier 'bar' will have same id as the old
> 'foo' identifier. This can
So I did some more in depth look, I do have couple of comments.
I would really like to have something like "Logical Replication
Infrastructure" doc section that would have both decoding and
identifiers (and possibly even CommitTs) underneath.
There is typo in docs:
+
+ The optiona
On Mon, Feb 16, 2015 at 4:46 AM, Andres Freund wrote:
>> At a quick glance, this basic design seems workable. I would suggest
>> expanding the replication IDs to regular 4 byte oids. Two extra bytes is a
>> small price to pay, to make it work more like everything else in the system.
>
> I don't kn
On 24/03/15 16:33, Andres Freund wrote:
Hi,
Here's the next version of this patch. I've tried to address the biggest
issue (documentation) and some more. Now that both the more flexible
commit WAL record format and the BKI_FORCE_NOT_NULL thing is in, it
looks much cleaner.
Nice, I see you als
Hi,
Here's the next version of this patch. I've tried to address the biggest
issue (documentation) and some more. Now that both the more flexible
commit WAL record format and the BKI_FORCE_NOT_NULL thing is in, it
looks much cleaner.
Changes:
* Loads of documentation and comments
* Revamped locki
On 2015-02-22 10:03:59 -0500, Peter Eisentraut wrote:
> On 2/15/15 7:24 PM, Andres Freund wrote:
> > On 2015-02-16 01:21:55 +0100, Andres Freund wrote:
> >> Here's my next attept attempt at producing something we can agree
> >> upon.
> >>
> >> The major change that might achieve that is that I've n
On 2/15/15 7:24 PM, Andres Freund wrote:
> On 2015-02-16 01:21:55 +0100, Andres Freund wrote:
>> Here's my next attept attempt at producing something we can agree
>> upon.
>>
>> The major change that might achieve that is that I've now provided a
>> separate method to store the origin_id of a node.
On 22/02/15 09:57, Andres Freund wrote:
On 2015-02-19 00:49:50 +0100, Petr Jelinek wrote:
On 16/02/15 10:46, Andres Freund wrote:
On 2015-02-16 11:34:10 +0200, Heikki Linnakangas wrote:
At a quick glance, this basic design seems workable. I would suggest
expanding the replication IDs to regul
On 2015-02-19 00:49:50 +0100, Petr Jelinek wrote:
> On 16/02/15 10:46, Andres Freund wrote:
> >On 2015-02-16 11:34:10 +0200, Heikki Linnakangas wrote:
> >
> >>At a quick glance, this basic design seems workable. I would suggest
> >>expanding the replication IDs to regular 4 byte oids. Two extra byt
On 2015-02-22 04:59:30 +0100, Petr Jelinek wrote:
> Now that the issue with padding seems to no longer exists since the patch
> works both with and without padding, I went through the code and here are
> some comments I have (in no particular order).
>
> In CheckPointReplicationIdentifier:
> >+ *
Now that the issue with padding seems to no longer exists since the
patch works both with and without padding, I went through the code and
here are some comments I have (in no particular order).
In CheckPointReplicationIdentifier:
+ * FIXME: Add a CRC32 to the end.
The function already does
On 16/02/15 10:46, Andres Freund wrote:
On 2015-02-16 11:34:10 +0200, Heikki Linnakangas wrote:
At a quick glance, this basic design seems workable. I would suggest
expanding the replication IDs to regular 4 byte oids. Two extra bytes is a
small price to pay, to make it work more like everythin
On 2015-02-16 11:34:10 +0200, Heikki Linnakangas wrote:
> Yes, thanks. Note to self and everyone else looking at this: It's important
> to keep in mind is that the replication IDs are completely internal to the
> local cluster. They are *not* sent over the wire.
Well, if you *want* to, you could s
On 02/16/2015 11:18 AM, Andres Freund wrote:
On 2015-02-16 11:07:09 +0200, Heikki Linnakangas wrote:
How does this work if you have multiple replication systems in use in the
same cluster? You might use Slony to replication one table to one system,
and BDR to replication another table with anoth
On 2015-02-16 11:07:09 +0200, Heikki Linnakangas wrote:
> On 02/16/2015 02:21 AM, Andres Freund wrote:
> >Furthermore the fact that the origin of records is recorded allows to
> >avoid decoding them in logical decoding. That has both efficiency
> >advantages (we can do so before they are stored in
On 02/16/2015 02:21 AM, Andres Freund wrote:
Furthermore the fact that the origin of records is recorded allows to
avoid decoding them in logical decoding. That has both efficiency
advantages (we can do so before they are stored in memory/disk) and
functionality advantages. Imagine using a logica
On 2015-02-16 01:21:55 +0100, Andres Freund wrote:
> Here's my next attept attempt at producing something we can agree
> upon.
>
> The major change that might achieve that is that I've now provided a
> separate method to store the origin_id of a node. I've made it
> conditional on !REPLICATION_IDE
Hi,
Here's my next attept attempt at producing something we can agree
upon.
The major change that might achieve that is that I've now provided a
separate method to store the origin_id of a node. I've made it
conditional on !REPLICATION_IDENTIFIER_REUSE_PADDING, to show both
paths. That new method
58 matches
Mail list logo
