Re: propagating replica identity to partitions

2019-09-11 Thread Alvaro Herrera from 2ndQuadrant
I've not had time to work on this issue, and there are a few remaining problems in the patch; mostly that I will need to change the way pg_dump represents the replica identity so that it is only defined when the index is dumped in all partitions rather than immediately after creating the index. Al

Re: propagating replica identity to partitions

2019-04-04 Thread Alvaro Herrera
On 2019-Mar-29, Peter Eisentraut wrote: > On 2019-03-28 18:16, Simon Riggs wrote: > > SET TABLESPACE should not recurse because it copies the data, while > > holding long locks. If that was ever fixed so it happened concurrently, > > I would agree this could recurse by default. > > Since a partit

Re: propagating replica identity to partitions

2019-03-29 Thread Simon Riggs
On Fri, 29 Mar 2019 at 09:51, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 2019-03-28 18:16, Simon Riggs wrote: > > SET TABLESPACE should not recurse because it copies the data, while > > holding long locks. If that was ever fixed so it happened concurrently, > > I would agree

Re: propagating replica identity to partitions

2019-03-29 Thread Peter Eisentraut
On 2019-03-28 18:16, Simon Riggs wrote: > SET TABLESPACE should not recurse because it copies the data, while > holding long locks. If that was ever fixed so it happened concurrently, > I would agree this could recurse by default. Since a partitioned table has no storage, what is the meaning of mo

Re: propagating replica identity to partitions

2019-03-29 Thread Peter Eisentraut
On 2019-03-28 17:46, Alvaro Herrera wrote: > Thanks, Michael and Peter, for responding; however there is a second > part to the question, which is "should I change the recursivity of > REPLICA IDENTITY, while not simultaneously changing the recusivity of > the TABLESPACE and OWNER TO forms of ALTER

Re: propagating replica identity to partitions

2019-03-28 Thread Simon Riggs
On Thu, 28 Mar 2019 at 16:46, Alvaro Herrera wrote: > Thanks, Michael and Peter, for responding; however there is a second > part to the question, which is "should I change the recursivity of > REPLICA IDENTITY, while not simultaneously changing the recusivity of > the TABLESPACE and OWNER TO for

Re: propagating replica identity to partitions

2019-03-28 Thread Simon Riggs
On Sat, 23 Mar 2019 at 01:34, Michael Paquier wrote: > I'm slightly baffled that we would even allow having different owners on > > different partitions, but that seems to be a separate discussion. > > Different owners can make sense for multiple layers of partitions > where the children have les

Re: propagating replica identity to partitions

2019-03-28 Thread Alvaro Herrera
Thanks, Michael and Peter, for responding; however there is a second part to the question, which is "should I change the recursivity of REPLICA IDENTITY, while not simultaneously changing the recusivity of the TABLESPACE and OWNER TO forms of ALTER TABLE?" I think everyone agrees that REPLICA IDEN

Re: propagating replica identity to partitions

2019-03-22 Thread Michael Paquier
On Fri, Mar 22, 2019 at 07:55:11PM +0100, Peter Eisentraut wrote: > If you are operating on a partitioned table and set the replica identity > to the primary key or a partitioned index of that partitioned table, > then I think, by definition of what it means to be a partitioned index, > that applie

Re: propagating replica identity to partitions

2019-03-22 Thread Peter Eisentraut
On 2019-03-22 17:52, Alvaro Herrera wrote: > To recap: my proposed change is to make > ALTER TABLE ... REPLICA IDENTITY > when applied on a partitioned table affect all of its partitions instead > of expecting the user to invoke the command for each partition. If you are operating on a partiti

Re: propagating replica identity to partitions

2019-03-22 Thread Alvaro Herrera
On 2019-Mar-22, Robert Haas wrote: > On Thu, Mar 21, 2019 at 5:01 PM Alvaro Herrera > wrote: > > I already argued that TABLESPACE and OWNER TO are documented to work > > that way, and have been for a long time, whereas REPLICA IDENTITY has > > never been. If you want to change long-standing beh

Re: propagating replica identity to partitions

2019-03-22 Thread Robert Haas
On Thu, Mar 21, 2019 at 5:01 PM Alvaro Herrera wrote: > I already argued that TABLESPACE and OWNER TO are documented to work > that way, and have been for a long time, whereas REPLICA IDENTITY has > never been. If you want to change long-standing behavior, be my guest, > but that's not my patch.

Re: propagating replica identity to partitions

2019-03-21 Thread Alvaro Herrera
On 2019-Mar-21, Robert Haas wrote: > On Wed, Mar 20, 2019 at 4:42 PM Alvaro Herrera > wrote: > > Unless there are any objections to fixing the REPLICA IDENTITY bug, I > > intend to push that tomorrow. > > I still think that this is an ill-considered, piecemeal approach to a > problem that deser

Re: propagating replica identity to partitions

2019-03-21 Thread Robert Haas
On Wed, Mar 20, 2019 at 4:42 PM Alvaro Herrera wrote: > Unless there are any objections to fixing the REPLICA IDENTITY bug, I > intend to push that tomorrow. I still think that this is an ill-considered, piecemeal approach to a problem that deserves a better solution. -- Robert Haas EnterpriseD

Re: propagating replica identity to partitions

2019-03-20 Thread Alvaro Herrera
Unless there are any objections to fixing the REPLICA IDENTITY bug, I intend to push that tomorrow. People can continue to discuss changing the behavior of other subcommands where reasonable (OWNER TO) or even for the cases others consider not reasonable (TABLESPACE), but there is no consensus of

Re: propagating replica identity to partitions

2019-03-14 Thread Peter Eisentraut
On 2019-02-28 23:41, Alvaro Herrera wrote: > Maybe ADD GENERATED AS IDENTITY / DROP IDENTITY should recurse; not > really sure about this one. Peter? No, the intention is that only the partition root has the identity property. If you wanted to make it recurse, then you'd need to arrange it so th

Re: propagating replica identity to partitions

2019-03-01 Thread Robert Haas
On Thu, Feb 28, 2019 at 6:13 PM Alvaro Herrera wrote: > Tablespaces already behave a little bit especially in another sense: > it doesn't recurse to indexes. I think it's not possible to implement a > three-way flag, where you tell it to move only the table, or to recurse > to child tables but le

Re: propagating replica identity to partitions

2019-02-28 Thread Michael Paquier
On Thu, Feb 28, 2019 at 07:41:11PM -0300, Alvaro Herrera wrote: > We all seem to agree that REPLICA IDENTITY should recurse. (entering the ring) FWIW, I agree that having REPLICA IDENTITY recurse on partitions feels more natural, as much as being able to use ALTER TABLE ONLY to only update the def

Re: propagating replica identity to partitions

2019-02-28 Thread Alvaro Herrera
Added Peter E to CC; question at the very end. On 2019-Feb-20, Robert Haas wrote: > Yeah, we could. I wonder, though, whether we should just make > everything recurse. I think that's what people are commonly going to > want, at least for partitioned tables, and it doesn't seem to me that > it w

Re: propagating replica identity to partitions

2019-02-28 Thread Alvaro Herrera
On 2019-Feb-20, Robert Haas wrote: > On Wed, Feb 20, 2019 at 12:38 PM Alvaro Herrera > wrote: > > Maybe the ALL IN TABLESPACE and OWNED BY sub-forms should be split to a > > separate para. I suggest: > > > > : This form changes the table's tablespace to the specified tablespace > > : and mo

Re: propagating replica identity to partitions

2019-02-20 Thread Simon Riggs
On Wed, 20 Feb 2019 at 18:51, Robert Haas wrote: I don't buy Simon's argument that we should treat TABLESPACE > differently because the tables might be really big and take a long > time to move. I agree that this could well be true, but nobody is > proposing to remove the ability to move tables

Re: propagating replica identity to partitions

2019-02-20 Thread Robert Haas
On Wed, Feb 20, 2019 at 12:38 PM Alvaro Herrera wrote: > > I don't see that in the NOTES section for ALTER TABLE. Are you > > looking at some patch, or at master? > > I was looking here: > https://www.postgresql.org/docs/11/sql-altertable.html OK, I was looking at the wrong thing. Not enough ca

Re: propagating replica identity to partitions

2019-02-20 Thread Simon Riggs
On Wed, 20 Feb 2019 at 17:38, Alvaro Herrera wrote: > > Fair enough. I don't think it's entirely useful to think about this > > in terms of which operations do and don't recurse; the question in my > > mind is WHY some things recurse and other things don't, and what will > > be easiest for user

Re: propagating replica identity to partitions

2019-02-20 Thread Alvaro Herrera
On 2019-Feb-20, Robert Haas wrote: > On Tue, Feb 19, 2019 at 5:41 PM Alvaro Herrera > wrote: > > OK, let me concede that point -- it's not rewriting that makes things > > act differently, but rather TABLESPACE (as well as some other things) > > behave that way. ALTER TABLE ... SET DATA TYPE is

Re: propagating replica identity to partitions

2019-02-20 Thread Robert Haas
On Tue, Feb 19, 2019 at 5:41 PM Alvaro Herrera wrote: > OK, let me concede that point -- it's not rewriting that makes things > act differently, but rather TABLESPACE (as well as some other things) > behave that way. ALTER TABLE ... SET DATA TYPE is the obvious > counterexample. > > The Notes sec

Re: propagating replica identity to partitions

2019-02-19 Thread Alvaro Herrera
On 2019-Feb-19, Robert Haas wrote: > On Tue, Feb 19, 2019 at 3:40 PM Alvaro Herrera > wrote: > > Maybe we should be using the inheritance marker in the command to note > > whether to recurse to partitions? That is, if you say > > ALTER TABLE ONLY parent SET REPLICA IDENTITY > > then we don't

Re: propagating replica identity to partitions

2019-02-19 Thread Robert Haas
On Tue, Feb 19, 2019 at 3:40 PM Alvaro Herrera wrote: > Maybe we should be using the inheritance marker in the command to note > whether to recurse to partitions? That is, if you say > ALTER TABLE ONLY parent SET REPLICA IDENTITY > then we don't recurse and just change the parent table and futu

Re: propagating replica identity to partitions

2019-02-19 Thread Alvaro Herrera
On 2019-Feb-19, Robert Haas wrote: > It's not unreasonable to use the parent's REPLICA IDENTITY setting as > the default for new partitions, much as we now do for the TABLESPACE, > because the parent's replica identity is otherwise without meaning. > But I'm less convinced that it's reasonable to

Re: propagating replica identity to partitions

2019-02-19 Thread Robert Haas
On Mon, Feb 4, 2019 at 11:30 AM Alvaro Herrera wrote: > If you do ALTER TABLE .. REPLICA IDENTITY to a partitioned table, the > command operates on the parent table itself and does not propagate to > partitions. Why is this? Maybe not recursing was the right call when > we only had regular inher

Re: propagating replica identity to partitions

2019-02-18 Thread Dmitry Dolgov
> On Fri, Feb 15, 2019 at 10:02 PM Alvaro Herrera > wrote: > > On 2019-Feb-07, Dmitry Dolgov wrote: > > > Could there be a race condition somewhere? The idea and the code looks > > reasonable, but when I'm testing ALTER TABLE ... REPLICA IDENTITY with this > > patch and concurrent partition creat

Re: propagating replica identity to partitions

2019-02-15 Thread Alvaro Herrera
On 2019-Feb-07, Dmitry Dolgov wrote: > Could there be a race condition somewhere? The idea and the code looks > reasonable, but when I'm testing ALTER TABLE ... REPLICA IDENTITY with this > patch and concurrent partition creation, I've got the following error on > ATTACH > PARTITION: > > ERROR:

Re: propagating replica identity to partitions

2019-02-09 Thread Alvaro Herrera
On 2019-Feb-07, Dmitry Dolgov wrote: > Could there be a race condition somewhere? The idea and the code looks > reasonable, but when I'm testing ALTER TABLE ... REPLICA IDENTITY with this > patch and concurrent partition creation, I've got the following error on > ATTACH > PARTITION: > > ERROR:

Re: propagating replica identity to partitions

2019-02-07 Thread Dmitry Dolgov
> On Tue, Feb 5, 2019 at 12:54 AM Alvaro Herrera > wrote: > > Actually, index-based replica identities failed in pg_dump: we first > create the index ONLY on the partitioned table (marking it as invalid), > so when we immediately do the ALTER TABLE/REPLICA IDENTITY, it rejects > the invalid index

Re: propagating replica identity to partitions

2019-02-04 Thread Alvaro Herrera
Actually, index-based replica identities failed in pg_dump: we first create the index ONLY on the partitioned table (marking it as invalid), so when we immediately do the ALTER TABLE/REPLICA IDENTITY, it rejects the invalid index. If we change the code to allow invalid indexes on partitioned table

Re: propagating replica identity to partitions

2019-02-04 Thread Alvaro Herrera
On 2019-Feb-04, Alvaro Herrera wrote: > I'll now look more carefully at the cases involving indexes; thus far I > was looking at the ones using FULL. Those seem to work as intended. Yeah, that didn't work so well -- it was trying to propagate the command verbatim to each partition, and obviously

Re: propagating replica identity to partitions

2019-02-04 Thread Alvaro Herrera
> I propose to change the behavior to: ... this patch. I'll now look more carefully at the cases involving indexes; thus far I was looking at the ones using FULL. Those seem to work as intended. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support,

propagating replica identity to partitions

2019-02-04 Thread Alvaro Herrera
Hello If you do ALTER TABLE .. REPLICA IDENTITY to a partitioned table, the command operates on the parent table itself and does not propagate to partitions. Why is this? Maybe not recursing was the right call when we only had regular inheritance (back in 9.4), but since partitioned tables got i