Re: CLUSTER on partitioned index

2022-04-25 Thread Alvaro Herrera
On 2022-Apr-26, Michael Paquier wrote: > On Sat, Apr 16, 2022 at 08:58:50PM +0900, Michael Paquier wrote: > > Well, I am a bit annoyed that we don't actually check that a CLUSTER > > command does not block when doing a CLUSTER on a partitioned table > > while a lock is held on one of its partition

Re: CLUSTER on partitioned index

2022-04-25 Thread Michael Paquier
On Sat, Apr 16, 2022 at 08:58:50PM +0900, Michael Paquier wrote: > Well, I am a bit annoyed that we don't actually check that a CLUSTER > command does not block when doing a CLUSTER on a partitioned table > while a lock is held on one of its partitions. So, attached is a > proposal of patch to imp

Re: CLUSTER on partitioned index

2022-04-16 Thread Michael Paquier
On Thu, Apr 14, 2022 at 10:37:06PM +0200, Alvaro Herrera wrote: > Thanks for the patch -- I have pushed it now, with some wording changes > and renaming the role to regress_* to avoid buildfarm's ire. Cool, thanks. > Michaël in addition proposes an isolation test. I'm not sure; is it > worth the

Re: CLUSTER on partitioned index

2022-04-15 Thread Alvaro Herrera
Thanks for the patch -- I have pushed it now, with some wording changes and renaming the role to regress_* to avoid buildfarm's ire. Michaël in addition proposes an isolation test. I'm not sure; is it worth the additional test run time? It doesn't seem a critical issue. But if anybody feels like

Re: CLUSTER on partitioned index

2022-04-13 Thread Michael Paquier
On Wed, Apr 13, 2022 at 05:52:14AM -0500, Justin Pryzby wrote: > Are you sure? The ownership re-check in cluster_rel() occurs after acquiring > locks. Yep, you are right. However, the SQL test does not check for this blocking scenario. In short, removing the new ACL check in get_tables_to_clust

Re: CLUSTER on partitioned index

2022-04-13 Thread Justin Pryzby
On Wed, Apr 13, 2022 at 03:50:15PM +0900, Michael Paquier wrote: > > > That dates to a556549d7 (see also cbe24a6dd8 for an earlier commit in > > CLUSTER > > itself). The reason was to avoid blocking if an unprivileged user runs > > VACUUM > > FULL which would try to lock things (including share

Re: CLUSTER on partitioned index

2022-04-12 Thread Michael Paquier
On Mon, Apr 11, 2022 at 09:06:09AM -0500, Justin Pryzby wrote: > On Sat, Apr 02, 2022 at 07:21:11PM +0200, Alvaro Herrera wrote: >> 1. in VACUUM FULL we only process partitions that are owned by the >> invoking user. We don't have this test in the new code. I'm not sure >> why do we do that there

Re: CLUSTER on partitioned index

2022-04-11 Thread Zhihong Yu
On Mon, Apr 11, 2022 at 7:06 AM Justin Pryzby wrote: > On Sat, Apr 02, 2022 at 07:21:11PM +0200, Alvaro Herrera wrote: > > Small things here. > > > 1. in VACUUM FULL we only process partitions that are owned by the > > invoking user. We don't have this test in the new code. I'm not sure > > why

Re: CLUSTER on partitioned index

2022-04-11 Thread Justin Pryzby
On Sat, Apr 02, 2022 at 07:21:11PM +0200, Alvaro Herrera wrote: > Small things here. > 1. in VACUUM FULL we only process partitions that are owned by the > invoking user. We don't have this test in the new code. I'm not sure > why do we do that there; is it worth doing the same here? That dates

Re: CLUSTER on partitioned index

2022-04-04 Thread Justin Pryzby
On Sat, Apr 02, 2022 at 07:11:47PM +0200, Alvaro Herrera wrote: > Thanks, pushed. Thank you for revisiting it, and thanks to Zhihong Yu for earlier review. I'll look into your outstanding questions later this week. -- Justin

Re: CLUSTER on partitioned index

2022-04-02 Thread Alvaro Herrera
Small things here. 1. in VACUUM FULL we only process partitions that are owned by the invoking user. We don't have this test in the new code. I'm not sure why do we do that there; is it worth doing the same here? 2. We should silently skip a partition that's a foreign table, I suppose. 3. We d

Re: CLUSTER on partitioned index

2022-04-02 Thread Alvaro Herrera
Thanks, pushed. -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/ "La fuerza no está en los medios físicos sino que reside en una voluntad indomable" (Gandhi)

Re: CLUSTER on partitioned index

2022-03-31 Thread Justin Pryzby
On Thu, Mar 31, 2022 at 12:54:36PM +0200, Alvaro Herrera wrote: > I realized after posting that we used to allow clustering toast tables, > but after my changes we no longer do. (Justin's version had a > RELKIND_HAS_STORAGE test here instead, which seemed a little too lax.) I > don't know why we a

Re: CLUSTER on partitioned index

2022-03-31 Thread Justin Pryzby
On Wed, Mar 30, 2022 at 10:51:43PM +0200, Alvaro Herrera wrote: > On 2022-Feb-23, Justin Pryzby wrote: > > > I hope that Alvaro will comment on the simplified patches. If multiple > > people > > think the patch isn't worth it, feel free to close it. But I don't see how > > complexity could be t

Re: CLUSTER on partitioned index

2022-03-31 Thread Robert Haas
On Thu, Mar 31, 2022 at 6:54 AM Alvaro Herrera wrote: > I realized after posting that we used to allow clustering toast tables, > but after my changes we no longer do. (Justin's version had a > RELKIND_HAS_STORAGE test here instead, which seemed a little too lax.) I > don't know why we allowed it

Re: CLUSTER on partitioned index

2022-03-31 Thread Alvaro Herrera
I realized after posting that we used to allow clustering toast tables, but after my changes we no longer do. (Justin's version had a RELKIND_HAS_STORAGE test here instead, which seemed a little too lax.) I don't know why we allowed it and I don't know of anyone who has ever used that feature and

Re: CLUSTER on partitioned index

2022-03-30 Thread Alvaro Herrera
On 2022-Feb-23, Justin Pryzby wrote: > I hope that Alvaro will comment on the simplified patches. If multiple people > think the patch isn't worth it, feel free to close it. But I don't see how > complexity could be the reason. I gave your patch a look and it seems a reasonable thing to do. Ma

Re: CLUSTER on partitioned index

2022-02-23 Thread Justin Pryzby
On Tue, Jul 20, 2021 at 08:27:02PM -0400, Alvaro Herrera wrote: > I have to wonder if there really *is* a use case for CLUSTER in the > first place on regular tables, let alone on partitioned tables, which > are likely to be large and thus take a lot of time. What justifies > spending so much time

Re: CLUSTER on partitioned index

2022-01-08 Thread Justin Pryzby
On Fri, Dec 03, 2021 at 10:16:24AM +0900, Michael Paquier wrote: > On Thu, Sep 23, 2021 at 06:56:26PM -0500, Justin Pryzby wrote: > > On Thu, Sep 23, 2021 at 08:18:41PM +0200, Matthias van de Meent wrote: > >> Note: The following review is based on the assumption that this v11 > >> revision was mea

Re: CLUSTER on partitioned index

2021-12-02 Thread Michael Paquier
On Thu, Sep 23, 2021 at 06:56:26PM -0500, Justin Pryzby wrote: > On Thu, Sep 23, 2021 at 08:18:41PM +0200, Matthias van de Meent wrote: >> Note: The following review is based on the assumption that this v11 >> revision was meant to contain only one patch. I put this up as a note, >> because it seem

Re: CLUSTER on partitioned index

2021-09-23 Thread Justin Pryzby
On Thu, Sep 23, 2021 at 08:18:41PM +0200, Matthias van de Meent wrote: > On Sun, 12 Sept 2021 at 22:10, Justin Pryzby wrote: > > > > On Tue, Jul 20, 2021 at 08:27:02PM -0400, Alvaro Herrera wrote: > > > I have to wonder if there really *is* a use case for CLUSTER in the > > > first place on regula

Re: CLUSTER on partitioned index

2021-09-23 Thread Matthias van de Meent
On Sun, 12 Sept 2021 at 22:10, Justin Pryzby wrote: > > On Tue, Jul 20, 2021 at 08:27:02PM -0400, Alvaro Herrera wrote: > > I have to wonder if there really *is* a use case for CLUSTER in the > > first place on regular tables, let alone on partitioned tables, which > > are likely to be large and t

Re: CLUSTER on partitioned index

2021-09-12 Thread Justin Pryzby
be required, which is a step forward from not supporting cluster on partitioned index at all. As attached. It's arguably true that the follow-up patches supporting indisclustered on partitioned indexes aren't worth the trouble. For sure CLUSTER is useful, see eg. https://github.com/bucardo/c

Re: CLUSTER on partitioned index

2021-07-21 Thread Laurenz Albe
On Tue, 2021-07-20 at 20:27 -0400, Alvaro Herrera wrote: > I have to wonder if there really *is* a use case for CLUSTER in the > first place on regular tables, let alone on partitioned tables, which > are likely to be large and thus take a lot of time. What justifies > spending so much time on thi

Re: CLUSTER on partitioned index

2021-07-20 Thread Michael Paquier
On Tue, Jul 20, 2021 at 08:27:02PM -0400, Alvaro Herrera wrote: > I have to wonder if there really *is* a use case for CLUSTER in the > first place on regular tables, let alone on partitioned tables, which > are likely to be large and thus take a lot of time. What justifies > spending so much time

Re: CLUSTER on partitioned index

2021-07-20 Thread Alvaro Herrera
I have to wonder if there really *is* a use case for CLUSTER in the first place on regular tables, let alone on partitioned tables, which are likely to be large and thus take a lot of time. What justifies spending so much time on this implementation? My impression is that CLUSTER is pretty much a

Re: CLUSTER on partitioned index

2021-04-03 Thread Zhihong Yu
Hi, For v10-0002-Implement-CLUSTER-of-partitioned-table.patch : or that an partitioned index was previously set clustered. 'an partitioned index' -> a partitioned index + * Return a List of tables and associated index, where each index is a associated index -> associated indices For cluster():

Re: CLUSTER on partitioned index

2021-04-02 Thread Justin Pryzby
@cfbot: rebased >From 686cd8fc644f1f86d81d7748b66feddd634c4dc8 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Thu, 26 Nov 2020 14:37:08 -0600 Subject: [PATCH v10 1/8] pg_dump: make CLUSTER ON a separate dump object.. ..since it needs to be restored after any child indexes are restored *and at

Re: CLUSTER on partitioned index

2021-03-06 Thread Justin Pryzby
On Wed, Feb 10, 2021 at 02:04:58PM -0600, Justin Pryzby wrote: > On Sat, Feb 06, 2021 at 08:45:49AM -0600, Justin Pryzby wrote: > > On Mon, Jan 18, 2021 at 12:34:59PM -0600, Justin Pryzby wrote: > > > On Sat, Nov 28, 2020 at 08:03:02PM -0600, Justin Pryzby wrote: > > > > On Sun, Nov 15, 2020 at 07:

Re: CLUSTER on partitioned index

2021-02-10 Thread Justin Pryzby
On Sat, Feb 06, 2021 at 08:45:49AM -0600, Justin Pryzby wrote: > On Mon, Jan 18, 2021 at 12:34:59PM -0600, Justin Pryzby wrote: > > On Sat, Nov 28, 2020 at 08:03:02PM -0600, Justin Pryzby wrote: > > > On Sun, Nov 15, 2020 at 07:53:35PM -0600, Justin Pryzby wrote: > > > > On Wed, Nov 04, 2020 at 08:

Re: CLUSTER on partitioned index

2021-02-06 Thread Zhihong Yu
Hi, For v7-0002-Implement-CLUSTER-of-partitioned-table.patch: +* We have to build the list in a different memory context so it will +* survive the cross-transaction processing +*/ + old_context = MemoryContextSwitchTo(cluster_context); cluster_context is not modified

Re: CLUSTER on partitioned index

2021-02-06 Thread Justin Pryzby
On Mon, Jan 18, 2021 at 12:34:59PM -0600, Justin Pryzby wrote: > On Sat, Nov 28, 2020 at 08:03:02PM -0600, Justin Pryzby wrote: > > On Sun, Nov 15, 2020 at 07:53:35PM -0600, Justin Pryzby wrote: > > > On Wed, Nov 04, 2020 at 08:23:56PM -0600, Justin Pryzby wrote: > > > > On Tue, Oct 27, 2020 at 07:

Re: CLUSTER on partitioned index

2021-01-18 Thread Justin Pryzby
On Sat, Nov 28, 2020 at 08:03:02PM -0600, Justin Pryzby wrote: > On Sun, Nov 15, 2020 at 07:53:35PM -0600, Justin Pryzby wrote: > > On Wed, Nov 04, 2020 at 08:23:56PM -0600, Justin Pryzby wrote: > > > On Tue, Oct 27, 2020 at 07:33:12PM -0500, Justin Pryzby wrote: > > > > I'm attaching a counter-pro

Re: CLUSTER on partitioned index

2020-11-28 Thread Justin Pryzby
On Sun, Nov 15, 2020 at 07:53:35PM -0600, Justin Pryzby wrote: > On Wed, Nov 04, 2020 at 08:23:56PM -0600, Justin Pryzby wrote: > > On Tue, Oct 27, 2020 at 07:33:12PM -0500, Justin Pryzby wrote: > > > I'm attaching a counter-proposal to your catalog change, which preserves > > > indisclustered on c

Re: CLUSTER on partitioned index

2020-11-15 Thread Justin Pryzby
On Wed, Nov 04, 2020 at 08:23:56PM -0600, Justin Pryzby wrote: > On Tue, Oct 27, 2020 at 07:33:12PM -0500, Justin Pryzby wrote: > > I'm attaching a counter-proposal to your catalog change, which preserves > > indisclustered on children of clustered, partitioned indexes, and > > invalidates > > ind

Re: CLUSTER on partitioned index

2020-11-04 Thread Justin Pryzby
@cfbot: rebased On Tue, Oct 27, 2020 at 07:33:12PM -0500, Justin Pryzby wrote: > I'm attaching a counter-proposal to your catalog change, which preserves > indisclustered on children of clustered, partitioned indexes, and invalidates > indisclustered when attaching unclustered indexes. ..and now

CLUSTER on partitioned index

2020-10-27 Thread Justin Pryzby
Forking this thread, since the existing CFs have been closed. https://www.postgresql.org/message-id/flat/20200914143102.GX18552%40telsasoft.com#58b1056488451f8594b0f0ba40996afd On Tue, Oct 06, 2020 at 01:38:23PM +0900, Michael Paquier wrote: > On Mon, Oct 05, 2020 at 10:07:33PM -0500, Justin Pryzb