Re: Partial aggregates pushdown

2021-11-01 Thread Ilya Gladyshev
anly on top of current master. Thanks, Ilya Gladyshev

Re: Partial aggregates pushdown

2021-11-01 Thread Ilya Gladyshev
On 01.11.2021 13:30, Alexander Pyhalov wrote: Peter Eisentraut писал 2021-11-01 12:47: On 21.10.21 12:55, Alexander Pyhalov wrote: Now aggregates with internal states can be pushed down, if they are marked as pushdown safe (this flag is set to true for min/max/sum), have internal states and a

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-03-14 Thread Ilya Gladyshev
> 14 марта 2023 г., в 18:34, Justin Pryzby написал(а): > > On Mon, Mar 13, 2023 at 10:42:59AM -0400, Tom Lane wrote: >> Justin Pryzby writes: >>> On Sun, Mar 12, 2023 at 06:25:13PM -0400, Tom Lane wrote: I agree that adding such a field to IndexStmt would be a very bad idea. However

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-03-16 Thread Ilya Gladyshev
> 16 марта 2023 г., в 04:07, Justin Pryzby написал(а): > > On Tue, Mar 14, 2023 at 06:58:14PM +0400, Ilya Gladyshev wrote: >>> The only change from the current patch is (3). (1) still calls >>> count_leaf_partitions(), but only once. I'd prefer that to rearra

Re: Progress report of CREATE INDEX for nested partitioned tables

2022-12-12 Thread Ilya Gladyshev
> Could you check what I've written as a counter-proposal ? I think that this might be a good solution to start with, it gives us the opportunity to improve the granularity later without any surprising changes for the end user. We could use this patch for previous versions and make more granul

Re: Progress report of CREATE INDEX for nested partitioned tables

2022-12-13 Thread Ilya Gladyshev
On Mon, 2022-12-12 at 22:43 -0600, Justin Pryzby wrote: > On Mon, Dec 12, 2022 at 11:39:23PM +0400, Ilya Gladyshev wrote: > > > > > Could you check what I've written as a counter-proposal ? > > > > I think that this might be a good solution to start with,

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-01-09 Thread Ilya Gladyshev
partitioned tables" patch, > which I think is otherwise done. I suggest that we move on with the IndexStmt patch and see what the committers have to say about it. I have brushed the patch up a bit, fixing TODOs and adding docs as per our discussion above. From 490d8afa7cb952e5b3947d8

backtrace_functions emits trace for any elog

2023-09-04 Thread Ilya Gladyshev
Hi, I used backtrace_functions to debug one of my ideas and found its behavior counter-intuitive and contradictory to it own docs. I think the GUC is supposed to be used to dump backtrace only on elog(ERROR) (should it also be done for higher levels? not sure about this), but, in fact, it does t

postgres_fdw binary protocol support

2022-11-21 Thread Ilya Gladyshev
and understand, whether this is a welcome feature, before proceeding with all of the abovementioned. Thanks, Ilya Gladyshev From 2cb72df03ed94d55cf51531a2d21a7d3369ae27b Mon Sep 17 00:00:00 2001 From: Ilya Gladyshev Date: Sat, 19 Nov 2022 17:47:49 +0400 Subject: [PATCH] postgres_fdw libpq binar

Re: postgres_fdw binary protocol support

2022-11-24 Thread Ilya Gladyshev
> 22 нояб. 2022 г., в 17:10, Ashutosh Bapat > написал(а): > > Hi Illya, > > On Mon, Nov 21, 2022 at 8:50 PM Ilya Gladyshev > wrote: >> >> Hi everyone, >> >> I have made a patch that introduces support for libpq binary protocol >> in

Re: CREATE INDEX CONCURRENTLY on partitioned index

2022-12-03 Thread Ilya Gladyshev
ink that fixing this problem requires some heavy code rewrite and I'm not quite sure how to go about it, if you have any ideas, I will be happy to try them out. Thanks, Ilya From 8eb9fd7ce7d34c5c323c47b60a7f883f360ef090 Mon Sep 17 00:00:00 2001 From: Ilya Gladyshev Date: Sat, 3 Dec 2022 18:20

Re: CREATE INDEX CONCURRENTLY on partitioned index

2022-12-09 Thread Ilya Gladyshev
On Sun, 2022-12-04 at 13:09 -0600, Justin Pryzby wrote: > > This beavior is fixed.  I re-factored and re-implented to use > DefineIndex() for building indexes concurrently rather than > reindexing. > That makes the patch smaller, actually, and has the added benefit of > splitting off the "Concurre

Progress report of CREATE INDEX for nested partitioned tables

2022-12-10 Thread Ilya Gladyshev
rrent behaviour to report the total number of partitions in the inheritance tree and fixed recursion in the attached patch. I used a static variable to keep the counter to avoid ABI breakage of DefineIndex, so that we could backpatch this to previous versions. Thanks, Ilya Gladyshev

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-01-31 Thread Ilya Gladyshev
> 17 янв. 2023 г., в 23:44, Tomas Vondra > написал(а): > Do we actually need the new parts_done field? I mean, we already do > track the value - at PROGRESS_CREATEIDX_PARTITIONS_DONE index in the > st_progress_param array. Can't we simply read it from there? Then we > would not have ABI issues w

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-01-31 Thread Ilya Gladyshev
> 1 февр. 2023 г., в 08:29, Justin Pryzby написал(а): > > On Tue, Jan 31, 2023 at 07:32:20PM +0400, Ilya Gladyshev wrote: >>> 17 янв. 2023 г., в 23:44, Tomas Vondra >>> написал(а): >>> Do we actually need the new parts_done field? I mean, we

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-02-01 Thread Ilya Gladyshev
> 1 февр. 2023 г., в 16:01, Alvaro Herrera написал(а): > > Hmm, count_leaf_partitions has to scan pg_inherits and do a syscache > lookup for every single element therein ... this sounds slow. > > In one of the callsites, we already have the partition descriptor > available. We could just scan

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-02-01 Thread Ilya Gladyshev
> 1 февр. 2023 г., в 20:27, Matthias van de Meent > написал(а): > > On Wed, 1 Feb 2023 at 16:53, Justin Pryzby <mailto:pry...@telsasoft.com>> wrote: >> >> On Wed, Feb 01, 2023 at 04:21:35PM +0100, Matthias van de Meent wrote: >>> On Wed, 1 Feb

Re: Segfault on logical replication to partitioned table with foreign children

2022-10-31 Thread Ilya Gladyshev
On Sun, 2022-10-30 at 16:52 +0100, Alvaro Herrera wrote: > On 2022-Oct-28, ilya.v.gladys...@gmail.com wrote: > > > This will cause a segfault or raise an assert, because inserting > > into > > foreign tables via logical replication is not possible. The > > solution I > > propose is to add recursiv

Re: Segfault on logical replication to partitioned table with foreign children

2022-10-31 Thread Ilya Gladyshev
ted > > > Regards, > Shi yu I have verified that the current patch handles the attaching of new partitions to the target partitioned table by throwing an error on attempt to insert into a foreign table inside the logical replication worker. I have refactored the code to minimize cache l

Re: Segfault on logical replication to partitioned table with foreign children

2022-11-03 Thread Ilya Gladyshev
n the previous patch, I think we can just use it. Attached a separate patch with the refactoring. From 004c63a8eba777be739f062cdc9b7ddcf2eac253 Mon Sep 17 00:00:00 2001 From: Ilya Gladyshev Date: Thu, 3 Nov 2022 11:39:24 +0400 Subject: [PATCH] Delay namespace and relname lookups until error ---

Re: Update of partition key on foreign server

2021-08-26 Thread Ilya Gladyshev
ate should be replaced with a foreign delete + local/foreign insert. In addition, a direct update should be forbidden when the query modifies the partition key. I’m probably missing a lot of details (feel free to point out), but is the general idea correct? I will be grateful for any

Per query FDW network stat collection

2021-08-24 Thread Ilya Gladyshev
now what the community thinks about the patch. Regards, Ilya Gladyshev >From 3ffbe071480672189c2e03d7e54707c77ba58b0b Mon Sep 17 00:00:00 2001 From: Ilya Gladyshev Date: Mon, 23 Aug 2021 21:37:31 +0300 Subject: [PATCH] adds per query FDW network usage stats Adds means for collecting network

Re: Per query FDW network stat collection

2021-08-24 Thread Ilya Gladyshev
On 24.08.2021 12:19, Julien Rouhaud wrote: However I'm not sure that having a new "network" option is the best way for that. It seems confusing as IIUC it won't be catching all network activity (like fe/be activity, or network disk...) but only FDW activity. I think it would be better to have

Re: CREATE INDEX CONCURRENTLY on partitioned index

2024-06-15 Thread Ilya Gladyshev
On 28.05.2024 07:05, Alexander Pyhalov wrote: Ilya Gladyshev писал(а) 2024-05-28 02:52: Also I'd like to note that in new patch version there's a strange wording in documentation: "This can be very convenient as not only will all existing partitions be  indexed, but any fu

Re: CREATE INDEX CONCURRENTLY on partitioned index

2024-06-20 Thread Ilya Gladyshev
On 15.06.2024 20:40, Justin Pryzby wrote: On Thu, May 23, 2024 at 10:14:57PM +0100, Ilya Gladyshev wrote: Hi, I think it's well worth the effort to revive the patch, so I rebased it on master, updated it and will return it back to the commitfest. Alexander, Justin feel free to add yours

Re: CREATE INDEX CONCURRENTLY on partitioned index

2024-07-11 Thread Ilya Gladyshev
counters. > 11 июля 2024 г., в 13:31, Justin Pryzby написал(а): > > On Sat, Jun 15, 2024 at 07:56:38PM +0100, Ilya Gladyshev wrote: >> In addition, I noticed that progress tracking is once again broken for >> partitioned tables, while looking at REINDEX implementation, attachin

REINDEX not updating partition progress

2024-07-12 Thread Ilya Gladyshev
ge-id/55cfae76-2ffa-43ed-a7e7-901bffbebee4%40gmail.com From 18baa028e1cc5c39347b9126ec1a96eb99e8e3e1 Mon Sep 17 00:00:00 2001 From: Ilya Gladyshev Date: Tue, 11 Jun 2024 17:48:08 +0100 Subject: [PATCH] make REINDEX track partition progress --- src/backend/catalog/index.c | 11 -- src/ba

Re: CREATE INDEX CONCURRENTLY on partitioned index

2024-07-12 Thread Ilya Gladyshev
last version of the relevant patch here to avoid any confusion. [1] https://www.postgresql.org/message-id/b72f2d89-820a-4fa2-9058-b155cf646f4f%40gmail.com From acf5cf5d4a984c0f8635a25e03c23409601c0c93 Mon Sep 17 00:00:00 2001 From: Ilya Gladyshev Date: Thu, 23 May 2024 18:13:41 +0100 Subject

Re: CREATE INDEX CONCURRENTLY on partitioned index

2024-05-23 Thread Ilya Gladyshev
ba70febc32017bc04c0a2 Mon Sep 17 00:00:00 2001 From: Ilya Gladyshev Date: Thu, 23 May 2024 18:13:41 +0100 Subject: [PATCH v2] Allow CREATE INDEX CONCURRENTLY on partitioned table --- doc/src/sgml/ddl.sgml | 7 +- doc/src/sgml/ref/create_index.sgml | 14 +- src/backe

Re: CREATE INDEX CONCURRENTLY on partitioned index

2024-05-27 Thread Ilya Gladyshev
On 24.05.2024 10:04, Alexander Pyhalov wrote: Ilya Gladyshev писал(а) 2024-05-24 00:14: Hi, Hi. I think it's well worth the effort to revive the patch, so I rebased it on master, updated it and will return it back to the commitfest. Alexander, Justin feel free to add yourselv

Re: optimizing pg_upgrade's once-in-each-database steps

2024-08-30 Thread Ilya Gladyshev
LGTM in general, but here are some final nitpicks: + if (maxFd != 0) + (void) select(maxFd + 1, &input_mask, &output_mask, &except_mask, NULL); It’s a good idea to check for the return value of select, in case it returns any errors. + dbs_complete++; +

Re: optimizing pg_upgrade's once-in-each-database steps

2024-09-03 Thread Ilya Gladyshev
On 01.09.2024 22:05, Nathan Bossart wrote: I think we can actually just use PQstatus() here. But furthermore, I think the way I was initiating connections was completely bogus. IIUC before calling PQconnectPoll() the first time, we should wait for a write indicator from select(), and then we s

Re: REINDEX not updating partition progress

2024-07-20 Thread Ilya Gladyshev
19 июля 2024 г., в 04:17, Michael Paquier написал(а): On Fri, Jul 12, 2024 at 11:07:49PM +0100, Ilya Gladyshev wrote: While working on CIC for partitioned tables [1], I noticed that REINDEX for partitioned tables is not tracking keeping progress of partitioned tables, so I'm creat

Re: REINDEX not updating partition progress

2024-07-21 Thread Ilya Gladyshev
Gladyshev написал(а):19 июля 2024 г., в 04:17, Michael Paquier  написал(а):On Fri, Jul 12, 2024 at 11:07:49PM +0100, Ilya Gladyshev wrote:While working on CIC for partitioned tables [1], I noticed that REINDEX forpartitioned tables is not tracking keeping progress of partitioned tables,so I'm creat

Re: pg_rewind with cascade standby doesn't work well

2024-07-22 Thread Ilya Gladyshev
> Hi, Thank you for addressing this issue! The patch needs to be rebased as it doesn’t apply on master anymore, but here are some thoughts on the patch in general without testing: 1. Regarding the approach to force a checkpoint on every restartpoint record, I wonder if it has any performan

Re: REINDEX not updating partition progress

2024-07-25 Thread Ilya Gladyshev
25 июля 2024 г., в 09:55, Michael Paquier написал(а): On Sun, Jul 21, 2024 at 11:41:43AM +0100, Ilya Gladyshev wrote: Forgot to update partition_relid in reindex_index in the second patch. Fixed in attachment.    relid oid       -   OID of the table on which the index is

Re: optimizing pg_upgrade's once-in-each-database steps

2024-07-31 Thread Ilya Gladyshev
On 22.07.2024 21:07, Nathan Bossart wrote: On Fri, Jul 19, 2024 at 04:21:37PM -0500, Nathan Bossart wrote: However, while looking into this, I noticed that only one get_query callback (get_db_subscription_count()) actually customizes the generated query using information in the provided DbInfo

Re: optimizing pg_upgrade's once-in-each-database steps

2024-08-04 Thread Ilya Gladyshev
On 01.08.2024 22:41, Nathan Bossart wrote: Here is a new patch set. Besides rebasing, I've added the recursive call to process_slot() mentioned in the quoted text, and I've added quite a bit of commentary to async.c. That's much better now, thanks! Here's my code review, note that I haven't t