Re: speedup COPY TO for partitioned table.

2025-04-15 Thread Kirill Reshke
On Thu, 10 Apr 2025 at 17:37, jian he wrote: > > > I think the current test example is fine. Ok, let it be so. I changed status to RFQ as I have no more input here, and other reviewers in thread remain silent (so I assume they are fine with v10) -- Best regards, Kirill Reshke

Re: speedup COPY TO for partitioned table.

2025-04-10 Thread jian he
On Thu, Apr 10, 2025 at 4:25 PM Kirill Reshke wrote: > > On Thu, 10 Apr 2025 at 07:45, jian he wrote: > > > > hi. > > > > rebase and simplify regress tests. > > HI! > You used CREATE TABLE PARTITION OF syntax for the second level of > partitioning scheme, but not for the first level. Is there any

Re: speedup COPY TO for partitioned table.

2025-04-10 Thread Kirill Reshke
On Thu, 10 Apr 2025 at 07:45, jian he wrote: > > hi. > > rebase and simplify regress tests. HI! You used CREATE TABLE PARTITION OF syntax for the second level of partitioning scheme, but not for the first level. Is there any reason? Also about column names. how about +CREATE TABLE pp (year int,

Re: speedup COPY TO for partitioned table.

2025-04-09 Thread jian he
hi. rebase and simplify regress tests. From f56c94ccb018928e41cc35e162174831cb016c1d Mon Sep 17 00:00:00 2001 From: jian he Date: Thu, 10 Apr 2025 10:41:40 +0800 Subject: [PATCH v10 1/1] support COPY partitioned_table TO CREATE TABLE pp (id int, val int ) PARTITION BY RANGE (id); CREATE TABLE p

Re: speedup COPY TO for partitioned table.

2025-04-07 Thread Kirill Reshke
On Fri, 4 Apr 2025, 15:17 Kirill Reshke, wrote: > Hi! > > First of all, a commit message does not need to contain SQL examples > of what it does. We should provide human-readable explanations and > that's it. > > Next, about changes to src/test/regress/sql/copy2.sql. I find the sql > you used to

Re: speedup COPY TO for partitioned table.

2025-04-07 Thread Kirill Reshke
Sorry, wrong thread Best regards, Kirill Reshke On Mon, 7 Apr 2025, 19:54 Kirill Reshke, wrote: > > On Fri, 4 Apr 2025, 15:17 Kirill Reshke, wrote: > >> Hi! >> >> First of all, a commit message does not need to contain SQL examples >> of what it does. We should provide human-readable explanati

Re: speedup COPY TO for partitioned table.

2025-04-04 Thread Kirill Reshke
Hi! First of all, a commit message does not need to contain SQL examples of what it does. We should provide human-readable explanations and that's it. Next, about changes to src/test/regress/sql/copy2.sql. I find the sql you used to test really unintuitive. How about CREATE TABLE ... PARTITION OF

Re: speedup COPY TO for partitioned table.

2025-04-03 Thread jian he
On Tue, Apr 1, 2025 at 1:38 PM vignesh C wrote: > > On Tue, 1 Apr 2025 at 06:31, jian he wrote: > > > > On Mon, Mar 31, 2025 at 4:05 PM Kirill Reshke > > wrote: > > > > Thanks for doing the benchmark. > > Few comments to improve the comments, error message and remove > redundant assignment: > 1

Re: speedup COPY TO for partitioned table.

2025-03-31 Thread vignesh C
On Tue, 1 Apr 2025 at 06:31, jian he wrote: > > On Mon, Mar 31, 2025 at 4:05 PM Kirill Reshke wrote: > > Thanks for doing the benchmark. Few comments to improve the comments, error message and remove redundant assignment: 1) How about we change below: /* * partition's rowtype might differ from

Re: speedup COPY TO for partitioned table.

2025-03-31 Thread jian he
On Mon, Mar 31, 2025 at 4:05 PM Kirill Reshke wrote: > > Hi! > I reviewed v7. Maybe we should add a multi-level partitioning case > into copy2.sql regression test? > sure. > I also did quick benchmarking for this patch: > > DDL > > create table ppp(i int) partition by range (i); > > genddl

Re: speedup COPY TO for partitioned table.

2025-03-31 Thread Kirill Reshke
Hi! I reviewed v7. Maybe we should add a multi-level partitioning case into copy2.sql regression test? I also did quick benchmarking for this patch: DDL create table ppp(i int) partition by range (i); genddl.sh: for i in `seq 0 200`; do echo "create table p$i partition of ppp for values

Re: speedup COPY TO for partitioned table.

2025-03-30 Thread jian he
On Sun, Mar 30, 2025 at 9:14 AM vignesh C wrote: > > On Sat, 29 Mar 2025 at 12:08, jian he wrote: > > > > > > I consolidated it into a new function: CopyThisRelTo. > > Few comments: > 1) Here the error message is not correct, we are printing the original > table from where copy was done which is

Re: speedup COPY TO for partitioned table.

2025-03-29 Thread vignesh C
On Sat, 29 Mar 2025 at 12:08, jian he wrote: > > > I consolidated it into a new function: CopyThisRelTo. Few comments: 1) Here the error message is not correct, we are printing the original table from where copy was done which is a regular table and not a foreign table, we should use childreloid

Re: speedup COPY TO for partitioned table.

2025-03-28 Thread jian he
On Fri, Mar 28, 2025 at 9:03 PM vignesh C wrote: > > On Fri, 28 Mar 2025 at 08:39, jian he wrote: > > > > hi. > > > > I made a mistake. > > The regress test sql file should have a new line at the end of the file. > > Couple of suggestions: > 1) Can you add some comments here, this is the only cod

Re: speedup COPY TO for partitioned table.

2025-03-28 Thread vignesh C
On Fri, 28 Mar 2025 at 08:39, jian he wrote: > > hi. > > I made a mistake. > The regress test sql file should have a new line at the end of the file. Couple of suggestions: 1) Can you add some comments here, this is the only code that is different from the regular table handling code: +

Re: speedup COPY TO for partitioned table.

2025-03-27 Thread jian he
hi. I made a mistake. The regress test sql file should have a new line at the end of the file. From a4c643ac3a9f40bbdf07dcacc38527ef6e86f1bc Mon Sep 17 00:00:00 2001 From: jian he Date: Fri, 28 Mar 2025 11:05:53 +0800 Subject: [PATCH v5 1/1] support COPY partitioned_table TO CREATE TABLE pp (id

Re: speedup COPY TO for partitioned table.

2025-03-27 Thread jian he
On Fri, Mar 21, 2025 at 6:13 PM vignesh C wrote: > > I find an issue with the patch: > > -- Setup > CREATE SERVER myserver FOREIGN DATA WRAPPER postgres_fdw OPTIONS > (dbname 'testdb', port '5432'); > CREATE TABLE t1(id int) PARTITION BY RANGE(id); > CREATE TABLE part1 PARTITION OF t1 FOR VALUES F

Re: speedup COPY TO for partitioned table.

2025-03-22 Thread vignesh C
On Tue, 11 Mar 2025 at 18:24, jian he wrote: > > after my change: > > COPY TO can be used only with plain tables, not views, and does not > copy rows from child tables, > however COPY TO can be used with partitioned tables. > For example, in a table inheritance hierarchy, COPY table T

Re: speedup COPY TO for partitioned table.

2025-03-17 Thread newtglobal postgresql_contributors
Hi Jian, Tested this patch with COPY sales TO STDOUT; ~ 1.909ms, improving performance over the older COPY (SELECT * FROM sales) TO STDOUT; ~ 3.80ms method. This eliminates query planning overhead and significantly speeds up data export from partitioned tables. Our test setup involved creating

Re: speedup COPY TO for partitioned table.

2025-03-11 Thread jian he
On Fri, Mar 7, 2025 at 6:41 PM jian he wrote: > > hi. > > rebased and polished patch attached, test case added. hi. I realized I need to change the doc/src/sgml/ref/copy.sgml Notes section. current doc note section: COPY TO can be used only with plain tables, not views, and does not copy rows fr

Re: speedup COPY TO for partitioned table.

2025-03-07 Thread jian he
hi. rebased and polished patch attached, test case added. However there is a case (the following) where ``COPY(partitioned_table)`` is much slower (around 25% in some cases) than ``COPY (select * from partitioned_table)``. If the partition attribute order is not the same as the partitioned table,

Re: speedup COPY TO for partitioned table.

2025-02-10 Thread David Rowley
On Tue, 11 Feb 2025 at 08:10, Melih Mutlu wrote: > jian he , 27 Oca 2025 Pzt, 04:47 tarihinde şunu > yazdı: >> so the find_all_inheritors output order is deterministic? > > You're right that order in find_all_inheritors is deterministic. But it's not > always the same with the order of SELECT ou

Re: speedup COPY TO for partitioned table.

2025-02-10 Thread Melih Mutlu
Hi, jian he , 27 Oca 2025 Pzt, 04:47 tarihinde şunu yazdı: > in the > > find_all_inheritors->find_inheritance_children->find_inheritance_children_extended > > find_inheritance_children_extended we have > """ > if (numoids > 1) > qsort(oidarr, numoids, sizeof(Oid), oid_cmp); > """ > >

Re: speedup COPY TO for partitioned table.

2025-01-27 Thread Nathan Bossart
On Wed, Jan 22, 2025 at 01:54:32AM +0300, Melih Mutlu wrote: > I confirm that the patch introduces some improvement in simple cases like > the one you shared. I looked around a bit to understand whether there is an > obvious reason why copying from a partitioned table is not allowed, but > couldn't

Re: speedup COPY TO for partitioned table.

2025-01-26 Thread jian he
On Wed, Jan 22, 2025 at 6:54 AM Melih Mutlu wrote: > > Hi Jian, > > Thanks for the patch. > > jian he , 19 Ara 2024 Per, 15:03 tarihinde şunu > yazdı: >> >> attached copy_par_regress_test.sql is a simple benchmark sql file, >> a partitioned table with 10 partitions, 2 levels of indirection. >> Th

Re: speedup COPY TO for partitioned table.

2025-01-21 Thread Melih Mutlu
Hi Jian, Thanks for the patch. jian he , 19 Ara 2024 Per, 15:03 tarihinde şunu yazdı: > attached copy_par_regress_test.sql is a simple benchmark sql file, > a partitioned table with 10 partitions, 2 levels of indirection. > The simple benchmark shows around 7.7% improvement in my local environme

speedup COPY TO for partitioned table.

2024-12-19 Thread jian he
2024 19:48:12 +0800 Subject: [PATCH v1 1/1] speedup COPY TO for partitioned table. COPY (select_query) generally slower than table_beginscan.. table_scan_getnextslot ..table_endscan. especially for partitioned table. so using table_beginscan.. table_scan_getnextslot ..table_endscan for COPY TO w