Re: A new strategy for pull-up correlated ANY_SUBLINK

2022-11-01 Thread Andrey Lepikhov
a has such corner cases too ? [1] https://www.postgresql.org/message-id/flat/CALNJ-vTa5VgvV1NPRHnypdnbx-fhDu7vWp73EkMUbZRpNHTYQQ%40mail.gmail.com -- regards, Andrey Lepikhov Postgres Professional

Re: join pushdown and issue with foreign update

2021-06-02 Thread Andrey Lepikhov
ed to be pushed to a foreign server? Potentially, I can imagine pushed-down JOIN with arbitrary ROW function in its target list. Amit's approach looks more safe for me. -- regards, Andrey Lepikhov Postgres Professional

Re: Asynchronous Append on postgres_fdw nodes.

2021-06-03 Thread Andrey Lepikhov
On 3/6/21 14:49, Etsuro Fujita wrote: On Tue, May 11, 2021 at 6:55 PM Etsuro Fujita wrote: On Tue, May 11, 2021 at 6:27 PM Andrey Lepikhov wrote: On 11/5/21 12:24, Etsuro Fujita wrote: -> Append (actual rows=3000 loops=1) -> Async Foreign Scan on f1 (actual

Fast COPY FROM based on batch insert

2021-06-04 Thread Andrey Lepikhov
ced to 50s. [1] https://www.postgresql.org/message-id/flat/3d0909dc-3691-a576-208a-90986e55489f%40postgrespro.ru -- regards, Andrey Lepikhov Postgres Professional From 715406ce4a98df4e0aecdfdf9d9f59cd3a13101e Mon Sep 17 00:00:00 2001 From: "Andrey V. Lepikhov" Date: Fri, 4 Jun 2021 13

Re: Fast COPY FROM based on batch insert

2021-06-06 Thread Andrey Lepikhov
On 4/6/21 13:45, tsunakawa.ta...@fujitsu.com wrote: From: Andrey Lepikhov We still have slow 'COPY FROM' operation for foreign tables in current master. Now we have a foreign batch insert operation And I tried to rewrite the patch [1] with this machinery. I haven't looked at

Re: Fast COPY FROM based on batch insert

2021-06-07 Thread Andrey Lepikhov
Second version of the patch fixes problems detected by the FDW regression tests and shows differences of error reports in tuple-by-tuple and batched COPY approaches. -- regards, Andrey Lepikhov Postgres Professional From 68ad02038d7477e005b65bf5aeeac4efbb41073e Mon Sep 17 00:00:00 2001 From

Add '--ignore-errors' into pg_regress

2021-06-28 Thread Andrey Lepikhov
ial and can be easy reverted if needed. An example: TEMP_CONFIG=/tmp/extra.config \ EXTRA_REGRESS_OPTS="--load-extension=aqo --ignore-errors --schedule=src/test/regress/parallel_schedule" \ make check-world Maybe I just don't know the right way? -- regards, Andrey

Re: Add '--ignore-errors' into pg_regress

2021-06-29 Thread Andrey Lepikhov
On 29/6/21 20:59, Tom Lane wrote: Andrey Lepikhov writes: BTW, I wonder if you can't get much or all of the same effect from "make -k check-world". Thank you, 'make -k' is suitable solution in such situation. -- regards, Andrey Lepikhov Postgres Professional

Re: Defer selection of asynchronous subplans until the executor initialization stage

2021-06-29 Thread Andrey Lepikhov
On 11/5/21 06:55, Zhihong Yu wrote: On Mon, May 10, 2021 at 8:45 PM Andrey Lepikhov mailto:a.lepik...@postgrespro.ru>> wrote: It seems the if statement is not needed: you can directly assign false to  subplan->async_capable. I have completely rewritten this patch. Main i

Re: Removing unneeded self joins

2021-06-30 Thread Andrey Lepikhov
NULL)) Rows Removed by Filter: 1000 It could reduce a number of selectivity mistakes, but increase CPU consumption. If we had such a clause analyzing machinery, we could trivially remove this unneeded qual. -- regards, Andrey Lepikhov Postgres Professional

Re: Merging statistics from children instead of re-sampling everything

2021-06-30 Thread Andrey Lepikhov
ther types of statistics. On 6/29/21 9:01 AM, Andrey Lepikhov wrote: On 30/3/21 03:51, Tomas Vondra wrote: Of course, that assumes the merge is cheaper than processing the list of statistics, but I find that plausible, especially the list needs to be processed multiple (e.g. when considering

Re: Removing unneeded self joins

2021-06-30 Thread Andrey Lepikhov
On 30/6/21 18:55, Laurenz Albe wrote: On Wed, 2021-06-30 at 14:21 +0300, Andrey Lepikhov wrote: I think, here we could ask more general question: do we want to remove a 'IS NOT NULL' clause from the clause list if the rest of the list implicitly implies it? Generally I have the impre

Re: Increase value of OUTER_VAR

2021-07-05 Thread Andrey Lepikhov
value. - exec_rt_fetch(Index) calls list_nth(int). - generate_subquery_vars() accepts an 'Index varno' value It looks sloppy. Do you plan to change this in the next commits? -- regards, Andrey Lepikhov Postgres Professional

Re: Asymmetric partition-wise JOIN

2021-07-05 Thread Andrey Lepikhov
On 18/6/21 15:02, Alexander Pyhalov wrote: Andrey Lepikhov писал 2021-05-27 07:27: Next version of the patch. For searching any problems I forced this patch during 'make check' tests. Some bugs were found and fixed. Hi. I've tested this patch and haven't found issues, but

Re: Removing unneeded self joins

2021-07-05 Thread Andrey Lepikhov
On 2/7/21 01:56, Hywel Carver wrote: On Wed, Jun 30, 2021 at 12:21 PM Andrey Lepikhov mailto:a.lepik...@postgrespro.ru>> wrote: I think, here we could ask more general question: do we want to remove a 'IS NOT NULL' clause from the clause list if the rest of the lis

Re: Asymmetric partition-wise JOIN

2021-07-06 Thread Andrey Lepikhov
On 5/7/21 23:15, Zhihong Yu wrote: On Mon, Jul 5, 2021 at 2:57 AM Andrey Lepikhov mailto:a.lepik...@postgrespro.ru>> wrote: +            * Can't imagine situation when join relation already exists. But in +            * the 'partition_join' regression test it happens. +  

Re: Removing unneeded self joins

2022-05-19 Thread Andrey Lepikhov
On 5/17/22 19:14, Ronan Dunklau wrote: Le vendredi 13 mai 2022, 07:07:47 CEST Andrey Lepikhov a écrit : New version of the feature. Here a minor bug with RowMarks is fixed. A degenerated case is fixed, when uniqueness of an inner deduced not from join quals, but from a baserestrictinfo clauses

Compare variables of composite type with slightly different column types

2022-05-26 Thread Andrey Lepikhov
, but don't found any rationale. -- Regards Andrey Lepikhov Postgres Professional

Re: Compare variables of composite type with slightly different column types

2022-05-28 Thread Andrey Lepikhov
On 26/5/2022 14:25, Andrey Lepikhov wrote: I guess, here the compatible_oper() routine can be used to find a appropriate operator, or something like that can be invented. I looked into the 2cd7084 and a4424c5, but don't found any rationale. In accordance to this idea I prepared a code.

Re: [PROPOSAL] Detecting plan changes with plan_id in pg_stat_activity

2022-06-15 Thread Andrey Lepikhov
://www.postgresql.org/message-id/flat/e0de3423-4bba-1e69-c55a-f76bf18dbd74%40postgrespro.ru -- regards, Andrey Lepikhov Postgres Professional

Postgres do not allow to create many tables with more than 63-symbols prefix

2022-06-23 Thread Andrey Lepikhov
Moved from the pgsql-bugs mailing list [1]. On 6/23/22 07:03, Masahiko Sawada wrote: > Hi, > > On Sat, Jun 4, 2022 at 4:03 AM Andrey Lepikhov > wrote: >> >> According to subj you can try to create many tables (induced by the case >> of partitioned table) with l

Re: Implement hook for self-join simplification

2022-06-24 Thread Andrey Lepikhov
free to use it in your research. [1] https://www.postgresql.org/message-id/a1d6290c-44e0-0dfc-3fca-66a68b310...@postgrespro.ru -- regards, Andrey Lepikhov Postgres Professional

Re: Implement hook for self-join simplification

2022-06-24 Thread Andrey Lepikhov
generalize parts of the core code and thus, reduce size of your code a lot. But if you want to use your code with many PG versions, even already working in production or you make just a research, without immediate practical result - your choice is an extension. -- regards, Andrey Lepikhov Postgres

Re: Postgres do not allow to create many tables with more than 63-symbols prefix

2022-06-26 Thread Andrey Lepikhov
On 6/27/22 06:38, Masahiko Sawada wrote: On Fri, Jun 24, 2022 at 2:12 PM Andrey Lepikhov wrote: On 6/23/22 07:03, Masahiko Sawada wrote: > On Sat, Jun 4, 2022 at 4:03 AM Andrey Lepikhov > wrote: >> It is very corner case, of course. But solution is easy and short. So, &

Re: Removing unneeded self joins

2022-06-30 Thread Andrey Lepikhov
On 19/5/2022 16:47, Ronan Dunklau wrote: I'll take a look at that one. New version of the patch, rebased on current master: 1. pgindent over the patch have passed. 2. number of changed files is reduced. 3. Some documentation and comments is added. -- regards, Andrey Lepikhov Pos

Re: Fast COPY FROM based on batch insert

2022-07-07 Thread Andrey Lepikhov
On 7/7/2022 06:14, Ian Barwick wrote: 2022年3月24日(木) 15:44 Andrey V. Lepikhov : > > On 3/22/22 06:54, Etsuro Fujita wrote: > > On Fri, Jun 4, 2021 at 5:26 PM Andrey Lepikhov > > wrote: > >> We still have slow 'COPY FROM' operation for foreign tables in

Re: Fast COPY FROM based on batch insert

2022-07-08 Thread Andrey Lepikhov
ghtly different change (see in attachment). -- regards, Andrey Lepikhov Postgres Professionaldiff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c index 245a260982..203289f7f2 100644 --- a/src/backend/commands/copyfrom.c +++ b/src/backend/commands/copyfrom.c @

Re: Fast COPY FROM based on batch insert

2022-07-10 Thread Andrey Lepikhov
On 11/7/2022 04:12, Ian Barwick wrote: On 09/07/2022 00:09, Andrey Lepikhov wrote: On 8/7/2022 05:12, Ian Barwick wrote: ERROR:  bind message supplies 0 parameters, but prepared statement "pgsql_fdw_prep_178" requires 6 CONTEXT:  remote SQL command: INSERT INTO public.fo

Re: Postgres picks suboptimal index after building of an extended statistics

2022-07-11 Thread Andrey Lepikhov
On 7/8/22 03:07, Tom Lane wrote: Andrey Lepikhov writes: On 12/8/21 04:26, Tomas Vondra wrote: I wonder if we should teach clauselist_selectivity about UNIQUE indexes, and improve the cardinality estimates directly, not just costing for index scans. I tried to implement this in different

Re: Fast COPY FROM based on batch insert

2022-07-19 Thread Andrey Lepikhov
, maybe to describe in documentation, if the value of batch_size is more than 1, the ExecForeignBatchInsert routine have a chance to be called? -- regards, Andrey Lepikhov Postgres Professional

Re: [PoC] Reducing planning time when tables have many partitions

2022-07-21 Thread Andrey Lepikhov
el free to reverse the status if you need more feedback. -- Regards Andrey Lepikhov Postgres Professional

Re: Fast COPY FROM based on batch insert

2022-07-21 Thread Andrey Lepikhov
On 7/20/22 13:10, Etsuro Fujita wrote: On Tue, Jul 19, 2022 at 6:35 PM Andrey Lepikhov wrote: On 18/7/2022 13:22, Etsuro Fujita wrote: I rewrote the decision logic to something much simpler and much less invasive, which reduces the patch size significantly. Attached is an updated patch

Re: Fast COPY FROM based on batch insert

2022-07-22 Thread Andrey Lepikhov
On 7/22/22 13:14, Etsuro Fujita wrote: On Fri, Jul 22, 2022 at 3:39 PM Andrey Lepikhov Why such cascade flush is really necessary, especially for BEFORE and INSTEAD OF triggers? BEFORE triggers on the chosen partition might query the parent table, not just the partition, so I think we need to

Re: Fast COPY FROM based on batch insert

2022-07-26 Thread Andrey Lepikhov
On 7/22/22 13:14, Etsuro Fujita wrote: On Fri, Jul 22, 2022 at 3:39 PM Andrey Lepikhov wrote: Analyzing multi-level heterogeneous partitioned configurations I realized, that single write into a partition with a trigger will flush buffers for all other partitions of the parent table even if the

Re: Fast COPY FROM based on batch insert

2022-08-14 Thread Andrey Lepikhov
r example: 'COPY %s (buffered)' or 'COPY FOREIGN TABLE %s' or, if instead of relname_only field to save a MultiInsertBuffer pointer, we might add min/max linenos into the report: 'COPY %s, line between %llu and %llu' -- Regards Andrey Lepikhov Postgres Professional

Re: Postgres picks suboptimal index after building of an extended statistics

2021-08-29 Thread Andrey Lepikhov
hat way and propose a new patch in attachment. -- regards, Andrey Lepikhov Postgres Professional From 41ce6007cd552afd1a73983f0b9c9cac0e125d58 Mon Sep 17 00:00:00 2001 From: "Andrey V. Lepikhov" Date: Mon, 30 Aug 2021 11:21:57 +0500 Subject: [PATCH] Estimating number of fetched rows in

Re: Asymmetric partition-wise JOIN

2021-09-14 Thread Andrey Lepikhov
regression test added. I thought more and realized there isn't necessary to recurse in the adjust_child_relids_multilevel() routine if required_outer contains only normal_relids. Also, regression tests were improved a bit. -- regards, Andrey Lepikhov Postgres Professional

Re: Increase value of OUTER_VAR

2021-09-14 Thread Andrey Lepikhov
erformance. -- regards, Andrey Lepikhov Postgres Professional

Re: Asymmetric partition-wise JOIN

2021-07-14 Thread Andrey Lepikhov
On 5/7/21 23:15, Zhihong Yu wrote: On Mon, Jul 5, 2021 at 2:57 AM Andrey Lepikhov mailto:a.lepik...@postgrespro.ru>> wrote: +            * Can't imagine situation when join relation already exists. But in +            * the 'partition_join' regression test it happens. +  

Re: Removing unneeded self joins

2021-07-15 Thread Andrey Lepikhov
On 6/7/21 13:49, Hywel Carver wrote: On Mon, Jul 5, 2021 at 2:20 PM Andrey Lepikhov mailto:a.lepik...@postgrespro.ru>> wrote: Looking through the email chain, a previous version of this patch added ~0.6% to planning time in the worst case tested - does that meet the "essen

Re: Extra code in commit_ts.h

2021-08-03 Thread Andrey Lepikhov
On 3/8/21 15:16, David Rowley wrote: it diff 73c986adde5~1.. | grep check_track_commit_timestamp I think this is waste code. May be delete it? -- regards, Andrey Lepikhov Postgres Professional

Representation of SubPlan testexpr in EXPLAIN

2021-08-24 Thread Andrey Lepikhov
) It is a bit annoying when you don't have original query or don't trust competence of a user who sent you this explain. In attachment - patch which solves this problem. I'm not completely sure that this option really needed and patch presents a proof of concept only. -- r

Re: POC, WIP: OR-clause support for indexes

2023-07-09 Thread Andrey Lepikhov
k we don't need to free it at all. -- regards, Andrey Lepikhov Postgres Professional diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c index 961ca3e482..f0fd63f05c 100644 --- a/src/backend/parser/parse_expr.c +++ b/src/backend/parser/parse_expr.c @@ -112,9 +112,6

Re: Generating code for query jumbling through gen_node_support.pl

2023-07-10 Thread Andrey Lepikhov
open a way for extensions to have easy-supporting custom queryIds. -- regards, Andrey Lepikhov Postgres Professional

Re: Generating code for query jumbling through gen_node_support.pl

2023-07-10 Thread Andrey Lepikhov
On 11/7/2023 12:35, Michael Paquier wrote: On Tue, Jul 11, 2023 at 12:29:29PM +0700, Andrey Lepikhov wrote: I vote for only one method based on a query tree structure. Noted BTW, did you think about different algorithms of queryId generation? Not really, except if you are referring to the

Re: POC, WIP: OR-clause support for indexes

2023-07-11 Thread Andrey Lepikhov
out: (((a)::text = 'ab'::text) OR ((a)::text = ANY ('{NULL,cd}'::text[]))) to (((a)::text = ANY ('{NULL,cd}'::text[])) OR ((a)::text = 'ab'::text)) Transformations, mentioned above, are correct, of course. But it can be a sign of possible unstable

Re: POC: GROUP BY optimization

2023-07-19 Thread Andrey Lepikhov
e current path according to the reasons uttered in the revert commit. -- regards, Andrey Lepikhov Postgres Professional From 913d55ee887dccfeba360f5f44ed347dd1ba9044 Mon Sep 17 00:00:00 2001 From: "Andrey V. Lepikhov" Date: Fri, 14 Jul 2023 10:29:36 +0700 Subject: [PATCH] When evaluating

Re: POC: GROUP BY optimization

2023-07-20 Thread Andrey Lepikhov
On 20/7/2023 18:46, Tomas Vondra wrote: On 7/20/23 08:37, Andrey Lepikhov wrote: On 3/10/2022 21:56, Tom Lane wrote: Revert "Optimize order of GROUP BY keys". This reverts commit db0d67db2401eb6238ccc04c6407a4fd4f985832 and several follow-on fixes. ... Since we're hard up agai

Re: POC: GROUP BY optimization

2023-07-23 Thread Andrey Lepikhov
On 20/7/2023 18:46, Tomas Vondra wrote: On 7/20/23 08:37, Andrey Lepikhov wrote: On 3/10/2022 21:56, Tom Lane wrote: Revert "Optimize order of GROUP BY keys". This reverts commit db0d67db2401eb6238ccc04c6407a4fd4f985832 and several follow-on fixes. ... Since we're hard up agai

Re: POC: GROUP BY optimization

2023-07-24 Thread Andrey Lepikhov
On 24/7/2023 16:56, Tomas Vondra wrote: On 7/24/23 04:10, Andrey Lepikhov wrote: On 20/7/2023 18:46, Tomas Vondra wrote: On 7/20/23 08:37, Andrey Lepikhov wrote: On 3/10/2022 21:56, Tom Lane wrote: Revert "Optimize order of GROUP BY keys". This reve

Re: [PoC] Reducing planning time when tables have many partitions

2023-07-27 Thread Andrey Lepikhov
test case in attachment. Here is three queries. Execution times: 1 - 8s; 2 - 30s; 3 - 131s (with your patch set). 1 - 5s; 2 - 10s; 3 - 33s (current master). Maybe it is a false alarm, but on my laptop I see this degradation at every launch. -- regards, Andrey Lepikhov Postgres Professional part

Re: [PoC] Reducing planning time when tables have many partitions

2023-08-02 Thread Andrey Lepikhov
is one possible option. You introduced list_ptr_cmp as an extern function of a List, but use it the only under USE_ASSERT_CHECKING ifdef. Maybe you hide it under USE_ASSERT_CHECKING or remove all the stuff? -- regards, Andrey Lepikhov Postgres Professional

Re: [PoC] Reducing planning time when tables have many partitions

2023-08-07 Thread Andrey Lepikhov
lated info in the output of EXPLAIN ANALYZE makes tests more complex because of architecture dependency. -- regards, Andrey Lepikhov Postgres Professional

Re: [PoC] Reducing planning time when tables have many partitions

2023-08-07 Thread Andrey Lepikhov
On 7/8/2023 19:15, Ashutosh Bapat wrote: On Mon, Aug 7, 2023 at 2:21 PM Andrey Lepikhov mailto:a.lepik...@postgrespro.ru>> wrote: >> Do you think that the memory measurement patch I have shared in those threads is useful in itself? If so, I will start another

Re: Report planning memory in EXPLAIN ANALYZE

2023-08-10 Thread Andrey Lepikhov
used' metric only. The patch looks good, passes the tests. -- regards, Andrey Lepikhov Postgres Professional

Re: Report planning memory in EXPLAIN ANALYZE

2023-08-13 Thread Andrey Lepikhov
reporting on memory consumption at each subquery and join level. But it looks too much for typical queries. -- regards, Andrey Lepikhov Postgres Professional

Re: Fast COPY FROM based on batch insert

2022-08-22 Thread Andrey Lepikhov
, which I think makes the range information less useful. Maybe I'm too worried about that, though. I got your point. Indeed, perharps such info doesn't really needed to be included into the core, at least for now. -- regards, Andrey Lepikhov Postgres Professional

Re: Removing unneeded self joins

2022-08-26 Thread Andrey Lepikhov
On 30/6/2022 17:11, Andrey Lepikhov wrote: On 19/5/2022 16:47, Ronan Dunklau wrote: I'll take a look at that one. New version of the patch, rebased on current master: 1. pgindent over the patch have passed. 2. number of changed files is reduced. 3. Some documentation and comments is

Re: Removing unneeded self joins

2022-08-28 Thread Andrey Lepikhov
e - many senseless search cycles of self-joins. And it may have higher limit than GUCs above. So I introduced a guc, called "self_join_search_limit" (so far undocumented) that is an explicit limit for a set of plain relations in FROM-list to search self-joins. -- Regards Andrey

[POC] Allow flattening of subquery with a link to upper query

2022-08-30 Thread Andrey Lepikhov
oach. [1] Kim, Won. “On optimizing an SQL-like nested query.” ACM Trans. Database Syst. 7 (1982): 443-469. -- Regards Andrey Lepikhov Postgres ProfessionalFrom 3f4247b23175388f8c6ee43740fb641d97e39d0b Mon Sep 17 00:00:00 2001 From: "Andrey V. Lepikhov" Date: Tue, 24 May 2022 15:59

Re: [HACKERS] PoC: custom signal handler for extensions

2022-09-01 Thread Andrey Lepikhov
reaks main concept of the auto_explain extension? -- Regards Andrey Lepikhov Postgres Professional

Re: [POC] Allow flattening of subquery with a link to upper query

2022-09-02 Thread Andrey Lepikhov
On 9/1/22 17:24, Richard Guo wrote: On Wed, Aug 31, 2022 at 2:35 PM Andrey Lepikhov mailto:a.lepik...@postgrespro.ru>> wrote: Before flattening procedure we just look through the quals of subquery, pull to the upper level OpExpr's containing variables from the upper r

Re: [POC] Allow flattening of subquery with a link to upper query

2022-09-05 Thread Andrey Lepikhov
On 9/5/22 12:22, Richard Guo wrote: On Fri, Sep 2, 2022 at 7:09 PM Andrey Lepikhov Yeah, it's not easy-to-solve problem. If I correctly understand the code, to fix this problem we must implement the same logic, as pull_up_subqueries (lowest_outer_join/safe_upper_varnos).

Re: [POC] Allow flattening of subquery with a link to upper query

2022-09-13 Thread Andrey Lepikhov
On 5/9/2022 12:22, Richard Guo wrote: On Fri, Sep 2, 2022 at 7:09 PM Andrey Lepikhov mailto:a.lepik...@postgrespro.ru>> wrote: > Hmm, I'm not sure this patch works correctly in all cases. It seems to > me this patch pulls up the subquery without checkin

Re: Removing unneeded self joins

2023-05-25 Thread Andrey Lepikhov
trying to use it in both 'Useless outer join' and 'Self join' elimination optimizations. Now, because of the 'ojrelid' field it looks too complicated. Do we need to split this routine again? -- Regards Andrey Lepikhov Postgres Professional From cb4340577dab0e8cf553

MergeJoin beats HashJoin in the case of multiple hash clauses

2023-06-15 Thread Andrey Lepikhov
lause components. But as for me, here we should go the same way, as estimation of groups. The attached patch shows a sketch of the solution. -- regards, Andrey Lepikhov Postgres Professional q2.sql Description: application/sql diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/opti

Re: eqjoinsel_semi still sucks ...

2023-06-23 Thread Andrey Lepikhov
orrectly and is limited. I've tried to understand the logic through commits 0d3b231eebf, 97930cf578e and 7f3eba30c9d. But it is still not clear. So, why the idea of clamping ndistinct is terrible in general? Could you explain your reasons a bit more? -- regards, Andrey Lepikhov Postgres Professional

Re: Problems with estimating OR conditions, IS NULL on LEFT JOINs

2023-06-26 Thread Andrey Lepikhov
'); ANALYZE l,r; EXPLAIN ANALYZE SELECT * FROM l LEFT OUTER JOIN r ON (r.id = l.id) WHERE r.v IS NULL; Here you can see the same kind of underestimation: Hash Left Join (... rows=500 width=14) (... rows=9 ...) So the eqjoinsel_unmatch_left() function should be modified for the case where nd1

Re: Removing unneeded self joins

2023-06-30 Thread Andrey Lepikhov
ade. -- regards, Andrey Lepikhov Postgres Professional From 4a342b9789f5be209318c13fb7ec336fcbd2aee5 Mon Sep 17 00:00:00 2001 From: Andrey Lepikhov Date: Mon, 15 May 2023 09:04:51 +0500 Subject: [PATCH] Remove self-joins. A Self Join Elimination (SJE) feature removes inner join of plain table t

Re: Removing unneeded self joins

2023-07-05 Thread Andrey Lepikhov
didn't replace SJs, defined by baserestrictinfos if no one self-join clause have existed for the join. Now, it is fixed, and the test has been added. To understand changes readily, see the delta file in the attachment. -- regards, Andrey Lepikhov Postgres Professional

Re: POC, WIP: OR-clause support for indexes

2023-07-06 Thread Andrey Lepikhov
, You can only merge here expressions like "F(X)=Const", not an 'F(X)=ConstExpression'. See delta.diff with mentioned changes in attachment. -- regards, Andrey Lepikhov Postgres Professional diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c index c9

Re: POC, WIP: OR-clause support for indexes

2023-07-06 Thread Andrey Lepikhov
clause. Constant side of the expression is detected by call of eval_const_expressions() and check each side on the Const type of node. See 'diff to diff' in attachment. -- regards, Andrey Lepikhov Postgres Professional diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/pa

Re: Asynchronous Append on postgres_fdw nodes.

2020-06-03 Thread Andrey Lepikhov
e not looked at the patch in any detail yet, so I'm not sure that that is the right status for the patch, though. I'd like to work on this for PG14 if I have time. Hi, This patch no longer applies cleanly. In addition, code comments contain spelling errors. -- Andrey Lepikhov Postgr

Re: Asynchronous Append on postgres_fdw nodes.

2020-06-09 Thread Andrey Lepikhov
n Scan on part_3 parts_4 (cost=100.00..177.80 rows=2260 width=8) -- Andrey Lepikhov Postgres Professional https://postgrespro.com The Russian Postgres Company diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c index b04b6a0e54..4406a9c3b3 100644 --- a/contrib/pos

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2020-06-21 Thread Andrey Lepikhov
19.06.2020 19:58, Etsuro Fujita пишет: On Tue, Jun 2, 2020 at 2:51 PM Andrey Lepikhov wrote: Hiding the COPY code under the buffers management machinery allows us to generalize buffers machinery, execute one COPY operation on each buffer and simplify error handling. I'm not sure that

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2020-07-12 Thread Andrey Lepikhov
at foreign servers with different types of foreign wrapper. Not all wrappers can support CopyIn API. Also I ran the Tomas Vondra benchmark. At my laptop we have results: * regular: 5000 ms. * Tomas buffering patch: 11000 ms. * This CopyIn patch: 8000 ms. -- regards, Andrey Lepikhov Postgres P

Re: Partitioning and postgres_fdw optimisations for multi-tenancy

2020-07-16 Thread Andrey Lepikhov
s only rough outline of a possible solution... [1] https://www.postgresql.org/message-id/CAN_9JTzo_2F5dKLqXVtDX5V6dwqB0Xk%2BihstpKEt3a1LT6X78A%40mail.gmail.com [2] https://www.postgresql.org/message-id/502.1586032...@sss.pgh.pa.us -- regards, Andrey Lepikhov Postgres Professional

Re: Global snapshots

2020-05-12 Thread Andrey Lepikhov
Rebased onto current master (fb544735f1). -- Andrey Lepikhov Postgres Professional https://postgrespro.com The Russian Postgres Company >From 29183c42a8ae31b830ab5af0dfcfdaadd6229700 Mon Sep 17 00:00:00 2001 From: "Andrey V. Lepikhov" Date: Tue, 12 May 2020 08:29:54 +0500 Subjec

[POC] Fast COPY FROM command for the table with foreign partitions

2020-06-01 Thread Andrey Lepikhov
heoretical minimum here (with infinite buffer size) is 40 seconds. A couple of questions: 1. Can this feature be interesting for the PostgreSQL core or not? 2. If this is a useful feature, is the correct way chosen? -- Andrey Lepikhov Postgres Professional https://postgrespro.com The

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2020-06-01 Thread Andrey Lepikhov
ne of main questions here is to use COPY TO machinery for serializing a tuple. It is needed (if you will take a look into the patch) to transform the CopyTo() routine to an iterative representation: start/next/finish. May it be acceptable? In the attachment there is a patch with the correction of

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2019-02-04 Thread Andrey Lepikhov
On 04.02.2019 10:04, Michael Paquier wrote: > On Tue, Dec 18, 2018 at 10:41:48AM +0500, Andrey Lepikhov wrote: >> Ok. It is used only for demonstration. > > The latest patch set needs a rebase, so moved to next CF, waiting on > author as this got no reviews. The new ver

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2019-02-06 Thread Andrey Lepikhov
I used approach No.3. On 04.02.2019 10:04, Michael Paquier wrote: > On Tue, Dec 18, 2018 at 10:41:48AM +0500, Andrey Lepikhov wrote: >> Ok. It is used only for demonstration. > > The latest patch set needs a rebase, so moved to next CF, waiting on > author as this got no reviews.

Re: [PATCH] xlogreader: do not read a file block twice

2019-02-12 Thread Andrey Lepikhov
ponding line in commit 7fcbf6a405f but have another semantics: the targetPageOff value can't be more or equal XLOG_BLCKSZ, but the reqLen value can be. It may be a reason of appearance of possible mistake, introduced by commit 7fcbf6a405f. -- Andrey Lepikhov Postgres Professional https://postgrespro.com The Russian Postgres Company

Re: WAL insert delay settings

2019-02-13 Thread Andrey Lepikhov
ity limit parameter (as Tom proposed) and may some another. -- Andrey Lepikhov Postgres Professional https://postgrespro.com The Russian Postgres Company

Re: Fixing findDependentObjects()'s dependency on scan order (regressions in DROP diagnostic messages)

2018-12-05 Thread Andrey Lepikhov
of test tools, not DBMS. And here we can use 'verbose terse'. 2. Print all dependencies in findDependentObjects() on a drop error (see attachment as a prototype). -- Andrey Lepikhov Postgres Professional https://postgrespro.com The Russian Postgres Company >From e6056363889a00699

Re: Fixing findDependentObjects()'s dependency on scan order (regressions in DROP diagnostic messages)

2018-12-06 Thread Andrey Lepikhov
On 06.12.2018 11:52, Peter Geoghegan wrote: On Wed, Dec 5, 2018 at 10:35 PM Andrey Lepikhov wrote: This solution changes pg_depend relation for solve a problem, which exists only in regression tests. Very rarely it can be in the partitioning cases. Or is it not? I don't think i

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2018-12-17 Thread Andrey Lepikhov
patched version and master, but probably in a more noisy way. Ok. It is used only for demonstration. -- Andrey Lepikhov Postgres Professional https://postgrespro.com The Russian Postgres Company

Re: Fixing findDependentObjects()'s dependency on scan order (regressions in DROP diagnostic messages)

2018-12-17 Thread Andrey Lepikhov
ch needs opinion of an another reviewer. [1] https://www.postgresql.org/message-id/425db134-8bba-005c-b59d-56e50de3b41e%40postgrespro.ru [2] https://www.postgresql.org/message-id/f49bb262-d246-829d-f835-3950ddac503c%40postgrespro.ru -- Andrey Lepikhov Postgres Professional https://postgrespro.com The Russian Postgres Company

Re: Removing unneeded self joins

2019-06-26 Thread Andrey Lepikhov
FUNCTIONS, TABLEFUNCS and VALUES plan nodes uses direct link to the rte table. We need to change varno references to relid which will be kept. Version v.17 of the patch that fix the bug see in attachment. -- Andrey Lepikhov Postgres Professional https://postgrespro.com The Russian Postgres Com

Re: [HACKERS] Partition-wise aggregation/grouping

2019-07-04 Thread Andrey Lepikhov
ugh I > realize that has unresolved issues. I discouraged by this logic. Now I use set_rel_pathlist_hook and make some optimizations at partition scanning paths. But apply_scanjoin_target_to_paths() deletes pathlist and violates all optimizations. May be it is possible to introduce some fl

Insecure initialization of required_relids field

2019-07-14 Thread Andrey Lepikhov
, analyzejoins.c, line 434,435: rinfo->required_relids = bms_copy(rinfo->required_relids); rinfo->required_relids = bms_del_member(rinfo->required_relids, relid); [2] https://commitfest.postgresql.org/23/1712/ -- Andrey Lepikhov Postgres Professional https://postgrespro.com The Russian Postgres C

Re: Insecure initialization of required_relids field

2019-07-15 Thread Andrey Lepikhov
On 15/07/2019 18:48, Tom Lane wrote: Andrey Lepikhov writes: commit a31ad27fc5d introduced required_relids field. By default, it links to the clause_relids. It works good while we do not modify clause_relids or required_relids. But in the case of modification such initialization demands us

Re: [WIP] [B-Tree] Retail IndexTuple deletion

2018-07-22 Thread Andrey Lepikhov
VACUUM. Though, of course, it's still very complicated. -- --- Andrey Lepikhov Postgres Professional https://postgrespro.com The Russian Postgres Company >From 022338178180b9968e365ed4157d35be7fb4c664 Mon Sep 17 00:00:00 2001 From: "Andrey V. Lepikhov" Date: Mon, 23 Jul 2018 09:35:

Re: [WIP] [B-Tree] Retail IndexTuple deletion

2018-07-22 Thread Andrey Lepikhov
in efforts is involved to tuning of the worker. --- Andrey Lepikhov Postgres Professional https://postgrespro.com The Russian Postgres Company

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2018-07-30 Thread Andrey Lepikhov
s WAL size: 1.25 GB patched Latency average: 18175.064 ms WAL size: 0.63 GB Test: pgbench -f spgist-WAL-test.sql -t 5: -- master: Latency average: 11529.384 ms WAL size: 1.07 GB patched Latency average: 9330.828 ms WAL size: 0.6 GB -- Andrey Lepikhov Pos

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2018-08-01 Thread Andrey Lepikhov
arking workloads like this one. [1] https://postgr.es/m/cah2-wzmf0fvvhu+sszpgw4qe9t--j_dmxdx3it5jcdb8ff2...@mail.gmail.com -- Andrey Lepikhov Postgres Professional https://postgrespro.com The Russian Postgres Company collate_bug.tar.gz Description: application/gzip updatable_views.tar.gz Description: application/gzip

Re: [WIP] [B-Tree] Retail IndexTuple deletion

2018-08-07 Thread Andrey Lepikhov
ot be removed yet, oldest xmin: 6339174 There were 75478 unused item pointers. Skipped 0 pages due to buffer pins, 0 frozen pages. 0 pages are entirely empty. CPU: user: 12.27 s, system: 4.03 s, elapsed: 31.43 s. -- Andrey Lepikhov Postgres Professional https://postgrespro.com The Russian Postgr

Re: [WIP] [B-Tree] Retail IndexTuple deletion

2018-08-08 Thread Andrey Lepikhov
09.08.2018 05:19, Peter Geoghegan пишет: On Tue, Aug 7, 2018 at 12:19 AM, Andrey Lepikhov wrote: I wrote a background worker (hcleaner) to demonstrate application of Retail IndexTuple deletion (see patch at attachment). Like Autovacuum it utilizes concept of one launcher and many workers

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2020-09-20 Thread Andrey Lepikhov
hanges. Patches 0003 and 0004 are experimental and i will not support them before discussing on applicability. [1] https://www.postgresql.org/message-id/CAExHW5uAtyAVL-iuu1Hsd0fycqS5UHoHCLfauYDLQwRucwC9Og%40mail.gmail.com -- regards, Andrey Lepikhov Postgres Professional >From 05e0e9cf9de7a3893dd

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2020-09-21 Thread Andrey Lepikhov
COPY rem2, line 1 The problem here is that we run into an error after the COPY FROM command completes. And we need to translate lineno from foreign server to lineno of overall COPY command. -- regards, Andrey Lepikhov Postgres Professional

<    1   2   3   >