Re: Removing unneeded self joins

2022-03-04 Thread Andrey Lepikhov
works (See demo in attachment). Also, in new version of the patch I fixed one stupid bug: checking a self-join candidate expression operator - we can remove only expressions like F(arg1) = G(arg2). -- regards, Andrey Lepikhov Postgres ProfessionalFrom 70398361a0a0d9c6c3c7ddd1fd305ac11138e7b1 Mon

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2020-12-29 Thread Andrey Lepikhov
opy or +* after an error. +*/ +/* + * + * postgresExecForeignCopy +/* + * + * postgresBeginForeignCopy Thanks, fixed. The patch in attachment rebased on 107a2d4204. -- regards, Andrey Lepikhov Postgres Professional From 1c5439d802b7654ee50dc4326b9bc24fc7f44677 Mon Sep 17 00:00:00 2001

Increase value of OUTER_VAR

2021-03-03 Thread Andrey Lepikhov
as relevant to the size of the int type at that time. Maybe we will change these values to INT_MAX? (See the patch in attachment). -- regards, Andrey Lepikhov Postgres Professional From 98da77cdefd53dbf4ce0c740d1a0f356da970648 Mon Sep 17 00:00:00 2001 From: "Andrey V. Lepikhov" Date: We

Re: Increase value of OUTER_VAR

2021-03-03 Thread Andrey Lepikhov
partitions. -- regards, Andrey Lepikhov Postgres Professional From a3c1ee9d2e197dee40aed81cb6a08695a8fa2917 Mon Sep 17 00:00:00 2001 From: "Andrey V. Lepikhov" Date: Wed, 3 Mar 2021 11:22:32 +0300 Subject: [PATCH] Increase values of special varnos to 1 million. Use this value as a realistic

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2020-11-23 Thread Andrey Lepikhov
t done any serious review yet. Could I or my colleague continue this patch in a few days? It looks it's stalled over one month. I don't found any problems with this patch that needed to be corrected. It is wait for actions from committers side, i think. -- regards, Andrey Lepikhov Postgres Professional

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2020-11-23 Thread Andrey Lepikhov
On 11/24/20 9:27 AM, tsunakawa.ta...@fujitsu.com wrote: Andrey-san, Fujita-san, From: Etsuro Fujita On Mon, Nov 23, 2020 at 5:39 PM Andrey Lepikhov wrote: On 11/23/20 7:49 AM, tsunakawa.ta...@fujitsu.com wrote: Could I or my colleague continue this patch in a few days? It looks it&#

Re: Removing unneeded self joins

2020-11-28 Thread Andrey Lepikhov
d to removing unnecessary relation and replacing of one oid with another. We are trying to use remove_rel_from_query() machinery. Perhaps this will allow us to make the code shorter. -- regards, Andrey Lepikhov Postgres Professional

Cost overestimation of foreign JOIN

2020-11-30 Thread Andrey Lepikhov
ce of the JOIN push-down strategy. -- regards, Andrey Lepikhov Postgres Professional diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c index b6c72e1d1e..3047300c4b 100644 --- a/contrib/postgres_fdw/postgres_fdw.c +++ b/contrib/postgres_fdw/postgres_fdw.c @

Re: Cost overestimation of foreign JOIN

2020-11-30 Thread Andrey Lepikhov
On 30.11.2020 22:38, Tom Lane wrote: Andrey Lepikhov writes: Maybe it is needed to swap lines 2908 and 2909 (see attachment)? No; as explained in the comment immediately above here, we're assuming that the join conditions will be applied on the cross product of the input relations.

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2021-02-02 Thread Andrey Lepikhov
e not going to do them for the moment, can we rebase and/or further modify them so that they can be committed in PG 14? Of course, you can rebase it. -- regards, Andrey Lepikhov Postgres Professional

Re: Clarifying/rationalizing Vars' varno/varattno/varnoold/varoattno

2021-12-22 Thread Andrey Lepikhov
ctual now? -- regards, Andrey Lepikhov Postgres Professional

Re: Clarifying/rationalizing Vars' varno/varattno/varnoold/varoattno

2021-12-22 Thread Andrey Lepikhov
On 22/12/2021 20:42, Tom Lane wrote: Andrey Lepikhov writes: On 5/2/2020 01:24, Tom Lane wrote: I've not written any actual code, but am close to being ready to. This thread gives us hope to get started on solving some of the basic planner problems. But there is no activity for a long

Re: Add index scan progress to pg_stat_progress_vacuum

2021-12-23 Thread Andrey Lepikhov
for send/receive/representation implementation of progress? So AM would define a set of parameters to send into stat collector and show to users. -- regards, Andrey Lepikhov Postgres Professional

Multiple Query IDs for a rewritten parse tree

2022-01-08 Thread Andrey Lepikhov
le and simple for implementation. Any thoughts, comments, criticism ? -- regards, Andrey Lepikhov Postgres Professional

Re: Multiple Query IDs for a rewritten parse tree

2022-01-10 Thread Andrey Lepikhov
would be better for performance to collect pointers to all constant nodes during a process of hash generation. -- regards, Andrey Lepikhov Postgres Professional

Re: Multiple Query IDs for a rewritten parse tree

2022-01-10 Thread Andrey Lepikhov
On 10/1/2022 15:39, Julien Rouhaud wrote: On Mon, Jan 10, 2022 at 03:24:46PM +0500, Andrey Lepikhov wrote: On 10/1/2022 13:56, Julien Rouhaud wrote: Yes. the same input query string doesn't prove that frozen query plan can be used, because rewrite rules could be changed. So we use only a

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2021-03-22 Thread Andrey Lepikhov
t@@GLIBC_2.2.5 but in MacOS (and maybe somewhere else) we need to explicitly link libintl library in the Makefile: SHLIB_LINK += $(filter -lintl, $(LIBS) Also, we may not use gettext at all in this part of the code. -- regards, Andrey Lepikhov Postgres Professional

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2020-07-27 Thread Andrey Lepikhov
(patched Postgres binaries should be available in the PATH). It works well with master and fails with your patch applied. I used master a3ab7a707d and v5 version of the patch with your script. No errors found. Can you check your test case? -- regards, Andrey Lepikhov Postgres Professional

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2020-08-21 Thread Andrey Lepikhov
rone. Finally, I suggest renaming ri_usesBulkModify to ri_usesMultiInsert to reflect its scope. Please check the attached delta patch that applies on top of v5 to see what that would look like. I merged your delta patch (see v6 in attachment) to the main patch. Currently it seems more com

Re: Ideas about a better API for postgres_fdw remote estimates

2020-08-28 Thread Andrey Lepikhov
tovacuum operation. Its effectiveness depends on relevance of statistics on the remote server, but still. -- regards, Andrey Lepikhov Postgres Professional >From 329954981959ee3fc97e52266c89a436d02ddf5e Mon Sep 17 00:00:00 2001 From: "Andrey V. Lepikhov" Date: Tue, 4 Aug 2020 09:2

Re: Ideas about a better API for postgres_fdw remote estimates

2020-08-29 Thread Andrey Lepikhov
e go along a difficult route. Moreover, I believe this strategy should only work if we analyze a relation implicitly. If the user executes analysis explicitly by the command "ANALYZE ", we need to perform an fair analysis of the table. -- regards, Andrey Lepikhov Postgres Professional

Re: Ideas about a better API for postgres_fdw remote estimates

2020-08-31 Thread Andrey Lepikhov
g. In previous threads statistics was converted row-by-row. I want to suggest to serialize all statistics tuples for the relation into single json string. On apply phase we can filter unneeded attributes. -- regards, Andrey Lepikhov Postgres Professional

Re: Make query ID more portable

2021-10-12 Thread Andrey Lepikhov
s an example) for all oids in this code. It would allow an extension to intercept this call and replace oid with an arbitrary value. -- regards, Andrey Lepikhov Postgres Professional

Re: Make query ID more portable

2021-10-13 Thread Andrey Lepikhov
On 12/10/21 18:45, Bruce Momjian wrote: On Tue, Oct 12, 2021 at 09:40:47AM -0400, Tom Lane wrote: Andrey Lepikhov writes: But core jumbling code is good, fast and much easier in support. Also, the current code handles renames of schemas and objects, but this would not. Yes, It is good option

Re: Make query ID more portable

2021-10-13 Thread Andrey Lepikhov
On 12/10/21 18:40, Tom Lane wrote: Andrey Lepikhov writes: But core jumbling code is good, fast and much easier in support. A bigger issue is that query ID stability isn't something we are going to promise on a large scale --- for example, what if a new release adds some new fields to s

Re: Make query ID more portable

2021-10-14 Thread Andrey Lepikhov
On 14/10/21 10:40, Julien Rouhaud wrote: On Thu, Oct 14, 2021 at 12:37 PM Andrey Lepikhov wrote: On 12/10/21 18:45, Bruce Momjian wrote: On Tue, Oct 12, 2021 at 09:40:47AM -0400, Tom Lane wrote: Andrey Lepikhov writes: I think that there are just too many arbitrary decisions that could be

Re: Suggestions on message transfer among backends

2019-03-11 Thread Andrey Lepikhov
emory (see ShmemInitHash() + shmem_startup_hook) Thanks -- Andrey Lepikhov Postgres Professional https://postgrespro.com The Russian Postgres Company

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2019-03-26 Thread Andrey Lepikhov
On 25/03/2019 15:21, Heikki Linnakangas wrote: On 25/03/2019 09:57, David Steele wrote: On 2/6/19 2:08 PM, Andrey Lepikhov wrote: The patchset had a problem with all-zero pages, has appeared at index build stage: the generic_log_relation() routine sends all pages into the WAL. So  lsn field

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2019-04-01 Thread Andrey Lepikhov
On 26/03/2019 15:59, Heikki Linnakangas wrote: On 26/03/2019 11:29, Andrey Lepikhov wrote: On 25/03/2019 15:21, Heikki Linnakangas wrote: Hmm. When do we create all-zero pages during index build? That seems pretty surprising. GIST uses buffered pages. During GIST build it is possible (very

Re: Removing unneeded self joins

2019-08-05 Thread Andrey Lepikhov
On 02/08/2019 04:54, Thomas Munro wrote: On Thu, Jun 27, 2019 at 6:42 PM Andrey Lepikhov wrote: Version v.17 of the patch that fix the bug see in attachment. While moving this to the September CF, I noticed that it needs to be updated for the recent pg_list.h API changes. The patch was

Re: Removing unneeded self joins

2019-11-05 Thread Andrey Lepikhov
v.21 in attechment fix small bug: Now we move all non-mergejoinable clauses from joininfo to base restrict info because of the relation will not be joined. On 30/09/2019 13:29, Konstantin Knizhnik wrote: Slightly refactored version of the patch with more comments. -- Andrey Lepikhov

Re: pg_waldump and PREPARE

2019-11-07 Thread Andrey Lepikhov
rds, One issue is that your patch provides small information. WAL errors Investigation often requires information on xid, subxacts, delete-on-abort/commit rels; rarely - invalidation messages etc. -- Andrey Lepikhov Postgres Professional https://postgrespro.com The Russian Postgres Company

Re: pg_waldump and PREPARE

2019-11-07 Thread Andrey Lepikhov
On 08/11/2019 09:26, Kyotaro Horiguchi wrote: Hello. At Fri, 8 Nov 2019 08:23:41 +0500, Andrey Lepikhov wrote in Can I switch the status back to "Needs review"? Regards, One issue is that your patch provides small information. WAL errors Investigation often requires informat

Re: pg_waldump and PREPARE

2019-11-12 Thread Andrey Lepikhov
12.11.2019 12:41, Fujii Masao пишет: Ok, I changed the patch that way. Attached is the latest version of the patch. Regards, I did not see any problems in this version of the patch. The information displayed by pg_waldump for the PREPARE record is sufficient for use. -- Andrey Lepikhov

Optimization of NestLoop join in the case of guaranteed empty inner subtree

2019-12-11 Thread Andrey Lepikhov
x27; As you can see, executor scan each of 1e5 outer tuples despite the fact that inner can't return any tuples. Teodor Sigaev and I developed a patch to solve this problem. Result of explain analyze procedure can be found in the 'optimized_execution.txt'. -- Andrey Lepikhov Postgres P

Re: Optimization of NestLoop join in the case of guaranteed empty inner subtree

2019-12-15 Thread Andrey Lepikhov
On 12/11/19 8:49 PM, Tom Lane wrote: Andrey Lepikhov writes: During NestLoop execution we have bad corner case: if outer subtree contains tuples the join node will scan inner subtree even if it does not return any tuples. So the first question about corner-case optimizations like this is

Re: NOT IN subquery optimization

2020-01-04 Thread Andrey Lepikhov
omments of the patch) the subquery hashing has the same execution time with queries No.13-17. At the queries No.1-12 it is not so slow as without hashing, but works more slowly (up to 3 orders) than NOT IN optimization. On 12/2/19 9:25 PM, Li, Zheng wrote: Here is the latest rebased patch.

Re: NOT IN subquery optimization

2020-01-08 Thread Andrey Lepikhov
NOT EXISTS. The convert_EXISTS_sublink_to_join() routine can contain vars of the parent query. May be you give an trivial example for this problem? -- Andrey Lepikhov Postgres Professional https://postgrespro.com The Russian Postgres Company

Re: Removing unneeded self joins

2020-01-27 Thread Andrey Lepikhov
Rebased version v.22. - Added enable_self_join_removal GUC (true is default) - The joinquals of the relation that is being removed, redistributed in accordance with the remove_rel_from_query () machinery. -- Andrey Lepikhov Postgres Professional https://postgrespro.com The Russian Postgres

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2019-04-01 Thread Andrey Lepikhov
can see the influence of the patch on WAL growth. Results === AM | master | patch | GIN | 347 MB | 66 MB | GiST| 157 MB | 43 MB | SP-GiST | 119 MB | 38 MB | -- Andrey Lepikhov Postgres Professional https://postgrespro.com The Russian Postgres Company >From 2edd0ada13b4749487d0f

Re: Failure in contrib test _int on loach

2019-04-10 Thread Andrey Lepikhov
bug (see attachment). It utilizes the idea of linear increment of LSN/NSN. WAL write process is used for change NSN value to 1 for each block of index relation. I hope this can be a fairly clear and safe solution. -- Andrey Lepikhov Postgres Professional https://postgrespro.com

Re: Failure in contrib test _int on loach

2019-04-11 Thread Andrey Lepikhov
On 11/04/2019 13:14, Heikki Linnakangas wrote: On 11/04/2019 09:10, Andrey Lepikhov wrote: On 10/04/2019 20:25, Heikki Linnakangas wrote: On 09/04/2019 19:11, Anastasia Lubennikova wrote: After introducing GistBuildNSN this code path became unreachable. To fix it, I added new flag to

Do CustomScan as not projection capable node

2019-04-18 Thread Andrey Lepikhov
custom_scan_tlist) we will get errors in the set_customscan_references() call. -- Andrey Lepikhov Postgres Professional https://postgrespro.com The Russian Postgres Company >From 938904d179e0a4e31cbb20fb70243d2b980d8dc2 Mon Sep 17 00:00:00 2001 From: "Andrey V. Lepikhov" Date: Fri, 19 Apr

Re: Do CustomScan as not projection capable node

2019-04-22 Thread Andrey Lepikhov
into two patches: 1a8a4e5 and e7cb7ee. It is possible that custom_scan_tlist is designed too nontrivially, and it is possible that it needs some comments describing in more detail how to use it. -- Andrey Lepikhov Postgres Professional https://postgrespro.com The Russian Postgres Company

Re: [PATCH] XLogReadRecord returns pointer to currently read page

2018-10-22 Thread Andrey Lepikhov
On 22.10.2018 2:06, Heikki Linnakangas wrote: On 17/08/2018 06:47, Andrey Lepikhov wrote: I propose the patch for fix one small code defect. The XLogReadRecord() function reads the pages of a WAL segment that contain a WAL-record. Then it creates a readRecordBuf buffer in private memory of a

Re: [PATCH] XLogReadRecord returns pointer to currently read page

2018-10-22 Thread Andrey Lepikhov
packing or compression can add the memcpy() where it needs it. I agree with it. Eventually, placement of the WAL-record can be defined by comparison the record, readBuf and readRecordBuf pointers. In attachment new version of the patch. -- Andrey Lepikhov Postgres Professional https://postgrespro.com T

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2018-10-23 Thread Andrey Lepikhov
g the WAL-record writing process. Maybe you will do a test without writing any data to disk? -- Andrey Lepikhov Postgres Professional https://postgrespro.com The Russian Postgres Company

Re: [PATCH] XLogReadRecord returns pointer to currently read page

2018-10-25 Thread Andrey Lepikhov
On 26.10.2018 10:33, Kyotaro HORIGUCHI wrote: Hello. At Tue, 23 Oct 2018 10:25:27 +0500, Andrey Lepikhov wrote in <2553f2b0-0e39-eb0e-d382-6c0ed08ca...@postgrespro.ru> On 23.10.2018 0:53, Heikki Linnakangas wrote: I'd expect the decompression to read from the on-disk buffer,

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2018-11-02 Thread Andrey Lepikhov
rown twice. May be you move this to some service routine? -- Andrey Lepikhov Postgres Professional https://postgrespro.com The Russian Postgres Company

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2018-11-03 Thread Andrey Lepikhov
. You can verify it by tracking the object_address_present_add_flags() routine return value. Some doubts, however, may be regarding the 'triggers' test. May you specify the test failures do you mean? [1] https://www.postgresql.org/message-id/20180504022601.fflymidf7eoencb2%40alvherre.pgs

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2018-11-04 Thread Andrey Lepikhov
On 04.11.2018 9:31, Peter Geoghegan wrote: On Sat, Nov 3, 2018 at 8:52 PM Andrey Lepikhov wrote: I applied your patches at top of master. After tests corrections (related to TID ordering in index relations DROP...CASCADE operation) 'make check-world' passed successfully many tim

Re: Fixing findDependentObjects()'s dependency on scan order (regressions in DROP diagnostic messages)

2018-11-05 Thread Andrey Lepikhov
er objects depend on it -DETAIL: extension earthdistance depends on extension cube +DETAIL: extension earthdistance depends on function cube_out(cube) Can anyone think of a workable, scalable approach to fixing the processing order of this findDependentObjects() pg_depend scan so that we reliably g

Re: [PATCH] XLogReadRecord returns pointer to currently read page

2018-11-18 Thread Andrey Lepikhov
uf for the record contents? Let's not forget that the contents of XLogReaderState are public. According to my experience, I clarify some comments to avoid this mistakes in the future (see attachment). -- Andrey Lepikhov Postgres Professional https://postgrespro.com The Russian Postgre

Re: [PATCH] XLogReadRecord returns pointer to currently read page

2018-11-19 Thread Andrey Lepikhov
On 20.11.2018 6:30, Michael Paquier wrote: On Mon, Nov 19, 2018 at 10:48:06AM +0500, Andrey Lepikhov wrote: According to my experience, I clarify some comments to avoid this mistakes in the future (see attachment). No objections from here. - * The returned pointer (or *errormsg) points

Re: Removing unneeded self joins

2022-12-15 Thread Andrey Lepikhov
; Nested Loop -> HashAggregate This change in the test behaviour is induced by the a5fc4641 "Avoid making commutatively-duplicate clauses in EquivalenceClasses." Nothing special, as I see. Attached patch fixes this. -- Regards Andrey Lepikhov Postgres Professional From 3e546637561bf

Optimization issue of branching UNION ALL

2022-12-20 Thread Andrey Lepikhov
#x27;t it? In attachment you can see some sketch that reduces a number of planner cycles/copyings. -- Regards Andrey Lepikhov Postgres Professional t.sh Description: application/shellscript diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c in

Re: Optimization issue of branching UNION ALL

2022-12-22 Thread Andrey Lepikhov
just commit it. Thanks, I have written the letter because of some doubts too. But only one weak point I could imagine - if someday sql standard will be changed. Your code looks better, than previous attempt. -- regards, Andrey Lepikhov Postgres Professional

Re: POC, WIP: OR-clause support for indexes

2022-12-27 Thread Andrey Lepikhov
paths. So, we can avoid planning overhead and non-optimal BitmapOr in the case of many OR's possibly aggravated by many indexes on the relation. For example, such operation can be executed in create_index_paths() before passing rel->indexlist. -- Regards Andrey Lepikhov Postgres Professional demo.sql Description: application/sql

Re: [PATCH] random_normal function

2023-01-18 Thread Andrey Lepikhov
have the "isolation tests" option to create stable execution time-dependent tests now? Or I'm not aware about some test machinery? -- Regards Andrey Lepikhov Postgres Professional

Re: [PATCH] random_normal function

2023-01-18 Thread Andrey Lepikhov
On 1/19/23 11:01, Tom Lane wrote: Andrey Lepikhov writes: On 1/9/23 23:52, Tom Lane wrote: BTW, if this does bring the probability of failure down to the one-in-a-billion range, I think we could also nuke the whole "ignore:" business, simplifying pg_regress and allowing the random

[POC] Allow an extension to add data into Query and PlannedStmt nodes

2023-03-29 Thread Andrey Lepikhov
planner and executor hooks for some purposes. So, any thoughts will be useful. -- Regards Andrey Lepikhov Postgres ProfessionalFrom 944ce61d7ff934727240d90ee7620bfb69ad3a5a Mon Sep 17 00:00:00 2001 From: Andrey Lepikhov Date: Wed, 22 Mar 2023 16:59:30 +0500 Subject: [PATCH] Add on more

Re: [POC] Allow an extension to add data into Query and PlannedStmt nodes

2023-03-30 Thread Andrey Lepikhov
n idea here - if an extension holds custom struct and want to pass it along all planning and execution stages it should use extensible node with custom read/write/copy routines. -- regards, Andrey Lepikhov Postgres Professional From ab101322330684e9839e46c26f70ad5462e40dac Mon Sep 17 00:00:00 2001 From:

Re: Removing unneeded self joins

2023-09-12 Thread Andrey Lepikhov
On 5/7/2023 21:28, Andrey Lepikhov wrote: Hi, During the significant code revision in v.41 I lost some replacement operations. Here is the fix and extra tests to check this in the future. Also, Tom added the JoinDomain structure five months ago, and I added code to replace relids for that

Re: POC: GROUP BY optimization

2023-09-12 Thread Andrey Lepikhov
in the query plan in attachment. -- regards, Andrey Lepikhov Postgres Professional From 33953655c9ac3f9ec64b80c9f2a2ff38bd178745 Mon Sep 17 00:00:00 2001 From: "Andrey V. Lepikhov" Date: Wed, 13 Sep 2023 11:20:03 +0700 Subject: [PATCH] Explore alternative orderings of group-by pathk

Re: POC: GROUP BY optimization

2023-09-18 Thread Andrey Lepikhov
On 20/7/2023 18:46, Tomas Vondra wrote: On 7/20/23 08:37, Andrey Lepikhov wrote: On 3/10/2022 21:56, Tom Lane wrote: Revert "Optimize order of GROUP BY keys". This reverts commit db0d67db2401eb6238ccc04c6407a4fd4f985832 and several follow-on fixes. ... Since we're hard up agai

Re: [PoC] Reducing planning time when tables have many partitions

2023-09-19 Thread Andrey Lepikhov
g lists of derives or ec_members, we should go through all the index lists and fix them, which is a non-trivial operation. [1] https://www.postgresql.org/message-id/flat/64486b0b-0404-e39e-322d-0801154901f3%40postgrespro.ru -- regards, Andrey Lepikhov Postgres Professional

Re: Oversight in reparameterize_path_by_child leading to executor crash

2023-09-20 Thread Andrey Lepikhov
lids adjustment. So you can build a more general view of the problem with this patch. [1] Asymmetric partition-wise JOIN https://www.postgresql.org/message-id/flat/CAOP8fzaVL_2SCJayLL9kj5pCA46PJOXXjuei6-3aFUV45j4LJQ%40mail.gmail.com -- regards, Andrey Lepikhov Postgres Professional

Re: disfavoring unparameterized nested loops

2023-09-20 Thread Andrey Lepikhov
ut contains too small tuples. It solves the issue, Isn't it? [1] https://techcommunity.microsoft.com/t5/sql-server-blog/introducing-batch-mode-adaptive-joins/ba-p/385411 -- regards, Andrey Lepikhov Postgres Professional

Re: [PATCH] Add extra statistics to explain for Nested Loop

2023-09-22 Thread Andrey Lepikhov
of parameters? And at the end. If someone wants a lot of additional statistics, why not give them that by extension? It is only needed to add a hook into the point of the node explanation and some efforts to make instrumentation extensible. But here, honestly, I don't have code/ideas so fa

Re: Postgres picks suboptimal index after building of an extended statistics

2023-09-24 Thread Andrey Lepikhov
ndexes only. Maybe to change compare_path_costs_fuzzily() and add some heuristic, for example: "If selectivity of both paths gives us no more than 1 row, prefer to use a unique index or an index with least selectivity." -- regards, Andrey Lepikhov Postgres Professional

Re: POC: GUC option for skipping shared buffers in core dumps

2023-09-25 Thread Andrey Lepikhov
gards, Andrey Lepikhov Postgres Professional diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c index 5810f8825e..4d7bf2c0e4 100644 --- a/src/backend/bootstrap/bootstrap.c +++ b/src/backend/bootstrap/bootstrap.c @@ -325,7 +325,7 @@ BootstrapModeMain(int argc, char

Re: RFC: Logging plan of the running query

2023-09-25 Thread Andrey Lepikhov
x27;t find a problem either. I just feel uncomfortable if, at the moment of interruption, we have a descriptor of another query than the query have been executing and holding resources. -- regards, Andrey Lepikhov Postgres Professional

Re: POC: GROUP BY optimization

2023-09-25 Thread Andrey Lepikhov
ve extended statistics on distinct values and these statistics cover some set of first columns in the grouping list, we can optimize these positions. It also looks reliable. Any thoughts? -- regards, Andrey Lepikhov Postgres Professional

Re: RFC: Logging plan of the running query

2023-09-27 Thread Andrey Lepikhov
On 28/9/2023 09:04, torikoshia wrote: On 2023-09-25 18:49, Andrey Lepikhov wrote: On 25/9/2023 14:21, torikoshia wrote: On 2023-09-20 14:39, Lepikhov Andrei wrote: Hmm, as a test, I made sure to call ProcessLogQueryPlanInterrupt() on all CFI using v28-0002-Testing-attempt-logging-plan-on

Re: NOT IN subquery optimization

2020-04-01 Thread Andrey Lepikhov
s NOT NULL)..." may be change to "Return pullout predicate (x is NOT NULL or NOT EXISTS...)"? 4. is_node_nonnullable(): I think one more case of non-nullable var may be foreign key constraint. -- Andrey Lepikhov Postgres Professional https://postgrespro.com The Russian Postgres Company

Re: Removing unneeded self joins

2020-04-02 Thread Andrey Lepikhov
potentially self joined relations may belong to different rules of order restriction in join_info_list. 3. Add test for item 2. The CF entry has been updated to Needs review. -- Andrey Lepikhov Postgres Professional https://postgrespro.com The Russian Postgres Company >F

Re: POC: GROUP BY optimization

2022-01-21 Thread Andrey Lepikhov
growth of this value? - see in attachment patch to previous fixes. -- regards, Andrey Lepikhov Postgres Professionaldiff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c index 70af9c91d5..4e26cd275d 100644 --- a/src/backend/optimizer/path/costsize.c +++

Re: POC: GROUP BY optimization

2022-01-23 Thread Andrey Lepikhov
rt with the original path, not the modified one we built in the last iteration. Or am I missing something You are right, I misunderstood the idea of path_save variable. -- regards, Andrey Lepikhov Postgres Professional

Re: Multiple Query IDs for a rewritten parse tree

2022-01-31 Thread Andrey Lepikhov
e shared with other extension that might be dealing with similar concerns. I think, it depends on a specific purpose of an extension. -- regards, Andrey Lepikhov Postgres Professional

Re: Merging statistics from children instead of re-sampling everything

2022-02-10 Thread Andrey Lepikhov
return a sample like you read tuples locally. Also, to get such parts of samples asynchronously, we can get size of each partition on a preliminary step of analysis. In my opinion, even this solution can reduce heaviness of a problem drastically. -- regards, Andrey Lepikhov Postgres Professional

Re: explain analyze rows=%.0f

2022-11-06 Thread Andrey Lepikhov
ientific format like X.XXEXX. I vote for second option. -- regards, Andrey Lepikhov Postgres Professional

Re: [PoC] Reducing planning time when tables have many partitions

2022-11-06 Thread Andrey Lepikhov
As I see, everywhere access to these lists guides by eclass_source_indexes and eclass_derive_indexes correspondingly. Maybe to merge them? -- regards, Andrey Lepikhov Postgres Professional

Re: [PoC] Reducing planning time when tables have many partitions

2022-11-08 Thread Andrey Lepikhov
is not a mistake, maybe to add a comment why assertion here isn't failed? -- regards, Andrey Lepikhov Postgres Professional

Re: Removing unneeded self joins

2021-05-07 Thread Andrey Lepikhov
quot; filter on a non-nullable column. To be clear, this is still an improvement (to me) without that. New version of the feature. Deeply refactored with main goal - to reduce the code size) and rebased on current master. Here I didn't work on 'unnecessary IS NOT NULL filter'. -- r

Re: Asynchronous Append on postgres_fdw nodes.

2021-05-07 Thread Andrey Lepikhov
artition - 178 ms; 4 - 263; 8 - 450; 16 - 860; 32 - 1740. Without: 1 - 178 ms; 4 - 583; 8 - 1140; 16 - 2302; 32 - 4620. So, these results show that we have a reason to use async append in the case where there's only one foreign server. -- regards, Andrey Lepikhov Postgres Professional

Re: Asynchronous Append on postgres_fdw nodes.

2021-05-07 Thread Andrey Lepikhov
: InstrUpdateTupleCount called on node not yet executed Initialization script see in attachment. -- regards, Andrey Lepikhov Postgres Professional t1.sql Description: application/sql

Re: Asynchronous Append on postgres_fdw nodes.

2021-05-07 Thread Andrey Lepikhov
proach to this would be the patch you proposed before [1]. Right? Yes. I think, new solution will be better. -- regards, Andrey Lepikhov Postgres Professional

Re: Asynchronous Append on postgres_fdw nodes.

2021-05-10 Thread Andrey Lepikhov
On 10/5/21 08:03, Etsuro Fujita wrote: On Fri, May 7, 2021 at 7:32 PM Andrey Lepikhov I think a simple fix for this would be just remove the check whether the instr->running flag is set or not in InstrUpdateTupleCount(). Attached is an updated patch, in which I also updated a comment

Defer selection of asynchronous subplans until the executor initialization stage

2021-05-10 Thread Andrey Lepikhov
7de2d5cb1a3%40postgrespro.ru -- regards, Andrey Lepikhov Postgres Professional From 395b1d62389cf40520a4afd87c11301aa2b17df2 Mon Sep 17 00:00:00 2001 From: "Andrey V. Lepikhov" Date: Tue, 11 May 2021 08:43:03 +0500 Subject: [PATCH] Defer selection of asynchronous subplans to the ex

Re: Defer selection of asynchronous subplans until the executor initialization stage

2021-05-11 Thread Andrey Lepikhov
ubplan->async_capable.Thank you, I agree with you. Close look into the postgres_fdw regression tests show at least one open problem with this approach: we need to control situations when only one partition doesn't pruned and append isn't exist at all. -- regards, Andrey Lepikhov Postgres Professional

Re: Asynchronous Append on postgres_fdw nodes.

2021-05-11 Thread Andrey Lepikhov
On 11/5/21 12:24, Etsuro Fujita wrote: On Tue, May 11, 2021 at 11:58 AM Andrey Lepikhov The extra tuple, which is from f1 or f2, would have been kept in the Append node's as_asyncresults, not returned from the Append node to the Limit node. The async Foreign Scan nodes would fetch tuples b

Re: Asymmetric partition-wise JOIN

2021-05-26 Thread Andrey Lepikhov
Next version of the patch. For searching any problems I forced this patch during 'make check' tests. Some bugs were found and fixed. -- regards, Andrey Lepikhov Postgres Professional From 101614b504b0b17e201d2375c8af61cfc671e51d Mon Sep 17 00:00:00 2001 From: Andrey Lepikhov Date:

Re: [PoC] Reducing planning time when tables have many partitions

2023-02-14 Thread Andrey Lepikhov
geTblEntry instead? -- Regards Andrey Lepikhov Postgres Professional

Re: [POC] Allow flattening of subquery with a link to upper query

2022-10-03 Thread Andrey Lepikhov
On 9/13/22 16:40, Andrey Lepikhov wrote: On 5/9/2022 12:22, Richard Guo wrote: On Fri, Sep 2, 2022 at 7:09 PM Andrey Lepikhov mailto:a.lepik...@postgrespro.ru>> wrote: To resolve both issues, lower outer join passes through pull_sublinks_* into flattening routine (see attachment). I&#x

Re: [POC] Allow flattening of subquery with a link to upper query

2022-10-05 Thread Andrey Lepikhov
better if the fields have comments. Ok, I've added some comments. +                    * (for grouping, as an example). So, revert its status to +                    * a full valued entry. full valued -> fully valued Fixed -- regards, Andrey Lepikhov Postgres Professio

Re: Removing unneeded self joins

2022-10-05 Thread Andrey Lepikhov
New version, rebased onto current master. Nothing special, just rebase. -- regards, Andrey Lepikhov Postgres Professional From 03aab7a2431032166c9ea5f52fbcccaf7168abec Mon Sep 17 00:00:00 2001 From: "Andrey V. Lepikhov" Date: Wed, 5 Oct 2022 16:58:34 +0500 Subject: [PATCH] Remove self

Re: document the need to analyze partitioned tables

2022-10-05 Thread Andrey Lepikhov
UMMARY OFF) SELECT * FROM test t1, test t2 WHERE t1.id = t2.val; VACUUM ANALYZE test; EXPLAIN (ANALYZE, TIMING OFF, SUMMARY OFF) SELECT * FROM test t1, test t2 WHERE t1.id = t2.val; Here without actual statistics on parent table we make wrong prediction. -- Regards Andrey Lepikhov Postgres Professional

Re: Fast COPY FROM based on batch insert

2022-10-10 Thread Andrey Lepikhov
o prevent possible errors in future. -- Regards Andrey Lepikhov Postgres Professional

Re: Fast COPY FROM based on batch insert

2022-10-12 Thread Andrey Lepikhov
On 10/12/22 07:56, Etsuro Fujita wrote: On Tue, Oct 11, 2022 at 3:06 PM Andrey Lepikhov wrote: I reviewed the patch one more time. Only one question: bistate and ri_FdwRoutine are strongly bounded. Maybe to add some assertion on (ri_FdwRoutine XOR bistate) ? Just to prevent possible errors in

Re: Fast COPY FROM based on batch insert

2022-10-28 Thread Andrey Lepikhov
studied performance of this code in comparison to bulk INSERTions. This patch seems to improve speed of insertion by about 20%. Also, this patch is very invasive. So, I don't have any plans to work on it now. -- regards, Andrey Lepikhov Postgres Professional

  1   2   3   >