Re: Multidimensional Histograms

2024-01-10 Thread Andrei Lepikhov
er for one combination, building the dependency statistics is still massive work. So, in the multicolumn case, having something like a histogram may be more effective. -- regards, Andrei Lepikhov Postgres Professional

Re: Custom explain options

2024-01-10 Thread Andrei Lepikhov
On 10/1/2024 20:27, Konstantin Knizhnik wrote: On 10/01/2024 8:46 am, Michael Paquier wrote: On Wed, Jan 10, 2024 at 01:29:30PM +0700, Andrei Lepikhov wrote: What do you think about this really useful feature? Do you wish to develop it further? I am biased here.  This seems like a lot of

Re: introduce dynamic shared memory registry

2024-01-10 Thread Andrei Lepikhov
t; I kept this the same, as I didn't see any need to tie the key size to > NAMEDATALEN. IMO, we should avoid magic numbers whenever possible. Current logic according to which first users of this feature will be extensions naturally bonds this size to the size of the 'name' type

Re: POC: GROUP BY optimization

2024-01-13 Thread Andrei Lepikhov
additional comments and tests to make the code more understandable (see attachment). I intended to look into this part of the code more, but the tests show a difference between PostgreSQL v.15 and v.16, which causes changes in the code of this feature. -- regards, Andrei Lepikhov Postgres Pr

Re: POC: GROUP BY optimization

2024-01-14 Thread Andrei Lepikhov
On 13/1/2024 22:00, Alexander Korotkov wrote: On Sat, Jan 13, 2024 at 11:09 AM Andrei Lepikhov wrote: On 11/1/2024 18:30, Alexander Korotkov wrote: On Tue, Jan 9, 2024 at 1:14 PM Pavel Borisov wrote: Hmm, I don't see this old code in these patches. Resend 0002-* because of trailing s

Re: POC: GROUP BY optimization

2024-01-15 Thread Andrei Lepikhov
set looks good to me.  I'm going to push it if there are no objections. Seems I'm late for the party.  Can we hold for several more days?  I'd like to have a review on this patch. Get on board! It looks like this feature needs as much review as possible (likewise SJE). --

Re: POC: GROUP BY optimization

2024-01-18 Thread Andrei Lepikhov
pk_eclass->ec_sortref' is valid before calling get_sortgroupref_clause_noerr with it. Also, how can we guarantee that the returned GROUP BY item is sortable? Should we check that with OidIsValid(sgc->sortop)? Added. Reviewed it, looks good. -- regards, Andrei Lepikhov Postgre

Re: POC: GROUP BY optimization

2024-01-18 Thread Andrei Lepikhov
Just forgotten second attachment. -- regards, Andrei Lepikhov Postgres Professional diff --git a/src/backend/optimizer/path/pathkeys.c b/src/backend/optimizer/path/pathkeys.c index 1095b73dac..b612420547 100644 --- a/src/backend/optimizer/path/pathkeys.c +++ b/src/backend/optimizer/path

Re: POC: GROUP BY optimization

2024-02-01 Thread Andrei Lepikhov
ession run. Nor do I see any particular reason for pg_class to be especially suited to the test. Yeah, It is my fault. Please, see in the attachment the patch fixing that. -- regards, Andrei Lepikhov Postgres Professional From 11a049d95ee48e38ad569aab7663d8de91f946ad Mon Sep 17 00:00:00 2001 F

Re: POC: GROUP BY optimization

2024-02-01 Thread Andrei Lepikhov
l be already sorted before the MergeJoin. Why not use Incremental Sort on (x,z,w) instead of full sort? 2. For memo, IMO, this test shows us the future near perspective of this feature: It is cheaper to use grouping order (w,z) instead of (z,w). -- regards, Andrei Lepikhov Postgres Professional

Re: POC, WIP: OR-clause support for indexes

2024-02-07 Thread Andrei Lepikhov
il.gmail.com [2] https://www.postgresql.org/message-id/CAPpHfduJtO0s9E%3DSHUTzrCD88BH0eik0UNog1_q3XBF2wLmH6g%40mail.gmail.com -- regards, Andrei Lepikhov Postgres Professional From 0ac511ab94959e87d1525d5ea8c4855643a6ccbe Mon Sep 17 00:00:00 2001 From: Alena Rybakina Date: Fri, 2 Feb 2024 22:01:0

Re: pg_stat_advisor extension

2024-02-07 Thread Andrei Lepikhov
. -- regards, Andrei Lepikhov Postgres Professional

Re: POC, WIP: OR-clause support for indexes

2024-03-14 Thread Andrei Lepikhov
On 14/3/2024 16:31, Alexander Korotkov wrote: On Wed, Mar 13, 2024 at 2:16 PM Andrei Lepikhov mailto:a.lepik...@postgrespro.ru>> wrote: > On 13/3/2024 18:05, Alexander Korotkov wrote: > > On Wed, Mar 13, 2024 at 7:52 AM Andrei Lepikhov > > Given all of the

Re: POC, WIP: OR-clause support for indexes

2024-03-14 Thread Andrei Lepikhov
ust no > index selective for the whole query. However, splitting scan by the > OR qual lets use a combination of two selective indexes. Thanks for the case. I will try to resolve it. -- regards, Andrei Lepikhov Postgres Professional From 156c00c820a38e5e1856f07363af87b3109b5d77 Mon Sep 17

Re: POC, WIP: OR-clause support for indexes

2024-03-18 Thread Andrei Lepikhov
On 14/3/2024 16:31, Alexander Korotkov wrote: On Wed, Mar 13, 2024 at 2:16 PM Andrei Lepikhov As you can see this case is not related to partial indexes.  Just no index selective for the whole query.  However, splitting scan by the OR qual lets use a combination of two selective indexes. I

Re: POC, WIP: OR-clause support for indexes

2024-03-31 Thread Andrei Lepikhov
ing each array element as an OR clause; we can also provide a BitmapOr path, where SAOP is covered with a minimal number of partial indexes (likewise, previous version). -- regards, Andrei Lepikhov Postgres Professional From e42a7111a12ef82eecdb2e692d65e7ba6e43ad79 Mon Sep 17 00:00:00 2001 From:

Re: Table AM Interface Enhancements

2024-04-01 Thread Andrei Lepikhov
index and avoid huge underestimation. This makes sense, especially for multicolumn filters/clauses. Having a probing AM method, we may invent something for this challenge. -- regards, Andrei Lepikhov Postgres Professional

Re: Asymmetric partition-wise JOIN

2024-04-01 Thread Andrei Lepikhov
the multipartition case. But as I can predict, it is easier to implement and looks more natural for the architecture. What do you think about that? -- regards, Andrei Lepikhov Postgres Professional

Re: type cache cleanup improvements

2024-04-02 Thread Andrei Lepikhov
moving the entry and reducing the hash table's size might be more efficient. In toto, the 0001-* patch looks good, and I would be glad to see it in the core. [1] https://www.postgresql.org/message-id/flat/CAKcux6ktu-8tefLWtQuuZBYFaZA83vUzuRd7c1YHC-yEWyYFpg%40mail.gmail.com -- regards,

Re: post-freeze damage control

2024-04-08 Thread Andrei Lepikhov
to show that this was not commit-ready. It's up to you. On the one hand, I don't see any bugs or strong performance issues, and all the issues can be resolved further; on the other hand, I've got your good review and some ideas to work out. -- regards, Andrei Lepikhov Postgres Professional

Re: post-freeze damage control

2024-04-09 Thread Andrei Lepikhov
On 9/4/2024 12:55, Tom Lane wrote: Andrei Lepikhov writes: * I really, really dislike jamming this logic into prepqual.c, where it has no business being. I note that it was shoved into process_duplicate_ors without even the courtesy of expanding the header comment: Yeah, I preferred to do

Re: POC, WIP: OR-clause support for indexes

2024-02-11 Thread Andrei Lepikhov
Thanks for the review! It was the first version for discussion. Of course, refactoring and polishing cycles will be needed, even so we can discuss the general idea earlier. On 10/2/2024 12:00, jian he wrote: On Thu, Feb 8, 2024 at 1:34 PM Andrei Lepikhov 1235 | PredicatesData

Re: POC, WIP: OR-clause support for indexes

2024-02-12 Thread Andrei Lepikhov
1, haven't we? -- regards, Andrei Lepikhov Postgres Professional

Re: POC, WIP: OR-clause support for indexes

2024-02-13 Thread Andrei Lepikhov
igins of such behavior, but it seems to be an issue of parallel workers, not this specific optimization. -- regards, Andrei Lepikhov Postgres Professional

Re: POC, WIP: OR-clause support for indexes

2024-02-13 Thread Andrei Lepikhov
On 12/2/2024 17:51, Alena Rybakina wrote: On 12.02.2024 12:01, Andrei Lepikhov wrote: On 12/2/2024 15:55, Alena Rybakina wrote: As I understand it, match_clauses_to_index is necessary if you have a RestrictInfo (rinfo1) variable, so maybe we should run it after the make_restrictonfo procedure

Re: POC, WIP: OR-clause support for indexes

2024-02-13 Thread Andrei Lepikhov
On 13/2/2024 17:03, Andrei Lepikhov wrote: On 13/2/2024 07:00, jian he wrote: The time is the last result of the 10 iterations. I'm not sure about the origins of such behavior, but it seems to be an issue of parallel workers, not this specific optimization. Having written that, I&#x

Re: Memory consumed by child SpecialJoinInfo in partitionwise join planning

2024-02-13 Thread Andrei Lepikhov
some extensions, such as pg_hint_plan, call build_child_join_sjinfo. It is OK to break the interface with a major version. But what if they need child_sjinfo a bit longer and collect links to this structure? I don't think it is a real stopper, but it is worth additional analysis. -- rega

Re: Propagate pathkeys from CTEs up to the outer query

2024-02-13 Thread Andrei Lepikhov
nt backburner - pull parameterisation through the GROUP-BY and the query block fence up to the JOIN searching code of the parent query. -- regards, Andrei Lepikhov Postgres Professional

Re: Memory consumed by child SpecialJoinInfo in partitionwise join planning

2024-02-13 Thread Andrei Lepikhov
On 14/2/2024 13:32, Ashutosh Bapat wrote: On Wed, Feb 14, 2024 at 9:50 AM Andrei Lepikhov wrote: On 30/1/2024 12:44, Ashutosh Bapat wrote: Thanks Vignesh. PFA patches rebased on the latest HEAD. The patch addressing Amit's comments is still a separate patch for him to review. Thank

Re: Memory consumed by paths during partitionwise join planning

2024-02-14 Thread Andrei Lepikhov
message-id/flat/CAOP8fzaVL_2SCJayLL9kj5pCA46PJOXXjuei6-3aFUV45j4LJQ%40mail.gmail.com [2] https://www.postgresql.org/message-id/flat/CAMbWs496%2BN%3DUAjOc%3DrcD3P7B6oJe4rZw08e_TZRUsWbPxZW3Tw%40mail.gmail.com -- regards, Andrei Lepikhov Postgres Professional

Re: planner chooses incremental but not the best one

2024-02-14 Thread Andrei Lepikhov
f grouping? -- regards, Andrei Lepikhov Postgres Professional

Re: planner chooses incremental but not the best one

2024-02-14 Thread Andrei Lepikhov
add a test directly checking appendrel->tuples correction. -- regards, Andrei Lepikhov Postgres Professional

Re: planner chooses incremental but not the best one

2024-02-15 Thread Andrei Lepikhov
On 15/2/2024 18:10, Tomas Vondra wrote: On 2/15/24 07:50, Andrei Lepikhov wrote: On 18/12/2023 19:53, Tomas Vondra wrote: On 12/18/23 11:40, Richard Guo wrote: The challenge is where to get usable information about correlation between columns. I only have a couple very rought ideas of what

Re: Memory consumed by paths during partitionwise join planning

2024-02-15 Thread Andrei Lepikhov
On 15/2/2024 19:06, Ashutosh Bapat wrote: On Thu, Feb 15, 2024 at 9:41 AM Andrei Lepikhov But I'm not sure about freeing unreferenced paths. I would have to see alternatives in the pathlist. I didn't understand this. Can you please elaborate? A path in any pathlist is refe

Re: POC, WIP: OR-clause support for indexes

2024-02-15 Thread Andrei Lepikhov
On 16/2/2024 07:00, jian he wrote: On Wed, Feb 14, 2024 at 11:21 AM Andrei Lepikhov wrote: My OS: Ubuntu 22.04.3 LTS I already set the max_parallel_workers_per_gather to 10. So for all cases, it should use parallelism first? a better question would be: how to make the number of OR less than 29

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2024-02-18 Thread Andrei Lepikhov
ability on many partitions, we should introduce per-partition memory context and reset it in between. GEQO already has a short-lived memory context, making designing extensions a bit more challenging but nothing too painful. -- regards, Andrei Lepikhov Postgres Professional

Re: Removing unneeded self joins

2024-02-18 Thread Andrei Lepikhov
d you, please, recheck? I reviewed this patch. Why do you check only the target list? I guess these links can be everywhere. See the patch in the attachment with the elaborated test and slightly changed code. -- regards, Andrei Lepikhov Postgres Professional From 7f94a3c96fd410522b87e570240cdb96b30

Re: POC, WIP: OR-clause support for indexes

2024-02-19 Thread Andrei Lepikhov
or both patches. As I see it, the only general explanation of the idea is not addressed. I'm not sure how deeply we should explain it. -- regards, Andrei Lepikhov Postgres Professional From 3a3b6aa36320a06b64f2f608e3526255e53ed655 Mon Sep 17 00:00:00 2001 From: Alena Rybakina Date: Fri, 2 Feb

Re: Memory consumed by paths during partitionwise join planning

2024-02-19 Thread Andrei Lepikhov
On 19/2/2024 19:25, Ashutosh Bapat wrote: On Fri, Feb 16, 2024 at 8:42 AM Andrei Lepikhov wrote: Live example: right now, I am working on the code like MSSQL has - a combination of NestLoop and HashJoin paths and switching between them in real-time. It requires both paths in the path list at

Re: POC, WIP: OR-clause support for indexes

2024-02-19 Thread Andrei Lepikhov
in this operation */ + continue; + arrayconst = lsecond_node(Const, saop->args); + dest = makeNode(ScalarArrayOpExpr); Thanks for the review! I'm not sure I understand you clearly. Does the patch in attachment fix the issue you raised? -- regards, Andrei Lepikhov Postgres Professional

Re: Optimize planner memory consumption for huge arrays

2024-02-19 Thread Andrei Lepikhov
same memory piece for the next array element. It finds this piece more quickly than before that optimization. -- regards, Andrei Lepikhov Postgres Professional From 2e89dc8b743953068174c777d7a014e1ea71f659 Mon Sep 17 00:00:00 2001 From: "Andrey V. Lepikhov" Date: Tue, 20 Feb 2024 11:0

Re: POC, WIP: OR-clause support for indexes

2024-02-19 Thread Andrei Lepikhov
o, by reducing the clause list, we eliminate many calls of the equal() routine, too. `leftop operator rightop` the operator can also be volatile. Do we need to check (op_volatile(opno) == PROVOLATILE_VOLATILE) within transformBoolExprOr? As usual, could you provide a test case to discuss it mo

Re: Optimize planner memory consumption for huge arrays

2024-02-19 Thread Andrei Lepikhov
think we should design small memory contexts - one per scalable direction of memory utilization, like selectivity or partitioning (appending ?). My coding experience shows that short-lived GEQO memory context forces people to learn on Postgres internals more precisely :). -- regards, Andrei Le

Re: [POC] Allow flattening of subquery with a link to upper query

2024-02-20 Thread Andrei Lepikhov
ted to fix links from a non-parent query block. So, in my opinion, the reason for this patch still exists, and we can continue this work further, maybe elaborating on flattening LATERAL references - this needs some research. [1] https://www.postgresql.org/message-id/35c8a3e8-d080-dfa8-2be3-cf5fe702010

Re: [POC] Allow flattening of subquery with a link to upper query

2024-02-20 Thread Andrei Lepikhov
On 20/2/2024 17:43, David Rowley wrote: On Tue, 20 Feb 2024 at 22:57, Andrei Lepikhov wrote: I agree that it would be nice to teach the planner how to do this, but I think it just has to be a cost-based decision. Imagine how the transformed query would perform of pg_am had a billion rows and

Re: Unlinking Parallel Hash Join inner batch files sooner

2024-02-20 Thread Andrei Lepikhov
stuck into the unlink of tons of temporary files. So, are you going to do something with this code? [1] https://www.postgresql.org/message-id/18349-83d33dd3d0c855c3%40postgresql.org -- regards, Andrei Lepikhov Postgres Professional

Re: Removing unneeded self joins

2024-02-21 Thread Andrei Lepikhov
need two different subtrees for the same query. I will look into your fix. -- regards, Andrei Lepikhov Postgres Professional

Re: POC: GROUP BY optimization

2024-02-21 Thread Andrei Lepikhov
around 10 cost points. [1] https://www.postgresql.org/message-id/CACG=ezaYM1tr6Lmp8PRH1aeZq=rbkxeotwgzmclad5mphfw...@mail.gmail.com -- regards, Andrei Lepikhov Postgres Professional

Re: Unlinking Parallel Hash Join inner batch files sooner

2024-02-21 Thread Andrei Lepikhov
On 22/2/2024 06:42, Thomas Munro wrote: On Wed, Feb 21, 2024 at 7:34 PM Andrei Lepikhov wrote: I see in [1] that the reporter mentioned a delay between the error message in parallel HashJoin and the return control back from PSQL. Your patch might reduce this delay. Also, I have the same

Re: POC: GROUP BY optimization

2024-02-21 Thread Andrei Lepikhov
tension of the relation blocks. If Maxim will answer that it's enough to resolve his issue, why not? -- regards, Andrei Lepikhov Postgres Professional

Re: Removing unneeded self joins

2024-02-22 Thread Andrei Lepikhov
se I follow another commenting style, but technically, it's still OK. -- regards, Andrei Lepikhov Postgres Professional

Re: "type with xxxx does not exist" when doing ExecMemoize()

2024-02-25 Thread Andrei Lepikhov
. Here, you just allocate the value in some upper memory context. Also, I'm curious why such a trivial error hasn't been found for a long time -- regards, Andrei Lepikhov Postgres Professional

Re: "type with xxxx does not exist" when doing ExecMemoize()

2024-02-25 Thread Andrei Lepikhov
On 26/2/2024 09:52, Andrei Lepikhov wrote: On 25/2/2024 20:32, Tender Wang wrote: I think in prepare_probe_slot(), should called datumCopy as the attached patch does. Any thoughts? Thanks. Thanks for the report. I think it is better to invent a Runtime Memory Context; likewise, it is

Re: "type with xxxx does not exist" when doing ExecMemoize()

2024-02-25 Thread Andrei Lepikhov
On 26/2/2024 12:44, Tender Wang wrote: Andrei Lepikhov <mailto:a.lepik...@postgrespro.ru>> 于2024年2月26日周一 10:57写道: On 25/2/2024 20:32, Tender Wang wrote: > I think in prepare_probe_slot(), should called datumCopy as the attached > patch does. >

Re: "type with xxxx does not exist" when doing ExecMemoize()

2024-02-26 Thread Andrei Lepikhov
On 26/2/2024 18:34, Richard Guo wrote: On Mon, Feb 26, 2024 at 3:54 PM Andrei Lepikhov mailto:a.lepik...@postgrespro.ru>> wrote: On 26/2/2024 12:44, Tender Wang wrote: > Make sense. I found MemoizeState already has a MemoryContext, so I used it. > I upda

The const expression evaluation routine should always return a copy

2024-02-26 Thread Andrei Lepikhov
as a result, allow to return a Const instead of SubPlan? But at least we can return a flat copy of the SubPplan node just for the convention — the same thing for the AlternativeSubPlan. See the patch in the attachment. -- regards, Andrei Lepikhov Postgres Professiona

Re: The const expression evaluation routine should always return a copy

2024-02-27 Thread Andrei Lepikhov
On 28/2/2024 04:19, Tom Lane wrote: Andrei Lepikhov writes: IMO, the routine eval_const_expressions_mutator contains some stale code: I'd like to push back against the idea that eval_const_expressions is expected to return a freshly-copied tree. Its API specification contains no such

Re: POC, WIP: OR-clause support for indexes

2024-02-27 Thread Andrei Lepikhov
ant to be on the right side. 2. We should describe the second part of the feature, where the optimiser can split an array to fit the optimal BitmapOr scan path. -- regards, Andrei Lepikhov Postgres Professional

Re: "type with xxxx does not exist" when doing ExecMemoize()

2024-02-27 Thread Andrei Lepikhov
ch looks better. Also, You don't need to initialize tts_values[i] at all if tts_isnull[i] set to true. -- regards, Andrei Lepikhov Postgres Professional

Re: POC, WIP: OR-clause support for indexes

2024-02-28 Thread Andrei Lepikhov
transformBoolExprOr and generate_saop_pathlist (including cross-referencing each other). These are starting points to understand the transformation and, therefore, a good place for a detailed explanation. -- regards, Andrei Lepikhov Postgres Professional

Re: POC, WIP: OR-clause support for indexes

2024-02-29 Thread Andrei Lepikhov
e smoothly. All modifications are integrated into the two new patches. Feel free to add, change or totally rewrite these changes. -- regards, Andrei Lepikhov Postgres Professional From 015a564cc784139c806a7004f25bf5f7a4b4a29d Mon Sep 17 00:00:00 2001 From: Alena Rybakina Date: Fri, 2 Feb 2024 22:01:0

Re: POC, WIP: OR-clause support for indexes

2024-03-03 Thread Andrei Lepikhov
he logic above. What about arrays? As I see, arrays don't have typarray and we can avoid to spend more cycles after detection of TYPCATEGORY_ARRAY. I haven't done it yet because have a second thought: what if to combine arrays into the larger one? I'm unsure on that, so we can forbid it too. -- regards, Andrei Lepikhov Postgres Professional

Re: a wrong index choose when statistics is out of date

2024-03-03 Thread Andrei Lepikhov
rk, but it needs a new parameter in pg_type and a lot of additional code for such a rare case. I'm looking forward to the demo patch. -- regards, Andrei Lepikhov Postgres Professional

Re: a wrong index choose when statistics is out of date

2024-03-03 Thread Andrei Lepikhov
of the boundaries when an index shows us that we have min/max outside known statistics? Because it would be used for the values out of the histogram, it should only add an overhead with a reason. -- regards, Andrei Lepikhov Postgres Professional

Re: POC, WIP: OR-clause support for indexes

2024-03-04 Thread Andrei Lepikhov
On 4/3/2024 09:26, jian he wrote: On Thu, Feb 29, 2024 at 4:59 PM Andrei Lepikhov Feel free to add, change or totally rewrite these changes. On replacement of static ScalarArrayOpExpr dest with dynamic allocated one: After discussion [1] I agree with that replacement. Some style (and language

Re: POC, WIP: OR-clause support for indexes

2024-03-04 Thread Andrei Lepikhov
On 5/3/2024 12:30, Andrei Lepikhov wrote: On 4/3/2024 09:26, jian he wrote: ... and the new version of the patchset is attached. -- regards, Andrei Lepikhov Postgres Professional From 1c3ac3e006cd66ff40f1ddaaa09e3fc0f3a75ca5 Mon Sep 17 00:00:00 2001 From: Alena Rybakina Date: Fri, 2 Feb 2024

Re: "type with xxxx does not exist" when doing ExecMemoize()

2024-03-05 Thread Andrei Lepikhov
On 1/3/2024 14:18, Tender Wang wrote: During debug, I learned that numeric_add doesn't have type check like rangetype, so aboved query will not report "type with xxx does not exist". And I realize that  the test case added by Andrei Lepikhov  in v3 is right. So in v6 pat

Re: Hooking into ExplainOneQuery() complicated by missing standard_ExplainOneQuery

2024-03-05 Thread Andrei Lepikhov
3f677%40garret.ru -- regards, Andrei Lepikhov Postgres Professional

Re: "type with xxxx does not exist" when doing ExecMemoize()

2024-03-05 Thread Andrei Lepikhov
On 6/3/2024 10:10, Tender Wang wrote: Andrei Lepikhov <mailto:a.lepik...@postgrespro.ru>> 于2024年3月5日周二 17:36写道: On 1/3/2024 14:18, Tender Wang wrote: > During debug, I learned that numeric_add doesn't have type check like > rangetype, so aboved query w

Re: a wrong index choose when statistics is out of date

2024-03-07 Thread Andrei Lepikhov
esult: planned rows=8604, actual rows=3501 EXPLAIN ANALYZE SELECT * FROM norm_test WHERE val = 700; -- result: planned rows=8604, actual rows=1705 EXPLAIN ANALYZE SELECT * FROM norm_test WHERE val = 1000; -- result: planned rows=8604, actual rows=91 -- regards, Andrei Lepikhov Postgres Professional

Re: a wrong index choose when statistics is out of date

2024-03-08 Thread Andrei Lepikhov
On 7/3/2024 17:32, David Rowley wrote: On Thu, 7 Mar 2024 at 21:17, Andrei Lepikhov wrote: I would like to ask David why the var_eq_const estimator doesn't have an option for estimation with a histogram. Having that would relieve a problem with skewed data. Detecting the situation

Re: POC, WIP: OR-clause support for indexes

2024-03-10 Thread Andrei Lepikhov
On 7/3/2024 21:51, Alexander Korotkov wrote: Hi! On Tue, Mar 5, 2024 at 9:59 AM Andrei Lepikhov mailto:a.lepik...@postgrespro.ru>> wrote: > On 5/3/2024 12:30, Andrei Lepikhov wrote: > > On 4/3/2024 09:26, jian he wrote: > ... and the new version of the patchset is atta

Re: POC, WIP: OR-clause support for indexes

2024-03-11 Thread Andrei Lepikhov
s caused by this approach. Anyway, it should be done as quickly as possible to increase the effect of the optimization. -- regards, Andrei Lepikhov Postgres Professional

Re: a wrong index choose when statistics is out of date

2024-03-12 Thread Andrei Lepikhov
tion that makes the index-picking technique less dependent on the ordering of index lists [1]. [1] https://www.postgresql.org/message-id/9b3dbf6d-776a-4953-a5a4-609929393...@postgrespro.ru -- regards, Andrei Lepikhov Postgres Professional diff --git a/src/backend/utils/adt/like_support.c

Re: POC, WIP: OR-clause support for indexes

2024-03-12 Thread Andrei Lepikhov
On 12/3/2024 22:20, Alexander Korotkov wrote: On Mon, Mar 11, 2024 at 2:43 PM Andrei Lepikhov I think you are right. It is probably a better place than any other to remove duplicates in an array. I just think we should sort and remove duplicates from entry->consts in one pass. Thus, t

Re: POC, WIP: OR-clause support for indexes

2024-03-13 Thread Andrei Lepikhov
On 13/3/2024 18:05, Alexander Korotkov wrote: On Wed, Mar 13, 2024 at 7:52 AM Andrei Lepikhov Given all of the above, I think moving transformation to the canonicalize_qual() would be the right way to go. Ok, I will try to move the code. I have no idea about the timings so far. I recall the

Re: Asymmetric partition-wise JOIN

2023-10-14 Thread Andrei Lepikhov
On 15/10/2023 07:18, Alexander Korotkov wrote: Hi Alexander, Hi Andrey, Thank you for your work on this subject. On Mon, Jan 17, 2022 at 1:42 PM Alexander Pyhalov wrote: The patch does not longer apply cleanly, so I rebased it. Attaching rebased version. Not surprising that the patch doesn'

Re: Asymmetric partition-wise JOIN

2023-10-15 Thread Andrei Lepikhov
On 15/10/2023 17:25, Alexander Korotkov wrote: On Sun, Oct 15, 2023 at 8:40 AM Andrei Lepikhov wrote: Thanks for such detailed feedback! The rationale for this patch was to give the optimizer additional ways to push down more joins into foreign servers. And, because of asynchronous append, the

Re: Removing unneeded self joins

2023-10-16 Thread Andrei Lepikhov
On 12/10/2023 18:32, Alexander Korotkov wrote: On Thu, Oct 5, 2023 at 12:17 PM Andrei Lepikhov wrote: On 4/10/2023 14:34, Alexander Korotkov wrote: > Relid replacement machinery is the most contradictory code here. We used > a utilitarian approach and implemented a simplistic v

Re: Asymmetric partition-wise JOIN

2023-10-17 Thread Andrei Lepikhov
On 16/10/2023 23:21, Ashutosh Bapat wrote: On Mon, Oct 16, 2023 at 10:24 AM Andrei Lepikhov Whenever I visited this idea, I hit one issue prominently - how would we differentiate different scans of the non-partitioned relation. Normally we do that using different Relids but in this case we

Re: Allow ALTER SYSTEM SET on unrecognized custom GUCs

2023-10-17 Thread Andrei Lepikhov
On 17/10/2023 07:19, Tom Lane wrote: Currently we have this odd behavior (for a superuser): regression=# ALTER SYSTEM SET foo.bar TO 'baz'; ERROR: unrecognized configuration parameter "foo.bar" regression=# SET foo.bar TO 'baz'; SET regression=# ALTER SYSTEM SET foo.bar TO 'baz'; ALTER SYSTEM

Re: Asymmetric partition-wise JOIN

2023-10-17 Thread Andrei Lepikhov
On 17/10/2023 17:09, Ashutosh Bapat wrote: On Tue, Oct 17, 2023 at 2:05 PM Andrei Lepikhov wrote: On 16/10/2023 23:21, Ashutosh Bapat wrote: On Mon, Oct 16, 2023 at 10:24 AM Andrei Lepikhov Whenever I visited this idea, I hit one issue prominently - how would we differentiate different scans

Re: Allow ALTER SYSTEM SET on unrecognized custom GUCs

2023-10-17 Thread Andrei Lepikhov
On 18/10/2023 12:15, Tom Lane wrote: Andrei Lepikhov writes: "SET foo.bar TO 'smth'" can immediately alter the placeholder's value. But what is the reason that "ALTER SYSTEM SET foo.bar TO 'smth'" doesn't do the same? Because it's not s

Re: Oversight in reparameterize_path_by_child leading to executor crash

2023-10-18 Thread Andrei Lepikhov
On 18/10/2023 13:39, Richard Guo wrote: On Fri, Oct 13, 2023 at 6:18 PM Andrei Lepikhov mailto:a.lepik...@postgrespro.ru>> wrote: On 23/8/2023 12:37, Richard Guo wrote: > To fix it we may need to modify RelOptInfos for Path, BitmapHeapPath, > ForeignPath and Cus

Re: Add the ability to limit the amount of memory that can be allocated to backends.

2023-10-18 Thread Andrei Lepikhov
On 19/10/2023 02:00, Stephen Frost wrote: Greetings, * Andrei Lepikhov (a.lepik...@postgrespro.ru) wrote: On 29/9/2023 09:52, Andrei Lepikhov wrote: On 22/5/2023 22:59, reid.thomp...@crunchydata.com wrote: Attach patches updated to master. Pulled from patch 2 back to patch 1 a change that

Re: Removing unneeded self joins

2023-10-18 Thread Andrei Lepikhov
On 19/10/2023 01:50, Alexander Korotkov wrote: On Mon, Oct 16, 2023 at 11:28 AM Andrei Lepikhov wrote: On 12/10/2023 18:32, Alexander Korotkov wrote: On Thu, Oct 5, 2023 at 12:17 PM Andrei Lepikhov wrote: On 4/10/2023 14:34, Alexander Korotkov wrote: > Relid replacement machinery is

Re: Asymmetric partition-wise JOIN

2023-10-18 Thread Andrei Lepikhov
On 18/10/2023 16:59, Ashutosh Bapat wrote: On Wed, Oct 18, 2023 at 10:55 AM Andrei Lepikhov wrote: But the clauses of A parameterized by P will produce different translations for each of the partitions. I think we will need different RelOptInfos (for A) to store these translations. Does

Re: Add the ability to limit the amount of memory that can be allocated to backends.

2023-10-19 Thread Andrei Lepikhov
On 20/10/2023 05:06, Stephen Frost wrote: Greetings, * Andrei Lepikhov (a.lepik...@postgrespro.ru) wrote: On 19/10/2023 02:00, Stephen Frost wrote: * Andrei Lepikhov (a.lepik...@postgrespro.ru) wrote: On 29/9/2023 09:52, Andrei Lepikhov wrote: On 22/5/2023 22:59, reid.thomp

Re: Removing unneeded self joins

2023-10-22 Thread Andrei Lepikhov
On 22/10/2023 05:01, Alexander Korotkov wrote: On Thu, Oct 19, 2023 at 6:16 AM Andrei Lepikhov wrote: On 19/10/2023 01:50, Alexander Korotkov wrote: This query took 3778.432 ms with self-join removal disabled, and 3756.009 ms with self-join removal enabled. So, no measurable overhead

Re: Add the ability to limit the amount of memory that can be allocated to backends.

2023-10-23 Thread Andrei Lepikhov
On 20/10/2023 19:39, Stephen Frost wrote: Greetings, * Andrei Lepikhov (a.lepik...@postgrespro.ru) wrote: The only issue I worry about is the uncertainty and clutter that can be created by this feature. In the worst case, when we have a complex error stack (including the extension's

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2023-10-24 Thread Andrei Lepikhov
he changes proposed? Maybe we should intensively work on adding the 'stats_since' parameter and continue the discussion of the necessity of another one? -- regards, Andrei Lepikhov Postgres Professional

Re: A performance issue with Memoize

2023-10-25 Thread Andrei Lepikhov
.  That is to say, the memoize node's keyparamids is {1}. ... Any thoughts? Do you've thought about the case, fixed with the commit 1db5667? As I see, that bugfix still isn't covered by regression tests. Could your approach of a PARAM_EXEC slot reusing break that case? -- regards, Andrei Lepikhov Postgres Professional

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2023-10-26 Thread Andrei Lepikhov
On 25/10/2023 20:00, Andrei Zubkov wrote: Hi Andrei, On Wed, 2023-10-25 at 13:59 +0700, Andrei Lepikhov wrote: But minmax_stats_since and changes in the UI of the reset routine look like syntactic sugar here. I can't convince myself that it is really needed. Do you have some set of cases

Re: A performance issue with Memoize

2023-10-30 Thread Andrei Lepikhov
On 30/10/2023 14:55, Richard Guo wrote: On Thu, Oct 26, 2023 at 12:07 PM Andrei Lepikhov mailto:a.lepik...@postgrespro.ru>> wrote: Do you've thought about the case, fixed with the commit 1db5667? As I see, that bugfix still isn't covered by regression tests. Could y

Re: [PoC] Reducing planning time when tables have many partitions

2023-11-19 Thread Andrei Lepikhov
@mail.gmail.com -- regards, Andrei Lepikhov Postgres Professional

Re: POC, WIP: OR-clause support for indexes

2023-11-20 Thread Andrei Lepikhov
opose to generate expression hash instead + prove the equality of two expressions by calling equal(). -- regards, Andrei Lepikhov Postgres Professional diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c index 25a4235dbd..de27d2646c 100644 --- a/src/backend/parser/parse_e

Re: Postgres picks suboptimal index after building of an extended statistics

2023-11-21 Thread Andrei Lepikhov
h modest number of indexes) perhaps OK. It wouldn't work without a unique index, but I don't have a better idea. It looks a bit expensive for me. But I am ready to try, if current solution doesn't look applicable. -- regards, Andrei Lepikhov Postgres Professional From 21677861e

Re: POC, WIP: OR-clause support for indexes

2023-11-23 Thread Andrei Lepikhov
trates the positive effect of the transformation. -- regards, Andrei Lepikhov Postgres Professional From 5071d02426ac3430f4dd61a8ad32c2847ba6f8a5 Mon Sep 17 00:00:00 2001 From: "Andrey V. Lepikhov" Date: Thu, 23 Nov 2023 16:00:13 +0700 Subject: [PATCH] Transform OR clause to ANY expressions

Re: POC, WIP: OR-clause support for indexes

2023-11-23 Thread Andrei Lepikhov
On 23/11/2023 16:23, Andrei Lepikhov wrote: This code changes tests in many places. But, as I see it, it mostly demonstrates the positive effect of the transformation. I found out that the postgres_fdw tests were impacted by the feature. Fix it, because the patch is on the commitfest and

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2023-11-23 Thread Andrei Lepikhov
ndex names. The 'errors table' must inherit any right policies from the table, to which we do the copy. -- regards, Andrei Lepikhov Postgres Professional

  1   2   3   4   >