About the constant-TRUE clause in reconsider_outer_join_clauses

2023-03-25 Thread Richard Guo
I happened to notice a constant-TRUE clause with is_pushed_down being true while its required_relids not including the OJ being formed, which seems abnormal to me. It turns out that this clause comes from reconsider_outer_join_clauses(), as a dummy replacement if we've generated a derived clause.

Re: Regarding the order of the header file includes

2024-03-17 Thread Richard Guo
On Wed, Mar 13, 2024 at 10:07 PM Peter Eisentraut wrote: > On 12.03.24 12:47, Richard Guo wrote: > > > > On Wed, Mar 6, 2024 at 5:32 PM Richard Guo > <mailto:guofengli...@gmail.com>> wrote: > > > > Please note that this patch only addresses the o

Re: Check lateral references within PHVs for memoize cache keys

2024-03-18 Thread Richard Guo
On Fri, Feb 2, 2024 at 5:18 PM Richard Guo wrote: > The v4 patch does not apply any more. I've rebased it on master. > Nothing else has changed. > Here is another rebase over master so it applies again. I also added a commit message to help review. Nothing else has changed.

Re: A problem about partitionwise join

2024-03-18 Thread Richard Guo
(Sorry it takes me some time to get back to this thread.) On Thu, Mar 7, 2024 at 7:13 PM Ashutosh Bapat wrote: > I did a deeper review of the patch. Here are some comments > Thank you for the review! > Approach > > The equijoin condition between partition keys doesn't appear in the j

Re: Support run-time partition pruning for hash join

2024-03-18 Thread Richard Guo
On Tue, Jan 30, 2024 at 10:33 AM Richard Guo wrote: > Attached is an updated patch. Nothing else has changed. > Here is another rebase over master so it applies again. Nothing else has changed. Thanks Richard v7-0001-Support-run-time-partition-pruning-for-hash-join.patch Descr

Re: Oversight in reparameterize_path_by_child leading to executor crash

2024-03-21 Thread Richard Guo
On Wed, Mar 20, 2024 at 2:57 AM Tom Lane wrote: > Richard Guo writes: > > Here is the patch for HEAD. I simply re-posted v10. Nothing has > > changed. > > I got back to this finally, and pushed it with some minor cosmetic > adjustments. Thanks for pushing! Thanks Richard

Re: Add Index-level REINDEX with multiple jobs

2024-03-24 Thread Richard Guo
On Mon, Mar 25, 2024 at 10:07 AM Tom Lane wrote: > Alexander Korotkov writes: > > Here goes the revised patch. I'm going to push this if there are no > objections. > > Quite a lot of the buildfarm is complaining about this: > > reindexdb.c: In function 'reindex_one_database': > reindexdb.c:434:

Re: A problem about partitionwise join

2024-03-24 Thread Richard Guo
On Tue, Mar 19, 2024 at 3:40 PM Ashutosh Bapat wrote: > On Tue, Mar 19, 2024 at 8:18 AM Richard Guo > wrote: > >> On Thu, Mar 7, 2024 at 7:13 PM Ashutosh Bapat < >> ashutosh.bapat@gmail.com> wrote: >> >>> Approach >>> >>>

Re: Properly pathify the union planner

2024-03-24 Thread Richard Guo
On Mon, Mar 25, 2024 at 9:44 AM David Rowley wrote: > It seems ok that > the ec_indexes are not set for the top-level set RelOptInfo as > get_eclass_for_sort_expr() does not make use of ec_indexes while > searching for an existing EquivalenceClass. Maybe we should fix this > varno == 0 hack and

Re: Memory consumed by child SpecialJoinInfo in partitionwise join planning

2024-03-25 Thread Richard Guo
On Mon, Mar 25, 2024 at 5:17 PM Amit Langote wrote: > I've pushed this now. > > When updating the commit message, I realized that you had made the > right call to divide the the changes around not translating the dummy > SpecialJoinInfos into a separate patch. So, I pushed it like that. Sorry

Re: Propagate pathkeys from CTEs up to the outer query

2024-03-25 Thread Richard Guo
On Tue, Mar 26, 2024 at 1:39 AM Tom Lane wrote: > I got around to looking at this finally. I was a bit surprised by > your choice of data structure. You made a per-CTE-item cte_paths > list paralleling cte_plan_ids, but what I had had in mind was a > per-subplan list of paths paralleling glob->

Remove some redundant set_cheapest() calls

2024-03-26 Thread Richard Guo
I happened to notice that the set_cheapest() calls in functions set_namedtuplestore_pathlist() and set_result_pathlist() are redundant, as we've centralized the set_cheapest() calls in set_rel_pathlist(). Attached is a trivial patch to remove these calls. BTW, I suspect that the set_cheapest() ca

Re: Properly pathify the union planner

2024-03-26 Thread Richard Guo
On Wed, Mar 27, 2024 at 6:23 AM David Rowley wrote: > Because this field is set, it plans the CTE thinking it's a UNION > child and breaks when it can't find a SortGroupClause for the CTE's > target list item. Right. The problem here is that we mistakenly think that the CTE query is a subquery

Re: Propagate pathkeys from CTEs up to the outer query

2024-03-26 Thread Richard Guo
On Wed, Mar 27, 2024 at 1:20 AM Tom Lane wrote: > Richard Guo writes: > > I agree with your points. Previously I was thinking that CTEs were the > > only scenario where we needed to remember the best path and only > > required the best path's pathkeys. Howe

Re: Remove some redundant set_cheapest() calls

2024-03-27 Thread Richard Guo
On Wed, Mar 27, 2024 at 4:06 AM Tom Lane wrote: > Richard Guo writes: > > I happened to notice that the set_cheapest() calls in functions > > set_namedtuplestore_pathlist() and set_result_pathlist() are redundant, > > as we've centralized the set_cheapest()

Re: Remove some redundant set_cheapest() calls

2024-03-27 Thread Richard Guo
On Wed, Mar 27, 2024 at 10:59 PM Tom Lane wrote: > Richard Guo writes: > > On Wed, Mar 27, 2024 at 4:06 AM Tom Lane wrote: > >> I'm less convinced about changing this. I'd rather keep it consistent > >> with mark_dummy_rel. > > > Hm, I wonder if

Re: Properly pathify the union planner

2024-03-27 Thread Richard Guo
On Wed, Mar 27, 2024 at 6:34 PM David Rowley wrote: > On Wed, 27 Mar 2024 at 22:47, David Rowley wrote: > > I did wonder when first working on this patch if subquery_planner() > > should grow an extra parameter, or maybe consolidate some existing > > ones by passing some struct that provides the

Re: To what extent should tests rely on VACUUM ANALYZE?

2024-03-28 Thread Richard Guo
On Fri, Mar 29, 2024 at 1:33 AM Tom Lane wrote: > Tomas Vondra writes: > > Yeah. I think it's good to design the data/queries in such a way that > > the behavior does not flip due to minor noise like in this case. > > +1 Agreed. The query in problem is: -- we can pull up the sublink into the

Re: To what extent should tests rely on VACUUM ANALYZE?

2024-03-29 Thread Richard Guo
On Thu, Mar 28, 2024 at 11:00 PM Alexander Lakhin wrote: > When running multiple 027_stream_regress.pl test instances in parallel > (and with aggressive autovacuum) on a rather slow machine, I encountered > test failures due to the subselect test instability just as the following > failures on bu

Remove excessive trailing semicolons

2024-03-29 Thread Richard Guo
Noticed some newly introduced excessive trailing semicolons: $ git grep -E ";;$" -- *.c *.h src/include/lib/radixtree.h:int deletepos = slot - n4->children;; src/test/modules/test_tidstore/test_tidstore.c: BlockNumber prevblkno = 0;; Here is a trivial patch to remo

Incorrect handling of IS [NOT] NULL quals on inheritance parents

2024-04-09 Thread Richard Guo
In b262ad440e we introduced an optimization that drops IS NOT NULL quals on a NOT NULL column, and reduces IS NULL quals on a NOT NULL column to constant-FALSE. I happened to notice that this is not working correctly for traditional inheritance parents. Traditional inheritance parents might have

Re: Incorrect handling of IS [NOT] NULL quals on inheritance parents

2024-04-10 Thread Richard Guo
On Wed, Apr 10, 2024 at 1:13 PM David Rowley wrote: > I looked at the patch and I don't think it's a good idea to skip > recording NOT NULL constraints to fix based on the fact that it > happens to result in this particular optimisation working correctly. > It seems that just makes this work in f

Revise some error messages in split partition code

2024-04-10 Thread Richard Guo
I noticed some error messages in the split partition code that are not up to par. Such as: "new partitions not have value %s but split partition has" how about we revise it to: "new partitions do not have value %s but split partition does" Another one is: "any partition in the list should be

Re: Removing unneeded self joins

2024-02-20 Thread Richard Guo
On Mon, Feb 19, 2024 at 1:24 PM Andrei Lepikhov wrote: > On 18/2/2024 23:18, Alexander Korotkov wrote: > > On Sun, Feb 18, 2024 at 5:04 PM Alexander Korotkov > wrote: > >> On Sun, Feb 18, 2024 at 3:00 PM Alexander Lakhin > wrote: > >>> Please look at the following query which fails with an erro

Re: POC: GROUP BY optimization

2024-02-21 Thread Richard Guo
On Fri, Feb 2, 2024 at 12:40 PM Andrei Lepikhov wrote: > On 2/2/2024 11:06, Richard Guo wrote: > > > > On Fri, Feb 2, 2024 at 11:32 AM Richard Guo > <mailto:guofengli...@gmail.com>> wrote: > > > > On Fri, Feb 2, 2024 at 10:02 AM Tom Lane &

Re: A problem about partitionwise join

2024-02-21 Thread Richard Guo
On Tue, Aug 2, 2022 at 4:24 AM Jacob Champion wrote: > Once you think you've built up some community support and the patchset > is ready for review, you (or any interested party) can resurrect the > patch entry by visiting > > https://commitfest.postgresql.org/38/2266/ > > and changing the st

Re: POC: GROUP BY optimization

2024-02-21 Thread Richard Guo
On Wed, Feb 21, 2024 at 6:20 PM Alexander Korotkov wrote: > Hi, Richard! > > > What do you think about the revisions for the test cases? > > I've rebased your patch upthread. Did some minor beautifications. > > > * The table 'btg' is inserted with 1 tuples, which seems a bit > > expensive fo

Re: POC: GROUP BY optimization

2024-02-21 Thread Richard Guo
On Thu, Feb 22, 2024 at 12:18 PM Andrei Lepikhov wrote: > On 22/2/2024 09:09, Richard Guo wrote: > > I looked through the v2 patch and have two comments. > > > > * The test case under "Check we don't pick aggregate path key instead of > > grouping path key&qu

Re: "type with xxxx does not exist" when doing ExecMemoize()

2024-02-26 Thread Richard Guo
On Mon, Feb 26, 2024 at 3:54 PM Andrei Lepikhov wrote: > On 26/2/2024 12:44, Tender Wang wrote: > > Make sense. I found MemoizeState already has a MemoryContext, so I used > it. > > I update the patch. > This approach is better for me. In the next version of this patch, I > included a test case.

Re: Support "Right Semi Join" plan shapes

2024-03-04 Thread Richard Guo
On Mon, Mar 4, 2024 at 10:33 AM wenhui qiu wrote: > HI Richard > Now it is starting the last commitfest for v17, can you respond to > Alena Rybakina points? > Thanks for reminding. Will do that soon. Thanks Richard

Re: Support "Right Semi Join" plan shapes

2024-03-04 Thread Richard Guo
On Tue, Jan 30, 2024 at 2:51 PM Alena Rybakina wrote: > I have reviewed your patch and I think it is better to add an Assert for > JOIN_RIGHT_SEMI to the ExecMergeJoin and ExecNestLoop functions to > prevent the use of RIGHT_SEMI for these types of connections (NestedLoop > and MergeJoin). Hmm,

Re: Refactoring backend fork+exec code

2024-03-05 Thread Richard Guo
On Mon, Mar 4, 2024 at 1:40 AM Heikki Linnakangas wrote: > On 22/02/2024 02:37, Heikki Linnakangas wrote: > > Here's another patch version that does that. Yeah, I agree it's nicer in > > the end. > > > > I'm pretty happy with this now. I'll read through these patches myself > > again after sleepi

Get rid of the excess semicolon in planner.c

2024-03-05 Thread Richard Guo
I think this is a typo introduced in 0452b461b. +root->processed_groupClause = list_copy(parse->groupClause);; The extra empty statement is harmless in most times, but I still think it would be better to get rid of it. Attached is a trivial patch to do that. Thanks Richard v1-0001-Get-ri

Regarding the order of the header file includes

2024-03-06 Thread Richard Guo
I think we generally follow the rule that we include 'postgres.h' or 'postgres_fe.h' first, followed by system header files, and then postgres header files ordered in ASCII value. I noticed that in some C files we fail to follow this rule strictly. Attached is a patch to fix this. Back in 2019,

Re: Regarding the order of the header file includes

2024-03-06 Thread Richard Guo
On Wed, Mar 6, 2024 at 6:25 PM Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > On Wed, Mar 6, 2024 at 3:02 PM Richard Guo wrote: > > > > I think we generally follow the rule that we include 'postgres.h' or > > 'postgres_fe.h' f

Re: Get rid of the excess semicolon in planner.c

2024-03-06 Thread Richard Guo
On Wed, Mar 6, 2024 at 6:00 AM David Rowley wrote: > On Wed, 6 Mar 2024 at 00:43, Richard Guo wrote: > > > > I think this is a typo introduced in 0452b461b. > > > > +root->processed_groupClause = list_copy(parse->groupClause);; > > "git grep -E

Re: Properly pathify the union planner

2024-03-07 Thread Richard Guo
On Thu, Mar 7, 2024 at 7:16 PM David Rowley wrote: > On Thu, 15 Feb 2024 at 17:30, David Rowley wrote: > > > > On Tue, 6 Feb 2024 at 22:05, Richard Guo wrote: > > > I'm thinking that maybe it'd be better to move the work of sorting the > > >

Re: Invalid query generated by postgres_fdw with UNION ALL and ORDER BY

2024-03-08 Thread Richard Guo
On Fri, Mar 8, 2024 at 10:13 AM David Rowley wrote: > The fix could also be to use deparseConst() in appendOrderByClause() > and have that handle Const EquivalenceMember instead. I'd rather just > skip them. To me, that seems less risky than ensuring deparseConst() > handles all Const types corr

Re: Properly pathify the union planner

2024-03-10 Thread Richard Guo
On Fri, Mar 8, 2024 at 11:31 AM David Rowley wrote: > On Fri, 8 Mar 2024 at 00:39, Richard Guo wrote: > > I would like to have another look, but it might take several days. > > Would that be too late? > > Please look. Several days is fine. I'd like to start looking o

Re: Regarding the order of the header file includes

2024-03-12 Thread Richard Guo
On Fri, Mar 8, 2024 at 6:58 PM Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > On Thu, Mar 7, 2024 at 12:39 PM Richard Guo > wrote: > > > > While rebasing one of my patches I noticed that the header file includes > > in relnode.c are not sorted

Re: Regarding the order of the header file includes

2024-03-12 Thread Richard Guo
On Wed, Mar 6, 2024 at 5:32 PM Richard Guo wrote: > Please note that this patch only addresses the order of header file > includes in backend modules (and might not be thorough). It is possible > that other modules may have a similar issue, but I have not evaluated > them yet. >

Re: A problem about ParamPathInfo for an AppendPath

2022-12-12 Thread Richard Guo
On Tue, Dec 6, 2022 at 5:00 PM Richard Guo wrote: > As we can see, MemoizePath can be generated for partitioned AppendPath > but not for union-all AppendPath. > > For the fix I think we can relax the check in create_append_path and > always use get_baserel_parampathinfo if

Re: Optimization issue of branching UNION ALL

2022-12-21 Thread Richard Guo
On Thu, Dec 22, 2022 at 9:50 AM Tom Lane wrote: > Andrey Lepikhov writes: > > Superficial study revealed possibly unnecessary operations that could be > > avoided: > > 1. Walking across a query by calling substitute_phv_relids() even if > > lastPHId shows that no one phv is presented. > > Yeah,

An oversight in ExecInitAgg for grouping sets

2022-12-21 Thread Richard Guo
I happened to notice $subject. It happens when we build eqfunctions for each grouping set. /* for each grouping set */ for (int k = 0; k < phasedata->numsets; k++) { int length = phasedata->gset_lengths[k]; if (phasedata->eqfunctions[length - 1] != NULL)

Re: Avoid lost result of recursion (src/backend/optimizer/util/inherit.c)

2022-12-22 Thread Richard Guo
On Wed, Dec 21, 2022 at 9:45 AM David Rowley wrote: > Also, I think it might be better to take the opportunity to rewrite > the function to not use recursion. I don't quite see the need for it > here and it looks like that might have helped contribute to the > reported issue. Can't we just write

Re: Avoid lost result of recursion (src/backend/optimizer/util/inherit.c)

2022-12-22 Thread Richard Guo
On Thu, Dec 22, 2022 at 5:22 PM David Rowley wrote: > On Thu, 22 Dec 2022 at 21:18, Richard Guo wrote: > > My best guess is that this function is intended to share the same code > > pattern as in adjust_appendrel_attrs_multilevel. The recursion is > > needed as '

Re: Avoid lost result of recursion (src/backend/optimizer/util/inherit.c)

2022-12-22 Thread Richard Guo
On Fri, Dec 23, 2022 at 11:22 AM Amit Langote wrote: > Attached shows a test case I was able to come up with that I can see > is broken by a61b1f74 though passes after applying Richard's patch. > What's broken is that deparseUpdateSql() outputs a remote UPDATE > statement with the wrong SET colum

Re: An oversight in ExecInitAgg for grouping sets

2022-12-26 Thread Richard Guo
On Thu, Dec 22, 2022 at 2:02 PM Richard Guo wrote: > I happened to notice $subject. It happens when we build eqfunctions for > each grouping set. > > /* for each grouping set */ > for (int k = 0; k < phasedata->numsets; k++) > { > int

Re: Making Vars outer-join aware

2022-12-27 Thread Richard Guo
On Sat, Dec 24, 2022 at 2:20 AM Tom Lane wrote: > I shoved some preliminary refactoring into the 0001 patch, > notably splitting deconstruct_jointree into two passes. > 0002-0009 cover the same ground as they did before, though > with some differences in detail. 0010-0012 are new work > mostly a

Re: Making Vars outer-join aware

2022-12-28 Thread Richard Guo
On Tue, Dec 27, 2022 at 11:31 PM Tom Lane wrote: > The thing that I couldn't get around before is that if you have, > say, a mergejoinable equality clause in an outer join: > > select ... from a left join b on a.x = b.y; > > that equality clause can only be associated with the join domain > f

Re: Check lateral references within PHVs for memoize cache keys

2022-12-29 Thread Richard Guo
On Fri, Dec 9, 2022 at 5:16 PM Richard Guo wrote: > Actually we do have checked PHVs for lateral references, earlier in > create_lateral_join_info. But that time we only marked lateral_relids > and direct_lateral_relids, without remembering the lateral expressions. > So I'm won

Re: Removing redundant grouping columns

2022-12-30 Thread Richard Guo
On Wed, Dec 28, 2022 at 6:18 AM Tom Lane wrote: > This patch is aimed at being smarter about cases where we have > redundant GROUP BY entries, for example > > SELECT ... WHERE a.x = b.y GROUP BY a.x, b.y; > > It's clearly not necessary to perform grouping using both columns. > Grouping by either

Re: An oversight in ExecInitAgg for grouping sets

2023-01-03 Thread Richard Guo
On Tue, Jan 3, 2023 at 5:25 AM Tom Lane wrote: > Agreed, that's a latent bug. It's only latent because the word just > before a palloc chunk will never be zero, either in our historical > palloc code or in v16's shiny new implementation. Nonetheless it > is a horrible idea for ExecInitAgg to de

Some compiling warnings

2023-01-03 Thread Richard Guo
When trying Valgrind I came across some compiling warnings with USE_VALGRIND defined and --enable-cassert not configured. This is mainly because in this case we have MEMORY_CONTEXT_CHECKING defined while USE_ASSERT_CHECKING not defined. aset.c: In function ‘AllocSetFree’: aset.c:1027:10: warning:

Re: An oversight in ExecInitAgg for grouping sets

2023-01-04 Thread Richard Guo
On Thu, Jan 5, 2023 at 6:18 AM David Rowley wrote: > On Tue, 3 Jan 2023 at 10:25, Tom Lane wrote: > > The thing that I find really distressing here is that it's been > > like this for years and none of our automated testing caught it. > > You'd have expected valgrind testing to do so ... but it

Re: Allow DISTINCT to use Incremental Sort

2023-01-09 Thread Richard Guo
On Sat, Jan 7, 2023 at 5:47 PM David Rowley wrote: > While working on the regression tests added in a14a58329, I noticed > that DISTINCT does not make use of Incremental Sort. It'll only ever > do full sorts on the cheapest input path or make use of a path that's > already got the required pathk

Re: Allow DISTINCT to use Incremental Sort

2023-01-09 Thread Richard Guo
On Tue, Jan 10, 2023 at 10:14 AM David Rowley wrote: > > /* For explicit-sort case, always use the more rigorous clause */ > > if (list_length(root->distinct_pathkeys) < > > list_length(root->sort_pathkeys)) > > { > > needed_pathkeys = root->sort_pa

Some revises in adding sorting path

2023-01-10 Thread Richard Guo
While reviewing [1], I visited other places where sorting is needed, and have some findings. In add_paths_with_pathkeys_for_rel, we do not try incremental sort atop of the epq_path, which I think we can do. I'm not sure how useful this is in real world since the epq_path is used only for EPQ chec

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2023-01-10 Thread Richard Guo
On Tue, Jan 10, 2023 at 6:12 PM Dean Rasheed wrote: > While doing some random testing, I noticed that the following is broken in > HEAD: > > SELECT COUNT(DISTINCT random()) FROM generate_series(1,10); > > ERROR: ORDER/GROUP BY expression not found in targetlist > > It appears to have been broken

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2023-01-11 Thread Richard Guo
On Wed, Jan 11, 2023 at 12:13 PM David Rowley wrote: > postgres=# set enable_presorted_aggregate=0; > SET > postgres=# select string_agg(random()::text, ',' order by random()) > from generate_series(1,3); > string_agg > -

Re: Fix condition in shm_toc and remove unused function shm_toc_freespace.

2023-01-12 Thread Richard Guo
On Thu, Jan 12, 2023 at 2:50 PM Zhang Mingli wrote: > On Jan 12, 2023, 14:34 +0800, Zhang Mingli , wrote: > > Some conditions in shm_toc_insert and shm_toc_allocate are bogus, like: > if (toc_bytes + nbytes > total_bytes || toc_bytes + nbytes < toc_bytes) > Remove the condition `toc_bytes + nbyte

Improve LATERAL join case in test memoize.sql

2023-01-16 Thread Richard Guo
I happened to notice we have the case in memoize.sql that tests for memoize node with LATERAL joins, which is -- Try with LATERAL joins SELECT explain_memoize(' SELECT COUNT(*),AVG(t2.unique1) FROM tenk1 t1, LATERAL (SELECT t2.unique1 FROM tenk1 t2 WHERE t1.twenty = t2.unique1) t2 WHERE t1.unique1

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2023-01-16 Thread Richard Guo
On Tue, Jan 17, 2023 at 11:39 AM David Rowley wrote: > On Tue, 17 Jan 2023 at 13:16, Dean Rasheed > wrote: > > I took a look at this, and I agree that the best solution is probably > > to have make_pathkeys_for_groupagg() ignore Aggrefs that contain > > volatile functions. > > Thanks for giving

Re: Removing redundant grouping columns

2023-01-16 Thread Richard Guo
On Sun, Jan 15, 2023 at 5:23 AM Tom Lane wrote: > vignesh C writes: > > The patch does not apply on top of HEAD as in [1], please post a rebased > patch: > > Yeah, sideswiped by 3c6fc5820 apparently. No substantive change needed. I looked through these two patches and they look good to me. B

Re: Removing redundant grouping columns

2023-01-17 Thread Richard Guo
On Wed, Jan 18, 2023 at 6:51 AM Tom Lane wrote: > I wrote: > > Yeah, sideswiped by 3c6fc5820 apparently. No substantive change needed. > > And immediately sideswiped by da5800d5f. Yeah, I noticed this too yesterday. I reviewed through these two patches yesterday and I think they are in good s

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2023-01-18 Thread Richard Guo
On Tue, Jan 17, 2023 at 3:05 PM Tom Lane wrote: > Richard Guo writes: > > BTW, I wonder if we should have checked CoercionForm before > > fix_upper_expr_mutator steps into CoerceViaIO->arg to adjust the expr > > there. > > I will just quote what it says in

Re: About the constant-TRUE clause in reconsider_outer_join_clauses

2023-03-26 Thread Richard Guo
On Sat, Mar 25, 2023 at 11:41 PM Tom Lane wrote: > Richard Guo writes: > > Should we instead mark the constant-TRUE clause with required_relids > > plus the OJ relid? > > I do not think it matters. Yeah, I agree that it makes no difference currently. One day if w

Re: same query but different result on pg16devel and pg15.2

2023-04-03 Thread Richard Guo
On Tue, Apr 4, 2023 at 10:53 AM tender wang wrote: > Hi hackers, > I encounter a problem, as shown below: > > query: > select > ref_0.ps_suppkey as c0, > ref_1.c_acctbal as c1, > ref_2.o_totalprice as c2, > ref_2.o_orderpriority as c3, > ref_2.o_clerk as c4 > from > public.parts

Re: Using each rel as both outer and inner for JOIN_ANTI

2023-04-05 Thread Richard Guo
On Thu, Apr 6, 2023 at 8:18 AM Thomas Munro wrote: > On Thu, Apr 6, 2023 at 9:11 AM Tom Lane wrote: > > Richard Guo writes: > > > Thanks for reminding. Attached is the rebased patch, with no other > > > changes. I think the patch is ready for commit. > >

Re: Using each rel as both outer and inner for JOIN_ANTI

2023-04-05 Thread Richard Guo
On Thu, Apr 6, 2023 at 1:06 PM Tom Lane wrote: > This: > > > +#if 0 > > /* If any limit was set to zero, the user doesn't want a > > parallel scan. */ > > if (parallel_workers <= 0) > > return; > > +#endif > > seems like it adds a lot of new paths with a lot lower

Re: Using each rel as both outer and inner for JOIN_ANTI

2023-04-07 Thread Richard Guo
On Tue, Aug 2, 2022 at 3:13 PM Richard Guo wrote: > On Sun, Jul 31, 2022 at 12:07 AM Tom Lane wrote: > >> [ wanders away wondering if JOIN_RIGHT_SEMI should become a thing ... ] > > Maybe this is something we can do. Currently for the query below: > > # explain sele

Re: Using each rel as both outer and inner for JOIN_ANTI

2023-04-10 Thread Richard Guo
On Fri, Apr 7, 2023 at 3:28 PM Richard Guo wrote: > On Tue, Aug 2, 2022 at 3:13 PM Richard Guo wrote: > >> On Sun, Jul 31, 2022 at 12:07 AM Tom Lane wrote: >> >>> [ wanders away wondering if JOIN_RIGHT_SEMI should become a thing ... ] >> >> Maybe this is

Can we rely on the ordering of paths in pathlist?

2023-04-10 Thread Richard Guo
As the comment above add_path() says, 'The pathlist is kept sorted by total_cost, with cheaper paths at the front.' And it seems that get_cheapest_parallel_safe_total_inner() relies on this ordering (without being mentioned in the comments, which I think we should do). I'm wondering if this is the

A minor adjustment to get_cheapest_path_for_pathkeys

2023-04-11 Thread Richard Guo
The check for parallel_safe should be even cheaper than cost comparison so I think it's better to do that first. The attached patch does this and also updates the comment to mention the requirement about being parallel-safe. Thanks Richard v1-0001-Adjustment-to-get_cheapest_path_for_pathkeys.pa

Re: Protecting allocator headers with Valgrind

2023-04-11 Thread Richard Guo
On Tue, Apr 11, 2023 at 9:28 PM David Rowley wrote: > Over on [1], Tom mentioned that we might want to rethink the decision > to not protect chunk headers with Valgrind. That thread fixed a bug > that was accessing array element -1, which effectively was reading the > MemoryChunk at the start of

Re: v12: ERROR: subplan "InitPlan 2 (returns $4)" was not initialized

2023-04-11 Thread Richard Guo
On Wed, Apr 12, 2023 at 3:59 AM Tom Lane wrote: > The v1 patch attached is enough to fix the immediate issue, > but there's another thing not to like, which is that we're also > discarding the costs associated with the initplans. That's > strictly cosmetic given that all the planning decisions a

Wrong results from Parallel Hash Full Join

2023-04-12 Thread Richard Guo
I came across $subject and reduced the repro query as below. create table a (i int); create table b (i int); insert into a values (1); insert into b values (2); update b set i = 2; set min_parallel_table_scan_size to 0; set parallel_tuple_cost to 0; set parallel_setup_cost to 0; # explain (costs

Re: Unexpected (wrong?) result querying boolean partitioned table with NULL partition

2023-04-12 Thread Richard Guo
On Wed, Apr 12, 2023 at 7:13 PM David Rowley wrote: > There's already code to effectively handle <> operators. Just the > PartClauseInfo.op_is_ne needs to be set to true. > get_matching_list_bounds() then handles that by taking the inverse of > the partitions matching the equality operator. > > E

Re: Unexpected (wrong?) result querying boolean partitioned table with NULL partition

2023-04-12 Thread Richard Guo
On Thu, Apr 13, 2023 at 10:39 AM Richard Guo wrote: > On Wed, Apr 12, 2023 at 7:13 PM David Rowley wrote: > >> There's already code to effectively handle <> operators. Just the >> PartClauseInfo.op_is_ne needs to be set to true. >> get_matching_list_bound

Re: Allowing parallel-safe initplans

2023-04-13 Thread Richard Guo
On Thu, Apr 13, 2023 at 12:43 AM Tom Lane wrote: > Pursuant to the discussion at [1], here's a patch that removes our > old restriction that a plan node having initPlans can't be marked > parallel-safe (dating to commit ab77a5a45). That was really a special > case of the fact that we couldn't tr

Re: Allowing parallel-safe initplans

2023-04-16 Thread Richard Guo
On Thu, Apr 13, 2023 at 10:00 PM Tom Lane wrote: > Richard Guo writes: > > * For the diff in standard_planner, I was wondering why not move the > > initPlans up to the Gather node, just as we did before. So I tried that > > way but did not notice the breakage of regressi

Re: Allowing parallel-safe initplans

2023-04-17 Thread Richard Guo
On Mon, Apr 17, 2023 at 10:57 AM Richard Guo wrote: > The initPlan has been moved from the Result node to the Gather node. As > a result, when doing tuple projection for the Result node, we'd get a > ParamExecData entry with NULL execPlan. So the initPlan does not get > chan

Re: Incremental sort for access method with ordered scan support (amcanorderbyop)

2023-04-17 Thread Richard Guo
On Sun, Apr 16, 2023 at 1:20 AM Miroslav Bendik wrote: > Postgres allows incremental sort only for ordered indexes. Function > build_index_paths dont build partial order paths for access methods > with order support. My patch adds support for incremental ordering > with access method. I think t

Re: Allowing parallel-safe initplans

2023-04-18 Thread Richard Guo
On Mon, Apr 17, 2023 at 11:04 PM Tom Lane wrote: > Richard Guo writes: > > So now it seems that the breakage of regression tests is more severe > > than being cosmetic. I wonder if we need to update the comments to > > indicate the potential wrong results issue if we

Support "Right Semi Join" plan shapes

2023-04-18 Thread Richard Guo
In thread [1] which discussed 'Right Anti Join', Tom once mentioned 'Right Semi Join'. After a preliminary investigation I think it is beneficial and can be implemented with very short change. With 'Right Semi Join', what we want to do is to just have the first match for each inner tuple. For Ha

Re: Allowing parallel-safe initplans

2023-04-18 Thread Richard Guo
On Tue, Apr 18, 2023 at 9:33 PM Tom Lane wrote: > Richard Guo writes: > > It seems that in this case the top_plan does not have any extParam, so > > the Gather node that is added atop the top_plan does not have a chance > > to get its initParam filled in set_param_reference

Re: Incremental sort for access method with ordered scan support (amcanorderbyop)

2023-04-19 Thread Richard Guo
On Thu, Apr 20, 2023 at 6:38 AM David Rowley wrote: > That function is pretty new and was exactly added so we didn't have to > write list_truncate(list_copy(...), n) anymore. That gets pretty > wasteful when the input List is long and we only need a small portion > of it. I searched the codes

Improve list manipulation in several places

2023-04-21 Thread Richard Guo
There was discussion in [1] about improvements to list manipulation in several places. But since the discussion is not related to the topic in that thread, fork a new thread here and attach a patch to show my thoughts. Some are just cosmetic changes by using macros. The others should have perfor

Re: Incremental sort for access method with ordered scan support (amcanorderbyop)

2023-04-21 Thread Richard Guo
On Fri, Apr 21, 2023 at 5:43 AM David Rowley wrote: > On Thu, 20 Apr 2023 at 18:46, Richard Guo wrote: > > I searched the codes and found some other places where the manipulation > > of lists can be improved in a similar way. > I'd be happy to discuss our thought about L

Re: Improve list manipulation in several places

2023-04-22 Thread Richard Guo
On Sat, Apr 22, 2023 at 12:55 AM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > On 21.04.23 09:34, Richard Guo wrote: > > There was discussion in [1] about improvements to list manipulation in > > several places. But since the discussion is not related to

Re: Improve list manipulation in several places

2023-04-22 Thread Richard Guo
On Fri, Apr 21, 2023 at 7:16 PM Ranier Vilela wrote: > +lcons_copy(void *datum, const List *list) > +lappend_copy(const List *list, void *datum) > list param pointer can be const here not? > Correct. Good point. V2 patch does that. Thanks Richard

Re: Improving worst-case merge join performance with often-null foreign key

2023-04-23 Thread Richard Guo
On Sat, Apr 22, 2023 at 11:21 PM Tom Lane wrote: > Steinar Kaldager writes: > > First-time potential contributor here. We recently had an incident due > > to a sudden 1000x slowdown of a Postgres query (from ~10ms to ~10s) > > due to a join with a foreign key that was often null. We found that i

Re: Improving worst-case merge join performance with often-null foreign key

2023-04-24 Thread Richard Guo
On Sun, Apr 23, 2023 at 5:29 PM Richard Guo wrote: > On Sat, Apr 22, 2023 at 11:21 PM Tom Lane wrote: > >> Steinar Kaldager writes: >> > First-time potential contributor here. We recently had an incident due >> > to a sudden 1000x slowdown of a Postgres query (fro

Re: postgres_fdw: wrong results with self join + enable_nestloop off

2023-04-25 Thread Richard Guo
On Fri, Apr 14, 2023 at 8:51 PM Etsuro Fujita wrote: > I think that the root cause for this issue would be in the > create_scan_plan handling of pseudoconstant quals when creating a > foreign-join (or custom-join) plan. Yes exactly. In create_scan_plan, we are supposed to extract all the pseud

Re: [pg_rewind] use the passing callback instead of global function

2023-04-26 Thread Richard Guo
On Wed, Apr 26, 2023 at 9:51 AM Junwang Zhao wrote: > `local_traverse_files` and `libpq_traverse_files` both have a > callback parameter but instead use the global process_source_file > which is no good for function encapsulation. Nice catch. This should be a typo introduced by 37d2ff38. Whil

Introduce join_info_array for direct lookups of SpecialJoinInfo by ojrelid

2023-05-04 Thread Richard Guo
When working on the improper qual pushdown issue [1], there is a need in the proposed fix to avoid scanning all the SpecialJoinInfos, since that is too expensive. I think this might be a common requirement. In the current codes there are several places where we need to scan all the SpecialJoinInf

Re: Questionable coding in nth_value

2023-05-06 Thread Richard Guo
On Sat, May 6, 2023 at 4:44 PM Tatsuo Ishii wrote: > Currently Window function nth_value is coded as following: > > nth = DatumGetInt32(WinGetFuncArgCurrent(winobj, 1, &isnull)); > if (isnull) > PG_RETURN_NULL(); > const_offset = get_fn_expr_arg_stable(fcin

Re: Introduce join_info_array for direct lookups of SpecialJoinInfo by ojrelid

2023-05-07 Thread Richard Guo
On Thu, May 4, 2023 at 4:07 PM Richard Guo wrote: > When working on the improper qual pushdown issue [1], there is a need in > the proposed fix to avoid scanning all the SpecialJoinInfos, since that > is too expensive. I think this might be a common requirement. In the > current

Re: Improve list manipulation in several places

2023-05-08 Thread Richard Guo
On Mon, May 8, 2023 at 11:22 PM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > On 23.04.23 08:42, Richard Guo wrote: > > Thanks for the suggestion. I've split the patch into two as attached. > > 0001 is just a minor simplification by replacing lfirst

Re: Improve list manipulation in several places

2023-05-08 Thread Richard Guo
On Tue, May 9, 2023 at 1:26 AM Alvaro Herrera wrote: > The problem I see is that each of these new functions has a single > caller, and the only one that looks like it could have a performance > advantage is list_copy_move_nth_to_head() (which is the weirdest of the > lot). I'm inclined not to h

<    1   2   3   4   5   6   7   8   9   10   >