Re: Typo in src/backend/access/nbtree/README?

2023-06-08 Thread Richard Guo
On Fri, Jun 9, 2023 at 10:37 AM Gurjeet Singh wrote: > On Thu, Jun 8, 2023 at 7:11 AM Daniel Westermann (DWE) > wrote: > > > > ... shouldn't there be a "to" before "detect"? > > > > These two additions make it possible detect a concurrent page split > > Agreed. Attached is a small patch that fix

Re: Remove WindowClause PARTITION BY items belonging to redundant pathkeys

2023-06-09 Thread Richard Guo
On Fri, Jun 9, 2023 at 8:13 AM David Rowley wrote: > After looking again at nodeWindowAgg.c, I think it might be possible > to do a bit more work and apply this to ORDER BY items too. Without > an ORDER BY clause, all rows in the partition are peers of each other, > and if the ORDER BY column is

Re: ERROR: wrong varnullingrels (b 3) (expected (b)) for Var 2/1

2023-06-11 Thread Richard Guo
On Sat, Jun 10, 2023 at 12:08 AM Tom Lane wrote: > Richard Guo writes: > > We can identify in which form of identity 3 the plan is built up by > > checking the relids of the B/C join's outer rel. If it's in the first > > form, the outer rel's relids must

Re: Remove WindowClause PARTITION BY items belonging to redundant pathkeys

2023-06-12 Thread Richard Guo
On Mon, Jun 12, 2023 at 12:06 PM David Rowley wrote: > On Fri, 9 Jun 2023 at 20:57, Richard Guo wrote: > > On Fri, Jun 9, 2023 at 8:13 AM David Rowley > wrote: > >> It might be possible to make adjustments in nodeWindowAgg.c to have > >> the equality checks com

Re: ERROR: wrong varnullingrels (b 3) (expected (b)) for Var 2/1

2023-06-12 Thread Richard Guo
On Mon, Jun 12, 2023 at 10:02 PM Tom Lane wrote: > Richard Guo writes: > > Yeah, that makes sense. process_subquery_nestloop_params is a better > > place to do this adjustments. +1 to v2 patch. > > Pushed, then. Oh, wait ... It occurred to me that we may have this sa

Re: ERROR: wrong varnullingrels (b 3) (expected (b)) for Var 2/1

2023-06-13 Thread Richard Guo
On Wed, Jun 14, 2023 at 6:02 AM Tom Lane wrote: > I wrote: > > Richard Guo writes: > >> Oh, wait ... It occurred to me that we may have this same issue with > >> Memoize cache keys. > > > Good catch --- I'll take a closer look tomorrow. > > Pushed

Re: Replace (GUC_UNIT_MEMORY | GUC_UNIT_TIME) with GUC_UNIT in guc.c

2023-06-14 Thread Richard Guo
On Wed, Jun 14, 2023 at 1:07 PM Masahiko Sawada wrote: > On Wed, Jun 14, 2023 at 12:33 PM Japin Li wrote: > > Hi, hackers > > > > We use (GUC_UNIT_MEMORY | GUC_UNIT_TIME) instead of GUC_UNIT even though > we > > already define it in guc.h. > > > > Maybe using GUC_UNIT is better? Here is a patch

Re: Avoid unncessary always true test (src/backend/storage/buffer/bufmgr.c)

2023-06-14 Thread Richard Guo
On Tue, Jun 13, 2023 at 3:39 PM Kyotaro Horiguchi wrote: > Gurjeet has mentioned that eb.rel cannot be modified by another > process since the value or memory is in the local stack, and I believe > he's correct. > > If the pointed Relation had been blown out, eb.rel would be left > dangling, not

Re: Replace (GUC_UNIT_MEMORY | GUC_UNIT_TIME) with GUC_UNIT in guc.c

2023-06-14 Thread Richard Guo
On Wed, Jun 14, 2023 at 3:47 PM Michael Paquier wrote: > On Wed, Jun 14, 2023 at 03:38:10PM +0800, Richard Guo wrote: > > +1. BTW, it seems that GUC_UNIT_TIME is not used anywhere except in > > GUC_UNIT. I was wondering if we can retire it, but maybe we'd better > >

Re: postgres_fdw: wrong results with self join + enable_nestloop off

2023-06-24 Thread Richard Guo
On Wed, Jun 14, 2023 at 2:49 PM Etsuro Fujita wrote: > On Mon, Jun 5, 2023 at 10:19 PM Etsuro Fujita > wrote: > > To avoid this issue, I am wondering if we should modify > > add_paths_to_joinrel() in back branches so that it just disallows the > > FDW to consider pushing down joins when the rest

Re: Incremental sort for access method with ordered scan support (amcanorderbyop)

2023-06-25 Thread Richard Guo
On Thu, Apr 20, 2023 at 9:37 PM Miroslav Bendik wrote: > Thanks for this fix. Now the version > am_orderbyop_incremental_sort_v3.1.patch [1] works without issues > using the master branch. The v3.1 patch looks good to me, except that the comments around match_pathkeys_to_index still need some p

Re: Parallelize correlated subqueries that execute within each worker

2023-06-25 Thread Richard Guo
On Mon, Jun 12, 2023 at 10:23 AM James Coleman wrote: > BTW are you by any chance testing on ARM macOS? I reproduced the issue > there, but for some reason I did not reproduce the error (and the plan > wasn't parallelized) when I tested this on linux. Perhaps I missed > setting something up; it s

Re: Assert !bms_overlap(joinrel->relids, required_outer)

2023-06-27 Thread Richard Guo
On Tue, Jun 27, 2023 at 1:35 PM Michael Paquier wrote: > On Mon, Jun 26, 2023 at 11:05:43PM -0500, Jaime Casanova wrote: > > The attached query makes beta2 crash with attached backtrace. > > Interestingly the index on ref_6 is needed to make it crash, without > > it the query works fine. > > Issu

Re: Assert !bms_overlap(joinrel->relids, required_outer)

2023-06-27 Thread Richard Guo
On Tue, Jun 27, 2023 at 10:12 PM Tom Lane wrote: > Richard Guo writes: > > That's right. This issue has something to do with the > > outer-join-aware-Var changes. I reduced the repro to the query below. > > Thanks for the simplified test case. > > > Whe

Re: Another incorrect comment for pg_stat_statements

2023-06-27 Thread Richard Guo
On Wed, Jun 28, 2023 at 10:53 AM Japin Li wrote: > > Hi, hackers > > There has $subject that introduced by commit 6b4d23feef6. When we reset > the entries > if all parameters are avaiable, non-top-level entries removed first, then > top-level > entries. I did not see the diffs. Maybe uploaded

Re: Assert !bms_overlap(joinrel->relids, required_outer)

2023-06-27 Thread Richard Guo
On Wed, Jun 28, 2023 at 6:28 AM Tom Lane wrote: > For a real fix, I'm inclined to extend the loop that calculates > param_source_rels (in add_paths_to_joinrel) so that it also tracks > a set of incompatible relids that *must not* be present in the > parameterization of a proposed path. This woul

Re: Another incorrect comment for pg_stat_statements

2023-06-28 Thread Richard Guo
On Wed, Jun 28, 2023 at 3:04 PM Michael Paquier wrote: > On Wed, Jun 28, 2023 at 12:15:47PM +0800, Japin Li wrote: > > - /* Remove the key if it exists, starting with the > top-level entry */ > > + /* Remove the key if it exists, starting with the > non-top-level entry */

Re: Another incorrect comment for pg_stat_statements

2023-06-28 Thread Richard Guo
On Wed, Jun 28, 2023 at 3:36 PM Michael Paquier wrote: > On Wed, Jun 28, 2023 at 03:09:55PM +0800, Richard Guo wrote: > > +1. To nitpick, how about we remove the blank line just before removing > > the key for top level entry? > > > > - /* Also remove entri

Re: Assert !bms_overlap(joinrel->relids, required_outer)

2023-06-28 Thread Richard Guo
On Wed, Jun 28, 2023 at 10:09 PM Tom Lane wrote: > However, given that what we need is to exclude parameterization > that depends on the currently-formed OJ, it seems to me we can do > it more simply and without any new JoinPathExtraData field, > as attached. What do you think? I think it make

Trivial revise for the check of parameterized partial paths

2023-06-28 Thread Richard Guo
While working on the invalid parameterized join path issue [1], I noticed that we can simplify the codes for checking parameterized partial paths in try_partial_hashjoin/mergejoin_path, with the help of macro PATH_REQ_OUTER. - if (inner_path->param_info != NULL) - { - Rel

Re: Another incorrect comment for pg_stat_statements

2023-06-28 Thread Richard Guo
On Thu, Jun 29, 2023 at 8:19 AM Michael Paquier wrote: > Nothing much to add, so applied with the initial comment fix. Thanks for pushing it! Thanks Richard

Re: Assert !bms_overlap(joinrel->relids, required_outer)

2023-06-28 Thread Richard Guo
On Thu, Jun 29, 2023 at 10:39 AM Richard Guo wrote: > On Wed, Jun 28, 2023 at 10:09 PM Tom Lane wrote: > >> However, given that what we need is to exclude parameterization >> that depends on the currently-formed OJ, it seems to me we can do >> it more si

Re: Assert !bms_overlap(joinrel->relids, required_outer)

2023-06-29 Thread Richard Guo
On Wed, Jun 28, 2023 at 10:09 PM Tom Lane wrote: > Those cases will go through calc_non_nestloop_required_outer > which has > > /* neither path can require rels from the other */ > Assert(!bms_overlap(outer_paramrels, inner_path->parent->relids)); > Assert(!bms_overlap(inn

Re: Assert !bms_overlap(joinrel->relids, required_outer)

2023-06-29 Thread Richard Guo
On Fri, Jun 30, 2023 at 12:16 AM Tom Lane wrote: > Pushed with that and defenses added to try_mergejoin_path and > try_hashjoin_path. It looks like the various try_partial_xxx_path > functions already reject cases that could be problematic. (They > will not accept inner parameterization that wo

Re: Assert !bms_overlap(joinrel->relids, required_outer)

2023-06-29 Thread Richard Guo
On Fri, Jun 30, 2023 at 12:20 AM Tom Lane wrote: > Richard Guo writes: > > On Wed, Jun 28, 2023 at 10:09 PM Tom Lane wrote: > >> Those cases will go through calc_non_nestloop_required_outer > >> which has > >> /* neither path can require rels from t

Re: Assert !bms_overlap(joinrel->relids, required_outer)

2023-07-02 Thread Richard Guo
On Fri, Jun 30, 2023 at 11:00 PM Tom Lane wrote: > Richard Guo writes: > > I think it just makes these two assertions meaningless to skip it for > > non-nestloop joins if the input paths are for otherrels, because paths > > would never be parameterized by the member rel

Re: Improve list manipulation in several places

2023-07-03 Thread Richard Guo
On Mon, Jul 3, 2023 at 5:41 PM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > On 09.05.23 05:13, Richard Guo wrote: > > Yeah, maybe this is the reason I failed to devise a query that shows any > > performance gain. I tried with a query which makes the

Re: Check lateral references within PHVs for memoize cache keys

2023-07-04 Thread Richard Guo
On Fri, Dec 30, 2022 at 11:00 AM Richard Guo wrote: > On Fri, Dec 9, 2022 at 5:16 PM Richard Guo wrote: > >> Actually we do have checked PHVs for lateral references, earlier in >> create_lateral_join_info. But that time we only marked lateral_relids >> and direct

Re: Incremental sort for access method with ordered scan support (amcanorderbyop)

2023-07-04 Thread Richard Guo
On Sun, Jul 2, 2023 at 12:02 PM Miroslav Bendik wrote: > Thanks, for suggestions. > > On Sun 02. 07. 2023 at 10:18 Richard Guo wrote: > > 1. For comment "On success, the result list is ordered by pathkeys.", I > > think it'd be more accurate if we say someth

Re: Incremental sort for access method with ordered scan support (amcanorderbyop)

2023-07-04 Thread Richard Guo
On Tue, Jul 4, 2023 at 7:15 PM David Rowley wrote: > On Tue, 4 Jul 2023 at 20:12, Richard Guo wrote: > > The v4 patch looks good to me (maybe some cosmetic tweaks are still > > needed for the comments). I think it's now 'Ready for Committer'. > > I agree

Re: Small code simplification

2024-08-21 Thread Richard Guo
On Wed, Aug 21, 2024 at 6:25 PM Tender Wang wrote: >Oversight in 7ff9afbbd; I think we can do the same way for the > ATExecAddColumn(). LGTM. Pushed. Thanks Richard

Redundant Result node

2024-08-22 Thread Richard Guo
I ran into a query plan where the Result node seems redundant to me: create table t (a int, b int, c int); insert into t select i%10, i%10, i%10 from generate_series(1,100)i; create index on t (a, b); analyze t; set enable_hashagg to off; set enable_seqscan to off; explain (verbose, costs off) s

Re: Redundant Result node

2024-08-22 Thread Richard Guo
On Thu, Aug 22, 2024 at 8:03 PM David Rowley wrote: > On Thu, 22 Aug 2024 at 23:33, Peter Eisentraut wrote: > > > I wonder if we need to invent a function to compare two PathTargets. > > > > Wouldn't the normal node equal() work? > > It might. I think has_volatile_expr might be missing a > pg_nod

Re: Redundant Result node

2024-08-22 Thread Richard Guo
On Thu, Aug 22, 2024 at 3:34 PM Richard Guo wrote: > /* Add projection step if needed */ > if (sorted_path->pathtarget != target) > sorted_path = apply_projection_to_path(root, ordered_rel, >sorted_path, target); &g

Re: Redundant Result node

2024-08-22 Thread Richard Guo
On Fri, Aug 23, 2024 at 11:19 AM Tom Lane wrote: > Richard Guo writes: > > ... we'll always make a separate ProjectionPath on top of the SortPath > > in create_ordered_paths. It’s only when we create the plan node for > > the projection step in createplan.c that we

Re: Redundant Result node

2024-08-23 Thread Richard Guo
On Fri, Aug 23, 2024 at 11:56 AM Tom Lane wrote: > Richard Guo writes: > > I agree that it’s always desirable to postpone work from path-creation > > time to plan-creation time. In this case, however, it’s a little > > different. The projection step could actually be avoid

Re: Redundant Result node

2024-08-26 Thread Richard Guo
On Mon, Aug 26, 2024 at 8:58 PM Peter Eisentraut wrote: > On 23.08.24 10:27, Richard Guo wrote: > > Fair point. After looking at the code for a while, I believe it is > > sufficient to compare PathTarget.exprs after we've checked that the > > two targe

Re: Redundant Result node

2024-08-26 Thread Richard Guo
On Thu, Aug 22, 2024 at 9:02 PM Ranier Vilela wrote: > Em qui., 22 de ago. de 2024 às 04:34, Richard Guo > escreveu: >> This does not seem right to me, as PathTargets are not canonical, so >> we cannot guarantee that two identical PathTargets will have the same >> poi

Re: Significant Execution Time Difference Between PG13.14 and PG16.4 for Query on information_schema Tables.

2024-08-27 Thread Richard Guo
On Wed, Aug 28, 2024 at 5:52 AM Tom Lane wrote: > I realized that actually we do have the mechanism for making that > work: we could apply add_nulling_relids to the expression, if it > meets those same conditions. I think this should work, as long as we apply add_nulling_relids only to Vars/PHVs

Re: Significant Execution Time Difference Between PG13.14 and PG16.4 for Query on information_schema Tables.

2024-08-27 Thread Richard Guo
On Wed, Aug 28, 2024 at 11:30 AM Richard Guo wrote: > On Wed, Aug 28, 2024 at 5:52 AM Tom Lane wrote: > > I realized that actually we do have the mechanism for making that > > work: we could apply add_nulling_relids to the expression, if it > > meets those same conditi

Re: Significant Execution Time Difference Between PG13.14 and PG16.4 for Query on information_schema Tables.

2024-08-28 Thread Richard Guo
On Wed, Aug 28, 2024 at 12:15 AM Tom Lane wrote: > If we > are willing to accept a HEAD-only fix, it'd likely be better to > attack the other end and make it possible to remove no-op PHVs. > I think that'd require marking PHVs that need to be kept because > they are serving to isolate subexpressio

Re: Significant Execution Time Difference Between PG13.14 and PG16.4 for Query on information_schema Tables.

2024-08-28 Thread Richard Guo
On Wed, Aug 28, 2024 at 7:58 AM David Rowley wrote: > On Wed, 28 Aug 2024 at 11:37, Tom Lane wrote: > > Oh, scratch that, I see you mean this is an additional way to do it > > not the only way to do it. But I'm confused why it works for > > t1.two+1 AS c1 > > but not > > t1.two+t

Re: Significant Execution Time Difference Between PG13.14 and PG16.4 for Query on information_schema Tables.

2024-08-28 Thread Richard Guo
On Thu, Aug 29, 2024 at 4:47 AM Tom Lane wrote: > In the meantime, I think this test case is mighty artificial, > and it wouldn't bother me any to just take it out again for the > time being. Yeah, I think we can remove the 't1.two+t2.two' test case if we go with your proposed patch to address th

Re: Eager aggregation, take 3

2024-08-28 Thread Richard Guo
On Fri, Aug 23, 2024 at 11:59 PM Robert Haas wrote: > Here are some initial, high-level thoughts about this patch set. Thank you for your review and feedback! It helps a lot in moving this work forward. > 1. As far as I can see, there's no real performance testing on this > thread. I expect tha

Re: Eager aggregation, take 3

2024-08-28 Thread Richard Guo
On Wed, Aug 28, 2024 at 11:57 AM Tender Wang wrote: > Rectenly, I do some benchmark tests, mainly on tpch and tpcds. > tpch tests have no plan diff, so I do not continue to test on tpch. > tpcds(10GB) tests have 22 plan diff as below: > 4.sql, 5.sql, 8.sql,11.sql,19.sql,23.sql,31.sql, > 33.sql,39

Re: Eager aggregation, take 3

2024-08-28 Thread Richard Guo
On Wed, Aug 28, 2024 at 9:01 PM Robert Haas wrote: > On Tue, Aug 27, 2024 at 11:57 PM Tender Wang wrote: > > I haven't look all of them. I just pick few simple plan test(e.g. 19.sql, > > 45.sql). > > For example, 19.sql, eager agg pushdown doesn't get large gain, but a little > > performance reg

Remove no-op PlaceHolderVars

2024-09-02 Thread Richard Guo
In [1] there was a short discussion about removing no-op PlaceHolderVars. This thread is for continuing that discussion. We may insert PlaceHolderVars when pulling up a subquery that is within the nullable side of an outer join: if subquery pullup needs to replace a subquery-referencing Var that

Re: Remove no-op PlaceHolderVars

2024-09-03 Thread Richard Guo
On Tue, Sep 3, 2024 at 11:31 AM Tom Lane wrote: > Yeah. I've been mulling over how we could do this, and the real > problem is that the expression containing the PHV *has* been fully > preprocessed by the time we get to outer join strength reduction > (cf. file header comment in prepjointree.c).

Re: Inconsistency between try_mergejoin_path and create_mergejoin_plan

2024-09-03 Thread Richard Guo
On Fri, Jul 26, 2024 at 3:56 PM Richard Guo wrote: > Do you think it works if we place this test in equivclass.sql and > write a comment explaining why it's there, like attached? Now I’m > also starting to wonder if this change actually warrants such a test. The new test case

Re: Wrong results with grouping sets

2024-09-03 Thread Richard Guo
On Tue, Aug 6, 2024 at 4:17 PM Richard Guo wrote: > I fixed this issue in v13 by performing the replacement of GROUP Vars > after we've done with expression preprocessing on targetlist and > havingQual. An ensuing effect of this approach is that a HAVING > clause may contain exp

Re: Inconsistency between try_mergejoin_path and create_mergejoin_plan

2024-09-03 Thread Richard Guo
On Tue, Sep 3, 2024 at 5:51 PM Richard Guo wrote: > The new test case fails starting from adf97c156, and we have to > install a hash opfamily and a hash function for the hacked int8alias1 > type to make the test case work again. > > Now, I'm more dubious about whether we reall

Re: Support run-time partition pruning for hash join

2024-09-06 Thread Richard Guo
On Fri, Sep 6, 2024 at 9:22 AM David Rowley wrote: > Maybe instead of inventing a very pessimistic part prune Hash Join, it > might be better to make the above work without the LATERAL + OFFSET 0 > by creating the parameterized paths Seq Scan paths. That's going to be > an immense help when the no

Re: On disable_cost

2024-09-06 Thread Richard Guo
On Fri, Sep 6, 2024 at 5:00 PM Alexander Lakhin wrote: > static void > label_sort_with_costsize(PlannerInfo *root, Sort *plan, double limit_tuples) > { > ... > cost_sort(&sort_path, root, NIL, >lefttree->total_cost, >plan->plan.disabled_nodes, >

Re: On disable_cost

2024-09-06 Thread Richard Guo
On Fri, Sep 6, 2024 at 5:27 PM Richard Guo wrote: > On Fri, Sep 6, 2024 at 5:00 PM Alexander Lakhin wrote: > > static void > > label_sort_with_costsize(PlannerInfo *root, Sort *plan, double limit_tuples) > (I'm a little surprised that this does not cause any plan diffs in

Re: On disable_cost

2024-09-08 Thread Richard Guo
On Fri, Sep 6, 2024 at 5:27 PM Richard Guo wrote: > On Fri, Sep 6, 2024 at 5:00 PM Alexander Lakhin wrote: > > static void > > label_sort_with_costsize(PlannerInfo *root, Sort *plan, double limit_tuples) > > { > > ... > > cost_sort(&sort_path, r

Why don't we consider explicit Incremental Sort?

2024-09-09 Thread Richard Guo
While looking at label_sort_with_costsize() due to another issue, I noticed that we build explicit Sort nodes but not explicit Incremental Sort nodes. I wonder why this is the case. It seems to me that Incremental Sorts are preferable in some cases. For example: create table t (a int, b int); c

Re: Wrong results with grouping sets

2024-09-09 Thread Richard Guo
On Wed, Sep 4, 2024 at 9:16 AM Richard Guo wrote: > I'm seeking the possibility to push 0001 and 0002 sometime this month. > Please let me know if anyone thinks this is unreasonable. > > For 0003, it might be extended to remove all no-op PHVs except those > that ar

Allow pushdown of HAVING clauses with grouping sets

2024-09-10 Thread Richard Guo
In some cases, we may want to transfer a HAVING clause into WHERE in hopes of eliminating tuples before aggregation instead of after. Previously, we couldn't do this if there were any nonempty grouping sets, because we didn't have a way to tell if the HAVING clause referenced any columns that were

Re: Why don't we consider explicit Incremental Sort?

2024-09-12 Thread Richard Guo
On Mon, Sep 9, 2024 at 6:22 PM Tomas Vondra wrote: > I think we intentionally added incremental sort ... incrementally ;-) Haha, right. > I think one challenge with this case is that create_mergejoin_plan > creates these Sort plans explicitly - it's not "pathified" so it doesn't > go through the

Re: Why don't we consider explicit Incremental Sort?

2024-09-13 Thread Richard Guo
On Mon, Sep 9, 2024 at 6:22 PM Tomas Vondra wrote: > I have not thought about this particular case too much, but how likely > is it that mergejoin will win for this plan in practice? If we consider > a query that only needs a fraction of rows (say, thanks to LIMIT), > aren't we likely to pick a ne

Re: Why don't we consider explicit Incremental Sort?

2024-09-13 Thread Richard Guo
On Fri, Sep 13, 2024 at 7:35 PM Tomas Vondra wrote: > On 9/13/24 06:04, Richard Guo wrote: > > It seems to me that some parts of our code assume that, for a given > > input path that is partially ordered, an incremental sort is always > > preferable to a full sort (see com

Re: Why don't we consider explicit Incremental Sort?

2024-09-13 Thread Richard Guo
On Fri, Sep 13, 2024 at 7:51 PM Tomas Vondra wrote: > Sure, an incremental sort can improve things if things go well, no doubt > about that. But how significant can the improvement be, especially if we > need to sort everything? In your example it's ~15%, which is nice, and > maybe the benefits ca

Duplicate unique key values in inheritance tables

2024-07-15 Thread Richard Guo
I came across a query that returned incorrect results and I traced it down to being caused by duplicate unique key values in an inheritance table. As a simple example, consider create table p (a int primary key, b int); create table c () inherits (p); insert into p select 1, 1; insert into c sel

Re: Wrong results with grouping sets

2024-07-16 Thread Richard Guo
On Mon, Jul 15, 2024 at 4:38 PM Sutou Kouhei wrote: > I'm not familiar with related codes but here are my > comments: Thanks for reviewing this patchset! > +* Fields valid for a GROUP RTE (else NULL/zero): > > There is only one field and it's LIST. So how about using > the following? > >

Re: Possible incorrect row estimation for Gather paths

2024-07-16 Thread Richard Guo
I can reproduce this problem with the query below. explain (costs on) select * from tenk1 order by twenty; QUERY PLAN - Gather Merge (cost=772.11..830.93 rows=5882 width=244) Wor

Re: Wrong results with grouping sets

2024-07-17 Thread Richard Guo
On Wed, Jul 17, 2024 at 8:50 AM Paul George wrote: > > Since a subquery is a volatile expression, each of its instances > should be evaluated separately. I don't think this conclusion is correct. Look at: select random(), random() from t group by random(); random | random ---

Re: Wrong results with grouping sets

2024-07-17 Thread Richard Guo
On Thu, Jul 18, 2024 at 8:31 AM Richard Guo wrote: > I am confused. Does the SQL standard explicitly define or standardize > the behavior of grouping by volatile expressions? Does anyone know > about that? Just for the record, multiple instances of non-volatile grouping expressio

Redundant code in create_gather_merge_path

2024-07-17 Thread Richard Guo
In create_gather_merge_path, we should always guarantee that the subpath is adequately ordered, and we do not add a Sort node in createplan.c for a Gather Merge node. Therefore, the 'else' branch in the snippet from create_gather_merge_path is redundant. if (pathkeys_contained_in(pathkeys, su

Re: Redundant code in create_gather_merge_path

2024-07-17 Thread Richard Guo
On Thu, Jul 18, 2024 at 10:02 AM Richard Guo wrote: > I noticed this while reviewing patch [1], thinking that it might be > worth fixing. Any thoughts? Here is the patch. Thanks Richard v1-0001-Remove-redundant-code-in-create_gather_merge_path.patch Description: Binary data

Re: Should consider materializing the cheapest inner path in consider_parallel_nestloop()

2024-07-18 Thread Richard Guo
On Thu, Jul 18, 2024 at 4:00 PM Alexander Lakhin wrote: > Please look at a recent buildfarm failure [1], which shows some > instability of that test addition: > -- the joinrel is not parallel-safe due to the OFFSET clause in the subquery > explain (costs off) > select * from tenk1 t1, (s

Re: Should consider materializing the cheapest inner path in consider_parallel_nestloop()

2024-07-18 Thread Richard Guo
On Thu, Jul 18, 2024 at 4:11 PM Richard Guo wrote: > On Thu, Jul 18, 2024 at 4:00 PM Alexander Lakhin wrote: > > Please look at a recent buildfarm failure [1], which shows some > > instability of that test addition: > > -- the joinrel is not parallel-safe due to the

Re: Should consider materializing the cheapest inner path in consider_parallel_nestloop()

2024-07-21 Thread Richard Guo
On Fri, Jul 19, 2024 at 12:00 PM Alexander Lakhin wrote: > 18.07.2024 17:30, Richard Guo wrote: > > I have no idea why the underlying statistics changed, but it seems > > that this slight change is sufficent to result in a different plan. > > I think it could be caused by

Re: Possible incorrect row estimation for Gather paths

2024-07-22 Thread Richard Guo
On Mon, Jul 22, 2024 at 4:47 PM Anthonin Bonnefoy wrote: > On Wed, Jul 17, 2024 at 3:59 AM Richard Guo wrote: > > I can reproduce this problem with the query below. > > > > explain (costs on) select * from tenk1 order by twenty; > >

Re: Possible incorrect row estimation for Gather paths

2024-07-22 Thread Richard Guo
On Mon, Jul 22, 2024 at 5:55 PM Richard Guo wrote: > Otherwise I think the v3 patch looks good to me. > > Attached is an updated version of this patch with cosmetic changes and > comment updates. It also squishes the two patches together into one. > I'm planning to push

Re: Redundant code in create_gather_merge_path

2024-07-22 Thread Richard Guo
On Thu, Jul 18, 2024 at 11:08 AM Richard Guo wrote: > On Thu, Jul 18, 2024 at 10:02 AM Richard Guo wrote: > > I noticed this while reviewing patch [1], thinking that it might be > > worth fixing. Any thoughts? > > Here is the patch. This patch is quite straightforward to

Re: Reuse child_relids in try_partitionwise_join was Re: Assert failure on bms_equal(child_joinrel->relids, child_joinrelids)

2024-07-23 Thread Richard Guo
On Wed, Jun 5, 2024 at 3:48 PM Ashutosh Bapat wrote: > This is different. But it needs a rebase. PFA rebased patch. The revised > commit message explains the change better. I looked through this patch and I think it is in a good shape. Some minor comments: * I think it's better to declare 'chi

Re: apply_scanjoin_target_to_paths and partitionwise join

2024-07-23 Thread Richard Guo
On Wed, May 22, 2024 at 3:57 PM Ashutosh Bapat wrote: > I will create patches for the back-branches once the patch for master is in a > committable state. AFAIU, this patch prevents apply_scanjoin_target_to_paths() from discarding old paths of partitioned joinrels. Therefore, we can retain non-

Re: Inconsistency between try_mergejoin_path and create_mergejoin_plan

2024-07-26 Thread Richard Guo
On Thu, Jul 25, 2024 at 12:07 AM Tom Lane wrote: > I took a brief look at this. I think the basic idea is sound, > but I have a couple of nits: Thank you for reviewing this patch! > * It's not entirely obvious that the checks preceding these additions > are sufficient to ensure that the clauses

Re: Reuse child_relids in try_partitionwise_join was Re: Assert failure on bms_equal(child_joinrel->relids, child_joinrelids)

2024-07-26 Thread Richard Guo
On Wed, Jul 24, 2024 at 3:30 PM Ashutosh Bapat wrote: > Done. Are you suggesting it for aesthetic purposes or there's > something else (read, less defragmentation, performance gain etc.)? I > am curious. I think it's just for readability. > PFA patches: > 0001 - same as previous one > 0002 - add

Re: Simplify create_merge_append_path a bit for clarity

2024-07-28 Thread Richard Guo
On Fri, Jul 26, 2024 at 1:28 PM Paul A Jungwirth wrote: > Is there a reason you don't want to remove the required_outer > parameter altogether? I guess because it is such a common pattern to > pass it? I think it's best to keep this parameter unchanged to maintain consistency with other functions

Re: Reuse child_relids in try_partitionwise_join was Re: Assert failure on bms_equal(child_joinrel->relids, child_joinrelids)

2024-07-28 Thread Richard Guo
On Fri, Jul 26, 2024 at 5:44 PM Richard Guo wrote: > I've worked a bit more on the comments and commit message, and I plan > to push the attached soon, barring any objections or comments. Pushed. Thanks Richard

Re: A problem about partitionwise join

2024-07-30 Thread Richard Guo
On Wed, May 8, 2024 at 5:01 PM Richard Guo wrote: > Below is what I got on my local machine. > > -- on master > > measurement | average | maximum | minimum | std_dev | > std_dev_as_perc_of_avg > ---+--+--+--+-+---

Re: Trivial revise for the check of parameterized partial paths

2024-07-30 Thread Richard Guo
On Thu, Jan 25, 2024 at 3:21 PM Richard Guo wrote: > On Sun, Jan 21, 2024 at 8:36 PM vignesh C wrote: >> I'm seeing that there has been no activity in this thread for nearly 7 >> months, I'm planning to close this in the current commitfest unless >> someone

Re: Short-circuit sort_inner_and_outer if there are no mergejoin clauses

2024-07-30 Thread Richard Guo
On Fri, Apr 26, 2024 at 2:57 PM Richard Guo wrote: > I doubt that there is measurable performance improvement. But I found > that throughout the run of the regression tests, sort_inner_and_outer is > called a total of 44,424 times. Among these calls, there are 11,064 > ins

Query results vary depending on the plan cache mode used

2024-08-01 Thread Richard Guo
While working on the grouping sets patches for queries with GROUP BY items that are constants, I noticed $subject on master. As an example, consider prepare q1(int) as select $1 as c1, $1 as c2 from generate_series(1,2) t group by rollup(c1); set plan_cache_mode to force_custom_plan; execute q1(

Re: Query results vary depending on the plan cache mode used

2024-08-01 Thread Richard Guo
On Thu, Aug 1, 2024 at 10:34 PM Tom Lane wrote: > Richard Guo writes: > > While working on the grouping sets patches for queries with GROUP BY > > items that are constants, I noticed $subject on master. As an > > example, consider > > This particular example seems lik

Re: Wrong results with grouping sets

2024-08-02 Thread Richard Guo
I've been looking at cases where there are grouping-set keys that reduce to Consts, and I noticed a plan with v11 patch that is not very great. explain (verbose, costs off) select 1 as one group by rollup(one) order by one nulls first; QUERY PLAN --- Sort

Re: Wrong results with grouping sets

2024-08-06 Thread Richard Guo
On Wed, Jun 5, 2024 at 5:42 PM Richard Guo wrote: > I found a bug in the v6 patch. The following query would trigger the > Assert in make_restrictinfo that the given subexpression should not be > an AND clause. > > select max(a) from t group by a > b and a = b having a > b a

Re: A problem about partitionwise join

2024-08-11 Thread Richard Guo
On Sat, Aug 10, 2024 at 6:22 AM Alexey Dvoichenkov wrote: > I haven't read the entire thread so I might be missing something, but > one interesting consequence of this patch is that it kind of breaks > the initial pruning of generic plans. Given a query such as SELECT > ... WHERE A.PK = B.PK AND A

Re: A problem about partitionwise join

2024-08-11 Thread Richard Guo
On Mon, Mar 25, 2024 at 7:09 PM Ashutosh Bapat wrote: > I think we need some way to avoid two different ways of looking up partition > keys - if we can't teach the EC machinery to produce clauses with partition > keys (always), we need to teach EC to contain partition keys in case of outer > jo

Re: A problem about partitionwise join

2020-04-07 Thread Richard Guo
On Sun, Apr 5, 2020 at 4:38 AM Tom Lane wrote: > Richard Guo writes: > > Rebased the patch with latest master and also addressed the test case > > failure reported by PostgreSQL Patch Tester. > > I looked this patch over, but I don't like it too much: it seems ver

Re: A problem about partitionwise join

2020-04-08 Thread Richard Guo
On Thu, Apr 9, 2020 at 1:07 AM Tom Lane wrote: > Richard Guo writes: > > On Sun, Apr 5, 2020 at 4:38 AM Tom Lane wrote: > >> There is already something in equivclass.c that would almost do what > >> we want here: exprs_known_equal() would tell us whether the partke

Re: weird hash plan cost, starting with pg10

2020-04-13 Thread Richard Guo
On Sat, Apr 11, 2020 at 4:11 AM Tom Lane wrote: > Konstantin Knizhnik writes: > > On 25.03.2020 13:36, Richard Guo wrote: > >> I tried this recipe on different PostgreSQL versions, starting from > >> current master and going backwards. I was able to reproduce this

Re: weird hash plan cost, starting with pg10

2020-04-13 Thread Richard Guo
On Mon, Apr 13, 2020 at 9:53 PM Tom Lane wrote: > Richard Guo writes: > > At first I was wondering if we need to check whether HashState.hashtable > > is not NULL in ExecShutdownHash() before we decide to allocate save > > space for HashState.hinstrument. And then I convinc

Re: index paths and enable_indexscan

2020-04-14 Thread Richard Guo
On Tue, Apr 14, 2020 at 2:44 PM Amit Langote wrote: > Hi, > > Maybe I am missing something obvious, but is it intentional that > enable_indexscan is checked by cost_index(), that is, *after* creating > an index path? I was expecting that if enable_indexscan is off, then > no index paths would be

Re: sqlsmith crash incremental sort

2020-04-16 Thread Richard Guo
On Mon, Apr 13, 2020 at 8:09 AM Tomas Vondra wrote: > > I've been messing with this the whole day, without much progress :-( > > I'm 99.% sure it's the same issue described by the quoted comment, > because the plan looks like this: > > Nested Loop Left Join > -> Sample Scan on pg_names

Re: sqlsmith crash incremental sort

2020-04-16 Thread Richard Guo
On Wed, Apr 15, 2020 at 10:47 PM Tomas Vondra wrote: > > Well, yeah. The problem is the Unique simply compares the columns in the > order it sees them, and it does not match the column order desired by > incremental sort. But we don't push down this information at all :-( > This is a nice optimi

Re: sqlsmith crash incremental sort

2020-04-16 Thread Richard Guo
On Thu, Apr 16, 2020 at 6:35 PM Richard Guo wrote: > > On Wed, Apr 15, 2020 at 10:47 PM Tomas Vondra < > tomas.von...@2ndquadrant.com> wrote: > >> >> Well, yeah. The problem is the Unique simply compares the columns in the >> order it sees them, and it does n

Re: sqlsmith crash incremental sort

2020-04-17 Thread Richard Guo
On Fri, Apr 17, 2020 at 2:44 AM Tomas Vondra wrote: > On Thu, Apr 16, 2020 at 12:04:03PM -0400, James Coleman wrote: > >On Thu, Apr 16, 2020 at 8:22 AM Richard Guo > wrote: > >> On Thu, Apr 16, 2020 at 6:35 PM Richard Guo > wrote: > >> > >> Attached is

Re: sqlsmith crash incremental sort

2020-04-17 Thread Richard Guo
On Fri, Apr 17, 2020 at 7:13 AM Tomas Vondra wrote: > On Thu, Apr 16, 2020 at 08:44:16PM +0200, Tomas Vondra wrote: > > > >Yeah, that's not entirely close to me. But maybe it shows us where we to > >get the unprocessed target list? > > > > I think at the very least this needs to apply the same ch

<    2   3   4   5   6   7   8   9   10   >