On Sun, Jan 24, 2021 at 6:26 PM Andy Fan wrote:
> Hi Masahiko:
>
> On Fri, Jan 22, 2021 at 9:15 PM Masahiko Sawada
> wrote:
>
>> Hi Andy,
>>
>> On Mon, Dec 7, 2020 at 9:15 PM Andy Fan wrote:
>> >
>> >
>> >
>> > On Mon, Dec 7, 2020 at 4:16 PM Jesper Pedersen <
>> jesper.peder...@redhat.com> wrot
Hi Masahiko:
On Fri, Jan 22, 2021 at 9:15 PM Masahiko Sawada
wrote:
> Hi Andy,
>
> On Mon, Dec 7, 2020 at 9:15 PM Andy Fan wrote:
> >
> >
> >
> > On Mon, Dec 7, 2020 at 4:16 PM Jesper Pedersen <
> jesper.peder...@redhat.com> wrote:
> >>
> >> Hi,
> >>
> >> On 12/5/20 10:38 PM, Andy Fan wrote:
>
Hi Andy,
On Mon, Dec 7, 2020 at 9:15 PM Andy Fan wrote:
>
>
>
> On Mon, Dec 7, 2020 at 4:16 PM Jesper Pedersen
> wrote:
>>
>> Hi,
>>
>> On 12/5/20 10:38 PM, Andy Fan wrote:
>> > Currently the UniqueKey is defined as a List of Expr, rather than
>> > EquivalenceClasses.
>> > A complete discussion
On Sun, 6 Dec 2020 at 04:10, Andy Fan wrote:
> For anyone who is interested with these patchsets, here is my plan about this
> now. 1). I will try EquivalenceClass rather than Expr in UniqueKey and add
> opfamily
> if needed.
I agree that we should be storing them in EquivalenceClasses. Apart
On Sun, Dec 6, 2020 at 9:09 AM Andy Fan wrote:
>>
>> I have not been following this thread closely enough to understand
>> why we need a new "UniqueKeys" data structure at all.
>
>
> Currently the UniqueKey is defined as a List of Expr, rather than
> EquivalenceClasses.
> A complete discussion un
On Mon, Dec 7, 2020 at 4:16 PM Jesper Pedersen
wrote:
> Hi,
>
> On 12/5/20 10:38 PM, Andy Fan wrote:
> > Currently the UniqueKey is defined as a List of Expr, rather than
> > EquivalenceClasses.
> > A complete discussion until now can be found at [1] (The messages I
> replied
> > to also
> > care
Hi,
On 12/5/20 10:38 PM, Andy Fan wrote:
Currently the UniqueKey is defined as a List of Expr, rather than
EquivalenceClasses.
A complete discussion until now can be found at [1] (The messages I replied
to also
care a lot and the information is completed). This patch has stopped at
this place fo
Thank you Tom and Heikki for your input.
On Sun, Dec 6, 2020 at 4:40 AM Tom Lane wrote:
> Heikki Linnakangas writes:
> >> I can understand why we need EquivalenceClass for UniqueKey, but I can't
> >> understand why we need opfamily here.
>
> > Thinking a bit harder, I guess we don't. Because Eq
Heikki Linnakangas writes:
>> I can understand why we need EquivalenceClass for UniqueKey, but I can't
>> understand why we need opfamily here.
> Thinking a bit harder, I guess we don't. Because EquivalenceClass
> includes the operator family already, in the ec_opfamilies field.
No. Equivalenc
On 05/12/2020 17:10, Andy Fan wrote:
Actually I can't understand this, could you explain more? Based on my
current
knowledge, when we run "SELECT DISTINCT a FROM t", we never care about
which operator to use for the unique.
SortGroupClause includes 'eqop' field, which determines the opera
Thank you Heikki for your attention.
On Mon, Nov 30, 2020 at 11:20 PM Heikki Linnakangas wrote:
> On 30/11/2020 16:30, Jesper Pedersen wrote:
> > On 11/30/20 5:04 AM, Heikki Linnakangas wrote:
> >> On 26/11/2020 16:58, Andy Fan wrote:
> >>> This patch has stopped moving for a while, any suggest
Hi
I look into the patch again and have some comments.
1.
+ Size oid_cmp_len = sizeof(Oid) * ind1->ncolumns;
+
+ return ind1->ncolumns == ind2->ncolumns &&
+ ind1->unique == ind2->unique &&
+ memcmp(ind1->indexkeys, ind2->indexkeys, sizeof(int) *
ind1->nc
On 30/11/2020 16:30, Jesper Pedersen wrote:
On 11/30/20 5:04 AM, Heikki Linnakangas wrote:
On 26/11/2020 16:58, Andy Fan wrote:
This patch has stopped moving for a while, any suggestion about
how to move on is appreciated.
The question on whether UniqueKey.exprs should be a list of
Equivalen
Hi,
On 11/30/20 5:04 AM, Heikki Linnakangas wrote:
On 26/11/2020 16:58, Andy Fan wrote:
This patch has stopped moving for a while, any suggestion about
how to move on is appreciated.
The question on whether UniqueKey.exprs should be a list of
EquivalenceClasses or PathKeys is unresolved. I
On 26/11/2020 16:58, Andy Fan wrote:
This patch has stopped moving for a while, any suggestion about
how to move on is appreciated.
The question on whether UniqueKey.exprs should be a list of
EquivalenceClasses or PathKeys is unresolved. I don't have an opinion on
that, but I'd suggest that
This patch has stopped moving for a while, any suggestion about
how to move on is appreciated.
--
Best Regards
Andy Fan
On Thu, Oct 8, 2020 at 12:12 PM Hou, Zhijie
wrote:
> Hi
>
> I have a look over this patch and find some typos in 0002.
>
> 1.Some typos about unique:
> There are some spelling mistakes about "unique" in code comments and
> README.
> Such as: "+However we define the UnqiueKey as below."
>
> 2.func
On Thu, Oct 8, 2020 at 6:39 PM Dmitry Dolgov <9erthali...@gmail.com> wrote:
> > On Thu, Oct 08, 2020 at 09:34:51AM +0800, Andy Fan wrote:
> >
> > > Other than that I wanted to ask what are the plans to proceed with this
> > > patch? It's been a while since the question was raised in which format
>
> On Thu, Oct 08, 2020 at 09:34:51AM +0800, Andy Fan wrote:
>
> > Other than that I wanted to ask what are the plans to proceed with this
> > patch? It's been a while since the question was raised in which format
> > to keep unique key expressions, and as far as I can see no detailed
> > suggestion
Hi
I have a look over this patch and find some typos in 0002.
1.Some typos about unique:
There are some spelling mistakes about "unique" in code comments and README.
Such as: "+However we define the UnqiueKey as below."
2.function name about initililze_uniquecontext_for_joinrel:
May be it should
On Wed, Oct 7, 2020 at 9:55 PM Dmitry Dolgov <9erthali...@gmail.com> wrote:
> > On Wed, Sep 09, 2020 at 07:51:12AM +0800, Andy Fan wrote:
> >
> > Thank you Michael for checking it, I can reproduce the same locally after
> > rebasing to the latest master. The attached v11 has fixed it and includes
> On Wed, Sep 09, 2020 at 07:51:12AM +0800, Andy Fan wrote:
>
> Thank you Michael for checking it, I can reproduce the same locally after
> rebasing to the latest master. The attached v11 has fixed it and includes
> the fix Floris found.
>
> The status of this patch is we are still in discussion ab
On Tue, Aug 04, 2020 at 06:59:50AM +0800, Andy Fan wrote:
> You are correct, I would include this in the next version patch, Thank you
> for this checking!
Regression tests are failing with this patch set applied. The CF bot
says so, and I can reproduce that locally as well. Could you look at
th
03 AM
> *To:* Dmitry Dolgov <9erthali...@gmail.com>
> *Cc:* David Rowley ; PostgreSQL Hackers <
> pgsql-hackers@lists.postgresql.org>; Tom Lane ;
> Ashutosh Bapat ; rushabh.lat...@gmail.com
> *Subject:* Re: [PATCH] Keeps tracking the uniqueness with UniqueKey
> [External]
>
>
>
> Fixed a test case in v10.
>
>
>
> --
>
> Best Regards
>
> Andy Fan
>
--
Best Regards
Andy Fan
n
Sent: Sunday 19 July 2020 5:03 AM
To: Dmitry Dolgov <9erthali...@gmail.com>
Cc: David Rowley ; PostgreSQL Hackers
; Tom Lane ; Ashutosh
Bapat ; rushabh.lat...@gmail.com
Subject: Re: [PATCH] Keeps tracking the uniqueness with UniqueKey [External]
Fixed a test case in v10.
--
Best Regards
Andy Fan
> On Sun, Jun 07, 2020 at 06:51:22PM +1200, David Rowley wrote:
>
> > * one in create_distinct_paths as per current implementation
> >
> > with what seems to be similar content.
>
> I think we need to have UniqueKeys in RelOptInfo so we can describe
> what a relation is unique by. There's no point
On Sat, 6 Jun 2020 at 21:15, Dmitry Dolgov <9erthali...@gmail.com> wrote:
> My concerns are more about having two different sets of distinct
> uniquekeys:
>
> * one prepared in standard_qp_callback for skip scan (I guess those
> should be added to PlannerInfo?)
Yes. Those must be set so that we
> On Fri, Jun 05, 2020 at 12:26:15PM +1200, David Rowley wrote:
> On Mon, 25 May 2020 at 19:14, Dmitry Dolgov <9erthali...@gmail.com> wrote:
> >
> > > On Mon, May 25, 2020 at 06:34:30AM +1200, David Rowley wrote:
> > > The difference will be that you'd be setting some distinct_uniquekeys
> > > in s
On Fri, Jun 5, 2020 at 10:57 AM David Rowley wrote:
> On Fri, 5 Jun 2020 at 14:36, Andy Fan wrote:
> > On Mon, May 25, 2020 at 2:34 AM David Rowley
> wrote:
> >>
> >> On Sun, 24 May 2020 at 04:14, Dmitry Dolgov <9erthali...@gmail.com>
> wrote:
> >> >
> >> > > On Fri, May 22, 2020 at 08:40:17AM
On Fri, 5 Jun 2020 at 14:36, Andy Fan wrote:
> On Mon, May 25, 2020 at 2:34 AM David Rowley wrote:
>>
>> On Sun, 24 May 2020 at 04:14, Dmitry Dolgov <9erthali...@gmail.com> wrote:
>> >
>> > > On Fri, May 22, 2020 at 08:40:17AM +1200, David Rowley wrote:
>> > > I imagine we'll set some required Un
On Mon, May 25, 2020 at 2:34 AM David Rowley wrote:
> On Sun, 24 May 2020 at 04:14, Dmitry Dolgov <9erthali...@gmail.com> wrote:
> >
> > > On Fri, May 22, 2020 at 08:40:17AM +1200, David Rowley wrote:
> > > I imagine we'll set some required UniqueKeys during
> > > standard_qp_callback()
> >
> > I
On Mon, 25 May 2020 at 19:14, Dmitry Dolgov <9erthali...@gmail.com> wrote:
>
> > On Mon, May 25, 2020 at 06:34:30AM +1200, David Rowley wrote:
> > The difference will be that you'd be setting some distinct_uniquekeys
> > in standard_qp_callback() to explicitly request that some skip scan
> > paths
> On Mon, May 25, 2020 at 06:34:30AM +1200, David Rowley wrote:
>
> > For a simple distinct query those UniqueKeys would be set based on
> > distinct clause. If I understand correctly, the very same is implemented
> > right now in create_distinct_paths, just after building all index paths,
> > so w
On Sun, 24 May 2020 at 04:14, Dmitry Dolgov <9erthali...@gmail.com> wrote:
>
> > On Fri, May 22, 2020 at 08:40:17AM +1200, David Rowley wrote:
> > I imagine we'll set some required UniqueKeys during
> > standard_qp_callback()
>
> In standard_qp_callback, because pathkeys are computed at this point
> On Fri, May 22, 2020 at 08:40:17AM +1200, David Rowley wrote:
>
> The way I picture the two working together is that the core UniqueKey
> patch adds UniqueKeys to RelOptInfos to allow us to have knowledge
> about what they're unique on based on the base relation's unique
> indexes.
Yep, I'm onbo
On Fri, May 22, 2020 at 4:52 AM David Rowley wrote:
> On Thu, 14 May 2020 at 14:39, Andy Fan wrote:
> >
> > On Thu, May 14, 2020 at 6:20 AM David Rowley
> wrote:
> >> Having the "onerow" flag was not how I intended it to work.
> >>
> > Thanks for the detailed explanation. So I think we do need
> if I understand correctly those two are introducing the concept, and
others are just using it
You are understand it correctly.
--
Best Regards
Andy Fan
On Fri, May 22, 2020 at 4:40 AM David Rowley wrote:
> On Fri, 22 May 2020 at 07:49, Dmitry Dolgov <9erthali...@gmail.com> wrote:
> > * It seems populate_baserel_uniquekeys, which actually sets uniquekeys,
> > is called after create_index_paths, where index skip scan already
> > needs to use t
On Thu, 14 May 2020 at 14:39, Andy Fan wrote:
>
> On Thu, May 14, 2020 at 6:20 AM David Rowley wrote:
>> Having the "onerow" flag was not how I intended it to work.
>>
> Thanks for the detailed explanation. So I think we do need to handle onerow
> specially, (It means more things than adding eac
On Fri, 22 May 2020 at 07:49, Dmitry Dolgov <9erthali...@gmail.com> wrote:
> * It seems populate_baserel_uniquekeys, which actually sets uniquekeys,
> is called after create_index_paths, where index skip scan already
> needs to use them. Is it possible to call it earlier?
Seems reasonable. I o
> On Tue, Apr 14, 2020 at 09:09:31PM +1200, David Rowley wrote:
>
> The infrastructure (knowing the unique properties of a RelOptInfo), as
> provided by the patch Andy has been working on, which is based on my
> rough prototype version, I believe should be used for the skip scans
> patch as well.
On Wed, May 13, 2020 at 11:48 PM Andy Fan wrote:
>
>> My impression about the one row stuff, is that there is too much
>> special casing around it. We should somehow structure the UniqueKey
>> data so that one row unique keys come naturally rather than special
>> cased. E.g every column in such a
On Thu, May 14, 2020 at 6:20 AM David Rowley wrote:
> On Thu, 14 May 2020 at 03:48, Andy Fan wrote:
> > On Wed, May 13, 2020 at 8:04 PM Ashutosh Bapat <
> ashutosh.bapat@gmail.com> wrote:
> >> My impression about the one row stuff, is that there is too much
> >> special casing around it. We
On Thu, 14 May 2020 at 03:48, Andy Fan wrote:
> On Wed, May 13, 2020 at 8:04 PM Ashutosh Bapat
> wrote:
>> My impression about the one row stuff, is that there is too much
>> special casing around it. We should somehow structure the UniqueKey
>> data so that one row unique keys come naturally ra
Hi Ashutosh:
All your suggestions are followed except the UNION ALL one. I replied the
reason
below. For the suggestions about partitioned table, looks lot of cases to
handle, so
I summarize/enrich your idea in README and email thread, we can continue
to talk about that.
>> +
>> +foreach
On Wed, May 13, 2020 at 8:04 PM Ashutosh Bapat
wrote:
> On Fri, May 8, 2020 at 7:27 AM Andy Fan wrote:
>
> >> +else if (inner_is_onerow)
> >> +{
> >> +/* Since rows in innerrel can't be duplicated AND if
> >> innerrel is onerow,
> >> + * th
On Fri, May 8, 2020 at 7:27 AM Andy Fan wrote:
>> +else if (inner_is_onerow)
>> +{
>> +/* Since rows in innerrel can't be duplicated AND if
>> innerrel is onerow,
>> + * the join result will be onerow also as well. Note:
>> onerow implies
>>
Hi Ashutosh:
Appreciate for your comments!
On Thu, May 7, 2020 at 7:26 PM Ashutosh Bapat
wrote:
> Hi Andy,
> Sorry for delay in review.
I understand no one has obligation to do that, and it must take
reviewer's time
and more, so really appreciated for it! Hope I can provide more kindly
hel
Hi Andy,
Sorry for delay in review. Your earlier patches are very large and it
requires some time to review those. I didn't finish reviewing those
but here are whatever comments I have till now on the previous set of
patches. Please see if any of those are useful to the new set.
+/*
+ * Return tr
On Wed, Apr 29, 2020 at 8:34 AM Andy Fan wrote:
>
>
> On Wed, Apr 29, 2020 at 8:29 AM David Rowley wrote:
>
>> On Thu, 16 Apr 2020 at 14:17, Andy Fan wrote:
>> > V6 also includes:
>> > 1. Fix the comment misleading you mentioned above.
>> > 2. Fixed a concern case for `relation_has_uniquekeys
On Wed, Apr 29, 2020 at 8:29 AM David Rowley wrote:
> On Thu, 16 Apr 2020 at 14:17, Andy Fan wrote:
> > V6 also includes:
> > 1. Fix the comment misleading you mentioned above.
> > 2. Fixed a concern case for `relation_has_uniquekeys_for` function.
>
> Over on [1], Richard highlights a problem
On Thu, 16 Apr 2020 at 14:17, Andy Fan wrote:
> V6 also includes:
> 1. Fix the comment misleading you mentioned above.
> 2. Fixed a concern case for `relation_has_uniquekeys_for` function.
Over on [1], Richard highlights a problem in the current join removals
lack of ability to remove left join
On Thu, Apr 16, 2020 at 8:36 PM Ashutosh Bapat
wrote:
> On Thu, Apr 16, 2020 at 7:47 AM Andy Fan wrote:
>
> > (9 rows)
> >
> > With this feature:
> > explain analyze select a, sum(c) from grp2 group by a;
> > QUERY PLAN
> >
> --
On Thu, Apr 16, 2020 at 7:47 AM Andy Fan wrote:
> (9 rows)
>
> With this feature:
> explain analyze select a, sum(c) from grp2 group by a;
> QUERY PLAN
> ---
On Wed, 15 Apr 2020 at 12:19, Andy Fan wrote:
>
>> 2. I think 0002 is overly restrictive in its demands that
>> parse->hasAggs must be false. We should be able to just use a Group
>> Aggregate with unsorted input when the input_rel is unique on the
>> GROUP BY clause. This will save on hashing an
Hi David:
Thanks for your time.
> 1. Out of date comment in join.sql
>
> -- join removal is not possible when the GROUP BY contains a column that is
> -- not in the join condition. (Note: as of 9.6, we notice that b.id is a
> -- primary key and so drop b.c_id from the GROUP BY of the resulting
On Fri, 3 Apr 2020 at 15:18, Andy Fan wrote:
> All above 4 item Done.
Just to explain my view on this going forward for PG14. I do plan to
do a more thorough review of this soon. I wasn't so keen on pursuing
this for PG13 as the skip scans patch [1] needs to use the same
infrastructure this pat
On Fri, 3 Apr 2020 at 21:56, Andy Fan wrote:
>
>
>
> On Fri, Apr 3, 2020 at 12:08 PM David Rowley wrote:
>>
>> On Fri, 3 Apr 2020 at 16:40, Tom Lane wrote:
>> > (It occurs to me BTW that we've been overly conservative about using
>> > NOT NULL constraints in planning, because of failing to consi
On Fri, Apr 3, 2020 at 12:08 PM David Rowley wrote:
> On Fri, 3 Apr 2020 at 16:40, Tom Lane wrote:
> > (It occurs to me BTW that we've been overly conservative about using
> > NOT NULL constraints in planning, because of failing to consider that.
> > Addition or drop of NOT NULL has to cause a c
On Fri, 3 Apr 2020 at 16:40, Tom Lane wrote:
> (It occurs to me BTW that we've been overly conservative about using
> NOT NULL constraints in planning, because of failing to consider that.
> Addition or drop of NOT NULL has to cause a change in
> pg_attribute.attnotnull, which will definitely caus
David Rowley writes:
> For just planner smarts such as LEFT JOIN removal, Unique Joins, and
> all this Unique Key stuff, we really don't need to record the
> dependency as if the index or constraint is dropped, then that'll
> cause a relcache invalidation and we'll see the invalidation when we
> a
On Fri, 3 Apr 2020 at 15:18, Andy Fan wrote:
>
> The updated patch should fixed all the issues. See the comments below for
> more
> information.
>
> On Tue, Mar 31, 2020 at 9:44 AM David Rowley wrote:
>>
>> + * can only guarantee the uniqueness without considering the null values.
>> This
>> +
The updated patch should fixed all the issues. See the comments below for
more
information.
On Tue, Mar 31, 2020 at 9:44 AM David Rowley wrote:
> On Sun, 29 Mar 2020 at 20:50, Andy Fan wrote:
> > Some other changes made in the new patch:
> > 1. Fixed bug for UniqueKey calculation for OUTER j
On Wed, 1 Apr 2020 at 13:45, Andy Fan wrote:
>> 5. I think you should be performing a bms_del_member during join
>> removal rather than removing this Assert()
>>
>> - Assert(bms_equal(rel->relids, root->all_baserels));
>>
>> FWIW, it's far from perfect that you've needed to delay the left join
>>
Thanks David for your time, I will acknowledge every item you mentioned
with the updated patch. Now I just talk about part of them.
> 1. There seem to be some cases where joins are no longer being
> detected as unique. This is evident in postgres_fdw.out. We shouldn't
> be regressing any of the
[ not a review, just some drive-by comments on David's comments ]
David Rowley writes:
> 2. The following change does not seem like it should be part of this
> patch. I understand you perhaps have done as you think it will
> improve the performance of checking if an expression is in a list of
>
On Sun, 29 Mar 2020 at 20:50, Andy Fan wrote:
> Some other changes made in the new patch:
> 1. Fixed bug for UniqueKey calculation for OUTER join.
> 2. Fixed some typo error in comments.
> 3. Renamed the field "grantee" as "guarantee".
I've had a look over this patch. Thank for you doing fu
>
>
>> Just update the patch which do some test case changes.
> 1.add "ANALYZE" command before running the explain.
> 2.order by with an explicit collate settings.
>
Thanks Rushabh for pointing this out, or else I'd spend much more time to
figure
out why I get a different order on Windows
Because I replied the old thread, cfbot run a test based on the old patch
on that thread. I have detached the old thread from commitfest. Reply
this
email again to wake up Mr. cfbot with the right information.
v2-0001-Maintain-the-uniqueness-of-a-Query-from-bottom-to.patch
Description: Binary
On Mon, Mar 23, 2020 at 6:21 PM Andy Fan wrote:
> Greetings.
>
> This thread is a follow-up thread for [1], where I submit a patch for
> erasing the
> distinct node if we have known the data is unique for sure. But since the
> implementation has changed a lot from the beginning and they are not
70 matches
Mail list logo
