Re: Extracting only the columns needed for a query

2020-06-30 Thread Soumyadeep Chakraborty
Hello, Melanie and me will be posting a separate thread with the scanCols patch listed here, a patch to capture the set of cols in RETURNING and a group of patches to pass down the list of cols to various table AM functions together as a patch set. This will take some time. Thus, we are deregister

Re: Extracting only the columns needed for a query

2020-06-23 Thread Melanie Plageman
On Fri, Mar 13, 2020 at 12:09 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > I've also noticed that for partitioned tables every partition is > evaluated separately. IIRC they structure cannot differ, does it > makes sense then? Good point. At some point, we had discussed only collecting the

Re: Extracting only the columns needed for a query

2020-06-18 Thread Melanie Plageman
On Fri, Mar 13, 2020 at 12:09 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > In general implemented functionality looks good. I've checked how it > works on the existing tests, almost everywhere required columns were not > missing in scanCols (which is probably the most important part). > Somet

Re: Extracting only the columns needed for a query

2020-03-13 Thread Dmitry Dolgov
> On Tue, Feb 18, 2020 at 03:26:16PM -0800, Melanie Plageman wrote: > > > > I believe it would be beneficial to add this potential API extension > > patch into > > > the thread (as an example of an interface defining how scanCols could be > > used) > > > and review them together. > > > > As for inc

Re: Extracting only the columns needed for a query

2020-02-18 Thread Melanie Plageman
On Fri, Jan 31, 2020 at 9:52 AM Melanie Plageman wrote: > I'm bumping this to the next commitfest because I haven't had a chance > to address the questions posed by Dmitry. I'm sure I'll get to it in > the next few weeks. > > > I believe it would be beneficial to add this potential API extension

Re: Extracting only the columns needed for a query

2020-02-14 Thread Pengzhou Tang
> > > On Sat, Jun 15, 2019 at 10:02 AM Tom Lane wrote: > > > > > > Another reason for having the planner do this is that presumably, in > > > an AM that's excited about this, the set of fetched columns should > > > play into the cost estimates for the scan. I've not been paying > > > enough atten

Re: Extracting only the columns needed for a query

2020-01-31 Thread Ashwin Agrawal
On Wed, Jan 15, 2020 at 7:54 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > For UPDATE, we need all of the columns in the table because of the > > table_lock() API's current expectation that the slot has all of the > > columns populated. If we want UPDATE to only need to insert the column > >

Re: Extracting only the columns needed for a query

2020-01-31 Thread Melanie Plageman
I'm bumping this to the next commitfest because I haven't had a chance to address the questions posed by Dmitry. I'm sure I'll get to it in the next few weeks. > I believe it would be beneficial to add this potential API extension patch into > the thread (as an example of an interface defining how

Re: Extracting only the columns needed for a query

2020-01-15 Thread Dmitry Dolgov
> On Thu, Jan 02, 2020 at 05:21:55PM -0800, Melanie Plageman wrote: > > That makes me think that maybe the function name, > extract_used_columns() is bad, though. Maybe extract_scan_columns()? > I tried this in the attached, updated patch. Thanks, I'll take a look at the new version. Following you

Re: Extracting only the columns needed for a query

2020-01-07 Thread Melanie Plageman
I just wanted to address a question we got about making scanCols instead of using RelOptInfo->attr_needed. David Rowley had asked: > For planning, isn't this information already available via either > targetlists or from the RelOptInfo->attr_needed array combined with > min_attr and max_attr? at

Re: Extracting only the columns needed for a query

2020-01-02 Thread Melanie Plageman
On Tue, Dec 17, 2019 at 2:57 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > Thanks for the patch! If I understand correctly from this thread, > approach B is more preferable, so I've concentrated on the patch 0001 > and have a few commentaries/questions: > Thanks so much for the review! >

Re: Extracting only the columns needed for a query

2019-12-17 Thread Dmitry Dolgov
> On Tue, Jul 16, 2019 at 06:49:10PM -0700, Melanie Plageman wrote: > > We implemented Approach B in the attached patch set (patch 0001) and > then implemented Approach A (patch 0002) to sanity check the pruned > list of columns to scan we were getting at plan-time. > We emit a notice in SeqNext()

Re: Extracting only the columns needed for a query

2019-07-17 Thread Melanie Plageman
On Sat, Jun 15, 2019 at 10:01 AM Tom Lane wrote: > Melanie Plageman writes: > > While hacking on zedstore, we needed to get a list of the columns to > > be projected--basically all of the columns needed to satisfy the > > query. The two use cases we have for this is > > 1) to pass this column li

Re: Extracting only the columns needed for a query

2019-07-16 Thread Melanie Plageman
We implemented Approach B in the attached patch set (patch 0001) and then implemented Approach A (patch 0002) to sanity check the pruned list of columns to scan we were getting at plan-time. We emit a notice in SeqNext() if the two column sets differ. Currently, for all of the queries in the regres

Re: Extracting only the columns needed for a query

2019-06-16 Thread Ashwin Agrawal
On Sat, Jun 15, 2019 at 10:02 AM Tom Lane wrote: > > Approach B: after parsing and/or after planning > > If we wanted to do something about this, making the planner record > the set of used columns seems like the thing to do. We could avoid > the expense of doing it when it's not needed by setti

Re: Extracting only the columns needed for a query

2019-06-16 Thread Corey Huinker
> > The thing that most approaches to this have fallen down on is triggers --- > that is, a trigger function might access columns mentioned nowhere in the > SQL text. (See 8b6da83d1 for a recent example :-() If you have a plan > for dealing with that, then ... > Well, if we had a trigger languag

Re: Extracting only the columns needed for a query

2019-06-15 Thread Tom Lane
Melanie Plageman writes: > While hacking on zedstore, we needed to get a list of the columns to > be projected--basically all of the columns needed to satisfy the > query. The two use cases we have for this is > 1) to pass this column list down to the AM layer for the AM to leverage it > 2) for us

Re: Extracting only the columns needed for a query

2019-06-14 Thread David Rowley
On Sat, 15 Jun 2019 at 13:46, Melanie Plageman wrote: > > While hacking on zedstore, we needed to get a list of the columns to > be projected--basically all of the columns needed to satisfy the > query. The two use cases we have for this is > 1) to pass this column list down to the AM layer for th

Extracting only the columns needed for a query

2019-06-14 Thread Melanie Plageman
While hacking on zedstore, we needed to get a list of the columns to be projected--basically all of the columns needed to satisfy the query. The two use cases we have for this is 1) to pass this column list down to the AM layer for the AM to leverage it 2) for use during planning to improving costi