Re: partitioning - changing a slot's descriptor is expensive

2018-08-20 Thread David Rowley
On 20 August 2018 at 23:21, Andres Freund wrote: > On 2018-08-20 16:40:17 +0530, Amit Khandekar wrote: >> On 20 August 2018 at 14:45, Amit Langote >> wrote: >> > By the way, I think it might be a good idea to >> > try to merge this patch with the effort in the following thread. >> > >> > * Reduc

Re: partitioning - changing a slot's descriptor is expensive

2018-08-20 Thread Andres Freund
Hi, On 2018-08-20 16:40:17 +0530, Amit Khandekar wrote: > On 20 August 2018 at 14:45, Amit Langote > wrote: > > By the way, I think it might be a good idea to > > try to merge this patch with the effort in the following thread. > > > > * Reduce partition tuple routing overheads * > > https://com

Re: partitioning - changing a slot's descriptor is expensive

2018-08-20 Thread Amit Khandekar
On 20 August 2018 at 14:45, Amit Langote wrote: > Thanks for the review. > > On 2018/08/17 15:00, Amit Khandekar wrote: >> Thanks for the patch. I did some review of the patch (needs a rebase). >> Below are my comments. >> >> @@ -1936,12 +1936,11 @@ ExecPartitionCheckEmitError(ResultRelInfo >> *re

Re: partitioning - changing a slot's descriptor is expensive

2018-08-20 Thread Amit Langote
Thanks for the review. On 2018/08/17 15:00, Amit Khandekar wrote: > Thanks for the patch. I did some review of the patch (needs a rebase). > Below are my comments. > > @@ -1936,12 +1936,11 @@ ExecPartitionCheckEmitError(ResultRelInfo > *resultRelInfo, > + /* Input slot might be of a partition,

Re: partitioning - changing a slot's descriptor is expensive

2018-08-17 Thread Amit Khandekar
On 29 June 2018 at 11:53, Amit Langote wrote: > Other issues that you mentioned, such as needless heap_tuple_deform/form > being invoked, seem less localized (to me) than this particular issue, so > I created a patch for just this, which is attached with this email. I'm > thinking about how to fi

Re: partitioning - changing a slot's descriptor is expensive

2018-08-16 Thread Amit Khandekar
On 29 June 2018 at 11:53, Amit Langote wrote: > What I'm thinking of doing is something that's inspired by one of the > things that David Rowley proposes in his patch for PG 12 to remove > inefficiencies in the tuple routing code [1]. > > Instead of a single TupleTableSlot attached at partition_tu

Re: partitioning - changing a slot's descriptor is expensive

2018-06-29 Thread Amit Langote
On 2018/06/29 15:23, Amit Langote wrote: > Instead of a single TupleTableSlot attached at partition_tuple_slot, we > allocate an array of TupleTableSlot pointers of same length as the number > of partitions, as you mentioned upthread. We then call > MakeTupleTableSlot() only if a partition needs i

Re: partitioning - changing a slot's descriptor is expensive

2018-06-28 Thread Amit Langote
On 2018/06/29 14:59, Andres Freund wrote: > On 2018-06-29 11:20:53 +0530, Amit Khandekar wrote: >> On 27 June 2018 at 18:33, Ashutosh Bapat >>> But I am worried that the code will have to create thousand slots if >>> there are thousand partitions. I think we will need to see how much >>> effect tha

Re: partitioning - changing a slot's descriptor is expensive

2018-06-28 Thread Ashutosh Bapat
On Fri, Jun 29, 2018 at 11:29 AM, Andres Freund wrote: > On 2018-06-29 11:20:53 +0530, Amit Khandekar wrote: >> On 27 June 2018 at 18:33, Ashutosh Bapat >> wrote: >> > On Wed, Jun 27, 2018 at 10:39 AM, Andres Freund wrote: >> >> Unfortunately calling ExecSetSlotDescriptor() is far from cheap, it

Re: partitioning - changing a slot's descriptor is expensive

2018-06-28 Thread Andres Freund
On 2018-06-29 11:20:53 +0530, Amit Khandekar wrote: > On 27 June 2018 at 18:33, Ashutosh Bapat > wrote: > > On Wed, Jun 27, 2018 at 10:39 AM, Andres Freund wrote: > >> Unfortunately calling ExecSetSlotDescriptor() is far from cheap, it has > >> to reallocate the values/isnull arrays (and potentia

Re: partitioning - changing a slot's descriptor is expensive

2018-06-28 Thread Amit Khandekar
On 27 June 2018 at 18:33, Ashutosh Bapat wrote: > On Wed, Jun 27, 2018 at 10:39 AM, Andres Freund wrote: >> Hi, >> >> (sorry if I CCed the wrong folks, the code has changed a fair bit) >> >> I noticed that several places in the partitioning code look like: >> >> /* >> * If the tuple has

Re: partitioning - changing a slot's descriptor is expensive

2018-06-27 Thread Ashutosh Bapat
On Wed, Jun 27, 2018 at 10:39 AM, Andres Freund wrote: > Hi, > > (sorry if I CCed the wrong folks, the code has changed a fair bit) > > I noticed that several places in the partitioning code look like: > > /* > * If the tuple has been routed, it's been converted to the > * partition'

Re: partitioning - changing a slot's descriptor is expensive

2018-06-26 Thread Amit Langote
On 2018/06/27 14:55, Andres Freund wrote: > On 2018-06-27 14:46:26 +0900, Amit Langote wrote: >> There is however similar code that runs in non-error paths, such as in >> ExecPrepareTupleRouting(), that is executed *if* the leaf partition and >> the root parent have differing attribute numbers. So

Re: partitioning - changing a slot's descriptor is expensive

2018-06-26 Thread Andres Freund
On 2018-06-27 14:46:26 +0900, Amit Langote wrote: > Hi Andres, > > On 2018/06/27 14:09, Andres Freund wrote: > > Hi, > > > > (sorry if I CCed the wrong folks, the code has changed a fair bit) > > > > I noticed that several places in the partitioning code look like: > > > > /* > > * If

Re: partitioning - changing a slot's descriptor is expensive

2018-06-26 Thread Amit Langote
Hi Andres, On 2018/06/27 14:09, Andres Freund wrote: > Hi, > > (sorry if I CCed the wrong folks, the code has changed a fair bit) > > I noticed that several places in the partitioning code look like: > > /* > * If the tuple has been routed, it's been converted to the > * partition

partitioning - changing a slot's descriptor is expensive

2018-06-26 Thread Andres Freund
Hi, (sorry if I CCed the wrong folks, the code has changed a fair bit) I noticed that several places in the partitioning code look like: /* * If the tuple has been routed, it's been converted to the * partition's rowtype, which might differ from the root * table's. We must co