Re: [HACKERS] Parallel Append implementation

2017-11-12 Thread Amit Khandekar
Thanks a lot Robert for the patch. I will have a look. Quickly tried to test some aggregate queries with a partitioned pgbench_accounts table, and it is crashing. Will get back with the fix, and any other review comments. Thanks -Amit Khandekar On 9 November 2017 at 23:44, Robert Haas wrote

Re: [HACKERS] UPDATE of partition key

2017-11-09 Thread Amit Khandekar
On 9 November 2017 at 09:27, Thomas Munro wrote: > On Wed, Nov 8, 2017 at 5:57 PM, Amit Khandekar wrote: >> On 8 November 2017 at 07:55, Thomas Munro >> wrote: >>> On Tue, Nov 7, 2017 at 8:03 AM, Robert Haas wrote: >>>> The changes to trigger.c stil

Re: [HACKERS] UPDATE of partition key

2017-11-07 Thread Amit Khandekar
patch and check if the test passes ? The patch is to be applied on the main v22 patch. If the test passes, I will include these changes (also for list_parted) in the upcoming v23 patch. Thanks -Amit Khandekar regress_locale_changes.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] UPDATE of partition key

2017-11-06 Thread Amit Khandekar
but we are adding a new int[] array that maps subplans to leaf partitions. Will get back with how it looks finally. Robert, Amit , I will get back with your other review comments. -- Thanks, -Amit Khandekar EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing

Re: [HACKERS] Parallel Append implementation

2017-10-19 Thread Amit Khandekar
On 13 October 2017 at 00:29, Robert Haas wrote: > On Wed, Oct 11, 2017 at 8:51 AM, Amit Khandekar > wrote: >> [ new patch ] > > + parallel_append > + Waiting to choose the next subplan during Parallel Append > plan > + execution. > +

Re: [HACKERS] pgsql: Avoid coercing a whole-row variable that is already coerced

2017-10-13 Thread Amit Khandekar
hack. I'm not sure whether this case ever arises currently, but the pending patch for update tuple routing will cause it to arise. Amit Khandekar Discussion: http://postgr.es/m/caj3gd9cazfppe7-wwubabpcq4_0subkipfd1+0r5_dkvnwo...@mail.gmail.com Tom Lane wrote: > Robert Haa

Re: [HACKERS] Parallel Append implementation

2017-10-11 Thread Amit Khandekar
On 9 October 2017 at 16:03, Amit Kapila wrote: > On Fri, Oct 6, 2017 at 12:03 PM, Amit Khandekar > wrote: >> On 6 October 2017 at 08:49, Amit Kapila wrote: >>> >>> Okay, but why not cheapest partial path? >> >> I gave some thought on this point.

Re: [HACKERS] Parallel Append implementation

2017-10-05 Thread Amit Khandekar
On 6 October 2017 at 08:49, Amit Kapila wrote: > On Thu, Oct 5, 2017 at 4:11 PM, Amit Khandekar wrote: >> >> Ok. How about removing pa_all_partial_subpaths altogether , and >> instead of the below condition : >> >> /* >> * If all the child rels have p

Re: [HACKERS] Parallel Append implementation

2017-10-05 Thread Amit Khandekar
On 30 September 2017 at 19:21, Amit Kapila wrote: > On Wed, Sep 20, 2017 at 10:59 AM, Amit Khandekar > wrote: >> On 16 September 2017 at 10:42, Amit Kapila wrote: >>> >>> At a broader level, the idea is good, but I think it won't turn out >>> exactl

Re: [HACKERS] UPDATE of partition key

2017-10-03 Thread Amit Khandekar
On 30 September 2017 at 01:26, Robert Haas wrote: > On Fri, Sep 29, 2017 at 3:53 PM, Robert Haas wrote: >> On Fri, Sep 22, 2017 at 1:57 AM, Amit Khandekar >> wrote: >>> The patch for the above change is : >>> 0002-Prevent-a-redundant-ConvertRowtypeExpr-node.

Re: [HACKERS] Parallel Append implementation

2017-09-28 Thread Amit Khandekar
On 20 September 2017 at 11:32, Amit Khandekar wrote: > There is still the open point being > discussed : whether to have non-parallel-aware partial Append path, or > always have parallel-aware Append paths. Attached is the revised patch v16. In previous versions, we used to add a non

Re: [HACKERS] UPDATE of partition key

2017-09-28 Thread Amit Khandekar
ey, with all rows moved to other partitions. There are no indexed columns. UPDATE query used : UPDATE ptab set a = a + '2 years'::interval ; [1] https://drive.google.com/open?id=0B_YJCqIAxKjeN3hMXzdDejlNYmlpWVJpaU9mWUhFRVhXTG5Z -- Thanks, -Amit Khandekar EnterpriseDB Corporation The Po

Re: [HACKERS] UPDATE of partition key

2017-09-22 Thread Amit Khandekar
On 21 September 2017 at 19:52, amul sul wrote: > On Wed, Sep 20, 2017 at 9:27 PM, Amit Khandekar > wrote: >> >> On 20 September 2017 at 00:06, Robert Haas wrote: >> > On Fri, Sep 15, 2017 at 7:25 AM, Amit Khandekar >> > wrote: >> >

Re: [HACKERS] UPDATE of partition key

2017-09-20 Thread Amit Khandekar
On 20 September 2017 at 00:06, Robert Haas wrote: > On Fri, Sep 15, 2017 at 7:25 AM, Amit Khandekar > wrote: >> [ new patch ] > > This already fails to apply again. In general, I think it would be a > good idea to break this up into a patch series rather than have it as &g

Re: [HACKERS] Parallel Append implementation

2017-09-19 Thread Amit Khandekar
; add_paths_to_append_rel() > Added comments. Attached revised patch v15. There is still the open point being discussed : whether to have non-parallel-aware partial Append path, or always have parallel-aware Append paths. -- Thanks, -Amit Khandekar EnterpriseDB Corporation The Postgres Database Company ParallelAppend_v15.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Parallel Append implementation

2017-09-19 Thread Amit Khandekar
chooses different next plan for leader or worker, then that logic would most probably get complicated than the current state. Also, I don't see any performance issue if there is a leader is running backwards while the others are going forwards. -- Thanks, -Amit Khandekar EnterpriseDB Corp

Re: [HACKERS] UPDATE of partition key

2017-09-19 Thread Amit Khandekar
; > +} PartitionWalker; > + > > Same as above Yes, this was left out from the earlier implementation. Will have this removed in the next updated patch. -- Thanks, -Amit Khandekar EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Parallel Append implementation

2017-09-17 Thread Amit Khandekar
On 16 September 2017 at 11:45, Amit Kapila wrote: > On Thu, Sep 14, 2017 at 8:30 PM, Amit Khandekar > wrote: >> On 11 September 2017 at 18:55, Amit Kapila wrote: >>>> >>> >>> How? See, if you have four partial subpaths and two non-partial >&

Re: [HACKERS] Parallel Append implementation

2017-09-14 Thread Amit Khandekar
n non-parallel-aware partial Append is out of question, which we both agree. And the other case where we skip non-parallel-aware partial Append is when all the cheapest subpaths of the parallel-aware Append path are partial paths: we do not want parallel-aware and non-parallel-aware Append paths b

Re: [HACKERS] expanding inheritance in partition bound order

2017-09-14 Thread Amit Khandekar
ied that the leaf oids are ordered exaclty in the order of the UPDATE subplans output. -- Thanks, -Amit Khandekar EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.pos

Re: [HACKERS] expanding inheritance in partition bound order

2017-09-13 Thread Amit Khandekar
later for some reason, we can do that incrementally. Will review your latest patch by tomorrow. -- Thanks, -Amit Khandekar EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-13 Thread Amit Khandekar
On 13 September 2017 at 13:05, Ashutosh Bapat wrote: > On Wed, Sep 13, 2017 at 12:32 PM, Amit Khandekar > wrote: >> Hi, >> >> Rafia had done some testing on TPCH queries using Partition-wise join >> patch along with Parallel Append patch. >> >> The

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-13 Thread Amit Khandekar
Hi, Rafia had done some testing on TPCH queries using Partition-wise join patch along with Parallel Append patch. There, we had observed that for query 4, even though the partition wise joins are under a Parallel Append, the join are all non-partial. Specifically, the partition-wise join has non

Re: [HACKERS] why not parallel seq scan for slow functions

2017-09-12 Thread Amit Khandekar
tkind == RELOPT_BASEREL && root->simple_rel_array_size > 2) + generate_gather_paths(root, rel, NULL); Above, in case it's a partitioned table, root->simple_rel_array_size includes the child rels. So even if it's a simple select without a join rel, simple_rel_a

Re: [HACKERS] UPDATE of partition key

2017-09-12 Thread Amit Khandekar
On 12 September 2017 at 11:57, Dilip Kumar wrote: > On Tue, Sep 12, 2017 at 11:15 AM, Amit Khandekar > wrote: > >> But the statement level trigger function can refer to OLD TABLE and >> NEW TABLE, which will contain all the OLD rows and NEW rows >> respectively.

Re: [HACKERS] UPDATE of partition key

2017-09-11 Thread Amit Khandekar
On 11 September 2017 at 21:12, Dilip Kumar wrote: > On Thu, Sep 7, 2017 at 11:41 AM, Amit Khandekar > wrote: >> On 6 September 2017 at 21:47, Dilip Kumar wrote: > >> Actually, since transition tables came in, the functions like >> ExecARUpdateTriggers() or ExecA

Re: [HACKERS] Parallel Append implementation

2017-09-11 Thread Amit Khandekar
On 8 September 2017 at 19:17, Amit Kapila wrote: > On Fri, Sep 8, 2017 at 3:59 PM, Amit Khandekar wrote: >> On 7 September 2017 at 11:05, Amit Kapila wrote: >>> On Thu, Aug 31, 2017 at 12:47 PM, Amit Khandekar

Re: [HACKERS] expanding inheritance in partition bound order

2017-09-11 Thread Amit Khandekar
ponding to the internal partitions are multiplied by" anywhere in the patch. I think those comments are still valid, and important. -- Thanks, -Amit Khandekar EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Parallel Append implementation

2017-09-08 Thread Amit Khandekar
On 7 September 2017 at 11:05, Amit Kapila wrote: > On Thu, Aug 31, 2017 at 12:47 PM, Amit Khandekar > wrote: >> The last updated patch needs a rebase. Attached is the rebased version. >> > > Few comments on the first read of the patch: Thanks ! > >

Re: [HACKERS] UPDATE of partition key

2017-09-07 Thread Amit Khandekar
On 3 September 2017 at 17:10, Amit Khandekar wrote: > After recent commit 30833ba154, now the partitions are expanded in > depth-first order. It didn't seem worthwhile rebasing my partition > walker changes onto the latest code. So in the attached patch, I have > removed all the

Re: [HACKERS] Parallel Append implementation

2017-09-07 Thread Amit Khandekar
On 7 September 2017 at 13:40, Rafia Sabih wrote: > On Wed, Aug 30, 2017 at 5:32 PM, Amit Khandekar > wrote: >> Hi Rafia, >> >> On 17 August 2017 at 14:12, Amit Khandekar wrote: >>> But for all of the cases here, partial >>> subplans seem possibl

Re: [HACKERS] Parallel Append implementation

2017-09-05 Thread Amit Khandekar
On 30 August 2017 at 17:32, Amit Khandekar wrote: > On 16 August 2017 at 18:34, Robert Haas wrote: >> Thanks for the benchmarking results! >> >> On Tue, Aug 15, 2017 at 11:35 PM, Rafia Sabih >> wrote: >>> Q4 | 244 | 12 | PA and PWJ, time by only PWJ - 41 &g

Re: [HACKERS] expanding inheritance in partition bound order

2017-09-04 Thread Amit Khandekar
On 4 September 2017 at 06:34, Amit Langote wrote: > Hi Amit, > > On 2017/09/03 16:07, Amit Khandekar wrote: >> On 31 August 2017 at 13:06, Amit Langote >> wrote: >>>> Mind you, that idea has some problems anyway in the face of default >>>> partition

Re: [HACKERS] UPDATE of partition key

2017-09-03 Thread Amit Khandekar
On 31 August 2017 at 14:15, Amit Khandekar wrote: > Thanks Dilip. I am working on rebasing the patch. Particularly, the > partition walker in my patch depended on the fact that all the tables > get opened (and then closed) while creating the tuple routing info. > But in HEAD,

Re: [HACKERS] expanding inheritance in partition bound order

2017-09-03 Thread Amit Khandekar
On 31 August 2017 at 13:06, Amit Langote wrote: >> Mind you, that idea has some problems anyway in the face of default >> partitions, null partitions, and list partitions which accept >> non-contiguous values (e.g. one partition for 1, 3, 5; another for 2, >> 4, 6). We might need to mark the Part

Re: [HACKERS] UPDATE of partition key

2017-08-31 Thread Amit Khandekar
-b906-dec040a206f2%40lab.ntt.co.jp On 31 August 2017 at 12:09, Dilip Kumar wrote: > On Fri, Aug 11, 2017 at 10:44 AM, Amit Khandekar > wrote: >> On 4 August 2017 at 22:28, Amit Khandekar wrote: >>>> > > I am planning to review and test this patch, Seems like th

Re: [HACKERS] Parallel Append implementation

2017-08-31 Thread Amit Khandekar
The last updated patch needs a rebase. Attached is the rebased version. Thanks -Amit Khandekar ParallelAppend_v13_rebased_3.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] Parallel Append implementation

2017-08-30 Thread Amit Khandekar
Hi Rafia, On 17 August 2017 at 14:12, Amit Khandekar wrote: > But for all of the cases here, partial > subplans seem possible, and so even on HEAD it executed Partial > Append. So between a Parallel Append having partial subplans and a > Partial Append having partial subplan

Re: [HACKERS] expanding inheritance in partition bound order

2017-08-30 Thread Amit Khandekar
On 25 August 2017 at 23:58, Robert Haas wrote: > > That just leaves indexes. In a world where keystate, tupslot, and > tupmap are removed from the PartitionDispatchData, you must need > indexes or there would be no point in constructing a > PartitionDispatchData object in the first place; any app

Re: [HACKERS] expanding inheritance in partition bound order

2017-08-17 Thread Amit Khandekar
use it's essentially free to create > while we are walking the partition tree in > RelationGetPartitionDispatchInfo() and it seems undesirable to make the > caller compute that information (indexes) by traversing the partition tree > all over again, if it doesn't otherwise ha

Re: [HACKERS] expanding inheritance in partition bound order

2017-08-17 Thread Amit Khandekar
o just generate oids, and keep RelationGetPartitionDispatchInfo() intact, to be used only for tuple routing. But, I haven't yet checked Ashuthosh's requirements, which suggest that it does not help to even get the oid list. > > -- > Robert Haas > EnterpriseDB: http://www.ent

Re: [HACKERS] expanding inheritance in partition bound order

2017-08-17 Thread Amit Khandekar
On 17 August 2017 at 06:39, Amit Langote wrote: > Hi Amit, > > Thanks for the comments. > > On 2017/08/16 20:30, Amit Khandekar wrote: >> On 16 August 2017 at 11:06, Amit Langote >> wrote: >> >>> Attached updated patches. >> >> Thanks Am

Re: [HACKERS] Parallel Append implementation

2017-08-17 Thread Amit Khandekar
t be due to some other reasons. I will investigate this, and the other queries. -- Thanks, -Amit Khandekar EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] expanding inheritance in partition bound order

2017-08-16 Thread Amit Khandekar
the partition tree using these descriptors similar to how it is traversed in RelationGetPartitionDispatchInfo() ? May be to avoid code duplication for traversing, we can have a common API. Still looking at RelationGetPartitionDispatchInfo() changes ... -- Thanks, -Amit Khandekar EnterpriseDB Co

Re: [HACKERS] Parallel Append implementation

2017-08-09 Thread Amit Khandekar
On 9 August 2017 at 19:05, Robert Haas wrote: > On Wed, Jul 5, 2017 at 7:53 AM, Amit Khandekar wrote: >>> This is not applicable on the latest head i.e. commit -- >>> 08aed6604de2e6a9f4d499818d7c641cbf5eb9f7, looks like need a rebasing. >> >> Thanks for notifying

Re: [HACKERS] expanding inheritance in partition bound order

2017-08-06 Thread Amit Khandekar
eparing the ModifyTable plan; the PartitionDispatch data structure returned by RelationGetPartitionDispatchInfo() should be stored in that plan, and then the execution-time fields in PartitionDispatch would be populated in ExecInitModifyTable(). -- Thanks, -Amit Khandekar EnterpriseDB Corporation The Post

Re: [HACKERS] UPDATE of partition key

2017-08-04 Thread Amit Khandekar
> > Below are the TODOS at this point : > > Fix for bug reported by Rajkumar about update with join. I had explained the root issue of this bug here : [1] Attached patch includes the fix, which is explained below. Currently in the patch, there is a check if the tuple is concurrently deleted by ot

Re: [HACKERS] map_partition_varattnos() and whole-row vars

2017-08-02 Thread Amit Khandekar
On 3 August 2017 at 11:00, Amit Langote wrote: > Thanks for the review. > > On 2017/08/03 13:54, Amit Khandekar wrote: >> On 2 August 2017 at 11:51, Amit Langote wrote: >>> On 2017/08/02 1:33, Amit Khandekar wrote: >>>> Instead of callers of map_partition_vara

Re: [HACKERS] map_partition_varattnos() and whole-row vars

2017-08-02 Thread Amit Khandekar
On 2 August 2017 at 11:51, Amit Langote wrote: > Thanks Fuita-san and Amit for reviewing. > > On 2017/08/02 1:33, Amit Khandekar wrote: >> On 1 August 2017 at 15:11, Etsuro Fujita wrote: >>> On 2017/07/31 18:56, Amit Langote wrote: >>>> Yes, that's

Re: [HACKERS] UPDATE of partition key

2017-08-02 Thread Amit Khandekar
On 2 August 2017 at 14:38, Amit Langote wrote: > On 2017/07/29 2:45, Amit Khandekar wrote: >> On 28 July 2017 at 20:10, Robert Haas wrote: >>> On Wed, Jul 26, 2017 at 2:13 AM, Amit Langote wrote: >>>> I checked that we get the same result relation order with both

Re: [HACKERS] map_partition_varattnos() and whole-row vars

2017-08-01 Thread Amit Khandekar
ence here */ + if (found_whole_row) + elog(ERROR, "unexpected whole-row reference found in partition key"); Instead of callers of map_partition_varattnos() reporting error, we can have map_partition_varattnos() itself report error. Instead of the found_whole_row parameter of map_partition_varattnos(), we can have error_on_whole_row parameter. So callers who don't expect whole row, would pass error_on_whole_row=true to map_partition_varattnos(). This will simplify the resultant code a bit. -- Thanks, -Amit Khandekar EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] UPDATE of partition key

2017-07-28 Thread Amit Khandekar
not update) while inserting a routed tuple. Use getASTriggerResultRelInfo() for attrno mapping, rather than first resultrel, for generating child WCO/RETURNING expression. Address Robert's review comments on make_resultrel_ordered.patch. pgindent. [1] https://www.postgresql.org/message-id/d86d27e

Re: [HACKERS] map_partition_varattnos() and whole-row vars

2017-07-28 Thread Amit Khandekar
ons (and I think even RETURNING) can have a subquery. > > Thanks, > Amit > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers > -- Thanks, -Amit Khandekar EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] UPDATE of partition key

2017-07-26 Thread Amit Khandekar
artitionDispatchInfo(). > On another note, did you do anything about the suggestion Thomas made > in > http://postgr.es/m/CAEepm=3sc_j1zwqdyrbu4dtfx5rhcamnnuaxrkwzfgt9m23...@mail.gmail.com > ? This is still pending on me; plus I think there are some more points. I need to go over those an

Re: [HACKERS] UPDATE of partition key

2017-07-25 Thread Amit Khandekar
On 25 July 2017 at 15:02, Rajkumar Raghuwanshi wrote: > On Mon, Jul 24, 2017 at 11:23 AM, Amit Khandekar > wrote: >> >> >> Attached update-partition-key_v13.patch now contains this >> make_resultrels_ordered.patch changes. >> > > I have appl

Re: [HACKERS] Oddity in error handling of constraint violation in ExecConstraints for partitioned tables

2017-07-24 Thread Amit Khandekar
On 24 July 2017 at 12:11, Amit Langote wrote: > Hi Amit, > > On 2017/07/24 14:09, Amit Khandekar wrote: >>>> On 2017/07/10 14:15, Etsuro Fujita wrote: >>>> Another thing I noticed is the error handling in ExecWithCheckOptions; it >>>> doesn't

Re: [HACKERS] UPDATE of partition key

2017-07-23 Thread Amit Khandekar
On 13 July 2017 at 22:39, Amit Khandekar wrote: > Attached is a WIP patch (make_resultrels_ordered.patch) that generates > the result rels in canonical order. This patch is kept separate from > the update-partition-key patch, and can be applied on master branch. Attached update-partitio

Re: [HACKERS] Oddity in error handling of constraint violation in ExecConstraints for partitioned tables

2017-07-23 Thread Amit Khandekar
ws junk value in one of the columns of the row in the error message emitted for the WithCheckOption violation. Thanks, -Amit Khandekar EnterpriseDB Corporation The Postgres Database Company set_slot_descriptor.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] UPDATE of partition key

2017-07-13 Thread Amit Khandekar
On 5 July 2017 at 15:12, Amit Khandekar wrote: > Like I mentioned upthread... in expand_inherited_rtentry(), if we > replace find_all_inheritors() with something else that returns oids in > canonical order, that will change the order in which children tables > get locked, which i

Re: [HACKERS] Parallel Append implementation

2017-07-05 Thread Amit Khandekar
On 30 June 2017 at 15:10, Rafia Sabih wrote: > > On Tue, Apr 4, 2017 at 12:37 PM, Amit Khandekar > wrote: >> >> Attached is an updated patch v13 that has some comments changed as per >> your review, and also rebased on latest master. > > > This is not appli

Re: [HACKERS] UPDATE of partition key

2017-07-05 Thread Amit Khandekar
On 4 July 2017 at 15:23, Amit Khandekar wrote: > On 4 July 2017 at 14:48, Amit Khandekar wrote: >> On 4 July 2017 at 14:38, Amit Langote wrote: >>> On 2017/07/04 17:25, Etsuro Fujita wrote: >>>> On 2017/07/03 18:54, Amit Langote wrote: >>>>> On 201

Re: [HACKERS] UPDATE of partition key

2017-07-04 Thread Amit Khandekar
On 4 July 2017 at 14:48, Amit Khandekar wrote: > On 4 July 2017 at 14:38, Amit Langote wrote: >> On 2017/07/04 17:25, Etsuro Fujita wrote: >>> On 2017/07/03 18:54, Amit Langote wrote: >>>> On 2017/07/02 20:10, Robert Haas wrote: >>>>> T

Re: [HACKERS] UPDATE of partition key

2017-07-04 Thread Amit Khandekar
ely. find_inheritance_children() needs to return the oids in canonical order. So in find_inheritance_children () need to re-use part of RelationBuildPartitionDesc() where it generates those oids in that order. I am checking this part, and am going to come up with an approach based on findings.

Re: [HACKERS] UPDATE of partition key

2017-06-29 Thread Amit Khandekar
approach above: If we don't find any updated partition-keys in any of them, well and good. If we do find, failover to approach 3 : For each of the update resultrels, use the new rd_partcheckattrs bitmap to know if it uses any of the updated columns. This would be faster than pulling up at

Re: [HACKERS] UPDATE of partition key

2017-06-29 Thread Amit Khandekar
On 29 June 2017 at 07:42, Amit Langote wrote: > Hi Amit, > > On 2017/06/28 20:43, Amit Khandekar wrote: >> In attached patch v12 > > The patch no longer applies and fails to compile after the following > commit was made yesterday: > > commit 501ed02cf6f4f60c335

Re: [HACKERS] UPDATE of partition key

2017-06-28 Thread Amit Khandekar
On 22 June 2017 at 01:57, Robert Haas wrote: > On Wed, Jun 21, 2017 at 1:38 PM, Amit Khandekar > wrote: >>>> Yep, it's more appropriate to use >>>> ModifyTableState->rootResultRelationInfo->ri_RelationDesc somehow. That >>>> is, if answer t

Re: [HACKERS] UPDATE of partition key

2017-06-28 Thread Amit Khandekar
On 26 June 2017 at 08:37, Amit Khandekar wrote: > On 22 June 2017 at 01:41, Robert Haas wrote: >>>> Second, it will amount to a functional bug if you get a >>>> different answer than the planner did. >>> >>> Actually, the per-leaf WCOs are meant to

Re: [HACKERS] UPDATE of partition key

2017-06-25 Thread Amit Khandekar
On 22 June 2017 at 01:41, Robert Haas wrote: >>> Second, it will amount to a functional bug if you get a >>> different answer than the planner did. >> >> Actually, the per-leaf WCOs are meant to be executed on the >> destination partitions where the tuple is moved, while the WCOs >> belonging to t

Re: [HACKERS] UPDATE of partition key

2017-06-21 Thread Amit Khandekar
gt;>> for some reason, the comments don't explain what that reason is. >> >> Yep, it's more appropriate to use >> ModifyTableState->rootResultRelationInfo->ri_RelationDesc somehow. That >> is, if answer to the question I raised above is positive. >Fro

Re: [HACKERS] UPDATE of partition key

2017-06-21 Thread Amit Khandekar
On 21 June 2017 at 00:23, Robert Haas wrote: > On Tue, Jun 20, 2017 at 2:54 AM, Amit Khandekar > wrote: >>> I guess I don't see why it should work like this. In the INSERT case, >>> we must build withCheckOption objects for each partition because those >>&g

Re: [HACKERS] UPDATE of partition key

2017-06-19 Thread Amit Khandekar
On 20 June 2017 at 03:46, Robert Haas wrote: > On Thu, Jun 15, 2017 at 1:36 PM, Amit Khandekar > wrote: >> Attached patch v10 fixes the above. In the existing code, where it >> builds WCO constraints for each leaf partition; with the patch, that >> code now is applicable

Re: [HACKERS] UPDATE of partition key

2017-06-19 Thread Amit Khandekar
) we jump the current map position for each successive subplan, whereas in my patch, in ExecInsert() we deduce the position of the right map to be fetched using the position of the current resultRelInfo in the mtstate->resultRelInfo[] array. I think your way is more consistent with the existing c

Re: [HACKERS] UPDATE of partition key

2017-06-18 Thread Amit Khandekar
palloc0(sizeof(TupleConversionMap*) * nplans); On 15 June 2017 at 23:06, Amit Khandekar wrote: > On 13 June 2017 at 15:40, Amit Khandekar wrote: >> While rebasing my patch for the below recent commit, I realized that a >> similar issue exists for the

Re: [HACKERS] UPDATE of partition key

2017-06-15 Thread Amit Khandekar
On 13 June 2017 at 15:40, Amit Khandekar wrote: > While rebasing my patch for the below recent commit, I realized that a > similar issue exists for the uptate-tuple-routing patch as well : > > commit 78a030a441966d91bc7e932ef84da39c3ea7d970 > Author: Tom Lane > Date: Mon Ju

Re: [HACKERS] UPDATE of partition key

2017-06-13 Thread Amit Khandekar
While rebasing my patch for the below recent commit, I realized that a similar issue exists for the uptate-tuple-routing patch as well : commit 78a030a441966d91bc7e932ef84da39c3ea7d970 Author: Tom Lane Date: Mon Jun 12 23:29:44 2017 -0400 Fix confusion about number of subplans in partition

Re: [HACKERS] UPDATE of partition key

2017-06-09 Thread Amit Khandekar
use? >>> >>> I think before doing above check we can simply check if ctid.ip_blkid >>> contains InvalidBlockNumber, then return an error. >> >> Hmm, OK. That case never happens today? >> > > As per my understanding that case doesn&#

Re: [HACKERS] strcmp() tie-breaker for identical ICU-collated strings

2017-06-08 Thread Amit Khandekar
On 2 June 2017 at 23:52, Peter Geoghegan wrote: > On Fri, Jun 2, 2017 at 10:34 AM, Amit Khandekar > wrote: >> Ok. I was thinking we are doing the tie-breaker because specifically >> strcoll_l() was unexpectedly returning 0 for some cases. Now I get it, >> that we do th

Re: [HACKERS] UPDATE of partition key

2017-06-08 Thread Amit Khandekar
On 7 June 2017 at 20:19, Amit Khandekar wrote: > On 7 June 2017 at 16:42, Amit Khandekar wrote: >> The column bitmap set returned by GetUpdatedColumns() refer to >> attribute numbers w.r.t. to the root partition. And the >> mstate->resultRelInfo[] have attnos w.r.t. to

Re: [HACKERS] UPDATE of partition key

2017-06-07 Thread Amit Khandekar
On 7 June 2017 at 16:42, Amit Khandekar wrote: > The column bitmap set returned by GetUpdatedColumns() refer to > attribute numbers w.r.t. to the root partition. And the > mstate->resultRelInfo[] have attnos w.r.t. to the leaf partitions. So > we need to do somet

Re: [HACKERS] UPDATE of partition key

2017-06-07 Thread Amit Khandekar
On 6 June 2017 at 23:52, Robert Haas wrote: > On Fri, Jun 2, 2017 at 7:07 AM, Amit Khandekar wrote: >> So, according to that, below would be the logic : >> >> Run partition constraint check on the original NEW row. >> If it succeeds : >> { >> Fire BR U

Re: [HACKERS] UPDATE of partition key

2017-06-04 Thread Amit Khandekar
On 5 June 2017 at 11:27, Amit Kapila wrote: > On Fri, Jun 2, 2017 at 4:37 PM, Amit Khandekar wrote: >> On 2 June 2017 at 01:17, Robert Haas wrote: >>> On Thu, Jun 1, 2017 at 7:41 AM, Amit Khandekar >>> wrote: >>>>> Regarding the trigger issu

Re: [HACKERS] strcmp() tie-breaker for identical ICU-collated strings

2017-06-02 Thread Amit Khandekar
er, so as to be 100% conformant to ICU. I can't come up with an example, or may there isn't one, but , say , there is a locale which is supposed to sort only by lowest comparison strength (de@strength=1 ?? ). In that case, there might be many characters considered equal, but PG < op

Re: [HACKERS] UPDATE of partition key

2017-06-02 Thread Amit Khandekar
On 2 June 2017 at 01:17, Robert Haas wrote: > On Thu, Jun 1, 2017 at 7:41 AM, Amit Khandekar wrote: >>> Regarding the trigger issue, I can't claim to have a terribly strong >>> opinion on this. I think that practically anything we do here might >>> upset

[HACKERS] strcmp() tie-breaker for identical ICU-collated strings

2017-06-01 Thread Amit Khandekar
ified that strcoll() returns true for 1FFD > 0384. So, it looks like ICU API function ucol_strcoll() returns false by intention. That's the reason I feel like the strcmp-if-strtoll-returns-0 thing might not be applicable for ICU. But I may be wrong, please correct me if I may be mi

Re: [HACKERS] UPDATE of partition key

2017-06-01 Thread Amit Khandekar
if (concurrently_deleted) > +return NULL; > > I don't understand the motivation for this change, and there are no > comments explaining it that I can see. Yeah comments, I think, are missing. I thought in the ExecDelete() they are there, but they are not. If a

Re: [HACKERS] UPDATE of partition key

2017-05-28 Thread Amit Khandekar
On 24 May 2017 at 20:16, Amit Kapila wrote: > On Wed, May 24, 2017 at 8:14 PM, Amit Kapila wrote: >> Apart from above, there is one open issue [1] >> > > Forget to mention the link, doing it now. > > [1] - > https://www.postgresql.org/message-id/CAA4eK1KEZQ%2BCyXbBzfn1jFHoEfa_OemDLhLyy7xfD1QUZLo

Re: [HACKERS] UPDATE of partition key

2017-05-24 Thread Amit Khandekar
On 12 May 2017 at 09:27, Amit Kapila wrote: > > + is_partitioned_table = > + root_rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE; > + > + if (is_partitioned_table) > + ExecSetupPartitionTupleRouting( > + root_rel, > + /* Build WITH CHECK OPTION constraints for leaf partitions */ > + ExecInitPar

Re: [HACKERS] UPDATE of partition key

2017-05-24 Thread Amit Khandekar
not firing update triggers on any of the partitions. So, I prefer option 2 over option 3 , i.e. make sure to fire BR and AR update triggers. Actually option 2 is what Robert had proposed in the beginning. -- Thanks, -Amit Khandekar EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] UPDATE of partition key

2017-05-17 Thread Amit Khandekar
On 17 May 2017 at 17:29, Rushabh Lathia wrote: > > > On Wed, May 17, 2017 at 12:06 PM, Dilip Kumar wrote: >> >> On Fri, May 12, 2017 at 4:17 PM, Amit Khandekar >> wrote: >> > Option 3 >> > >> > >> > BR, AR delete triggers

Re: [HACKERS] UPDATE of partition key

2017-05-12 Thread Amit Khandekar
On 12 May 2017 at 14:56, Amit Kapila wrote: > I think it might be better to summarize all the options discussed > including what the patch has and see what most people consider as > sensible. Yes, makes sense. Here are the options that were discussed so far for ROW triggers : Option 1 : (the pat

Re: [HACKERS] UPDATE of partition key

2017-05-11 Thread Amit Khandekar
non-default partition, it moves into that partition. I think we can debate on whether the row should stay in the default partition or move. I think it should be moved, since now the row has a suitable partition. -- Thanks, -Amit Khandekar EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] UPDATE of partition key

2017-05-11 Thread Amit Khandekar
On 12 May 2017 at 08:30, Amit Kapila wrote: > On Thu, May 11, 2017 at 5:41 PM, Amit Khandekar > wrote: >> On 11 May 2017 at 17:23, Amit Kapila wrote: >>> On Fri, Mar 17, 2017 at 4:07 PM, Amit Khandekar >>> wrote: >>>> On 4 March 2017 at 12:49, Robert

Re: [HACKERS] UPDATE of partition key

2017-05-11 Thread Amit Khandekar
On 12 May 2017 at 10:01, Amit Kapila wrote: > On Fri, May 12, 2017 at 9:27 AM, Amit Kapila wrote: >> On Thu, May 11, 2017 at 5:45 PM, Amit Khandekar >> wrote: >>> On 11 May 2017 at 17:24, Amit Kapila wrote: >>>> Few comments: >>>> 1. >>&

Re: [HACKERS] UPDATE of partition key

2017-05-11 Thread Amit Khandekar
r will occur. > > Doesn't this error case indicate that this needs to be integrated with > Default partition patch of Rahila or that patch needs to take care > this error case? > Basically, if there is no matching partition, then move it to default > partition. Will have a

Re: [HACKERS] UPDATE of partition key

2017-05-11 Thread Amit Khandekar
On 11 May 2017 at 17:23, Amit Kapila wrote: > On Fri, Mar 17, 2017 at 4:07 PM, Amit Khandekar > wrote: >> On 4 March 2017 at 12:49, Robert Haas wrote: >>> On Thu, Mar 2, 2017 at 11:53 AM, Amit Khandekar >>> wrote: >>>> I think it does not make s

Re: [HACKERS] modeling parallel contention (was: Parallel Append implementation)

2017-05-04 Thread Amit Khandekar
ave a list of next few blocks, say : w1 : 1, 5, 9, 13 w2 : 2, 6, 10, 14 w3 : 3, 7, 11, 15. w4 : . May be the leader worker would do the accounting and store the instructions for each of the workers at individual locations in shared memory, so there won't be any contention while accessing

Re: [HACKERS] UPDATE of partition key

2017-05-02 Thread Amit Khandekar
On 2 May 2017 at 18:17, Robert Haas wrote: > On Tue, Apr 4, 2017 at 7:11 AM, Amit Khandekar wrote: >> Attached updated patch v7 has the above changes. > > This no longer applies. Please rebase. Thanks Robert for informing about this. My patch has a separate function for emittin

Re: [HACKERS] Declarative partitioning - another take

2017-04-26 Thread Amit Khandekar
t, > only the parent's triggers fire. I would also opt for this behaviour. Thanks, -Amit Khandekar EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Parallel Append implementation

2017-04-17 Thread Amit Khandekar
On 7 April 2017 at 20:35, Andres Freund wrote: >> But for costs such as (4, 4, 4, 20 times), the logic would give >> us 20 workers because we want to finish the Append in 4 time units; >> and this what we want to avoid when we go with >> don't-allocate-too-many-workers approach. > > I guess,

Re: [HACKERS] Parallel Append implementation

2017-04-06 Thread Amit Khandekar
On 6 April 2017 at 07:33, Andres Freund wrote: > On 2017-04-05 14:52:38 +0530, Amit Khandekar wrote: >> This is what the earlier versions of my patch had done : just add up >> per-subplan parallel_workers (1 for non-partial subplan and >> subpath->parallel_workers for p

  1   2   >