Re: Optimizing nested ConvertRowtypeExpr execution

2018-04-05 Thread Ashutosh Bapat
On Tue, Apr 3, 2018 at 10:48 AM, Ashutosh Bapat wrote: >> >> Why is this done appropriately at ExecInitExpr() time, rather than at >> plan time? Seems like eval_const_expressions() would be a bit more >> appropriate (being badly named aside...)? > > That seems to

Re: Get the name of the target Relation from Query struct?

2018-04-06 Thread Ashutosh Bapat
the lists are 0 based. But relations indexes are 1 based. Please use rt_fetch() or planner_rt_fetch() as suitable. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Get the name of the target Relation from Query struct? SOLVED!

2018-04-06 Thread Ashutosh Bapat
ble, > Query->resultRelation - 1) As I suggested in the other mail, please use rt_fetch() or planner_rt_fetch() that's future-proof in case we change relids to be something-other-than-one based (an unlikely event, but who knows). -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread Ashutosh Bapat
see a reason why this is bad? I don't think the concept is bad by itself. That's expected, in fact, we have added an option to pg_dump (dump through parent or some such) to handle exactly this case. What Amit seems to be complaining though is the regression test. We need to write regression tests so that they produce the same plans, pruning same partitions by name, on all architectures. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread Ashutosh Bapat
On Sat, Apr 7, 2018 at 8:37 AM, David Rowley wrote: > On 7 April 2018 at 15:03, Ashutosh Bapat > wrote: >> On Sat, Apr 7, 2018 at 7:25 AM, David Rowley >>> The only alternative would be to change all the hash functions so that >>> they normalise their endianness. It

Re: [HACKERS] path toward faster partition pruning

2018-04-06 Thread Ashutosh Bapat
On Sat, Apr 7, 2018 at 8:55 AM, David Rowley wrote: > On 7 April 2018 at 15:14, Ashutosh Bapat > wrote: >> On Sat, Apr 7, 2018 at 8:37 AM, David Rowley >>> Why is writing tests that produce the same output required? >>> >>> We have many tests with alt

Re: Optimizing nested ConvertRowtypeExpr execution

2018-04-09 Thread Ashutosh Bapat
r values cannot be assumed safely > composed each other for general inputs but it is known to be safe > for the ConvertRowtypeExpr case.. I think. I am not able to parse this statement. What output do you see without the patch? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Optimizing nested ConvertRowtypeExpr execution

2018-04-09 Thread Ashutosh Bapat
On Mon, Apr 9, 2018 at 3:53 PM, Ashutosh Bapat wrote: > On Mon, Apr 9, 2018 at 3:49 PM, Kyotaro HORIGUCHI > wrote: >> >> I don't think it is not only on constatns. With the patch, >> non-constants are .. getting a rather strange conversion. >> >> >&g

Re: Optimizing nested ConvertRowtypeExpr execution

2018-04-09 Thread Ashutosh Bapat
On Mon, Apr 9, 2018 at 4:29 PM, Kyotaro HORIGUCHI wrote: > At Mon, 9 Apr 2018 16:07:33 +0530, Ashutosh Bapat > wrote in > >> On Mon, Apr 9, 2018 at 3:53 PM, Ashutosh Bapat >> wrote: >> > On Mon, Apr 9, 2018 at 3:49 PM, Kyotaro HORIGUCHI >> > wrote: &

Re: Optimizing nested ConvertRowtypeExpr execution

2018-04-09 Thread Ashutosh Bapat
typeExpr, arg)->arg; > > This runs depth-first so the while loop seems to run at most > once. I suppose that the "arg =" and the while loop are > transposed as intention. Yes. I have modelled it after RelableType case few lines above in the same function. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] path toward faster partition pruning

2018-04-09 Thread Ashutosh Bapat
ould produce for a given input except by running the code. > If you do the kind of thing shown above, though, then you can easily > see by inspection that you're getting the right answer. +1. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] path toward faster partition pruning

2018-04-10 Thread Ashutosh Bapat
On Tue, Apr 10, 2018 at 5:32 PM, David Rowley wrote: > On 10 April 2018 at 20:56, Amit Langote wrote: >> On 2018/04/10 13:27, Ashutosh Bapat wrote: >>> On Mon, Apr 9, 2018 at 8:56 PM, Robert Haas wrote: >>>> CREATE OR REPLACE FUNCTION hashint4_noop(int4, int8)

Re: crash with sql language partition support function

2018-04-10 Thread Ashutosh Bapat
LOBBER_CACHE_ALWAYS should show failure. I am missing something here. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] path toward faster partition pruning

2018-04-11 Thread Ashutosh Bapat
ch is the point of having the inserts and the select just > above the actual pruning related tests. So, I'm not sure if adding the > satisfies_hash_partition against each pruning tests adds much. +1. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Creation of wiki page for open items of v11

2018-04-11 Thread Ashutosh Bapat
for resolved items, we can keep track of all items. If we just delete the resolved items, we wouldn't know if it was a mistake or it was intentional removal. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Creation of wiki page for open items of v11

2018-04-11 Thread Ashutosh Bapat
ore 11beta1" section is pretty much useless. > > Hm. There's definitely an argument to be made that it's not worth > tracking resolved items till after beta1. Once betas exist, the list > becomes useful to beta testers who may not be tracking events as closely > as hackers do. > +1 -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Creation of wiki page for open items of v11

2018-04-11 Thread Ashutosh Bapat
re patches to solve those and owners responsible in resolving them. We have all the infrastructure in the commitfest app to track those things, may be we could just relabel things. I haven't seen the commitfest app code (and don't plan to do that myself) so may be this is just a wild idea.. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Creation of wiki page for open items of v11

2018-04-12 Thread Ashutosh Bapat
On Thu, Apr 12, 2018 at 6:27 PM, Alvaro Herrera wrote: > Ashutosh Bapat wrote: > >> We use commitfest app for tracking the patches submitted. It has done >> well. Can we re-purpose the same for tracking open items? > > I think the rules are too different to cram both in

Re: Creation of wiki page for open items of v11

2018-04-12 Thread Ashutosh Bapat
#x27;s what I had in mind. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] path toward faster partition pruning

2018-04-12 Thread Ashutosh Bapat
ng? I think we need to have some testcases somwhere to test constraint exclusion on partitions and partitioned tables, those do not necessarily fit partition pruning. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Problem while updating a foreign table pointing to a partitioned table on foreign server

2018-04-16 Thread Ashutosh Bapat
er for the first and local foreign table's oid for the second. Right now, I don't see a simple way to do that. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c index 6e2fa14.

Re: Postgres 10 problem with UNION ALL of null value in "subselect"

2018-04-16 Thread Ashutosh Bapat
expected with that commit. We can work around this problem by casting null to integer like null::integer. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-04-17 Thread Ashutosh Bapat
On Tue, Apr 17, 2018 at 2:05 PM, Etsuro Fujita wrote: > (2018/02/20 18:13), Ashutosh Bapat wrote: >> >> Here's patchset implementing this solution. >> >> 0001 adds PVC_*_CONVERTROWTYPEEXPR to pull_var_clause() and adjusts its >> callers. >> >>

Re: Problem while updating a foreign table pointing to a partitioned table on foreign server

2018-04-18 Thread Ashutosh Bapat
t; Not just cumbersome, it's not going to be always right, if the things change on the foreign server e.g. OID of the table changes because it got dropped and recreated on the foreign server or OID remained same but the table got inherited and so on. I think we should try getting 0001 and 000

Re: Should we add GUCs to allow partition pruning to be disabled?

2018-04-18 Thread Ashutosh Bapat
thing else to mean the same thing. > That will still be useful for inheritance based partitioning. We might re-use constraint_exclusion = 'partition' to mean enable_partition_pruning (ok, I suggested a name as well) = true, although that's not my favourite. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

de-deduplicate code in DML execution hooks in postgres_fdw

2018-04-18 Thread Ashutosh Bapat
postgresql.org/message-id/CAFjFpRfcgwsHRmpvoOK-GUQi-n8MgAS+OxcQo=abdn1coyw...@mail.gmail.com -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company From 2b7156230423a9f58f2f8ee27ab24aed1156605d Mon Sep 17 00:00:00 2001 From: Ashutosh Bapat Date: Wed, 18 Apr 2018 15:52:32

Re: Should we add GUCs to allow partition pruning to be disabled?

2018-04-18 Thread Ashutosh Bapat
/CPU incurred in pruning, when users know for sure that pruning is not going to happen e.g. the cases like no qual on partition key or no equality qual on hash partition key etc. Do we know how much planning time can be saved this way? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Should we add GUCs to allow partition pruning to be disabled?

2018-04-19 Thread Ashutosh Bapat
On Thu, Apr 19, 2018 at 5:02 PM, Amit Langote wrote: > On 2018/04/19 13:32, Ashutosh Bapat wrote: >> On Thu, Apr 19, 2018 at 2:54 AM, David Rowley >>> The more I think about this the more undecided I am as to whether we >>> need to add a GUC for this at all, so I

Re: Should we add GUCs to allow partition pruning to be disabled?

2018-04-19 Thread Ashutosh Bapat
On Fri, Apr 20, 2018 at 7:37 AM, Amit Langote wrote: > On 2018/04/19 21:50, Ashutosh Bapat wrote: >> On Thu, Apr 19, 2018 at 5:02 PM, Amit Langote >>> I can imagine having a enable_partition_pruning which defaults to true, if >>> only to avoid the performance overhead

Re: minor fix for acquire_inherited_sample_rows

2018-04-23 Thread Ashutosh Bapat
tuple descs of two different tables stamped with the same tdtypeid. From that POV, equalTupleDescs seems to be a stronger check than what you have in the patch. BTW the patch you have posted also has a fix you proposed in some other thread. Probably you want to get rid of it. -- Best Wishes, Ashu

Re: minor fix for acquire_inherited_sample_rows

2018-04-23 Thread Ashutosh Bapat
On Mon, Apr 23, 2018 at 6:45 PM, Amit Langote wrote: > Thanks for the review. > > On Mon, Apr 23, 2018 at 8:25 PM, Ashutosh Bapat > wrote: >> On Mon, Apr 23, 2018 at 3:44 PM, Amit Langote >> wrote: >>> Hi. >>> >>> acquire_inherited_sample_rows

Re: minor fix for acquire_inherited_sample_rows

2018-04-23 Thread Ashutosh Bapat
t around to updating equalTupleDescs to cope, we will > need this call right where it is (and we'd have a hard time finding this > potential callsite later, I think). If this were a hot spot I would be > happy to change it, but it's not so I'd rather leave it alone. > +1 -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-04-25 Thread Ashutosh Bapat
On Tue, Apr 24, 2018 at 4:49 PM, Etsuro Fujita wrote: > (2018/04/17 19:00), Etsuro Fujita wrote: >> >> (2018/04/17 18:43), Ashutosh Bapat wrote: >>> >>> Here's updated patch-set. >> >> >> Will review. > > > I started reviewing this

Re: minor fix for acquire_inherited_sample_rows

2018-04-26 Thread Ashutosh Bapat
's invent a new > function that tests for the thing partitioning cares about (same > ordering of the same columns with the same type information) and call > it logicallyEqualTupleDescs() or something like that. Why don't we just rely on the output of convert_tuples_by_name(), which it seems is always called right now? What's advantage of adding another tuple descriptor comparison? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: WIP: a way forward on bootstrap data

2018-04-26 Thread Ashutosh Bapat
erwise, > that would have the effect of documenting which way is now considered > standard. > FWIW, I would like some standard naming convention one way or the other for in/out function names. Looking up pg_type.h for in/out functions when you know the built-in type name isn't great. But that itself may not be enough reason to change it. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: jitflags in _outPlannedStmt and _readPlannedStmt treated as bool type

2018-04-26 Thread Ashutosh Bapat
king of it is a copy paste mistake as the other members around the > initflags are of "bool" type or is there any specific reason to treat as > "bool" type? No, I don't see any. Here's patch for the same. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporatio

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2019-03-10 Thread Ashutosh Bapat
g merged_index explicitly somewhere else, other places may miss that explicit assignment. So it's better to move it inside this function. -- Best Wishes, Ashutosh Bapat

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2019-03-11 Thread Ashutosh Bapat
;> >> > > Ok, that can be fixed. > > Similarly, I think merge_null_partitions should set null_index instead of > asserting when null partitions missing from both the side, make sense? > I think not. null_index, once set shouldn't change and hence does not change with each pair of partitions being matched. So, it makes sense to make sure that null_index remains invalid if none of the tables have null partition. -- Best Wishes, Ashutosh Bapat

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2018-01-01 Thread Ashutosh Bapat
the atomic commit. Will > fix. > I haven't checked how much code it needs to track whether the local transaction wrote anything. But probably we can post-pone this optimization. If it's easy to incorporate, it's good to have in the first set itself. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] path toward faster partition pruning

2018-01-04 Thread Ashutosh Bapat
Key. But if we can do it that way, we can pass that structure to the functions. 2. we will need to construct the structure filling it with comparison information and pass it to the comparison functions. I think what we achieve out of this isn't worth the code we will need to add. I would prefer first approach over the other. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

setting estate in ExecInitNode() itself

2018-01-04 Thread Ashutosh Bapat
e any cases when a node wants to set a different estate there? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: CREATE ROUTINE MAPPING

2018-01-12 Thread Ashutosh Bapat
ble t > > WHERE t.active = true; > I think this is a desired feature. Being able to call a function on remote server through local server is often useful. PostgreSQL allows function overloading, which means that there can be multiple functions with same name differing in argument types. So

Re: CREATE ROUTINE MAPPING

2018-01-28 Thread Ashutosh Bapat
is fine. But whatever design we chose should be extensible to do that. A possible way to implement this may be to implement sql-med language handler which takes the responsibility to interact with FDW and evaluate the function. That way we can use existing function evaluation infrastructure. > > - Once it's established that the combinations could actually work, > execution gets pushed to the foreign server(s) > Overall this structure looks ok to me. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] PoC: full merge join on comparison clause

2018-01-28 Thread Ashutosh Bapat
so that it's easy to review patches. There's lot of executor code, which I don't understand myself. So, I won't be able to complete the review in this CF. Sorry. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Query related to alter table ... attach partition

2018-01-28 Thread Ashutosh Bapat
aints taken together contradict partition constraints or not. The downside is simply that the partition will remain empty forever, but then that's what user wants, since s/he has added the constraints. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: constraint exclusion and nulls in IN (..) clause

2018-01-31 Thread Ashutosh Bapat
gt; --- > Seq Scan on foo >Filter: (a = ANY ('{NULL,2,1}'::integer[])) > (2 rows) > > Thoughts? AFAIU, this doesn't look to be the right way to fix the problem; it assumes that the operators are strict. Sorry, if I have m

Re: constraint exclusion and nulls in IN (..) clause

2018-02-04 Thread Ashutosh Bapat
That's fine, but I am not sure whether this fits constraint exclusion's charter. Please add this patch to the next commitfest so that we can have a better opinion. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Crash in partition-wise join involving dummy partitioned relation

2018-02-05 Thread Ashutosh Bapat
ct this case during build_joinrel_partition_info(). One of the joining relations presented to that function will involve the dummy relation and would have been deemed as unpartitioned when it was processed. We don't need any dummy relation handling in try_partition_wise_join(). Here's pa

Re: Crash in partition-wise join involving dummy partitioned relation

2018-02-05 Thread Ashutosh Bapat
On Tue, Feb 6, 2018 at 4:04 AM, Robert Haas wrote: > On Mon, Feb 5, 2018 at 4:46 AM, Ashutosh Bapat > wrote: >> Here's patch taking that approach. > > I rewrote the comment in relation.h like this, which I think is more clear: > > /* > * Is given relation

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2018-02-07 Thread Ashutosh Bapat
On Fri, Dec 22, 2017 at 3:00 PM, Rajkumar Raghuwanshi wrote: > On Wed, Dec 20, 2017 at 5:21 PM, Ashutosh Bapat > wrote: >> >> Thanks. Here are some comments >> > Thanks Ashutosh for review and suggestions. > >> >> +-- test default partition beha

Re: [HACKERS] path toward faster partition pruning

2018-02-07 Thread Ashutosh Bapat
from bounds etc. which are metadata management kind 2. partition bound comparison functions, and other optimizer related functions. May be we should divide the file that way. The first category code remains in catalog/ as it is today. The second catagory functions move to optimizer/. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] path toward faster partition pruning

2018-02-07 Thread Ashutosh Bapat
On Wed, Feb 7, 2018 at 6:49 PM, Robert Haas wrote: > On Wed, Feb 7, 2018 at 3:42 AM, Ashutosh Bapat > wrote: >> partprune.c looks to much tied to one feature. I am sure that the >> functions used for partition pruning can be used by other >> optimizations as well. > &g

Re: [HACKERS] path toward faster partition pruning

2018-02-08 Thread Ashutosh Bapat
On Wed, Feb 7, 2018 at 7:17 PM, Robert Haas wrote: > On Wed, Feb 7, 2018 at 8:37 AM, Ashutosh Bapat > wrote: >> While looking at the changes in partition.c I happened to look at the >> changes in try_partition_wise_join(). They mark partitions deemed >> dummy by pruning a

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2018-02-08 Thread Ashutosh Bapat
On Thu, Feb 8, 2018 at 10:41 AM, Amit Langote wrote: > On 2018/02/08 11:55, Amit Langote wrote: >> Hi Ashutosh. >> >> On 2018/02/07 13:51, Ashutosh Bapat wrote: >>> Here's a new patchset with following changes >>> >>> 1. Rebased on the late

Re: Proposal: partition pruning by secondary attributes

2018-02-08 Thread Ashutosh Bapat
nsertion and update of secondary attribute will touch catalog and update if required. That will increase the contention on catalog. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2018-02-08 Thread Ashutosh Bapat
On Fri, Feb 9, 2018 at 11:26 AM, Amit Langote wrote: > On 2018/02/09 14:31, Ashutosh Bapat wrote: >>> I also noticed that a later patch adds partsupfunc to PartitionScheme, >>> which the pruning patch needs too. So, perhaps would be nice to take out >>> that porti

Re: Creation of wiki page for open items of v11

2018-02-08 Thread Ashutosh Bapat
ail.com > > Are there any objections in creating a wiki page to track all those bugs > and issues? We don't want to lose track of all that. +1. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Wait event names mismatch: oldserxid

2018-02-09 Thread Ashutosh Bapat
CHE_OLDSERXID_BUFFERS); -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Wait event names mismatch: oldserxid

2018-02-09 Thread Ashutosh Bapat
On Fri, Feb 9, 2018 at 7:23 PM, Michael Paquier wrote: > On Fri, Feb 09, 2018 at 06:04:39PM +0530, Ashutosh Bapat wrote: >> Name for wait event "LWTRANCHE_OLDSERXID_BUFFERS" is printed as >> "oldserxid", but documentation at >> https://www.postgresql.org/

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-02-13 Thread Ashutosh Bapat
t_relation_column_alias_ids() encounters a ConvertRowtypeExpr, it pulls out the embedded whole row reference and returns the corresponding column id. deparseExpr() calls deparseVar() by pulling out the embedded whole-row reference Var when it encouters ConvertRowtypeExpr. Comments, thoughts any other solutions? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: reorganizing partitioning code (was: Re: [HACKERS] path toward faster partition pruning)

2018-02-13 Thread Ashutosh Bapat
itioning related fields. Various partition > bound comparison and search functions (and then some) that work off of the > cached information are moved. Are you moving partition bound comparison functions to partcache.c? They will also used by optimizer, so may be leave them out of partcache.c? --

Re: spelling of enable_partition_wise_join

2018-02-14 Thread Ashutosh Bapat
e. Alvaro, Peter, Gavin have voted for partitionwise in this thread and Robert had similar objections earlier. Looks like we should change it. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-02-14 Thread Ashutosh Bapat
partitioned table. That will be too restrictive. I think the case of PHV and ConvertRowtypeExprs are different. The the former we need a subquery to push PHVs, whereas to deparse ConvertRowtypeExpr on the nullable side we can use the same strategy as whole-row expression deparsing. -- Best

Re: [HACKERS] why not parallel seq scan for slow functions

2018-02-15 Thread Ashutosh Bapat
e output will show that the parallel seq scan's targetlist has costly_func() in it. Isn't that what we want to test here? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Changing baserel to foreignrel in postgres_fdw functions

2018-02-15 Thread Ashutosh Bapat
Hi, I noticed that functions is_foreign_expr(), classifyConditions() and appendOrderByClause() had variables/arguments named baserel when the relations passed to those could be join or upper relation as well. Here's patch renaming those as foreignrel. -- Best Wishes, Ashutosh Bapat Enterpr

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2019-05-14 Thread Ashutosh Bapat
de to handle the cases when the partitions are missing on the nullable side. Tom mentioned the idea of using just projection to produce join tuples with rows on the outer side appended with null columns from the nullable side. If we can implement that, we can remove the restrictions in this patch. -- Best Wishes, Ashutosh Bapat

Re: [HACKERS] why not parallel seq scan for slow functions

2018-02-15 Thread Ashutosh Bapat
On Thu, Feb 15, 2018 at 7:47 PM, Amit Kapila wrote: > On Thu, Feb 15, 2018 at 4:18 PM, Ashutosh Bapat > wrote: >> I happened to look at the patch for something else. But here are some >> comments. If any of those have been already discussed, please feel >> free to ignore.

Re: spelling of enable_partition_wise_join

2018-02-18 Thread Ashutosh Bapat
On Fri, Feb 16, 2018 at 9:09 PM, Peter Eisentraut wrote: > On 2/14/18 03:28, Ashutosh Bapat wrote: >> On Wed, Feb 14, 2018 at 2:15 AM, Alvaro Herrera >> wrote: >>> Peter Eisentraut wrote: >>>> I wonder how others feel about this, but the spelling of >>

Re: [HACKERS] why not parallel seq scan for slow functions

2018-02-18 Thread Ashutosh Bapat
On Sat, Feb 17, 2018 at 8:17 AM, Amit Kapila wrote: > On Fri, Feb 16, 2018 at 9:29 AM, Ashutosh Bapat > wrote: >> On Thu, Feb 15, 2018 at 7:47 PM, Amit Kapila wrote: >>> On Thu, Feb 15, 2018 at 4:18 PM, Ashutosh Bapat >>> wrote: >>>> I happened to look

Re: [HACKERS] why not parallel seq scan for slow functions

2018-02-18 Thread Ashutosh Bapat
On Mon, Feb 19, 2018 at 9:35 AM, Ashutosh Bapat wrote: > On Sat, Feb 17, 2018 at 8:17 AM, Amit Kapila wrote: >> On Fri, Feb 16, 2018 at 9:29 AM, Ashutosh Bapat >> wrote: >>> On Thu, Feb 15, 2018 at 7:47 PM, Amit Kapila >>> wrote: >>>> On

Re: [HACKERS] Partition-wise aggregation/grouping

2018-02-18 Thread Ashutosh Bapat
Commit 2fb1abaeb016aeb45b9e6d0b81b7a7e92bb251b9, changed enable_partition_wise_join to enable_partitionwise_join. This patch too should use enable_partitionwise_agg instead of enable_partition_wise_agg.

Re: spelling of enable_partition_wise_join

2018-02-18 Thread Ashutosh Bapat
On Mon, Feb 19, 2018 at 9:08 AM, Ashutosh Bapat wrote: > > Thanks. There are functions like try_partition_wise_join(), > generate_partition_wise_join_paths() which use partition_wise > spelling. Should we update those as well? I see that the commit already did that. Sorry. Although,

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-02-20 Thread Ashutosh Bapat
On Tue, Feb 13, 2018 at 6:21 PM, Ashutosh Bapat wrote: > > 1. Push down ConvertRowtypeExpr and include it in the pushed down targetlist. > This would solve both the problems described above. Both set_plan_ref() and > get_relation_column_alias_ids() will find ConvertRowtypeExpr, they

Re: NEXT VALUE FOR sequence

2018-02-21 Thread Ashutosh Bapat
ed row" thing gets pretty complicated. Consider simple case. What happens when NextValueExpr appears in one of the conditions and that row gets eliminated, do we consider that as a processed row and increment the NextValueExpr or do not increment it? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

PlaceHolderVars in pushed down child-join cause error

2018-02-22 Thread Ashutosh Bapat
o push down a child-join with PlaceHolderVars in it and fails with error "unsupported expression type for deparse: 198". 198 being T_PlaceHolderVar. The fix is to use joinrel->top_parent_relids for a child-join. Attached patch for the same. -- Best Wishes, Ashutosh Bapat EnterpriseD

Re: PlaceHolderVars in pushed down child-join cause error

2018-02-25 Thread Ashutosh Bapat
On Thu, Feb 22, 2018 at 8:36 PM, Robert Haas wrote: > On Thu, Feb 22, 2018 at 7:41 AM, Ashutosh Bapat > wrote: >> postgres_fdw isn't expected to push down joins with placeholder vars. >> But the check for that in foreign_join_ok() only considers >> joinrel->

Optimizing nested ConvertRowtypeExpr execution

2018-02-26 Thread Ashutosh Bapat
code which handles nested ConvertRowtypeExprs like is_converted_whole_row_reference(). But I haven't tried that approach yet. Suggestions/comments welcome. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company diff --git a/src/backend/executor/execExpr.c b/src/backend/exe

Re: [HACKERS] path toward faster partition pruning

2018-02-28 Thread Ashutosh Bapat
be a relevant test in partition_join.sql? If yes, > attached a patch (partitionwise-join-collation-test-1.patch) to add one. A partition-wise join path will be created but discarded because of higher cost. This test won't see it in that case. So, please add some data like other tes

Re: [HACKERS] path toward faster partition pruning

2018-02-28 Thread Ashutosh Bapat
't explain why and if it did so it should do that in find_partition_scheme() not here. partsupfunc is another property of partition keys that is cached like parttyplen, parttypbyval. Why does it deserve a separate comment when others don't? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] path toward faster partition pruning

2018-02-28 Thread Ashutosh Bapat
On Thu, Mar 1, 2018 at 6:57 AM, Amit Langote wrote: > On 2018/02/28 19:14, Ashutosh Bapat wrote: >> On Wed, Feb 28, 2018 at 6:42 AM, Amit Langote wrote: >>> BTW, should there be a relevant test in partition_join.sql? If yes, >>> attached a patch (partitionwise-join-col

Re: parallel append vs. simple UNION ALL

2018-03-01 Thread Ashutosh Bapat
th = make_union_unique(op, ppath, tlist, root); We could probably push the grouping/sorting down to the parallel workers. But again not part of this patchset. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: postgres_fdw: perform UPDATE/DELETE .. RETURNING on a join directly

2018-03-04 Thread Ashutosh Bapat
has grown really large over the time, as we have added more pushdown. Since most of the queries in that file use the same tables created at the beginning of the file, changes somewhere in-between (esp. DMLs) affect the following queries. It's getting hard to add a test query in that file an

Re: [HACKERS] PoC: full merge join on comparison clause

2018-03-04 Thread Ashutosh Bapat
pstream}, but there are other ways also. Then you can use git rebase to rebase your patches periodically. If you are already doing that, sorry for the noise. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-06 Thread Ashutosh Bapat
t * -make_partial_grouping_target(PlannerInfo *root, PathTarget *grouping_target) +make_partial_grouping_target(PlannerInfo *root, + PathTarget *grouping_target, + Node *havingQual) This looks like a refactoring change. Should go to one of the r

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-06 Thread Ashutosh Bapat
al_paths_to_grouping_rel() and > add_paths_to_grouping_rel() to avoid passing can_sort, can_hash and costs > related details individually to them? I think so too. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Missing break statement after transformCallStmt in transformStmt

2018-03-06 Thread Ashutosh Bapat
g to cause problems in future when somebody adds a case after CallStmt. Here's patch with fix. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c index c3a9617..cf1a34e 100644 --- a/src

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-07 Thread Ashutosh Bapat
On Wed, Mar 7, 2018 at 10:04 AM, Ashutosh Bapat wrote: > On Tue, Mar 6, 2018 at 7:52 PM, Jeevan Chalke > wrote: > >> >> >> Changes look good to me and refactoring will be useful for partitionwise >> patches. >> >> However, will it be good if we add

Re: [HACKERS] Another oddity in handling of WCO constraints in postgres_fdw

2018-03-07 Thread Ashutosh Bapat
eed to use remote WCO. That means we create many foreign tables pointing to same local table on the foreign server through many views, but it's not impossible. [1] https://www.postgresql.org/message-id/20180117130021.GC2416%40tamriel.snowman.net [2] https://www.postgresql.org/message-id/5A058F21.2040201%40lab.ntt.co.jp [3] https://www.postgresql.org/message-id/a3955a1d-ad07-5b0a-7618-b6ef5ff0e1c5%40lab.ntt.co.jp -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-07 Thread Ashutosh Bapat
uch paths have different shapes from the ones that we create now and will they be better? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-08 Thread Ashutosh Bapat
upper-rel PathTargets we just computed into * root->upper_targets[]. The core code doesn't use this, but it * provides a convenient place for extensions to get at the info. For * consistency, we save all the intermediate targets, even though some * of t

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-09 Thread Ashutosh Bapat
the grouped relation and partially grouped relation do not have bare partition keys. So, for a relation sitting on top of this (partially) grouped relation the partition key doesn't exist. So, we can't consider grouped or partially grouped relation as partitioned. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-12 Thread Ashutosh Bapat
On Fri, Mar 9, 2018 at 4:21 PM, Ashutosh Bapat wrote: > On Thu, Mar 8, 2018 at 7:31 PM, Robert Haas wrote: >> >> This kind of goes along with the suggestion I made yesterday to >> introduce a new function, which at the time I proposed calling >> initialize_grouping_r

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-12 Thread Ashutosh Bapat
On Mon, Mar 12, 2018 at 7:49 PM, Jeevan Chalke wrote: > > > On Mon, Mar 12, 2018 at 6:07 PM, Ashutosh Bapat > wrote: >> >> On Fri, Mar 9, 2018 at 4:21 PM, Ashutosh Bapat >> >> GroupPathExtraData now completely absorbs members from and replaces >> OtherU

Re: [HACKERS] Another oddity in handling of WCO constraints in postgres_fdw

2018-03-12 Thread Ashutosh Bapat
On Mon, Mar 12, 2018 at 1:25 PM, Etsuro Fujita wrote: > (2018/03/09 20:55), Etsuro Fujita wrote: >> >> (2018/03/08 14:24), Ashutosh Bapat wrote: >>> >>> For local constraints to be enforced, we use remote >>> constraints. For local WCO we need to use

Re: parallel append vs. simple UNION ALL

2018-03-12 Thread Ashutosh Bapat
; > Good point. Changed. It looks like it was not changed in all the places. make falied. I have fixed all the instances of these two functions in the attached patchset (only 0003 changes). Please check. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporat

Re: [HACKERS] why not parallel seq scan for slow functions

2018-03-12 Thread Ashutosh Bapat
targetlist of parallel seq scan plan. The solution here addresses only parallel scan requirement. In future if we implement a solution which also addresses requirements of FDW and custom plan (i.e. ability to handle targetlists by FDW and custom plan), the changes made here will need to be reverted. That would be a painful exercsize. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Ambigous Plan - Larger Table on Hash Side

2018-03-13 Thread Ashutosh Bapat
ANALYZE on this query, produce outputs of two plans: one with larger table being hashed and second with the smaller one being hashed, you will see which of them performs better. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Additional Statistics Hooks

2018-03-13 Thread Ashutosh Bapat
e, I mean input argument tuple. In Mat's case the mapping cardinality will be 12. The number of distinct values that function may output is estimated as number of estimated rows / mapping cardinality of that function. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: Ambigous Plan - Larger Table on Hash Side

2018-03-13 Thread Ashutosh Bapat
on and can we fix this ? After analysing the small table, the first plan is chosen as the cheapest. This means that the plan with smaller table being hashed has cost higher than the plan with larger table being hashed. We need to examine that costing to see what went wrong in costing. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Comment fixes in create_grouping_paths() add_paths_to_append_rel()

2018-03-14 Thread Ashutosh Bapat
grouping sets to create_grouping_paths(), but forgot to update the prologue of the function. Fixed it. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company From 053a0f4e77ad14a08f7fb8fb96152fae4406bd9a Mon Sep 17 00:00:00 2001 From: Ashutosh Bapat Date: Wed, 14 Mar

<    1   2   3   4   5   6   7   8   9   10   >