Re: [HACKERS] GetExistingLocalJoinPath() vs. the docs

2016-03-01 Thread Ashutosh Bapat
ction just doesn't seem to fit into this section at > all. It's really quite different from the others listed there. How > about something like the attached instead? Right. Mentioning the function in the description of relevant function looks better and avoids some duplication. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Issue with NULLS LAST, with postgres_fdw sort pushdown

2016-03-02 Thread Ashutosh Bapat
; > create user mapping for db1 server db2_link_server options (user 'db2', > password 'db2'); > > --create a foreign table > create foreign table fdw_sort_test (id integer, name varchar(50)) server > db2_link_server; > > --run the below query and checkout th

Re: [HACKERS] Issue with NULLS LAST, with postgres_fdw sort pushdown

2016-03-02 Thread Ashutosh Bapat
On Thu, Mar 3, 2016 at 7:27 AM, Michael Paquier wrote: > On Wed, Mar 2, 2016 at 7:04 PM, Rajkumar Raghuwanshi > wrote: > > On Wed, Mar 2, 2016 at 2:35 PM, Ashutosh Bapat > > wrote: > >> > >> Thanks Rajkumar for your report. Let me know if the

Re: [HACKERS] Pushing down sorted joins

2016-03-08 Thread Ashutosh Bapat
there will be a single equivalence member A.c3 in the pathkeys and em_relids will indicate only A. Hence instead of equal, (which used to be OK for single relation join push-down) we have to use subset operation. We want an equivalence members whose relids are subset of relids contained by gi

Re: [HACKERS] Pushing down sorted joins

2016-03-09 Thread Ashutosh Bapat
On Wed, Mar 9, 2016 at 9:22 PM, Robert Haas wrote: > On Wed, Mar 9, 2016 at 2:23 AM, Ashutosh Bapat > wrote: > > [ new patch ] > > This looks OK to me. Committed! > > Thanks. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Adjusting the API of pull_var_clause()

2016-03-10 Thread Ashutosh Bapat
ough, I think this would have been OK when pull_var_clause was being written afresh. Now, that we have this API, I am not sure whether the effort is worth the result. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Obsolete comment in postgres_fdw.c

2016-03-14 Thread Ashutosh Bapat
; Those look fine. Sorry for missing those in the commit and thanks for providing a patch for the same. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Use %u to print user mapping's umid and userid

2016-03-14 Thread Ashutosh Bapat
On Mon, Mar 14, 2016 at 1:29 PM, Etsuro Fujita wrote: > Hi, > > On 2016/02/09 14:09, Ashutosh Bapat wrote: > >> Sorry, I was wrong. For public user mapping userid is 0 (InvalidOid), >> which is returned as is in UserMapping object. I confused InvalidOid >> with -1

Re: [HACKERS] Re: [COMMITTERS] pgsql: Only try to push down foreign joins if the user mapping OIDs mat

2016-03-14 Thread Ashutosh Bapat
ies that user mapping is meaningless to it. Should we allow the user mapping to be created but ignore it or do not allow it to be created? In the later case, what should happen to the existing user mappings? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Re: [COMMITTERS] pgsql: Only try to push down foreign joins if the user mapping OIDs mat

2016-03-15 Thread Ashutosh Bapat
h sides have > InvalidOid? > > Exactly. And we should make sure (possibly with a regression test) > that postgres_fdw handles that case correctly - i.e. with the right > error. > > -- > Robert Haas > EnterpriseDB: http://www.enterprisedb.com > The Enterprise PostgreSQL Company

Re: [HACKERS] Re: [COMMITTERS] pgsql: Only try to push down foreign joins if the user mapping OIDs mat

2016-03-16 Thread Ashutosh Bapat
On Wed, Mar 16, 2016 at 2:14 AM, Robert Haas wrote: > On Tue, Mar 15, 2016 at 6:44 AM, Ashutosh Bapat > wrote: > > Here's patch which fixes the issue using Robert's idea. > > Please at least check your patches with 'git diff --check' Thanks. > befo

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-18 Thread Ashutosh Bapat
sh down of higher operations. It seems very much possible after the upper pathification changes. We can not have a query sent to the foreign server for a relation, when pushdown_safe is false for that. Your patch does that for foreign base relation which try to fetch system columns. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-19 Thread Ashutosh Bapat
t means those attributes won't be set while fetching the row from the foreign server and will have garbage values in corresponding places. I guess that would work. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Re: [COMMITTERS] pgsql: Only try to push down foreign joins if the user mapping OIDs mat

2016-03-19 Thread Ashutosh Bapat
On Wed, Mar 16, 2016 at 10:22 PM, Tom Lane wrote: > Robert Haas writes: > > On Wed, Mar 16, 2016 at 4:10 AM, Ashutosh Bapat < > > ashutosh.ba...@enterprisedb.com> wrote: > >> In 9.5, postgres_fdw allowed to prepare statements involving foreign > >> table

Re: [HACKERS] Postgres_fdw join pushdown - getting server crash in left outer join of three table

2016-03-21 Thread Ashutosh Bapat
add pathkeys which will help merge joins. I have included the relevant tests rewriting them to use local tables, so that the entire join is not pushed down to the foreign server. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company diff --git a/contrib/postg

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-21 Thread Ashutosh Bapat
ns which can be used by FDWs, if they want, to return values for those columns. Something like Datum get_syscol_value(RelOptInfo/Relation, attno). The function will return Datum 0 for most of the columns and table's OID for tableoid. My 0.02. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-22 Thread Ashutosh Bapat
On Tue, Mar 22, 2016 at 5:05 PM, Etsuro Fujita wrote: > On 2016/03/22 14:54, Ashutosh Bapat wrote: > >> On Tue, Mar 22, 2016 at 8:03 AM, Etsuro Fujita >> mailto:fujita.ets...@lab.ntt.co.jp>> wrote: >> OK, I'll modify the patch so that the join is push

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-22 Thread Ashutosh Bapat
On Wed, Mar 23, 2016 at 8:20 AM, Etsuro Fujita wrote: > On 2016/03/22 21:10, Ashutosh Bapat wrote: > >> On Tue, Mar 22, 2016 at 5:05 PM, Etsuro Fujita >> mailto:fujita.ets...@lab.ntt.co.jp>> wrote: >> On 2016/03/22 14:54, Ashutosh Bapat wrote: >>

Re: [HACKERS] Postgres_fdw join pushdown - getting server crash in left outer join of three table

2016-03-24 Thread Ashutosh Bapat
> > Thanks for the report and the testing. I have committed the patch. > > Thanks. > Cool, I have refreshed the wiki page for open items accordingly. > Thanks. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-24 Thread Ashutosh Bapat
On Thu, Mar 24, 2016 at 9:31 AM, Etsuro Fujita wrote: > On 2016/03/23 13:44, Ashutosh Bapat wrote: > >> An FDW can choose not to use those functions, so I don't see a >> connection between scan list having simple Vars and existence of those >> functions (actually a

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-03-24 Thread Ashutosh Bapat
me of planning should be fine since change in tableoid between planning and execution will trigger plan cache invalidation. I haven't tried this though. Sorry for bringing this solution late to the table. On Thu, Mar 24, 2016 at 3:04 PM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com&g

Re: [HACKERS] Postgres_fdw join pushdown - INNER - FULL OUTER join combination generating wrong result

2016-03-29 Thread Ashutosh Bapat
lar case that was reported, the bug triggered because of the way conditions are handled for an inner join. For an inner join, all the conditions in ON as well as WHERE clause are treated like they are part of WHERE clause. This allows pushing down a join even if there are unpushable join clauses.

Re: [HACKERS] FDW join pushdown and scanclauses

2016-01-13 Thread Ashutosh Bapat
On Thu, Jan 14, 2016 at 9:31 AM, Etsuro Fujita wrote: > On 2016/01/08 22:05, Ashutosh Bapat wrote: > >> In add_paths_to_joinrel(), the FDW specific hook GetForeignJoinPaths() >> is called. This hook if implemented should add ForeignPaths for pushed >> down joins. cre

Re: [HACKERS] Limit and inherited tables

2016-01-17 Thread Ashutosh Bapat
ps to get sorted data from the foreign server. LIMIT pushdown is not supported yet, though. > > -- > Konstantin Knizhnik > Postgres Professional: http://www.postgrespro.com > The Russian Postgres Company > > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgres

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-01-18 Thread Ashutosh Bapat
9fOZ0dNNf9Z=gnyksb6wg...@mail.gmail.com I will be working next on (in that order) 1. eval_plan_qual fix for foreign join. (Considered as a must-have for 9.6) 2. Pushing down ORDER BY clause along with join pushdown 3. Parameterization of foreign join paths (Given the complexity of the feature this may not make it into 9.6) -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-01-20 Thread Ashutosh Bapat
On Wed, Jan 20, 2016 at 4:58 AM, Robert Haas wrote: > On Mon, Jan 18, 2016 at 6:47 AM, Ashutosh Bapat > wrote: > > Thanks Thom for bringing it to my notice quickly. Sorry for the same. > > > > Here are the patches. > > > > 1. pg_fdw_core_v2.patch: cha

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-01-20 Thread Ashutosh Bapat
I missed the example plan cache revalidation patch in the previous mail. Sorry. Here it is. On Wed, Jan 20, 2016 at 7:20 PM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com> wrote: > > > On Wed, Jan 20, 2016 at 4:58 AM, Robert Haas > wrote: > >> On Mon, Jan 18, 2016

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-01-20 Thread Ashutosh Bapat
in_v2.patch, postgresBeginForeignScan() obtained user mapping using GetUserMappingById() instead of the earlier way of fetching it by userid and serverid. So even that change will remain, right? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-01-21 Thread Ashutosh Bapat
On Thu, Jan 21, 2016 at 3:03 AM, Robert Haas wrote: > On Mon, Jan 18, 2016 at 6:47 AM, Ashutosh Bapat > wrote: > > 2. pg_fdw_join_v2.patch: postgres_fdw changes for supporting join > pushdown > > The very first hunk of this patch contains annoying whitespace > changes

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-01-21 Thread Ashutosh Bapat
ng oid (which will have invalid user id for public user mapping) and used userid from that structure, we will get rid of this problem. > > -- > Robert Haas > EnterpriseDB: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

[HACKERS] Using user mapping OID as hash key for connection hash

2016-01-27 Thread Ashutosh Bapat
u-vppq1d7ufsjaopbq+lgpxtchnuqfobjg2...@mail.gmail.com -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company pg_fdw_concache.patch.large Description: Binary data pg_fdw_concache.patch.short Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@post

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-01-29 Thread Ashutosh Bapat
On Fri, Jan 29, 2016 at 9:51 AM, Robert Haas wrote: > On Thu, Jan 28, 2016 at 11:26 AM, Ashutosh Bapat > wrote: > > 2. pg_fdw_join_v3.patch: changes to postgres_fdw - more description below > > This patch no longer quite applies because of conflicts with one of > your

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-01-29 Thread Ashutosh Bapat
On Fri, Jan 29, 2016 at 2:05 PM, Etsuro Fujita wrote: > On 2016/01/29 1:26, Ashutosh Bapat wrote: > >> Here's an updated version of the previous patches, broken up like before >> > > 2. pg_fdw_join_v3.patch: changes to postgres_fdw - more description below >>

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-02 Thread Ashutosh Bapat
On Tue, Feb 2, 2016 at 5:18 AM, Robert Haas wrote: > On Mon, Feb 1, 2016 at 8:27 AM, Ashutosh Bapat > wrote: > > Here are patches rebased on recent commit > > cc592c48c58d9c1920f8e2063756dcbcce79e4dd. Renamed original > deparseSelectSql > > as deparseSe

Re: [HACKERS] [POC] FETCH limited by bytes.

2016-02-03 Thread Ashutosh Bapat
ession tests added. > > Committed. > > -- > Robert Haas > EnterpriseDB: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company diff --git a/contrib/postgres_fdw/postgres_f

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-04 Thread Ashutosh Bapat
fdw_outerpath should be a local path set when paths for outerjoinpath->parent was being created. Am I missing something? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-04 Thread Ashutosh Bapat
rameterized join paths would be closer to parameterized foreign paths (if we were to produce those). Hence my statement. There is always a possibility that those two costs are way too different, hence I have used phrase "possibly" there. I could be wrong. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-04 Thread Ashutosh Bapat
On Thu, Feb 4, 2016 at 3:28 PM, Etsuro Fujita wrote: > On 2016/02/04 17:58, Etsuro Fujita wrote: > >> On 2016/02/04 8:00, Robert Haas wrote: >> >>> On Wed, Feb 3, 2016 at 5:56 PM, Robert Haas >>> wrote: >>> >>>> On Wed, Feb 3, 2016 at 12:

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-04 Thread Ashutosh Bapat
On Thu, Feb 4, 2016 at 4:30 AM, Robert Haas wrote: > On Wed, Feb 3, 2016 at 5:56 PM, Robert Haas wrote: > > On Wed, Feb 3, 2016 at 12:08 PM, Ashutosh Bapat > > wrote: > >> PFA patches with naming conventions similar to previous ones. > >> pg_fdw_core_v7.patch:

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-05 Thread Ashutosh Bapat
IUC, all of them output the same targetlist. We don't need to make sure that targetlist match as long as we are using the targetlist passed in by create_scan_plan(). Do you have a counter example? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-05 Thread Ashutosh Bapat
t. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c index fb72f45..7a2a67b 100644 --- a/contrib/postgres_fdw/deparse.c +++ b/contrib/postgres_fdw/deparse.c @@ -93,9 +93,10 @@ typedef str

Re: [HACKERS] Use %u to print user mapping's umid and userid

2016-02-08 Thread Ashutosh Bapat
nd userid. > > Pushed, thanks. > > regards, tom lane > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers > -- Best Wishes, A

Re: [HACKERS] Use %u to print user mapping's umid and userid

2016-02-08 Thread Ashutosh Bapat
Sorry, I was wrong. For public user mapping userid is 0 (InvalidOid), which is returned as is in UserMapping object. I confused InvalidOid with -1. Sorry for the confusion. On Tue, Feb 9, 2016 at 10:21 AM, Tom Lane wrote: > Ashutosh Bapat writes: > > Sorry to come to this late. >

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-09 Thread Ashutosh Bapat
Yay, finally! Thanks. On Wed, Feb 10, 2016 at 12:46 AM, Robert Haas wrote: > On Tue, Feb 9, 2016 at 8:39 AM, Ashutosh Bapat > wrote: > > Thanks Jeevan for your review and comments. PFA the patch which fixes > those. > > Committed with a couple more small adjustments

Re: [HACKERS] [COMMITTERS] pgsql: postgres_fdw: Push down joins to remote servers.

2016-02-09 Thread Ashutosh Bapat
> > -- > Robert Haas > EnterpriseDB: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers > -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-10 Thread Ashutosh Bapat
, 2016 at 8:39 AM, Ashutosh Bapat > > wrote: > >> Thanks Jeevan for your review and comments. PFA the patch which fixes > those. > > > > Committed with a couple more small adjustments. > > I'm getting a compiler warning which I think is coming from thi

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-15 Thread Ashutosh Bapat
s and thus lists of any higher level joins. On Mon, Feb 15, 2016 at 1:10 PM, Etsuro Fujita wrote: > On 2016/02/10 4:16, Robert Haas wrote: > >> On Tue, Feb 9, 2016 at 8:39 AM, Ashutosh Bapat >> wrote: >> >>> Thanks Jeevan for your review and comments. PFA the patc

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-15 Thread Ashutosh Bapat
: > On 2016/02/15 21:33, Ashutosh Bapat wrote: > >> Here's patch with better way to fix it. I think while concatenating the >> lists, we need to copy the lists being appended and in all the cases. If >> we don't copy, a change in those lists can cause changes in the

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-15 Thread Ashutosh Bapat
On Tue, Feb 16, 2016 at 12:26 PM, Etsuro Fujita wrote: > On 2016/02/16 15:22, Ashutosh Bapat wrote: > >> During join planning, the planner tries multiple combinations of joining >> relations, thus the same base or join relation can be part of multiple >> of combinatio

Re: [HACKERS] GetExistingLocalJoinPath() vs. the docs

2016-02-16 Thread Ashutosh Bapat
return a void) Thanks for pointing that out. > and the paragraph about what it's for could do > with some wordsmithing. > Any specific suggestions? > > A link from 54.2 to 54.3 which mentions it would be fine, of course. > Ok PFA patch fixing those things. -- Best

[HACKERS] Pushing down sorted joins

2016-02-17 Thread Ashutosh Bapat
-id/CAFjFpRfeKHiCmwJ72p4=zvuzrqsau9tbfyw7vwr-5ppvrcb...@mail.gmail.com -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company pg_join_sort_pd.patch Description: application/download \set num_samples 10 \set query 'SELECT lt1.val, ft1.val, ft2.val FROM lt1 join (ft1 join ft2 on

Re: [HACKERS] postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)

2016-02-18 Thread Ashutosh Bapat
On Thu, Feb 18, 2016 at 3:48 PM, Etsuro Fujita wrote: > On 2016/02/16 16:40, Etsuro Fujita wrote: > >> On 2016/02/16 16:02, Ashutosh Bapat wrote: >> >>> On Tue, Feb 16, 2016 at 12:26 PM, Etsuro Fujita >>> mailto:fujita.ets...@lab.ntt.co.jp>> >>>

Re: [HACKERS] Pushing down sorted joins

2016-02-23 Thread Ashutosh Bapat
their right place. On Wed, Feb 17, 2016 at 5:37 PM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com> wrote: > Hi All, > Now that we have join pushdown support in postgres_fdw, we can leverage > the sort pushdown mechanism for base relations to work for pushed down > joins as w

Re: [HACKERS] Declarative partitioning - another take

2016-08-21 Thread Ashutosh Bapat
e available for creating and managing partitioned table > and its partitions. Especially considering that it is no longer necessary > to define CHECK constraints and triggers/rules manually for constraint > exclusion and tuple routing, respectively. > > TODO (in short term): > *

Re: [HACKERS] Push down more full joins in postgres_fdw

2016-08-21 Thread Ashutosh Bapat
3, c4, c5, c6, > c7, c8), "C 1" FROM "S 1"."T 1") ss2(c1, c2) ON (TRUE)) WHERE ((ss1.c3 = > ss2.\ > c2)) ORDER BY ss1.c4 ASC NULLS LAST, ss1.c3 ASC NULLS LAST > (6 rows) > > I'll add this to the next CF. Comments are welcome! > > Best rega

Re: [HACKERS] Declarative partitioning - another take

2016-08-25 Thread Ashutosh Bapat
On Thu, Aug 25, 2016 at 12:22 PM, Amit Langote < langote_amit...@lab.ntt.co.jp> wrote: > On 2016/08/22 13:51, Ashutosh Bapat wrote: > > The parent-child relationship of multi-level partitioned tables is not > > retained when creating the AppendRelInfo nodes. We create RelOp

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2016-08-25 Thread Ashutosh Bapat
ginal patch, patch of pg_fdw_xact_resolver and > documentation are lacked. > I am not able to understand the last statement. Do you mean to say that your patches do not have pg_fdw_xact_resolver() and documentation that my patches had? OR you mean to say that my patches did not have (lacked) pg_fdw_xact_resolver() and documenation OR some combination of those? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2016-08-25 Thread Ashutosh Bapat
On Fri, Aug 26, 2016 at 11:37 AM, Masahiko Sawada wrote: > On Fri, Aug 26, 2016 at 3:03 PM, Ashutosh Bapat > wrote: > > > > > > On Fri, Aug 26, 2016 at 11:22 AM, Masahiko Sawada > > > wrote: > >> > >> On Fri, Aug 26, 2016 at

Re: [HACKERS] Declarative partitioning - another take

2016-09-01 Thread Ashutosh Bapat
-wise join. > Then as we build successively higher levels of > joinrels, new entries will be made for those joinrels for which we added > pairwise join paths, with relids matching the corresponding joinrels. > Does that make sense? > > I don't think we will make any ne

Re: [HACKERS] Declarative partitioning - another take

2016-09-01 Thread Ashutosh Bapat
nge partition, an error will be thrown. > > 0008-Update-DDL-Partitioning-chapter.patch > > This patch updates the partitioning section in the DDL chapter to reflect > the new methods made available for creating and managing partitioned table > and its partitions. Especially considering th

Re: [HACKERS] Declarative partitioning - another take

2016-09-01 Thread Ashutosh Bapat
0 or a < 600)). It will probably get excluded, if constraint exclusion is smart enough to understand ORing. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Declarative partitioning - another take

2016-09-01 Thread Ashutosh Bapat
On Fri, Sep 2, 2016 at 12:23 PM, Amit Langote wrote: > On 2016/09/02 15:22, Ashutosh Bapat wrote: > >> > >> > >>> 2. A combination of constraints on the partitions should be applicable > to > >>> the parent. We aren't doing that. > >&

Re: [HACKERS] Declarative partitioning - another take

2016-09-06 Thread Ashutosh Bapat
ve of parent's, selecting a parent's NOT NULL > column might return nulls from the child table that no longer has the > constraint. > > I recently came across a related proposal whereby dropping *inherited* NOT > NULL from child tables will be prevented. Problems in letting it

Re: [HACKERS] Declarative partitioning - another take

2016-09-06 Thread Ashutosh Bapat
This patch uses RangeBound structure. There's also a structure defined with the same name in rangetypes.h with some slight differences. Should we rename the one in partition.c as PartRangeBound or something like that to avoid the confusion? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corpor

Re: [HACKERS] Push down more full joins in postgres_fdw

2016-09-06 Thread Ashutosh Bapat
On Fri, Sep 2, 2016 at 3:55 PM, Etsuro Fujita wrote: > Hi Ashutosh, > > On 2016/08/22 15:49, Ashutosh Bapat wrote: > >> 1. deparsePlaceHolderVar looks odd - each of the deparse* function is >> named as deparse + > into>. PlaceHolderVar is not a parser node, so no s

Re: [HACKERS] Push down more UPDATEs/DELETEs in postgres_fdw

2016-09-06 Thread Ashutosh Bapat
ERE clause can be applied on it. But it's not needed if we are pushing down the query. If we eliminate the targetlist of the query, we could construct a remote query without having subquery in it, making it more readable. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Aggregate Push Down - Performing aggregation on foreign server

2016-09-08 Thread Ashutosh Bapat
of just assigning the pointer. 30. By the time postgresGetForeignUpperPaths() gets called, the core has already added its own paths, so it doesn't make much sense to set rows and width grouped_rel in create_foreign_grouping_paths(). 31. fpinfo->server and user fields are being set twice, on

Re: [HACKERS] Push down more full joins in postgres_fdw

2016-09-12 Thread Ashutosh Bapat
a clue as to what error context is >> about. >> > > I don't think so. Consider an example of the conversion error message, > which is from the regression test: > > SELECT ft1.c1, ft2.c2, ft1 FROM ft1, ft2 WHERE ft1.c1 = ft2.c1 AND > ft1.c1 = 1; > ERROR: invalid input syntax for integer: "foo" > CONTEXT: whole-row reference to foreign table "ft1" > > As shown in the example, the error message is displayed under a remote > query for execution. So, ISTM it's reasonable to print something like > "expression at position %d in select list" in the context if an expression > in a PHV. > I missed it. Sorry. Looks ok. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company

Re: [HACKERS] Nested loop join condition does not get pushed down to foreign scan

2016-09-13 Thread Ashutosh Bapat
ths for that foreign relation. If using a parameterized path reduces cost of the join, it will use a nested loop join with inner relation parameterized by the outer relation, pushing join conditions down into the foreign scan. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Push down more full joins in postgres_fdw

2016-09-13 Thread Ashutosh Bapat
On Tue, Sep 13, 2016 at 10:28 PM, Robert Haas wrote: > On Tue, Sep 6, 2016 at 9:07 AM, Ashutosh Bapat > wrote: >> That's not true with the alias information. As long as we detect which >> relations need subqueries, their RTIs are enough to create unique aliases >>

[HACKERS] Printing bitmap objects in the debugger

2016-09-14 Thread Ashutosh Bapat
and compile it for debugging. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c index 29b7712..b4cae11 100644 --- a/src/backend/nodes/outfuncs.c +++ b/src/backend/nodes/outfuncs.c @

[HACKERS] Re: [HACKERS] Error running custom plugin: “output plugins have to declare the _PG_output_plugin_init symbol”

2016-09-14 Thread Ashutosh Bapat
_decoding/test_decoding.c > > I'm not sure if the deployment process is ok (just copying the dll) or if > there is some other step to take. Can anyone shed some light? > It's hard to tell what's wrong exactly, without seeing the changes you have made. But, it looks like wh

Re: [HACKERS] Printing bitmap objects in the debugger

2016-09-14 Thread Ashutosh Bapat
tion like the one I have in the patch makes things easy for all the debuggers and may be developers not familiar with python. > > The complete API reference is available here > http://lldb.llvm.org/python_reference/index.html > > Looks like an interesting SoC project to write useful

Re: [HACKERS] Printing bitmap objects in the debugger

2016-09-14 Thread Ashutosh Bapat
here are other places where we manipulate Bitmapsets. And not every Relids object computed is contained in a Node. So, pprint() doesn't help much. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Printing bitmap objects in the debugger

2016-09-14 Thread Ashutosh Bapat
On Wed, Sep 14, 2016 at 8:45 PM, Alvaro Herrera wrote: > Tom Lane wrote: >> Ashutosh Bapat writes: >> > While working on partition-wise join, I had to examine Relids objects >> > many times. Printing the Bitmapset::words[] in binary format and then >> > in

Re: [HACKERS] Push down more full joins in postgres_fdw

2016-09-14 Thread Ashutosh Bapat
On Wed, Sep 14, 2016 at 8:52 PM, Robert Haas wrote: > On Tue, Sep 13, 2016 at 11:38 PM, Ashutosh Bapat > wrote: >> On Tue, Sep 13, 2016 at 10:28 PM, Robert Haas wrote: >>> On Tue, Sep 6, 2016 at 9:07 AM, Ashutosh Bapat >>> wrote: >>>> That's not

Re: [HACKERS] Declarative partitioning - another take

2016-09-15 Thread Ashutosh Bapat
vation 3 : Getting cache lookup failed, when selecting list >> partition table containing array. >> >> CREATE TABLE test_array ( i int,j int[],k text[]) PARTITION BY LIST (j); >> CREATE TABLE test_array_p1 PARTITION OF test_array FOR VALUES IN ('{1}'); >> CREAT

Re: [HACKERS] Parallel sec scan in plpgsql

2016-09-15 Thread Ashutosh Bapat
ql. So, I guess, answer to your question is yes. If you are expecting something else, more context will help. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscript

Re: [HACKERS] Printing bitmap objects in the debugger

2016-09-16 Thread Ashutosh Bapat
ids) $4 = 0x2105998 "(b 1 3 4)" (gdb) p bmsToString(joinrel->lateral_relids) $5 = 0x2105db0 "(b)" (gdb) p joinrel->lateral_relids $6 = (Relids) 0x0 -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company diff --git a/src/backend/nodes/outfu

Re: [HACKERS] Why postgres take RowExclusiveLock on all partition

2016-09-16 Thread Ashutosh Bapat
On Fri, Sep 16, 2016 at 4:31 PM, Sachin Kotwal wrote: > Hi Hackers, > > > I checked if there is update transaction on master table involved in > partition. > Postgresql takes RowExclusiveLock on all partition tables. > > constraint exclusion is set to on. I checked this under the debugger and f

Re: [HACKERS] Improvements in psql hooks for variables

2016-09-18 Thread Ashutosh Bapat
ove changes, > I think we should hear it first. > > [1] > https://www.postgresql.org/message-id/f2cb5838-0ee9-4fe3-acc0-df77aeb7d4c7%40mm > [2] > https://www.postgresql.org/message-id/4695.1473961140%40sss.pgh.pa.us > > > Best regards, > -- > Daniel Vérité > Pos

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2016-09-18 Thread Ashutosh Bapat
On Fri, Sep 16, 2016 at 6:00 PM, Rajkumar Raghuwanshi wrote: > > On Fri, Sep 9, 2016 at 3:17 PM, Ashutosh Bapat > wrote: >> >> Hi All, >> >> PFA the patch to support partition-wise joins for partitioned tables. The >> patch >> is based on the declarat

Re: [HACKERS] Printing bitmap objects in the debugger

2016-09-18 Thread Ashutosh Bapat
Thanks a lot. On Fri, Sep 16, 2016 at 7:07 PM, Tom Lane wrote: > Ashutosh Bapat writes: >>> I'd suggest that this is parallel to nodeToString() and therefore >>> (a) should be placed beside it, > >> Done. Added it after nodeToString(). > > Pushed, th

[HACKERS] Calculation of param_source_rels in add_paths_to_joinrel

2016-09-19 Thread Ashutosh Bapat
path(). Also, the way this code has been written, the declaration of variable sjinfo masks the earlier declaration with the same name. I am not sure if that's intentional, but may be we should use another variable name for the inner sjinfo. I have not included that change in the patch. -- Best

[HACKERS] Re: [HACKERS] Error running custom plugin: “output plugins have to declare the _PG_output_plugin_init symbol”

2016-09-20 Thread Ashutosh Bapat
at https://wiki.postgresql.org/wiki/Getting_a_stack_trace_of_a_running_PostgreSQL_backend_on_Windows. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2016-09-20 Thread Ashutosh Bapat
Hi Rajkumar, On Fri, Sep 16, 2016 at 6:00 PM, Rajkumar Raghuwanshi wrote: > > On Fri, Sep 9, 2016 at 3:17 PM, Ashutosh Bapat > wrote: >> >> Hi All, >> >> PFA the patch to support partition-wise joins for partitioned tables. The >> patch >> is base

Re: [HACKERS] Declarative partitioning - another take

2016-09-21 Thread Ashutosh Bapat
) PARTITION BY LIST (j); >> CREATE TABLE test_array_p1 PARTITION OF test_array FOR VALUES IN ('{1}'); >> CREATE TABLE test_array_p2 PARTITION OF test_array FOR VALUES IN ('{2,2}'); >> >> INSERT INTO test_array (i,j[1],k[1]) VALUES (1,1,1); >> INS

Re: [HACKERS] Declarative partitioning - another take

2016-09-22 Thread Ashutosh Bapat
scheme match to the OIDs array to find matching RelOptInfos for partition-wise join. On Thu, Sep 22, 2016 at 11:12 AM, Ashutosh Bapat wrote: > Hi Amit, > Following sequence of DDLs gets an error > -- > -- multi-leveled partitions > -- > CREATE TABLE prt1_l (a int, b int, c var

Re: [HACKERS] Declarative partitioning - another take

2016-09-22 Thread Ashutosh Bapat
On Thu, Sep 22, 2016 at 1:02 PM, Ashutosh Bapat wrote: > For list partitions, the ListInfo stores the index maps for values > i.e. the index of the partition to which the value belongs. Those > indexes are same as the indexes in partition OIDs array and come from > the catalogs. In

Re: [HACKERS] Aggregate Push Down - Performing aggregation on foreign server

2016-09-26 Thread Ashutosh Bapat
1 where c1 = 13) q(a) right join ft2 on (q.a = ft2.c1) where ft2.c1 between 10 and 15 group by q.a order by 1 nulls last; 26. Instead of the testcase below a test which has window aggregates over a pushed down aggregate makes sense in the context of aggregate pushdown. +-- WindowAgg +explain (verb

Re: [HACKERS] Push down more full joins in postgres_fdw

2016-09-26 Thread Ashutosh Bapat
On Mon, Sep 26, 2016 at 1:05 PM, Etsuro Fujita wrote: > On 2016/09/15 15:29, Ashutosh Bapat wrote: >> >> On Wed, Sep 14, 2016 at 8:52 PM, Robert Haas >> wrote: > > >>> I'm not sure why it wouldn't work >>> to just use the lowest RTI in

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2016-09-26 Thread Ashutosh Bapat
optimized. One of the things I wanted to do is see, if those optimizations are applicable here as well. Have you considered that? [1]. https://www.postgresql.org/message-id/74355FCF-AADC-4E51-850B-47AF59E0B215%40postgrespro.ru On Fri, Aug 26, 2016 at 11:43 AM, Ashutosh Bapat wrote: > > &g

Re: [HACKERS] Push down more full joins in postgres_fdw

2016-09-26 Thread Ashutosh Bapat
On Mon, Sep 26, 2016 at 4:06 PM, Etsuro Fujita wrote: > On 2016/09/26 18:06, Ashutosh Bapat wrote: >> >> On Mon, Sep 26, 2016 at 1:05 PM, Etsuro Fujita >> wrote: > > >>> ISTM that the use of the same RTI for subqueries in multi-levels in a >>> remote

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2016-09-26 Thread Ashutosh Bapat
On Mon, Sep 26, 2016 at 5:25 PM, Masahiko Sawada wrote: > On Mon, Sep 26, 2016 at 7:28 PM, Ashutosh Bapat > wrote: >> My original patch added code to manage the files for 2 phase >> transactions opened by the local server on the remote servers. This >> code was mostly i

Re: [HACKERS] Aggregate Push Down - Performing aggregation on foreign server

2016-09-26 Thread Ashutosh Bapat
pt the FIXME over there as > I think it should have done by the core itself. > > 30, 31, 32, 33. All done. > > Let me know your views. > > Thanks > -- > Jeevan B Chalke > Principal Software Engineer, Product Development > EnterpriseDB Corporation > The Enterprise Post

Re: [HACKERS] Push down more full joins in postgres_fdw

2016-09-26 Thread Ashutosh Bapat
On Tue, Sep 27, 2016 at 8:48 AM, Etsuro Fujita wrote: > On 2016/09/26 20:20, Ashutosh Bapat wrote: >> >> On Mon, Sep 26, 2016 at 4:06 PM, Etsuro Fujita >> wrote: >>> >>> On 2016/09/26 18:06, Ashutosh Bapat wrote: >>>> >>>

Re: [HACKERS] Declarative partitioning - another take

2016-09-26 Thread Ashutosh Bapat
as corresponding OIDs. Partition-wise join simply joins the RelOptInfos at the same positions from both the parent RelOptInfos. I can not spot an error in this logic too. -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company -- Sent via pgsql-hackers mailing l

Re: [HACKERS] Declarative partitioning - another take

2016-09-27 Thread Ashutosh Bapat
tch is > certainly to be blamed here; if I revert the patch, the problem goes away. > > Based on 2 above, I attempted to add logic for AppendPath in > reparameterize_path() as in the attached. It fixes the reported problem > and does not break any regression tests. If it's

Re: [HACKERS] Declarative partitioning - another take

2016-09-27 Thread Ashutosh Bapat
On Tue, Sep 27, 2016 at 2:46 PM, Amit Langote wrote: > On 2016/09/27 15:44, Ashutosh Bapat wrote: >>> By the way, I fixed one thinko in your patch as follows: >>> >>> -result->oids[i] = oids[mapping[i]]; >>> +result->oids[mapping[

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2016-09-27 Thread Ashutosh Bapat
On Tue, Sep 27, 2016 at 2:54 PM, Masahiko Sawada wrote: > On Mon, Sep 26, 2016 at 9:07 PM, Ashutosh Bapat > wrote: >> On Mon, Sep 26, 2016 at 5:25 PM, Masahiko Sawada >> wrote: >>> On Mon, Sep 26, 2016 at 7:28 PM, Ashutosh Bapat >>> wrote: >>>>

  1   2   3   4   5   6   7   8   9   10   >