Re: [HACKERS] Status of FDW pushdowns

2013-12-04 Thread Tom Lane
David Fetter writes: > On Wed, Dec 04, 2013 at 03:04:31PM -0500, Tom Lane wrote: >> If that's the argument, why not just use dblink or dbilink, and be >> happy? This discussion sounds a whole lot like it's trending to a >> conclusion of wanting one of those in core, which is not where I'd >> like

Re: [HACKERS] Status of FDW pushdowns

2013-12-04 Thread David Fetter
On Wed, Dec 04, 2013 at 03:04:31PM -0500, Tom Lane wrote: > David Fetter writes: > > The idea here is that such a happy situation will not obtain until > > much later, if ever, and meanwhile, we need a way to get things > > accomplished even if it's inelegant, inefficient, etc. The > > alternativ

Re: [HACKERS] Status of FDW pushdowns

2013-12-04 Thread Tom Lane
Merlin Moncure writes: > The downside of SQL-MED, particularly the way postgres implemented the > driver API, is that each driver is responsible for for all > optimization efforts and I think this is bad. There was never any intention that that would be the final state of things. All the FDW API

Re: [HACKERS] Status of FDW pushdowns

2013-12-04 Thread Merlin Moncure
On Wed, Dec 4, 2013 at 1:39 PM, David Fetter wrote: > On Wed, Dec 04, 2013 at 12:43:44PM -0600, Merlin Moncure wrote: >> On Mon, Dec 2, 2013 at 10:26 PM, David Fetter wrote: >> > On Tue, Dec 03, 2013 at 11:15:36AM +0800, Craig Ringer wrote: >> >> On 11/28/2013 03:24 AM, David Fetter wrote: >> >>

Re: [HACKERS] Status of FDW pushdowns

2013-12-04 Thread Tom Lane
David Fetter writes: > The idea here is that such a happy situation will not obtain until > much later, if ever, and meanwhile, we need a way to get things > accomplished even if it's inelegant, inefficient, etc. The > alternative is that those things simply will not get accomplished at > all. I

Re: [HACKERS] Status of FDW pushdowns

2013-12-04 Thread David Fetter
On Wed, Dec 04, 2013 at 12:43:44PM -0600, Merlin Moncure wrote: > On Mon, Dec 2, 2013 at 10:26 PM, David Fetter wrote: > > On Tue, Dec 03, 2013 at 11:15:36AM +0800, Craig Ringer wrote: > >> On 11/28/2013 03:24 AM, David Fetter wrote: > >> > WITH, or SRF, or whatever, the point is that we need to b

Re: [HACKERS] Status of FDW pushdowns

2013-12-04 Thread Merlin Moncure
On Mon, Dec 2, 2013 at 10:26 PM, David Fetter wrote: > On Tue, Dec 03, 2013 at 11:15:36AM +0800, Craig Ringer wrote: >> On 11/28/2013 03:24 AM, David Fetter wrote: >> > WITH, or SRF, or whatever, the point is that we need to be able to >> > specify what we're sending--probably single opaque string

Re: [HACKERS] Status of FDW pushdowns

2013-12-02 Thread David Fetter
On Tue, Dec 03, 2013 at 11:15:36AM +0800, Craig Ringer wrote: > On 11/28/2013 03:24 AM, David Fetter wrote: > > WITH, or SRF, or whatever, the point is that we need to be able to > > specify what we're sending--probably single opaque strings delimited > > just as we do other strings--and what we mi

Re: [HACKERS] Status of FDW pushdowns

2013-12-02 Thread Craig Ringer
On 11/28/2013 03:24 AM, David Fetter wrote: > WITH, or SRF, or whatever, the point is that we need to be able to > specify what we're sending--probably single opaque strings delimited > just as we do other strings--and what we might get back--errors only, > rows, [sets of] refcursors are the ones I

Re: [HACKERS] Status of FDW pushdowns

2013-11-29 Thread Atri Sharma
Sent from my iPad > On 28-Nov-2013, at 16:13, Dimitri Fontaine wrote: > > Tom Lane writes: >> I'm not real sure what this'd buy us that wouldn't be done as well or >> better by creating a view on the remote side. (IOW, there's nothing >> that says that the remote object backing a foreign tab

Re: [HACKERS] Status of FDW pushdowns

2013-11-28 Thread Dimitri Fontaine
Tom Lane writes: > I'm not real sure what this'd buy us that wouldn't be done as well or > better by creating a view on the remote side. (IOW, there's nothing > that says that the remote object backing a foreign table can't be a > view.) Agreed, for those remote sides that know what a view is. -

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread Tom Lane
Dimitri Fontaine writes: > Atri Sharma writes: >> Can we add a function to the FDW API to define a SQL to foreign server >> side conversion? > I think that's not tenable. Even if you limit the discussion to the > postgres_fdw, some queries against past version will stop working > against new ver

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread Dimitri Fontaine
Atri Sharma writes: > Can we add a function to the FDW API to define a SQL to foreign server > side conversion? I think that's not tenable. Even if you limit the discussion to the postgres_fdw, some queries against past version will stop working against new version (keywords changes, catalogs, de

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread David Fetter
On Thu, Nov 28, 2013 at 01:29:46AM +0530, Atri Sharma wrote: > On Thu, Nov 28, 2013 at 12:54 AM, David Fetter wrote: > > On Wed, Nov 27, 2013 at 10:29:34AM -0300, Alvaro Herrera wrote: > >> Shigeru Hanada escribió: > >> > >> > SQL/MED standard doesn't say much about PASS THROUGH mode, especially >

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread Atri Sharma
On Thu, Nov 28, 2013 at 12:54 AM, David Fetter wrote: > On Wed, Nov 27, 2013 at 10:29:34AM -0300, Alvaro Herrera wrote: >> Shigeru Hanada escribió: >> >> > SQL/MED standard doesn't say much about PASS THROUGH mode, especially >> > about interaction between client. Besides it, I think it would be

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread David Fetter
On Wed, Nov 27, 2013 at 10:29:34AM -0300, Alvaro Herrera wrote: > Shigeru Hanada escribió: > > > SQL/MED standard doesn't say much about PASS THROUGH mode, especially > > about interaction between client. Besides it, I think it would be > > nice to allow arbitrary FDW as backend of dblink interfa

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread Atri Sharma
On Wed, Nov 27, 2013 at 11:08 PM, Dimitri Fontaine wrote: > Atri Sharma writes: >> This would work,but how can we do it for FDWs which do not parse SQL? >> Am I missing something here? > > Worst case: > > CREATE FOREIGN VIEW foo > AS $$ > whatever syntax is accepted on the other side Th

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread Dimitri Fontaine
Atri Sharma writes: > This would work,but how can we do it for FDWs which do not parse SQL? > Am I missing something here? Worst case: CREATE FOREIGN VIEW foo AS $$ whatever syntax is accepted on the other side $$; Regards, -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : E

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread Atri Sharma
> I guess the view query would have to be validated by the FDW, which > would just receive a text. +1 This is not exactly in context, but I and David Fetter discussed recently how we could do similar thing. This would work,but how can we do it for FDWs which do not parse SQL? Am I missing somet

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread Dimitri Fontaine
Shigeru Hanada writes: > I'm sorry but I don't see the point here. Do you mean that user > executes CREATE FOREIGN VIEW in advance and uses the view in a Yes that's what I mean. > I think it's nice to support executing ad-hoc remote query written in > the syntax which is valid only on remote da

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread Shigeru Hanada
2013/11/27 Dimitri Fontaine : > Alvaro Herrera writes: >> Seems to me that if you want to read remote tables without creating a >> foreign table, you could define them locally using something like the >> WITH syntax and then use them normally in the rest of the query. > > I guess what's needed her

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread Merlin Moncure
On Wed, Nov 27, 2013 at 4:20 AM, Shigeru Hanada wrote: > Hi Merlin, > > 2013/11/22 Merlin Moncure : >> On Thu, Nov 21, 2013 at 6:43 PM, Shigeru Hanada >> wrote: >>> 2013/11/22 Tom Lane : Merlin Moncure writes: > On Thu, Nov 21, 2013 at 9:05 AM, Bruce Momjian wrote: >> I know join p

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread Dimitri Fontaine
Alvaro Herrera writes: > Seems to me that if you want to read remote tables without creating a > foreign table, you could define them locally using something like the > WITH syntax and then use them normally in the rest of the query. I guess what's needed here is a kind of barrier that allows pus

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread Alvaro Herrera
Shigeru Hanada escribió: > SQL/MED standard doesn't say much about PASS THROUGH mode, especially > about interaction between client. Besides it, I think it would be > nice to allow arbitrary FDW as backend of dblink interface like this: > > postgres=> SELECT dblink_connect('con1', 'server name o

Re: [HACKERS] Status of FDW pushdowns

2013-11-27 Thread Shigeru Hanada
Hi Merlin, 2013/11/22 Merlin Moncure : > On Thu, Nov 21, 2013 at 6:43 PM, Shigeru Hanada > wrote: >> 2013/11/22 Tom Lane : >>> Merlin Moncure writes: On Thu, Nov 21, 2013 at 9:05 AM, Bruce Momjian wrote: > I know join pushdowns seem insignificant, but it helps to restrict what > da

Re: [HACKERS] Status of FDW pushdowns

2013-11-25 Thread Dimitri Fontaine
Merlin Moncure writes: > By 'insignificant' you mean 'necessary to do any non-trivial real > work'. Personally, I'd prefer it if FDW was extended to allow > arbitrary parameterized queries like every other database connectivity > API ever made ever. But in lieu of that, I'll take as much push d

Re: [HACKERS] Status of FDW pushdowns

2013-11-22 Thread David Fetter
On Fri, Nov 22, 2013 at 08:25:05AM -0600, Merlin Moncure wrote: > On Thu, Nov 21, 2013 at 6:43 PM, Shigeru Hanada > wrote: > > 2013/11/22 Tom Lane : > >> Merlin Moncure writes: > >>> On Thu, Nov 21, 2013 at 9:05 AM, Bruce Momjian wrote: > I know join pushdowns seem insignificant, but it hel

Re: [HACKERS] Status of FDW pushdowns

2013-11-22 Thread Merlin Moncure
On Thu, Nov 21, 2013 at 6:43 PM, Shigeru Hanada wrote: > 2013/11/22 Tom Lane : >> Merlin Moncure writes: >>> On Thu, Nov 21, 2013 at 9:05 AM, Bruce Momjian wrote: I know join pushdowns seem insignificant, but it helps to restrict what data must be passed back because you would only pas

Re: [HACKERS] Status of FDW pushdowns

2013-11-21 Thread David Fetter
On Thu, Nov 21, 2013 at 10:46:14AM -0500, Tom Lane wrote: > Merlin Moncure writes: > > On Thu, Nov 21, 2013 at 9:05 AM, Bruce Momjian wrote: > >> I know join pushdowns seem insignificant, but it helps to restrict what > >> data must be passed back because you would only pass back joined rows. >

Re: [HACKERS] Status of FDW pushdowns

2013-11-21 Thread Shigeru Hanada
2013/11/22 Tom Lane : > Merlin Moncure writes: >> On Thu, Nov 21, 2013 at 9:05 AM, Bruce Momjian wrote: >>> I know join pushdowns seem insignificant, but it helps to restrict what >>> data must be passed back because you would only pass back joined rows. > >> By 'insignificant' you mean 'necessar

Re: [HACKERS] Status of FDW pushdowns

2013-11-21 Thread Shigeru Hanada
2013/11/22 Kohei KaiGai : > 2013/11/21 Bruce Momjian : >> Where are we on the remaining possible pushdowns for foreign data >> wrappers, particularly the Postgres one? I know we do WHERE restriction >> pushdowns in 9.3, but what about join and aggregate pushdowns? Is >> anyone working on those? >

Re: [HACKERS] Status of FDW pushdowns

2013-11-21 Thread Tom Lane
Merlin Moncure writes: > On Thu, Nov 21, 2013 at 9:05 AM, Bruce Momjian wrote: >> I know join pushdowns seem insignificant, but it helps to restrict what >> data must be passed back because you would only pass back joined rows. > By 'insignificant' you mean 'necessary to do any non-trivial real

Re: [HACKERS] Status of FDW pushdowns

2013-11-21 Thread Kohei KaiGai
2013/11/21 Bruce Momjian : > Where are we on the remaining possible pushdowns for foreign data > wrappers, particularly the Postgres one? I know we do WHERE restriction > pushdowns in 9.3, but what about join and aggregate pushdowns? Is > anyone working on those? > > I know join pushdowns seem in

Re: [HACKERS] Status of FDW pushdowns

2013-11-21 Thread Merlin Moncure
On Thu, Nov 21, 2013 at 9:05 AM, Bruce Momjian wrote: > Where are we on the remaining possible pushdowns for foreign data > wrappers, particularly the Postgres one? I know we do WHERE restriction > pushdowns in 9.3, but what about join and aggregate pushdowns? Is > anyone working on those? > > I

Re: [HACKERS] Status of FDW pushdowns

2013-11-21 Thread Tom Lane
Kohei KaiGai writes: > Right now, it performs on relation scan or join only. However, we will be > able to apply same concept on aggregation. > For example, an aggregation node on a foreign table scan is a good > candidate to push down because it can be replaced with a custom- > logic that scans a

[HACKERS] Status of FDW pushdowns

2013-11-21 Thread Bruce Momjian
Where are we on the remaining possible pushdowns for foreign data wrappers, particularly the Postgres one? I know we do WHERE restriction pushdowns in 9.3, but what about join and aggregate pushdowns? Is anyone working on those? I know join pushdowns seem insignificant, but it helps to restrict