On Mon, Sep 7, 2015 at 1:33 PM, Ahsan Hadi <ahsan.h...@enterprisedb.com> wrote: > I > > On Monday, September 7, 2015, Ashutosh Bapat > <ashutosh.ba...@enterprisedb.com> wrote: >> >> >> >> On Sat, Sep 5, 2015 at 4:22 AM, Ozgun Erdogan <oz...@citusdata.com> wrote: >>> >>> Hey Robert, >>> >>>> Now the question is, where should the code that does all of this live? >>>> postgres_fdw? Some new, sharding-specific FDW? In core? I don't >>>> know for sure, but what I do know is that we could make a lot of >>>> progress over where we are today by just improving postgres_fdw, and I >>>> don't think those improvements are even all that difficult. If we >>>> decide we need to implement something new, it's going to be a huge >>>> project that will take years to complete, with uncertain results. I'd >>>> rather have a postgres_fdw-based implementation that is imperfect and >>>> can't handle some kinds of queries in 9.6 than a promise that by 9.9 >>>> we'll have something really great that handles MPP perfectly. >>> >>> >>> Distributed shuffles (Map/Reduce) are hard. When we looked at using FDWs >>> for pg_shard, we thought that Map/Reduce would require a comprehensive >>> revamp of the APIs. >>> >>> For Citus, a second part of the question is as FDW writers. We >>> implemented cstore_fdw, json_fdw, and mongo_fdw, and these wrappers don't >>> benefit from even the simple join pushdown that doesn't require Map/Reduce. >> >> >> I didn't get this. Join pushdown infrastructure (chiefly set of hooks >> provided in join planning paths) is part of 9.5. Isn't that sufficient to >> implement join push-down for above FDWs? Or FDW writers are facing problems >> while implementing those hooks. In either case that should be reported on >> hackers. >> > > > I don't think any FDW writer (other the postgres_fdw) has tried to implement > join push down in the respective FDW's using the new API.
Well, 'jdbc_fdw2' seems to implement deparsing at some level: https://github.com/heimir-sverrisson/jdbc2_fdw/blob/master/deparse.c ...but this was likely a copy/paste job from the postgres_fdw. It should not escape note that the deparsing strategy has dependencies on the optimizer and the parser. This is not good; asking the FDW implementations to implement SQL optimizations is not a sustainable solution. They should be indicating, for example, "I support ANSI-92 SQL", and the postgres planner should be deparsing the foreign table definition and rewriting it, not the other way around. merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
