On Wed, 13 Oct 2010 19:10:42 -0400
Robert Haas wrote:
> On Tue, Oct 12, 2010 at 2:27 AM, Shigeru HANADA
> wrote:
> > Thanks for the review!
> > I'll finish the SQL/MED patch by applying your comments.
>
> I think this patch has gotten as much review as we can reasonably give
> it this CommitFest
On Tue, Oct 12, 2010 at 2:27 AM, Shigeru HANADA
wrote:
> Thanks for the review!
> I'll finish the SQL/MED patch by applying your comments.
I think this patch has gotten as much review as we can reasonably give
it this CommitFest (which is about to end), so I'm going to mark it
Returned with Feedb
On Mon, 11 Oct 2010 13:22:13 -0400
Robert Haas wrote:
> On Thu, Oct 7, 2010 at 7:20 AM, Shigeru HANADA
> wrote:
> > In this step, optimizer calls FdwRoutine->GetStatistics() to get
> > estimated costs of the foreign scan. ?FDW can calculate costs by
> > itself with some statistics, or get costs f
On Thu, Oct 7, 2010 at 7:20 AM, Shigeru HANADA
wrote:
> [ design for FDW ]
This seems mostly sensible.
> In this step, optimizer calls FdwRoutine->GetStatistics() to get
> estimated costs of the foreign scan. FDW can calculate costs by
> itself with some statistics, or get costs from remote-sid
On Tue, 05 Oct 2010 14:59:27 -0400
Tom Lane wrote:
> Robert Haas writes:
> > I wonder if we
> > should focus on first efforts on really simple cases like CSV files
> > (as you mentioned) and perhaps something like memcached, which has
> > different properties than a CSV file, but extremely simple
Robert Haas writes:
> I'm somewhat afraid that a remote-PG adapter will turn into a can of
> worms. If we give up on remote statistics, does that mean we're also
> giving up on index use on the remote side? I fear that we'll end up
> crafting partial solutions that will only end up getting throw
On Tue, Oct 5, 2010 at 12:49 PM, Tom Lane wrote:
> Robert Haas writes:
>> On Tue, Oct 5, 2010 at 11:06 AM, Tom Lane wrote:
>>> IMO this is a *clear* case of premature optimization being the root of
>>> all evil. We should get it working first and then see what needs to be
>>> optimized by measu
Robert Haas writes:
> On Tue, Oct 5, 2010 at 11:06 AM, Tom Lane wrote:
>> IMO this is a *clear* case of premature optimization being the root of
>> all evil. We should get it working first and then see what needs to be
>> optimized by measuring, rather than guessing in a vacuum.
> I have no pro
On Tue, Oct 5, 2010 at 11:06 AM, Tom Lane wrote:
> Robert Haas writes:
>> On Tue, Oct 5, 2010 at 10:41 AM, Tom Lane wrote:
>>> (I'd also say that your performance estimate is miles in advance of any
>>> facts; but even if it's true, the caching ought to be inside the FDW,
>>> because we have no
Heikki Linnakangas writes:
> On 05.10.2010 17:56, Robert Haas wrote:
>> I can't imagine how an FDW could possibly be expected to perform well
>> without some persistent local data storage.
> It doesn't seem completely out of the question to me.
One other point that could be made here is that it'
On 05.10.2010 17:56, Robert Haas wrote:
On Tue, Oct 5, 2010 at 10:41 AM, Tom Lane wrote:
(I'd also say that your performance estimate is miles in advance of any
facts; but even if it's true, the caching ought to be inside the FDW,
because we have no clear idea of what it will need to cache.)
Robert Haas writes:
> On Tue, Oct 5, 2010 at 10:41 AM, Tom Lane wrote:
>> (I'd also say that your performance estimate is miles in advance of any
>> facts; but even if it's true, the caching ought to be inside the FDW,
>> because we have no clear idea of what it will need to cache.)
> I can't im
On Tue, Oct 5, 2010 at 10:41 AM, Tom Lane wrote:
> Robert Haas writes:
>> On Tue, Oct 5, 2010 at 10:25 AM, Tom Lane wrote:
>>> This whole discussion seems to me to be about trying to do things outside
>>> the FDW that should properly be left inside the FDW. Who's to say that
>>> the remote side
Robert Haas writes:
> On Tue, Oct 5, 2010 at 10:25 AM, Tom Lane wrote:
>> This whole discussion seems to me to be about trying to do things outside
>> the FDW that should properly be left inside the FDW. Who's to say that
>> the remote side even *has* statistics of the sort that PG creates?
>>
On Tue, Oct 5, 2010 at 8:10 AM, Shigeru HANADA
wrote:
> On Mon, 4 Oct 2010 19:31:52 -0400
> Robert Haas wrote:
>> On Thu, Sep 30, 2010 at 3:48 AM, Shigeru HANADA
>> wrote:
>> > How about having cost hints in generic option of the foreign table or
>> > its columns? ?Generic options are storage fo
On Tue, Oct 5, 2010 at 10:25 AM, Tom Lane wrote:
> Shigeru HANADA writes:
>> Can we treat statistics of a foreign table separately?
>
>> 1. Same as local tables (maybe required)
>> (pg_statistic.*, pg_class.reltuples/relpages)
>
> This whole discussion seems to me to be about trying to do thin
Shigeru HANADA writes:
> Can we treat statistics of a foreign table separately?
> 1. Same as local tables (maybe required)
>(pg_statistic.*, pg_class.reltuples/relpages)
This whole discussion seems to me to be about trying to do things outside
the FDW that should properly be left inside the
On Mon, 4 Oct 2010 19:31:52 -0400
Robert Haas wrote:
> On Thu, Sep 30, 2010 at 3:48 AM, Shigeru HANADA
> wrote:
> > How about having cost hints in generic option of the foreign table or
> > its columns? ?Generic options are storage for wrappers, not for
> > PostgreSQL core modules. ?Wrappers can
On Thu, Sep 30, 2010 at 3:48 AM, Shigeru HANADA
wrote:
> On Thu, 30 Sep 2010 09:26:54 +0300
> Heikki Linnakangas wrote:
>> > FYI, HiRDB, that implements FDW routines, has CREATE FOREIGN INDEX.
>> > I think it is a little ugly and won't work in some cases -- for example,
>> > index organized table
On Wed, Sep 29, 2010 at 3:56 AM, Shigeru HANADA
wrote:
> In current patch, fdwoptions for relations have been separated from
> reloptins by introducing pg_foreign_table catalog. As mentioned in
> wiki, integration into rel/attoptions is nothing but an idea, we're
> willing to add pg_foreign_attri
On Fri, Oct 01, 2010 at 10:54:47AM -0400, Tom Lane wrote:
> David Fetter writes:
> > On Thu, Sep 30, 2010 at 10:29:56PM -0400, Robert Haas wrote:
> >> Yeah, that might be better. Is it reasonable to assume we always
> >> want to push down as much as possible, or do we need to think about
> >> loc
David Fetter writes:
> On Thu, Sep 30, 2010 at 10:29:56PM -0400, Robert Haas wrote:
>> Yeah, that might be better. Is it reasonable to assume we always
>> want to push down as much as possible, or do we need to think about
>> local work vs. remote work trade-offs?
> In cases where the foreign si
On Thu, Sep 30, 2010 at 10:29:56PM -0400, Robert Haas wrote:
> On Sep 29, 2010, at 10:09 AM, Alvaro Herrera
> wrote:
> > Excerpts from Robert Haas's message of mar sep 28 10:26:54 -0400 2010:
> >
> >> Then:
> >>
> >> - Begin a sequential scan with the following set of quals.
> >> - Begin an ind
On Sep 29, 2010, at 10:09 AM, Alvaro Herrera wrote:
> Excerpts from Robert Haas's message of mar sep 28 10:26:54 -0400 2010:
>
>> Then:
>>
>> - Begin a sequential scan with the following set of quals.
>> - Begin an index scan using the index called X with the following set of
>> quals.
>> - Fet
On Thu, 30 Sep 2010 09:26:54 +0300
Heikki Linnakangas wrote:
> > FYI, HiRDB, that implements FDW routines, has CREATE FOREIGN INDEX.
> > I think it is a little ugly and won't work in some cases -- for example,
> > index organized tables -- but evidently it's a realistic solution.
>
> A long time
On 30.09.2010 04:30, Itagaki Takahiro wrote:
On Wed, Sep 29, 2010 at 11:09 PM, Alvaro Herrera
wrote:
I'm not sure that it's a good idea to embed into the FDW API the set of
operations known to the executor. For example your proposal fails to
consider bitmap scans. Seems simpler and more gene
On Wed, Sep 29, 2010 at 11:09 PM, Alvaro Herrera
wrote:
> I'm not sure that it's a good idea to embed into the FDW API the set of
> operations known to the executor. For example your proposal fails to
> consider bitmap scans. Seems simpler and more general to hand the quals
> over saying "I need
Excerpts from Robert Haas's message of mar sep 28 10:26:54 -0400 2010:
> Then:
>
> - Begin a sequential scan with the following set of quals.
> - Begin an index scan using the index called X with the following set of
> quals.
> - Fetch next tuple.
> - End scan.
I'm not sure that it's a good ide
On Tue, 28 Sep 2010 22:14:22 +0300
Heikki Linnakangas wrote:
> On 09/28/10 17:26, Robert Haas wrote:
> > First, it seems totally wrong to assume that the same functions and
> > operators will be defined on the remote side as you have locally;
> > indeed, for CSV files, you won't have anything defi
On Tue, 28 Sep 2010 10:26:42 -0400
Robert Haas wrote:
> On Mon, Sep 27, 2010 at 2:50 AM, SAKAMOTO Masahiko
> wrote:
> > ?http://wiki.postgresql.org/wiki/SQL/MED
> With regard to what is written here, it strikes me that it would be an
> extremely bad idea to try to mix reloptions or attoptions wit
On 09/28/10 17:26, Robert Haas wrote:
First, it seems totally wrong to assume that the same functions and
operators will be defined on the remote side as you have locally;
indeed, for CSV files, you won't have anything defined on the remote
side at all. You need some kind of a discovery mechanis
On Mon, Sep 27, 2010 at 2:50 AM, SAKAMOTO Masahiko
wrote:
> http://wiki.postgresql.org/wiki/SQL/MED
With regard to what is written here, it strikes me that it would be an
extremely bad idea to try to mix reloptions or attoptions with
fdwoptions. fdwoptions are options to be passed transparently
On Mon, Sep 27, 2010 at 2:50 AM, SAKAMOTO Masahiko
wrote:
> http://wiki.postgresql.org/wiki/SQL/MED
With regard to what is written here, it strikes me that it would be an
extremely bad idea to try to mix reloptions or attoptions with
fdwoptions. fdwoptions are options to be passed transparently
Hi hackers,
On Mon, 27 Sep 2010 15:50:34 +0900
SAKAMOTO Masahiko wrote:
> Right. In any case, I should clearify what this API could cover
> by this patch and what could not.
> # And also how far I and my collaborator can implement..
>
> As Itagaki points out, we have two types of FDW implementat
sorry for my slow response.
(2010/09/24 21:13), Robert Haas wrote:
This is a good point. On the one hand, I do agree that the API for
simple things like processing CSV files shouldn't be overly complex.
So perhaps we could start with a simple API and extend it later. On
the other hand, underst
On Fri, Sep 24, 2010 at 5:56 AM, Itagaki Takahiro
wrote:
> I think there are two type of FDWs. One is a simple flat file wrapper
> used by COPY FROM now, that doesn't require any planner hooks.
> Another is a connector to an external database, like as dblink, that
> should be integrated with the p
On Thu, Sep 23, 2010 at 11:26 PM, Itagaki Takahiro
wrote:
> There are no active discussions :-( I think the author tried his best, so if
> other developers think it's a bad design, alternate plan must be proposed.
>
> Also, if the syntax change is trivial, that's why we merge it at
> earlier comm
On Fri, Sep 24, 2010 at 6:12 PM, Heikki Linnakangas
wrote:
> It's not that the design is bad, it's that it's non-existent. I haven't seen
> any design on how this integrates with the planner.
In my understanding, the DDL part is independent from planner integration,
and that's why the author extr
On 24/09/10 06:26, Itagaki Takahiro wrote:
On Fri, Sep 24, 2010 at 12:08 PM, Robert Haas wrote:
I think we need to further discuss how this is eventually going to get
integrated with the query planner and the executor before we commit
anything. The syntax support by itself is quite trivial.
On Fri, Sep 24, 2010 at 12:08 PM, Robert Haas wrote:
> I think we need to further discuss how this is eventually going to get
> integrated with the query planner and the executor before we commit
> anything. The syntax support by itself is quite trivial.
There are no active discussions :-( I th
On Thu, Sep 23, 2010 at 10:55 PM, Itagaki Takahiro
wrote:
> Overall, I think the FOREIGN TABLE DDL part is almost ready to commit if we
> accept the proposal itself. Of course we need more discussions about FDW
> Routines, but it must be developed based on the infrastructure.
I think we need to f
On Fri, Sep 17, 2010 at 7:47 PM, SAKAMOTO Masahiko
wrote:
> Thanks for your comment. I've updated the patches.
I reviewed fdw_table20100917.patch.gz. It adds FOREIGN TABLE in the
SQL standard,
including CREATE/ALTER/DROP FOREIGN TABLE, information_schema (foreign_table
and foreign_table_options),
2010/9/15 SAKAMOTO Masahiko :
> This is a proposal patch for SQL/MED for 9.1.
> (1) foreign table DDL support (this proposal):
> - support for foreign table DDL syntax (CREATE/ALTER FOREIGN TABLE)
> - Definition of FDW routine interface and system catalogs for it.
I checked the patch. It
On Wed, Sep 15, 2010 at 10:05:00PM -0400, Tom Lane wrote:
> Mark Kirkwood writes:
> > On 16/09/10 13:22, Tom Lane wrote:
> >> What exactly do those get you that an ordinary index, or at worst an
> >> index-organized table, doesn't get you?
>
> > It is pretty rare to see key value stores vs relati
On Thu, Sep 16, 2010 at 3:15 PM, SAKAMOTO Masahiko
wrote:
> This is designed to be used in the executor module.
> The API has some simple interfaces, such as FreeConnection,
> Open, Iterate, Close, ReOpen, and OnError.
Of course they are similar APIs mentioned in the SQL standard, but
I'm not sur
And this is sample script to play the feature of this patch.
It includes:
- create foreign data wrapper and foreign server(PG at localhost)
- select foreign PostgreSQL table.
*Important*
This script initializes your database cluster(specified by $PGDATA).
Run with care
Regards,
SAKAMOTO
2010/9/16 Mark Kirkwood :
> On 16/09/10 14:05, Robert Haas wrote:
>>
>> On Wed, Sep 15, 2010 at 9:22 PM, Tom Lane wrote:
>>
>>>
>>> Hitoshi Harada writes:
>>>
2010/9/16 Robert Haas:
>
> Oh, key-value store, I bet. Yeah, that would be cool.
>
>>>
>>>
That's it
On 16/09/10 14:05, Robert Haas wrote:
On Wed, Sep 15, 2010 at 9:22 PM, Tom Lane wrote:
Hitoshi Harada writes:
2010/9/16 Robert Haas:
Oh, key-value store, I bet. Yeah, that would be cool.
That's it. Like Redis, Tokyo Cabinet, or something.
What exa
2010/9/15 Heikki Linnakangas :
> The API needs to be simple and version-independent, so that you can
> write simple wrappers like the flat file wrapper easily. At the same
> time, it needs to be very flexible, so that it allows safely pushing
> down all kinds constructs like functions, aggregates a
On Wed, Sep 15, 2010 at 9:22 PM, Tom Lane wrote:
> Hitoshi Harada writes:
>> 2010/9/16 Robert Haas :
>>> Oh, key-value store, I bet. Yeah, that would be cool.
>
>> That's it. Like Redis, Tokyo Cabinet, or something.
>
> What exactly do those get you that an ordinary index, or at worst an
> index
Mark Kirkwood writes:
> On 16/09/10 13:22, Tom Lane wrote:
>> What exactly do those get you that an ordinary index, or at worst an
>> index-organized table, doesn't get you?
> It is pretty rare to see key value stores vs relational engines
> discussed without a descent into total foolishiness, b
On 16/09/10 13:22, Tom Lane wrote:
Hitoshi Harada writes:
2010/9/16 Robert Haas:
Oh, key-value store, I bet. Yeah, that would be cool.
That's it. Like Redis, Tokyo Cabinet, or something.
What exactly do those get you that an ordinary index, or at worst an
index-o
Hitoshi Harada writes:
> 2010/9/16 Robert Haas :
>> Oh, key-value store, I bet. Yeah, that would be cool.
> That's it. Like Redis, Tokyo Cabinet, or something.
What exactly do those get you that an ordinary index, or at worst an
index-organized table, doesn't get you?
r
2010/9/16 Robert Haas :
> On Wed, Sep 15, 2010 at 8:57 PM, Robert Haas wrote:
>> On Wed, Sep 15, 2010 at 8:53 PM, Hitoshi Harada wrote:
>>> 2010/9/16 Robert Haas :
On Wed, Sep 15, 2010 at 3:00 PM, Tom Lane wrote:
> Yet there are other cases that probably *could* work well based on a
>>>
On Wed, Sep 15, 2010 at 8:57 PM, Robert Haas wrote:
> On Wed, Sep 15, 2010 at 8:53 PM, Hitoshi Harada wrote:
>> 2010/9/16 Robert Haas :
>>> On Wed, Sep 15, 2010 at 3:00 PM, Tom Lane wrote:
Yet there are other cases that probably *could* work well based on a
storage-level abstraction bo
2010/9/16 Robert Haas :
> On Wed, Sep 15, 2010 at 3:00 PM, Tom Lane wrote:
>> Yet there are other cases that probably *could* work well based on a
>> storage-level abstraction boundary; index-organized tables for instance.
>> So I think we need to have some realistic idea of what we want to
>> sup
On Wed, Sep 15, 2010 at 8:53 PM, Hitoshi Harada wrote:
> 2010/9/16 Robert Haas :
>> On Wed, Sep 15, 2010 at 3:00 PM, Tom Lane wrote:
>>> Yet there are other cases that probably *could* work well based on a
>>> storage-level abstraction boundary; index-organized tables for instance.
>>> So I think
On Wed, Sep 15, 2010 at 3:00 PM, Tom Lane wrote:
> Well, the problem is to not draw the abstraction boundary so high that
> your plugins have to reimplement the world to get anything done.
> mysql got this wrong IMO, and are still paying the price in the form of
> bizarre functional incompatibilit
Robert Haas writes:
> 2010/9/15 Heikki Linnakangas :
>> I've been thinking about that for some time. Here's one sketch I made a
>> while ago:
>> http://archives.postgresql.org/pgsql-hackers/2008-07/msg00395.php.
> I wonder if we might not think of the foreign data wrapper problem as
> an instance
On Wed, Sep 15, 2010 at 2:28 PM, Heikki Linnakangas
wrote:
> On 15/09/10 21:21, Robert Haas wrote:
>>
>> I wonder if we might not think of the foreign data wrapper problem as
>> an instance of an even more general problem. Right now, we have
>> semi-pluggable index access methods - it's probably
On 15/09/10 21:21, Robert Haas wrote:
I wonder if we might not think of the foreign data wrapper problem as
an instance of an even more general problem. Right now, we have
semi-pluggable index access methods - it's probably not quite possible
to implement one as a standalone module because of XL
2010/9/15 Heikki Linnakangas :
> In my mind the key question is: what does the API for writing foreign
> data wrappers look like? I couldn't figure that out by looking at the patch.
>
> The API needs to be simple and version-independent, so that you can
> write simple wrappers like the flat file wr
On 15/09/10 08:46, SAKAMOTO Masahiko wrote:
> This is a proposal patch for SQL/MED for 9.1.
>
> At the prev. CF, this patch had so many features to make it hard
> to review all of them. So I devided it into smaller parts:
> (1) foreign table DDL support (this proposal):
> - support for fo
Hello.
This is a proposal patch for SQL/MED for 9.1.
At the prev. CF, this patch had so many features to make it hard
to review all of them. So I devided it into smaller parts:
(1) foreign table DDL support (this proposal):
- support for foreign table DDL syntax (CREATE/ALTER FOREIGN TABLE)
64 matches
Mail list logo
