Re: Foreign Data Wrappers

2024-09-06 Thread Brent Wood
ery useful functionality. Brent Wood On Saturday, September 7, 2024 at 12:55:37 PM GMT+12, Gus Spier wrote: I find myself in new territory, Foreign Data Wrappers (FDW). I stumble closer to success through the documentation, youtube videos, and various google resources. But I'v

Re: Foreign Data Wrappers

2024-09-06 Thread Gus Spier
Thanks for the pointers! I'll resume my task in the morning. If I get anywhere, I will post the outcome. R, Gus On Fri, Sep 6, 2024 at 9:35 PM Tom Lane wrote: > Gus Spier writes: > > If I understand the concepts correctly, FDW not only makes other > databases > > available, FDW also offers ac

Re: Foreign Data Wrappers

2024-09-06 Thread Tom Lane
Gus Spier writes: > If I understand the concepts correctly, FDW not only makes other databases > available, FDW also offers access to .csv files, plain text, or just about > anything that can be bullied into some kind of query-able order. Has anyone > ever tried to connect to redis or elasticache?

Re: Foreign Data Wrappers

2024-09-06 Thread Ian Lawrence Barwick
2024年9月7日(土) 9:55 Gus Spier : > > I find myself in new territory, Foreign Data Wrappers (FDW). I stumble closer > to success through the documentation, youtube videos, and various google > resources. But I've come across a concept that intrigues me very deeply. > > If I

Re: Foreign Data Wrappers

2024-09-06 Thread Christophe Pettus
> On Sep 6, 2024, at 17:55, Gus Spier wrote: > If I understand the concepts correctly, FDW not only makes other databases > available, FDW also offers access to .csv files, plain text, or just about > anything that can be bullied into some kind of query-able order. There two parts to FDWs: T

Foreign Data Wrappers

2024-09-06 Thread Gus Spier
I find myself in new territory, Foreign Data Wrappers (FDW). I stumble closer to success through the documentation, youtube videos, and various google resources. But I've come across a concept that intrigues me very deeply. If I understand the concepts correctly, FDW not only makes

Re: Implementing foreign data wrappers and avoiding n+1 querying

2022-12-23 Thread Brad White
We had a similar situation in a completely different context. Our eventual solution was to fire off a request as soon as one came in. Then we batched further requests until the first returned. Whenever a request returned, we sent any pending requests. Any single request not sent immediately was slo

Re: Implementing foreign data wrappers and avoiding n+1 querying

2022-12-22 Thread David Gilman
I apologize that my post was not super clear, I am thinking about implementing a fdw from scratch, and the target database is one of those NoSQL databases where you have to send JSON over a HTTP connection for each query. I have reviewed the postgres fdw code to see how it works and to see what's

Re: Implementing foreign data wrappers and avoiding n+1 querying

2022-12-21 Thread David Rowley
On Thu, 22 Dec 2022 at 13:31, David Gilman wrote: > > When a fdw table participates in query planning and finds itself as > part of a join it can output a parameterized path. If chosen, Postgres > will dutifully call the fdw over and over via IterateForeignScan to > fetch matching tuples. Many fdw

Implementing foreign data wrappers and avoiding n+1 querying

2022-12-21 Thread David Gilman
When a fdw table participates in query planning and finds itself as part of a join it can output a parameterized path. If chosen, Postgres will dutifully call the fdw over and over via IterateForeignScan to fetch matching tuples. Many fdw extensions do network traffic, though, and it would be benef