Re: Implementing foreign data wrappers and avoiding n+1 querying

2022-12-23 Thread Brad White
We had a similar situation in a completely different context. Our eventual solution was to fire off a request as soon as one came in. Then we batched further requests until the first returned. Whenever a request returned, we sent any pending requests. Any single request not sent immediately was slo

Re: Implementing foreign data wrappers and avoiding n+1 querying

2022-12-22 Thread David Gilman
I apologize that my post was not super clear, I am thinking about implementing a fdw from scratch, and the target database is one of those NoSQL databases where you have to send JSON over a HTTP connection for each query. I have reviewed the postgres fdw code to see how it works and to see what's

Re: Implementing foreign data wrappers and avoiding n+1 querying

2022-12-21 Thread David Rowley
On Thu, 22 Dec 2022 at 13:31, David Gilman wrote: > > When a fdw table participates in query planning and finds itself as > part of a join it can output a parameterized path. If chosen, Postgres > will dutifully call the fdw over and over via IterateForeignScan to > fetch matching tuples. Many fdw