On Sun, Apr 12, 2020 at 09:18:28PM -0400, Stephen Frost wrote: > * Bruce Momjian (br...@momjian.us) wrote: > > On Fri, Apr 10, 2020 at 10:54:10AM -0400, Stephen Frost wrote: > > > * Robert Haas (robertmh...@gmail.com) wrote: > > > > On Thu, Apr 9, 2020 at 6:44 PM Bruce Momjian <br...@momjian.us> wrote: > > > > > Good point, but if there are multiple APIs, it makes shell script > > > > > flexibility even more useful. > > > > > > > > This is really the key point for me. There are so many existing tools > > > > that store a file someplace that we really can't ever hope to support > > > > them all in core, or even to have well-written extensions that support > > > > them all available on PGXN or wherever. We need to integrate with the > > > > tools that other people have created, not try to reinvent them all in > > > > PostgreSQL. > > > > > > So, this goes to what I was just mentioning to Bruce independently- you > > > could have made the same argument about FDWs, but it just doesn't > > > actually hold any water. Sure, some of the FDWs aren't great, but > > > there's certainly no shortage of them, and the ones that are > > > particularly important (like postgres_fdw) are well written and in core. > > > > No, no one made that argument. It isn't clear how a shell script API > > would map to relational database queries. The point is how well the > > APIs match, and then if they are close, does it give us the flexibility > > we need. You can't just look at flexibility without an API match. > > If what we're talking about is the file_fdw, which certainly isn't very > complicated, it's not hard to see how you could use shell scripts for > it. What happens is that it starts to get harder and require custom > code when you want to do something more complex- which is very nearly > what we're talking about here too. Sure, for a simple 'bzip2' filter, a > shell script might be alright, but it's not going to cut it for the more > complex use-cases that users, today, expect solutions to.
Well, file_fdw is the simplest FDW, and we might have been able to do that in shell script, but almost all the other FDWs couldn't, so we might as well choose a C API for FDWs and use the same one for file_fdw. It seems like basic engineering that you choose the closest API that meets most of your deployment requirements, and meets all of the required ones. > To that end, if we contemplate adding support for some cloud vendor's > storage, as an example, and discover that the command line tools for it > suck or don't meet our expectations, I'd expect us to either refuse to > support it, or to forgo using the command-line tools and instead > implement support for talking to the cloud storage interface directly, > if it works well. Do we choose a more inflexible API on a hypothetical risk? -- Bruce Momjian <br...@momjian.us> https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
