There are a couple of things that pg_basebackup can't do that might be an issue for some users. One of them is that you might want to do something like encrypt your backup. Another is that you might want to store someplace other than in the filesystem, like maybe S3. We could certainly teach pg_basebackup how to do specifically those things, and maybe that is worthwhile. However, I wonder if it would be useful to provide a more general capability, either instead of doing those more specific things or in addition to doing those more specific things.
What I'm thinking about is: suppose we add an option to pg_basebackup with a name like --pipe-output. This would be mutually exclusive with -D, but would work at least with -Ft and maybe also with -Fp. The argument to --pipe-output would be a shell command to be executed once per output file. Any instance of %f in the shell command would be replaced with the name of the file that would have been written (and %% would turn into a single %). The shell command itself would be executed via system(). So if you want to compress, but using some other compression program instead of gzip, you could do something like: pg_basebackup -Ft --pipe-output 'bzip > %f.bz2' And if you want to encrypt, you could do something like: pg_basebackup -Ft --pipe-output 'gpg -e -o %f.gpg' And if you want to ship it off to be stored in a concrete bunker deep underground, you can just do something like: pg_basebackup -Ft --pipe-output 'send-to-underground-storage.sh backup-2020-04-03 %f' You still have to write send-to-underground-storage.sh, of course, and that may involve some work, and maybe also some expensive construction. But what you don't have to do is first copy the entire backup to your local filesystem and then as a second step figure out how to put it through whatever post-processing it needs. Instead, you can simply take your backup and stick it anywhere you like. Thoughts? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
