On Fri, Mar 11, 2022 at 8:52 PM Andres Freund <and...@anarazel.de> wrote: > You could also just append a manifest as a compresed tar to the compressed tar > stream. Unfortunately GNU tar requires -i to read concated compressed > archives, so perhaps that's not quite an alternative.
s/Unfortunately/Fortunately/ :-p I think we've already gone way too far in the direction of making this stuff rely on specific details of the tar format. What if someday we wanted to switch to pax, cpio, zip, 7zip, whatever, or even just have one of those things as an option? It's not that I'm dying to have PostgreSQL produce rar or arj files, but I think we box ourselves into a corner when we just assume tar everywhere. As an example of a similar issue with real consequences, consider the recent discovery that we can't easily add support for LZ4 or ZSTD compression of pg_wal.tar. The problem is that the existing code tells the gzip library to emit the tar header as part of the compressed stream without actually compressing it, and then it goes back and overwrites that data later! Unsurprisingly, that's not a feature every compression library offers. -- Robert Haas EDB: http://www.enterprisedb.com
