On Mon, Nov 6, 2017 at 9:24 PM, Ryan Schmidt <ryandes...@macports.org> wrote:
> > On Nov 6, 2017, at 21:10, Eric A. Borisch wrote: > > > Just wondering what everyone would think of applying this patch? > > Are you aware of prior discussion in this ticket? > > https://trac.macports.org/ticket/36560 Yes. I thought I'd start here to cast a wider net. I actually had been running with my own patch to run afsctool (in parallel) over extracted files, and then discovered that bsdtar has the --hfsCompression option; we can use _a_ tar, just like we usually do, with one extra flag, and it's done. Unfortunately even on High Sierra the native bsdtar is 2.8.3, and it looks [1] like this went in in 3.1.0. As you can see from the patch there are only four new actions: two tests (bsdtar in path? bsdtar accepts --hfsCompression?) and two assignments if those are both true (use tar=bsdtar, and use --hfsCompression in the flags). It's a slam dunk from my perspective. Compared with five years ago, the things that make it beneficial (trading CPU cores sitting idle for disk space) are just more in abundance, and with more systems with soldered-on storage, drive capacity is still at a premium. The only question (to me) is if we should bundle (~2MB installed, ~900k packaged) libarchive, or test for it at runtime similar to lbzip2. (Which is another everyone-should-use-it tool in the multicore age...) I can see arguments for both, but the test-at-activation-time is certainly lower impact to begin with, and reduces the required footprint. The (by far) biggest downside is that it negatively impacts activation time. This as due to the extra (expensive) compression step performed done during activation, but it doesn't have to write the file and then re-process it (afsctool route) -- bsdtar does it in-line. (If a file doesn't compress well, it ends up writing the compressed one, and then re-writing the decompressed one, from my quick read of the code.) Also of (new) note, it appears to still work on APFS volumes. There were mixed messages in early reviews of APFS if this would be the case, but at least the finder is reporting a file extracted via bsdtar with --hfsCompression clearly shows the file as compressed and reports both the size and space used on disk -- and they're vastly different for a trivially compressible (repetitive) text file. Some quick sizes (High Sierra) boost: 290MB w/o compression; 190MB with qt5-qtbase: 56MB / 30MB llvm-4.0: 135MB / 60MB I'm sure there will be those that will question how it will impact performance. I'm sure there are tests one could devise where it is not beneficial, but I'll exit with one final note: Apple HFS+ compresses just about everything I've checked (again High Sierra) in the system / default applications. Take a look for yourself; 'open /usr/include' and then use the finder to get info on a header file. curses.h as a nice large one to see. - Eric (I'll post a link to this thread in the ticket for good measure.) [1] https://github.com/libarchive/libarchive/commit/fb67a295 4f6ad067a272373eca797278e8232223 [Sorry for the duplicate Ryan; didn't hit reply-all; edited third paragraph from previous email that just went to you, FWIW]
