Michael Gmelin <gre...@freebsd.org> writes: >> On 12. Apr 2017, at 05:43, Alexey Dokuchaev <da...@freebsd.org> wrote: >> >> Hi there porters, >> >> It had been annoying me for a while, so I figured I try asking here: >> LLVM ports (particularly, llvm39-3.9.1) take abnormally long time to >> package, e.g. right now it took 43 minutes (tmpfs-backed build area, >> pkg-1.10.0_2.txz, AMD A8 APU @1900 MHz, plenty of RAM, no swap). >> >> pkg was eating 100% (of a single core) during that time, so it looks >> like it's actually busy (not waiting for I/O or something), but with >> what? And can it be fixed? LLVM ports are real PITA to (re)build, >> and while ccache helps to greatly reduce the build times, 3/4-hour >> package time still cripples it. >> > > It's probably busy compressing the package. Try setting > PKG_NOCOMPRESS=1 in /etc/make.conf (works ok when just building a > package with make, didn't test with poudriere[0]). When creating > packages, the current compression algorithm can't take advantage of > multiple cores, so it's kind of a bottleneck on modern machines.
Nah, libarchive just defaults to 1 thread instead of using hw.ncpu.
The tricky part is propagating the fix into old releases or setting
compression options from within pkg(8).
Index: lib/libarchive/config_freebsd.h
===================================================================
--- lib/libarchive/config_freebsd.h (revision 316719)
+++ lib/libarchive/config_freebsd.h (working copy)
@@ -139,6 +139,7 @@
#define HAVE_LONG_LONG_INT 1
#define HAVE_LSTAT 1
#define HAVE_LUTIMES 1
+#define HAVE_LZMA_STREAM_ENCODER_MT 1
#define HAVE_MBRTOWC 1
#define HAVE_MEMMOVE 1
#define HAVE_MEMORY_H 1
Index: contrib/libarchive/libarchive/archive_write_add_filter_xz.c
===================================================================
--- contrib/libarchive/libarchive/archive_write_add_filter_xz.c (revision
316719)
+++ contrib/libarchive/libarchive/archive_write_add_filter_xz.c (working copy)
@@ -152,7 +152,11 @@ common_setup(struct archive_write_filter *f)
}
f->data = data;
data->compression_level = LZMA_PRESET_DEFAULT;
+#ifdef HAVE_LZMA_STREAM_ENCODER_MT
+ data->threads = lzma_cputhreads();
+#else
data->threads = 1;
+#endif
f->open = &archive_compressor_xz_open;
f->close = archive_compressor_xz_close;
f->free = archive_compressor_xz_free;
Index: contrib/libarchive/libarchive/archive_write_set_format_xar.c
===================================================================
--- contrib/libarchive/libarchive/archive_write_set_format_xar.c
(revision 316719)
+++ contrib/libarchive/libarchive/archive_write_set_format_xar.c
(working copy)
@@ -384,7 +384,11 @@ archive_write_set_format_xar(struct archive *_a)
/* Set default compression type, level, and number of threads. */
xar->opt_compression = GZIP;
xar->opt_compression_level = 6;
+#ifdef HAVE_LZMA_STREAM_ENCODER_MT
+ xar->opt_threads = lzma_cputhreads();
+#else
xar->opt_threads = 1;
+#endif
a->format_data = xar;
signature.asc
Description: PGP signature
