On 28/02/2022 13:22, Paul Spooren wrote: > Hi team, > >> On 19. Feb 2022, at 19:21, Phillip Lougher <phillip.loug...@gmail.com> wrote: >> >> On Sat, Feb 19, 2022 at 4:01 PM Felix Fietkau <n...@nbd.name> wrote: >>> On 19.02.22 16:54, Stijn Tintel wrote: >>>> Drop the -processors argument from the mksquashfs4 call, so it will use >>>> all available processors. This dramatically reduces the time to create >>>> squashfs filesystems. >>>> >>>> The times below are observed when building an image for my main router, >>>> the WatchGuard Firebox M300 (qoriq target): >>>> >>>> Before: >>>> real 4m45,973s >>>> >>>> After: >>>> real 0m23,497s >>>> >>>> Signed-off-by: Stijn Tintel <st...@linux-ipv6.be> >>> We need to verify that this doesn't break reproducible builds... >>> >> It won't break reproducible builds. Since Mksquashfs version 4.4 the >> ordering is guaranteed to be the same, irrespective of how many >> threads or processors are used. > I think OpenWrt uses `squashfskit`[1] which is a fork some squashfs-tools > version. I think it happened due to stalled development some years ago. > Anyway, if squashfs-tools is actively maintained we should consider moving > back to the upstream version. > > Some time ago I already wanted to make the move but squashfskit uses some > downstream patches which expose more compression options, are those now part > of squashfs-tools[2]? If not, do we need those anyway? Not aware if we do or don't. > > Thanks for all further information, I’d be happy to have upstream > squashfs-tools with reproducible multithread support within OpenWrt! > > Sunshine, > Paul > > [1]: https://github.com/squashfskit/squashfskit > [2]: > https://github.com/squashfskit/squashfskit/commit/5568b6a7d7db01c8f60a502d549a431a2b7219ae
If you decide to switch back to squashfs-tools, and incorporate the change to let it use all processors, we should probably respect the number of jobs the user passes to make, and limit mksquashfs to use at most that number of processors. This was suggested on #openwrt-devel, after my initial submission. Unfortunately I have not found a way to do so, as MAKE_JOBSERVER is exported in include/toplevel.mk, which is not included in include/image.mk, and I am not familiar enough with this part of the code to understand the impact of adding that include. Stijn _______________________________________________ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
