Re: [idea]: Switch default compression from "xz" to "zstd" for .deb packages

Leandro Cunha Mon, 18 Sep 2023 12:35:13 -0700

Hi,

On Mon, Sep 18, 2023 at 4:20 PM Helmut Grohne <hel...@subdivi.de> wrote:
>
> Hi,
>
> On Sat, Sep 16, 2023 at 10:31:20AM +0530, Hideki Yamane wrote:
> >  Today I want to propose you to change default compression format in .deb,
> >  {data,control}.tar."xz" to ."zst".
> >
> >  I want to hear your thought about this.
>
> I am not very enthusiastic about this idea. I skip over those arguments
> already raised by others and add one that I haven't seen thus far. zstd
> is quite optimized for 64bit CPUs and for amd64 in particular. amd64 is
> the only architecture for which zstd provides a hufmann implementation
> in assembly.
>
> > ## More CPUs
> >
> >  2012: ThinkPad L530 has Core i5-3320M (2 cores, 4 threads)
> >  2023: ThinkPad L15 has Core i5-1335U (10 cores, 12 threads)
> >
> >  
> > https://www.cpubenchmark.net/compare/817vs5294/Intel-i5-3320M-vs-Intel-i5-1335U
> >   - i5-3320M: single 1614, multicore 2654
> >   - i5-1335U: single 3650, multicore 18076 points.
>
> While the majority of CPUs in active deployments is amd64, I'd also like
> to see numbers for 32bit CPUs and non-x86 ones. While I personally find
> the trade-off by zstd fit for a number of my use cases, I was also
> surprised just how slow it decompresses on armhf.
>
> I found some arm board with some linux kernel package sized 36MB.
>
>  algo     | compressed size | decompression time
> ----------+-----------------+-------------------
>  xz       |     36MB        |  14.7s
>  zstd     |     52MB        |   5.2s
>  zstd -9  |     48MB        |   5.2s
>  zstd -11 |     47MB        |   5.4s
>  zstd -19 |     41MB        |   5.7s
>
> Not as slow as I remembered apparently, but it still has a more than 10%
> size overhead. The size ratio is consistent with Robert Edmond's
> numbers, but we no longer see that 10-fold speedup. And this did not
> look at decompression memory requirements.
>
> I am decompressing a *lot* of .debs (dedup.d.n, multiarch hinter,
> crossqa.d.n, dumat). All of these applications would benefit from zstd
> compressed .debs in terms of decompression speed. Yet, that has never
> been the bottleneck to me. To me, download speed matters more and
> swapping out a 1GBit link for a faster one isn't that easy.
>
> I'd vote against this given the data we have now.
>
> Can we defer the discussion until there are more convincing numbers?
>
> Helmut
>


If even Helmut voted against, I also vote and I package both for Arch
and Debian, I didn't notice that much of a difference in performance
for compression/decompression. But I use an NVME on the machine (it's
usually very fast along with a processor that isn't slow either) I use
to compile the packages I work with. But when you say that the numbers
are not that convincing, I agree.

-- 
Cheers,
Leandro Cunha

Re: [idea]: Switch default compression from "xz" to "zstd" for .deb packages

Reply via email to