I built from HEAD, named it gzip_vanilla, rebuilt with my CRC code and
named it gzip_8_slice. Input file is a 115MB file gzipped (default
settings) to 61M
sam@DESKTOP-R64B0KJ:~/gziptest$ time ../code/gzip/gzip_8_slice -k -d -c
large_file.gz > /dev/null
real 0m0.307s
user 0m0.298s
sys 0m0.000s
sam@DESKTOP-R64B0KJ:~/gziptest$ time ../code/gzip/gzip_8_slice -k -d -c
large_file.gz > /dev/null
real 0m0.312s
user 0m0.310s
sys 0m0.000s
sam@DESKTOP-R64B0KJ:~/gziptest$ time ../code/gzip/gzip_8_slice -k -d -c
large_file.gz > /dev/null
real 0m0.306s
user 0m0.307s
sys 0m0.000s
sam@DESKTOP-R64B0KJ:~/gziptest$ time ../code/gzip/gzip_8_slice -k -d -c
large_file.gz > /dev/null
real 0m0.319s
user 0m0.316s
sys 0m0.000s
sam@DESKTOP-R64B0KJ:~/gziptest$ time ../code/gzip/gzip_vanilla -k -d -c
large_file.gz > /dev/null
real 0m0.485s
user 0m0.484s
sys 0m0.000s
sam@DESKTOP-R64B0KJ:~/gziptest$ time ../code/gzip/gzip_vanilla -k -d -c
large_file.gz > /dev/null
real 0m0.498s
user 0m0.476s
sys 0m0.020s
sam@DESKTOP-R64B0KJ:~/gziptest$ time ../code/gzip/gzip_vanilla -k -d -c
large_file.gz > /dev/null
real 0m0.488s
user 0m0.478s
sys 0m0.010s
Looks to be about a ~35% reduction in time
On Mon, 14 Oct 2024 at 17:44, Bruno Haible <br...@clisp.org> wrote:
> Sam Russell wrote:
> > As for your question on speed, I noticed between zstd (which uses zlib
> as a
> > backend) and gzip there seems to be an improvement of maybe 30-40% for
> > decompressing a 100MB file (part of this is due to multithreading
> though),
>
> If you compare two scenarios which differ in 4 aspects:
> - different general codebase,
> - different CRC32 implementation,
> - different multithreading,
> - code compared with -fPIC (shared library) vs. without -fPIC (gzip
> program),
> you cannot draw any conclusion. In order to make a valid point, you need to
> vary _one_ aspect only.
>
> > and gprof shows the CRC calculation being maybe 40-50% of the CPU cycles
>
> Don't blindly trust gprof figures. gprof is sensitive to a function's size
> and invocation frequency. There are better profilers [1], but in this case
> what I asked for is not a profiling but a comparison of two scenarios that
> differ in 1 aspect, each compiled in the usual way ("gcc -O2", with a
> recent
> gcc, on x86_64 or arm64).
>
> Bruno
>
> [1] https://gitlab.com/ghwiki/gnow-how/-/wikis/Profiling
>
>
>
>
