Package: tar
Version: 1.34+dfsg-1.2+deb12u1
Severity: normal
X-Debbugs-Cc: hpal...@seznam.cz
Dear Maintainer,
While extracting big .tar.zst archive (23GB with lot of small files, created
with same GNU tar) with tar xpf archive.tar.zst the tar starts gradually
using 100% user% CPU, while disk and decompressor is idle. This slow
down increases with time - making practically impossible to extract such
archive in reasonable time
* What exactly did you do (or not do) that was effective (or
ineffective)?
There was easy workaround - using bsdtar - it utilized both disk and zstd
decompressor
- unpacking .tar.zst in expected time.
It was not possible to use cpio -H ustar -iVdm, because it does not
understand BigLing extension of GNU tar.
* What was the outcome of this action?
GNU tar extracting archive slowed down eating 100% user CPU (disk idle) making
impossible
to extract archive in reasonable time. There was nearly no I/O activity,
meaning that tar was busy with some housekeeping but not extracting
files.
* What outcome did you expect instead?
I expect that tar will extract files utilizing all resources - bsdtar utilizes
both
disk I/O and zstd decompress process - finishing extraction in finite time.
Debug information:
- here is typical top output - only user% CPU used (zstd somewhere at
bottom of list):
top - 16:21:47 up 1:09, 7 users, load average: 1.19, 1.12, 1.22
Tasks: 223 total, 2 running, 221 sleeping, 0 stopped, 0 zombie
%Cpu(s):100.0 us, 0.0 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
MiB Mem : 7939.9 total, 176.2 free, 573.7 used, 7499.9 buff/cache
MiB Swap: 8192.0 total, 8191.5 free, 0.5 used. 7366.2 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3341 root 20 0 20700 19188 2144 R 100.0 0.2 45:52.56 tar
5745 root 20 0 8608 4960 2916 R 12.5 0.1 0:00.02 top
1 root 20 0 104444 14188 8812 S 0.0 0.2 0:46.04 systemd
- lsof - and /proc/PID/fd - most of time there is no file open for writing -
another
proof that GNU tar is not spending time extracting content.
- output from perf record -g and perf report -g
Samples: 334K of event 'cycles', Event count (approx.): 167377283731
Children Self Command Shared Object Symbol
- 48.18% 0.00% tar [unknown] [.] 0x742f646c6975622f
- 0x742f646c6975622f
43.37% find_delayed_link_source
- 48.00% 47.68% tar tar [.] extract_link
extract_link
- 43.37% 43.07% tar tar [.] find_delayed_link_source
43.07% 0x742f646c6975622f
find_delayed_link_source
- 4.10% 4.09% tar libc.so.6 [.] __strcmp_sse2
4.08% 0x742f646c6975622f
+ 3.52% 0.01% tar [kernel.kallsyms] [k]
entry_SYSCALL_64_after_hwframe
+ 3.51% 0.01% tar [kernel.kallsyms] [k] do_syscall_64
+ 1.67% 0.01% tar libc.so.6 [.] linkat
+ 1.62% 0.00% tar [kernel.kallsyms] [k] __x64_sys_linkat
>From perf report I suspect that one or both these functions in GNU TAR cause
>this problem:
a) extract_link
b) find_delayed_link_source
Unfrotunately I'm unable to upload such archive for testing - it is too
big. In my case I basically packed complete Yocto Linux source and build
files to .tar.zst
-- System Information:
Debian Release: 12.6
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500,
'stable-debug'), (500, 'stable')
Architecture: amd64 (x86_64)
Kernel: Linux 6.1.0-23-amd64 (SMP w/2 CPU threads; PREEMPT)
Locale: LANG=C, LC_CTYPE=C.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
Versions of packages tar depends on:
ii libacl1 2.3.1-3
ii libc6 2.36-9+deb12u7
ii libselinux1 3.4-1+b6
tar recommends no packages.
Versions of packages tar suggests:
ii bzip2 1.0.8-5+b1
pn ncompress <none>
pn tar-doc <none>
pn tar-scripts <none>
ii xz-utils 5.4.1-0.2
-- no debconf information
