05.07.2023 09:26, Andres Salomon пишет:
On Wed, Jul 5 2023 at 08:42:23 AM +03:00:00, Michael Tokarev <[email protected]>
wrote:
Package: lsof Version: 4.95.0-1 Severity: normal Previously, I was able to see open but deleted files on a given filesystem (for example, after
upgrading a library, with old .so files still open by older processes) by doing this: lsof / | grep DEL This gave output like this (after recent
libX11 update): Xwayland 1998 2007 Xwayland: mjt DEL REG 0,23 339489 /usr/lib/x86_64-linux-gnu/libX11-xcb.so.1.0.0 Xwayland 1998 2007 Xwayland: mjt
DEL REG 0,23 339461 /usr/lib/x86_64-linux-gnu/libX11.so.6.4.0 But with bookworm lsof, it does not work, `lsof /' is not finding these DEL files
anymore. It only works without the filesystem argument, eg lsof | grep DEL but obviously it shows far more than I wanted it to show, and works
*dramatically* slower. It looks like something changed within lsof in the filesystem matching
That's odd, it still works for me on bookworm:
dilinger@5310:~$ lsof / |grep DEL|grep chromiu chromium 146607 dilinger DEL REG 254,1 54575015
/home/dilinger/.config/chromium/BrowserMetrics/BrowserMetrics-64A3BB92-23CAF.pma chromium 146607 dilinger DEL REG 254,1 54623911
/home/dilinger/.config/dconf/user dilinger@5310:~$
Hm. Interesting. And odd, indeed.
What does your /proc/mounts look like? Maybe there's something different with how your / is mounted? Or, maybe lsof lacks permission to access
something? Does it work if you run lsof / as root? Running it under strace could also provide a hint.
I did some more tests. The prob isn't lsof itself but something else, since
lsof from bullseye shows exactly the same behavour.
For example:
# lsof -p 1 | grep /lib/
systemd 1 root txt REG 0,24 92544 111995
/usr/lib/systemd/systemd
systemd 1 root mem REG 0,22 111995
/usr/lib/systemd/systemd (path dev=0,24)
systemd 1 root mem REG 0,22 118104
/usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.11.2 (path dev=0,24)
systemd 1 root mem REG 0,22 22227
/usr/lib/x86_64-linux-gnu/libm.so.6 (path dev=0,24)
systemd 1 root mem REG 0,22 147312
/usr/lib/x86_64-linux-gnu/libcrypto.so.3 (path dev=0,24)
systemd 1 root mem REG 0,22 124846
/usr/lib/x86_64-linux-gnu/libgpg-error.so.0.33.1 (path dev=0,24)
systemd 1 root mem REG 0,22 117583
/usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 (path dev=0,24)
systemd 1 root mem REG 0,22 22103
/usr/lib/x86_64-linux-gnu/liblzma.so.5.4.1 (path dev=0,24)
...
# lsof / | grep ' 1 '
systemd 1 root cwd DIR 0,24 184 256 /
systemd 1 root rtd DIR 0,24 184 256 /
systemd 1 root txt REG 0,24 92544 111995
/usr/lib/systemd/systemd
(digging more..)
It looks like this is btrfs-specific. Note the pathnames above - the "(path
dev=0,24)" tail thing.
It is not shown when root fs is on ext4:
anothersystem# lsof -p 1 | grep /lib/
systemd 1 root txt REG 9,1 92544 21522
/usr/lib/systemd/systemd
systemd 1 root mem REG 9,1 157768 438
/usr/lib/x86_64-linux-gnu/libgpg-error.so.0.33.1
systemd 1 root mem REG 9,1 629384 7184
/usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.11.2
systemd 1 root mem REG 9,1 907784 10122
/usr/lib/x86_64-linux-gnu/libm.so.6
systemd 1 root mem REG 9,1 190456 10205
/usr/lib/x86_64-linux-gnu/liblzma.so.5.4.1
systemd 1 root mem REG 9,1 4709656 4362
/usr/lib/x86_64-linux-gnu/libcrypto.so.3
systemd 1 root mem REG 9,1 30704 2770
/usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0
...
and it shows expected set of files when run as `lsof /'
I noticed this on a few systems after upgrading to bookworm, but I also updated
filesystem layout and switched to btrfs. Had no idea this can ever be related
to a filesystem.
I'll try to dig further here. strace didn't reveal anything interesting so far.
(and yes, all tests are done as root, as is the initial issue. Here's my
/prpc/mounts:
udev /dev devtmpfs
rw,nosuid,noexec,relatime,size=8156784k,nr_inodes=2039196,mode=755,inode64 0 0
runfs /run tmpfs rw,nosuid,relatime,size=3265800k,mode=755,inode64 0 0
/dev/md1 / btrfs
rw,relatime,compress=zstd:4,space_cache=v2,subvolid=256,subvol=/rootfs 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0
tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0
devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0
cgroup2 /sys/fs/cgroup cgroup2
rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0
pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0
efivarfs /sys/firmware/efi/efivars efivarfs rw,nosuid,nodev,noexec,relatime 0 0
bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0
hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0
mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0
debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0
tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0
fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0
configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0
ramfs /run/credentials/systemd-sysctl.service ramfs
ro,nosuid,nodev,noexec,relatime,mode=700 0 0
ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs
ro,nosuid,nodev,noexec,relatime,mode=700 0 0
tmpfs /tmp tmpfs rw,nosuid,nodev,relatime,inode64 0 0
/dev/sda5 /squid/b btrfs
rw,nosuid,nodev,noexec,relatime,compress=zstd:5,space_cache,subvolid=5,subvol=/
0 0
/dev/sdc1 /boot/efi vfat
rw,nosuid,nodev,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro
0 0
/dev/sdc5 /squid/a btrfs
rw,nosuid,nodev,noexec,relatime,compress=zstd:5,space_cache,subvolid=5,subvol=/
0 0
/dev/md2p1 /var ext4 rw,nosuid,nodev,relatime 0 0
/dev/md2p3 /ora ext4 rw,nosuid,nodev,relatime 0 0
/dev/md0 /stage btrfs rw,nosuid,nodev,relatime,space_cache,subvolid=5,subvol=/
0 0
/dev/md2p4 /ws/ws btrfs
rw,nosuid,nodev,relatime,compress=zstd:5,space_cache,subvolid=258,subvol=/ws 0 0
/dev/md2p4 /home btrfs
rw,nosuid,nodev,relatime,compress=zstd:5,space_cache,subvolid=257,subvol=/home
0 0
/dev/md2p4 /share btrfs
rw,nosuid,nodev,relatime,compress=zstd:5,space_cache,subvolid=260,subvol=/soft
0 0
/dev/md2p4 /home/mail btrfs
rw,nosuid,nodev,relatime,compress=zstd:5,space_cache,subvolid=259,subvol=/mail
0 0
ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs
ro,nosuid,nodev,noexec,relatime,mode=700 0 0
)
Thanks,
/mjt
