On 2020/01/07 10:12, Michael Jeanson wrote:
> On 2020-01-06 7:46 p.m., Benjamin Poirier wrote:
> > I'm not sure if it's related but I saw almost the same error on last
> > upgrade (but for 5.4.0-2):
> >
> > depmod: ERROR: ../libkmod/libkmod.c:515 lookup_builtin_file() could not
> > open builtin file
> > '/var/tmp/mkinitramfs_J2sneW/lib/modules/5.4.0-2-amd64/modules.builtin.bin'
> >
> > and I now get:
> >
> > root@vsid:~# lttng list --kernel
> > Error: Unable to list kernel events: Kernel tracer not available
> > root@vsid:~# journalctl -u lttng-sessiond.service
> > [...]
> > Jan 07 09:33:20 vsid lttng-sessiond[403]: Error: Failed to load kmod
> > library resources
> > Jan 07 09:33:20 vsid lttng-sessiond[403]: Warning: No kernel tracer
> > available
> >
> > lttng-modules-dkms is installed. I can load the modules manually but I
> > still get the same error.
> >
>
> Hi,
>
> If you had just installed the lttng-modules-dkms and lttng-tools packages,
> it's possible that the lttng-sessiond deamon was started before the kernel
> modules were built and so it couldn't load them.
Looks like the modules are built before lttng-sessiond is started:
Setting up lttng-modules-dkms (2.11.0-2) ...
Loading new lttng-modules-2.11.0 DKMS files...
Building for 5.4.0-2-amd64
Building initial module for 5.4.0-2-amd64
Done.
lttng-lib-ring-buffer.ko:
Running module version sanity check.
- Original module
- No original module exists within this kernel
- Installation
- Installing to /lib/modules/5.4.0-2-amd64/updates/dkms/
[...]
depmod...
DKMS: install completed.
Setting up linux-headers-5.4.0-2-amd64 (5.4.8-1) ...
/etc/kernel/header_postinst.d/dkms:
dkms: running auto installation service for kernel 5.4.0-2-amd64:.
Setting up sudo (1.8.29-1) ...
Setting up babeltrace (1.5.7-2) ...
Setting up liburcu6:amd64 (0.11.1-2) ...
Setting up linux-headers-amd64 (5.4.8-1) ...
Setting up liblttng-ctl0:amd64 (2.11.0-3) ...
Setting up lttng-tools (2.11.0-3) ...
Still, it doesn't work.
> Simply restarting the sessiond should fix this.
I tried restarting lttng-sessiond or rebooting the machine but it was no
help, lttng-sessiond always reports:
Error: Failed to load kmod library resources
Warning: No kernel tracer available
A quick look into the code shows that is:
src/bin/lttng-sessiond/modprobe.c
kmod_set_log_fn(*ctx, log_kmod, NULL);
ret = kmod_load_resources(*ctx);
if (ret < 0) {
ERR("Failed to load kmod library resources");
goto error;
}
I didn't dig into libkmod, but I noticed (using opensnoop.bt) the
following:
8071 lttng-sessiond 2 0
/lib/modules/5.4.0-2-amd64/modules.dep.bin
8071 lttng-sessiond 2 0
/lib/modules/5.4.0-2-amd64/modules.alias.bin
8071 lttng-sessiond 2 0
/lib/modules/5.4.0-2-amd64/modules.symbols.bin
8071 lttng-sessiond 2 0
/lib/modules/5.4.0-2-amd64/modules.builtin.alias.bin
On another machine which I haven't yet updated and where lttng still
works, I see:
193519 lttng-sessiond 2 0
/lib/modules/5.4.0-1-amd64/modules.dep.bin
193519 lttng-sessiond 2 0
/lib/modules/5.4.0-1-amd64/modules.alias.bin
193519 lttng-sessiond 2 0
/lib/modules/5.4.0-1-amd64/modules.symbols.bin
193519 lttng-sessiond 2 0
/lib/modules/5.4.0-1-amd64/modules.builtin.bin
Not sure if /lib/modules/5.4.0-2-amd64/modules.builtin.alias.bin is
relevant but it's an empty file...
After downgrading libkmod2 from
Version: 26+20191223-1
to
Version: 26-3
the issue with lttng is no longer apparent:
root@vsid:/tmp# lttng list --kernel
Kernel events:
-------------
asoc_snd_soc_bias_level_start (loglevel: TRACE_EMERG (0)) (type:
tracepoint)
asoc_snd_soc_bias_level_done (loglevel: TRACE_EMERG (0)) (type:
tracepoint)
asoc_snd_soc_dapm_start (loglevel: TRACE_EMERG (0)) (type:
tracepoint)
[...]
