Hello, I have recently upgraded from lttng 2.10 to 2.13 as part of servers migration from buster to bookworm.
lttng 2.10.6 - KeKriek to lttng 2.13.9 - Nordicité I see an increase in memory consumption upto 11Gig after my recent upgrade. In buster root@localhost:~# top -b -n 1 | grep lttng PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 4296 root 20 0 1003676 19616 4608 S 0.0 0.1 0:00.67 lttng-sessiond 4297 root 20 0 44260 800 0 S 0.0 0.0 0:00.00 lttng-runas 4643 root 20 0 5838512 10396 9204 S 0.0 0.0 0:09.24 lttng-consumerd 4646 root 20 0 48048 544 0 S 0.0 0.0 0:00.00 lttng-runas In bookworm In bookworm. root@edgecore-40XKE-j2-101-32:~# top -b -n 1 | grep lttng PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 4824 root 20 0 1098824 13336 5980 S 11.8 0.0 0:03.46 lttng-s+ 4825 root 20 0 48872 2188 1012 S 0.0 0.0 0:00.00 lttng-r+ 4841 root 20 0 11.5g 11.0g 11.0g S 0.0 35.4 2:33.33 lttng-c+ 4843 root 20 0 3680 1160 816 S 0.0 0.0 0:00.23 lttng-r+ I have a python script which was used to enable channels, these were the parameters I was using large_channels_sz = 4000000 medium_channels_sz = 200000 small_channels_sz = 10000 num_large = 4 num_medium = 4 num_small = 4 def enable_channel(channels, session, subbuf_size, subbuf_num): for c in channels: call(['lttng', 'enable-channel', '-u', c, '-s', session, '--subbuf-size', str(subbuf_size), '--num-subbuf', str(subbuf_num),], stdout=devnull, stderr=subprocess.STDOUT) enable_channel(large_channels, def_session, large_channels_sz, num_large) enable_channel(medium_channels, def_session, medium_channels_sz, num_medium) enable_channel(small_channels, def_session, small_channels_sz, num_small) there are total 16 large channels and 6 medium channels and no small channels. The total size for all these session cominded turn out to be machine roughly 7GB on my 32 core machine, which is still less that the amount of memory used by the lttng-consumerd. I only have default session configured and nothing else root@edgecore-40XKE-j2-101-32:~# lttng list Available recording sessions: 1) default [active snapshot] I have seen that when only the channel are enable and no process which is using lttng is running then, I see normal memory consumption. root@edgecore-40XKE-j2-101-32:~# top -b -n 1 | grep lttng PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 4824 root 20 0 1098184 12732 5376 S 0.0 0.0 0:03.15 lttng-s+ 4825 root 20 0 48872 2188 1012 S 0.0 0.0 0:00.00 lttng-r+ 4841 root 20 0 536240 4284 2400 S 0.0 0.0 2:09.32 lttng-c+ 4843 root 20 0 3680 1160 816 S 0.0 0.0 0:00.23 lttng-r+ But, as soon as I run even an single process which is linked to lttng, and which is *not* using any lttng channel liblttng-ust.so.1 => /lib/x86_64-linux-gnu/liblttng-ust.so.1 (0x 00007fa867ea3000) liblttng-ust-common.so.1 => /lib/x86_64-linux-gnu/liblttng-ust-common.so.1 ( 0x00007fa86752d000) liblttng-ust-tracepoint.so.1 => /lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1 (0x00007fa86750b000) I see an very high memory consumption of 5.5g and when i run all the process then i see the total memory consumption reaching uptop 11G. Do you have any hints on why this could be happening, Thank You.
