Re: [lttng-dev] 回复： 回复： shm leak in traced application?

[Jonathan Rajotte-Julien via lttng-dev]

Hi zhenyu.ren,

Please open a bug on our bug tracker and provide a reproducer against the latest
stable version (2.13.x).

https://bugs.lttng.org/

Please follow the guidelines: https://bugs.lttng.org/#Bug-reporting-guidelines

Cheers

On Fri, Feb 25, 2022 at 12:47:34PM +0800, zhenyu.ren via lttng-dev wrote:
> Hi, lttng-dev team 
>When lttng-sessiond exits, the ust applications should call 
> lttng_ust_objd_table_owner_cleanup() and clean up all shm resource(unmap and 
> close). Howerver I do find that the ust applications keep opening "all" of 
> the shm fds("/dev/shm/ust-shm-consumer-81132 (deleted)") and do NOT free shm.
>If we run lttng-sessiond again, ust applications can get a new piece of 
> shm and a new list of shm fds so double shm usages. Then if we kill 
> lttng-sessiond, what the mostlikely happened is ust applications close the 
> new list of shm fds and free new shm resource but keeping old shm still. In 
> other word, we can not free this piece of shm unless we killing ust 
> applications!!!
>   So Is there any possilbe that ust applications failed calling 
> lttng_ust_objd_table_owner_cleanup()? Do you have ever see this problem? Do 
> you have any advice to free the shm without killling ust applications(I tried 
> to dig into kernel shm_open and /dev/shm, but not found any ideas)?
> 
> Thanks in advance
> zhenyu.ren 
> 
> 
> 
> --
> 发件人：zhenyu.ren via lttng-dev 
> 发送时间：2022年2月23日(星期三) 23:09
> 收件人：lttng-dev 
> 主　题：[lttng-dev] 回复： shm leak in traced application?
> 
> >"I found these items also exist in a traced application which is a long-time 
> >running daemon"
>  Even if lttng-sessiond has been killed!!
> 
> Thanks
> zhenyu.ren
> --
> 发件人：zhenyu.ren via lttng-dev 
> 发送时间：2022年2月23日(星期三) 22:44
> 收件人：lttng-dev 
> 主　题：[lttng-dev] shm leak in traced application?
> 
> Hi, 
>There are many items such as "/dev/shm/ust-shm-consumer-81132 (deleted)" 
> exist in lttng-sessiond fd spaces. I know it is the result of shm_open() and 
> shm_unlnik() in create_posix_shm(). 
>However, today, I found these items also exist in a traced application 
> which is a long-time running daemon. The most important thing I found is that 
> there seems no reliable way to release share memory.
>I tried to kill lttng-sessiond but not always release share memory. 
> Sometimes I need to kill the traced application to free share memoryBut 
> it is not a good idea to kill these applications.
>My questions are: 
>1. Is there any way to release share memory without killing any traced 
> application?
>2. Is it normal that many items such as "/dev/shm/ust-shm-consumer-81132 
> (deleted)" exist in the traced application?
> 
> Thanks
> zhenyu.ren
> 
> 

> ___
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev


-- 
Jonathan Rajotte-Julien
EfficiOS
___
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

[lttng-dev] [PATCH lttng-tools] Fix: consumer-stream: use-after-free of metadata bucket

[Vincent Whitchurch via lttng-dev]

When consumer_stream_destroy() is called from, for example, the error
path in setup_metadata(), consumer_stream_free() can end up being called
twice on the same stream.  Since the stream->metadata_bucket is not set
to NULL after being destroyed, it leads to a use-after-free:

 ERROR: AddressSanitizer: heap-use-after-free on address 0x60400318
 READ of size 8 at 0x60400318 thread T7
 #0 in metadata_bucket_destroy
 #1 in consumer_stream_free
 #2 in consumer_stream_destroy
 #3 in setup_metadata
 #4 in lttng_ustconsumer_recv_cmd
 #5 in lttng_consumer_recv_cmd
 #6 in consumer_thread_sessiond_poll
 #7 in start_thread nptl/pthread_create.c:481
 #8 in clone (/lib/x86_64-linux-gnu/libc.so.6+0xfcbde)

 0x60400318 is located 8 bytes inside of 48-byte region 
[0x60400310,0x60400340)
 freed by thread T7 here:
 #0 in __interceptor_free
 #1 in metadata_bucket_destroy
 #2 in consumer_stream_free
 #3 in consumer_stream_destroy
 #4 in clean_channel_stream_list
 #5 in consumer_del_channel
 #6 in consumer_stream_destroy
 #7 in setup_metadata
 #8 in lttng_ustconsumer_recv_cmd
 #9 in lttng_consumer_recv_cmd
 #10 in consumer_thread_sessiond_poll
 #11 in start_thread nptl/pthread_create.c:481

 previously allocated by thread T7 here:
 #0 in __interceptor_calloc
 #1 in zmalloc
 #2 in metadata_bucket_create
 #3 in consumer_stream_enable_metadata_bucketization
 #4 in lttng_ustconsumer_set_stream_ops
 #5 in lttng_ustconsumer_on_recv_stream
 #6 in lttng_consumer_on_recv_stream
 #7 in create_ust_streams
 #8 in ask_channel
 #9 in lttng_ustconsumer_recv_cmd
 #10 in lttng_consumer_recv_cmd
 #11 in consumer_thread_sessiond_poll
 #12 in start_thread nptl/pthread_create.c:481

 Thread T7 created by T0 here:
 #0 in __interceptor_pthread_create
 #1 in main
 #2 in __libc_start_main ../csu/libc-start.c:332

 SUMMARY: AddressSanitizer: heap-use-after-free in metadata_bucket_destroy

Signed-off-by: Vincent Whitchurch 
---
 src/common/consumer/consumer-stream.cpp | 1 +
 1 file changed, 1 insertion(+)

diff --git a/src/common/consumer/consumer-stream.cpp 
b/src/common/consumer/consumer-stream.cpp
index 2dc3f002b..481611c3e 100644
--- a/src/common/consumer/consumer-stream.cpp
+++ b/src/common/consumer/consumer-stream.cpp
@@ -988,6 +988,7 @@ void consumer_stream_free(struct lttng_consumer_stream 
*stream)
LTTNG_ASSERT(stream);
 
metadata_bucket_destroy(stream->metadata_bucket);
+   stream->metadata_bucket = NULL;
call_rcu(&stream->node.head, free_stream_rcu);
 }
 
-- 
2.34.1

___
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev