On Tue, Mar 4, 2025 at 11:32 AM David Bell <davidbellou...@gmail.com> wrote: > > Why is memory increasing indefinitely with CGO?
Because when the Go scheduler creates threads, it keeps them around, on the theory that if they were needed once, they will be needed again. Certainly we wouldn't want all threads to exit as soon as they no longer needed, as for most programs that would waste time exiting and recreating threads. Your program creates a lot of threads. Developing a better approach for this is https://go.dev/issue/14592. It's not a problem in practice for most programs, because most programs do not run 5000 executions of C code in parallel. But as you've seen it's certainly possible to write a program that behaves badly in this regard. > Why are threads not getting properly cleaned up after CGO calls? Same answer. > Is there a way to force the Go runtime to reclaim memory allocated for CGO > threads? Not really, no. As others have pointed out you can use runtime.LockOSThread in a goroutine, which will cause the thread used for that goroutine to exit when the goroutine exits. But it's not a very elegant solutoin. > Is there a better approach to handling CGO calls that spawn short-lived > threads? There are various approaches, but I think they all start with: why are you doing this? Your example program demonstrates the problem but is clearly pointless, so what does your real program do? > Would using runtime.UnlockOSThread() help in this case, or is this purely a > CGO threading issue? runtime.UnlockOSThread won't do anything, but as noted runtime.LockOSThread can change the behavior. > Is there a way to track down where the memory is being held? Since pprof does > not show high memory usage, what other tools can I use? It sounds like most of the memory is being held by thread stacks created by the C library. Unfortunately, as far as I know, Go doesn't have a good way to track that. Ian -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/CAOyqgcWKm8Pw7PyqHmzVB2NH75fL3WbqyGfA0X2vmbf7O%2BChvQ%40mail.gmail.com.
