I suspect your implementation is incorrect in some way, or you have a memory
leak of another sort.
When I used your original code, the memory usage according to the activity
monitor was > 100 mb.
When I changed it to use a semaphore to limit the concurrency to 64 threads, it
was about 16mb.
Y
I wanted to add some clarification on my misunderstanding and the
approaches I tried.
I tested two different strategies to handle the CGO calls:
1.
*Worker Pool Approach* (50 workers handling CGO calls)
- I set up a worker pool with 50 goroutines, each making CGO calls in a
Hi Ian and Robert,
First, thank you both for taking the time to answer and provide insights! I
really appreciate it.
To clarify, my real program is a long-lived process that receives network
traffic from C++ code and processes it in Go.
The CGO calls in my program seemed to be the likely cul
On Tue, Mar 4, 2025 at 11:32 AM David Bell wrote:
>
> Why is memory increasing indefinitely with CGO?
Because when the Go scheduler creates threads, it keeps them around,
on the theory that if they were needed once, they will be needed
again. Certainly we wouldn't want all threads to exit as soon
Thanks for the response Robert,
I tried to put a semaphore to control the max amount CGO calls
simultaneously,
similar to what they did here
https://github.com/golang/go/blob/master/src/net/net.go#L794-L818
i noticed it does help somewhat, but memory consumption keeps going up
uncontrollably.
Or at the github issue points out, if the thread will exit, LockOSThread my
work.
On Tuesday, March 4, 2025 at 2:23:28 PM UTC-6 robert engels wrote:
> One way you can address this is to put a semaphore on the Go side around
> the C call, so you ensure only so many C calls are made simultaneousl
This is going to create 5000 OS threads. The loop runs really quickly, and
since CGO cannot use Go routines, you quickly allocate 5000 OS threads.
See https://github.com/golang/go/issues/14592
> On Mar 4, 2025, at 1:26 PM, David Bell wrote:
>
> Hi everyone,
>
> I'm relatively new to Go and ev
