Thank you very much for the fast, clear, and detailed answer :) On Fri, 1 Nov 2019 at 15:08, Ian Lance Taylor <i...@golang.org> wrote:
> On Fri, Nov 1, 2019 at 6:31 AM Tom Payne <twpa...@gmail.com> wrote: > > > > cgo is often used to provide bindings to C libraries. Any memory > allocated in the C library is not visible to Go, so Go does not have an > accurate view of the program's memory usage and does not run the garbage > collector or any finalizers often enough. Consequently, memory usage for a > Go server that uses cgo heavily can grow very large, with Go itself being > utterly unaware of it. > > > > If the C functions allocate memory, historically you could set a > finalizer to free the memory sometime after there are no remaining > references to it in Go, as, for example, described in this blog post. > However, the current Go docs on runtime.SetFinalizer state: > > > > > There is no guarantee that finalizers will run before a program exits, > so typically they are useful only for releasing non-memory resources > associated with an object during a long-running program. > > > > Are there any other ways to automatically release memory resources > associated with an object? Or telling Go's memory manager that the small > object it sees in the Go world is but the tip of an iceberg of memory > allocated in the C world and therefore should be finalized? > > > > Not-good options include: > > - Requiring the programmer to make explicit calls to free the memory > resources when they believe the object is no longer needed, but this takes > us back to the painful world of C's manual memory management and is easy to > get wrong. > > - Padding Go objects associated with C memory resources with large, > unused fields (e.g. an [1024]byte) in the hope that the Go garbage > collector will be more likely to finalize and free them when they are > unused. > > - Avoiding cgo in any server code. > > > > Are there any good options? > > > If you are using C code, you are using C's memory resource policy. So > the best approach is to use explicit calls to free. I agree that that > takes you back to the painful world of C's manual memory management, > but, after all, if you are using C then you can't pretend that you are > not using C. > > That said, although Go is careful not to promise that it will actually > execute finalizers, in practice it does run them. Padding the Go > object won't make any difference as to when a finalizer is run. It > will be run in the next full GC cycle after the Go value is no longer > referenced. If there are times when the program knows that there is a > lot of C memory that is no longer required, it can help by calling > runtime.GC itself. > > In practice I think the best approach is a hybrid: free the C memory > explicitly, but also add a finalizer. In the finalizer, free the C > memory, and log the fact that you are doing so, along with any > information that will help identify where the C memory was allocated. > Then periodically check your logs for cases where the finalizer ran, > and add the necessary calls to explicitly free the C memory manually. > Of course, when you free the C memory manually, don't forget to clear > the finalizer. > > Ian > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAHY_QbR2NB1FFobtTW6dkXN9h-Zj5uw3bfodVnFzghL_1%3DdUJg%40mail.gmail.com.
