Thank you Ian, you reply is very much appreciated. ... what if a compiler would do a bit of helping "magic"? I would suspect it does know that you return a subslice of an allocated array (something that compiler still sees up the call stack). Would it be possible to put a code for "This big alloc is going out of scope. If resulting (surviving) slice is less than a X% of that original thing, copy needed part, and point a new slice there".
Simple cases only - ints, strings, bytes. Struct of anything with pointers are no-go (pardon the pun) - they might point to another item in the same array. For cases of huge allocs, running-close-to-mem-limits, X will have to be smaller, so you would not OOM right there. Unless I miss something big and obvious - which is very highly possible - that sounds like a possible way to drop wasted space, keeping small part required on a side. Again, thank you very much for the answer. Andrey On Thursday, July 18, 2019 at 10:57:26 AM UTC-6, Ian Lance Taylor wrote: > > On Thu, Jul 18, 2019 at 8:35 AM Andrey Tcherepanov > <xnow4f...@sneakemail.com <javascript:>> wrote: > > > > > Basically, it's hard to be sure that there isn't any other slice > somewhere that might permit references to that trailing memory > > > > Ian, I am kind of curious about this case. I understand that "hard" is > not "impossible", but still - if I copy the needed part to a smaller array, > how does the GC knows that there are no more slices pointing to a part of > an old allocation? > > I'm not sure precisely what you are asking. Conceptually, the GC > starts with global variables and goroutine stacks and traces all > pointers to find all reachable memory. Unreachable memory is > discarded. > > What makes this case harder is that we need to not only track > pointers, we need to also look at the slice information to turn the > pointer to the backing array into a sort of bounded pointer: a pointer > to only part of the object, not all of it. That means that the GC has > to understand slice structures, which are more complex than pointers. > It means that the compiler has to build slice structures in a way that > the GC can understand, which it currently does not do. That may sound > simple, but currently the compiler splits up local slices into three > independent local variables, and that would either become impossible > or would require significantly more data to be recorded for the GC. > > Another wrinkle is that the current memory allocator does not support > shrinking memory allocations at all, since that currently never > happens. The allocator is based on storing objects of the same size > together, so it's not easy to split an existing object into smaller > sizes. That said, very large objects do have to be handled > differently and it might be more feasible to shrink those objects; I'm > not sure. > > So those are some of the reasons why it is hard. > > Ian > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/6160bc33-1ec0-4c17-ae50-e817e822fd3a%40googlegroups.com.
