Hello everyone, I might be saying something stupid but I found this topic really interesting and I was wondering:
Instead of trying to get the notion of "slice" to replace that of "pointer" in the compiler/runtime reference frame, wouldn't it be possible / less complex to keep track of the largest subslice of a slice that is currently used? Again this is not applicable for slices of elements of complex type. But at least with native basic Go types, it would become possible to say: I have a slice S that is shared by N goroutines / different parts of the program, the largest subslice of S referenced right now is T, and T is less than X% of the length of S, so we can retrieve the remaining memory to the OS. Michel, Le ven. 19 juil. 2019 à 07:15, Andrey Tcherepanov < xnow4fippy...@sneakemail.com> a écrit : > Thank you Ian, you reply is very much appreciated. > > ... what if a compiler would do a bit of helping "magic"? I would suspect > it does know that you return a subslice of an allocated array (something > that compiler still sees up the call stack). Would it be possible to put a > code for "This big alloc is going out of scope. If resulting (surviving) > slice is less than a X% of that original thing, copy needed part, and point > a new slice there". > > Simple cases only - ints, strings, bytes. Struct of anything with pointers > are no-go (pardon the pun) - they might point to another item in the same > array. > > For cases of huge allocs, running-close-to-mem-limits, X will have to be > smaller, so you would not OOM right there. > > Unless I miss something big and obvious - which is very highly possible - > that sounds like a possible way to drop wasted space, keeping small part > required on a side. > > Again, thank you very much for the answer. > > Andrey > > > On Thursday, July 18, 2019 at 10:57:26 AM UTC-6, Ian Lance Taylor wrote: >> >> On Thu, Jul 18, 2019 at 8:35 AM Andrey Tcherepanov >> <xnow4f...@sneakemail.com> wrote: >> > >> > > Basically, it's hard to be sure that there isn't any other slice >> somewhere that might permit references to that trailing memory >> > >> > Ian, I am kind of curious about this case. I understand that "hard" is >> not "impossible", but still - if I copy the needed part to a smaller array, >> how does the GC knows that there are no more slices pointing to a part of >> an old allocation? >> >> I'm not sure precisely what you are asking. Conceptually, the GC >> starts with global variables and goroutine stacks and traces all >> pointers to find all reachable memory. Unreachable memory is >> discarded. >> >> What makes this case harder is that we need to not only track >> pointers, we need to also look at the slice information to turn the >> pointer to the backing array into a sort of bounded pointer: a pointer >> to only part of the object, not all of it. That means that the GC has >> to understand slice structures, which are more complex than pointers. >> It means that the compiler has to build slice structures in a way that >> the GC can understand, which it currently does not do. That may sound >> simple, but currently the compiler splits up local slices into three >> independent local variables, and that would either become impossible >> or would require significantly more data to be recorded for the GC. >> >> Another wrinkle is that the current memory allocator does not support >> shrinking memory allocations at all, since that currently never >> happens. The allocator is based on storing objects of the same size >> together, so it's not easy to split an existing object into smaller >> sizes. That said, very large objects do have to be handled >> differently and it might be more feasible to shrink those objects; I'm >> not sure. >> >> So those are some of the reasons why it is hard. >> >> Ian >> > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/golang-nuts/6160bc33-1ec0-4c17-ae50-e817e822fd3a%40googlegroups.com > <https://groups.google.com/d/msgid/golang-nuts/6160bc33-1ec0-4c17-ae50-e817e822fd3a%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CANgi335naByTb%3DtsY1oEyKQ_mEN6%2BLyFqFyc%2BRHhMfsxJ4UWow%40mail.gmail.com.
