I don't think it is a GC bug, but rather a trade-off of several desirable properties.
The system does the following: - allocates 2gb - Since the current heap size is 0 the 10% gc limit is reached and a GC is started concurrently with the program - While the program processes the array, the GC finishes (using a separate CPU core probably) - The heap is now 2gb - The next 2gb is requested - Since the current heap size is 2gb, the limit of 10% is around 200mb. This is reached, so a new concurrent allocation is started - The system mallocs the 2gb in - At this point, your program needs 4gb memory, but the GC is running and immediately frees up 2 gigabytes - ... - The 2gigabytes are given back to the OS if nothing else in the program needs it. TL;DR: You must construct additional pylons! You have two options if the malloc is denied by the operating system: - Crash the program. This is the current behavior - Block the program, try to GC and see if that frees up enough space. I think Java did the blocking solution for a while. The problem with blocking is that your program now has latency problems. Worse, a system that is about to die ends up in a perpetual state where is grinds to a halt while trying to squeeze out more bytes from the GC, but fails to do so, which retrigger another blocking GC, and so on. Trying to recover from lack of memory is almost always a futile endeavor which rarely ends up with good results. Go is a language created at solving systems problems. Systems problems generally want low latency operation over high throughput, so a GC which blocks the program is always undesirable. Also, your program is somewhat extreme because it requests almost all of the systems available memory very quickly. I know of no garbage collected language implementation which fares well in that situation :) On Tue, May 2, 2017 at 12:59 PM Serhat Sevki Dincer <jfcga...@gmail.com> wrote: > The allocation request (make) is made to the runtime which covers GC, > right? GC percent is also set to 10%. > After 1st call returns, my app has about 2gb ram allocated. When 2nd call > requests allocation, runtime cannot: > - first allocate another 2gb > - free the 1st buffer later > due to the definition of the GOGC (GC percent). > I think this is a GC bug. > > 2 May 2017 07:09 tarihinde "Sokolov Yura" <funny.fal...@gmail.com> yazdı: > >> GC is triggered *after* allocation than crosses boundary. So your second >> allocation is actually tries to complete before first allocation freed. And >> Ubuntu with 4GB memory doesn't allow to allocate 4GB memory cause >> overcommit is not enabled by default. >> >> Use C/C++, or buy more memory, or change your datastructure from slice to >> slice of slice and allocate second dimension lazely, or call runtime.GC() >> explicitely between calls to f() (it will block until GC finishes). >> >> -- >> > You received this message because you are subscribed to a topic in the >> Google Groups "golang-nuts" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/golang-nuts/HrZpsyfb3i0/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> golang-nuts+unsubscr...@googlegroups.com. > > >> For more options, visit https://groups.google.com/d/optout. >> > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
