Hi, guys, I have a question about memory management. I created a ballast in my go application, I didn't create the ballast in the very beginning like initializing a global variable or do this in package level init functions. I initialize it later after some setup process.
Now the problem begins. I expect the ballast like `ballast := make([]byte, 1<<30)` shouldn't take up any physical memory because there's no any writing to it. While when I use `top` command or `pmap` command to check the RSS field, it takes up the physical memory, also `pmap` reports RSS size is equal to DIRTY size. This makes me confused. After some thinking and testing, I find this is relevant with go Garbage Collector and Memory Allocator. I use 2 manners to verify this: - 1st manner, turn off the GC. I run the application with GOGC=off, then ballast takes up no physical space. - 2nd manner, turn off the GC in package level init function using `debug.SetGCPercent(-1)`, then after the necessary setup, explicitly trigger GC `debug.FreeOSMemory()`, then I create the ballast and reset the GC `ballast := make([]byte, 1<<30); debug.SetGCPercent(100)`. During these two tests, ballast works as expected. Actually If we initialize the ballast as a global variable, it will work as expected, too. These tests make me think the ballast takes up RSS is relevant with go Garbage Collector and Memory Allocator. After some time digging into go source code, I think it maybe caused like this: - during package initialization, maybe there're heap space allocation and recycle by using mmap and madvise. Maybe some pages allocated to create mcentral/mspan to prepare space for small objects. The pages are written, so they're dirty pages. Then after objects recycled, the pages maybe put into p.mcache or mcentral or page cache heap. - then we create a ballast with 1GB virtual space, it's a large object, go runtime allocate it directly by heap. It may use mmap and advise OS to provide some hugepages (actually on my OS hugepage is configured to [never]). I know the dirty size is incremented by pagesize when a specific mapped page is written. But I notice the dirty size is about 1GB (which is the same as RSS and ballast size). Who write the space? I didn't write it explicity even once. I just work around this problem using solutions mentioned above. But I am still confused what happened. That's interesting. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/8c2376a6-03bd-4797-a295-e20b1fba071en%40googlegroups.com.
