On Wed, Sep 6, 2017 at 2:58 AM, Jesper Louis Andersen <
jesper.louis.ander...@gmail.com> wrote:
> It is correct that you get superpages/hugepages automatically if they are
> enabled. But your software must also allocate pages in a way which lets the
> operating system carve out a huge page through a mmap() call (or similar).
> If the hugepage is 2 megabytes and your garbage collector allocates in
> 512kilobyte chunks for instance, you don't get the advantage.
>
Thanks for the explanation. Few questions.
1. OS( sorry only worked on Linux kernel), usually keeps track of per CPU
pages for each of the cores on the Machine. What this means is that when a
page fault occurs and there is no mapping for the page in current address
space, the underlying implementation of alloc() is invoked, which usually
will be a mmap or a setbrk or something similar to grow the address space
for current process, which then in turn shall land into a slab allocator in
the kernel which in turn is built on top of the per CPU page cache(??). So,
when you allocate, you cannot get anything less than a page (which can be
4k or 2MB or 2GB or 100PB depending on MMU implementation of the underlying
hardware). The userspace allocator then shall carve the chunk out of these
Virtual pages which are mapped to real physical pages by the OS. But, the
allocator you use can be a rudimentary one or a mature once like glibc
allocator or its variant which is smart enough to allocate correctly sized
chunks. I am wondering, wouldn't it make sense for Go runtime to use glibc
allocator and just continue working in whatever chunks it wants as long as
system allocator keeps giving you Virtual pages?
2. What would be the advantage of having to reinvent the wheel to write
allocator in the first place? Is this because of Syscall overhead? If yes,
it is anyway one time overhead and the same memory can be reused perhaps in
the same way Erlang uses carriers?
Thanks,
--Pradeep
>
> The tunable you often want is a way to tell the GC that it should allocate
> its "carriers[0]" in multiples of 2 megabyte.
>
> Does it matter? If you run outside a virtualized environment it does
> matter quite a lot in some workloads as you can quickly fill a TLB after a
> shootdown, which then improves the efficiency of the system. For other
> workloads, I haven't been able to measure any difference.
>
> [0] This terminology is taken from Erlang, for the lack of a better
> definition in Go-land; I don't know too much about the GC internals of Go.
> The VM allocates a huge page as a "carrier" to the scheduler, and
> individual processes then carve out process memory from the carrier through
> a normal allocation strategy such as best-fit. Process memory is garbage
> collected, extended and released to/from the carrier.
>
>
> On Tue, Sep 5, 2017 at 7:04 PM Pradeep Singh <raute...@gmail.com> wrote:
>
>> What is the issue you are facing?
>>
>> You do not need to do anything special to enable Hugepages for Go.
>> Huge page size is a kernel property usually set at Kernel compilation
>> time or set by your Linux Distribution.
>>
>> Go does not care about what the underlying page size is, the page size is
>> transparent to Go or any other application.
>>
>> Unless, you are doing some kind of direct IO or memory mapped voodoo
>> which require intricate knowledge where your page boundaries are.
>>
>> Thanks,
>>
>>
>> On Monday, 4 September 2017 21:17:27 UTC-7, Linker Lin wrote:
>>
>>> Hi, Guys!
>>> I have a lot of docker running go micro-services which have a big-heap.
>>> My question is how to make go work with hugepagesize in linux/amd64 ?
>>> Thanks,
>>>
>>> --
>>>
>>
>>>
>>> *Regards,Linker linlinker...@gmail.com*
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
--
Pradeep Singh
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
