I think you had a bug in your Memory implementation. That is the correct way to 
implement and it’s straightforward. Not sure why you ran into a problem. 
> On Oct 10, 2024, at 9:31 AM, 'Timo Beckers' via golang-nuts 
> <golang-nuts@googlegroups.com> wrote:
> 
> On Thu, 10 Oct 2024 at 14:48, Jan Mercl <0xj...@gmail.com> wrote:
>> 
>>> On Thu, Oct 10, 2024 at 2:23 PM 'Timo Beckers' via golang-nuts
>>> <golang-nuts@googlegroups.com> wrote:
>>> 
>>> I've been searching around for some info or existing conversations around 
>>> this topic, but that hasn't turned up anything useful so far. I had a 
>>> question around some implicit behaviour of Go's heap allocator.
>> 
>> From the specs point of view, there's no heap and no allocator. The
>> language specification covers allocation at
>> https://go.dev/ref/spec#Allocation, but the allocator is an
>> implementation detail with no particular guarantees.
> 
> Thanks, I didn't realize this was mentioned in the spec. I guess that
> puts an end to the conversation immediately. :)
> I'll allocate an extra page to make sure we can always align to a page 
> boundary.
> 
>> 
>>> I'm working on implementing BPF map operations through direct shared memory 
>>> access (without going through syscalls). To make the API somewhat 
>>> ergonomic, I've settled on mmapping BPF map (kernel) memory over a part of 
>>> the Go heap using MAP_FIXED. Feel free to tell me how bad of an idea this 
>>> is, I can elaborate if needed.
>> 
>> I'm not sure what exactly do you mean by "over" as there is more than
>> one possibility I can think of. Go managed memory and manually mmaped
>> memory can intermix under certain conditions. One of them is that the
>> runtime must know which is which, ie. be able to distinguish pointers
>> to memory allocated by itself from memory allocated by something else
>> - manually mmaped, C allocated etc.
>> 
> 
> Precisely, we want this memory to be both tracked by the GC _and_
> backed by shared kernel memory.
> I can only hope this isn't too common. :)
> 
>>> In order for this to work and to minimize allocations, I need a heap 
>>> allocation that starts on a page boundary. Initially, I experimented with 
>>> //go:linkname'ing mallocgc(), but I figured allocating a regular slice the 
>>> size of a page has basically the same effect. Here's a playground link: 
>>> https://go.dev/play/p/ua2NJ-rEIlC. As long as the slice/backing array is a 
>>> multiple of the architecture's page size, it seems to start on a page 
>>> boundary. I've tried allocating a bunch of ballast, forcing GCs, etc. and 
>>> it hasn't failed once.
>> 
>> Those are guarantees possibly provided by the kernel. No promises from Go.
> 
> The kernel guarantees handing out page-aligned slabs of memory to Go,
> but after that point, isn't it up to Go to slice that up however it
> pleases.
> 
>> 
>>> Here's my question: which property of the Go allocator is this banking on? 
>>> Intuitively, this makes sense to me. An 8-byte alloc needs to be 8-byte 
>>> aligned in case it's a pointer. Does a 4k allocation need to be 4k-aligned 
>>> as well (e.g. in case it's a struct), since a compiler would align members 
>>> to the start of the struct? I'm reading larger (> 8 or 16 bytes depending 
>>> on arch?) allocs have an alignment of 1 byte, and 
>>> unsafe.Alignof([4096]byte) tells me the same, but that's not the behaviour 
>>> exhibited by the allocator.
>> 
>> Implementation details that can change. You cannot rely on it. You
>> have to code it manually and look for platform details to work out.
>> 
>> -j
> 
> Yep, clear. Thanks again for the pointer to the spec, not sure why I
> didn't check there first.
> 
> Take care,
> 
> Timo
> 
> --
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/CANgQc9gD3uphTpX3V46caqQvuSyNJ3ieOuz%2BOJJcyXPThTi%3D1Q%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/1FED7365-9DBE-4FF8-9187-FB22E3D3DE43%40ix.netcom.com.

Reply via email to