On Thu, Oct 10, 2024 at 2:23 PM 'Timo Beckers' via golang-nuts <golang-nuts@googlegroups.com> wrote:
> I've been searching around for some info or existing conversations around > this topic, but that hasn't turned up anything useful so far. I had a > question around some implicit behaviour of Go's heap allocator. >From the specs point of view, there's no heap and no allocator. The language specification covers allocation at https://go.dev/ref/spec#Allocation, but the allocator is an implementation detail with no particular guarantees. > I'm working on implementing BPF map operations through direct shared memory > access (without going through syscalls). To make the API somewhat ergonomic, > I've settled on mmapping BPF map (kernel) memory over a part of the Go heap > using MAP_FIXED. Feel free to tell me how bad of an idea this is, I can > elaborate if needed. I'm not sure what exactly do you mean by "over" as there is more than one possibility I can think of. Go managed memory and manually mmaped memory can intermix under certain conditions. One of them is that the runtime must know which is which, ie. be able to distinguish pointers to memory allocated by itself from memory allocated by something else - manually mmaped, C allocated etc. > In order for this to work and to minimize allocations, I need a heap > allocation that starts on a page boundary. Initially, I experimented with > //go:linkname'ing mallocgc(), but I figured allocating a regular slice the > size of a page has basically the same effect. Here's a playground link: > https://go.dev/play/p/ua2NJ-rEIlC. As long as the slice/backing array is a > multiple of the architecture's page size, it seems to start on a page > boundary. I've tried allocating a bunch of ballast, forcing GCs, etc. and it > hasn't failed once. Those are guarantees possibly provided by the kernel. No promises from Go. > Here's my question: which property of the Go allocator is this banking on? > Intuitively, this makes sense to me. An 8-byte alloc needs to be 8-byte > aligned in case it's a pointer. Does a 4k allocation need to be 4k-aligned as > well (e.g. in case it's a struct), since a compiler would align members to > the start of the struct? I'm reading larger (> 8 or 16 bytes depending on > arch?) allocs have an alignment of 1 byte, and unsafe.Alignof([4096]byte) > tells me the same, but that's not the behaviour exhibited by the allocator. Implementation details that can change. You cannot rely on it. You have to code it manually and look for platform details to work out. -j -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAA40n-Vot0daq2w0eCLUEU1gvJP3J32n-E86tkDwxMGT8YBMSQ%40mail.gmail.com.
