On Thu, 10 Oct 2024 at 14:34, Robert Engels <reng...@ix.netcom.com> wrote: > > I’m not going to tell you how bad an idea it is… > > But I would like to ask ”why”? Why not just map the address using mmap - why > does it need to be in the heap memory space?
Ha! I had a feeling someone would bite. :) The full explanation is in memory.go in this PR: https://github.com/cilium/ebpf/pull/1572/files. Essentially, what this does is: - create bpf map - allocate a slice on the Go heap - mmap the contents of the bpf map (by fd) over this slice's backing array - set finalizer on the backing array to undo (mmap(MAP_FIXED|MAP_PRIVATE|MAP_ANONYMOUS)) the mapping and patch the hole in the slab - synthesize Go objects around parts of this mmapped backing array, in particular sync/atomic types and structs This has the nice property of being completely hands-off wrt. resource management. Since the GC tracks any pointers into this part of the heap, it will call the finalizer for us when all pointers into the region have disappeared. If we'd instead create this mapping off-heap, we'd need to tie its lifecycle to a 'proxy' Go object/accessor. The first version of this implementation featured a Memory object that received this finalizer calling the 'undo' mmap(), But this proved dangerous since access to the underlying memory took place through one of those synthesized atomic.Uint64s that were not tracked by the GC. For example: ``` type Variable struct { *atomic.Uint64 m *Memory } ``` Technically, when calling Variable.Uint64.Add(), the Variable and its underlying Memory can become unreachable before the atomic access takes place, which is pretty disastrous as it would cause an access to unmapped memory. Any such call would need to be followed by a runtime.Keepalive(v) to be perfectly safe, which doesn't make for great API. Hopefully this response satisfies your curiosity! Timo > > On Oct 10, 2024, at 7:23 AM, 'Timo Beckers' via golang-nuts > <golang-nuts@googlegroups.com> wrote: > > > Hello, > > I've been searching around for some info or existing conversations around > this topic, but that hasn't turned up anything useful so far. I had a > question around some implicit behaviour of Go's heap allocator. > > I'm working on implementing BPF map operations through direct shared memory > access (without going through syscalls). To make the API somewhat ergonomic, > I've settled on mmapping BPF map (kernel) memory over a part of the Go heap > using MAP_FIXED. Feel free to tell me how bad of an idea this is, I can > elaborate if needed. > > In order for this to work and to minimize allocations, I need a heap > allocation that starts on a page boundary. Initially, I experimented with > //go:linkname'ing mallocgc(), but I figured allocating a regular slice the > size of a page has basically the same effect. Here's a playground link: > https://go.dev/play/p/ua2NJ-rEIlC. As long as the slice/backing array is a > multiple of the architecture's page size, it seems to start on a page > boundary. I've tried allocating a bunch of ballast, forcing GCs, etc. and it > hasn't failed once. > > Here's my question: which property of the Go allocator is this banking on? > Intuitively, this makes sense to me. An 8-byte alloc needs to be 8-byte > aligned in case it's a pointer. Does a 4k allocation need to be 4k-aligned as > well (e.g. in case it's a struct), since a compiler would align members to > the start of the struct? I'm reading larger (> 8 or 16 bytes depending on > arch?) allocs have an alignment of 1 byte, and unsafe.Alignof([4096]byte) > tells me the same, but that's not the behaviour exhibited by the allocator. > > So, is this just luck or is this behaviour we can rely on? > > Thanks, > > Timo > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/golang-nuts/94808426-074f-4cb8-b647-f47664aed273n%40googlegroups.com. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CANgQc9iGokpOw8R0bhWmkqS04uPwGnAUjMmCpMcKXfxkBjzk9Q%40mail.gmail.com.
