singhpratech opened a new issue, #1297:
URL: https://github.com/apache/arrow-go/issues/1297
**Describe the enhancement requested**
`memory.NewGoAllocator` (which is `memory.DefaultAllocator` unless the
program is built with the
`mallocator` tag) returns buffers whose address is aligned to Go's 8 KiB
heap span but not to the
16 KiB page of Apple silicon: over repeated builds of a 10,000,000-element
int64 array, the values
buffer sits at offset 0 or 8192 within the page, and which one depends on
heap state. In this run all
twenty were at 8192; earlier runs on the same machine gave between 5 and 15
of 20 at offset 0. Nothing
in the `memory` package documents what alignment any allocator guarantees,
and the allocator the cdata
documentation recommends when C retains pointers is behind the `ccalloc`
build tag and needs a C++
Arrow installation.
Why it matters: a consumer that can use a buffer in place only when it is
page aligned (a GPU
runtime wrapping shared memory, an mmap-based store, an io_uring or DMA
path) gets a copy or a
zero-copy import at random from the same code. #70 (closed) covered the C
Data Interface storing
unpinned Go pointers, which `runtime.Pinner` now addresses; this is about
alignment only.
Go 1.27.1, macOS 26.6 (arm64), arrow-go v18.7.0; identical on v18.8.0-rc1.
```go
const page = 16384
vals := make([]int64, 10_000_000)
var offsets []uintptr
for i := 0; i < 20; i++ {
b := array.NewInt64Builder(memory.NewGoAllocator())
b.AppendValues(vals, nil)
arr := b.NewArray()
p := uintptr(unsafe.Pointer(&arr.Data().Buffers()[1].Bytes()[0]))
offsets = append(offsets, p%page)
arr.Release(); b.Release()
}
fmt.Println(offsets)
```
Output:
```
[8192 8192 8192 8192 8192 8192 8192 8192 8192 8192 8192 8192 8192 8192 8192
8192 8192 8192 8192 8192]
```
**What would help**
A documented alignment guarantee per allocator, and an allocator option (or
a small aligned
allocator in the `memory` package, built on `make([]byte, n+align)` and a
sliced start) that returns
buffers aligned to a caller-chosen power of two, so a C consumer can rely on
it without cgo or a C++
install. A 40-line version of that allocator exists in the ArrowMetal Go
binding
(https://github.com/singhpratech/ArrowMetal/blob/main/go/arrowmetal,
`PageAlignedAllocator`) and can
be contributed if the shape is acceptable.
**Component(s)**
Go, Memory
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
