In the runtime we use structs like these, but with unsafe.Pointer data
fields (runtime.stringStruct and runtime.slice). They are much safer to use
than reflect's types with uintptr Data fields. Unfortunately we can't
change reflect's types because of the Go 1 compatibility guarantee.
You can do the same thing the runtime does. Something like this would work,
and hopefully catch any future changes which would break the implementation:
type stringHeader struct {
data unsafe.Pointer
len int
}
type sliceHeader struct {
data unsafe.Pointer
len int
cap int
}
func init() {
// Check to make sure string header is what reflect thinks it is.
// They should be the same except for the type of the Data field.
if unsafe.Sizeof(stringHeader{}) != unsafe.Sizeof(reflect.StringHeader{}) {
panic("string layout has changed")
}
x := stringHeader{}
y := reflect.StringHeader{}
x.data = unsafe.Pointer(y.Data)
y.Data = uintptr(x.data)
x.len = y.Len
// same for slice
}
On Monday, September 23, 2019 at 9:38:27 AM UTC-7, Robert Engels wrote:
>
> As someone that has worked with a lot of similar libraries in the HFT
> space - things like UnsafeString or FastString in Java I would caution
> against doing this in Go - especially as proposed here. Taking an immutable
> object like string and making it mutable by accident is a recipe for
> disaster. You are almost always better mapping a struct with accessors and
> letting Go escape analysis perform the work on the stack and keep the
> safety.
>
>
>
> On Sep 23, 2019, at 10:09 AM, Francis <francis...@gmail.com <javascript:>>
> wrote:
>
> So I think the current state of unsafe conversions of string <-> []byte is
> roughly
>
> 1. Use the reflect Slice/StringHeader struct. These structs give you clear
> fields to set and read from. If the runtime representation of a string or
> []byte ever changes then these structs should change to reflect this (they
> have a non-backwards compatibility carve out in the comments). But this
> also means that you run into all these exotic problems because these two
> structs have a `uintpr` an `unsafe.Pointer` so for a short time the GC
> won't realise you are reading/writing a pointer. This makes correct use of
> these structs very difficult.
> 2. You can just cast between these two types going through
> `unsafe.Pointer` on the way. This works, because these two types have
> almost identical layouts. We don't use any uintptr at all and so the GC
> probably won't get confused. But, if the representations of string or
> []byte ever change then you code breaks silently, and could have very
> weird/hard to track down problems.
>
> So I don't think `neither is safer than the other` is quite the right
> description in this context. They both have problems, so they are both
> not-perfect. But their problems are quite distinct. At the least if we
> choose one over the other we can describe clearly which set of problems we
> want to have.
>
> My hope was that someone had thought through these problems and could
> indicate the right way to do it.
>
> On a related note. I was trying to track down where the Slice/StringHeader
> was first introduced. It was a long time ago
>
> <Rob Pike> (10 years ago) 29e6eb21ec (HEAD)
>
> make a description of the slice header public
>
> R=rsc
> DELTA=18 (3 added, 0 deleted, 15 changed)
> OCL=31086
> CL=31094
>
> Although I couldn't open that CL in gerrit (I assume user-error). From
> reading the commit I think the intention was for these header structs to be
> used for this or similar things. But the data was represented as a uintptr
> and a comment explicitly states that these structs are of no use without
> `unsafe.Pointer`. I have seen roughly three other CL which try to change
> the data field to `unsafe.Pointer` but are rejected because they change the
> reflect packages API.
>
> There is also this issue
>
> https://github.com/golang/go/issues/19367
>
> Which proposes that Slice/StringHeader be moved/duplicated in unsafe and
> use `unsafe.Pointer`. As far as I can tell once we have this then all the
> subtle problems disappear and lovingly crafted examples like
>
> https://github.com/m3db/m3x/blob/master/unsafe/string.go#L62
>
> just become the right way to do it.
>
> Until then maybe we should just rely on the structural similarities
> between the two types and cast between them. This seems especially
> appealing as Jan pointed out above that at least one of the hypothetical
> problems isn't hypothetical at all.
>
>
> On Monday, 23 September 2019 12:43:34 UTC+2, kortschak wrote:
>>
>> Any particular reason for that? Neither is safer than the other and
>> it's not clear to me that you can actually achieve the goal of having a
>> compile-time check for the correctness of this type of conversion.
>>
>> On Mon, 2019-09-23 at 02:36 -0700, fra...@adeven.com wrote:
>> > But this relies on a string's representation being the same as, but a
>> > bit smaller thabn, a []byte. I would prefer to use
>> > the Slice/StringHeader.
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golan...@googlegroups.com <javascript:>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/422ca2bd-d6c8-4ebe-9578-8dd3cd8317e9%40googlegroups.com
>
> <https://groups.google.com/d/msgid/golang-nuts/422ca2bd-d6c8-4ebe-9578-8dd3cd8317e9%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/a0d74779-a957-473f-a541-7e4cb2a9d6e0%40googlegroups.com.
