Robert Engels, I am not familiar with the two libraries you named. But from your description I think (I'm not sure) that we have different uses in mind.
The escape analysis that would be required for us to avoid using unsafe is _possible_, but does not yet exist in the Go compiler. The compiler facilities needed to negate our need for unsafe are best described in this issue https://github.com/golang/go/issues/2205 So the pattern that exists in many Go programs is that you have a variable of type string([]byte) and you have a function which takes type []byte(string) and the function only reads from []byte(string) argument. Often this comes in the form of 'I have a string, and I want to write its contents via some interface which takes []byte. Most often it is about code-deduplication, we want to read from some kind of sequence of bytes and []byte and string would both do fine. While this does open us up to a class of bugs, which are both dangerous and potentially hard to diagnose the places I see it used are usually very self contained and the benefits, if that read string/[]byte lies on a hot path are potentially significant. Although I don't know what UnsafeString/SafeString are used for I _suspect_ they are for high-performance unsafe manipulation of strings. I have never seen anyone try to use unsafe to do this in Go. Someone probably does, but the overwhelmingly most common use case that I see is 'turn this string into a []byte and use this function to read from it' going the other way is less common. Personally I would _love_ to see the read-only bytes escape analysis built into the compiler so we can throw away all of this unsafe code. On Monday, September 23, 2019 at 6:38:27 PM UTC+2, Robert Engels wrote: > > As someone that has worked with a lot of similar libraries in the HFT > space - things like UnsafeString or FastString in Java I would caution > against doing this in Go - especially as proposed here. Taking an immutable > object like string and making it mutable by accident is a recipe for > disaster. You are almost always better mapping a struct with accessors and > letting Go escape analysis perform the work on the stack and keep the > safety. > > > > On Sep 23, 2019, at 10:09 AM, Francis <francis...@gmail.com <javascript:>> > wrote: > > So I think the current state of unsafe conversions of string <-> []byte is > roughly > > 1. Use the reflect Slice/StringHeader struct. These structs give you clear > fields to set and read from. If the runtime representation of a string or > []byte ever changes then these structs should change to reflect this (they > have a non-backwards compatibility carve out in the comments). But this > also means that you run into all these exotic problems because these two > structs have a `uintpr` an `unsafe.Pointer` so for a short time the GC > won't realise you are reading/writing a pointer. This makes correct use of > these structs very difficult. > 2. You can just cast between these two types going through > `unsafe.Pointer` on the way. This works, because these two types have > almost identical layouts. We don't use any uintptr at all and so the GC > probably won't get confused. But, if the representations of string or > []byte ever change then you code breaks silently, and could have very > weird/hard to track down problems. > > So I don't think `neither is safer than the other` is quite the right > description in this context. They both have problems, so they are both > not-perfect. But their problems are quite distinct. At the least if we > choose one over the other we can describe clearly which set of problems we > want to have. > > My hope was that someone had thought through these problems and could > indicate the right way to do it. > > On a related note. I was trying to track down where the Slice/StringHeader > was first introduced. It was a long time ago > > <Rob Pike> (10 years ago) 29e6eb21ec (HEAD) > > make a description of the slice header public > > R=rsc > DELTA=18 (3 added, 0 deleted, 15 changed) > OCL=31086 > CL=31094 > > Although I couldn't open that CL in gerrit (I assume user-error). From > reading the commit I think the intention was for these header structs to be > used for this or similar things. But the data was represented as a uintptr > and a comment explicitly states that these structs are of no use without > `unsafe.Pointer`. I have seen roughly three other CL which try to change > the data field to `unsafe.Pointer` but are rejected because they change the > reflect packages API. > > There is also this issue > > https://github.com/golang/go/issues/19367 > > Which proposes that Slice/StringHeader be moved/duplicated in unsafe and > use `unsafe.Pointer`. As far as I can tell once we have this then all the > subtle problems disappear and lovingly crafted examples like > > https://github.com/m3db/m3x/blob/master/unsafe/string.go#L62 > > just become the right way to do it. > > Until then maybe we should just rely on the structural similarities > between the two types and cast between them. This seems especially > appealing as Jan pointed out above that at least one of the hypothetical > problems isn't hypothetical at all. > > > On Monday, 23 September 2019 12:43:34 UTC+2, kortschak wrote: >> >> Any particular reason for that? Neither is safer than the other and >> it's not clear to me that you can actually achieve the goal of having a >> compile-time check for the correctness of this type of conversion. >> >> On Mon, 2019-09-23 at 02:36 -0700, fra...@adeven.com wrote: >> > But this relies on a string's representation being the same as, but a >> > bit smaller thabn, a []byte. I would prefer to use >> > the Slice/StringHeader. >> >> -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golan...@googlegroups.com <javascript:>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/golang-nuts/422ca2bd-d6c8-4ebe-9578-8dd3cd8317e9%40googlegroups.com > > <https://groups.google.com/d/msgid/golang-nuts/422ca2bd-d6c8-4ebe-9578-8dd3cd8317e9%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/3a9c18d9-27c3-4c79-bbc5-8a9925e39ca6%40googlegroups.com.
