(sorry about the code formatting gone wrong, I replied in gmail it it seems
to have removed all indentation!)
On Wednesday, 16 December 2020 at 10:15:07 UTC Arnaud Delobelle wrote:
> Hi Ben, that's an interesting idea. I considered it at the start but
> didn't go for it in the end (I can't remember why exactly, probably
> because that would make it quite a big struct for Lua). There is a
> possibility that I could adapt it a bit and have something like
>
> type Value struct {
> scalar uint64
> iface interface{}
> }
>
> The type could be always obtained from the iface field (it would be
> its concrete type), but the value could be encoded in the scalar field
> for a few types such as int64, float64, bool. There would be no
> storage overhead for int64 and floa64, as the extra 8 bytes used for
> the scalar field are saved by having a "constant" iface field. The
> overhead for other non-scalar values would be only 8 bytes.
>
> I would need some reusable "dummy" interface values for the types
> encoded in the scalar:
>
> var (
> dummyInt64 interface{} = int64(0)
> dummyFloat64 interface{} = float64(0)
> dummyBool interface{} = false
> )
>
> Then I could create Value instances like this:
>
> func IntValue(n int64) Value {
> return Value{uint64(n), dummyInt64}
> }
>
> func FloatValue(f float64) Value {
> return Value{*(*uint64)(unsafe.Pointer(&f)), dummyFloat64}
> }
>
> func BoolValue(b bool) Value {
> var s uint64
> if b {
> s = 1
> }
> return Value{s, dummyBool}
> }
>
> func StringValue(s string) Value {
> return Value{iface: s}
> }
>
> func TableValue(t Table) Value {
> return Value{iface: t}
> }
>
> We could obtain the type of Values like this:
>
> type ValueType uint8
>
> const (
> IntType ValueType = iota
> FloatType
> BoolType
> StringType
> TableType
> )
>
> func (v Value) Type() ValueType {
> switch v.iface.(type) {
> case int64:
> return IntType
> case float64:
> return FloatType
> case bool:
> return BoolType
> case string:
> return StringType
> case Table:
> return TableType
> default:
> panic("invalid type")
> }
> }
>
> Methods like this could extract the concrete value out a Value instance:
>
> func (v Value) AsInt() int64 {
> return int64(v.scalar)
> }
>
> func (v Value) AsFloat() float64 {
> return *(*float64)(unsafe.Pointer(&v.scalar))
> }
>
> func (v Value) AsBool() bool {
> return v.scalar != 0
> }
>
> func (v Value) AsString() string {
> return v.iface.(string)
> }
>
> func (v Value) AsTable() Table {
> return v.iface.(Table)
> }
>
> Interoperability with Go code is not as good but still OK. There is
> no need to maintain a pool of reusable values, which is a bonus. I'll
> have to see how much modification to the codebase it requires, but
> that sounds interesting.
>
> --
> Arnaud
>
> On Tue, 15 Dec 2020 at 20:06, ben...@gmail.com <ben...@gmail.com> wrote:
> >
> > Nice project!
> >
> > It's a pity Go doesn't have C-like unions for cases like this (though I
> understand why). In my implementation of AWK in Go, I modelled the value
> type as a pseudo-union struct, passed by value:
> >
> > type value struct {
> > typ valueType // Type of value (Null, Str, Num, NumStr)
> > s string // String value (for typeStr)
> > n float64 // Numeric value (for typeNum and typeNumStr)
> > }
> >
> > Code here:
> https://github.com/benhoyt/goawk/blob/22bd82c92461cedfd02aa7b8fe1fbebd697d59b5/interp/value.go#L22-L27
> >
> > Initially I actually used "type Value interface{}" as well, but I
> switched to the above primarily to model the funky AWK "numeric string"
> concept. However, I seem to recall that it had a significant performance
> benefit too, as passing everything by value avoided a number of allocations.
> >
> > Lua has more types to deal with, but you could try something similar. Or
> maybe include int64 (for bool as well) and string fields, and everything
> else falls back to interface{}? It'd be a fairly large struct, so not sure
> it would help ... you'd have to benchmark it. But I'm thinking something
> like this:
> >
> > type Value struct {
> > typ valueType
> > i int64 // for typ = bool, integer
> > s string // for typ = string
> > v interface{} // for typ = float, other
> > }
> >
> > -Ben
> >
> > On Wednesday, December 16, 2020 at 6:50:05 AM UTC+13 arn...@gmail.com
> wrote:
> >>
> >> Hi
> >>
> >> The context for this question is that I am working on a pure Go
> implementation of Lua [1] (as a personal project). Now that it is more or
> less functionally complete, I am using pprof to see what the main CPU
> bottlenecks are, and it turns out that they are around memory management.
> The first one was to do with allocating and collecting Lua "stack frame"
> data, which I improved by having add-hoc pools for such objects.
> >>
> >> The second one is the one that is giving me some trouble. Lua is a
> so-called "dynamically typed" language, i.e. values are typed but variables
> are not. So for easy interoperability with Go I implemented Lua values with
> the type
> >>
> >> // Go code
> >> type Value interface{}
> >>
> >> The scalar Lua types are simply implemented as int64, float64, bool,
> string with their type "erased" by putting them in a Value interface. The
> problem is that the Lua runtime creates a great number of short lived Value
> instances. E.g.
> >>
> >> -- Lua code
> >> for i = 0, 1000000000 do
> >> n = n + i
> >> end
> >>
> >> When executing this code, the Lua runtime will put the values 0 to 1
> billion into the register associated with the variable "i" (say, r_i). But
> because r_i contains a Value, each integer is converted to an interface
> which triggers a memory allocation. The critical functions in the Go
> runtime seem to be convT64 and mallocgc.
> >>
> >> I am not sure how to deal with this issue. I cannot easily create a
> pool of available values because Go presents say Value(int64(1000)) as an
> immutable object to me, so I cannot keep it around for later use to hold
> the integer 1001. To be more explicit
> >>
> >> // Go code
> >> i := int64(1000)
> >> v := Value(i) // This triggers an allocation (because the interface
> needs a pointer)
> >> // Here the Lua runtime can work with v (containing 1000)
> >> j := i + 1
> >> // Even though v contains a pointer to a heap location, I cannot modify
> it
> >> v := Value(j) // This triggers another allocation
> >> // Here the Lua runtime can work with v (containing 1001)
> >>
> >>
> >> I could perhaps use a pointer to an integer to make a Value out of.
> This would allow reuse of the heap location.
> >>
> >> // Go code
> >> p :=new(int64) // Explicit allocation
> >> vp := Value(p)
> >> i :=int64(1000)
> >> *p = i // No allocation
> >> // Here the Lua runtime can work with vp (contaning 1000)
> >> j := i + 1
> >> *p = j // No allocation
> >> // Here the Lua runtime can work with vp (containing 1001)
> >>
> >> But the issue with this is that Go interoperability is not so good, as
> Go int64 now map to (interfaces holding) *int64 in the Lua runtime.
> >>
> >> However, as I understand it, in reality interfaces holding an int64 and
> an *int64 both contain the same thing (with a different type annotation): a
> pointer to an int64.
> >>
> >> Imagine that if somehow I had a function that can turn an *int64 to a
> Value holding an int64 (and vice-versa):
> >>
> >> func Int64PointerToInt64Iface(p *int16) interface{} {
> >> // returns an interface that has concrete type int64, and points at p
> >> }
> >>
> >> func int64IfaceToInt64Pointer(v interface{}) *int64 {
> >> // returns the pointer that v holds
> >> }
> >>
> >> then I would be able to "pool" the allocations as follows:
> >>
> >> func NewIntValue(n int64) Value {
> >> v = getFromPool()
> >> if p == nil {
> >> return Value(n)
> >> }
> >> *p = n
> >> return Int64PointerToint64Iface(p)
> >> }
> >>
> >> func ReleaseIntValue(v Value) {
> >> addToPool(Int64IPointerFromInt64Iface(v))
> >> }
> >>
> >> func getFromPool() *int64 {
> >> // returns nil if there is no available pointer in the pool
> >> }
> >>
> >> func addToPool(p *int64) {
> >> // May add p to the pool if there is spare capacity.
> >> }
> >>
> >> I am sure that this must leak an abstraction and that there are good
> reasons why this may be dangerous or impossible, but I don't know what the
> specific issues are. Could someone enlighten me?
> >>
> >> Or even better, would there be a different way of modelling Lua values
> that would allow good Go interoperability and allow controlling heap
> allocations?
> >>
> >> If you got to this point, thank you for reading!
> >>
> >> Arnaud Delobelle
> >>
> >> [1] https://github.com/arnodel/golua
> >
> > --
> > You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> > To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/163s0WdXYIU/unsubscribe.
> > To unsubscribe from this group and all its topics, send an email to
> golang-nuts...@googlegroups.com.
> > To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/dcd07f38-1ead-4359-90f3-f6b514c7d541n%40googlegroups.com
> .
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/841a70b9-ee31-40e9-9841-adc70c008962n%40googlegroups.com.
