Re: [go-nuts] An efficient algorithm to write binary data

2017-10-30 Thread roger peppe
Something like this is a bit simpler, avoids the need for unsafe, and may be a little faster too: https://play.golang.org/p/4jayHwLroR It may be quicker to use a type switch: https://play.golang.org/p/4uu2XeVM9m It's a pity that about the need for a dynamic type conversion on every field, but the

Re: [go-nuts] An efficient algorithm to write binary data

2017-10-29 Thread DrGo
Thanks Michael, Here is the cleaner current version, thanks to your suggestion: //writeData loops over the field vectors and write their binary representation to an io.Writer func (sf *File) writeData(w io.Writer) error { if sf.NoObs == 0 { return nil } if len(sf.fields) == 0

Re: [go-nuts] An efficient algorithm to write binary data

2017-10-29 Thread DrGo
Michael, here is the cleaner (thanks to your suggestion) current version: //writeData loops over the field vectors and write their binary representation to an io.Writer func (sf *File) writeData(w io.Writer) error { if sf.NoObs == 0 { return nil } if len(sf.fields) == 0 {

[go-nuts] An efficient algorithm to write binary data

2017-10-29 Thread DrGo
Thanks Tamas, I agree re float32bits. I think the compiler will inline it. Right? Re binary.put*, I am essentially inlining their implementation and avoiding unnecessary use of reflection for serializing the floats. I am not sure I understand your point re the type switch, can you provide an e

[go-nuts] An efficient algorithm to write binary data

2017-10-29 Thread DrGo
Thanks, I agree re float32bits. I think the compiler will inline it. Right? Re binary.put*, I am essentially inlining their implementation and avoiding unnecessary use of reflection for serializing the floats. I am not sure I understand your point re the type switch, can you provide an example

Re: [go-nuts] An efficient algorithm to write binary data

2017-10-28 Thread Michael Jones
you could do your unsafe cast to an appropriately-sized byte array and write that as a single call or use it to access the bytes: base := *(*[8]byte)(unsafe.Pointer(&doubleThing)) bs[offset+0]=base[0] bs[offset+1]=base[1] : offset+=8 On Fri, Oct 27, 2017 at 10:18 PM, Tamás Gulácsi wrote: > Woul

[go-nuts] An efficient algorithm to write binary data

2017-10-27 Thread Tamás Gulácsi
Wouldn't math.Float64bits be easier? And encoding/binary.LittleEndian.PutUint64 would help the readability, too. If you need more speed, and the records have the same structure, you could do the type swith once: create a slice of type-specific functions, and later just invoke them. -- You rec

[go-nuts] An efficient algorithm to write binary data

2017-10-27 Thread DrGo
Hello, I am looking for help reviewing the function below. I need to generate millions of large files in a specific binary format. I do not care about portability (only used on Little Endian machines). The file format writes the data in row order, so need to loop over slices of different types a