Re: [go-nuts] Shiny gldriver Texture.Fill

[Nigel Tao]

On Sun, Feb 19, 2017 at 6:11 PM, Dmitry Kravchenko  wrote:
> When will Texture.Fill be implemented in Shiny gldriver?

I mailed out https://go-review.googlesource.com/37415

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Support for the race detector on ARM

['Dmitry Vyukov' via golang-nuts]

C++ ThreadSanitizer works on arm64. So making it work for Go should be
a modest amount of work.
ThreadSanitizer does not work on any 32-bit platforms. It assumes that
it can reserve huge continuous chunks of address space for various
things.
ThreadSanitizer is not dependend on underlying hardware memory model,
it check against abstract language memory model. So even if you
production code runs on arm32, you can well test it for races on
x86_64.

On Fri, Feb 24, 2017 at 4:28 AM, Ian Lance Taylor  wrote:
> [ +dvyukov ]
>
> On Thu, Feb 23, 2017 at 1:34 PM, Owen Waller  wrote:
>> Hi Ian,
>>
>> Go's race detector is based on and uses Thread Sanitizer, which has
>> only been implemented for amd64.  I'm not aware of any effort to
>> extend thread sanitizer to other processors.  That would have been
>> done before there is any likelihood of Go supporting it.
>>
>> Ian
>>
>>
>> Thanks for the help.
>>
>> After a little digging, the best I can come up with is that there might be
>> an AARCH64 port. That would cover ARM8 64-bit at least.
>> The reference I've found is on the thread sanitizer google group is, here:
>> https://groups.google.com/forum/#!searchin/thread-sanitizer/Port$20to$20ARMv7|sort:relevance/thread-sanitizer/14rRfPAr8vE/dJJ1fzrYAAAJ
>> [1]
>> Which leads to a discussion on llvm.org which suggests it's done( maybe...)
>> https://reviews.llvm.org/D11484
>>
>> What would be the best way to confirm if the AArch64 support exists, as that
>> seems to be a prerequisite for at least an ARM8 race detector?
>>
>> I can't find any existing solution to the 32bit ARMs - v5, v6, v7. Thread
>> Sanitizer seems to need a whole heap of memory so there is a very real risk
>> out running out of RAM. But then the typical program sizes that these cores
>> run are much smaller. If we were talking in terms of programs that only used
>> 10s to 100's of Megs of RAM - you'd have a reasonably good chance of x5-10
>> those amounts being free. So maybe there would be a case to try and port it
>> to 32 bit platforms, but I admit that is a much bigger question.
>>
>> Owen
>>
>> [1] This post may also be related as it discusses getting the thread
>> sanitizer running on a AMD AArch64 board with a 42 bit VA space.
>> https://groups.google.com/forum/#!searchin/thread-sanitizer/mips$20support%7Csort:relevance/thread-sanitizer/QQ3fOtA7qcM/tycOcC1jq3AJ
>> [2] There also seems to be PPC64 port too
>> https://groups.google.com/forum/#!searchin/thread-sanitizer/ppc$20support|sort:relevance/thread-sanitizer/D-ku2r5cDtU/BbFcY2Gf8qAJ
>> If this is also true then it also opens up using the race detector on the
>> PPC64 go ports.
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Shiny gldriver Texture.Fill

[Dmitry Kravchenko]

Thank you

On Friday, February 24, 2017 at 11:53:30 AM UTC+3, Nigel Tao wrote:
>
> On Sun, Feb 19, 2017 at 6:11 PM, Dmitry Kravchenko  > wrote: 
> > When will Texture.Fill be implemented in Shiny gldriver? 
>
> I mailed out https://go-review.googlesource.com/37415 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] errors.New("...") vs package level error vars

[lee . hambley]

I'm currently upgrading an application to Go 1.8 (thanks for the compile 
time speed boost, by the way).

I'm falling over a lot of issues with URL parsing now blowing up on RFC3986 
compliance issues 
(see 
https://github.com/golang/go/blob/ea5529de155cfd3f2c31698344b1ca001e0f8819/src/net/url/url.go#L517)
 
and something occured to me.

In net/http for example the vars ErrBodyNotAllowed, ErrHijacked, 
ErrContentLength etc are provided at the top level, whereas in the changes 
to the URL package (linked above) simply use errors.New() inline.

It strikes me that the stdlib is very inconsistent in this regard, and it 
makes my job in checking for valid/invalid URLs more difficult as I'm now 
doing sub string matching on err.Error() which leaves a bad taste in my 
mouth.

What're people's thoughts, would it be worth my investing time to prepare a 
patchset for 1.9 to normalize this, so that all errors in the stdlib are 
handled the same way that net/http handles them, as top level package vars?

Cheers, Lee

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: correct/working 9p2000 library in Go?

[Ilya Kostarev]

Hi, David
thanks for the code,
slightly offtopic, seems you heavily use Acme which can explore 9p file 
system directly without mounting, but I forget the syntax on plan9port 
and can't find appropriate documentation. Can you kindly give me a hint.

Cheers
__
Ilya

On 02/24/2017 10:20 AM, David Arroyo wrote:
I'm the author of this one[1], and I hate to contribute to the OP's 
initial complaint, but it is not complete. I plan to make it as easy 
to use as net/http, but I can only find so much time to work on it. 
I've also fixed a number of bugs that arose from my misunderstanding 
of the protocol, so it would only be compliant by luck right now. I 
did use it to write a toy server[2], so it's functional,  but 
definitely a moving target.


Regards,
David

[1]: https://aqwari.net/net/styx
[2]: https://github.com/droyo/jsonfs (go get github.com/droyo/jsonfs)



--
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Support for the race detector on ARM

[Owen Waller]

Hi Dimitry,

> > C++ ThreadSanitizer works on arm64. So making it work for Go should
be
> a modest amount of work.

Thanks for confirming this. Should I now Open a issue on Github so we
can track adding this?
Also, does it exist for any other platforms?
There are hints of PPC64. What about MIPS64? or any of the other
platforms that Go supports (at least on linux) - SPARC64?

> ThreadSanitizer does not work on any 32-bit platforms. It assumes that
it can reserve huge continuous chunks of address space for various
things.

I have no idea how the ThreadSanitizer works, but can you clarify what
you mean by "huge"?
I'm coming at this from am embedded angle, So huge
to me is a program that swallows 10's of Megs of RAM.

32bit support would potentially also allow support on Android - on current 
hardware.


> ThreadSanitizer is not dependend on underlying hardware memory model,
it check against abstract language memory model. So even if you
production code runs on arm32, you can well test it for races on
x86_64.


Unfortunately I'm in a case where this isn't possible. I'm trying to
make sure that an library I am writing is go routine safe. The library
itself  writes directly to the i2c bus on a Raspberry Pi to control an
IC. The code isn't ARM only (per say) but the hardware element means it
is.

Owen

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Reflection: Constructing a struct type that satisfies an interface

[Ivan Vučica]

Cool, that's exactly the stuff I was interested in.

(I was actually trying to figure out why implementing "grpc-web" is not a
trivial problem. :))

On Thu, Feb 23, 2017 at 12:33 PM, Axel Wagner  wrote:

> From what I know, a) with the normal protobuf package, you can not en- or
> decode and then reflect messages by the proto descriptor, you need to
> compile in the generated code - adding methods to a reflect-created struct
> alone doesn't help, as the package also has some assumptions about the
> representation of the type, from what I can tell. And b) you can not invoke
> reflected RPC calls with the go grpc implementation. Related issues are
> https://github.com/golang/protobuf/issues/199
> https://github.com/grpc/grpc-go/issues/866
>
> On Thu, Feb 23, 2017 at 1:16 PM, Ivan Vučica  wrote:
>
>> What I want, in the end, is to send a gRPC request based on an ASCII or
>> JSON representation of a protobuf, receive a response and display it.
>>
>> Essentially, if you take grpc_cli tool present in the repo of C
>> implementation of gRPC, I would like to implement its "call" functionality.
>>
>> Even though trying to construct a binary protobuf /might/ be a dead end
>> depending on how I would have to invoke the RPC itself, building a
>> proto.Message out of a remote protobuf descriptor happens to be an
>> interesting problem in itself.
>>
>> It'd be less than fun if the solution is to write yet another proto
>> serializer/deserializer. And I didn't even check if, when using grpc-go, I
>> can invoke a remote gRPC method using just its name...
>>
>> Of course, if you have another shortcut that I missed in grpc-go's APIs
>> or in Go protobuf's APIs, that'd be great, too ☺️
>>
>> On Thu, Feb 23, 2017, 02:57 Matt Harden  wrote:
>>
>>> Is the intermediate Go struct necessary, or do you just want to convert
>>> a text proto to a binary representation?
>>>
>>> On Wed, Feb 22, 2017 at 6:10 PM  wrote:
>>>
>>> Hi,
>>>
>>> I'm fiddling with gRPC and its service reflection. I discovered a neat
>>> package (github.com/jhump/protoreflect) that let me quickly enumerate
>>> the services that are exposed by the gRPC server, the RPCs that are in
>>> those gRPC services, and finally the proto messages that are used as inputs
>>> and outputs.
>>>
>>> Later, I'll worry about how to actually send out the RPC.
>>>
>>> For now, I'm trying to unserialize a text proto message into a
>>> dynamically constructed Go struct, which I'd then serialize back into a
>>> binary proto.
>>>
>>> I'd like to make the generated struct satisfy the proto.Message
>>> interface. There are three methods required by that interface.
>>>
>>> How would one go about attaching the required methods onto the newly
>>> constructed type?
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golang-nuts+unsubscr...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Debugging long GC pauses

[Oliver Beattie]

Thanks for getting back to me. I have posted the output from a 30-second 
runtime/trace session here, which appears to show three 80ms GC's:

https://www.dropbox.com/s/wc3hp6k17txaofh/staging-trace.tar.gz?dl=0

I'm struggling a bit to interpret what's there though.

On Thursday, February 23, 2017 at 7:46:42 PM UTC, Austin Clements wrote:
>
> AFAIK, the only thing that can cause this in Go 1.8 is a non-preemptible 
> loop. It's not related to the heap size at all.
>
> To test this theory, you can set GOEXPERIMENT=preemptibleloops and rebuild 
> your Go tree (the compiler has to be built with this, so you can't just 
> turn it on to build your project). I wouldn't recommend running in 
> production with this, but if it eliminates the long pauses, we'll at least 
> know that's the culprit.
>
> Since these are quite long, the other thing you can do is run with the 
> execution tracer (https://godoc.org/runtime/trace). You'll be able to see 
> exactly what's happening at the beginning of each GC cycle. If you do have 
> non-preemptible loops, you should also see goroutines executing for much 
> longer than 10ms at a time, which is the default preemption bound.
>
> On Thu, Feb 23, 2017 at 1:46 PM, Oliver Beattie  > wrote:
>
>> I am looking for some advice about how I can debug some long GC pauses I 
>> am observing in our production workloads under go 1.8 (the problem is not 
>> specific to 1.8, though). This is a very simple network server – basically 
>> a HTTP ping endpoint – but I regularly see tail request latencies of 
>> >100ms. I have enabled GODEBUG=gctrace=1, and I can see some quite long 
>> STW pauses amid lots of much less worrying pauses:
>>
>> gc 54 @348.007s 0%: 0.061+81+0.040 ms clock, 0.12+0.39/81/81+0.081 ms 
>> cpu, 4->4->1 MB, 5 MB goal, 2 P
>> gc 55 @358.007s 0%: 0.21+83+0.019 ms clock, 0.43+80/2.7/81+0.039 ms cpu, 
>> 4->4->1 MB, 5 MB goal, 2 P
>> *gc 56 @367.507s 0%: 80+1.3+0.065 ms clock, 161+0.080/1.2/82+0.13 ms cpu, 
>> 4->4->1 MB, 5 MB goal, 2 P*
>> gc 57 @377.726s 0%: 0.054+63+0.023 ms clock, 0.10+0.68/61/0.44+0.046 ms 
>> cpu, 4->4->1 MB, 5 MB goal, 2 P
>> gc 58 @388.007s 0%: 0.033+81+0.036 ms clock, 0.067+0.32/80/81+0.072 ms 
>> cpu, 4->4->1 MB, 5 MB goal, 2 P
>> gc 59 @398.007s 0%: 0.021+82+0.019 ms clock, 0.043+0.17/80/82+0.038 ms 
>> cpu, 4->4->1 MB, 5 MB goal, 2 P
>> gc 60 @407.630s 0%: 0.012+57+0.031 ms clock, 0.025+0.25/0.64/57+0.063 ms 
>> cpu, 4->4->1 MB, 5 MB goal, 2 P
>> gc 61 @418.007s 0%: 0.19+1.0+79 ms clock, 0.38+0.28/0.69/0.98+159 ms cpu, 
>> 4->4->1 MB, 5 MB goal, 2 P
>> gc 62 @427.507s 0%: 0.21+81+0.29 ms clock, 0.42+81/0.96/81+0.58 ms cpu, 
>> 4->4->1 MB, 5 MB goal, 2 P
>> gc 63 @437.507s 0%: 0.015+81+0.053 ms clock, 0.031+0.29/0.98/80+0.10 ms 
>> cpu, 4->4->1 MB, 5 MB goal, 2 P
>> *gc 64 @443.507s 0%: 81+1.2+0.032 ms clock, 162+0.040/1.2/0.44+0.065 ms 
>> cpu, 4->4->1 MB, 5 MB goal, 2 P*
>> scvg2: inuse: 4, idle: 2, sys: 7, released: 0, consumed: 7 (MB)
>> gc 65 @453.507s 0%: 0.13+81+0.051 ms clock, 0.26+0.20/81/82+0.10 ms cpu, 
>> 4->4->1 MB, 5 MB goal, 2 P
>>
>> If I am reading this correctly, some of these STW pauses are 80+ 
>> milliseconds, in order to scan a minuscule heap. I am not experienced with 
>> debugging the GC in Go, so I'd appreciate any pointers as to why this could 
>> happening and what I can do to get to the bottom of the behaviour. Many 
>> thanks :)
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Support for the race detector on ARM

['Dmitry Vyukov' via golang-nuts]

On Fri, Feb 24, 2017 at 2:39 PM, Owen Waller  wrote:
> Hi Dimitry,
>
> C++ ThreadSanitizer works on arm64. So making it work for Go should be
> a modest amount of work.
>
>
> Thanks for confirming this. Should I now Open a issue on Github so we can
> track adding this?

If there is somebody who is willing to work on this, then yes. Otherwise, maybe.

> Also, does it exist for any other platforms? There are hints of PPC64. What
> about MIPS64? or any of the other platforms that Go supports (at least on
> linux) - SPARC64?

Supported platforms are enumerated in
llvm/projects/compiler-rt/lib/tsan/rtl/tsan_platform.h
I see x86_64, aarch64, mips64 and powerpc64


> ThreadSanitizer does not work on any 32-bit platforms. It assumes that
> it can reserve huge continuous chunks of address space for various
> things.
>
>
> I have no idea how the ThreadSanitizer works, but can you clarify what you
> mean by "huge"?
> I'm coming at this from am embedded angle, So huge to me is a program that
> swallows 10's of Megs of RAM.

Tsan reserves 4X for shadow memory (where X is amount of memory where
user data resides), 0.5X for another shadow memory, 1TB for heap and
128GB for thread info. You can see the details in
llvm/projects/compiler-rt/lib/tsan/rtl/tsan_platform.h.



> 32bit support would potentially also allow support on Android - on current
> hardware.

There are 64-bit androids already. I would rather test with tsan
there. That's what we do for kernel ASAN.


> ThreadSanitizer is not dependend on underlying hardware memory model,
> it check against abstract language memory model. So even if you
> production code runs on arm32, you can well test it for races on
> x86_64.
>
>
> Unfortunately I'm in a case where this isn't possible. I'm trying to make
> sure that an library I am writing is go routine safe. The library itself
> writes directly to the i2c bus on a Raspberry Pi to control an IC. The code
> isn't ARM only (per say) but the hardware element means it is.

Maybe it's possible to stub the i2c part? That would make testing much
simpler regardless of tsan.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] errors.New("...") vs package level error vars

[Steven Hartland]

Internal errors are one of my pet peeves about the core go libs, as you 
say they are inconsistent and you end up doing all sorts of string 
checks which are not only hackey but are very fragile too. We had a 
break in our code due to just this when updating to 1.8.


There are very few reason why errors should be private in my mind 
especially simple errors.New(...) ones.


I would personally love to see this fixed.

Regards
Steve


On 24/02/2017 10:28, lee.hamb...@gmail.com wrote:
I'm currently upgrading an application to Go 1.8 (thanks for the 
compile time speed boost, by the way).


I'm falling over a lot of issues with URL parsing now blowing up on 
RFC3986 compliance issues 
(see https://github.com/golang/go/blob/ea5529de155cfd3f2c31698344b1ca001e0f8819/src/net/url/url.go#L517) 
and something occured to me.


In net/http for example the vars ErrBodyNotAllowed, ErrHijacked, 
ErrContentLength etc are provided at the top level, whereas in the 
changes to the URL package (linked above) simply use errors.New() inline.


It strikes me that the stdlib is very inconsistent in this regard, and 
it makes my job in checking for valid/invalid URLs more difficult as 
I'm now doing sub string matching on err.Error() which leaves a bad 
taste in my mouth.


What're people's thoughts, would it be worth my investing time to 
prepare a patchset for 1.9 to normalize this, so that all errors in 
the stdlib are handled the same way that net/http handles them, as top 
level package vars?


Cheers, Lee
--
You received this message because you are subscribed to the Google 
Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to golang-nuts+unsubscr...@googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] errors.New("...") vs package level error vars

[Manlio Perillo]

Il giorno venerdì 24 febbraio 2017 14:30:54 UTC+1, Steven Hartland ha 
scritto:
>
> Internal errors are one of my pet peeves about the core go libs, as you 
> say they are inconsistent and you end up doing all sorts of string checks 
> which are not only hackey but are very fragile too. We had a break in our 
> code due to just this when updating to 1.8.
>
> There are very few reason why errors should be private in my mind 
> especially simple errors.New(...) ones.
>
>
AFAIK, the developers of the Go standard library tend to keep the API as 
small as possible.

> [...]

Manlio

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] errors.New("...") vs package level error vars

[Konstantin Khomoutov]

On Fri, 24 Feb 2017 02:28:39 -0800 (PST)
lee.hamb...@gmail.com wrote:

> I'm currently upgrading an application to Go 1.8 (thanks for the
> compile time speed boost, by the way).
> 
> I'm falling over a lot of issues with URL parsing now blowing up on
> RFC3986 compliance issues 
> (see
> https://github.com/golang/go/blob/ea5529de155cfd3f2c31698344b1ca001e0f8819/src/net/url/url.go#L517)
> and something occured to me.
> 
> In net/http for example the vars ErrBodyNotAllowed, ErrHijacked, 
> ErrContentLength etc are provided at the top level, whereas in the
> changes to the URL package (linked above) simply use errors.New()
> inline.
> 
> It strikes me that the stdlib is very inconsistent in this regard,
> and it makes my job in checking for valid/invalid URLs more difficult
> as I'm now doing sub string matching on err.Error() which leaves a
> bad taste in my mouth.

In this particular case, isn't it subsequently wrapped in a custom
error type in an outer function [2]?

That is, your client code will actually get an error value of type
net/url.Error with its Op field being set to "parse".
I reckon your code relies on coercing these types into strings via
their Error() method and then checking the string rep.
If I'm correct, a better approach is to type-assert the error returned
to see if it's of type net/url.Error, and then checking whether its
Op field equals "parse".

> What're people's thoughts, would it be worth my investing time to
> prepare a patchset for 1.9 to normalize this, so that all errors in
> the stdlib are handled the same way that net/http handles them, as
> top level package vars?

Well, it's not me who decides but this looks to me as a "where should
be draw the line?" kind of situation: as it appears, the "url" package
already tells its clients which "kind" of error has happened ("parse"
in your case), and this pretty much matches, say, what the "os" package
does for errors related to operations on files.  Maybe an additional
field in net/url.Error which would contain some errno-like value of a
set exported from the package would be OK?

2. 
https://github.com/golang/go/blob/ea5529de155cfd3f2c31698344b1ca001e0f8819/src/net/url/url.go#L460

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] GoQt fatal error: QAbstractAnimation :no such file or directory

[Nikita Melnikov]

I tryed to compile GoQt for developing desctop applecations. I followed the 
instruction from the official wiki guied on Qt website. When i tryed to go 
run an exsample from the examples folder i grabbed an error go run 
'/home/pinkya_rabbit/workspace/go1programs/src/qt/qt/internal/examples/uitools/calculator/calculator.go'
 
workspace/go1programs/src/github.com/therecipe/qt/core/core.cpp:9:30: fatal 
error: QAbstractAnimation: no such file or directory compilation terminated.
 . 
How can i solve it?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Code review/Advice - reading and editing a CSV

[Robbie Wright]

Hi this is my first go program and I am looking for some feedback and help 
writing buffer to csv. To give you an idea of the structure of the data I 
have run head on the input file gives output seen at the bottom.

package main

import (
   "bytes"
   "encoding/csv"
   "fmt"
   "io"
   "log"
   "strconv"
   "strings"
   "bufio"
   "io/ioutil"
)

// Calculate economic block value given a 40x20x12 block of density sg and grade
func calculateE(sg float64, grade float64) float64 {
   return 40 * 20 * 12 * sg * grade * 80 / 10 * 0.66
}

// Credit: Stack Overflow
func processCSV(rc io.Reader) (ch chan []string) {
   ch = make(chan []string, 10)
   go func() {
  r := csv.NewReader(rc)
  if _, err := r.Read(); err != nil { //read header
 log.Fatal(err)
  }
  defer close(ch)
  for {
 rec, err := r.Read()

 if err != nil {
if err == io.EOF {
   break
}
log.Fatal(err)

 }
 ch <- rec
  }
   }()
   return
}

// Credit: Stack Overflow
func Readln(r *bufio.Reader) (string, error) {
   var (
  isPrefix bool  = true
  err  error = nil
  line, ln []byte
   )
   for isPrefix && err == nil {
  line, isPrefix, err = r.ReadLine()
  ln = append(ln, line...)
   }
   return string(ln), err
}

func main() {
   var buf bytes.Buffer

   w := csv.NewWriter(&buf)
   file := "C:/ORIG.csv"
   dat, _ := ioutil.ReadFile(file)
   f := string(dat)

   for rec := range processCSV(strings.NewReader(f)) {
  sg, err := strconv.ParseFloat(rec[17], 64)
  grade, err := strconv.ParseFloat(rec[13], 64)

  if grade < 0 {
 grade = 0
  } else {
 grade = 0.1*grade
  }
  if sg < 0 {
 sg = 0
  }
  fmt.Println(grade)
  // get economic block value
  e := calculateE(sg, grade)
  gradeStr := strconv.FormatFloat(grade, 'f', 8, 64)
  sgStr := strconv.FormatFloat(sg, 'f', 8, 64)
  eStr := strconv.FormatFloat(e, 'f', 8, 64)
  rec := append(rec[0:4], eStr, sgStr, gradeStr)
  if err = w.Write(rec); err != nil {
 log.Fatal(err)
  }
   }
   w.Flush()
   if err := w.Error(); err != nil {
  log.Fatal(err)
   }
   //fmt.Println(buf.String())
   //f2, err := os.Create("C:/Users/root/result.csv")
   //defer f2.Close()
   //writer := csv.NewWriter(f2)
   //writer.Write(buf.String())
}

// 500mb csv with head of the data looks like:
const data = 
`xcentre,ycentre,zcentre,xlength,ylength,zlength,fe_percent,fe_recovery,oxide,rescat,sg,mat_type_8,fillpc,mass_recovery,mass_recovery_percent,air,al2o3,cao,k2o,loi,mgo,mno,phos,sio2,tio2
556960.000,6319980.000,-1100.000,40.000,20.000,12.000,-99.,66.,-99,4,2.8490,2,91,0.,0.,0,0.,0.,0.,0.,0.,0.,0.,0.,0.
557000.000,6319980.000,-1100.000,40.000,20.000,12.000,-99.,66.,-99,4,2.8490,2,91,0.,0.,0,0.,0.,0.,0.,0.,0.,0.,0.,0.
556960.000,632.000,-1100.000,40.000,20.000,12.000,-99.,66.,-99,4,2.8490,2,91,0.,0.,0,0.,0.,0.,0.,0.,0.,0.,0.,0.
557000.000,632.000,-1100.000,40.000,20.000,12.000,-99.,66.,-99,4,2.8490,2,91,0.,0.,0,0.,0.,0.,0.,0.,0.,0.,0.,0.
556960.000,6319980.000,-1088.000,40.000,20.000,12.000,-99.,66.,-99,4,2.8490,2,100,0.,0.,0,0.,0.,0.,0.,0.,0.,0.,0.,0.
557000.000,6319980.000,-1088.000,40.000,20.000,12.000,-99.,66.,-99,4,2.8490,2,100,0.,0.,0,0.,0.,0.,0.,0.,0.,0.,0.,0.
556960.000,632.000,-1088.000,40.000,20.000,12.000,-99.,66.,-99,4,2.8490,2,100,0.,0.,0,0.,0.,0.,0.,0.,0.,0.,0.,0.
557000.000,632.000,-1088.000,40.000,20.000,12.000,-99.,66.,-99,4,2.8490,2,100,0.,0.,0,0.,0.,0.,0.,0.,0.,0.,0.,0.
557040.000,6319980.000,-1

[go-nuts] Variadic parameters to SQL-queries, specifically gopkg.in/rana/ora.V*

[trond . kandal]

Hello

Maybe this is a very simple and stupid, but I have not found
out how to specify variadic arguments to SQL-queries.  I am
a novice and I do not understand how this work in Go.

I have some REST-endpoints to implement, each endpoint may
receive GET-parameters which I want to pass on to the SQL-queries.
They may be of different types, both strings and integers.

Can anybody help me with this?

Thank you

Trond.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Variadic parameters to SQL-queries, specifically gopkg.in/rana/ora.V*

[Tamás Gulácsi]

params:=[]interface{}{1,time.Now(),"a"}
sql.exex("select count(0) from table where id=:1 and dt<:2 and z<>:3", 
params...)

See "variadic parameters".

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Variadic parameters to SQL-queries, specifically gopkg.in/rana/ora.V*

[Trond Kandal]

Thank Your for your answer, Sir!

Hmmm...
I do not seem to get it working

sqlQry := "SELECT * FROM TIA_EMNEINFO WHERE EMNEKODE = :1"

stmt, err := ses.Prep(sqlQuery,
ora.OraI64, ora.OraS, ora.OraS, ora.OraS, ora.OraS, ora.OraS, ora.OraI64,
ora.OraS, ora.OraI64, ora.L, ora.L, ora.OraI64)

defer stmt.Close()

if err != nil {

...

}

params = []interface{}{"BI101714"}
rst, err := stmt.Qry(params)

...


(The parameter is supposed to be a string)


Then I get:

error = Stmt.qryC Stmt.bind Invalid bind parameter. ([]interface{}) 
([BI101714]).


By the way, how do I extend the params with more parameters as I
collect the GET-parameters?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Variadic parameters to SQL-queries, specifically gopkg.in/rana/ora.V*

[pierre . curto]

Do a:

rst, err := stmt.Qry(params...)



The 3 dots will expand the slice for the variadic function.


Le vendredi 24 février 2017 16:05:42 UTC+1, Trond Kandal a écrit :
>
> Thank Your for your answer, Sir!
>
> Hmmm...
> I do not seem to get it working
>
> sqlQry := "SELECT * FROM TIA_EMNEINFO WHERE EMNEKODE = :1"
>
> stmt, err := ses.Prep(sqlQuery,
> ora.OraI64, ora.OraS, ora.OraS, ora.OraS, ora.OraS, ora.OraS, ora.OraI64,
> ora.OraS, ora.OraI64, ora.L, ora.L, ora.OraI64)
>
> defer stmt.Close()
>
> if err != nil {
>
> ...
>
> }
>
> params = []interface{}{"BI101714"}
> rst, err := stmt.Qry(params)
>
> ...
>
>
> (The parameter is supposed to be a string)
>
>
> Then I get:
>
> error = Stmt.qryC Stmt.bind Invalid bind parameter. ([]interface{}) 
> ([BI101714]).
>
>
> By the way, how do I extend the params with more parameters as I
> collect the GET-parameters?
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Minimizing golang docker images

[Sankar]

Hi

I have a bunch of services (all sources in a single git project) which I 
want to dockerize and deploy in kubernetes (and run CI too preferably via 
minikube).

I have seen that it is possible to generate small docker images for golang 
HTTP servers by building them separately outside the Dockerfile that will 
be used for deployment, such 
as: 
http://samuraiprogrammer.net/blog/way-to-reduce-docker-image-size-with-tarball/

But the above approach of two Dockerfiles for each service, makes it very 
difficult to manage the sources. I wanted to know if there are any 
hacks/best-practices for reducing the docker image sizes for golang 
microservices but without creating two Dockerfiles for each service. Any 
help ?

Thanks.

Sankar

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Variadic parameters to SQL-queries, specifically gopkg.in/rana/ora.V*

[Trond Kandal]

Thank You very much, Sir!

That did the trick!
I am not used to the small and subtle tricks of Go yet.

Is there any way I can append parameters to the variable as I go, before
passing it to the query?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Go Compiler Intermediate Representation

[Arpit Aggarwal]

Thank you very much adon...
Its very useful for my project.
Again, thanks a lot

Regards
Arpit

On Wednesday, February 22, 2017 at 2:13:25 PM UTC-8, adon...@google.com 
wrote:
>
> On Tuesday, 21 February 2017 12:23:44 UTC-5, Arpit Aggarwal wrote:
>>
>> I am doing a project in which I need Go compiler's intermediate 
>> representation(IR) (which is semantics preserving and I can get all all 
>> info like line number and data type and other features) (human readable) to 
>> convert to another IR of a tool.
>>
>
> The most convenient way to access the semantics of a Go program is to use 
> the golang.org/x/tools/go/ssa package, which builds a high-level 
> SSA-based intermediate representation.
>
> $ cat fib.go
> package fib
>
> func fib(x int) int {
> if x < 2 {
> return x
> }
> return fib(x-2) + fib(x-1)
> }
>
> $ go build golang.org/x/tools/cmd/ssadump
> $ ./ssadump -build=F fib.go
> # Name: fib.fib
> # Package: fib
> # Location: fib.go:3:6
> func fib(x int) int:
> 0:entry 
> P:0 S:2
> t0 = x < 2:int 
> bool
> if t0 goto 1 else 2
> 1:  if.then 
> P:1 S:0
> return x
> 2:  if.done 
> P:1 S:0
> t1 = x - 2:int 
>  int
> t2 = fib(t1)   
>  int
> t3 = x - 1:int 
>  int
> t4 = fib(t3)   
>  int
> t5 = t2 + t4   
>  int
> return t5
>
> # Name: fib.init
> # Package: fib
> # Synthetic: package initializer
> func init():
> 0:entry 
> P:0 S:2
> t0 = *init$guard   
> bool
> if t0 goto 2 else 1
> 1:   init.start 
> P:1 S:1
> *init$guard = true:bool
> jump 2
> 2:init.done 
> P:2 S:0
> return
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Variadic parameters to SQL-queries, specifically gopkg.in/rana/ora.V*

[pierre . curto]

Yes you can, use append:
https://golang.org/ref/spec#Appending_and_copying_slices

Le vendredi 24 février 2017 17:09:12 UTC+1, Trond Kandal a écrit :
>
> Thank You very much, Sir!
>
> That did the trick!
> I am not used to the small and subtle tricks of Go yet.
>
> Is there any way I can append parameters to the variable as I go, before
> passing it to the query?
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Variadic parameters to SQL-queries, specifically gopkg.in/rana/ora.V*

[Trond Kandal]

Thank You so much again, Sir!

Sorry, I have been searching around, but not found that documentation.
Now I have learned a lot!

Trond.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Code review/Advice - reading and editing a CSV

[peterGo]

Robbie,

It's hard to understand what you are doing.

The header says that there is no grade and sg is not where you say it is. 
Why are you using a channel? Why are you limiting the size of your input by 
reading the entire file into memory? And so on.

My best guess is that you want to do something like this:

Go Playground: https://play.golang.org/p/4Aok43E6AC

Peter

On Friday, February 24, 2017 at 8:58:52 AM UTC-5, Robbie Wright wrote:
>
> Hi this is my first go program and I am looking for some feedback and help 
> writing buffer to csv. To give you an idea of the structure of the data I 
> have run head on the input file gives output seen at the bottom.
>
> package main
>
> import (
>"bytes"
>"encoding/csv"
>"fmt"
>"io"
>"log"
>"strconv"
>"strings"
>"bufio"
>"io/ioutil"
> )
>
> // Calculate economic block value given a 40x20x12 block of density sg and 
> grade
> func calculateE(sg float64, grade float64) float64 {
>return 40 * 20 * 12 * sg * grade * 80 / 10 * 0.66
> }
>
> // Credit: Stack Overflow
> func processCSV(rc io.Reader) (ch chan []string) {
>ch = make(chan []string, 10)
>go func() {
>   r := csv.NewReader(rc)
>   if _, err := r.Read(); err != nil { //read header
>  log.Fatal(err)
>   }
>   defer close(ch)
>   for {
>  rec, err := r.Read()
>
>  if err != nil {
> if err == io.EOF {
>break
> }
> log.Fatal(err)
>
>  }
>  ch <- rec
>   }
>}()
>return
> }
>
> // Credit: Stack Overflow
> func Readln(r *bufio.Reader) (string, error) {
>var (
>   isPrefix bool  = true
>   err  error = nil
>   line, ln []byte
>)
>for isPrefix && err == nil {
>   line, isPrefix, err = r.ReadLine()
>   ln = append(ln, line...)
>}
>return string(ln), err
> }
>
> func main() {
>var buf bytes.Buffer
>
>w := csv.NewWriter(&buf)
>file := "C:/ORIG.csv"
>dat, _ := ioutil.ReadFile(file)
>f := string(dat)
>
>for rec := range processCSV(strings.NewReader(f)) {
>   sg, err := strconv.ParseFloat(rec[17], 64)
>   grade, err := strconv.ParseFloat(rec[13], 64)
>
>   if grade < 0 {
>  grade = 0
>   } else {
>  grade = 0.1*grade
>   }
>   if sg < 0 {
>  sg = 0
>   }
>   fmt.Println(grade)
>   // get economic block value
>   e := calculateE(sg, grade)
>   gradeStr := strconv.FormatFloat(grade, 'f', 8, 64)
>   sgStr := strconv.FormatFloat(sg, 'f', 8, 64)
>   eStr := strconv.FormatFloat(e, 'f', 8, 64)
>   rec := append(rec[0:4], eStr, sgStr, gradeStr)
>   if err = w.Write(rec); err != nil {
>  log.Fatal(err)
>   }
>}
>w.Flush()
>if err := w.Error(); err != nil {
>   log.Fatal(err)
>}
>//fmt.Println(buf.String())
>//f2, err := os.Create("C:/Users/root/result.csv")
>//defer f2.Close()
>//writer := csv.NewWriter(f2)
>//writer.Write(buf.String())
> }
>
> // 500mb csv with head of the data looks like:
> const data = 
> `xcentre,ycentre,zcentre,xlength,ylength,zlength,fe_percent,fe_recovery,oxide,rescat,sg,mat_type_8,fillpc,mass_recovery,mass_recovery_percent,air,al2o3,cao,k2o,loi,mgo,mno,phos,sio2,tio2
> 556960.000,6319980.000,-1100.000,40.000,20.000,12.000,-99.,66.,-99,4,2.8490,2,91,0.,0.,0,0.,0.,0.,0.,0.,0.,0.,0.,0.
> 557000.000,6319980.000,-1100.000,40.000,20.000,12.000,-99.,66.,-99,4,2.8490,2,91,0.,0.,0,0.,0.,0.,0.,0.,0.,0.,0.,0.
> 556960.000,632.000,-1100.000,40.000,20.000,12.000,-99.,66.,-99,4,2.8490,2,91,0.,0.,0,0.,0.,0.,0.,0.,0.,0.,0.,0.
> 557000.000,632.000,-1100.000,40.000,20.000,12.000,-99.,66.,-99,4,2.8490,2,91,0.,0.,0,0.,0.,0.,0.,0.,0.,0.,0.,0.
> 556960.000,6319980.000,-1088.000,40.000,20.000,12.000,-99.,66.,-99,4,2.8490,2,100,0.,0.,0,0.,0.,0.,0.,0.,0.,0.,0.,0.
> 557000.000,6319980.000,-1088.000,40.

[go-nuts] Re: Code review/Advice - reading and editing a CSV

[peterGo]

Robbie,

Fixed a typo: https://play.golang.org/p/JbKGJMw4r3

Peter

On Friday, February 24, 2017 at 12:48:44 PM UTC-5, peterGo wrote:
>
> Robbie,
>
> It's hard to understand what you are doing.
>
> The header says that there is no grade and sg is not where you say it is. 
> Why are you using a channel? Why are you limiting the size of your input by 
> reading the entire file into memory? And so on.
>
> My best guess is that you want to do something like this:
>
> Go Playground: https://play.golang.org/p/4Aok43E6AC
>
> Peter
>
> On Friday, February 24, 2017 at 8:58:52 AM UTC-5, Robbie Wright wrote:
>>
>> Hi this is my first go program and I am looking for some feedback and 
>> help writing buffer to csv. To give you an idea of the structure of the 
>> data I have run head on the input file gives output seen at the bottom.
>>
>> package main
>>
>> import (
>>"bytes"
>>"encoding/csv"
>>"fmt"
>>"io"
>>"log"
>>"strconv"
>>"strings"
>>"bufio"
>>"io/ioutil"
>> )
>>
>> // Calculate economic block value given a 40x20x12 block of density sg and 
>> grade
>> func calculateE(sg float64, grade float64) float64 {
>>return 40 * 20 * 12 * sg * grade * 80 / 10 * 0.66
>> }
>>
>> // Credit: Stack Overflow
>> func processCSV(rc io.Reader) (ch chan []string) {
>>ch = make(chan []string, 10)
>>go func() {
>>   r := csv.NewReader(rc)
>>   if _, err := r.Read(); err != nil { //read header
>>  log.Fatal(err)
>>   }
>>   defer close(ch)
>>   for {
>>  rec, err := r.Read()
>>
>>  if err != nil {
>> if err == io.EOF {
>>break
>> }
>> log.Fatal(err)
>>
>>  }
>>  ch <- rec
>>   }
>>}()
>>return
>> }
>>
>> // Credit: Stack Overflow
>> func Readln(r *bufio.Reader) (string, error) {
>>var (
>>   isPrefix bool  = true
>>   err  error = nil
>>   line, ln []byte
>>)
>>for isPrefix && err == nil {
>>   line, isPrefix, err = r.ReadLine()
>>   ln = append(ln, line...)
>>}
>>return string(ln), err
>> }
>>
>> func main() {
>>var buf bytes.Buffer
>>
>>w := csv.NewWriter(&buf)
>>file := "C:/ORIG.csv"
>>dat, _ := ioutil.ReadFile(file)
>>f := string(dat)
>>
>>for rec := range processCSV(strings.NewReader(f)) {
>>   sg, err := strconv.ParseFloat(rec[17], 64)
>>   grade, err := strconv.ParseFloat(rec[13], 64)
>>
>>   if grade < 0 {
>>  grade = 0
>>   } else {
>>  grade = 0.1*grade
>>   }
>>   if sg < 0 {
>>  sg = 0
>>   }
>>   fmt.Println(grade)
>>   // get economic block value
>>   e := calculateE(sg, grade)
>>   gradeStr := strconv.FormatFloat(grade, 'f', 8, 64)
>>   sgStr := strconv.FormatFloat(sg, 'f', 8, 64)
>>   eStr := strconv.FormatFloat(e, 'f', 8, 64)
>>   rec := append(rec[0:4], eStr, sgStr, gradeStr)
>>   if err = w.Write(rec); err != nil {
>>  log.Fatal(err)
>>   }
>>}
>>w.Flush()
>>if err := w.Error(); err != nil {
>>   log.Fatal(err)
>>}
>>//fmt.Println(buf.String())
>>//f2, err := os.Create("C:/Users/root/result.csv")
>>//defer f2.Close()
>>//writer := csv.NewWriter(f2)
>>//writer.Write(buf.String())
>> }
>>
>> // 500mb csv with head of the data looks like:
>> const data = 
>> `xcentre,ycentre,zcentre,xlength,ylength,zlength,fe_percent,fe_recovery,oxide,rescat,sg,mat_type_8,fillpc,mass_recovery,mass_recovery_percent,air,al2o3,cao,k2o,loi,mgo,mno,phos,sio2,tio2
>> 556960.000,6319980.000,-1100.000,40.000,20.000,12.000,-99.,66.,-99,4,2.8490,2,91,0.,0.,0,0.,0.,0.,0.,0.,0.,0.,0.,0.
>> 557000.000,6319980.000,-1100.000,40.000,20.000,12.000,-99.,66.,-99,4,2.8490,2,91,0.,0.,0,0.,0.,0.,0.,0.,0.,0.,0.,0.
>> 556960.000,632.000,-1100.000,40.000,20.000,12.000,-99.,66.,-99,4,2.8490,2,91,0.,0.,0,0.,0.,0.,0.,0.,0.,0.,0.,0.
>> 557000.000,632.000,-1100.000,40.000,20.000,12.000,-99.,66.,-99,4,2.8490,2,91,0.,0.,0,0.,0.,0.,0.,0.,0.,0.,0.,0.0

[go-nuts] Custom serializer issue

[Fabrice Vaillant]

Hi

For a custom project, I'm trying to write a serializer for a trie 
datastructure. I was using gob but it is slow due to it's use of 
reflect, and I wanted to learn something new.


The struct is mainly composed of nodes:

// Node implements a node of the tree
type Node struct {
// rw is a RWMutex, can be hold by either
// 1 writer or many reader
rw sync.RWMutex
// Sons and Radix holds information about the "descendant" of this node
Sons  []*Node
Radix []string
// Refs hold information about the word ending at this node
Refs []Ref
}

The refs slice is data about the node itself (my trie is used as an 
index). Basically a list of id and a list of float64.


Sons is a slice of pointer to other nodes, and Radix is a slice of 
strings which are the radix matching each of the sons, both slice have 
the same length.


The code relevant to the serialization is here 
https://github.com/Succo/rechercheInfoWeb/blob/customSerializer/encoder.go


I'm trying to encode the trie recursively. For each node I first encode 
the refs slice length, then all the int in order, and then all floats in 
order. Then I encode the length of the string slice, and then each 
string in order, by first writing the string length, and then 
[]byte(str). I finish by recursively encoding all nodes in the Sons slice.


When I try to decode the encoded file, it crashes at some point (always 
the same) because one of the length is absurdly high (either the slice 
of string itself or one of the string). Which I suppose mean it's 
reading data corresponding to something else. until this point all read 
data seems consistent as far as I could see.


I have helper function to encode and decode int, floats and []string, 
all seem to pass basic test. I tried to use a smaller trie and it was 
successfully encoded and decoded.  If I remove the part about encoding 
and decoding the slice of string it works, but obviously it's missing data.


I really have no idea what causes the corrupted, I imagine it's because 
at some point some data don't get committed but I can't see why.


Can anybody help me with this?

Thank you

Fabrice

--
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Code review/Advice - reading and editing a CSV

[peterGo]

Robbie,

Fixed some typos: https://play.golang.org/p/GGjoeOYfRN

Peter

On Friday, February 24, 2017 at 12:48:44 PM UTC-5, peterGo wrote:
>
> Robbie,
>
> It's hard to understand what you are doing.
>
> The header says that there is no grade and sg is not where you say it is. 
> Why are you using a channel? Why are you limiting the size of your input by 
> reading the entire file into memory? And so on.
>
> My best guess is that you want to do something like this:
>
> Go Playground: https://play.golang.org/p/4Aok43E6AC
>
> Peter
>
> On Friday, February 24, 2017 at 8:58:52 AM UTC-5, Robbie Wright wrote:
>>
>> Hi this is my first go program and I am looking for some feedback and 
>> help writing buffer to csv. To give you an idea of the structure of the 
>> data I have run head on the input file gives output seen at the bottom.
>>
>> package main
>>
>> import (
>>"bytes"
>>"encoding/csv"
>>"fmt"
>>"io"
>>"log"
>>"strconv"
>>"strings"
>>"bufio"
>>"io/ioutil"
>> )
>>
>> // Calculate economic block value given a 40x20x12 block of density sg and 
>> grade
>> func calculateE(sg float64, grade float64) float64 {
>>return 40 * 20 * 12 * sg * grade * 80 / 10 * 0.66
>> }
>>
>> // Credit: Stack Overflow
>> func processCSV(rc io.Reader) (ch chan []string) {
>>ch = make(chan []string, 10)
>>go func() {
>>   r := csv.NewReader(rc)
>>   if _, err := r.Read(); err != nil { //read header
>>  log.Fatal(err)
>>   }
>>   defer close(ch)
>>   for {
>>  rec, err := r.Read()
>>
>>  if err != nil {
>> if err == io.EOF {
>>break
>> }
>> log.Fatal(err)
>>
>>  }
>>  ch <- rec
>>   }
>>}()
>>return
>> }
>>
>> // Credit: Stack Overflow
>> func Readln(r *bufio.Reader) (string, error) {
>>var (
>>   isPrefix bool  = true
>>   err  error = nil
>>   line, ln []byte
>>)
>>for isPrefix && err == nil {
>>   line, isPrefix, err = r.ReadLine()
>>   ln = append(ln, line...)
>>}
>>return string(ln), err
>> }
>>
>> func main() {
>>var buf bytes.Buffer
>>
>>w := csv.NewWriter(&buf)
>>file := "C:/ORIG.csv"
>>dat, _ := ioutil.ReadFile(file)
>>f := string(dat)
>>
>>for rec := range processCSV(strings.NewReader(f)) {
>>   sg, err := strconv.ParseFloat(rec[17], 64)
>>   grade, err := strconv.ParseFloat(rec[13], 64)
>>
>>   if grade < 0 {
>>  grade = 0
>>   } else {
>>  grade = 0.1*grade
>>   }
>>   if sg < 0 {
>>  sg = 0
>>   }
>>   fmt.Println(grade)
>>   // get economic block value
>>   e := calculateE(sg, grade)
>>   gradeStr := strconv.FormatFloat(grade, 'f', 8, 64)
>>   sgStr := strconv.FormatFloat(sg, 'f', 8, 64)
>>   eStr := strconv.FormatFloat(e, 'f', 8, 64)
>>   rec := append(rec[0:4], eStr, sgStr, gradeStr)
>>   if err = w.Write(rec); err != nil {
>>  log.Fatal(err)
>>   }
>>}
>>w.Flush()
>>if err := w.Error(); err != nil {
>>   log.Fatal(err)
>>}
>>//fmt.Println(buf.String())
>>//f2, err := os.Create("C:/Users/root/result.csv")
>>//defer f2.Close()
>>//writer := csv.NewWriter(f2)
>>//writer.Write(buf.String())
>> }
>>
>> // 500mb csv with head of the data looks like:
>> const data = 
>> `xcentre,ycentre,zcentre,xlength,ylength,zlength,fe_percent,fe_recovery,oxide,rescat,sg,mat_type_8,fillpc,mass_recovery,mass_recovery_percent,air,al2o3,cao,k2o,loi,mgo,mno,phos,sio2,tio2
>> 556960.000,6319980.000,-1100.000,40.000,20.000,12.000,-99.,66.,-99,4,2.8490,2,91,0.,0.,0,0.,0.,0.,0.,0.,0.,0.,0.,0.
>> 557000.000,6319980.000,-1100.000,40.000,20.000,12.000,-99.,66.,-99,4,2.8490,2,91,0.,0.,0,0.,0.,0.,0.,0.,0.,0.,0.,0.
>> 556960.000,632.000,-1100.000,40.000,20.000,12.000,-99.,66.,-99,4,2.8490,2,91,0.,0.,0,0.,0.,0.,0.,0.,0.,0.,0.,0.
>> 557000.000,632.000,-1100.000,40.000,20.000,12.000,-99.,66.,-99,4,2.8490,2,91,0.,0.,0,0.,0.,0.,0.,0.,0.,0.,0.

Re: [go-nuts] Support for the race detector on ARM

[Owen Waller]

Hi Dimitry,

> 
> If there is somebody who is willing to work on this, then yes. Otherwise, 
> maybe.


I have created https://github.com/golang/go/issues/19273 in order to
track this. 

> > > > ThreadSanitizer does not work on any 32-bit platforms. It assumes
that
> > it can reserve huge continuous chunks of address space for various
> > things.
> > 
> > 
> 
> Tsan reserves 4X for shadow memory (where X is amount of memory where
> user data resides), 0.5X for another shadow memory, 1TB for heap and
> 128GB for thread info. You can see the details in
> llvm/projects/compiler-rt/lib/tsan/rtl/tsan_platform.h.
> 

Those ranges are well beyond the 32-bit address space. But, having had
a quick look at the tsan_platform.h file. It looks like the file
contains memory layouts for hardware with 39, 42, 44, 46 and 64 bit
address spaces, at least for C/C++ programs if not Go.

Given that these ranges are just defined as constants in the code is
there a fundamental reason why these ranges could not be shrunk to fit
into a 32 bit address space? Does the tread sanitizer require some
(assembly) instructions that are only found in 64 bit hardware for
example?

> 
Maybe it's possible to stub the i2c part? That would make testing much
simpler regardless of tsan.


In this specific case, maybe, yes. It's not my i2c drier so just how
much work and how much change would be required I'd need to
investigate.

Owen

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Support for the race detector on ARM

['Dmitry Vyukov' via golang-nuts]

On Fri, Feb 24, 2017 at 9:24 PM, Owen Waller  wrote:
> Hi Dimitry,
>
>
> If there is somebody who is willing to work on this, then yes. Otherwise,
> maybe.
>
>
> I have created https://github.com/golang/go/issues/19273 in order to track
> this.
>
> ThreadSanitizer does not work on any 32-bit platforms. It assumes that
> it can reserve huge continuous chunks of address space for various
> things.
>
>
>
>
> Tsan reserves 4X for shadow memory (where X is amount of memory where
> user data resides), 0.5X for another shadow memory, 1TB for heap and
> 128GB for thread info. You can see the details in
> llvm/projects/compiler-rt/lib/tsan/rtl/tsan_platform.h.
>
>
> Those ranges are well beyond the 32-bit address space. But, having had a
> quick look at the tsan_platform.h file. It looks like the file contains
> memory layouts for hardware with 39, 42, 44, 46 and 64 bit address spaces,
> at least for C/C++ programs if not Go.
>
> Given that these ranges are just defined as constants in the code is there a
> fundamental reason why these ranges could not be shrunk to fit into a 32 bit
> address space? Does the tread sanitizer require some (assembly) instructions
> that are only found in 64 bit hardware for example?

No fundamental reason.
Tsan requires 64-bit atomic loads and stores.


> Maybe it's possible to stub the i2c part? That would make testing much
> simpler regardless of tsan.
>
>
> In this specific case, maybe, yes. It's not my i2c drier so just how much
> work and how much change would be required I'd need to investigate.
>
> Owen

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Support for the race detector on ARM

[Owen Waller]

Hi Dimitry,

> > > > ThreadSanitizer does not work on any 32-bit platforms. It assumes
that
> > it can reserve huge continuous chunks of address space for various
> > things.
> > 
> > 
> > 
> > 
> > > > Tsan reserves 4X for shadow memory (where X is amount of memory
where
> > > > user data resides), 0.5X for another shadow memory, 1TB for heap
and
> > 128GB for thread info. You can see the details in
> > llvm/projects/compiler-rt/lib/tsan/rtl/tsan_platform.h.
> > 
> > 
> > > > Those ranges are well beyond the 32-bit address space. But, having
had a
> > > > quick look at the tsan_platform.h file. It looks like the file
contains
> > > > memory layouts for hardware with 39, 42, 44, 46 and 64 bit address
spaces,
> > at least for C/C++ programs if not Go.
> > 
> > > > Given that these ranges are just defined as constants in the code
is there a
> > > > fundamental reason why these ranges could not be shrunk to fit into
a 32 bit
> > > > address space? Does the tread sanitizer require some (assembly)
instructions
> > that are only found in 64 bit hardware for example?
> 
> No fundamental reason.
> Tsan requires 64-bit atomic loads and stores.

This leads me to ask two things.

Why are 64-bit atomic load and stores required? To take an example.
ARM6 cores have have load stores for a very long time[1]. But being a
32-bit core that's usually attached to a 32 bit memory bus, the
instructions are 32-bit. So is it just that an atomic pair of load and
store operations are required? I am of course assuming that gcc or
clang on these platforms can make use of these instructions.

If all that is needed are atomic loads and stores then that leads to
the second question. Why hasn't a 32-bit port (with a reduced memory
map) of the thread sanitizer already appeared? There are lots of other
32 bit cores with atomic load sores - MIPS32, PPC, SPARC etc...

At the minute I feel I am missing something _very_ important. Otherwise
this looks like it should be a solved problem. 

[1] http://infocenter.arm.com/help/topic/com.arm.doc.dht0008a/DHT0008A_
arm_synchronization_primitives.pdf Section 1.2

Owen

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] first try with templates

['simon place' via golang-nuts]

https://play.golang.org/p/NGU4kstcT-

just trying to put one var into one template and i'm failing!, see above, 
i've tried the docs, googling and randomly guessing. please someone put me 
out of my misery.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] first try with templates

[Ian Davis]

On Fri, 24 Feb 2017, at 11:40 PM, 'simon place' via golang-nuts wrote:

> https://play.golang.org/p/NGU4kstcT-

> 

> just trying to put one var into one template and i'm failing!, see
> above, i've tried the docs, googling and randomly guessing. please
> someone put me out of my misery.


Hopefully this variation will help you get started:



https://play.golang.org/p/npB-83GnXV




-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Go Language tests

[bryontrott]

Hello,
I am testing Go language on Nano Server.

Are there any Acceptance tests or tests the Go team has created to tests Go 
builds?

If possible I would like to run these tests on Nano Server.

Thanks,
Bryon Trott

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Code review/Advice - reading and editing a CSV

[Mandolyte]

I deal with CSV files a lot. I have made most available here:

https://github.com/mandolyte/csv-utils

Hope it helps,
Cecil

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] first try with templates

['simon place' via golang-nuts]

thanks, but

the handed-in data, on each execute, doesn't seem like a good fit for a 
global var, and there will always be the actual one-off 'context' data i 
will need to push in, so its not available.

so if you really can't do this

option1:  i could use a function, but that seemed like overkill.

option2: i could keep rebuilding the template with the var directly 
inserted into the template definition, but that kind of defeats the point 
of templates, i may as well do the whole thing that way.

option3: i could embed the context data in a struct with the global stuff 
as pointer members.


On Saturday, 25 February 2017 00:34:38 UTC, Ian Davis wrote:
>
>
> On Fri, 24 Feb 2017, at 11:40 PM, 'simon place' via golang-nuts wrote:
>
> https://play.golang.org/p/NGU4kstcT-
>
> just trying to put one var into one template and i'm failing!, see above, 
> i've tried the docs, googling and randomly guessing. please someone put me 
> out of my misery.
>
>
> Hopefully this variation will help you get started:
>
> https://play.golang.org/p/npB-83GnXV
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Go Language tests

[Ian Lance Taylor]

On Fri, Feb 24, 2017 at 4:51 PM,   wrote:
>
> I am testing Go language on Nano Server.
>
> Are there any Acceptance tests or tests the Go team has created to tests Go
> builds?
>
> If possible I would like to run these tests on Nano Server.

Are you looking for any specific kind of tests?  There are all sorts
of tests in the Go source tree, run by src/run.bash (or src/all.bash,
which invokes run.bash).

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: first try with templates

['simon place' via golang-nuts]

well option3 seems to work

https://play.golang.org/p/XMih6ocmGY

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Minimizing golang docker images

[Steve Roth]

Hello, Sankar,

The minimal Docker image for a Go webapp contains absolutely nothing in its 
filesystem other than the binary itself.  Here's an example Dockerfile for 
such an image:
FROM scratch
COPY appname /
EXPOSE 8000
CMD ["/appname"]


However, to build the image, you have to have a full Go compiler toolchain, 
which means you need a full OS distro under it.  This is (relatively) huge. 
 It doesn't have to be in a separate Docker image; it can be run natively 
outside of Docker.  But you can't have the build in the Dockerfile for the 
image you're building, because that would require you to have the full OS 
distro and Go toolchain in that image, and then it wouldn't be small.

Regards,
Steve



On Friday, February 24, 2017 at 7:57:45 AM UTC-8, Sankar wrote:
>
> Hi
>
> I have a bunch of services (all sources in a single git project) which I 
> want to dockerize and deploy in kubernetes (and run CI too preferably via 
> minikube).
>
> I have seen that it is possible to generate small docker images for golang 
> HTTP servers by building them separately outside the Dockerfile that will 
> be used for deployment, such as: 
> http://samuraiprogrammer.net/blog/way-to-reduce-docker-image-size-with-tarball/
>
> But the above approach of two Dockerfiles for each service, makes it very 
> difficult to manage the sources. I wanted to know if there are any 
> hacks/best-practices for reducing the docker image sizes for golang 
> microservices but without creating two Dockerfiles for each service. Any 
> help ?
>
> Thanks.
>
> Sankar
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Expressing test dependence

[kpratt]

What I want is a way to say if Test_A fails then don't run Test_B because 
it's guaranteed to fail and that just creates noise in the output.


So instinctively I'd like something like annotations for this. 

// @DependsOn [ "Test_A" ]
func Test_B(t *testing.T) {
  .test stuff
}

https://github.com/MarcGrol/golangAnnotations

If I use something like magic comments will it be possible for `go test` to 
see these? if not, What would be the recommended approach to add this 
functionality to `go test` ?


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Expressing test dependence

[Dave Cheney]

Within the testing package you choice is t.Skip and some set of package level 
variables.  

Maybe the more involved testing frameworks like convoy or gocheck offer more 
final version of t.Fatal.

However, from the situation you've presented it feels to me that your solving 
the wrong problem. If a single change causes all your tests to go from working 
to not working you probably want to back that out, not add features to the 
testing package to make those tests failures less verbose. 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] is it a waste of time to study old synchronization algorithms?

[Marwan abdel moneim]

Algorithms like Peterson’s algorithm 
 and others 
,
 
which i think was developed before languages provide Mutexes or Channels 
(or something similar)

If i want to be able to develop complex concurrent programs, should i study 
those algorithms? or is it a waste of time?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.