Re: [go-nuts] Huge map[string]*Object and GC where *Object is from sync.Pool

[Dan Kortschak]

Please re-read what I was answering: "I know what a uintptr is but what
would you put in it if not a pointer to another object?".

If you have done that and still believe what I wrote is not true,
please explain what I have got wrong.

I am afraid I do not understand at all what you think is not true.

On Fri, 2018-09-28 at 22:26 -0500, Robert Engels wrote:
> That is just not true. Delete an item while who? some? hold an index
> reference. You have no way of knowing. That is the crux of memory
> management. If everything is static it is far simpler but most more
> than trivial applications need dynamic data structures. 
> 
> Sent from my iPhone
> 
> > 
> > On Sep 28, 2018, at 10:14 PM, Dan Kortschak  > .edu.au> wrote:
> > 
> > You put in it a number that has the same bit pattern as a pointer
> > to a
> > value. There is nothing more to it than that. It does not refer to
> > anything. They exist so you can do uncomfortable pointer
> > arithmetic.
> > 
> > > 
> > > On Fri, 2018-09-28 at 15:14 +0200, Henrik Johansson wrote:
> > > I know what a uintptr is but what would you put in it if not a
> > > pointer to
> > > another object?
> > > Isn't this very analogous to what you said: "a weak hashmap uses
> > > weak
> > > references to refer to the contained objects so that they will be
> > > collected
> > > if nothing else refers to them".
> > > 
> > > Maybe I am missing something. I never meant to imply that they
> > > worked
> > > the
> > > same way _internally_ but at a conceptual level.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Huge map[string]*Object and GC where *Object is from sync.Pool

[Peter Mogensen]

On 09/28/2018 03:14 PM, Peter Mogensen wrote:
> In other words.. using a map as index into another datastructure without
> the map values being considered by the garbage collector as references.

I just quickly read about the java.util.WeakHashMap and it says that the
keys are the weak references - not the values.
So it's not exactly what I was thinking of. I wanted the values as weak
references.
But as Ian Lance Taylor pointed out, the unsafe packages doesn't allow
this, since:
"Conversion of a uintptr back to Pointer is not valid in general. "

... which would be needed to use the map as an index.

I guess that's basically the summary of this thread. Thanks for your input.

/Peter

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Unsupported File Format error while building for linux-amd64 on mac os

[roger peppe]

I've also found that multipass (https://github.com/CanonicalLtd/multipass)
works well for running Ubuntu on different platforms FWIW.

On Fri, 28 Sep 2018, 9:28 pm Tamás Gulácsi,  wrote:

> 2018. szeptember 28., péntek 15:59:14 UTC+2 időpontban Ankit Gupta a
> következőt írta:
>>
>> Thanks Ian for responding. Can you point me as to how to get the cross
>> compiler. I am farely new to Mac.
>>
>> On Fri, 28 Sep 2018, 19:23 Ian Lance Taylor,  wrote:
>>
>>> On Fri, Sep 28, 2018 at 2:08 AM, Ankit Gupta
>>>  wrote:
>>> >
>>> > I am working with confluent-kafka-go library
>>> > (https://github.com/confluentinc/confluent-kafka-go) which builds
>>> fine on
>>> > the mac machine (mac OS 10.13.6) along with the code files I wrote. In
>>> order
>>> > to deploy it on Linux server (Ubuntu 64 bit), I try this -
>>> >
>>> > $> CGO_ENABLED=1 GOOS=linux GOARCH=amd64 go build program.go
>>> >
>>> >
>>> > I get below linking error -
>>> >
>>> >
>>> > /usr/local/go/pkg/tool/darwin_amd64/link: running clang failed: exit
>>> status
>>> > 1
>>> >
>>> > ld: warning: ignoring file
>>> >
>>> /var/folders/fy/9ph54yjs6cq1kyxgs6cc9rvjvs6t0m/T/go-link-060577916/go.o,
>>> > file was built for unsupported file format ( 0x7F 0x45 0x4C 0x46 0x02
>>> 0x01
>>> > 0x01 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 ) which is not the
>>> > architecture being linked (x86_64):
>>> > /var/folders/fy/9ph54yjs6cq1kyxgs6cc9rvjvs6t0m/T/go-link-060577916/go.o
>>> >
>>> > Undefined symbols for architecture x86_64:
>>> >
>>> >   "__cgo_topofstack", referenced from:
>>> >
>>> >   __cgo_f2fa82ea8f11_Cfunc_rd_kafka_conf_new in 03.o
>>> >
>>> >   __cgo_f2fa82ea8f11_Cfunc_rd_kafka_conf_set in 03.o
>>> >
>>> >   __cgo_f2fa82ea8f11_Cfunc_rd_kafka_topic_conf_new in 03.o
>>> >
>>> >   __cgo_f2fa82ea8f11_Cfunc_rd_kafka_topic_conf_set in 03.o
>>> >
>>> >   __cgo_f2fa82ea8f11_Cfunc__c_rdkafka_topic_partition_list_entry in
>>> > 04.o
>>> >
>>> >   __cgo_f2fa82ea8f11_Cfunc_rd_kafka_assign in 04.o
>>> >
>>> >   __cgo_f2fa82ea8f11_Cfunc_rd_kafka_assignment in 04.o
>>> >
>>> >   ...
>>> >
>>> >   "_main", referenced from:
>>> >
>>> >  implicit entry/start for main executable
>>> >
>>> >  (maybe you meant:
>>> __cgo_f2fa82ea8f11_Cfunc_rd_kafka_queue_get_main)
>>> >
>>> > ld: symbol(s) not found for architecture x86_64
>>> >
>>> > clang: error: linker command failed with exit code 1 (use -v to see
>>> > invocation)
>>> >
>>> >
>>> > It points to /var/folders which I am given to understand is a temp
>>> location
>>> > in mac.
>>> >
>>> >
>>> > Any ideas on how to fix it?
>>>
>>> When cross-compiling a Go program that uses cgo, you need to be using
>>> a C cross-compiler.  It looks like you are using the native Darwin C
>>> compiler.  You need a cross-compiler from Darwin to GNU/Linux.
>>>
>>> Ian
>>>
>>
> Native compilation is much easier.
> If you have Docker installed, https://github.com/karalabe/xgo can help a
> lot!
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Are C allocated memory completely isolated to Go memory in any cases?

[Tamás Gulácsi]

You should LockOSThread and process every pango-facing call in that goroutine.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Are C allocated memory completely isolated to Go memory in any cases?

[changkun]

As discussed before, the every pango call is called in a non-Go newly 
created thread by C. 
runtime.LockOSThread doesn't work.

On Saturday, September 29, 2018 at 1:24:21 PM UTC+2, Tamás Gulácsi wrote:
>
> You should LockOSThread and process every pango-facing call in that 
> goroutine.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Are C allocated memory completely isolated to Go memory in any cases?

[Tamás Gulácsi]

Yes, but is that a one and only goroutine? 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Are C allocated memory completely isolated to Go memory in any cases?

[changkun]

On Saturday, September 29, 2018 at 2:08:03 PM UTC+2, Tamás Gulácsi wrote:
>
> Yes, but is that a one and only goroutine?


No. The cgo call is opened for every new incoming user request.

Let's summarize:

- Every network request creates a goroutine without response 
  processing result to a user of that goroutine.
- The goroutine instantly proceeds a cgo call and the cgo call 
  creates a non-Go thread, then a Pango call pango_font_map_get_default()
  is involved in the non-Go thread.
- According to pango_font_map_get_default(), it holds a static thread-safe 
variable. 
- The original pure C code is able to proceed execution without involving 
Go.
  But stuck at the Pango call when involving cgo
- runtime.LockOSThread doesn't work:
go func() {
runtime.LockOSThread()
handleConnection(timeout)
runtime.UnlockOSThread()
}()

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Are C allocated memory completely isolated to Go memory in any cases?

[Robert Engels]

Your description makes no sense. Why are you creating another C thread, you 
should just lock the OS thread the Go routine is on. Otherwise if you create 
another C thread, in the cgo call you need to block that thread or somehow 
communicate back into Go some other way - otherwise what is the point?

> On Sep 29, 2018, at 7:31 AM, changkun  wrote:
> 
> 
>> On Saturday, September 29, 2018 at 2:08:03 PM UTC+2, Tamás Gulácsi wrote:
>> Yes, but is that a one and only goroutine?
> 
> No. The cgo call is opened for every new incoming user request.
> 
> Let's summarize:
> 
> - Every network request creates a goroutine without response 
>   processing result to a user of that goroutine.
> - The goroutine instantly proceeds a cgo call and the cgo call 
>   creates a non-Go thread, then a Pango call pango_font_map_get_default()
>   is involved in the non-Go thread.
> - According to pango_font_map_get_default(), it holds a static thread-safe 
> variable. 
> - The original pure C code is able to proceed execution without involving Go.
>   But stuck at the Pango call when involving cgo
> - runtime.LockOSThread doesn't work:
> go func() {
> runtime.LockOSThread()
> handleConnection(timeout)
> runtime.UnlockOSThread()
> }()
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] go mod tidy pulls in too much

[Scott Cotton]

Hi all,

I think this is related to https://github.com/golang/go/issues/26913 which 
is closed but still has commentary that seems unresolved after it was 
closed .

Scott


On Friday, 28 September 2018 20:11:29 UTC+2, Harmen wrote:
>
> On Fri, Sep 28, 2018 at 10:19:50AM -0700, thepud...@gmail.com 
>  wrote: 
> > Hi Harmen, 
> > 
> > And my first sentence might not have been clear. When I said "even in 
> your 
> > current situation, 'go build' is still pulling in exactly what it 
> needs", I 
> > was trying to reference the actual compilation process. 
> > 
> > In other words, I was just trying to make it clear that even if you have 
> > "extra" dependencies appearing in your go.mod file, the resulting binary 
> > produced by 'go build' does not have anything extra or any unused 
> > dependencies. 
>
> Hi thepudds, 
> thanks for your anwers. 
>
> It doesn't hurt per-se, but all those unused modules will be vendored in 
> the 
> repo (or the CI will have to download them every time). Maybe the consul 
> repo 
> is a particularly unlucky repo, since the /api package is small compared 
> to the 
> rest of the repo, but still. 
>
> As for the argument that it's for test reproducibility, I do not follow 
> that. 
> Everything to test /api is there, and the stuff I don't import doesn't 
> need to 
> be tested in the first place. 
>
> > In any event, I wanted to share at least my personal understanding, but 
> of 
> > course happy to learn more... 
>
> I'm trying to do the same :) 
> Thanks! 
>
> > 
> > --thepudds 
> > 
> > 
> > On Friday, September 28, 2018 at 1:00:17 PM UTC-4, thepud...@gmail.com 
> > wrote: 
> > > 
> > >  > "So if consul adds a go.mod file in the root, then `mod tidy` will 
> > > suddenly 
> > > behave as I would expect (i.e. not pull in unused dependencies)? " 
> > > 
> > > Hi Harmen, 
> > > 
> > > Just in case this isn't already clear-- note that even in your current 
> > > situation, 'go build' is still pulling in exactly what it needs (and 
> not 
> > > pulling in unused dependencies). 
> > > 
> > > I think there is not a significant harm in your go.mod containing 
> these 
> > > indirect dependencies (aside from of course seeing that longer list, 
> etc.). 
> > > 
> > > And there is some benefit -- this behavior is part of what provides 
> for 
> > > 100% reproducible builds and tests. The modules system records precise 
> > > dependency version information, and in your case, that precise 
> dependency 
> > > version information for some of your indirect dependencies is being 
> > > recorded in your go.mod (given that it is not yet recorded in the 
> > > non-existent go.mod of some of your direct dependencies).   
> > > 
> > > As an example, this behavior helps make sure that `go test all` is 
> 100% 
> > > reproducible for you (where `go test all` runs tests for both your 
> module, 
> > > your direct dependencies, and your indirect dependencies, which is 
> valuable 
> > > as one way of validating that the currently selected packages versions 
> are 
> > > compatible -- the number of possible version combinations is 
> exponential in 
> > > the number of modules, so in general you cannot expect your 
> dependencies to 
> > > have tested against all possible combinations of *their* 
> dependencies). 
> > > 
> > > --thepudds 
> > > 
> > > On Friday, September 28, 2018 at 12:48:20 PM UTC-4, Harmen wrote: 
> > >> 
> > >> On Fri, Sep 28, 2018 at 04:48:32PM +0100, Paul Jolly wrote: 
> > >> > Hi Harmen 
> > >> > 
> > >> > I described the problem on 
> https://github.com/golang/go/issues/27920, 
> > >> which 
> > >> > > got 
> > >> > > closed within three minutes as being "documented", and "works as 
> > >> > > expected" (which I assume also means "works as intended"). 
> > >> > > Is this really the intented behaviour? It seems unexpected to me. 
> Or 
> > >> > > should I 
> > >> > > simply stay away from `go mod tidy`? 
> > >> > > 
> > >> > 
> > >> > I replied to your issue earlier. But could arguably have been 
> slightly 
> > >> more 
> > >> > detailed in my response beyond simply linking to 
> > >> > 
> > >> 
> https://github.com/golang/go/wiki/Modules#why-does-go-mod-tidy-record-indirect-and-test-dependencies-in-my-gomod
>  
> > >> > 
> > >> > As you note, github.com/hashicorp/consul/api is a package. It is a 
> > >> package 
> > >> > in the module github.com/hashicorp/consul. Despite there being no 
> > >> go.mod in 
> > >> > github.com/hashicorp/consul, the go tool simulates it as a module. 
> > >> > 
> > >> > Given there are no go.mod files in any subdirectories below 
> > >> > github.com/hashicorp/consul, then github.com/hashicorp/consul is 
> the 
> > >> module 
> > >> > for all the packages github.com/hashicorp/consul/... 
> > >> > 
> > >> > Hence go mod tidy (per the link in my response to your issue) is 
> > >> pulling 
> > >> > in the transitive dependencies needed for tests in 
> > >> > github.com/hashicorp/consul/... The reason you see all of these in 
> > >> your 
> > >> > go.

Re: [go-nuts] Are C allocated memory completely isolated to Go memory in any cases?

[Tamás Gulácsi]

Create One (1) goroutine with LockOSThread and let it get the todo and response 
channel from a channel and send the response back when ready, and make all 
other goroutines communicate with this one and only one pango-dedicated 
goroutine. Which is locked to a thread.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Are C allocated memory completely isolated to Go memory in any cases?

[changkun]

On Saturday, September 29, 2018 at 3:46:41 PM UTC+2, Robert Engels wrote:
>
> Your description makes no sense. Why are you creating another C thread, 
> you should just lock the OS thread the Go routine is on. Otherwise if you 
> create another C thread, in the cgo call you need to block that thread or 
> somehow communicate back into Go some other way - otherwise what is the 
> point?
>

As I mentioned on the top floor, I am migrating a very old pure C project 
to Go, and 
eventually become a pure Go project. the original project was linked by 
different components, and I am replacing them one by one...

I know it doesn't make sense (apparently, before creating a non-Go thread, 
there are some 
internal preprocessing they are irrelevant to Pango)  but to archive what 
you describe (in one goroutine) 
will consume loads of time for reading, understanding, and implementation, 
time is limited.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Are C allocated memory completely isolated to Go memory in any cases?

[changkun]

On Saturday, September 29, 2018 at 4:03:42 PM UTC+2, Tamás Gulácsi wrote:
>
> Create One (1) goroutine with LockOSThread and let it get the todo and 
> response channel from a channel and send the response back when ready, and 
> make all other goroutines communicate with this one and only one 
> pango-dedicated goroutine. Which is locked to a thread.
>

Can't do that at the moment, reason in above. 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] go mod tidy pulls in too much

[t hepudds]

Hi Scott,

Regarding your comment, a related issue (which is still open) is this one:

   #26955: “cmd/go: provide straightforward way to see non-test dependencies”
  https://github.com/golang/go/issues/26955

--thepudds 

Terseness courtesy of my mobile device

> On Sep 29, 2018, at 9:53 AM, Scott Cotton  wrote:
> 
> Hi all,
> 
> I think this is related to https://github.com/golang/go/issues/26913 which is 
> closed but still has commentary that seems unresolved after it was closed .
> 
> Scott
> 
> 
>> On Friday, 28 September 2018 20:11:29 UTC+2, Harmen wrote:
>> On Fri, Sep 28, 2018 at 10:19:50AM -0700, thepud...@gmail.com wrote: 
>> > Hi Harmen, 
>> > 
>> > And my first sentence might not have been clear. When I said "even in your 
>> > current situation, 'go build' is still pulling in exactly what it needs", 
>> > I 
>> > was trying to reference the actual compilation process. 
>> > 
>> > In other words, I was just trying to make it clear that even if you have 
>> > "extra" dependencies appearing in your go.mod file, the resulting binary 
>> > produced by 'go build' does not have anything extra or any unused 
>> > dependencies. 
>> 
>> Hi thepudds, 
>> thanks for your anwers. 
>> 
>> It doesn't hurt per-se, but all those unused modules will be vendored in the 
>> repo (or the CI will have to download them every time). Maybe the consul 
>> repo 
>> is a particularly unlucky repo, since the /api package is small compared to 
>> the 
>> rest of the repo, but still. 
>> 
>> As for the argument that it's for test reproducibility, I do not follow 
>> that. 
>> Everything to test /api is there, and the stuff I don't import doesn't need 
>> to 
>> be tested in the first place. 
>> 
>> > In any event, I wanted to share at least my personal understanding, but of 
>> > course happy to learn more... 
>> 
>> I'm trying to do the same :) 
>> Thanks! 
>> 
>> > 
>> > --thepudds 
>> > 
>> > 
>> > On Friday, September 28, 2018 at 1:00:17 PM UTC-4, thepud...@gmail.com 
>> > wrote: 
>> > > 
>> > >  > "So if consul adds a go.mod file in the root, then `mod tidy` will 
>> > > suddenly 
>> > > behave as I would expect (i.e. not pull in unused dependencies)? " 
>> > > 
>> > > Hi Harmen, 
>> > > 
>> > > Just in case this isn't already clear-- note that even in your current 
>> > > situation, 'go build' is still pulling in exactly what it needs (and not 
>> > > pulling in unused dependencies). 
>> > > 
>> > > I think there is not a significant harm in your go.mod containing these 
>> > > indirect dependencies (aside from of course seeing that longer list, 
>> > > etc.). 
>> > > 
>> > > And there is some benefit -- this behavior is part of what provides for 
>> > > 100% reproducible builds and tests. The modules system records precise 
>> > > dependency version information, and in your case, that precise 
>> > > dependency 
>> > > version information for some of your indirect dependencies is being 
>> > > recorded in your go.mod (given that it is not yet recorded in the 
>> > > non-existent go.mod of some of your direct dependencies).   
>> > > 
>> > > As an example, this behavior helps make sure that `go test all` is 100% 
>> > > reproducible for you (where `go test all` runs tests for both your 
>> > > module, 
>> > > your direct dependencies, and your indirect dependencies, which is 
>> > > valuable 
>> > > as one way of validating that the currently selected packages versions 
>> > > are 
>> > > compatible -- the number of possible version combinations is exponential 
>> > > in 
>> > > the number of modules, so in general you cannot expect your dependencies 
>> > > to 
>> > > have tested against all possible combinations of *their* dependencies). 
>> > > 
>> > > --thepudds 
>> > > 
>> > > On Friday, September 28, 2018 at 12:48:20 PM UTC-4, Harmen wrote: 
>> > >> 
>> > >> On Fri, Sep 28, 2018 at 04:48:32PM +0100, Paul Jolly wrote: 
>> > >> > Hi Harmen 
>> > >> > 
>> > >> > I described the problem on https://github.com/golang/go/issues/27920, 
>> > >> which 
>> > >> > > got 
>> > >> > > closed within three minutes as being "documented", and "works as 
>> > >> > > expected" (which I assume also means "works as intended"). 
>> > >> > > Is this really the intented behaviour? It seems unexpected to me. 
>> > >> > > Or 
>> > >> > > should I 
>> > >> > > simply stay away from `go mod tidy`? 
>> > >> > > 
>> > >> > 
>> > >> > I replied to your issue earlier. But could arguably have been 
>> > >> > slightly 
>> > >> more 
>> > >> > detailed in my response beyond simply linking to 
>> > >> > 
>> > >> https://github.com/golang/go/wiki/Modules#why-does-go-mod-tidy-record-indirect-and-test-dependencies-in-my-gomod
>> > >>  
>> > >> > 
>> > >> > As you note, github.com/hashicorp/consul/api is a package. It is a 
>> > >> package 
>> > >> > in the module github.com/hashicorp/consul. Despite there being no 
>> > >> go.mod in 
>> > >> > github.com/hashicorp/consul, the go tool simulates it as a module. 
>> > >>

Re: [go-nuts] go mod tidy pulls in too much

[thepudds1460]

Hi Scott,

Regarding your comment, a related issue (which is still open) is this one:

   #26955: “cmd/go: provide straightforward way to see non-test dependencies”
  https://github.com/golang/go/issues/26955

—thepudds 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Are C allocated memory completely isolated to Go memory in any cases?

[Tamás Gulácsi]

2018. szeptember 29., szombat 17:10:25 UTC+2 időpontban changkun a 
következőt írta:
>
>
>
> On Saturday, September 29, 2018 at 4:03:42 PM UTC+2, Tamás Gulácsi wrote:
>>
>> Create One (1) goroutine with LockOSThread and let it get the todo and 
>> response channel from a channel and send the response back when ready, and 
>> make all other goroutines communicate with this one and only one 
>> pango-dedicated goroutine. Which is locked to a thread.
>>
>
> Can't do that at the moment, reason in above. 
>

I tought something like: 

type todo struct {
  FuncToCall string
  Args []interface{}
  Response ch chan<- response
}
type response struct {
  ResultCode int
  Err error
}

var todoCh = make(chan todo)

go func(ch <- todo) {
runtime.LockOSThread()
defer runtime.UnlockOSThread()
for t := range ch {
  switch t.FuncToCall {
  ...
  }
  t.Response <- response{Err:nil, ResultCode:0}
}
}()

func handleConnection(timeout) error {
ch := make(chan response, 1)
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
select {
case <-ctx.Done():
  // timeout
  return ctx.Err()
case todoCh <- todo{FuncToCall:..., Response:ch}:
  select {
case <-ctx.Done():
  return ctx.Err()
case resp := <-ch:
  // success!
  return nil
  }
}
}
 

I don't feel that abstracting out the C (Pango) calls into an interface 
with maybe several implementations (C or Go) would be such a great 
undertaking,
but would ease up writing correct code.

And anyway you have to call that Pango thing from one thread.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Are C allocated memory completely isolated to Go memory in any cases?

[Wojciech S. Czarnecki]

On Fri, 28 Sep 2018 22:27:04 -0700 (PDT)
changkun  wrote:

> Not really, the original C code calls pango in every new created thread.
> [...]
> the Pango call is always called in a newly created thread.

IIUC its your old C code spinning a new thread for each Pango
call (?). If so, you should not bridge such code directly with Go. You need
to make a wrapper on the C side (in C), init it from the **single** goroutine
you pin to a single (go side) thread then communicate only via said
wrapper calls.

[Go] <-go-channels-> [comm goroutine] <-cgo-call(s)-> [wrapper] <-C->
[library]

You can try to leverage existing C implementation of Go channels for
the wrapper:

https://github.com/tylertreat/chan

[IDNK if these are still functioning with go1.11, ofc]

Example:
https://github.com/matiasinsaurralde/cgo-channels

> Best,
> changkun
> 

Hope this helps,

-- 
Wojciech S. Czarnecki
 << ^oo^ >> OHIR-RIPE

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] go mod tidy pulls in too much

[Scott Cotton]

As a followup,

I have been informed that there is a related open issue which seems like a 
good place to direct followup

https://github.com/golang/go/issues/26955 


Best,
Scott

On Saturday, 29 September 2018 15:53:53 UTC+2, Scott Cotton wrote:
>
> Hi all,
>
> I think this is related to https://github.com/golang/go/issues/26913 
> which is closed but still has commentary that seems unresolved after it was 
> closed .
>
> Scott
>
>
> On Friday, 28 September 2018 20:11:29 UTC+2, Harmen wrote:
>>
>> On Fri, Sep 28, 2018 at 10:19:50AM -0700, thepud...@gmail.com wrote: 
>> > Hi Harmen, 
>> > 
>> > And my first sentence might not have been clear. When I said "even in 
>> your 
>> > current situation, 'go build' is still pulling in exactly what it 
>> needs", I 
>> > was trying to reference the actual compilation process. 
>> > 
>> > In other words, I was just trying to make it clear that even if you 
>> have 
>> > "extra" dependencies appearing in your go.mod file, the resulting 
>> binary 
>> > produced by 'go build' does not have anything extra or any unused 
>> > dependencies. 
>>
>> Hi thepudds, 
>> thanks for your anwers. 
>>
>> It doesn't hurt per-se, but all those unused modules will be vendored in 
>> the 
>> repo (or the CI will have to download them every time). Maybe the consul 
>> repo 
>> is a particularly unlucky repo, since the /api package is small compared 
>> to the 
>> rest of the repo, but still. 
>>
>> As for the argument that it's for test reproducibility, I do not follow 
>> that. 
>> Everything to test /api is there, and the stuff I don't import doesn't 
>> need to 
>> be tested in the first place. 
>>
>> > In any event, I wanted to share at least my personal understanding, but 
>> of 
>> > course happy to learn more... 
>>
>> I'm trying to do the same :) 
>> Thanks! 
>>
>> > 
>> > --thepudds 
>> > 
>> > 
>> > On Friday, September 28, 2018 at 1:00:17 PM UTC-4, thepud...@gmail.com 
>> > wrote: 
>> > > 
>> > >  > "So if consul adds a go.mod file in the root, then `mod tidy` will 
>> > > suddenly 
>> > > behave as I would expect (i.e. not pull in unused dependencies)? " 
>> > > 
>> > > Hi Harmen, 
>> > > 
>> > > Just in case this isn't already clear-- note that even in your 
>> current 
>> > > situation, 'go build' is still pulling in exactly what it needs (and 
>> not 
>> > > pulling in unused dependencies). 
>> > > 
>> > > I think there is not a significant harm in your go.mod containing 
>> these 
>> > > indirect dependencies (aside from of course seeing that longer list, 
>> etc.). 
>> > > 
>> > > And there is some benefit -- this behavior is part of what provides 
>> for 
>> > > 100% reproducible builds and tests. The modules system records 
>> precise 
>> > > dependency version information, and in your case, that precise 
>> dependency 
>> > > version information for some of your indirect dependencies is being 
>> > > recorded in your go.mod (given that it is not yet recorded in the 
>> > > non-existent go.mod of some of your direct dependencies).   
>> > > 
>> > > As an example, this behavior helps make sure that `go test all` is 
>> 100% 
>> > > reproducible for you (where `go test all` runs tests for both your 
>> module, 
>> > > your direct dependencies, and your indirect dependencies, which is 
>> valuable 
>> > > as one way of validating that the currently selected packages 
>> versions are 
>> > > compatible -- the number of possible version combinations is 
>> exponential in 
>> > > the number of modules, so in general you cannot expect your 
>> dependencies to 
>> > > have tested against all possible combinations of *their* 
>> dependencies). 
>> > > 
>> > > --thepudds 
>> > > 
>> > > On Friday, September 28, 2018 at 12:48:20 PM UTC-4, Harmen wrote: 
>> > >> 
>> > >> On Fri, Sep 28, 2018 at 04:48:32PM +0100, Paul Jolly wrote: 
>> > >> > Hi Harmen 
>> > >> > 
>> > >> > I described the problem on 
>> https://github.com/golang/go/issues/27920, 
>> > >> which 
>> > >> > > got 
>> > >> > > closed within three minutes as being "documented", and "works as 
>> > >> > > expected" (which I assume also means "works as intended"). 
>> > >> > > Is this really the intented behaviour? It seems unexpected to 
>> me. Or 
>> > >> > > should I 
>> > >> > > simply stay away from `go mod tidy`? 
>> > >> > > 
>> > >> > 
>> > >> > I replied to your issue earlier. But could arguably have been 
>> slightly 
>> > >> more 
>> > >> > detailed in my response beyond simply linking to 
>> > >> > 
>> > >> 
>> https://github.com/golang/go/wiki/Modules#why-does-go-mod-tidy-record-indirect-and-test-dependencies-in-my-gomod
>>  
>> > >> > 
>> > >> > As you note, github.com/hashicorp/consul/api is a package. It is 
>> a 
>> > >> package 
>> > >> > in the module github.com/hashicorp/consul. Despite there being no 
>> > >> go.mod in 
>> > >> > github.com/hashicorp/consul, the g

[go-nuts] Is this implementation of weak pointers safe?

[Ben Lubar]

https://play.golang.org/p/iBAii-f84Sq

vet is complaining because the unsafe.Pointer usage would normally be 
dangerous around the garbage collector, but since I have a finalizer on the 
"real" pointer and there is no way the code could access the uintptr with 
the same value as the real pointer once the finalizer has run or been 
removed by Close, shouldn't this code be safe?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Is this implementation of weak pointers safe?

[Ian Denhardt]

Quoting Ben Lubar (2018-09-29 14:40:28)
>[1]https://play.golang.org/p/iBAii-f84Sq
>vet is complaining because the unsafe.Pointer usage would normally be
>dangerous around the garbage collector, but since I have a finalizer on
>the "real" pointer and there is no way the code could access the
>uintptr with the same value as the real pointer once the finalizer has
>run or been removed by Close, shouldn't this code be safe?

>From the unsafe package's documentation:

> Even if a uintptr holds the address of some object, the garbage
> collector will not update that uintptr's value if the object moves

One valid implementation strategy for a garbage collector is to do arena
allocation, and on a GC pass copy all *live* objects out of an arena
into another one for longer-lived objects, and then free the old arena.
In this case, your uintptrs would still still contain the old addresses,
even though the objects have moved. Note that the finalizer is never run
in this case, because the object has been moved without being garbage.

Whether what you're doing is actually safe depends heavily on the
implementation details of the garbage collector. It may or may not
be possible for it to break given the current implementation;
I don't know enough about it to be sure. But a correct implementation
of the runtime could very well corrupt memory given the code you
linked; even if it works now, it's possible in a future release of
Go this could break.

Hope this helps,

-Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Are C allocated memory completely isolated to Go memory in any cases?

[K Davidson]

Not sure if it would be of any help, but maybe you can gleem some insight 
from the way these packages did things?

https://github.com/mattn/go-gtk
https://github.com/gotk3/gotk3

-K

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] cgo use question

[Scott Cotton]

Howdy Gophers,

I've a got cgo question.

Suppose I want to use cgo in 2 packages and have one C function used in C 
code in both packages.

Is there a way to do this with the go tool only?  I am trying to avoid a 
build that needs make or the like to build a c library,
or copying and renaming the C functions.

My attempts so far have all yielded link errors.  Either the C function is 
undefined in one of the two Go packages (at build or runtime, depending on 
how I use "extern") or I get multiple symbol definitions.

Scott

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] using 'go function_call()' inside Go program

[Hemant Singh]

I have a question.  I have changed the open goping.go code to return error 
from functions. 

The following function is implemented in a .goping.go file.

func ping(config *Config, address, name string, pattern *regexp.Regexp) 
error {  }

Another function shown below, in the same .go file, calls 'go ping()'.  As 
one can see above, the ping function returns an error.  How do I change the 
'go ping()' code below to grab the error returned from calling ping()?  I 
could also use a URL to where a description exists for using 'go 
function_call' inside a Go program.

func ping_main(address string) error {
go ping(config, address, "DstIP", LATENCY_PATTERN)
}

Best,

Hemant

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] using 'go function_call()' inside Go program

[Justin Israel]

On Sun, Sep 30, 2018 at 3:47 PM Hemant Singh  wrote:

> I have a question.  I have changed the open goping.go code to return error
> from functions.
>
> The following function is implemented in a .goping.go file.
>
> func ping(config *Config, address, name string, pattern *regexp.Regexp)
> error {  }
>
> Another function shown below, in the same .go file, calls 'go ping()'.  As
> one can see above, the ping function returns an error.  How do I change the
> 'go ping()' code below to grab the error returned from calling ping()?  I
> could also use a URL to where a description exists for using 'go
> function_call' inside a Go program.
>
> func ping_main(address string) error {
> go ping(config, address, "DstIP", LATENCY_PATTERN)
> }
>

In this current form, it is not possible to grab the error because you are
not waiting on the ping() to finish when you call it asynchronously in a
goroutine. If you need the error, the most obvious thing to do is to call
it in a blocking fashion:

func ping_main(address string) error {
return ping(config, address, "DstIP", LATENCY_PATTERN)
}

​

But if you need to call it in a non-blocking way, then you have to arrange
a way to receive the error value later once the ping is complete. Wrapping
the existing ping() function might be a way to go:

func ping_main(address string) <-chan error {
err := make(chan error, 1)
go func() {
err <- ping(config, address, "DstIP", LATENCY_PATTERN)
}()
return err
}

​

This means the ping still runs in a non-blocking way, but will report the
return value when it is complete.


>
> Best,
>
> Hemant
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Is this implementation of weak pointers safe?

[Ben Lubar]

On Saturday, September 29, 2018 at 2:49:19 PM UTC-5, Ian Denhardt wrote:
>
> Quoting Ben Lubar (2018-09-29 14:40:28) 
> >[1]https://play.golang.org/p/iBAii-f84Sq 
> >vet is complaining because the unsafe.Pointer usage would normally be 
> >dangerous around the garbage collector, but since I have a finalizer 
> on 
> >the "real" pointer and there is no way the code could access the 
> >uintptr with the same value as the real pointer once the finalizer 
> has 
> >run or been removed by Close, shouldn't this code be safe? 
>
> From the unsafe package's documentation: 
>
> > Even if a uintptr holds the address of some object, the garbage 
> > collector will not update that uintptr's value if the object moves 
>
> One valid implementation strategy for a garbage collector is to do arena 
> allocation, and on a GC pass copy all *live* objects out of an arena 
> into another one for longer-lived objects, and then free the old arena. 
> In this case, your uintptrs would still still contain the old addresses, 
> even though the objects have moved. Note that the finalizer is never run 
> in this case, because the object has been moved without being garbage. 
>
> Whether what you're doing is actually safe depends heavily on the 
> implementation details of the garbage collector. It may or may not 
> be possible for it to break given the current implementation; 
> I don't know enough about it to be sure. But a correct implementation 
> of the runtime could very well corrupt memory given the code you 
> linked; even if it works now, it's possible in a future release of 
> Go this could break. 
>
> Hope this helps, 
>
> -Ian 
>

Is there any way to implement this safely, or should I just give up on the 
weak pointer idea?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Is this implementation of weak pointers safe?

[Tamás Gulácsi]

Why do you want a weak pointer? What for?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.