[go-nuts] golang gc stop the world how to stop cpu busy goroutine?

[刘桂祥]

when have one or two cpu busy goroutine in my server (example for loop 
empty),  the gc stop the world how to stop the goroutine ?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] golang gc stop the world how to stop cpu busy goroutine?

[刘桂祥]

package main

import "runtime"

// GOMAXPROCS=1
var sum1, summain int64

func add1() {
for i := 0; i < 1; i++ {
for j := 0; j < 1; j++ {
sum1++
}
}
}

func bar() {
for {
add1()
}
}

func main() {
go bar()
runtime.Gosched()
println("debug>>>")
}

above is my example code,when I set GOMAXPROCS=1 and build with -gcflags 
"-N -l"  the println("debug>>>") don't  execute 
I just doubt the go runtime support peempt schedule in call function,   it 
don't work ??



在 2017年9月4日星期一 UTC+8下午2:20:37，John Souvestre写道：
>
> Although a goroutine isn’t pre-emptible by the Go scheduler, there is an 
> opportunity whenever it does something which blocks or when it calls a 
> non-inlined function.  So generally the GC’s STW can take place pretty 
> quickly.  But if you are doing something with is totally compute bound then 
> it can be a problem.  An easy solution is to insert a call to 
> runtime.Gosched() every so often.
>
>  
>
> I believe that there is an ongoing discussion about a way for Go to plan 
> ahead and be able to handle even these cases, but it’s something for the 
> future.
>
>  
>
> John
>
> John Souvestre - New Orleans LA
>
>  
>
> *From:* golan...@googlegroups.com  [mailto:
> golan...@googlegroups.com ] *On Behalf Of *???
> *Sent:* 2017 September 04, Mon 00:47
> *To:* golang-nuts
> *Subject:* [go-nuts] golang gc stop the world how to stop cpu busy 
> goroutine?
>
>  
>
> when have one or two cpu busy goroutine in my server (example for loop 
> empty),  the gc stop the world how to stop the goroutine ?
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts...@googlegroups.com .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] golang gc stop the world how to stop cpu busy goroutine?

[刘桂祥]

Yes, I just want to let the bar func goroutine run first  or replace 
runtime.Gosched() 
with time.Sleep,
But I still doubt why add1 in bar func don't happen preempt schedule ?

在 2017年9月4日星期一 UTC+8下午3:25:04，John Souvestre写道：
>
> I think that you put the call to runtime.Gosched() in the wrong place.  
> You want to place it in the routine which needs to be pre-empted.  In your 
> example, in add1() inside the “for i” loop or in bar() in the “for” loop.
>
>  
>
> John
>
> John Souvestre - New Orleans LA
>
>  
>
> *From:* golan...@googlegroups.com  [mailto:
> golan...@googlegroups.com ] *On Behalf Of *???
> *Sent:* 2017 September 04, Mon 02:10
> *To:* golang-nuts
> *Subject:* Re: [go-nuts] golang gc stop the world how to stop cpu busy 
> goroutine?
>
>  
>
> package main
>
>  
>
> import "runtime"
>
>  
>
> // GOMAXPROCS=1
>
> var sum1, summain int64
>
>  
>
> func add1() {
>
> for i := 0; i < 1; i++ {
>
> for j := 0; j < 1; j++ {
>
> sum1++
>
> }
>
> }
>
> }
>
>  
>
> func bar() {
>
> for {
>
> add1()
>
> }
>
> }
>
>  
>
> func main() {
>
> go bar()
>
> runtime.Gosched()
>
> println("debug>>>")
>
> }
>
>
> above is my example code,when I set GOMAXPROCS=1 and build with -gcflags 
> "-N -l"  the println("debug>>>") don't  execute 
>
> I just doubt the go runtime support peempt schedule in call function,   it 
> don't work ??
>
>
>
> 在 2017年9月4日星期一 UTC+8下午2:20:37，John Souvestre写道：
>
> Although a goroutine isn’t pre-emptible by the Go scheduler, there is an 
> opportunity whenever it does something which blocks or when it calls a 
> non-inlined function.  So generally the GC’s STW can take place pretty 
> quickly.  But if you are doing something with is totally compute bound then 
> it can be a problem.  An easy solution is to insert a call to 
> runtime.Gosched() every so often.
>
>  
>
> I believe that there is an ongoing discussion about a way for Go to plan 
> ahead and be able to handle even these cases, but it’s something for the 
> future.
>
>  
>
> John
>
> John Souvestre - New Orleans LA
>
>  
>
> *From:* golan...@googlegroups.com [mailto:golan...@googlegroups.com] *On 
> Behalf Of *???
> *Sent:* 2017 September 04, Mon 00:47
> *To:* golang-nuts
> *Subject:* [go-nuts] golang gc stop the world how to stop cpu busy 
> goroutine?
>
>  
>
> when have one or two cpu busy goroutine in my server (example for loop 
> empty),  the gc stop the world how to stop the goroutine ?
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts...@googlegroups.com .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] golang gc stop the world how to stop cpu busy goroutine?

[刘桂祥]

But I build with -gcflags "-N -l"  
here is gdb code:
(gdb)
Dump of assembler code for function main.main:
   0x2100 <+0>: mov%gs:0x8a0,%rcx
   0x2109 <+9>: cmp0x10(%rcx),%rsp
   0x210d <+13>: jbe0x216c 
   0x210f <+15>: sub$0x18,%rsp
   0x2113 <+19>: mov%rbp,0x10(%rsp)
   0x2118 <+24>: lea0x10(%rsp),%rbp
   0x211d <+29>: movl   $0x0,(%rsp)
   0x2124 <+36>: lea0x6abfd(%rip),%rax# 0x6cd28 

   0x212b <+43>: mov%rax,0x8(%rsp)
   0x2130 <+48>: callq  0x2bf60 
   0x2135 <+53>: callq  0x253c0 
   0x213a <+58>: callq  0x242d0 
   0x213f <+63>: lea0x643b1(%rip),%rax# 0x664f7 

   0x2146 <+70>: mov%rax,(%rsp)
   0x214a <+74>: movq   $0x8,0x8(%rsp)
   0x2153 <+83>: callq  0x24bf0 
   0x2158 <+88>: callq  0x24550 
   0x215d <+93>: callq  0x24360 
   0x2162 <+98>: mov0x10(%rsp),%rbp
   0x2167 <+103>: add$0x18,%rsp
   0x216b <+107>: retq
   0x216c <+108>: callq  0x493c0 
   0x2171 <+113>: jmp0x2100 
End of assembler dump.
(gdb) disas bar
No symbol "bar" in current context.
(gdb) disas main.bar
Dump of assembler code for function main.bar:
   0x20d0 <+0>: mov%gs:0x8a0,%rcx
   0x20d9 <+9>: cmp0x10(%rcx),%rsp
   0x20dd <+13>: jbe0x20ec 
   0x20df <+15>: jmp0x20e1 
   0x20e1 <+17>: jmp0x20e3 
   0x20e3 <+19>: callq  0x2040 
   0x20e8 <+24>: jmp0x20ea 
   0x20ea <+26>: jmp0x20e1 
   0x20ec <+28>: callq  0x493c0 
   0x20f1 <+33>: jmp0x20d0 



在 2017年9月4日星期一 UTC+8下午3:54:13，John Souvestre写道：
>
> time.Sleep() will do the job, but you probably wouldn’t want to use it in 
> a compute-bound loop.
>
>  
>
> Perhaps the call to add1() got inlined in bar().  The compiler might have 
> simplified add1() first, since sum1 wasn’t being used anywhere.  Then 
> perhaps it inlined what was left.
>
>  
>
> You might want to use sum1 by printing it (not inside the loop!).
>
>  
>
> John
>
> John Souvestre - New Orleans LA
>
>  
>
> *From:* golan...@googlegroups.com  [mailto:
> golan...@googlegroups.com ] *On Behalf Of *???
> *Sent:* 2017 September 04, Mon 02:42
> *To:* golang-nuts
> *Subject:* Re: [go-nuts] golang gc stop the world how to stop cpu busy 
> goroutine?
>
>  
>
> Yes, I just want to let the bar func goroutine run first  or replace 
> runtime.Gosched() 
> with time.Sleep,
>
> But I still doubt why add1 in bar func don't happen preempt schedule ?
>
>
> 在 2017年9月4日星期一 UTC+8下午3:25:04，John Souvestre写道：
>
> I think that you put the call to runtime.Gosched() in the wrong place.  
> You want to place it in the routine which needs to be pre-empted.  In your 
> example, in add1() inside the “for i” loop or in bar() in the “for” loop.
>
>  
>
> John
>
> John Souvestre - New Orleans LA
>
>  
>
> *From:* golan...@googlegroups.com [mailto:golan...@googlegroups.com] *On 
> Behalf Of *???
> *Sent:* 2017 September 04, Mon 02:10
> *To:* golang-nuts
> *Subject:* Re: [go-nuts] golang gc stop the world how to stop cpu busy 
> goroutine?
>
>  
>
> package main
>
>  
>
> import "runtime"
>
>  
>
> // GOMAXPROCS=1
>
> var sum1, summain int64
>
>  
>
> func add1() {
>
> for i := 0; i < 1; i++ {
>
> for j := 0; j < 1; j++ {
>
> sum1++
>
> }
>
> }
>
> }
>
>  
>
> func bar() {
>
> for {
>
> add1()
>
> }
>
> }
>
>  
>
> func main() {
>
> go bar()
>
> runtime.Gosched()
>
> println("debug>>>")
>
> }
>
>
> above is my example code,when I set GOMAXPROCS=1 and build with -gcflags 
> "-N -l"  the println("debug>>>") don't  execute 
>
> I just doubt the go runtime support peempt schedule in call function,   it 
> don't work ??
>
>
>
> 在 2017年9月4日星期一 UTC+8下午2:20:37，John Souvestre写道：
>
> Although a goroutine isn’t pre-emptible by the Go scheduler, there is an 
> opportunity whenever it does something which blocks or when it calls a 
> non-inlined function.  So generally the GC’s STW can take place pretty 
> quickly.  But if you are doing something with is totally compute bound then 
> it can be a problem.  An easy solution is to insert a call to 
> runtime.Gosched() every so often.
>
>  
>
> I believe that there is an ongoing discussion about a way for Go to plan 
> ahead and be able to handle even these cases, but it’s something for the 
> future.
>
>  
>
> John
>
> John Souvestre - New Orleans LA
>
>  
>
> *From:* golan...@googlegroups.com [mailto:golan...@googlegroups.com] *On 
> Behalf Of *???
> *Sent:* 2017 September 04, Mon 00:47
> *To:* golang-nuts
> *Subject:* [go-nuts] golang gc stop the world how to stop cpu busy 
> goroutine?
>
>  
>
> when have one or two cpu busy goroutine in my ser

[go-nuts] golang preempt schedule how work ?

[刘桂祥]

 

I have a question about golang preempt schedule,  here is my example code;

I doubt why the bar groutine don't happen preempt schedule? (I build with 
-gcflags “-N -l” to stop func inline)

More if my server with one or two cpu bound worker goroutine runing, when 
the gc need stop the world, how it can stop the worker? 


package main

import “time"


// GOMAXPROCS=1

var sum1 int64


func add1() {

   for i := 0; i < 1; i++ {

   for j := 0; j < 1; j++ {

   sum1++

   }

   }

}


func bar() {

   for {

   add1()

   }

}


func main() {


go bar()


time.Sleep(1*time.Second)


println("debug>>>")

}


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] golang preempt schedule how work ?

[刘桂祥]

Thanks a lot!

在 2017年9月8日星期五 UTC+8下午8:23:39，Ian Lance Taylor写道：
>
> On Mon, Sep 4, 2017 at 10:02 PM, 刘桂祥 > 
> wrote: 
> > 
> > I have a question about golang preempt schedule,  here is my example 
> code; 
> > 
> > I doubt why the bar groutine don't happen preempt schedule? (I build 
> with 
> > -gcflags “-N -l” to stop func inline) 
> > 
> > More if my server with one or two cpu bound worker goroutine runing, 
> when 
> > the gc need stop the world, how it can stop the worker? 
>
> This is https://golang.org/issue/10958 . 
>
> Ian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] about golang escape analysis

[刘桂祥]

package main

import "testing"

var gbuf []byte
var gi int

func BenchmarkCopy1(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
m := make(map[string]int, 5)
for i := 0; i < 5; i++ {
m["100"] = i
}
for key := range m {
copyIface1(key, key)
}

}
}

func BenchmarkCopy2(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
m := make(map[string]int, 5)
for i := 0; i < 5; i++ {
m["100"] = i
}
for key := range m {
copyIface2(key, key)
}

}
}

func copyIface1(a ...interface{}) {
if gi > 0 {
copy(gbuf, a[0].(string))
copy(gbuf, a[1].(string))
}
}

func copyIface2(a interface{}, b interface{}) {
if gi > 0 {
copy(gbuf, a.(string))
copy(gbuf, b.(string))
}
}
BenchmarkCopy1-4300452 ns/op   32 B/op2 
allocs/op
BenchmarkCopy2-4500323 ns/op0 B/op0 
allocs/op

BenchmarkCopy1 key escapes to heap
BenchmarkCopy2 key does not escape
why ??? 


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: about golang escape analysis

[刘桂祥]

very very thanks!

cannot inline copyIface1: has ... args
from *a[0].(string) (indirection)
key escapes to heap


在 2017年12月3日星期日 UTC+8下午4:08:41，Dave Cheney写道：
>
> Add -gcflags=“-m -m” will tell you. 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] golang gc time STW1=>Mark=>STW2

[刘桂祥]

package main

import (
   "runtime"

   _ "net/http/pprof"

   "net/http"
   "time"
)


var ch = make(chan struct{},1)

func printMem() {
   var mem runtime.MemStats

   runtime.ReadMemStats(&mem)

   println(mem.HeapObjects)


}


func easy() {
   <-ch
}

func bussy() {
   sum := 0
   for i:=0;i<1000;i++ {
  for j:=0;j<1*10;j++ {
 sum++
  }
  time.Sleep(time.Duration(100+i/10)*time.Millisecond)
   }
}

func main() {

   for i:=0;i<1;i++ {
  //go easy()
  go bussy()
   }

   ticker := time.NewTicker(3*time.Second)
   for range ticker.C {
  runtime.GC()
   }
   http.ListenAndServe(":9090",nil)
}


gc 1 @3.007s 0%: 195+0.26+0.025 ms clock, 587+0/0.21/0.44+0.077 ms cpu, 
1->1->0 MB, 4 MB goal, 4 P (forced)
gc 2 @6.006s 0%: 0.065+0.82+0.041 ms clock, 0.26+0/0.71/0.51+0.16 ms cpu, 
0->0->0 MB, 4 MB goal, 4 P (forced)
gc 3 @9.005s 0%: 242+0.32+0.016 ms clock, 971+0/0.045/0.36+0.064 ms cpu, 
0->0->0 MB, 4 MB goal, 4 P (forced)
gc 4 @12.005s 0%: 0.010+0.37+0.039 ms clock, 0.040+0/0.32/0.68+0.15 ms cpu, 
0->0->0 MB, 4 MB goal, 4 P (forced)
gc 5 @15.004s 0%: 233+0.33+0.014 ms clock, 932+0/0.050/0.36+0.059 ms cpu, 
0->0->0 MB, 4 MB goal, 4 P (forced)
gc 6 @18.006s 0%: 0.016+0.19+0.016 ms clock, 0.064+0/0.16/0.33+0.066 ms 
cpu, 0->0->0 MB, 4 MB goal, 4 P (forced)
gc 7 @21.003s 0%: 276+0.30+0.049 ms clock, 1104+0/0.044/0.33+0.19 ms cpu, 
0->0->0 MB, 4 MB goal, 4 P (forced)
gc 8 @24.002s 0%: 52+0.23+0.019 ms clock, 211+0/0.20/0.37+0.076 ms cpu, 
0->0->0 MB, 4 MB goal, 4 P (forced)
gc 9 @27.002s 0%: 270+0.23+0.013 ms clock, 1080+0/0.21/0.37+0.053 ms cpu, 
0->0->0 MB, 4 MB goal, 4 P (forced)
gc 10 @30.002s 0%: 59+0.48+0.029 ms clock, 239+0/0.44/0.10+0.11 ms cpu, 
0->0->0 MB, 4 MB goal, 4 P (forced)
gc 11 @33.002s 0%: 339+0.32+0.014 ms clock, 1356+0/0.29/0.44+0.058 ms cpu, 
0->0->0 MB, 4 MB goal, 4 P (forced)
gc 12 @36.004s 0%: 286+0.45+0.038 ms clock, 1145+0/0.40/0.13+0.15 ms cpu, 
0->0->0 MB, 4 MB goal, 4 P (forced)
gc 13 @39.006s 0%: 123+0.38+0.023 ms clock, 493+0/0.36/0.66+0.094 ms cpu, 
0->0->0 MB, 4 MB goal, 4 P (forced)
gc 14 @42.005s 0%: 0.016+0.26+0.015 ms clock, 0.064+0/0.25/0.41+0.062 ms 
cpu, 0->0->0 MB, 4 MB goal, 4 P (forced)
gc 15 @45.005s 0%: 278+0.33+0.020 ms clock, 1113+0/0.29/0.28+0.082 ms cpu, 
0->0->0 MB, 4 MB goal, 4 P (forced)
gc 16 @48.004s 0%: 128+0.22+0.039 ms clock, 514+0/0.13/0.40+0.15 ms cpu, 
0->0->0 MB, 4 MB goal, 4 P (forced)
gc 17 @51.004s 0%: 0.014+0.34+0.020 ms clock, 0.056+0/0.053/0.39+0.081 ms 
cpu, 0->0->0 MB, 4 MB goal, 4 P (forced)
gc 18 @54.006s 0%: 331+0.23+0.017 ms clock, 1325+0/0.19/0.35+0.070 ms cpu, 
0->0->0 MB, 4 MB goal, 4 P (forced)
gc 19 @57.002s 0%: 179+0.30+0.017 ms clock, 717+0/0.044/0.32+0.068 ms cpu, 
0->0->0 MB, 4 MB goal, 4 P (forced)
gc 20 @60.004s 0%: 0.009+0.47+0.033 ms clock, 0.038+0/0.36/0.77+0.13 ms 
cpu, 0->0->0 MB, 4 MB goal, 4 P (forced)


version golang 1.9.2
I doubt the gc time: STW1=>Mark=>STW2
the long time cal goroutine make the STW1 long time but why the STW2 also 
need stop the world and the time is little ?? 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: golang gc time STW1=>Mark=>STW2

[刘桂祥]

195+0.26+0.025 ms
0.065+0.82+0.041 ms
242+0.32+0.016 ms
0.010+0.37+0.039 ms
233+0.33+0.014 ms

version go1.9.2

I want to ask why the third phrase mark-termination time is always short, 
it also need STW ??

在 2017年12月27日星期三 UTC+8下午5:14:52，Dave Cheney写道：
>
> I’m sorry I don’t understand the question you are asking but what I see 
> from those gc times is your go busy() goroutine is blocking the STW1 phase 
> because Go cannot currently preempt running functions. 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] go compile for memory allocation

[刘桂祥]

//  example1.go
packge main

func main() {
 s := make([]byte, 1024, 1024)
 _ = s
}

s will be allocated in stack and lookup assembl 
e
 
code  not call runtime.makeslice

// example2.go
packge main

func main() {
 cap := 1024
 s := make([]byte, 1024, cap)
 _ = s
}

s will be allocated in heap and lookup assemble code there is 
runtime.makeslice   why this ???

// example3.go
package main

func main() {
   a := 100
   if a>1 {
 a = 1000
   }
   b := interface{}(a)
   _ = b
}

0x 0 (example1.go:3) TEXT "".main(SB), $64-0
0x 0 (example1.go:3) MOVQ (TLS), CX
0x0009 9 (example1.go:3) CMPQ SP, 16(CX)
0x000d 00013 (example1.go:3) JLS 93
0x000f 00015 (example1.go:3) SUBQ $64, SP
0x0013 00019 (example1.go:3) MOVQ BP, 56(SP)
0x0018 00024 (example1.go:3) LEAQ 56(SP), BP
0x001d 00029 (example1.go:3) FUNCDATA $0, gclocals·33cdeebe80329
f1fdbee7f5874cb(SB)
0x001d 00029 (example1.go:3) FUNCDATA $1, gclocals·33cdeebe80329
f1fdbee7f5874cb(SB)
0x001d 00029 (example1.go:8) MOVQ $1000, "".autotmp_0+48(SP)
0x0026 00038 (example1.go:8) MOVQ $0, "".autotmp_1+40(SP)
0x002f 00047 (example1.go:8) LEAQ type.int(SB), AX
0x0036 00054 (example1.go:8) MOVQ AX, (SP)
0x003a 00058 (example1.go:8) LEAQ "".autotmp_0+48(SP), AX
0x003f 00063 (example1.go:8) MOVQ AX, 8(SP)
0x0044 00068 (example1.go:8) LEAQ "".autotmp_1+40(SP), AX
0x0049 00073 (example1.go:8) MOVQ AX, 16(SP)
0x004e 00078 (example1.go:8) PCDATA $0, $0
0x004e 00078 (example1.go:8) CALL runtime.convT2E(SB)
0x0053 00083 (example1.go:10) MOVQ 56(SP), BP
0x0058 00088 (example1.go:10) ADDQ $64, SP
0x005c 00092 (example1.go:10) RET
0x005d 00093 (example1.go:10) NOP
0x005d 00093 (example1.go:3) CALL runtime.morestack_noctxt(SB)
0x0062 00098 (example1.go:3) JMP 0


lookup the assemble code and see the compiler is very clever to opt that 
but in example2.go why don't do this 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] go escape analysis

[刘桂祥]

// example1.go
package main


func main() {

for i := 0; i < 2; i++ {
m := make(map[int]int) 

   
m[1] = 100 
}   


}


main make(map[int]int) does not escape


// example2.go
package main


func main() {
var m map[int]int
for i := 0; i < 2; i++ {
m = make(map[int]int)   

 
m[1] = 100 
}   


}

 make(map[int]int) escapes to heapwhy ???


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: go escape analysis

[刘桂祥]

go 1.7

在 2016年9月28日星期三 UTC+8下午10:41:09，Dave Cheney写道：
>
> Which version of Go?
>
> On Thursday, 29 September 2016 00:18:29 UTC+10, 刘桂祥 wrote:
>>
>> // example1.go
>> package main
>>
>>
>> func main() {
>>
>> for i := 0; i < 2; i++ {
>> m := make(map[int]int)   
>> 
>>  
>> m[1] = 100 
>> }   
>>
>>
>> }
>>
>>
>> main make(map[int]int) does not escape
>>
>>
>> // example2.go
>> package main
>>
>>
>> func main() {
>> var m map[int]int
>> for i := 0; i < 2; i++ {
>> m = make(map[int]int)   
>> 
>>  
>> m[1] = 100 
>> }   
>>
>>
>> }
>>
>>  make(map[int]int) escapes to heapwhy ???
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] go analysis memory mallocs

[刘桂祥]

// example1.go
package main


import "runtime"


func main() {
m := new(runtime.MemStats)
runtime.ReadMemStats(m)
println(m.Alloc, m.Mallocs)
cap := 1024 * 1024 * 3
s := make([]byte, cap)
_ = s
runtime.ReadMemStats(m)
println(m.Alloc, m.Mallocs)
}

go run example1.go
32744 65
3178472 66

// example2.go
package main


import "runtime"


func main() {
m := new(runtime.MemStats)
runtime.ReadMemStats(m)
println(m.Alloc, m.Mallocs)
cap := 1024 * 1024 * 4
s := make([]byte, cap)
_ = s
runtime.ReadMemStats(m)
println(m.Alloc, m.Mallocs)
}

go run example2.go
32744 65
4228984 77

why it have 12 times mallocs ??

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: go compile for memory allocation

[刘桂祥]

https://github.com/golang/go/issues/17275

在 2016年9月29日星期四 UTC+8下午2:57:08，Dave Cheney写道：
>
> Don't forget to log that issue. 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: go analysis memory mallocs

[刘桂祥]

In my two examples, I use variable in makeslice that it will all escape to 
the heap ,  But I just want to use runtime.ReadMemStats to lookup it
and find strange 
ly
 
example2.go memstats.Mallocs is many times  don't know why ...

在 2016年9月29日星期四 UTC+8下午2:56:38，Dave Cheney写道：
>
> Be careful that the compiler isnt removing some or all of your program. 
> Check the asm to assert that your program is not being optimised away.
>
> Then check -gcflags=-m to see if the compiler is choosing a different 
> escape analysis depending on the size of your allocation. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: go analysis memory mallocs

[刘桂祥]

package main

import "runtime"

func init() {
runtime.MemProfileRate = 1
}
func main() {
m := new(runtime.MemStats)
runtime.ReadMemStats(m)
println(m.Alloc, m.Mallocs)
cap := 1024 * 1024 * 4
s := make([]byte, cap)
_ = s
runtime.ReadMemStats(m)
println(m.Alloc, m.Mallocs)
}

I add this to example2.go  but the results is the same . Did I use right ??

在 2016年9月29日星期四 UTC+8下午5:07:23，Dave Cheney写道：
>
> One way to do this might be to enable memory profiling in your program 
> with the rate set to 1. Hopefully this will record the stack trace of every 
> allocation. The data may need some interpretation

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: go analysis memory mallocs

[刘桂祥]

package main

import "runtime"

func init() {
runtime.MemProfileRate = 1
}
func main() {
m := new(runtime.MemStats)
runtime.ReadMemStats(m)
println(m.Alloc, m.Mallocs)
cap := 1024 * 1024 * 4
s := make([]byte, cap)
_ = s
runtime.ReadMemStats(m)
println(m.Alloc, m.Mallocs)
}
go run example2.go
32744 65
4228984 76
sorry  add this to example2.go  and the results is a little different  but 
I also don't konw the reason to the results

在 2016年9月29日星期四 UTC+8下午5:07:23，Dave Cheney写道：
>
> One way to do this might be to enable memory profiling in your program 
> with the rate set to 1. Hopefully this will record the stack trace of every 
> allocation. The data may need some interpretation

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: go escape analysis

[刘桂祥]

veyr thanks and want to know more about go's compile and escape rules 

在 2016年9月29日星期四 UTC+8上午1:39:09，Chris Manghane写道：
>
> In the first example, make does not escape the scope of the for statement. 
> In the second example, make is assigned to m, which is outside of the scope 
> of the for statement, which means the make operation escapes its scope and 
> subsequently, is heap allocated. If you want more information about why 
> something escapes, try compiling with -gcflags "-m -m" for an explanation 
> of the escape analysis information.
>
> On Wed, Sep 28, 2016 at 7:56 AM, 刘桂祥 > 
> wrote:
>
>> go 1.7
>>
>> 在 2016年9月28日星期三 UTC+8下午10:41:09，Dave Cheney写道：
>>
>>> Which version of Go?
>>>
>>> On Thursday, 29 September 2016 00:18:29 UTC+10, 刘桂祥 wrote:
>>>>
>>>> // example1.go
>>>> package main
>>>>
>>>>
>>>> func main() {
>>>>
>>>> for i := 0; i < 2; i++ {
>>>> m := make(map[int]int) 
>>>>
>>>>  
>>>>
>>>> m[1] = 100 
>>>> }   
>>>>
>>>>
>>>> }
>>>>
>>>>
>>>> main make(map[int]int) does not escape
>>>>
>>>>
>>>> // example2.go
>>>> package main
>>>>
>>>>
>>>> func main() {
>>>> var m map[int]int
>>>> for i := 0; i < 2; i++ {
>>>> m = make(map[int]int) 
>>>>
>>>>  
>>>>
>>>> m[1] = 100 
>>>> }   
>>>>
>>>>
>>>> }
>>>>
>>>>  make(map[int]int) escapes to heapwhy ???
>>>>
>>>>
>>>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: go analysis memory mallocs

[刘桂祥]

package main

import "runtime"

func main() {
m := new(runtime.MemStats)
runtime.ReadMemStats(m)
cap := 1024
println("debug0")
s := make([]byte, cap)
println("debug1")
_ = s
runtime.ReadMemStats(m)
println("memstats:", m.Alloc, m.Mallocs)
}

I use go1.4.2 and In go source code add mallocgc debug info 
here is the result:
debug0
mallocgc: 1024
mallocgc: 272
mallocgc: 32
debug1

I doubt there is other goroutine call the mallocgc  
 
but if I use go test -bench="." makeslice_test.go
func BenchmarkMakeSlice2(b *testing.B) {
b.ReportAllocs()
cap := 1024
for i := 0; i < b.N; i++ {
s := make([]byte, cap, cap)
_ = s
}
}
the output is:
14120 ns/op1024 B/op   1 allocs/op
I infer the go test know all the mallocgc and also know which is  user's 
source code to call it  

my guess

在 2016年9月29日星期四 UTC+8下午5:30:54，Dave Cheney写道：

> Sorry, you'll have to work harder to generate a memory profile. My 
> github.com/pkg/profile package may help automate some of the work, but 
> may introduce a few allocations of its own which you will have to filter 
> out. 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: go analysis memory mallocs

[刘桂祥]

if use go1.7
in runtime/malloc.go file add mallocgc debug info and go install and go run 
x.goI will get a error "signal: segmentation fault"
in go1.4.2 will not

func mallocgc(size uintptr, typ *_type, needzero bool) unsafe.Pointer {
println("mallocgc:size=", size, "kind=", typ.kind, "needzero=", 
needzero)

在 2016年9月29日星期四 UTC+8下午9:04:58，Dave Cheney写道：
>
> You should use Go 1.7.1 or later. 
>
> On Thursday, 29 September 2016 23:00:25 UTC+10, 刘桂祥 wrote:
>>
>> package main
>>
>> import "runtime"
>>
>> func main() {
>> m := new(runtime.MemStats)
>> runtime.ReadMemStats(m)
>> cap := 1024
>> println("debug0")
>> s := make([]byte, cap)
>> println("debug1")
>> _ = s
>> runtime.ReadMemStats(m)
>> println("memstats:", m.Alloc, m.Mallocs)
>> }
>>
>> I use go1.4.2 and In go source code add mallocgc debug info 
>> here is the result:
>> debug0
>> mallocgc: 1024
>> mallocgc: 272
>> mallocgc: 32
>> debug1
>>
>> I doubt there is other goroutine call the mallocgc  
>>  
>> but if I use go test -bench="." makeslice_test.go
>> func BenchmarkMakeSlice2(b *testing.B) {
>> b.ReportAllocs()
>> cap := 1024
>> for i := 0; i < b.N; i++ {
>> s := make([]byte, cap, cap)
>> _ = s
>> }
>> }
>> the output is:
>> 14120 ns/op1024 B/op   1 allocs/op
>> I infer the go test know all the mallocgc and also know which is  user's 
>> source code to call it  
>>
>> my guess
>>
>> 在 2016年9月29日星期四 UTC+8下午5:30:54，Dave Cheney写道：
>>
>>> Sorry, you'll have to work harder to generate a memory profile. My 
>>> github.com/pkg/profile package may help automate some of the work, but 
>>> may introduce a few allocations of its own which you will have to filter 
>>> out. 
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: go escape analysis

[刘桂祥]

package main

func main() {
var m map[int]int

{
m = make(map[int]int)
}

_ = m
}

if I do this m will not escape just want to know what's the scope rule for 
escape ?  puzzled 
<http://www.baidu.com/link?url=ApC817U9uUoCFHhS_dqb5JzUWJQsslUUA6_TDv3LDZBJgaA-G2ZbRfWA-2cGajgU_MHmTiXVEouMmdPN53mMMXxYd1nvCWWJJvfu5Mmg4Ca>
 

在 2016年9月29日星期四 UTC+8上午1:39:09，Chris Manghane写道：
>
> In the first example, make does not escape the scope of the for statement. 
> In the second example, make is assigned to m, which is outside of the scope 
> of the for statement, which means the make operation escapes its scope and 
> subsequently, is heap allocated. If you want more information about why 
> something escapes, try compiling with -gcflags "-m -m" for an explanation 
> of the escape analysis information.
>
> On Wed, Sep 28, 2016 at 7:56 AM, 刘桂祥 > 
> wrote:
>
>> go 1.7
>>
>> 在 2016年9月28日星期三 UTC+8下午10:41:09，Dave Cheney写道：
>>
>>> Which version of Go?
>>>
>>> On Thursday, 29 September 2016 00:18:29 UTC+10, 刘桂祥 wrote:
>>>>
>>>> // example1.go
>>>> package main
>>>>
>>>>
>>>> func main() {
>>>>
>>>> for i := 0; i < 2; i++ {
>>>> m := make(map[int]int) 
>>>>
>>>>  
>>>>
>>>> m[1] = 100 
>>>> }   
>>>>
>>>>
>>>> }
>>>>
>>>>
>>>> main make(map[int]int) does not escape
>>>>
>>>>
>>>> // example2.go
>>>> package main
>>>>
>>>>
>>>> func main() {
>>>> var m map[int]int
>>>> for i := 0; i < 2; i++ {
>>>> m = make(map[int]int) 
>>>>
>>>>  
>>>>
>>>> m[1] = 100 
>>>> }   
>>>>
>>>>
>>>> }
>>>>
>>>>  make(map[int]int) escapes to heapwhy ???
>>>>
>>>>
>>>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Re: go escape analysis

[刘桂祥]

 

Hi  Chris:


  I am gopher from Beijing China . I am very like programing and adore 
the old hack culture . and NBA too!


I am very happy in google groups and receive the answer so quickly. 
 


go on practice programing and improve my badly english 


 

   question:  


   package main


func main() {

var a []int


Label1:

a = make([]int, 0, 2)

_ = a

}


 could’t you give me an example that happens escape ?

在 2016年10月1日星期六 UTC+8上午12:19:17，Chris Manghane写道：
>
> Hmm, I'm not so sure if that's the rule. The rules are expressed in the 
> code in 
> https://github.com/golang/go/blob/master/src/cmd/compile/internal/gc/esc.go, 
> which is unlikely to make any significant changes in the near future. Those 
> rules aren't written in plain english, but we should not discourage any 
> attempts to do so.
>
> Chris
>
> On Thu, Sep 29, 2016 at 11:42 PM, 'Axel Wagner' via golang-nuts <
> golan...@googlegroups.com > wrote:
>
>> The only real rule is "when the compiler can prove, that it doesn't 
>> escape, it doesn't". There is no guarantee or fixed policy around escape 
>> analysis.
>> If you are convinced that something doesn't escape and that the compiler 
>> should be able to figure it out, you can try filing an issue about that 
>> with the code and output of -gcflags=-m.
>>  
>>
> On Fri, Sep 30, 2016 at 5:20 AM, 刘桂祥 > 
>> wrote:
>>
>>> package main
>>>
>>> func main() {
>>> var m map[int]int
>>>
>>> {
>>> m = make(map[int]int)
>>> }
>>>
>>> _ = m
>>> }
>>>
>>> if I do this m will not escape just want to know what's the scope rule 
>>> for escape ?  puzzled 
>>> <http://www.baidu.com/link?url=ApC817U9uUoCFHhS_dqb5JzUWJQsslUUA6_TDv3LDZBJgaA-G2ZbRfWA-2cGajgU_MHmTiXVEouMmdPN53mMMXxYd1nvCWWJJvfu5Mmg4Ca>
>>>  
>>>
>>> 在 2016年9月29日星期四 UTC+8上午1:39:09，Chris Manghane写道：
>>>>
>>>> In the first example, make does not escape the scope of the for 
>>>> statement. In the second example, make is assigned to m, which is outside 
>>>> of the scope of the for statement, which means the make operation escapes 
>>>> its scope and subsequently, is heap allocated. If you want more 
>>>> information 
>>>> about why something escapes, try compiling with -gcflags "-m -m" for an 
>>>> explanation of the escape analysis information.
>>>>
>>>> On Wed, Sep 28, 2016 at 7:56 AM, 刘桂祥  wrote:
>>>>
>>>>> go 1.7
>>>>>
>>>>> 在 2016年9月28日星期三 UTC+8下午10:41:09，Dave Cheney写道：
>>>>>
>>>>>> Which version of Go?
>>>>>>
>>>>>> On Thursday, 29 September 2016 00:18:29 UTC+10, 刘桂祥 wrote:
>>>>>>>
>>>>>>> // example1.go
>>>>>>> package main
>>>>>>>
>>>>>>>
>>>>>>> func main() {
>>>>>>>
>>>>>>> for i := 0; i < 2; i++ {
>>>>>>> m := make(map[int]int) 
>>>>>>> 
>>>>>>> 
>>>>>>>
>>>>>>> m[1] = 100 
>>>>>>> }   
>>>>>>>
>>>>>>>
>>>>>>> }
>>>>>>>
>>>>>>>
>>>>>>> main make(map[int]int) does not escape
>>>>>>>
>>>>>>>
>>>>>>> // example2.go
>>>>>>> package main
>>>>>>>
>>>>>>>
>>>>>>> func main() {
>>>>>>> var m map[int]int
>>>>>>> for i := 0; i < 2; i++ {
>>>>>>> m = make(map[int]int)   
>>>>>>> 
>>>>>>> 
>>>>>>>  
>>>>>>> m[1] = 100 
>>>>>>> }   
>>>>>>>
>>>>>>>
>>>>>>> }
>>>>>>>
>>>>>>>  make(map[int]int) escapes to heapwhy ???
>>>>>>>
>>>>>>>
>>>>>>> -- 
>>>>> You received this message because you are subscribed to the Google 
>>>>> Groups "golang-nuts" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send 
>>>>> an email to golang-nuts...@googlegroups.com.
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>>
>>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to golang-nuts...@googlegroups.com .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] about interface data pointer

[刘桂祥]

package main


import (
"fmt"
)
func main() {
a := interface{}(100)
println(&a, a)
fmt.Sprint(a)
}

go run x.go  the output:

0xc42003bf18 (0x89040,0xc42000a2c0)

I just want to ask 0xc42000a2c0 as the a (interface{}) 's data pointer is 
point to where ? heap ?

also I test this:
func BenchmarkFmtEscap3(b *testing.B) {
b.ReportAllocs()


for i := 0; i < b.N; i++ {
a := interface{}(100)
fmt.Sprint(a)
}
}

BenchmarkFmtEscap3-4 1000   141 ns/op  16 B/op   2 
allocs/op

why is 2 allocs and 16bytes ??

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: about interface data pointer

[刘桂祥]

  Yes  you are right . thanks a lot and I try more debug info and have 
looked that:


func main() {
println("debug")
a := interface{}(100)
println(&a, a)
b := fmt.Sprint(a)
println("debug1", b, &b, Pointer(*((*uintptr)(Pointer(&b, *((*int)(
Pointer(uintptr(Pointer(&b)) + 8
}

and add debug info to runtime/mallocgc.go file
here is the output:

debug
mallocgc: 8
mallocgc return2: 0xc20800a220
0xc208039f68 (0xa00a0,0xc20800a220)
mallocgc: 3
mallocgc return2: 0xc20800a230
debug1 100 0xc208039f48 0xc20800a230 3

在 2016年10月3日星期一 UTC+8上午9:47:18，Dave Cheney写道：
>
> On Monday, 3 October 2016 12:39:34 UTC+11, 刘桂祥  wrote: 
> > package main 
> > 
> > 
> > import ( 
> > "fmt" 
> > ) 
> > func main() { 
> > a := interface{}(100) 
> > println(&a, a) 
> > fmt.Sprint(a) 
> > } 
> > 
> > 
> > go run x.go  the output: 
> > 
> > 
> > 0xc42003bf18 (0x89040,0xc42000a2c0) 
> > 
> > 
> > 
> > I just want to ask 0xc42000a2c0 as the a (interface{}) 's data pointer 
> is point to where ? heap ? 
>
> It is on the heap, the value escapes when passed to fmt,Sprint 
>
> > 
> > also I test this: 
> > 
> > 
> > func BenchmarkFmtEscap3(b *testing.B) { 
> > b.ReportAllocs() 
> > 
> > 
> > for i := 0; i < b.N; i++ { 
> > a := interface{}(100) 
> > fmt.Sprint(a) 
> > } 
> > } 
> > 
> > 
> > BenchmarkFmtEscap3-41000   141 ns/op 
>  16 B/op   2 allocs/op 
> > 
> > 
> > 
> > why is 2 allocs and 16bytes ?? 
>
> One for a, which escapes to the heap when passed to fmt.Sprint, and one 
> for the value returned from fmt,Sprint 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] go closure escape analysis

[刘桂祥]

// example1.go

package main


func main() {
 var y int // BAD: y escapes
 func(p *int, x int) {
 *p = x
 }(&y, 42)
}

example1.go  y escape to the heap

// example2.go

package main

func main() {
 var y int
 func(x int) {
 y = x
 }(42)
}

  example2.go y is not escaped and why ??

// example3.go

package main

func main() {
 a := 100
 y := &a
 func(x int) {
 *y = x
 }(42)
}

 example3.go y is not escaped and why ??

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] go closure escape analysis

[刘桂祥]

  In go when closure call the out varible will pass  pointer to the closure 
func so I doubt in example2.go y is passed &y to closure 

  I don't know the compile how to analysis this and decide that can be 
allocated in stack correctly ? 

在 2016年10月4日星期二 UTC+8下午11:54:02，Chris Manghane写道：
>
> In example1, the // BAD: y escapes is there because y should not escape 
> from that function, but the current algorithm needs to be improved for this 
> case. In that closure, the address of y is captured as the closure argument 
> p and since p is only dereferenced, the analysis should not consider it to 
> escape.
>
> In example3, the analysis above applies and y should not escape. y is 
> closed over and dereferenced, but we correctly do not mark it as escaping.
>
> In example2, y contains no pointers; it can never escape. The test was to 
> see if capturing/closing over a non-pointer value would cause it to escape.
>
> It's a bit confusing, but to restate, in example1, y should not escape 
> even though it currently does.
>
> On Mon, Oct 3, 2016 at 11:09 PM, 刘桂祥 > 
> wrote:
>
>> // example1.go
>>
>> package main
>>
>>
>> func main() {
>>  var y int // BAD: y escapes
>>  func(p *int, x int) {
>>  *p = x
>>  }(&y, 42)
>> }
>>
>> example1.go  y escape to the heap
>>
>> // example2.go
>>
>> package main
>>
>> func main() {
>>  var y int
>>  func(x int) {
>>  y = x
>>  }(42)
>> }
>>
>>   example2.go y is not escaped and why ??
>>
>> // example3.go
>>
>> package main
>>
>> func main() {
>>  a := 100
>>  y := &a
>>  func(x int) {
>>  *y = x
>>  }(42)
>> }
>>
>>  example3.go y is not escaped and why ??
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] when the runtime decide to threadcreate

[刘桂祥]

package main


import (
"bufio"
"net/http"
_ "net/http/pprof"
"os"
"time"
)


func main() {
go func() {
http.ListenAndServe(":8080", nil)
}()
reader := bufio.NewReader(os.Stdin)
for i := 0; i < 100; i++ {
go func(id int) {
for {
data, _, _ := reader.ReadLine()
println(id, string(data))
}
}(i)
time.Sleep(1 * time.Second)
}
println("create finish!")
time.Sleep(5 * time.Second)
}

  I use curl http://localhost:8080/debug/pprof/threadcreate   see that the 
theadcreate num is always grow.
  want to ask when the runtime decide to threadcreate : long time not 
return syscall ? any others ??

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] go closure escape analysis

[刘桂祥]

  1: In example2_modified.go (y=x line should be *y=x ?) and that is the 
same as example1.go ???
   but in example1.go y is escapted and example2.go is not.
 2:how do I see the compile handle closure call results ? compile para ? 

在 2016年10月5日星期三 UTC+8下午11:38:42，Chris Manghane写道：
>
> Sorry, poor explanation again.
>
> When a variable is used from outside a closure's scope, it is *captured* 
> by the closure. Usually that means that the closure function is modified to 
> include a reference to that variable when it is called as a regular 
> function. An example from the compiler: 
> https://github.com/golang/go/blob/master/src/cmd/compile/internal/gc/closure.go#L311
> .
>
> In example2, this
>
> // example2.go
>
> package main
>
> func main() {
>  var y int
>  func(x int) {
>  y = x
>  }(42)
> }
>
> is transformed into something like
>
> // example2_modified.go
>
> package main
>
> func main() {
>  var y int
>  func(y *int, x int) {
>  y = x
>  }(&y, 42)
> }
>
> and then the same analysis from above is applied. y should not escape to 
> the heap here, similar to example3.go.
>
> Hope that clarifies,
> Chris
>
> On Wed, Oct 5, 2016 at 6:18 AM, 刘桂祥 > 
> wrote:
>
>>   In go when closure call the out varible will pass  pointer to the 
>> closure func so I doubt in example2.go y is passed &y to closure 
>>
>>   I don't know the compile how to analysis this and decide that can be 
>> allocated in stack correctly ? 
>>
>> 在 2016年10月4日星期二 UTC+8下午11:54:02，Chris Manghane写道：
>>>
>>> In example1, the // BAD: y escapes is there because y should not escape 
>>> from that function, but the current algorithm needs to be improved for this 
>>> case. In that closure, the address of y is captured as the closure argument 
>>> p and since p is only dereferenced, the analysis should not consider it to 
>>> escape.
>>>
>>> In example3, the analysis above applies and y should not escape. y is 
>>> closed over and dereferenced, but we correctly do not mark it as escaping.
>>>
>>> In example2, y contains no pointers; it can never escape. The test was 
>>> to see if capturing/closing over a non-pointer value would cause it to 
>>> escape.
>>>
>>> It's a bit confusing, but to restate, in example1, y should not escape 
>>> even though it currently does.
>>>
>>> On Mon, Oct 3, 2016 at 11:09 PM, 刘桂祥  wrote:
>>>
>>>> // example1.go
>>>>
>>>> package main
>>>>
>>>>
>>>> func main() {
>>>>  var y int // BAD: y escapes
>>>>  func(p *int, x int) {
>>>>  *p = x
>>>>  }(&y, 42)
>>>> }
>>>>
>>>> example1.go  y escape to the heap
>>>>
>>>> // example2.go
>>>>
>>>> package main
>>>>
>>>> func main() {
>>>>  var y int
>>>>  func(x int) {
>>>>  y = x
>>>>  }(42)
>>>> }
>>>>
>>>>   example2.go y is not escaped and why ??
>>>>
>>>> // example3.go
>>>>
>>>> package main
>>>>
>>>> func main() {
>>>>  a := 100
>>>>  y := &a
>>>>  func(x int) {
>>>>  *y = x
>>>>  }(42)
>>>> }
>>>>
>>>>  example3.go y is not escaped and why ??
>>>>
>>>> -- 
>>>> You received this message because you are subscribed to the Google 
>>>> Groups "golang-nuts" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send 
>>>> an email to golang-nuts...@googlegroups.com.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] go closure escape analysis

[刘桂祥]

  ok  very thanks!

在 2016年10月6日星期四 UTC+8上午7:49:03，Chris Manghane写道：
>
> To see the escape analysis results, compile with -gcflags "-m -m -m -m".
> The code in each of the example should be relatively similar, which is why 
> it is a bug that one of them causes the variable to escape to the heap.
>
> On Wed, Oct 5, 2016 at 4:46 PM, 刘桂祥 > 
> wrote:
>
>>   1: In example2_modified.go (y=x line should be *y=x ?) and that is the 
>> same as example1.go ???
>>but in example1.go y is escapted and example2.go is not.
>>  2:how do I see the compile handle closure call results ? compile para ? 
>>
>> 在 2016年10月5日星期三 UTC+8下午11:38:42，Chris Manghane写道：
>>>
>>> Sorry, poor explanation again.
>>>
>>> When a variable is used from outside a closure's scope, it is *captured* 
>>> by the closure. Usually that means that the closure function is modified to 
>>> include a reference to that variable when it is called as a regular 
>>> function. An example from the compiler: 
>>> https://github.com/golang/go/blob/master/src/cmd/compile/internal/gc/closure.go#L311
>>> .
>>>
>>> In example2, this
>>>
>>> // example2.go
>>>
>>> package main
>>>
>>> func main() {
>>>  var y int
>>>  func(x int) {
>>>  y = x
>>>  }(42)
>>> }
>>>
>>> is transformed into something like
>>>
>>> // example2_modified.go
>>>
>>> package main
>>>
>>> func main() {
>>>  var y int
>>>  func(y *int, x int) {
>>>  y = x
>>>  }(&y, 42)
>>> }
>>>
>>> and then the same analysis from above is applied. y should not escape to 
>>> the heap here, similar to example3.go.
>>>
>>> Hope that clarifies,
>>> Chris
>>>
>>> On Wed, Oct 5, 2016 at 6:18 AM, 刘桂祥  wrote:
>>>
>>>>   In go when closure call the out varible will pass  pointer to the 
>>>> closure func so I doubt in example2.go y is passed &y to closure 
>>>>
>>>>   I don't know the compile how to analysis this and decide that can be 
>>>> allocated in stack correctly ? 
>>>>
>>>> 在 2016年10月4日星期二 UTC+8下午11:54:02，Chris Manghane写道：
>>>>>
>>>>> In example1, the // BAD: y escapes is there because y should not 
>>>>> escape from that function, but the current algorithm needs to be improved 
>>>>> for this case. In that closure, the address of y is captured as the 
>>>>> closure 
>>>>> argument p and since p is only dereferenced, the analysis should not 
>>>>> consider it to escape.
>>>>>
>>>>> In example3, the analysis above applies and y should not escape. y is 
>>>>> closed over and dereferenced, but we correctly do not mark it as escaping.
>>>>>
>>>>> In example2, y contains no pointers; it can never escape. The test was 
>>>>> to see if capturing/closing over a non-pointer value would cause it to 
>>>>> escape.
>>>>>
>>>>> It's a bit confusing, but to restate, in example1, y should not escape 
>>>>> even though it currently does.
>>>>>
>>>>> On Mon, Oct 3, 2016 at 11:09 PM, 刘桂祥  wrote:
>>>>>
>>>>>> // example1.go
>>>>>>
>>>>>> package main
>>>>>>
>>>>>>
>>>>>> func main() {
>>>>>>  var y int // BAD: y escapes
>>>>>>  func(p *int, x int) {
>>>>>>  *p = x
>>>>>>  }(&y, 42)
>>>>>> }
>>>>>>
>>>>>> example1.go  y escape to the heap
>>>>>>
>>>>>> // example2.go
>>>>>>
>>>>>> package main
>>>>>>
>>>>>> func main() {
>>>>>>  var y int
>>>>>>  func(x int) {
>>>>>>  y = x
>>>>>>  }(42)
>>>>>> }
>>>>>>
>>>>>>   example2.go y is not escaped and why ??
>>>>>>
>>>>>> // example3.go
>>>>>>
>>>>>> package main
>>>>>>
>>>>>> func main() {
>>>>>>  a := 100
>>>>>>  y := &a
>>>>>>  func(x int) {
>>>>>>  *y = x
>>>>>>  }(42)
>>>>>> }
>>>>>>
>>>>>>  example3.go y is not escaped and why ??
>>>>>>
>>>>>> -- 
>>>>>> You received this message because you are subscribed to the Google 
>>>>>> Groups "golang-nuts" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it, 
>>>>>> send an email to golang-nuts...@googlegroups.com.
>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>
>>>>>
>>>>> -- 
>>>> You received this message because you are subscribed to the Google 
>>>> Groups "golang-nuts" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send 
>>>> an email to golang-nuts...@googlegroups.com.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] golang memory allocation

[刘桂祥]

package main


import (
"fmt"
_ "net/http/pprof"
"testing"
)


// 8 Bytes/op   1 allocs/op
func BenchmarkFmt1(b *testing.B) {   
 b.ReportAllocs()
for i := 0; i < b.N; i++ {
fmt.Println(100)
}
}


// 16 Bytes/op   1 allocs/op
func BenchmarkFmt2(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
fmt.Println("hello")
}
}


// 32 Bytes/op   1 allocs/op
func BenchmarkFmt3(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
fmt.Println([]int{})
  

  }
}


// 48 Bytes/op   3 allocs/op
func BenchmarkFmt4(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
fmt.Println([]int{100})
}
}


// 64 Bytes/op   4 allocs/op
func BenchmarkFmt5(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
fmt.Println([]int{100, 200})
}
}

 go version 1.7
 
 Fmt1 : the runtime alloc 100 in heap
 Fmt2:  the runtime alloc a string header (addr len) in heap 
 Fmt3:  why the runtime alloc 32 bytes for slice header ??
 Fmt4:  why have 3 allocs and 48 bytes ??
 Fmt5:  why have 4 allocs and 64 bytes ??

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] golang too many json unmarshal trigger gc frequency to many

[刘桂祥]

  In my server many requests need to handle a loop to unmarshal json 
strings and I find gc is very frequency 



-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] golang closure variable

[刘桂祥]

In golang closure func ,the needed outer variable is passed by reference 

















but when the variable self is a reference in example.go , the closure func 
paras is *[&map[int]int,...]  is so ?

and I doubt what kind of datatype the paras in closure func?

// example.go

package main


import "time"


func main() {
 m := map[int]int{1: 100, 2: 200}
 println(&m, m)
 go func() {
 time.Sleep(1 * time.Second)
 println(m[1])
 }()
 m = map[int]int{3: 300, 4: 400}
 time.Sleep(2 * time.Second)
}

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] golang closure variable

[刘桂祥]

 

I just want to look what variable datastructure is passed to the closure 
func   but I am confused with the assemble code 


// example.go

package main


import "time"


func main() {

s := []int{100, 200}

func() {

s[0] = 300

s = []int{300, 400}

}()

time.Sleep(1 * time.Second)

}



And I am confused with the assemble code 


"".main t=1 size=113 args=0x0 locals=0x38

0x 0 (main.go:5) TEXT "".main(SB), $56-0

0x 0 (main.go:5) MOVQ (TLS), CX

0x0009 9 (main.go:5) CMPQ SP, 16(CX)

0x000d 00013 (main.go:5) JLS 106

0x000f 00015 (main.go:5) SUBQ $56, SP

0x0013 00019 (main.go:5) MOVQ BP, 48(SP)

0x0018 00024 (main.go:5) LEAQ 48(SP), BP

0x001d 00029 (main.go:5) FUNCDATA $0, 
gclocals·69c1753bd5f81501d95132d08af04464(SB)

0x001d 00029 (main.go:5) FUNCDATA $1, 
gclocals·2c033e7f4f4a74cc7e9f368d1fec9f60(SB)

0x001d 00029 (main.go:6) MOVUPS "".statictmp_0(SB), X0

0x0024 00036 (main.go:6) MOVUPS X0, "".autotmp_2+8(SP)

0x0029 00041 (main.go:6) LEAQ "".autotmp_2+8(SP), AX

0x002e 00046 (main.go:6) MOVQ AX, "".s+24(SP)

0x0033 00051 (main.go:6) MOVQ $2, "".s+32(SP)

0x003c 00060 (main.go:6) MOVQ $2, "".s+40(SP)

0x0045 00069 (main.go:6) LEAQ "".s+24(SP), AX

0x004a 00074 (main.go:10) MOVQ AX, (SP)

0x004e 00078 (main.go:10) PCDATA $0, $1

0x004e 00078 (main.go:10) CALL "".main.func1(SB)

0x0053 00083 (main.go:11) MOVQ $10, (SP)

0x005b 00091 (main.go:11) PCDATA $0, $1

0x005b 00091 (main.go:11) CALL time.Sleep(SB)

0x0060 00096 (main.go:12) MOVQ 48(SP), BP

0x0065 00101 (main.go:12) ADDQ $56, SP

0x0069 00105 (main.go:12) RET

0x006a 00106 (main.go:12) NOP

0x006a 00106 (main.go:5) CALL runtime.morestack_noctxt(SB)

0x006f 00111 (main.go:5) JMP 0

在 2016年11月3日星期四 UTC+8下午2:43:40，Ian Lance Taylor写道：
>
> On Wed, Nov 2, 2016 at 8:03 PM, 刘桂祥 > 
> wrote: 
> > 
> > In golang closure func ,the needed outer variable is passed by reference 
> > 
> > but when the variable self is a reference in example.go , the closure 
> func 
> > paras is *[&map[int]int,...]  is so ? 
>
> I'm not completely sure what you are asking, but I think the answer is 
> yes. 
>
> > and I doubt what kind of datatype the paras in closure func? 
>
> Each function that has a closure has a custom type for the closure 
> variable.  It's essentially a struct each of whose fields is a pointer 
> to the type of the variable being closed over. 
>
> Ian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] golang closure variable

[刘桂祥]

package main

import "time"

func main() {
s := []int{100, 200}
println(&s)
go func() {
s[0] = 300
s = []int{300, 400}
}()
time.Sleep(1 * time.Second)
}

just curious about this ,can you help explain the assemble code here 

"".main t=1 size=241 args=0x0 locals=0x28
0x 0 (main.go:5) TEXT "".main(SB), $40-0
0x 0 (main.go:5) MOVQ (TLS), CX
0x0009 9 (main.go:5) CMPQ SP, 16(CX)
0x000d 00013 (main.go:5) JLS 231
0x0013 00019 (main.go:5) SUBQ $40, SP
0x0017 00023 (main.go:5) MOVQ BP, 32(SP)
0x001c 00028 (main.go:5) LEAQ 32(SP), BP
0x0021 00033 (main.go:5) FUNCDATA $0, 
gclocals·69c1753bd5f81501d95132d08af04464(SB)
0x0021 00033 (main.go:5) FUNCDATA $1, 
gclocals·0c8aa8e80191a30eac23f1a218103f16(SB)
0x0021 00033 (main.go:6) LEAQ type.[]int(SB), AX
0x0028 00040 (main.go:6) MOVQ AX, (SP)
0x002c 00044 (main.go:6) PCDATA $0, $0
0x002c 00044 (main.go:6) CALL runtime.newobject(SB)
0x0031 00049 (main.go:6) MOVQ 8(SP), AX
0x0036 00054 (main.go:6) MOVQ AX, "".&s+24(SP)
0x003b 00059 (main.go:6) LEAQ type.[2]int(SB), CX
0x0042 00066 (main.go:6) MOVQ CX, (SP)
0x0046 00070 (main.go:6) PCDATA $0, $1
0x0046 00070 (main.go:6) CALL runtime.newobject(SB)
0x004b 00075 (main.go:6) MOVQ 8(SP), AX
0x0050 00080 (main.go:6) MOVUPS "".statictmp_2(SB), X0
0x0057 00087 (main.go:6) MOVUPS X0, (AX)
0x005a 00090 (main.go:6) MOVQ "".&s+24(SP), CX
0x005f 00095 (main.go:6) MOVQ $2, 8(CX)
0x0067 00103 (main.go:6) MOVQ $2, 16(CX)
0x006f 00111 (main.go:6) MOVL runtime.writeBarrier(SB), DX
0x0075 00117 (main.go:6) TESTB DL, DL
0x0077 00119 (main.go:6) JNE $0, 210
0x0079 00121 (main.go:6) MOVQ AX, (CX)
0x007c 00124 (main.go:7) PCDATA $0, $1
0x007c 00124 (main.go:7) CALL runtime.printlock(SB)
0x0081 00129 (main.go:7) MOVQ "".&s+24(SP), AX
0x0086 00134 (main.go:7) MOVQ AX, (SP)
0x008a 00138 (main.go:7) PCDATA $0, $1
0x008a 00138 (main.go:7) CALL runtime.printpointer(SB)
0x008f 00143 (main.go:7) PCDATA $0, $1
0x008f 00143 (main.go:7) CALL runtime.printnl(SB)
0x0094 00148 (main.go:7) PCDATA $0, $1
0x0094 00148 (main.go:7) CALL runtime.printunlock(SB)
0x0099 00153 (main.go:11) MOVQ "".&s+24(SP), AX
0x009e 00158 (main.go:11) MOVQ AX, 16(SP)
0x00a3 00163 (main.go:11) MOVL $8, (SP)
0x00aa 00170 (main.go:11) LEAQ "".main.func1·f(SB), AX
0x00b1 00177 (main.go:11) MOVQ AX, 8(SP)
0x00b6 00182 (main.go:11) PCDATA $0, $0
0x00b6 00182 (main.go:11) CALL runtime.newproc(SB)
0x00bb 00187 (main.go:12) MOVQ $10, (SP)
0x00c3 00195 (main.go:12) PCDATA $0, $0
0x00c3 00195 (main.go:12) CALL time.Sleep(SB)
0x00c8 00200 (main.go:13) MOVQ 32(SP), BP
0x00cd 00205 (main.go:13) ADDQ $40, SP
0x00d1 00209 (main.go:13) RET
0x00d2 00210 (main.go:6) MOVQ CX, (SP)
0x00d6 00214 (main.go:6) MOVQ AX, 8(SP)
0x00db 00219 (main.go:6) PCDATA $0, $1
0x00db 00219 (main.go:6) CALL runtime.writebarrierptr(SB)
0x00e0 00224 (main.go:6) MOVQ "".&s+24(SP), CX
0x00e5 00229 (main.go:7) JMP 124
0x00e7 00231 (main.go:7) NOP
0x00e7 00231 (main.go:5) CALL runtime.morestack_noctxt(SB)
0x00ec 00236 (main.go:5) JMP 0


在 2016年11月3日星期四 UTC+8下午2:56:10，Ian Lance Taylor写道：
>
> On Wed, Nov 2, 2016 at 11:48 PM, 刘桂祥 > 
> wrote: 
> > I just want to look what variable datastructure is passed to the closure 
> > func   but I am confused with the assemble code 
> > 
> > 
> > // example.go 
> > 
> > package main 
> > 
> > 
> > import "time" 
> > 
> > 
> > func main() { 
> > 
> > s := []int{100, 200} 
> > 
> > func() { 
> > 
> > s[0] = 300 
> > 
> > s = []int{300, 400} 
> > 
> > }() 
> > 
> > time.Sleep(1 * time.Second) 
> > 
> > } 
>
> In this example you are calling the function directly, so the compiler 
> doesn't use a closure at all.  It simply passes the closure values as 
> arguments to the function.  There is no need for an actual closure 
> when you always call the function. 
>
> Ian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Multiple-reader single-writer map access is safe ?

[刘桂祥]

// example.go

package main

var gMap = make(map[int]int)

func w() {
temp := make(map[int]int)
temp[1] = 100
temp[2] = 200
gMap = temp// Does the compiler or cpu will reorder temp[1]=100, 
temp[2]=200, gMap=temp ??
}

func r() {
local := gMap
println(local[1], local[2])
}

func main() {

go w()
go r()

// ...
}

I have one goroutine to read the map and one goroutine to rewrite the 
global map variable does this safe ??




-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Multiple-reader single-writer map access - is this lockless workaround safe?

[刘桂祥]

sorry could you provide a complete example ?

I try this  but not find question
package main


import "time"


type T struct{ x int }


var global *T


func f() {
 p := new(T)
 p.x = 1
 global = p // "publish" the new T (racy!)
}


func g() {
 p := global
 if p != nil {
 // println(p.x) // may print "0" due to data race
 if p.x == 0 {
 panic("ca")
 }
 }
}
func main() {
 for i := 0; i < 1; i++ {
 go func() {
 for j := 0; j < 1000; j++ {
 f()
 }
 }()
 go func() {
 for j := 0; j < 1000; j++ {
 g()
 }
 }()
 }
 time.Sleep(100 * time.Second)
}


在 2016年9月13日星期二 UTC+8上午5:28:37，adon...@google.com写道：
>
> On Monday, 12 September 2016 12:04:30 UTC-4, sqweek E. wrote:
>>
>> Yes, through plain assignment. What problems arise from that?
>>
>
> Here's the general form of the problem.  In the code below, one goroutine 
> (f) creates a variable, initializes it (x=1), then shares its address with 
> another goroutine (in this case by assigning to a global variable, but 
> that's a detail).  Another goroutine (g) reads the pointer out of the 
> global variable and inspects the variable to which it points.
>
> type T struct { x int }
>
> var global *T
>
> func f() {
> p := new(T)
> p.x = 1
> global = p // "publish" the new T (racy!)
> }
>
> func g() {
> p = global
> if p != nil {
> println(p.x) // may print "0" due to data race
> }
> }
>
> go f()
> go g()
>
> One might naively think that the print statement always prints 1, but in 
> fact it can print 0.  The reason is that the compiler or CPU is free to 
> reorder the assignments p.x=1 and global=p, since f can't tell.  But 
> reordering means other goroutines might observe a non-nil pointer in global 
> before the thing it points to has been fully initialized.
>
> To safely "publish" a variable initialized by one goroutine so that other 
> goroutines see it properly initialized, you need "barrier semantics", that 
> is, you need to tell the compiler and CPU not to reorder those assignments. 
> That's what atomic.Value does.
>
> But you shouldn't reach for atomic.Value without data.  In most programs, 
> a mutex is simpler to reason about and plenty fast enough.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Multiple-reader single-writer map access is safe ?

[刘桂祥]

  can you explain why whis ? 

在 2016年11月4日星期五 UTC+8下午1:16:39，Ian Lance Taylor写道：
>
> On Thu, Nov 3, 2016 at 8:37 PM, 刘桂祥 > 
> wrote: 
> > // example.go 
> > 
> > package main 
> > 
> > var gMap = make(map[int]int) 
> > 
> > func w() { 
> > temp := make(map[int]int) 
> > temp[1] = 100 
> > temp[2] = 200 
> > gMap = temp// Does the compiler or cpu will reorder temp[1]=100, 
> > temp[2]=200, gMap=temp ?? 
> > } 
> > 
> > func r() { 
> > local := gMap 
> > println(local[1], local[2]) 
> > } 
> > 
> > func main() { 
> > 
> > go w() 
> > go r() 
> > 
> > // ... 
> > } 
> > 
> > I have one goroutine to read the map and one goroutine to rewrite the 
> global 
> > map variable does this safe ?? 
>
> No.  Use a lock. 
>
> Ian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Multiple-reader single-writer map access is safe ?

[刘桂祥]

 In my write goroutine I don't modify the map key (data structure) but set 
the variable point to another map address  ,this is a pointer assignment 
and is atomic 

在 2016年11月4日星期五 UTC+8下午1:22:20，Ian Lance Taylor写道：
>
> On Thu, Nov 3, 2016 at 10:19 PM, 刘桂祥 > 
> wrote: 
> >   can you explain why whis ? 
>
> A map is basically a pointer to a complex data structure.  Setting a 
> value in a map changes that data structure.  If one goroutine is 
> reading from the data structure while a different goroutine is writing 
> to the data structure, the results are completely unpredictable.  In 
> the worst case they could even cause the program to crash. 
>
> Ian 
>
>
> > 在 2016年11月4日星期五 UTC+8下午1:16:39，Ian Lance Taylor写道： 
> >> 
> >> On Thu, Nov 3, 2016 at 8:37 PM, 刘桂祥  wrote: 
> >> > // example.go 
> >> > 
> >> > package main 
> >> > 
> >> > var gMap = make(map[int]int) 
> >> > 
> >> > func w() { 
> >> > temp := make(map[int]int) 
> >> > temp[1] = 100 
> >> > temp[2] = 200 
> >> > gMap = temp// Does the compiler or cpu will reorder 
> temp[1]=100, 
> >> > temp[2]=200, gMap=temp ?? 
> >> > } 
> >> > 
> >> > func r() { 
> >> > local := gMap 
> >> > println(local[1], local[2]) 
> >> > } 
> >> > 
> >> > func main() { 
> >> > 
> >> > go w() 
> >> > go r() 
> >> > 
> >> > // ... 
> >> > } 
> >> > 
> >> > I have one goroutine to read the map and one goroutine to rewrite the 
> >> > global 
> >> > map variable does this safe ?? 
> >> 
> >> No.  Use a lock. 
> >> 
> >> Ian 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups 
> > "golang-nuts" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an 
> > email to golang-nuts...@googlegroups.com . 
> > For more options, visit https://groups.google.com/d/optout. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Multiple-reader single-writer map access is safe ?

[刘桂祥]

   very thanks ,  I am unfamiliar with the cpu *out-of-order execution , 
but I doubt if my three line code is related ,it will reorder it ??*

在 2016年11月4日星期五 UTC+8下午1:38:12，Ian Lance Taylor写道：
>
> On Thu, Nov 3, 2016 at 10:28 PM, 刘桂祥 > 
> wrote: 
> > 
> >  In my write goroutine I don't modify the map key (data structure) but 
> set 
> > the variable point to another map address  ,this is a pointer assignment 
> and 
> > is atomic 
>
> You're right, my previous reply was incorrect.  My apologies.  The 
> real reason your code is incorrect is more subtle. 
>
> A map is a pointer to a complex data structure.  Setting values in a 
> map is a series of memory writes.  From the point of view of the 
> processor doing the writes, they all appear in order.  From the 
> pointer of view of a different processor, they need not.  It is 
> possible the the other processor to see the final write, to gMap, 
> before it sees the other writes, setting the values in the map.  The 
> read from gMap will then be trying to access an incomplete data 
> structure. 
>
> In order to force all the writes to be visible to the reading 
> processor, you need to use a sync.Mutex, a channel operation, or a 
> sync/atomic.Store function. 
>
> There is a simple rule to follow when it comes to communicating 
> between processes: always use channels or locks.  Don't try to be 
> clever.  Please read 
>
> https://software.intel.com/en-us/blogs/2013/01/06/benign-data-races-what-could-possibly-go-wrong.
>  
>
>
> Ian 
>
> > 在 2016年11月4日星期五 UTC+8下午1:22:20，Ian Lance Taylor写道： 
> >> 
> >> On Thu, Nov 3, 2016 at 10:19 PM, 刘桂祥  wrote: 
> >> >   can you explain why whis ? 
> >> 
> >> A map is basically a pointer to a complex data structure.  Setting a 
> >> value in a map changes that data structure.  If one goroutine is 
> >> reading from the data structure while a different goroutine is writing 
> >> to the data structure, the results are completely unpredictable.  In 
> >> the worst case they could even cause the program to crash. 
> >> 
> >> Ian 
> >> 
> >> 
> >> > 在 2016年11月4日星期五 UTC+8下午1:16:39，Ian Lance Taylor写道： 
> >> >> 
> >> >> On Thu, Nov 3, 2016 at 8:37 PM, 刘桂祥  wrote: 
> >> >> > // example.go 
> >> >> > 
> >> >> > package main 
> >> >> > 
> >> >> > var gMap = make(map[int]int) 
> >> >> > 
> >> >> > func w() { 
> >> >> > temp := make(map[int]int) 
> >> >> > temp[1] = 100 
> >> >> > temp[2] = 200 
> >> >> > gMap = temp// Does the compiler or cpu will reorder 
> >> >> > temp[1]=100, 
> >> >> > temp[2]=200, gMap=temp ?? 
> >> >> > } 
> >> >> > 
> >> >> > func r() { 
> >> >> > local := gMap 
> >> >> > println(local[1], local[2]) 
> >> >> > } 
> >> >> > 
> >> >> > func main() { 
> >> >> > 
> >> >> > go w() 
> >> >> > go r() 
> >> >> > 
> >> >> > // ... 
> >> >> > } 
> >> >> > 
> >> >> > I have one goroutine to read the map and one goroutine to rewrite 
> the 
> >> >> > global 
> >> >> > map variable does this safe ?? 
> >> >> 
> >> >> No.  Use a lock. 
> >> >> 
> >> >> Ian 
> >> > 
> >> > -- 
> >> > You received this message because you are subscribed to the Google 
> >> > Groups 
> >> > "golang-nuts" group. 
> >> > To unsubscribe from this group and stop receiving emails from it, 
> send 
> >> > an 
> >> > email to golang-nuts...@googlegroups.com. 
> >> > For more options, visit https://groups.google.com/d/optout. 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups 
> > "golang-nuts" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an 
> > email to golang-nuts...@googlegroups.com . 
> > For more options, visit https://groups.google.com/d/optout. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] Multiple-reader single-writer map access is safe ?

[刘桂祥]

should I use ??

mu.Lock()
temp := make(map[int]int)
temp[1] = 100
temp[2] = 200
gMap = temp 
mu.Unlock()

or

temp := make(map[int]int)
temp[1] = 100
temp[2] = 200
mu.Lock()
gMap = temp
mu.Unlock() 

在 2016年11月4日星期五 UTC+8下午9:59:45，Ian Lance Taylor写道：
>
> On Thu, Nov 3, 2016 at 11:30 PM, 刘桂祥 > 
> wrote: 
> > 
> >very thanks ,  I am unfamiliar with the cpu out-of-order execution , 
> but 
> > I doubt if my three line code is related ,it will reorder it ?? 
>
> I want to be clear that the real problem is not out-of-order 
> execution, it is memory ordering.  On modern multi-core processors 
> there is no guarantee that one core will see memory writes in the 
> order they are done by a different core.  Making that guarantee 
> requires an explicit memory barrier instruction, and there is no such 
> instruction in your example program. 
>
> Ian 
>
>
> > 在 2016年11月4日星期五 UTC+8下午1:38:12，Ian Lance Taylor写道： 
> >> 
> >> On Thu, Nov 3, 2016 at 10:28 PM, 刘桂祥  wrote: 
> >> > 
> >> >  In my write goroutine I don't modify the map key (data structure) 
> but 
> >> > set 
> >> > the variable point to another map address  ,this is a pointer 
> assignment 
> >> > and 
> >> > is atomic 
> >> 
> >> You're right, my previous reply was incorrect.  My apologies.  The 
> >> real reason your code is incorrect is more subtle. 
> >> 
> >> A map is a pointer to a complex data structure.  Setting values in a 
> >> map is a series of memory writes.  From the point of view of the 
> >> processor doing the writes, they all appear in order.  From the 
> >> pointer of view of a different processor, they need not.  It is 
> >> possible the the other processor to see the final write, to gMap, 
> >> before it sees the other writes, setting the values in the map.  The 
> >> read from gMap will then be trying to access an incomplete data 
> >> structure. 
> >> 
> >> In order to force all the writes to be visible to the reading 
> >> processor, you need to use a sync.Mutex, a channel operation, or a 
> >> sync/atomic.Store function. 
> >> 
> >> There is a simple rule to follow when it comes to communicating 
> >> between processes: always use channels or locks.  Don't try to be 
> >> clever.  Please read 
> >> 
> >> 
> https://software.intel.com/en-us/blogs/2013/01/06/benign-data-races-what-could-possibly-go-wrong.
>  
>
> >> 
> >> Ian 
> >> 
> >> > 在 2016年11月4日星期五 UTC+8下午1:22:20，Ian Lance Taylor写道： 
> >> >> 
> >> >> On Thu, Nov 3, 2016 at 10:19 PM, 刘桂祥  wrote: 
> >> >> >   can you explain why whis ? 
> >> >> 
> >> >> A map is basically a pointer to a complex data structure.  Setting a 
> >> >> value in a map changes that data structure.  If one goroutine is 
> >> >> reading from the data structure while a different goroutine is 
> writing 
> >> >> to the data structure, the results are completely unpredictable.  In 
> >> >> the worst case they could even cause the program to crash. 
> >> >> 
> >> >> Ian 
> >> >> 
> >> >> 
> >> >> > 在 2016年11月4日星期五 UTC+8下午1:16:39，Ian Lance Taylor写道： 
> >> >> >> 
> >> >> >> On Thu, Nov 3, 2016 at 8:37 PM, 刘桂祥  
> wrote: 
> >> >> >> > // example.go 
> >> >> >> > 
> >> >> >> > package main 
> >> >> >> > 
> >> >> >> > var gMap = make(map[int]int) 
> >> >> >> > 
> >> >> >> > func w() { 
> >> >> >> > temp := make(map[int]int) 
> >> >> >> > temp[1] = 100 
> >> >> >> > temp[2] = 200 
> >> >> >> > gMap = temp// Does the compiler or cpu will reorder 
> >> >> >> > temp[1]=100, 
> >> >> >> > temp[2]=200, gMap=temp ?? 
> >> >> >> > } 
> >> >> >> > 
> >> >> >> > func r() { 
> >> >> >> > local := gMap 
> >> >> >> > println(local[1], local[2]) 
> >> >> >> > } 
> >> >> >> > 
> >> >> >> > func main() { 
> >> >> >> > 
> >> >> >

[go-nuts] golang cpu usage question

[刘桂祥]

As we know go have optimise its gc stw-pause-time ,but it has some 
trade-off ;
it implements concurent gc and will grab cpu resources with your 
application 

In some high CPU usage application, there are long time request case caused 
by gc

I want to know How to monitor my application in CPU usage ;  appication 
self use and gc use ??

I find rutime.memstats   gccpufration is about gc CPU usage but it is quota 
from the server start 
Have some quota  over time?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: golang cpu usage question

[刘桂祥]

Hi Dave:
   very thanks your timely response
   two more question
   1.  GODEBUG=gctrace=1 Do have some method  get these quota from 
runtime package apis ?  
   2.  In go1.7  the concurent gc could occupy all cpu core ??  does it 
have cpu usgae limit ?
   

在 2017年3月27日星期一 UTC+8下午5:28:38，Dave Cheney写道：
>
> You can enable monitoring of the gc with this environment variable
>
> GODEBUG=gctrace=1
>
> The format of the output is described on this page.
>
> https://golang.org/pkg/runtime/
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: golang cpu usage question

[刘桂祥]

now In my application  48 core set GODEBUG=gctrace=1  the gc CPU percent is 
17%  
but my server request have many long time case 
want to know the CPU percent in witch percentage is hight??

在 2017年3月27日星期一 UTC+8下午6:55:47，Dave Cheney写道：
>
>
>
> On Monday, 27 March 2017 21:12:44 UTC+11, 刘桂祥 wrote:
>>
>> Hi Dave:
>>very thanks your timely response
>>two more question
>>1.  GODEBUG=gctrace=1 Do have some method  get these quota 
>> from runtime package apis ?  
>>
>
> no
>  
>
>>2.  In go1.7  the concurent gc could occupy all cpu core ??  does 
>> it have cpu usgae limit ?
>>
>
> no
>
>  
>
>>
>>
>> 在 2017年3月27日星期一 UTC+8下午5:28:38，Dave Cheney写道：
>>>
>>> You can enable monitoring of the gc with this environment variable
>>>
>>> GODEBUG=gctrace=1
>>>
>>> The format of the output is described on this page.
>>>
>>> https://golang.org/pkg/runtime/
>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: golang cpu usage question

[刘桂祥]

package main

import (
"fmt"
"time"

   )

func foo() {
str := ""
 for i := 0; i < 1000; i++ {
 str += "aa"
 }
}
func main() {

for i := 0; i < 200; i++ {
go func() {
for {
foo()
time.Sleep(10 * time.Millisecond)
}
}()
}

for {
fmt.Printf("ticker>>>\n")
time.Sleep(3 * time.Second)
}
}



above is my worse example code
1. GOMAXPROCS=1 GODEBUG=gctrace=1 go run worse.go
...
gc 1276 @1.700s 58%: 0.001+0.75+0.005 ms clock, 0.001+0/0.75/0+0.005 ms 
cpu, 7->7->3 MB, 8 MB goal, 1 P
gc 1277 @1.702s 58%: 0.001+0.92+0.005 ms clock, 0.001+0/0.91/0+0.005 ms 
cpu, 7->7->3 MB, 8 MB goal, 1 P
gc 1278 @1.703s 58%: 0+0.76+0.005 ms clock, 0+0/0.76/0+0.005 ms cpu, 
7->7->3 MB, 8 MB goal, 1 P
gc 1279 @1.704s 58%: 0+0.75+0.005 ms clock, 0+0/0.75/0+0.005 ms cpu, 
7->7->3 MB, 8 MB goal, 1 P
gc 1280 @1.706s 58%: 0.001+0.73+0.005 ms clock, 0.001+0/0.72/0+0.005 ms 
cpu, 7->7->3 MB, 8 MB goal, 1 P
gc 1281 @1.707s 58%: 0+0.74+0.008 ms clock, 0+0/0.74/0+0.008 ms cpu, 
7->7->3 MB, 8 MB goal, 1 P
gc 1282 @1.708s 58%: 0.001+0.76+0.005 ms clock, 0.001+0/0.76/0+0.005 ms 
cpu, 7->7->3 MB, 8 MB goal, 1 P
gc 1283 @1.709s 58%: 0+0.76+0.005 ms clock, 0+0/0.76/0+0.005 ms cpu, 
7->7->3 MB, 8 MB goal, 1 P
gc 1284 @1.710s 58%: 0.001+0.77+0.005 ms clock, 0.001+0/0.76/0+0.005 ms 
cpu, 7->7->3 MB, 8 MB goal, 1 P
gc 1285 @1.712s 58%: 0.001+0.72+0.005 ms clock, 0.001+0/0.72/0+0.005 ms 
cpu, 7->7->3 MB, 8 MB goal, 1 P
gc 1286 @1.713s 58%: 0.001+0.75+0.005 ms clock, 0.001+0/0.74/0+0.005 ms 
cpu, 7->7->3 MB, 8 MB goal, 1 P
gc 1287 @1.714s 58%: 0+0.75+0.005 ms clock, 0+0/0.74/0+0.005 ms cpu, 
7->7->3 MB, 8 MB goal, 1 P
gc 1288 @1.715s 58%: 0+0.78+0.005 ms clock, 0+0/0.77/0+0.005 ms cpu, 
7->7->3 MB, 8 MB goal, 1 P

2.  GOMAXPROCS=4 GODEBUG=gctrace=1 go run worse.go
...
gc 1446 @1.457s 26%: 0.012+0.55+0.076 ms clock, 0.038+0.088/0.54/0.40+0.22 
ms cpu, 5->8->4 MB, 8 MB goal, 4 P
gc 1447 @1.458s 26%: 0.011+0.50+0.074 ms clock, 0.035+0.077/0.49/0.44+0.22 
ms cpu, 5->9->3 MB, 8 MB goal, 4 P
gc 1448 @1.459s 26%: 0.012+0.50+0.062 ms clock, 0.036+0.080/0.48/0.43+0.18 
ms cpu, 5->9->3 MB, 7 MB goal, 4 P
gc 1449 @1.460s 26%: 0.011+0.51+0.064 ms clock, 0.035+0.078/0.47/0.45+0.19 
ms cpu, 5->8->4 MB, 7 MB goal, 4 P
gc 1450 @1.460s 26%: 0.013+0.51+0.064 ms clock, 0.039+0.078/0.50/0.40+0.19 
ms cpu, 5->9->3 MB, 8 MB goal, 4 P
gc 1451 @1.461s 26%: 0.007+0.54+0.060 ms clock, 0.030+0.077/0.54/0.38+0.24 
ms cpu, 5->9->3 MB, 7 MB goal, 4 P
gc 1452 @1.462s 26%: 0.011+0.50+0.063 ms clock, 0.035+0.078/0.46/0.45+0.19 
ms cpu, 5->8->3 MB, 7 MB goal, 4 P
gc 1453 @1.463s 26%: 0.010+0.48+0.055 ms clock, 0.032+0.078/0.47/0.45+0.16 
ms cpu, 5->8->3 MB, 7 MB goal, 4 P
gc 1454 @1.464s 26%: 0.014+0.50+0.059 ms clock, 0.042+0.082/0.50/0.43+0.17 
ms cpu, 4->9->3 MB, 7 MB goal, 4 P
gc 1455 @1.464s 26%: 0.013+0.50+0.060 ms clock, 0.039+0.077/0.49/0.46+0.18 
ms cpu, 5->9->4 MB, 7 MB goal, 4 P
gc 1456 @1.465s 26%: 0.014+0.51+0.060 ms clock, 0.044+0.077/0.51/0.37+0.18 
ms cpu, 5->9->3 MB, 8 MB goal, 4 P
gc 1457 @1.466s 26%: 0.015+0.52+0.058 ms clock, 0.047+0.078/0.51/0.47+0.17 
ms cpu, 4->8->3 MB, 7 MB goal, 4 P
gc 1458 @1.467s 26%: 0.012+0.55+0.061 ms clock, 0.038+0.078/0.53/0.59+0.18 
ms cpu, 5->9->3 MB, 7 MB goal, 4 P


3.  GOMAXPROCS=8 GODEBUG=gctrace=1 go run worse.go
...
gc 2229 @1.659s 22%: 0.022+0.50+0.24 ms clock, 0.089+0.17/0.93/0.27+0.97 ms 
cpu, 11->19->9 MB, 19 MB goal, 8 P
gc 2230 @1.660s 22%: 0.018+0.43+0.071 ms clock, 0.075+0.14/0.81/0.34+0.28 
ms cpu, 10->18->9 MB, 18 MB goal, 8 P
gc 2231 @1.660s 22%: 0.019+0.38+0.23 ms clock, 0.078+0.16/0.71/0.28+0.95 ms 
cpu, 10->18->9 MB, 18 MB goal, 8 P
gc 2232 @1.661s 22%: 0.025+0.38+0.084 ms clock, 0.10+0.16/0.72/0.26+0.33 ms 
cpu, 10->18->9 MB, 19 MB goal, 8 P
gc 2233 @1.662s 22%: 0.023+0.38+0.083 ms clock, 0.095+0.12/0.71/0.30+0.33 
ms cpu, 10->18->9 MB, 19 MB goal, 8 P
gc 2234 @1.663s 22%: 0.018+0.39+0.23 ms clock, 0.074+0.11/0.72/0.34+0.95 ms 
cpu, 10->18->9 MB, 19 MB goal, 8 P
gc 2235 @1.664s 22%: 0.021+0.40+0.090 ms clock, 0.084+0.13/0.72/0.31+0.36 
ms cpu, 10->18->9 MB, 19 MB goal, 8 P
gc 2236 @1.664s 22%: 0.019+0.36+0.077 ms clock, 0.079+0.17/0.67/0.36+0.31 
ms cpu, 11->18->9 MB, 19 MB goal, 8 P
gc 2237 @1.665s 22%: 0.019+0.39+0.23 ms clock, 0.079+0.14/0.72/0.30+0.93 ms 
cpu, 10->18->10 MB, 18 MB goal, 8 P
gc 2238 @1.666s 22%: 0.022+0.39+0.076 ms clock, 0.091+0.15/0.71/0.28+0.30 
ms cpu, 11->19->9 MB, 20 MB goal, 8 P
gc 2239 @1.667s 22%: 0.020+0.40+0.11 ms clock, 0.081+0.18/0.75/0.42+0.44 ms 
cpu, 14->19->9 MB, 19 MB goal, 8 P
gc 2240 @1.667s 22%: 0.019+0.45+0.10 ms clock, 0.077+0.15/0.86/0.23+0.42 ms 
cpu, 10->18->10 MB, 18 MB goal, 8 P


4.GOMAXPROCS=48 GODEBUG=gctrace=1 go run worse.go
..
gc 658 @1.909s 5%: 1.3+1.1+0.46 ms clock, 5.2+3.1/4.6/0+1.8 ms cpu, 
22->43->22 MB, 39 MB goal, 48 P
gc 659 @1.913s 5%: 0.23+1.3+0.37 ms c

[go-nuts] Re: golang cpu usage question

[刘桂祥]

my production program data:
...
gc 149 @91.330s 6%: 6.8+557+0.50+1.0+10 ms clock, 20+557+0+1841/0/1304+31 
ms cpu, 3694->3793->2016 MB, 3789 MB goal, 48 P
gc 150 @92.012s 6%: 2.9+505+0.32+0.58+9.5 ms clock, 
11+505+0+1756/0.014/1574+38 ms cpu, 3739->3805->1987 MB, 3835 MB goal, 48 P
gc 151 @92.632s 6%: 5.4+420+0.38+0.25+9.9 ms clock, 
21+420+0+1690/0.17/1241+39 ms cpu, 3748->3810->1993 MB, 3844 MB goal, 48 P
gc 152 @93.187s 6%: 7.3+471+0.14+0.21+10 ms clock, 
29+471+0+1792/0.001/978+42 ms cpu, 3762->3861->1989 MB, 3858 MB goal, 48 P
gc 153 @93.806s 6%: 2.7+481+0.20+0.51+10 ms clock, 
10+481+0+1807/0.094/1261+40 ms cpu, 3685->3727->1965 MB, 3780 MB goal, 48 P
gc 154 @94.408s 6%: 3.3+477+0.46+0.54+10 ms clock, 
13+477+0+1697/0.007/1497+41 ms cpu, 3751->3848->2043 MB, 3847 MB goal, 48 P
gc 155 @95.011s 6%: 3.2+534+0.83+0.46+9.7 ms clock, 
12+534+0+1780/0.008/1457+38 ms cpu, 3794->3912->2068 MB, 3891 MB goal, 48 P
gc 156 @95.660s 6%: 5.4+504+0.96+0.57+9.5 ms clock, 21+504+0+1653/0/1604+38 
ms cpu, 3802->3900->2075 MB, 3899 MB goal, 48 P
gc 157 @96.290s 6%: 2.7+482+0.68+0.38+10 ms clock, 
10+482+0+1732/0.011/1160+41 ms cpu, 3852->3909->2043 MB, 3950 MB goal, 48 P
...
gc 375 @272.575s 8%: 4.1+628+0.75+0.75+15 ms clock, 
24+628+0+5656/0.006/2881+93 ms cpu, 5847->5933->3115 MB, 5997 MB goal, 48 P
gc 376 @273.400s 8%: 3.7+739+0.36+0.96+15 ms clock, 
22+739+0+4737/0.50/3957+94 ms cpu, 5907->6032->3161 MB, 6058 MB goal, 48 P
gc 377 @274.333s 8%: 3.2+700+0.22+0.41+14 ms clock, 
19+700+0+6550/0.011/3101+88 ms cpu, 5920->6056->3170 MB, 6072 MB goal, 48 P
gc 378 @275.219s 8%: 3.1+681+0.33+0.61+16 ms clock, 
18+681+0+5050/0.002/3335+99 ms cpu, 5922->6036->3155 MB, 6073 MB goal, 48 P
gc 379 @276.085s 8%: 3.2+764+0.38+0.83+15 ms clock, 
19+764+0+4807/0.001/3859+91 ms cpu, 5932->6062->3170 MB, 6084 MB goal, 48 P
gc 380 @277.032s 8%: 3.0+731+0.44+0.61+14 ms clock, 
18+731+0+6406/0.002/3522+89 ms cpu, 5931->6067->3175 MB, 6083 MB goal, 48 P
gc 381 @277.947s 8%: 3.1+739+0.30+0.45+15 ms clock, 
19+739+0+4654/0.001/3375+90 ms cpu, 5930->6063->3160 MB, 6082 MB goal, 48 P
gc 382 @278.871s 8%: 3.3+645+0.42+0.57+15 ms clock, 
19+645+0+4512/0.020/3602+91 ms cpu, 5905->5990->3121 MB, 6056 MB goal, 48 P
...
gc 475 @368.078s 9%: 3.3+790+0.31+0.56+19 ms clock, 
20+790+0+4356/0.001/5125+117 ms cpu, 6137->6255->3274 MB, 6294 MB goal, 48 P
gc 476 @369.047s 9%: 3.3+695+2.3+0.37+16 ms clock, 
20+695+0+7353/0.37/4180+97 ms cpu, 6157->6307->3335 MB, 6314 MB goal, 48 P
gc 477 @369.922s 9%: 3.4+754+0.17+0.38+17 ms clock, 
20+754+0+4769/0.64/5090+103 ms cpu, 6215->6385->3378 MB, 6374 MB goal, 48 P
gc 478 @370.857s 9%: 3.6+727+0.59+0.43+16 ms clock, 
22+727+0+4726/0.009/5578+99 ms cpu, 6258->6405->3405 MB, 6417 MB goal, 48 P
gc 479 @371.766s 9%: 3.3+778+0.14+0.71+17 ms clock, 
20+778+0+4581/0.031/5694+107 ms cpu, 6354->6492->3414 MB, 6517 MB goal, 48 P
gc 480 @372.729s 9%: 3.4+796+0.27+0.63+16 ms clock, 
20+796+0+4576/0.070/5559+101 ms cpu, 6394->6529->3417 MB, 6557 MB goal, 48 P
gc 481 @373.712s 9%: 3.3+771+0.12+0.41+16 ms clock, 
20+771+0+4918/0.001/5195+101 ms cpu, 6404->6521->3399 MB, 6568 MB goal, 48 P
gc 482 @374.667s 9%: 3.3+763+0.37+0.29+16 ms clock, 
20+763+0+4805/0.056/5169+100 ms cpu, 6404->6547->3422 MB, 6568 MB goal, 48 P
...
gc 508 @401.881s 10%: 3.5+715+0.15+0.47+19 ms clock, 
21+715+0+4336/0.18/5210+114 ms cpu, 6240->6385->3394 MB, 6400 MB goal, 48 P
gc 509 @402.781s 10%: 3.8+842+0.17+0.31+18 ms clock, 
22+842+0+5586/0.001/5674+110 ms cpu, 6337->6478->3418 MB, 6500 MB goal, 48 P
gc 510 @403.798s 10%: 3.8+830+0.36+0.30+17 ms clock, 
23+830+0+4356/0/5296+105 ms cpu, 6394->6502->3391 MB, 6558 MB goal, 48 P
gc 511 @404.808s 10%: 3.4+830+0.70+0.42+18 ms clock, 
20+830+0+4347/0.065/5628+110 ms cpu, 6407->6509->3382 MB, 6571 MB goal, 48 P
gc 512 @405.819s 10%: 3.6+832+0.65+0.46+19 ms clock, 
22+832+0+4450/0.17/5178+119 ms cpu, 6400->6545->3428 MB, 6563 MB goal, 48 P
gc 513 @406.827s 10%: 3.4+784+0.17+0.27+17 ms clock, 
20+784+0+4530/0.045/5297+104 ms cpu, 6404->6542->3431 MB, 6568 MB goal, 48 P
gc 514 @407.788s 10%: 4.3+735+0.63+1.2+19 ms clock, 
26+735+0+4576/0.009/5267+115 ms cpu, 6424->6536->3383 MB, 6589 MB goal, 48 P
gc 515 @408.717s 10%: 8.0+783+0.30+0.25+18 ms clock, 
48+783+0+5859/0.012/6100+111 ms cpu, 6382->6536->3428 MB, 6544 MB goal, 48 P
gc 516 @409.685s 10%: 4.1+712+0.76+0.70+18 ms clock, 
25+712+0+5675/0.018/5834+111 ms cpu, 6385->6507->3421 MB, 6549 MB goal, 48 P
gc 517 @410.567s 10%: 3.9+806+0.40+0.74+19 ms clock, 
23+806+0+5823/0.003/5744+115 ms cpu, 6437->6503->3399 MB, 6602 MB goal, 48 P
...
gc 638 @547.759s 10%: 9.0+877+0.16+0.36+24 ms clock, 
45+877+0+7723/0.25/7339+121 ms cpu, 7527->7693->4028 MB, 7719 MB goal, 48 P
gc 639 @548.866s 10%: 7.9+935+0.75+0.61+25 ms clock, 
39+935+0+6085/0.002/6937+129 ms cpu, 7532->7629->3958 MB, 7724 MB goal, 48 P
gc 640 @550.026s 10%: 5.3+972+0.27+0.34+24 ms clock, 
26+972+0+6123/0.49/7629+123 ms cpu, 7531->7692->4021 MB, 7722 MB goal, 48 P
gc 641 @551.214s 10%: 7.7+889+0.51+0.35+27 ms 

[go-nuts] Re: golang cpu usage question

[刘桂祥]

yes ,I do a str += str0  so many in my program
now I change to use bytes,Buffer to solve it

And I really need to do a gc monitor that notice me my code is worse
now gc frequency and pause time do not notice me
does the runtime gc_cpu_fraciton (the same as GODEBUG=gctrace=1  CPU occupy 
)  say something?  but I am confused with the value?
 


在 2017年3月28日星期二 UTC+8下午3:07:53，Dave Cheney写道：
>
> It looks like your program is generating in excess of 4gb of garbage a 
> second. I suggest generating a memory profile with pkg/profile and looking 
> at your allocation hotspots. 
>
> gc 708 @641.202s 11%: 9.2+1025+0.50+0.31+32 ms clock, 
> 55+1025+0+9983/0.013/9124+194 ms cpu, 8189->8363->4389 MB, 8399 MB goal, 48 
> P
> gc 709 @642.498s 11%: 9.4+1074+0.31+0.53+29 ms clock, 
> 56+1074+0+7404/0.004/9114+178 ms cpu, 8227->8380->4357 MB, 8438 MB goal, 48 
> P
> gc 710 @643.854s 11%: 5.7+1033+0.40+0.38+30 ms clock, 
> 34+1033+0+9576/0.005/9297+183 ms cpu, 8200->8351->4324 MB, 8410 MB goal, 48 
> P
>
> At the rate your program is allocating I'm sure the garbage collector will 
> be in "assist" mode most of the time. Assist mode causes goroutines to 
> "pay" for their allocation by doing some mark and sweep work each time they 
> allocate. 
>
> The execution trace can show this more clearly. 
>
> On Tuesday, 28 March 2017 17:59:19 UTC+11, 刘桂祥 wrote:
>>
>> my production program data:
>> ...
>> gc 149 @91.330s 6%: 6.8+557+0.50+1.0+10 ms clock, 20+557+0+1841/0/1304+31 
>> ms cpu, 3694->3793->2016 MB, 3789 MB goal, 48 P
>> gc 150 @92.012s 6%: 2.9+505+0.32+0.58+9.5 ms clock, 
>> 11+505+0+1756/0.014/1574+38 ms cpu, 3739->3805->1987 MB, 3835 MB goal, 48 P
>> gc 151 @92.632s 6%: 5.4+420+0.38+0.25+9.9 ms clock, 
>> 21+420+0+1690/0.17/1241+39 ms cpu, 3748->3810->1993 MB, 3844 MB goal, 48 P
>> gc 152 @93.187s 6%: 7.3+471+0.14+0.21+10 ms clock, 
>> 29+471+0+1792/0.001/978+42 ms cpu, 3762->3861->1989 MB, 3858 MB goal, 48 P
>> gc 153 @93.806s 6%: 2.7+481+0.20+0.51+10 ms clock, 
>> 10+481+0+1807/0.094/1261+40 ms cpu, 3685->3727->1965 MB, 3780 MB goal, 48 P
>> gc 154 @94.408s 6%: 3.3+477+0.46+0.54+10 ms clock, 
>> 13+477+0+1697/0.007/1497+41 ms cpu, 3751->3848->2043 MB, 3847 MB goal, 48 P
>> gc 155 @95.011s 6%: 3.2+534+0.83+0.46+9.7 ms clock, 
>> 12+534+0+1780/0.008/1457+38 ms cpu, 3794->3912->2068 MB, 3891 MB goal, 48 P
>> gc 156 @95.660s 6%: 5.4+504+0.96+0.57+9.5 ms clock, 
>> 21+504+0+1653/0/1604+38 ms cpu, 3802->3900->2075 MB, 3899 MB goal, 48 P
>> gc 157 @96.290s 6%: 2.7+482+0.68+0.38+10 ms clock, 
>> 10+482+0+1732/0.011/1160+41 ms cpu, 3852->3909->2043 MB, 3950 MB goal, 48 P
>> ...
>> gc 375 @272.575s 8%: 4.1+628+0.75+0.75+15 ms clock, 
>> 24+628+0+5656/0.006/2881+93 ms cpu, 5847->5933->3115 MB, 5997 MB goal, 48 P
>> gc 376 @273.400s 8%: 3.7+739+0.36+0.96+15 ms clock, 
>> 22+739+0+4737/0.50/3957+94 ms cpu, 5907->6032->3161 MB, 6058 MB goal, 48 P
>> gc 377 @274.333s 8%: 3.2+700+0.22+0.41+14 ms clock, 
>> 19+700+0+6550/0.011/3101+88 ms cpu, 5920->6056->3170 MB, 6072 MB goal, 48 P
>> gc 378 @275.219s 8%: 3.1+681+0.33+0.61+16 ms clock, 
>> 18+681+0+5050/0.002/3335+99 ms cpu, 5922->6036->3155 MB, 6073 MB goal, 48 P
>> gc 379 @276.085s 8%: 3.2+764+0.38+0.83+15 ms clock, 
>> 19+764+0+4807/0.001/3859+91 ms cpu, 5932->6062->3170 MB, 6084 MB goal, 48 P
>> gc 380 @277.032s 8%: 3.0+731+0.44+0.61+14 ms clock, 
>> 18+731+0+6406/0.002/3522+89 ms cpu, 5931->6067->3175 MB, 6083 MB goal, 48 P
>> gc 381 @277.947s 8%: 3.1+739+0.30+0.45+15 ms clock, 
>> 19+739+0+4654/0.001/3375+90 ms cpu, 5930->6063->3160 MB, 6082 MB goal, 48 P
>> gc 382 @278.871s 8%: 3.3+645+0.42+0.57+15 ms clock, 
>> 19+645+0+4512/0.020/3602+91 ms cpu, 5905->5990->3121 MB, 6056 MB goal, 48 P
>> ...
>> gc 475 @368.078s 9%: 3.3+790+0.31+0.56+19 ms clock, 
>> 20+790+0+4356/0.001/5125+117 ms cpu, 6137->6255->3274 MB, 6294 MB goal, 48 P
>> gc 476 @369.047s 9%: 3.3+695+2.3+0.37+16 ms clock, 
>> 20+695+0+7353/0.37/4180+97 ms cpu, 6157->6307->3335 MB, 6314 MB goal, 48 P
>> gc 477 @369.922s 9%: 3.4+754+0.17+0.38+17 ms clock, 
>> 20+754+0+4769/0.64/5090+103 ms cpu, 6215->6385->3378 MB, 6374 MB goal, 48 P
>> gc 478 @370.857s 9%: 3.6+727+0.59+0.43+16 ms clock, 
>> 22+727+0+4726/0.009/5578+99 ms cpu, 6258->6405->3405 MB, 6417 MB goal, 48 P
>> gc 479 @371.766s 9%: 3.3+778+0.14+0.71+17 ms clock, 
>> 20+778+0+4581/0.031/5694+107 ms cpu, 6354->6492->3414 MB, 6517 MB goal, 48 P
>> gc 480 @372.729s 9%: 3.4+796+0.27+0.63+16 ms clock, 
>> 20+796+0+4576/0.070/5

[go-nuts] golang precise gc

[刘桂祥]

package main

import (
"fmt"
"runtime"
"unsafe"
)

func main() {
var m runtime.MemStats

runtime.ReadMemStats(&m)
println(m.HeapObjects, m.Mallocs)

runtime.GC()

runtime.ReadMemStats(&m)
println(m.HeapObjects, m.Mallocs)

runtime.GC()

runtime.ReadMemStats(&m)
println(m.HeapObjects, m.Mallocs)

p := foo()
runtime.ReadMemStats(&m)
println(m.HeapObjects, m.Mallocs)

runtime.GC()

runtime.ReadMemStats(&m)
println(m.HeapObjects, m.Mallocs)
println(*((*int)(unsafe.Pointer(p

foo1()

println(*((*int)(unsafe.Pointer(p
}

func foo() uintptr {
i := new(int)
*i = 1000
if false {
fmt.Sprintf("%v", i)
}

return uintptr(unsafe.Pointer(i))
}
func foo1() uintptr {
i := new(int)
*i = 2000
if false {
fmt.Sprintf("%v", i)
}

return uintptr(unsafe.Pointer(i))
}



-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] go build -gcflags '-N -l' don't work

[刘桂祥]

// example.go
package main


import "runtime"


type S struct{}


func main() {
 var stats runtime.MemStats


 runtime.ReadMemStats(&stats)
 println(stats.Mallocs)
 var x S
 runtime.ReadMemStats(&stats)
 println(stats.Mallocs)
 _ = *ref(x)
 runtime.ReadMemStats(&stats)
 println(stats.Mallocs)
}


func ref(z S) *S {
 return &z
}

go build -gcflags '-N -l' example.go && ./example
or
go run -gcflags '-N -l' example.go

results:
./example.go:21: &z escapes to heap
./example.go:20: moved to heap: z
./example.go:10: main &stats does not escape
./example.go:13: main &stats does not escape
./example.go:16: main &stats does not escape
61
61
61


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] go build -gcflags '-N -l' don't work

[刘桂祥]

go1.7.1

go run -gcflags '-N -l -m' example.go

在 2017年4月14日星期五 UTC+8下午1:18:26，Ian Lance Taylor写道：
>
> On Thu, Apr 13, 2017 at 8:33 PM, 刘桂祥 > 
> wrote: 
> > // example.go 
> > package main 
> > 
> > 
> > import "runtime" 
> > 
> > 
> > type S struct{} 
> > 
> > 
> > func main() { 
> >  var stats runtime.MemStats 
> > 
> > 
> >  runtime.ReadMemStats(&stats) 
> >  println(stats.Mallocs) 
> >  var x S 
> >  runtime.ReadMemStats(&stats) 
> >  println(stats.Mallocs) 
> >  _ = *ref(x) 
> >  runtime.ReadMemStats(&stats) 
> >  println(stats.Mallocs) 
> > } 
> > 
> > 
> > func ref(z S) *S { 
> >  return &z 
> > } 
> > 
> > go build -gcflags '-N -l' example.go && ./example 
> > or 
> > go run -gcflags '-N -l' example.go 
> > 
> > results: 
> > ./example.go:21: &z escapes to heap 
> > ./example.go:20: moved to heap: z 
> > ./example.go:10: main &stats does not escape 
> > ./example.go:13: main &stats does not escape 
> > ./example.go:16: main &stats does not escape 
> > 61 
> > 61 
> > 61 
>
> That looks like the output from -gcflags="-m -l".  Are you really sure 
> you typed -N?  What version of Go are you using? 
>
> Ian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] go build -gcflags '-N -l' don't work

[刘桂祥]

go build -gcflags '-N -l' example.go && ./example
If I use this to compile example.go it shound disable function inline and 
the compile code should call ref() in example.go and will call mallocgc 
runtimestats.Mallocs can see it  but it is not.

在 2017年4月14日星期五 UTC+8下午10:01:25，Ian Lance Taylor写道：
>
> On Thu, Apr 13, 2017 at 10:45 PM, 刘桂祥 > 
> wrote: 
> > go1.7.1 
> > 
> > go run -gcflags '-N -l -m' example.go 
>
> It is the -m flag that is generating the output you see. 
>
> If there is some other bug, please let us know what it is. 
>
> Ian 
>
> > 在 2017年4月14日星期五 UTC+8下午1:18:26，Ian Lance Taylor写道： 
> >> 
> >> On Thu, Apr 13, 2017 at 8:33 PM, 刘桂祥  wrote: 
> >> > // example.go 
> >> > package main 
> >> > 
> >> > 
> >> > import "runtime" 
> >> > 
> >> > 
> >> > type S struct{} 
> >> > 
> >> > 
> >> > func main() { 
> >> >  var stats runtime.MemStats 
> >> > 
> >> > 
> >> >  runtime.ReadMemStats(&stats) 
> >> >  println(stats.Mallocs) 
> >> >  var x S 
> >> >  runtime.ReadMemStats(&stats) 
> >> >  println(stats.Mallocs) 
> >> >  _ = *ref(x) 
> >> >  runtime.ReadMemStats(&stats) 
> >> >  println(stats.Mallocs) 
> >> > } 
> >> > 
> >> > 
> >> > func ref(z S) *S { 
> >> >  return &z 
> >> > } 
> >> > 
> >> > go build -gcflags '-N -l' example.go && ./example 
> >> > or 
> >> > go run -gcflags '-N -l' example.go 
> >> > 
> >> > results: 
> >> > ./example.go:21: &z escapes to heap 
> >> > ./example.go:20: moved to heap: z 
> >> > ./example.go:10: main &stats does not escape 
> >> > ./example.go:13: main &stats does not escape 
> >> > ./example.go:16: main &stats does not escape 
> >> > 61 
> >> > 61 
> >> > 61 
> >> 
> >> That looks like the output from -gcflags="-m -l".  Are you really sure 
> >> you typed -N?  What version of Go are you using? 
> >> 
> >> Ian 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups 
> > "golang-nuts" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an 
> > email to golang-nuts...@googlegroups.com . 
> > For more options, visit https://groups.google.com/d/optout. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] go build -gcflags '-N -l' don't work

[刘桂祥]

thanks a lot

在 2017年4月16日星期日 UTC+8上午12:22:13，peterGo写道：
>
>
> For this version of your program with
>
> type S struct{ B byte }
>
> There is an allocation,
>
> // example.go
> package main
>
> import (
> "runtime"
> "unsafe"
> )
>
> type S struct{ B byte }
>
> func main() {
> var stats runtime.MemStats
>
> runtime.ReadMemStats(&stats)
> println(stats.Mallocs)
> var x S
> runtime.ReadMemStats(&stats)
> println(stats.Mallocs)
> _ = *ref(x)
> runtime.ReadMemStats(&stats)
> println(stats.Mallocs)
>
> println(unsafe.Sizeof(x))
> }
>
> func ref(z S) *S {
> return &z
> }
>
> Output:
>
> $ go version
> go version devel +bc29313 Fri Apr 14 16:47:25 2017 + linux/amd64
> $ go run -gcflags '-N -l -m' example.go
> # command-line-arguments
> ./example.go:27:9: &z escapes to heap
> ./example.go:26:16: moved to heap: z
> ./example.go:14:23: main &stats does not escape
> ./example.go:17:23: main &stats does not escape
> ./example.go:20:23: main &stats does not escape
> 83
> 83
> 84
> 1
> $
>
> Peter
>
> On Saturday, April 15, 2017 at 12:14:48 PM UTC-4, peterGo wrote:
>>
>> 刘桂祥,
>>
>> Size and alignment guarantees
>> https://golang.org/ref/spec#Size_and_alignment_guarantees
>>
>> A struct or array type has size zero if it contains no fields (or 
>> elements, respectively) that have a size greater than zero. Two distinct 
>> zero-size variables may have the same address in memory.
>>
>> You have
>>
>> type S struct{}
>>
>> Why do you expect runtime.MemStats to show an allocation for zero bytes?
>>
>> Peter
>>
>> On Saturday, April 15, 2017 at 10:12:34 AM UTC-4, 刘桂祥 wrote:
>>>
>>>
>>> go build -gcflags '-N -l' example.go && ./example
>>> If I use this to compile example.go it shound disable function inline 
>>> and the compile code should call ref() in example.go and will call mallocgc 
>>> runtimestats.Mallocs can see it  but it is not.
>>>
>>> 在 2017年4月14日星期五 UTC+8下午10:01:25，Ian Lance Taylor写道：
>>>>
>>>> On Thu, Apr 13, 2017 at 10:45 PM, 刘桂祥  wrote: 
>>>> > go1.7.1 
>>>> > 
>>>> > go run -gcflags '-N -l -m' example.go 
>>>>
>>>> It is the -m flag that is generating the output you see. 
>>>>
>>>> If there is some other bug, please let us know what it is. 
>>>>
>>>> Ian 
>>>>
>>>> > 在 2017年4月14日星期五 UTC+8下午1:18:26，Ian Lance Taylor写道： 
>>>> >> 
>>>> >> On Thu, Apr 13, 2017 at 8:33 PM, 刘桂祥  wrote: 
>>>> >> > // example.go 
>>>> >> > package main 
>>>> >> > 
>>>> >> > 
>>>> >> > import "runtime" 
>>>> >> > 
>>>> >> > 
>>>> >> > type S struct{} 
>>>> >> > 
>>>> >> > 
>>>> >> > func main() { 
>>>> >> >  var stats runtime.MemStats 
>>>> >> > 
>>>> >> > 
>>>> >> >  runtime.ReadMemStats(&stats) 
>>>> >> >  println(stats.Mallocs) 
>>>> >> >  var x S 
>>>> >> >  runtime.ReadMemStats(&stats) 
>>>> >> >  println(stats.Mallocs) 
>>>> >> >  _ = *ref(x) 
>>>> >> >  runtime.ReadMemStats(&stats) 
>>>> >> >  println(stats.Mallocs) 
>>>> >> > } 
>>>> >> > 
>>>> >> > 
>>>> >> > func ref(z S) *S { 
>>>> >> >  return &z 
>>>> >> > } 
>>>> >> > 
>>>> >> > go build -gcflags '-N -l' example.go && ./example 
>>>> >> > or 
>>>> >> > go run -gcflags '-N -l' example.go 
>>>> >> > 
>>>> >> > results: 
>>>> >> > ./example.go:21: &z escapes to heap 
>>>> >> > ./example.go:20: moved to heap: z 
>>>> >> > ./example.go:10: main &stats does not escape 
>>>> >> > ./example.go:13: main &stats does not escape 
>>>> >> > ./example.go:16: main &stats does not escape 
>>>> >> > 61 
>>>> >> > 61 
>>>> >> > 61 
>>>> >> 
>>>> >> That looks like the output from -gcflags="-m -l".  Are you really 
>>>> sure 
>>>> >> you typed -N?  What version of Go are you using? 
>>>> >> 
>>>> >> Ian 
>>>> > 
>>>> > -- 
>>>> > You received this message because you are subscribed to the Google 
>>>> Groups 
>>>> > "golang-nuts" group. 
>>>> > To unsubscribe from this group and stop receiving emails from it, 
>>>> send an 
>>>> > email to golang-nuts...@googlegroups.com. 
>>>> > For more options, visit https://groups.google.com/d/optout. 
>>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: Large GC pauses with large map

[刘桂祥]

try use value type
example:
map[string]*struct  => map[[20]byte]struct  

在 2017年4月20日星期四 UTC+8下午9:49:49，Lee Armstrong写道：
>
> See attached graph which shows the GC pauses of an application we have.
>
> I am frequently seeing pauses of 1-1.5 seconds. This is using Go 1.8.1 and 
> have a large map that is frequently accessed and items are removed and 
> added to it.  These can be of some size.
>
> Is there a way to get these pauses down at all?  Would forcing a GC() 
> after removing a batch of elements help at all?
>
> Alongside the pauses I see some substantial CPU usage showing up in traces 
> for the GC scan.
>
> Thanks in advance!
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] golang thrift server 28k/s request client many dial tcp timeout ,why??

[刘桂祥]

   I have a very simple golang thrift server. the handle is very easy 
just to return
   But when I use another machine to benchmark the server ,when the qps 
to 28K/s , client have many dial tcp io/timeout 
   I don't konw the reason ? is the client or server's problem? 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] golang thrift server 28k/s request client many dial tcp timeout ,why??

[刘桂祥]

client: dial tcp 100.70.186.38:9783: i/o timeout
when I modify the server 
/proc/sys/net/core/netdev_max_backlog16384
/proc/sys/net/core/somaxconn  16384

the client dial timeout decrease some percent

在 2017年6月9日星期五 UTC+8上午12:10:15，Shawn Milochik写道：
>
> What error are you getting along with the timeout? I suspect that it has 
> something to do with the limit of filehandles on the client. 
>
> Have a look at this:
>
> https://www.cyberciti.biz/faq/linux-increase-the-maximum-number-of-open-files/
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] golang thrift server 28k/s request client many dial tcp timeout ,why??

[刘桂祥]

 I just want to bench the simple server accept new connection
 now I create a new simple tcp server and use tcp short connection to 
bench it
 the result is also that  client have many dial i/o timeout
 does it is client's question ??

在 2017年6月9日星期五 UTC+8下午9:28:55，Shawn Milochik写道：
>
> I think the setting is more likely to need an increase on the client than 
> the server. 
>
> Are you using HTTP/1 or HTTP/2? If HTTP/1, set the MaxIdleConnsPerHost 
> value on your client's Transport.
>
> More info:
>
> https://golang.org/pkg/net/http/#Transport
> https://paperairoplane.net/?p=556
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] dial io timeout and tcp short connection performance bottleneck

[刘桂祥]

> I write a simple tcp server with golang
> and use tcp short connect client to bench it.
> when qps to 28K,  client have many dial io timeout
> 
> I don't know what is the performance bottleneck??
> 
> I modify the server:
> cat /proc/sys/net/ipv4/tcp_max_syn_backlog1024
> cat cat /proc/sys/net/core/somaxconn  1024

// tcpserver.go
// a simple golang tcp server
package main

import (
"io"
"log"
"net"
)

func init() {
log.SetFlags(log.LstdFlags | log.Lshortfile)
}
func main() {
tcpAddr, err := net.ResolveTCPAddr("tcp", ":")
if err != nil {
log.Println(err)
return
}   
tcpListener, err := net.ListenTCP("tcp", tcpAddr)
if err != nil {
log.Println(err)
   return
}   
defer tcpListener.Close()

for {
tcpConn, err := tcpListener.AcceptTCP()
if err != nil {
log.Println(err)
continue
}
go handle(tcpConn)
}

}
// handle
func handle(conn *net.TCPConn) {
//log.Println(conn.RemoteAddr().String())

buf := make([]byte, 1024)
for {
_, err := conn.Read(buf)
if err != nil {
if err != io.EOF {
log.Println(err)
}
return
}

_, err = conn.Write([]byte{'h', 'e', 'l', 'l', 'o'})

if err != nil {
log.Println(err)
}
}
}

// tcpclient.go
// tcp short connection bench
package main

import (
"log"
"net"
"sync"
"time"
)

var (
wg sync.WaitGroup
)

func init() {
log.SetFlags(log.LstdFlags | log.Lshortfile)
}
func main() {

for i := 0; i < 150; i++ {
wg.Add(1)
go func() {
for {
req()
time.Sleep(10 * time.Millisecond)
}
wg.Done()
}()
}
wg.Wait()
}
// tcp req
func req() {
conn, err := net.DialTimeout("tcp", "192.168.67.133:", 
300*time.Millisecond)
if err != nil {
log.Println(err)
return
}
// conn.SetDeadline(time.Now().Add(300 * time.Millisecond))
defer conn.Close()
_, err = conn.Write([]byte("client"))
if err != nil {
log.Println(err)
return
}
buf := make([]byte, 1024)
_, err = conn.Read(buf)
if err != nil {
log.Println(err)
}
}




-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] dial io timeout and tcp short connection performance bottleneck

[刘桂祥]

//question
> I write a simple golang tcp server
> and use tcp short connect client to bench it.
> when qps up to 28K,  client have many dial i/o timeout
> 
> I don't know what is the performance bottleneck??
> 
> I modify the server:
> cat /proc/sys/net/ipv4/tcp_max_syn_backlog1024
> cat /proc/sys/net/core/somaxconn  1024

//a simple golang tcp server

   package main

import (
"io"
"log"
"net"
)

func init() {
log.SetFlags(log.LstdFlags | log.Lshortfile)
}
func main() {
tcpAddr, err := net.ResolveTCPAddr("tcp", ":")
if err != nil {
log.Println(err)
return
}   
tcpListener, err := net.ListenTCP("tcp", tcpAddr)
if err != nil {
log.Println(err)
   return
}   
defer tcpListener.Close()

for {
tcpConn, err := tcpListener.AcceptTCP()
if err != nil {
log.Println(err)
continue
}
go handle(tcpConn)
}

}
func handle(conn *net.TCPConn) {
//log.Println(conn.RemoteAddr().String())

buf := make([]byte, 1024)
for {
_, err := conn.Read(buf)
if err != nil {
if err != io.EOF {
log.Println(err)
}
return
}

_, err = conn.Write([]byte{'h', 'e', 'l', 'l', 'o'})

if err != nil {
log.Println(err)
}
}
}

// tcp short connection bench

package main

import (
"log"
"net"
"sync"
"time"
)

var (
wg sync.WaitGroup
)

func init() {
log.SetFlags(log.LstdFlags | log.Lshortfile)
}
func main() {

for i := 0; i < 150; i++ {
wg.Add(1)
go func() {
for {
req()
time.Sleep(10 * time.Millisecond)
}
wg.Done()
}()
}
wg.Wait()
}
// tcp req
func req() {
conn, err := net.DialTimeout("tcp", "192.168.67.133:", 
300*time.Millisecond)
if err != nil {
log.Println(err)
return
}
// conn.SetDeadline(time.Now().Add(300 * time.Millisecond))
defer conn.Close()
_, err = conn.Write([]byte("client"))
if err != nil {
log.Println(err)
return
}
buf := make([]byte, 1024)
_, err = conn.Read(buf)
if err != nil {
log.Println(err)
}
}

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: dial io timeout and tcp short connection performance bottleneck

[刘桂祥]

HI Matthew:
   Because I don't konw the performance bottleneck;  do you mean the 
tcp syn queue (half open queue) is full ??but I don't see the syn flood 
warn message in /var/log/messages

在 2017年6月11日星期日 UTC+8下午7:03:58，Matthew Stevenson写道：
>
> Are you exhausting the local TCP sockets? Some (50k?) in a wait state? Try 
> using a few IP addresses on the test box. Increase 
> /proc/sys/net/core/somaxconn to a larger value, if you have a 20 
> millisecond pause at 28kreqs/sec then thats 500 ish connections so 1024 may 
> not be enough.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [go-nuts] golang thrift server 28k/s request client many dial tcp timeout ,why??

[刘桂祥]

I truely try with two client server to bench it; and the result  is the 
same, so I doubt the performance bottleneck 
<https://groups.google.com/forum/#!topic/golang-nuts/aHj0T4mx6Uc> is in the 
server side 
but I don't konw the concrete  
<http://www.baidu.com/link?url=-DlmohrXhQQnIfxnBK-Tt3rYAF5agKj8uD4t6EqlOf8pAAFMZxXO_ufb0GT4arfhGV7Sxry5aWm9C4XAupL3dobAL-LE-ISgDILB94JLQATdeO_F-S7xy7Y8Tf0r_VMh>
question 


在 2017年6月11日星期日 UTC+8下午8:10:49，Jesper Louis Andersen写道：
>
> Hi!
>
> One thing you should check is the amount of TCP sessions you are starting 
> from one host. Since a TCP session is identified by the IP/Port in both 
> ends, and you communicate where some of those numbers stay the same, you 
> are probably limited to no more than 64000 ports at most (or such).
>
> If you run 28k reqs/s and you have a high time-wait timeout, you will run 
> out of ports quickly.
>
> A simple test is to have two "client" hosts and then checking if you can 
> run about the double amount through before getting into trouble.
>
> The other thing you should worry about is that if you are truly building 
> new TCP sessions every time you have a request, you pay the session setup 
> time on every request. You could maybe look into batch-framing some of them 
> which will improve the TCP window and likely increase throughput.
>
> On Sat, Jun 10, 2017 at 11:51 AM 刘桂祥 > 
> wrote:
>
>>  I just want to bench the simple server accept new connection
>>  now I create a new simple tcp server and use tcp short connection to 
>> bench it
>>  the result is also that  client have many dial i/o timeout
>>  does it is client's question ??
>>
>> 在 2017年6月9日星期五 UTC+8下午9:28:55，Shawn Milochik写道：
>>
>>> I think the setting is more likely to need an increase on the client 
>>> than the server. 
>>>
>>> Are you using HTTP/1 or HTTP/2? If HTTP/1, set the MaxIdleConnsPerHost 
>>> value on your client's Transport.
>>>
>>> More info:
>>>
>>> https://golang.org/pkg/net/http/#Transport
>>> https://paperairoplane.net/?p=556
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[go-nuts] Re: dial io timeout and tcp short connection performance bottleneck

[刘桂祥]

Hi Darkofday:
   I doubt that if the client's local port is  used out, it should't 
 return dail io timeout
   I use one client with less worker it don't have dial timeout and 
when another client with the same worker count to bench the server,  now 
 two clients have dial io  timeout  
   so I guess the performance bottleneck  is in the server

在 2017年6月12日星期一 UTC+8上午8:54:20，Darkofday Darkofday写道：
>
> Maybe your client's local ports is used out. threre are only about 65000 
> ports you can use to make a connection from a client to server per IP. You 
> can check output of netstat to confirm that. 
>
> On Sunday, June 11, 2017 at 9:48:30 AM UTC+8, 刘桂祥 wrote:
>>
>> //question
>> > I write a simple golang tcp server
>> > and use tcp short connect client to bench it.
>> > when qps up to 28K,  client have many dial i/o timeout
>> > 
>> > I don't know what is the performance bottleneck??
>> > 
>> > I modify the server:
>> > cat /proc/sys/net/ipv4/tcp_max_syn_backlog1024
>> > cat /proc/sys/net/core/somaxconn  1024
>>
>> //a simple golang tcp server
>> 
>>package main
>> 
>> import (
>> "io"
>> "log"
>> "net"
>> )
>> 
>> func init() {
>> log.SetFlags(log.LstdFlags | log.Lshortfile)
>> }
>> func main() {
>> tcpAddr, err := net.ResolveTCPAddr("tcp", ":")
>> if err != nil {
>> log.Println(err)
>> return
>> }   
>> tcpListener, err := net.ListenTCP("tcp", tcpAddr)
>> if err != nil {
>> log.Println(err)
>>return
>> }   
>> defer tcpListener.Close()
>> 
>> for {
>> tcpConn, err := tcpListener.AcceptTCP()
>> if err != nil {
>> log.Println(err)
>> continue
>> }
>> go handle(tcpConn)
>> }
>> 
>> }
>> func handle(conn *net.TCPConn) {
>> //log.Println(conn.RemoteAddr().String())
>> 
>> buf := make([]byte, 1024)
>> for {
>> _, err := conn.Read(buf)
>> if err != nil {
>> if err != io.EOF {
>> log.Println(err)
>> }
>> return
>> }
>> 
>> _, err = conn.Write([]byte{'h', 'e', 'l', 'l', 'o'})
>> 
>> if err != nil {
>> log.Println(err)
>> }
>> }
>> }
>> 
>> // tcp short connection bench
>>
>> package main
>> 
>> import (
>> "log"
>> "net"
>> "sync"
>> "time"
>> )
>> 
>> var (
>> wg sync.WaitGroup
>> )
>> 
>> func init() {
>> log.SetFlags(log.LstdFlags | log.Lshortfile)
>> }
>> func main() {
>> 
>> for i := 0; i < 150; i++ {
>> wg.Add(1)
>> go func() {
>> for {
>> req()
>> time.Sleep(10 * time.Millisecond)
>> }
>> wg.Done()
>> }()
>> }
>> wg.Wait()
>> }
>> // tcp req
>> func req() {
>> conn, err := net.DialTimeout("tcp", "192.168.67.133:", 
>> 300*time.Millisecond)
>> if err != nil {
>> log.Println(err)
>> return
>> }
>> // conn.SetDeadline(time.Now().Add(300 * time.Millisecond))
>> defer conn.Close()
>> _, err = conn.Write([]byte("client"))
>> if err != nil {
>> log.Println(err)
>> return
>> }
>> buf := make([]byte, 1024)
>> _, err = conn.Read(buf)
>> if err != nil {
>> log.Println(err)
>> }
>> }
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.