Re: [go-nuts] Profiling runtime.futex

2019-02-05 Thread Andrew Medvedev
Profiling shows ~150_000 CS for channel version. For some reason I thought scheduling should be done by Go runtime, but it is not. Anyway you are right, I will look closer into syscall part, may be I will get rid of netlink library, or something.. On Monday, February 4, 2019 at 6:59:33 PM UT

Re: [go-nuts] Profiling runtime.futex

2019-02-04 Thread robert engels
And if that is the case, the most likely way to speed up you program would be to send the route updates in a batch to the kernel - not sure that this is even possible. > On Feb 4, 2019, at 9:56 AM, robert engels wrote: > > Also, to clarify, in the absence of more details, I would consider > n

Re: [go-nuts] Profiling runtime.futex

2019-02-04 Thread robert engels
Also, to clarify, in the absence of more details, I would consider netlink.RouteReplace to be an IO operation - it is calling into the kernel (most likely). Whether that should be considered IO is probably a matter of opinion. > On Feb 4, 2019, at 9:32 AM, Andrew Medvedev > wrote: > > Hi Rob

Re: [go-nuts] Profiling runtime.futex

2019-02-04 Thread robert engels
That doesn’t seem to match up with the pprof report - although it would be easier to determine if you showed the back-traced the syscalls. The original pprof report you stated was from without channels, thus the syscalls are almost certainly related to IO. Maybe you could clarify why you think i

Re: [go-nuts] Profiling runtime.futex

2019-02-04 Thread Andrew Medvedev
Hi Robert The program is not IO bound. Sorry for not giving out the source code in the first place, here is a no-channel version https://github.com/andrewmed/loadroutes/blob/master/cmd/main.go and here is a slower channels version https://github.com/andrewmed/loadroutes/blob/test/channels/cmd/

Re: [go-nuts] Profiling runtime.futex

2019-02-04 Thread Robert Engels
Reading and writing to a channel has locks behind the scenes. Locks are implemented with futexes. If your program is IO bound the simplest solution is often to use buffering in the IO to perform more work per IO operation. > On Feb 3, 2019, at 5:00 PM, Andrew Medvedev > wrote: > > Hi every

[go-nuts] Profiling runtime.futex

2019-02-03 Thread Andrew Medvedev
Hi everybody I have a simple program which parses IP addresses and adds them to the system routing table using the netlink library. The number is routes is huge (400_000). My parsing part is very simple and fast, and most of the time is spent in system calls: File: loadroutes Type: cpu T