RE: NioEventLoop consumes most of the CPU

2015-05-06 Thread Kruse, Sebastian
it is for networking and serialization. Cheers, Sebastian -Original Message- From: ewenstep...@gmail.com [mailto:ewenstep...@gmail.com] On Behalf Of Stephan Ewen Sent: Mittwoch, 6. Mai 2015 12:25 To: dev@flink.apache.org Subject: Re: NioEventLoop consumes most of the CPU Uful has a good

Re: NioEventLoop consumes most of the CPU

2015-05-06 Thread Stephan Ewen
Uful has a good point The NIO epoll wait method leaves the thread in state RUNNABLE. That may explain things. Still, would be good to have more information on your setup. Stephan Am 06.05.2015 10:15 schrieb "Ufuk Celebi" : > I agree with Stephan's points. Thanks for reporting and let's investig

Re: NioEventLoop consumes most of the CPU

2015-05-06 Thread Ufuk Celebi
I agree with Stephan's points. Thanks for reporting and let's investigate this further. To keep in mind: I think VisualVM is using hprof for CPU sampling, which has some known issues ( http://www.brendangregg.com/blog/2014-06-09/java-cpu-sampling-using-hprof.html). For one thing, it's profiling Ja

Re: NioEventLoop consumes most of the CPU

2015-05-05 Thread Stephan Ewen
Hi! That does not sound right, I agree. Can you tell us a bit more? - What version of Flink are you using? - I assume the NIO loop is executed by a Netty thread. Can you tell us whether it is from a "io.netty.*" thread, or a "org.jboss.netty.*" thread? The former is from Flink's data network thr

NioEventLoop consumes most of the CPU

2015-05-05 Thread Kruse, Sebastian
Hi everyone, Everytime when I am running jvisualvm on one of the machines in our cluster during a Flink job, I see that NioEventLoop.select() is taking 50% to 70% CPU self-time. I wonder how severe this is. It might be busy-waiting time that cannot be filled otherwise, but I wanted to ask you i