Re: Monitoring backpressure

2016-12-07 Thread Chesnay Schepler
ple as 2|1|0, so that we can plot backpressure state over time per stage. This would also allow users to set alerts on backpressure state. How might we get access to this backpressure state information? Thanks, Alan -- View this message in context: http://apache-flink-mailing-list-archive.10

Re: Monitoring backpressure

2016-12-06 Thread a...@opsclarity.com
this backpressure state information? Thanks, Alan -- View this message in context: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Monitoring-backpressure-tp9472p14868.html Sent from the Apache Flink Mailing List archive. mailing list archive at Nabble.com.

Re: Monitoring backpressure

2015-12-07 Thread Gyula Fóra
Thanks Stephan, I will try with the profiler for now. Gyula Stephan Ewen ezt írta (időpont: 2015. dec. 7., H, 10:51): > I discussed about this quite a bit with other people. > > It is not totally straightforward. One could try and measure exhaustion of > the output buffer pools, but that fluct

Re: Monitoring backpressure

2015-12-07 Thread Stephan Ewen
I discussed about this quite a bit with other people. It is not totally straightforward. One could try and measure exhaustion of the output buffer pools, but that fluctuates a lot - it would need some work to get a stable metric from that... If you have a profiler that you can attach to the proce

Monitoring backpressure

2015-12-07 Thread Gyula Fóra
Hey guys, Is there any way to monitor the backpressure in the Flink job? I find it hard to debug slow operators because of the backpressure mechanism so it would be good to get some info out of the network layer on what exactly caused the backpressure. For example: task1 -> task2 -> task3 -> tas