Re: debug for Flink

2018-04-20 Thread Kien Truong
Hi, We put a lot of logging in our code the flow of events as they flow between operators, this is done at the DEBUG level. Normally this logging would be turn off by setting the logging level to INFO or above, because logging everything is expensive and a cluttered log is harder to analy

Re: debug for Flink

2018-04-19 Thread Qian Ye
Thanks for your kind reply. But I still have some question. What does the logging level mean in your system? Why do you need to re-deploy the cluster to change the logging level? As far as I know, the debugging information can be divided into level like info, warn, error, etc. Is these informa

Re: debug for Flink

2018-04-19 Thread Qian Ye
That sounds nice. But the lazy evaluation feature of Flink seems to cause the debug process more different with a regular Java/Scala application. Do you know how to debug a Flink application, e.g., tracing some local variables, in IDE? Best, Stephen > On Apr 19, 2018, at 6:29 AM, Fabian Hueske

Re: debug for Flink

2018-04-19 Thread Kien Truong
Hi, Our most useful tool when debugging Flink is actually the simple log files, because debugger just slow things down too much for us. However, having to re-deploy the entire cluster to change the logging level is a pain (we use YARN), so we would really like an easier method to change the

Re: debug for Flink

2018-04-19 Thread Fabian Hueske
Hi, You can run Flink applications locally in your IDE and debug a Flink program just like a regular Java/Scala application. Best, Fabian 2018-04-19 0:53 GMT+02:00 Qian Ye : > Hi > > I’m wondering if new debugging methods/tools are urgent for Flink > development. I know there already exists so