Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-16 Thread Rui Fan
Thanks Yun for the update, LGTM. Best, Rui On Mon, Oct 16, 2023 at 1:34 PM Yu Chen wrote: > Hi David. > > Thanks for your detailed comments. > The async-profiler has a lot of features, but there are some requirements > to use it (some cases encountered in production are listed below): > 1. Prio

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-15 Thread Yu Chen
Hi David. Thanks for your detailed comments. The async-profiler has a lot of features, but there are some requirements to use it (some cases encountered in production are listed below): 1. Prior to JDK 11, alloc mode requires HotSpot debug symbols, and for OpenJDK, you need to install them additio

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-15 Thread Yun Tang
Christle Sent: Saturday, October 14, 2023 4:11 To: dev@flink.apache.org Subject: Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers In the Wiki, this FLIP is motivated by: - That the current flamegraph functionality can only see operator-level stack traces, while

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-13 Thread David Christle
In the Wiki, this FLIP is motivated by: - That the current flamegraph functionality can only see operator-level stack traces, while async-profiler provides CPU/allocation/locks information, along with deeper Java & system call stack information. - Low configurability (e.g. cannot set the sampling

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-13 Thread Rui Fan
One minor comment: In general, the generic java profiler includes memory analysis, cpu, thread, deadlock, etc. The FLIP title is java profiler, but the FLIP just supports flamegraph at process level. So the `powerful java profiler` title may not be suitable. Would you mind updating the FLIP title?

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-13 Thread Yu Chen
Hi all. If there are no further questions, we will start a vote on FLIP-375 next week. Best regards, Yu Chen Yu Chen 于2023年10月9日周一 17:24写道: > Hi all, > > Yun Tang and I are opening this thread to discuss our proposal to > integrate async-profiler's capabilities for profiling taskmananger (e.g.

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-10 Thread Jing Ge
> > > [1] https://github.com/async-profiler/async-profiler/issues/14 > > Best > Yun Tang > > > From: Jing Ge > Sent: Tuesday, October 10, 2023 12:05 > To: dev@flink.apache.org > Subject: Re: [DISCUSS] FLIP-375: Built-in cross-pla

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-10 Thread Yun Tang
ed on taskmanager side. They flamegraph-files will be cleanup > automatically once reached to rest.profiling.history-size. > > Best > Yun Tang > > > > ________________ > From: Rui Fan <1996fan...@gmail.com> > Sent: Tuesday, October 10, 2023 10:10

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-09 Thread Jing Ge
ey flamegraph-files will be cleanup > automatically once reached to rest.profiling.history-size. > > Best > Yun Tang > > > > ________ > From: Rui Fan <1996fan...@gmail.com> > Sent: Tuesday, October 10, 2023 10:10 > To: dev@flink.apache.org > Subjec

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-09 Thread Yun Tang
for the job manager > > as > > > well. In the OLAP scenario and sometimes in the streaming scenario > (when > > > there're some heavy operations during execution plan generation or in > > > operator coordinators), the JM can have bottleneck as well. > >

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-09 Thread Rui Fan
he streaming scenario > (when > > > there're some heavy operations during execution plan generation or in > > > operator coordinators), the JM can have bottleneck as well. > > > > > > Best, > > > Zhanghao Chen > > > ___

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-09 Thread Jing Ge
> > > > Best, > > Zhanghao Chen > > ________________ > > From: Yu Chen > > Sent: Monday, October 9, 2023 17:24 > > To: dev@flink.apache.org > > Subject: [DISCUSS] FLIP-375: Built-in cross-platform powerful java > > profi

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-09 Thread Yu Chen
re're some heavy operations during execution plan generation or in > operator coordinators), the JM can have bottleneck as well. > > Best, > Zhanghao Chen > > From: Yu Chen > Sent: Monday, October 9, 2023 17:24 > To: dev@flink.apache.org >

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-09 Thread Rui Fan
rio (when > there're some heavy operations during execution plan generation or in > operator coordinators), the JM can have bottleneck as well. > > Best, > Zhanghao Chen > > From: Yu Chen > Sent: Monday, October 9, 2023 17:24 > To: dev@

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-09 Thread Zhanghao Chen
nt: Monday, October 9, 2023 17:24 To: dev@flink.apache.org Subject: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers Hi all, Yun Tang and I are opening this thread to discuss our proposal to integrate async-profiler's capabilities for profiling ta

[DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-09 Thread Yu Chen
Hi all, Yun Tang and I are opening this thread to discuss our proposal to integrate async-profiler's capabilities for profiling taskmananger (e.g., generating flame graphs) in the Flink Web [1]. Currently, Flink provides ThreadDump and Operator-Level Flame Graphs by sampling task threads. The re