ary 11, 2022 10:11
To: dev@flink.apache.org
Subject: RE: Re: [DISCUSS] FLIP-213: TaskManager's Flame Graphs
Our flink application is on k8s.Yes, user can use the async-profiler directly,
but it is not convenient for user, who should download the jars and need to
know how to use it. And some
Hi Jacky,
Some performance improvements for Flame Graphs related to how the stack
traces are collected were recently merged [1]. Since you are working on a
real-life environment with high parallelism, could you maybe run a couple
of tests to verify that everything still works as expected after the
Hi Jacky,
I think it’s a very useful feature, can get flame graph easily and avoid
run script personally.
But I think can package async-profiler library into flink, and use JNI
execute the profiling, doesn’t depend on local environment.
We did this way in out internal version, and provide api to do
Pyroscope[1] and Parca[2] are other options for less-intrusive profiling (&
great fits for k8s) that move the burden from Flink & its UI to tools that
are purpose-built for this use case. Perhaps we could investigate what it
would take (if anything) to make Flink compatible with those?
Best,
Austi
Are you sure the UI is the bottleneck? The UI gets back a JSON
representation of this data structure:
https://github.com/apache/flink/blob/2e21321f9c9d9aada7e4ad8ca90d915c34f58015/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/threadinfo/JobVertexFlameGraph.java
All samples from
There are already tools [1] that simplify this for the user.
I honestly don't know, it feels like it can bring more problems that actual
benefits as this heavily relies on the environment. It can easily break for
some users, eg. because of the kernel settings; their architecture might
not be suppo
Our flink application is on k8s.Yes, user can use the async-profiler directly,
but it is not convenient for user, who should download the jars and need to
know how to use it. And some users don’t know the tool.if we integrate it, user
will benefit a lot.
On 2022/01/26 18:56:17 David Morávek wro
Our flink application is on k8s.Yes, user can use the async-profiler directly,
but it is not convenient for user, who should download the jars and need to
know how to use it. And some users don’t know the tool.if we integrate it, user
will benefit a lot.
On 2022/01/26 18:56:17 David Morávek wro
Our flink application is on k8s.Yes, user can use the async-profiler directly,
but it is not convenient for user, who should download the jars and need to
know how to use it. And some users don’t know the tool.if we integrate it, user
will benefit a lot.
On 2022/01/26 18:56:17 David Morávek wro
Hi Alexander:
Sorry for late response for Chinese Spring Festival.
The bottleneck is rendering on the browser side.
For 1) we support user define script capability like yarn. And the flame
graph script just encapsulate async profiler. So we should make it secure.
For 2) yeah, we use di
Hi jacky,
Thanks for bring up this discussion, I think it's an useful feature
which can make the performance tuning more portable, +1 for this
Best,
Aitozi
Jacky Lau <281293...@qq.com.invalid> 于2022年1月24日周一 16:48写道：
> Hi All,
> I would like to start the discussion on FLIP-213 <
> https:
-platforms problems?
Best,
Yun Tang
From: David Morávek
Sent: Thursday, January 27, 2022 2:56
To: dev
Subject: Re: [DISCUSS] FLIP-213: TaskManager's Flame Graphs
I'd second to Alex's concerns. Is there a reason why you can't use the
async-p
I'd second to Alex's concerns. Is there a reason why you can't use the
async-profiler directly? In what kind of environment are your Flink
clusters running (YARN / k8s / ...)?
Best,
D.
On Wed, Jan 26, 2022 at 4:32 PM Alexander Fedulov
wrote:
> Hi Jacky,
>
> Could you please clarify what kind of
Hi Jacky,
Could you please clarify what kind of *problems* you experience with the
large parallelism? You referred to D3, is it something related to rendering
on the browser side or is it about the samples collection process? Were you
able to identify the bottleneck?
Fundamentally I have some con
This is an expected feature, as we also experienced browser crashes on existing
operator-level flame graphs
Best,
Echo Lee
> 在 2022年1月24日，下午6:16，David Morávek 写道：
>
> Hi Jacky,
>
> The link seems to be broken, here is the correct one [1].
>
> [1]
> https://cwiki.apache.org/confluence/displa
Hi Jacky,
The link seems to be broken, here is the correct one [1].
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-213%3A+TaskManager%27s+Flame+Graphs
Best,
D.
On Mon, Jan 24, 2022 at 9:48 AM Jacky Lau <281293...@qq.com.invalid> wrote:
> Hi All,
> I would like to start the disc
16 matches
Mail list logo
