Thanks for drafting the FLIP and starting this discussion, Yadong.
I have some comments:
- I can see that the proposed memory and cpu usage to be displayed (in
section 1.1) are aligned with the current ResourceProfile fields. However,
we are working on changing the memory fields in 1.10 with FLIP-49 [1]. I
suggest we align the UI design with the new FLIP-49 memory fields.
- The task executor overview design (in section 1.2) is based on the
current slot model. The coming FLIP-56 [2] which is also planned for 1.10
is changing the model so that task executors no longer have fixed number of
slots, but allocated slots (may have different resources) and available
resources.
- I can see that there's discussions in the google doc about using
different color for available resources. However, the resource
availability
for different fields can be different, and may not be simply
displayed by a
different color. E.g., a task executor may have two slot, while slot 1
takes (20% cpu, 10% heap mem, 50% managed mem, etc.), slot 2 takes (10%
cpu, 35% heap mem, 0% managed mem etc.), and the remaining resources in
the task executor are (70% cpu, 55% heap mem, 50% managed mem, etc.). How
do you plan to display that?
- I would suggest to have multiple bars for each task executor, while
each bar represents one of the resource fields. In addition, we
may have a
number (or some other figures) showing how many slots are allocated from
the task executor.
- Is there any way we provide access to logs of terminated task
executors? It occurs to us a lot that a job failed due to a task executor
fail/lost. And we have to find the logs of failed task executors by
manually accessing the file system. I think it would be helpful if we can
find the logs of failed task executors directly in flink webui.
- Regarding log pagination, is there any way to provide keyword
searching across all the pages?
Thank you~
Xintong Song
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-49%3A+Unified+Memory+Configuration+for+TaskExecutors
[2]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-56%3A+Dynamic+Slot+Allocation
On Fri, Sep 27, 2019 at 3:57 PM Paul Lam <paullin3...@gmail.com> wrote:
> Filed a jira to track this[1]. Thanks a lot.
>
> [1] https://issues.apache.org/jira/browse/FLINK-14242 <
> https://issues.apache.org/jira/browse/FLINK-14242>
>
> Best,
> Paul Lam
>
> > 在 2019年9月27日，14:34，Yadong Xie <vthink...@gmail.com> 写道：
> >
> > Hi Paul
> > Thanks for your suggestion.
> > I think it is easy to implement, could you create a JIRA for me?
> >
> > Paul Lam <paullin3...@gmail.com> 于2019年9月27日周五 上午11:11写道：
> >
> >> Hi Yadong,
> >>
> >> Thanks a lot for summing up the Web UI efforts.
> >>
> >> I have a minor suggestion: can we provide a collapse button for the task
> >> names in job graph visualization? For some complex jobs, especially SQL
> >> jobs, the task names are quite long which makes the job graph hard to
> read.
> >>
> >> Best,
> >> Paul Lam
> >>
> >>> 在 2019年9月27日，10:13，Yadong Xie <vthink...@gmail.com> 写道：
> >>>
> >>> Hi all
> >>>
> >>> Flink Web UI is the main platform for most users to monitor their jobs
> >> and
> >>> clusters. We have reconstructed Flink web in 1.9.0 version, but there
> are
> >>> still some shortcomings.
> >>>
> >>> This discussion thread aims to provide a better experience for Flink UI
> >>> users.
> >>>
> >>> Here is the design doc I drafted:
> >>>
> >>>
> >>
> https://docs.google.com/document/d/1tIa8yN2prWWKJI_fa1u0t6h1r6RJpp56m48pXEyh6iI/edit?usp=sharing
> >>>
> >>>
> >>> The FLIP can be found at [2].
> >>>
> >>> Please keep the discussion here, in the mailing list.
> >>>
> >>> Looking forward to your opinions, any feedbacks are welcome.
> >>>
> >>> [1]:
> >>>
> >>
> https://docs.google.com/document/d/1tIa8yN2prWWKJI_fa1u0t6h1r6RJpp56m48pXEyh6iI/edit?usp=sharing
> >>> <
> >>
> https://docs.google.com/document/d/1tIa8yN2prWWKJI_fa1u0t6h1r6RJpp56m48pXEyh6iI/edit#
> >>>
> >>> [2]:
> >>>
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-75%3A+Flink+Web+UI+Improvement+Proposal
> >>
> >>
>
>
