@Jark
First, let me try to clarify that, while this FLIP is about adding JM
metrics, the discussion of having different colors distinguishing the
memory usage applies for both JM and TM.
IMO, I don't think there's a good way to define how should memory
utilization be mapped to colors in general.
- Direct memory
- JM: ATM, we do not specify -XX:MaxDirectMemorySize.
- TM: Direct memory consists of network memory and framework/task
off-heap memory, the former should always be 100% while the
latter may not.
Therefore, the utilization of direct memory really depends on the
configured size of network memory and framework/task off-heap memory.
- Heap memory: We might observe that the memory usage keeps growing
until GC is triggered, thus eventually the utilization might fluctuates at
somewhere close to 100%.
In general, a low memory utilization probably suggests that the memory size
is configured too large, but a high memory utilization does not necessarily
suggest the configured memory size need to be increased, thus, not sure
about rendering it in red.
Thank you~
Xintong Song
On Tue, Feb 25, 2020 at 3:13 PM Yadong Xie <vthink...@gmail.com> wrote:
> Hi all
> we have updated the POC web, and added unit to GC metrics
> check it here http://101.132.122.69:8081/web/#/job-manager/metrics
> thanks for all the response
>
> Jark Wu <imj...@gmail.com> 于2020年2月24日周一 下午8:48写道：
>
>> Hi Yadong,
>>
>> > what is the boundary between red and green?
>> Yes. I think that's the point we need to discuss. My gut feeling is "<60%"
>> => green, "60%~80%" => yellow, ">80%" => red.
>> But I guess directed memory is always 100%, so it is not suitable for
>> that?
>> Maybe @Xintong Song <tonysong...@gmail.com> has a better understanding on
>> the memory threshold.
>>
>> Best,
>> Jark
>>
>> On Mon, 24 Feb 2020 at 15:41, Yadong Xie <vthink...@gmail.com> wrote:
>>
>> > Hi Jark
>> > thanks for your suggestion
>> >
>> > > I think we can use different color to distinguish the memory usage
>> (from
>> > green to red?).
>> >
>> > It is a good idea, but what is the boundary between red and green?
>> giving a
>> > magic number boundary may mislead the users. any suggestions?
>> >
>> > > Besides, I think we should add an unit on the "Garbage Collection" ->
>> > "Time", it's hard to know what the value mean. Would be better to
>> display
>> > the value like "10ms", "5ns".
>> >
>> > I will add the unit later, thanks for your advice.
>> >
>> >
>> > Xintong Song <tonysong...@gmail.com> 于2020年2月21日周五 下午6:02写道：
>> >
>> > > FYI, there's an effort planned for 1.11 to improve the memory
>> > configuration
>> > > of the Flink master process, similar to FLIP-49 but definitely less
>> > > complexity.
>> > >
>> > > I would not consider the memory configuration improvement as a blocker
>> > for
>> > > this effort. As far as I can see, there's nothing in conflict. Just
>> after
>> > > the memory configuration improvement, we might be able to present more
>> > > information on the JM metrics page, which are tightly corresponding to
>> > the
>> > > configuration options, like what we planned for the TM metrics page in
>> > > FLIP-102. Therefore, it might make sense to proceed this FLIP
>> afterwards.
>> > >
>> > > I'm neutral on this, and would leave the call to Yandong and Lining.
>> > >
>> > > Thank you~
>> > >
>> > > Xintong Song
>> > >
>> > >
>> > >
>> > > On Fri, Feb 21, 2020 at 2:47 PM Jark Wu <imj...@gmail.com> wrote:
>> > >
>> > > > Thanks Yadong,
>> > > >
>> > > > I think we can use different color to distinguish the memory usage
>> > (from
>> > > > green to red?).
>> > > > Besides, I think we should add an unit on the "Garbage Collection"
>> ->
>> > > > "Time", it's hard to know what the value mean.
>> > > > Would be better to display the value like "10ms", "5ns".
>> > > >
>> > > > Best,
>> > > > Jark
>> > > >
>> > > > On Thu, 20 Feb 2020 at 17:58, Yadong Xie <vthink...@gmail.com>
>> wrote:
>> > > >
>> > > > > Hi all
>> > > > >
>> > > > > I want to start the vote for FLIP-104, which proposes to add more
>> > > metrics
>> > > > > to job manager.
>> > > > >
>> > > > > To help everyone better understand the proposal, we spent some
>> > efforts
>> > > on
>> > > > > making an online POC
>> > > > >
>> > > > > previous web: http://101.132.122.69:8081/#/job-manager/config
>> > > > > POC web: http://101.132.122.69:8081/web/#/job-manager/metrics
>> > > > >
>> > > > >
>> > > > > The vote will last for at least 72 hours, following the consensus
>> > > voting
>> > > > > process.
>> > > > >
>> > > > > FLIP wiki:
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-104%3A+Add+More+Metrics+to+Jobmanager
>> > > > >
>> > > > > Discussion thread:
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
>> > > > >
>> > > > > Thanks,
>> > > > >
>> > > > > Yadong
>> > > > >
>> > > >
>> > >
>> >
>>
>
