Hi Xintong thanks for your advice, the POC web and the FLIP doc was updated now here is the new link: http://101.132.122.69:8081/web/#/task-manager/7e7cf0293645c8537caab915c829aa73/metrics
Xintong Song <tonysong...@gmail.com> 于2020年2月21日周五 下午12:00写道： > > > > 1. Should the managed memory be part of direct memory? > > > The answer is no. Managed memory is currently allocated by accessing to > private field of Unsafe. It is not accounted for in JVM's direct memory > limit and corresponding metrics. To that end, it is equivalent to > native memory. > > > > 2. Should the shuffle memory also be part of the managed memory? > > I don't think so. Shuffle (Network) memory is allocated with direct > buffers, and accounted for in JVM's direct memory limit and corresponding > metrics. Moreover, the FLIP-49 memory model expose network memory and > managed memory as two independent components of the overall memory > footprint. > > > Thank you~ > > Xintong Song > > > > On Fri, Feb 21, 2020 at 11:45 AM Kurt Young <ykt...@gmail.com> wrote: > > > Some questions related to "managed memory": > > > > 1. Should the managed memory be part of direct memory? > > 2. Should the shuffle memory also be part of the managed memory? > > > > Best, > > Kurt > > > > > > On Fri, Feb 21, 2020 at 10:41 AM Xintong Song <tonysong...@gmail.com> > > wrote: > > > > > Thanks for driving this FLIP, Yadong. > > > > > > +1 (non-binding) for the FLIP in general. I think this really helps our > > > users to understand and use the new FLIP-49 memory configuration. > > > > > > I have a few minor comments. > > > - There's a frame "Other" in the frame "Non-Heap", besides "JVM > Overhead" > > > and "JVM Metaspace". IIUC, the purpose of this is to explain the > > > mismatching between the metric "non-heap maximum" and the sum of the > > > configurations "JVM metaspace" & "JVM Overhead". However, from the > > > perspective of FLIP-49, JVM Overhead accounts for all the JVM non-heap > > > memory usages except for metaspace. The metrics does not match the > > > configuration because we did not set the a JVM parameter for "max > > non-heap > > > memory" (actually I'm not sure whether it can be specified in java 8). > > The > > > current UI might confuse people making them think there are other > > non-heap > > > memory usages not accounted by the configurations. Therefore, I would > > > suggest to remove the "Other" frame, but add another frame inside "JVM > > > Overhead", besides "Configuration", with "JVM limit" as the title and > > > "non-heap max metric minus metaspace configuration" as the value . > > > > > > - In the final release, we have changed "shuffle memory" to "network > > > memory" because the latter is easier to understand for users. I think > we > > > should be updated it in this FLIP as well. > > > > > > - There's a typo "Directed" (should be "Direct") at the direct memory > > > metric. > > > > > > Thank you~ > > > > > > Xintong Song > > > > > > > > > > > > On Thu, Feb 20, 2020 at 5:52 PM Yadong Xie <vthink...@gmail.com> > wrote: > > > > > > > Hi all > > > > > > > > I want to start the vote for FLIP-102, which proposes to add more > > metrics > > > > to the task manager in web UI. > > > > > > > > To help everyone better understand the proposal, we spent some > efforts > > on > > > > making an online POC > > > > > > > > previous web: > > > > > > > > > > > > > > http://101.132.122.69:8081/#/task-manager/6df6c5f37b2bff125dbc3a7388128559/metrics > > > > POC web: > > > > > > > > > > > > > > http://101.132.122.69:8081/web/#/task-manager/6df6c5f37b2bff125dbc3a7388128559/metrics > > > > > > > > > > > > The vote will last for at least 72 hours, following the consensus > > voting > > > > process. > > > > > > > > FLIP wiki: > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-102%3A+Add+More+Metrics+to+TaskManager > > > > > > > > Discussion thread: > > > > > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html > > > > > > > > Thanks, > > > > > > > > Yadong > > > > > > > > > >
