Hello, Igniters. I assigned ticket [1] created by Denis and want to clarify how to log committed size. The metric offHeapSize (in DataRegionMetricsImpl) is always calculated, but getOffHeapSize returns zero if memory metrics are disabled for this data region.
So I see the following options: 1. Modify method getOffHeapSize so that it always returns actual value offHeapSize. 2. Add another offHeapSize() method. 3. Output to log max size instead of committed (change "comm" to "max" in log output). 4. Don't bother about disabling metrics and output to log value returned by getOffHeapSize. Any thoughts? [1] https://issues.apache.org/jira/browse/IGNITE-9305 сб, 18 авг. 2018 г. в 3:17, Denis Magda <dma...@apache.org>: > > Vova, the things are even simpler - we have this > > ignite.dataRegionMetrics().getPhysicalMemorySize() that returns the > number equal/comparabel to pageNumber X pageSize. > > > Igniters, if you believe that we need to do more work here then let's > do it iteratively. Let's fix the off-heap occupied size the way above > (just print out getPhysicalMemorySize() for every data region). Then > do the rest. This needs to be fixed in 2.7. > > > -- > > Denis > > > On Fri, Aug 17, 2018 at 10:20 AM Vladimir Ozerov <voze...@gridgain.com> > wrote: > > > Folks, > > > > We already have this: > > >>> PageMemory [pages=6997377] > > > > Then we can multiply it by page size and get occupied memory. Am I wrong? > > > > On Fri, Aug 17, 2018 at 12:56 PM Dmitriy Pavlov <dpavlov....@gmail.com> > > wrote: > > > > > Hi Maxim, > > > > > > thank you for stepping in and for finding these issues. Yes, these > > tickets > > > are correct. > > > > > > I can move https://issues.apache.org/jira/browse/IGNITE-5583 to > > unassigned > > > if someone would like to implement this change. I will not have enough > > time > > > to complete it in 1 month (before 2.7 release). > > > > > > Sincerely, > > > Dmitriy Pavlov > > > > > > пт, 17 авг. 2018 г. в 11:04, Maxim Muzafarov <maxmu...@gmail.com>: > > > > > > > Igniters, > > > > > > > > Suppose, Dmitry is talking about IGNITE-5583 [1] - `Switch non-heap > > > memory > > > > metrics > > > > to new page memory semantics` and related previous disscustions to it > > > [4]. > > > > > > > > Also we have some additional improvements to CacheMetrics: > > > > IGNITE-5490 [2] - `Implement replacement for obsolete > > > > CacheMetrics#getOffHeapAllocatedSize` > > > > IGNITE-5765 [3] - `CacheMetrics interface cleanup, documentation and > > > fixes` > > > > > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-5583 > > > > [2] https://issues.apache.org/jira/browse/IGNITE-5490 > > > > [3] https://issues.apache.org/jira/browse/IGNITE-5765 > > > > [4] > > > > > > > > > > > > > http://apache-ignite-developers.2346864.n4.nabble.com/Negative-non-heap-memory-maximum-td17990.html > > > > > > > > On Fri, 17 Aug 2018 at 10:14 Dmitriy Pavlov <dpavlov....@gmail.com> > > > wrote: > > > > > > > > > Hi Igniters, > > > > > > > > > > It is not an easy fix, so I'm not sure it is possible to do in 2.7. > > > > > > > > > > Offheap size is not reported by VM (it returns -1). To implement it > > we > > > > need > > > > > totally migrate off-heap memory metrics to durable memory data. > > > > > > > > > > I think this issue was reported and I'll find the duplicate. > > > > > > > > > > Sincerely, > > > > > Dmitriy Pavlov > > > > > > > > > > пт, 17 авг. 2018 г. в 6:10, Denis Magda <dma...@apache.org>: > > > > > > > > > > > Yes, it was at the end of my wordy email :) > > > > > > https://issues.apache.org/jira/browse/IGNITE-9305 > > > > > > > > > > > > -- > > > > > > Denis > > > > > > > > > > > > On Thu, Aug 16, 2018 at 11:03 PM Dmitriy Setrakyan < > > > > > dsetrak...@apache.org> > > > > > > wrote: > > > > > > > > > > > > > Is there a blocker ticket for 2.7? > > > > > > > > > > > > > > On Thu, Aug 16, 2018, 19:59 Denis Magda <dma...@apache.org> > > wrote: > > > > > > > > > > > > > > > Igniters, > > > > > > > > > > > > > > > > Was troubleshooting an Ignite deployment today and couldn't > > find > > > > out > > > > > > from > > > > > > > > the logs what was the actual off-heap space used. > > > > > > > > > > > > > > > > Those were the given memory resoures (Ignite 2.6): > > > > > > > > > > > > > > > > [2018-08-16 15:07:49,961][INFO ][main][GridDiscoveryManager] > > > > Topology > > > > > > > > snapshot [ver=1, servers=1, clients=0, CPUs=64, > > *offheap=30.0GB*, > > > > > > > > heap=24.0GB] > > > > > > > > > > > > > > > > And that weird stuff was reported by the node (pay attention to > > > the > > > > > > last > > > > > > > > line): > > > > > > > > > > > > > > > > [2018-08-16 15:45:50,211][INFO > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ][grid-timeout-worker-#135%cluster_31-Dec-2017%][IgniteKernal%cluster_31-Dec-2017] > > > > > > > > Metrics for local node (to disable set 'metricsLogFrequency' to > > > 0) > > > > > > > > ^-- Node [id=c033026e, name=cluster_31-Dec-2017, > > > > > > uptime=00:38:00.257] > > > > > > > > ^-- H/N/C [hosts=1, nodes=1, CPUs=64] > > > > > > > > ^-- CPU [cur=0.03%, avg=5.54%, GC=0%] > > > > > > > > ^-- PageMemory [pages=6997377] > > > > > > > > ^-- Heap [used=9706MB, free=61.18%, comm=22384MB] > > > > > > > > * ^-- Non heap [used=144MB, free=-1%, comm=148MB] - this > > line > > > is > > > > > > > always > > > > > > > > the same!* > > > > > > > > > > > > > > > > Had to change the code by using > > > dataRegion.getPhysicalMemoryPages() > > > > > to > > > > > > > find > > > > > > > > out that actual off-heap usage size was > > > > > > > > >>> Physical Memory Size: 28651614208 => 27324 MB, 26 GB > > > > > > > > > > > > > > > > Let's fix this issue in 2.7, I proposed a new format. Please > > > review > > > > > and > > > > > > > > share your thoughts: > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-9305 > > > > > > > > > > > > > > > > -- > > > > > > > > Denis > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > -- > > > > Maxim Muzafarov > > > > > > > > >
