[
https://issues.apache.org/jira/browse/IGNITE-12096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16914510#comment-16914510
]
Colin Cassidy commented on IGNITE-12096:
----------------------------------------
Thanks for the response. Some observations:
* I'm using the technique recommended to me by GG support - but happy to be
corrected.
* The memory usage calculation page recommends using DataStorageMetrics. This
returns null for me even with setMetricsEnabled(true) - presumably because I am
not using native persistence.
* The above code worked fine up to Ignite 2.6 - so I assume there has been
some change to the purging logic to cause this.
* If using the DataRegion allocatedSize, it appears not to be account for the
fill factor - so this value doesn't drop on cache purge even in Ignite 2.6.
* My entries are 1MB each - comfortably larger than the page size. So I expect
fragmentation is probably be minimal.
* Memory is not reported as free even when my cache is destroyed.
* If I remove all entries from the cache and then write them back again, the
memory usage stays static - it doesn't drop at any point, even if the cache was
close to full. The memory must be reclaimed at some point before it is reused -
or is the problem that they are overwritten and never actually purged? Prior to
2.7, I would see the fill factor drop to near 0 indicating that the pages are
still allocated but are now considered to be empty.
For many use cases, it's important to have a timely and reasonably accurate
estimate of memory usage because in a pure in-memory configuration (no native
persistence) there is no other way to avoid an OOM condition. OOM is considered
a critical error and causes the node to stop. Although this can be overridden,
I am told that this is not a good idea.
> Ignite memory metrics incorrect on cache usage contraction
> ----------------------------------------------------------
>
> Key: IGNITE-12096
> URL: https://issues.apache.org/jira/browse/IGNITE-12096
> Project: Ignite
> Issue Type: Bug
> Components: cache
> Affects Versions: 2.7
> Reporter: Colin Cassidy
> Priority: Critical
>
> When using the Ignite metrics API to measure available memory, the usage
> figures appear to be accurate while memory is being consumed - but when
> memory is freed the metrics do not drop. They appear to report that memory
> has not been freed up, even though it has.
> Reproducer below. This affects Ignite 2.7+.
> {{}}{{import org.apache.ignite.failure.NoOpFailureHandler; }}
> {{import org.junit.Test; }}
> {{public class MemoryTest2 { }}
> {{ private static final String CACHE_NAME = "cache"; }}
> {{ private static final String DEFAULT_MEMORY_REGION = "Default_Region"; }}
> {{ private static final long MEM_SIZE = 100L * 1024 * 1024; }}
> {{ @Test }}
> {{ public void testOOM() throws InterruptedException { }}
> {{ try (Ignite ignite = startIgnite("IgniteMemoryMonitorTest1")) { }}
> {{ fillDataRegion(ignite); }}
> {{ CacheConfiguration<Object, Object> cfg = new }}
> {{CacheConfiguration<>(CACHE_NAME); }}
> {{ cfg.setStatisticsEnabled(true); }}
> {{ IgniteCache<Object, Object> cache = }}
> {{ignite.getOrCreateCache(cfg); }}
> {{ // Clear all entries from the cache to free up memory }}
> {{ memUsed(ignite); }}
> {{ cache.clear(); }}
> {{ cache.removeAll(); }}
> {{ cache.put("Key", "Value"); }}
> {{ memUsed(ignite); }}
> {{ cache.destroy(); }}
> {{ Thread.sleep(5000); }}
> {{ // Should now report close to 0% but reports 59% still }}
> {{ memUsed(ignite); }}
> {{ } }}
> {{ } }}
> {{ }}
> {{ private Ignite startIgnite(String instanceName) { }}
> {{ IgniteConfiguration cfg = new IgniteConfiguration(); }}
> {{ cfg.setIgniteInstanceName(instanceName); }}
> {{ cfg.setDataStorageConfiguration(createDataStorageConfiguration());
> }}
> {{ cfg.setFailureHandler(new NoOpFailureHandler()); }}
> {{ return Ignition.start(cfg); }}
> {{ } }}
> {{ private DataStorageConfiguration createDataStorageConfiguration() { }}
> {{ return new DataStorageConfiguration() }}
> {{ .setDefaultDataRegionConfiguration( }}
> {{ new DataRegionConfiguration() }}
> {{ .setName(DEFAULT_MEMORY_REGION) }}
> {{ .setInitialSize(MEM_SIZE) }}
> {{ .setMaxSize(MEM_SIZE) }}
> {{ .setMetricsEnabled(true)); }}
> {{ } }}
> {{ private void fillDataRegion(Ignite ignite) { }}
> {{ byte[] megabyte = new byte[1024 * 1024]; }}
> {{ IgniteCache<Object, Object> cache = }}
> {{ ignite.getOrCreateCache(CACHE_NAME); }}
> {{ for (int i = 0; i < 50; i++) { }}
> {{ cache.put(i, megabyte); }}
> {{ memUsed(ignite); }}
> {{ } }}
> {{ } }}
> {{ private void memUsed(Ignite ignite) { }}
> {{ DataRegionConfiguration defaultDataRegionCfg = }}
> {{ignite.configuration() }}
> {{ .getDataStorageConfiguration() }}
> {{ .getDefaultDataRegionConfiguration(); }}
> {{ String regionName = defaultDataRegionCfg.getName(); }}
> {{ DataRegionMetrics metrics = ignite.dataRegionMetrics(regionName); }}
> {{ float usedMem = metrics.getPagesFillFactor() * }}
> {{metrics.getTotalAllocatedPages() * metrics.getPageSize(); }}
> {{ float pctUsed = 100 * usedMem / defaultDataRegionCfg.getMaxSize();
> }}
> {{ System.out.println("Memory used: " + pctUsed + "%"); }}
> {{ } }}
> {{} }}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
