Sergey, looks good. Can you also provide a list of metrics that you have
removed?
I think it's best to include this into the Jira ticket.
D.
On Thu, Mar 16, 2017 at 10:22 AM, Sergey Chugunov <sergey.chugu...@gmail.com
> wrote:
> Denis,
>
> Here is a version of CacheMetrics interface with all changes how I see them
> (pretty long list :)).
>
> public interface CacheMetrics {
>
> public long getCacheHits();
>
> public float getCacheHitPercentage();
>
> public long getCacheMisses();
>
> public float getCacheMissPercentage();
>
> public long getCacheGets();
>
> public long getCachePuts();
>
> public long getCacheRemovals();
>
> public long getCacheEvictions();
>
> public float getAverageGetTime();
>
> public float getAveragePutTime();
>
> public float getAverageRemoveTime();
>
> public float getAverageTxCommitTime();
>
> public float getAverageTxRollbackTime();
>
> public long getCacheTxCommits();
>
> public long getCacheTxRollbacks();
>
> public String name();
>
> public long getOverflowSize();
>
> public long getOffHeapGets();
>
> public long getOffHeapPuts();//removing as it duplicates cachePuts
>
> public long getOffHeapRemovals();
>
> public long getOffHeapEvictions();
>
> public long getOffHeapHits();
>
> public float getOffHeapHitPercentage();
>
> public long getOffHeapMisses();//removing as it duplicates cacheMisses
>
> public float getOffHeapMissPercentage();//removing as it duplicates
> cacheMissPercentage
>
> public long getOffHeapEntriesCount();
>
> public long getOffHeapPrimaryEntriesCount();
>
> public long getOffHeapBackupEntriesCount();
>
> public long getOffHeapAllocatedSize();
>
> public long getOffHeapMaxSize();
>
> public long getSwapGets();
>
> public long getSwapPuts();
>
> public long getSwapRemovals();
>
> public long getSwapHits();
>
> public long getSwapMisses();
>
> public long getSwapEntriesCount();
>
> public long getSwapSize();
>
> public float getSwapHitPercentage();
>
> public float getSwapMissPercentage();
>
> public int getSize();
>
> public int getKeySize();
>
> public boolean isEmpty();
>
> public int getDhtEvictQueueCurrentSize();
>
> public int getTxThreadMapSize();
>
> public int getTxXidMapSize();
>
> public int getTxCommitQueueSize();
>
> public int getTxPrepareQueueSize();
>
> public int getTxStartVersionCountsSize();
>
> public int getTxCommittedVersionsSize();
>
> public int getTxRolledbackVersionsSize();
>
> public int getTxDhtThreadMapSize();
>
> public int getTxDhtXidMapSize();
>
> public int getTxDhtCommitQueueSize();
>
> public int getTxDhtPrepareQueueSize();
>
> public int getTxDhtStartVersionCountsSize();
>
> public int getTxDhtCommittedVersionsSize();
>
> public int getTxDhtRolledbackVersionsSize();
>
> public boolean isWriteBehindEnabled();
>
> public int getWriteBehindFlushSize();
>
> public int getWriteBehindFlushThreadCount();
>
> public long getWriteBehindFlushFrequency();
>
> public int getWriteBehindStoreBatchSize();
>
> public int getWriteBehindTotalCriticalOverflowCount();
>
> public int getWriteBehindCriticalOverflowCount();
>
> public int getWriteBehindErrorRetryCount();
>
> public int getWriteBehindBufferSize();
>
> public String getKeyType();
>
> public String getValueType();
>
> public boolean isStoreByValue();
>
> public boolean isStatisticsEnabled();
>
> public boolean isManagementEnabled();
>
> public boolean isReadThrough();
>
> public boolean isWriteThrough();
>
> public long getTotalAllocatedPages();
>
> public long getTotalEvictedPages();
>
> }
>
>
> Also I suggest to introduce new interface for MemoryPolicy metrics and make
> it available through *IgniteCacheDatabaseSharedManager*:
>
>
> public interface IgniteMemoryPolicyMetrics {
>
> /**
>
> * @return Memory policy name.
>
> */
>
> public String getName();
>
>
> /**
>
> * @return Total number of allocated pages.
>
> */
>
> public long getTotalAllocatedPages();
>
>
> /**
>
> * @return Amount (in bytes) of not yet allocated space in PageMemory.
>
> */
>
> public long getAvailableSpace();
>
>
> /**
>
> * @return Number of allocated pages per second within PageMemory.
>
> */
>
> public float getAllocationRate();
>
>
> /**
>
> * @return Number of evicted pages per second within PageMemory.
>
> */
>
> public float getEvictionRate();
>
>
> /**
>
> * Large entities bigger than page are split into fragments so each
> fragment can fit into a page.
>
> *
>
> * @return Percentage of pages fully occupied by large entities.
>
> */
>
> public long getLargeEntriesPagesPercentage();
>
>
> //---FreeList-related metrics
>
>
> /**
>
> * @return Free space to overall size ratio across all pages in
> FreeList.
>
> */
>
> public float getPagesFillFactor();
>
>
> /**
>
> * @return Percentage of pages in FreeList with free space >= 8 and <
> 16 bytes
>
> */
>
> public float getPagesPercentage_8_16_freeBytes();
>
>
> /**
>
> * @return Percentage of pages in FreeList with free space >= 16 and <
> 64 bytes
>
> */
>
> public float getPagesPercentage_16_64_freeBytes();
>
>
> /**
>
> * @return Percentage of pages in FreeList with free space >= 64 and <
> 256 bytes
>
> */
>
> public float getPagesPercentage_64_256_freeBytes();
>
> }
>
> In my mind last three methods provide some kind of hist to give an insight
> about memory fragmentation.
> If there are a lot of pages with relatively big free chunks and less with a
> smaller chunks it may indicate that memory is fragmented and it may be
> reasonable to adjust page sizes.
>
> Thanks,
> Sergey.
>
>
>
> On Thu, Mar 16, 2017 at 1:29 AM, Denis Magda <dma...@apache.org> wrote:
>
> > Hi Sergey,
> >
> > >> In memory management scheme based on MemoryPolicies it may be useful
> > (and
> > >> easier) to collect some metrics not for individual caches but for
> whole
> > >> MemoryPolicies where several caches may reside.
> > >>
> >
> > I would collect the metrics for every single MemoryPolicy as well as for
> > individual caches. It makes sense to expose which cache contributes more
> to
> > memory utilization.
> >
> > >> - free space / used space tracking;
> > >> - allocation / eviction rate;
> >
> > Please consider this as well:
> > - total number of pages;
> > - total number of enters (how hard to support?).
> >
> > >> - metrics to track memory fragmentation: e.g. % of pages with only 8
> > >> bytes free, 16 bytes free and so on;
> > >> - % of big fragmented entries in cache: may be useful to adjust page
> > >> size.
> >
> > >
> > How do you see this in the metrics interface?
> >
> >
> > > 3. Useful, not going to remove:
> > > getOffHeapGets //useful as there still may be deserialized entries
> > > residing on-heap
> > > getOffHeapHitPercentage
> > > getOffHeapHits //overall hits include offheap and onheap
> > > getOffHeapMisses //I think in new model is the same as getCacheMisses
> > > getOffHeapMissPercentage //same as above
> > > getOffHeapPuts //same as above
> > > getOffHeapRemovals //same as above
> >
> > Could you please prepare an updated version of the cache metrics adding
> > new methods and renaming existing ones (only if necessary)? It will be
> > simpler to keep up the discussion relying on this updated interface.
> >
> > —
> > Denis
> >
> > > On Mar 15, 2017, at 8:32 AM, Sergey Chugunov <
> sergey.chugu...@gmail.com>
> > wrote:
> > >
> > > Also I looked through current set of metrics available on
> > > *CacheMetrics *interface
> > > and suggest following changes:
> > >
> > >
> > > 1. All methods related to tracking swap space (including
> > > *getOverflowSize*) to be removed.
> > >
> > > 2. Useless/hard to calculate in new memory management approach:
> > > getOffHeapAllocatedSize //max size is constrained by MemoryPolicy
> > config
> > > getOffHeapEntriesCount //all cache entries live offheap
> > > getOffHeapEvictions //will be captured on MemoryPolicyMetrics level;
> > > getOffHeapMaxSize //same as the first one
> > >
> > > 3. Useful, not going to remove:
> > > getOffHeapGets //useful as there still may be deserialized entries
> > > residing on-heap
> > > getOffHeapHitPercentage
> > > getOffHeapHits //overall hits include offheap and onheap
> > > getOffHeapMisses //I think in new model is the same as getCacheMisses
> > > getOffHeapMissPercentage //same as above
> > > getOffHeapPuts //same as above
> > > getOffHeapRemovals //same as above
> > >
> > > Please share your thought if I miss something here.
> > >
> > > Thanks,
> > > Sergey Chugunov.
> > >
> > > On Wed, Mar 15, 2017 at 4:51 PM, Sergey Chugunov <
> > sergey.chugu...@gmail.com>
> > > wrote:
> > >
> > >> Hello Igniters,
> > >>
> > >> As part of [1] cache metrics need to be updated as some of them like
> > swap
> > >> hits are not applicable anymore.
> > >>
> > >> In memory management scheme based on MemoryPolicies it may be useful
> > (and
> > >> easier) to collect some metrics not for individual caches but for
> whole
> > >> MemoryPolicies where several caches may reside.
> > >>
> > >> I suggest the following list of new metrics to collect for each
> > >> MemoryPolicy:
> > >>
> > >> - free space / used space tracking;
> > >> - allocation / eviction rate;
> > >> - metrics to track memory fragmentation: e.g. % of pages with only 8
> > >> bytes free, 16 bytes free and so on;
> > >> - % of big fragmented entries in cache: may be useful to adjust page
> > >> size.
> > >>
> > >>
> > >> Please suggest any other metrics that may be worth tracking.
> > >>
> > >> [1] https://issues.apache.org/jira/browse/IGNITE-3477
> > >>
> > >> Thanks,
> > >> Sergey Chugunov.
> > >>
> >
> >
>
