Hi,
I used ignite in version 2.7.6 (but I have also seen this behaviour on
other 2.7.x versions) and there aren't any near or local cache.
I expect that if I ask distributed cache about key which does not exist
then the miss metric will be incremented.
śr., 11 mar 2020 o 11:35 Andrey Gura <ag...@apache.org> napisał(a):
> Denis,
>
> I don't sure that I understand what is expected behavior should be.
> There are local and aggregated cluster wide metrics. I don't know
> which one used by Visor because I never used it :)
>
> Also it would be great to know what version of Apache Ignite used in
> described case. I remember some bug with metrics aggregation during
> discovery metrics message round trip.
>
> On Wed, Mar 11, 2020 at 12:05 AM Denis Magda <dma...@apache.org> wrote:
> >
> > @Nikolay Izhikov <nizhi...@apache.org>, @Andrey Gura <ag...@apache.org>,
> > could you folks check out this thread?
> >
> > I have a feeling that what Dominik is describing was talked out before
> and
> > rather some sort of a limitation than an issue with the current
> > implementation.
> >
> > -
> > Denis
> >
> >
> > On Tue, Mar 3, 2020 at 11:41 PM Dominik Przybysz <alien11...@gmail.com>
> > wrote:
> >
> > > Hi,
> > > I am trying to use partitioned cache on server nodes to which I connect
> > > with client node. Statistics of cache in the cluster are updated, but
> only
> > > for hits metric - misses metric is always 0.
> > >
> > > To reproduce this problem I created cluster of two nodes:
> > >
> > > Server node 1 adds 100 random test cases and prints cache statistics
> > > continuously:
> > >
> > > public class IgniteClusterNode1 {
> > > public static void main(String[] args) throws InterruptedException
> {
> > > IgniteConfiguration igniteConfiguration = new
> > > IgniteConfiguration();
> > >
> > > CacheConfiguration cacheConfiguration = new
> CacheConfiguration();
> > > cacheConfiguration.setName("test");
> > > cacheConfiguration.setCacheMode(CacheMode.PARTITIONED);
> > > cacheConfiguration.setAtomicityMode(CacheAtomicityMode.ATOMIC);
> > > cacheConfiguration.setStatisticsEnabled(true);
> > > igniteConfiguration.setCacheConfiguration(cacheConfiguration);
> > >
> > > TcpCommunicationSpi communicationSpi = new
> TcpCommunicationSpi();
> > > communicationSpi.setLocalPort(47500);
> > > igniteConfiguration.setCommunicationSpi(communicationSpi);
> > >
> > > TcpDiscoverySpi discoverySpi = new TcpDiscoverySpi();
> > > discoverySpi.setLocalPort(47100);
> > > discoverySpi.setLocalPortRange(100);
> > > TcpDiscoveryVmIpFinder ipFinder = new TcpDiscoveryVmIpFinder();
> > > ipFinder.setAddresses(Arrays.asList("127.0.0.1:47100..47200",
> > > "127.0.0.1:48100..48200"));
> > > igniteConfiguration.setDiscoverySpi(discoverySpi);
> > >
> > > try (Ignite ignite = Ignition.start(igniteConfiguration)) {
> > > try (IgniteCache<String, String> cache =
> > > ignite.getOrCreateCache("test")) {
> > > new Random().ints(1000).map(i -> Math.abs(i %
> > > 1000)).distinct().limit(100).forEach(i -> {
> > > String key = "data_" + i;
> > > String value =
> UUID.randomUUID().toString();
> > > cache.put(key, value);
> > > }
> > > );
> > > }
> > > while (true) {
> > > System.out.println(ignite.cache("test").metrics());
> > > Thread.sleep(5000);
> > > }
> > > }
> > > }
> > > }
> > >
> > > Server node 2 only prints cache statistics continuously:
> > >
> > > public class IgniteClusterNode2 {
> > > public static void main(String[] args) throws InterruptedException
> {
> > > IgniteConfiguration igniteConfiguration = new
> > > IgniteConfiguration();
> > >
> > > CacheConfiguration cacheConfiguration = new
> CacheConfiguration();
> > > cacheConfiguration.setName("test");
> > > cacheConfiguration.setCacheMode(CacheMode.PARTITIONED);
> > > cacheConfiguration.setStatisticsEnabled(true);
> > > igniteConfiguration.setCacheConfiguration(cacheConfiguration);
> > >
> > > TcpCommunicationSpi communicationSpi = new
> TcpCommunicationSpi();
> > > communicationSpi.setLocalPort(48500);
> > > igniteConfiguration.setCommunicationSpi(communicationSpi);
> > >
> > > TcpDiscoverySpi discoverySpi = new TcpDiscoverySpi();
> > > discoverySpi.setLocalPort(48100);
> > > discoverySpi.setLocalPortRange(100);
> > > TcpDiscoveryVmIpFinder ipFinder = new TcpDiscoveryVmIpFinder();
> > > ipFinder.setAddresses(Arrays.asList("127.0.0.1:47100..47200",
> > > "127.0.0.1:48100..48200"));
> > > igniteConfiguration.setDiscoverySpi(discoverySpi);
> > >
> > > try (Ignite ignite = Ignition.start(igniteConfiguration)) {
> > > while (true) {
> > > System.out.println(ignite.cache("test").metrics());
> > > Thread.sleep(5000);
> > > }
> > > }
> > > }
> > > }
> > >
> > > Next I start a client node which continuously read data from the
> cluster:
> > >
> > > public class CacheClusterReader {
> > > public static void main(String[] args) throws InterruptedException
> {
> > > IgniteConfiguration cfg = new IgniteConfiguration();
> > > cfg.setClientMode(true);
> > >
> > > TcpDiscoverySpi spi = new TcpDiscoverySpi();
> > > TcpDiscoveryVmIpFinder tcMp = new TcpDiscoveryVmIpFinder();
> > > tcMp.setAddresses(Arrays.asList("127.0.0.1:47100..47200",
> > > "127.0.0.1:48100..48200"));
> > > spi.setIpFinder(tcMp);
> > > cfg.setDiscoverySpi(spi);
> > >
> > > CacheConfiguration<String, String> cacheConfig = new
> > > CacheConfiguration<>("test");
> > > cacheConfig.setStatisticsEnabled(true);
> > > cacheConfig.setCacheMode(CacheMode.PARTITIONED);
> > > cfg.setCacheConfiguration(cacheConfig);
> > >
> > > try (Ignite ignite = Ignition.start(cfg)) {
> > > System.out.println(ignite.cacheNames());
> > >
> > > while (true) {
> > > try (IgniteCache<String, String> cache =
> > > ignite.getOrCreateCache(cacheConfig)) {
> > > cache.enableStatistics(true);
> > >
> > > new Random().ints(1000).map(i -> Math.abs(i %
> > > 2000)).distinct().limit(100).forEach(i -> {
> > > String key = "data_" + i;
> > > String value = cache.get(key);
> > > System.out.println("Found " + key + "
> with
> > > value " + value);
> > > }
> > > );
> > >
> > > System.out.println(cache.metrics());
> > > System.out.println(cache.localMetrics());
> > > }
> > > Thread.sleep(3000);
> > > }
> > > }
> > > }
> > > }
> > >
> > > On server and client nodes cluster statistics are synchronized, but
> misses
> > > counter is always equal to 0, e. g.
> > >
> > > CacheMetricsSnapshot [reads=93, puts=100, entryProcessorPuts=0,
> > > entryProcessorReadOnlyInvocations=0,
> > > entryProcessorAverageInvocationTime=0.0, entryProcessorInvocations=0,
> > > entryProcessorRemovals=0, entryProcessorMisses=0, entryProcessorHits=0,
> > > entryProcessorMissPercentage=0.0, entryProcessorHitPercentage=0.0,
> > > entryProcessorMaxInvocationTime=0.0,
> entryProcessorMinInvocationTime=0.0, *hits=93,
> > > misses=0*, txCommits=0, txRollbacks=0, evicts=0, removes=0,
> > > putAvgTimeNanos=259.4794, getAvgTimeNanos=0.0, rmvAvgTimeNanos=0.0,
> > > commitAvgTimeNanos=0.0, rollbackAvgTimeNanos=0.0, cacheName=test,
> > > offHeapGets=0, offHeapPuts=0, offHeapRemoves=0, offHeapEvicts=0,
> > > offHeapHits=0, offHeapMisses=0, offHeapEntriesCnt=100,
> heapEntriesCnt=0,
> > > offHeapPrimaryEntriesCnt=100, offHeapBackupEntriesCnt=0,
> > > offHeapAllocatedSize=0, size=56, cacheSize=56, keySize=56,
> isEmpty=false,
> > > dhtEvictQueueCurrSize=-1, txThreadMapSize=0, txXidMapSize=0,
> > > txCommitQueueSize=0, txPrepareQueueSize=0, txStartVerCountsSize=0,
> > > txCommittedVersionsSize=0, txRolledbackVersionsSize=0,
> > > txDhtThreadMapSize=0, txDhtXidMapSize=-1, txDhtCommitQueueSize=0,
> > > txDhtPrepareQueueSize=0, txDhtStartVerCountsSize=0,
> > > txDhtCommittedVersionsSize=-1, txDhtRolledbackVersionsSize=-1,
> > > isWriteBehindEnabled=false, writeBehindFlushSize=-1,
> > > writeBehindFlushThreadCnt=-1, writeBehindFlushFreq=-1,
> > > writeBehindStoreBatchSize=-1, writeBehindTotalCriticalOverflowCnt=-1,
> > > writeBehindCriticalOverflowCnt=-1, writeBehindErrorRetryCnt=-1,
> > > writeBehindBufSize=-1, totalPartitionsCnt=1024,
> rebalancingPartitionsCnt=0,
> > > rebalancedKeys=44, estimatedRebalancingKeys=44, keysToRebalanceLeft=0,
> > > rebalancingKeysRate=0, rebalancingBytesRate=0, rebalanceStartTime=0,
> > > rebalanceFinishTime=0, rebalanceClearingPartitionsLeft=0,
> > > keyType=java.lang.Object, valType=java.lang.Object, isStoreByVal=true,
> > > isStatisticsEnabled=true, isManagementEnabled=false,
> isReadThrough=false,
> > > isWriteThrough=false, isValidForReading=true, isValidForWriting=true]
> > >
> > > In comparision, I also have also a thin client which connects to server
> > > and read some random data once:
> > >
> > > public class CacheClusterThinClientReader {
> > > public static void main(String[] args) throws Exception {
> > > ClientConfiguration clientConfiguration = new
> ClientConfiguration()
> > > .setAddresses("127.0.0.1:10800");
> > >
> > > try (IgniteClient ignite =
> > > Ignition.startClient(clientConfiguration)) {
> > >
> > > ClientCache<String, String> cache =
> > > ignite.getOrCreateCache("test");
> > >
> > > Set<String> misses = new HashSet<>();
> > > new Random().ints(1000).map(i -> Math.abs(i %
> > > 2000)).distinct().limit(100).forEach(i -> {
> > > String key = "data_" + i;
> > > String value = cache.get(key);
> > > System.out.println("Found " + key + " with
> value "
> > > + value);
> > > if (value == null) {
> > > misses.add(key);
> > > }
> > > }
> > > );
> > > System.out.println("Misses count: " + misses.size());
> > > }
> > > }
> > > }
> > >
> > > and finally prints (in my try):
> > >
> > > Misses count: 97
> > >
> > > On the server nodes misses were counted, but only for one server node:
> > >
> > > CacheMetricsSnapshot [reads=51, puts=100, entryProcessorPuts=0,
> > > entryProcessorReadOnlyInvocations=0,
> > > entryProcessorAverageInvocationTime=0.0, entryProcessorInvocations=0,
> > > entryProcessorRemovals=0, entryProcessorMisses=0, entryProcessorHits=0,
> > > entryProcessorMissPercentage=0.0, entryProcessorHitPercentage=0.0,
> > > entryProcessorMaxInvocationTime=0.0,
> entryProcessorMinInvocationTime=0.0, *hits=3,
> > > misses=48*, txCommits=0, txRollbacks=0, evicts=0, removes=0,
> > > putAvgTimeNanos=276.13132, getAvgTimeNanos=850.90735,
> rmvAvgTimeNanos=0.0,
> > > commitAvgTimeNanos=0.0, rollbackAvgTimeNanos=0.0, cacheName=test,
> > > offHeapGets=0, offHeapPuts=0, offHeapRemoves=0, offHeapEvicts=0,
> > > offHeapHits=0, offHeapMisses=0, offHeapEntriesCnt=100,
> heapEntriesCnt=0,
> > > offHeapPrimaryEntriesCnt=100, offHeapBackupEntriesCnt=0,
> > > offHeapAllocatedSize=0, size=53, cacheSize=53, keySize=53,
> isEmpty=false,
> > > dhtEvictQueueCurrSize=-1, txThreadMapSize=0, txXidMapSize=0,
> > > txCommitQueueSize=0, txPrepareQueueSize=0, txStartVerCountsSize=0,
> > > txCommittedVersionsSize=0, txRolledbackVersionsSize=0,
> > > txDhtThreadMapSize=0, txDhtXidMapSize=-1, txDhtCommitQueueSize=0,
> > > txDhtPrepareQueueSize=0, txDhtStartVerCountsSize=0,
> > > txDhtCommittedVersionsSize=-1, txDhtRolledbackVersionsSize=-1,
> > > isWriteBehindEnabled=false, writeBehindFlushSize=-1,
> > > writeBehindFlushThreadCnt=-1, writeBehindFlushFreq=-1,
> > > writeBehindStoreBatchSize=-1, writeBehindTotalCriticalOverflowCnt=-1,
> > > writeBehindCriticalOverflowCnt=-1, writeBehindErrorRetryCnt=-1,
> > > writeBehindBufSize=-1, totalPartitionsCnt=1024,
> rebalancingPartitionsCnt=0,
> > > rebalancedKeys=47, estimatedRebalancingKeys=47, keysToRebalanceLeft=0,
> > > rebalancingKeysRate=0, rebalancingBytesRate=0, rebalanceStartTime=0,
> > > rebalanceFinishTime=0, rebalanceClearingPartitionsLeft=0,
> > > keyType=java.lang.Object, valType=java.lang.Object, isStoreByVal=true,
> > > isStatisticsEnabled=true, isManagementEnabled=false,
> isReadThrough=false,
> > > isWriteThrough=false, isValidForReading=true, isValidForWriting=true]
> > >
> > > Visor confirms my observation:
> > >
> > > Nodes for: test(@c0)
> > >
> > >
> +==================================================================================================================+
> > > | Node ID8(@), IP | CPUs | Heap Used | CPU Load | Up Time
> > > | Size (Primary / Backup) | Hi/Mi/Rd/Wr |
> > >
> > >
> +==================================================================================================================+
> > > | D09713CD(@n1), 10.10.10.14 | 8 | 2.81 % | 0.53 % |
> 00:15:00.291
> > > | Total: 47 (47 / 0) | Hi: 0 |
> > > | | | | |
> > > | Heap: 0 (0 / <n/a>) | Mi: 0 |
> > > | | | | |
> > > | Off-Heap: 47 (47 / 0) | Rd: 0 |
> > > | | | | |
> > > | Off-Heap Memory: <n/a> | Wr: 0 |
> > >
> > >
> +----------------------------+------+-----------+----------+--------------+--------------------------+-------------+
> > > | B8CCBCAB(@n0), 10.10.10.14 | 8 | 1.15 % | 0.50 % |
> 00:15:04.122
> > > | Total: 53 (53 / 0) | Hi: 3 |
> > > | | | | |
> > > | Heap: 0 (0 / <n/a>) | Mi: 48 |
> > > | | | | |
> > > | Off-Heap: 53 (53 / 0) | Rd: 51 |
> > > | | | | |
> > > | Off-Heap Memory: <n/a> | Wr: 100 |
> > >
> > >
> +------------------------------------------------------------------------------------------------------------------+
> > >
> > > I have seen the same behaviour with rest api as with the thin client,
> but
> > > I don't have prepared test case for this.
> > >
> > > Is there a good way to correctly configure collecting statistics on
> server
> > > nodes?
> > > Should I create an issue for this problem?
> > >
> > > --
> > > Pozdrawiam / Regards,
> > > Dominik Przybysz
> > >
>
--
Pozdrawiam / Regards,
Dominik Przybysz
