[
https://issues.apache.org/jira/browse/IGNITE-7918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16395087#comment-16395087
]
Zbyszek B commented on IGNITE-7918:
-----------------------------------
{noformat}
C:\Dev>"C:\Program Files\Java\jdk1.8.0_111\bin\java" -Xmx4G -XX:+UseG1GC
"-javaagent:C:\Program Files\JetBrains\IntelliJ IDEA Community Edition
2017.2.5\lib\ide
a_rt.jar=52512:C:\Program Files\JetBrains\IntelliJ IDEA Community Edition
2017.2.5\bin" -Dfile.encoding=UTF-8 -classpath "C:\Program
Files\Java\jdk1.8.0_111\jre
\lib\charsets.jar;C:\Program
Files\Java\jdk1.8.0_111\jre\lib\deploy.jar;C:\Program
Files\Java\jdk1.8.0_111\jre\lib\ext\access-bridge-64.jar;C:\Program Files\Jav
a\jdk1.8.0_111\jre\lib\ext\cldrdata.jar;C:\Program
Files\Java\jdk1.8.0_111\jre\lib\ext\dnsns.jar;C:\Program
Files\Java\jdk1.8.0_111\jre\lib\ext\jaccess.jar;C:\P
rogram Files\Java\jdk1.8.0_111\jre\lib\ext\jfxrt.jar;C:\Program
Files\Java\jdk1.8.0_111\jre\lib\ext\localedata.jar;C:\Program
Files\Java\jdk1.8.0_111\jre\lib\ex
t\nashorn.jar;C:\Program
Files\Java\jdk1.8.0_111\jre\lib\ext\sunec.jar;C:\Program
Files\Java\jdk1.8.0_111\jre\lib\ext\sunjce_provider.jar;C:\Program Files\Java\
jdk1.8.0_111\jre\lib\ext\sunmscapi.jar;C:\Program
Files\Java\jdk1.8.0_111\jre\lib\ext\sunpkcs11.jar;C:\Program
Files\Java\jdk1.8.0_111\jre\lib\ext\zipfs.jar;C:\
Program Files\Java\jdk1.8.0_111\jre\lib\javaws.jar;C:\Program
Files\Java\jdk1.8.0_111\jre\lib\jce.jar;C:\Program
Files\Java\jdk1.8.0_111\jre\lib\jfr.jar;C:\Prog
ram Files\Java\jdk1.8.0_111\jre\lib\jfxswt.jar;C:\Program
Files\Java\jdk1.8.0_111\jre\lib\jsse.jar;C:\Program
Files\Java\jdk1.8.0_111\jre\lib\management-agent.j
ar;C:\Program Files\Java\jdk1.8.0_111\jre\lib\plugin.jar;C:\Program
Files\Java\jdk1.8.0_111\jre\lib\resources.jar;C:\Program
Files\Java\jdk1.8.0_111\jre\lib\rt.
jar;C:\Dev\IgniteMemLeak4\out\production\IgniteDemo;C:\Dev\IgniteMemLeak4\lib\2.3\cache-api-1.0.0.jar;C:\Dev\IgniteMemLeak4\lib\2.3\commons-codec-1.6.jar;C:\Dev
\IgniteMemLeak4\lib\2.3\h2-1.4.195.jar;C:\Dev\IgniteMemLeak4\lib\2.3\ignite-core-2.3.0.jar;C:\Dev\IgniteMemLeak4\lib\2.3\ignite-indexing-2.3.0.jar;C:\Dev\Ignite
MemLeak4\lib\2.3\lucene-analyzers-common-5.5.2.jar;C:\Dev\IgniteMemLeak4\lib\2.3\lucene-core-5.5.2.jar;C:\Dev\IgniteMemLeak4\lib\2.3\lucene-queryparser-5.5.2.ja
r" leak.Demo
>>> Starting Ignite on MyLeakingNode; work directory C:\Users\user1\ignite ...
Mar 11, 2018 5:24:06 PM java.util.logging.LogManager$RootLogger log
SEVERE: Failed to resolve default logging config file:
config/java.util.logging.properties
[17:24:06] __________ ________________
[17:24:06] / _/ ___/ |/ / _/_ __/ __/
[17:24:06] _/ // (7 7 // / / / / _/
[17:24:06] /___/\___/_/|_/___/ /_/ /___/
[17:24:06]
[17:24:06] ver. 2.3.0#20171028-sha1:8add7fd5
[17:24:06] 2017 Copyright(C) Apache Software Foundation
[17:24:06]
[17:24:06] Ignite documentation: http://ignite.apache.org
[17:24:06]
[17:24:06] Quiet mode.
[17:24:06] ^-- To see **FULL** console log here add -DIGNITE_QUIET=false or
"-v" to ignite.{sh|bat}
[17:24:06]
[17:24:06] OS: Windows 7 6.1 amd64
[17:24:06] VM information: Java(TM) SE Runtime Environment 1.8.0_111-b14 Oracle
Corporation Java HotSpot(TM) 64-Bit Server VM 25.111-b14
[17:24:06] Initial heap size is 256MB (should be no less than 512MB, use
-Xms512m -Xmx512m).
[17:24:06] Configured plugins:
[17:24:06] ^-- None
[17:24:06]
[17:24:06] Security status [authentication=off, tls/ssl=off]
[17:24:08] Performance suggestions for grid (fix if possible)
[17:24:08] To disable, set -DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true
[17:24:08] ^-- Set max direct memory size if getting 'OOME: Direct buffer
memory' (add '-XX:MaxDirectMemorySize=<size>[g|G|m|M|k|K]' to JVM options)
[17:24:08] ^-- Disable processing of calls to System.gc() (add
'-XX:+DisableExplicitGC' to JVM options)
[17:24:08] Refer to this page for more performance suggestions:
https://apacheignite.readme.io/docs/jvm-and-system-tuning
[17:24:08]
[17:24:08] To start Console Management & Monitoring run ignitevisorcmd.{sh|bat}
[17:24:08]
[17:24:08] Ignite node started OK (id=991d7791)
[17:24:08] Topology snapshot [ver=1, servers=1, clients=0, CPUs=8, heap=4.0GB]
>>> Ignite started on MyLeakingNode (991d7791-5cc8-4765-82a4-6aff1a6e74c6)
>>> successfully!
Mar 11, 2018 11:35:28 PM org.apache.ignite.logger.java.JavaLogger error
SEVERE: TcpDiscoverSpi's message worker thread failed abnormally. Stopping the
node in order to prevent cluster wide instability.
java.lang.OutOfMemoryError: Java heap space
at
org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNodesRing.serverNodes(TcpDiscoveryNodesRing.java:608)
at
org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNodesRing.coordinator(TcpDiscoveryNodesRing.java:439)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.resolveCoordinator(ServerImpl.java:1491)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.resolveCoordinator(ServerImpl.java:1472)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.isLocalNodeCoordinator(ServerImpl.java:1442)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.access$500(ServerImpl.java:171)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.sendMetricsUpdateMessage(ServerImpl.java:5486)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.noMessageLoop(ServerImpl.java:2688)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6646)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2533)
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
Mar 11, 2018 11:35:28 PM org.apache.ignite.logger.java.JavaLogger error
SEVERE: Caught unhandled exception in NIO worker thread (restart the node).
java.lang.OutOfMemoryError: Java heap space
Mar 11, 2018 11:35:28 PM org.apache.ignite.logger.java.JavaLogger error
SEVERE: Unexpected exception in discovery worker thread (ignored).
java.lang.OutOfMemoryError: Java heap space
Mar 11, 2018 11:35:28 PM org.apache.ignite.logger.java.JavaLogger error
SEVERE: Error when executing timeout callback: CancelableTask
[id=cbec1e51261-5c626802-5363-4ccc-a435-f2664758f69e, endTime=1520807664027,
period=5000, cancel=f
alse,
task=o.a.i.i.processors.cache.query.continuous.CacheContinuousQueryManager$BackupCleaner@721fae23]
java.lang.OutOfMemoryError: Java heap space
at
java.util.concurrent.ConcurrentSkipListMap.doPut(ConcurrentSkipListMap.java:850)
at
java.util.concurrent.ConcurrentSkipListMap.putIfAbsent(ConcurrentSkipListMap.java:1962)
at
org.apache.ignite.internal.util.GridConcurrentSkipListSet.add(GridConcurrentSkipListSet.java:143)
at
org.apache.ignite.internal.processors.timeout.GridTimeoutProcessor.addTimeoutObject(GridTimeoutProcessor.java:95)
at
org.apache.ignite.internal.processors.timeout.GridTimeoutProcessor$CancelableTask.onTimeout(GridTimeoutProcessor.java:267)
at
org.apache.ignite.internal.processors.timeout.GridTimeoutProcessor$TimeoutWorker.body(GridTimeoutProcessor.java:163)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at java.lang.Thread.run(Thread.java:745)
Mar 11, 2018 11:35:28 PM org.apache.ignite.logger.java.JavaLogger error
SEVERE: Runtime error caught during grid runnable execution: GridWorker
[name=nio-acceptor, igniteInstanceName=null, finished=false,
hashCode=1326643198, interr
upted=false, runner=nio-acceptor-#29]
java.lang.OutOfMemoryError: Java heap space
at
org.jsr166.ConcurrentHashMap8$KeySetView.iterator(ConcurrentHashMap8.java:3612)
at
org.apache.ignite.internal.util.GridSetWrapper.iterator(GridSetWrapper.java:94)
at
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.reset0(GridNioServer.java:2689)
at
org.apache.ignite.internal.util.nio.GridNioServer$SizeBasedBalancer.run(GridNioServer.java:4016)
at
org.apache.ignite.internal.util.nio.GridNioServer$GridNioAcceptWorker.accept(GridNioServer.java:2777)
at
org.apache.ignite.internal.util.nio.GridNioServer$GridNioAcceptWorker.body(GridNioServer.java:2744)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at java.lang.Thread.run(Thread.java:745)
Exception in thread "nio-acceptor-#29" java.lang.OutOfMemoryError: Java heap
space
at
org.jsr166.ConcurrentHashMap8$KeySetView.iterator(ConcurrentHashMap8.java:3612)
at
org.apache.ignite.internal.util.GridSetWrapper.iterator(GridSetWrapper.java:94)
at
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.reset0(GridNioServer.java:2689)
at
org.apache.ignite.internal.util.nio.GridNioServer$SizeBasedBalancer.run(GridNioServer.java:4016)
at
org.apache.ignite.internal.util.nio.GridNioServer$GridNioAcceptWorker.accept(GridNioServer.java:2777)
at
org.apache.ignite.internal.util.nio.GridNioServer$GridNioAcceptWorker.body(GridNioServer.java:2744)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at java.lang.Thread.run(Thread.java:745)
Mar 11, 2018 11:39:30 PM org.apache.ignite.logger.java.JavaLogger error
SEVERE: Runtime error caught during grid runnable execution: GridWorker
[name=grid-timeout-worker, igniteInstanceName=null, finished=false,
hashCode=185468496,
interrupted=false, runner=grid-timeout-worker-#23]
java.lang.OutOfMemoryError: Java heap space
at
java.util.concurrent.ConcurrentSkipListMap.doPut(ConcurrentSkipListMap.java:850)
at
java.util.concurrent.ConcurrentSkipListMap.putIfAbsent(ConcurrentSkipListMap.java:1962)
at
org.apache.ignite.internal.util.GridConcurrentSkipListSet.add(GridConcurrentSkipListSet.java:143)
at
org.apache.ignite.internal.processors.timeout.GridTimeoutProcessor.addTimeoutObject(GridTimeoutProcessor.java:95)
at
org.apache.ignite.internal.processors.timeout.GridTimeoutProcessor$CancelableTask.onTimeout(GridTimeoutProcessor.java:267)
at
org.apache.ignite.internal.processors.timeout.GridTimeoutProcessor$TimeoutWorker.body(GridTimeoutProcessor.java:163)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at java.lang.Thread.run(Thread.java:745)
Exception in thread "grid-timeout-worker-#23" java.lang.OutOfMemoryError: Java
heap space
at
java.util.concurrent.ConcurrentSkipListMap.doPut(ConcurrentSkipListMap.java:850)
at
java.util.concurrent.ConcurrentSkipListMap.putIfAbsent(ConcurrentSkipListMap.java:1962)
at
org.apache.ignite.internal.util.GridConcurrentSkipListSet.add(GridConcurrentSkipListSet.java:143)
at
org.apache.ignite.internal.processors.timeout.GridTimeoutProcessor.addTimeoutObject(GridTimeoutProcessor.java:95)
at
org.apache.ignite.internal.processors.timeout.GridTimeoutProcessor$CancelableTask.onTimeout(GridTimeoutProcessor.java:267)
at
org.apache.ignite.internal.processors.timeout.GridTimeoutProcessor$TimeoutWorker.body(GridTimeoutProcessor.java:163)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at java.lang.Thread.run(Thread.java:745)
Exception in thread "grid-nio-worker-client-listener-0-#30" Exception in thread
"disco-event-worker-#41" java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
Mar 12, 2018 12:02:16 AM org.apache.ignite.logger.java.JavaLogger error
SEVERE: Runtime error caught during grid runnable execution: IgniteSpiThread
[name=tcp-disco-msg-worker-#2]
java.lang.OutOfMemoryError: Java heap space
at
org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNodesRing.serverNodes(TcpDiscoveryNodesRing.java:608)
at
org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNodesRing.coordinator(TcpDiscoveryNodesRing.java:439)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.resolveCoordinator(ServerImpl.java:1491)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.resolveCoordinator(ServerImpl.java:1472)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.isLocalNodeCoordinator(ServerImpl.java:1442)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.access$500(ServerImpl.java:171)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.sendMetricsUpdateMessage(ServerImpl.java:5486)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.noMessageLoop(ServerImpl.java:2688)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6646)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2533)
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
Exception in thread "tcp-disco-msg-worker-#2" java.lang.OutOfMemoryError: Java
heap space
at
org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNodesRing.serverNodes(TcpDiscoveryNodesRing.java:608)
at
org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNodesRing.coordinator(TcpDiscoveryNodesRing.java:439)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.resolveCoordinator(ServerImpl.java:1491)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.resolveCoordinator(ServerImpl.java:1472)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.isLocalNodeCoordinator(ServerImpl.java:1442)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.access$500(ServerImpl.java:171)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.sendMetricsUpdateMessage(ServerImpl.java:5486)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.noMessageLoop(ServerImpl.java:2688)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6646)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2533)
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
javax.cache.CacheException: class org.apache.ignite.IgniteCheckedException:
Operation has been cancelled (node is stopping).{noformat}
> Huge memory leak when data streamer used together with local cache
> ------------------------------------------------------------------
>
> Key: IGNITE-7918
> URL: https://issues.apache.org/jira/browse/IGNITE-7918
> Project: Ignite
> Issue Type: Bug
> Components: cache
> Affects Versions: 2.3
> Reporter: Zbyszek B
> Priority: Blocker
>
> Dear Igniters,
> We observe huge memory leak when data streamer used together with local cache.
> In the attached demo producer produces local cache with single binary object
> and passes this to queue. Consumer picks up the cache from the queue,
> constructs different binary object from it, adds it to global partitioned
> cache and destroys local cache.
> This design causes a significant leak - the whole heap it takes within
> minutes (no matter if this is 4G or 24G).
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
