[ https://issues.apache.org/jira/browse/IGNITE-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Semen Boikov resolved IGNITE-1183. ---------------------------------- Resolution: Fixed This issue was fixed as part of changes done for IGNITE-264, now IgniteAtomicLongChangingTopologySelfTest pass. > IgniteAtomicLong closing and getting during rolling restart causes hangs > ------------------------------------------------------------------------ > > Key: IGNITE-1183 > URL: https://issues.apache.org/jira/browse/IGNITE-1183 > Project: Ignite > Issue Type: Bug > Reporter: Andrey Gura > Assignee: Alexey Goncharuk > Attachments: AtomicLongChangingTopologySelfTest.java, log.zip, > thread_dump.zip > > > During rolling restart of nodes some threads hang on closing or getting > {{IgniteAtomicLong}}. > Stack trace for thread that hangs on {{IgniteAtomicLong.close()}}: > {noformat} > "Test worker" #11 prio=5 os_prio=31 tid=0x00007fbbf4ae7800 nid=0x5b03 waiting > on condition [0x0000000122f93000] > java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x000000076db515a0> (a > org.apache.ignite.internal.util.future.GridFutureAdapter$ChainFuture) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) > at > org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:115) > at > org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:4376) > at > org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:1448) > at > org.apache.ignite.internal.processors.cache.GridCacheProxyImpl.get(GridCacheProxyImpl.java:288) > at > org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor$14.call(DataStructuresProcessor.java:1086) > at > org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor$14.call(DataStructuresProcessor.java:1082) > at > org.apache.ignite.internal.processors.cache.GridCacheUtils.outTx(GridCacheUtils.java:939) > at > org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.removeInternal(DataStructuresProcessor.java:1081) > at > org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.access$800(DataStructuresProcessor.java:51) > at > org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor$4.applyx(DataStructuresProcessor.java:453) > at > org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor$4.applyx(DataStructuresProcessor.java:448) > at > org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.removeDataStructure(DataStructuresProcessor.java:507) > at > org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.removeAtomicLong(DataStructuresProcessor.java:448) > at > org.apache.ignite.internal.processors.datastructures.GridCacheAtomicLongImpl.close(GridCacheAtomicLongImpl.java:368) > {noformat} > Stack trace for thread that hangs on {{Ignite.atomicLong()}}: > {noformat} > "datastructures.AtomicLongChangingTopologySelfTest-1" prio=10 > tid=0x00007f9cd800a800 nid=0x2bb3 waiting on condition [0x00007f9c49cfc000] > java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x00000007f106e580> (a > org.apache.ignite.internal.util.future.GridEmbeddedFuture) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:964) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1282) > at > org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:113) > at > org.apache.ignite.internal.processors.cache.GridCacheAdapter$24.op(GridCacheAdapter.java:2089) > at > org.apache.ignite.internal.processors.cache.GridCacheAdapter$24.op(GridCacheAdapter.java:2081) > at > org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:3990) > at > org.apache.ignite.internal.processors.cache.GridCacheAdapter.invoke(GridCacheAdapter.java:2081) > at > org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.getAtomic(DataStructuresProcessor.java:449) > at > org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.atomicLong(DataStructuresProcessor.java:364) > at > org.apache.ignite.internal.IgniteKernal.atomicLong(IgniteKernal.java:2725) > at > org.apache.ignite.internal.processors.cache.datastructures.AtomicLongChangingTopologySelfTest$1.run(AtomicLongChangingTopologySelfTest.java:180) > at > org.apache.ignite.testframework.GridTestUtils$7.call(GridTestUtils.java:911) > at > org.apache.ignite.testframework.GridTestThread.run(GridTestThread.java:86) > {noformat} > During testing I've noticed also that {{IgniteAtomicLong.getAndIncrement}} > sometimes returns inconsistent values (sequence has gaps). > See also full log ([^log.zip]) and full thread dump for second case > ([^thread_dump.zip]). > See attached test [^AtomicLongChangingTopologySelfTest.java] that reproduces > this problem (not stable, thus can't be part of some test suite of Ignite). > Note that {{CHECK_CONSISTENCY}} parameter controls whether test check > consistency (in this case {{IgniteAtmocInstance}} will not be closed) or not. -- This message was sent by Atlassian JIRA (v6.3.4#6332)