Ok, seems something goes wrong on node with id=36edbfd5-4feb-417e-b965-bdc34a0a6f4f If you still have a problem, can u send here or directly by me these logs ?
>And finally this on the coordinator node.... > >[14:07:41,282][WARNING][exchange-worker-#42%xxxxxx%][GridDhtPartitionsExchangeFuture] > Unable to await partitions release latch within timeout. Some nodes have not >sent acknowledgement for latch completion. It's possible due to unfinishined >atomic updates, transactions or not released explicit locks on that nodes. >Please check logs for errors on nodes with ids reported in latch `pendingAcks` >collection [latch=ServerLatch [permits=1, pendingAcks=HashSet >[36edbfd5-4feb-417e-b965-bdc34a0a6f4f], super=CompletableLatch >[id=CompletableLatchUid [id=exchange, topVer=AffinityTopologyVersion >[topVer=103, minorTopVer=0]]]]] >On Tue, 5 Oct 2021 at 10:07, John Smith < java.dev....@gmail.com > wrote: >>And I see this... >> >>[14:04:15,150][WARNING][exchange-worker-#43%raange%][GridDhtPartitionsExchangeFuture] >> Unable to await partitions release latch within timeout. For more details >>please check coordinator node logs [crdNode=TcpDiscoveryNode >>[id=36ad785d-e344-43bb-b685-e79557572b54, >>consistentId=8172e45d-3ff8-4fe4-aeda-e7d30c1e11e2, addrs=ArrayList >>[127.0.0.1, xxxxxx.65], sockAddrs=HashSet [xxxxxx-0002/xxxxxx.65:47500, / >>127.0.0.1:47500 ], discPort=47500, order=1, intOrder=1, >>lastExchangeTime=1633370987399, loc=false, ver=2.8.1#20200521-sha1:86422096, >>isClient=false]] [latch=ClientLatch [coordinator=TcpDiscoveryNode >>[id=36ad785d-e344-43bb-b685-e79557572b54, >>consistentId=8172e45d-3ff8-4fe4-aeda-e7d30c1e11e2, addrs=ArrayList >>[127.0.0.1, xxxxxx.65], sockAddrs=HashSet [xxxxxx-0002/xxxxxx.65:47500, / >>127.0.0.1:47500 ], discPort=47500, order=1, intOrder=1, >>lastExchangeTime=1633370987399, loc=false, ver=2.8.1#20200521-sha1:86422096, >>isClient=false], ackSent=true, super=CompletableLatch [id=CompletableLatchUid >>[id=exchange, topVer=AffinityTopologyVersion [topVer=103, minorTopVer=0]]]]] >>On Tue, 5 Oct 2021 at 10:02, John Smith < java.dev....@gmail.com > wrote: >>>Actually to be more clear... >>> >>>http://xxxxxx-0001:8080/ignite?cmd=version responds immediately. >>> >>>http://xxxxxx-0001:8080/ignite?cmd=size&cacheName=my-cache doesn't respond >>>at all. >>>On Tue, 5 Oct 2021 at 09:59, John Smith < java.dev....@gmail.com > wrote: >>>>Yeah ever since I got this erro for example the REST APi wont return and >>>>the request are slower. But when I connect with visor I can get stats I can >>>>scan the cache etc... >>>> >>>>Is it possible that these async futures/threads are not released? >>>>On Tue, 5 Oct 2021 at 04:11, Zhenya Stanilovsky < arzamas...@mail.ru > >>>>wrote: >>>>>Hi, this is just a warning shows that something suspicious observed. >>>>>There is no simple reply for your question, in common case all these >>>>>messages are due to cluster (resources or settings) limitation. >>>>>Check documentation for tuning performance [1] >>>>> >>>>>[1] >>>>>https://ignite.apache.org/docs/latest/perf-and-troubleshooting/general-perf-tips >>>>> >>>>>>Hi, using 2.8.1 I understand the message as in my async TRX is taking >>>>>>longer but is there a way to prevent it? >>>>>> >>>>>>When this happened I was pushing about 50, 000 get/puts per second from >>>>>>my API. >>>>> >>>>> >>>>> >>>>>
