Hello! This does look like some control.sh issue, such as that it is unable to complete request and activation never happens. Can you try to activate this node in some other way, such as connecting a thick client node to it and doing client.cluster().state(ACTIVE) from there?
Regards, -- Ilya Kasnacheev пн, 25 янв. 2021 г. в 16:35, Edwind <[email protected]>: > Hello! > > Here is my full logs with Stephen's image :) I think I have enabled INFO > logging but I'm not sure tell me if you need more. > > > Ignite Logs wrote > > [15:55:08] __________ ________________ > > [15:55:08] / _/ ___/ |/ / _/_ __/ __/ > > [15:55:08] _/ // (7 7 // / / / / _/ > > [15:55:08] /___/\___/_/|_/___/ /_/ /___/ > > [15:55:08] > > [15:55:08] ver. 2.8.1#20200521-sha1:86422096 > > [15:55:08] 2020 Copyright(C) Apache Software Foundation > > [15:55:08] > > [15:55:08] Ignite documentation: http://ignite.apache.org > > [15:55:08] > > [15:55:08] Quiet mode. > > [15:55:08] ^-- Logging to file > > '/opt/ignite/apache-ignite/work/log/ignite-2b0807c4.0.log' > > [15:55:08] ^-- Logging by 'JavaLogger [quiet=true, config=null]' > > [15:55:08] ^-- To see **FULL** console log here add > -DIGNITE_QUIET=false > > or "-v" to ignite.{sh|bat} > > [15:55:08] > > [15:55:09] OS: Linux 5.4.79-v7+ arm > > [15:55:09] VM information: OpenJDK Runtime Environment 1.8.0_212-b04 > > IcedTea OpenJDK Zero VM 25.212-b04 > > [15:55:09] Please set system property '-Djava.net.preferIPv4Stack=true' > to > > avoid possible problems in mixed environments. > > [15:55:09] Initial heap size is 16MB (should be no less than 512MB, use > > -Xms512m -Xmx512m). > > [15:55:11] Configured plugins: > > [15:55:11] ^-- None > > [15:55:11] > > [15:55:11] Configured failure handler: [hnd=StopNodeOrHaltFailureHandler > > [tryStop=false, timeout=0, super=AbstractFailureHandler > > [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, > > SYSTEM_CRITICAL_OPERATION_TIMEOUT]]]] > > [15:55:14] Message queue limit is set to 0 which may lead to potential > > OOMEs when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes > > due to message queues growth on sender and receiver sides. > > [15:55:15] Security status [authentication=off, tls/ssl=off] > > [15:56:30] Nodes started on local machine require more than 80% of > > physical RAM what can lead to significant slowdown due to swapping > (please > > decrease JVM heap size, data region size or checkpoint buffer size) > > [required=574MB, available=924MB] > > [15:56:46,041][SEVERE][tcp-disco-msg-worker-[crd]-#2][G] Blocked > > system-critical thread has been detected. This can lead to cluster-wide > > undefined behaviour [workerName=disco-notifier-worker, > > threadName=disco-notifier-worker-#43, blockedFor=15s] > > [15:56:46] Possible failure suppressed accordingly to a configured > handler > > [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, > > super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet > > [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], > > failureCtx=FailureContext [type=SYSTEM_WORKER_BLOCKED, err=class > > o.a.i.IgniteException: GridWorker [name=disco-notifier-worker, > > igniteInstanceName=null, finished=false, heartbeatTs=1611417390753]]] > > [15:56:58,064][SEVERE][tcp-disco-msg-worker-[crd]-#2][G] Blocked > > system-critical thread has been detected. This can lead to cluster-wide > > undefined behaviour [workerName=disco-notifier-worker, > > threadName=disco-notifier-worker-#43, blockedFor=27s] > > [15:56:58] Possible failure suppressed accordingly to a configured > handler > > [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, > > super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet > > [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], > > failureCtx=FailureContext [type=SYSTEM_WORKER_BLOCKED, err=class > > o.a.i.IgniteException: GridWorker [name=disco-notifier-worker, > > igniteInstanceName=null, finished=false, heartbeatTs=1611417390753]]] > > [15:57:09] Performance suggestions for grid (fix if possible) > > [15:57:09] To disable, set -DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true > > [15:57:09] ^-- Enable server mode for JVM (add '-server' to JVM > options) > > [15:57:09] ^-- Enable G1 Garbage Collector (add '-XX:+UseG1GC' to JVM > > options) > > [15:57:09] ^-- Specify JVM heap max size (add '-Xmx > > <size> > > [g|G|m|M|k|K]' to JVM options) > > [15:57:09] ^-- Set max direct memory size if getting 'OOME: Direct > > buffer memory' (add '-XX:MaxDirectMemorySize= > > <size> > > [g|G|m|M|k|K]' to JVM options) > > [15:57:09] ^-- Disable processing of calls to System.gc() (add > > '-XX:+DisableExplicitGC' to JVM options) > > [15:57:09] Refer to this page for more performance suggestions: > > https://apacheignite.readme.io/docs/jvm-and-system-tuning > > [15:57:09] > > [15:57:09] To start Console Management & Monitoring run > > ignitevisorcmd.{sh|bat} > > [15:57:09] Data Regions Configured: > > [15:57:09] ^-- default_region [initSize=50.0 MiB, maxSize=50.0 MiB, > > persistence=false, lazyMemoryAllocation=true] > > [15:57:09] ^-- persistence_region [initSize=100.0 MiB, maxSize=100.0 > > MiB, persistence=true, lazyMemoryAllocation=true] > > [15:57:09] > > [15:57:09] Ignite node started OK (id=2b0807c4) > > [15:57:09] >>> Ignite cluster is not active (limited functionality > > available). Use control.(sh|bat) script or IgniteCluster interface to > > activate. > > [15:57:09] Topology snapshot [ver=1, locNode=2b0807c4, servers=1, > > clients=0, state=INACTIVE, CPUs=4, offheap=0.15GB, heap=0.22GB] > > [15:57:16] New version is available at ignite.apache.org: 2.9.1 > > [16:05:22,842][SEVERE][rest-#109][GridTcpRestProtocol] Failed to process > > client request [ses=GridSelectorNioSessionImpl > > [worker=ByteBufferNioClientWorker [readBuf=java.nio.HeapByteBuffer[pos=0 > > lim=8192 cap=8192], super=AbstractNioClientWorker [idx=1, bytesRcvd=0, > > bytesSent=0, bytesRcvd0=0, bytesSent0=0, select=true, super=GridWorker > > [name=grid-nio-worker-tcp-rest-1, igniteInstanceName=null, > finished=false, > > heartbeatTs=1611417921389, hashCode=4009713, interrupted=false, > > runner=grid-nio-worker-tcp-rest-1-#36]]], writeBuf=null, readBuf=null, > > inRecovery=null, outRecovery=null, closeSocket=true, > > outboundMessagesQueueSizeMetric=null, super=GridNioSessionImpl > > [locAddr=/127.0.0.1:11211, rmtAddr=/127.0.0.1:33872, > > createTime=1611417824153, closeTime=1611417905352, bytesSent=633, > > bytesRcvd=519, bytesSent0=5, bytesRcvd0=5, sndSchedTime=1611417922803, > > lastSndTime=1611417901655, lastRcvTime=1611417901655, readsPaused=false, > > filterChain=FilterChain[filters=[GridNioCodecFilter > > [parser=GridTcpRestParser [marsh=JdkMarshaller > > [clsFilter=o.a.i.marshaller.MarshallerUtils$1@1864f0b], > > routerClient=false], directMode=false]], accepted=true, > > markedForClose=false]], msg=GridClientAuthenticationRequest > > [super=GridClientAbstractMessage [reqId=3, > > id=227a26f2-3ada-4be3-8150-5879fdfb48c1, destId=null, > > > super=o.a.i.i.processors.rest.client.message.GridClientAuthenticationRequest@177bade > ]]] > > class org.apache.ignite.IgniteCheckedException: Failed to send message > > (connection was closed): GridSelectorNioSessionImpl > > [worker=ByteBufferNioClientWorker [readBuf=java.nio.HeapByteBuffer[pos=0 > > lim=8192 cap=8192], super=AbstractNioClientWorker [idx=1, bytesRcvd=0, > > bytesSent=0, bytesRcvd0=0, bytesSent0=0, select=true, super=GridWorker > > [name=grid-nio-worker-tcp-rest-1, igniteInstanceName=null, > finished=false, > > heartbeatTs=1611417921389, hashCode=4009713, interrupted=false, > > runner=grid-nio-worker-tcp-rest-1-#36]]], writeBuf=null, readBuf=null, > > inRecovery=null, outRecovery=null, closeSocket=true, > > outboundMessagesQueueSizeMetric=null, super=GridNioSessionImpl > > [locAddr=/127.0.0.1:11211, rmtAddr=/127.0.0.1:33872, > > createTime=1611417824153, closeTime=1611417905352, bytesSent=633, > > bytesRcvd=519, bytesSent0=5, bytesRcvd0=5, sndSchedTime=1611417922803, > > lastSndTime=1611417901655, lastRcvTime=1611417901655, readsPaused=false, > > filterChain=FilterChain[filters=[GridNioCodecFilter > > [parser=GridTcpRestParser [marsh=JdkMarshaller > > [clsFilter=org.apache.ignite.marshaller.MarshallerUtils$1@1864f0b], > > routerClient=false], directMode=false]], accepted=true, > > markedForClose=false]] > > at > > org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7507) > > at > > > org.apache.ignite.internal.util.future.GridFutureAdapter.resolve(GridFutureAdapter.java:260) > > at > > > org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:172) > > at > > > org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:141) > > at > > > org.apache.ignite.internal.processors.rest.protocols.tcp.GridTcpRestNioListener$1$1.apply(GridTcpRestNioListener.java:277) > > at > > > org.apache.ignite.internal.processors.rest.protocols.tcp.GridTcpRestNioListener$1$1.apply(GridTcpRestNioListener.java:274) > > at > > > org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:399) > > at > > > org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:354) > > at > > > org.apache.ignite.internal.processors.rest.protocols.tcp.GridTcpRestNioListener$1.apply(GridTcpRestNioListener.java:274) > > at > > > org.apache.ignite.internal.processors.rest.protocols.tcp.GridTcpRestNioListener$1.apply(GridTcpRestNioListener.java:242) > > at > > > org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:399) > > at > > > org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:347) > > at > > > org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:335) > > at > > > org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:511) > > at > > > org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:490) > > at > > > org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:467) > > at > > > org.apache.ignite.internal.processors.rest.GridRestProcessor$2$1.apply(GridRestProcessor.java:184) > > at > > > org.apache.ignite.internal.processors.rest.GridRestProcessor$2$1.apply(GridRestProcessor.java:181) > > at > > > org.apache.ignite.internal.util.future.GridFinishedFuture.listen(GridFinishedFuture.java:124) > > at > > > org.apache.ignite.internal.processors.rest.GridRestProcessor$2.body(GridRestProcessor.java:181) > > at > > > org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) > > at > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > > at > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > at java.lang.Thread.run(Thread.java:748) > > Caused by: java.io.IOException: Failed to send message (connection was > > closed): GridSelectorNioSessionImpl [worker=ByteBufferNioClientWorker > > [readBuf=java.nio.HeapByteBuffer[pos=0 lim=8192 cap=8192], > > super=AbstractNioClientWorker [idx=1, bytesRcvd=0, bytesSent=0, > > bytesRcvd0=0, bytesSent0=0, select=true, super=GridWorker > > [name=grid-nio-worker-tcp-rest-1, igniteInstanceName=null, > finished=false, > > heartbeatTs=1611417921389, hashCode=4009713, interrupted=false, > > runner=grid-nio-worker-tcp-rest-1-#36]]], writeBuf=null, readBuf=null, > > inRecovery=null, outRecovery=null, closeSocket=true, > > outboundMessagesQueueSizeMetric=null, super=GridNioSessionImpl > > [locAddr=/127.0.0.1:11211, rmtAddr=/127.0.0.1:33872, > > createTime=1611417824153, closeTime=1611417905352, bytesSent=633, > > bytesRcvd=519, bytesSent0=5, bytesRcvd0=5, sndSchedTime=1611417922803, > > lastSndTime=1611417901655, lastRcvTime=1611417901655, readsPaused=false, > > filterChain=FilterChain[filters=[GridNioCodecFilter > > [parser=GridTcpRestParser [marsh=JdkMarshaller > > [clsFilter=org.apache.ignite.marshaller.MarshallerUtils$1@1864f0b], > > routerClient=false], directMode=false]], accepted=true, > > markedForClose=false]] > > at > > > org.apache.ignite.internal.util.nio.GridNioServer.send0(GridNioServer.java:638) > > at > > > org.apache.ignite.internal.util.nio.GridNioServer.send(GridNioServer.java:579) > > at > > > org.apache.ignite.internal.util.nio.GridNioServer$HeadFilter.onSessionWrite(GridNioServer.java:3635) > > at > > > org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedSessionWrite(GridNioFilterAdapter.java:121) > > at > > > org.apache.ignite.internal.util.nio.GridNioCodecFilter.onSessionWrite(GridNioCodecFilter.java:96) > > at > > > org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedSessionWrite(GridNioFilterAdapter.java:121) > > at > > > org.apache.ignite.internal.util.nio.GridNioFilterChain$TailFilter.onSessionWrite(GridNioFilterChain.java:269) > > at > > > org.apache.ignite.internal.util.nio.GridNioFilterChain.onSessionWrite(GridNioFilterChain.java:192) > > at > > > org.apache.ignite.internal.util.nio.GridNioSessionImpl.send(GridNioSessionImpl.java:113) > > at > > > org.apache.ignite.internal.processors.rest.protocols.tcp.GridTcpRestNioListener$1.apply(GridTcpRestNioListener.java:271) > > ... 15 more > > [16:57:10] New version is available at ignite.apache.org: 2.9.1 > > > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
