hello, seems you really use all available memory, of course without heap dump i have no clue why it`s happen, plz increase -Xmx and check once more.
> >We are planning to upgrade from 2.9 to 2.11.1. But as soon as we start the >2.11.1 or 2.12 ignite instance crashes with OOME error. > >[20:15:05,474][SEVERE][grid-nio-worker-client-listener-2-#34][] Critical >system error detected. Will be handled accordingly to configured handler >[hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, >super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet >[SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], >failureCtx=FailureContext [type=CRITICAL_ERROR, >err=java.lang.OutOfMemoryError: Direct buffer memory]] >java.lang.OutOfMemoryError: Direct buffer memory > >Some of the pointers > >1) 2.9 has no problem. >2) 2.11 and 2.12 crashes even with default supplied config of >ignite.sh. OOM is within a min of startup. >3) No other application is running on the server >4) Details about the environment >[oracle@jmngd1s05ppv014 log]$ free -g > total used free shared buff/cache available >Mem: 46 18 21 0 5 23 >Swap: 47 0 47 > >[oracle@jmngd1s05ppv014 log]$ java -version >java version "1.8.0_301" >Java(TM) SE Runtime Environment (build 1.8.0_301-b25) >Java HotSpot(TM) 64-Bit Server VM (build 25.301-b25, mixed mode) > >[oracle@jmngd1s05ppv014 log]$ uname -a >Linux jmngd1s05ppv014 3.10.0-1160.49.1.el7.x86_64 #1 SMP Tue Nov 9 16:09:48 >UTC 2021 x86_64 x86_64 x86_64 GNU/Linux > >[oracle@jmngd1s05ppv014 log]$ cat /etc/os-release >NAME="Red Hat Enterprise Linux Server" >VERSION="7.9 (Maipo)" >ID="rhel" >ID_LIKE="fedora" >VARIANT="Server" >VARIANT_ID="server" >VERSION_ID="7.9" >PRETTY_NAME="Red Hat Enterprise Linux Server 7.9 (Maipo)" >ANSI_COLOR="0;31" >CPE_NAME="cpe:/o:redhat:enterprise_linux:7.9:GA:server" >HOME_URL=" https://www.redhat.com/ " >BUG_REPORT_URL=" https://bugzilla.redhat.com/ " > >REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7" >REDHAT_BUGZILLA_PRODUCT_VERSION=7.9 >REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" >REDHAT_SUPPORT_PRODUCT_VERSION="7.9" >5) I have also attached the ignite log. >6) O/S administrator confirmed there are no hardware failure. > >If anyone can help how to further diagnosis the issue. We will be grateful. > > >Thanks, >Sachin. > >" Confidentiality Warning : This message and any attachments are intended only >for the use of the intended recipient(s), are confidential and may be >privileged. If you are not the intended recipient, you are hereby notified >that any review, re-transmission, conversion to hard copy, copying, >circulation or other use of this message and any attachments is strictly >prohibited. If you are not the intended recipient, please notify the sender >immediately by return email and delete this message and any attachments from >your system. >Virus Warning: Although the company has taken reasonable precautions to ensure >no viruses are present in this email. The company cannot accept responsibility >for any loss or damage arising from the use of this email or attachment."