We are running Kafka 2.0.1 and lately the brokers have been crashing with several different SIGSEGV errors as listed below. I was wondering if anybody has encountered similar errors or might have advice on what the problem could be?
Thanks, David 1. # # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x00007f883195b06d, pid=5999, tid=0x00007f7d9c1ee700 # # JRE version: Java(TM) SE Runtime Environment (8.0_192-b12) (build 1.8.0_192-b12) # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.192-b12 mixed mode linux-amd64 compressed oops) # Problematic frame: # J 21026 C2 org.apache.kafka.common.network.Selector.pollSelectionKeys(Ljava/util/Set;ZJ)V (543 bytes) @ 0x00007f883195b06d [0x00007f883195ab40+0x52d] # # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again # # If you would like to submit a bug report, please visit: # http://bugreport.java.com/bugreport/crash.jsp # Stack: [0x00007f7d9c0ee000,0x00007f7d9c1ef000], sp=0x00007f7d9c1ed830, free space=1022k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) J 21026 C2 org.apache.kafka.common.network.Selector.pollSelectionKeys(Ljava/util/Set;ZJ)V (543 bytes) @ 0x00007f883195b06d [0x00007f883195ab40+0x52d] J 19626 C2 org.apache.kafka.common.network.Selector.poll(J)V (396 bytes) @ 0x00007f883121f42c [0x00007f883121eca0+0x78c] J 19419 C2 kafka.network.Processor.poll()V (66 bytes) @ 0x00007f883001afa8 [0x00007f883001af60+0x48] j kafka.network.Processor.run()V+31 j java.lang.Thread.run()V+11 v ~StubRoutines::call_stub V [libjvm.so+0x685d2b] JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*, Thread*)+0xddb V [libjvm.so+0x6835f3] JavaCalls::call_virtual(JavaValue*, KlassHandle, Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x263 V [libjvm.so+0x683bb7] JavaCalls::call_virtual(JavaValue*, Handle, KlassHandle, Symbol*, Symbol*, Thread*)+0x47 V [libjvm.so+0x6efb3c] thread_entry(JavaThread*, Thread*)+0x6c V [libjvm.so+0xa77a5b] JavaThread::thread_main_inner()+0xdb V [libjvm.so+0xa77d61] JavaThread::run()+0x2d1 V [libjvm.so+0x90a8b2] java_start(Thread*)+0x102 C [libpthread.so.0+0x7de5] start_thread+0xc5 2. # # JRE version: Java(TM) SE Runtime Environment (8.0_192-b12) (build 1.8.0_192-b12) # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.192-b12 mixed mode linux-amd64 compressed oops) # Problematic frame: # J 16655 C2 sun.nio.ch.FileChannelImpl.size()J (239 bytes) @ 0x00007fda693c4c80 [0x00007fda693c4c60+0x20] # # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again # # If you would like to submit a bug report, please visit: # http://bugreport.java.com/bugreport/crash.jsp # --------------- T H R E A D --------------- Current thread (0x00007fda7d048800): JavaThread "kafka-network-thread-93732870-ListenerName(PLAINTEXT)-PLAINTEXT-4" [_thread_in_Java, id=15967, stack(0x00007fcfb84fc000,0x00007fcfb85fd000)] siginfo: si_signo: 11 (SIGSEGV), si_code: 2 (SEGV_ACCERR), si_addr: 0x00007fda693c4c80 Stack: [0x00007fcfb84fc000,0x00007fcfb85fd000], sp=0x00007fcfb85fb818, free space=1022k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) J 16655 C2 sun.nio.ch.FileChannelImpl.size()J (239 bytes) @ 0x00007fda693c4c80 [0x00007fda693c4c60+0x20] C 0x000000055ea5b690 3. # # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x00007fbfc45c7080, pid=2548, tid=0x00007fba8f24c700 # # JRE version: Java(TM) SE Runtime Environment (8.0_192-b12) (build 1.8.0_192-b12) # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.192-b12 mixed mode linux-amd64 compressed oops) # Problematic frame: # J 15764 C2 org.apache.kafka.common.record.FileRecords.writeTo(Ljava/nio/channels/GatheringByteChannel;JI)J (151 bytes) @ 0x00007fbfc45c7080 [0x00007fbfc45c7060+0x20] # # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again # # If you would like to submit a bug report, please visit: # http://bugreport.java.com/bugreport/crash.jsp # --------------- T H R E A D --------------- Current thread (0x00007fbfd9b60000): JavaThread "kafka-network-thread-93732870-ListenerName(PLAINTEXT)-PLAINTEXT-7" [_thread_in_Java, id=2679, stack(0x00007fba8f14c000,0x00007fba8f24d000)] siginfo: si_signo: 11 (SIGSEGV), si_code: 2 (SEGV_ACCERR), si_addr: 0x00007fbfc45c7080 Stack: [0x00007fba8f14c000,0x00007fba8f24d000], sp=0x00007fba8f24b838, free space=1022k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) J 15764 C2 org.apache.kafka.common.record.FileRecords.writeTo(Ljava/nio/channels/GatheringByteChannel;JI)J (151 bytes) @ 0x00007fbfc45c7080 [0x00007fbfc45c7060+0x20] C 0x0000000000000001
