Looks our case is triggered from caffeine cache, different from https://bugs.openjdk.org/browse/JDK-8285835 where it happens in facet module.
--------------- T H R E A D --------------- Current thread (0x00007f96b1286f40): JavaThread "C2 CompilerThread0" daemon [_thread_in_native, id=16637, stack(0x00007f85f50ab000,0x00007f85f51ac000)] Current CompileTask: C2:148683516 28882 ! 4 com.github.benmanes.caffeine.cache.BoundedLocalCache::put (731 bytes) Stack: [0x00007f85f50ab000,0x00007f85f51ac000], sp=0x00007f85f51a6e40, free space=1007k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) V [libjvm.so+0x9e13a1] PhaseIdealLoop::build_loop_late_post_work(Node*, bool)+0x111 V [libjvm.so+0x9e1910] PhaseIdealLoop::build_loop_late(VectorSet&, Node_List&, Node_Stack&)+0x180 V [libjvm.so+0x9e2321] PhaseIdealLoop::build_and_optimize(LoopOptsMode)+0x841 V [libjvm.so+0x5b90da] PhaseIdealLoop::optimize(PhaseIterGVN&, LoopOptsMode)+0x17a V [libjvm.so+0x5b6b5e] Compile::Optimize()+0x84e V [libjvm.so+0x5b883c] Compile::Compile(ciEnv*, ciMethod*, int, bool, bool, bool, bool, bool, DirectiveSet*)+0x13ec V [libjvm.so+0x4f87da] C2Compiler::compile_method(ciEnv*, ciMethod*, int, bool, DirectiveSet*)+0xba V [libjvm.so+0x5c0c8d] CompileBroker::invoke_compiler_on_method(CompileTask*)+0xf8d V [libjvm.so+0x5c15d8] CompileBroker::compiler_thread_loop()+0x588 V [libjvm.so+0xde6bb8] JavaThread::thread_main_inner()+0x158 V [libjvm.so+0xde6cf9] JavaThread::run()+0x129 V [libjvm.so+0xde9bce] Thread::call_run()+0x7e V [libjvm.so+0xb337b1] thread_native_entry(Thread*)+0xe1 On Thu, Oct 13, 2022 at 12:00 PM Wei <weiwan...@gmail.com> wrote: > We also observed the same SIGSEGV crash during our test with Solr 9.0.0. > It happens very rarely though, seem 2 occurences during one month of > testing. We use Zulu JDK 17.34. > > > - > > A fatal error has been detected by the Java Runtime Environment: > SIGSEGV (0xb) at pc=0x00007f96b852d3a1, pid=16597, tid=16637 > # JRE version: OpenJDK Runtime Environment Zulu17.34+20-SA (17.0.3+7) > (build 17.0.3+7-LTS) > # Java VM: OpenJDK 64-Bit Server VM Zulu17.34+20-SA (17.0.3+7-LTS, > mixed mode, sharing, tiered, compressed class ptrs, g1 gc, linux-amd64) > # Problematic frame: > # V [libjvm.so+0x9e13a1] > PhaseIdealLoop::build_loop_late_post_work(Node*, bool)+0x111 > > > Do we know which lucene/solr component would trigger this error? > > Regards, > Wei > > On Thu, Oct 13, 2022 at 10:51 AM solr <fred...@rodland.no> wrote: > >> Hi. >> >> We tried to upgrade a number of solar images from 8.11 to 9.0.0 this week >> and ran into the same problems with the official docker image as well >> (approximately 30 running pods in Kubernetes, crashing on average once an >> hour). >> >> We even tried to build a docker image with another base image >> (bellsoft/liberica-openjdk-debian:17), with the same result (even though >> the actual error differed somewhat (optimize instead of >> build_loop_late_post_work)). >> >> For now we ended up with building a docker image of solr 9.0.0 based on >> openjdk:11-jre and it has been running several hours without crashes. >> Maybe the official docker image should be reverted to this until the bug is >> fixed? >> >> >> For reference: here are 2 crashes (hs_err_pidXX.log and replay_pidXX.log >> can be provided on request): >> >> >> Temurin-17.0.4.1+1: >> # >> # A fatal error has been detected by the Java Runtime Environment: >> # >> # SIGSEGV (0xb) at pc=0x00007fd81c7e6153, pid=14, tid=85 >> # >> # JRE version: OpenJDK Runtime Environment Temurin-17.0.4.1+1 >> (17.0.4.1+1) (build 17.0.4.1+1) >> # Java VM: OpenJDK 64-Bit Server VM Temurin-17.0.4.1+1 (17.0.4.1+1, mixed >> mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, >> linux-amd64) >> # Problematic frame: >> # V [libjvm.so+0xacc153] >> PhaseIdealLoop::build_loop_late_post_work(Node*, bool)+0x153 >> # >> # Core dump will be written. Default location: /core.%e.14.%t >> # >> # An error report file with more information is saved as: >> # /opt/solr-9.0.0/server/hs_err_pid14.log >> {# >> # Compiler replay data is saved as: >> # /opt/solr-9.0.0/server/replay_pid14.log >> # >> # If you would like to submit a bug report, please visit: >> # https://github.com/adoptium/adoptium-support/issues >> # >> /opt/scripts/startSolr.sh: line 54: 14 Aborted >> /opt/solr/bin/solr -f -p ${solrport} -m ${memory} >> -Dsolr.jetty.request.header.size=65535 -Dsolr.disable.allowUrls=true >> -Dsolrindex=${profile} -Dlog4j2.formatMsgNoLookups=true >> -Dfinnapp=${finnapp} -Dsolr.master.host=${masterhost} -Denable.slave=true >> -Denable.replication=true -Dsolr.http1=true >> -Dsolr.disable.shardsWhitelist=true -Dodin.job.host=solr-cl-job >> -Dodin.bap.host=solr-cl-bap -Dodin.motor.host=solr-cl-motor >> -Dodin.estate.host=solr-cl-estate >> >> >> bellsoft/liberica-openjdk-debian:17 >> # >> # A fatal error has been detected by the Java Runtime Environment: >> # >> # SIGSEGV (0xb) at pc=0x00007f96e2515e7f, pid=13, tid=76 >> # >> # JRE version: OpenJDK Runtime Environment (17.0.4.1+1) (build >> 17.0.4.1+1-LTS) >> # Java VM: OpenJDK 64-Bit Server VM (17.0.4.1+1-LTS, mixed mode, tiered, >> compressed oops, compressed class ptrs, g1 gc, linux-amd64) >> # Problematic frame: >> # V [libjvm.so+0x515e7f] PhaseIdealLoop::optimize(PhaseIterGVN&, >> LoopOptsMode)+0x13bf >> # >> # Core dump will be written. Default location: /core.%e.13.%t >> # >> # An error report file with more information is saved as: >> # /opt/solr-9.0.0/server/hs_err_pid13.log >> # >> # Compiler replay data is saved as: >> # /opt/solr-9.0.0/server/replay_pid13.log >> # >> # If you would like to submit a bug report, please visit: >> # https://bell-sw.com/support >> # >> /opt/scripts/startSolr.sh: line 54: 13 Aborted >> /opt/solr/bin/solr -f -p ${solrport} -m ${memory} >> -Dsolr.jetty.request.header.size=65535 -Dsolr.disable.allowUrls=true >> -Dsolrindex=${profile} -Dlog4j2.formatMsgNoLookups=true >> -Dfinnapp=${finnapp} -Dsolr.master.host=${masterhost} -Denable.slave=true >> -Denable.replication=true -Dsolr.http1=true >> -Dsolr.disable.shardsWhitelist=true -Dodin.job.host=solr-cl-job >> -Dodin.bap.host=solr-cl-bap -Dodin.motor.host=solr-cl-motor >> -Dodin.estate.host=solr-cl-estate >> >> >> >> >> >> >> Regards, >> >> >> Fredrik >> >> >> -- >> >> Fredrik Rødland Cell: +47 99 21 98 17 >> Maisen Pedersens vei 1 Twitter: @fredrikr >> NO-1363 Høvik, NORWAY flickr: >> http://www.flickr.com/fmmr/ >> http://rodland.no about.me http://about.me/fmr >> >>