Including my reply that by mistake was sent directly to Sathish for reference. Turned out that upgrading to JDK17 resolved the issue -----
Thanks for the extra info. There is some loop during class loading of OTEL client's http client that causes stack overflow. > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) > ~[?:?] > at okhttp3.internal.Util.<clinit>(Util.kt:604) ~[?:?] > at okhttp3.internal.concurrent.TaskRunner.<clinit>(TaskRunner.kt:309) > ~[?:?] > at okhttp3.ConnectionPool.<init>(ConnectionPool.kt:41) ~[?:?] > at okhttp3.ConnectionPool.<init>(ConnectionPool.kt:47) ~[?:?] > at okhttp3.OkHttpClient$Builder.<init>(OkHttpClient.kt:471) ~[?:?] > at > io.opentelemetry.exporter.sender.okhttp.internal.OkHttpGrpcSender.<init>(OkHttpGrpcSender.java:82) > ~[?:?] > at > io.opentelemetry.exporter.sender.okhttp.internal.OkHttpGrpcSenderProvider.createSender(OkHttpGrpcSenderProvider.java:45) > ~[?:?] Have not seen this before. Not even sure what class it is attempting to load. What exact version of Java are you running? Could not see it from the logs. I have a suspicion that your java11 package is very old. A long shot would be to try eclipse-temurin-17 latest version to week out potential JDK bug. Also you could try Solr 9.6.1 but I cannot see any upgraded dependencies related to OTEL. I fear taht increasing stack size may not help if this is a bug/loop. PS: It is a bad practice to run solr as root user. Have you considered installing it with the install_solr_service.sh script? It will create a separate user account and a daemon script to run Solr as that user on startup. Jan > 8. aug. 2024 kl. 02:14 skrev Sathish Ponnusamy <sathishrp...@gmail.com>: > > Hi Jan, > > Thanks for checking on this issue. Here is the solr.in.sh > <http://solr.in.sh/> file & Error logs for your reference.and other details > as requested. > Solr is working fine and able to see the admin console perfectly if the open > telemetry module is not enabled. > OTEL Collector is installed on the same VM and it is running fine. But I > didn't spend much time on it since my goal is to start the solr server with > opentelemetry module and fix the collector issue if any later. > > > &&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&& Server Details &&&&&&&&&&&&&&&&&&&&&&&&& > > > > GCP VM > > OS Image - Customized Linux RHEL-9 family > > GCP Machine Type - c3d-highmem-16 > > > > --------------------------------------------------------------------------------------------------------------------------- > > > > > > > > &&&&&&&&&&&&&&& Solr Instllation Steps &&&&&&&&&&&&&&&&&&&&&&&&&&&&&&& > > > > sudo dnf install java-11-openjdk > > > > cd data > > dnf install wget > > wget https://archive.apache.org/dist/solr/solr/9.6.0/solr-9.6.0.tgz > > /data/solr-9.6.0/bin/solr start -force -cloud > > --------------------------------------------------------------------------------------------------------------------------- > > > > > > &&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&& solr.in.sh <http://solr.in.sh/> > &&&&&&&&&&&&&&&&&&&&&&&& > > > > # SOLR_ADMIN_UI_DISABLED=false > > > > # Solr is by default allowed to read and write data from/to SOLR_HOME and a > few other well defined locations > > # Sometimes it may be necessary to place a core or a backup on a different > location or a different disk > > # This parameter lets you specify file system path(s) to explicitly allow. > The special value of '*' will allow any path > > #SOLR_OPTS="$SOLR_OPTS -Dsolr.allowPaths=/mnt/bigdisk,/other/path" > > > > # Solr can attempt to take a heap dump on out of memory errors. To enable > this, uncomment the line setting > > # SOLR_HEAP_DUMP below. Heap dumps will be saved to SOLR_LOG_DIR/dumps by > default. Alternatively, you can specify any > > # other directory, which will implicitly enable heap dumping. Dump name > pattern will be solr-[timestamp]-pid[###].hprof > > # When using this feature, it is recommended to have an external service > monitoring the given dir. > > # If more fine grained control is required, you can manually add the > appropriate flags to SOLR_OPTS > > # See > https://docs.oracle.com/en/java/javase/11/troubleshoot/command-line-options1.html > > # You can test this behavior by setting SOLR_HEAP=25m > > #SOLR_HEAP_DUMP=true > > #SOLR_HEAP_DUMP_DIR=/var/log/dumps > > > > # Before version 9.0, Solr required a copy of solr.xml file in $SOLR_HOME. > Now Solr will use a default file if not found. > > # To restore the old behavior, set the variable below to true > > #SOLR_SOLRXML_REQUIRED=false > > > > # Some previous versions of Solr use an outdated log4j dependency. If you are > unable to use at least log4j version 2.15.0 > > # then enable the following setting to address CVE-2021-44228 > > # SOLR_OPTS="$SOLR_OPTS -Dlog4j2.formatMsgNoLookups=true" > > > > # The bundled plugins in the "modules" folder can easily be enabled as a > comma-separated list in SOLR_MODULES variable > > SOLR_MODULES=extraction,ltr,scripting,gcs-repository,opentelemetry > > #SOLR_MODULES=extraction,ltr,scripting,gcs-repository > > > > # Configure the default replica placement plugin to use if one is not > configured in cluster properties > > # See > https://solr.apache.org/guide/solr/latest/configuration-guide/replica-placement-plugins.html > for details > > #SOLR_PLACEMENTPLUGIN_DEFAULT=simple > > > > # Solr internally doesn't use cookies other than for modules such as > Kerberos/Hadoop Auth. If you don't need any of those > > # And you don't need them for an external system (such as a load balancer), > you can disable the use of a CookieStore with: > > # SOLR_OPTS="$SOLR_OPTS -Dsolr.http.disableCookies=true" > > #################################################################### > > > > ################################################################### > > #SOLR_HOST="0.0.0.0" > > SOLR_JETTY_HOST="0.0.0.0" > > ####################################################################### > > SOLR_SSL_ENABLED=true > > SOLR_SSL_KEY_STORE=/data/solr-9.6.0/server/etc/stg.anap-api.prod.gcp.net.jks > > SOLR_SSL_KEY_STORE_PASSWORD=Password@123 > > SOLR_SSL_KEY_STORE_TYPE=jks > > SOLR_SSL_TRUST_STORE=/data/solr-9.6.0/server/etc/solr_trust_store.jks > > SOLR_SSL_TRUST_STORE_PASSWORD=Password@123 > > SOLR_SSL_TRUST_STORE_TYPE=jks > > SOLR_SSL_NEED_CLIENT_AUTH=false > > SOLR_SSL_WANT_CLIENT_AUTH=true > > ################################# CIRCUIT BREAKER > ################################# > > SOLR_CIRCUITBREAKER_QUERY_MEM=85 > > SOLR_CIRCUITBREAKER_UPDATE_MEM=85 > > ############################################################################### > > ZK_HOST="geapizoos1.stg.anap-api.prod.gcp.net:2181 > <http://geapizoos1.stg.anap-api.prod.gcp.net:2181/>,geapizoos2.stg.anap-api.prod.gcp.net:2181 > > <http://geapizoos2.stg.anap-api.prod.gcp.net:2181/>,geapizoos3.stg.anap-api.prod.gcp.net:2181 > <http://geapizoos3.stg.anap-api.prod.gcp.net:2181/>" > > SOLR_HOST="geapislrs1.stg.anap-api.prod.gcp.net > <http://geapislrs1.stg.anap-api.prod.gcp.net/>" > > SOLR_PORT="6010" > > SOLR_HEAP="12g" > > SOLR_HOME="/data//cloud/node1/solr" > > ############################################################################# > > OTEL_SERVICE_NAME="ASA-ANAP-API-SOLR-STG" > > OTEL_SDK_DISABLED="false" > > OTEL_TRACES_EXPORTER="otlp" > > OTEL_EXPORTER_OTLP_PROTOCOL="grpc" > > OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 <http://localhost:4317/> > OTEL_TRACES_SAMPLER="parentbased_always_on" > > OTEL_PROPAGATORS="tracecontext,baggage" > > > > ------------------------------------------------------------------------------------------------------------------------------------ > > > &&&&&&&&&&&&&&&&&&&&&&&&& Complete Error trace > &&&&&&&&&&&&&&&&&&&&&&&&&&&&&&& > > 2024-08-07 11:53:16.331 ERROR (main) [c: s: r: x: t:] > o.a.s.s.CoreContainerProvider Could not start Solr. Check solr/home property > and the logs => java.lang.StackOverflowError > > at > java.base/java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1541) > > java.lang.StackOverflowError: null > > at > java.base/java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1541) > ~[?:?] > > at > java.base/java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:673) > ~[?:?] > > at > java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:591) > ~[?:?] > > at > java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:579) > ~[?:?] > > at > java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:581) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:581) > ~[?:?] > > at > java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) > ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) > ~[?:?] > > at > java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) > ~[?:?] > > at > java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) > ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) > ~[?:?] > > at java.base/java.security.AccessController.doPrivileged(Native > Method) ~[?:?] > > at > java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) > ~[?:?] > > at > java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) > ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) > ~[?:?] > > at > java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) > ~[?:?] > > at > java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) > ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) > ~[?:?] > > at java.base/java.security.AccessController.doPrivileged(Native > Method) ~[?:?] > > at > java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) > ~[?:?] > > at > java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) > ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) > ~[?:?] > > at > java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) > ~[?:?] > > at > java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) > ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) > ~[?:?] > > at java.base/java.security.AccessController.doPrivileged(Native > Method) ~[?:?] > > at > java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) > ~[?:?] > > at > java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) > ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) > ~[?:?] > > at > java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) > ~[?:?] > > at > java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) > ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) > ~[?:?] > > at java.base/java.security.AccessController.doPrivileged(Native > Method) ~[?:?] > > at > java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) > ~[?:?] > > at > java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) > ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) > ~[?:?] > > at > java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) > ~[?:?] > > at > java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) > ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) > ~[?:?] > > at java.base/java.security.AccessController.doPrivileged(Native > Method) ~[?:?] > > at > java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) > ~[?:?] > > at > java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) > ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) > ~[?:?] > > at > java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) > ~[?:?] > > at > java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) > ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) > ~[?:?] > > at java.base/java.security.AccessController.doPrivileged(Native > Method) ~[?:?] > > at > java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) > ~[?:?] > > at > java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) > ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) > ~[?:?] > > at > java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) > ~[?:?] > > at > java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) > ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) > ~[?:?] > > at java.base/java.security.AccessController.doPrivileged(Native > Method) ~[?:?] > > at > java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) > ~[?:?] > > at > java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) > ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) > ~[?:?] > > at > java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) > ~[?:?] > > at > java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) > ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) > ~[?:?] > > at java.base/java.security.AccessController.doPrivileged(Native > Method) ~[?:?] > > at > java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) > ~[?:?] > > at > java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) > ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) > ~[?:?] > > at > java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) > ~[?:?] > > at > java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) > ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) > ~[?:?] > > at java.base/java.security.AccessController.doPrivileged(Native > Method) ~[?:?] > > at > java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) > ~[?:?] > > at > java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) > ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) > ~[?:?] > > at > java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) > ~[?:?] > > at > java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) > ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) > ~[?:?] > > at java.base/java.security.AccessController.doPrivileged(Native > Method) ~[?:?] > > at > java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) > ~[?:?] > > at > java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) > ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) > ~[?:?] > > at > java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) > ~[?:?] > > at > java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) > ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) > ~[?:?] > > at java.base/java.security.AccessController.doPrivileged(Native > Method) ~[?:?] > > at > java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) > ~[?:?] > > at > java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) > ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) > ~[?:?] > > at > java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) > ~[?:?] > > at > java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) > ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) > ~[?:?] > > at java.base/java.security.AccessController.doPrivileged(Native > Method) ~[?:?] > > at > java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) > ~[?:?] > > at > java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) > ~[?:?] > > at okhttp3.internal.Util.<clinit>(Util.kt:604) ~[?:?] > > at okhttp3.internal.concurrent.TaskRunner.<clinit>(TaskRunner.kt:309) > ~[?:?] > > at okhttp3.ConnectionPool.<init>(ConnectionPool.kt:41) ~[?:?] > > at okhttp3.ConnectionPool.<init>(ConnectionPool.kt:47) ~[?:?] > > at okhttp3.OkHttpClient$Builder.<init>(OkHttpClient.kt:471) ~[?:?] > > at > io.opentelemetry.exporter.sender.okhttp.internal.OkHttpGrpcSender.<init>(OkHttpGrpcSender.java:82) > ~[?:?] > > at > io.opentelemetry.exporter.sender.okhttp.internal.OkHttpGrpcSenderProvider.createSender(OkHttpGrpcSenderProvider.java:45) > ~[?:?] > > at > io.opentelemetry.exporter.internal.grpc.GrpcExporterBuilder.build(GrpcExporterBuilder.java:191) > ~[?:?] > > at > io.opentelemetry.exporter.otlp.trace.OtlpGrpcSpanExporterBuilder.build(OtlpGrpcSpanExporterBuilder.java:220) > ~[?:?] > > at > io.opentelemetry.exporter.otlp.internal.OtlpSpanExporterProvider.createExporter(OtlpSpanExporterProvider.java:72) > ~[?:?] > > at > io.opentelemetry.sdk.autoconfigure.internal.SpiHelper.lambda$loadConfigurable$0(SpiHelper.java:69) > ~[?:?] > > at > io.opentelemetry.sdk.autoconfigure.internal.NamedSpiManager.tryLoadImplementationForName(NamedSpiManager.java:51) > ~[?:?] > > at > java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1705) > ~[?:?] > > at > io.opentelemetry.sdk.autoconfigure.internal.NamedSpiManager.getByName(NamedSpiManager.java:41) > ~[?:?] > > at > io.opentelemetry.sdk.autoconfigure.SpanExporterConfiguration.configureExporter(SpanExporterConfiguration.java:91) > ~[?:?] > > at > io.opentelemetry.sdk.autoconfigure.SpanExporterConfiguration.configureSpanExporters(SpanExporterConfiguration.java:67) > ~[?:?] > > at > io.opentelemetry.sdk.autoconfigure.TracerProviderConfiguration.configureTracerProvider(TracerProviderConfiguration.java:55) > ~[?:?] > > at > io.opentelemetry.sdk.autoconfigure.AutoConfiguredOpenTelemetrySdkBuilder.build(AutoConfiguredOpenTelemetrySdkBuilder.java:416) > ~[?:?] > > at > io.opentelemetry.sdk.autoconfigure.AutoConfiguredOpenTelemetrySdk.initialize(AutoConfiguredOpenTelemetrySdk.java:34) > ~[?:?] > > at > org.apache.solr.opentelemetry.OtelTracerConfigurator.getTracer(OtelTracerConfigurator.java:75) > ~[?:?] > > at > org.apache.solr.core.TracerConfigurator.autoConfigOTEL(TracerConfigurator.java:122) > ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - > 2024-04-22 23:20:52] > > at > org.apache.solr.core.TracerConfigurator.lambda$loadTracer$1(TracerConfigurator.java:65) > ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - > 2024-04-22 23:20:52] > > at > io.opentracing.util.GlobalTracer.registerIfAbsent(GlobalTracer.java:121) > ~[opentracing-util-0.33.0.jar:?] > > at > org.apache.solr.core.TracerConfigurator.loadTracer(TracerConfigurator.java:65) > ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - > 2024-04-22 23:20:52] > > at > org.apache.solr.core.CoreContainer.loadInternal(CoreContainer.java:795) > ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - > 2024-04-22 23:20:52] > > at org.apache.solr.core.CoreContainer.load(CoreContainer.java:762) > ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - > 2024-04-22 23:20:52] > > at > org.apache.solr.servlet.CoreContainerProvider.createCoreContainer(CoreContainerProvider.java:427) > ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - > 2024-04-22 23:20:52] > > at > org.apache.solr.servlet.CoreContainerProvider.init(CoreContainerProvider.java:246) > ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - > 2024-04-22 23:20:52] > > at > org.apache.solr.servlet.CoreContainerProvider.contextInitialized(CoreContainerProvider.java:116) > ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - > 2024-04-22 23:20:52] > > at > org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1049) > ~[jetty-server-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:624) > ~[jetty-servlet-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:984) > ~[jetty-server-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:740) > ~[jetty-servlet-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:392) > ~[jetty-servlet-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1304) > ~[jetty-webapp-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:901) > ~[jetty-server-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:306) > ~[jetty-servlet-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:532) > ~[jetty-webapp-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:121) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) > ~[jetty-server-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) > ~[jetty-server-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:121) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) > ~[jetty-server-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) > ~[jetty-server-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) > ~[jetty-server-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:221) > ~[jetty-server-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) > ~[jetty-util-10.0.20.jar:10.0.20] > > at org.eclipse.jetty.server.Server.start(Server.java:470) > ~[jetty-server-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) > ~[jetty-server-10.0.20.jar:10.0.20] > > at org.eclipse.jetty.server.Server.doStart(Server.java:415) > ~[jetty-server-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1919) > ~[jetty-xml-10.0.20.jar:10.0.20] > > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) ~[?:?] > > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > ~[?:?] > > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > ~[?:?] > > at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[?:?] > > at org.eclipse.jetty.start.Main.invokeMain(Main.java:229) > ~[start.jar:10.0.20] > > at org.eclipse.jetty.start.Main.start(Main.java:528) > ~[start.jar:10.0.20] > > at org.eclipse.jetty.start.Main.main(Main.java:76) > ~[start.jar:10.0.20] > > 2024-08-07 11:53:16.331 WARN (main) [c: s: r: x: t:] o.e.j.w.WebAppContext > Failed startup of context > o.e.j.w.WebAppContext@5b69d40d{solr-jetty-context.xml,/solr,file:///data/solr-9.6.0/server/solr-webapp/webapp/,UNAVAILABLE}{/data/solr-9.6.0/server/solr-webapp/webapp} > => java.lang.StackOverflowError > > at > java.base/java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1541) > > java.lang.StackOverflowError: null > > at > java.base/java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1541) > ~[?:?] > > at > java.base/java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:673) > ~[?:?] > > at > java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:591) > ~[?:?] > > at > java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:579) > ~[?:?] > > at > java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:581) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:581) > ~[?:?] > > at > java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) > ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) > ~[?:?] > > at > java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) > ~[?:?] > > at > java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) > ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) > ~[?:?] > > at java.base/java.security.AccessController.doPrivileged(Native > Method) ~[?:?] > > at > java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) > ~[?:?] > > at > java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) > ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) > ~[?:?] > > at > java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) > ~[?:?] > > at > java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) > ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) > ~[?:?] > > at java.base/java.security.AccessController.doPrivileged(Native > Method) ~[?:?] > > at > java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) > ~[?:?] > > at > java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) > ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) > ~[?:?] > > at > java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) > ~[?:?] > > at > java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) > ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) > ~[?:?] > > at java.base/java.security.AccessController.doPrivileged(Native > Method) ~[?:?] > > at > java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) > ~[?:?] > > at > java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) > ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) > ~[?:?] > > at > java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) > ~[?:?] > > at > java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) > ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) > ~[?:?] > > at java.base/java.security.AccessController.doPrivileged(Native > Method) ~[?:?] > > at > java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) > ~[?:?] > > at > java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) > ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) > ~[?:?] > > at > java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) > ~[?:?] > > at > java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) > ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) > ~[?:?] > > at java.base/java.security.AccessController.doPrivileged(Native > Method) ~[?:?] > > at > java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) > ~[?:?] > > at > java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) > ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) > ~[?:?] > > at > java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) > ~[?:?] > > at > java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) > ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) > ~[?:?] > > at java.base/java.security.AccessController.doPrivileged(Native > Method) ~[?:?] > > at > java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) > ~[?:?] > > at > java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) > ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) > ~[?:?] > > at > java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) > ~[?:?] > > at > java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) > ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) > ~[?:?] > > at java.base/java.security.AccessController.doPrivileged(Native > Method) ~[?:?] > > at > java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) > ~[?:?] > > at > java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) > ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) > ~[?:?] > > at > java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) > ~[?:?] > > at > java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) > ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) > ~[?:?] > > at java.base/java.security.AccessController.doPrivileged(Native > Method) ~[?:?] > > at > java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) > ~[?:?] > > at > java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) > ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) > ~[?:?] > > at > java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) > ~[?:?] > > at > java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) > ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) > ~[?:?] > > at java.base/java.security.AccessController.doPrivileged(Native > Method) ~[?:?] > > at > java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) > ~[?:?] > > at > java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) > ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) > ~[?:?] > > at > java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) > ~[?:?] > > at > java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) > ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) > ~[?:?] > > at java.base/java.security.AccessController.doPrivileged(Native > Method) ~[?:?] > > at > java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) > ~[?:?] > > at > java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) > ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) > ~[?:?] > > at > java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) > ~[?:?] > > at > java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) > ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) > ~[?:?] > > at java.base/java.security.AccessController.doPrivileged(Native > Method) ~[?:?] > > at > java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) > ~[?:?] > > at > java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) > ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] > > at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) > ~[?:?] > > at > java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) > ~[?:?] > > at > java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) > ~[?:?] > > at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) > ~[?:?] > > at java.base/java.security.AccessController.doPrivileged(Native > Method) ~[?:?] > > at > java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) > ~[?:?] > > at > java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) > ~[?:?] > > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) > ~[?:?] > > at okhttp3.internal.Util.<clinit>(Util.kt:604) ~[?:?] > > at okhttp3.internal.concurrent.TaskRunner.<clinit>(TaskRunner.kt:309) > ~[?:?] > > at okhttp3.ConnectionPool.<init>(ConnectionPool.kt:41) ~[?:?] > > at okhttp3.ConnectionPool.<init>(ConnectionPool.kt:47) ~[?:?] > > at okhttp3.OkHttpClient$Builder.<init>(OkHttpClient.kt:471) ~[?:?] > > at > io.opentelemetry.exporter.sender.okhttp.internal.OkHttpGrpcSender.<init>(OkHttpGrpcSender.java:82) > ~[?:?] > > at > io.opentelemetry.exporter.sender.okhttp.internal.OkHttpGrpcSenderProvider.createSender(OkHttpGrpcSenderProvider.java:45) > ~[?:?] > > at > io.opentelemetry.exporter.internal.grpc.GrpcExporterBuilder.build(GrpcExporterBuilder.java:191) > ~[?:?] > > at > io.opentelemetry.exporter.otlp.trace.OtlpGrpcSpanExporterBuilder.build(OtlpGrpcSpanExporterBuilder.java:220) > ~[?:?] > > at > io.opentelemetry.exporter.otlp.internal.OtlpSpanExporterProvider.createExporter(OtlpSpanExporterProvider.java:72) > ~[?:?] > > at > io.opentelemetry.sdk.autoconfigure.internal.SpiHelper.lambda$loadConfigurable$0(SpiHelper.java:69) > ~[?:?] > > at > io.opentelemetry.sdk.autoconfigure.internal.NamedSpiManager.tryLoadImplementationForName(NamedSpiManager.java:51) > ~[?:?] > > at > java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1705) > ~[?:?] > > at > io.opentelemetry.sdk.autoconfigure.internal.NamedSpiManager.getByName(NamedSpiManager.java:41) > ~[?:?] > > at > io.opentelemetry.sdk.autoconfigure.SpanExporterConfiguration.configureExporter(SpanExporterConfiguration.java:91) > ~[?:?] > > at > io.opentelemetry.sdk.autoconfigure.SpanExporterConfiguration.configureSpanExporters(SpanExporterConfiguration.java:67) > ~[?:?] > > at > io.opentelemetry.sdk.autoconfigure.TracerProviderConfiguration.configureTracerProvider(TracerProviderConfiguration.java:55) > ~[?:?] > > at > io.opentelemetry.sdk.autoconfigure.AutoConfiguredOpenTelemetrySdkBuilder.build(AutoConfiguredOpenTelemetrySdkBuilder.java:416) > ~[?:?] > > at > io.opentelemetry.sdk.autoconfigure.AutoConfiguredOpenTelemetrySdk.initialize(AutoConfiguredOpenTelemetrySdk.java:34) > ~[?:?] > > at > org.apache.solr.opentelemetry.OtelTracerConfigurator.getTracer(OtelTracerConfigurator.java:75) > ~[?:?] > > at > org.apache.solr.core.TracerConfigurator.autoConfigOTEL(TracerConfigurator.java:122) > ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - > 2024-04-22 23:20:52] > > at > org.apache.solr.core.TracerConfigurator.lambda$loadTracer$1(TracerConfigurator.java:65) > ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - > 2024-04-22 23:20:52] > > at > io.opentracing.util.GlobalTracer.registerIfAbsent(GlobalTracer.java:121) > ~[opentracing-util-0.33.0.jar:?] > > at > org.apache.solr.core.TracerConfigurator.loadTracer(TracerConfigurator.java:65) > ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - > 2024-04-22 23:20:52] > > at > org.apache.solr.core.CoreContainer.loadInternal(CoreContainer.java:795) > ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - > 2024-04-22 23:20:52] > > at org.apache.solr.core.CoreContainer.load(CoreContainer.java:762) > ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - > 2024-04-22 23:20:52] > > at > org.apache.solr.servlet.CoreContainerProvider.createCoreContainer(CoreContainerProvider.java:427) > ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - > 2024-04-22 23:20:52] > > at > org.apache.solr.servlet.CoreContainerProvider.init(CoreContainerProvider.java:246) > ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - > 2024-04-22 23:20:52] > > at > org.apache.solr.servlet.CoreContainerProvider.contextInitialized(CoreContainerProvider.java:116) > ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - > 2024-04-22 23:20:52] > > at > org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1049) > ~[jetty-server-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:624) > ~[jetty-servlet-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:984) > ~[jetty-server-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:740) > ~[jetty-servlet-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:392) > ~[jetty-servlet-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1304) > ~[jetty-webapp-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:901) > ~[jetty-server-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:306) > ~[jetty-servlet-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:532) > ~[jetty-webapp-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:121) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) > ~[jetty-server-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) > ~[jetty-server-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:121) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) > ~[jetty-server-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) > ~[jetty-server-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) > ~[jetty-server-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:221) > ~[jetty-server-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) > ~[jetty-util-10.0.20.jar:10.0.20] > > at org.eclipse.jetty.server.Server.start(Server.java:470) > ~[jetty-server-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) > ~[jetty-server-10.0.20.jar:10.0.20] > > at org.eclipse.jetty.server.Server.doStart(Server.java:415) > ~[jetty-server-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) > ~[jetty-util-10.0.20.jar:10.0.20] > > at > org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1919) > ~[jetty-xml-10.0.20.jar:10.0.20] > > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) ~[?:?] > > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > ~[?:?] > > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > ~[?:?] > > at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[?:?] > > at org.eclipse.jetty.start.Main.invokeMain(Main.java:229) > ~[start.jar:10.0.20] > > at org.eclipse.jetty.start.Main.start(Main.java:528) > ~[start.jar:10.0.20] > > at org.eclipse.jetty.start.Main.main(Main.java:76) > ~[start.jar:10.0.20] > > 2024-08-07 11:53:16.340 INFO (main) [c: s: r: x: t:] > o.e.j.s.RequestLogWriter Opened > /data/solr-9.6.0/server/logs/2024_08_07.request.log > > 2024-08-07 11:53:16.348 INFO (main) [c: s: r: x: t:] > o.e.j.u.s.SslContextFactory x509=X509@46a795de(stg.anap-api.prod.gcp.net > <http://stg.anap-api.prod.gcp.net/>,h=[stg.anap-api.prod.gcp.net > <http://stg.anap-api.prod.gcp.net/>, geapislrs1.stg.anap-api.prod.gcp.net > <http://geapislrs1.stg.anap-api.prod.gcp.net/>, > geapislrs2.stg.anap-api.prod.gcp.net > <http://geapislrs2.stg.anap-api.prod.gcp.net/>, > geapislrs3.stg.anap-api.prod.gcp.net > <http://geapislrs3.stg.anap-api.prod.gcp.net/>],a=[],w=[]) for > Server@44f0ff2b[provider=null,keyStore=file:///data/solr-9.6.0/server/etc/stg.anap-api.prod.gcp.net.jks,trustStore=file:///data/solr-9.6.0/server/etc/solr_trust_store.jks] > > 2024-08-07 11:53:16.362 INFO (main) [c: s: r: x: t:] > o.e.j.s.AbstractConnector Started ServerConnector@182f1e9a{SSL, (ssl, alpn, > h2, http/1.1)}{0.0.0.0:6010 <http://0.0.0.0:6010/>} > > 2024-08-07 11:53:16.363 INFO (main) [c: s: r: x: t:] o.e.j.s.Server Started > Server@173b9122{STARTING}[10.0.20,sto=0] @3075ms > ----------------------------------------------------------------------------------------------------------------------------------------------------------------- > > On Wed, Aug 7, 2024 at 11:28 PM Jan Høydahl <jan....@cominvent.com > <mailto:jan....@cominvent.com>> wrote: >> Hi, >> >> Please provide more information, such as the complete stack trace, the >> complete solr.in <http://solr.in/>.sh file, all ENV variables etc, how Solr >> was intstalled, what Linux flavor etc. >> Is it so that Solr starts fine if opentelementry module is not enabled? >> Do you have an OTEL collector setup on another VM? >> Can you reproduce the same error locally on your laptop? >> >> Jan >> >>> 7. aug. 2024 kl. 18:56 skrev Sathish Ponnusamy <sathishrp...@gmail.com >>> <mailto:sathishrp...@gmail.com>>: >>> >>> Hi Team, >>> >>> We are using solr/zk 3+3 nodes running on GCP VM while enabling open >>> telemetry module getting below error in solr.log and the solr node is down >>> with 503 unavailable error in the console, after starting the node. How to >>> fix the error and make the solr node up with the open telemetry module >>> enabled? >>> >>> Configuration: >>> * Java - 11 >>> * GCP VM >>> * solr version - 9.6.0 >>> * Enabled opentelemetry module via solr.in.sh <http://solr.in.sh/> file >>> * OTEL_SEVICE_NAME value also set in solr.in.sh <http://solr.in.sh/> file >>> >>> *Error in the solr.log:* >>> >>> "CoreContainerProvider Could not start Solr. Check solr/home property and >>> the logs => java.lang.StackOverflowError >>> >>> at >>> java.base/java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1541) >>> >>> java.lang.StackOverflowError: null" >>> >>> >>> Regards >>> >>> Sathish Ponnusamy >>> >>> + 91 9962331981 >>
