Hi Jan, Thanks for checking on this issue. Here is the solr.in.sh file & Error logs for your reference.and other details as requested. Solr is working fine and able to see the admin console perfectly if the open telemetry module is not enabled. OTEL Collector is installed on the same VM and it is running fine. But I didn't spend much time on it since my goal is to start the solr server with opentelemetry module and fix the collector issue if any later.
&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&& Server Details &&&&&&&&&&&&&&&&&&&&&&&&& GCP VM OS Image - Customized Linux RHEL-9 family GCP Machine Type - c3d-highmem-16 --------------------------------------------------------------------------------------------------------------------------- &&&&&&&&&&&&&&& Solr Instllation Steps &&&&&&&&&&&&&&&&&&&&&&&&&&&&&&& sudo dnf install java-11-openjdk cd data dnf install wget wget https://archive.apache.org/dist/solr/solr/9.6.0/solr-9.6.0.tgz /data/solr-9.6.0/bin/solr start -force -cloud --------------------------------------------------------------------------------------------------------------------------- &&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&& solr.in.sh &&&&&&&&&&&&&&&&&&&&&&&& # SOLR_ADMIN_UI_DISABLED=false # Solr is by default allowed to read and write data from/to SOLR_HOME and a few other well defined locations # Sometimes it may be necessary to place a core or a backup on a different location or a different disk # This parameter lets you specify file system path(s) to explicitly allow. The special value of '*' will allow any path #SOLR_OPTS="$SOLR_OPTS -Dsolr.allowPaths=/mnt/bigdisk,/other/path" # Solr can attempt to take a heap dump on out of memory errors. To enable this, uncomment the line setting # SOLR_HEAP_DUMP below. Heap dumps will be saved to SOLR_LOG_DIR/dumps by default. Alternatively, you can specify any # other directory, which will implicitly enable heap dumping. Dump name pattern will be solr-[timestamp]-pid[###].hprof # When using this feature, it is recommended to have an external service monitoring the given dir. # If more fine grained control is required, you can manually add the appropriate flags to SOLR_OPTS # See https://docs.oracle.com/en/java/javase/11/troubleshoot/command-line-options1.html # You can test this behavior by setting SOLR_HEAP=25m #SOLR_HEAP_DUMP=true #SOLR_HEAP_DUMP_DIR=/var/log/dumps # Before version 9.0, Solr required a copy of solr.xml file in $SOLR_HOME. Now Solr will use a default file if not found. # To restore the old behavior, set the variable below to true #SOLR_SOLRXML_REQUIRED=false # Some previous versions of Solr use an outdated log4j dependency. If you are unable to use at least log4j version 2.15.0 # then enable the following setting to address CVE-2021-44228 # SOLR_OPTS="$SOLR_OPTS -Dlog4j2.formatMsgNoLookups=true" # The bundled plugins in the "modules" folder can easily be enabled as a comma-separated list in SOLR_MODULES variable SOLR_MODULES=extraction,ltr,scripting,gcs-repository,opentelemetry #SOLR_MODULES=extraction,ltr,scripting,gcs-repository # Configure the default replica placement plugin to use if one is not configured in cluster properties # See https://solr.apache.org/guide/solr/latest/configuration-guide/replica-placement-plugins.html for details #SOLR_PLACEMENTPLUGIN_DEFAULT=simple # Solr internally doesn't use cookies other than for modules such as Kerberos/Hadoop Auth. If you don't need any of those # And you don't need them for an external system (such as a load balancer), you can disable the use of a CookieStore with: # SOLR_OPTS="$SOLR_OPTS -Dsolr.http.disableCookies=true" #################################################################### ################################################################### #SOLR_HOST="0.0.0.0" SOLR_JETTY_HOST="0.0.0.0" ####################################################################### SOLR_SSL_ENABLED=true SOLR_SSL_KEY_STORE=/data/solr-9.6.0/server/etc/stg.anap-api.prod.gcp.net.jks SOLR_SSL_KEY_STORE_PASSWORD=Password@123 SOLR_SSL_KEY_STORE_TYPE=jks SOLR_SSL_TRUST_STORE=/data/solr-9.6.0/server/etc/solr_trust_store.jks SOLR_SSL_TRUST_STORE_PASSWORD=Password@123 SOLR_SSL_TRUST_STORE_TYPE=jks SOLR_SSL_NEED_CLIENT_AUTH=false SOLR_SSL_WANT_CLIENT_AUTH=true ################################# CIRCUIT BREAKER ################################# SOLR_CIRCUITBREAKER_QUERY_MEM=85 SOLR_CIRCUITBREAKER_UPDATE_MEM=85 ############################################################################### ZK_HOST="geapizoos1.stg.anap-api.prod.gcp.net:2181, geapizoos2.stg.anap-api.prod.gcp.net:2181, geapizoos3.stg.anap-api.prod.gcp.net:2181" SOLR_HOST="geapislrs1.stg.anap-api.prod.gcp.net" SOLR_PORT="6010" SOLR_HEAP="12g" SOLR_HOME="/data//cloud/node1/solr" ############################################################################# OTEL_SERVICE_NAME="ASA-ANAP-API-SOLR-STG" OTEL_SDK_DISABLED="false" OTEL_TRACES_EXPORTER="otlp" OTEL_EXPORTER_OTLP_PROTOCOL="grpc" OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 OTEL_TRACES_SAMPLER="parentbased_always_on" OTEL_PROPAGATORS="tracecontext,baggage" ------------------------------------------------------------------------------------------------------------------------------------ &&&&&&&&&&&&&&&&&&&&&&&&& Complete Error trace &&&&&&&&&&&&&&&&&&&&&&&&&&&&&&& 2024-08-07 11:53:16.331 ERROR (main) [c: s: r: x: t:] o.a.s.s.CoreContainerProvider Could not start Solr. Check solr/home property and the logs => java.lang.StackOverflowError at java.base/java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1541) java.lang.StackOverflowError: null at java.base/java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1541) ~[?:?] at java.base/java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:673) ~[?:?] at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:591) ~[?:?] at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:579) ~[?:?] at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:581) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:581) ~[?:?] at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) ~[?:?] at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) ~[?:?] at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) ~[?:?] at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) ~[?:?] at java.base/java.security.AccessController.doPrivileged(Native Method) ~[?:?] at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) ~[?:?] at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) ~[?:?] at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) ~[?:?] at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) ~[?:?] at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) ~[?:?] at java.base/java.security.AccessController.doPrivileged(Native Method) ~[?:?] at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) ~[?:?] at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) ~[?:?] at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) ~[?:?] at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) ~[?:?] at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) ~[?:?] at java.base/java.security.AccessController.doPrivileged(Native Method) ~[?:?] at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) ~[?:?] at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) ~[?:?] at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) ~[?:?] at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) ~[?:?] at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) ~[?:?] at java.base/java.security.AccessController.doPrivileged(Native Method) ~[?:?] at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) ~[?:?] at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) ~[?:?] at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) ~[?:?] at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) ~[?:?] at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) ~[?:?] at java.base/java.security.AccessController.doPrivileged(Native Method) ~[?:?] at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) ~[?:?] at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) ~[?:?] at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) ~[?:?] at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) ~[?:?] at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) ~[?:?] at java.base/java.security.AccessController.doPrivileged(Native Method) ~[?:?] at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) ~[?:?] at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) ~[?:?] at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) ~[?:?] at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) ~[?:?] at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) ~[?:?] at java.base/java.security.AccessController.doPrivileged(Native Method) ~[?:?] at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) ~[?:?] at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) ~[?:?] at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) ~[?:?] at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) ~[?:?] at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) ~[?:?] at java.base/java.security.AccessController.doPrivileged(Native Method) ~[?:?] at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) ~[?:?] at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) ~[?:?] at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) ~[?:?] at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) ~[?:?] at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) ~[?:?] at java.base/java.security.AccessController.doPrivileged(Native Method) ~[?:?] at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) ~[?:?] at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) ~[?:?] at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) ~[?:?] at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) ~[?:?] at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) ~[?:?] at java.base/java.security.AccessController.doPrivileged(Native Method) ~[?:?] at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) ~[?:?] at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) ~[?:?] at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) ~[?:?] at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) ~[?:?] at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) ~[?:?] at java.base/java.security.AccessController.doPrivileged(Native Method) ~[?:?] at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) ~[?:?] at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) ~[?:?] at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) ~[?:?] at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) ~[?:?] at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) ~[?:?] at java.base/java.security.AccessController.doPrivileged(Native Method) ~[?:?] at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) ~[?:?] at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) ~[?:?] at okhttp3.internal.Util.<clinit>(Util.kt:604) ~[?:?] at okhttp3.internal.concurrent.TaskRunner.<clinit>(TaskRunner.kt:309) ~[?:?] at okhttp3.ConnectionPool.<init>(ConnectionPool.kt:41) ~[?:?] at okhttp3.ConnectionPool.<init>(ConnectionPool.kt:47) ~[?:?] at okhttp3.OkHttpClient$Builder.<init>(OkHttpClient.kt:471) ~[?:?] at io.opentelemetry.exporter.sender.okhttp.internal.OkHttpGrpcSender.<init>(OkHttpGrpcSender.java:82) ~[?:?] at io.opentelemetry.exporter.sender.okhttp.internal.OkHttpGrpcSenderProvider.createSender(OkHttpGrpcSenderProvider.java:45) ~[?:?] at io.opentelemetry.exporter.internal.grpc.GrpcExporterBuilder.build(GrpcExporterBuilder.java:191) ~[?:?] at io.opentelemetry.exporter.otlp.trace.OtlpGrpcSpanExporterBuilder.build(OtlpGrpcSpanExporterBuilder.java:220) ~[?:?] at io.opentelemetry.exporter.otlp.internal.OtlpSpanExporterProvider.createExporter(OtlpSpanExporterProvider.java:72) ~[?:?] at io.opentelemetry.sdk.autoconfigure.internal.SpiHelper.lambda$loadConfigurable$0(SpiHelper.java:69) ~[?:?] at io.opentelemetry.sdk.autoconfigure.internal.NamedSpiManager.tryLoadImplementationForName(NamedSpiManager.java:51) ~[?:?] at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1705) ~[?:?] at io.opentelemetry.sdk.autoconfigure.internal.NamedSpiManager.getByName(NamedSpiManager.java:41) ~[?:?] at io.opentelemetry.sdk.autoconfigure.SpanExporterConfiguration.configureExporter(SpanExporterConfiguration.java:91) ~[?:?] at io.opentelemetry.sdk.autoconfigure.SpanExporterConfiguration.configureSpanExporters(SpanExporterConfiguration.java:67) ~[?:?] at io.opentelemetry.sdk.autoconfigure.TracerProviderConfiguration.configureTracerProvider(TracerProviderConfiguration.java:55) ~[?:?] at io.opentelemetry.sdk.autoconfigure.AutoConfiguredOpenTelemetrySdkBuilder.build(AutoConfiguredOpenTelemetrySdkBuilder.java:416) ~[?:?] at io.opentelemetry.sdk.autoconfigure.AutoConfiguredOpenTelemetrySdk.initialize(AutoConfiguredOpenTelemetrySdk.java:34) ~[?:?] at org.apache.solr.opentelemetry.OtelTracerConfigurator.getTracer(OtelTracerConfigurator.java:75) ~[?:?] at org.apache.solr.core.TracerConfigurator.autoConfigOTEL(TracerConfigurator.java:122) ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - 2024-04-22 23:20:52] at org.apache.solr.core.TracerConfigurator.lambda$loadTracer$1(TracerConfigurator.java:65) ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - 2024-04-22 23:20:52] at io.opentracing.util.GlobalTracer.registerIfAbsent(GlobalTracer.java:121) ~[opentracing-util-0.33.0.jar:?] at org.apache.solr.core.TracerConfigurator.loadTracer(TracerConfigurator.java:65) ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - 2024-04-22 23:20:52] at org.apache.solr.core.CoreContainer.loadInternal(CoreContainer.java:795) ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - 2024-04-22 23:20:52] at org.apache.solr.core.CoreContainer.load(CoreContainer.java:762) ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - 2024-04-22 23:20:52] at org.apache.solr.servlet.CoreContainerProvider.createCoreContainer(CoreContainerProvider.java:427) ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - 2024-04-22 23:20:52] at org.apache.solr.servlet.CoreContainerProvider.init(CoreContainerProvider.java:246) ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - 2024-04-22 23:20:52] at org.apache.solr.servlet.CoreContainerProvider.contextInitialized(CoreContainerProvider.java:116) ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - 2024-04-22 23:20:52] at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1049) ~[jetty-server-10.0.20.jar:10.0.20] at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:624) ~[jetty-servlet-10.0.20.jar:10.0.20] at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:984) ~[jetty-server-10.0.20.jar:10.0.20] at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:740) ~[jetty-servlet-10.0.20.jar:10.0.20] at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:392) ~[jetty-servlet-10.0.20.jar:10.0.20] at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1304) ~[jetty-webapp-10.0.20.jar:10.0.20] at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:901) ~[jetty-server-10.0.20.jar:10.0.20] at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:306) ~[jetty-servlet-10.0.20.jar:10.0.20] at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:532) ~[jetty-webapp-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:121) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) ~[jetty-server-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) ~[jetty-server-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:121) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) ~[jetty-server-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) ~[jetty-server-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) ~[jetty-server-10.0.20.jar:10.0.20] at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:221) ~[jetty-server-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.server.Server.start(Server.java:470) ~[jetty-server-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) ~[jetty-server-10.0.20.jar:10.0.20] at org.eclipse.jetty.server.Server.doStart(Server.java:415) ~[jetty-server-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1919) ~[jetty-xml-10.0.20.jar:10.0.20] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?] at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[?:?] at org.eclipse.jetty.start.Main.invokeMain(Main.java:229) ~[start.jar:10.0.20] at org.eclipse.jetty.start.Main.start(Main.java:528) ~[start.jar:10.0.20] at org.eclipse.jetty.start.Main.main(Main.java:76) ~[start.jar:10.0.20] 2024-08-07 11:53:16.331 WARN (main) [c: s: r: x: t:] o.e.j.w.WebAppContext Failed startup of context o.e.j.w.WebAppContext@5b69d40d{solr-jetty-context.xml,/solr,file:///data/solr-9.6.0/server/solr-webapp/webapp/,UNAVAILABLE}{/data/solr-9.6.0/server/solr-webapp/webapp} => java.lang.StackOverflowError at java.base/java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1541) java.lang.StackOverflowError: null at java.base/java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1541) ~[?:?] at java.base/java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:673) ~[?:?] at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:591) ~[?:?] at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:579) ~[?:?] at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:581) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:581) ~[?:?] at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) ~[?:?] at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) ~[?:?] at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) ~[?:?] at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) ~[?:?] at java.base/java.security.AccessController.doPrivileged(Native Method) ~[?:?] at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) ~[?:?] at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) ~[?:?] at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) ~[?:?] at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) ~[?:?] at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) ~[?:?] at java.base/java.security.AccessController.doPrivileged(Native Method) ~[?:?] at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) ~[?:?] at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) ~[?:?] at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) ~[?:?] at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) ~[?:?] at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) ~[?:?] at java.base/java.security.AccessController.doPrivileged(Native Method) ~[?:?] at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) ~[?:?] at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) ~[?:?] at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) ~[?:?] at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) ~[?:?] at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) ~[?:?] at java.base/java.security.AccessController.doPrivileged(Native Method) ~[?:?] at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) ~[?:?] at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) ~[?:?] at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) ~[?:?] at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) ~[?:?] at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) ~[?:?] at java.base/java.security.AccessController.doPrivileged(Native Method) ~[?:?] at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) ~[?:?] at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) ~[?:?] at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) ~[?:?] at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) ~[?:?] at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) ~[?:?] at java.base/java.security.AccessController.doPrivileged(Native Method) ~[?:?] at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) ~[?:?] at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) ~[?:?] at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) ~[?:?] at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) ~[?:?] at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) ~[?:?] at java.base/java.security.AccessController.doPrivileged(Native Method) ~[?:?] at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) ~[?:?] at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) ~[?:?] at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) ~[?:?] at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) ~[?:?] at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) ~[?:?] at java.base/java.security.AccessController.doPrivileged(Native Method) ~[?:?] at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) ~[?:?] at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) ~[?:?] at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) ~[?:?] at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) ~[?:?] at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) ~[?:?] at java.base/java.security.AccessController.doPrivileged(Native Method) ~[?:?] at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) ~[?:?] at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) ~[?:?] at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) ~[?:?] at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) ~[?:?] at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) ~[?:?] at java.base/java.security.AccessController.doPrivileged(Native Method) ~[?:?] at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) ~[?:?] at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) ~[?:?] at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) ~[?:?] at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) ~[?:?] at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) ~[?:?] at java.base/java.security.AccessController.doPrivileged(Native Method) ~[?:?] at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) ~[?:?] at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) ~[?:?] at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022) ~[?:?] at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174) ~[?:?] at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458) ~[?:?] at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452) ~[?:?] at java.base/java.security.AccessController.doPrivileged(Native Method) ~[?:?] at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594) ~[?:?] at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904) ~[?:?] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527) ~[?:?] at okhttp3.internal.Util.<clinit>(Util.kt:604) ~[?:?] at okhttp3.internal.concurrent.TaskRunner.<clinit>(TaskRunner.kt:309) ~[?:?] at okhttp3.ConnectionPool.<init>(ConnectionPool.kt:41) ~[?:?] at okhttp3.ConnectionPool.<init>(ConnectionPool.kt:47) ~[?:?] at okhttp3.OkHttpClient$Builder.<init>(OkHttpClient.kt:471) ~[?:?] at io.opentelemetry.exporter.sender.okhttp.internal.OkHttpGrpcSender.<init>(OkHttpGrpcSender.java:82) ~[?:?] at io.opentelemetry.exporter.sender.okhttp.internal.OkHttpGrpcSenderProvider.createSender(OkHttpGrpcSenderProvider.java:45) ~[?:?] at io.opentelemetry.exporter.internal.grpc.GrpcExporterBuilder.build(GrpcExporterBuilder.java:191) ~[?:?] at io.opentelemetry.exporter.otlp.trace.OtlpGrpcSpanExporterBuilder.build(OtlpGrpcSpanExporterBuilder.java:220) ~[?:?] at io.opentelemetry.exporter.otlp.internal.OtlpSpanExporterProvider.createExporter(OtlpSpanExporterProvider.java:72) ~[?:?] at io.opentelemetry.sdk.autoconfigure.internal.SpiHelper.lambda$loadConfigurable$0(SpiHelper.java:69) ~[?:?] at io.opentelemetry.sdk.autoconfigure.internal.NamedSpiManager.tryLoadImplementationForName(NamedSpiManager.java:51) ~[?:?] at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1705) ~[?:?] at io.opentelemetry.sdk.autoconfigure.internal.NamedSpiManager.getByName(NamedSpiManager.java:41) ~[?:?] at io.opentelemetry.sdk.autoconfigure.SpanExporterConfiguration.configureExporter(SpanExporterConfiguration.java:91) ~[?:?] at io.opentelemetry.sdk.autoconfigure.SpanExporterConfiguration.configureSpanExporters(SpanExporterConfiguration.java:67) ~[?:?] at io.opentelemetry.sdk.autoconfigure.TracerProviderConfiguration.configureTracerProvider(TracerProviderConfiguration.java:55) ~[?:?] at io.opentelemetry.sdk.autoconfigure.AutoConfiguredOpenTelemetrySdkBuilder.build(AutoConfiguredOpenTelemetrySdkBuilder.java:416) ~[?:?] at io.opentelemetry.sdk.autoconfigure.AutoConfiguredOpenTelemetrySdk.initialize(AutoConfiguredOpenTelemetrySdk.java:34) ~[?:?] at org.apache.solr.opentelemetry.OtelTracerConfigurator.getTracer(OtelTracerConfigurator.java:75) ~[?:?] at org.apache.solr.core.TracerConfigurator.autoConfigOTEL(TracerConfigurator.java:122) ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - 2024-04-22 23:20:52] at org.apache.solr.core.TracerConfigurator.lambda$loadTracer$1(TracerConfigurator.java:65) ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - 2024-04-22 23:20:52] at io.opentracing.util.GlobalTracer.registerIfAbsent(GlobalTracer.java:121) ~[opentracing-util-0.33.0.jar:?] at org.apache.solr.core.TracerConfigurator.loadTracer(TracerConfigurator.java:65) ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - 2024-04-22 23:20:52] at org.apache.solr.core.CoreContainer.loadInternal(CoreContainer.java:795) ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - 2024-04-22 23:20:52] at org.apache.solr.core.CoreContainer.load(CoreContainer.java:762) ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - 2024-04-22 23:20:52] at org.apache.solr.servlet.CoreContainerProvider.createCoreContainer(CoreContainerProvider.java:427) ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - 2024-04-22 23:20:52] at org.apache.solr.servlet.CoreContainerProvider.init(CoreContainerProvider.java:246) ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - 2024-04-22 23:20:52] at org.apache.solr.servlet.CoreContainerProvider.contextInitialized(CoreContainerProvider.java:116) ~[solr-core-9.6.0.jar:9.6.0 f8e5a93c11267e13b7b43005a428bfb910ac6e57 - gus - 2024-04-22 23:20:52] at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1049) ~[jetty-server-10.0.20.jar:10.0.20] at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:624) ~[jetty-servlet-10.0.20.jar:10.0.20] at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:984) ~[jetty-server-10.0.20.jar:10.0.20] at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:740) ~[jetty-servlet-10.0.20.jar:10.0.20] at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:392) ~[jetty-servlet-10.0.20.jar:10.0.20] at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1304) ~[jetty-webapp-10.0.20.jar:10.0.20] at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:901) ~[jetty-server-10.0.20.jar:10.0.20] at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:306) ~[jetty-servlet-10.0.20.jar:10.0.20] at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:532) ~[jetty-webapp-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:121) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) ~[jetty-server-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) ~[jetty-server-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:121) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) ~[jetty-server-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) ~[jetty-server-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) ~[jetty-server-10.0.20.jar:10.0.20] at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:221) ~[jetty-server-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:171) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.server.Server.start(Server.java:470) ~[jetty-server-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:89) ~[jetty-server-10.0.20.jar:10.0.20] at org.eclipse.jetty.server.Server.doStart(Server.java:415) ~[jetty-server-10.0.20.jar:10.0.20] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93) ~[jetty-util-10.0.20.jar:10.0.20] at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1919) ~[jetty-xml-10.0.20.jar:10.0.20] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?] at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[?:?] at org.eclipse.jetty.start.Main.invokeMain(Main.java:229) ~[start.jar:10.0.20] at org.eclipse.jetty.start.Main.start(Main.java:528) ~[start.jar:10.0.20] at org.eclipse.jetty.start.Main.main(Main.java:76) ~[start.jar:10.0.20] 2024-08-07 11:53:16.340 INFO (main) [c: s: r: x: t:] o.e.j.s.RequestLogWriter Opened /data/solr-9.6.0/server/logs/2024_08_07.request.log 2024-08-07 11:53:16.348 INFO (main) [c: s: r: x: t:] o.e.j.u.s.SslContextFactory x509=X509@46a795de(stg.anap-api.prod.gcp.net,h=[ stg.anap-api.prod.gcp.net, geapislrs1.stg.anap-api.prod.gcp.net, geapislrs2.stg.anap-api.prod.gcp.net, geapislrs3.stg.anap-api.prod.gcp.net],a=[],w=[]) for Server@44f0ff2b [provider=null,keyStore=file:///data/solr-9.6.0/server/etc/stg.anap-api.prod.gcp.net.jks,trustStore=file:///data/solr-9.6.0/server/etc/solr_trust_store.jks] 2024-08-07 11:53:16.362 INFO (main) [c: s: r: x: t:] o.e.j.s.AbstractConnector Started ServerConnector@182f1e9a{SSL, (ssl, alpn, h2, http/1.1)}{0.0.0.0:6010} 2024-08-07 11:53:16.363 INFO (main) [c: s: r: x: t:] o.e.j.s.Server Started Server@173b9122{STARTING}[10.0.20,sto=0] @3075ms ----------------------------------------------------------------------------------------------------------------------------------------------------------------- On Wed, Aug 7, 2024 at 11:28 PM Jan Høydahl <jan....@cominvent.com> wrote: > Hi, > > Please provide more information, such as the complete stack trace, the > complete solr.in.sh file, all ENV variables etc, how Solr was intstalled, > what Linux flavor etc. > Is it so that Solr starts fine if opentelementry module is not enabled? > Do you have an OTEL collector setup on another VM? > Can you reproduce the same error locally on your laptop? > > Jan > > 7. aug. 2024 kl. 18:56 skrev Sathish Ponnusamy <sathishrp...@gmail.com>: > > Hi Team, > > We are using solr/zk 3+3 nodes running on GCP VM while enabling open > telemetry module getting below error in solr.log and the solr node is down > with 503 unavailable error in the console, after starting the node. How to > fix the error and make the solr node up with the open telemetry module > enabled? > > Configuration: > * Java - 11 > * GCP VM > * solr version - 9.6.0 > * Enabled opentelemetry module via solr.in.sh file > * OTEL_SEVICE_NAME value also set in solr.in.sh file > > *Error in the solr.log:* > > "CoreContainerProvider Could not start Solr. Check solr/home property and > the logs => java.lang.StackOverflowError > > at > > java.base/java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1541) > > java.lang.StackOverflowError: null" > > > Regards > > Sathish Ponnusamy > > + 91 9962331981 > > >
