Hi,
Looks like you have configured Zookeper with client port 3181 on all three
nodes, so why don't you use that in ZK_HOST?
And you now say that you use SSL. How did you initialize SSL, did you update
urlScheme in zookeeper? I feel you are leaving details out.
It would perhaps help for both you and us if you created minimal reproduction
steps. I.e. from scratch instructions for how to reproduce your state. Chances
are that in the process of creating that list of steps, you'll find the
configuration error yourself. Example:
1. Downlaod 9.5.0.tgz from solr and unzip
2. Run install_solr_service.sh with these arguments
3. Edit /etc/default/solr.in.sh as follows xxxx
4. Install Zookeeper as follows yyyy
5. Start Solr with systemctl start solr
6. etc.....
Jan
> 17. apr. 2024 kl. 11:09 skrev HariBabu kuruva <hari2708.kur...@gmail.com>:
>
> Hi Jan,
>
> Thanks for the response.
>
> Please find the details below:
>
> *Exactly how did you install Solr, and what is your ZK_HOST connection
> string?:*
> I got the solr-9.5.0.tg.zip file from Solr website and unzipped it.
> ZK_HOST = "ZK_host1.corp.abc.com:3181,ZK_Host2.corp.abc.com:3182,
> ZK_Host3.corp.equinbcax.com:3183"
> - *Exactly how did you install ZK, and what is your zoo.cfg, on each of the
> three nodes.*
> I have got the zip file and uninstalled it.
> *zoo.cfg:*
>
> *---------------*tickTime=2000
> initLimit=10
> syncLimit=5
> 4lw.commands.whitelist=*
> dataDir=/gdev/zk1/zoo/data/1
> clientPort=3181
> autopurge.snapRetainCount=3
> autopurge.purgeInterval=12
> server.1=ZKHost1.corp.equinix.com:4888:5888
> server.2=ZKHost2.corp.equinix.com:4889:5889
> server.3=ZKHost3.corp.equinix.com:4890:5890
> admin.serverPort=8081
> -----------------------------------
> *- Which command did you use to create the collection?*
> ------- Commands: -------------------------
> update_command="/gdev/opt/eisp/solr-9.5.0/bin/solr zk -upconfig -z
> $zoo_keeper_host -n $config_name -d
> /gdev/opt/eisp/solr/configsets/$store_name"
> create_command="curl -k --user solr:SolrRocks
> '$solr_admin_url/solr/admin/collections?
> action=CREATE&name=$store_name&numShards=1&replicationFactor=2&maxShardsPerNode=1&collection.configName=$config_name'"
>
> *Note: *Configsets folder , i am using the same files of current running
> version solr-8.11.1
> ---------------------------------------
> -* Does this happen every time you try to create a collection, even if you
> create a test collection with _default configset from the UI?*
> *Yes, i can see the same error from UI also and also with _default
> configset as well*
>
> *- Did the Solr node crash or recover during this process?*
> No
> *- Are there ERRORs in Zookeeper's log files?*
> No errors in ZK logs.
>
> Please let me know if any other info is required.
>
>
>
> On Wed, Apr 17, 2024 at 12:39 PM Jan Høydahl <jan....@cominvent.com> wrote:
>
>> Hi,
>>
>> Do you mind sharing the details with us?
>>
>> - Exactly how did you install Solr, and what is your ZK_HOST connection
>> string?
>> - Exactly how did you install ZK, and what is your zoo.cfg, on each of the
>> three nodes.
>> - Which command did you use to create the collection?
>> - Does this happen every time you try to create a collection, even if you
>> create a test collection with _default configset from the UI?
>> - Did the Solr node crash or recover during this process?
>> - Are there ERRORs in Zookeeper's log files?
>>
>> Jan
>>
>>> 17. apr. 2024 kl. 02:37 skrev HariBabu kuruva <hari2708.kur...@gmail.com
>>> :
>>>
>>> HI , I am working on a POC, to set up solr-9.5.0. Currently I have
>>> Solr-8.11.1 installed and running.
>>>
>>> As part of the POC. I have installed Solr-9.5.0(on one node) and
>> Zookeeper
>>> 3.9.2 (on 3 nodes). This setup is done on the same server where I have
>>> solr-8.5 running but with different ports.
>>>
>>> Solr and Zookeeper are running on separate nodes and are running fine.
>>>
>>> When I try to create the collection using the below command. I get this
>>> error. Please help me.
>>>
>>>
>>>
>>> 2024-04-16 05:17:09.045 ERROR (OverseerThreadFactory-17-thread-5)
>> [c:test1
>>> s: r: x: t:] o.a.s.c.a.c.CollectionHandlingUtils Collect
>>> ion test1, operation create failed =>
>> org.apache.solr.common.SolrException:
>>> Underlying core creation failed while creating collecti
>>> on: test1
>>> at
>>>
>> org.apache.solr.cloud.api.collections.CreateCollectionCmd.call(CreateCollectionCmd.java:447)
>>> org.apache.solr.common.SolrException: Underlying core creation failed
>> while
>>> creating collection: test1
>>> at
>>>
>> org.apache.solr.cloud.api.collections.CreateCollectionCmd.call(CreateCollectionCmd.java:447)
>>> ~[solr-core-9.5.0.jar:9.5.0
>>> cdd27dd15c3a6574032e9b1b92b148ab4e383599 - gerlowskija - 2024-02-07
>>> 15:10:39]
>>> at
>>>
>> org.apache.solr.cloud.api.collections.CollApiCmds$TraceAwareCommand.call(CollApiCmds.java:225)
>>> ~[solr-core-9.5.0.jar:9.5
>>> .0 cdd27dd15c3a6574032e9b1b92b148ab4e383599 - gerlowskija - 2024-02-07
>>> 15:10:39]
>>> at
>>>
>> org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.j
>>> ava:130) ~[solr-core-9.5.0.jar:9.5.0
>>> cdd27dd15c3a6574032e9b1b92b148ab4e383599 - gerlowskija - 2024-02-07
>>> 15:10:39]
>>> at
>>>
>> org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:564)
>>> ~[solr-core-9.5.0.jar:9.5.0 cdd27
>>> dd15c3a6574032e9b1b92b148ab4e383599 - gerlowskija - 2024-02-07 15:10:39]
>>> at
>>>
>> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:299)
>>> ~[solr-solrj
>>> -9.5.0.jar:9.5.0 cdd27dd15c3a6574032e9b1b92b148ab4e383599 - gerlowskija -
>>> 2024-02-07 15:10:39]
>>> at
>>>
>> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>>> ~[?:?]
>>> at
>>>
>> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>>> ~[?:?]
>>> at java.base/java.lang.Thread.run(Thread.java:834) [?:?]
>>> 2024-04-16 05:17:09.054 ERROR
>>> (qtp1119242936-23-vmclxeissldi101.corp.equinix.com-50) [c:test1 s: r: x:
>>> t:vmclxeissldi101.corp.equin
>>> ix.com-50] o.a.s.h.RequestHandlerBase Client exception =>
>>> org.apache.solr.common.SolrException: Underlying core creation failed whi
>>> le creating collection: test1
>>> at
>>>
>> org.apache.solr.client.solrj.SolrResponse.getException(SolrResponse.java:56)
>>> org.apache.solr.common.SolrException: Underlying core creation failed
>> while
>>> creating collection: test1
>>> at
>>>
>> org.apache.solr.client.solrj.SolrResponse.getException(SolrResponse.java:56)
>>> ~[solr-solrj-9.5.0.jar:9.5.0 cdd27dd15c3a65
>>> 74032e9b1b92b148ab4e383599 - gerlowskija - 2024-02-07 15:10:39]
>>> at
>>>
>> org.apache.solr.handler.admin.api.CreateCollection.createCollection(CreateCollection.java:133)
>>> ~[solr-core-9.5.0.jar:9.5
>>> .0 cdd27dd15c3a6574032e9b1b92b148ab4e383599 - gerlowskija - 2024-02-07
>>> 15:10:39]
>>> at
>>>
>> org.apache.solr.handler.admin.CollectionsHandler$CollectionOperation.lambda$static$0(CollectionsHandler.java:517)
>>> ~[solr
>>> -core-9.5.0.jar:9.5.0 cdd27dd15c3a6574032e9b1b92b148ab4e383599 -
>>> gerlowskija - 2024-02-07 15:10:39]
>>> at
>>>
>> org.apache.solr.handler.admin.CollectionsHandler$CollectionOperation.execute(CollectionsHandler.java:1264)
>>> ~[solr-core-9
>>> .5.0.jar:9.5.0 cdd27dd15c3a6574032e9b1b92b148ab4e383599 - gerlowskija -
>>> 2024-02-07 15:10:39]
>>> at
>>>
>> org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:319)
>>> ~[solr-core-9.5.0.jar:9.5.0 c
>>> dd27dd15c3a6574032e9b1b92b148ab4e383599 - gerlowskija - 2024-02-07
>> 15:10:39]
>>> at
>>>
>> org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:297)
>>> ~[solr-core-9.5.0.jar:9.
>>> 5.0 cdd27dd15c3a6574032e9b1b92b148ab4e383599 - gerlowskija - 2024-02-07
>>> 15:10:39]
>>> at
>>>
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:226)
>>> ~[solr-core-9.5.0.jar:9.5.0 cdd27d
>>> d15c3a6574032e9b1b92b148ab4e383599 - gerlowskija - 2024-02-07 15:10:39]
>>> at
>>> org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:932)
>>> ~[solr-core-9.5.0.jar:9.5.0 cdd27dd15c3a6574032e
>>> 9b1b92b148ab4e383599 - gerlowskija - 2024-02-07 15:10:39]
>>> at
>>>
>> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:880)
>>> ~[solr-core-9.5.0.jar:9.5.0 cdd27dd15c3a6
>>> 574032e9b1b92b148ab4e383599 - gerlowskija - 2024-02-07 15:10:39]
>>> at
>> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:541)
>>> ~[solr-core-9.5.0.jar:9.5.0 cdd27dd15c3a6574032e9b1b92b
>>> 148ab4e383599 - gerlowskija - 2024-02-07 15:10:39]
>>> at
>>>
>> org.apache.solr.servlet.SolrDispatchFilter.dispatch(SolrDispatchFilter.java:262)
>>> ~[solr-core-9.5.0.jar:9.5.0 cdd27dd15c3
>>> a6574032e9b1b92b148ab4e383599 - gerlowskija - 2024-02-07 15:10:39]
>>> at
>>>
>> org.apache.solr.servlet.SolrDispatchFilter.lambda$doFilter$0(SolrDispatchFilter.java:219)
>>> ~[solr-core-9.5.0.jar:9.5.0 cd
>>> d27dd15c3a6574032e9b1b92b148ab4e383599 - gerlowskija - 2024-02-07
>> 15:10:39]
>>> at
>>>
>> org.apache.solr.servlet.ServletUtils.traceHttpRequestExecution2(ServletUtils.java:249)
>>> ~[solr-core-9.5.0.jar:9.5.0 cdd27
>>> dd15c3a6574032e9b1b92b148ab4e383599 - gerlowskija - 2024-02-07 15:10:39]
>>> at
>>>
>> org.apache.solr.servlet.ServletUtils.rateLimitRequest(ServletUtils.java:215)
>>> ~[solr-core-9.5.0.jar:9.5.0 cdd27dd15c3a657
>>> 4032e9b1b92b148ab4e383599 - gerlowskija - 2024-02-07 15:10:39]
>>> at
>>>
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:213)
>>> ~[solr-core-9.5.0.jar:9.5.0 cdd27dd15c3
>>> a6574032e9b1b92b148ab4e383599 - gerlowskija - 2024-02-07 15:10:39]
>>> at
>>>
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
>>> ~[solr-core-9.5.0.jar:9.5.0 cdd27dd15c3
>>> a6574032e9b1b92b148ab4e383599 - gerlowskija - 2024-02-07 15:10:39]
>>> at
>>> org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:210)
>>> ~[jetty-servlet-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)
>>> ~[jetty-servlet-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:527)
>>> ~[jetty-servlet-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:131)
>>> ~[jetty-server-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:598)
>>> ~[jetty-security-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)
>>> ~[jetty-server-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:223)
>>> ~[jetty-server-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1580)
>>> ~[jetty-server-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221)
>>> ~[jetty-server-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1384)
>>> ~[jetty-server-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:176)
>>> ~[jetty-server-10.0.19.jar:10.0.19]
>>> at
>>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:484)
>>> ~[jetty-servlet-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1553)
>>> ~[jetty-server-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:174)
>>> ~[jetty-server-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1306)
>>> ~[jetty-server-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129)
>>> ~[jetty-server-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:149)
>>> ~[jetty-server-10.0.
>>> 19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.server.handler.InetAccessHandler.handle(InetAccessHandler.java:228)
>>> ~[jetty-server-10.0.19.jar:10.0.19
>>> ]
>>> at
>>>
>> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:141)
>>> ~[jetty-server-10.0.19.jar:10.0.19
>>> ]
>>> at
>>>
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)
>>> ~[jetty-server-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:301)
>>> ~[jetty-rewrite-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)
>>> ~[jetty-server-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:822)
>>> ~[jetty-server-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)
>>> ~[jetty-server-10.0.19.jar:10.0.19]
>>> at org.eclipse.jetty.server.Server.handle(Server.java:563)
>>> ~[jetty-server-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.server.HttpChannel$RequestDispatchable.dispatch(HttpChannel.java:1598)
>>> ~[jetty-server-10.0.19.jar:10.0
>>> .19]
>>> at
>>> org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:753)
>>> ~[jetty-server-10.0.19.jar:10.0.19]
>>> at
>>> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:501)
>>> ~[jetty-server-10.0.19.jar:10.0.19]
>>> at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:461)
>>> ~[jetty-server-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:421)
>>> ~[jetty-uti
>>> l-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:390)
>>> ~[jetty
>>> -util-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:277)
>>> ~[jetty-
>>> util-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:193)
>>> ~[jetty-uti
>>> l-10.0.19.jar:10.0.19]
>>> at
>>> org.eclipse.jetty.http2.HTTP2Connection.produce(HTTP2Connection.java:208)
>>> ~[http2-common-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.http2.HTTP2Connection.onFillable(HTTP2Connection.java:155)
>>> ~[http2-common-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.http2.HTTP2Connection$FillableCallback.succeeded(HTTP2Connection.java:450)
>>> ~[http2-common-10.0.19.jar:
>>> 10.0.19]
>>> at
>>> org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)
>>> ~[jetty-io-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:558)
>>> ~[jetty-io-10.0.19.jar:10.0.
>>> 19]
>>> at
>>> org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:379)
>>> ~[jetty-io-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:146)
>>> ~[jetty-io-10.0.19.jar:10.0.19]
>>> at
>>> org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)
>>> ~[jetty-io-10.0.19.jar:10.0.19]
>>> at
>>> org.eclipse.jetty.io
>> .SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53)
>>> ~[jetty-io-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:421)
>>> ~[jetty-uti
>>> l-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:390)
>>> ~[jetty
>>> -util-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:277)
>>> ~[jetty-
>>> util-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:199)
>>> ~[jetty-util-10
>>> .0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:411)
>>> ~[jetty-util-10
>>> .0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
>>> ~[jetty-util-10.0.19.jar:10.0.19]
>>> at
>>>
>> org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
>>> ~[jetty-util-10.0.19.jar:10.0
>>> .19]
>>> at
>>>
>> org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
>>> ~[jetty-util-10.0.19.jar:10.0.19]
>>> at java.base/java.lang.Thread.run(Thread.java:834) [?:?]
>>> 2024-04-16 05:17:09.054 INFO
>>> (qtp1119242936-23-vmclxeissldi101.corp.equinix.com-50) [c:test1 s: r: x:
>>> t:vmclxeissldi101.corp.equin
>>> ix.com-50] o.a.s.s.HttpSolrCall [admin] webapp=null
>> path=/admin/collections
>>> params={replicationFactor=1&collection.configName=_defa
>>> ult&name=test1&router.name
>> =compositeId&action=CREATE&numShards=1&wt=json&_=1713244112747}
>>> status=400 QTime=405
>>> 2024-04-16 05:17:10.667 INFO
>>> (
>> OverseerCollectionConfigSetProcessor-72062388420345865-vmclxeissldi101.corp.equinix.com:8985
>>> _solr-n_
>>> 0000000008) [c: s: r: x: t:] o.a.s.c.OverseerTaskQueue Response ZK path:
>>> /overseer/collection-queue-work/qnr-0000000046 doesn't exi
>>> st. Requestor may have disconnected from ZooKeeper
>>>
>>> --
>>>
>>> Thanks and Regards,
>>> Hari
>>> Mobile:9790756568
>>
>>
>
> --
>
> Thanks and Regards,
> Hari
> Mobile:9790756568
