: Is there any way whatsoever to solve this on the Solr side only? : : Only I can think of is to send all requests to a 3rd node in the cluster : that does not have a core for the collection, then it will balance : between the two :)
correct -- you can create a Solr node w/o any cores that will act as a "load balancer" to other solr nodes. : Or create a new, empty collection on the node, which acts as a routing : collection only to the target collection? no -- this won't work, because the requerst your remote client sends will need to specify the actual collection you want to query, and when the node gets this it will hand it to the local core for that collection -- it won't care that there is another local collection that's unrelated. : Sounds like there should be a way to explicitly disable the : "optimization" of always handling the request locally in single-shard : collections, i.e. always try to balance unless shards.preference=local? that seems... dangerous. you could easily wind up in a situation where nodes just keep trying to forward forever? : : Jan : : > 10. mar. 2021 kl. 19:06 skrev Chris Hostetter <hossman_luc...@fucit.org>: : > : > : > : Ah, I missed "single shard" ... this looks relevant: : > : https://issues.apache.org/jira/browse/SOLR-12217 : > : > That improvement still isn't going to impact Jan's situation where the : > *client* isn't SolrJ ... as the description says: : > : >>> NOTE: This Jira doesn't cover the single-sharded collections cases when : >>> not using the CloudSolrClient or Streaming Expressions (i.e. if you do : >>> a non-streaming curl request to a random node in the cluster, the : >>> shards.preference parameter is not considered in the case of single : >>> shards collections). : > : > : > : : > : On Wed, Mar 10, 2021 at 12:43 PM Jan Høydahl <jan....@cominvent.com> wrote: : > : : > : > We have not set any shard.preference, and I also think preferLocal : > : > defaults to false, i.e random : > : > : > : > Earlier we had 2 shares for the same collection (both existed on both : > : > nodes) and then requests were distributed to both nodes. That’s why, when : > : > we went to 1 shard, I was wondering if the “single-shard” code path perhaps : > : > never attempts to utilize replicas?? But have not looked in code yet. : > : > : > : > Guess next step is to setup a small local test cluster and see what : > : > happens. : > : > : > : > Jan Høydahl : > : > : > : > > 10. mar. 2021 kl. 15:46 skrev Michael Gibney <mich...@michaelgibney.net : > : > >: : > : > > : > : > > You say not "anything fancy" -- depending on how you define "fancy", if : > : > you : > : > > have an explicit `shards.preference` param, based on the version you're : > : > > running (8.4) you might also take a look at : > : > > https://issues.apache.org/jira/browse/SOLR-14471. (If SOLR-14471 is the : > : > > problem, removing the explicit `shards.preference` param should restore : > : > > default "shuffling" routing). : > : > > : > : > > I haven't dug too deep, but it looks like for 8.4 preferLocalShards : > : > > actually defaults to false? I might be missing something though: : > : > > : > : > https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.4.1/solr/solrj/src/java/org/apache/solr/client/solrj/routing/RequestReplicaListTransformerGenerator.java#L85 : > : > > : > : > > : > : > > : > : > >> On Wed, Mar 10, 2021 at 9:10 AM Houston Putman <houstonput...@gmail.com : > : > > : > : > >> wrote: : > : > >> : > : > >> I could be wrong, but i dont think preferLocalShards is the default in : > : > >> multi-shard use cases. : > : > >> : > : > >>> On Wed, Mar 10, 2021 at 9:07 AM Mike Drob <md...@mdrob.com> wrote: : > : > >>> : > : > >>> I believe a server will always try to prefer local cores. Can you do an : > : > >>> experiment with 3 nodes, and send http queries to the node not hosting : > : > >> any : > : > >>> replicas? That should confirm the balanced distribution. : > : > >>> : > : > >>> If you have multiple shards, the receiving server will forward the : > : > >> requests : > : > >>> for shards it doesn’t have, but would still prefer local shards when : > : > they : > : > >>> are available. : > : > >>> : > : > >>> On Wed, Mar 10, 2021 at 8:00 AM Jan Høydahl <jan....@cominvent.com> : > : > >> wrote: : > : > >>> : > : > >>>> Hi, : > : > >>>> : > : > >>>> A client has a SolrCloud 8.4 setup with two nodes, and one collection : > : > >>> with : > : > >>>> one shard and replicationFactor=2. : > : > >>>> Of course we want search traffic to be evenly distributed between the : > : > >> two : > : > >>>> replicas. : > : > >>>> The client is using plain HTTP requests, no SolrJ or anything fancy, : > : > >> and : > : > >>>> sends all requests to one of the two nodes. : > : > >>>> I was expecting Solr to forward about 50% of those requests to the : > : > >> other : > : > >>>> replica, but it is serving them all locally. : > : > >>>> : > : > >>>> I know we can setup an LB in front or re-program the client to do : > : > round : > : > >>>> robin, but that is not my question. : > : > >>>> Is the select-random-replica logic only active when we have a sharded : > : > >>>> oollection, and not for a single-shard? : > : > >>>> : > : > >>>> Jan : > : > >>> : > : > >> : > : > : > : : > : > -Hoss : > http://www.lucidworks.com/ : : -Hoss http://www.lucidworks.com/
