Aha, I'm starting to see what is happening here. So on the server side, a node hosting one of N replicas for a shard, and that collection is single-sharded, then no randomization or forwarding will ever take place. Before SOLR-12217 it would not happen when using SolrJ either, but after SOLR-12217, SolrJ will load balance between replicas when selecting a node to send request to.
So in this case the client is a .NET app and has no SolrJ. Is there any way whatsoever to solve this on the Solr side only? Only I can think of is to send all requests to a 3rd node in the cluster that does not have a core for the collection, then it will balance between the two :) Or create a new, empty collection on the node, which acts as a routing collection only to the target collection? Sounds like there should be a way to explicitly disable the "optimization" of always handling the request locally in single-shard collections, i.e. always try to balance unless shards.preference=local? Jan > 10. mar. 2021 kl. 19:06 skrev Chris Hostetter <hossman_luc...@fucit.org>: > > > : Ah, I missed "single shard" ... this looks relevant: > : https://issues.apache.org/jira/browse/SOLR-12217 > > That improvement still isn't going to impact Jan's situation where the > *client* isn't SolrJ ... as the description says: > >>> NOTE: This Jira doesn't cover the single-sharded collections cases when >>> not using the CloudSolrClient or Streaming Expressions (i.e. if you do >>> a non-streaming curl request to a random node in the cluster, the >>> shards.preference parameter is not considered in the case of single >>> shards collections). > > > : > : On Wed, Mar 10, 2021 at 12:43 PM Jan Høydahl <jan....@cominvent.com> wrote: > : > : > We have not set any shard.preference, and I also think preferLocal > : > defaults to false, i.e random > : > > : > Earlier we had 2 shares for the same collection (both existed on both > : > nodes) and then requests were distributed to both nodes. That’s why, when > : > we went to 1 shard, I was wondering if the “single-shard” code path > perhaps > : > never attempts to utilize replicas?? But have not looked in code yet. > : > > : > Guess next step is to setup a small local test cluster and see what > : > happens. > : > > : > Jan Høydahl > : > > : > > 10. mar. 2021 kl. 15:46 skrev Michael Gibney <mich...@michaelgibney.net > : > >: > : > > > : > > You say not "anything fancy" -- depending on how you define "fancy", if > : > you > : > > have an explicit `shards.preference` param, based on the version you're > : > > running (8.4) you might also take a look at > : > > https://issues.apache.org/jira/browse/SOLR-14471. (If SOLR-14471 is the > : > > problem, removing the explicit `shards.preference` param should restore > : > > default "shuffling" routing). > : > > > : > > I haven't dug too deep, but it looks like for 8.4 preferLocalShards > : > > actually defaults to false? I might be missing something though: > : > > > : > > https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.4.1/solr/solrj/src/java/org/apache/solr/client/solrj/routing/RequestReplicaListTransformerGenerator.java#L85 > : > > > : > > > : > > > : > >> On Wed, Mar 10, 2021 at 9:10 AM Houston Putman <houstonput...@gmail.com > : > > > : > >> wrote: > : > >> > : > >> I could be wrong, but i dont think preferLocalShards is the default in > : > >> multi-shard use cases. > : > >> > : > >>> On Wed, Mar 10, 2021 at 9:07 AM Mike Drob <md...@mdrob.com> wrote: > : > >>> > : > >>> I believe a server will always try to prefer local cores. Can you do > an > : > >>> experiment with 3 nodes, and send http queries to the node not hosting > : > >> any > : > >>> replicas? That should confirm the balanced distribution. > : > >>> > : > >>> If you have multiple shards, the receiving server will forward the > : > >> requests > : > >>> for shards it doesn’t have, but would still prefer local shards when > : > they > : > >>> are available. > : > >>> > : > >>> On Wed, Mar 10, 2021 at 8:00 AM Jan Høydahl <jan....@cominvent.com> > : > >> wrote: > : > >>> > : > >>>> Hi, > : > >>>> > : > >>>> A client has a SolrCloud 8.4 setup with two nodes, and one collection > : > >>> with > : > >>>> one shard and replicationFactor=2. > : > >>>> Of course we want search traffic to be evenly distributed between the > : > >> two > : > >>>> replicas. > : > >>>> The client is using plain HTTP requests, no SolrJ or anything fancy, > : > >> and > : > >>>> sends all requests to one of the two nodes. > : > >>>> I was expecting Solr to forward about 50% of those requests to the > : > >> other > : > >>>> replica, but it is serving them all locally. > : > >>>> > : > >>>> I know we can setup an LB in front or re-program the client to do > : > round > : > >>>> robin, but that is not my question. > : > >>>> Is the select-random-replica logic only active when we have a sharded > : > >>>> oollection, and not for a single-shard? > : > >>>> > : > >>>> Jan > : > >>> > : > >> > : > > : > > -Hoss > http://www.lucidworks.com/
