[
https://issues.apache.org/jira/browse/SOLR-18433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris M. Hostetter updated SOLR-18433:
--------------------------------------
Description:
I'm creating this Jira to serve as a "parent" issue for a few loosely collected
concerns I have regarding how CloudSolrClient decides which node to send
requests to when they are not collection specific.
The broader context of why these concerns ... concern me ... is in [the
recommendations we
make|https://solr.apache.org/guide/solr/latest/upgrade-notes/major-changes-in-solr-10.html#solrcloud-overseer]
for people who want to use {{SOLR_CLOUD_OVERSEER_ENABLED=false}} (emphasis
added)...
{quote}Creating a replica (either via collection creation or other
circumstances) can take more time without the Overseer if these creation
commands are delivered to many nodes around the cluster. *That can be avoided
simply by sending admin requests to a consistent node.*
{quote}
If we're recommending that people send admin commands to a consistent node,
then that should be possible (and easy) using {{CloudSolrClient}}
But in looking into this, other concerns popped up.
----
For reference, here is the psuedo-code/flow for
{{CloudSolrClient.sendRequest(request)}} as it currently exists on main...
{noformat}
if (request.getType == UPDATE) {
if (we can send direct to leaders) {{
// NOTE: not focusing on this code path in this jira
return directUpdate(request)
}
}
List<Endpoint> endpoints = new List()
if (request is V2) {
endpoints.add(pickOneAtRandom(liveNodes)
} else if (! request.requiresCollection() )
// implementation detail: liveNodes are a SortedSet in "natural" order of name
endpoints.add(liveNodes)
} else {
collectionNames = ...
if (! request.getPreferredNodes().isEmpty()) {
endpoints.add(
request.getPreferredNodes().stream()
.mapToReplicasOf(collectionNames)
return getLbClient().request(request, endpoints)
}
endpoints.add(
collectionNames
.getAllReplicas()
.sortAndGroupBy(replicaListTransformer))
}
return getLbClient().request(request, endpoints)
}
{noformat}
was:
I'm creating this Jira to serve as a "parent" issue for a few loosely collected
concerns I have regarding how CloudSolrClient decides which node to send
requests to when they are not collection specific.
The broader context of why these concerns ... concern me ... is in [the
recommendations we
make|https://solr.apache.org/guide/solr/latest/upgrade-notes/major-changes-in-solr-10.html#solrcloud-overseer]
for people who want to use {{SOLR_CLOUD_OVERSEER_ENABLED=false}} (emphasis
added)...
{quote}Creating a replica (either via collection creation or other
circumstances) can take more time without the Overseer if these creation
commands are delivered to many nodes around the cluster. *That can be avoided
simply by sending admin requests to a consistent node.*
{quote}
If we're recommending that people send admin commands to a consistent node,
then that should be possible (and easy) using {{CloudSolrClient}}
But in looking into this, other concerns popped up.
----
For reference, here is the psuedo-code/flow for
{{CloudSolrClient.sendRequest(request)}} as it currently exists on main...
{noformat}
if (request.getType == UPDATE) {
if (we can send direct to leaders) {{
// NOTE: not focusing on this code path in this jira
return directUpdate(request)
}
}
List<Endpoint> endpoints = new List()
if (request is V2) {
endpoints.add(pickOneAtRandom(liveNodes)
} else if (! request.requiresCollection() )
endpoints.add(liveNodes)
} else {
collectionNames = ...
if (! request.getPreferredNodes().isEmpty()) {
endpoints.add(
request.getPreferredNodes().stream()
.mapToReplicasOf(collectionNames)
return getLbClient().request(request, endpoints)
}
endpoints.add(
collectionNames
.getAllReplicas()
.sortAndGroupBy(replicaListTransformer))
}
return getLbClient().request(request, endpoints)
}
{noformat}
> CloudSolrClient inconsistencies in routing non-collection requests
> ------------------------------------------------------------------
>
> Key: SOLR-18433
> URL: https://issues.apache.org/jira/browse/SOLR-18433
> Project: Solr
> Issue Type: Improvement
> Reporter: Chris M. Hostetter
> Priority: Major
>
> I'm creating this Jira to serve as a "parent" issue for a few loosely
> collected concerns I have regarding how CloudSolrClient decides which node to
> send requests to when they are not collection specific.
>
> The broader context of why these concerns ... concern me ... is in [the
> recommendations we
> make|https://solr.apache.org/guide/solr/latest/upgrade-notes/major-changes-in-solr-10.html#solrcloud-overseer]
> for people who want to use {{SOLR_CLOUD_OVERSEER_ENABLED=false}} (emphasis
> added)...
> {quote}Creating a replica (either via collection creation or other
> circumstances) can take more time without the Overseer if these creation
> commands are delivered to many nodes around the cluster. *That can be avoided
> simply by sending admin requests to a consistent node.*
> {quote}
> If we're recommending that people send admin commands to a consistent node,
> then that should be possible (and easy) using {{CloudSolrClient}}
> But in looking into this, other concerns popped up.
> ----
> For reference, here is the psuedo-code/flow for
> {{CloudSolrClient.sendRequest(request)}} as it currently exists on main...
> {noformat}
> if (request.getType == UPDATE) {
> if (we can send direct to leaders) {{
> // NOTE: not focusing on this code path in this jira
> return directUpdate(request)
> }
> }
> List<Endpoint> endpoints = new List()
> if (request is V2) {
> endpoints.add(pickOneAtRandom(liveNodes)
> } else if (! request.requiresCollection() )
> // implementation detail: liveNodes are a SortedSet in "natural" order of
> name
> endpoints.add(liveNodes)
> } else {
> collectionNames = ...
> if (! request.getPreferredNodes().isEmpty()) {
> endpoints.add(
> request.getPreferredNodes().stream()
> .mapToReplicasOf(collectionNames)
> return getLbClient().request(request, endpoints)
> }
> endpoints.add(
> collectionNames
> .getAllReplicas()
> .sortAndGroupBy(replicaListTransformer))
> }
> return getLbClient().request(request, endpoints)
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
