[
https://issues.apache.org/jira/browse/SOLR-7344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17356766#comment-17356766
]
Mark Robert Miller commented on SOLR-7344:
------------------------------------------
[~atris] i missed that comment. Taking advantage of the QosFilter is really the
best strategy. These other issues around thread pools don’t solve the problem
at all. Given that you only want as many threads as you have cores + what may
be needed while blocking or waiting (that can be reduced huge as well), current
thread usage is nowhere near optimal or good. It’s extremely costly, circuit
breaker approaches are better than nothing but equivalent to a hack compared to
what you actually want.
But yeah, I doubt that gets solved in Solr. But the largest issue is the
unlimited jetty pool. This can be improved on a few fronts and should be.
Beyond the obvious issues with the unlimited pool, jetty looks at the max
threads config to configure and control behavior. Sizing the reserve executor.
Deciding how many threads to keep spinning up in ensureThreads, etc. The
certainly don’t expect to be consulting Integer.MAX_VALUE.
There are all kinds of pitfalls with choosing a reasonable limit with Solr
though. With the QoSFilter you can work towards a nicer model than hard thread
limits anyway. You can dynamically limit the number of external concurrent
requests.
In the end, you don’t want to actually “dumbly” started limited resources. So
you mark internal requests with a header and limit external requests. That
keeps in flight requests moving and this is how you work towards achieving more
optimal throughout under load rather than death spirals or everyone pays
starvation and mass partial work done rejections.
External requests can be momentarily paused and then if necessary suspended -
holding onto 0 threads while they wait for availability. If that takes too
long, then you can reject with a proper response that keeps clients from retry
bombarding.
The goal should be high throughput and a consistent thread count not far above
your available cpu cores. It takes way more than the above to do that, but
circuit breakers and hard limits would never come anywhere near the optimal
situation. Just a more brute force, better than nothing approach. Jetty allows
for that with its own thread pool via its configurable low resource detection
sledgehammer.
> Allow Jetty thread pool limits while still avoiding distributed deadlock.
> -------------------------------------------------------------------------
>
> Key: SOLR-7344
> URL: https://issues.apache.org/jira/browse/SOLR-7344
> Project: Solr
> Issue Type: Improvement
> Components: SolrCloud
> Reporter: Mark Miller
> Priority: Major
> Attachments: SOLR-7344.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
