[
https://issues.apache.org/jira/browse/SOLR-17707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17936606#comment-17936606
]
David Smiley commented on SOLR-17707:
-------------------------------------
Definitely want to see a test for this, as I'll have little faith we've fixed
it without one.
> HttpJdkSolrClient threads stuck on exceptions
> ---------------------------------------------
>
> Key: SOLR-17707
> URL: https://issues.apache.org/jira/browse/SOLR-17707
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrJ
> Affects Versions: 9.6.1
> Reporter: Artem Abeleshev
> Priority: Major
> Labels: SolrJ, pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> There is a problem with _HttpJdkSolrClient_ used by {_}SolrJ{_}. In case of
> exception occurs on sending data (in my case it was {_}connection reset by
> peer{_}) the thread from the http client executor pool get stuck. Internally,
> _HttpJdkSolrClient_ using _PipedOutputStream_ and _PipedInputStream_ and in
> case of exception it ends up waiting forever. Seems like _PipedOutputStream_
> buffer remains full and _PipedInputStream_ stuck in a waiting state waiting
> for the avaialble space to write. This will consume one thread from the pool
> making it blocked. By default, _HttpJdkSolrClient_ created with an executor
> of a _4_ threads, if all of them are blocked the _SolrJ_ will be unable to
> work.
> There seems to be no problem with that if switching to use _Http2SolrClient_
> instead of _HttpJdkSolrClient_ as _Http2SolrClient_ doesn't use
> _PipedOutputStream_ and {_}PipedInputStream{_} facility.
> Here is a part of threaddump showing the http client threads get stuck:
> {code:java}
> "HttpJdkSolrClient-1-thread-1-processing-system
> ad571f96-762d-4e9a-8f9c-f5580413397d" prio=5 Id=52 TIMED_WAITING on
> java.io.PipedInputStream@67168d76
> at java.base@17.0.12/java.lang.Object.wait(Native Method)
> - waiting on java.io.PipedInputStream@67168d76
> at java.base@17.0.12/java.io.PipedInputStream.awaitSpace(Unknown Source)
> at java.base@17.0.12/java.io.PipedInputStream.receive(Unknown Source)
> at java.base@17.0.12/java.io.PipedOutputStream.write(Unknown Source)
> at
> org.apache.solr.common.util.FastOutputStream.flush(FastOutputStream.java:207)
> at
> org.apache.solr.common.util.FastOutputStream.flushBuffer(FastOutputStream.java:200)
> at
> org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:170)
> at
> org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:100)
> ...
> Number of locked synchronizers = 1
> - java.util.concurrent.ThreadPoolExecutor$Worker@bdec8d6
> "HttpJdkSolrClient-1-thread-2-processing-system
> ad571f96-762d-4e9a-8f9c-f5580413397d" prio=5 Id=53 TIMED_WAITING on
> java.io.PipedInputStream@571e7f87
> at java.base@17.0.12/java.lang.Object.wait(Native Method)
> - waiting on java.io.PipedInputStream@571e7f87
> at java.base@17.0.12/java.io.PipedInputStream.awaitSpace(Unknown Source)
> at java.base@17.0.12/java.io.PipedInputStream.receive(Unknown Source)
> at java.base@17.0.12/java.io.PipedOutputStream.write(Unknown Source)
> at
> org.apache.solr.common.util.FastOutputStream.flush(FastOutputStream.java:207)
> at
> org.apache.solr.common.util.FastOutputStream.flushBuffer(FastOutputStream.java:200)
> at
> org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:170)
> at
> org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:100)
> ...
> Number of locked synchronizers = 1
> - java.util.concurrent.ThreadPoolExecutor$Worker@7e6cf6f2
> "HttpJdkSolrClient-1-thread-3-processing-system
> ad571f96-762d-4e9a-8f9c-f5580413397d" prio=5 Id=54 TIMED_WAITING on
> java.io.PipedInputStream@497c4c19
> at java.base@17.0.12/java.lang.Object.wait(Native Method)
> - waiting on java.io.PipedInputStream@497c4c19
> at java.base@17.0.12/java.io.PipedInputStream.awaitSpace(Unknown Source)
> at java.base@17.0.12/java.io.PipedInputStream.receive(Unknown Source)
> at java.base@17.0.12/java.io.PipedOutputStream.write(Unknown Source)
> at
> org.apache.solr.common.util.FastOutputStream.flush(FastOutputStream.java:207)
> at
> org.apache.solr.common.util.FastOutputStream.flushBuffer(FastOutputStream.java:200)
> at
> org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:170)
> at
> org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:100)
> ...
> Number of locked synchronizers = 1
> - java.util.concurrent.ThreadPoolExecutor$Worker@7a2b5f30
> "HttpJdkSolrClient-1-thread-4-processing-system
> ad571f96-762d-4e9a-8f9c-f5580413397d" prio=5 Id=55 TIMED_WAITING on
> java.io.PipedInputStream@77907562
> at java.base@17.0.12/java.lang.Object.wait(Native Method)
> - waiting on java.io.PipedInputStream@77907562
> at java.base@17.0.12/java.io.PipedInputStream.awaitSpace(Unknown Source)
> at java.base@17.0.12/java.io.PipedInputStream.receive(Unknown Source)
> at java.base@17.0.12/java.io.PipedOutputStream.write(Unknown Source)
> at
> org.apache.solr.common.util.FastOutputStream.flush(FastOutputStream.java:207)
> at
> org.apache.solr.common.util.FastOutputStream.write(FastOutputStream.java:97)
> at
> org.apache.solr.common.util.JavaBinCodec.writeStr(JavaBinCodec.java:916)
> at
> org.apache.solr.common.util.JavaBinCodec.writePrimitive(JavaBinCodec.java:1062)
> ...
> Number of locked synchronizers = 1
> - java.util.concurrent.ThreadPoolExecutor$Worker@2fb4c889
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
