[
https://issues.apache.org/jira/browse/SOLR-3544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295913#comment-13295913
]
Dušan Omerčević commented on SOLR-3544:
---------------------------------------
Ad 1) We're using Tomcat.
Ad 2) We fetch up to 40K documents with each document being a few KB in size.
Total response is some 100MB in size. The cut off really happened at arbitrary
position, from a few KB into the response up to several MB into the response.
Ad 3) I've checked it thoroughly and I couldn't find any pattern. It really
seems that the problem is independent of the data.
Ad 4) The full message is:
2012-06-11 12:23:34 ERROR: ClientAbortException: java.net.SocketException:
Broken pipe
at
org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:370)
at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:323)
at
org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:396)
at
org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:385)
at
org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:89)
at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:202)
at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:263)
at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:106)
at java.io.OutputStreamWriter.write(OutputStreamWriter.java:190)
at org.apache.solr.common.util.FastWriter.flush(FastWriter.java:113)
at
org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:344)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:265)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:873)
at
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
at
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
at
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at
org.apache.coyote.http11.InternalOutputBuffer.realWriteBytes(InternalOutputBuffer.java:750)
at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:432)
at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:347)
at
org.apache.coyote.http11.InternalOutputBuffer$OutputStreamOutputBuffer.doWrite(InternalOutputBuffer.java:773)
at
org.apache.coyote.http11.filters.IdentityOutputFilter.doWrite(IdentityOutputFilter.java:127)
at
org.apache.coyote.http11.InternalOutputBuffer.doWrite(InternalOutputBuffer.java:583)
at org.apache.coyote.Response.doWrite(Response.java:560)
at
org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:365)
... 25 more
2012-06-11 12:23:34 ERROR: Servlet.service() for servlet default threw exception
java.lang.IllegalStateException
at
org.apache.catalina.connector.ResponseFacade.sendError(ResponseFacade.java:405)
at
org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:380)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:283)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:873)
at
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
at
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
at
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
at java.lang.Thread.run(Thread.java:662)
But I believe that this is just a secondary effect of client closing the
connection. It seems that the client is closing the connection because he
doesn't have anything left to read. And it seems that it doesn't have anything
to read because server abruptly stops sending data. In catalina.out there were
no other error messages that would indicate why the server stopped sending data.
I hope this helps at least a bit. I didn't post this bug report because I
expected it to be resolved (it's quite weird indeed), but mostly as reference
point if somebody else is experiencing similar problems. But if it would get
resolved that would be even better, of course :)
Dušan
> Under heavy load json response is cut at some arbitrary position
> ----------------------------------------------------------------
>
> Key: SOLR-3544
> URL: https://issues.apache.org/jira/browse/SOLR-3544
> Project: Solr
> Issue Type: Bug
> Components: search
> Affects Versions: 3.1
> Environment: Linux version 2.6.32-5-amd64 (Debian 2.6.32-38)
> ([email protected]) (gcc version 4.3.5 (Debian 4.3.5-4) )
> Reporter: Dušan Omerčević
>
> We query solr for 30K documents using json as the response format. Normally
> this works perfectly fine. But when the machine comes under heavy load (all
> cores utilized) the response got interrupted at arbitrary position. We
> circumvented the problem by switching to xml response format.
> I've written the full description here:
> http://restreaming.wordpress.com/2012/06/14/the-curious-case-of-solr-malfunction/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
