Hi, I am running ab, Apache's HTTP server benchmarking tool to evaluate the query performance of a Solr/Lucene 3.1 instance.
The test index has 12 million documents. The search returns the first 10 rows of 8 stored fields that match a standard query (q=field:value). The box has 256G RAM and 32 cores, the VM is run with -Xms512M -Xmx100G Up to a ab concurrency level of 4 (-c4), everything works fine, above that I am getting timeout exceptions from ab and org.mortbay.jetty.EofException (Caused by: java.net.SocketException: Broken pipe) exceptions in the Solr log file. I ran the same test on a smaller box using Solr/Lucene 1.4.1 with a similar index (8 million docs) in the past and I got results up to a concurrency level of 512. Here are the details for the Solr/Lucene 3.1 run: AB setting: concurrency: 2 complete requests: 2000 [ab -n2000 -c2] works fine concurrency: 4 complete requests: 4000 [ab -n4000 -c4] works fine concurrency: 8 complete requests: 8000 [ab -n8000 -c8] apr_poll: The timeout specified has expired (70007) solr-log: May 4, 2011 10:24:27 AM org.apache.solr.common.SolrException log SEVERE: org.mortbay.jetty.EofException at org.mortbay.jetty.HttpGenerator.flush(HttpGenerator.java:791) at org.mortbay.jetty.AbstractGenerator$Output.flush(AbstractGenerator.java:569) at org.mortbay.jetty.HttpConnection$Output.flush(HttpConnection.java:1012) at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:278) at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:122) at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:212) at org.apache.solr.common.util.FastWriter.flush(FastWriter.java:115) at org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:344) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:265) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228) at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:451) Any explanations for this ? Do you recommend to switch to Tomcat instead of JETTY ? Thanks a lot, Johannes