RE: field sorted searches with unbounded hit count

2011-06-30 Thread Tim Eck
searches with unbounded hit count On Fri, Jun 24, 2011 at 2:14 PM, Tim Eck wrote: > I'm currently using the "real-time" readers from IndexWriter.getReader() and never closing my IndexWriter. I was (perhaps wrongly) assuming that those readers can observe mutations that have occurred a

Re: field sorted searches with unbounded hit count

2011-06-26 Thread Michael McCandless
On Fri, Jun 24, 2011 at 2:14 PM, Tim Eck wrote: > I'm currently using the "real-time" readers from IndexWriter.getReader() and > never closing my IndexWriter. I was (perhaps wrongly) assuming that those > readers can observe mutations that have occurred after creating them. Actually, the NRT r

RE: field sorted searches with unbounded hit count

2011-06-24 Thread Tim Eck
Simon Willnauer [mailto:simon.willna...@googlemail.com] Sent: Thursday, June 23, 2011 10:15 PM To: java-user@lucene.apache.org Subject: Re: field sorted searches with unbounded hit count On Thu, Jun 23, 2011 at 10:41 PM, Tim Eck wrote: > Thanks for the idea Ian. I still need to think about

RE: field sorted searches with unbounded hit count

2011-06-23 Thread Toke Eskildsen
On Thu, 2011-06-23 at 22:41 +0200, Tim Eck wrote: > I don't want to accuse anyone of bad code but always preallocating a > potentially large array in org.apache.lucene.util.PriorityQueue seems > non-ideal for the search I want to run. The current implementation of IndexSearcher uses threaded s

Re: field sorted searches with unbounded hit count

2011-06-23 Thread Simon Willnauer
nal Message- > From: Ian Lea [mailto:ian@gmail.com] > Sent: Thursday, June 23, 2011 3:12 AM > To: java-user@lucene.apache.org > Subject: Re: field sorted searches with unbounded hit count > > One possibility would be to execute the search first just to get the > number of

RE: field sorted searches with unbounded hit count

2011-06-23 Thread Tim Eck
o dig into some more lucene code :-) FYI: TotalHitCountCollector looks like it was added in 3.1.0 -Original Message- From: Ian Lea [mailto:ian@gmail.com] Sent: Thursday, June 23, 2011 3:12 AM To: java-user@lucene.apache.org Subject: Re: field sorted searches with unbounded hit count

Re: field sorted searches with unbounded hit count

2011-06-23 Thread Ian Lea
One possibility would be to execute the search first just to get the number of hits - see TotalHitCountCollector in recent versions of lucene, not sure when it was added - and use the hit count from that as the max docs to return. The counting only search would typically be very quick, certainly m