Re: Continuously iterate over documents in index

2010-07-15 Thread Max Lynch
Erick, This is what I ended up doing. I initially avoided it because I was storing dates using Solr's date type which AFAIK aren't usable in Lucene, but I ended up using DateTools to store a lucene readable version that seems to work well. Thanks! On Wed, Jul 14, 2010 at 7:59 PM, Erick Erickson

Re: Continuously iterate over documents in index

2010-07-14 Thread Erick Erickson
H, if you somehow know the last date you processed, why wouldn't using a range query work for you? I.e. date:[ TO ]? Best Erick On Wed, Jul 14, 2010 at 10:37 AM, Max Lynch wrote: > You could have a field within each doc say "Processed" and store a > > > value Yes/No, next run a searcher que

Re: Continuously iterate over documents in index

2010-07-14 Thread Erick Erickson
Kiran: Please start a new thread when asking a new question. From Hossman's apache page: When starting a new discussion on a mailing list, please do not reply to an existing message, instead start a fresh email. Even if you change the subject line of your email, other mail headers still track whi

Re: Continuously iterate over documents in index

2010-07-14 Thread Kiran Kumar
All, Issue: Unable to get the proper results after searching. I added sample code which I used in the application. If I used *numHitPerPage* value as 1000 its giving expected results. ex: The expected results is 32 docs but showing 32 docs Instead If I use *numHitPerPage* as 2^32-1 its not giving

Re: Continuously iterate over documents in index

2010-07-14 Thread Max Lynch
You could have a field within each doc say "Processed" and store a > value Yes/No, next run a searcher query which should give you the > collection of unprocessed ones. > That sounds like a reasonable idea, and I just realized that I could have done that in a way specific to my application. Howe

Re: Continuously iterate over documents in index

2010-07-13 Thread Shashi Kant
On Tue, Jul 13, 2010 at 5:17 PM, Max Lynch wrote: > Hi, > I would like to continuously iterate over the documents in my lucene index > as the index is updated.  Kind of like a "stream" of documents.  Is there a > way I can achieve this? > > Would something like this be sufficient (untested): > >  

Continuously iterate over documents in index

2010-07-13 Thread Max Lynch
Hi, I would like to continuously iterate over the documents in my lucene index as the index is updated. Kind of like a "stream" of documents. Is there a way I can achieve this? Would something like this be sufficient (untested): int currentDocId = 0; while(true) { for(; currentDocId < r