RE: ParallelReader

2011-02-21 Thread Uwe Schindler
Hi David, With current Lucene versions, the usage of ParallelReader is very complicated to keep in sync. The problem is how merges occur. For ParallelReader to work, all internal document ids (the integers) must be parallel. As the new MergePolicies now work on size of documents and also may work

Re: ParallelReader

2010-10-14 Thread Rob Bygrave
>> Any case where it would break? If a query uses multiple fields it would break. That is, usually all the fields need to be in doc in index 2 - not just the modified one. On Fri, Oct 15, 2010 at 2:35 PM, Erick Erickson wrote: > This seems like far too much work if I'm reading things right. You

Re: ParallelReader

2010-10-14 Thread Erick Erickson
This seems like far too much work if I'm reading things right. You can't update a field, but you #can# update a document which actually re-index that document under the covers (you have to have a way to uniquely identify the doc). Then, when you reopen your index reader, you'll only see the new val

Re: ParallelReader

2010-10-14 Thread Nilesh Vijaywargiay
Hey Erick, Sure. * * *What I am trying to achieve:* A) Update a field in Index A B) When searching for that old field, it should be a miss. *How I achieved it* *Index 1 * Doc 1 - Field1, Value 1 Doc 2 - Field1, Value 1 *Index 2* Doc 1 - Field1, Modified_Value 1 Doc 2 - EMPTY Add index 2 before

Re: ParallelReader

2010-10-14 Thread Erick Erickson
No. And you don't even want to try... Document IDs are NOT invariant. Particularly when you delete a document and optimize an index, all the documents that come after the deleted one get new doc IDs. Trying to keep these two indexes in synch will be a nightmare. Perhaps you could explain what you'

Re: ParallelReader fails on Deletion

2006-08-27 Thread neils
Hi Erik, i made some several test as follows: Create new indicies with only on document with one Value ("ID"=100). Open it with parallelreader. Delete Document with parallelreader.delete(new Term("ID","100")) Close Reader with parallelreader.close. After this i open the index with luke and coul

Re: ParallelReader fails on Deletion

2006-08-27 Thread Erick Erickson
The first place I'd look is whether you're closing your *searcher* between times in both cases. If that leads nowhere, I'd make sure I had a copy of Luke so I could reassure myself that the document was actually in both cases. After that, I'd think about posting a fragment of both your deletion

Re: ParallelReader

2005-10-10 Thread Erik Hatcher
The use case is when there is some data that changes frequently, but some data is static, _and_ that the volatile index can be rebuilt in the same order that the static one was built. The indexes must be "parallel" in terms of the document index order. If you delete, then you should delet

Re: ParallelReader

2005-10-10 Thread John Smith
Sorry to bug people on this again and again. I might be missing something or confused totally, But what is the use case for a ParallelReader if the use case is not addressing the situation where we have a index changing frequently( meaning deletes and reindex) and index not changing , but has s

Re: ParallelReader

2005-10-10 Thread John Smith
A while ago I had asked a question on what would be a good solution for a situation mentioned below and I was pointed in the direction of Parallel Reader. Looks like that will not work. Thank you for alerting me on this. So other than delete and reindex the document to a single index, there is

Re: ParallelReader

2005-10-10 Thread Daniel Naber
On Montag 10 Oktober 2005 20:24, John Smith wrote: > My understanding is ParallelReader works for situations where you have a > static index and a dynamic index. That's no correct. Quoting the documentation: It is up to you to make sure all indexes are created and modified the same way. For exam

Re: ParallelReader and Date Filter

2005-09-12 Thread John Smith
Thank you. I will try that JS Erik Hatcher <[EMAIL PROTECTED]> wrote: On Sep 12, 2005, at 2:04 PM, John Smith wrote: > I have Lucene 1.4.3 codebase and I got Parallel Reader from the > trunk along with a few changes that need to go on top of it to make > it compile. I highly recommend you si

Re: ParallelReader and Date Filter

2005-09-12 Thread Erik Hatcher
On Sep 12, 2005, at 2:04 PM, John Smith wrote: I have Lucene 1.4.3 codebase and I got Parallel Reader from the trunk along with a few changes that need to go on top of it to make it compile. I highly recommend you simply compile the trunk and use it instead of trying to patch these classe