Re: CheckIndex: pos -1 is out of bounds

2008-03-29 Thread Chris Hostetter
: Hey guys, a Solr user just encountered an interesting situation... sorry, i forgot to paste the refrence... http://www.nabble.com/WordDelimiterFilter%2BLenghtFilter-results-in-termPosition%3D%3D-1-to16306788.html -Hoss -

CheckIndex: pos -1 is out of bounds

2008-03-29 Thread Chris Hostetter
Hey guys, a Solr user just encountered an interesting situation... ...due to a naive "LengthFilter", an Analyzer is produce a TokenStream where the first Token has a positionIncrement of "0" which seems to produce this error from CheckIndex... WARNING: would remove reference to this segm

Re: setPositionIncrement questions

2008-03-29 Thread Chris Hostetter
: Breaking proximity data has been discussed several times before, and : concluded that setPositionIncrement is the way to go. In regards of it: : : 1. Where should it be called exactly to create the gap properly? any part of your Analyzer can set the position increment on any token to indicat

Re: is it possible to change the way score from different field combine to give final lucene score

2008-03-29 Thread Grant Ingersoll
http://lucene.apache.org/java/2_3_1/api/org/apache/lucene/search/function/package-summary.html Sorry, they are called FunctionQuery in Solr, but Lucene calls 'em FieldScoreQuery, etc. Sounds like CustomScoreQuery may do the trick. HTH -Grant On Mar 26, 2008, at 4:34 AM, John Wang wrote: H

Re: Lucene 2.3.1 IndexReader and IndexWriter Synchronization Issues

2008-03-29 Thread Jamie
H Michael Well spotted! Thanks for the tip and your precious time. Thanks! Jamie Michael McCandless wrote: OK indeed from your code you are manually removing files from the index (the "deleteIndex" method). You shouldn't do that. I think what's happening is deleteIndex is called when an In

Re: Lucene 2.3.1 IndexReader and IndexWriter Synchronization Issues

2008-03-29 Thread Michael McCandless
OK indeed from your code you are manually removing files from the index (the "deleteIndex" method). You shouldn't do that. I think what's happening is deleteIndex is called when an IndexReader has the index open (do you see those "failed to delete file in existing index" log messages?), then you

Re: Lucene 2.3.1 IndexReader and IndexWriter Synchronization Issues

2008-03-29 Thread Jamie
By the way, the line numbers in the last email apply to the SVN trunk code. I switch over to the latest code to see if this would make a difference. Alas, the problem still exists! Jamie wrote: Hi Michael Glad you asked and thanks in advance for your help! I am trying to reindex 325 emails.

Re: Lucene 2.3.1 IndexReader and IndexWriter Synchronization Issues

2008-03-29 Thread Jamie
Hi Michael Glad you asked and thanks in advance for your help! I am trying to reindex 325 emails. Its a pretty straight forward use of Lucene. I create an IndexWriter, write a whole bunch of documents and then close the Index every 2 seconds. See the attached source code. In case, you are won

Re: Lucene 2.3.1 IndexReader and IndexWriter Synchronization Issues

2008-03-29 Thread Michael McCandless
That use case of Lucene should be fine, ie no further synchronization should be necessary. Your debug output below is great, but it doesn't seem to cover the occurrence of that exception. Can you post the full debug output? Can you describe how you reindex in more detail? Are you manual