BitSet Filter ArrayIndexOutOfBoundsException?

2009-04-15 Thread Ryan McKinley
I am working on a Filter that uses an RTree to test for inclusion. This Filter works great *most* of the time -- if the index is optimized, it works all of the time. I feel like I am missing something basic, but not sure what it could be. Each time the reader opens (and the index has chan

Re: Lucene SnowBall unexpected behavior for some terms

2009-04-15 Thread Otis Gospodnetic
I seem to recall some Snowball-related incompatibilities around Lucene 2.4.0 release, but can't find the exact details at the moment... but this was documented somewhere (Wiki?) Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: oleg_gnatov

Re: Lucene SnowBall unexpected behavior for some terms

2009-04-15 Thread Grant Ingersoll
A quick search (http://www.lucidimagination.com/search/?q=snowball+back+compatibility ) yields that Snowball was updated in LUCENE-1142 (http://issues.apache.org/jira/browse/LUCENE-1142 ) and that we discussed a little bit about back compatibility with regards to contrib. In all likelihood, t

Re: BitSet Filter ArrayIndexOutOfBoundsException?

2009-04-15 Thread Michael McCandless
Maybe it's because you're using the MultiReader docID space but getDocIdSet(IndexReader) expects you to use the docID space for that IndexReader (ie, a single segment)? Mike On Wed, Apr 15, 2009 at 1:37 PM, Ryan McKinley wrote: > I am working on a Filter that uses an RTree to test for inclusion.

Re: BitSet Filter ArrayIndexOutOfBoundsException?

2009-04-15 Thread Ryan McKinley
Are you saying there lucene document could have different ids in the MultiReader and the IndexReader? I have assumed that the ids have not changed as long as the lastmodified time has not changed: long lastmodified = IndexReader.lastModified( reader.directory() ); Is this assumption correc

RE: BitSet Filter ArrayIndexOutOfBoundsException?

2009-04-15 Thread Uwe Schindler
Use the index reader given to getDocIdSet. The Ids are only valid for that index reader. This is new in Lucene 2.9: filters are executed against each segment of an index separately, so the docids of the MultiReader/DirectoryIndexReader are different to the local ones. - Uwe Schindler H.-H.-Mei

Re: BitSet Filter ArrayIndexOutOfBoundsException?

2009-04-15 Thread Ryan McKinley
uggg. So there is no longer a consistent docId I can use in a filter? I have an operation that is quite expensive that I am hoping to run only once for each time the index changes. Is the How would I get all the doc ids with a given (stored) field from a Reader? I am trying: TermDocs

Re: BitSet Filter ArrayIndexOutOfBoundsException?

2009-04-15 Thread Yonik Seeley
On Wed, Apr 15, 2009 at 8:35 PM, Ryan McKinley wrote: > uggg.  So there is no longer a consistent docId I can use in a filter? There are ways around this... I had to do something for Solr's external file field to get it to work because of these changes. We can continue this part of the discussio

Re: BitSet Filter ArrayIndexOutOfBoundsException?

2009-04-15 Thread Ryan McKinley
thanks -- I'll move this discussion to solr-user since I am now delving into SolrIndexReader... On Apr 15, 2009, at 9:06 PM, Yonik Seeley wrote: On Wed, Apr 15, 2009 at 8:35 PM, Ryan McKinley wrote: uggg. So there is no longer a consistent docId I can use in a filter? There are ways ar

RE: BitSet Filter ArrayIndexOutOfBoundsException?

2009-04-15 Thread Uwe Schindler
> >> How would I get all the doc ids with a given (stored) field from a > >> Reader? > >> I am trying: > >> > >> TermDocs td = reader.termDocs(); > > > > Relatively new in 2.9, you can pass null to enumerate over all non- > > deleted docs: > > TermDocs td = reader.termDocs(null); Or even faster: