Re: ThreadLocal in SegmentReader

2008-07-11 Thread Roman Puchkovskiy
Well, this 'replacement' of the ThreadLocal does not solve the initial problem. As there's always at least one ThreadLocal which binds the object loaded by the web-app to the Thread which is _not_ loaded by the web-app, the classloader never may be unloaded. You are right, this is not the 'leak'

Re: how to get total hit count for each Searchable?

2008-07-11 Thread Erik Hatcher
On Jul 11, 2008, at 1:13 PM, xin liu wrote: I have individual index files for Audio, Image and PDF files. We build common meta fields for these different data types. When I search for a string, I want the search to return mixed search results from these 3 different index based on relevancy.

Re: Searching for instances within a document

2008-07-11 Thread Karl Wettin
11 jul 2008 kl. 15.28 skrev jnance: The TermFrequencyVector works perfectly for normal query strings. But if I add a wild card (*) onto words to search for different forms of the word I get an ArrayIndexOutOfBoundsException because the index is -1. Why does this happen? And is there anyw

how to get total hit count for each Searchable?

2008-07-11 Thread xin liu
Hi, I have individual index files for Audio, Image and PDF files. We build common meta fields for these different data types. When I search for a string, I want the search to return mixed search results from these 3 different index based on relevancy. So I use ParallelMultiSearcher class to do

Re: Searching for instances within a document

2008-07-11 Thread jnance
The TermFrequencyVector works perfectly for normal query strings. But if I add a wild card (*) onto words to search for different forms of the word I get an ArrayIndexOutOfBoundsException because the index is -1. Why does this happen? And is there anyway to avoid it? Thanks, James jnance wrot

Re: Best practice for updating an index when reindexing is not an option

2008-07-11 Thread Michael McCandless
OK, sounds good. Fall will be here before you know it! Mike Christopher Kolstad wrote: The only way to make this work with svn is if you can have svn perform a switch without doing any removal, then restart your IndexSearcher, then do a normal svn switch to remove the now unused files.

Re: Best practice for updating an index when reindexing is not an option

2008-07-11 Thread Christopher Kolstad
> > The only way to make this work with svn is if you can have svn perform a > switch without doing any removal, then restart your IndexSearcher, then do a > normal svn switch to remove the now unused files. Does svn have an option > to "switch but don't remove any removed files"? Because IndexSe

Re: ThreadLocal in SegmentReader

2008-07-11 Thread Michael McCandless
After discussing this on java-dev: http://mail-archives.apache.org/mod_mbox/lucene-java-dev/200807.mbox/[EMAIL PROTECTED] it seems that this is not in fact a leak but rather a delayed GC issue. The objects are eventually freed, on Sun 1.4, 1.5 and 1.6. When a ThreadLocal instance beco

Re: Best practice for updating an index when reindexing is not an option

2008-07-11 Thread Michael McCandless
OK, got it. The only way to make this work with svn is if you can have svn perform a switch without doing any removal, then restart your IndexSearcher, then do a normal svn switch to remove the now unused files. Does svn have an option to "switch but don't remove any removed files"? Bec

Re: Deletions

2008-07-11 Thread Michael McCandless
The deleted docs are actually stored separately, per segment, into files named _X_N.del, where X is the segment name and N is a generation count (keeps increasing by 1 every time new deletes are committed to that segment). Normal segment merging will also collapse the deletes in those s

Re: .fdt file

2008-07-11 Thread Michael McCandless
I think the FieldCache API fits for this... Its purpose is to do a one-time caching of a particular field's values, rendered as a straight array keyed by document ID. So building that cache initially takes time and memory, but then getting the value for a particular document is extremely

Re: Can we update a field on the current index

2008-07-11 Thread Michael McCandless
You're welcome! And, feel free to go vote for that issue ... Jira's voting system lets us gauge importance of features like this over time :) Mike Aditi Goyal wrote: Thanks Mike for your valuable time. Regards, Aditi On Thu, Jul 10, 2008 at 5:36 PM, Michael McCandless < [EMAIL PROTECTED

Re: Best practice for updating an index when reindexing is not an option

2008-07-11 Thread Christopher Kolstad
Hi. First, thanks for the reply. Why does SubversionUpdate require shutting down the IndexSearcher? What > goes wrong? > SubversionUpdate requires shutting down the IndexSearcher in our current implementation because the old index files are deleted in the tag we're switching to. Sorry, just rea