RE: TopFieldDocCollector and v3.0.0

2009-12-07 Thread Uwe Schindler
That's a java docs issue and already fixed for the 3.0 branch. You have to use the no longer experimental API. Any by the way: Experimental means not "bad" or "functionally unstable", it just means that the API might change on later releases. - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Brem

Re: heap memory issues when sorting by a string field

2009-12-07 Thread Jason Rutherglen
TCK, CSIndexInput is returned by SegmentReader.getFieldCacheKey() If you think it's an issue, then it'd be good to open an issue and submit some code as a patch, maybe a test case showing the WHM isn't removing values like it's supposed to. Jason On Mon, Dec 7, 2009 at 10:45 PM, TCK wrote: > T

Re: heap memory issues when sorting by a string field

2009-12-07 Thread Jason Rutherglen
> It's an apache license - but you mentioned something about no third party > libraries. Is that a policy for Lucene? Pretty much... Though one can always submit a patch anyways. On Mon, Dec 7, 2009 at 4:57 PM, Tom Hill wrote: > Hey, that's a nice little Class! I hadn't see it before. But it so

Re: heap memory issues when sorting by a string field

2009-12-07 Thread TCK
Thanks for the feedback guys. The evidence I have collected does point to an issue either in the java WeakHashMap implementation or in Lucene's use of it. In particular, I used reflection to replace the WeakHashMap instances with my own dummy Map that does a no-op for the put operation, and althoug

TopFieldDocCollector and v3.0.0

2009-12-07 Thread Antony Bowesman
I'm on 2.3.2 and looking to move to 2.9.1 or 3.0.0 In 2.9.1 TopFieldDocCollector is "Deprecated. Please use TopFieldCollector instead." in 3.0.0 TopFieldCollector says NOTE: This API is experimental and might change in incompatible ways in the next release What is the suggested path for mig

Re: IndexWriter creates multiple .cfs files

2009-12-07 Thread Cool The Breezer
Thanks Jason for quick reply and suggestion. I am definitely going to change directory implementation policy to have one .cfs file instead of hundreds in one directory. regards, Ranjan - Original Message From: Jason Rutherglen To: java-user@lucene.apache.org Sent: Tue, December 8, 2

Re: IndexWriter creates multiple .cfs files

2009-12-07 Thread Jason Rutherglen
RB, That's expected behavior, each .cfs corresponds to all of a segment's files. You could write your own directory implementation that underneath writes to a single file. It's usually good to present what you're trying to accomplish (i.e. the why). Jason On Mon, Dec 7, 2009 at 10:25 PM, Cool Th

IndexWriter creates multiple .cfs files

2009-12-07 Thread Cool The Breezer
Hello Group, I am continuously updating an index and at the same time searcher also searches the index, which resulted in multiple .cfs files for each commit by IndexWriter. I am not sure whether this is an expected behavior or I need to merge each time after IndexWriter commit

Re: Lucene-3.0.0 web demo problem

2009-12-07 Thread brian li
Thanks. And now I know where to go if there are more issues :) On Tue, Dec 8, 2009 at 11:54 AM, Robert Muir wrote: > thanks for reporting this, i opened a jira issue at > https://issues.apache.org/jira/browse/LUCENE-2132 > > On Mon, Dec 7, 2009 at 7:58 PM, brian li wrote: > >> Hi, >> >> I was ch

Re: Lucene-3.0.0 web demo problem

2009-12-07 Thread Robert Muir
thanks for reporting this, i opened a jira issue at https://issues.apache.org/jira/browse/LUCENE-2132 On Mon, Dec 7, 2009 at 7:58 PM, brian li wrote: > Hi, > > I was checking out Lucene-3.0.0, and bumped into one little problem. > > I followed the steps in http://lucene.apache.org/java/3_0_0/dem

Lucene-3.0.0 web demo problem

2009-12-07 Thread brian li
Hi, I was checking out Lucene-3.0.0, and bumped into one little problem. I followed the steps in http://lucene.apache.org/java/3_0_0/demo3.html to run the basic web demo of lucene. Then I got an error message that in result.jsp QueryParser(String, Analyzer) was undefined. I went to the source cod

Re: heap memory issues when sorting by a string field

2009-12-07 Thread Tom Hill
Hey, that's a nice little Class! I hadn't see it before. But it sounds like the asynchronous cleanup might deal with the problem I mentioned above (but I haven't looked at the code yet). It's an apache license - but you mentioned something about no third party libraries. Is that a policy for Lucen

Re: heap memory issues when sorting by a string field

2009-12-07 Thread Jason Rutherglen
I wonder if Google Collections (even though we don't use third party libraries) concurrent map, which supports weak keys, handles the removal of weakly referenced keys in a more elegant way than Java's WeakHashMap? On Mon, Dec 7, 2009 at 4:38 PM, Tom Hill wrote: > Hi - > > If I understand correct

Re: heap memory issues when sorting by a string field

2009-12-07 Thread Tom Hill
Hi - If I understand correctly, WeakHashMap does not free the memory for the value (cached data) when the key is nulled, or even when the key is garbage collected. It requires one more step: a method on WeakHashMap must be called to allow it to release its hard reference to the cached data. It ap

Re: heap memory issues when sorting by a string field

2009-12-07 Thread TCK
Thanks for the response. But I'm definitely calling close() on the old reader and opening a new one (not using reopen). Also, to simplify the analysis, I did my test with a single-threaded requester to eliminate any concurrency issues. I'm doing: sSearcher.getIndexReader().close(); sSearcher.close

Re: heap memory issues when sorting by a string field

2009-12-07 Thread Erick Erickson
What this sounds like is that you're not really closing your readers even though you think you are. Sorting indeed uses up significant memory when it populates internal caches and keeps it around for later use (which is one of the reasons that warming queries matter). But if you really do close the

heap memory issues when sorting by a string field

2009-12-07 Thread TCK
Hi, I'm having heap memory issues when I do lucene queries involving sorting by a string field. Such queries seem to load a lot of data in to the heap. Moreover lucene seems to hold on to references to this data even after the index reader has been closed and a full GC has been run. Some of the con

Looking for a MappingCharFilter that accepts regular expressions

2009-12-07 Thread Paul Taylor
I want my search to treat 'No. 1' and 'No.1' the same, because in our context its one token I want 'No. 1' to become 'No.1', I need to do this before tokenizing because the tokenizer would split one value into two terms and one into just one term. I already use a NormalizeMapFilter to map &' t

Re: [ERR] Only a type can be imported

2009-12-07 Thread Ghazal Gharooni
Thank you! I added all the jar files but accessed them as an external jar files. Since I copied them all in WEB-INF/lib the problem has been solved. On Mon, Dec 7, 2009 at 1:58 AM, Ian Lea wrote: > > Using google to search for "Generated servlet error: Only a type can > be imported." leads to po

Re: english dictionary for spelling

2009-12-07 Thread Benjamin Heilbrunn
If you are searching for a dictionary this might be a good ressource for you: http://wiki.services.openoffice.org/wiki/Dictionaries 2009/12/7 m.harig : > > hello all > >      i've a doubt in spell checker , am creating spell index from my > original index , but my original index itself has some mi

Re: [ERR] Only a type can be imported

2009-12-07 Thread Ian Lea
Using google to search for "Generated servlet error: Only a type can be imported." leads to posts that say it is caused by your webapp not being able to find a jar. Double check that you've got the lucene-core-x.x.x.jar file in webapps/whatever/WEB-INF/lib, assuming that's the right place for what