Re: Lucene 4.0 PerFieldAnalyzerWrapper question

2012-09-25 Thread Chris Male
Mike, On Wed, Sep 26, 2012 at 1:05 PM, Mike O'Leary wrote: > Hi Chris, > So if I change my analyzer to inherit from AnalyzerWrapper, I need to > define a getWrappedAnalyzer function and a wrapComponents function. I think > getWrappedAnalyzer is straightforward, but I don't understand who is > ca

RE: Lucene 4.0 PerFieldAnalyzerWrapper question

2012-09-25 Thread Mike O'Leary
Hi Chris, So if I change my analyzer to inherit from AnalyzerWrapper, I need to define a getWrappedAnalyzer function and a wrapComponents function. I think getWrappedAnalyzer is straightforward, but I don't understand who is calling wrapComponents and for what purpose, so I don't know how to def

Re: Lucene 4.0 PerFieldAnalyzerWrapper question

2012-09-25 Thread Chris Male
Ah I see. The problem is that we don't really encourage wrapping of Analyzers. Your Analyzer wraps a PerFieldAnalyzerWrapper consequently it needs to extend AnalyzerWrapper, not Analyzer. AnalyzerWrapper handles the createComponents call and just requires you to give it the Analyzer(s) you've wr

RE: Lucene 4.0 PerFieldAnalyzerWrapper question

2012-09-25 Thread Mike O'Leary
Hi Chris, In a nutshell, my question is, what should I put in place of ??? to make this into a Lucene 4.0 analyzer? public class MyPerFieldAnalyzer extends Analyzer { PerFieldAnalyzerWrapper _analyzer; public MyPerFieldAnalyzer() { Map analyzerMap = new HashMap(); analyzerMap.put("I

Re: Lucene 4.0 PerFieldAnalyzerWrapper question

2012-09-25 Thread Chris Male
Hi Mike, I don't really understand what problem you're having. PerFieldAnalyzerWrapper, like all AnalyzerWrappers, uses Analyzer.PerFieldReuseStrategy which means it caches the TokenStreamComponents per field. The TokenStreamComponents cached are created by by retrieving the wrapped Analyzer thr

Lucene 4.0 PerFieldAnalyzerWrapper question

2012-09-25 Thread Mike O'Leary
I am updating an analyzer that uses a particular configuration of the PerFieldAnalyzerWrapper to work with Lucene 4.0. A few of the fields use a custom analyzer and StandardTokenizer and the other fields use the KeywordAnalyzer and KeywordTokenizer. The older version of the analyzer looks like

RE: Memory issues with Lucene deployment

2012-09-25 Thread Uwe Schindler
Hi, Without a full output of "free -h" we cannot say anything. But the total Linux memory use should always used by 100% on a good server otherwise it's useless (because full memory includes cache usage, too). I think, -Xmx may be too less for your Java deployment? We have no information about

Memory issues with Lucene deployment

2012-09-25 Thread Paul Taylor
Doing Lucene search within a jetty servlet container, the machine has 16gb of memory. Using 64bit JVM and Lucene 3.6 and files are memory mapped so I just allocate a max of 512mb to jetty itself, understanding that the remaining memory can be used to memory map lucene files. Monitoring total

Re: multireader.deleteDocuments transfer to IndexWriter.deleteDocuments ?

2012-09-25 Thread Ian Lea
So you've got a MultiReader over some number of indexes, and in order to delete stuff matched with that MultiReader you need an IndexWriter for the specific index that holds the selected term. In Lucene 4.0. Is that right? There are getSequentialSubReaders() and readerIndex(int docID) methods in