BitSet in Filters

2014-08-11 Thread Sandeep Khanzode
Hi,   The current usage of BitSets in filters in Lucene is limited to applying only on docIDs i.e. I can only construct a filter out of a BitSet if I have the DocumentIDs handy. However, with every update/delete i.e. CRUD modification, these will change, and I have to again redo the whole proce

Re: Problem of calling indexWriterConfig.clone()

2014-08-11 Thread Vitaly Funstein
I honestly don't understand what DWPT pool has to do with IndexWriterConfig instances not being reusable for new IndexWriter instances. If you have the need to open a new IndexWriter with the same configuration as the one you used before, why not save the original config as the "template", then sim

Re: Problem of calling indexWriterConfig.clone()

2014-08-11 Thread Sheng
>From src code of DocumentsWriterPerThreadPool, the variable numThreadStatesActive seems to be always increasing, which explains why asserting on numThreadStatesActive == 0 before cloning this object fails. So what should be the most appropriate way of re-opening an indexwriter if what you have are

Re: Problem of calling indexWriterConfig.clone()

2014-08-11 Thread Vitaly Funstein
I only have the source to 4.6.1, but if you look at the constructor of IndexWriter there, it looks like this: public IndexWriter(Directory d, IndexWriterConfig conf) throws IOException { conf.setIndexWriter(this); // prevent reuse by other instances The setter throws an exception if the con

Re: Problem of calling indexWriterConfig.clone()

2014-08-11 Thread Sheng
So the indexWriterConfig.clone() failed at this step: clone.indexerThreadPool = indexerThreadPool .clone

Re: Problem of calling indexWriterConfig.clone()

2014-08-11 Thread Vitaly Funstein
Looks like you have to clone it prior to using with any IndexWriter instances. On Mon, Aug 11, 2014 at 2:49 PM, Sheng wrote: > I tried to create a clone of indexwriteconfig with > "indexWriterConfig.clone()" for re-creating a new indexwriter, but I then I > got this very annoying illegalstateex

Re: Can't get case insensitive keyword analyzer to work

2014-08-11 Thread Milind
I found the problem. But it makes no sense to me. If I set the field type to be tokenized, it works. But if I set it to not be tokenized the search fails. i.e. I have to pass in true to the method. theFieldType.setTokenized(storeTokenized); I want the field to be stored as un-tokenized. B

Re: escaping characters

2014-08-11 Thread Jack Krupansky
You need to manually enable automatic generation of phrase queries - it defaults to disabled, which simply treats the sub-terms as individual terms subject to the default operator. See: http://lucene.apache.org/core/4_9_0/queryparser/org/apache/lucene/queryparser/classic/QueryParserBase.html#se

Problem of calling indexWriterConfig.clone()

2014-08-11 Thread Sheng
I tried to create a clone of indexwriteconfig with "indexWriterConfig.clone()" for re-creating a new indexwriter, but I then I got this very annoying illegalstateexception: "clone this object before it is used". Why does this exception happen, and how can I get around it? Thanks!

Re: Can't get case insensitive keyword analyzer to work

2014-08-11 Thread Milind
It does look like the lowercase is working. The following code Document theDoc = theIndexReader.document(0); System.out.println(theDoc.get("sn")); IndexableField theField = theDoc.getField("sn"); TokenStream theTokenStream = theField.tokenStream(theAnalyzer);

RE: escaping characters

2014-08-11 Thread Chris Salem
I'm not using Solr. Here's my code: FSDirectory fsd = FSDirectory.open(new File("C:\\indexes\\Lucene4")); IndexReader reader = DirectoryReader.open(fsd); IndexSearcher searcher = new IndexSearcher(reader); Analyzer analyzer = new StandardAnalyzer(Version.LUCENE_4_9, getS

Re: escaping characters

2014-08-11 Thread Erick Erickson
Take a look at the adnim/analysis page for the field in question. The next bit of critical information is adding & debug=query to the URL. The former will tell you what happens to the input stream at query and index time, the latter will tell you how the query got through the query parsing process.

escaping characters

2014-08-11 Thread Chris Salem
Hi everyone, I'm trying to escape special characters and it doesn't seem to be working. If I do a search like resume_text: (LS\/MS) it searches for LS AND MS instead of LS/MS. How would I escape the slash so it searches for LS/MS? Thanks

Re: Is housekeeping of Lucene indexes block index update but allow search ?

2014-08-11 Thread Gaurav gupta
Kumaran, Below is the code snippet for concurrent writes (i.e. concurrent updates/deletes etc.) alongwith Search operation using the NRT Manger APIs. Let me know if you need any other details or have any suggesstion for me :- public class LuceneEngineInstance implements IndexEngineInstance { p