Re: Securing stored data using Lucene

2013-06-23 Thread Alon Muchnick
hi Rafaela, one option you can try and look at is to do the below : 1.add an additional "permissions" field to each of the document you are indexing , this field can contain a string representing a specific "user id" or a "user group/s" that will have permission to read this document. 2.when runn

Re: Handle expression in the index

2013-02-06 Thread Alon Muchnick
hi Nicolas , if i understand correctly what you are describing is that your tag field will contain Lucine queries syntax - one word = exact match , 2 words "xx yy" = phrase match , and so on . there is a search method called "Prospective search" which fits this situation . you can try and use th

Re: Lucene 4.0 WhitespaceAnalyzer problem

2013-01-15 Thread Alon Muchnick
hi Maxim , you need to reset the tokenStream before the while loop - tokenStream .reset () check out http://lucene.apache.org/core/4_0_0/core/org/apache/lucene/analysis/package-summary.html look under "invoking the analyzer" : "ts.reset(); // Resets this stream to the beginning. (Required)"

Re: extensive minor garbage collection when using RAMDirectory on Lucune 3.6.2

2013-01-09 Thread Alon Muchnick
attaching the second screen shot of live recorded objects . thanks again Alon On Wed, Jan 9, 2013 at 7:34 PM, Alon Muchnick wrote: > hi , > after upgrading to Lucune 3.6.2 i noticed there is an extensive minor > garbage collection operations. once or twice a second , and the

Fwd: extensive minor garbage collection when using RAMDirectory on Lucune 3.6.2

2013-01-09 Thread Alon Muchnick
hi , after upgrading to Lucune 3.6.2 i noticed there is an extensive minor garbage collection operations. once or twice a second , and the amount of memory being freed is about 600 MB each time for a load of 60 searches per second : 2013-01-09T18:57:24.350+0200: 174200.121: [GC [PSYoungGen: 630064

Re: potential memory leak when using RAMDirectory ,CloseableThreadLocal and a thread pool .

2013-01-04 Thread Alon Muchnick
o attached a screen shot). if you like i can upload the full heap dump to some FTP. thanks again . Alon On Thu, Jan 3, 2013 at 11:36 PM, Robert Muir wrote: > On Thu, Jan 3, 2013 at 12:16 PM, Alon Muchnick wrote: > > > value org.apache.lucene.index.TermInfos

Re: potential memory leak when using RAMDirectory ,CloseableThreadLocal and a thread pool .

2013-01-03 Thread Alon Muchnick
hi Mike , at the peak there are 500 live threads going through Lucune (not all of them at the same time , tomcat thread pool uses round robin ) ,regarding the Directory impl we are using RAMDirectory. the object that takes most of the heap is the "hardRefs" WeakHashMap class member in the Close

potential memory leak when using RAMDirectory ,CloseableThreadLocal and a thread pool .

2013-01-02 Thread Alon Muchnick
hi MIke, thank you very much for replying . i am not sure when the doClose() method of the SegementReader will be called , but i think you are correct regarding the SegmentReader still being live , we have an singleton object with a hash map of several RamDirectory indexes and their IndexReader,

potential memory leak when using RAMDirectory ,CloseableThreadLocal and a thread pool .

2013-01-02 Thread Alon Muchnick
Hello All , we are using Lucune 3.6.2 in our web application on tomcat 5.5 and recently we started testing our application on tomcat 7, unfortunately we seem to encounter a memory link in Lucune's CloseableThreadLocal class , any help with solving the below issue would be much appreciated. we ar