Re: Multiple searcher blocked threads

2010-10-06 Thread Yannis Pavlidis
I would recommend you use NIOFSDirectory. We had similar issues and after we switched to NIOFSDirectory these issues disappeared (dramatically reduced). Also if you are using filters what has really helped us is to use cached filters (CachingWrapperFilters). Hope that helps. Yannis. On Oct 6

Multiple searcher blocked threads

2010-10-06 Thread Shailendra Mudgal
Hi Everyone, Recently we have migrated from lucene 2.2 to lucene 2.9.3. We are having some issues in search. During the load, searchers are getting hung up. When we took a process stack, we sound that there are around 450 threads are in the blocked state. Thread t...@885: (state = BLOCKED) - o

Re: How to find performance bottleneck

2010-10-06 Thread Sergey
I do some warmup queries on both machines. I also checked heap size, it is 1G on 32 bit and 4G on 64 bit machine. I also have SATA disk on 32 bit and SCSI on 64 bit. So I/O should not be a problem. Actually if I'm using MMapDirectory and doing the same query over and over again I would guess the

RE: Issue with sentence specific search

2010-10-06 Thread Sirish Vadala
Awesome! Thanks a lot Steven! This is exactly what I wanted. Hi Sirish, Have you looked at SpanQuery's yet?: http://lucene.apache.org/java/3_0_2/api/all/org/apache/lucene/search/spans/package-summary.html See also this Lucid Imagination blog post by Mark Miller: http://www.lucidimagination.c

RE: Issue with sentence specific search

2010-10-06 Thread Steven A Rowe
Hi Sirish, Have you looked at SpanQuery's yet?: http://lucene.apache.org/java/3_0_2/api/all/org/apache/lucene/search/spans/package-summary.html See also this Lucid Imagination blog post by Mark Miller: http://www.lucidimagination.com/blog/2009/07/18/the-spanquery/ One common technique, instead

RE: Issue with sentence specific search

2010-10-06 Thread Sirish Vadala
Hmmm... My mistake. In fact it is not a phrase search, but its a proximity search. My screen gives four options to the user: -All words, -Exact phrase, -At least one word, -Within proximity of xx words. In case of -All words and -At least one word, this is irrelevant an everything works fine.

RE: Numeric range query not returning results

2010-10-06 Thread Todd Nine
I've determined the problem. It's the same end bug as we experienced with the Cassandra encoding and the term enumeration not being properly returned. I've outlined the issues in this bug on Lucandra. http://github.com/tjake/Lucandra/issues/#issue/40 As you can see, the enumeration of the Lucan

RE: Issue with sentence specific search

2010-10-06 Thread Steven A Rowe
Hi Sirish, I think I understand "within sentence" phrase search - you want the entire phrase to be within a single sentence. But can you give an example of "non sentence specific phrase search"? It's not clear to me how useful such capability would be. Steve > -Original Message- > F

Issue with sentence specific search

2010-10-06 Thread Sirish Vadala
Hello All: Can any one suggest me the best way to implement both sentence specific and non sentence specific phrase search? The user is going to have a check box for phrase search on the screen that says 'within sentence'. If s/he selects 'within sentence', then I should perform sentence specifi

Re: How to find performance bottleneck

2010-10-06 Thread Erick Erickson
Also note that if you've fired some warmup queries at one machine but not the other that'll skew your results... Erick On Wed, Oct 6, 2010 at 7:22 AM, Sergey wrote: > Thanks, I'll give it a try. > > 06.10.10, 14:47, "Toke Eskildsen" : > > > On Wed, 2010-10-06 at 12:22 +0200, Sergey wrote: > >

Re: Merge and commit behaviour - changed between 2.4 and 2.9?

2010-10-06 Thread mark harwood
> OK. I'll double check the reports. The issue I'm tackling in this app is that multiple servers are applying equal-size volumes of updates in a distributed transaction and one means of node failure detection is based on detecting a server's failure to prepare a batch of updates in a timescal

Re: How to find performance bottleneck

2010-10-06 Thread Sergey
Thanks, I'll give it a try. 06.10.10, 14:47, "Toke Eskildsen" : > On Wed, 2010-10-06 at 12:22 +0200, Sergey wrote: > > When running application on Windows XP 32 bit machine the search time is > 0.5 second. JVM is IBM Java 5 for 32 bit. > > But when running the same application on much more pow

Re: How to find performance bottleneck

2010-10-06 Thread Toke Eskildsen
On Wed, 2010-10-06 at 12:22 +0200, Sergey wrote: > When running application on Windows XP 32 bit machine the search time is 0.5 > second. JVM is IBM Java 5 for 32 bit. > But when running the same application on much more powerfull Windows Server > 2007 64 bit machine the search time is 3 seconds.

Re: Restore documents marked as deleted

2010-10-06 Thread Simon Willnauer
On Wed, Oct 6, 2010 at 12:18 PM, Michael McCandless wrote: > There's no quick way to recover from this. > > You'd need to derive the segments_N file by opening the files that did > get written.  Conceptually this should be possible (there was a recent > thread about it), but I don't think anyone h

How to find performance bottleneck

2010-10-06 Thread Sergey
When running application on Windows XP 32 bit machine the search time is 0.5 second. JVM is IBM Java 5 for 32 bit. But when running the same application on much more powerfull Windows Server 2007 64 bit machine the search time is 3 seconds. JVM is IBM Java 5 for 64 bit. It just does not make any

Re: Restore documents marked as deleted

2010-10-06 Thread Michael McCandless
There's no quick way to recover from this. You'd need to derive the segments_N file by opening the files that did get written. Conceptually this should be possible (there was a recent thread about it), but I don't think anyone has actually created the tool to do so... Mike On Wed, Oct 6, 2010 a

Restore documents marked as deleted

2010-10-06 Thread Philippe Thomas
Hi, I was indexing some documents, but my program crashed after several days of work. If I reopen this index it is empty. I guess the reason is that auto-commit was not set and I never performed a commit. (Lesson learned) So probably all documents are marked as "deleted" and re-opening the i