Re: background merge hit exception

2011-09-09 Thread Koji Sekiguchi
Also: what java version are they running? We added this check originally as a workaround for a JRE bug... but usually when that bug strikes the file size is very close (like off by just 1 byte or 8 bytes or something). I think they are using 1.6, but I should ask the minor number. Could you show

Re: background merge hit exception

2011-09-09 Thread Koji Sekiguchi
Hi Mike, This wouldn't be caused by the "NFS happily deletes open files" problem (= Stale NFS file handle error). But this could in theory be caused by the NFS client somehow being wrong about the file's metadata (file length). It's sort of odd because I would expect since the client wrote the

Re: Search highlighter for custom Query implementations - how to?

2011-09-09 Thread Michael Sokolov
Lukas there really isn't any support for custom Query types in Highlighter, as you've found. If you inherit from one of the types it does support, or rewrite your query to one of them, that should work, but the Query class just doesn't provide enough support for Highlighter to work with in the

Re: optimize with num segments > 1 index keeps growing

2011-09-09 Thread Robert Muir
On Fri, Sep 9, 2011 at 3:07 PM, Uwe Schindler wrote: > Hi, > > This is still some kind of bug, because expungeDeletes is documented to > remove all deletes. Maybe we need to modify MergePolicy? > we should correct the javadocs for expungeDeletes here I think: so that its more consistent with the

Re: optimize with num segments > 1 index keeps growing

2011-09-09 Thread Uwe Schindler
Hi, This is still some kind of bug, because expungeDeletes is documented to remove all deletes. Maybe we need to modify MergePolicy? Uwe -- Uwe Schindler H.-H.-Meier-Allee 63, 28213 Bremen http://www.thetaphi.de Michael McCandless schrieb: TieredMergePolicy by default will only merge a segm

Re: optimize with num segments > 1 index keeps growing

2011-09-09 Thread Michael McCandless
TieredMergePolicy by default will only merge a segment if it has > 10% deletions. Can you try calling .setExpungeDeletesPctAllowed(0.0) and then expunge again? Mike McCandless http://blog.mikemccandless.com On Fri, Sep 9, 2011 at 1:41 PM, wrote: > Hi, > > this post is quite old, but I would l

RE: optimize with num segments > 1 index keeps growing

2011-09-09 Thread v . sevel
Hi, this post is quite old, but I would like to share some recen developments. I applied the recommandation. my process became: expunge deletes and optimize 2 segments. at the time I was with lucene 3.1 and that solved my issue. recently I moved to lucene 3.3, and I tried playing with the new

Re: Lucene killing JVM

2011-09-09 Thread Robert Muir
Why not? its not our bug, its Oracle's bug. Personally I would us a more stable JRE version rather than one so new, like 1.6.0_18 or 1.6.0_19 On Fri, Sep 9, 2011 at 6:05 AM, Dragan Jotanovic wrote: > Looks like the problem is somewhere in MMapFSDirectory. As soon as I > switched to NIOFSDirector

Re: background merge hit exception

2011-09-09 Thread Michael McCandless
Interesting... This wouldn't be caused by the "NFS happily deletes open files" problem (= Stale NFS file handle error). But this could in theory be caused by the NFS client somehow being wrong about the file's metadata (file length). It's sort of odd because I would expect since the client wrote

background merge hit exception

2011-09-09 Thread Koji Sekiguchi
A user here hit the exception the title says when optimizing. They're using Solr 1.4 (Lucene 2.9) running on a server that mounts NFS for index. I think I know the famous "Stale NFS File Handle IOException" problem, but I think it causes FileNoutFoundException. Is there any chance to hit the exc

Re: OR operator in lucene

2011-09-09 Thread Ian Lea
Documents with both terms are likely to be returned first because they are likely to score higher, but that isn't guaranteed. Google "lucene scoring" for details. If you absolutely must have docs with both terms first, you could execute 2 searches: the first with AND and the second with OR, then

Search highlighter for custom Query implementations - how to?

2011-09-09 Thread Lukáš Vlček
Hi, What is the recommended way to use custom Query implementations with Lucene (3.3.0) Highlighter framework? In short, what worries me a bit is the fact that WeightedSpanTermExtractor#extract(Query, Map) accepts a general Query parameter but inside it does a lot of tests against particular impl

OR operator in lucene

2011-09-09 Thread dyzc2010
Hi, I am trying to use OR operator, and want to know if it returns the results comprising of both keywords contained first and then either keyword later. To be more clear, if A OR B is enforced, should results with A and B are returned first? If not, how can I make sure that the OR operator

Re: Lucene killing JVM

2011-09-09 Thread Dragan Jotanovic
Looks like the problem is somewhere in MMapFSDirectory. As soon as I switched to NIOFSDirectory, JVM stopped crashing. Maybe MMapFSDirectory shouldn't be default FSDirectory on 64bit Linux ater all? Dragan On 9/1/2011 11:51 AM, Uwe Schindler wrote: Hi, If the other suggestions for checking

Checkout SearchWorkings.org - it just went live!

2011-09-09 Thread Simon Willnauer
Hey folks, Some of you might have heard, myself and a small group of other passionate search technology professionals have been working hard in the last few months to launch a community site known as SearchWorkings.org [1]. This initiative has been set up for other search professionals to have a s