Re: Excessive use of IOException without proper documentation

2012-11-02 Thread Simon Willnauer
Hey, On Fri, Nov 2, 2012 at 2:20 PM, Michael-O <1983-01...@gmx.net> wrote: > Hi, > > why does virtually every method (exaggerating) throw an IOE? I know there > might be a failure in the underlying IO (corrupt files, passing checked exc > up, etc) but > > 1. Almost none of the has a JavaDoc on i

Re: App supplied docID in lucene possible?

2012-11-02 Thread Michael McCandless
I suspect app-controlled docID will be a challenge, but I haven't thought it through much. One possible solution might be to use joins? Either index time or query time Ie, make a document that has the big text field that never change, and a separate document that has all the little fields th

RE: "read past EOF" when merge

2012-11-02 Thread Markus Jelsma
No this is not using NFS but EXT3 on SSD. Thanks -Original message- > From:Michael McCandless > Sent: Fri 02-Nov-2012 16:22 > To: java-user@lucene.apache.org > Subject: Re: "read past EOF" when merge > > On Fri, Nov 2, 2012 at 6:53 AM, Markus Jelsma > wrote: > > Hi, > > > > For what

Re: "read past EOF" when merge

2012-11-02 Thread Michael McCandless
On Fri, Nov 2, 2012 at 6:53 AM, Markus Jelsma wrote: > Hi, > > For what it's worth, we have seen similar issues with Lucene/Solr from this > week's trunk. The issue manifests itself when it want to replicate. The > servers have not been taken offline and did not crash when this happenend. > > 20

Excessive use of IOException without proper documentation

2012-11-02 Thread Michael-O
Hi, why does virtually every method (exaggerating) throw an IOE? I know there might be a failure in the underlying IO (corrupt files, passing checked exc up, etc) but 1. Almost none of the has a JavaDoc on it 2. Throwing an IOE from most of the methods doesn't really help. You cannot create se

Re: App supplied docID in lucene possible?

2012-11-02 Thread Deepak Shakya
Why don't you try using your app supplied DOC ID as one of the fields of the document. You can add, update and delete documents based on that. On Fri, Nov 2, 2012 at 4:55 PM, Ravikumar Govindarajan < ravikumar.govindara...@gmail.com> wrote: > I am aware of ExternalFileField, but docID solution l

Re: App supplied docID in lucene possible?

2012-11-02 Thread Ravikumar Govindarajan
I am aware of ExternalFileField, but docID solution looks more elegant and performant Our re-indexing rate daily is around 35-40% of index additions. When a small int-value/boolean value in a lucene document changes, I need to re-index an entire 5-10MB content again. This is the reason why I am l

RE: "read past EOF" when merge

2012-11-02 Thread Markus Jelsma
Hi, For what it's worth, we have seen similar issues with Lucene/Solr from this week's trunk. The issue manifests itself when it want to replicate. The servers have not been taken offline and did not crash when this happenend. 2012-10-30 16:12:51,061 WARN [solr.handler.ReplicationHandler] - [h

Re: "read past EOF" when merge

2012-11-02 Thread Michael McCandless
Are you able to reproduce the corruption? If at any time you accidentally had two writers open on the same index, it could have created this corruption. Writing to an index over NFS ought to be OK, however, it's not well tested. You should use SimpleFSLockFactory (not the default NativeFSLockFac

Re: How to correctly use SearcherManager#close?

2012-11-02 Thread Michael-O
> hey michael, > > On Thu, Nov 1, 2012 at 11:30 PM, Michael-O <1983-01...@gmx.net> wrote: > > Thanks for the quick response. Any chance this could be clearer in the > JavaDoc of this class? > > sure thing, do you wanna open an issues / create a patch I am happy to > commit it. Thanks Simon, Iss

Re: How to correctly use SearcherManager#close?

2012-11-02 Thread Michael-O
> That would be great! > > Hmm which version are you looking at? In 4.0 it currently says this: > > /** >* Close this ReferenceManager to future {@link #acquire() acquiring}. > Any >* references that were previously {@link #acquire() acquired} won't be >* affected, and they should

Re: ComplexPhraseQueryParser and stop words

2012-11-02 Thread Mark Harwood
Hi Brandon, Can you start by calling toString on the parse result (the Query object) to see what is being produced and post that here. On the face of it it sounds like it should work OK. What happens if you use the "normal" query parser on your query "time to leave" - that should parse ok as

Re: 4.0 tokenStream or SimpleAnalyzer bug?

2012-11-02 Thread Simon Willnauer
hey scott, this is intentional see the javadoc step 2: http://lucene.apache.org/core/4_0_0/core/org/apache/lucene/analysis/TokenStream.html simon On Fri, Nov 2, 2012 at 2:07 AM, Scott Smith wrote: > I was doing some tokenizer/filter analysis attempting to fix a bug I have in > highlighting und