Re: Separating IndeWriter with NRT

2011-07-22 Thread duschhaube
Hi, I think you have to use IndexReader.isCurrent and then reopen the reader if there are changes. You will need an automatism that periodically checks if any changes where made to the index. If that happens und need to reopen the your reader. But a real lucene expert maybe can give you a bet

Separating IndeWriter with NRT

2011-07-22 Thread Mihai Caraman
I trust that some of you had to run the indexing as a service/jar and the search as a servlet/war. How can i obtain this while still keeping the search near real time(this is difficult because IndexReader needs direct access to the IndexWriter instance). If no lucene users know this, where else sho

remove queryNorm only for boosted score

2011-07-22 Thread Elaine Li
Hi, I have a same question to this one which was filed long time ago( 1969?:) ). : Is it possible to remove the QueryNorm?? : so all the bf boost became an add of the solr score?? http://search.lucidimagination.com/search/document/d5605388d7fa38be/querynorm_and_fieldnorm I asked it on the solr

Re: Index one huge text file

2011-07-22 Thread starz10de
I have no problem with indexing performance. I indexed the 60 000 (sentences) text files with only few minutes. I have performance problem split the huge file that contains 60 000 sentences into 60 000 text files even I can have an index in sentence level. I asked if I could read the one huge fi

Re: Index one huge text file

2011-07-22 Thread Shashi Kant
Alternatively, you could create a multivalued field whereby each sentence is in the same document, but retrievable in order. On Fri, Jul 22, 2011 at 11:10 AM, Glen Newton wrote: > So to use Lucene-speak, each sentence is a document. > > I don't know how you are indexing and what code you are usi

Re: Index one huge text file

2011-07-22 Thread Glen Newton
So to use Lucene-speak, each sentence is a document. I don't know how you are indexing and what code you are using (and what hardware, etc.), but you if you are not already, should consider multi-threading the indexing which should give you a significant indexing performance boost. -Glen On Fri

Re: Index one huge text file

2011-07-22 Thread starz10de
I can save the sentences in lucene index as extra field which i can call for example "sentence_content" -- View this message in context: http://lucene.472066.n3.nabble.com/Index-one-huge-text-file-tp3191605p3191637.html Sent from the Lucene - Java Users mailing list archive at Nabble.com. --

Re: Index one huge text file

2011-07-22 Thread starz10de
I am interested to search in sentence level. It is a parallel corpora , each sentence in the first language is equivalence to sentence in the second language. I want to index each sentence and have some id for each sentence in order when I retrieve it I go easily and retrieve its equivalence in th

Re: Index one huge text file

2011-07-22 Thread Glen Newton
Could you elaborate what you want to do with the index of large documents? Do you want to search at the document or sentence level? This can drive how to index this content. -Glen On Fri, Jul 22, 2011 at 10:52 AM, starz10de wrote: > Hi, > > I have one text file that contains 60 000 sentences. Is

Index one huge text file

2011-07-22 Thread starz10de
Hi, I have one text file that contains 60 000 sentences. Is there a possibility to index this file sentence by sentence where each sentence is treated as one document? What I do now is splitting the huge text files into 60 000 sentences then index them. This work is not easy because I have few hug