RE: search time & number of segments

2014-05-20 Thread Toke Eskildsen
De Simone, Alessandro [alessandro.desim...@bvdinfo.com] wrote: > We have stopped optimizing the index because everybody told us it was a bad > idea. > It makes sense if you think about it. When you reopen the index not all > segments must be reopened then you have: > (1) better reload time >

RE: Question about multi-valued fields

2014-05-20 Thread Allison, Timothy B.
Chris, Good to see you over here. There's probably an easier way... I ran into this with geo queries, and the answer there is to test every value in the multi field for the document that is a hit. For the text search question, though, you could use analysis and then run a SpanQuery against y

Question about multi-valued fields

2014-05-20 Thread Chris Bamford
Hi, Does anyone know a way of identifying which entry in a multi-value field actually matches during a search? e.g. in this example: String[] entryList = { "entry one", "entry two", "entry three", "entry four" }; Document doc = new Document(); for (String entry : Arrays.asList(entryList))

Re: How to add machine learning to Apache lucene

2014-05-20 Thread Diego Fernandez
Great! That's good to know that at least it's being discussed. The jforests package seems really interesting, I'll start looking a bit more into it. Although I'm not a Java programmer per se, I'd be interested in helping out with this effort. If there's anything I can do, let me know! Diego

RE: search time & number of segments

2014-05-20 Thread De Simone, Alessandro
> You are changing a system from being heavily optimized towards search to be > balanced between updates and search. There seems to be an assumption that > this will be without a change to hardware requirements, which I find to be > quite optimistic. We have stopped optimizing the index because

RE: NewBie To Lucene || Perfect configuration on a 64 bit server

2014-05-20 Thread Shruthi
-Original Message- From: Toke Eskildsen [mailto:t...@statsbiblioteket.dk] Sent: Tuesday, May 20, 2014 3:48 PM To: java-user@lucene.apache.org Subject: Re: NewBie To Lucene || Perfect configuration on a 64 bit server On Tue, 2014-05-20 at 11:56 +0200, Shruthi wrote: Toke: > Is 20 seco

Re: search time & number of segments

2014-05-20 Thread Toke Eskildsen
On Tue, 2014-05-20 at 15:04 +0200, De Simone, Alessandro wrote: Toke: > > Using the calculator, I must admit that it is puzzling that you have > 2432 / 143 = 17.001 times the amount of seeks with 16 segments. > > Do you have any clue? Is there something I could test? If your segmented index was

RE: search time & number of segments

2014-05-20 Thread De Simone, Alessandro
Hi again! > Using the calculator, I must admit that it is puzzling that you have 2432 / 143 = 17.001 times the amount of seeks with 16 segments. Do you have any clue? Is there something I could test? > Is the total file size of the optimized index about the same as the segmented > one? Yes i

[ANNOUNCE] Apache Lucene 4.8.1 released

2014-05-20 Thread Robert Muir
May 2014, Apache Lucene™ 4.8.1 available The Lucene PMC is pleased to announce the release of Apache Lucene 4.8.1 Apache Lucene is a high-performance, full-featured text search engine library written entirely in Java. It is a technology suitable for nearly any application that requires full-text

Re: NewBie To Lucene || Perfect configuration on a 64 bit server

2014-05-20 Thread Toke Eskildsen
On Tue, 2014-05-20 at 11:56 +0200, Shruthi wrote: Toke: > Is 20 second an acceptable response time for your users? > > Shruthi: Its definitely not acceptable. PFA the piece of code that we > are using..Its taking 20seconds. That’s why I drafted this ticket to > see where I was going wrong. Inde

RE: NewBie To Lucene || Perfect configuration on a 64 bit server

2014-05-20 Thread Shruthi
-Original Message- From: Toke Eskildsen [mailto:t...@statsbiblioteket.dk] Sent: Tuesday, May 20, 2014 3:01 PM To: java-user@lucene.apache.org Subject: Re: NewBie To Lucene || Perfect configuration on a 64 bit server On Tue, 2014-05-20 at 10:40 +0200, Shruthi wrote: > Just the indexing t

Re: NewBie To Lucene || Perfect configuration on a 64 bit server

2014-05-20 Thread Toke Eskildsen
On Tue, 2014-05-20 at 10:40 +0200, Shruthi wrote: > Just the indexing took 20 seconds L That's more than I expected, but it leaves the same question: Is 20 second an acceptable response time for your users? I don't know your document size, but unless they are very large, the response times from a

RE: NewBie To Lucene || Perfect configuration on a 64 bit server

2014-05-20 Thread Shruthi
-Original Message- From: Toke Eskildsen [mailto:t...@statsbiblioteket.dk] Sent: Tuesday, May 20, 2014 12:57 PM To: java-user@lucene.apache.org Subject: Re: NewBie To Lucene || Perfect configuration on a 64 bit server On Mon, 2014-05-19 at 12:40 +0200, Shruthi wrote: > 1. Client

Re: NewBie To Lucene || Perfect configuration on a 64 bit server

2014-05-20 Thread Toke Eskildsen
On Mon, 2014-05-19 at 12:40 +0200, Shruthi wrote: > 1. Client makes a request with a search phrase. Lucene > application indexes a list of 500 documents(at max. ) and searches the > phrase on the index constructed. Fetching from NAS + indexing sounds like something that would take a second o