Re: Did you Mean search on Indexes created by Different Files.

2013-07-31 Thread Varun Thacker
Hi, I'm not quite sure on what you mean when you are saying that the suggestions are coming from a dictionary instead of the index. In your example, which looks okay, the suggestion will come from the the reader which is reading from your index. One thing I noticed, when you are creating the term

Re: Did you Mean search on Indexes created by Different Files.

2013-07-31 Thread Ankit Murarka
Can anyone please guide me on how to implement Did You Mean Search using indexes created from the supplied bunch of files as an input. On 7/31/2013 11:15 AM, Ankit Murarka wrote: Any help on this will be highly appreciated..I have been trying all possible different option but to no avail. Al

RE: lucene 4.3 seems to be much slower in indexing than lucene 3.6?

2013-07-31 Thread Zhang, Lisheng
Hi Mike, I retested and results are the same: 1/ I did not use sort (so FieldCache should not enter picture?) 2/ I created indexed data from scratch separately for 361 and 43 based on same text (text files), and I ran test from command line separately against each index folder, so seems a

Re: sorting with lucene 4.3

2013-07-31 Thread Nicolas Guyot
ok i see. I tried the same test with randomized values on the numeric DV and now the search speed is low and constant. It's not gonna solve our issue since the values are relatively ordered in our case but it's good to know. On Wed, Jul 31, 2013 at 12:05 PM, Yonik Seeley wrote: > On Wed, Jul 3

Re: sorting with lucene 4.3

2013-07-31 Thread Yonik Seeley
On Wed, Jul 31, 2013 at 2:51 PM, Nicolas Guyot wrote: > I have written a quick test to reproduce the slower sorting with numeric DV. > In this test case, it happens only when reverse sorting. Right - I bet your numeric field is relatively ordered in the index. When this happens, there is always o

Re: sorting with lucene 4.3

2013-07-31 Thread Nicolas Guyot
thanks Adrien for the explanation, it's really much appreciated. I have written a quick test to reproduce the slower sorting with numeric DV. In this test case, it happens only when reverse sorting. About the sorting by page i mentioned, it was due to a mistake in our testcase, sorry about that.

Re: lucene 4.3 seems to be much slower in indexing than lucene 3.6?

2013-07-31 Thread Michael McCandless
On Tue, Jul 30, 2013 at 6:13 PM, Zhang, Lisheng wrote: > Hi Mike, > > I did more tests with realistic text from different languages (typical > text for 8 different languages, English one is novel "Animal Farm"). > > What I found seems to be: > > ## Indexing: > 36 and 43 comparable (your previous c

RE: why lucene not use DirectByteBuffer in NIOFSDirectory

2013-07-31 Thread Uwe Schindler
Hi, There is a misunderstanding: Just by allocating a direct buffer, there is still no difference to a heap buffer in the workflow! NIO will read the data from file, copy it to FS cache and then the positional read() system call (used by NIO) copies the FS cache contents to the direct buffer,

why lucene not use DirectByteBuffer in NIOFSDirectory

2013-07-31 Thread wangzhijiang999
I read this article "Use Lucene's MMapDirectory on 64bit platforms, please!" and it said the MMapDirectory is better than other Directory because it will void copy data between file system cache and java heap.   I checked the source code of NIOFSDirectory, and in new Buffer method it called "Byt