Re: manually merging Directories

2014-12-23 Thread Erick Erickson
I doubt this is going to work. I have to ask why you're worried about the I/O; this smacks of premature optimization. Not only do the files have to be moved, but the right control structures need to be in place to inform Solr (well, Lucene) exactly what files are current. There's a lot of room for

manually merging Directories

2014-12-23 Thread Shaun Senecal
Hi I have a number of Directories which are stored in various paths on HDFS, and I would like to merge them into a single index. The obvious way to do this is to use IndexWriter.addIndexes(...), however, I'm hoping I can do better. Since I have created each of the separate indexes using Map/R

Lucene search/count performance abrupt degradation (MMapDirectory)

2014-12-23 Thread Piotr Idzikowski
Hello. I have a performance problem with Lucene search queries running on Java 7u71 Linux platform. I use the newest 4.10 version, but we also experience the same problem with Lucene 4.7.2. I observed that after some time (1 or 2 days), even without any heavy use, Lucene search and count slow dow

RE: BTRFS ?

2014-12-23 Thread Uwe Schindler
Hi Dawid, Unfortunately, for that to work, Solr needs to solely use NIO.2, too. Only Lucene disallows java.io.File and related classes, Solr is excluded from this forbidden-check. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > --

RE: "batch-update"-pattern, NoMergeScheduler?

2014-12-23 Thread nnagarajayya
You can try out the TimedSerialMergeScheduler. It allows you to set a merge schedule to a time in the evening or after n number of merge requests ... http://rankingalgorithm.1050964.n5.nabble.com/TimedSerialMergerScheduler-java-allows-merges-to-be-deferred-to-a-known-time-like-11pm-or-1am-td5706

Help for Implementing Most relevance Search algorithm in lucene for my project

2014-12-23 Thread Nitin Chauhan
Hi, I wanted to implement "most relevant search" in Lucene for my project. I am currently using the Lucene provided by Hybris 5.3 i.e. Lucene 4.6.1. The scenario is that I have type ahead functionality (autosuggest) implemented already in the project so when user starts typing in the input box,

Re: BTRFS ?

2014-12-23 Thread Dawid Weiss
> This could speed up tests, especially Solr where some dirs are copied over > and over for every test case. :-) A wild idea, but since there's NIO everywhere now you could use an in-memory filesystem for tests and avoid going to disk entirely :D https://github.com/google/jimfs Dawid -

Re: "batch-update"-pattern, NoMergeScheduler?

2014-12-23 Thread Ian Lea
Hi I can't give an exact answer to your question but my experience has been that it's best to leave all the merge/buffer/etc settings alone. If you are doing a bulk update of a large number of docs then it's no surprise that you are seeing a heavy IO load. If you can, it's likely to be worth giv