Re: Lucene cluster with NFS or synchronization tool such as rsync

2016-07-07 Thread Michael McCandless
Alas, there are no more docs than the classes themselves, in the lucene/replicator module, under the oal.replicator.nrt package. Essentially, you create a PrimaryNOde (equivalent of IndexWriter) for indexing documents, in a JVM on machine 1, and a ReplicaNode in a JVM on machine 2, but you must su

Re: Lucene cluster with NFS or synchronization tool such as rsync

2016-07-05 Thread Sanne Grinovero
I had a similar need some ~5 years ago, and contributed this Lucene extension to the Infinispan project: - http://infinispan.org/docs/8.2.x/user_guide/user_guide.html#_infinispan_as_a_storage_for_lucene_indexes It since matured and is now being actively maintained by several other people using i

Re: Lucene cluster with NFS or synchronization tool such as rsync

2016-07-04 Thread Desteny Child
Hi Mike, Thanks you very much for your response. I would be really grateful if you can please provide me with an information where I can read(may be with examples) about new near-real-time replication ? Thanks, Alex 2016-07-04 12:57 GMT+03:00 Michael McCandless : > NFS is dangerous if differen

Re: Lucene cluster with NFS or synchronization tool such as rsync

2016-07-04 Thread Michael McCandless
NFS is dangerous if different nodes may take turns writing to the shared index. Locking sometimes doesn't work correctly, client-side metadata caching (e.g. the directory entry) can cause problems, NFS doesn't support "delete on final close" semantics that Lucene relies on. rsync-like behavior ca

Re: Lucene cluster with NFS or synchronization tool such as rsync

2016-07-03 Thread Evert Wagenaar
Hello, I know that Nutch does it this way, already for years (Since version 1.0). If you want to know exactly how it's done you should look into Nutch's Source Code. The latest version of Nutch actually uses SolrCloud, therefore I would look into earlier versions of Nutch, say 1.xx. Don't let Map

Lucene cluster with NFS or synchronization tool such as rsync

2016-07-03 Thread Desteny Child
I need to organize a cluster for my stateless application based on Lucene 5.2.1. Right now I'm looking for a solution in order to share Lucene index via NFS or rsync between different Lucene nodes. Is it a good idea to use NFS for this purpose and if so will it be possible to read/write from diffe