RE: Lucene Sharding and Replication.

2014-04-08 Thread Umashanker, Srividhya
We are looking to run a federated index across different nodes. You are right, we need similar distributed feature as solrcloud or the elasticsearch provides. But we want to evaluate building the sharding and replication on our own framework, which is developed with vanilla lucene. -Vidhya

Re: Lucene Sharding and Replication.

2014-04-08 Thread Furkan KAMACI
Hi; Are you looking for something like SolrCloud? Thank; Furkan KAMACI 2014-04-08 17:08 GMT+03:00 Umashanker, Srividhya < srividhya.umashan...@hp.com>: > HI Group - > > Is there anyone who has tried or researched on manual sharding and > replication with Lucene? > > We are also evaluating ES,

Lucene Sharding and Replication.

2014-04-08 Thread Umashanker, Srividhya
HI Group - Is there anyone who has tried or researched on manual sharding and replication with Lucene? We are also evaluating ES, but trying to see if we can enhance our existing framework to do manual sharding and replication When I looked for details. I found the MultiPassIndexSplitter - t

Using Sentence Information for Lucene Highlighting

2014-04-08 Thread Furkan KAMACI
Hi; I could not get an answer for my question at Solr list and I wanted to ask it here because I think that it is more Lucene specific question. I have indexed my documents and there is a special character sequence that shows the end of a string. It is: *|* For example: The quick brown fox jum

Re: background merge hit exception

2014-04-08 Thread Jose Carlos Canova
Hi Jason, No, the StrackTrace shows clearly the cause of the errror occurred during the merge into a single index file segment(forgeMerge parameter defines the number of desired segments at end). During the indexing of a document, Lucene might decide to create a new segment of the information ext

Re: IndexReplication Client and IndexWriter

2014-04-08 Thread Michael McCandless
You might be able to use a class on the NRT replication branch (LUCENE-5438), InfosRefCounts (weird name), whose purpose is to do what IndexFileDeleter does for IndexWriter, ie keep track of which files are still referenced, delete them when they are done, etc. This could used on the client side t

Re: IndexReplication Client and IndexWriter

2014-04-08 Thread Shai Erera
IndexRevision uses the IndexWriter for deleting unused files when the revision is released, as well as to obtain the SnapshotDeletionPolicy. I think that you will need to implement two things on the "client" side: * Revision, which doesn't use IndexWriter. * Replicator which keeps track of how ma

Re: IndexReplication Client and IndexWriter

2014-04-08 Thread Michael McCandless
It's not safe also opening an IndexWriter on the client side. But I agree, supporting tree topology would make sense; it seems like we just need a way for the ReplicationClient to also be a Replicator. It seems like it should be possible, since it's clearly aware of the SessionToken it's pulled fr

IndexReplication Client and IndexWriter

2014-04-08 Thread Christoph Kaser
Hi all, I am trying out the (highly useful) index replicator module (with the HttpReplicator) and have stumbled upon a question: It seems, the IndexReplicationHandler is working directly on the index directory, without using an indexwriter. Could there be a problem if I open an IndexWriter on

Re: background merge hit exception

2014-04-08 Thread Jason Wee
Hello Jose, Thank you for your response, I took a closer look. Below are my responses: > Seems that you want to force a max number of segments to 1, // you're done adding documents to it): // writer.forceMerge(1); writer.close(); Yes, the line of code is uncommented be