Replicating Lucene Index with out SOLR

2008-08-27 Thread rahul_k123
I have the following requirement Right now we have multiple indexes serving our web application. Our indexes are around 30 GB size. We want to replicate the index data so that we can use them to distribute the search load. This is what we need ideally. A – (supports writes and reads) A1 –Rep

Re: Replicating Lucene Index with out SOLR

2008-08-27 Thread rahul_k123
ng the index, only the new/modified files be copied. This > is what Solr replication scripts do, too. > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > - Original Message >> From: rahul_k123 <[EMAIL PROTECTED]> >&

Re: Replicating Lucene Index with out SOLR

2008-08-28 Thread rahul_k123
he design of the Lucene >> file >> format and therefore uses minimal network traffic and places no >> restrictions on >> the IndexWriter merge policies I may choose to use to optimise search >> speed. >> >> Keen to explore the pros and cons of these dif

IndexReader.isCurrent()

2008-09-16 Thread rahul_k123
what is the behaviour of IndexReader.current() if i modify the index manually? Will it returns false? One more question The index is on Linux if my indexReader is open and some of the files in index are deleted, what is the behaviour of this? Will it give any exception like File Not found??

Re: IndexReader.isCurrent()

2008-09-17 Thread rahul_k123
I am doing replication and i am running scripts to sync the index b/w master and slave. Michael McCandless-2 wrote: > > > rahul_k123 wrote: > >> what is the behaviour of IndexReader.current() if i modify the index >> manually? Will it returns false? > >

Rsync causing search timeouts on master

2008-09-23 Thread rahul_k123
Hi, I am using snappuller to sync my slave with master, i am not using rsync daemon, i am doing Rsync using remote shell. When i am serving requests from the master when the snappuller is running (after optimization, total index is arnd 4 gb it doing the transfer of whole index), the performance

Caused by: java.io.IOException: read past EOF on Slave

2008-09-25 Thread rahul_k123
First of all, thanks to all the people who helped me in getting the lucene replication setup working and right now its live in our production :-) Everything working fine, except that i am seeing some exceptions on slaves. The following is the one which is occuring more often on slaves at java

Re: Caused by: java.io.IOException: read past EOF on Slave

2008-09-26 Thread rahul_k123
g accessed through? > > Mike > > rahul_k123 wrote: > >> >> First of all, thanks to all the people who helped me in getting the >> lucene >> replication setup working and right now its live in our production :-) >> >> Everything working fi

Re: Caused by: java.io.IOException: read past EOF on Slave

2008-09-26 Thread rahul_k123
ontext of these IndexReader.document(...) calls? How are > you getting the doc numbers that you're passing to them? In both > cases, an invalid doc number would explain your exception. Are you > doing any search caching, where you cache hits and then much later try > to lo

Document.getBinaryValue returning null after upgrading to 2.4 for the data which was indexed using 2.3.1

2008-12-16 Thread rahul_k123
The data was indexed using 2.3.1 as follows doc.add(new Field(Fields.DETAILS, byte[] bytes, Field.Store.YES)); When i reindex this particular item using 2.4 and when i try to retrive it, it works fine. Then for the items which were indexed using 2.3.1 and not reindexed using 2.4 t

Re: Document.getBinaryValue returning null after upgrading to 2.4 for the data which was indexed using 2.3.1

2008-12-16 Thread rahul_k123
: > > rahul_k123 wrote: >> The data was indexed using 2.3.1 as follows >> >> doc.add(new Field(Fields.DETAILS, byte[] bytes, >> Field.Store.YES)); >> >> When i reindex this particular item using 2.4 and when i try to retrive >> it, >> it wo