[
https://issues.apache.org/jira/browse/LUCENE-4975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shai Erera updated LUCENE-4975:
-------------------------------
Attachment: LUCENE-4975.patch
New patch changes how handlers work:
* Beasting found a seed which uncovered a major problem with their current
operation. They were trying to be "too honest" with the index and e.g.
revert/delete upon any exception that occurred.
* Thanks to MDW, Mike and I decided to keep the handlers simple -- if a handler
successfully copies + syncs the revision files, then this is considered the
"new revision".
* Kissing the index is now done not through IndexWriter, but rather deleting
all files not referenced by last commit.
** That cleanup is a best-effort only ... if it fails, it just logs this
information and not act on it. Cleanup can happen later too.
* That means that if you have a really nasty crazy IO system (like MDW
sometimes acts), the Replicator is not the one that's going to care about it.
The app will hit those weird errors in other places too, e.g. when it tries to
refresh SearcherManager or perform search.
** These errors are not caused by the Replicator or bad handler operation. I.e.
if the handler successfully called fsync(), yet the IO system decides to fail
later ... that's really not the handler's problem.
* Therefore the handlers are now simpler, don't use IW (and the crazy need to
rollback()), and once files were successfully copied + sync'd, no more
exceptions can occur by the handler (except callback may fail, but that's ok).
* I also removed the timeout behavior the test employed -- now that
ReplicationClient has isUpdateThreadAlive(), assertHandlerRevision loops as
long as the client is alive. If there's a serious bug, test-framework will
terminate the test after 2 hours ...
* ReplicationClient.startUpdateThread is nicer -- allows starting the thread if
updateThread != null, but !isAlive.
Now beasting this patch.
> Add Replication module to Lucene
> --------------------------------
>
> Key: LUCENE-4975
> URL: https://issues.apache.org/jira/browse/LUCENE-4975
> Project: Lucene - Core
> Issue Type: New Feature
> Reporter: Shai Erera
> Assignee: Shai Erera
> Attachments: LUCENE-4975.patch, LUCENE-4975.patch, LUCENE-4975.patch,
> LUCENE-4975.patch, LUCENE-4975.patch, LUCENE-4975.patch, LUCENE-4975.patch,
> LUCENE-4975.patch, LUCENE-4975.patch, LUCENE-4975.patch, LUCENE-4975.patch,
> LUCENE-4975.patch
>
>
> I wrote a replication module which I think will be useful to Lucene users who
> want to replicate their indexes for e.g high-availability, taking hot backups
> etc.
> I will upload a patch soon where I'll describe in general how it works.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
