[
https://issues.apache.org/jira/browse/SOLR-12999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Smiley updated SOLR-12999:
--------------------------------
Description:
Index replication could optionally delete files that it knows will not be
needed _first_. This would reduce disk capacity requirements of Solr, and it
would reduce some disk fragmentation when space get tight.
Solr (IndexFetcher) already grabs the remote file list, and it could see which
files it has locally, then delete the others. Today it asks Lucene to
{{deleteUnusedFiles}} at the end. This new mode would probably only be useful
if there is no SolrIndexSearcher open, since it would prevent the removal of
files.
The motivating scenario is a SolrCloud replica that is going into full
recovery. It ought to not be fielding searches. The code changes would not
depend on SolrCloud though.
This option would have some danger the user should be aware of. If the
replication fails, leaving the local files incomplete/corrupt, the only
recourse is to try full replication again. You can't just give up and field
queries.
was:
Index replication could optionally delete files that it knows will not be
needed _first_. This would reduce disk capacity requirements of Solr, and it
would reduce some disk fragmentation when space get tight.
Solr (IndexFetcher) already grabs the remote file list, and it could see which
files it has locally, then delete the others. Today it asks Lucene to
{{deleteUnusedFiles}} at the end. This new mode would only be useful if there
is no SolrIndexSearcher open, since it would prevent the removal of files.
> Index replication could delete segments first
> ---------------------------------------------
>
> Key: SOLR-12999
> URL: https://issues.apache.org/jira/browse/SOLR-12999
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: replication (java)
> Reporter: David Smiley
> Priority: Major
>
> Index replication could optionally delete files that it knows will not be
> needed _first_. This would reduce disk capacity requirements of Solr, and it
> would reduce some disk fragmentation when space get tight.
> Solr (IndexFetcher) already grabs the remote file list, and it could see
> which files it has locally, then delete the others. Today it asks Lucene to
> {{deleteUnusedFiles}} at the end. This new mode would probably only be
> useful if there is no SolrIndexSearcher open, since it would prevent the
> removal of files.
> The motivating scenario is a SolrCloud replica that is going into full
> recovery. It ought to not be fielding searches. The code changes would not
> depend on SolrCloud though.
> This option would have some danger the user should be aware of. If the
> replication fails, leaving the local files incomplete/corrupt, the only
> recourse is to try full replication again. You can't just give up and field
> queries.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
