[
https://issues.apache.org/jira/browse/SOLR-16697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anshum Gupta updated SOLR-16697:
--------------------------------
Priority: Major (was: Blocker)
> New API support to import index files generated by Embedded SOLR into SOLR
> Cloud
> --------------------------------------------------------------------------------
>
> Key: SOLR-16697
> URL: https://issues.apache.org/jira/browse/SOLR-16697
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Backup/Restore
> Reporter: Indumathy Rajagopalan
> Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Offline indexing is a popular option when really large data sets needs to be
> indexed into SOLR.
> Data is loaded from data source ( eg. c*) and index creation pipelines
> produce index files per shard using embedded SOLR.
>
> With older versions of SOLR, we would copy these index files into SOLR Cloud
> data directories using a custom tools and reload the collection to be able to
> search/update on the newly uploaded collection.
> Ideally, we should use the Restore API to import the index files from backup
> repository. However, the file structure expected for the Restore API to work
> is complex enough that massaging the index files in every shard into Restore
> compatible format is infeasible.
>
> It would be good for SOLR to support a 'Restore' like API that would allow us
> to import index files generated by embedded SOLR into SOLR Cloud ? This API
> should operate on shard level and be able to import the index files into a
> single shard (per invocation)
>
> *With the new API , offline indexing could look like this :*
>
> 1. Generate index files per shard using embedded SOLR as a part of hadoop MR
> /Spark jobs and copy all index files for every shard into backup repository.
>
> 2. The New API should be able to import the index from backup repository
> location into each shard on SOLR Cloud. The API would handle things like
> marking the collection as read-only, trigger replication etc. along the lines
> of what the 'RESTORE' API currently does.
>
> The new API should be able to support relevant parameters from Restore API (
> location & repository )
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
