I’m guessing this is not simply retrieving all documents through API using pagination and sending them to index 🤔 About being in-place, how can it work when a new Solr version requires a different schema or config file, because time to time old definitions don’t work in a new version.
-ufuk — > On Mar 30, 2025, at 10:33, Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD A) > <lkotzanie...@bloomberg.net> wrote: > > Hi Rahul, > > This sounds very interesting! > > I enjoyed the discussion at CoC and would be very > interested to hear more about the technical details. > > I am also curious to know more what you mean by "in-place" > and what the expectation is around downtime. > > Either way I am sure this would be a great addition to > the tool belt for getting people to finally move off > ancient versions of Solr. > > Look forward to discussing this more on the JIRA! > > Luke > > From: users@solr.apache.org At: 03/28/25 01:05:57 UTC-4:00To: > users@solr.apache.org > Subject: Automatic upgrade of Solr indexes over multiple versions > > Today upgrading from Solr version X to X+2 requires complete reingestion of > data from source. This comes from Lucene's constraint which only guarantees > index compatibility between the version the index was created in and the > immediate next version. > > > This reindexing usually comes with added downtime and/or cost. Especially > in case of deployments which are in customer environments and not > completely in control of the vendor, this proposition of having to > completely reindex the data can become a hard sell. > > > I have developed a way which achieves this reindexing in-place on the same > index. Also, the process automatically keeps "upgrading" the indexes over > multiple subsequent Solr upgrades without needing manual intervention. > > > It does come with a limitation that all *source* fields need to be either > stored=true or docValues=true. Any copyField destination fields can be > stored=false of course, but as long as the source field (or in general, the > fields you care about preserving) is either stored or docValues true , the > tool can reindex in-place and legitimately "upgrade" the index. For indexes > where this limitation is not a problem (it wasn't for us!), this tool can > remove a lot of operational headaches, especially in environments with > hundreds/thousands of very large indexes. > > > I had a conversation about this with some of you during "Apache Community > over Code 2024" in Denver, and I could sense some interest. If this feature > sounds appealing, I would like to contribute it to Solr on behalf of my > employer, Commvault. Please let me know if I should create a JIRA and get > the discussion rolling! > > > Thanks, > Rahul Goswami > >
