Erick, Typically, while creating collections, a replicationFactor is specified. Thus, the meta data about the collection does have information about what the "desired" replicationFactor is for the collection. If that's the case, when a Solr node joins the cluster, there could be a pro-active add-replica operation that can be initiated if the Solr detects that the current replicas are less than the desired replicationFactor and pull the collection data from the leader.
Isn't that what the attribute "autoAddReplicas" does for HDFS - can this be done for non-shared filesystem? As a side note, we do this for our customers as that's baked into our cloud provisioning software, but it would be nice if Solr supports that OOTB. Are there any underlying flaws of doing that? Thanks, -- *Sameer Maggon* www.measuredsearch.com <https://mailtrack.io/trace/link/663333fad5b85359bf1b21be04166edea6c7d13e?url=http%3A%2F%2Fmeasuredsearch.com%2F&signature=6dbc74f0abef4882> | Deploy, Scale & Manage Solr in the cloud of your choice. On Wed, Dec 9, 2015 at 11:19 AM, Erick Erickson <erickerick...@gmail.com> wrote: > Not that I know of. The two systems are somewhat disconnected. > AWS doesn't know that Solr lives on those nodes, it's just spinning > one up, right? Albeit with Solr running. > > There's nothing in Solr that auto-detects the existence of a new > Solr node and automagically assigns collections and/or replicas. > > How would either system intuit that this new node is replacing > something else and "do the right thing"? > > I'll tell you how, by interrogating Zookeeper and seeing that for some > specific collection, shardX had fewer replicas than other shards and > issuing the Collections API ADDREPLICA command. > > But now there are _three_ systems that need to be coordinated and > doing the right thing in your situation would be the wrong thing in > another. The last thing many sys ops want is having replicas started > without their knowledge. > > And on top of that, I have doubts about the model. Having AWS > elastically spin up a new replica is a heavyweight operation from > Solr's perspective. I mean this potentially copies a many G set of > index files from one place to another which could take a long time, > is that really what's desired here? > > I have seen some folks spin up/down Solr instances based on a > schedule if they know roughly when the peak load will be, but again > there's nothing built in to handle this. > > Best, > Erick > > On Wed, Dec 9, 2015 at 10:15 AM, Ugo Matrangolo > <ugo.matrang...@gmail.com> wrote: > > Hi, > > > > I was trying to setup a SolrCloud cluster in AWS backed by an ASG (auto > > scaling group) serving a replicated collection. I have just came across a > > case when one of the Solr node became unresponsive with AWS killing it > and > > spinning a new one. > > > > Unfortunately, this new Solr node did not join as a replica of the > existing > > collection requiring human intervention to configure it as a new replica. > > > > I was wondering if there is around something that will make this process > > fully automated by detecting that a new node just joined the cluster and > > instructing it (e.g. via Collections API) to join as a replica of a given > > collection. > > > > Best > > Ugo >
