Re: Node roles vs SIP-20 Separation of Compute and Storage

With the SIP-20 proposal, there is a single copy of a shard on shared storage, and any existing or future replica of that shard (leader or not leader) accesses the same storage area. With HdfsDirectory there’s a copy on shared storage for each replica. During writes, the usual SolrCloud replicatio

Re: Node roles vs SIP-20 Separation of Compute and Storage

Don’t we already have implementation for shared storage backend using HDFS (and S3 transitively through the HDFS-S3 connectors)? On Wed, Jan 17, 2024 at 5:26 AM Ilan Ginzburg wrote: > HI, > Thanks for asking that question. > > The separation of compute and storage would be relevant for the nodes

Re: Node roles vs SIP-20 Separation of Compute and Storage

HI, Thanks for asking that question. The separation of compute and storage would be relevant for the nodes having the "data" role, i.e. nodes that host indexes. SIP-20 offers a way for these indexes to be on shared storage (S3/GCS etc) and not persisted long term on each individual node, making t

Node roles vs SIP-20 Separation of Compute and Storage

Hi All, Saw a post on the dev-mailing list about SIP-20 Separation of Compute and Storage . Trying to understand what extra features it adds when compared to configuring a solrcloud clust