Hi, There are no transient nodes. All nodes are the same. If you have transient replication enabled each node will transiently replicate some ranges instead of fully replicating them.
Capacity requirements are reduced evenly across all nodes in the cluster. Nodes are not temporarily transient replicas during expansion. They need to stream data like a full replica for the transient range before they can serve reads. There is a pending state similar to how there is a pending state for full replicas. Transient replicas also always receive writes when they are pending. There may be some room to relax how that is handled, but for now we opt to send pending transient ranges a bit more data and avoid reading from them when maybe we could. This doesn't change how expansion works with vnodes. The same restrictions still apply. We won't officially support vnodes until we have done more testing and really thought through the corner cases. It's quite possible we will relax the restriction on creating transient keyspaces with vnodes in 4.0.x. Ariel On Fri, Aug 31, 2018, at 2:07 PM, Carl Mueller wrote: > I put these questions on the ticket too... Sorry if some of them are > stupid. > > So are (basically) these transient nodes basically serving as centralized > hinted handoff caches rather than having the hinted handoffs cluttering up > full replicas, especially nodes that have no concern for the token range > involved? I understand that hinted handoffs aren't being replaced by this, > but is that kind of the idea? > > Are the transient nodes "sitting around"? > > Will the transient nodes have cheaper/lower hardware requirements? > > During cluster expansion, does the newly streaming node acquiring data > function as a temporary transient node until it becomes a full replica? > Likewise while shrinking, does a previously full replica function as a > transient while it streams off data? > > Can this help vnode expansion with multiple concurrent nodes? Admittedly > I'm not familiar with how much work has gone into fixing cluster expansion > with vnodes, it is my understanding that you typically expand only one node > at a time or in multiples of the datacenter size > > On Mon, Aug 27, 2018 at 12:29 PM Ariel Weisberg <ar...@weisberg.ws> wrote: > > > Hi all, > > > > I wanted to give everyone an update on how development of Transient > > Replication is going and where we are going to be as of 9/1. Blake > > Eggleston, Alex Petrov, Benedict Elliott Smith, and myself have been > > working to get TR implemented for 4.0. Up to now we have avoided merging > > anything related to TR to trunk because we weren't 100% sure we were going > > to make the 9/1 deadline and even minimal TR functionality requires > > significant changes (see 14405). > > > > We focused on getting a minimal set of deployable functionality working, > > and want to avoid overselling what's going to work in the first version. > > The feature is marked explicitly as experimental and has to be enabled via > > a feature flag in cassandra.yaml. The expected audience for TR in 4.0 is > > more experienced users who are ready to tackle deploying experimental > > functionality. As it is deployed by experienced users and we gain more > > confidence in it and remove caveats the # of users it will be appropriate > > for will expand. > > > > For 4.0 it looks like we will be able to merge TR with support for normal > > reads and writes without monotonic reads. Monotonic reads require blocking > > read repair and blocking read repair with TR requires further changes that > > aren't feasible by 9/1. > > > > Future TR support would look something like > > > > 4.0.next: > > * vnodes (https://issues.apache.org/jira/browse/CASSANDRA-14404) > > > > 4.next: > > * Monotonic reads ( > > https://issues.apache.org/jira/browse/CASSANDRA-14665) > > * LWT (https://issues.apache.org/jira/browse/CASSANDRA-14547) > > * Batch log (https://issues.apache.org/jira/browse/CASSANDRA-14549) > > * Counters (https://issues.apache.org/jira/browse/CASSANDRA-14548) > > > > Possibly never: > > * Materialized views > > > > Probably never: > > * Secondary indexes > > > > The most difficult changes to support Transient Replication should be > > behind us. LWT, Batch log, and counters shouldn't be that hard to make > > transient replication aware. Monotonic reads require some changes to the > > read path, but are at least conceptually not that hard to support. I am > > confident that by 4.next TR will have fewer tradeoffs. > > > > If you want to take a peek the current feature branch is > > https://github.com/aweisberg/cassandra/tree/14409-7 although we will be > > moving to 14409-8 to rebase on to trunk. > > > > Regards, > > Ariel > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > For additional commands, e-mail: dev-h...@cassandra.apache.org > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
