You know what they say: Go big or go home. Right now candidates are Cassandra itself but embedded or on the side not on the actual data clusters, zookeeper (yuck) , Kafka (which needs zookeeper, yuck) , S3 (outside service dependency, so no go. )
Jeff, Those are great patterns. ESP. Second one. Have used it several times. Cassandra is a great place to store data in transport. Rahul On Sep 10, 2018, 5:21 PM -0400, DuyHai Doan <doanduy...@gmail.com>, wrote: > Also using Calvin means having to implement a distributed monotonic sequence > as a primitive, not trivial at all ... > > > On Mon, Sep 10, 2018 at 3:08 PM, Rahul Singh <rahul.xavier.si...@gmail.com> > > wrote: > > > In response to mimicking Advanced replication in DSE. I understand the > > > goal. Although DSE advanced replication does one way, those are use cases > > > with limited value to me because ultimately it’s still a master slave > > > design. > > > > > > I’m working on a prototype for this for two way replication between > > > clusters or databases regardless of dB tech - and every variation I can > > > get to comes down to some implementation of the Calvin protocol which > > > basically verifies the change in either cluster , sequences it according > > > to impact to underlying data, and then schedules the mutation in a > > > predictable manner on both clusters / DBS. > > > > > > All that means is that I need to sequence the change before it happens so > > > I can predictably ensure it’s Scheduled for write / Mutation. So I’m > > > Back to square one: having a definitive queue / ledger separate from the > > > individual commit log of the cluster. > > > > > > > > > Rahul Singh > > > Chief Executive Officer > > > m 202.905.2818 > > > > > > Anant Corporation > > > 1010 Wisconsin Ave NW, Suite 250 > > > Washington, D.C. 20007 > > > > > > We build and manage digital business technology platforms. > > > On Sep 10, 2018, 3:58 AM -0400, Dinesh Joshi > > > <dinesh.jo...@yahoo.com.invalid>, wrote: > > > > > On Sep 9, 2018, at 6:08 AM, Jonathan Haddad <j...@jonhaddad.com> > > > > > wrote: > > > > > > > > > > There may be some use cases for it.. but I'm not sure what they are. > > > > > It might help if you shared the use cases where the extra complexity > > > > > is required? When does writing to Cassandra which then dedupes and > > > > > writes to Kafka a preferred design then using Kafka and simply > > > > > writing to Cassandra? > > > > > > > > From the reading of the proposal, it seems bring functionality similar > > > > to MySQL's binlog to Kafka connector. This is useful for many > > > > applications that want to be notified when certain (or any) rows change > > > > in the database primarily for a event driven application architecture. > > > > > > > > Implementing this in the database layer means there is a standard > > > > approach to getting a change notification stream. Downstream > > > > subscribers can then decide which notifications to act on. > > > > > > > > LinkedIn's databus is similar in functionality - > > > > https://github.com/linkedin/databus However it is for heterogenous > > > > datastores. > > > > > > > > > > On Thu, Sep 6, 2018 at 1:53 PM Joy Gao <j...@wepay.com.invalid> > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > We have a WIP design doc that goes over this idea in details. > > > > > > > > > > > > > > We haven't sort out all the edge cases yet, but would love to get > > > > > > > some feedback from the community on the general feasibility of > > > > > > > this approach. Any ideas/concerns/questions would be helpful to > > > > > > > us. Thanks! > > > > > > > > > > > > > > > Interesting idea. I did go over the proposal briefly. I concur with Jon > > > > about adding more use-cases to clarify this feature's potential > > > > use-cases. > > > > > > > > Dinesh >
