Also using Calvin means having to implement a distributed monotonic sequence as a primitive, not trivial at all ...
On Mon, Sep 10, 2018 at 3:08 PM, Rahul Singh <rahul.xavier.si...@gmail.com> wrote: > In response to mimicking Advanced replication in DSE. I understand the > goal. Although DSE advanced replication does one way, those are use cases > with limited value to me because ultimately it’s still a master slave > design. > > I’m working on a prototype for this for two way replication between > clusters or databases regardless of dB tech - and every variation I can get > to comes down to some implementation of the Calvin protocol which basically > verifies the change in either cluster , sequences it according to impact to > underlying data, and then schedules the mutation in a predictable manner on > both clusters / DBS. > > All that means is that I need to sequence the change before it happens so > I can predictably ensure it’s Scheduled for write / Mutation. So I’m > Back to square one: having a definitive queue / ledger separate from the > individual commit log of the cluster. > > > Rahul Singh > Chief Executive Officer > m 202.905.2818 > > Anant Corporation > 1010 Wisconsin Ave NW, Suite 250 > <https://maps.google.com/?q=1010+Wisconsin+Ave+NW,+Suite+250+%0D%0AWashington,+D.C.+20007&entry=gmail&source=g> > Washington, D.C. 20007 > > We build and manage digital business technology platforms. > On Sep 10, 2018, 3:58 AM -0400, Dinesh Joshi <dinesh.jo...@yahoo.com.invalid>, > wrote: > > On Sep 9, 2018, at 6:08 AM, Jonathan Haddad <j...@jonhaddad.com> wrote: > > There may be some use cases for it.. but I'm not sure what they are. It > might help if you shared the use cases where the extra complexity is > required? When does writing to Cassandra which then dedupes and writes to > Kafka a preferred design then using Kafka and simply writing to Cassandra? > > > From the reading of the proposal, it seems bring functionality similar to > MySQL's binlog to Kafka connector. This is useful for many applications > that want to be notified when certain (or any) rows change in the database > primarily for a event driven application architecture. > > Implementing this in the database layer means there is a standard approach > to getting a change notification stream. Downstream subscribers can then > decide which notifications to act on. > > LinkedIn's databus is similar in functionality - > https://github.com/linkedin/databus However it is for heterogenous > datastores. > > On Thu, Sep 6, 2018 at 1:53 PM Joy Gao <j...@wepay.com.invalid> wrote: > >> >> >> We have a* WIP design doc >> <https://wepayinc.box.com/s/fmdtw0idajyfa23hosf7x4ustdhb0ura>* that goes >> over this idea in details. >> >> We haven't sort out all the edge cases yet, but would love to get some >> feedback from the community on the general feasibility of this approach. >> Any ideas/concerns/questions would be helpful to us. Thanks! >> >> > Interesting idea. I did go over the proposal briefly. I concur with Jon > about adding more use-cases to clarify this feature's potential use-cases. > > Dinesh > >
