Re: Using CDC Feature to Stream C* to Kafka (Design Proposal)

2018-09-11 Thread Joy Gao
2018 at 8:38 PM Joy Gao wrote: > Thank you all for the feedback so far. > > The immediate use case for us is setting up a real-time streaming data > pipeline from C* to our Data Warehouse (BigQuery), where other teams can > access the data for reporting/analytics/ad-hoc query. W

Re: Using CDC Feature to Stream C* to Kafka (Design Proposal)

2018-09-11 Thread Joy Gao
Kafka connector. This is useful for many applications >> that want to be notified when certain (or any) rows change in the database >> primarily for a event driven application architecture. >> >> Implementing this in the database layer means there is a standard >> approach to

Using CDC Feature to Stream C* to Kafka (Design Proposal)

2018-09-06 Thread Joy Gao
Hi all, We are fairly new to Cassandra. We began looking into the CDC feature introduced in 3.0. As we spent more time looking into it, the complexity began to add up (i.e. duplicated mutation based on RF, out of order mutation, mutation does not contain full row of data, etc). These limitations h

CDC and TTL

2018-06-18 Thread Joy Gao
Hi all! I recently started to look into Cassandra CDC implementation. One question that occurred to me is how/if TTL is handled for CDC. For example, If I insert some data with TTL enabled and expiring in 60 seconds, will CDC be aware of these changes 60 seconds later when the TTL expired? If not,