Re: CDC usability and future development

2018-02-01 Thread Jay Zhuang
We did a POC to improve CDC feature as an interface ( https://github.com/ngcc/ngcc2017/blob/master/CassandraDataIngestion.pdf), so the user doesn't have to read the commit log directly. We deployed the change to a test cluster and doing more tests for production traffics, will send out the design p

Re: CDC usability and future development

2018-01-31 Thread Josh McKenzie
> > CDC provides only the mutation as opposed to the full column value, which > tends to be of limited use for us. Applications might want to know the full > column value, without having to issue a read back. We also see value in > being able to publish the full column value both before and after t

Re: CDC usability and future development

2018-01-30 Thread Jeff Jirsa
Here's a deck of some proposed additions, discussed at one of the NGCC sessions last fall: https://github.com/ngcc/ngcc2017/blob/master/CassandraDataIngestion.pdf On Tue, Jan 30, 2018 at 5:10 PM, Andrew Prudhomme wrote: > Hi all, > > We are currently designing a system that allows our Cassand

CDC usability and future development

2018-01-30 Thread Andrew Prudhomme
Hi all, We are currently designing a system that allows our Cassandra clusters to produce a stream of data updates. Naturally, we have been evaluating if CDC can aid in this endeavor. We have found several challenges in using CDC for this purpose. CDC provides only the mutation as opposed to the

CDC usability and future development

2018-01-30 Thread Andrew Prudhomme
Hi all, We are currently designing a system that allows our Cassandra clusters to produce a stream of data updates. Naturally, we have been evaluating if CDC can aid in this endeavor. We have found several challenges in using CDC for this purpose. CDC provides only the mutation as opposed to the