Thanks Alain for the help. I will give these options a try. On Dec 18, 2016 10:01 PM, "Alain RODRIGUEZ" <arodr...@gmail.com> wrote:
> Hi Sagar, > > >> But this is a known anti pattern to not use Cassandra as a queue causing >> tombstones etc. >> But I could not think of any other way. Does anyone have any other >> suggestion so as to not delete after a pair is created > > > I believe you could try using a fixed TTL (defined at the table level for > example), then use a TWCS compaction strategy and compactions options that > would efficiently manage tombstones. A colleague at The Last Pickle just > wrote an article about TWCS: http://thelastpickle. > com/blog/2016/12/08/TWCS-part1.html, and there is a lot more information > around, including a talk this year at the summit from Jeff who contributed > with TWCS to Apache Cassandra: https://www.youtube.com/watch?v=PWtekUWCIaw > . > > Also using a time buckets in the partition key could help making sure > tombstones will be correctly removed and are not being scanned when > requesting new data. Yet do not use *only* a time bucket in the partition > key as it would lead to hotspots. For a given date, only one node (+ > replicas) would handle the write / read load. > > So using "day + something else" as a partition key and "TWCS + Fixed TTLs" > *could* be a good way to move forward. > > I would give it a try with the cassandra-stress tool that is shipped > alongside Apache Cassandra and allows the use of a user defined schema. > > C*heers, > > Alain > > 2016-12-17 21:02 GMT+01:00 Sagar Jambhulkar <sagar.jambhul...@gmail.com>: > >> Hi, >> Needed a suggestion for a schema query. I want to build a reconciliation >> using Cassandra. Basically two or more systems send message to a >> reconciliation process. The reconciliation process first does a level one >> match of id's and than does complete comparison of messages. >> >> The best I could think of is a like a queue table with id's. My consumer >> thread/s would, poll this table, create a pair and would have to delete >> from this table. But this is a known anti pattern to not use Cassandra as a >> queue causing tombstones etc. >> But I could not think of any other way. Does anyone have any other >> suggestion so as to not delete after a pair is created. Is Cassandra not >> the correct technology for a recon process? >> >> Thanks, >> Sagar >> > >
