Hi Sagar,
> But this is a known anti pattern to not use Cassandra as a queue causing > tombstones etc. > But I could not think of any other way. Does anyone have any other > suggestion so as to not delete after a pair is created I believe you could try using a fixed TTL (defined at the table level for example), then use a TWCS compaction strategy and compactions options that would efficiently manage tombstones. A colleague at The Last Pickle just wrote an article about TWCS: http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html, and there is a lot more information around, including a talk this year at the summit from Jeff who contributed with TWCS to Apache Cassandra: https://www.youtube.com/watch?v=PWtekUWCIaw. Also using a time buckets in the partition key could help making sure tombstones will be correctly removed and are not being scanned when requesting new data. Yet do not use *only* a time bucket in the partition key as it would lead to hotspots. For a given date, only one node (+ replicas) would handle the write / read load. So using "day + something else" as a partition key and "TWCS + Fixed TTLs" *could* be a good way to move forward. I would give it a try with the cassandra-stress tool that is shipped alongside Apache Cassandra and allows the use of a user defined schema. C*heers, Alain 2016-12-17 21:02 GMT+01:00 Sagar Jambhulkar <sagar.jambhul...@gmail.com>: > Hi, > Needed a suggestion for a schema query. I want to build a reconciliation > using Cassandra. Basically two or more systems send message to a > reconciliation process. The reconciliation process first does a level one > match of id's and than does complete comparison of messages. > > The best I could think of is a like a queue table with id's. My consumer > thread/s would, poll this table, create a pair and would have to delete > from this table. But this is a known anti pattern to not use Cassandra as a > queue causing tombstones etc. > But I could not think of any other way. Does anyone have any other > suggestion so as to not delete after a pair is created. Is Cassandra not > the correct technology for a recon process? > > Thanks, > Sagar >
