Using cassandra as a queue is possible if you really really understand the data model, but most people will do it wrong the first few times
Cap your partition size. The times I’ve seen this done were near 10mb partitions and used a special hook into internals to track partition size via index offsets so they knew when to switch to the next partition. Don’t delete records, delete partitions. Maybe use CAS to know when to flip to the next partition. Maybe use CAS to track your consumed offset within a partition CQL Row level tombstones don’t matter in cassandra 3+ - they’re just point deletes after the storage engine rewrite. You’re still probably better off running Kafka in the spare cpu and memory you’d use for this. Understand it’s nontrivial to setup but it’s also nontrivial to do this properly. > On May 23, 2020, at 9:26 AM, Laxmikant Upadhyay <laxmikant....@gmail.com> > wrote: > > > Thanks you so much for quick response. I completely agree with Jeff and > Gabor that it is an anti-pattern to build queue in Cassandra. But plan is to > reuse the existing Cassandra infrastructure without any additional cost (like > kafka). > So even if the data is partioned properly (max 10mb per date ) ..so still it > will be an issue if I read the partition only once a day ? Even with update > status and don't delete the row? > >> On Sat, May 23, 2020, 4:36 PM Gábor Auth <auth.ga...@gmail.com> wrote: >> Hi, >> >>> On Sat, May 23, 2020 at 4:09 PM Laxmikant Upadhyay >>> <laxmikant....@gmail.com> wrote: >>> I think that we should avoid tombstones specially row-level so should go >>> with option-1. Kindly suggest on above or any other better approach ? >> >> Why don't you use a queue implementation, like AcitiveMQ, Kafka and >> something? Cassandra is not suitable for this at all, it is anti-pattern in >> the Cassandra world. >> >> -- >> Bye, >> Auth Gábor (https://iotguru.cloud)
