Thanks all for your answer. Thanks Jeff for clarification. Only thing I could not get is how CAS(I assume you r talking about compare and set) will help track the offset consumed within partition. But I got an good idea what you r trying to explain. Capping partition size and deleting by partition r two important point to remember. Thanks for your help.
On Sat, May 23, 2020, 6:23 PM Jeff Jirsa <jji...@gmail.com> wrote: > > Using cassandra as a queue is possible if you really really understand the > data model, but most people will do it wrong the first few times > > Cap your partition size. The times I’ve seen this done were near 10mb > partitions and used a special hook into internals to track partition size > via index offsets so they knew when to switch to the next partition. > Don’t delete records, delete partitions. > Maybe use CAS to know when to flip to the next partition. > Maybe use CAS to track your consumed offset within a partition > CQL Row level tombstones don’t matter in cassandra 3+ - they’re just point > deletes after the storage engine rewrite. > > You’re still probably better off running Kafka in the spare cpu and memory > you’d use for this. Understand it’s nontrivial to setup but it’s also > nontrivial to do this properly. > > > > On May 23, 2020, at 9:26 AM, Laxmikant Upadhyay <laxmikant....@gmail.com> > wrote: > > > Thanks you so much for quick response. I completely agree with Jeff and > Gabor that it is an anti-pattern to build queue in Cassandra. But plan is > to reuse the existing Cassandra infrastructure without any additional cost > (like kafka). > So even if the data is partioned properly (max 10mb per date ) ..so still > it will be an issue if I read the partition only once a day ? Even with > update status and don't delete the row? > > On Sat, May 23, 2020, 4:36 PM Gábor Auth <auth.ga...@gmail.com> wrote: > >> Hi, >> >> On Sat, May 23, 2020 at 4:09 PM Laxmikant Upadhyay < >> laxmikant....@gmail.com> wrote: >> >>> I think that we should avoid tombstones specially row-level so should go >>> with option-1. Kindly suggest on above or any other better approach ? >>> >> >> Why don't you use a queue implementation, like AcitiveMQ, Kafka and >> something? Cassandra is not suitable for this at all, it is anti-pattern in >> the Cassandra world. >> >> -- >> Bye, >> Auth Gábor (https://iotguru.cloud) >> >
