Thanks for the excellent advice, this was extremely helpful! Did not know about TWCS... curing a lot of headache.
Adam Am Mi., 28. Nov. 2018 um 20:47 Uhr schrieb Jeff Jirsa <jji...@gmail.com>: > Probably fine as long as there’s some concept of time in the partition key > to keep them from growing unbounded. > > Use TWCS, TTLs and something like 5-10 minute buckets. Don’t use RF=1, but > you can write at CL ONE. TWCS will largely just drop whole sstables as they > expire (especially with 3.11 and the more aggressive expiration logic there) > > > > -- > Jeff Jirsa > > > > On Nov 28, 2018, at 11:24 AM, Adam Smith <adamsmith8...@gmail.com> > wrote: > > > > Hi All, > > > > I need to use C* somehow as fluent data storage - maybe this is > different to the queue antipattern? Lots of data come in (10MB/sec/node), > remains for e.g. 1 hour and should then be evicted. It is somehow not > critical when data would occasionally disappear/get lost. > > > > Thankful for any advice! > > > > Is this nowadays possible without suffering too much from compactation? > I would not have ranged tombstones, and depending on a possible solution > only using point deletes (PK+CK). There is only one CK, could also be empty. > > > > 1) The data is usually 1 MB. Can I just update with empty data? PK + CK > would remain, but I would not carry about that. Would this create > tombstones or is equivalent to a DELETE? > > > > 2) Like 1) and later then set a TTL == small amount of data to be > deleted then? And hopefully small compactation? > > > > 3) Simply setting TTL 1h and hoping the best, because I am wrong with my > worries? > > > > 4) Any optimization strategies like setting the RF to 1? Which > compactation strategy is advised? > > > > 5) Are there any recent performance benchmarks for one of the scenarios? > > > > What else could I do? > > > > Thanks a lot! > > Adam > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org > For additional commands, e-mail: user-h...@cassandra.apache.org > >
