Re: Deleting Compaction Strategy for Cassandra 3.0?
On Sat, Feb 29, 2020 at 8:49 AM Jeff Jirsa wrote: > If you’re really really advanced you MIGHT be able to use spark + > cqlsstablewriter to create a ton of sstables with just tombstones one them > representing deletes, then either nodetool refresh or sstableloader them > into the cluster > > If you create sstables on the right timestamp boundaries to match your > twcs windows, each one will compact with the data file or the same window > and delete the data. > > Will be a ton of compaction though. Not as efficient as the deleting > strategy. Also not sure if the offline cqlsstablewriter actually supports > deletes because I’m on my phone and too lazy to check. If it doesn’t it > probably wouldn’t be that hard to add. > Yeah, even if that would work with the CQLSSTableWriter, the ton of user-defined compaction is what we would like to avoid. We are OK with rewriting all files once, though. Assuming, we get it running on our server version: do I get it right that running `nodetool upgradesstables -a` is going to rewrite all the SSTable files subject to the defined compaction strategy? -- Alex
Deleting data from future
Greetings, In our C* cluster, one node lost time sync and it went to future(16 Mar 2020) for a while. After fixing timesync, we couldnt update or delete records which were inserted while node's system time was from the future. Inspecting sstables, we found that timestamp(TS field value) of these records was 1584349956844022, Monday, 16 March 2020 09:12:36.844. We neither delete these records nor truncate table. Is there anyway to manipulate records inside sstable manually?
Re: Deleting Compaction Strategy for Cassandra 3.0?
> On Mar 2, 2020, at 2:02 AM, Oleksandr Shulgin > wrote: > > >> On Sat, Feb 29, 2020 at 8:49 AM Jeff Jirsa wrote: > >> If you’re really really advanced you MIGHT be able to use spark + >> cqlsstablewriter to create a ton of sstables with just tombstones one them >> representing deletes, then either nodetool refresh or sstableloader them >> into the cluster >> >> If you create sstables on the right timestamp boundaries to match your twcs >> windows, each one will compact with the data file or the same window and >> delete the data. >> >> Will be a ton of compaction though. Not as efficient as the deleting >> strategy. Also not sure if the offline cqlsstablewriter actually supports >> deletes because I’m on my phone and too lazy to check. If it doesn’t it >> probably wouldn’t be that hard to add. > > Yeah, even if that would work with the CQLSSTableWriter, the ton of > user-defined compaction is what we would like to avoid. We are OK with > rewriting all files once, though. > > Assuming, we get it running on our server version: do I get it right that > running `nodetool upgradesstables -a` is going to rewrite all the SSTable > files subject to the defined compaction strategy? You don’t need to do user defined compaction here As soon as the data files are on the server, the next time TWCS looks for compaction candidates (e.g. next flush, so “nodetool flush”), it’ll find all of the extra sstables and start putting them into the right windows. Note that you have to have the sstables lined up properly - when you build them, they must stop on the right timestamp boundaries or this doesn’t work. You can try a day at a time though - process all of the deletes for one time window and load them in. (Again, presumes this works with the cqlsstablewriter which I haven’t looked at in years)
Re: Deleting data from future
You can issue a delete using a future timestamp. http://cassandra.apache.org/doc/latest/cql/dml.html#grammar-token-update-parameter Look for USING TIMESTAMP. Jon On Mon, Mar 2, 2020, 3:28 AM Furkan Cifci wrote: > Greetings, > In our C* cluster, one node lost time sync and it went to future(16 Mar > 2020) for a while. > After fixing timesync, we couldnt update or delete records which were > inserted while node's system time was from the future. > Inspecting sstables, we found that timestamp(TS field value) of these > records was 1584349956844022, Monday, 16 March 2020 09:12:36.844. > We neither delete these records nor truncate table. > Is there anyway to manipulate records inside sstable manually? > > > > >
