A row is TTLed once all its columns are TTLed. If you want a row to be TTLed at
once just set the same TTL on all its columns.
________________________________
From: Charulata Sharma (charshar) <chars...@cisco.com>
Sent: Friday, March 23, 2018 9:52:28 PM
To: user@cassandra.apache.org
Subject: Re: Using Spark to delete from Transactional Cluster
Yes agree on “let really old data expire” . However, I could not find a way to
TTL an entire row. Only columns can be TTLed.
Charu
From: Rahul Singh <rahul.xavier.si...@gmail.com>
Reply-To: "user@cassandra.apache.org" <user@cassandra.apache.org>
Date: Friday, March 23, 2018 at 1:45 PM
To: "user@cassandra.apache.org" <user@cassandra.apache.org>,
"user@cassandra.apache.org" <user@cassandra.apache.org>
Subject: Re: Using Spark to delete from Transactional Cluster
I think there are better ways to leverage parallel processing than to use it to
delete data. As I said , it works for one of my projects for the same exact
reason you stated : business rules.
Deleting data is an old way of thinking. Why not store the data and just use
the relevant data .. let really old data expire ..
--
Rahul Singh
rahul.si...@anant.us
Anant Corporation
On Mar 23, 2018, 11:38 AM -0700, Charulata Sharma (charshar)
<chars...@cisco.com>, wrote:
Hi Rahul,
Thanks for your answer. Why do you say that deleting from spark is not
elegant?? This is the exact feedback I want. Basically why is it not elegant?
I can either delete using delete prepared statements or through spark. TTL
approach doesn’t work for us
Because first of all ttl is there at a column level and there are business
rules for purge which make the TTL solution not very clean in our case.
Thanks,
Charu
From: Rahul Singh <rahul.xavier.si...@gmail.com>
Reply-To: "user@cassandra.apache.org" <user@cassandra.apache.org>
Date: Thursday, March 22, 2018 at 5:08 PM
To: "user@cassandra.apache.org" <user@cassandra.apache.org>,
"user@cassandra.apache.org" <user@cassandra.apache.org>
Subject: Re: Using Spark to delete from Transactional Cluster
Short answer : it works. You can even run “delete” statements from within Spark
once you know which keys to delete. Not elegant but it works.
It will create a bunch of tombstones and you may need to spread your deletes
over days. Another thing to consider is instead of deleting setting a TTL which
will eventually get cleansed.
--
Rahul Singh
rahul.si...@anant.us
Anant Corporation
On Mar 22, 2018, 2:19 PM -0500, Charulata Sharma (charshar)
<chars...@cisco.com>, wrote:
Hi,
Wanted to know the community’s experiences and feedback on using Apache
Spark to delete data from C* transactional cluster.
We have spark installed in our analytical C* cluster and so far we have been
using Spark only for analytics purposes.
However, now with advanced features of Spark 2.0, I am considering using
spark-cassandra connector for deletes instead of a series of Delete Prepared
Statements
So essentially the deletes will happen on the analytical cluster and they will
be replicated over to transactional cluster by means of our keyspace
replication strategies.
Are there any risks involved in this ??
Thanks,
Charu
