Using TTL for data purge
Hi, I'm looking for suggestions/caveats on using TTL as a subsitute for a manual data purge job. We have few tables that hold user information - this could be guest or registered users, and there could be between 500K to 1M records created per day per table. Currently, these tables have a secondary indexed updated_date column which is populated on each update. However, we have been getting timeouts when running queries using updated_date when the number of records are high, so i don't think this would be a reliable option in the long term when we need to purge records that have not been used for the last X days. In this scenario, is it advisable to include a high enough TTL (i.e the amount of time we want these to last, could be 3 to 6 months) when inserting/updating records? There could be cases where the TTL may get reset after couple of days/weeks, when the user visits the site again. The tables have fixed number of columns, except for one which has a clustering key, and may have max 10 entries per partition key. I need to know the overhead of having so many rows with TTL hanging around for a relatively longer duration (weeks/months), and the impacts it could have on performance/storage. If this is not a recommended approach, what would be an alternate design which could be used for a manual purge job, without using secondary indices. We are using Cassandra 2.0.x. Thanks, Joseph
CQL Composite Key Seen After Table Creation
Hello,
Defining the “column_metadata” after table creation vs during table
creation seems to affect the cql schema. We are running a large
high-throughput Cassandra 2.1.10 cluster. During table creation, the client
Cassandra-cli was used with the following commands:
“create column family Test with comparator = UTF8Type and
key_validation_class=UTF8Type;”
“update column family Test with column_metadata = [ {column_name: title,
validation_class: UTF8Type}]”
Though these commands had the desirable effect in thrift, in cql it causes
the composite key ((key), column1) to be seen, where column1 is the “title”
value. Despite the composite key issue in cql, the underlying data on disk
is correct. This article seemed to best describe what might be happening
on the backend: http://www.datastax.com/dev/blog/thrift-to-cql3. Instead
of recreating the table with an explicit column definition, which would
look like this:
create column family Test with comparator = UTF8Type
and key_validation_class=UTF8Type
and column_metadata = [ {column_name: title, validation_class: UTF8Type}]
Is there a way to repair the cql schema to recognize that the table is not
dynamic, ultimately removing the composite key?
Thanks,
Amir
Cassandra Java Driver
Hi, Which Java Driver is suited for Cassandra 2.2.x. ? I see datastax 3.0.0 beta1 and datastax 2.2.0 rc3... Are they suited for production? Is there anything better? Thanks for your comments and replies? Jean
Is CQLSSTableWriter tied to C* version?
Hi, Can sstables created by CQLSSTableWriter in cassandra-all.jar 2.1.12 be loaded into C* 2.2.4? Or they have to be on the same version?
Re: Is CQLSSTableWriter tied to C* version?
The streaking format is directly tied to the sstable format. So, in general, if the format changes between versions, you can't stream. I don't think the format changed between these 2 versions, but I'm typing this on my phone and can't verify. On Tue, Dec 22, 2015 at 6:36 PM Kai Wang wrote: > Hi, > > Can sstables created by CQLSSTableWriter in cassandra-all.jar 2.1.12 be > loaded into C* 2.2.4? Or they have to be on the same version? >
Re: Is CQLSSTableWriter tied to C* version?
Jon, Thanks. So I will just update to use cassandra-all.jar 2.2.4 to be sure. On Wed, Dec 23, 2015 at 12:10 AM, Jonathan Haddad wrote: > The streaking format is directly tied to the sstable format. So, in > general, if the format changes between versions, you can't stream. I don't > think the format changed between these 2 versions, but I'm typing this on > my phone and can't verify. > > On Tue, Dec 22, 2015 at 6:36 PM Kai Wang wrote: > >> Hi, >> >> Can sstables created by CQLSSTableWriter in cassandra-all.jar 2.1.12 be >> loaded into C* 2.2.4? Or they have to be on the same version? >> >
