Should we use Materialised Views or ditch them ?

2020-02-28 Thread Tobias Eriksson
Hi A debate has surfaced in my company, whether to keep or remove Materialized Views The Datastax FAQ says sure thing, go ahead and use it https://docs.datastax.com/en/dse/5.1/cql/cql/cql_using/faqMV.html But know the limitations https://docs.datastax.com/en/dse/5.1/cql/cql/cql_using/knownLimitat

Deleting Compaction Strategy for Cassandra 3.0?

2020-02-28 Thread Oleksandr Shulgin
Hi, We have a task where we would need to remove roughly 25% of the data from the SSTables that shouldn't be there anymore. The rows to be removed can be identified by the first component of its partitioning key, e.g. remove all rows where type='A' or type='B'. The 2nd component can be anything

Re: Should we use Materialised Views or ditch them ?

2020-02-28 Thread Max C.
The general view of the community is that you should *NOT* use them in production, due to multiple serious outstanding issues (see Jira). We used them quite a bit when they first came out and have since rolled back all uses except for the absolute most basic cases (ex: a table with 30K rows th

Re: Should we use Materialised Views or ditch them ?

2020-02-28 Thread Jon Haddad
I also recommend avoiding them. I've seen too many clusters fall over as a result of their usage. On Fri, Feb 28, 2020 at 9:52 AM Max C. wrote: > The general view of the community is that you should *NOT* use them in > production, due to multiple serious outstanding issues (see Jira). We used

Re: Should we use Materialised Views or ditch them ?

2020-02-28 Thread Tobias Eriksson
It is interresting, cause when they arrived we started using them, but then there surfaced some posts on issues, even Datastax seemed to indicate that you shouldn’t use them But now lately, from the articles/blogposts below, it seems like that is not the case anymore…. I wonder if there has been

Re: Should we use Materialised Views or ditch them ?

2020-02-28 Thread Erick Ramirez
Personally, I think MVs are still experimental and not ready for primetime. It works for some but if you run into issues, fixing them have a huge impact to your application. For example if the view updates get too far behind, there's no effective way to resolve them other than having to drop the MV

Re: Deleting Compaction Strategy for Cassandra 3.0?

2020-02-28 Thread Erick Ramirez
I'm not personally aware of anyone who is using it successfully other than ProtectWise where it was a good fit for their narrow use case. My limited knowledge of it is that it has some sharp edges which is the reason they haven't pushed for it to be added to Cassandra (that's second hand info so pl

Re: Deleting Compaction Strategy for Cassandra 3.0?

2020-02-28 Thread Oleksandr Shulgin
On Fri, 28 Feb 2020, 23:02 Erick Ramirez, wrote: > I'm not personally aware of anyone who is using it successfully other > than ProtectWise where it was a good fit for their narrow use case. My > limited knowledge of it is that it has some sharp edges which is the reason > they haven't pushed for

Re: Deleting Compaction Strategy for Cassandra 3.0?

2020-02-28 Thread Jeff Jirsa
If you’re really really advanced you MIGHT be able to use spark + cqlsstablewriter to create a ton of sstables with just tombstones one them representing deletes, then either nodetool refresh or sstableloader them into the cluster If you create sstables on the right timestamp boundaries to ma