Re: Migrating to CQL and Non Compact Storage

2016-04-11 Thread Jack Krupansky
At this point I have no additional advice to offer. There seems to be intense resistance to follow the modeling approach I have recommended, so there is nothing more I can offer on that front. The bottom line is that if the techniques referenced in the blog post are not sufficient, then nothing sho

Re: Migrating to CQL and Non Compact Storage

2016-04-11 Thread Jim Ancona
On Mon, Apr 11, 2016 at 4:19 PM, Jack Krupansky wrote: > Some of this may depend on exactly how you are using so-called COMPACT > STORAGE. I mean, if your tables really are modeled as all but exactly one > column in the primary key, then okay, COMPACT STORAGE may be a reasonable > model, but that

Re: Migrating to CQL and Non Compact Storage

2016-04-11 Thread Jack Krupansky
Some of this may depend on exactly how you are using so-called COMPACT STORAGE. I mean, if your tables really are modeled as all but exactly one column in the primary key, then okay, COMPACT STORAGE may be a reasonable model, but that seems to be a very special, narrow use case, so for all other ca

Re: Migrating to CQL and Non Compact Storage

2016-04-11 Thread Anuj Wadehra
Thanks Jim. I think you understand the pain of migrating TBs of data to new tables. There is no command to change from compact to non compact storage and the fastest solution to migrate data using Spark is too slow for production systems. And the pain gets bigger when your performance dips after

Re: Migrating to CQL and Non Compact Storage

2016-04-11 Thread Anuj Wadehra
Thanks Jack.Let me rephrase and try to fetch some help :) Cql tables always have schema but Thrift allowed you to have a cf with mix of statically declared columns in schema and dynamic columns i.e. columns not part of schema and created as and when needed at runtime. When you drop Thrift code a

Re: Migrating to CQL and Non Compact Storage

2016-04-11 Thread Jim Ancona
Jack, the Datastax link he posted ( http://www.datastax.com/dev/blog/thrift-to-cql3) says that for column families with mixed dynamic and static columns: "The only solution to be able to access the column family fully is to remove the declared columns from the thrift schema altogether..." I think t

Re: Migrating to CQL and Non Compact Storage

2016-04-11 Thread Jack Krupansky
Sorry, but your message is too confusing - you say "reading dynamic columns in CQL" and "make the table schema less", but neither has any relevance to CQL! 1. CQL tables always have schemas. 2. All columns in CQL are statically declared (even maps/collections are statically declared columns.) Grant

Re: Migrating to CQL and Non Compact Storage

2016-04-11 Thread Anuj Wadehra
Any comments or suggestions on this one?  ThanksAnuj Sent from Yahoo Mail on Android On Sun, 10 Apr, 2016 at 11:39 PM, Anuj Wadehra wrote: Hi We are on 2.0.14 and Thrift. We are planning to migrate to CQL soon but facing some challenges. We have a cf with a mix of statically defined colum