Re: Ultra wide row anti pattern

2014-02-04 Thread Edward Capriolo
You could use another column of CAS as a management layer. You only have to consult it when picking up new rows. On Tue, Feb 4, 2014 at 3:45 PM, DuyHai Doan wrote: > Great idea for implementing queue pattern. Thank you Edward. > > However with your design there are still corner cases for 2 cons

Re: Ultra wide row anti pattern

2014-02-04 Thread DuyHai Doan
Great idea for implementing queue pattern. Thank you Edward. However with your design there are still corner cases for 2 consumers to read from the same queue. Reading and writing with QUORUM does not prevent race conditions. I believe the new CAS feature of C* 2.0 might be useful here but with th

Re: Ultra wide row anti pattern

2014-02-04 Thread Edward Capriolo
Generally you need to make a wide row because the row keys in cassandra are ordered by their md5/murmer code. As a result you have no way of locating "new rows", but if the row name is predictable the columns inside the row are ordered. On Tue, Feb 4, 2014 at 12:02 PM, Yogi Nerella wrote: > Sor

Re: Ultra wide row anti pattern

2014-02-04 Thread Yogi Nerella
Sorry, I am not understanding the problem, and I am new to Cassandra, and want to understand this issue. Why do we need to use wide row for this situation, why not a simple table in cassandra? todolist (user, state) ==> is there any other information in this table which needs for processing to

Re: Ultra wide row anti pattern

2014-02-04 Thread Edward Capriolo
I have actually been building something similar in my space time. You can hang around and wait for it or build your own. Here is the basics. Not perfect but it will work. Create column family queue with gc_grace_period=[1 day] set queue [timeuuid()] ["z"+timeuuid()] = [ work do do] The producer

Re: Ultra wide row anti pattern

2014-01-31 Thread DuyHai Doan
Thanks Nat for your ideas. >This could be as simple as adding year and month to the primary key (in the form >'mm'). Alternatively, you could add this in the partition in the definition. Either way, it >then becomes pretty easy to re-generate these based on the query parameters. The thing is

Re: Ultra wide row anti pattern

2014-01-31 Thread Nate McCall
> > > The only drawback for ultra wide row I can see is point 1). But if I use > leveled compaction with a sufficiently large value for "sstable_size_in_mb" > (let's say 200Mb), will my read performance be impacted as the row grows ? > For this use case, you would want to use SizeTieredCompaction

Re: Ultra wide row anti pattern

2014-01-31 Thread DuyHai Doan
Durable write has been removed from the entire keyspace already. I'll run a bench on a 24*10⁶ wide row and give feedback soon On Fri, Jan 31, 2014 at 7:55 PM, Robert Coli wrote: > On Fri, Jan 31, 2014 at 6:52 AM, DuyHai Doan wrote: > >> 4) hard limit of 2*10⁹ columns per "physical" row >>

Re: Ultra wide row anti pattern

2014-01-31 Thread Robert Coli
On Fri, Jan 31, 2014 at 6:52 AM, DuyHai Doan wrote: > 4) hard limit of 2*10⁹ columns per "physical" row > b. maximum number of items to be processed is 24*10⁶, far below the hard > limit of 2*10⁹ columns so point 4) does not apply either > Before discarding this point, try writing an example

Ultra wide row anti pattern

2014-01-31 Thread DuyHai Doan
Hello all I've read some materials on the net about Cassandra anti patterns, among which is mentionned the very large wide-row anti pattern. The main rationale to avoid too wide rows are: 1) fragmentation of data on multiple SStables when the row is very wide, leading to very slow reads by sl