Re: new version of Cassandra-stress doesn't support random read benchmark?

2015-05-25 Thread graham sanderson
Phone > >> On May 25, 2015, at 2:34 AM, graham sanderson wrote: >> >> Hey Benedict; >> >> I screwed up on email after a bachelor party, and sent something to external >> cassandra-users not internal users (drunken drivel) >> >> I never said

Re: new version of Cassandra-stress doesn't support random read benchmark?

2015-05-24 Thread graham sanderson
Hey Benedict; I screwed up on email after a bachelor party, and sent something to external cassandra-users not internal users (drunken drivel) I never said anything about it because I hoped no one noticed it. That said, I was wondering if my data was helpful for your injector post. We haven’t

Re: DateTieredCompactionStrategy and static columns

2015-05-01 Thread graham sanderson
porting Inner joins if the partition key is the same in both >> tables. >>> >>> I'd rather see join support personally :) >>> >>> Jon >>> >>> On Fri, May 1, 2015 at 6:38 AM graham sanderson wrote: >>> >>>> I 100%

Re: DateTieredCompactionStrategy and static columns

2015-05-01 Thread graham sanderson
. If we do, we should definitely keep > it all inline. If not, it probably permits a lot better behaviour to > separate them, since it's easier to reason about and improve their distinct > characteristics. > > > On Fri, May 1, 2015 at 1:24 AM, graham sanderson wrote: > >>

Re: DateTieredCompactionStrategy and static columns

2015-04-30 Thread graham sanderson
. > On Apr 30, 2015, at 7:13 PM, Jonathan Haddad wrote: > > If you want it in a separate sstable, just use a separate table. There's > nothing that warrants making the codebase more complex to accomplish > something it already does. > > On Thu, Apr 30, 2015 at 5:07

Fwd: DateTieredCompactionStrategy and static columns

2015-04-30 Thread graham sanderson
th an sstable that's > almost entirely TTL'ed with a few static columns that will never get > compacted or dropped. Pretty much the worst scenario I can think of. > > > > On Thu, Apr 30, 2015 at 11:21 AM graham sanderson <mailto:gra...@vast.com>> wrote: >

Re: Cassandra Schema Change Performance Improvement

2014-07-24 Thread graham sanderson
Just as a general observation, there is a third possible solution depending on the number of columns involved per K1 (which looks reasonable based on your numbers) but also on how much contention there is for updates to particular (K1, K2) values (which are always unique partitions in your optio

Hinted Handoff/Atomic Updates & SnapTree replacement with BTree

2014-07-12 Thread graham sanderson
Running on 2.0.5 we’ve noticed occasional MASSIVE spikes in memory allocation that happen under write load, but way out of proportion to the amount of data actually being written. My suspicion is the problem is related to hinted handoff, and basically follows the “some sort of non trivial GC pa

Static columns & when is a feature mature enough for a change to be a regression

2014-07-02 Thread graham sanderson
There is no regression here yet, however static columns turn out to be insanely useful (perhaps for reasons other than originally intended), so I am curious what the general approach is to changes in what is essentially a very new feature. In this context, I am showing something that works as i

Re: static columns and TTL - wouldn't it be nice if static columns played nice with a partition whose partition keys have all (TTL) expired

2014-06-23 Thread graham sanderson
absence of any column names prefixed by the partition key On Jun 24, 2014, at 12:26 AM, graham sanderson wrote: > Note, that as I think about it, if you had the new OnDiskAtom time with TTL > and no value, then you wouldn’t need anything special about static columns, > you’d just need a C

Re: static columns and TTL - wouldn't it be nice if static columns played nice with a partition whose partition keys have all (TTL) expired

2014-06-23 Thread graham sanderson
Note, that as I think about it, if you had the new OnDiskAtom time with TTL and no value, then you wouldn’t need anything special about static columns, you’d just need a CQL syntax to update/set the TTL for a column which might be useful for lots of things. On Jun 24, 2014, at 12:22 AM, graham

static columns and TTL - wouldn't it be nice if static columns played nice with a partition whose partition keys have all (TTL) expired

2014-06-23 Thread graham sanderson
So, I was thinking about a new use case, where an ideal situation would be to have something like CREATE TABLE series { id uuid, inserted timeuuid, small_thing blob, large_static_thing blob static, PRIMARY KEY (id, inserted) } So this is my first use of st

Re: CMS GC / fragmentation / memtables etc

2014-06-15 Thread graham sanderson
memtable under offheap_objects (again dependent on > workload, but as much as 8x in extreme cases) which means more queries are > answerable from memtable, and write amplification is reduced accordingly, > improving write throughput. > On 15 Jun 2014 13:32, "graham sanderson" wrot

Re: CMS GC / fragmentation / memtables etc

2014-06-15 Thread graham sanderson
), so I don't expect this change will be > introduced until zero-copy offheap memtables are introduced, which have > been shelved for the moment. > > > On 15 Jun 2014 10:53, "graham sanderson" wrote: > >> Hi Benedict, >> >> So I had a look at t

Re: CMS GC / fragmentation / memtables etc

2014-06-15 Thread graham sanderson
ile a ticket for that, although > it won't be back ported to 2.0. > > > On 21 May 2014 00:57, graham sanderson wrote: > >> So i’ve been tinkering a bit with CMS config because we are still seeing >> fairly frequent full compacting GC due to framgentation/promotion

Re: CMS GC / fragmentation / memtables etc

2014-05-21 Thread graham sanderson
of objects at the same time, however any time slabs are allocated >> under load, we end up promoting them with whatever other live stuff in eden >> is still there. If we only do this once and reuse the slabs, we are likely >> to minimize our promotion problem later (at least for th

CMS GC / fragmentation / memtables etc

2014-05-20 Thread graham sanderson
in eden is still there. If we only do this once and reuse the slabs, we are likely to minimize our promotion problem later (at least for these large objects) On May 16, 2014, at 9:37 PM, graham sanderson wrote: > Excellent - thank you… > > On May 16, 2014, at 7:08 AM, Samuel CA

Re: Things that are about 1M big

2014-05-16 Thread graham sanderson
Excellent - thank you… On May 16, 2014, at 7:08 AM, Samuel CARRIERE wrote: > Hi, > This is arena allocation of memtables. See here for more infos : > http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-0-performance > > > > > De :graham

Things that are about 1M big

2014-05-16 Thread graham sanderson
So just throwing this out there for those for whom this might ring a bell. I’m debugging some CMS memory fragmentation issues on 2.0.5 - and interestingly enough most of the objects giving us promotion failures are of size 131074 (dwords) - GC logging obviously doesn’t say what those are, but I’

Re: IDEA code style settings

2014-02-23 Thread graham sanderson
thanks On Feb 22, 2014, at 6:06 PM, Jonathan Ellis wrote: > I've attached mine to https://wiki.apache.org/cassandra/CodeStyle. > > On Fri, Feb 21, 2014 at 5:37 PM, graham sanderson wrote: >> Does anyone have IDEA already configured with Cassandra coding style >>

IDEA code style settings

2014-02-21 Thread graham sanderson
Does anyone have IDEA already configured with Cassandra coding style (without using the eclipse formatting plugin)… If so could they please export them and attach? Thanks, Graham P.S. otherwise I’ll go thru and set them up, and attach myself for others to use. smime.p7s Description: S/MIME c

Error reporting

2014-02-06 Thread graham sanderson
I’ve seen a few JIRA issues here and there, but haven’t heard much about any goal towards more context in error reporting. For almost all of the errors we see in production, it would be useful to include KS/CF in the error (and from the code that information is available in most cases). We have

Re: can test/Test.iml be removed?

2013-12-14 Thread graham sanderson
Thank you. On Dec 14, 2013, at 9:57 AM, Jonathan Ellis wrote: > Done > > On Sat, Dec 14, 2013 at 11:28 AM, graham sanderson wrote: >> This seemed to get introduced (by mistake?) in ef33f9543 > > > > -- > Jonathan Ellis > Project Chair, Ap

can test/Test.iml be removed?

2013-12-13 Thread graham sanderson
This seemed to get introduced (by mistake?) in ef33f9543 smime.p7s Description: S/MIME cryptographic signature

Re: Issue with SizeTieredCompactionStrategy in 2.0.3

2013-12-13 Thread graham sanderson
as how it's off by default, the default max_cold_reads_ratio is 0.0, > so filterColdSSTables() won't actually filter anything. I can add a check > to skip that function if maxColdReadsRatio is 0.0 in the 6483 patch. > > > On Thu, Dec 12, 2013 at 7:00 PM, graham sanders

Issue with SizeTieredCompactionStrategy in 2.0.3

2013-12-12 Thread graham sanderson
I just created https://issues.apache.org/jira/browse/CASSANDRA-6483 for an issue introduced it seems by https://issues.apache.org/jira/browse/CASSANDRA-6109 Note that the latter feature claims to be “off” by default, however it isn’t immediately clear to me from the patch how that “off” is impl