Yep. The "trouble" with FIOs is that they almost completely remove your disk throughput problems, so then you're constrained by CPU. Concurrent compactors and concurrent writes are two params that come to mind but there are likely others.
@kevin. I hear you. 5TB is sort of a maximum that DataStax has been recommending per an instance with SSDs (there's lots of factors at play though; YMMV and I encourage you to test it out). I would also note that the more RAM you can get for Cassandra, the better. Newer version of C* store caches, indexes, and memtables in native memory and many of those are proportional to the size of the datasets on the box. 64GB would probably work for around 1TB without row caching, but if you have a lot of indexes or column families 128GB is not unreasonable. Otherwise, your proposed ideal config would be...ideal. Chris On Thu, Nov 6, 2014 at 3:05 PM, jeeyoung kim <jeeyou...@gmail.com> wrote: > I've been running with FIOs and we've been CPU bound most of the time. But > I'm not using native transport yet, and is hoping that it would make things > faster. > > On Thu, Nov 6, 2014 at 12:54 PM, Christopher Brodt <ch...@uberbrodt.net> > wrote: > >> You should get pretty great performance with those FusionIO cards. One >> thing I watch out for whenever scaling Cassandra vertically is compaction >> times, which probably won't matter here. However, you have to take into >> account that you lose some resiliency to failures with less nodes. >> >> On Thu, Nov 6, 2014 at 2:48 PM, Russ Bradberry <rbradbe...@gmail.com> >> wrote: >> >>> I've heard of people running dense nodes (8+ TB) using fusion I/O, but >>> with 10GBe connections. I mean why buy a Ferrari and never leave first gear? >>> >>> As far as saturating the network goes, I guess that all depends on your >>> workload, and how often you need to repair. >>> >>> Sent from my iPhone >>> >>> On Nov 6, 2014, at 3:40 PM, Kevin Burton <bur...@spinn3r.com> wrote: >>> >>> We’re looking at switching data centers and they’re offering pretty >>> aggressive pricing on boxes with fusion IO cards. >>> >>> 2x 1.2TB Fusion IO >>> 128GB RAM >>> 20 cores. >>> >>> now.. this isn’t the typical cassandra box. Most people are running >>> multiple nodes to scale out vs scale vertically. But these boxes are >>> priced aggressively and honestly I think that cassandra would be able to >>> saturate the gigabit ethernet port on these machines. >>> >>> so it *might* be that these are TOO powerful in a way. >>> >>> Curious if others are running in this config and what tuning options >>> were required to get it to work. >>> >>> Kevin >>> >>> -- >>> >>> Founder/CEO Spinn3r.com >>> Location: *San Francisco, CA* >>> blog: http://burtonator.wordpress.com >>> … or check out my Google+ profile >>> <https://plus.google.com/102718274791889610666/posts> >>> <http://spinn3r.com> >>> >>> >> > > > -- > Jeeyoung Kim > http://kimjeeyoung.com/ > >
