Hello,
We are evaluating Cassandra performance and would like to know if the
numbers we've gotten can be improved upon. Right now we're just testing a
simple key/value store using the cassandra-stress tool. A quick summary of
our setup:
* Using the datastax 2.2 distribution, Oracle HotSpot Java 1.8 default
config
except as noted below
* 3 node cluster, each node with a 3.3GHz Xeon, 4 cores w/ 2 hyperthreads
each, 32G memory and large SSDs
* stress.yaml files run with cassandra-stress on other machine(s) as
described
below
First, vanilla cassandra-stress run like so
cassandra-stress write duration=10m cl=QUORUM -mode native cql3 \
-schema "replication(factor=3)" -node 10.40.252.5 -rate threads=700
-errors ignore
gives 98k ops/s, latency median 5.1, max 887, and
cassandra-stress read duration=10m cl=QUORUM -mode native cql3 \
-schema "replication(factor=3)" -node 10.40.252.5 -rate threads=700
-errors ignore
gives 105k ops/s, latency median 6.3, max 184.3.
Next, we run with our "user" profile. The results are as follows, where
"small" denotes 5B keys/2B values and "medium" denotes 128B keys/512B
values.
one test machine:
small: consistency QUORUM insert, 900 threads, 120k ops/s
small: consistency QUORUM read, 800 threads, 132k ops/s
small: consistency ONE insert, 600 threads, 185k ops/s
small: consistency ONE read, 900 threads, 194k ops/s
medium: consistency QUORUM insert, 550 threads, 89k ops/s
medium: consistency QUORUM read, 880 threads, 55k ops/s
two test machines, total number threads listed:
medium: consistency ONE insert, 800 threads, 108k ops/s
medium: consistency ONE read, 1200 threads, 136k ops/s
Do these numbers seem reasonable? Is there any low hanging fruit that would
change them dramatically? Thanks!
More details: the stress.yaml user configuration is below. The numbers in
the
"columnspec" key change with the above key/value sizes, but that's the only
change.
---
keyspace: smstress
keyspace_definition: |
CREATE KEYSPACE smstress WITH replication = {'class': 'SimpleStrategy',
'replication_factor': 3};
table: kvtestsmkey
table_definition: |
CREATE TABLE kvtestsmkey (
key blob PRIMARY KEY,
value blob
) WITH COMPACT STORAGE;
columnspec:
- name: key
size: fixed(5)
- name: value
size: fixed(2)
insert:
partitions: fixed(1)
select: fixed(1)/1
batchtype: UNLOGGED
queries:
read:
cql: select * from kvtestsmkey where key = ? LIMIT 1
fields: samerow
---
The default config is used except for the following changes:
* concurrent_reads and _writes is set to 64,
* memtable_allocation_type is set to offheap_objects,
* memtable_flush_writers is set to 8,
* request timeouts have been increased somewhat
cassandra-stress is run like this, mutatis mutandis, for user profiles:
cassandra-stress user duration=10m cl=ONE profile=stress.yaml
ops\(insert=1\) \
-mode native cql3 -node (ip address) -rate threads=(number) -errors
ignore
I have system data that seem to indicate that net and disk IO are not
bottlenecks.
Thanks very much for your help!
--
Sean Bowman
Software Developer, Epoch Labs
