Write performance expectations...

2013-02-13 Thread kadey
Hello, New member here, and I have (yet another) question on write performance. I'm using Apache Cassandra version 1.1, Python 2.7 and Pycassa 1.7. I have a cluster of 2 datacenters, each with 3 nodes, on AWS EC2 using EBS and the RandomPartioner. I'm writing to a column family in a keyspace

Re: Write performance expectations...

2013-02-13 Thread kadey
I'm not using multi-threads/processes. I'll try multi-threading to see if I get a boost. Thanks. Ken - Original Message - From: "Tyler Hobbs" To: user@cassandra.apache.org Sent: Wednesday, February 13, 2013 11:06:30 AM Subject: Re: Write performance expectations... 2500

Re: Write performance expectations...

2013-02-14 Thread kadey
> Is there a particular reason for you to use EBS ? Instance Store are > recommended because they improve performances by reducing the I/O throttling. > You should give the EC2 instance type (m1.xlarge / m1.large / ...) if you > want some feedback about the 2500 w/s, and also give the mean si

Re: Write performance expectations...

2013-02-14 Thread kadey
Using multithreading, inserting 2000 per thread, resulted in no throughput increase. Each thread is taking about 4 seconds per, indicating a bottleneck elsewhere. Ken - Original Message - From: "Tyler Hobbs" To: user@cassandra.apache.org Sent: Wednesday, February 13, 2013 11:

Re: Write performance expectations...

2013-02-14 Thread kadey
Alain, I found out that the client node is an m1.small, and the cassandra nodes are m1.large. This is what is contained in each row: {dev1-dc1r-redir-0.unica.net/B9tk: {batchID: 2486272}}. Not a whole lot of data. If you don't use EBS, how is data persistence then maintained in the event

Re: data model advice needed

2013-02-27 Thread kadey
One possibility would be to use dynamic columns, with each column name being a composite made from a timestamp, and the value of each containing serialized json of the details. The host could be the key. Then you could slice the data by column name. Ken - Original Message - Fro