What replication factor are you using? Even if your writes use CL.ONE, Cassandra will be attempting writes to the replica nodes in the background.
Are your writes "token aware"? If not, the receiving node has the overhead of forwarding the request to the node that owns the token for the primary key. For the record, Cassandra is not designed and optimized for so-called "fat nodes". The design focus is "commodity hardware" and "distributed cluster" (typically a dozen or more nodes.) That said, it would be good if we had a rule of thumb for how many simultaneous requests a node can handle, both external requests and inter-node traffic. I think there is an open Jira to enforce a limit on inflight requests so that nodes don't overloaded and start failing in the middle of writes as you seem to be seeing. -- Jack Krupansky On Sun, Dec 6, 2015 at 9:29 AM, jerry <xutom2...@126.com> wrote: > Dear All, > > Now I have a 4 nodes Cassandra cluster, and I want to know the highest > performance of my Cassandra cluster. I write a JAVA client to batch insert > datas into ALL 4 nodes Cassandra, when I start less than 30 subthreads in > my client applications to insert datas into cassandra, it will be ok for > everything, but when I start more than 80 or 100 subthreads in my client > applications, there will be too much timeout Exceptions (Such as: Cassandra > timeout during write query at consistency ONE (1 replica were required but > only 0 acknowledged the write)). And no matter how many subthreads or even > I start multiple clients with multiple subthreads on different computers, I > can get the highest performance for about 60000 - 80000 TPS. By the way, > each row I insert into cassandra is about 130 Bytes. > My 4 nodes of Cassandra is : > CPU: 4*15 > Memory: 512G > Disk: flash card (only one disk but better than SSD) > My cassandra configurations are: > MAX_HEAP_SIZE: 60G > NEW_HEAP_SIZE: 40G > > When I insert datas into my cassandra cluster, each nodes has NOT > reached bottleneck such as CPU or Memory or Disk. Each of the three main > hardwares is idlećSo I think maybe there is something wrong about my > configuration of cassandra cluster. Can somebody please help me to My > Cassandra Tuning? Thanks in advances! >
