What does your schema look like? Are you seeing any warnings or errors in the
server log?
Dinesh
On Tuesday, March 13, 2018, 3:43:33 PM PDT, Faraz Mateen <fmat...@an10.io>
wrote:
Hi everyone,
I seem to have hit a problem in which writing to cassandra through a python
script fails and also occasionally causes cassandra node to crash. Here are the
details of my problem.
I have a python based streaming application that reads data from kafka at a
high rate and pushes it to cassandra through datastax's cassandra driver for
python. My cassandra setup consists of 3 nodes and a replication factor of 2.
Problem is that my python application crashes after writing ~12000 records with
the following error:
Exception: Error from server: code=1100 [Coordinator node timed out waiting for
replica nodes' responses] message="Operation timed out - received only 0
responses." info={'received_responses':
0, 'consistency': 'LOCAL_ONE', 'required_responses': 1}Sometimes the python
application crashes with this traceback:
cassandra.OperationTimedOut: errors={'10.128.1.1': 'Client request timeout. See
Session.execute[_async](timeou t)'}, last_host=10.128.1.1With the error above,
one of the cassandra node crashes as well. When I look at cassandra system logs
(/var/log/cassandra/system.log ), I see the following exception:
https://gist.github.com/ farazmateen/ e7aa5749f963ad2293f8be0ca1ccdc 22/
e3fd274af32c20eb9f534849a31734 dcd33745b4
According to the suggestion in post linked below, I have set my JVM Heap size
to 8GB but the problem still persists.:
https://dzone.com/articles/dia gnosing-and-fixing-cassandra- timeouts
Cluster:
- Cassandra version 3.9
- 3 nodes, with 8 cores and 30GB of RAM each.
- Keyspace has a replication factor of 2.
- Write consistency is LOCAL_ONE
- MAX HEAP SIZE is set to 8GB.
Any help will be greatly appreciated.
--
Faraz
