[
https://issues.apache.org/jira/browse/CASSANDRA-3640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13170485#comment-13170485
]
Edward Capriolo commented on CASSANDRA-3640:
--------------------------------------------
I did happen to capture the snitch information from another node at the time of
the event.
/10.71.71.51=3.15
/10.71.74.30=26.88
/10.71.71.62=1.67
/10.71.71.66=5.19
/10.71.71.73=3.76
/10.71.71.76=0.68
/10.71.74.34=1.66
/10.71.71.63=2.42
/10.71.71.72=0.82
/10.71.71.59=3.44
/10.71.74.33=1.21
/10.71.71.64=1.21
/10.71.71.60=2.19
/10.71.71.71=1.75
/10.71.74.32=106.55
/10.71.71.54=86.69
/10.71.71.53=5.14
/10.71.74.27=5.93
/10.71.74.31=3.11
/10.71.71.69=1.15
/10.71.71.56=2.73
/10.71.74.37=2.16
/10.71.71.70=2.85
/10.71.71.58=0.77
/10.71.71.55=5.83
/10.71.74.38=1.14
/10.71.74.35=3.61
/10.71.71.68=0.81
/10.71.71.67=0.69
/10.71.71.74=3.64
/10.71.71.57=1.21
/10.71.71.52=2.37
/10.71.71.65=6.78
/10.71.71.61=2.8
/10.71.71.75=4.12
/10.71.74.36=3.22
These are the two systems that I show as getting a lot of IO
/10.71.74.29=1.64
/10.71.74.28=3.49
We are doing mostly READ.ONE with a low read repair chance. ring is balanced
and data/node is the same. badness_threshold is 0.2
No user load is hitting these two machines directly. Thus it is hard for me to
understand how dynamic snitch is routing so much traffic to these machines that
they are more burdened then other nodes. As I understand the dynamic snitch
these machines should be the least burdened ones.
> Dynamic Snitch does not compute scores if no direct reads hit the node.
> -----------------------------------------------------------------------
>
> Key: CASSANDRA-3640
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3640
> Project: Cassandra
> Issue Type: Improvement
> Affects Versions: 0.8.7
> Reporter: Edward Capriolo
> Priority: Minor
>
> We can into an interesting situation. We added 2 nodes to our cluster.
> Strangely these nodes were performing worse then other nodes. They had more
> IOwait for example. The impact was not major but it was noticeable. Later I
> determined that these Cassandra node were not in our client's list of nodes
> and our clients do not auto discover. I confirmed that the hosts did not have
> any scores inside it's dynamic snitch.
> It is counter intuitive that a node receiving less or no direct user requests
> would perform worse then others. I am not sure of the dynamic that caused
> this.
> I understand that DSnitch is supposed to have it's own view of the world,
> maybe it could share information with neighbours. Again this is more of a
> client configuration issue then a direct Cassandra issue, but I found it
> interesting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
