HI Jason;
Whats in the log files at the moment jstat shows 100%. What is the activity on
the cluster & the node at the specific point in time (reads/ writes/ joins etc)
Jan/
On Wednesday, March 4, 2015 5:59 AM, Jason Wee <peich...@gmail.com> wrote:
Hi, our cassandra node using java 7 update 72 and we ran jstat on one of the
node, and notice some strange behaviour as indicated by output below. any idea
why when eden space stay the same for few seconds like 100% and 18.02% for few
seconds? we suspect such "stalling" cause timeout to our cluster.
any idea what happened, what went wrong and what could cause this?
$ jstat -gcutil 32276 1s
0.00 5.78 91.21 70.94 60.07 2657 73.437 4 0.056 73.493
0.00 5.78 100.00 70.94 60.07 2657 73.437 4 0.056 73.493 0.00
5.78 100.00 70.94 60.07 2657 73.437 4 0.056 73.493 0.00 5.78
100.00 70.94 60.07 2657 73.437 4 0.056 73.493 0.00 5.78
100.00 70.94 60.07 2657 73.437 4 0.056 73.493 0.00 5.78
100.00 70.94 60.07 2657 73.437 4 0.056 73.493 0.00 5.78
100.00 70.94 60.07 2657 73.437 4 0.056 73.493 0.00 5.78
100.00 70.94 60.07 2657 73.437 4 0.056 73.493 0.00 5.78
100.00 70.94 60.07 2657 73.437 4 0.056 73.493 0.00 5.78
100.00 70.94 60.07 2657 73.437 4 0.056 73.493 0.00 5.78
100.00 70.94 60.07 2657 73.437 4 0.056 73.493 0.00 5.78
100.00 70.94 60.07 2657 73.437 4 0.056 73.493 0.00 5.78
100.00 70.94 60.07 2657 73.437 4 0.056 73.493 0.00 5.78
100.00 70.94 60.07 2657 73.437 4 0.056 73.493 0.00 5.78
100.00 70.94 60.07 2657 73.437 4 0.056 73.493 0.00 5.78
100.00 70.94 60.07 2657 73.437 4 0.056 73.493 0.00 5.78
100.00 70.94 60.07 2657 73.437 4 0.056 73.493 0.00 4.65
29.66 71.00 60.07 2659 73.488 4 0.056 73.544 0.00 4.65 70.88
71.00 60.07 2659 73.488 4 0.056 73.544 0.00 4.65 71.58
71.00 60.07 2659 73.488 4 0.056 73.544 0.00 4.65 72.15 71.00
60.07 2659 73.488 4 0.056 73.544 0.00 4.65 72.33 71.00
60.07 2659 73.488 4 0.056 73.544 0.00 4.65 72.73 71.00 60.07
2659 73.488 4 0.056 73.544 0.00 4.65 73.20 71.00 60.07
2659 73.488 4 0.056 73.544 0.00 4.65 73.71 71.00 60.07 2659
73.488 4 0.056 73.544 0.00 4.65 73.84 71.00 60.07 2659
73.488 4 0.056 73.544 0.00 4.65 73.91 71.00 60.07 2659
73.488 4 0.056 73.544 0.00 4.65 74.18 71.00 60.07 2659
73.488 4 0.056 73.544 0.00 4.65 74.29 71.00 60.07 2659
73.488 4 0.056 73.544 0.00 4.65 74.29 71.00 60.07 2659
73.488 4 0.056 73.544 0.00 4.65 74.29 71.00 60.07 2659
73.488 4 0.056 73.544 0.00 4.65 74.29 71.00 60.07 2659
73.488 4 0.056 73.544 0.00 4.65 74.29 71.00 60.07 2659
73.488 4 0.056 73.544 0.00 4.65 74.29 71.00 60.07 2659
73.488 4 0.056 73.544 0.00 4.65 74.29 71.00 60.07 2659
73.488 4 0.056 73.544 0.00 5.43 12.64 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 18.02 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 18.02 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 18.02 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 18.02 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 18.02 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 18.02 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 18.02 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 18.02 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 18.02 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 18.02 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 18.02 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 18.02 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 18.02 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 18.02 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 18.02 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 18.02 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 18.02 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 69.24 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 78.05 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 78.97 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 79.07 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 79.18 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 80.09 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 80.36 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 80.51 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 80.70 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 81.31 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 81.63 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 82.94 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 82.94 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 82.94 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 82.94 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 82.94 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 82.94 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 5.43 82.94 71.09 60.07 2661
73.534 4 0.056 73.590 0.00 9.37 65.64 71.21 60.07 2663
73.583 4 0.056 73.639 0.00 3.49 11.89 71.25 60.10 2665
73.623 4 0.056 73.679 0.00 3.49 66.11 71.25 60.10 2665
73.623 4 0.056 73.679 14.26 0.00 9.91 71.25 60.10 2666
73.649 4 0.056 73.705 14.26 0.00 73.25 71.25 60.10 2666
73.649 4 0.056 73.705 0.00 10.49 14.43 71.29 60.10 2667
73.675 4 0.056 73.731 0.00 10.49 66.07 71.29 60.10 2667
73.675 4 0.056 73.731 0.00 10.49 100.00 71.29 60.10 2667
73.675 4 0.056 73.731 0.00 10.49 100.00 71.29 60.10 2667
73.675 4 0.056 73.731 0.00 10.49 100.00 71.29 60.10 2667
73.675 4 0.056 73.731 0.00 10.49 100.00 71.29 60.10 2667
73.675 4 0.056 73.731 0.00 10.49 100.00 71.29 60.10 2667
73.675 4 0.056 73.731 0.00 10.49 100.00 71.29 60.10 2667
73.675 4 0.056 73.731 0.00 10.49 100.00 71.29 60.10 2667
73.675 4 0.056 73.731 0.00 10.49 100.00 71.29 60.10 2667
73.675 4 0.056 73.731 0.00 10.49 100.00 71.29 60.10 2667
73.675 4 0.056 73.731 0.00 10.49 100.00 71.29 60.10 2667
73.675 4 0.056 73.731 0.00 10.49 100.00 71.29 60.10 2667
73.675 4 0.056 73.731 0.00 10.49 100.00 71.29 60.10 2667
73.675 4 0.056 73.731 0.00 10.49 100.00 71.29 60.10 2667
73.675 4 0.056 73.731 0.00 10.49 100.00 71.29 60.10 2667
73.675 4 0.056 73.731 0.00 10.49 100.00 71.29 60.10 2667
73.675 4 0.056 73.731
thanks.
jason
