Re: Mixing incremental repair with sequential

2015-06-26 Thread Carl Hu
Alain, The reduction of compaction is having significant impact lowering response time, especially at the 90th percentile level, for us. For the record, we are using AWS's i2.2xl instance types (these are ssd). We were running compaction_throughput_mb_per_sec at 18. Now we are running at 10. Late

Re: Mixing incremental repair with sequential

2015-06-26 Thread Alain RODRIGUEZ
Here is something I wrote some time ago: http://planetcassandra.org/blog/interview/video-advertising-platform-teads-chose-cassandra-spm-and-opscenter-to-monitor-a-personalized-ad-experience/ Monitoring absolutely necessary to understand what is happening in the system. There is no magic in there

Re: Mixing incremental repair with sequential

2015-06-26 Thread Carl Hu
Thank you, Alain, for the response. We're using 2.1 indeed. I've lowered compaction threshhold from 18 to 10mb/s. Will see what happens. > I hope you have a monitoring tool up and running and an easy way to detect errors on your logs. We do not have this. What do you use for this? Thank you, Ca

Re: Mixing incremental repair with sequential

2015-06-26 Thread Alain RODRIGUEZ
"It is not possible to mix sequential repair and incremental repairs." I guess that is a system limitation, even if I am not sure of it (I don't have used C*2.1 yet) I would focus on tuning your repair by : - Monitoring performance / logs (see why the cluster hangs) - Use range repairs (as a work

Mixing incremental repair with sequential

2015-06-26 Thread Carl Hu
Dear colleagues, We are using incremental repair and have noticed that every few repairs, the cluster experiences pauses. We run the repair with the following command: nodetool repair -par -inc I have tried to run it not in parallel, but get the following error: "It is not possible to mix sequen