Re: Incremental repair for the first time

2017-01-09 Thread Oskar Kjellin
There is no harm in running it tho. If it's not needed it will simply terminate. Better to be safe Sent from my iPhone > On 9 Jan 2017, at 08:13, Amit Singh F wrote: > > Hi , > > Generally Upgradesstables are only recommended when you plan to move with > Major version like from 2.0 to 2.1

Re: Incremental repair for the first time

2017-01-09 Thread Kathiresan S
Thanks Amit & Oskar Thanks, Kathir On Mon, Jan 9, 2017 at 3:23 AM, Oskar Kjellin wrote: > There is no harm in running it tho. If it's not needed it will simply > terminate. Better to be safe > > Sent from my iPhone > > On 9 Jan 2017, at 08:13, Amit Singh F wrote: > > Hi , > > > > Generally Upg

Re: Help

2017-01-09 Thread Edward Capriolo
On Sun, Jan 8, 2017 at 11:30 PM, Anshu Vajpayee wrote: > Gossip shows - all nodes are up. > > But when we perform writes , coordinator stores the hints. It means - > coordinator was not able to deliver the writes to few nodes after meeting > consistency requirements. > > The nodes for which wr

Re: Help

2017-01-09 Thread Chris Lohfink
Do you have any monitoring setup around garbage collections? A GC + network latency > write timeout will cause intermittent hints. On Sun, Jan 8, 2017 at 10:30 PM, Anshu Vajpayee wrote: > Gossip shows - all nodes are up. > > But when we perform writes , coordinator stores the hints. It means

collecting metrics at column-family level

2017-01-09 Thread Jacob Shadix
Is it possible to report on the following metrics at table level? 1. count of number of records inserted within given timeframe 2. data growth 3. write latency -- Jacob Shadix

Re: collecting metrics at column-family level

2017-01-09 Thread Jacob Shadix
I found a couple of metrics that should suffice for #2,3 - - > TBL: Live Disk Used & TBL: Local Write Latency. 1. count of number of records inserted within given timeframe 2. data growth 3. write latency -- Jacob Shadix On Mon, Jan 9, 2017 at 2:29 PM, Jacob Shadix wrote: > Is it poss

Trying to identify the cause of these errors.

2017-01-09 Thread Gopal, Dhruva
My colleague (Richard Ney) has already been in touch with you on a couple of other issues we’ve seen in the past. Our development been trying to track down some new issues we’ve been seeing on one of our pre-prod environments where we’ve been having consistent failures very often (every day or e

Re: Cassandra cluster performance

2017-01-09 Thread Branislav Janosik -T (bjanosik - AAP3 INC at Cisco)
Hi we have made some changes to our code and benchmarking and now it seems to have the scalability. Async writes plus the changes made the difference. So for now, thank you very much everyone for help. Very appreciated. Branislav From: Jonathan Haddad Reply-To: "user@cassandra.apache.org" Da

Strange issue wherein cassandra not being started from cron

2017-01-09 Thread Ajay Garg
Hi All. Facing a very weird issue, wherein the command */etc/init.d/cassandra start* causes cassandra to start when the command is run from command-line. However, if I put the above as a cron job ** * * * * /etc/init.d/cassandra start* cassandra never starts. I have checked, and "cron" se

Re: Strange issue wherein cassandra not being started from cron

2017-01-09 Thread Bhuvan Rawal
Hi Ajay, Have you had a look at cron logs? - mine is in path /var/log/cron Thanks & Regards, On Tue, Jan 10, 2017 at 9:45 AM, Ajay Garg wrote: > Hi All. > > Facing a very weird issue, wherein the command > > */etc/init.d/cassandra start* > > causes cassandra to start when the command is run fr

Incremental Repair Migration

2017-01-09 Thread Amit Singh F
Hi All, We are thinking of migrating from primary range repair (-pr) to incremental repair. Environment : * Cassandra 2.1.16 * 25 Node cluster , * RF 3 * Data size up to 450 GB per nodes We found that running full repair will be taking around 8 hrs per node which means 20

Re: Strange issue wherein cassandra not being started from cron

2017-01-09 Thread Jonathan Haddad
Last I checked, cron doesn't load the same, full environment you see when you log in. Also, why put Cassandra on a cron? On Mon, Jan 9, 2017 at 9:47 PM Bhuvan Rawal wrote: > Hi Ajay, > > Have you had a look at cron logs? - mine is in path /var/log/cron > > Thanks & Regards, > > On Tue, Jan 10, 20

Re: Incremental Repair Migration

2017-01-09 Thread Jonathan Haddad
Your best bet is to just run repair constantly. We maintain an updated fork of Spotify's reaper tool to help manage it: https://github.com/thelastpickle/cassandra-reaper On Mon, Jan 9, 2017 at 10:04 PM Amit Singh F wrote: > Hi All, > > > > We are thinking of migrating from primary range repair (-

OutOfMemoryError in startup process

2017-01-09 Thread Yuji Ito
Hi all, I got OutOfMemoryError in startup process as below. I have 3 questions about the error. 1. Why did Cassandra built by myself cause OutOfMemory errors? OutOfMemory errors happened in startup process in some (not all) nodes on Cassandra 2.2.8 which I got from github and built by myself. Ho

RE: Incremental Repair Migration

2017-01-09 Thread Amit Singh F
Hi Jonathan, Really appreciate your response. It will not be possible for us to move to Reaper as of now, we are in process to migrate to Incremental repair. Also Running repair constantly will be costly affair in our case . For migrating to incremental repair with large set of dataset will ta