Hi Paul,
I have tried running 'nodetool compact' and the situation remains the same
after I deleted the files that caused 'nodetool compact' to generate an
exception in the first place.
My concern is that if I delete some sstable sets from a directory or even if I
completely eliminate the sstables in a directory on one machine, run 'nodetool
repair' followed by 'nodetool compact', that directory remains empty. My
understanding has been that these equivalently named directories should contain
roughly the same amount of content.
Thanks,
Jason
From: Paul Fife <paulf...@gmail.com>
To: user@cassandra.apache.org; Jason Kania <jason.ka...@ymail.com>
Sent: Wednesday, June 8, 2016 12:55 PM
Subject: Re: Nodetool repair inconsistencies
Hi Jason -
Did you run a major compaction after the repair completed? Do you have other
reasons besides the number/size of sstables to believe all nodes don't have a
copy of the current data at the end of the repair operation?
Thanks,Paul
On Wed, Jun 8, 2016 at 8:12 AM, Jason Kania <jason.ka...@ymail.com> wrote:
Hi Romain,
The problem is that there is no error to share. I am focusing on the
inconsistency that when I run nodetool repair, get no errors and yet the
content in the same directory on the different nodes is vastly different. This
lack of an error is nature of my question, not the nodetool compact error.
Thanks,
Jason
From: Romain Hardouin <romainh...@yahoo.fr>
To: "user@cassandra.apache.org" <user@cassandra.apache.org>; Jason Kania
<jason.ka...@ymail.com>
Sent: Wednesday, June 8, 2016 8:30 AM
Subject: Re: Nodetool repair inconsistencies
Hi Jason,
It's difficult for the community to help you if you don't share the error
;-)What the logs said when you ran a major compaction? (i.e. the first error
you encountered)
Best,
Romain
Le Mercredi 8 juin 2016 3h34, Jason Kania <jason.ka...@ymail.com> a écrit :
I am running a 3 node cluster of 3.0.6 instances and encountered an error when
running nodetool compact. I then ran nodetool repair. No errors were returned.
I then attempted to run nodetool compact again, but received the same error so
the repair made no correction and reported no errors.
After that, I moved the problematic files out of the directory, restarted
cassandra and attempted the repair again. The repair again completed without
errors, however, no files were added to the directory that had contained the
corrupt files. So nodetool repair does not seem to be making actual repairs.
I started looking around and numerous directories have vastly different amounts
of content across the 3 nodes. There are 3 replicas so I would expect to find
similar amounts of content in the same data directory on the different nodes.
Is there any way to dig deeper into this? I don't want to be caught because
replication/repair is silently failing. I noticed that there is always an "some
repair failed" amongst the repair output but that is so completely unhelpful
and has always been present.
Thanks,
Jason
