Re: Table Id mismatches

2022-01-25 Thread Amandeep Srivastava
That is very helpful, thank you so much, Bowen. I dropped the keyspace and created a new one (like with a new name). Things look good now. Will modify the service bootups to introduce the checks you've mentioned, and that should mitigate the issue for any new clusters we'd setup. Regards, Aman

Re: Table Id mismatches

2022-01-25 Thread Bowen Song
Ouch, all nodes are affected? That's a bit unusual... If you don't have any useful data in the table, try drop the table and then manually delete the data folder on all nodes. TBH I'm not even sure if this is going to work, but worth to give it a try as it's simple enough. Alternatively, if t

Re: Table Id mismatches

2022-01-25 Thread Amandeep Srivastava
Hi, I tried looking for the Id we got from schema table in all 18 nodes of my cluster (we maintain 9+9 hot/warm cluster), but none of the node seems to have the same Id in its data folder. Is there a mechanism in cass which would've cleaned the duplicate/redundant id table? To summarise, we have

Re: Unsolicited emails from IRONMAN Monttremblant

2022-01-25 Thread Paul Chandler
Could it be that email address is in the user group? If so your email could have triggered that automatic response, however I have not received anything after my recent emails There was an email on the dev list on 14/4/2020 that said the following, so they do seem to have had an interest in C

Re: Table Id mismatches

2022-01-25 Thread Amandeep Srivastava
Ah, was expecting concurrency to be the cause. Lucky, we caught it before we started ingesting any data in cassandra :) Thanks for your help, Bowen, let me try this. Regards, Aman On Tue, 25 Jan, 2022, 7:14 pm Bowen Song, wrote: > The usual cause of this is the CQL statement CREATE TABLE ...

Re: Table Id mismatches

2022-01-25 Thread Bowen Song
The usual cause of this is the CQL statement CREATE TABLE ... and/or CREATE TABLE ... IF NOT EXISTS were ran on two nodes in a short period of time (i.e. before the schema is fully synced between the nodes). If both tables (same name, different UUIDs) have data in them, fixing the issue without

Re: Cassandra 4.0 hanging on restart

2022-01-25 Thread Bowen Song
That would indicate the "isSuperseded(session)" call returned false. After looking at the source code, it seems the subrange incremental repair is likely causing this. Would you mind to try either subrange full repair or full range incremental repair? You may need to reset the "repairedAt" val

Unsolicited emails from IRONMAN Monttremblant

2022-01-25 Thread Bowen Song
Hello everyone, Did anyone who has been participating discussions in this mailing list receive unsolicited emails from the following email address? IRONMAN Monttremblant The emails have subject line looks like this: A ticket has been opened for your request [ TKT--] ref:#

Re: Hanging repairs in Cassandra

2022-01-25 Thread Bowen Song
TBH, 10 minutes is pretty low. That's more suitable for a web server than a database server. If it's easy to do, you may prefer to increase that on the firewall instead of tuning Cassandra. Cassandra won't be the only thing affected by it, and you may just save yourself some debugging time in t

Re: Cassandra 4.0 hanging on restart

2022-01-25 Thread Paul Chandler
Hi Bowen, Yes there are a large number of "Skipping delete of FINALIZED LocalSession” messages. We have a script that repairs ranges, stepping through the complete range in 5 days, this should create 1600 ranges over the 5 days, this runs commands like this: nodetool -h localhost -p 7199 repa