Yes, I would add the 6 new nodes, then decommission the 6 original nodes, then
upgrade to 5.0.
Remember to change the seed node values before you decommission the old nodes.
Thanks
Paul
> On 30 Jan 2025, at 14:42, Luciano Greiner <luciano.grei...@gmail.com> wrote:
>
> Ok, so you mean I setup these new 6 nodes as 4.1.3, then just upgrade
> the software, correct?
>
> Thank you!
>
> Luciano
>
>
> On Thu, Jan 30, 2025 at 6:57 AM Paul Chandler <p...@redshots.com> wrote:
>>
>> Hi Luciano,
>>
>> The problem occurs due to Cassandra 5 making changes to the system tables,
>> so the cluster will be in schema mismatch during the upgrade process, until
>> all the nodes are on 5.0
>>
>> Normally this would not be a problem, as the system tables are not
>> replicated anyway, but, as you are finding, it will stop you adding, or
>> removing nodes during an upgrade process.
>>
>> Given this, it maybe best to split this operation, firstly to provision the
>> new nodes with 4.1.3, then upgrade the nodes to Cassandra 5.
>>
>> Thanks
>>
>> Paul
>>
>>
>>
>>> On 30 Jan 2025, at 02:38, Luciano Greiner <luciano.grei...@gmail.com> wrote:
>>>
>>> Hello.
>>>
>>> I am upgrading a small Cassandra 4.1.3 cluster (2 sites, 3 nodes each)
>>> to Cassandra 5.
>>>
>>> Given we're using an old Centos OS in our nodes, I decided to get new
>>> nodes provisioned in the cluster (with version 5 on compatibility
>>> mode), and then decommission the old nodes when all is completed.
>>>
>>> There is an issue about schema agreement when I try to bootstrap the
>>> first Cassandra 5 node into the cluster:
>>>
>>> WARN [main] 2025-01-30 01:55:40,012
>>> DefaultSchemaUpdateHandler.java:131 - There are nodes in the cluster
>>> with a different schema version than us, from which we did not merge
>>> schemas: our version: (59adb24e-f3cd-3e02-97f0-5b395827453f),
>>> outstanding versions -> endpoints:
>>> {9bd8dc68-76f1-3b55-b60a-bb69ea16e53a=[/10.1.9.10:7000,
>>> /10.1.9.11:7000, /10.2.8.20:7000, /10.1.9.13:7000, /10.2.8.24:7000,
>>> /10.2.8.25:7000]}. Use -Dcassandra.skip_schema_check=true to ignore
>>> this, -Dcassandra.skip_schema_check_for_endpoints=<ep1[,epN]> to skip
>>> specific endpoints, or
>>> -Dcassandra.skip_schema_check_for_versions=<ver1[,verN]> to skip
>>> specific schema versions
>>> ERROR [main] 2025-01-30 01:55:40,012
>>> DefaultSchemaUpdateHandler.java:142 - Didn't receive schemas for all
>>> known versions within the PT30S. Use
>>> -Dcassandra.skip_schema_check=true to skip this check.
>>> ERROR [main] 2025-01-30 01:55:40,014 CassandraDaemon.java:887 -
>>> Exception encountered during startup
>>> java.lang.IllegalStateException: Could not achieve schema readiness in PT30S
>>>
>>> So I ran the describecluster. All 6 nodes are using the same schema
>>> version 9bd8dc68-76f1-3b55-b60a-bb69ea16e53a. Still, I've gone ahead
>>> and ran resetlocalschema and restarted each node, one at a time. No
>>> luck.
>>>
>>> Then I tried to skip the schema check, but then I get several table
>>> not found errors raised on the new node logs until it eventually shuts
>>> down.
>>>
>>> I am running out of ideas here.
>>>
>>> Does anybody have any thoughts?
>>>
>>> Thank you
>>>
>>> Luciano
>>
