Greetings,
I am working on the downgradesstables code and seem to have a problem with
ordering of the downgrade or perhaps the Directories.SSTableLister
I lifted the code from upgradesstables to select the files to downgrade.
The only difference in the code that selects the files to downgrade is the
actual selection of the file. There is no change to the ordering of the
files that are evaluated for inclusion. Yet I think the downgrade ordering
is incorrect.
My process is to start 3.1 version to create the tables and then use the
4.0 code base to run the standaloneupgrader and then the
standalonedowngrader
When running the standaloneupgrader on system local I see the following
{{noformat}}
Found 3 sstables that need upgrading.
Upgrading
BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/ma-1-big-Data.db')
Upgrade of
BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/ma-1-big-Data.db')
complete.
Upgrading
BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/ma-2-big-Data.db')
Upgrade of
BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/ma-2-big-Data.db')
complete.
Upgrading
BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/ma-3-big-Data.db')
Upgrade of
BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/ma-3-big-Data.db')
complete.
{{noformat}}
when running the standalonedowngrader is see
{{noformat}}
Found 3 sstables that need downgrading.
Downgrading
BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/nb-6-big-Data.db')
Downgrade of
BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/nb-6-big-Data.db')
complete.
Downgrading
BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/nb-4-big-Data.db')
Downgrade of
BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/nb-4-big-Data.db')
complete.
Downgrading
BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/nb-5-big-Data.db')
Downgrade of
BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/nb-5-big-Data.db')
complete.
{{noformat}}}
Note the order of the generations in the downgrader (I have seen similar
out of order issues with the upgrader, but infrequently)
The difference between the upgrader and downgrader code in the questionable
section (
https://github.com/Claudenw/cassandra/blob/CASSANDRA-8928/src/java/org/apache/cassandra/tools/StandaloneDowngrader.java#:~:text=new%20ArrayList%3C%3E()%3B-,//%20Downgrade%20sstables,%7D,-int%20numSSTables%20%3D)
is on line 101 where the files are selected and put into a list. I think
this means that the Directories.SSTableLister on occasion returns files in
the incorrect order during a call to lister.list().entrySet()
I believe that the files are processed in the order specified and that the
generations get switched around. This is evidenced by the file size of the
Data file associated with the generations as it moves through the process.
In this case we expect the nb-6 to become ma-7 as per the output from the
run. In actuality we want nb-6 to be nb-9.
{{noformat}}
ma nb ma
1 212 4 212 8 212
2 64 5 70 9 70
3 4876 6 4883 7 4883
{{noformat}}
So now my question, has anyone seen the behaviour before?
Oh, to make things more interesting I am using Docker images of 3.1 and a
modified 4.0 that turns off the execution so I can just run the upgrade and
downgrade.
Any help would be appreciated,
Claude
