> I think this means that the Directories.SSTableLister on occasion returns
> files in the incorrect order during a call to lister.list().entrySet()
This seems easy enough to verify by looping it and examining the results.
Kind Regards,
Brandon
On Tue, Jan 10, 2023 at 4:44 AM Claude Warren, Jr via dev
<dev@cassandra.apache.org> wrote:
>
> Greetings,
>
> I am working on the downgradesstables code and seem to have a problem with
> ordering of the downgrade or perhaps the Directories.SSTableLister
>
> I lifted the code from upgradesstables to select the files to downgrade. The
> only difference in the code that selects the files to downgrade is the actual
> selection of the file. There is no change to the ordering of the files that
> are evaluated for inclusion. Yet I think the downgrade ordering is incorrect.
>
> My process is to start 3.1 version to create the tables and then use the 4.0
> code base to run the standaloneupgrader and then the standalonedowngrader
>
> When running the standaloneupgrader on system local I see the following
> {{noformat}}
> Found 3 sstables that need upgrading.
> Upgrading
> BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/ma-1-big-Data.db')
> Upgrade of
> BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/ma-1-big-Data.db')
> complete.
> Upgrading
> BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/ma-2-big-Data.db')
> Upgrade of
> BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/ma-2-big-Data.db')
> complete.
> Upgrading
> BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/ma-3-big-Data.db')
> Upgrade of
> BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/ma-3-big-Data.db')
> complete.
> {{noformat}}
>
> when running the standalonedowngrader is see
> {{noformat}}
> Found 3 sstables that need downgrading.
> Downgrading
> BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/nb-6-big-Data.db')
> Downgrade of
> BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/nb-6-big-Data.db')
> complete.
> Downgrading
> BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/nb-4-big-Data.db')
> Downgrade of
> BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/nb-4-big-Data.db')
> complete.
> Downgrading
> BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/nb-5-big-Data.db')
> Downgrade of
> BigTableReader(path='/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/nb-5-big-Data.db')
> complete.
> {{noformat}}}
>
> Note the order of the generations in the downgrader (I have seen similar out
> of order issues with the upgrader, but infrequently)
>
> The difference between the upgrader and downgrader code in the questionable
> section
> (https://github.com/Claudenw/cassandra/blob/CASSANDRA-8928/src/java/org/apache/cassandra/tools/StandaloneDowngrader.java#:~:text=new%20ArrayList%3C%3E()%3B-,//%20Downgrade%20sstables,%7D,-int%20numSSTables%20%3D)
> is on line 101 where the files are selected and put into a list. I think
> this means that the Directories.SSTableLister on occasion returns files in
> the incorrect order during a call to lister.list().entrySet()
>
> I believe that the files are processed in the order specified and that the
> generations get switched around. This is evidenced by the file size of the
> Data file associated with the generations as it moves through the process.
> In this case we expect the nb-6 to become ma-7 as per the output from the
> run. In actuality we want nb-6 to be nb-9.
>
> {{noformat}}
> ma nb ma
> 1 212 4 212 8 212
> 2 64 5 70 9 70
> 3 4876 6 4883 7 4883
> {{noformat}}
>
> So now my question, has anyone seen the behaviour before?
>
> Oh, to make things more interesting I am using Docker images of 3.1 and a
> modified 4.0 that turns off the execution so I can just run the upgrade and
> downgrade.
>
> Any help would be appreciated,
> Claude
