I made some modifications and implemented a replica data migration tool

### Motivation
We have a scenario where a large number of bookie nodes are offline, and we 
find that the bookie's offline solution is very time-consuming.


Bookie offline steps are as follows:
1. Set the bookie node to be offline to readOnly;
2. Wait for the Pulsar data on the Bookie node to be offline to expire and 
delete;
3. When most of the data on these offline nodes is expired and cleaned up, 
there will still be some data that cannot be expired and deleted;
4. Stop a bookie, and then use the decommission command to migrate the data 
that has not been expired and cleaned up to the new node:
bin/bookkeeper shell decommissionbookie -bookieid xx
5. When the data on one bookie node is migrated, continue to the next bookie 
node;


Step 4 is very time-consuming. We found that waiting for a bookie data 
migration to complete, it takes about 1 hour, and we have 125 bookie nodes to 
be offline.




### Proposal
To solve this problem, we developed a replica data migration tool.
After having this tool, our offline steps are as follows:
1. Execute the data migration command:
bin/bookkeeper shell replicasMigration --bookieIds bookie1,bookie2 --ledgerIds 
ALL --readOnly true
2. When the data migration is completed, stop all bookie nodes to be offline;


In addition, this command can also migrate the replica data on some bookie 
nodes to other nodes, for example:
bin/bookkeeper shell replicasMigration --bookieIds bookie1,bookie2 --ledgerIds 
ledger1,ledger2,ledger3 --readOnly false

Reply via email to