Hi!
I have two identical 4Tb usb external drives, Western Digital Model WDC
WD40NDZW-11A8JS1. My computer is a Dell Inspiron 15 3000 Model 3511 (a
very modest laptop), from early 2024, running Debian 12 Stable, always
kept updated.
The first drive, Drive 1, is my "backup drive". I backup daily using
Borgbackup Version 1.2.4 from the Debian Stable repositories, and
rsnapshot Version 1.4.5-1, also from the Debian Stable repositories.
It also has a whole bunch of other archival programs, data and image
files on it as well.
sudo df -h /media/user/DRIVE1
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 3.6T 1.5T 2.0T 42% /media/user/DRIVE1
sudo du -sh /media/user/DRIVE1
1.5T /media/user/DRIVE1
I am trying to use the other drive, DRIVE2, as an exact copy of DRIVE1
(except for the unused space on the drive). The only difference between
the two drives is that I just re-formatted DRIVE2 and set it up so that
the disk is now LUKS 1 encrypted.
sudo df -h /media/user/DRIVE2
Filesystem Size Used Avail
Use% Mounted on
/dev/mapper/luks-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx 3.6T 1.2T 2.3T
35% /media/user/DRIVE2
sudo du -sh /media/user/DRIVE2
1.2T /media/user/DRIVE2
Every night, I have been using rsync to copy from DRIVE1 to DRIVE2,
doing:
time sudo rsync -avvv --human-readable --delete --numeric-ids
--info=progress2,stats2,name2 --
exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media
/*",
"/lost+found"} /media/user/DRIVE1/ /media/user/DRIVE2/ ; date
And each time, DRIVE2 would be using more and more space, much more
than DRIVE1!
Finally, I decided to re-format DRIVE2 and set it up for LUKS. Then I
tied again to use rsync to write everything from DRIVE1 to DRIVE2, all
at once. It ran for 39 hours, using more and more space until it ran
out of space, without finishing.
So, I deleted everything from DRIVE2 (which surprisingly took many
hours), keeping it set up for LUKS, and used rsync again to copy
everything from DRIVE1 to DRIVE2, this time trying:
time sudo rsync -aHAXUSxvvv --human-readable --delete --numeric-ids --
info=progress2,stats2,name2 --
exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media
/*","/lost+found"} /media/user/DRIVE1/ /media/user/DRIVE2/ ; date
which again took many hours, so it was done over multiple sessions.
(BTW, I don't know if the -U option helps; it may just screw things
up.)
Now, DRIVE2 is still using considerably LESS space than DRIVE1!
Is that due to using -H for hard links? Or -S for sparse files? Or
both? Or neither?
I have not yet done any further copying with rsync to see if the rate
of change to DRIVE1 and DRIVE2 will be equal. But I really doubt it.
So, back to the original question: what in the world am I supposed to
do to have rsync copy so that the size change in the two drives is
equal, and DRIVE2 has (theoretically) the same data, taking up the same
space, as DRIVE1?
-----------------------------------------------------------------------
BTW, I forgot to mention, FWIW, that the Borgbackup and rsnapshot
backups are of /home/user and its subdirectories only. Everything else
(all the system stuff) is backed up using Timeshift. The Timeshift
data, like everything else, is part of the stuff on DRIVE1.
