Den sön 22 dec. 2024 kl 09:33 skrev Bo Berglund <bo.bergl...@gmail.com>:
> Background: > ----------- > In 2018 we migrated from CVSNT to VisualSvn and at the time I ran the > conversion, which produced a mirror repository from CVS but in subversion. > > This worked fine and we could use that new repository successfully from > then on. > > Then about a year later I set up a backup scheme by creating an Ubuntu > Server > 18.04.1 LTS machine at another location (actually a different country). > > I transfered the repositories from the main server as svn dump files and > imported these to the mirror server. This also worked fine. > > Then I created a bat file on the main server in order to run a sync > nightly to > save new added data to the mirror server making them identical. > > This has been running until now. > And it seemingly has the same data as witnessed by tests like checking the > revision numbers of each repository. > > Problem > ------- > Then on Dec 4th 2024 disaster happened and the main Windows server failed > following a power outage that took out some parts of the VMWARE ESX system > under > which the main server was running. Apparently there was a drive failure. > > IT has managed to restore the server (Windows Server 2016 I believe) from > a 5 > months old system backup, but it now lacks all commits between July and > December.... > Oh, sorry to hear. 5 month old backups really suck. > > So I need to sync in the other direction to bring the data missing on the > now 5 > months old server from the backup server. > > > Questions > --------- > > How do I proceed to do this restore? > > The batch file used on the main server to sync nightly to the backup server > contains a command like this for each main repository on the server: > > SET SYNCCMD="C:\Program Files\VisualSVN Server\bin\svnsync.exe" > > %SYNCCMD% synchronize https://backupserverurl.com/svn/reponame > https://visualsvn-name/svn/reponame > > There are 11 such sync commands in the bat file. > > Do I just reverse the two https commands in order to do the reverse sync? > Nope, svnsync will not let you sync "backwards". > And execute from the backup server? Then it must be connected to the > company LAN > by VPN, I believe... > > And are there some permission problems that can pop up, for example how is > it > specified which user is doing the restore sync? > > Dump files? > ----------- > Maybe I should transfer the backup repository as dump files and import > these to > the restored main server? > You should be able to do an incremental dump on the backup server starting from the first revision that is missing on the main server then load this on the main server. > If so can dump files be created starting at a specified commit date and if > there > are overlaps when using the dump files will duplicated revisions be > created or > will the revisions in the dump be ignored if they also exist on the target > server? > You should make the dump based on revision number rather than date, since the date is an unversioned property which could be changed (if the repository is configured to allow it). > > If the partial dump files strategy works how could that be done exactly? > I believe a full example is better, although it is slightly long. The interesting part comes towards the end. Please note that I'm using the file:// protocol, you may replace this with http:// or svn:// depending on your deployment situation. Also note that I'm doing this on a single machine so you need to consider which machine to do each step. First I setup two repositories, one main and one which will be the mirror: [[[ dsg@daniel-23:~/temp$ svnadmin create repo_main dsg@daniel-23:~/temp$ svn co file://`pwd`/repo_main wc Checked out revision 0. dsg@daniel-23:~/temp$ svnadmin create repo_mirror ]]] Then I configure the mirror to allow changing of ANY revision properties, this is NOT best-practise but for the sake of brevity I'll do it in this example. The SVN Book has a variation of the hook script checking for the svnsync user ( https://svnbook.red-bean.com/en/1.7/svn-book.html#svn.reposadmin.maint.replication.svnsync ). [[[ dsg@daniel-23:~/temp$ cat <<EOF >repo_mirror/hooks/pre-revprop-change #!/bin/sh exit 0 EOF dsg@daniel-23:~/temp$ chmod +x repo_mirror/hooks/pre-revprop-change ]]] Next, I'll initialize svnsync and add a few revisions and sync those [[[ dsg@daniel-23:~/temp$ svnsync init file://`pwd`/repo_mirror file://`pwd`/repo_main Copied properties for revision 0. dsg@daniel-23:~/temp$ echo "1">wc/fil dsg@daniel-23:~/temp$ svn add wc/fil A wc/fil dsg@daniel-23:~/temp$ svn ci wc/fil -m "rev 1" Adding wc/fil Transmitting file data .done Committing transaction... Committed revision 1. dsg@daniel-23:~/temp$ echo "2">wc/fil dsg@daniel-23:~/temp$ svn ci wc/fil -m "rev 2" Sending wc/fil Transmitting file data .done Committing transaction... Committed revision 2. dsg@daniel-23:~/temp$ svnsync sync file://`pwd`/repo_mirror file://`pwd`/repo_main Transmitting file data . Committed revision 1. Copied properties for revision 1. Transmitting file data . Committed revision 2. Copied properties for revision 2. ]]] Now I have two revisions in both the main and the mirror repo. I'll make a backup (this is your backup from 5 month ago) and then commit two more revisions and sync those. [[[ dsg@daniel-23:~/temp$ cp -r repo_main/ repo_bak/ dsg@daniel-23:~/temp$ echo "3">wc/fil dsg@daniel-23:~/temp$ svn ci wc/fil -m "rev 3" Sending wc/fil Transmitting file data .done Committing transaction... Committed revision 3. dsg@daniel-23:~/temp$ echo "4">wc/fil dsg@daniel-23:~/temp$ svn ci wc/fil -m "rev 4" Sending wc/fil Transmitting file data .done Committing transaction... Committed revision 4. dsg@daniel-23:~/temp$ svnsync sync file://`pwd`/repo_mirror file://`pwd`/repo_main Transmitting file data . Committed revision 3. Copied properties for revision 3. Transmitting file data . Committed revision 4. Copied properties for revision 4. ]]] Then disaster strikes - we lost our main repository. So I restore the backup which is now "old". I believe this is has replicated the situation you are in right now. When I check the Last Changed Rev, I can see the main repo is on rev 2 and the mirror is on rev 4. [[[ dsg@daniel-23:~/temp$ mv repo_main/ repo_lost/ dsg@daniel-23:~/temp$ cp -r repo_bak/ repo_main/ dsg@daniel-23:~/temp$ svn info file://`pwd`/repo_main Path: repo_main URL: file:///home/dsg/temp/repo_main Relative URL: ^/ Repository Root: file:///home/dsg/temp/repo_main Repository UUID: b59fc87c-8933-423b-ac61-4985cfc5a6ad Revision: 2 Node Kind: directory Last Changed Author: dsg Last Changed Rev: 2 Last Changed Date: 2024-12-22 16:52:38 +0100 (Sun, 22 Dec 2024) dsg@daniel-23:~/temp$ svn info file://`pwd`/repo_mirror Path: repo_mirror URL: file:///home/dsg/temp/repo_mirror Relative URL: ^/ Repository Root: file:///home/dsg/temp/repo_mirror Repository UUID: 2a340f81-40ba-467e-9d23-dc1940d0c4cb Revision: 4 Node Kind: directory Last Changed Author: dsg Last Changed Rev: 4 Last Changed Date: 2024-12-22 16:53:01 +0100 (Sun, 22 Dec 2024) ]]] Here comes the restore part. I'll start by dumping the mirror repository from the first missing revision up to HEAD, please note the --incremental option. If you want, you could pipe this through your favourite compression program to get a slightly smaller dump file. [[[ dsg@daniel-23:~/temp$ svnadmin dump --incremental repo_mirror -r 3:HEAD>dump * Dumped revision 3. * Dumped revision 4. ]]] Then on the main server I'll load the dump file. (You would obviously have to transfer the dump file in some way from the mirror server to the main server). [[[ dsg@daniel-23:~/temp$ cat dump | svnadmin load repo_main <<< Started new transaction, based on original revision 3 * editing path : fil ... done. ------- Committed revision 3 >>> <<< Started new transaction, based on original revision 4 * editing path : fil ... done. ------- Committed revision 4 >>> ]]] If I check the main repo and the mirror repo both are on revision 4. [[[ dsg@daniel-23:~/temp$ svn info file://`pwd`/repo_main/ Path: repo_main URL: file:///home/dsg/temp/repo_main Relative URL: ^/ Repository Root: file:///home/dsg/temp/repo_main Repository UUID: b59fc87c-8933-423b-ac61-4985cfc5a6ad Revision: 4 Node Kind: directory Last Changed Author: dsg Last Changed Rev: 4 Last Changed Date: 2024-12-22 16:53:01 +0100 (Sun, 22 Dec 2024) dsg@daniel-23:~/temp$ svn info file://`pwd`/repo_mirror/ Path: repo_mirror URL: file:///home/dsg/temp/repo_mirror Relative URL: ^/ Repository Root: file:///home/dsg/temp/repo_mirror Repository UUID: 2a340f81-40ba-467e-9d23-dc1940d0c4cb Revision: 4 Node Kind: directory Last Changed Author: dsg Last Changed Rev: 4 Last Changed Date: 2024-12-22 16:53:01 +0100 (Sun, 22 Dec 2024) ]]] The main repo is also identical to the lost repo [[[ dsg@daniel-23:~/temp$ diff -ur repo_main/ repo_lost/ ]]] I can successfully make a change in my existing working copy, commit it and do a new sync to the mirror repo [[[ dsg@daniel-23:~/temp$ echo "5">wc/fil dsg@daniel-23:~/temp$ svn ci wc/fil -m "rev 5" Sending wc/fil Transmitting file data .done Committing transaction... Committed revision 5. dsg@daniel-23:~/temp$ svnsync sync file://`pwd`/repo_mirror file://`pwd`/repo_main Transmitting file data . Committed revision 5. Copied properties for revision 5. ]]] The only remaining obstacle is if there were revisions committed to the main repository that were not yet synced with the mirror repository. In that case, the working copies with these changes would contain invalid data, in which case your only option would be to remove the .svn administrative folder and make a new checkout. Of course, before you attempt any of the above, make sure you have a copy of the main repository (as it was restored from the backup). I should note that you would be able to dump-load cycle without having access to either server using the svnrdump command, but this means you would have to setup a hook script in the main repository. I was able to do the following, however it will not work out of the box on your server since you need the pre-revprop-change hookscript to be a Windows batch file (pre-revprop-change.bat should work). [[[ dsg@daniel-23:~/temp$ cat <<EOF >repo_main/hooks/pre-revprop-change #!/bin/sh exit 0 EOF dsg@daniel-23:~/temp$ chmod +x repo_main/hooks/pre-revprop-change dsg@daniel-23:~/temp$ svnrdump dump --incremental -r 3:HEAD file://`pwd`/repo_mirror | svnrdump load file://`pwd`/repo_main * Dumped revision 3. * Dumped revision 4. * Loaded revision 3. * Loaded revision 4. ]]] > The VisualSVN server on Windows is on svn version 1.9.7 because I never > understood how I could upgrade it. > VisualSVN has a comprehensive checklist on their website. https://www.visualsvn.com/support/topic/00233/ I'm not sure what version of VisualSVN shipped with Subversion 1.9.7, but it seems kind of old so you should probably upgrade to get all the security fixes since that old version. > > The backup server otoh is upgraded with Ubuntu when it went from 18.04.1 to > 20.04.1 and then to 24.04.1 LTS where it is now. > Current svn version on the Ubuntu Server is 1.14.3 (r1914484) > > Finally: > The combined size of the repositories on Ubuntu is 36 GB and they reside > in a > separate partition. > > And activity from July until the crash has been rather low, so the data > between > then and now is probably very small compared to the total. That is why I > want to > run a reverse sync or a partial dump to reduce the amount of data to move. > > Please advice. > > TIA > > Good luck! Cheers, Daniel > > -- > Bo Berglund > Developer in Sweden > >
