Hello, our backup-client is a zfs-storage, which has 48 harddisk which is configured as RAID10 with SSDs for read and write cache, before I backup my data I create a snapshot.
So far so good, my problem is that the incremental backup took around 32minutes every time, no matter how many files or how many gigabytes, for me the bottleneck is not the network, nor the backup storage, nor the client. When the backup run the bareos-fd client has a cpu utilisation of around 15%, iowait and also the load is around 0-1(%). SO can somebody tell me why is it so slow and how can i tune this? A fullbackup (around 6,5TB) takes 26hours, I expect a read/backup speed of >150MB/s. The Rate differs between 1709.4 KB/s and 143351.1 KB/s from my last 50 inc Backups Here two examples: 23-Jan 02:00 bareos-dir JobId 1063: Start Backup JobId 1063, Job=clouddata.2019-01-23_02.00.00_30 23-Jan 02:00 bareos-dir JobId 1063: There are no more Jobs associated with Volume "Incremental-0188". Marking it purged. 23-Jan 02:00 bareos-dir JobId 1063: All records pruned from Volume "Incremental-0188"; marking it "Purged" 23-Jan 02:00 bareos-dir JobId 1063: Recycled volume "Incremental-0188" 23-Jan 02:00 bareos-dir JobId 1063: Using Device "FileStorage" to write. 23-Jan 01:54 zfs-serv1-fd JobId 1063: DIR and FD clocks differ by -324 seconds, FD automatically compensating. 23-Jan 01:54 zfs-serv1-fd JobId 1063: shell command: run ClientBeforeJob "/sbin/zfs snapshot -r nc_storage@bareos" 23-Jan 02:00 bareos-sd JobId 1063: Recycled volume "Incremental-0188" on device "FileStorage" (/backupdata), all previous data lost. 23-Jan 02:00 bareos-dir JobId 1063: Max Volume jobs=1 exceeded. Marking Volume "Incremental-0188" as Used. 23-Jan 02:25 zfs-serv1-fd JobId 1063: shell command: run ClientAfterJob "/sbin/zfs destroy -r nc_storage@bareos" 23-Jan 02:31 bareos-sd JobId 1063: Elapsed time=00:31:12, Transfer rate=5.970 M Bytes/second 23-Jan 02:31 bareos-dir JobId 1063: sql_create.c:872 Insert of attributes batch table done 23-Jan 02:31 bareos-dir JobId 1063: Bareos bareos-dir 17.2.4 (21Sep17): Build OS: x86_64-pc-linux-gnu debian Debian GNU/Linux 9.3 (stretch) JobId: 1063 Job: clouddata.2019-01-23_02.00.00_30 Backup Level: Incremental, since=2019-01-22 02:00:00 Client: "zfs-serv1.my.domain.de" 17.2.4 (21Sep17) x86_64-pc-linux-gnu,debian,Debian GNU/Linux 9.3 (stretch),Debian_9.0,x86_64 FileSet: "backup-zfs-serv" 2018-03-06 12:19:01 Pool: "Incremental" (From Job IncPool override) Catalog: "MyCatalog" (From Client resource) Storage: "File" (From Job resource) Scheduled time: 23-Jan-2019 02:00:00 Start time: 23-Jan-2019 02:00:00 End time: 23-Jan-2019 02:31:13 Elapsed time: 31 mins 13 secs Priority: 10 FD Files Written: 2,257 SD Files Written: 2,257 FD Bytes Written: 11,175,399,254 (11.17 GB) SD Bytes Written: 11,175,894,854 (11.17 GB) Rate: 5966.6 KB/s Software Compression: 72.8 % (lz4) VSS: no Encryption: no Accurate: no Volume name(s): Incremental-0188 Volume Session Id: 267 Volume Session Time: 1539762885 Last Volume Bytes: 11,189,726,587 (11.18 GB) Non-fatal FD errors: 0 SD Errors: 0 FD termination status: OK SD termination status: OK Termination: Backup OK 07-Mar 02:00 bareos-dir JobId 1180: Start Backup JobId 1180, Job=clouddata.2019-03-07_02.00.00_06 07-Mar 02:00 bareos-dir JobId 1180: There are no more Jobs associated with Volume "Incremental-0045". Marking it purged. 07-Mar 02:00 bareos-dir JobId 1180: All records pruned from Volume "Incremental-0045"; marking it "Purged" 07-Mar 02:00 bareos-dir JobId 1180: Recycled volume "Incremental-0045" 07-Mar 02:00 bareos-dir JobId 1180: Using Device "FileStorage" to write. 07-Mar 01:59 zfs-serv1-fd JobId 1180: DIR and FD clocks differ by -30 seconds, FD automatically compensating. 07-Mar 01:59 zfs-serv1-fd JobId 1180: shell command: run ClientBeforeJob "/sbin/zfs snapshot -r nc_storage@bareos" 07-Mar 02:00 bareos-sd JobId 1180: Recycled volume "Incremental-0045" on device "FileStorage" (/backupdata), all previous data lost. 07-Mar 02:00 bareos-dir JobId 1180: Max Volume jobs=1 exceeded. Marking Volume "Incremental-0045" as Used. 07-Mar 02:31 zfs-serv1-fd JobId 1180: shell command: run ClientAfterJob "/sbin/zfs destroy -r nc_storage@bareos" 07-Mar 02:31 bareos-sd JobId 1180: Elapsed time=00:31:58, Transfer rate=9.753 M Bytes/second 07-Mar 02:32 bareos-dir JobId 1180: sql_create.c:872 Insert of attributes batch table done 07-Mar 02:32 bareos-dir JobId 1180: Bareos bareos-dir 17.2.4 (21Sep17): Build OS: x86_64-pc-linux-gnu debian Debian GNU/Linux 9.3 (stretch) JobId: 1180 Job: clouddata.2019-03-07_02.00.00_06 Backup Level: Incremental, since=2019-03-06 02:00:01 Client: "zfs-serv1.my.domain.de" 17.2.4 (21Sep17) x86_64-pc-linux-gnu,debian,Debian GNU/Linux 9.3 (stretch),Debian_9.0,x86_64 FileSet: "backup-zfs-serv" 2018-03-06 12:19:01 Pool: "Incremental" (From Job IncPool override) Catalog: "MyCatalog" (From Client resource) Storage: "File" (From Job resource) Scheduled time: 07-Mar-2019 02:00:00 Start time: 07-Mar-2019 02:00:00 End time: 07-Mar-2019 02:32:00 Elapsed time: 32 mins Priority: 10 FD Files Written: 27,685 SD Files Written: 27,685 FD Bytes Written: 18,699,788,372 (18.69 GB) SD Bytes Written: 18,707,482,165 (18.70 GB) Rate: 9739.5 KB/s Software Compression: 51.9 % (lz4) VSS: no Encryption: no Accurate: no Volume name(s): Incremental-0045 Volume Session Id: 384 Volume Session Time: 1539762885 Last Volume Bytes: 18,725,855,238 (18.72 GB) Non-fatal FD errors: 0 SD Errors: 0 FD termination status: OK SD termination status: OK Termination: Backup OK -- You received this message because you are subscribed to the Google Groups "bareos-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. For more options, visit https://groups.google.com/d/optout.
