On 9/14/23 15:35, Rob Gerber wrote:
Bacula is transferring data at a fraction of the available link speed.
I am backing up an SMB share hosted on a fast NAS appliance. The share
is mounted on the bacula server in /mnt/NAS/sharename. I have
dedicated 10gbe copper interfaces on the NAS and the bacula server.
When backing up the NAS, cifsiostat shows around 250MB/s during the
spooling phase (and 0 kb/s during the despool phase). When using cp to
copy files from the NAS to the Bacula server, I can easily saturate my
10gbe link (avg throughput around 1GB/s, or a little lower).
So that tells you that there's nothing wrong with the underlying SMB
file system. The Bacula client just reads the files like any other
directory it's backing up.
I think the problem lies in Bacula because I can copy data much faster
using cp instead of bacula. Obviously bacula is doing a lot more than
cp, so there will be differences. However I would hope for transfer
speeds closer to the available link speed.
top shows that a couple cores are maxed out during the spooling
process. Maybe hashing speed is the limitation here? If so, could
multicore hashing support speed this up? I have two e5-2676 v3
processors in this server. I am using SHA512 right now, but I saw
similar speeds from bacula when using MD5.
The hashing speed doesn't account for a 4x slower transfer, and likely
not for saturating 2 cores. Do you have compression enabled for the job?
Or encryption? You definitely do not want compression, since the tape
drive will handle compression itself. Also, the client and sd are the
same machine in this case, but make sure it is not configured to use TLS
connections.
Average write speed to LTO-8 media winds up being about 120-150MB/s
once the times to spool and despool are considered.
My spool is on a 76GB ramdisk (spool size is 75G in bacula dir conf),
so I don't think spool disk access speed is a factor.
Might be overkill. A NVMe SSD is plenty fast enough for both the 10G
network and for despooling to the LTO8 drive. If the catalog DB is also
on this server, then you might be better off with the spool on SSD and
far more RAM dedicated to postgresql. If the DB is on another server,
then the attributes are being despooled to the DB over the 1G network.
I have not tested to see if bacula could back up faster if it wasn't
accessing a share via SMB. I don't think SMB should be an issue here
but I have to consider every possibility. The SMB share I'm backing up
is mounted on /mnt/NAS/sharename. Bacula is backing that mount folder up.
Currently, my only access to the NAS appliance is via SMB. The
appliance does support iscsi in read only mode but i'm not sure if
there would be any performance improvements.
I don't think the traffic could be going out through the wrong
interface. The NAS is directly attached to my bacula server using a
short cat6 cable. The NAS and my server each have 10gbe copper
interfaces. The relevant interfaces have ip addresses statically
assigned. These addresses are unique to the LAN configuration (local
lan is 10.1.1.0/24 <http://10.1.1.0/24>, 10gbe interfaces assigned to
192.168.6.25 and 192.168.6.100). My bacula server's only other
connection is to the gigabit LAN switch.
Is there any information that I could provide to help the list help
me, or does anyone have any thoughts for me?
Regards,
Robert Gerber
402-237-8692
r...@craeon.net
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
