Thank you for your reply, josh. Sorry for delay in reply - have been on vacation. See replies below.
Robert Gerber 402-237-8692 r...@craeon.net On Fri, Sep 15, 2023, 9:52 AM Josh Fisher via Bacula-users < bacula-users@lists.sourceforge.net> wrote: > > On 9/14/23 15:35, Rob Gerber wrote: > > Bacula is transferring data at a fraction of the available link speed. I > am backing up an SMB share hosted on a fast NAS appliance. The share is > mounted on the bacula server in /mnt/NAS/sharename. I have dedicated 10gbe > copper interfaces on the NAS and the bacula server. > > When backing up the NAS, cifsiostat shows around 250MB/s during the > spooling phase (and 0 kb/s during the despool phase). When using cp to copy > files from the NAS to the Bacula server, I can easily saturate my 10gbe > link (avg throughput around 1GB/s, or a little lower). > > > So that tells you that there's nothing wrong with the underlying SMB file > system. The Bacula client just reads the files like any other directory > it's backing up. > > > > I think the problem lies in Bacula because I can copy data much faster > using cp instead of bacula. Obviously bacula is doing a lot more than cp, > so there will be differences. However I would hope for transfer speeds > closer to the available link speed. > > top shows that a couple cores are maxed out during the spooling process. > Maybe hashing speed is the limitation here? If so, could multicore hashing > support speed this up? I have two e5-2676 v3 processors in this server. I > am using SHA512 right now, but I saw similar speeds from bacula when using > MD5. > > > The hashing speed doesn't account for a 4x slower transfer, and likely not > for saturating 2 cores. Do you have compression enabled for the job? Or > encryption? You definitely do not want compression, since the tape drive > will handle compression itself. Also, the client and sd are the same > machine in this case, but make sure it is not configured to use TLS > connections. > I have not affirmatively enabled compression. Is there any sort of default compression mode that I could disable? I'll see what I can find regarding TLS connections. I haven't set up such a thing, but maybe it works that way by default? > > > Average write speed to LTO-8 media winds up being about 120-150MB/s once > the times to spool and despool are considered. > > My spool is on a 76GB ramdisk (spool size is 75G in bacula dir conf), so I > don't think spool disk access speed is a factor. > > > Might be overkill. A NVMe SSD is plenty fast enough for both the 10G > network and for despooling to the LTO8 drive. If the catalog DB is also on > this server, then you might be better off with the spool on SSD and far > more RAM dedicated to postgresql. If the DB is on another server, then the > attributes are being despooled to the DB over the 1G network. > I was using a sata SSD for my spool, but switched to a ramdisk for troubleshooting purposes. I have about 50gb ram available (not counting ramdisk). Top shows about 1.6gb ram in use, though I don't know how accurate that is. My postgres DB server is on the same machine. I haven't specifically configured postgres to use a certain amount of ram. I don't know what performance enhancements I might be able to use with postgres. I'm tempted to try setting up a direct to disk backup and test with local files, or files stored on my TrueNAS server (which has its own dedicated optical 10gbe link, which I can also a saturate when using cp). Is there a way I could troubleshoot and try to pinpoint the location of my bottleneck? I will investigate the questions you raised in your email and report back. Some delay is to be expected because a 124tb backup is running and is currently at 98tb complete, averaging about 125MiB/s according to bacularis. > > I have not tested to see if bacula could back up faster if it wasn't > accessing a share via SMB. I don't think SMB should be an issue here but I > have to consider every possibility. The SMB share I'm backing up is mounted > on /mnt/NAS/sharename. Bacula is backing that mount folder up. > > Currently, my only access to the NAS appliance is via SMB. The appliance > does support iscsi in read only mode but i'm not sure if there would be any > performance improvements. > > I don't think the traffic could be going out through the wrong interface. > The NAS is directly attached to my bacula server using a short cat6 cable. > The NAS and my server each have 10gbe copper interfaces. The relevant > interfaces have ip addresses statically assigned. These addresses are > unique to the LAN configuration (local lan is 10.1.1.0/24, 10gbe > interfaces assigned to 192.168.6.25 and 192.168.6.100). My bacula > server's only other connection is to the gigabit LAN switch. > > Is there any information that I could provide to help the list help me, or > does anyone have any thoughts for me? > > Regards, > Robert Gerber > 402-237-8692 > r...@craeon.net > > > > > > _______________________________________________ > Bacula-users mailing > listBacula-users@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/bacula-users > > _______________________________________________ > Bacula-users mailing list > Bacula-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-users >
_______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
