On 21/07/10, Craig Miskell (craig.misk...@opus.co.nz) wrote: > Rory Campbell-Lange wrote: > >On 19/07/10, Rory Campbell-Lange (r...@campbell-lange.net) wrote: > > > I ran a full backup of a volume last week on Bacula v2.4.4 and I > > > have now upgraded to 5.0.2. I'm getting half the throughput last > > > week as I am now. > > > > > > Last week's test volume was a 2.1TB xfs volume and it read/wrote > > > on average at 70MB/s. I am now backing up a 4.4TB ext3 volume and > > > bconsole's status client shows backup running at less than 30MB/s. > > > Hardware compression has been enabled in both cases. > > > > > > Both volumes are RAID5 Coraid (ATA over Ethernet) devices. I'd be > > > grateful for any pointers for getting to the bottom of this. > > > > With reference to > > http://www.mail-archive.com/bacula-users@lists.sourceforge.net/msg36340.html > > I wonder if I should configure the following settings: > > > > Maximum File Size = 5G > > Maximum Spool Size = 370g > > Maximum Job Spool Size = 20g > > Maximum Block Size = 262144 > > Maximum Network Buffer Size = 262144 > > > > for faster throughput. I haven't set any of these parameters at the moment. > > I can vouch for Maximum File Size = 5G at least; it's a totally > safe/backwards compatible change to make, and doing that took my > backups from 38MB/s to 46Mb/s (~20% improvement). And I can give > some tentative support to Maximum Block Size = 262144; in some small > scale tests yesterday (on a small number of large files only), I > went from 47MB/s to 64Mb/s (~25%). I haven't yet done full-scale > tests though. > > Be aware that it seems probable that if you change Maximum Block > Size on a device, you may (probably?) need to change that back to > the default on the device if you wish to restore from backups done > with the default block size. > > The other settings (Spool size and Network Buffer Size) will help in > getting your data onto the backup server and ensuring there's enough > data for your tape to chew on, and are good. Actual sizing for the > Spool settings will depend on how big your backups are; I don't have > enough experience (or data) to comment much.
Thanks very much for the comments, Craig. I'm leaving the majority of the previous emails above because others may find it germane to anyone else who may wish to comment on this debate. I'll definitely set the Maximum File Size as you suggest. Following your comments, Craig. From other comments on the list about Maximum Block Size it appears that this could be a dangerous change to make. In our case we may not be the party retrieving the data and we have to make that process as easy as possible. It would be great to have a comment on the safety of using Maximum Block Size from others on this list. Regarding spooling, I re-read the entry here: http://www.bacula.org/manuals/en/concepts/concepts/Data_Spooling.html Spooling specifically is used to avoid shoe-shining, so in my case it looks like a good idea because the tape seems to be waiting for data. The iotop tool shows quite big differences in read IO from the target volume (i.e 100MB/s one moment, 18MB/s the next. If I've got a 2.2TB backup writing to LTO4 tapes at just over 1TB/tape using hardware compression I wonder what the approprate spool sizes would be? I've only got a 300G internal volume available for spooling. Spool Size = 250GB # maximum spool size for this job Maximum Spool Size = 250GB # only one job at a time Maximum Job Spool Size = 250000000 # 250GB in bytes Spool Directory = /tapespool # XFS filesystem preferred? I'd be grateful on any comments about spool sizes and whether this is appropriate for my situtation with locally attached storage and and LTO4 tape library. The central issue is that the locally attached storage can be read at well over 150MB/s and I'm only getting to tape at under 40MB/s. Regards Rory -- Rory Campbell-Lange r...@campbell-lange.net ------------------------------------------------------------------------------ This SF.net email is sponsored by Sprint What will you do first with EVO, the first 4G phone? Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first _______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
