On Wed, 31 Oct 2007, Ralf Gross wrote: > I did some tiobenchmarks, but I'm not spooling more than one job at > the same time. The RAID is capable of ~140 MB/s (seq. writes).
This is slightly lower than what I'm seeing with 4 3Gb/s 10,000RPM SATA drives. > But spooling is not my problem (I didn't get you point regarding > shoe-shining). As long as you can read data off the filesystems fast enough, spooling is not necessary in single full backup mode. > Spooling will always cost extra time, thus I'd like to avoid spooling at > at all for jobs that take days to finish. The first thing to point out is that when one job is spooling, another can be despooling - ie, a backup run for 2 jobs on one tape drive while using spooling is only a tiny time penalty over 1 job and the spooled blocks of data will be interleaved on the tape. The problem is that as soon as you start parallising operations like this, the disk heads start seeking and performance suffers - ie, the combined read/write rate will be about 120Mb/s, not 140Mb/s. As you add more parallel spooling processes the seek penalty increases - running 4 parallel operations (mimicking 2 tape drives being used for backups) the overall speed drops to about 90Mb/s I have 4 tape drives and allow up to 3 parallel operations per drive. Any more than this usually results in several jobs spooling, then pausing while they wait their turn to despool. At full steam (6 backups running simultaneously) I can _just_ keep up with 2 LTO2 tape drives with compression ratios not exceeding 1.5:1. More compressible data, more tape drives, or faster tape drives would mean this spooling model wouldn't work - and a lot more spindles would be necessary, which tends to come with a hefty controller price tag to suit. As most of the performance penalty is disk seeking, I feel that using SSDs would provide a fairly reasonable upgrade path. The next step after that may well be an external array controller stuffed with SSDs. > On the other hand I would like to avoid shoe-shining, but I can't say > for sure if it happens at all during direct backup over net. I only > see that bacula writes to tape with 75 MB/s (no spooling) and 77 MB/s > with spooling, but not if the drive has to stop 10, 20 or 100 times > during this backup. Even the iostat stats of the /dev/st0 tape device > are not clear. With or without spooling they sometimes drop below 40 > MB/s. Try running 2 backups at once - check the overall time for the jobs to complete. You should find that for approximately equal size filesystems, as long as there are no other bottlenecks (ie source arrays are separate and the spool disk can keep up), 2 backup jobs will complete in about 10% more time than one job alone. AB ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
