On Tue, Nov 04, 2008 at 10:21:54PM +0100, Ulrich Leodolter wrote:
>
> On Tue, 2008-11-04 at 22:38 +0200, Pasi Kärkkäinen wrote:
> > On Tue, Nov 04, 2008 at 09:02:46PM +0100, Ulrich Leodolter wrote:
> > > Hi,
> > >
> > > Problem: Migrate/Copy jobs from disk pool (DiskBackup)
> > > to tape (DiskCopy) only get overall speed of 10-20MB/s.
> > >
> > > full backup job size varies from 10-50GB.
> > >
> > > Pool setup is simple, just one pool for full an incremental backups
> > > to disk (automatic recycle works good)
> > >
> > > Pool {
> > > Name = DiskBackup
> > > Pool Type = Backup
> > > Recycle = yes
> > > RecyclePool = DiskBackup
> > > AutoPrune = yes
> > > Volume Retention = 15 days
> > > Volume Use Duration = 6 days
> > > Maximum Volume Bytes = 4G
> > > Label Format = Backup-
> > > Next Pool = DiskCopy
> > > }
> > >
> > > DiskCopy pool goes to LTO4 tape device.
> > > DiskBackup goes to SATA external raid (6T ext3).
> > >
> >
> > Which raid-level your SATA external raid is?
> >
>
> Raid 5
> >
> > > concurrency for DiskBackup jobs is 15, jobs are spread
> > > over DiskBackup Valumes (maybe thats the main problem)
> > >
> > > i can read write/continuous on both devices at about 70MB/s
> > > (SATA is not fast)
> > >
> >
> > So your SATA-raid can do max 70 MB/sec..
> >
> > How much your LTO4 can do?
>
> 70MB/sec
>
Have you tried with dd if=/dev/null of=/dev/tape bs=<bigblocksize>
to get the actual max tape drive performance.. ?
> >
> > > i tried spooling to local SAS raid, but overall speed is lower
> > > than direct writing to tape.
> > > despooling from SAS raid to Tape runs at Tape maximum spped.
> > >
> >
> > So SAS-RAID to Tape is faster than SATA-RAID to Tape?
> >
>
> about 180MB/s continuous write
>
Yep. Over 2x compared to SATA-Raid.
So clearly it's the RAID that's limiting your performance.
> > >
> > >
> > > I need some Performance tuning tips, maybe:
> > >
> > > Limit jobs per Volume in DiskBackup pools?
> > > Split DiskBackup into DiskFull and DiskIncr pools?
> > >
> >
> > First determine the bottleneck and then work around it..
> >
>
> Thats why i am asking :-)
>
> i tried to monitor disk using dstat
> http://dag.wieers.com/home-made/dstat/
> i reports 70MB/s read while bacula Copy Job is running.
>
> looks like some overhead is in bacula itself:
>
> avg iostat while bacula CopyDiskToTape
>
> user 9.12
> system 2.79
> iowait 17.00
>
> avg iostat while EMC Networker (Legato) Clone Disk to Tape
> (on the same hardware)
>
> user 2.14
> system 2.02
> iowait 12.84
>
> as u can see user and iowait are higher for bacula.
> total Copy Jobs size is about 800GB (one LTO4 Tape)
>
> Networker backup jobs to disk run in parallel (max 10),
> i dont think backup file sets are continous on disk.
>
Hmm.. have you adjusted block sizes bacula uses?
"iostat" is also useful tool.. I guess it's part of "sysstat" package.
>
> > Btw. could you post your 'copy' job? I'm in the process of trying it out but
> > I'm still stuck with trying to implement 'copy only uncopied
> > jobs'-feature..
> >
> > -- Pasi
> >
>
> Job {
> Name = "CopyDiskToTape"
> Type = Copy
> Client = dir-fd
> Level = Full # must be defined, but is ignored
> FileSet = "Full Set" # must be defined, but is ignored
> Pool = DiskBackup
> Storage = File
> Messages = Standard
> Selection Type = PoolUncopiedJobs
> Maximum Concurrent Jobs = 10
> # SpoolData = yes
> }
>
>
> PoolUncopiedJobs is based on an SQLQuery i posted in the devel list.
> i am running 2.5.17 svn at the server side
>
Oh nice, I missed that.
I'll try it out:)
-- Pasi
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
