2011/5/31 Sean Clark <[email protected]>: > On 05/30/2011 02:11 PM, reiserfs wrote: >> Hello, im new with Bacula scene, i have used the HP Dataprotector with a HP >> Library fiber channel backup. >> >> With the Dataprotector i got 1gbps interfaces and switch and all jobs get >> done very fast, with transfer like 50-80MB/s. >> >> Now im using Bacula with a DELL TL2000 iSCSI, and with my first experience i >> got only 6MB/s transfer with 1gbps interfaces and switch. >> >> So what im missing >> >> Used to tes: >> Bacula Director runing on Slackware64 13.1 >> Bacula Cliente Windows 2003 Server > Turning on software gzip compression on the client is definitely a major > performance killer, unfortunately, so that would be my first guess as > well. This looks like a good place to mention some testing I've done. > > I've been doing some testing lately due to also being somewhat > aggravated as the apparently slow transfer rates I get during Bacula > backups, but it's starting to look like it's not really Bacula's fault > most of the time. Most of the time, it looks like the problem is just > how fast the client can read files off of the disk and send them. The > network (at least on Gb) is not usually the problem, nor even database > activity on the director (attribute spooling will help if you DO have > any problems with that). > > Encryption and gzip compression by the client introduce major latency > that unavoidably slows down the transfer, and this isn't specifically a > bacula client issue. Other things I have seen that cause major > slowdowns are antivirus software on Windows (particularly "on-access > scanning") and active use of the computer while the backup is running. > > Regarding compression, specifically, though - testing on my laptop here, > I tested just reading files from /usr and /home with "tar", piping them > through "pv" to get the transfer rate (and then dumping them directly to > /dev/null). I repeated the tests then with some different compression > schemes inserted. > for example: > > tar -cf - /usr | pv -b -r -a > /dev/null ("No Compression") > tar -cf - /usr | gzip -c | pv -b -r -a > /dev/null ("GZIP") > tar -cf - /usr | gzip -1 -c | pv -b -r -a > /dev/null ("GZIP1") > tar -cf - /usr/ | lzop -c | pv -b -r -a > /dev/null ("LZO") > > (and repeated for /home) > > Here are my results: > > /usr > No Compression: 5.58GB total data, Avg 13.1MB/s (436s to finish) > GZIP: 2.11GB Total data, Avg 2.97MB/s (727s to finish) > GZIP1: 2.36GB Total data, Avg 4.13MB/s (585s to finish) > LZO: 2.82GB Total Data, Avg 6.48MB/s (445s to finish) > > /home (includes a lot of e.g. media files that are not very compressible) > No Compression: 91.56GB Total Data, 34.5MB/s Avg,, (~2700s to finish) > GZIP: 77.1GB Total Data, 9.78MB/s Avg, (8072s to finish) > GZIP1: 77.6GB Total Data, 11.7MB/s Avg, (~6790s to finish) > LZO: 80.6GB Total Data, 28.3MB/s Avg, (~2900s to finish) > > So, yes, if you have gzip compression turned on, you'll almost certainly > see a huge increase in speed if you turn it off (I believe most tape > drives can or will do compression in hardware, so you don't need to > pre-compress at the client). > > If you are backing up to disk as I am (or for some reason aren't doing > hardware compression on the tape drive), you can also get a small speed > increase by dropping the gzip compression down to the minimum > ("Compress=GZIP1" in the FileSet), which seem to compress almost as well > overall but induces less latency. > > FEATURE REQUEST: > However, assuming my tests so far are representative, it looks like LZO > compression can get backup jobs transferred in almost the same amount of > time as no compression at all, while still substantially reducing the > amount of data transferred and stored (not as much as GZIP does, but > still a noteworthy amount). Is it possible we could get a > "Compress=LZOP" capability added to bacula-fd? > > tl;dr: Turn off compression until and unless an LZO compression option > is implemented, unless you are desperate for space on your backup media, > in which case you'll just have to cope with the slow backups. > > ------------------------------------------------------------------------------ > Simplify data backup and recovery for your virtual environment with vRanger. > Installation's a snap, and flexible recovery options mean your data is safe, > secure and there when you need it. Data protection magic? > Nope - It's vRanger. Get your free trial download today. > http://p.sf.net/sfu/quest-sfdev2dev > _______________________________________________ > Bacula-users mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/bacula-users >
+1, LZO seems really faster, that would be a good feature I want to add that I don't think that the whole directory is in a "tar". I think that each file is compressed and encrypted separately. Can someone tell us the algorithm that is used by bacula to compress and encrypt ( by block, by file, … ) ? ------------------------------------------------------------------------------ Simplify data backup and recovery for your virtual environment with vRanger. Installation's a snap, and flexible recovery options mean your data is safe, secure and there when you need it. Data protection magic? Nope - It's vRanger. Get your free trial download today. http://p.sf.net/sfu/quest-sfdev2dev _______________________________________________ Bacula-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/bacula-users
