>>>>> "BM" == Bill Moran <[EMAIL PROTECTED]> writes:
BM> In response to "Anders Boström" <[EMAIL PROTECTED]>: >> >>>>> "BM" == Bill Moran <[EMAIL PROTECTED]> writes: >> BM> In response to "Anders Boström" <[EMAIL PROTECTED]>: >> >> I did some new performance-tests: >> >> >> >> All operations are against a directory-tree with 7,255,659,224 bytes >> >> data in 98,025 files. >> >> >> >> | test1 | test2 | test3 | >> >> --------------------------------+-------+-------+-------+ >> >> bacula-fd, no compression, md5: | 10:25 | 10:42 | 10:15 | >> >> bacula-fd, GZIP, md5: | 16:09 | 15:46 | 17:02 | >> >> tar, local (1): | 8:37 | 8:53 | 8:54 | >> >> tar + nc (2): | 9:48 | 9:52 | 9:43 | >> >> tar + gzip + nc (3): | 14:11 | 14:26 | 15:03 | >> >> --------------------------------+-------+-------+-------+ >> BM> OK. This indicates to me that bacula is doing a damn good job. Only BM> 15% overhead to add checksumming and cataloging features to backup. BM> If you ask me, that's a hell of a deal. >> >> >> (1) time /bin/sh -c "tar cf - directory | cat >/dev/null" >> >> (2) time /bin/sh -c "tar cf - directory | nc -q 0 backup_server 4711" >> >> (3) time /bin/sh -c "tar czf - directory | nc -q 0 backup_server 4711" >> >> >> >> This round of tests is more in line with what I expected, and the >> >> bacula performance is quite good. The only major difference compared >> >> to my previous tests is that the file-server disc-performance is much >> >> better. It seems like bacula suffers much more than tar from slow >> >> disc-performance on the file-server. backup-server and network >> >> performance don't seem to be an issue at all in the tests, even if >> >> write to TCP is a bit slower than /dev/null. >> >> >> >> However, both tar and bacula suffers from quite large slow-down when >> >> gzip is used. This is on an Athlon 64 X2 3800+ (2-core), running >> >> >50% idle during backup, leading me to believe that there are room for >> >> improvement. But part of the problem might be in the linux-kernel >> >> (2.6.17.8). At least when tar was running, the gzip process seemed to >> >> move from one CPU-core to the other very frequently. >> BM> Improvement, maybe, but not for Bacula, as far as I can see. If a BM> dual-core system is running at 50%, then 1 core is maxed out. Since >> >> No, this was not the case. Most of the time, both CPU's was idle >30% >> of the time, according to top. BM> It's hard to help when your facts keep changing. Further up, you mention BM> that the CPU is >50% idle, now you say that each independent core is >30% BM> idle. OK, I wrote "both CPU's was idle >30% of the time" when I should have written "both core's was idle >30% of the time". Does that clear up thinks for you? BM> This is not meant as an attack, but you will get absolutely nowhere in BM> performance optimization if you can't get the details right. Details BM> are terribly important. Yes, I totally agree, details are *very* important. But I think you are unfair when you state that "your facts keep changing" for a single wrong word (CPU vs. core) on one occasion. BM> the gzip process is serialized, it can only run on one core at a time, BM> which means the CPU is the limiting factor at this time. >> >> gzip on this computer, on one CPU, reach about 18 Mbyte/s. bacula with >> gzip only reach ~7.7 Mbyte/s. This leads me to believe that there are >> room for improvement. BM> Again, the story changes. Above, you indicate that tar+gzip ran about BM> 15% faster than bacula with gzip, which seems reasonable. Now you're BM> saying that gzip is ~twice as fast as Bacula + gzip. Where did this new BM> number come from? Are you taking in to account networking on this new BM> test? If I state that "gzip on this computer, on one CPU, reach about 18 Mbyte/s", I mean just that, nothing else. To clarify, this means that pure gzip-performance on this computer, using just one gzip-process, is 18 Mbyte/s. Also, my original debian-bug-report, posted on this list by John Goerzen, includes this very same information, so it's hardly new. An important methodology in performance debugging is to try to isolate different factors from each other. This is what I'm trying to do by running a separate gzip-performance-test. You seems to have missed the point: Software-compression makes bacula (and tar) much slower *even* when both CPU-cores are >30% idle most of the time. >> No, I don't expect better performance than the disc-performance. In >> fact, a lot lower than disc-performance is acceptable. The 7.7 Mbyte/s >> we reach now is OK. However, full backup times are long, but as long >> as we can run a full backup in less than 48 hours, it's OK. BM> Again, I'm glad it's working for you. However, as someone who would BM> otherwise be interested in researching this, I don't have any motivation BM> to do so. "full backup times are long" sounds arbitrary enough to BM> disinterest me. Performance optimization of bacula probably isn't the top priority, but I imagine that backup-performance is an issue in many cases. Even if we currently perform our full backup in less than 24 hours, this doesn't mean that backup-performance isn't important. We still haven't migrated all our backup-data to bacula and the data requiring backup is rising. Others are probably in the same situation. One conclusion from my tests is that it probably is possible to improve the backup-performance when using software compression. At least in the case when network isn't a limiting factor, and when 2 or more CPU-cores are available. Another conclusion from my tests is that compression in the storage-daemon, instead of in FD, might improve performance for us. (I say might, I haven't tested this.) Don't take this as a negative criticism against bacula, take it as an opportunity to make bacula even better! We did an evaluation of different backup-software (mostly commercial), and bacula was the best alternative for us. I think that bacula is a great backup-program! / Anders ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
