Hello. > Every time backup starts server slows down significantly, disk > operations become very slow. It may take up to 10 seconds to stat() a > file that is not in filesystem cache. At the same time, rsync on > remote server does not affect disk load much, server works without > slowdown.
Thank you all for the answers. Matthew, yes, I know about --bwlimit in rsync, but this will mostly slow down the transfer of data and not stat() operations that are used for comparing files. I afraid bwlimiting will not make any better (I tried it some time ago without success). Daan, thanks for the patch! I will try it. A lot of impact also produced by rm -rf of old backups. I assume that low performance is also related to a large numbers of hardlinks. There was a moment when I had ~15 backups hardlinked by rsync, and rm -rf of single backup was VERY slow and slowed down the server dramatically. I had no choice except installing a new clean disk for backups, limiting number of future backups and issuing a newfs over old backup disk. When there is less number of hardlinked copies, backups cleanup works much better. Can large number of hardlinks produce such an impact on filesystem operations? May be it is possible to increase disk performance somehow? Server has a lot of memory. At this time vfs.ufs.dirhash_maxmem = 67108864 (max monitored value for vfs.ufs.dirhash_mem was 52290119) and kern.maxvnodes = 500000 (max monitored value for vfs.numvnodes was 450567). Can increasing of these (or other) sysctls help? I ask because (as you can see) these tunables are already incremented, and I am not sure further increment really makes sense. Also, is it possible to limit disk operations for rm -rf somehow? The only idea I have at the moment is to replace rm -rf with 'find | slow_down_script | xargs rm' (or use similar patch as for rsync)... Or, probably, it is possible to limit the IO bandwith for a particular device somehow? I would then limit disk operations for SATA backup disk to prevent it affecting the rest of services that work on SAS mirror. And also, maybe there are other ways to create incremental backups instead of using rsync/hardlinks? I was thinking about generating list of changed files with own script and packing it with tar, but I did not find a way to remove old backups with such an easy way as it is with hardlnks.. Thanks in advance! -- // cronfy _______________________________________________ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
