Hello, I have a web-server (nginx + apache + mysql, FreeBSD 7.3) with many sites. Every night it creates a backup of /home on a separate disk. /home is a RAID1 mirror on Adaptec 3405 (128M write cache) with SAS drives; /backup is a single SATA drive on the same controller.
Rsync creates backups using hardlinks, it stores 7 daily and 4 weekly copies. Total amount of data is ~300G and 11M of files. The server is under heavy web load every time (appox 100 queries/sec). Every time backup starts server slows down significantly, disk operations become very slow. It may take up to 10 seconds to stat() a file that is not in filesystem cache. At the same time, rsync on remote server does not affect disk load much, server works without slowdown. I think that problem can be caused by two reasons: * either bulk of reads on SATA /backup drive, that fills OS filesystem cache and many file access operations require real disk read. * or bulk of writes on /backup fills controller write cache and geom disk operations queue grown, causing all disk operations to wait. This is only my assumption of course, I may be wrong. How can I find a real reason of these slowdowns, to either conclude that it is not possible to solve this because of hardware/software limits, or tune my software/hardware system to make this all work at an acceptable speed? Here is my current sysctl setup (what should I tune?): kern.maxvnodes=500000 vfs.ufs.dirhash_maxmem=67108864 vfs.lookup_shared=1 kern.dirdelay=6 kern.metadelay=5 kern.filedelay=7 sysctl's counters (which others should I monitor?): vfs.numvnodes: 407690 vfs.ufs.dirhash_mem: 27158118 I tried to enable async (in hope it will make rsync faster) or even disable softupdates on /backup partition (in hope it will make rsync slower and OS filesytem cache will not be flushed by backups), it did not help. I also want to try to upgrate to Adaptec 5405 (it has 256M of write cache) or move mysql databases on a separate SAS disk, but I just not quite sure what will help better. What would I start from to diagnose the issue? Thanks in advance! -- // cronfy _______________________________________________ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
