On 9/26/07, Ken Naim <[EMAIL PROTECTED]> wrote: > As for bad io experiences, our core production system use raw devices for > our databases so we don't have the same issue(s), this is our production > reporting system that gets cloned over nightly. So the process removes all > the existing files and then writes new versions of them from a backup onto a > file system. I have noticed the poor io performance since I came onsite but > the unix team keeps saying everything is fine.
They probably just mean that nothing is reporting an error. Unless you have a performance SLA agreed with them, that is. [If you do have a performance SLA with them you should start by measuring how many blocks of I/O you are issuing from your host while trying to delete the files] > This rm issue is causing the > database clone process to exceed its allocated downtime window so I thought > I'd start there. > > If anyone can point me to any specific information on tuning ext3 file > system I'd appreciate it. I am googling it now. If you are not space limited, you will find it faster in terms of wall time to move the old files to a staging area (using either mv(1) or rename(2)) on the same filesystem, launch a file removal process in the background, and meanwhile create the new files (from the backup you refer to). This will probably get you the biggest win (assuming there are a reasonable number of spindles to spread the I/O load). Another option here is to issue all the unlink system calls in parallel. Again, this assumes that the performance limitation is due to SAN latency rather than I/O bandwidth. If these files are the only things on this filesystem, the ext3 journalling is gaining you nothing except a limited fsck time. You could try turning off journalling or moving the journal to a different LUN. However, if you go down this route you will need to script something to avoid fsck on startup after unclean shutdown (e.g. by making a new, empty, filesystem with mkfs). Alternatively, if your production database supports point-in-time recovery, you could just snapshot the LUNs on the production system, copy the transaction logs to the clone system, attach the snapshot LUNs to it, switch the snapshots to read-write, perform point-in-time recovery with the transaction logs, and then use the copy of the database you have there. James. _______________________________________________ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
