* Sander ([EMAIL PROTECTED]) wrote: > Dr. David Alan Gilbert wrote (ao):
> > I was using rsync, but the problem with rsync is that I have > > a back up server then filled with lots and lots of small files > > - I want larger files for spooling to tape. > > (Other suggestions welcome) > > Can't you just tar the small files from the backupserver to tape? (or, > what is the problem with that?). Lots of small files->slow; it is an LTO-2 tape drive that is spec'd at 35MByte/s - it won't get that if I'm feeding it from something seeking all over. > > write a lot more data. > > You also do incremental backups? I could - but they are a pain at restore time. > > I've benchmarked write performance on the filesystem at > > 60-70MB/s for a single write process (as shown with iostat) > > for a simple dd if=/dev/zero of=abigfile bs=1024k > > > > My problem is that with the parallel writes iostat is showing > > I'm actually getting ~3MB/s write bandwidth - that stinks! > > How many parallel streams can the system currently handle before the > write bandwith gets unacceptable? I'll be honest I don't know; this was running with 9 streams; but I know the overall speed of the backup goes up as I increase the parallelism from 5 through 9 - but it still sucks. > > The machine is a dual xeon with 1GB of RAM, an intel GigE > > card and a 2.6.11 kernel, a 3ware-9000 series pci-x controller > > with a 1.5TB RAID5 partition running Reiser3. > > What mount options? And how many disks? 7 active discs, raid5; mounted with noatime, nodiratime > > Reiser3 is used because I couldn't get ext3 stable on a filesystem of > > this size (-64ZByte free shown in df), > > That is not a sign of instability per se AFAIK. When I fsck it fixes it - this to me is an indication something is wrong with the ondisc data; now it might only be the freespace totals - but the fact that the disc contents are wrong makes me worry - I don't like having to fsck a 1.5TB partition. > > and xfs didn't seem stable on recovering from an arbitrarily placed > > reset. The 3ware has write caching (with battery backup). > > How is the cache configured in the bios? Write cache is on in the 3ware bios as is the battery backup. > > I'm open for all suggestions. > > Would it be possible to test software raid to see if that gives > different numbers? Erm I guess I could - but the controller does manage 60/70MB/s write as a raw stream, so as far as I can tell if I can persuade the kernel not to chop my writes into silly small chunks things should be good. Dave -- -----Open up your eyes, open up your mind, open up your code ------- / Dr. David Alan Gilbert | Running GNU/Linux on Alpha,68K| Happy \ \ gro.gilbert @ treblig.org | MIPS,x86,ARM,SPARC,PPC & HPPA | In Hex / \ _________________________|_____ http://www.treblig.org |_______/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
