All,
One big thing I failed to mention is I was running rsync inside a
cygwin windows 8.1 setup.
I moved it to a linux box and the behavior is much better. I get a
nice smooth 85-90 MB/sec. That might be the max speed of the source
drive.
I'd still like a way to improve rsync's performance in cygwin, but I
can understand it is a low priority.
Thanks
Greg
--
Greg Freemyer
www.IntelligentAvatar.net
On Mon, Apr 11, 2016 at 4:08 PM, Greg Freemyer <greg.freem...@gmail.com> wrote:
> I hope this isn't a FAQ.
>
> Per the man page I see ways to control the blocksize for hash
> comparison reasons, but no way to control it for i/o performance
> reasons.
>
> I'm using rsync to copy folder trees full of large files and I'd like
> to have control of how much data is read / written at a time. Maybe
> read 10 MB, write 10 MB, etc.
>
> Is there an existing way to do that?
>
> == details ==
>
> When copying a bunch of 1.5 GB files with rsync, I'm only seeing 50%
> of the throughput I hope to see.
>
> I haven't looked at the code, or even run strace, but it seems like
> the code is doing something like:
>
> while (files) {
> read 1.5 GB file to ram
> write 1.5 GB file from ram
> fsync() ensure 1.5 GB file is on disk
> } endwhile
>
> I say that because I see several seconds of high-speed reading, then no reads.
>
> When the reads stop, I see writes kick in, then they stop and reads
> start up again.
>
> The end result is I'm only using 50% of the available bandwidth.
>
> Not that I'm copying my source folder tree to a newly created folder
> tree, so there is not any reading of the destination needed.
> My ultimate would be something like:
>
> while (files) {
> while (data_in_file) {
> read user_defined_blocksize to ram from file
> write user_defined_blocksize from ram to file
> }
> fsync() ensure 1.5 GB file is on disk
> } endwhile
>
> Thanks
> Greg
> --
> Greg Freemyer
> www.IntelligentAvatar.net
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
