Re: RFC: slow-down option

2014-04-08 Thread Dave Taht
The biggest problem we've found with modern day gateways is that they are dramatically overbuffered and this messes with tcp's congestion avoidance mode, a lot. Innumerable resources on "bufferbloat" are on bufferbloat.net to this effect: https://www.bufferbloat.net/projects/bloat/wiki/TechnicalI

Re: RFC: slow-down option

2014-04-08 Thread devzero
regarding dynamic slowdown, you may also have a look at: https://bugzilla.samba.org/show_bug.cgi?id=7120 regards Roland List: rsync Subject:Re: RFC: slow-down option From: Marian Marinov Date: 2014-04-03 12:52:53 Message-ID: 533D59A5.4080503 () yuhu ! biz [Download

Re: RFC: slow-down option

2014-04-04 Thread Satish Shukla
With multiple rsync (I run around 10) streams the bwlimit becomes complicated w.r.t. optimizing total bandwidth, its no way near close to what I want to achieve. I want it to be dynamically scale up and down within maximum threshold depending on overall load. trickle (http://monkey.org/~marius/

Re: RFC: slow-down option

2014-04-04 Thread Frank Terhaar-Yonkers
I needed to back up some of my NAS over the WAN to another (friends) NAS (3/4TB-total).  A lot of expensive hi-def music files, mostly very large.  He backs up to mine, vice versa for disaster recovery. There were two issues:  1) sucking up all my UL bandwidth, only

Re: RFC: slow-down option

2014-04-03 Thread Marian Marinov
On 04/03/2014 03:35 PM, Christoph Biedl wrote: Joe wrote... This is way beyond my level of expertise, but wouldn't something like ionice help with that? Although I'm not Marian, probably not. The ionice program does a reasonable good job when it's about prioritizing read operation. The contex

Re: RFC: slow-down option

2014-04-03 Thread Marian Marinov
On 04/03/2014 02:48 PM, Christoph Biedl wrote: Marian Marinov wrote... I've been using rsync on some backup servers for years. In 2011 we had a situation where the FS of the backup server was behaving strange, even thou there was enough available I/O, the fs(ext4 on 16TB partition with a lot of

Re: RFC: slow-down option

2014-04-03 Thread Christoph Biedl
Joe wrote... > This is way beyond my level of expertise, but wouldn't something like > ionice help with that? Although I'm not Marian, probably not. The ionice program does a reasonable good job when it's about prioritizing read operation. The context makes me guess it's rather about writing. >

Re: RFC: slow-down option

2014-04-03 Thread Joe
This is way beyond my level of expertise, but wouldn't something like ionice help with that? Also, check out: 2 more pipe utilities Viewer & throttle http://www.ivarch.com/programs/pv.shtml Throttle - limits bandwidth of a pipe - for use with network transfers http://linux.die.net/man/1/throttl

Re: RFC: slow-down option

2014-04-03 Thread Christoph Biedl
Marian Marinov wrote... > I've been using rsync on some backup servers for years. In 2011 we > had a situation where the FS of the backup server was behaving > strange, even thou there was enough available I/O, the fs(ext4 on > 16TB partition with a lot of inodes) was lagging. After much testing >

RFC: slow-down option

2014-04-03 Thread Marian Marinov
Hello, I've been using rsync on some backup servers for years. In 2011 we had a situation where the FS of the backup server was behaving strange, even thou there was enough available I/O, the fs(ext4 on 16TB partition with a lot of inodes) was lagging. After much testing we found that rsync was