Re: time variable throttling rsync traffic

2009-09-17 Thread Eric S. Johansson
On 9/16/2009 11:11 AM, Andrew Gideon wrote: On Tue, 15 Sep 2009 22:01:04 +, Andrew Gideon wrote: It can also potentially be extended in other directions. For one crazy example, the utility (or some other utility that modifies the first utilities configuration) could listen on a port for me

Re: time variable throttling rsync traffic

2009-09-16 Thread Andrew Gideon
On Tue, 15 Sep 2009 22:01:04 +, Andrew Gideon wrote: > It can also potentially be extended in other directions. For one crazy > example, the utility (or some other utility that modifies the first > utilities configuration) could listen on a port for messages from - > presumably - the receivin

Re: time variable throttling rsync traffic

2009-09-15 Thread Andrew Gideon
On Tue, 15 Sep 2009 12:11:03 -0400, Eric S. Johansson wrote: > run rsync till a given time deadline, killing off the original program > instance and then restart with a new bandwidth limit. I would probably > use a small program invoking rsync and then sending a signal when "it's > Time" then sta

Re: time variable throttling rsync traffic

2009-09-15 Thread Matthias Schniedermeyer
On 15.09.2009 12:11, Eric S. Johansson wrote: > On 9/14/2009 3:55 PM, Andrew Gideon wrote: > >>> If there is one or more bottleneck link in the network (places where >>> traffic feeds from one or more links with aggregate larger capacity >>> into a link with smaller capacity) then it makes sense th

Re: time variable throttling rsync traffic

2009-09-15 Thread Eric S. Johansson
On 9/14/2009 3:55 PM, Andrew Gideon wrote: If there is one or more bottleneck link in the network (places where traffic feeds from one or more links with aggregate larger capacity into a link with smaller capacity) then it makes sense that this work has to be done on the router that is on the se

Re: time variable throttling rsync traffic

2009-09-14 Thread Andrew Gideon
> I would think priority queuing is > better than shaping in this case. I'm afraid I'm not following you here. As I've learned it, priority queuing is one of several tools available to achieve shaping. No? [...] > > If there is one or more bottleneck link in the network (places where > traf

Re: time variable throttling rsync traffic

2009-09-14 Thread Phil Vandry
On Mon, 14 Sep 2009 13:25:55 + (UTC), Andrew Gideon wrote: > If a router is involved, it can do egress shaping on the local side. > That's best. If a router is not involved, then the server must do I don't think that's best at all. I would think priority queuing is better than shaping in t

Re: time variable throttling rsync traffic

2009-09-14 Thread Andrew Gideon
On Mon, 14 Sep 2009 13:09:41 -0400, Eric S. Johansson wrote: > On 9/14/2009 9:25 AM, Andrew Gideon wrote: > >> So control is most effective at the sending rsync, which suggests that >> bwlimit is a good approach. But the most information is available at >> the receiving router, suggesting that s

Re: time variable throttling rsync traffic

2009-09-14 Thread Eric S. Johansson
On 9/14/2009 9:25 AM, Andrew Gideon wrote: So control is most effective at the sending rsync, which suggests that bwlimit is a good approach. But the most information is available at the receiving router, suggesting that shaping at the router is also a good approach. Interesting. Exactly wha

Re: time variable throttling rsync traffic

2009-09-14 Thread Andrew Gideon
On Mon, 14 Sep 2009 14:45:02 +1200, Nathan Ward wrote: > Unless you do it properly, and do your QoS on routers in the middle. This is true. But there are considerations. I became curious about this, so I did some reading to refresh my memory. First, keep in mind that we're talking about contr

Re: time variable throttling rsync traffic

2009-09-13 Thread Nathan Ward
On 14/09/2009, at 2:22 PM, Eric S. Johansson wrote: Unfortunately, the QOS solution only works for the platform you develop it for. On the other hand, the bwlimit solution works for almost every platform but doesn't behave well if there are multiple rsync clients talking to one hopes. The

Re: time variable throttling rsync traffic

2009-09-13 Thread Andrew Gideon
On Sun, 13 Sep 2009 22:22:34 -0400, Eric S. Johansson wrote: > It is all within one tool and there's no way you can hurt or damage > anyone else through its use. It is also within one instance of the tool. What if two of your remote users rsync at the same time? Twenty? What if someone has a

Re: time variable throttling rsync traffic

2009-09-13 Thread Eric S. Johansson
On 9/13/2009 9:20 PM, Matt McCutchen wrote: On Sun, 2009-09-13 at 21:02 -0400, Eric S. Johansson wrote: I am using rsync to back up across a VPN. Unfortunately, every so often the home office miscreants drop a big block of data into the backup and that particular backup cycle takes many hours.

Re: time variable throttling rsync traffic

2009-09-13 Thread Andrew Gideon
On Sun, 13 Sep 2009 21:20:01 -0400, Matt McCutchen wrote: > How about the suggestions you were given on the rsnapshot list? Assuming that you're using Linux somewhere in the mix, its ability to put different network traffic into different pools for purposes of rate management is (1) admittedly

Re: time variable throttling rsync traffic

2009-09-13 Thread Matt McCutchen
On Sun, 2009-09-13 at 21:02 -0400, Eric S. Johansson wrote: > I am using rsync to back up across a VPN. Unfortunately, every so often the > home office miscreants drop a big block of data into the backup and that > particular backup cycle takes many hours. These same people also complain > whe