On 12/19/2010 02:16 PM, Joel Jaeggli wrote:
On 12/9/10 7:20 AM, Mikael Abrahamsson wrote:
On Thu, 9 Dec 2010, Vasil Kolev wrote:
I wonder why this hasn't made the rounds here. From what I see, a
change in this part (e.g. lower buffers in customer routers, or a
change (yet another) to the congestion control algorithms) would do
miracles for end-user perceived performance and should help in some
way with the net neutrality dispute.
It's really hard to replace all the home user's hardware. Trying to
"fix" the problem by fixing all of that is much more painful (and
expensive) than fixing the network to not have the buffers.
I'd say this is common knowledge and has been for a long time.
Common knowledge among whom? I'm hardly a naive Internet user.
And the statement is wrong: the large router buffers have effectively
destroyed TCP's congestion avoidance altogether.
In the world of CPEs, lowest price and simplicity is what counts, so
nobody cares about buffer depth and AQM, that's why you get ADSL CPEs
with 200+ ms of upstream FIFO buffer (no AQM) in most devices.
200ms is good; but it is often up to multiple *seconds*. Resulting
latencies on broadband gears are often horrific: see the netalyzr plots
that I posted in my blog. See:
http://gettys.wordpress.com/2010/12/06/whose-house-is-of-glasse-must-not-throw-stones-at-another/
Dave Clark first discovered bufferbloat on his DSLAM: he used the 6
second latency he saw to DDOS his son's excessive WOW playing.
All broadband technologies are affected, as are, it turns out, all
operating systems and likely all home routers as well (see other posts
I've made recently). DSL, cable and FIOS all have problems.
How many of retail ISP's service calls have been due to this terrible
performance?
I know I was harassing Comcast with multiple service calls over a year
ago over what I now think was bufferbloat. And periodically for a
number of years before that (roughly since DOCSIS 2 deployed, I would
guess).
"The Internet is slow today, Daddy" was usually Daddy saturating the
home link, and bufferbloat the cause. Every time they would complain,
I'd stop what I was doing, and the problem would vanish. A really nice
willow the wisp...
you're going to see more of it, at a minimum cpe are going to have to be
able to drain a gig-e into a port that may be only 100Mb/s. The QOS
options available in a ~$100 cpe router are adequate for the basic purpose.
But the port may only be 1 Mb/second; 802.11g is 20Mbps tops; but drops
to 1Mbps in extremis.
So the real dynamic range is at least a factor of 1000 to 1.
d-link dir-825 or 665 are examples of such devices
Yes, and E3000's and others. Some are half measures, and have a single
knob for both shaping uplinks and downlink bandwidth.
The QOS features in home routers can help, but does not solve all
problems.
In part, because as broadband bandwidth increases, the bottleneck link
may shift/often shifts to the home router to edge device links, and
there are similar (or even worse) bufferbloat problems in both the home
routers and operating systems.
Personally I have MQC configured on my interface which has assured bw
for small packets and ssh packets, and I also run fair-queue to make tcp
sessions get a fair share. I don't know any non-cisco devices that does
this.
the consumer cpe that care seem to be mostly oriented along keeping
gaming and voip from being interfereed with by p2p and file transfers.
An unappreciated issue is that these buffers have destroyed TCP (and all
other congestion avoiding protocols) congestion avoidance.
Secondly, any modern operating system (anything other than Windows XP),
implements window scaling, and will, within about 10 seconds, *fill* the
buffers with a single TCP connection, and they stay full until traffic
drops enough to allow them to empty (which may take seconds). Since
congestion avoidance has been defeated, you get nasty behaviour out of TCP.
Congestion avoidance depends on *timely* notification to the end points
of congestion: these buffers have destroyed the *timely* requirement of
a fundamental presumption of internet protocol design.
If you think that simultaneously:
1) destroying congestion avoidance
2) destroying slow-start, as many major web sites are
by increasing their initial window
3) browsers are now using many TCP connections simultaneously
4) while the TCP traffic shifts to window scaling, enabling
even a single TCP connection to fill these buffers.
5) increasing numbers of large uploads/downloads (not just
bittorrent, HD movie delivery to disk, backup, crash dump
uploads, etc)
is a good idea, you aren't old enough to have experienced the NSFnet
collapse during the 1980's (as I did). I have post-traumatic stress
disorder from that experience; I'm worried about the confluence of these
changes, folks.
And there are network neutrality aspects to bufferbloat: since carriers
have been provisioning their telephony service separate from their
internet service, *and* there are these bloated buffers, *and* there is
no classification that end users can perform over their broadband
connections, you can't do as well as a carrier even with fancy home
routers for any low latency service such as voip. See:
http://gettys.wordpress.com/2010/12/07/bufferbloat-and-network-neutrality-back-to-the-past/
Personally, I don't think this was by malice of forethought, but it's
not a good situation.
The best you can do is what Ooma has done; bandwidth shaping along with
being closest to the broadband connection (or by fancy home routers with
classification and bandwidth shaping). That won't help the downstream
direction where a single other user (or yourself), can inject large
packet bursts routinely by browsing web sites like YouTube or Google
images (unless some miracle occurs, and the broadband head ends are
classifying traffic in the downstream direction over those links).
- Jim
