It looks like one or more circuits are down, so you're seeing asymmetrical routing over congested paths in one direction.
-mel via cell > On Feb 12, 2017, at 7:14 PM, Miles Fidelman <mfidel...@meetinghouse.net> > wrote: > > Hi Folks, > > I'm visiting AZ, and seeing some really really poor performance accessing > some of our servers via Cox broadband. The folks at Cox technical support > are useless - all they say is "well you're on a DOCSIS 2 modem." Meanwhile, > everything I'm seeing is several hops upstream of the local segment - and all > that Cox level2 tech support will say is "if there was a backbone problem, > our backbone people would have dealt with it." > > I'm having problems reaching both our own server, and sites like google, > facebook, windows update. > > Traceroutes to and from our server are illustrative - note that for most of > the past week, the average ping time was 85msec. Now we're seeing this: > > From 98.177.135.186 - the public IP address on Cox's local broadband service. > To 107.154.13.58 (ntcorp.server - one of our servers, sitting in a Tierpoint > data center, near Boston) > traceroute to ntcorp.com (207.154.13.58), 64 hops max, 52 byte packets > 1 10.128.128.1 (10.128.128.1) 199.893 ms 75.319 ms 27.295 ms > 2 100.127.69.178 (100.127.69.178) 38.710 ms 40.075 ms 43.598 ms > 3 72.215.229.22 (72.215.229.22) 39.674 ms 201.368 ms * > 4 lag-157.bear2.phoenix1.level3.net (4.28.82.53) 686.499 ms 1837.141 ms > 16.273 ms > 5 ae-1-60.edge1.losangeles6.level3.net (4.69.144.16) 35.498 ms 964.377 ms * > 6 ae-3-80.edge1.losangeles6.level3.net (4.69.144.144) 551.760 ms 525.014 ms > ae-1-60.edge1.losangeles6.level3.net (4.69.144.16) 2061.191 ms > 7 be3036.ccr41.lax04.atlas.cogentco.com (154.54.14.129) 847.778 ms 87.601 > ms 71.504 ms > 8 be2965.ccr22.lax01.atlas.cogentco.com (154.54.45.1) 79.060 ms 225.647 ms > be2964.ccr21.lax01.atlas.cogentco.com (154.54.44.77) 60.306 ms > 9 * be2931.ccr21.phx02.atlas.cogentco.com (154.54.44.85) 2264.071 ms > 185.180 ms > 10 be2929.ccr21.elp01.atlas.cogentco.com (154.54.42.66) 61.208 ms > be2930.ccr21.elp01.atlas.cogentco.com (154.54.42.78) 386.149 ms 1278.868 > ms > 11 be2928.ccr22.iah01.atlas.cogentco.com (154.54.30.161) 384.136 ms > be2927.ccr41.iah01.atlas.cogentco.com (154.54.29.221) 2339.833 ms 615.415 > ms > 12 be2690.ccr42.atl01.atlas.cogentco.com (154.54.28.129) 233.061 ms > be2687.ccr41.atl01.atlas.cogentco.com (154.54.28.69) 87.902 ms > be2690.ccr42.atl01.atlas.cogentco.com (154.54.28.129) 861.159 ms > 13 be2113.ccr42.dca01.atlas.cogentco.com (154.54.24.221) 998.858 ms > be2112.ccr41.dca01.atlas.cogentco.com (154.54.7.157) 249.930 ms * > 14 be2807.ccr42.jfk02.atlas.cogentco.com (154.54.40.109) 768.461 ms > be2806.ccr41.jfk02.atlas.cogentco.com (154.54.40.105) 136.772 ms > be2807.ccr42.jfk02.atlas.cogentco.com (154.54.40.109) 288.225 ms > 15 be2094.ccr21.bos01.atlas.cogentco.com (154.54.30.14) 271.736 ms 166.224 > ms > be2096.ccr22.bos01.atlas.cogentco.com (154.54.30.42) 565.015 ms > 16 te0-4-1-7.agr22.bos01.atlas.cogentco.com (154.54.80.34) 1944.479 ms > te0-4-1-6.agr22.bos01.atlas.cogentco.com (154.54.80.10) 149.803 ms > te0-4-1-6.agr21.bos01.atlas.cogentco.com (154.54.47.230) 897.115 ms > 17 te0-0-2-0.nr11.b000254-0.bos01.atlas.cogentco.com (154.24.9.82) 107.207 > ms > te0-0-2-1.nr11.b000254-0.bos01.atlas.cogentco.com (154.24.9.78) 295.881 > ms 185.453 ms > 18 38.122.127.18 (38.122.127.18) 115.652 ms 461.168 ms 615.526 ms > 19 207.154.0.57 (207.154.0.57) 1871.023 ms 1987.832 ms 2165.248 ms > 20 server1.ntcorp.com (207.154.13.58) 587.560 ms 263.328 ms 333.542 ms > > > Traceroute in the reverse direction: > 1 207.154.13.47 (207.154.13.47) 0.000 ms 0.000 ms 0.000 ms > 2 * * * > 3 h130.207.190.173.static.ip.windstream.net (173.190.207.130) 0.000 ms > 0.000 ms 0.000 ms > 4 xe1-2-0-0.cr01.cley01-oh.us.windstream.net (40.128.250.166) 12.000 ms > 12.000 ms 12.000 ms > 5 et11-0-0-0.cr01.chcg01-il.us.windstream.net (40.128.248.71) 20.000 ms > 20.000 ms 20.000 ms > 6 10gigabitethernet4-1.core1.chi1.he.NET (206.223.119.37) 72.001 ms 20.000 > ms 16.000 ms > 7 chgobbrj01pos010100.r2.ch.cox.net (68.105.30.193) 20.000 ms 20.000 ms > 20.000 ms > 8 chnddsrj01-ae1.0.rd.ph.cox.net (68.1.5.211) 72.001 ms 72.001 ms 72.001 > ms > 9 * * * > 10 * * * > 11 * * * > <snip> > 30 * * * > <looks like a lot of the later hops don't respond to pings> > > Two things jump out at me: > 1. The rather large number of hops from cox to ntcorp - with high delays > from several nodes in both the level3 and cogent networks. > 2. That there's a rather more direct path from the datacenter to cox, that > shows up in the reverse direction. > > Some kind of routing or peering issue, perhaps? (And I also note the earlier > string of messages regarding youtube streaming problems - that also seemed to > involve cox and level3. > > Thanks for any insight (and better, for any fixes!). > > Miles Fidelman > > > > > -- > In theory, there is no difference between theory and practice. > In practice, there is. .... Yogi Berra >
