On Thu, Sep 15, 2011 at 4:13 PM, Steve Bohrer <skboh...@simons-rock.edu> wrote: > On Sep 15, 2011, at 3:39 PM, Christopher Morrow wrote: > >> On Thu, Sep 15, 2011 at 3:34 PM, Brian Gold <bg...@simons-rock.edu> wrote: >>> >>> Over the past week, we've discovered that there is an issue with the way >>> some Verizon DSL customers are being routed in Western Massachusetts that >>> is >>> preventing them from reaching my employers public IPs. The problem is >>> only >>> limited to Verizon DSL customers, everyone else can reach these IP >>> addresses >>> just fine. After many hours on the phone with Verizon tech support, I >>> finally managed to get myself and one of my coworker's home dsl >>> connections >>> switched from a "redback router" to a "juniper router" which resolved the >>> issue, but only for us. > > [...] >> >> If you buy verizon services at your day job you can probably make >> noise through your sales droids better than here (sadly)... verizon >> likes to jump when customers have problems, if the customer is a large >> corporation or other 'important' customer. > > > That is just the problem! The college does not buy any Verizon network stuff > directly, so we don't really have any access to their support. (We have a > few cell phones, but not enough to be "important".) > > Brian Gold (who first posted) happens to have their DSL to his house, and he > was one of five who have reported the problem, so that gave him a slight in. > But the only techs he could reach as an "end user" were not high enough up > to fix this problem in a general way. After pressing them for literally > hours, he was able to get transfered to their NOC, and get the problem > resolved for his one address. But, they would not give him the NOC contact, > and he had to repeat this multi-hour process to get it fixed for an other > user. Verizon's DSL support suggested that we get our bandwith provider > involved, and so they tried to pitch in, but they don't have any Verizon NOC > contact either, especially since this issue is purely within a small corner > of Verizon's DSL network, not on any of Verizon's links to our provider. > > This issue hits only a few Verizon DSL users in NW Mass. It does not really > seem like a routing problem, because the affected users can reach many of > the servers in our AS, but not some addresses. Unfortunately, the "blocked" > addresses include our web server and our mail server, so our staff who live > out there noticed the issue pretty quickly. Traceroutes from Brian's house > show that for our blocked hosts, the users don't get beyond Verizon's NAT.
I wasn't aware verizon implemented CGN already... way to be a 'first mover' in this field verizon! > > The Verizon tech's "fix" of re-patching Brian's DSL line in to a different > router feels to me like there is a config problem in the other router, but > the tech we got is not authorized to alter the config. It would be nice if > we could reach someone who could actually edit the broken config and make it > right. Anyone from Verzion's NOC for Western Mass reading this? Or, does > anyone else have useful contact info for them? you probably want someone in the NOC which is (I think) stil in Reston, va... I don't think they have separate noc's per region. The first-line tech folks you chat with on the phone really arent' able (even to login really) to fix devices in the field :( anyways, this looks crappy :( but yeah for CGN being all it's cracked up to be! -chris > > FWIW, Simon's Rock is 208.81.88.0/21, AS 19345. Here are a failed and a good > trace from Brian's house, to different servers on our campus : > > FAILS: > Tracing route to wilbur.simons-rock.edu [208.81.88.15] > over a maximum of 30 hops: > > 1 <1 ms <1 ms <1 ms 192.168.10.1 > 2 1 ms 1 ms 1 ms 192.168.1.1 > 3 53 ms 104 ms 116 ms 10.14.1.1 > 4 * * * Request timed out. > 5 * * * Request timed out. > 6 * * * Request timed out. > 7 * * * Request timed out. > > WORKS: > Tracing route to dev.simons-rock.edu [208.81.88.25] > over a maximum of 30 hops: > > 1 <1 ms <1 ms <1 ms 192.168.10.1 > 2 1 ms 1 ms 1 ms 192.168.1.1 > 3 87 ms 54 ms 54 ms 10.14.1.1 > 4 99 ms 109 ms 103 ms at-0-3-0-1711.WMA-CORE-RTR2.verizon-gni.net > [130.81.10.77] > 5 16 ms 18 ms 16 ms so-7-3-1-0.NY5030-BB-RTR2.verizon-gni.net > [130.81.20.6] > 6 19 ms 17 ms 17 ms 0.xe-3-1-0.BR3.NYC4.ALTER.NET [152.63.2.81] > 7 18 ms 21 ms 18 ms 204.255.168.194 > 8 108 ms 188 ms 116 ms pos5-0-2488M.cr1.BOS1.gblx.net [67.17.94.57] > 9 24 ms 28 ms 23 ms pos0-0-0-155M.ar1.BOS1.gblx.net [67.17.70.162] > 10 121 ms 160 ms 127 ms 64.213.79.250 > 11 77 ms 77 ms 78 ms 208.81.88.25 > > Trace complete. > > Anyways, thanks for any suggestions you can offer. > > Steve Bohrer > Network Administrator > ITS, Bard College at Simon's Rock > 413-528-7645 > > > >
