On Thu, Aug 28, 2014 at 10:18 AM, Bacon, Ricky (RJ) <rj.ba...@verizon.com> wrote: > Hi Chris!
howdy! > @Ryan, XO was accurate, the peering connection between 6 and 7 is not I had thought Ryan wasn't at XO but an XO customer... I could be mistaken though. congested, nor is it taking any sort of errors. > > I took a look. The significant increase in rtt seems to happen between 7 and > 8 (an Verizon LSP consisting of multiple physical routers). That hop > traverses the yikes :) lsp's from XT -> LCR now, neat. it's also possible (and likely) that the return path from VZ customer -> Ryan is across either the NYC or DCA links to XO (nyc4 or dca6?)... I agree that the ~65ms cross-country seems on-par with SJC -> PHL on a good day though. continental US from San Jose to Philadelphia, so you would expect that, and I don't see anything going on in that path. The end to end rtt is pretty normal, and in any case, it shouldn't prevent your customers from connecting. I am not seeing any packet loss on that path. It's possible that whatever they saw was a transient issue and may be over now. > > So, please double check with your customers to verify that they still have > problems. If so, continue to escalate the ticket. > hopefully ryan's having a better day today :) > -- rj > > -----Original Message----- > From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Christopher Morrow > Sent: Wednesday, August 27, 2014 10:52 PM > To: ryanruuska+na...@gmail.com > Cc: nanog list > Subject: Re: Verizon connectivity issues > > On Mon, Aug 25, 2014 at 1:54 PM, Ryan Ruuska <ryanruuska+na...@gmail.com> > wrote: >> Hello all, >> >> We have heard from many of our customers in the Northeast region, >> specifically PA, MD, and VA who have difficulty connecting to our >> website > > there's probably vz customer folk on-list, maybe 'what should they test for > you' would be nice? :) > >> (very slow loading times). We have noticed that if our data center >> provider specifically routes our outbound traffic over Hurricane >> Electric rather than XO Communications, that connectivity is restored >> to normal for our customers. The HE path goes from Salt Lake City to >> Denver where it is handed off to TeliaSonera, and they send it to >> Chicago where it gets handed off to Verizon. This works great and >> generally has a ping response of 20-25ms less than the bad route as posted >> below from one of our servers: >> >> Tracing route to pool-108-52-xxx-xxx.phlapa.fios.verizon.net >> [108.52.xxx.xxx] >> over a maximum of 30 hops: >> >> 1 <1 ms <1 ms <1 ms 172.xxx.xxx.xxx (Internal IP) >> 2 <1 ms 10 ms <1 ms 209-41-xxx-xxx.c7dc.com [209.41.xxx.xxx] >> 3 <1 ms <1 ms <1 ms e1-5.rt08.gp1.c7dc.com [192.41.0.97] >> 4 2 ms 1 ms 1 ms ip65-46-51-113.z51-46-65.customer.algx.net >> [65.46.51.113] >> 5 19 ms 18 ms 19 ms vb1611.rar3.sanjose-ca.us.xo.net >> [216.156.0.5] >> 6 21 ms 18 ms 18 ms 207.88.14.226.ptr.us.xo.net [207.88.14.226] >> 7 18 ms 18 ms 18 ms 206.111.6.122.ptr.us.xo.net [206.111.6.122] >> >>>> Verizon owned device with an XO IP address >> 8 87 ms 86 ms 87 ms b100.phlapa-lcr-22.verizon-gni.net >> [130.81.209.187] >> >>>> Next hop goes from CA to PA, probably just hidden routing info >> 9 * * * Request timed out. >> 10 88 ms 89 ms 90 ms pool-108-52-xxx-xxx.phlapa.fios.verizon.net >> [108.52.xxx.xxx] >> >> As you can see, XO sends this traffic from Salt Lake City to San Jose >> where it is handed off to Verizon. We've worked with XO and >> determined that there are no connectivity issues along their path, >> which seems to point to an issue within Verizon's network. I have a >> couple of tickets open with Verizon at the moment on behalf of some of >> our customers, but we have had trouble breaking through Tier 1. >> >> Our first thought was saturation of the peering point, but XO states >> that the link between hops 6 and 7 is using 5Gbps out of 20Gbps >> capacity, and the latency on hop 7 (the Verizon owned device with an >> XO IP) seems normal whenever we test. >> >> Any Verizon engineers out there willing to take a quick peek at this >> route for any obvious issues? It would be a lot easier for us if >> Verizon provided Looking Glass servers, but alas... > > they have route data sent to routeviews at least...
