Re: Level 3 voice outage

2016-10-04 Thread Mel Beckman
Possibly somebody YANGed when they should have yinged :) -mel beckman On Oct 4, 2016, at 1:06 PM, Marco Teixeira mailto:ad...@marcoteixeira.com>> wrote: Yeap, i know, it was what i understood, as it is my opinion that a zero day would fit better... in the pure speculation world :) At the end

Re: Level 3 voice outage

2016-10-04 Thread Marco Teixeira
Yeap, i know, it was what i understood, as it is my opinion that a zero day would fit better... in the pure speculation world :) At the end of the day... maybe some undocumented fault int some obscure functionality that was activated/deployed a long time ago, and just revealed it self now... There

Re: Level 3 voice outage

2016-10-04 Thread Mel Beckman
Just so. We are just speculating, but in light of recent GRE attack developments, the ddos possibility is worth considering. -mel beckman On Oct 4, 2016, at 12:54 PM, Shawn Ritchie mailto:shawnritc...@gmail.com>> wrote: Well, Level3 has by no means said that this was the result of a DDoS, tha

Re: Level 3 voice outage

2016-10-04 Thread Marco Teixeira
... but back then they didn't have ITIL and the likes (at least not that i knew... but then again, i was just starting with networking back then), where a ChM process must produce risk, impact, etc, reports, and at least one peer review meeting for approval :) = Marco On Tue, Oct 4, 2016 at 8:33

Re: Level 3 voice outage

2016-10-04 Thread Marco Teixeira
I won't believe a company like Level3 would not deploy backplane protection/policing on routers. Also, 1Tb/s aggregated DDoS towards OVH network didn't pause or rebooted routers. And i guess both companies have had their share of (D)DoS in the past, so they had the time to get up to the challenge.

Re: Level 3 voice outage

2016-10-04 Thread Valdis . Kletnieks
On Tue, 04 Oct 2016 20:10:44 +0100, Marco Teixeira said: > Had it been an approved chance it would have been > rolled back i guess... See the 1990 ATT long-distance collapse for a worked example. pgp_Sro2vzM5h.pgp Description: PGP signature

Re: Level 3 voice outage

2016-10-04 Thread Mel Beckman
765 Gbps per second directed at a router’s interface IP might give the router pause, so to speak :) -mel On Oct 4, 2016, at 12:10 PM, Marco Teixeira mailto:ad...@marcoteixeira.com>> wrote: Multiple reboots across several markets... Does not seem something that full pipes would trigger. Had

Re: Level 3 voice outage

2016-10-04 Thread Marco Teixeira
Multiple reboots across several markets... Does not seem something that full pipes would trigger. Had it been an approved chance it would have been rolled back i guess... On the other hand, a zero day could apply... Em 04/10/2016 19:54, "Mel Beckman" escreveu: > Sure. The recent release of the I

Re: Level 3 voice outage

2016-10-04 Thread Mel Beckman
Sure. The recent release of the IoT DDoS attack code in the wild. -mel > On Oct 4, 2016, at 11:42 AM, valdis.kletni...@vt.edu wrote: > > On Tue, 04 Oct 2016 18:14:54 -, Mel Beckman said: > >> This could be DoS attack. > > Or a missing comma in a code update. > > Or a fumble-fingered NOC

Re: Level 3 voice outage

2016-10-04 Thread Valdis . Kletnieks
On Tue, 04 Oct 2016 18:14:54 -, Mel Beckman said: > This could be DoS attack. Or a missing comma in a code update. Or a fumble-fingered NOC monkey. Or You have any reason to suspect a DoS attack rather than all the other possibilities? pgp03Ijae5BSG.pgp Description: PGP signature

Re: Level 3 voice outage

2016-10-04 Thread Mel Beckman
We are seeing high latencies and loss of transit through Level3 in Southern California. Level 3 has opened Global Ticket 115724: Network devices in multiple markets in North America are rebooting, impacting IP services.” This could be DoS attack. > On Oct 4, 2016, at 11:06 AM, Jason Hellenth

Re: 10G tester recommendations?

2016-10-04 Thread Saku Ytti
On 3 October 2016 at 22:09, Bryan Holloway wrote: > We're in the market for a hand-held 10G ethernet tester, and I was curious > if the NANOG community had any recommendations or experiences they would be > willing to share, negative or positive. What are you looking to test? Are you interested i

Re: Level 3 voice outage

2016-10-04 Thread Jason Hellenthal
Patience Obi Wan ! They are investigating the root cause and like most root causes they don’t just hold out a flag and say here I am !!! > On Oct 4, 2016, at 10:32, Ivaylo Katovski wrote: > > When will L3 notify their customers for the outage?!? According to l3 > twitter account their are aware

Re: Level 3 voice outage

2016-10-04 Thread Ivaylo Katovski
When will L3 notify their customers for the outage?!? According to l3 twitter account their are aware of the voice impact and working on it On Oct 4, 2016 11:03 AM, "Mark Stevens" wrote: > Is anyone noticing issue with Level 3 voice? I can't even call their 800 > number using one of my other car

RE: [BULK] Re: Level 3 voice outage

2016-10-04 Thread Asche, Ron
Boise Idaho is affected... -Original Message- From: NANOG [mailto:nanog-bounces+ascher=slhs@nanog.org] On Behalf Of Benjamin Hatton Sent: Tuesday, October 04, 2016 9:04 AM To: nanog@nanog.org Subject: [BULK] Re: Level 3 voice outage Importance: High WARNING: This email originated

RE: Level 3 voice outage

2016-10-04 Thread Ken Warnimont
There's all sorts of people online reporting problems with Level3 voice services - we're down for outbound calling, inbound is spotty. Their portal is unreachable, can't get to any of their #'s...seems like the world is on fire over there. We were a previous TW Telecom customer before Level3 b

Re: Level 3 voice outage

2016-10-04 Thread Shane Heupel
We can't make any LD calls through Level3 either. Thank you, Shane Heupel > On Oct 4, 2016, at 10:06 AM, Benjamin Hatton wrote: > > We just got a notification from one of our upstream Voice providers that > they are seeing problems both inbound and outbound that seems to be Level 3 > related >

Re: Level 3 voice outage

2016-10-04 Thread Mark Horton
Major Level 3 outage; they are currently not taking support calls. On 10/4/16, 8:01 AM, "NANOG on behalf of Mark Stevens" wrote: >Is anyone noticing issue with Level 3 voice? I can't even call their 800 >number using one of my other carriers. > >Mark

Re: Level 3 voice outage

2016-10-04 Thread C. Jon Larsen
On Tue, 4 Oct 2016, Mark Stevens wrote: Is anyone noticing issue with Level 3 voice? I can't even call their 800 number using one of my other carriers. Yes, major outage: http://downdetector.com/status/level3 Customers all over the mid-atlantic region are down.

RE: 10G tester recommendations?

2016-10-04 Thread Robert Jacobs
JDSU 5800 series very user friendly... nice reports... VNC remote connection so you can trigger test with out rolling a tech.. Robert Jacobs | Network Director/Architect Direct:  832-615-7742 Main:   832-615-8000 Fax:    713-510-1650 5959 Corporate Dr. Suite 3300; Houston, TX 77036 A

Re: Legislative proposal sent to my Congressman

2016-10-04 Thread Anne Mitchell
(Interesting and inarguably well-intentioned, and possibly even sound, idea snipped, but noted.) There are a handful of reasons that this will never happen (well, I'm 98% certain it will never happen, nothing is every 100% sure when it comes to the law, and legislation)... among them the manufa

RE: Level 3 voice outage

2016-10-04 Thread timrutherford
We are back in action here in MA. -Original Message- From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of timrutherf...@c4.net Sent: Tuesday, October 4, 2016 11:20 AM To: nanog@nanog.org Subject: RE: Level 3 voice outage Our upstream Voice provider is also having inbound issues with

Re: Level 3 voice outage

2016-10-04 Thread Sean
Twitter is blowing up ( https://twitter.com/hashtag/outage ) and their website (www.level3.com) is down for me so yeah... I'd say they're having a frowny face day. On Tue, Oct 4, 2016 at 10:20 AM, wrote: > Our upstream Voice provider is also having inbound issues with numbers > that were ported

Re: Level 3 voice outage

2016-10-04 Thread Adair Winter
Our provider is down as well. Saying Level 3 issues. Anyone know what's going on or and ETR? On Tue, Oct 4, 2016 at 10:06 AM, Kevin Kadow wrote: > Level 3 has issues nationwide today. We have voice outages at our offices > in multiple cities, Chicago and out west. > > On Tue, Oct 4, 2016 at 11:

RE: Level 3 voice outage

2016-10-04 Thread timrutherford
Our upstream Voice provider is also having inbound issues with numbers that were ported into their system, but the main trunk number is working. -Tim -Original Message- From: NANOG [mailto:nanog-bounces+timrutherford=c4@nanog.org] On Behalf Of Benjamin Hatton Sent: Tuesday, October

Re: Level 3 voice outage

2016-10-04 Thread Matt Freitag
There's a long thread over at the outages list. Sign-up for the outages list is at https://puck.nether.net/mailman/listinfo/outages and the archive for the whole thread starts at https://puck.nether.net/pipermail/outages/2016-October/009609.html. Matt Freitag Network Engineer I Information Technol

Re: Level 3 voice outage

2016-10-04 Thread Kevin Kadow
Level 3 has issues nationwide today. We have voice outages at our offices in multiple cities, Chicago and out west. On Tue, Oct 4, 2016 at 11:01 AM, Mark Stevens wrote: > Is anyone noticing issue with Level 3 voice? I can't even call their 800 > number using one of my other carriers. > > Mark >

Re: Level 3 voice outage

2016-10-04 Thread Benjamin Hatton
We just got a notification from one of our upstream Voice providers that they are seeing problems both inbound and outbound that seems to be Level 3 related *Ben Hatton* Network Engineer Haefele TV Inc. d:(607)589-8000 bhat...@htva.net www.htva.net On Tue, Oct 4, 2016 at 11:01 AM, Mark Steve

Level 3 voice outage

2016-10-04 Thread Mark Stevens
Is anyone noticing issue with Level 3 voice? I can't even call their 800 number using one of my other carriers. Mark

Re: 10G tester recommendations?

2016-10-04 Thread Brian Mengel
We've used the Veex TX300 (now discontinued and replaced with the TX230S I believe) and been satisfied. Our primary use is RFC 2544 testing, so I don't know much about it's capabilities beyond that. On Tue, Oct 4, 2016 at 9:39 AM, Dustin Jurman wrote: > We utilize the EXFO product line and it's

RE: 10G tester recommendations?

2016-10-04 Thread Dustin Jurman
We utilize the EXFO product line and it's a quality product. Dustin Dustin Jurman CEO Rapid Systems Corporation 1211 N. West Shore Blvd. Suite 711 Tampa, FL 33607 Ph: 813-232-4887 Cell: 813-892-7006 http://www.rapidsys.com “Building Better Infrastructure”  -Original Message- Fr

ATT contact?

2016-10-04 Thread Ken Breeman
Could someone from AT&T please contact me off list? Your residential Internet DNS servers are redirecting all traffic destined for Akamai hosted sites to your homepage... Thanks, Ken

Re:Kudos to Rogers Wireless on IPv6 deployment

2016-10-04 Thread Ida Leung
Yes, we have started the IPv6 enablement of our wireless network. West was completed dual stack on Sept 29, Ontario will come next then east region. Rogers Internet has completed all the IPv6 enablement. Fido Internet is coming! Please email me directly for your IPv6 experience with Rogers.

10G tester recommendations?

2016-10-04 Thread Bryan Holloway
We're in the market for a hand-held 10G ethernet tester, and I was curious if the NANOG community had any recommendations or experiences they would be willing to share, negative or positive. Currently we're looking at the EXFO MAX-800 and NetScout's AT 10G, but we're open to other suggestions.