"On 5/13/19 12:11 PM, dan...@pyranah.com wrote: > Does anyone have contacts at Charter (Spectrum) and Cox? For some reason, > our IP has been blocked by them and our customers are unable to send email > via their charter/cox accounts. Thanks
Would you be talking about port 25/tcp outbound? Lots of ISPs will block port 25 as a rule; you have to call to have it opened. At the same time, you can request the PTR record you need to keep big mail receivers happy." It's the outgoing smtp ports, 587 for charter and 465 for cox. They are open on my end but our IP address that most of our customers are natted through is blocked by the email servers. Daniel Temple VP of Operations Pyranah Communications, LLC 828-743-2470 ext.205 Pyranah.com Like us on Facebook! -----Original Message----- From: NANOG <nanog-boun...@nanog.org> On Behalf Of nanog-requ...@nanog.org Sent: Tuesday, May 14, 2019 8:00 AM To: nanog@nanog.org Subject: NANOG Digest, Vol 136, Issue 14 Send NANOG mailing list submissions to nanog@nanog.org To subscribe or unsubscribe via the World Wide Web, visit https://mailman.nanog.org/mailman/listinfo/nanog or, via email, send a message with subject or body 'help' to nanog-requ...@nanog.org You can reach the person managing the list at nanog-ow...@nanog.org When replying, please edit your Subject line so it is more specific than "Re: Contents of NANOG digest..." Today's Topics: 1. Issues with SIP packets between VZ Fios and NTT (Dovid Bender) 2. Re: Ownership of Routers on Both Ends of Transnational Links (Pengxiong Zhu) 3. Re: Issues with SIP packets between VZ Fios and NTT (Brielle Bruns) 4. Re: Ownership of Routers on Both Ends of Transnational Links (Christopher Morrow) 5. Re: Issues with SIP packets between VZ Fios and NTT (Dovid Bender) 6. Re: Ownership of Routers on Both Ends of Transnational Links (Pengxiong Zhu) 7. Re: Issues with SIP packets between VZ Fios and NTT (Brielle Bruns) 8. Re: Issues with SIP packets between VZ Fios and NTT (Dovid Bender) 9. Re: Ownership of Routers on Both Ends of Transnational Links (Christopher Morrow) 10. Re: Issues with SIP packets between VZ Fios and NTT (Dovid Bender) 11. Re: Issues with SIP packets between VZ Fios and NTT (Jared Mauch) 12. Re: Issues with SIP packets between VZ Fios and NTT (Dovid Bender) 13. Re: Issues with SIP packets between VZ Fios and NTT (Pete Rohrman) 14. Charter and Cox contacts (dan...@pyranah.com) 15. Mostly name and shame... (b...@theworld.com) 16. Re: Ownership of Routers on Both Ends of Transnational Links (Randy Bush) 17. Re: Charter and Cox contacts (Stephen Satchell) 18. RE: FCC Hurricane Michael after-action report (frnk...@iname.com) 19. Re: FCC Hurricane Michael after-action report (Mike Bolitho) 20. Re: Issues with SIP packets between VZ Fios and NTT (Saku Ytti) 21. Re: NTP for ASBRs? (Mark Tinka) 22. Re: NTP for ASBRs? (colin johnston) 23. Fwd: [afnog] SAFNOG-5 & iWeek 2019 Registrations Open (Mark Tinka) 24. Re: Issues with SIP packets between VZ Fios and NTT (Dovid Bender) ---------------------------------------------------------------------- Message: 1 Date: Mon, 13 May 2019 11:21:12 -0400 From: Dovid Bender <do...@telecurve.com> To: NANOG <nanog@nanog.org> Subject: Issues with SIP packets between VZ Fios and NTT Message-ID: <cam3tth2qpkwod2dpy2gu-9hhkkux8gvexfjymmq0yeecyub...@mail.gmail.com> Content-Type: text/plain; charset="utf-8" Hi, Over the last 48 hours we have been getting a lot of alerts of customers phones losing registrations to us. All the complaints are coming from customers that are on VZ Fios in the NYC area. Anyone else see anything strange going on? TIA. Dovid -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190513/37343019/attachment-0001.html> ------------------------------ Message: 2 Date: Sat, 11 May 2019 17:24:27 -0700 From: Pengxiong Zhu <pzhu...@ucr.edu> To: "North American Network Operators' Group" <nanog@nanog.org> Cc: Zhiyun Qian <zhiy...@cs.ucr.edu>, Zhongjie Wang <zwang...@ucr.edu>, Keyu Man <kman...@ucr.edu> Subject: Re: Ownership of Routers on Both Ends of Transnational Links Message-ID: <caenbxkuzz8zdqafn3cnn6q5zo3ydsmimsj83mejhl5wf_my...@mail.gmail.com> Content-Type: text/plain; charset="utf-8" Thanks again for your insightful responses! The case we discuss above is Chinese ISPs renting routers located outside China and the IPs belong to other ISPs. How about the case that the IP belongs to a Chinese ISP and is located in US(from RTT result), can we say it is very likely or definitely owned/operated by the Chinese ISP? Why would some ISP try to rent routers of Chinese ISP in US? For example, a traceroute from Ohio to an IP in China. Hop 17 and hop 18 should be located in US based on the RTT, and yet they belong to a Chinese AS(China Telecom). Does this mean that Chinese Telecom is managing these two hops? HOST: Loss% Snt Last Avg Best Wrst StDev 6. AS??? 100.65.11.97 0.0% 100 2.0 1.0 0.4 12.6 1.3 7. AS??? 52.93.15.238 0.0% 100 2.4 2.0 1.5 11.4 1.1 8. AS??? 52.93.14.134 0.0% 100 21.9 26.3 4.2 54.4 11.3 9. AS??? 52.93.14.119 0.0% 100 2.6 2.1 1.6 10.8 1.2 10. AS??? 100.91.27.86 0.0% 100 25.8 26.2 25.6 34.9 1.2 11. AS??? 54.239.42.197 0.0% 100 25.5 25.9 25.4 35.8 1.5 12. AS??? 100.91.4.218 0.0% 100 25.9 26.2 25.1 38.3 1.6 13. AS??? 100.91.4.217 0.0% 100 25.4 26.0 25.3 41.4 2.0 14. AS??? 100.91.5.85 0.0% 100 25.3 25.8 25.2 29.1 0.9 15. AS??? 54.239.103.86 0.0% 100 25.6 30.0 25.2 49.1 3.8 16. AS??? 54.239.103.77 0.0% 100 25.3 25.6 25.2 28.1 0.5 17. AS4134 218.30.53.1 0.0% 100 28.0 29.1 25.2 33.1 2.3 18. AS4134 202.97.50.21 0.0% 100 32.4 29.1 25.2 33.5 2.4 19. AS??? ??? 100.0 100 0.0 0.0 0.0 0.0 0.0 20. AS??? ??? 100.0 100 0.0 0.0 0.0 0.0 0.0 21. AS4134 202.97.94.121 0.0% 100 186.8 185.6 181.8 189.8 2.3 22. AS4816 119.147.222.6 0.0% 100 182.6 183.5 182.4 195.8 1.8 23. AS4816 183.2.182.130 0.0% 100 181.7 183.3 181.5 207.0 3.9 24. AS??? ??? 100.0 100 0.0 0.0 0.0 0.0 0.0 25. AS45102 116.251.113.158 0.0% 100 176.7 177.9 176.5 186.7 2.1 26. AS45102 116.251.115.141 0.0% 100 213.2 213.4 213.1 218.5 0.6 Best, Pengxiong Zhu Department of Computer Science and Engineering University of California, Riverside On Tue, Apr 16, 2019 at 7:37 PM Erik Sundberg <esundb...@nitelusa.com> wrote: > May sure when you are dealing with transnational links to watch the > latency so you can tell when the link goes international. Just because you > are going from a US Network provider to China Telecom doesn't mean that > your not connecting to them in the united states. > > > > For example a traceroute from Denver to 27.29.128.1 which is an IP in > China Telecom's network. > > It's about 26ms between Denver and Los Angeles. Hop 5 to Hop 6 > > China Telecom connects to GTT in Los Angeles Hop7/8 > > On Hop 8 is in the United State and Hop 9 is across the pacific. Because > the latency goes from 31 ms to 183 ms. > > > Just something to keep in mind. > > > > Packets Pings > Host > Loss% Snt Last Avg Best Wrst StDev > 1. _gateway > 0.0% 14 1.0 1.2 1.0 2.8 0.5 > 2. te-0-0-26.ear2.den1.us.nitelusa.net > 0.0% 14 0.9 1.0 0.8 2.1 0.4 > 3. te-0-0-26.ear1.den1.us.nitelusa.net > 0.0% 14 1.1 1.6 1.1 2.9 0.7 > 4. te-0-0-1-0.cr1.den1.us.nitelusa.net > 0.0% 14 1.0 1.0 1.0 1.1 0.0 > 5. ae1-122.cr0-den2.ip4.gtt.net > 0.0% 14 0.5 1.2 0.3 6.9 2.0 > 6. et-0-0-47.cr3-lax2.ip4.gtt.net > 0.0% 14 26.5 26.4 26.2 26.7 0.2 > 7. as4134.lax20.ip4.gtt.net > 0.0% 14 27.7 28.7 26.8 30.1 1.1 > 8. 202.97.50.29 > 0.0% 14 31.4 30.6 26.8 34.1 2.4 > 9. 202.97.41.129 > 0.0% 14 183.3 187.1 183.3 190.8 2.4 > 10. 202.97.94.101 > 0.0% 14 187.9 188.6 186.1 211.2 6.8 > 11. 202.97.94.141 > 0.0% 13 177.8 180.7 177.2 184.2 2.3 > 12. 202.97.67.54 > 0.0% 13 199.5 201.2 197.4 205.1 2.6 > 13. 111.177.110.62 > 0.0% 13 205.9 206.3 205.9 208.2 0.7 > 14. 27.29.128.1 > 0.0% 13 202.6 202.8 202.5 203.9 0.4 > > > Erik Sundberg > > Sr. Network Engineer > > Nitel > > 350 N Orleans Street > > Suite 1300N > > Chicago, Il 60654 > > Desk: 773-661-5532 > > Cell: 708-710-7419 > > NOC: 866-892-0915 > > Email: esundb...@nitelusa.com > > web: www.nitelusa.com > > ------------------------------ > *From:* Zhiyun Qian <zhiy...@cs.ucr.edu> > *Sent:* Tuesday, April 16, 2019 1:02:36 PM > *To:* Erik Sundberg > *Cc:* Pengxiong Zhu; Zhiyun Qian; Zhongjie Wang; Keyu Man > *Subject:* Re: Ownership of Routers on Both Ends of Transnational Links > > Erik, > > Thanks a lot for the information! This is extremely helpful. We are > conducting an analysis on performance/policy-related study on transnational > links. We are hoping to submit a paper soon. Will be glad to share all the > details once we have a draft! > > Best, > -Zhiyun > > > On Tue, Apr 16, 2019 at 10:35 AM Erik Sundberg <esundb...@nitelusa.com> > wrote: > > CPE is usually ran by the customer. Some provider do offer managed routers > for a fee. Kinda like renting a cable modem from your provider. > > > What are your guys trying to accomplish or find out? > > Erik > > > > Erik Sundberg > Sr. Network Engineer > Nitel > 350 N Orleans Street > Suite 1300N > Chicago, Il 60654 > Desk: 773-661-5532 > Cell: 708-710-7419 > NOC: 866-892-0915 > Email: esundb...@nitelusa.com > web: www.nitelusa.com > > ------------------------------ > *From:* Pengxiong Zhu <pzhu...@ucr.edu> > *Sent:* Tuesday, April 16, 2019 12:32 PM > *To:* Erik Sundberg > *Cc:* Zhiyun Qian; Zhongjie Wang; Keyu Man > *Subject:* Re: Ownership of Routers on Both Ends of Transnational Links > > Thanks a lot! > > Are the Customer Devices managed by Telia or the customer? > > Best, > Pengxiong Zhu > Department of Computer Science and Engineering > University of California, Riverside > > > On Tue, Apr 16, 2019 at 7:43 AM Erik Sundberg <esundb...@nitelusa.com> > wrote: > > I hope this helps with the breakdown for telia. > > > > > Telia i think is using /31's for there serial blocks now > > 62.115.170.56 (Telia Edge Rotuer) > > 62.115.170.57 (Customer Device) > > > > chinaunicom-ic-341501-sjo-b21.c.telia.net. > > > <Customername>-<CircuitID>-<POP>-<router>.c.telia.net > > > > Customer: ChinaUnicom > > Telia Circuit ID's are: ic-123456 > > POP: SJO (Airport code) > > Router: b21 > > Doamin: c.telia.net "Customer.telia.net" > > > > > Erik Sundberg > > Sr. Network Engineer > > Nitel > > 350 N Orleans Street > > Suite 1300N > > Chicago, Il 60654 > > Desk: 773-661-5532 > > Cell: 708-710-7419 > > NOC: 866-892-0915 > > Email: esundb...@nitelusa.com > > web: www.nitelusa.com > > ------------------------------ > *From:* NANOG <nanog-boun...@nanog.org> on behalf of Pengxiong Zhu < > pzhu...@ucr.edu> > *Sent:* Monday, April 15, 2019 11:36:45 PM > *To:* nanog@nanog.org > *Cc:* Keyu Man; Zhiyun Qian; Zhongjie Wang > *Subject:* Ownership of Routers on Both Ends of Transnational Links > > Howdy folks, > > We are a group of researchers at UC Riverside conducting some measurement > about transnational networks. In particular, we are interested in studying > the ownership of routers on the two sides of transnational links. > > We have some concrete questions which we hope someone can shed some light > on. Basically when we send packets from US/Canada to China, through > traceroute and the RTT of each hop, we can locate the last hop in the US > before the packets enter China (*there is a large jump of RTT of 100+ms > from this hop onwards*). Oftentimes the ownership of such routers is > ambiguous. > > These hops whose IPs seem to belong to US or European ISPs (*according to > BGP info*) but their reverse DNS names have *chinaunicom* in it, which is > a Chinese ISP. > AS1299 Telia Company AB > 62.115.170.57 name = chinaunicom-ic-341501-sjo-b21.c.telia.net. > 62.115.33.230 name = chinaunicom-ic-302366-las-bb1.c.telia.net. > 213.248.73.190 name = chinaunicom-ic-127288-sjo-b21.c.telia.net. > > AS701 Verizon Business > 152.179.103.254 name = chinaunicom-gw.customer.alter.net. > > While the following routers, they don't have a reverse DNS name at all, > which seem to be uncommon if they were managed by US or European ISPs but > quite common for Chinese ISPs. > AS6453 TATA COMMUNICATIONS (AMERICA) INC > 63.243.205.90 > 66.110.59.118 > > Can anyone confirm that these are indeed managed by the Chinese ISPs (even > though they are physically located in the US according to the traceroute > and RTT analysis)? > > > Best, > Pengxiong Zhu > Department of Computer Science and Engineering > University of California, Riverside > > ------------------------------ > > CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files > or previous e-mail messages attached to it may contain confidential > information that is legally privileged. If you are not the intended > recipient, or a person responsible for delivering it to the intended > recipient, you are hereby notified that any disclosure, copying, > distribution or use of any of the information contained in or attached to > this transmission is STRICTLY PROHIBITED. If you have received this > transmission in error please notify the sender immediately by replying to > this e-mail. You must destroy the original transmission and its attachments > without reading or saving in any manner. Thank you. > > > ------------------------------ > > CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files > or previous e-mail messages attached to it may contain confidential > information that is legally privileged. If you are not the intended > recipient, or a person responsible for delivering it to the intended > recipient, you are hereby notified that any disclosure, copying, > distribution or use of any of the information contained in or attached to > this transmission is STRICTLY PROHIBITED. If you have received this > transmission in error please notify the sender immediately by replying to > this e-mail. You must destroy the original transmission and its attachments > without reading or saving in any manner. Thank you. > > > ------------------------------ > > CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files > or previous e-mail messages attached to it may contain confidential > information that is legally privileged. If you are not the intended > recipient, or a person responsible for delivering it to the intended > recipient, you are hereby notified that any disclosure, copying, > distribution or use of any of the information contained in or attached to > this transmission is STRICTLY PROHIBITED. If you have received this > transmission in error please notify the sender immediately by replying to > this e-mail. You must destroy the original transmission and its attachments > without reading or saving in any manner. Thank you. > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190511/4d6a3b8d/attachment-0001.html> ------------------------------ Message: 3 Date: Mon, 13 May 2019 09:38:53 -0600 From: Brielle Bruns <br...@2mbit.com> To: nanog@nanog.org Subject: Re: Issues with SIP packets between VZ Fios and NTT Message-ID: <98cf5423-4167-2242-62f7-782a9c500...@2mbit.com> Content-Type: text/plain; charset=windows-1252; format=flowed On 5/13/2019 9:21 AM, Dovid Bender wrote: > Hi, > > Over the last 48 hours we have been getting a lot of alerts of customers > phones losing registrations to us. All the complaints are coming from > customers that are on VZ Fios in the NYC area. Anyone else see anything > strange going on? > While you are diagnosing, might check to make sure that the SIP ALG is disabled on all of their routers too. -- Brielle Bruns The Summit Open Source Development Group http://www.sosdg.org / http://www.ahbl.org ------------------------------ Message: 4 Date: Mon, 13 May 2019 11:51:58 -0400 From: Christopher Morrow <morrowc.li...@gmail.com> To: Pengxiong Zhu <pzhu...@ucr.edu> Cc: "North American Network Operators' Group" <nanog@nanog.org>, Keyu Man <kman...@ucr.edu>, Zhiyun Qian <zhiy...@cs.ucr.edu>, Zhongjie Wang <zwang...@ucr.edu> Subject: Re: Ownership of Routers on Both Ends of Transnational Links Message-ID: <CAL9jLabwtcfU=ndpryo2wfxmzzmrxe0jzk2tytfwjxxdh+1...@mail.gmail.com> Content-Type: text/plain; charset="UTF-8" On Mon, May 13, 2019 at 11:38 AM Pengxiong Zhu <pzhu...@ucr.edu> wrote: > > Thanks again for your insightful responses! > > The case we discuss above is Chinese ISPs renting routers located outside > China and the IPs belong to other ISPs. > I think you are using all of the wrong verbs here... 'renting' does not make sense here, I'm unclear on what you actually mean, please try again with a different verb OR more clarifying text. \ ------------------------------ Message: 5 Date: Mon, 13 May 2019 12:20:10 -0400 From: Dovid Bender <do...@telecurve.com> To: Brielle Bruns <br...@2mbit.com> Cc: NANOG <nanog@nanog.org> Subject: Re: Issues with SIP packets between VZ Fios and NTT Message-ID: <cam3tth2wcwqcwbjjnphxqbwasowxgarev61ovota3az3d5-...@mail.gmail.com> Content-Type: text/plain; charset="utf-8" Thought of that. Customers have their own CPE's. So far the only thing mutual here is that it's NTT -> VZ. Here is what I found so far looking at two Polycom phones using non standard ports (e.g. not 5060) 1) PhoneA tries to register multiple extensions and for each request we send a 401. We expect to get back a REGISTER request with a no-once but we don't. This happens for a while and then magically it starts working. 2) PhoneB tries to register the time time as PhoneA and has no issues. At first I thought it was something possibly with the SIP call-ID but I ruled that out since in the same SIP DIALOG it was not working then it started. Also the seems to be per phone each phone is behind NAT and the traffic is coming from a different NAT'd port. Seems like there is some device in the middle that is randomly dropping traffic on specific sessions. On Mon, May 13, 2019 at 11:40 AM Brielle Bruns <br...@2mbit.com> wrote: > On 5/13/2019 9:21 AM, Dovid Bender wrote: > > Hi, > > > > Over the last 48 hours we have been getting a lot of alerts of customers > > phones losing registrations to us. All the complaints are coming from > > customers that are on VZ Fios in the NYC area. Anyone else see anything > > strange going on? > > > > > While you are diagnosing, might check to make sure that the SIP ALG is > disabled on all of their routers too. > > > > -- > Brielle Bruns > The Summit Open Source Development Group > http://www.sosdg.org / http://www.ahbl.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190513/ea4a26ee/attachment-0001.html> ------------------------------ Message: 6 Date: Mon, 13 May 2019 09:31:23 -0700 From: Pengxiong Zhu <pzhu...@ucr.edu> To: Christopher Morrow <morrowc.li...@gmail.com> Cc: "North American Network Operators' Group" <nanog@nanog.org>, Keyu Man <kman...@ucr.edu>, Zhiyun Qian <zhiy...@cs.ucr.edu>, Zhongjie Wang <zwang...@ucr.edu> Subject: Re: Ownership of Routers on Both Ends of Transnational Links Message-ID: <CAEnbXKswsW41UmSW4djXhRGdTWX=fyspnq8i9npavhdy68b...@mail.gmail.com> Content-Type: text/plain; charset="utf-8" Sorry for the confusion. I mean the IPs belong to non-Chinese ISPs but are actually controlled/managed by Chinese ISPs. Best, Pengxiong Zhu Department of Computer Science and Engineering University of California, Riverside On Mon, May 13, 2019 at 8:52 AM Christopher Morrow <morrowc.li...@gmail.com> wrote: > On Mon, May 13, 2019 at 11:38 AM Pengxiong Zhu <pzhu...@ucr.edu> wrote: > > > > Thanks again for your insightful responses! > > > > The case we discuss above is Chinese ISPs renting routers located > outside China and the IPs belong to other ISPs. > > > > I think you are using all of the wrong verbs here... 'renting' does > not make sense here, I'm unclear on what you actually mean, please try > again with a different verb OR more clarifying text. > \ > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190513/d31cb249/attachment-0001.html> ------------------------------ Message: 7 Date: Mon, 13 May 2019 10:48:17 -0600 From: Brielle Bruns <br...@2mbit.com> To: NANOG <nanog@nanog.org> Subject: Re: Issues with SIP packets between VZ Fios and NTT Message-ID: <361d0ac7-c356-dbbf-b40b-d75536da8...@2mbit.com> Content-Type: text/plain; charset=windows-1252; format=flowed On 5/13/2019 10:20 AM, Dovid Bender wrote: > Thought of that. Customers have their own CPE's. So far the only thing > mutual here is that it's NTT -> VZ. Here is what I found so far looking > at two Polycom phones using non standard ports (e.g. not 5060) > 1) PhoneA tries to register multiple extensions and for each request we > send a 401. We expect to get back a REGISTER request with a no-once but > we don't. This happens for a while and then magically it starts working. > 2) PhoneB tries to register the time time as PhoneA and has no issues. > > At first I thought it was something possibly with the SIP call-ID but I > ruled that out since in the same SIP DIALOG it was not working then it > started. Also the seems to be per phone each phone is behind NAT and the > traffic is coming from a different NAT'd port. Seems like there is some > device in the middle that is randomly dropping traffic on specific sessions. Are you using TLS encrypted SIP or just plain ol' cleartext? If its encrypted, I'd look at possibly there being a MTU/MSS issue somewhere along the path possibly? -- Brielle Bruns The Summit Open Source Development Group http://www.sosdg.org / http://www.ahbl.org ------------------------------ Message: 8 Date: Mon, 13 May 2019 13:04:45 -0400 From: Dovid Bender <do...@telecurve.com> To: Brielle Bruns <br...@2mbit.com> Cc: NANOG <nanog@nanog.org> Subject: Re: Issues with SIP packets between VZ Fios and NTT Message-ID: <CAM3TTh3QT9ezBe=u4GeeMsK11Wb+Gcbfg15jg4pBB=gmyxp...@mail.gmail.com> Content-Type: text/plain; charset="utf-8" Good ol UDP encrypted. On Mon, May 13, 2019 at 12:49 PM Brielle Bruns <br...@2mbit.com> wrote: > > On 5/13/2019 10:20 AM, Dovid Bender wrote: > > Thought of that. Customers have their own CPE's. So far the only thing > > mutual here is that it's NTT -> VZ. Here is what I found so far looking > > at two Polycom phones using non standard ports (e.g. not 5060) > > 1) PhoneA tries to register multiple extensions and for each request we > > send a 401. We expect to get back a REGISTER request with a no-once but > > we don't. This happens for a while and then magically it starts working. > > 2) PhoneB tries to register the time time as PhoneA and has no issues. > > > > At first I thought it was something possibly with the SIP call-ID but I > > ruled that out since in the same SIP DIALOG it was not working then it > > started. Also the seems to be per phone each phone is behind NAT and the > > traffic is coming from a different NAT'd port. Seems like there is some > > device in the middle that is randomly dropping traffic on specific > sessions. > > > Are you using TLS encrypted SIP or just plain ol' cleartext? > > If its encrypted, I'd look at possibly there being a MTU/MSS issue > somewhere along the path possibly? > > > -- > Brielle Bruns > The Summit Open Source Development Group > http://www.sosdg.org / http://www.ahbl.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190513/6d84c2d2/attachment-0001.html> ------------------------------ Message: 9 Date: Mon, 13 May 2019 13:40:11 -0400 From: Christopher Morrow <morrowc.li...@gmail.com> To: Pengxiong Zhu <pzhu...@ucr.edu> Cc: "North American Network Operators' Group" <nanog@nanog.org>, Keyu Man <kman...@ucr.edu>, Zhiyun Qian <zhiy...@cs.ucr.edu>, Zhongjie Wang <zwang...@ucr.edu> Subject: Re: Ownership of Routers on Both Ends of Transnational Links Message-ID: <CAL9jLaYfTGuK165y9tGuJPJfSBciZxEAw=Vp4QXnzc0=g9w...@mail.gmail.com> Content-Type: text/plain; charset="UTF-8" On Mon, May 13, 2019 at 12:31 PM Pengxiong Zhu <pzhu...@ucr.edu> wrote: > > Sorry for the confusion. I mean the IPs belong to non-Chinese ISPs but are > actually controlled/managed by Chinese ISPs. > this is, as I think was said earlier, normal practice. Sometimes you accept a /31 from your "provider" or "peer", sometimes they accept yours... sometimes this is because of seasons/reasons/etc, sometimes because it's how folk denote who's paying for the link in between. Those ips are not useful as a signal, which I think was also said previously in this thread. > Best, > Pengxiong Zhu > Department of Computer Science and Engineering > University of California, Riverside > > > On Mon, May 13, 2019 at 8:52 AM Christopher Morrow <morrowc.li...@gmail.com> > wrote: >> >> On Mon, May 13, 2019 at 11:38 AM Pengxiong Zhu <pzhu...@ucr.edu> wrote: >> > >> > Thanks again for your insightful responses! >> > >> > The case we discuss above is Chinese ISPs renting routers located outside >> > China and the IPs belong to other ISPs. >> > >> >> I think you are using all of the wrong verbs here... 'renting' does >> not make sense here, I'm unclear on what you actually mean, please try >> again with a different verb OR more clarifying text. >> \ ------------------------------ Message: 10 Date: Mon, 13 May 2019 14:32:37 -0400 From: Dovid Bender <do...@telecurve.com> To: Brielle Bruns <br...@2mbit.com> Cc: NANOG <nanog@nanog.org> Subject: Re: Issues with SIP packets between VZ Fios and NTT Message-ID: <cam3tth3jip8cevub+n8z7z4sy6myxfnrdr4p9cwphyxwjqa...@mail.gmail.com> Content-Type: text/plain; charset="utf-8" FYI: More than one person reached out to me off list. The issue is clearly with VZ. Traces by the others were done and NTT was not in the mix. The only common denominator was 401 SIP packets hitting VZ Fios IP's in the NY area. On Mon, May 13, 2019 at 1:04 PM Dovid Bender <do...@telecurve.com> wrote: > Good ol UDP encrypted. > > > > On Mon, May 13, 2019 at 12:49 PM Brielle Bruns <br...@2mbit.com> wrote: > >> >> On 5/13/2019 10:20 AM, Dovid Bender wrote: >> > Thought of that. Customers have their own CPE's. So far the only thing >> > mutual here is that it's NTT -> VZ. Here is what I found so far looking >> > at two Polycom phones using non standard ports (e.g. not 5060) >> > 1) PhoneA tries to register multiple extensions and for each request we >> > send a 401. We expect to get back a REGISTER request with a no-once but >> > we don't. This happens for a while and then magically it starts working. >> > 2) PhoneB tries to register the time time as PhoneA and has no issues. >> > >> > At first I thought it was something possibly with the SIP call-ID but I >> > ruled that out since in the same SIP DIALOG it was not working then it >> > started. Also the seems to be per phone each phone is behind NAT and >> the >> > traffic is coming from a different NAT'd port. Seems like there is some >> > device in the middle that is randomly dropping traffic on specific >> sessions. >> >> >> Are you using TLS encrypted SIP or just plain ol' cleartext? >> >> If its encrypted, I'd look at possibly there being a MTU/MSS issue >> somewhere along the path possibly? >> >> >> -- >> Brielle Bruns >> The Summit Open Source Development Group >> http://www.sosdg.org / http://www.ahbl.org >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190513/585bb7a9/attachment-0001.html> ------------------------------ Message: 11 Date: Mon, 13 May 2019 14:43:45 -0400 From: Jared Mauch <ja...@puck.nether.net> To: Dovid Bender <do...@telecurve.com> Cc: Brielle Bruns <br...@2mbit.com>, NANOG <nanog@nanog.org> Subject: Re: Issues with SIP packets between VZ Fios and NTT Message-ID: <3b38b91f-ef60-4838-ba72-df8ea3691...@puck.nether.net> Content-Type: text/plain; charset="utf-8" This matches my experience with running SIP on networks. Slowly over the years it became more unreliable as “helper” ALGs were in the path. Eventually we moved some devices off 5060 to alleviate the problem. Sent from my iCar > On May 13, 2019, at 2:32 PM, Dovid Bender <do...@telecurve.com> wrote: > > FYI: More than one person reached out to me off list. The issue is clearly > with VZ. Traces by the others were done and NTT was not in the mix. The only > common denominator was 401 SIP packets hitting VZ Fios IP's in the NY area. > > > >> On Mon, May 13, 2019 at 1:04 PM Dovid Bender <do...@telecurve.com> wrote: >> Good ol UDP encrypted. >> >> >> >>> On Mon, May 13, 2019 at 12:49 PM Brielle Bruns <br...@2mbit.com> wrote: >>> >>> On 5/13/2019 10:20 AM, Dovid Bender wrote: >>> > Thought of that. Customers have their own CPE's. So far the only thing >>> > mutual here is that it's NTT -> VZ. Here is what I found so far looking >>> > at two Polycom phones using non standard ports (e.g. not 5060) >>> > 1) PhoneA tries to register multiple extensions and for each request we >>> > send a 401. We expect to get back a REGISTER request with a no-once but >>> > we don't. This happens for a while and then magically it starts working. >>> > 2) PhoneB tries to register the time time as PhoneA and has no issues. >>> > >>> > At first I thought it was something possibly with the SIP call-ID but I >>> > ruled that out since in the same SIP DIALOG it was not working then it >>> > started. Also the seems to be per phone each phone is behind NAT and the >>> > traffic is coming from a different NAT'd port. Seems like there is some >>> > device in the middle that is randomly dropping traffic on specific >>> > sessions. >>> >>> >>> Are you using TLS encrypted SIP or just plain ol' cleartext? >>> >>> If its encrypted, I'd look at possibly there being a MTU/MSS issue >>> somewhere along the path possibly? >>> >>> >>> -- >>> Brielle Bruns >>> The Summit Open Source Development Group >>> http://www.sosdg.org / http://www.ahbl.org -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190513/b0a29b6a/attachment-0001.html> ------------------------------ Message: 12 Date: Mon, 13 May 2019 14:44:40 -0400 From: Dovid Bender <do...@telecurve.com> To: Jared Mauch <ja...@puck.nether.net> Cc: Brielle Bruns <br...@2mbit.com>, NANOG <nanog@nanog.org> Subject: Re: Issues with SIP packets between VZ Fios and NTT Message-ID: <CAM3TTh3rn1Ms+cs51Bc8=_oSyx8SNBu=T=otxwcnav1we3b...@mail.gmail.com> Content-Type: text/plain; charset="utf-8" In our case we are not using 5060. The issue seems exclusive to VZ. On Mon, May 13, 2019 at 2:43 PM Jared Mauch <ja...@puck.nether.net> wrote: > This matches my experience with running SIP on networks. Slowly over the > years it became more unreliable as “helper” ALGs were in the path. > > Eventually we moved some devices off 5060 to alleviate the problem. > > Sent from my iCar > > On May 13, 2019, at 2:32 PM, Dovid Bender <do...@telecurve.com> wrote: > > FYI: More than one person reached out to me off list. The issue is clearly > with VZ. Traces by the others were done and NTT was not in the mix. The > only common denominator was 401 SIP packets hitting VZ Fios IP's in the NY > area. > > > > On Mon, May 13, 2019 at 1:04 PM Dovid Bender <do...@telecurve.com> wrote: > >> Good ol UDP encrypted. >> >> >> >> On Mon, May 13, 2019 at 12:49 PM Brielle Bruns <br...@2mbit.com> wrote: >> >>> >>> On 5/13/2019 10:20 AM, Dovid Bender wrote: >>> > Thought of that. Customers have their own CPE's. So far the only thing >>> > mutual here is that it's NTT -> VZ. Here is what I found so far >>> looking >>> > at two Polycom phones using non standard ports (e.g. not 5060) >>> > 1) PhoneA tries to register multiple extensions and for each request >>> we >>> > send a 401. We expect to get back a REGISTER request with a no-once >>> but >>> > we don't. This happens for a while and then magically it starts >>> working. >>> > 2) PhoneB tries to register the time time as PhoneA and has no issues. >>> > >>> > At first I thought it was something possibly with the SIP call-ID but >>> I >>> > ruled that out since in the same SIP DIALOG it was not working then it >>> > started. Also the seems to be per phone each phone is behind NAT and >>> the >>> > traffic is coming from a different NAT'd port. Seems like there is >>> some >>> > device in the middle that is randomly dropping traffic on specific >>> sessions. >>> >>> >>> Are you using TLS encrypted SIP or just plain ol' cleartext? >>> >>> If its encrypted, I'd look at possibly there being a MTU/MSS issue >>> somewhere along the path possibly? >>> >>> >>> -- >>> Brielle Bruns >>> The Summit Open Source Development Group >>> http://www.sosdg.org / http://www.ahbl.org >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190513/5ae2701b/attachment-0001.html> ------------------------------ Message: 13 Date: Mon, 13 May 2019 13:52:11 -0400 From: Pete Rohrman <prohr...@stage2networks.com> To: <nanog@nanog.org> Subject: Re: Issues with SIP packets between VZ Fios and NTT Message-ID: <688fc03b-8ab8-3380-7652-dadb15202...@stage2networks.com> Content-Type: text/plain; charset="utf-8"; Format="flowed" Dovid Bender, I'm seeing the same sort of thing. Polycom phones. Multiple customers getting to me from Verizon in NYC area. I'm seeing phones register for a while, then drop off, then I see them trying to re-reg resulting in your 401 below. Call me. 212 497 8015. Let's look at this. Pete Pete Rohrman Stage2 Support 212 497 8000, Opt. 2 On 5/13/19 12:20 PM, Dovid Bender wrote: > Thought of that. Customers have their own CPE's. So far the only thing > mutual here is that it's NTT -> VZ. Here is what I found so far > looking at two Polycom phones using non standard ports (e.g. not 5060) > 1) PhoneA tries to register multiple extensions and for each request > we send a 401. We expect to get back a REGISTER request with a no-once > but we don't. This happens for a while and then magically it starts > working. > 2) PhoneB tries to register the time time as PhoneA and has no issues. > > At first I thought it was something possibly with the SIP call-ID but > I ruled that out since in the same SIP DIALOG it was not working then > it started. Also the seems to be per phone each phone is behind NAT > and the traffic is coming from a different NAT'd port. Seems like > there is some device in the middle that is randomly dropping traffic > on specific sessions. > > > > > > On Mon, May 13, 2019 at 11:40 AM Brielle Bruns <br...@2mbit.com > <mailto:br...@2mbit.com>> wrote: > > On 5/13/2019 9:21 AM, Dovid Bender wrote: > > Hi, > > > > Over the last 48 hours we have been getting a lot of alerts of > customers > > phones losing registrations to us. All the complaints are coming > from > > customers that are on VZ Fios in the NYC area. Anyone else see > anything > > strange going on? > > > > > While you are diagnosing, might check to make sure that the SIP > ALG is > disabled on all of their routers too. > > > > -- > Brielle Bruns > The Summit Open Source Development Group > http://www.sosdg.org / http://www.ahbl.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190513/adc79d9d/attachment-0001.html> ------------------------------ Message: 14 Date: Mon, 13 May 2019 15:11:03 -0400 From: <dan...@pyranah.com> To: <nanog@nanog.org> Subject: Charter and Cox contacts Message-ID: <052201d509bf$9c000c30$d4002490$@pyranah.com> Content-Type: text/plain; charset="utf-8" Does anyone have contacts at Charter (Spectrum) and Cox? For some reason, our IP has been blocked by them and our customers are unable to send email via their charter/cox accounts. Thanks Daniel Temple VP of Operations Pyranah Communications, LLC 828-743-2470 ext.205 <http://www.pyranah.com/> Pyranah.com <https://www.facebook.com/cashiersvalleyelectronics/> Like us on Facebook! -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190513/c0972bf8/attachment-0001.html> ------------------------------ Message: 15 Date: Mon, 13 May 2019 17:39:39 -0400 From: b...@theworld.com To: nanog@nanog.org Cc: ab...@outlook.com Subject: Mostly name and shame... Message-ID: <23769.58395.683601.559...@gargle.gargle.howl> Content-Type: text/plain; charset=us-ascii Why has OUTLOOK.COM allowed daily dictionary spammers to operate from their net, FOR YEARS? It can't be that hard to detect and block. 2019-05-13T17:00:18.194103-04:00 pcls6 sendmail[14128]: NOUSER: proctor5 relay=mail-eopbgr740053.outbound.protection.outlook.com [40.107.74.53] 2019-05-13T17:00:18.444848-04:00 pcls6 sendmail[14128]: NOUSER: proctor4 relay=mail-eopbgr740053.outbound.protection.outlook.com [40.107.74.53] 2019-05-13T17:00:18.698977-04:00 pcls6 sendmail[14128]: NOUSER: proctor2 relay=mail-eopbgr740053.outbound.protection.outlook.com [40.107.74.53] 2019-05-13T17:00:18.952548-04:00 pcls6 sendmail[14128]: NOUSER: proctor10 relay=mail-eopbgr740053.outbound.protection.outlook.com [40.107.74.53] 2019-05-13T17:10:27.523597-04:00 pcls6 sendmail[19471]: NOUSER: proctor6 relay=mail-eopbgr810043.outbound.protection.outlook.com [40.107.81.43] 2019-05-13T17:10:27.775984-04:00 pcls6 sendmail[19471]: NOUSER: proctor5 relay=mail-eopbgr810043.outbound.protection.outlook.com [40.107.81.43] 2019-05-13T17:10:28.029744-04:00 pcls6 sendmail[19471]: NOUSER: proctor4 relay=mail-eopbgr810043.outbound.protection.outlook.com [40.107.81.43] 2019-05-13T17:10:28.283016-04:00 pcls6 sendmail[19471]: NOUSER: proctor2 relay=mail-eopbgr810043.outbound.protection.outlook.com [40.107.81.43] 2019-05-13T17:10:28.537106-04:00 pcls6 sendmail[19471]: NOUSER: proctor10 relay=mail-eopbgr810043.outbound.protection.outlook.com [40.107.81.43] 2019-05-13T17:30:47.045677-04:00 pcls6 sendmail[31621]: NOUSER: proctor6 relay=mail-eopbgr810072.outbound.protection.outlook.com [40.107.81.72] 2019-05-13T17:30:47.299131-04:00 pcls6 sendmail[31621]: NOUSER: proctor5 relay=mail-eopbgr810072.outbound.protection.outlook.com [40.107.81.72] 2019-05-13T17:30:47.552492-04:00 pcls6 sendmail[31621]: NOUSER: proctor4 relay=mail-eopbgr810072.outbound.protection.outlook.com [40.107.81.72] 2019-05-13T17:30:47.804233-04:00 pcls6 sendmail[31621]: NOUSER: proctor2 relay=mail-eopbgr810072.outbound.protection.outlook.com [40.107.81.72] 2019-05-13T17:30:48.056635-04:00 pcls6 sendmail[31621]: NOUSER: proctor10 relay=mail-eopbgr810072.outbound.protection.outlook.com [40.107.81.72] 2019-05-13T17:35:05.867715-04:00 pcls6 sendmail[1352]: NOUSER: proctor9 relay=mail-eopbgr50127.outbound.protection.outlook.com [40.107.5.127] 2019-05-13T17:35:06.120021-04:00 pcls6 sendmail[1352]: NOUSER: proctor7 relay=mail-eopbgr50127.outbound.protection.outlook.com [40.107.5.127] 2019-05-13T17:35:06.372603-04:00 pcls6 sendmail[1352]: NOUSER: proctor6 relay=mail-eopbgr50127.outbound.protection.outlook.com [40.107.5.127] 2019-05-13T17:35:06.627583-04:00 pcls6 sendmail[1352]: NOUSER: proctor5 relay=mail-eopbgr50127.outbound.protection.outlook.com [40.107.5.127] 2019-05-13T17:35:06.885218-04:00 pcls6 sendmail[1352]: NOUSER: proctor relay=mail-eopbgr50127.outbound.protection.outlook.com [40.107.5.127] etc etc etc etc. -- -Barry Shein Software Tool & Die | b...@theworld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo* ------------------------------ Message: 16 Date: Mon, 13 May 2019 14:53:01 -0700 From: Randy Bush <ra...@psg.com> To: Christopher Morrow <morrowc.li...@gmail.com> Cc: Pengxiong Zhu <pzhu...@ucr.edu>, Keyu Man <kman...@ucr.edu>, North American Network Operators' Group <nanog@nanog.org>, Zhiyun Qian <zhiy...@cs.ucr.edu>, Zhongjie Wang <zwang...@ucr.edu> Subject: Re: Ownership of Routers on Both Ends of Transnational Links Message-ID: <m236li58ea.wl-ra...@psg.com> Content-Type: text/plain; charset=US-ASCII i suspect the OP is down the rabbit hole of what is known as "anti-aliasing," trying to find out whether IP address A on some router is actually on the same router as IP address B, and what AS(s) those IPs are in. your point is that an inter-as link may have IPs from either of the providers. yup. and, because it is an INTER-as link, it does not really belong to one or t'other. this particular rabbit digs deep holes. an early entrance to the burrow is the classic from the uw crew inproceedings{Spring:2002:MIT:633025.633039, author = {Spring, Neil and Mahajan, Ratul and Wetherall, David}, title = {Measuring ISP Topologies with Rocketfuel}, booktitle = {Proceedings of the 2002 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications}, series = {SIGCOMM '02}, year = {2002}, isbn = {1-58113-570-X}, location = {Pittsburgh, Pennsylvania, USA}, pages = {133--145}, numpages = {13}, url = {http://doi.acm.org/10.1145/633025.633039}, doi = {10.1145/633025.633039}, acmid = {633039}, publisher = {ACM}, address = {New York, NY, USA}, } randy ------------------------------ Message: 17 Date: Mon, 13 May 2019 15:29:43 -0700 From: Stephen Satchell <l...@satchell.net> To: nanog@nanog.org Subject: Re: Charter and Cox contacts Message-ID: <1d87ab31-ab93-d69b-be66-c29bf97de...@satchell.net> Content-Type: text/plain; charset=windows-1252 On 5/13/19 12:11 PM, dan...@pyranah.com wrote: > Does anyone have contacts at Charter (Spectrum) and Cox? For some reason, > our IP has been blocked by them and our customers are unable to send email > via their charter/cox accounts. Thanks Would you be talking about port 25/tcp outbound? Lots of ISPs will block port 25 as a rule; you have to call to have it opened. At the same time, you can request the PTR record you need to keep big mail receivers happy. ------------------------------ Message: 18 Date: Mon, 13 May 2019 23:48:02 -0500 From: <frnk...@iname.com> To: "'Mel Beckman'" <m...@beckman.org>, "Mike Bolitho" <mikeboli...@gmail.com> Cc: <nanog@nanog.org> Subject: RE: FCC Hurricane Michael after-action report Message-ID: <003e01d50a10$36d52330$a47f6990$@iname.com> Content-Type: text/plain; charset="utf-8" This webinar may be of some interest to those in this group: https://www.fcc.gov/small-rural-communications-provider-network-resiliency-webinar Here’s some additional color commentary on the FCC’s concerns: https://urgentcomm.com/2019/05/10/backhaul-problems-disjointed-recovery-efforts-key-causes-of-unacceptable-extended-wireless-outage-after-hurricane-michael-fcc-report-says/ "“Uniti Fiber (Uniti) provides backhaul services to Verizon Wireless in Bay and Gulf Counties. Uniti indicates it experienced at least 33 separate fiber cuts during the recovery effort. These fiber cuts included damage to sections that already had been repaired. Commenters attributed fiber cuts to debris-removal crews, power-company restorations, and returning homeowners clearing their property.” One of my takeaways from that article was that burying fiber underground could likely have avoided many/most of these fiber cuts, though I’m not familiar enough with the terrain to know how feasible that is. Frank From: NANOG <nanog-boun...@nanog.org> On Behalf Of Mel Beckman Sent: Saturday, May 11, 2019 9:52 AM To: Mike Bolitho <mikeboli...@gmail.com> Cc: nanog@nanog.org Subject: Re: FCC Hurricane Michael after-action report This is what I tell outage complainers during natural disasters, such as the fires in California that recently took out a lot of power and communications: “Stop whining about how long it is taking to repair your Internet, your cell phone service, or your cable TV. You didn’t pay anything extra to recover from natural disasters, and none of us in the field are getting paid anything extra to restore your services. No, we don’t know how long it will take. It takes what it takes. That you don’t get instant gratification doesn’t make us incompetent. It makes you ungrateful. It’s a natural disaster. These are not scheduled. Your outage is nobody’s fault. We don’t have a duty to mitigate all conceivable failures. It takes time to repair. We’re not cheating you, or loafing around. We don’t owe you any special attention because of your status or reputation. So quit whining and be thankful you’re alive, and hopefully you haven’t lost too much. Maybe pitch in and help those who have.“ I also send this to ignorant journalists and grandstanding politicians. -mel via cell On May 11, 2019, at 4:29 AM, Mike Bolitho <mikeboli...@gmail.com <mailto:mikeboli...@gmail.com> > wrote: Trying not to get political, here goes... Something important to keep in mind: The current administration has been getting slammed for their lack of response in the aftermath of Michael since the hurricane hit. A lot of that criticism revolves around communications infrastructure and FEMA's lack of assistance. The current administration has, time and time again, used federal agencies (specifically their presidential appointees) to defend the administration's actions or inactions. I have read the full report and it is more or less a thinly veiled hit piece. I'm not going to link them here (they are easy enough to find via Google) but there are several very good articles written by reputable tech journalists that go into greater detail responding to the report. Worth checking out. I say all of that because most of us like to hate on telecom companies (many times rightly so) but I don't think they are entirely to blame here. There's nothing Verizon or AT&T can do if their backhaul is cut by a tree or some third party clean up crew. The report is a gross oversimplification of how telecommunication infrastructure works. I think anyone here that has ever worked a storm like this can attest to the complexity and difficulty you run into during recovery. Hanlon's Razor and all but this is the FCC and I would hope they would know better. Speaking specifically to point 51, it's impossible to coordinate between the thousands of crews working to clean things up and repair physical infrastructure after a massive storm like this. Many of the people doing physical cleanup are volunteers that are fully independent of any governing body or company. It is not a telco's responsibility to know when and where those crews are working. Further, even if those crews we're calling in and letting each telco know exactly where they were, what does that provide other than an impossibly large and fluid dataset to parse for any meaningful information. - Mike Bolitho On Thu, May 9, 2019, 4:43 PM Sean Donelan wrote: The FCC has released its report and analysis of Hurricane Michael impact on communications: preparation, effect and recovery. https://www.fcc.gov/document/fcc-releases-report-communication-impacts-hurricane-michael-0 Conclusions and Recommendations 51. Backhaul outages loomed large as an impediment to communications recovery. Uncoordinated post-storm recovery efforts between and among communications, utility, and debris removal teams created unnecessary delays to a speedy return to service. Customers who had communications service restored – only to lose it again almost immediately because of a fiber cut – provide a clear example of how better cross-sector coordination could have improved the restoration process. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190513/f13f13d1/attachment-0001.html> ------------------------------ Message: 19 Date: Mon, 13 May 2019 22:20:22 -0700 From: Mike Bolitho <mikeboli...@gmail.com> Cc: nanog@nanog.org Subject: Re: FCC Hurricane Michael after-action report Message-ID: <CACoNLryTaFJYDMUSL0duWxSKr913r6X=xrbn6d2c_k-nozy...@mail.gmail.com> Content-Type: text/plain; charset="utf-8" In Florida, especially the panhandle, it's not possible to bury it. The water table is way too high. On Mon, May 13, 2019, 9:47 PM <frnk...@iname.com> wrote: > This webinar may be of some interest to those in this group: > > > https://www.fcc.gov/small-rural-communications-provider-network-resiliency-webinar > > > > Here’s some additional color commentary on the FCC’s concerns: > > > https://urgentcomm.com/2019/05/10/backhaul-problems-disjointed-recovery-efforts-key-causes-of-unacceptable-extended-wireless-outage-after-hurricane-michael-fcc-report-says/ > > "“Uniti Fiber (Uniti) provides backhaul services to Verizon Wireless in > Bay and Gulf Counties. Uniti indicates it experienced at least 33 separate > fiber cuts during the recovery effort. These fiber cuts included damage to > sections that already had been repaired. Commenters attributed fiber cuts > to debris-removal crews, power-company restorations, and returning > homeowners clearing their property.” > > One of my takeaways from that article was that burying fiber underground > could likely have avoided many/most of these fiber cuts, though I’m not > familiar enough with the terrain to know how feasible that is. > > Frank > > > > *From:* NANOG <nanog-boun...@nanog.org> *On Behalf Of *Mel Beckman > *Sent:* Saturday, May 11, 2019 9:52 AM > *To:* Mike Bolitho <mikeboli...@gmail.com> > *Cc:* nanog@nanog.org > *Subject:* Re: FCC Hurricane Michael after-action report > > > > This is what I tell outage complainers during natural disasters, such as > the fires in California that recently took out a lot of power and > communications: > > > > “Stop whining about how long it is taking to repair your Internet, your > cell phone service, or your cable TV. You didn’t pay anything extra to > recover from natural disasters, and none of us in the field are getting > paid anything extra to restore your services. > > > > No, we don’t know how long it will take. It takes what it takes. That you > don’t get instant gratification doesn’t make us incompetent. It makes you > ungrateful. > > > > It’s a natural disaster. These are not scheduled. Your outage is nobody’s > fault. We don’t have a duty to mitigate all conceivable failures. > > > > It takes time to repair. We’re not cheating you, or loafing around. We > don’t owe you any special attention because of your status or reputation. > > > > So quit whining and be thankful you’re alive, and hopefully you haven’t > lost too much. Maybe pitch in and help those who have.“ > > > > I also send this to ignorant journalists and grandstanding politicians. > > -mel via cell > > > On May 11, 2019, at 4:29 AM, Mike Bolitho <mikeboli...@gmail.com> wrote: > > Trying not to get political, here goes... > > > > Something important to keep in mind: The current administration has been > getting slammed for their lack of response in the aftermath of Michael > since the hurricane hit. A lot of that criticism revolves around > communications infrastructure and FEMA's lack of assistance. The current > administration has, time and time again, used federal agencies > (specifically their presidential appointees) to defend the administration's > actions or inactions. I have read the full report and it is more or less a > thinly veiled hit piece. I'm not going to link them here (they are easy > enough to find via Google) but there are several very good articles written > by reputable tech journalists that go into greater detail responding to the > report. Worth checking out. > > > > I say all of that because most of us like to hate on telecom companies > (many times rightly so) but I don't think they are entirely to blame here. > There's nothing Verizon or AT&T can do if their backhaul is cut by a tree > or some third party clean up crew. The report is a gross oversimplification > of how telecommunication infrastructure works. I think anyone here that has > ever worked a storm like this can attest to the complexity and difficulty > you run into during recovery. Hanlon's Razor and all but this is the FCC > and I would hope they would know better. > > > > Speaking specifically to point 51, it's impossible to coordinate between > the thousands of crews working to clean things up and repair physical > infrastructure after a massive storm like this. Many of the people doing > physical cleanup are volunteers that are fully independent of any governing > body or company. It is not a telco's responsibility to know when and where > those crews are working. Further, even if those crews we're calling in and > letting each telco know exactly where they were, what does that provide > other than an impossibly large and fluid dataset to parse for any > meaningful information. > > > > - Mike Bolitho > > > > On Thu, May 9, 2019, 4:43 PM Sean Donelan wrote: > > > The FCC has released its report and analysis of Hurricane Michael impact > on communications: preparation, effect and recovery. > > > > https://www.fcc.gov/document/fcc-releases-report-communication-impacts-hurricane-michael-0 > > Conclusions and Recommendations > > 51. Backhaul outages loomed large as an impediment to communications > recovery. Uncoordinated post-storm recovery efforts between and among > communications, utility, and debris removal teams created unnecessary > delays to a speedy return to service. Customers who had communications > service restored – only to lose it again almost immediately because of a > fiber cut – provide a clear example of how better cross-sector > coordination could have improved the restoration process. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190513/9f6d6901/attachment-0001.html> ------------------------------ Message: 20 Date: Tue, 14 May 2019 09:03:30 +0300 From: Saku Ytti <s...@ytti.fi> To: prohr...@stage2networks.com Cc: "nanog@nanog.org" <nanog@nanog.org> Subject: Re: Issues with SIP packets between VZ Fios and NTT Message-ID: <caaeewd_nowcrvkf7_om9gs2na4bvfvbfu36pmtpkmiegv3s...@mail.gmail.com> Content-Type: text/plain; charset="UTF-8" Can someone try to recreate the problem with TCP/5060. Or do iperf test on equivalent ports with UDP+TCP, to determine if the problem is related specifically to UDP. Most networks have some form of limits to even transit traffic, UDP is most typical L4 to have policers. On Tue, 14 May 2019 at 00:12, Pete Rohrman <prohr...@stage2networks.com> wrote: > > Dovid Bender, > > I'm seeing the same sort of thing. Polycom phones. Multiple customers > getting to me from Verizon in NYC area. I'm seeing phones register for a > while, then drop off, then I see them trying to re-reg resulting in your 401 > below. > > Call me. 212 497 8015. Let's look at this. > > Pete > > Pete Rohrman > Stage2 Support > 212 497 8000, Opt. 2 > > On 5/13/19 12:20 PM, Dovid Bender wrote: > > Thought of that. Customers have their own CPE's. So far the only thing mutual > here is that it's NTT -> VZ. Here is what I found so far looking at two > Polycom phones using non standard ports (e.g. not 5060) > 1) PhoneA tries to register multiple extensions and for each request we send > a 401. We expect to get back a REGISTER request with a no-once but we don't. > This happens for a while and then magically it starts working. > 2) PhoneB tries to register the time time as PhoneA and has no issues. > > At first I thought it was something possibly with the SIP call-ID but I ruled > that out since in the same SIP DIALOG it was not working then it started. > Also the seems to be per phone each phone is behind NAT and the traffic is > coming from a different NAT'd port. Seems like there is some device in the > middle that is randomly dropping traffic on specific sessions. > > > > > > On Mon, May 13, 2019 at 11:40 AM Brielle Bruns <br...@2mbit.com> wrote: >> >> On 5/13/2019 9:21 AM, Dovid Bender wrote: >> > Hi, >> > >> > Over the last 48 hours we have been getting a lot of alerts of customers >> > phones losing registrations to us. All the complaints are coming from >> > customers that are on VZ Fios in the NYC area. Anyone else see anything >> > strange going on? >> > >> >> >> While you are diagnosing, might check to make sure that the SIP ALG is >> disabled on all of their routers too. >> >> >> >> -- >> Brielle Bruns >> The Summit Open Source Development Group >> http://www.sosdg.org / http://www.ahbl.org -- ++ytti ------------------------------ Message: 21 Date: Tue, 14 May 2019 09:41:55 +0200 From: Mark Tinka <mark.ti...@seacom.mu> To: nanog@nanog.org Subject: Re: NTP for ASBRs? Message-ID: <0c74f5cb-9299-e6f9-b0e6-46863ca96...@seacom.mu> Content-Type: text/plain; charset=utf-8 On 9/May/19 02:47, Bryan Holloway wrote: > > > Hawai'i and Arizona can add/subtract without looking at the damn > calendar. I'm just sayin' I'd like to see more of that. Well, 2 months ago, the EU parliament voted to scrap daylight saving time from 2021. This would also apply to the UK if it chooses to remain in the EU, or during the extended transition period that Theresa May is currently working. It's now up to the various EU member states to decide whether they want to remain permanently in winter or permanently in summer. Of course, the UK government aren't necessarily amused. Mark. ------------------------------ Message: 22 Date: Tue, 14 May 2019 09:10:16 +0100 From: colin johnston <col...@gt86car.org.uk> To: Mark Tinka <mark.ti...@seacom.mu> Cc: nanog@nanog.org Subject: Re: NTP for ASBRs? Message-ID: <dd2322bd-6fa2-4f84-bb58-a33b81b9d...@gt86car.org.uk> Content-Type: text/plain; charset=us-ascii > On 14 May 2019, at 08:41, Mark Tinka <mark.ti...@seacom.mu> wrote: > > > > On 9/May/19 02:47, Bryan Holloway wrote: > >> >> >> Hawai'i and Arizona can add/subtract without looking at the damn >> calendar. I'm just sayin' I'd like to see more of that. > > Well, 2 months ago, the EU parliament voted to scrap daylight saving > time from 2021. This would also apply to the UK if it chooses to remain > in the EU, or during the extended transition period that Theresa May is > currently working. > > It's now up to the various EU member states to decide whether they want > to remain permanently in winter or permanently in summer. > > Of course, the UK government aren't necessarily amused. > > Mark. > Dst Time works great for Scotland as allows kids to go to school during lighter hours. It has been proved to save road deaths Col ------------------------------ Message: 23 Date: Tue, 14 May 2019 11:31:57 +0200 From: Mark Tinka <mark.ti...@seacom.mu> To: "nanog@nanog.org" <nanog@nanog.org> Subject: Fwd: [afnog] SAFNOG-5 & iWeek 2019 Registrations Open Message-ID: <a58f9917-1c39-1dbf-bf23-bb993719c...@seacom.mu> Content-Type: text/plain; charset="utf-8" FYI. Mark. -------- Forwarded Message -------- Subject: [afnog] SAFNOG-5 & iWeek 2019 Registrations Open Date: Mon, 13 May 2019 11:06:53 +0000 From: Portia Rabonda <portia.rabo...@safnog.net> To: af...@afnog.org <af...@afnog.org> Greetings All, It gives us great pleasure to announce that the SAFNOG-5 & iWeek 2019 event registration is now open! SAFNOG, in collaboration with iWeek, is proud to announce that this year’s event registration is free of charge. SAFNOG-5 and iWeek 2019 will be held between the 26th- 28th August, 2019, in Johannesburg, South Africa. SAFNOG-5 seats are limited to 120 people, to reserve your spot, register through our website: http://safnog.org/. Please take note to select “SAFNOG-5” as the main event of attendance to secure your free seat. Information about the event programme, the venue, travel information and other important updates will be made available on http://safnog.org/ in due time. We are officially 105 days from the big showdown in Jozi! We look forward to seeing you there! Regards, Portia Rabonda on Behalf of SAFNOG MC -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190514/ba3116d6/attachment-0001.html> -------------- next part -------------- _______________________________________________ afnog mailing list https://www.afnog.org/mailman/listinfo/afnog ------------------------------ Message: 24 Date: Tue, 14 May 2019 07:48:25 -0400 From: Dovid Bender <do...@telecurve.com> To: Saku Ytti <s...@ytti.fi> Cc: Peter Rohrman <prohr...@stage2networks.com>, "nanog@nanog.org" <nanog@nanog.org> Subject: Re: Issues with SIP packets between VZ Fios and NTT Message-ID: <cam3tth2+4zxjgylxrfmxazudrmkoee9w8ddk+3qwrj2ntna...@mail.gmail.com> Content-Type: text/plain; charset="utf-8" It's not strictly UDP. I spoke with someone yesterday that was re-producing it with curl. On Tue, May 14, 2019 at 2:04 AM Saku Ytti <s...@ytti.fi> wrote: > Can someone try to recreate the problem with TCP/5060. Or do iperf > test on equivalent ports with UDP+TCP, to determine if the problem is > related specifically to UDP. > > Most networks have some form of limits to even transit traffic, UDP is > most typical L4 to have policers. > > On Tue, 14 May 2019 at 00:12, Pete Rohrman <prohr...@stage2networks.com> > wrote: > > > > Dovid Bender, > > > > I'm seeing the same sort of thing. Polycom phones. Multiple > customers getting to me from Verizon in NYC area. I'm seeing phones > register for a while, then drop off, then I see them trying to re-reg > resulting in your 401 below. > > > > Call me. 212 497 8015. Let's look at this. > > > > Pete > > > > Pete Rohrman > > Stage2 Support > > 212 497 8000, Opt. 2 > > > > On 5/13/19 12:20 PM, Dovid Bender wrote: > > > > Thought of that. Customers have their own CPE's. So far the only thing > mutual here is that it's NTT -> VZ. Here is what I found so far looking at > two Polycom phones using non standard ports (e.g. not 5060) > > 1) PhoneA tries to register multiple extensions and for each request we > send a 401. We expect to get back a REGISTER request with a no-once but we > don't. This happens for a while and then magically it starts working. > > 2) PhoneB tries to register the time time as PhoneA and has no issues. > > > > At first I thought it was something possibly with the SIP call-ID but I > ruled that out since in the same SIP DIALOG it was not working then it > started. Also the seems to be per phone each phone is behind NAT and the > traffic is coming from a different NAT'd port. Seems like there is some > device in the middle that is randomly dropping traffic on specific sessions. > > > > > > > > > > > > On Mon, May 13, 2019 at 11:40 AM Brielle Bruns <br...@2mbit.com> wrote: > >> > >> On 5/13/2019 9:21 AM, Dovid Bender wrote: > >> > Hi, > >> > > >> > Over the last 48 hours we have been getting a lot of alerts of > customers > >> > phones losing registrations to us. All the complaints are coming from > >> > customers that are on VZ Fios in the NYC area. Anyone else see > anything > >> > strange going on? > >> > > >> > >> > >> While you are diagnosing, might check to make sure that the SIP ALG is > >> disabled on all of their routers too. > >> > >> > >> > >> -- > >> Brielle Bruns > >> The Summit Open Source Development Group > >> http://www.sosdg.org / http://www.ahbl.org > > > > -- > ++ytti > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190514/060b4add/attachment-0001.html> End of NANOG Digest, Vol 136, Issue 14 **************************************