Re: 95th billing and automation

[Jason Canady]

We use rtg2, which stores data in MySQL.  I use PHP to calculate percentiles.  
It allows for most flexibility.  

> 
> On Dec 10, 2020, at 13:29, Mehmet Akcin  wrote:
> 
> 
> hi there,
> 
> i have asked about this in the past. What is the best tool out there to do 
> 95th percentile billing. I have decided to use observium and librenms as 
> result of responses but there seems to be some kind of billing module issue 
> with these tools (thy are basically the same code). 
> 
> What are other systems besides observium and librenms (and old fashion cacti) 
> people are using these days with 95th billing and integration with a CRM like 
> salesforce/zoho, etc. I appreciate the responses.
> 
> Mehmet

Re: NDAA passed: Internet and Online Streaming Services Emergency Alert Study

[Jason Canady]

I agree with Mike on this.


On 1/4/21 10:17 AM, Mike Hammett wrote:
Every device that would be capable of doing anything also has an OS. 
That OS is likely shared amongst multiple device models.


The only involvement ISPs should have is ensuring that they have 
proper IP <-> geolocation information and your standard IP forwarding 
principles. ISPs should not be involved in the processing or design of 
any of this. It simply doesn't involve them.




-
Mike Hammett
Intelligent Computing Solutions 

Midwest Internet Exchange 

The Brothers WISP 


*From: *"Masataka Ohta" 
*To: *nanog@nanog.org
*Sent: *Monday, January 4, 2021 9:01:57 AM
*Subject: *Re: NDAA passed: Internet and Online Streaming Services 
Emergency Alert Study


Mike Hammett wrote:

> What makes the most sense is the underlying OS does the work and not
> each individual app.

It all depends on not OSes but devices.

Any device with speaker should produce audible alert and any
device with display should produce visible alert.

As devices are identified at the IP layer, the alert must be
distributed at the IP layer, that is, by ISPs.

Masataka Ohta

Re: New minimum speed for US broadband connections

[Jason Canady]

I second Mike.


On 5/28/21 8:37 AM, Mike Hammett wrote:

I don't think it needs to change.



-
Mike Hammett
Intelligent Computing Solutions
http://www.ics-il.com

Midwest-IX
http://www.midwest-ix.com


*From: *"Sean Donelan" 
*To: *nanog@nanog.org
*Sent: *Thursday, May 27, 2021 7:29:08 PM
*Subject: *New minimum speed for US broadband connections


What should be the new minimum speed for "broadband" in the U.S.?


This is the list of past minimum broadband speed definitions by year

year  speed

1999  200 kbps in both directions (this was chosen as faster than
dialup/ISDN speeds)

2000  200 kbps in at least one direction (changed because too many 
service

providers had 128 kbps upload)

2010   4 mbps down / 1 mbps up

2015   25 Mbps down / 3 Mbps up (wired)
         5 Mbps down / 1 Mbps up (wireless)

2021   ??? / ??? (some Senators propose 100/100 mbps)

Not only in major cities, but also rural areas

Note, the official broadband definition only means service providers 
can't

advertise it as "broadband" or qualify for subsidies; not that they must
deliver better service.

Re: Can't Port from a Particular Rate Center

[Jason Canady]

Another trick I've used is to use a separate number and forward the old 
number to the new.  Set the caller ID to the original number, test 911.  
You may want to run the 911 with the new number instead though.  With 
this setup, you can try porting again down the road, but at least you 
have a solution in the current time.


On 6/10/21 7:32 AM, Ray Orsini wrote:
If there's wireless you can always try porting to wireless. We do that 
in a few rate centers

OIT Website  
Ray Orsini​
Chief Executive Officer
OIT, LLC

	*305.967.6756 x1009*  	 | 		*305.571.6272* 



	*r...@oit.co*  	 | 	https://www.oit.co 
 	* www.oit.co* 


 oit.co/ray

Facebook 


LinkedIn 


Twitter 


YouTube 

*Headed to ASCII: Ohio on June 16th - 17th?Come meet the OITVOIP family!
​​Find your city and register for FREE using code "OIT" 
https://go.oit.co/ASCII2021* 



*From:* NANOG  on behalf of Peter 
Beckman 

*Sent:* Thursday, June 10, 2021 12:33:45 AM
*To:* Mike Hammett 
*Cc:* NANOG Operators' Group 
*Subject:* EXTERNAL: Re: Can't Port from a Particular Rate Center
CAUTION: This email originated from outside of the organization. Do 
not click links or open attachments unless you recognize the sender 
and know the content is safe. If you are unsure, please forward this 
email to the CSE team for review.



I had this happen to me recently.

Customer came in with a number that had very little coverage, but our
carrier had a 1,000 block in the same ratecenter, so we held out some 
hope.


Once we dug into it, the 1,000 block was designated for a different
"service offering" with the carrier. They were not offering portability in
that Ratecenter, despite having coverage, or even hardware or leased
hardware there.

So we had to send the customer off. There really were only about 5 
carriers

serving the Ratecenter, 3 of them wireless, one very local, and our
carrier.

If your carrier decides not to port a number, even when they seem to be
present in the ratecenter in question, they are not required by any law or
rule to port, AFAIK.

If a company will port in, the other carrier must (IMHO) port out. If not,
then you can't port. There may be some subtleties to that, but this is my
understanding.

Fun!

Beckman

On Wed, 9 Jun 2021, Mike Hammett wrote:

> I first asked on a list much more narrow in scope, but failing to get
> sufficient data points, I've expanded my scope.
>
> Assuming the number isn't held by someone exempt from porting, what 
would

> prevent someone from being able to port a number from a particular rate
> center in a LATA they have coverage in?
>
> We picked up a particular carrier for our out-of-area needs and the 
first

> thing we throw at them in a LATA we know they have coverage in, they
> can't do. They have a non-useful reason why. It doesn't appear to have
> moved to a state where they contacted the losing provider as the 
response

> was very fast, so my provider rejected the port, not theirs.
>
> When I started at this company (where we do our own porting), I made 
sure

> to port a bunch of numbers from all over our LATA to see what would
> happen. All successful. That seems to indicate that it doesn't matter
> which xLEC or tandem currently serves that number, it can move 
elsewhere.

>
>
>
> -
> Mike Hammett
> Intelligent Computing Solutions
> http://www.ics-il.com 
>
> Midwest-IX
> http://www.midwest-ix.com 
>
>

---
Peter Beckman Internet Guy
beck...@angryox.com http://www.angryox.com/ 
---

Re: akamai yesterday - what in the world was that

[Jason Canady]

We saw a higher load overnight, a little bit of a spike last night, but 
really hard to tell overall with our traffic.  Updates were still going 
at 8am today.  We run a local/regional WISP.


On 2/12/20 9:46 AM, Brandon Martin wrote:

On 2/11/20 6:41 PM, Tom Deligiannis wrote:
There is a major update that has released today, how's everything 
looking for everyone?


I run a couple distinct very small networks.  Both are transit-only 
with no direct peering or local caching and generally sub-gbps.


One set a new 1-min 95% record and did so by nearly 25% over its 
previous record.  The other matched its existing record.  The former 
thankfully has ample capacity, while the latter thankfully did so 
before primetime.


These are definitely going to be harder to manage than the primetime 
hump.  It would be nice if things could drop overnight to hopefully 
spread things out during the daytime lull some.  I understand that 
some people won't pick it up until they get home and turn on devices 
(which is often after school hours for these type of game updates), 
but spreading things out would be really nice especially for those of 
us without local caching.

Re: The great Netflix vpn debacle!

[Jason Canady]

We don’t NAT IPv4 and we’ve had a few new issues with Netflix (had to fix it a 
few years ago too).They resolved it this time, thankfully!  

> On Aug 31, 2021, at 18:15, Mark Andrews  wrote:
> 
> Force the traffic to these companies to use IPv6.  Advise your customers that
> you are doing this, why you are doing this and what steps they need to take
> to enable IPv6 on their equipment. Your customers can’t be in a worse 
> position.
> 
> "Dear customer,
>   if you want to reach … you will need to enable IPv6 support in
> your home network.  The world ran out of enough IPv4 for everyone several 
> years
> back and we have been sharing IPv4 between customers to allow you to reach 
> IPv4
> only sites.  The afore mentioned companies are now blocking IPv4 connections 
> from
> ISPs that have to share IPv4 addresses.  To give you a better service we are
> blocking IPv4 connections to these companies so you will get a more reliable 
> service
> over IPv6.
> 
> For instructions on how to enable IPv6 connectivity on you home router see 
> this
> page ….
> 
> If your home router does not support IPv6 you will need to upgrade it to one 
> that does."
> 
>> On 1 Sep 2021, at 06:36, Bryan Holloway  wrote:
>> 
>> Thanks, Owen ... good point.
>> 
>> Now hearing reports for these same prefixes with Disney+ too.
>> 
>> So the common denominators are:
>> 
>> HBO
>> Hulu
>> Netflix
>> Amazon Prime
>> Disney+
>> 
>> ... there has _got_ to be some new-fangled DB somewhere. This all started in 
>> the last month or so.
>> 
>> All of our RR objects, whois, DNS is solid ... dehr?
>> 
>> Fun times.
>> 
>> 
>> On 8/31/21 9:16 PM, Owen DeLong wrote:
>> 
>> [snip]
>> 
>>> Geolocate and VPN or Not are often kind of tied to the same kinds of 
>>> reporting services and it may well be that whatever provider HBO is using 
>>> for one is also being used for the other.
>>> Owen
> 
> -- 
> Mark Andrews, ISC
> 1 Seymour St., Dundas Valley, NSW 2117, Australia
> PHONE: +61 2 9871 4742  INTERNET: ma...@isc.org
> 

Re: AS6461 issues in Montreal

[Jason Canady]

We're in Indianapolis / Chicago and seeing 854,787 routes.

On 9/24/21 11:17 AM, Eric Dugas via NANOG wrote:

Hello,

Anyone else seeing a large withdrawal of routes on their Zayo AS6461 
sessions? We've lost about 400k routes at around 10:40 EDT.


Nothing in their Network Status so far

Eric

Re: What’s up with Comcast in Philadelphia area

[Jason Canady]

It's also affecting the Midwest.  No update on what's going on.

On 11/9/21 9:25 AM, Justin Keller wrote:

Hello.
Anyone know what's up with Comcast in the Philadelphia area? There
seems to be a lot of outages both residential and business

Justin

Re: FCC proposes higher speed goals (100/20 Mbps) for USF providers

[Jason Canady]

On 6/6/22 10:56 AM, Casey Russell via NANOG wrote:



For a long time now...

I have had the opinion that we have reached the age of "peak
bandwidth", that nearly nobody's 4 person home needs more than 50Mbit
with good queue management. Certainly increasing upload
speeds dramatically (and making static IP addressing and saner
firewalling feasible) might shift some resources from the cloud, which
I'd like (anyone using tailscale here?), but despite
8k video (which nobody can discern), it's really hard to use up >
50Mbit for more than a second or three with current applications.


One single digital game download to a console (xbox, playstation, 
etc.) can be over 80Gb of data.  That's half of your Saturday just 
waiting to play a game.  That assumes you'r'e getting the full 50Mbit 
(your provider isn't oversubscribing) to yourself in the home.  It 
also assumes your console (and all the games on it) is fully updated 
when you fired it up to download that new game. Hope you didn't want a 
couple of new games (after Christmas or a birthday). I admit, it's not 
a daily activity, and it might not look like much in a monthly 
average.  But I'd argue there are plenty of applications where 50Mbit 
equals HOURS of download wait for "average families" already today, 
not seconds.
At what price, is that worth though, Casey?  Simply set the game to 
download overnight.  It's better than standing in line outside of a store!

bfd & IPv6 on Cisco 4948E-E / IOS 15.2

[Jason Canady]

I am attempting to setup bfd with IPv6 on Cisco 4948E-E running IOS 
15.2.  bfd on IPv4 works great, but I'm having troubles with IPv6 and 
spent hours on it.  The trouble exists whether I use OSPFv3 or BGP.  
Each side transmits, but the other side doesn't receive.  Same interface 
works fine on IPv4. Any and all help would be appreciated!


Using this on the interface of each switch:

 ospfv3 1 bfd
 ospfv3 1 ipv6 area 0
 ospfv3 1 ipv6 bfd
 bfd interval 500 min_rx 500 multiplier 40

#show bfd neighbors details
IPv6 Sessions
NeighAddr  LD/RD RH/RS State Int
FE80::A2EC:F9FF:FE2B:B33F  68/0  Down Down  Te1/52
Session Host: Software
OurAddr: FE80::BA38:61FF:FE65:20BF
Handle: 1
Local Diag: 0, Demand mode: 0, Poll bit: 0
MinTxInt: 100, MinRxInt: 100, Multiplier: 40
Received MinRxInt: 0, Received Multiplier: 0
Holddown (hits): 0(0), Hello (hits): 1000(566)
Rx Count: 0, Rx Interval (ms) min/max/avg: 0/0/0 last: 23068067 ms ago
Tx Count: 566, Tx Interval (ms) min/max/avg: 756/1000/875 last: 268 ms ago
Elapsed time watermarks: 0 0 (last: 0)
Registered protocols: OSPFv3 CEF
Last packet: Version: 1  - Diagnostic: 0
 State bit: AdminDown    - Demand bit: 0
 Poll bit: 0 - Final bit: 0
 C bit: 0
 Multiplier: 0   - Length: 0
 My Discr.: 0    - Your Discr.: 0
 Min tx interval: 0  - Min rx interval: 0
 Min Echo interval: 0
#

Re: bfd & IPv6 on Cisco 4948E-E / IOS 15.2

[Jason Canady]

Hi Tom,

Thank you!   None of this is working and unless I've missed something, 
it doesn't seem to be supported.  I tried downgrading to 15.0 and it 
didn't even have bfd support for anything at all.


I may just have to go without bfd for IPv6 and turn the timers down on 
OSPFv3.  I'm open to any further suggestions or thoughts!


Best Regards,

Jason

On 6/7/23 2:25 PM, Tom Hill wrote:

On 07/06/2023 04:13, Jason Canady wrote:


Using this on the interface of each switch:

  ospfv3 1 bfd
  ospfv3 1 ipv6 area 0
  ospfv3 1 ipv6 bfd
  bfd interval 500 min_rx 500 multiplier 40

#show bfd neighbors details
IPv6 Sessions
NeighAddr  LD/RD RH/RS State Int
FE80::A2EC:F9FF:FE2B:B33F  68/0  Down Down  
Te1/52

Session Host: Software
OurAddr: FE80::BA38:61FF:FE65:20BF



There's literally one command here in the docs, and it doesn't look 
like you're using it. You are using one that isn't documented, too. Woo!


https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_ospf/configuration/15-e/iro-15-e-book/ip6-route-ospfv3.html 



I'd suggest taking the 'ospfv3 1 ipv6 bfd' command out and seeing if 
that still gives you an IPv6 session attempt? If in doubt, add the 
'all- interfaces' to 'ospfv3 1 bfd' & look again.


Old IOS and old hardware. Great gear at the time, but I can't imagine 
anyone at Cisco will be interested in fixing it if it's not quite 
working right.


The only other thing I've be interested to know is if you can specify 
a pair of static link-local neighbour addresses under 'ospfv3 1 ipv6 
bfd ...'?  Something like fe80::1 and fe80::2? As opposed to relying 
on autoconf addresses.

Formula1 / F1TV Contact

[Jason Canady]

Does anyone here work at Formula1 / F1TV or have a contact?  They are 
blocking our network and we need to get this resolved.


Thank you!

Best Regards,

Jason Canady
Unlimited Net / AS11990

Re: Contact for Hulu

[Jason Canady]

Digital Element helped promptly last time, it was mainly with Hulu Live. 
ipad...@hulu.com has helped before, but they didn't last time when 
Digital Element did.


https://www.digitalelement.com/contact-us/

Hope this helps!

On 9/27/23 11:46 AM, Brad Bendy wrote:

Can anyone at Hulu contact me off list? Have issues with some new
subnets we have and our end users cannot access the Hulu service with
various error messages.

Thanks

Re: Host.us DDOS attack

[Jason Canady]

Strange that they cannot send a BGP blackhole upstream to keep everyone 
else online within their advertised route.


On 8/3/16 5:27 PM, Tony Wicks wrote:

Further to that, and I would suggest it should be part of the overall 
discussion here. It appears the IPv4 IP block my VM is in is not currently 
advertised on the world route table. I assume hostus.us's transit provider has 
dropped their ipv4 BGP to save themselves. This is really the ultimate reward 
for the extortionists as they don't even need to sustain the DDOS to attack 
their target. While I see the transit providers point of view, it’s a pretty 
shitty situation for their customer, and their customers/customers.



-Original Message-
From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Tony Wicks
Sent: Thursday, 4 August 2016 9:10 AM
To: 'NANOG list' 
Subject: RE: Host.us DDOS attack

Interestingly my VM (LA) with them has been effectively down for half a day as 
far as IPv4 is concerned. IPv6 traffic seems unaffected.

Re: Two BGP peering sessions on single Comcast Fiber Connection?

[Jason Canady]

I completely concur.  We spread our uplinks across separate boxes and we have 
/29 allocations.  Get the best of all worlds. But if I only had one provider, 
I'd want to have multiple BGP sessions for this reason.  

> On Oct 17, 2016, at 08:30, Mike Hammett  wrote:
> 
> It really seems like it's a grave oversight to *NOT* support multiple BGP 
> sessions. I drop to two routers for that same reason, I can do maintenance on 
> one, while the other carries traffic. 
> 
> 
> 
> 
> - 
> Mike Hammett 
> Intelligent Computing Solutions 
> http://www.ics-il.com 
> 
> Midwest-IX 
> http://www.midwest-ix.com 
> 
> - Original Message -
> 
> From: "Mike Poublon"  
> To: "rar" , nanog@nanog.org 
> Sent: Thursday, October 13, 2016 2:04:29 PM 
> Subject: Re: Two BGP peering sessions on single Comcast Fiber Connection? 
> 
> I started a thread around the same topic back on 10/16 of 2014. A 
> Comcast engineer (who ultimately spoke to the national product manager) 
> came back after discussing and said the same thing "We don't support 
> that". I got a slightly longer explanation of: 
> 
>  
> 
> In a nutshell, when we design a product we do it to accommodate the most 
> typical customer cases. 
> Given that the design includes a single fiber path and thus the fiber 
> path and device that terminates on either end each are a single point of 
> failure, adding extra BGP sessions doesn’t seem to add value in the 
> typical failure scenarios. In order to achieve the simplest and most 
> scalable solution to address the market, we rely on narrowing the 
> possible combinations of parameters. 
> 
>  
> 
> I explained to them that their interpretation prevents me from being 
> able to do concurrent maintenance on my side (single router 
> reboot/upgrade, etc). Never got anywhere with it though. 
> 
> I'm still interested in having this set up, but have given up on it ever 
> really coming to reality. Luckily ALL of my other providers were more 
> than happy to set up an extra session. 
> 
> If anyone from Comcast is listening, there is customer demand for this. 
> It's not about making it better for Comcast, it's about allowing 
> customers to have more flexibility. 
> 
> Mike Poublon 
> 
> /Senior Datacenter Network Engineer/ 
> 
> *Secant Technologies* 
> 
> 6395 Technology Ave. Suite A 
> 
> Kalamazoo, MI 49009 
> 
>> On 10/13/2016 1:48 PM, rar wrote: 
>> After a many month wait, we were ready to turn up our BGP peering sessions 
>> on a new Comcast fiber connection. 
>> 
>> With our other providers (Level 3 and Verizon) we have edge routers that 
>> directly connect between the provider's on premise connection and our 
>> primary and a backup core routers. Each core router has a multihop BGP 
>> session with the provider's BGP router. The goal is to keep the single BGP 
>> router from being a single point of failure. 
>> 
>> Comcast said they could not support two separate BGP peering sessions on the 
>> same circuit. Does anyone have any counter examples? We used to have this 
>> setup with Comcast 5+ years ago, but now they say they can't support it. 
>> 
>> 
>> Bob Roswell 
>> brosw...@syssrc.com 
>> 410-771-5544 ext 4336 
>> 
>> Computer Museum Highlights
> 
> 

Re: backbones filtering unsanctioned sites

[Jason Canady]

Cogent's best friend to the rescue: http://bgp.he.net/ip/104.31.18.30#_dns

Looks like mostly proxy/torrent sites on that IP address.

-- 

Jason Canady
Unlimited Net, LLC
Responsive, Reliable, Secure

On 2/11/17 5:11 PM, Marco Teixeira wrote:
> So... i doubt CloudFlare allocates one ip per domain served... which means
> Cogent customers will be unable to access other CloudFlare proxied site,
> served by this same IP, for a particular geographic zone?
>
>
> ---
> Marco
>
>
>
> On Sat, Feb 11, 2017 at 8:44 PM, Alistair Mackenzie 
> wrote:
>
>> Cogent confirmed on the phone that they are the ones who put the blackhole
>> in place. This is after they closed our ticket twice without response.
>>
>> Purposely didn't mention a website in the ticket yet they asked on the
>> phone if it was regarding thepiratebay so they are very aware of this...
>>
>> On 11 February 2017 at 15:18, Bryan Holloway  wrote:
>>
>>> Yup, they do indeed. And for fun, I black-listed one of our IPs, and sure
>>> enough, the next-hop shows up as 10.255.255.255, and the communities are
>>> the same aside from what appear to be regional things.
>>>
>>> --
>>>
>>> BGP routing table entry for 66.253.214.90/32, version 638637516
>>> Paths: (1 available, best #1, table Default-IP-Routing-Table)
>>> Flag: 0x820
>>>   23473
>>> 10.255.255.255 (metric 10177050) from 154.54.66.21 (154.54.66.21)
>>>   Origin IGP, localpref 150, valid, internal, best
>>>   Community: 174:990 174:20912 174:21001 174:22013
>>>   Originator: 66.28.1.228, Cluster list: 154.54.66.21, 66.28.1.9
>>>
>>>
>>>
>>> On 2/10/17 1:49 PM, Alistair Mackenzie wrote:
>>>
>>>> Cogent also have a blackhole route-server that they will provide to you
>> to
>>>> announce /32's for blackholing.
>>>>
>>>> The address for this is 66.28.1.228 which is the originator for the
>>>> 104.31.19.30/3 <http://104.31.19.30/32>2 and 104.31.18.30/32 routes.
>>>>
>>>>
>>>> On 10 February 2017 at 18:46, Jason Rokeach  wrote:
>>>>
>>>> This looks pretty intentional to me.  From
>>>>> http://www.cogentco.com/en/network/looking-glass:
>>>>>
>>>>> BGP routing table entry for 104.31.18.30/32, version 611495773
>>>>> Paths: (1 available, best #1, table Default-IP-Routing-Table)
>>>>>   Local
>>>>> 10.255.255.255 (metric 10177050) from 154.54.66.21 (154.54.66.21)
>>>>>   Origin IGP, metric 0, localpref 150, valid, internal, best
>>>>>   Community: 174:990 174:20912 174:21001
>>>>>   Originator: 66.28.1.228, Cluster list: 154.54.66.21, 66.28.1.9
>>>>>
>>>>> BGP routing table entry for 104.31.19.30/32, version 611495772
>>>>> Paths: (1 available, best #1, table Default-IP-Routing-Table)
>>>>>   Local
>>>>> 10.255.255.255 (metric 10177050) from 154.54.66.21 (154.54.66.21)
>>>>>   Origin IGP, metric 0, localpref 150, valid, internal, best
>>>>>   Community: 174:990 174:20912 174:21001
>>>>>   Originator: 66.28.1.228, Cluster list: 154.54.66.21, 66.28.1.9
>>>>>
>>>>>
>>>>> Call it a "hunch" but I doubt 10.255.255.255 is a valid next-hop
>> router.
>>>>> On Fri, Feb 10, 2017 at 1:39 PM, Mike Hammett 
>> wrote:
>>>>> Have we determined that this is intentional vs. some screw up?
>>>>>>
>>>>>>
>>>>>>
>>>>>> -
>>>>>> Mike Hammett
>>>>>> Intelligent Computing Solutions
>>>>>> http://www.ics-il.com
>>>>>>
>>>>>> Midwest-IX
>>>>>> http://www.midwest-ix.com
>>>>>>
>>>>>> - Original Message -
>>>>>>
>>>>>> From: "Brielle Bruns" 
>>>>>> To: nanog@nanog.org
>>>>>> Sent: Friday, February 10, 2017 12:28:53 PM
>>>>>> Subject: Re: backbones filtering unsanctioned sites
>>>>>>
>>>>>> On 2/9/17 9:18 PM, Ken Chase wrote:
>>>>>>
>>>>>>> https://torrentfreak.com/internet-backbone-provider-
>>>>>>>
>>>>>> cogent-blocks-pirate-bay-and-other-pirate-sites-170209/
>>>>>>
>>>>>>> /kc
>>>>>>>
>>>>>>>
>>>>>> Funny. Someone else got back:
>>>>>>
>>>>>> "Abuse cannot not provide you a list of websites that may be
>>>>>> encountering reduced visibility via Cogent"
>>>>>>
>>>>>> I almost wish I had a Cogent circuit just to bring this up with an
>>>>>> account rep. Almost.
>>>>>>
>>>>>> I'd very much so view this as a contractual violation on Cogent's
>> part.
>>>>>> Cogent keeps contacting me every year wanting to sell me service. This
>>>>>> will be a good one to bring up when they call me next time.
>>>>>>
>>>>>> --
>>>>>> Brielle Bruns
>>>>>> The Summit Open Source Development Group
>>>>>> http://www.sosdg.org / http://www.ahbl.org
>>>>>>
>>>>>>
>>>>>>

JCPenny Contact

[Jason Canady]

Hello,

We are looking to contact someone at JCPenny in the abuse/NOC 
department.  Can anyone contact me from JCPenny or send me information 
off-list to someone there that can help me?


Thank you!

Jason

Hulu / ESPN: Commercial IP Address

[Jason Canady]

Hello,

I have a customer that is using Hulu Live to stream ESPN, however it 
isn't showing up in their Channel list.  They reached out to Hulu and 
it's because their IP address is 'commercial'.  We have many customers 
using Hulu without problems, but it seems specific to ESPN.  Anyone else 
have this issue?  Do you reach out to ESPN or Hulu?


If anyone has any information, please share it.  Appreciate your help in 
advance!


Best Regards,

Jason Canady
Unlimited Net, LLC
Responsive, Reliable, Secure

Re: Cogent charging 50/mo for BGP (not IPs, the service)

[Jason Canady]

Yes.  Our service didn't start out this way, but a few years ago they 
added that.  At least my rep at the time quoted me out with the fee 
added into it.  I believe IPv6 BGP is free.


On 10/17/18 11:47 AM, Josh Luthman wrote:
Has anyone else dealt with this mess?  Even my Cogent rep admits it's 
unique to their business.


Josh Luthman
Office: 937-552-2340
Direct: 937-552-2343
1100 Wayne St
Suite 1337
Troy, OH 45373

Re: Whats going on at Cogent

[Jason Canady]

It's been slow for quite some time now. I only find it useful for 
billing purposes.  It's a shame carriers don't have a good ticket system.


Jason Canady
Unlimited Net, LLC
Responsive, Reliable, Secure

On 10/19/18 5:47 PM, Aaron1 wrote:

Yes I noticed that last week, it is very slow

Aaron

On Oct 19, 2018, at 4:43 PM, Ryan Gelobter 
<mailto:rya...@andthenwegotpaycuts.com>> wrote:


Has anyone else noticed their ecogent portal is super fucking slow? 
Back in the day it used to be fast


On Thu, Oct 18, 2018 at 2:12 PM Troy Mursch <mailto:t...@wolvtech.com>> wrote:


Cogent has done well to remediate the compromised MikroTik
routers on their network. 3,000 IPv4 hosts were found on Aug. 25
(https://twitter.com/bad_packets/status/1033256704941514752) and
today, only a hundred:

https://censys.io/ipv4?q=%28%28%28%22CoinHive.Anonymous%22%29+AND+%28MikroTik%29%29+AND+location.country_code%3A+US%29+AND+autonomous_system.description.raw%3A+%22COGENT-174+-+Cogent+Communications%22&;

__

*Troy Mursch*



On Thu, Oct 18, 2018 at 12:05 PM Aaron Gould mailto:aar...@gvtc.com>> wrote:

I guess those bots have to sit somewhere.  I don’t know that
they would be in routers as much as they would be in
Microsoft Windows… so if that’s what you meant, then I see
what you mean Michael

Niels, I like my cogent and telia internet connections… I
just recall seeing more ddos on cogent then I did on my
previous att, and current spectrum… telia is showing a good
bit of ddos also

Let’s put it this way, I can thank Cogent and Telia for
helping my get better in my ddos mitigation skills ☺… there’s
a bright side to everything huh

Aaron

*From:*NANOG [mailto:nanog-boun...@nanog.org
<mailto:nanog-boun...@nanog.org>] *On Behalf Of *Michael Crapse
*Sent:* Tuesday, October 16, 2018 8:37 PM
*To:* NANOG list
*Subject:* Re: Whats going on at Cogent

Or he's saying that cogent has the biggest network of
compromised users. Usually ipv4 only eyeball networks tend to
have the most bots on net.

On Tue, 16 Oct 2018 at 19:22, Niels Bakker
mailto:na...@bakker.net>> wrote:

* aar...@gvtc.com <mailto:aar...@gvtc.com> (Aaron1) [Wed
17 Oct 2018, 00:17 CEST]:
>However Cogent seems to be the dirtiest in regards to
DDOS...
>however Telia might be catching up... in times past when
I receive
>volumetric DDOS, Cogent typically ranks with the highest
on my
>providers ... AT&T and spectrum seem to be a bit cleaner

So you're saying, Cogent and Telia have the best
backbones and
interconnects and thus deliver the most of your traffic
to you,
even at times of peak utilization?


        -- Niels.

Re: Spectrum technical contact

[Jason Canady]

The /32 should override any static route they are sending you with a 
larger prefix.


Jason Canady
Unlimited Net, LLC
Responsive, Reliable, Secure

On 12/22/18 11:30 AM, Josh Luthman wrote:
I do BGP with them, but of course the issue is an IP that they route 
to me.


My issue is with ASN 10796

Josh Luthman
Office: 937-552-2340
Direct: 937-552-2343
1100 Wayne St
Suite 1337
Troy, OH 45373


On Fri, Dec 21, 2018 at 4:55 PM Aaron1 <mailto:aar...@gvtc.com>> wrote:


If you BGP neighbor with them you can send-community /32
advertisement to them, and the will remotely black hole it

Aaron

> On Dec 21, 2018, at 3:51 PM, Josh Luthman
mailto:j...@imaginenetworksllc.com>>
wrote:
>
> We have had a DOS attack for over 12 hours.  I simply want them
to null route or black hole an address.  The traffic is filling
one of our circus with them.
>
> The farthest I got was them telling me they can't do route
changes because we're not public safety.
>
> Josh Luthman
> Office: 937-552-2340
> Direct: 937-552-2343
> 1100 Wayne St
> Suite 1337
> Troy, OH 45373

Re: Spectrum technical contact

[Jason Canady]

Your upstream provider is null routing it when you send them the command via 
BGP, no longer filling your pipe. 

> On Dec 22, 2018, at 19:24, Josh Luthman  wrote:
> 
> But if they route it to me and I null it, the traffic is already fillimg my 
> pipe (which is my issue).
> 
> Josh Luthman
> Office: 937-552-2340
> Direct: 937-552-2343
> 1100 Wayne St
> Suite 1337
> Troy, OH 45373
> 
>> On Sat, Dec 22, 2018, 11:32 AM Jason Canady > The /32 should override any static route they are sending you with a larger 
>> prefix.
>> Jason Canady
>> Unlimited Net, LLC
>> Responsive, Reliable, Secure
>>> On 12/22/18 11:30 AM, Josh Luthman wrote:
>>> I do BGP with them, but of course the issue is an IP that they route to me.
>>> 
>>> My issue is with ASN 10796
>>> 
>>> Josh Luthman
>>> Office: 937-552-2340
>>> Direct: 937-552-2343
>>> 1100 Wayne St
>>> Suite 1337
>>> Troy, OH 45373
>>> 
>>> 
>>>> On Fri, Dec 21, 2018 at 4:55 PM Aaron1  wrote:
>>>> If you BGP neighbor with them you can send-community /32 advertisement to 
>>>> them, and the will remotely black hole it 
>>>> 
>>>> Aaron
>>>> 
>>>> > On Dec 21, 2018, at 3:51 PM, Josh Luthman  
>>>> > wrote:
>>>> > 
>>>> > We have had a DOS attack for over 12 hours.  I simply want them to null 
>>>> > route or black hole an address.  The traffic is filling one of our 
>>>> > circus with them.
>>>> > 
>>>> > The farthest I got was them telling me they can't do route changes 
>>>> > because we're not public safety.
>>>> > 
>>>> > Josh Luthman
>>>> > Office: 937-552-2340
>>>> > Direct: 937-552-2343
>>>> > 1100 Wayne St
>>>> > Suite 1337
>>>> > Troy, OH 45373
>>>> 
>> 

Re: "Hypothetical" Datacenter Overheating

[Jason Canady]

Our Zayo circuit just came up 30 minutes ago and it routes through 350 E 
Cermak.  Chillers were all messed up.  No hypothetical there.  :-) It 
was down for over 16 hours!


On 1/15/24 10:04 AM, Bryan Holloway wrote:

I think we're beyond "hypothetical" at this point, Mike ... ;)


On 1/15/24 15:49, Mike Hammett wrote:

Coincidence indeed   ;-)



-
Mike Hammett
Intelligent Computing Solutions 
 


Midwest Internet Exchange 
 


The Brothers WISP 
 



*From: *"Clayton Zekelman" 
*To: *"Mike Hammett" , "NANOG" 
*Sent: *Monday, January 15, 2024 8:23:37 AM
*Subject: *Re: "Hypothetical" Datacenter Overheating




At 09:08 AM 2024-01-15, Mike Hammett wrote:
 >Let's say that hypothetically, a datacenter you're in had a cooling
 >failure and escalated to an average of 120 degrees before
 >mitigations started having an effect. What are normal QA procedures
 >on your behalf? What is the facility likely to be doing?
 >What  should be expected in the aftermath?

One would hope they would have had disaster recovery plans to bring
in outside cold air, and have executed on it quickly, rather than
hoping the chillers got repaired.

All our owned facilities have large outside air intakes, automatic
dampers and air mixing chambers in case of mechanical cooling
failure, because cooling systems are often not designed to run well
in extreme cold.  All of these can be manually run incase of controls
failure, but people tell me I'm a little obsessive over backup plans
for backup plans.

You will start to see premature failure of equipment over the coming
weeks/months/years.

Coincidentally, we have some gear in a data centre in the Chicago
area that is experiencing that sort of issue right now... :-(

Roku Network Contact

[Jason Canady]

Does anyone here have a network contact for Roku?  Need some 
assistance.  Thank you!


Best Regards,

Jason

Re: Why are paper LOAs still used?

[Jason Canady]

We just switched over to IRR routing with Cogent, it is available.  It's 
just not on by default.


Best Regards,

Jason

On 2/26/24 3:14 PM, Aaron Wendel wrote:
I don't have any examples of anyone still using paper LOAs except for 
Cogent.


Aaron


On 2/26/2024 12:57 PM, Seth Mattinen via NANOG wrote:
Why do companies still insist on, or deploy new systems that rely on 
paper LOA for IP and ASN resources? How can this be considered more 
trustworthy than RIR based IRR records?


And I'm not even talking about old companies, I have a situation 
right now where a VPS provider I'm using will no longer use IRR and 
only accepts new paper LOAs. In the year 2024. I don't understand how 
anyone can go backwards like that.


~Seth

Re: lots of internet starting at ~3 a.m. cst

[Jason Canady]

Yes, looks like all the families with young/middle aged teens are 
downloading updates.


On 7/23/24 09:03, Aaron Gould wrote:
Anyone else see a lot of Internet traffic starting at 3 a.m. and 
continuing even now?  Seems to be spiky tcp.

Re: hbo max geolocation issue

[Jason Canady]

We've had HBO blocked at same time as Hulu and Digital Element helped: 
https://www.digitalelement.com/contact-us/


On 8/27/24 13:16, Howard, Lee via NANOG wrote:


My source for geolocation updates is 
https://thebrotherswisp.com/index.php/geo-and-vpn and they say:


HBO: ctiaengine...@hbo.com

Lee

*From:*NANOG  
*On Behalf Of *Mehmet

*Sent:* Tuesday, August 27, 2024 12:00 PM
*To:* nanog 
*Subject:* hbo max geolocation issue




You don't often get email from meh...@akcin.net. Learn why this is 
important 




*This message is from an EXTERNAL SENDER - be CAUTIOUS, particularly 
with links and attachments.*


hello there

i am dealing with hbo max geolocation issue, is there anyone on the 
list from HBO team who can assist (or point me where i can look for a 
solution)


thank you

Google G Suite Email Contact

[Jason Canady]

Is anyone here from Google's G Suite or email department?  I recently 
acquired a brand who's domain is being blocked by Google Mail ("G 
Suite").  I have followed all of the steps to be compliant (SPF and 
DKIM), but email is still going into customer's Spam folder at Google.  
There is not a massive amount of emails sent, just basic communication 
such as monthly invoices, support tickets, etc.


If anyone has a contact, please share it with me on or off list. I would 
greatly appreciate it. Thank you!


Jason Canady
Unlimited Net, LLC

Re: SLA Monitoring

[Jason Canady]

We use various tools for monitoring here. Pingdom for external 
monitoring, Observium for internal and SmokePing for internal/external.


As far as Pingdom goes, we ended up paying for 3 years to lock in 
pricing because it keeps going up and the service doesn't improve, but 
it is useful.  I need to find an alternative prior to the next renewal.


--

Jason Canady
Unlimited Net, LLC
Responsive, Reliable, Secure

On 4/11/17 6:57 PM, Mike Hammett wrote:

What do you guys use for monitoring of SLAs, be it an upstream or a downstream 
SLA? I know of a couple services, just looking to see who's doing what and how 
they like it.




-
Mike Hammett
Intelligent Computing Solutions
http://www.ics-il.com

Midwest-IX
http://www.midwest-ix.com

Anyone here from Netflix? | VPN Detection Problem

[Jason Canady]

Hello,

We have IP addresses being blocked due to proxy servers that were once
on the IP address space.  They have now been reclaimed and will be used
for ISP services to end-users.  We need to ensure that customers can
watch Netflix without any issues.

Would someone please reach out to me to resolve this?

Thank you!

Best Regards,

Jason Canady
Unlimited Net, LLC

Amazon Streaming Department

[Jason Canady]

Hello all,

A while back I wrote in regarding an update on Netflix services / our IP
addresses being blocked.  I had helpful feedback in getting this
resolved.  Does anyone have a contact at Amazon for resolving IP
blacklist issues on their Amazon Prime / streaming services?  I have
contacted their Customer Service, but they aren't very easy to work with.

Thank you!

Best Regards,

Jason Canady
Unlimited Net, LLC

Target.com NOC Contact

[Jason Canady]

Hello,

Would anyone here have a contact for Target.com NOC?

Thank you!

- Jason

Re: ticketmaster.com 403 Forbidden

[Jason Canady]

Has anyone found a resolution to this?  Our network has been blocked and 
I had a customer mention it to me the other day, so I would like to get 
it resolved.


Thank you!

Best Regards,

--

Jason Canady
Unlimited Net, LLC
Responsive, Reliable, Secure

On 2/7/17 12:05 PM, Manser, Charles J wrote:

All,

Thank you for the suggestions. All (3) of the e-mail addresses associated with 
their ARIN records bounced back.

Remote Server returned '< #5.7.133 smtp;550 5.7.133 
RESOLVER.RST.SenderNotAuthenticatedForGroup; authentication required; Delivery 
restriction check failed because the sender was not authenticated when sending to 
this group>'

It can be difficult for consumers to work these issues individually, so we 
reached out to the NANOG community for an assist. The problem seemed widespread 
and not isolated to single customers and referring them to a web form did not 
seem like an option.

Good news: I am making some progress with the Live Nation/Ticketmaster team.

"Thank you for bringing this to our attention. We are conducting an 
investigation on suspicious activity that has been observed on the range of IP's are 
associated to your connectivity and will make every effort to do this as fast as 
possible."

Thank you all again for the help and I will keep the archive updated if we 
reach a repeatable resolution.

Regards,
  
Charles Manser | Principal Engineer I, Network Security

charles.man...@charter.com

-Original Message-
From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of joel jaeggli
Sent: Monday, February 06, 2017 7:38 PM
To: Suresh Ramasubramanian ; mike.l...@gmail.com; Ethan E. Dee 

Cc: Niels Bakker ; nanog@nanog.org
Subject: Re: ticketmaster.com 403 Forbidden

On 2/6/17 8:49 AM, Suresh Ramasubramanian wrote:

My guess is you have or had sometime in the long distant past a scalper 
operating on your network, using automated ticket purchase bots.

If you still have that scalper around, you might want to turf him.  If he’s 
ancient history, saying so might induce them to remove the block.

Note that scalper bots benefit from pools of residential ip addresses to
work with in subverting the anti-bot countermeasures of ticket sale
platforms. so there are the legitimate possibility that subverted hosts
are being used for that sort of thing.

--srs

On 06/02/17, 8:45 AM, "nanog-boun...@nanog.org on behalf of mike.l...@gmail.com" 
 wrote:

 Yup, i have a /22 that has the same problem. Support is useless...
 
 > On Feb 6, 2017, at 08:35, Ethan E. Dee  wrote:

 >
 > It gives me a Forbidden error.
 > It has for over a year.
 > There support says they are not allowed to me why by their policy.
 > it is across an entire /19.
 > I gave up after the fifth time and encourage the customers to call them 
individually.
 >
 >> On 02/06/2017 11:09 AM, Niels Bakker wrote:
 >> * charles.man...@charter.com (Manser, Charles J) [Mon 06 Feb 2017, 
16:21 CET]:
 >>> It seems that browsing to ticketmaster.com or any of the associated IP 
addresses results in a 403 Forbidden for our customers today. Is anyone else having this 
issue?
 >>
 >> 
http://help.ticketmaster.com/why-am-i-getting-a-blocked-forbidden-or-403-error-message/
 >>
 >>
 >>-- Niels.
 >
 






E-MAIL CONFIDENTIALITY NOTICE:
The contents of this e-mail message and any attachments are intended solely for 
the addressee(s) and may contain confidential and/or legally privileged 
information. If you are not the intended recipient of this message or if this 
message has been addressed to you in error, please immediately alert the sender 
by reply e-mail and then delete this message and any attachments. If you are 
not the intended recipient, you are notified that any use, dissemination, 
distribution, copying, or storage of this message or any attachment is strictly 
prohibited.

Re: congestion between Cogent and CenturyLink

[Jason Canady]

I'm already seeing a huge improvement to Comcast after Netflix moved a lot of 
traffic off of the ports.  


On Feb 27, 2014, at 22:21, Stephen Frost  wrote:

> * Paul S. (cont...@winterei.se) wrote:
>> +1, which semi-large eyeball does Cogent NOT have capacity problems to?
> 
> Soon, Comcast...  Given what's going on w/ them and Netflix.
> 
>Thanks,
> 
>Stephen

Re: dedicated server providers in Mexico?

[Jason Canady]

I have no experience with dedicated hosting providers in Mexico, but 
that list is incorrect.  I know that Steadfast does not have servers 
located in Mexico.  I believe other providers are also incorrectly listed.


You should search for providers on Web Hosting Talk, 
http://www.webhostingtalk.com


Regards,

Jason

On 4/29/14, 2:06 PM, Carlos Kamtha wrote:

Hi everyone,

I am currently not happy with out MX server provider, and so, inquiring
with anyone that can give a recommendation based on experience?

I found this list via google.

http://www.webhostingsearch.com/dedicated-server/mexico.php

I wondering if anyone can speak to any of the providers on this list
(outside of 1n1). offlist..

Feedback as always greatly appreciated.

Cheers,

Carlos.

Re: Major Level3 Issues

[Jason Canady]

Dean,

Best I have is from their portal:

The IP NOC reports that a module failed on a device in Chicago, IL causing 
additional impact to IP services across Multiple Markets. The IP NOC has 
confirmed that the module initially failed and recovered prior to manual 
intervention. A short time later, the module failed again, restored of its own 
accord and continued to bounce thereafter. Services have since been restored 
and stable as of approximately 23:54 GMT. The equipment vendor is being engaged 
to assist in isolation efforts to determine a root cause and the IP NOC is 
continuing to monitor at this time.

We rerouted away majority traffic from Level 3 and will leave it till late 
tonight or early morning. We experienced two major blips tonight. 

Jason

On Oct 16, 2014, at 21:25, Dean Perrine  wrote:

> Just had major outages on our Level3 connections across the country. (LA,
> TX, ORD)
> 
> Anyone have any news / updates which can be shared?
> 
> Looks like AT&T and XO had loss of connectivity to Level3?
> 
> Thank you!
> Dean

Re: Facebook outage?

[Jason Canady]

Instagram appears to be down as well, but that would make sense since they are 
part of Facebook. 

On Jan 27, 2015, at 1:50, Damien Burke  wrote:

> I hear that AIM and hipchat is also having issues. 
> 
> Any other major company down too?
> 
> -Original Message-
> From: John van Oppen [mailto:jvanop...@spectrumnet.us] 
> Sent: Monday, January 26, 2015 10:49 PM
> To: Damien Burke; nanog@nanog.org
> Subject: RE: Facebook outage?
> 
> Dead here at AS11404 from all locations where we PNI or public peer...   
> 
> must be bad over there, v4 dies at their edge, v6 makes it in but no page 
> loads.
> 
> John

Re: Indiana Dark Fiber

[Jason Canady]

Hi Mike,

Have you checked with Axia Technology Partners?  www.axiatp.com

They have fiber throughout Indiana. Doubtful they have their own dark fiber 
from Indy to Ft Wayne, but worth checking into!

-- 

Jason Canady
Unlimited Net, LLC
www.unlimitednet.us
twitter: @unlimitednet


On Jan 31, 2015, at 11:25, Mike Hammett  wrote:

> Any recommendations for dark fiber between Indianapolis and Fort Wayne? I see 
> a few routes, but that doesn't mean they actually sell dark fiber. The 
> company that we've partnered with asked a couple companies (Zayo, IFN and one 
> of the Time Warners) and came up short. Seems odd because Zayo lists dark 
> fiber routes to within 1.5 miles of the Fort Wayne location, but figured I'd 
> look to see what companies I was missing that actually did dark fiber. 
> 
> 
> 
> 
> - 
> Mike Hammett 
> Intelligent Computing Solutions 
> http://www.ics-il.com 
> 
> 
> 

Re: Provider to Blend with Level3

[Jason Canady]

Another good choice would be Cogent, AS174.  We use Cogent along with 
Level 3.  I'd say 2/3 of our traffic is on Level 3 and 1/3 on Cogent.  
It's been a great blend for us.  Justin Wilson recently made some great 
comments about Cogent on Feb 6, reference subject: Re: Input Regarding 
Cogent and NTT.


At this point, I would fully rely on Cogent for maintenance or outages 
on Level 3.  A year and a half ago, they had some problems between 
providers such as Comcast.  With Netflix getting direct connections, 
this is now resolved.


--

Jason Canady
Unlimited Net, LLC
www.unlimitednet.us
twitter: @unlimitednet

On 2/9/15 2:29 AM, vaibhav nikam wrote:

Hi,

You can check with CenturyLink

Regards

Vaib

On Mon, Feb 9, 2015 at 12:46 PM, Max Tulyev  wrote:


Hi!

If you have he.net there - it will be the best choise.

On 06.02.15 19:26, Colton Conor wrote:

We have a network that is single homed with Level3 at this time in

Dallas.

They already have BGP and their own ASN and IP setup. Who would you
recommend for a second provider in Dallas to blend with Level3? Assuming
Level3 and this other provider would be the only two in the blend for a
long time to come? Client was talking to TWT, but now that they are being
bought by Level3 that doesn't make much sense.

Re: GTT NOC

[Jason Canady]

Hi Ammar,

Sorry to hear this has happened. I do not have any contact info, but have you 
tried announcing more specific prefixes to override the hijacker?  

Jason


On Feb 13, 2015, at 20:10, Ammar Zuberi  wrote:

> Hi all,
> 
> Does anyone know of a direct phone number for someone with somewhat authority 
> at GTT? Our prefix has been hijacked by a customer of theirs and we haven’t 
> received any kind of response to our email and the guys on the phone seem to 
> not speak very good English.
> 
> Any ideas?
> 
> Ammar.

Re: Level(3) ex-twtelecom midwest packet loss (4323)

[Jason Canady]

Mike, I would take it to mean someone screwed something up and they 
don't want to admit to it.  :-)  That's just a guess.


--

Jason Canady
Unlimited Net, LLC
Responsive, Reliable, Secure

www.unlimitednet.us
ja...@unlimitednet.us
twitter: @unlimitednet

On 8/28/15 12:00 AM, Mike Hammett wrote:

08/28/2015 3:08 AM GMT
Event Conclusion Summary

Start: August 27, 2015 13:20 GMT
Stop: August 28, 2015 00:00 GMT

Root Cause: A protocol issue impacted IP services in multiple markets.
Fix Action: Adjustments were made to clear the errors.

Summary:
The IP NOC began investigating the root cause with Tier III Technical Support. 
It was reported that the issue was causing packet loss for customers. 
Operations Engineering teams were engaged, and Field Services were dispatched 
to a site in Chicago, IL to assist with investigations. Troubleshooting 
identified a protocol issue, and Operations Engineering worked with Tier III 
Technical Support to perform adjustments on the links. It was confirmed that 
the errors cleared. The traffic load was also lowered on cards in Chicago to 
alleviate any further issues. Should any additional impact be experienced, 
please contact the Level 3 Technical Service Center.

What the hell is a "protocol issue"?

I'm not an idiot, you can tell me specifically what happened...




-
Mike Hammett
Intelligent Computing Solutions
http://www.ics-il.com

- Original Message -

From: "Ryan Gelobter" 
To: "Mel Beckman" 
Cc: "" 
Sent: Thursday, August 27, 2015 3:14:59 PM
Subject: Re: Level(3) ex-twtelecom midwest packet loss (4323)

If you have access to the Level3 portal you should see ticket #9639047
under Network Events now.

Event Summary:IP Network Event ~ Multiple Markets

08/27/2015 8:05 PM GMT

Level 3 Tier III and Operations Engineering teams have identified Internet
Protocols dropping, affecting customer services. Restoration efforts are
in progress, but an estimated time of restoral is not available at this
time.

08/27/2015 6:36 PM GMT

IP and Transport Tier III, Operations Engineering and Field Services
continue collaboratively working the issue.


08/27/2015 4:59 PM GMT

Operations Engineering is engaged and Field Services is on site in Chicago,
IL investigating the issue.


08/27/2015 4:38 PM GMT

The engineers are currently migrating traffic in efforts of restoring
services while troubleshooting continues. Field Services is being
dispatched to a Chicago, IL site to assist.

08/27/2015 4:21 PM GMT

IP services are affected across multiple markets and the root cause is
currently under investigation. The IP NOC and IP and Transport Tier III are
actively troubleshooting and working to isolate the cause. The engineers
have detected peering issues which are resulting in packet loss for
customers. Please be advised that updates will be provided at minimum of
hourly unless otherwise noted.

Re: Dial Up Solutions

[Jason Canady]

I'm going to go with Justin's suggestion and go with a wholesale 
provider such as DialupUSA.  It's not worth paying for the lines and 
keeping a T1 or better for just a few users.  DialupUSA use to charge 
around $5/user.  They also had hourly and per port options. Looks like 
you can port existing numbers to them now.  I used them 9-10 years ago 
and they were great to work with!  IKANO bought them out since then, but 
they still operate under DialupUSA.net.  They have DSL and T1 options too.


- Jason

On 10/17/15 10:23 AM, Clayton Zekelman wrote:


3Com TC here.  9 users online at the moment.  Surprises me that it's 
that high.  Last reboot on the HiperARC was 399 days ago. I almost 
forgot how to log on to the damned thing.


At one time we had over 3000 DS0s worth of dialup capacity.

At 09:37 AM 17/10/2015, frnk...@iname.com wrote:
We're still using USR Robotics/3com TotalControls and were able to 
get some spare parts from our statewide telecom partner when they 
shut down their stuff.  Most common problem we see now are fan 
failures, but we just cannabilize existing the fans out of a fan 
tray.  The volume of calls are so low that there are no hours that no 
one is dialed in, and at most we see two people connected at one time.


Frank

-Original Message-
From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Will Duquette
Sent: Friday, October 16, 2015 2:29 PM
To: nanog@nanog.org
Subject: Dial Up Solutions

Does anyone have any suggestions on equipment for our ISP that is still
supporting dial up customers?

At the moment we are running 3Com Total Control 1000's but are 
running out
of spare parts as we have failures.  Given that this gear is so old 
trying

to source spare parts is proving to be difficult.

We do have access to an Cisco AS5200 but are looking for maybe a SIP 
based

solution that could possibly run on our VM farm?  Has anyone heard of
anything like that or does it even exist?

What kind of gear are you running if you still are supporting dial up
customers?

Thanks in advance

--
Will Duquette
GWI
Network Systems Engineer
www.gwi.net

Re: Shared cabinet "security"

[Jason Canady]

Mike,

Are you leasing a full cabinet and sub-leasing out portions of it? Not 
sure how you can define what other customers do, unless they're your 
customers.  Split cabinets are ideal, as you the sections are 
compartmentalized.


--

Jason Canady
Unlimited Net, LLC
Responsive, Reliable, Secure

www.unlimitednet.us
ja...@unlimitednet.us
twitter: @unlimitednet

On 2/13/16 11:25 AM, Mike Hammett wrote:

Right, but that doesn't limit one's ability (intentional or not) to pull out the wrong 
power cord or smack someone's loosely ran cables, etc. We're sorting out some standards 
now and I think it'll largely involve color coding, wire looms, horizontal cable 
management and a "cabinet practices" document defining standards for use in the 
cabinet. This is meant to protect customers from themselves and each other.

IE: Someone is removing a power cable and the pull the wrong one out of the 
PDU. Maybe they pull the right one out of the PDU, but it's wrapped around 
someone else's power cable and theirs gets pulled out along the way. Stuff like 
that.




-
Mike Hammett
Intelligent Computing Solutions
http://www.ics-il.com

Midwest-IX
http://www.midwest-ix.com

- Original Message -

From: "Greg Sowell" 
To: "Mike Hammett" 
Cc: "NANOG list" 
Sent: Saturday, February 13, 2016 10:16:17 AM
Subject: Re: Shared cabinet "security"


Mike,
I've seen people use shelves to segregate cabinets. I've seen some that screw 
from both sides and eat very little space.
Greg
On Feb 13, 2016 8:07 AM, "Mike Hammett" < na...@ics-il.net > wrote:


Getting a cabinet in someone else's datacenter (Equinix, Coresite, Telx, etc.) 
and having sub-tenants. Most networks aren't going to need more than a handful 
of U in a datacenter, but the more significant the datacenter, the less likely 
they are to provide partial cabinets... which makes no sense. Sure, some 
networks need large chassis routers chewing up 10U - 20U, but there are far 
more networks that need routers that take up 1U, 2U, something like that. For 
many networks, the sheer cost of the space in the datacenter doubles their 
overall cost per megabit.




-
Mike Hammett
Intelligent Computing Solutions
http://www.ics-il.com

Midwest-IX
http://www.midwest-ix.com

- Original Message -

From: "Bevan Slattery" < be...@slattery.net.au >
To: "Mike Hammett" < na...@ics-il.net >
Cc: "North American Network Operators' Group" < nanog@nanog.org >
Sent: Saturday, February 13, 2016 2:36:34 AM
Subject: Re: Shared cabinet "security"


Sorry. I'm not sure I get from which angle you are coming at this from. Happy 
to clarify for you and anyone interested if you can help me out here.


Cheers

[b]

On 13 Feb 2016, at 12:58 PM, Mike Hammett < na...@ics-il.net > wrote:





There are more options when you're not just using someone else's datacenter.




-
Mike Hammett
Intelligent Computing Solutions
http://www.ics-il.com

Midwest-IX
http://www.midwest-ix.com

- Original Message -

From: "Bevan Slattery" < be...@slattery.net.au >
To: "Mike Hammett" < na...@ics-il.net >
Cc: "North American Network Operators' Group" < nanog@nanog.org >
Sent: Friday, February 12, 2016 4:44:34 PM
Subject: Re: Shared cabinet "security"

In a past life we worked with our supplier to create physically separate 
sub-enclosures.1/2 and 1/3. Able to build in a separate and secure cable path 
for interconnects to the meet-me-room and connection to power supplies.

Can be done and I think there are now rack suppliers that do this as standard. 
Been out of DC space for a few years now.

[b]


On 13 Feb 2016, at 6:58 AM, Mike Hammett < na...@ics-il.net > wrote:


That moment when you hit send and remember a couple things…

Of course labeling of the cables.

Maybe colored wire loom for fiber and DACs in the vertical spaces to go along 
with the previously mentioned color scheme?




-
Mike Hammett
Intelligent Computing Solutions
http://www.ics-il.com

Midwest-IX
http://www.midwest-ix.com

- Original Message -

From: "Mike Hammett" < na...@ics-il.net >
To: "North American Network Operators' Group" < nanog@nanog.org >
Sent: Friday, February 12, 2016 2:53:17 PM
Subject: Re: Shared cabinet "security"


I am finding a bunch of covers for the front. I do wish they stuck out more 
than an inch (like two).
http://www.middleatlantic.com/~/media/middleatlantic/documents/techdocs/s_sf%20series%20security%20covers_96-035/96_035s_sf.ashx

It looks like these guys stick out 1.5”. That may be workable… 
http://www.lowellmfg.com/tinymce/jscripts/tiny_mce/plugins/filemanager/files/1717-SSCV.pdf

I guess those covers are really only useful for servers. That really woul

Re: mrtg alternative

[Jason Canady]

A friend was just showing me grafana this morning. I use rtg for a lot of 
bandwidth data / graphs, but I also have observium for a lot of extra stuff. 

Kicked cacti to the curb a long time ago.  rtg is really flexible, but the 
graphing isn't pretty. 

Sent from my iPhone

> On Feb 27, 2016, at 20:42, B  wrote:
> 
> Welcome to the future.
> Graphite/grafana.
> 
>> On Fri, Feb 26, 2016 at 06:30:02PM -0500, Shawn L wrote:
>> 
>> We use observium.  It has most of what you're looking for.   Used to use 
>> cacti but switched a couple of months ago
>> 
>> 
>> -Original Message-
>> From: "Baldur Norddahl" 
>> Sent: Friday, February 26, 2016 6:18pm
>> To: "nanog@nanog.org" 
>> Subject: mrtg alternative
>> 
>> 
>> 
>> Hi
>> 
>> I am currently using MRTG and RRD to make traffic graphs. I am searching
>> for more modern alternatives that allows the user to dynamically zoom and
>> scroll the timeline.
>> 
>> Bonus points if the user can customize the graphs directly in the
>> webbrowse. For example he might be able to add or remove individual peers
>> from the graph by simply clicking a checkbox.
>> 
>> What is the 2016 tool for this?
>> 
>> Regards,
>> 
>> Baldur

Re: Cogent Communications

[Jason Canady]

As a Cogent customer, I'm sure I can get you in touch with someone. Have 
you tried emailing supp...@cogentco.com ?  If not, email me off list and 
I'll find someone you can speak to.


- Jason

On 3/24/16 4:20 AM, Brandon Vincent wrote:

Does anyone have a NOC/SOC contact for Cogent? I found a improperly
secured router on the Internet and I'd like to report it.

Thank you,
Brandon Vincent

Re: BGP Multihoming 2 providers full or partial?

[Jason Canady]

If your traffic is small, you could setup a VyOS box.  You can still get 
redundancy by having two switches, each one connected to an upstream 
provider receiving a default route.  Then hookup your VyOS router to 
each switch and receive full routes to that.  You will need a /29 subnet 
from your providers to pull this off.  If your VyOS box goes down for 
whatever reason, you will failover to using one or the other switch.  
Announce your prefixes using the BGP session on each switch so that your 
inbound traffic doesn't hit the VyOS box.


--

Jason Canady
Unlimited Net, LLC
Responsive, Reliable, Secure

www.unlimitednet.us
ja...@unlimitednet.us
twitter: @unlimitednet

On 5/29/15 4:36 AM, Maqbool Hashim wrote:

Hi,


We are an enterprise that are eBGP multihoming to two ISPs. We wish to load 
balance in inbound and outbound traffic thereby using our capacity as 
efficiently as possible. My current feeling is that it would be crazy for us to 
take a full Internet routing table from either ISP. I have read this document 
from NANOG presentations:


https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0CCoQFjAA&url=https%3A%2F%2Fwww.nanog.org%2Fmeetings%2Fnanog41%2Fpresentations%2FBGPMultihoming.pdf&ei=cyRnVb--FeWY7gbq4oHoAQ&usg=AFQjCNFsMx3NZ0Vn4bJ5zJpzFz3senbaqg&bvm=bv.93990622,d.ZGU


The above document reenforces my opinion that we do not need full routing tables. However 
I was seeking some clarity as there are other documents which suggest taking a full 
routing table would be optimal. I "guess" it depends on our criteria and 
requirements for load balancing:


- Just care about roughly balancing link utilisation

- Be nice to make some cost savings


We have PI space and two Internet routers one for each ISP. Either of our links is 
sufficient to carry all our traffic, but we want to try and balance utilisation to remain 
within our commits if possible. I am thinking a "rough" approach for us would 
be:


- Take partial (customer) routes from both providers

- Take defaults from both and pref one


Maybe we can refine the above a bit more, any suggestions would be most welcome!


Many Thanks

Re: Setting Up a Looking Glass

[Jason Canady]

I totally agree, it would be awesome if it had routing table lookups / 
BGP queries.  We also have a LG running the original system, 
https://github.com/telephone/LookingGlass.  It would probably be pretty 
simple to add in BGP options.


There's a nice system called bgplg that is part of OpenBSD.  A quick 
Internet search will bring up many providers that utilize it so that you 
can check it out.


--

Jason Canady
Unlimited Net, LLC
Responsive, Reliable, Secure

On 6/13/15 12:53 PM, Shane Ronan wrote:

This would be even more AWESOME if you added routing table lookup.


On 6/13/15 12:38 PM, Jim Popovitch wrote:

On Sat, Jun 13, 2015 at 12:29 PM, Mike Hammett  wrote:
What's out there for setting up your own looking glass? I saw lots 
of lists of dead projects or projects that hadn't received any love 
in years. Being as most the people I work with don't run Cisco, 
Juniper, etc. for routers, likely having those capabilities with the 
LG would be nice.



Here's a relatively new and fresh perspective on it:

https://github.com/ramnode/LookingGlass

You can see it in action here:
http://lg.nyc.ramnode.com/

-Jim P.

Removal of Stale Level 3 IRR Object

[Jason Canady]

Would someone from Level 3 please contact me off-list regarding removal 
of stale IRR removal?  Our ASN was previously used by another 
organization and they had an IRR listed with Level 3.


AS11990

Thank you!

--

Jason Canady
Unlimited Net, LLC
Responsive, Reliable, Secure

www.unlimitednet.us
ja...@unlimitednet.us
twitter: @unlimitednet

Re: bgp traceroute tool?

[Jason Canady]

I agree, ASN lookup in mtr would be awesome!  I'll have to look into that 
sometime. 

Jason


On Nov 30, 2013, at 19:58, Jason Lixfeld  wrote:

> It would be slick if someone could patch mtr to do this too.
> 
> Sent from my iPhone
> 
>> On Nov 30, 2013, at 7:19 PM, Rene Wilhelm  wrote:
>> 
>> 
>>> On 11/30/13 1:18 AM, Lee Clark wrote:
>>> The traceroute variant  included with CentOS 6.4 & Mint 13 has an -A
>>> flag which does ASN lookups. ntraceroute on FreeBSD supports it as
>>> well. I believe the Linux port is traceroute-nanog.
>>> 
>>> Lee
>> 
>> traceroute -A  consults the internet routing registry which is know
>> to beincomplete and at times incorrect when it comes to IP to
>> BGP origin AS mapping. For this reason we developed riswhois.ripe.net,
>> a whois style interface to the BGP data collected by RIPE NCC's Routing
>> information service (http://www.ripe.net/data-tools/stats/ris)
>> 
>> Reporting in the same format as the IRR, riswhois is plugin
>> compatible with whois.radb.net. If your linux traceroutederives
>> from http://traceroute.sourceforge.net/ all it takes to switch to
>> using true BGP info in traceroute is setting the environment variable
>> RA_SERVER to "riswhois.ripe.net"
>> 
>> 
>> -- Rene
>> 
>> P.S. the LFT tool metioned earlier in this thread can also use RISwhois
>> to lookup ASNs; just pass it the -r option on the command line.
>> 
>> 
>>> 
>>> 
>>> [user@box ~]# traceroute -V Modern traceroute for  Linux, version
>>> 2.0.14, Nov 11 2010 Copyright (c) 2008  Dmitry  Butskoy,   License:
>>> GPL v2 or any later
>>> 
>>> [user@box ~]# traceroute -A www.google.ca traceroute  to www.google.ca
>>> (74.125.226.127), 30 hops max, 60 byte packets   6  72.14.197.33
>>> (72.14.197.33) [AS15169]  73.927 ms  69.254 ms69.305 ms 7
>>> 209.85.254.130 (209.85.254.130) [AS15169]  69.436 ms  209.85.254.122
>>> (209.85.254.122) [AS15169]  79.554 ms  64.269 ms 872.14.237.130
>>> (72.14.237.130) [AS15169]  64.979 ms  65.975 ms  209.85.254.238
>>> (209.85.254.238) [AS15169]  66.700 ms 9216.239.46.161
>>> (216.239.46.161) [AS15169]  71.293 ms  72.251 ms73.521 ms 10
>>> 209.85.250.207 (209.85.250.207) [AS15169]  74.454  ms  74.920 ms
>>> 75.889 ms 11  yyz08s13-in-f31.1e100.net  (74.125.226.127) [AS15169]
>>> 76.628 ms  77.105 ms  70.928 ms
>>> 
>>> 
>>> -Original Message- From: John Conner
>>> [mailto:bs7...@gmail.com] Sent: Friday, November 29, 2013 5:04 PM To:
>>> nanog@nanog.org Subject: bgp traceroute tool?
>>> 
>>> Hi there, is there any tools available under linux which can do bgp
>>> traceroute? (print bgp AS numbers for each traceroute hop ) , i
>>> googled and found nothing.
>>> 
>>> thanks
>>> 
>>> John
> 

Re: Cogent & Level 3 routing issue?

[Jason Canady]

Unfortunately Cogent has a lot of peering issues.  We use them in our 
network blend and we have been having lots of problems with traffic 
outbound to Comcast.  It looks like from South Bend, Indiana on Cogent 
to Chicago / Level 3 we are getting a very tiny amount of packet loss 
and a higher than 'normal' latency of 35ms+.


Where are you connected to Cogent at?  And what destination are you 
going to on Level 3?


Best Regards,

--

Jason Canady
Unlimited Net, LLC
Responsive, Reliable, Secure

www.unlimitednet.us
ja...@unlimitednet.us
twitter: @unlimitednet

On 12/7/13 3:14 PM, Matthew Crocker wrote:

Anyone seeing issues between Cogent & Level3 in NYC?

I have Sprint & Cogent for bandwidth.   Everything has been humming along for a 
couple years just fine.   Yesterday around 8:00AM my BGP session with Cogent 
flapped.  Now, when my Cogent BGP is up I get 100% packet loss in level3 land.  
When Cogent BGP is down (i.e. I’m running solely on Sprint)  Everything is fine.

I have an open ticket with Cogent.  They say they have a ‘capacity issue’ with 
level3 that has been escalated to executive levels.

With Sprint & Cogent BGP UP
  I see traceroutes showing traffic leaving me on Sprint but returning on 
Cogent (and failing at level3).  I’m guessing it is the level3/cogent border

With Sprint UP & Cogent Down
  I see trace routes showing traffic on to/from on Sprint just fine.


Anyone else having issues?

-Matt

--
Matthew S. Crocker
President
Crocker Communications, Inc.
PO BOX 710
Greenfield, MA 01302-0710

E: matt...@crocker.com
P: (413) 746-2760
F: (413) 746-3704
W: http://www.crocker.com

McAfee SiteAdvisor Removal

[Jason Canady]

Our web site has been incorrectly listed on McAfee's SiteAdvisor service 
as "SPAM URLs".  We offer dedicated servers to clients and I suspect 
that one of them was in the same /24 block of IPs.  I have tried 
numerous times to get removed and have been unsuccessful.


Does anyone have a contact at SiteAdvisor or can someone contact me 
off-list to get this taken care of?   Thanks in advance!


Best Regards,

--

Jason Canady
Unlimited Net, LLC
Responsive, Reliable, Secure

www.unlimitednet.us
ja...@unlimitednet.us
twitter: @unlimitednet