CGNAT - Seeking Real World Experience
I'm crunching the numbers on the cost effectiveness of implementing CGN vs IPv4 auctions. The determining factor is how many ephemeral ports are reserved for each customer. This is for a residential broadband environment. Is anybody doing deterministic NAT/PAT (i.e. each customer gets X ports - no more, no less)? My thinking is that X=8192 would cover even the power users. However, with only 8 customers per public IPv4 address, CGN is not worth the trouble. With X=8192, our money and time would better be spent acquiring additional IPv4 space. Are people successfully using a smaller deterministic port allocation? What's your X? If I can't make the numbers work for deterministic NAT, I might be able to live with dynamic port assignments. Specifically, I'm referring to what vendor J calls "Port Block Allocation". I was thinking 1024 ports per block, with up to 8 blocks per customer (and a bunch of log correlation to determine who was using which ip:port tuple at a given datetime). I *can* make the math work out in favor of CGN if the average customer uses <= 3072 ports (3 blocks). But is that going to be enough? I'd love to hear other people's experiences. Thanks! -Adam
Verizon to AT&T Outage from East Coast?
I've seen sporadic connectivity from Wash DC to SanFran CA today. At 1700 ET the connection dropped entirely. Anybody aware of network issues between VZ and AT&T? -- Failed Path -- 1<1 ms<1 ms<1 ms 192.168.1.1 2 7 ms 6 ms 7 ms L100.WASHDC-VFTTP-125.verizon-gni.net[173.66.17 0.1] 312 ms13 ms 9 ms G0-8-0-3.WASHDC-LCR-22.verizon-gni.net[130.81.2 16.72] 4 7 ms 7 ms 7 ms ae5-0.RES-BB-RTR1.verizon-gni.net[130.81.209.22 2] 540 ms11 ms11 ms 0.xe-5-1-0.BR1.IAD8.ALTER.NET [152.63.37.69] 6 *** Request timed out. 7 *** Request timed out. 8 *** Request timed out. 9 *** Request timed out. -- Working Path -- 1 1 ms<1 ms 1 ms 192.168.1.1 2 8 ms 6 ms 6 ms L100.WASHDC-VFTTP-125.verizon-gni.net[173.66.17 0.1] 322 ms 9 ms10 ms G0-8-0-3.WASHDC-LCR-22.verizon-gni.net[130.81.2 16.72] 4 8 ms 6 ms 7 ms ae5-0.RES-BB-RTR1.verizon-gni.net[130.81.209.22 2] 512 ms11 ms11 ms 0.xe-5-1-0.BR1.IAD8.ALTER.NET [152.63.37.69] 610 ms 9 ms 9 ms 192.205.36.141 787 ms88 ms86 ms cr2.wswdc.ip.att.net [12.122.81.250] 885 ms87 ms86 ms cr1.cgcil.ip.att.net [12.122.18.21] 989 ms92 ms89 ms cr1.sffca.ip.att.net [12.122.4.121] 1087 ms89 ms83 ms cr83.sffca.ip.att.net [12.123.15.110] 1184 ms84 ms84 ms 12.122.137.161 1290 ms88 ms90 ms 12.127.32.62 Thanks, Adam
Re: Industry practice for BGP costs - one time or fixed/monthly?
Edward's response nailed this one on the head. It has to do with the additional support/hardware required to support a BGP session. Granted, once a BGP session is established it rarely requires any tweaking, but I've spent hours troubleshooting a downed BGP session because the client's IPS signature update decided TCP/179 was malicious. You also have to implement additional filters to protect yourself from what your client can advertise. I'm lucky enough to work for a major ISP with pretty sophisticated filters built off the public route registry, but not all ISPs have this functionality. Adam On Fri, May 25, 2012 at 12:18 PM, Edward J. Dore < edward.d...@freethought-internet.co.uk> wrote: > The only thing that I can really think of is that the BGP sessions do take > up extra CPU time and memory on the routing engine, so there is an > additional cost to the provider in terms of needing more routers and/or > bigger routers if they have lots of customers speaking BGP to them that > they may not have factored in to their standard pricing. > > I guess there is also some extra cost in terms of NOC staff and systems to > monitor the sessions as well as providing any troubleshooting etc. that > they wouldn't have to do with "standard" customers that are statically > routed. > > Edward Dore > Freethought Internet > > - Original Message - > From: "Anurag Bhatia" > To: "NANOG Mailing List" > Sent: Friday, 25 May, 2012 5:01:11 PM > Subject: Industry practice for BGP costs - one time or fixed/monthly? > > Hello everyone > > > I have been aggressively looking for deals in servers in Europe for > anycasting. One thing which surprises me is the "setup costs" for BGP. Few > providers quoted additional $50-100 which looks OK but a few of them quoted > as high as $150 *extra every month* just for having BGP (no full routing > table, but just default route pointing). Is there's any technical logic > behind such heavy costs? I mean at the end of day we are all talking at > layer 3 and thus it does not involves any hard connection/physical work. > What other members pay for BGP setup costs? > > > > Thanks! > > -- > > Anurag Bhatia > anuragbhatia.com > or simply - http://[2001:470:26:78f::5] if you are on IPv6 connected > network! > > Linkedin <http://in.linkedin.com/in/anuragbhatia21> | > Twitter<https://twitter.com/anurag_bhatia>| > Google+ <https://plus.google.com/118280168625121532854> > >
Re[2]: Thank you, Comcast.
I'd expect the Colo's to start "locking this down" about the same time I'd expect ISP's to start implementing BCP38 in earnest. Adam -- Original Message -- From: "Dovid Bender" To: "Damian Menscher" Cc: "Mody, Nirmal" ; "NANOG list" Sent: 2/26/2016 3:43:34 PM Subject: Re: Thank you, Comcast. Lawsuits? There is no reason the dedicated server I have with a 100meg pipe for $65.00 per month is able to spoof IP's. The colo's should be doing a better job to lock this down. Regards, Dovid -Original Message- From: Damian Menscher Date: Fri, 26 Feb 2016 11:47:43 To: Dovid B Cc: Jared Mauch; Jason Livingood; Mody, Nirmal; NANOG list Subject: Re: Thank you, Comcast. "We all know..." followed by a false statement is amusing. A significant portion of spoofing originates from North America. In a recent attack I'm reviewing, the top sources of spoofing were the southwestern US, the northwestern US, and east Asia (and almost none from Europe). If ISPs understood how to collect and review netflow we might get somewhere... why is this so hard, and how do we fix it? Damian On Fri, Feb 26, 2016 at 10:48 AM, Dovid Bender wrote: We all know what countries this traffic is coming from. While you can threaten the local ISP's the ones over seas where the traffic is coming from won't care. Regards, Dovid -Original Message- From: Damian Menscher via NANOG Sender: "NANOG" Date: Fri, 26 Feb 2016 08:02:52 To: Jared Mauch; Jason Livingood< jason_living...@cable.comcast.com>; Mody, Nirmal< nirmal_m...@cable.comcast.com> Reply-To: Damian Menscher Cc: NANOG list Subject: Re: Thank you, Comcast. On Fri, Feb 26, 2016 at 6:28 AM, Jared Mauch wrote: > As a community we need to determine if this background radiation and these > responses are proper. I think it's a good response since vendors can't do > uRPF at line rate and the major purchasers of BCM switches don't ask for it > and aren't doing it, so it's not optimized or does not exist. /sigh > I don't agree with the approach of going after individual reflectors (open*project) or blocking specific ports (Comcast's action here) as both are reactive, unlikely to be particularly effective (there are still millions of reflectors and plenty of open ports available), and don't solve the root problem (spoofed packets making it onto the public internet). What I'd much rather see Comcast do is use their netflow to trace the source of the spoofed packets (one of their peers or transit providers, no doubt) and strongly encourage (using their legal or PR team as needed) them to trace back and stop the spoofing. This benefits everyone in a much more direct and scalable way. Until some of the larger providers start doing that, amplification attacks and other spoofed-source attacks (DNS and synfloods) will continue to thrive. (I've contacted several ISPs about the spoofed traffic they send to us. The next major hurdle is that so many don't have netflow or other useful monitoring of their networks) Damian
AS33132 / Crown Castle pulling a Lumen
If anybody that can help with CCF continuing to announce a prefix four hours after the originating session was shutdown, please contact me unicast. A ticket has been opened with their NOC but progress is not forthcoming. Thanks, Adam
Incorrect GeoIP filtering of 185.83.72.0/22
Hello, We are reaching out to NANOG since the following issue is mostly observed in US-based service providers. We are advertising the prefix *185.83.72.0/22 <http://185.83.72.0/22>*, that seems to be blocked by various popular US-based services, thus our customers originating from this prefix have trouble reaching such services. Indicatively: Atlassian (AMAZON), Adobe (AKAMAI) . To the best of our understanding, the blocking is enforced due to US / EU sanctions against Iran. The prefix was purchased by us - Lamda Hellix SA ( *AS56910* based in *GREECE, EUROPE* ) - approximately 8 months ago. We followed the necessary process as instructed by RIPE for changing the ownership of the prefix. Therefore, we kindly ask those of you that use/maintain/operate GeoIP databases to update your records. Most importantly, we would be grateful if you could share with us ( n...@lamdahellix.com) which Geo databases are mainly used in the US for this purpose. Kind regards, Adam Pavlidis
Re: Strange connectivity issue Frontier EVPL
As it happens, I've just recently turned up a peering circuit with PSL in Houston, and their senior engineer is clue++ Naturally, he's on vacation this week, but [Aaron] ping me unicast if I might be able to assist/lend eyeballs/make an introduction of you guys next week. --Adam On 11/9/20, 8:05 AM, "NANOG on behalf of Tim Burke" wrote: I'm amazed you can get *anything* to work with Logix involved. Haven't heard of many issues with PSLightwave in Houston, however... they seem to be one of the only halfway decent options here. On 11/6/20 2:57 PM, aar...@gvtc.com wrote: > My coworker is having similar issues with PS Lightwave and Alpheus/Logix > from San Antonio to Houston whereas some things work and somethings don't > > -Aaron > >
Telus Technical Network Contact
Hello All! I am hoping to find a Technical network contact inside Telus; I have reached out to the listed address on their ARIN records a couple of times, and haven't heard back. If someone can point me in the right direction, it would be much appreciated. Thank you! Adam Burnworth (he/him) Terrestrial Network Engineer adam.burnwo...@spacex.com<mailto:adam.burnwo...@spacex.com> SpaceX Starlink
Re: Petition for Board of Trustees adoption consideration of ARIN-2020-2
Frankly, at this point, I'll support whatever lets me never have to hear about the subject again. -Adam (Yes, I know Outlook has filters, thank you. In fact, I think I'll go create one right now.) Adam Thompson Consultant, Infrastructure Services [1593169877849] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: ARIN-PPML on behalf of Jon Bachtold Sent: January 12, 2021 05:56 To: arin-p...@arin.net Subject: [arin-ppml] Petition for Board of Trustees adoption consideration of ARIN-2020-2 I continue to support Reinstatement ie, ARIN-2020-2, and I support this petition to move the policy consideration to the Board of Trustees. Jon Bachtold CIRBN ARIN-2020-2: Reinstatement of Organizations Removed from Waitlist by Implementation of ARIN-2019-16 ARIN-2020-2 reverted to Draft Policy on 6 January following a Last Call period. Per ARIN's Policy Development Process (PDP), all Recommended Draft Policies not sent to the ARIN Board of Trustees for adoption consideration within 60 days of Last Call completion will revert to Draft Policy status. Jon Bachtold Chief Technology Officer CIRBN, LLC Tel:309-820-7321 Email: jcb...@cirbn.org<mailto:jcb...@cirbn.org> Web:www.cirbn.org<https://www.cirbn.org/> [CIRBN]<http://www.cirbn.org/> 200 W Front St, Ste 500A Bloomington,IL 61701 [Facebook]<https://www.facebook.com/cirbn> [LinkedIn]<https://www.linkedin.com/company/3729509/> [CIRBN Twitter]<https://twitter.com/CIRBN_LLC>
MS Teams orig/term carrier?
Despite having worked for an ILEC in network ops for 10yrs, I can't figure out how to answer this without an SS7 sniffer (which I obviously no longer have, nor access to an SSP or SCP). What 3rd-party carrier(s), if any, does Microsoft use to originate & terminate MS Teams-to-PSTN voice calls in Canada? Pointers to rabbit holes welcome, if you think there’s an actual answer at the end ;-). Thanks, -Adam Adam Thompson Consultant, Infrastructure Services [[MERLIN LOGO]]<https://www.merlin.mb.ca/> 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/>
Re: Zayo or HE for IP transit
IMHO: * Zayo = worse coverage/connectivity, adequate connectivity to other Tier1/Tier2, massive fiber network * HE = less redundancy built into their network, but best-connected to leaves/edges of the internet (and still good connectivity to core) Others may have had different experiences, but that's what I see from where I sit. -Adam Adam Thompson Consultant, Infrastructure Services [1593169877849] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG on behalf of James Lumby Sent: April 19, 2021 16:30 To: nanog@nanog.org Subject: Zayo or HE for IP transit What is the current experience with Zayo or HE? I’m looking at possibly adding one of them into a mix of cogent and a mix from my datacenter. Would be using BGP full routes. Any experiences would be appreciated. Sincerely, James
IS-IS and IPv6 LLA next-hop - just Arista, or everyone?
Hey, just checking as I don’t have any Cisco or Extreme or Juniper gear running IS-IS to verify myself… On current Arista (7280SR2K) and older Brocade (MLXe) routers, the IPv6 next-hop address in IS-IS seems to always be the link-local address of the neighbour, instead of any manually-assigned address on the interface. I believe this is *not* the case with Cisco, in particular. Not sure about other vendors, I don’t have any handy that can run IS-IS. I don’t appear to have any knobs to control this behaviour, and haven’t found anything related in the docs. Can anyone confirm that Cisco or other vendors does/do not do this (prefer LLA over GUA)? Thanks, -Adam Adam Thompson Consultant, Infrastructure Services [[MERLIN LOGO]]<https://www.merlin.mb.ca/> 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/>
Re: IS-IS and IPv6 LLA next-hop - just Arista, or everyone?
Thank you, both! ...that kinda sucks, though. I don't see any rationale in RFC 5308 for why the HELLO packet may only contain the LLA - does anyone know/remember why? (I'm hoping that understanding the rationale will make this an easier pill to swallow.) Obviously this behaviour/requirement is net-new to the IPv6 TLVs, as there's no LLA-cognate in IPv4 (APIPA doesn't count). There is in OSI, I think, but I'm still too sane to read those docs. It makes sense that you would not want LLAs in LSPs, only GUAs, but does that imply that you must use either ULAs or GUAs in order to establish IPv6 routes in IS-IS, in an IPv6 environment? That makes about as much sense to me as forcing LLAs for next-hops. -Adam Adam Thompson Consultant, Infrastructure Services [1593169877849] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG on behalf of Saku Ytti Sent: May 4, 2021 01:44 To: Mark Tinka Cc: nanog list Subject: Re: IS-IS and IPv6 LLA next-hop - just Arista, or everyone? On Tue, 4 May 2021 at 07:24, Mark Tinka wrote: > Junos: >> 2c0f:feb0::1/128 *[IS-IS/18] 02:43:49, metric 5870 >to fe80::1205:caff:fe86:4ac3 via et-4/0/2.0 >to fe80::5287:89ff:fef3:25c3 via et-4/0/2.0 >to fe80::1205:caff:fe86:4b10 via et-5/0/2.0 > > to fe80::5287:89ff:fef3:2610 via et-5/0/2.0 > > IOS XE: > I2 2C0F:FEB0::1/128 [115/6410] > via FE80::1205:CAFF:FE86:4AC3, TenGigabitEthernet1/0/0 > via FE80::1205:CAFF:FE86:4B10, TenGigabitEthernet0/0/0 > via FE80::5287:89FF:FEF3:25C3, TenGigabitEthernet1/0/0 > via FE80::5287:89FF:FEF3:2610, TenGigabitEthernet0/0/0 > > IOS XR: > i L2 2c0f:feb0::1/128 > [115/5870] via fe80::1205:caff:fe86:4b10, 02:45:22, HundredGigE0/3/0/0 > (!) > [115/5810] via fe80::f60f:1bff:feb0:75c4, 02:45:22, HundredGigE0/2/0/1 > SROS: 2001:218:0:1000::1/128Remote ISIS 48d22h13m 18 fe80::42de:adff:fe98:87e4-"lag1" 25301 https://tools.ietf.org/html/rfc5308 For Hello PDUs, the "Interface Address" TLV MUST contain only the link-local IPv6 addresses assigned to the interface that is sending the Hello. For LSPs, the "Interface Address" TLVs MUST contain only the non-link-local IPv6 addresses assigned to the IS. These are hello derived: A:y...@a04.chcgil09.us.bb# show router isis adjacency r22.chcgil09.us.bb-re0 detail |match Neigh IPv6 Neighbor : fe80::42de:adff:fe98:87e4 IPv4 Neighbor : 129.250.3.205 Vendors do not have the option to use GUA while being RFC5308 compliant. -- ++ytti
Re: IS-IS and IPv6 LLA next-hop - just Arista, or everyone?
I don't believe APIPA and Link-Local are precisely equivalent, but I agree it's the closest thing IPv4 has. IS-IS/IPv4 would presumably use APIPA addresses if nothing else were assigned to the interface, based on my reading of the RFC. I'm unsure what the RFC authors think should happen in a HELLO packet when the interface has multiple IPv4 addresses, but none of that is my problem here. I don't like LLAs because they are - intrinsically - meaningless. In the context of my L3 core, I know that for any subnet, .1/::1 is such-and-such a router, .2/::2 is that one, .3/::3, is the other one, etc., etc. (Yes, I have a very small & topologically simple L3 core. Let's not talk about L2!) When I look at my IPv4 routing table, I know which next-hop is which just by looking at it, and I can spot anomalies very easily. When I look at my IPv6 routing table, the next-hops are all... well... gibberish, at least to me. My experience is that LLAs are not durable, so memorizing them is not IMHO a useful task. Figuring out an (IS-IS) IPv6 route currently involves a couple of extra steps to locate the LLA's interface route, find the MAC address of that LLA on that link, and then identify the router from its MAC address. Am I missing something obvious? Thanks! -Adam Adam Thompson Consultant, Infrastructure Services [1593169877849] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: Saku Ytti Sent: May 4, 2021 10:20 To: Adam Thompson Cc: Mark Tinka ; nanog Subject: Re: IS-IS and IPv6 LLA next-hop - just Arista, or everyone? On Tue, 4 May 2021 at 18:15, Adam Thompson wrote: Hey Adam, > I don't see any rationale in RFC 5308 for why the HELLO packet may only > contain the LLA - does anyone know/remember why? (I'm hoping that > understanding the rationale will make this an easier pill to swallow.) > Obviously this behaviour/requirement is net-new to the IPv6 TLVs, as there's > no LLA-cognate in IPv4 (APIPA doesn't count). There is in OSI, I think, but > I'm still too sane to read those docs. IPv4 link local is 169.254/16, you may have seen them in Windows. > It makes sense that you would not want LLAs in LSPs, only GUAs, but does that > imply that you must use either ULAs or GUAs in order to establish IPv6 routes > in IS-IS, in an IPv6 environment? That makes about as much sense to me as > forcing LLAs for next-hops. The list may benefit from the context you have, it is not obvious to me why you'd want anything but LLA. -- ++ytti
RE: IS-IS and IPv6 LLA next-hop - just Arista, or everyone?
LOLOLOL. “%VXLAN-4-IPV6_UNDERLAY_UNSUPPORTED: VXLAN encapsulation using IPv6 VTEP addresses is not supported on this platform” Guess it’s going to be a non-issue for me, at this time, since VxLAN was the main reason for this entire setup… Thanks for all the responses! -Adam Adam Thompson Consultant, Infrastructure Services [[MERLIN LOGO]]<https://www.merlin.mb.ca/> 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG On Behalf Of Adam Thompson Sent: Tuesday, May 4, 2021 10:29 AM To: Saku Ytti Cc: nanog Subject: Re: IS-IS and IPv6 LLA next-hop - just Arista, or everyone? I don't believe APIPA and Link-Local are precisely equivalent, but I agree it's the closest thing IPv4 has. IS-IS/IPv4 would presumably use APIPA addresses if nothing else were assigned to the interface, based on my reading of the RFC. I'm unsure what the RFC authors think should happen in a HELLO packet when the interface has multiple IPv4 addresses, but none of that is my problem here. I don't like LLAs because they are - intrinsically - meaningless. In the context of my L3 core, I know that for any subnet, .1/::1 is such-and-such a router, .2/::2 is that one, .3/::3, is the other one, etc., etc. (Yes, I have a very small & topologically simple L3 core. Let's not talk about L2!) When I look at my IPv4 routing table, I know which next-hop is which just by looking at it, and I can spot anomalies very easily. When I look at my IPv6 routing table, the next-hops are all... well... gibberish, at least to me. My experience is that LLAs are not durable, so memorizing them is not IMHO a useful task. Figuring out an (IS-IS) IPv6 route currently involves a couple of extra steps to locate the LLA's interface route, find the MAC address of that LLA on that link, and then identify the router from its MAC address. Am I missing something obvious? Thanks! -Adam Adam Thompson Consultant, Infrastructure Services [1593169877849] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: Saku Ytti mailto:s...@ytti.fi>> Sent: May 4, 2021 10:20 To: Adam Thompson mailto:athomp...@merlin.mb.ca>> Cc: Mark Tinka mailto:mark@tinka.africa>>; nanog mailto:nanog@nanog.org>> Subject: Re: IS-IS and IPv6 LLA next-hop - just Arista, or everyone? On Tue, 4 May 2021 at 18:15, Adam Thompson mailto:athomp...@merlin.mb.ca>> wrote: Hey Adam, > I don't see any rationale in RFC 5308 for why the HELLO packet may only > contain the LLA - does anyone know/remember why? (I'm hoping that > understanding the rationale will make this an easier pill to swallow.) > Obviously this behaviour/requirement is net-new to the IPv6 TLVs, as there's > no LLA-cognate in IPv4 (APIPA doesn't count). There is in OSI, I think, but > I'm still too sane to read those docs. IPv4 link local is 169.254/16, you may have seen them in Windows. > It makes sense that you would not want LLAs in LSPs, only GUAs, but does that > imply that you must use either ULAs or GUAs in order to establish IPv6 routes > in IS-IS, in an IPv6 environment? That makes about as much sense to me as > forcing LLAs for next-hops. The list may benefit from the context you have, it is not obvious to me why you'd want anything but LLA. -- ++ytti
RE: Juniper hardware recommendation
Hi, Javier! MX series: Full-featured – sings, dances, walks the cat, etc. But painful racking (as you noted). Very nice and comprehensive boxes otherwise. Interfaces are more expensive, but often modular and wider variety. EX/QFX series: Nice switches, OK L3 routers. Lots of limitations in MPLS and various other corner-case limitations. My personal opinion: * Skip the MX480 (and up), it’s just too expensive. Consider an EX9200 instead, which can do 90% of the same functions. (If you can afford an MX480 or MX960, by all means, get one!) * MX240 is reasonable, but dated. A pair of MX204s in HA would make more sense, to me. * Skip the MX 2k/10k series – they don’t support SFP+ interfaces! (“No 10G WDM for you!”) Also no 1G, you need a separate step-down switch for that. I don’t know what SP Juniper thinks they’re targeting with these. * 1U/2U EX/QFX are reasonable edge devices as long as you’ve verified they can do what you need. Not core-router class IMHO. * If you don’t already know that you want a PTX, then you don’t want a PTX. The product is fine, but niche, and has the same interface limitations as MX10k. * ACX: MEF-compliant mini-MX, basically. Edge device only, pairs well with an MX480 (IIRC). Top-end are exceptions: ACX5k/7k might work, depending on what you need it to do. Not normally deployed as a core router. My experience is that you never fill up an EX9208 or MX480 chassis, but the MX240 is too small. YMMV. MX480 line cards are stupid expensive compared to, well, everything else. I’m probably out-of-date on some (or much) of my knowledge, let’s see what everyone else here has to say! -Adam Adam Thompson Consultant, Infrastructure Services [[MERLIN LOGO]]<https://www.merlin.mb.ca/> 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG On Behalf Of Javier Gutierrez Guerra Sent: Friday, May 7, 2021 3:55 PM To: nanog@nanog.org Subject: Juniper hardware recommendation Hi, Just out of curiosity, what would you recommend using for a core router/switch from Juniper? MX208,480,10K Datasheets show them all as very nice and powerful devices (although they do use a lot of rack space and side to side airflow is painful) but I’m just wondering here what most people use and how good or bad of an experience you have with it 😊 Thanks, Javier Gutierrez Guerra Network Analyst CCNA R&S, JNCIA Westman Communications Group Phone: 204-717-2827 Email: guer...@westmancom.com<mailto:guer...@westmancom.com> [WCG_Corp_Logo_horiz_cFullcolorHR]<http://westmancom.com/personal/> [cisco-certified-network-associate-routing-and-switching-ccna-routing-and-switching]
RE: Juniper hardware recommendation
OK, enough people have pointed it out :-). Clearly I was wrong about the MX 2K family, I missed the SFP+ MIC completely. That is good to know. However, the MX 10k family still only shows as being compatible with two QSFP cards. And yes, you can get a QSFP-SFP+ breakout cable, but those don't let you use SFP+ CWDM/DWDM transceivers. -Adam Adam Thompson Consultant, Infrastructure Services MERLIN 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca www.merlin.mb.ca > -Original Message- > From: Bjørn Mork > Sent: Saturday, May 8, 2021 6:32 AM > To: Adam Thompson > Cc: Javier Gutierrez Guerra ; nanog@nanog.org > Subject: Re: Juniper hardware recommendation > > Adam Thompson writes: > > > > * Skip the MX 2k/10k series – they don’t support SFP+ interfaces! > > https://apps.juniper.net/hct/model/?component=MX2K-MPC6E > https://apps.juniper.net/hct/model/?component=MIC6-10G > > > Bjørn
RE: Juniper hardware recommendation
I did not know such a thing existed! Cool! Holy murdering your port density, though. Ouch$$$. Adam Thompson Consultant, Infrastructure Services MERLIN 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca www.merlin.mb.ca > -Original Message- > From: Nick Hilliard > Sent: Friday, May 14, 2021 9:40 AM > To: Adam Thompson > Cc: nanog@nanog.org > Subject: Re: Juniper hardware recommendation > > Adam Thompson wrote on 14/05/2021 14:30: > > However, the MX 10k family still only shows as being compatible with > > two QSFP cards. And yes, you can get a QSFP-SFP+ breakout cable, but > > those don't let you use SFP+ CWDM/DWDM transceivers. > you can also get QSA adapters to convert from a QSFP form factor port to a > SFP+ port. This should allow SFP+ WDM transceivers. > > Nick
RE: Juniper hardware recommendation
At least it isn’t Arista, where SVI egress counters are disabled by default, and once enabled count everything UNLESS the packet egresses via a LAG! Talk about being “impactful”, we’re having to buy new routers to insert behind them, just to count packets so we can bill accurately, and for that matter, have traffic graphs that work at all. :-( Adam Thompson Consultant, Infrastructure Services [[MERLIN LOGO]]<https://www.merlin.mb.ca/> 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG On Behalf Of Michael Fiumano Sent: Friday, May 14, 2021 12:06 PM To: nanog@nanog.org Subject: RE: Juniper hardware recommendation If accurate interface stats are important to you, MX’s don’t support accurate SNMP Interface Utilization, ie they don’t comply with RFC2665/3635, which seems like a fairly basic thing to do but they decided not to, and has been impactful to me in the past. So, any SNMP monitoring of an interface will always show less utilization than what is actually occurring, possibly leading to a false sense of security, or delay in augmentation. Would also affect usage based billing, if you do that. https://www.juniper.net/documentation/us/en/software/junos/network-mgmt/topics/topic-map/snmp-mibs-and-traps-supported-by-junos-os.html For M Series, T Series, and MX Series, the SNMP counters do not count the Ethernet header and frame check sequence (FCS). Therefore, the Ethernet header bytes and the FCS bytes are not included in the following four tables: ifInOctets ifOutOctets ifHCInOctets ifHCOutOctets Thanks, Michael Fiumano From: NANOG On Behalf Of Mark Tinka Sent: Monday, May 10, 2021 10:25 AM To: nanog@nanog.org<mailto:nanog@nanog.org> Subject: Re: Juniper hardware recommendation On 5/10/21 16:19, aar...@gvtc.com<mailto:aar...@gvtc.com> wrote: I prefer MX204 over the ACX5048. The ACX5048 can’t add L3 interface to an mpls layer 2 type of service. There are other limitations to the ACX5048 that cause me to want to possibly replace them with MX204’s. But in defense of the ACX5048, we have gotten some good mileage (a few years now) of good resi/busi bb over vrf’s and also carrier ethernet for businesses and lots of cell backhaul… so they are good for that. I’ve heard the ACX5448 was even better. Trio will always provide better features, but come with the price tag to boot. I’m looking at the MX240 for the SCB3E MPC10E hefty with 100 gig ports You might want to look at the MX10003, in that case, as well. We are deploying those for 100Gbps service (customer-facing). Works out cheaper than offering 100Gbps service on the MX240/480/960 for the same task. Mark.
Re: MPLS/MEF Switches and NIDs
Extreme has excellent MEF implementations. I've never used their MPLS implementations, but it's definitely there on, I think, all their products. I only have the X620 model in my network, which may or may not work for you. Beware using any EXOS-based product (anything that starts with "X") unless you're already familiar with EXOS! I cannot emphasise this enough! Extreme's other product lines come from Nortel/Avaya and Broadcom heritage, and also have good MEF implementations (and more-or-less-sane OSes). They have MPLS support, but again, no experience with it. I can't give much advice on pricing as I get both edu & gov discounts, but they are competitive with Arista and Cisco when we go to RFP. Also, Juniper's MX (and maybe PTX?) families support MEF if that's a hard requirement. I know some but not all EX switches have had both MEF and MPLS, too. Beware many EX models have pretty minimalist MPLS implementations (e.g. no VPLS). Agreed on their pricing, though, which is why I don't have any 🙂. But for 4x10G the MX104 is a very nice box - if you can afford it. Lastly, have you seen https://www.mef.net/certify/technology-registry/ ? -Adam Adam Thompson Consultant, Infrastructure Services [1593169877849] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG on behalf of Colton Conor Sent: May 26, 2021 11:39 To: NANOG Subject: MPLS/MEF Switches and NIDs For MPLS and MEF switches, I know Juniper, Cisco, and Nokia are commonly talked about on this list. However, I was wondering if anyone has evaluated other brands? We are not interested in looking at chinese based vendors, so ZTE and Huawei are not an option. Anyone else worth looking into? We have used Juniper's ACX line primarily, but there is a big gap in their product line. The ACX2200 has only two 10G ports. The next jump up from there is the ACX710 with 24 10G ports. They have nothing in between that has 4-12 10G ports. Not to mention, Juniper is very proud price wise. We are looking for cost efficient 10G NIDs with at least 4 10G ports on them and aggregation boxes with at least 12 10G ports on them with 25g/100G uplinks. Ciena seems to have multiple options available with Segment Routing, MPLS, and streaming telemetry support. I am probably most interested in what Ciena has to offer. Has anyone deployed the 3000 or 5000 product line of Ciena? How does it compare to Juniper? The Ciena 3924 is sub $1000 for example, and has 4 10G ports on it. Adva has quite a few options as well, but I don't think their routing stack is as strong as Ciena's. Tejas was an unknown player to me, but they seem to have a couple of options that fit the bill. Price wise, I have heard the run circles around everyone. RAD has some options, but their pricing looks much higher than Ciena. Accedian looked interesting, but it seems they don't make aggregation switches, only NIDs. ECI Telecom / Ribbon seems to have some options, but I have not talked to them. What does Nokia and Cisco have in this space, and price wise is it going to compare to these less known vendors?
Re: MPLS/MEF Switches and NIDs
EXOS is a perfectly good OS that bears absolutely no resemblance to anything else you've ever used in your career. If you start from scratch without training courses, you're looking at wasting 6 months (maybe more) just learning the OS well enough to figure out how to configure your desired deployment. Then you get to the usual month or so of fine-tuning required that every product needs. Given how many companies will pay for training nowadays, it's a relevant concern IMHO. Now that I'm used to EXOS, I like it. But I would never recommend an EXOS newbie start a project with a product that runs EXOS, without some jump-start training. It really is/was that painful. It's like giving a 100% Windows admin a UNIX box to get the new service running on, with no training. (Or vice-versa.) NOTE: Extreme's goal (supposedly targeting 2021) was to ship one common hardware platform that could run any of their 3 OSes. I don't know if they're achieved it, but generally speaking, for any EXOS box, there's two more products, one running the Nortel/Avaya OS and one IronWare (Foundry/Broadcom), both of which are fairly "normal". -Adam Adam Thompson Consultant, Infrastructure Services [1593169877849] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> ____ From: Colton Conor Sent: May 31, 2021 15:30 To: Adam Thompson Cc: NANOG Subject: Re: MPLS/MEF Switches and NIDs Adam. When you say "Beware using any EXOS-based product (anything that starts with "X") unless you're already familiar with EXOS!" Are you saying stay away from this line completely, or what do you mean by this statement. I have heard good things about Extreme for deploying service provider G.8032 and MPLS functions. Yes, I was aware of https://www.mef.net/certify/technology-registry/ and have gone through pretty much every vendor looking at their solutions. Extreme for example is not listed at all, so I guess they didn't want to pay those fees! There are quite a few Chinese vendors we can't use. On Mon, May 31, 2021 at 12:44 PM Adam Thompson mailto:athomp...@merlin.mb.ca>> wrote: Extreme has excellent MEF implementations. I've never used their MPLS implementations, but it's definitely there on, I think, all their products. I only have the X620 model in my network, which may or may not work for you. Beware using any EXOS-based product (anything that starts with "X") unless you're already familiar with EXOS! I cannot emphasise this enough! Extreme's other product lines come from Nortel/Avaya and Broadcom heritage, and also have good MEF implementations (and more-or-less-sane OSes). They have MPLS support, but again, no experience with it. I can't give much advice on pricing as I get both edu & gov discounts, but they are competitive with Arista and Cisco when we go to RFP. Also, Juniper's MX (and maybe PTX?) families support MEF if that's a hard requirement. I know some but not all EX switches have had both MEF and MPLS, too. Beware many EX models have pretty minimalist MPLS implementations (e.g. no VPLS). Agreed on their pricing, though, which is why I don't have any 🙂. But for 4x10G the MX104 is a very nice box - if you can afford it. Lastly, have you seen https://www.mef.net/certify/technology-registry/ ? -Adam Adam Thompson Consultant, Infrastructure Services [1593169877849] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG mailto:merlin.mb...@nanog.org>> on behalf of Colton Conor mailto:colton.co...@gmail.com>> Sent: May 26, 2021 11:39 To: NANOG mailto:nanog@nanog.org>> Subject: MPLS/MEF Switches and NIDs For MPLS and MEF switches, I know Juniper, Cisco, and Nokia are commonly talked about on this list. However, I was wondering if anyone has evaluated other brands? We are not interested in looking at chinese based vendors, so ZTE and Huawei are not an option. Anyone else worth looking into? We have used Juniper's ACX line primarily, but there is a big gap in their product line. The ACX2200 has only two 10G ports. The next jump up from there is the ACX710 with 24 10G ports. They have nothing in between that has 4-12 10G ports. Not to mention, Juniper is very proud price wise. We are looking for cost efficient 10G NIDs with at least 4 10G ports on them and aggregation boxes with at least 12 10G ports on them with 25g/100G uplinks. Ciena seems to have multiple options available with Segment Routing, MPLS, and streaming telemetry support. I am
Re: A survey on BGP MRAI timer values in practice
+1 to Saku's concerns - I simply ignored the survey because I wasn't sure what MRAI was, and I wasn't sure what my values would be. But I have time to be interested right now, so a-spelunking I go... The term "MRAI" does not appear anywhere in Arista's or Extreme's documentation. Nor does this timer interval appear in any BGP-related "show" output, on any of my platforms, that I can see. I've found out that "out-delay" (not "delay out") is synonymous with MRAI and seems widely used. I found "out-delay" in an Arista technote, and now I know how to override it on Aristas, and the default is zero (0). Unfortunately, "show" commands on EOS will only show you the current out-delay if it is non-zero, which makes reporting it a bit difficult. Extreme's MLXe platform doesn't appear to support an out-delay/MRAI knob at all, at least as far as I can tell. I know there are several other current and former MLXe operators here, maybe one of them will know? Based on my limited history with NANOG-L, I guess your initial email might have been seen by perhaps 20 people who immediately knew what you were talking about, and 2000 who didn't. (I don't actually know subscriber numbers, that's totally a WAG. And maybe more people have touched out-delay knobs than I think. Dunno.) This illustrates the gap between academia and industry - academics can research a narrow topic, and come at issues from a theoretic standpoint using unfamiliar terminology. As a practitioner, I get told to add carrier X as a peer by end-of-day Friday, and "just make it work". I wasn't even able to understand your initial question, because I haven't spent a semester understanding the intricacies of BGP propagation - I just know it usually works, knobs exist that I shouldn't fiddle with, and that's good enough for my job. If your work results in actionable recommendations such as "don't use BGP out-delay timers to mitigate XYZ in circumstance LMNO, do ABC instead", that's fantastic. Please keep us advised, and do post aggregated survey results here once you close the survey. I am specifically interested in the answer to "Have you ever had to adjust BGP out-delay with any of your peers, and why?" It would be great if we could derive that answer from the survey results, but anecdotal replies here would also be helpful. All you larger(-than-me) network operators out there: when would I need to use out-delay? Why? What does it accomplish? Good luck in reformulating your survey to get better engagement, -Adam Adam Thompson Consultant, Infrastructure Services [1593169877849] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG on behalf of Saku Ytti Sent: June 8, 2021 01:06 To: shahr...@cs.umass.edu Cc: nanog list ; Arun Venkataramani Subject: Re: A survey on BGP MRAI timer values in practice On Mon, 7 Jun 2021 at 19:32, wrote: > We often read that the Internet (i.e. BGP) has a long convergence delay. > But why is it so slow? And can we (researchers) do anything about it? Create business incentives to improve it. This is a non-technical problem, we've long had technical tools to make it fast, there just isn't incentive to make it fast. Customers are not asking operators for better convergence speeds. > Please help us out to find out by answering our short anonymous survey > (<10 minutes). Can you tell me what have you done so far? What are the default MRAI values for each AFI/SAFI for IOS, IOS-XR, Junos, SROS, VRP and EOS? Then people responding don't have to check what their NOS does, they can refer to your table and tell the default value, since this is what >99% will be using. Now your survey has built-in selection bias, people who answer it are people who know what it is and who are concerned about it and have changed it, this is not a representative group and you will start your work with very bad data. -- ++ytti
Re: A survey on BGP MRAI timer values in practice
My question at this point is, what slow global convergence? When I (or any of my downstreams) adjusts a prefix, I nearly always see global propagation in well under 60 seconds. Among networks where I can check, at least. I understand it could be technically possible to see near-instantaneous global convergence in more like <5sec, but... on a global scale, neither I nor my customers care about the difference between 5s and 60s. Do other people need <60s propagation? -Adam From: NANOG on behalf of Mark Tinka Sent: June 10, 2021 02:36 To: nanog@nanog.org Subject: Re: A survey on BGP MRAI timer values in practice On 6/10/21 08:26, Saku Ytti wrote: > I don't understand the question, but the way I read the question it > may be unanswerable even if I did understand it. As the reader would > self-define negligible and well acceptable and answer yes/no based on > the definition they used, which might be different to the definition > writer intended. It's possible we've become accustom to a slow, global BGP, due to a perception of fragility (and complexity), which favours stability over speed. I suppose the size of the current BGP and the nature of the FSM it lends itself to does some to account for those perceptions. At a per-ISP level, it is not impossible to speed up (i)BGP convergence. On a global scale, taking the least common denominator to allow for all manner of network we don't know about, allowing the ship a wide turn in BGP waters, at least on a perceptive level, seems like an unsigned social agreement amongst autonomous systems. Ultimately, I feel we aren't talking enough about this, and hopefully, this thread gets us to that point. Mark.
Re: 1G/10G BaseT switch recommendation
If you've already looked at Cisco, Juniper, Extreme, Juniper, and Arista, that's the big ones. Everything else is increasingly niche vendors. -Adam Adam Thompson Consultant, Infrastructure Services [1593169877849] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG on behalf of Drew Weaver Sent: July 22, 2021 13:46 To: 'nanog@nanog.org' Subject: 1G/10G BaseT switch recommendation Hello everyone, I’m looking for recommendations from the community on 48x10G RJ45/4-6 SFP28 (uplink ports) switches that people actually like working with. Features are VPC or non-vendor specific equivalent, L2/L3 BGP/OSPFv3, ACLs, functional CoPP and some sort of API to manage them. [the CLI would work, my lib can handle most Networking OS CLIs anyway] My problem point is coming from the RJ45 requirement, most vendors have one switch that they sell that is RJ45 at 10G or at the most one in each line (enterprise/datacenter) and they seem to be almost an afterthought. [probably because SFP28 is better in every way if you are already using fiber at the endpoint] sadly, we are not. I just want to make sure I am not excluding any vendors from my research. I appreciate any suggestions or recommendations. Can even keep it off-list if you want. Thanks, -Drew
RE: 1G/10G BaseT switch recommendation
True. I forget carrier space often, these days. Adam Thompson Consultant, Infrastructure Services [[MERLIN LOGO]]<https://www.merlin.mb.ca/> 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: Saku Ytti Sent: Thursday, July 22, 2021 2:34 PM To: Adam Thompson Cc: Drew Weaver ; nanog@nanog.org Subject: Re: 1G/10G BaseT switch recommendation On Thu, 22 Jul 2021 at 22:32, Adam Thompson mailto:athomp...@merlin.mb.ca>> wrote If you've already looked at Cisco, Juniper, Extreme, Juniper, and Arista, that's the big ones. Everything else is increasingly niche vendors. Extreme is a mom and pop shop compared to Nokia and Huawei, and I guess quite selection of names. -- ++ytti
Re: 1G/10G BaseT switch recommendation
While acknowledging that some people love Rucks for legitimate reasons, our experience with them can be summed up as "never again". YMMV. -Adam Adam Thompson Consultant, Infrastructure Services [1593169877849] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG on behalf of Jörg Kost Sent: July 22, 2021 14:39 To: Drew Weaver Cc: nanog@nanog.org Subject: Re: 1G/10G BaseT switch recommendation Ruckus ICX 7650-48ZP has - 24x 1GB RJ45 - 24x 2.5/5/10G RJ45 - stacking - uplink 100G | 40G | 10G uplink module - BGP, OSPF, ACL https://de.commscope.com/product-type/enterprise-networking/ethernet-switches/icx7650 On 22 Jul 2021, at 21:29, Adam Thompson wrote: > If you've already looked at Cisco, Juniper, Extreme, Juniper, and > Arista, that's the big ones. Everything else is increasingly niche > vendors. > -Adam > > > Adam Thompson
Re: Anycast but for egress
Without any sarcasm: to make it harder to block. If, say, Google, always crawled your site from 8.8.1.2 (random made-up example) then you would see a not-insignificant number of hosts and networks null-routing that IP. I have no idea why someone would do so, but I've seen it done many times. Mostly by people who don't understand how un-special they are on the internet. Also it would trigger IDS/IPS systems all over the place, having gobs and gobs of connections coming from a single IP. That's setting aside the technical issues involved; routing is often asymmetric, i.e. the return packet takes a different path than the inbound packet. So it would, as Owen implied, be nearly impossible to ensure the reply packets got back to the correct TCP stack. As an example, I'm multi-homed and use path-prepending, so if a packet claiming to be from 8.8.8.8 arrived on one of my commercial links, I would send the reply out the cheapest link, which in my case is a flat-rate R&E network (that has a path to Google), thus ensuring the reply does not get to the originating anycast node. When my clients make connections outbound to anycast addresses, the destination is more-or-less stable, and the replies come back to the client's unique IP, so anycast works in that direction. The guarantees are not present in the reverse direction. The logical extremity of this is that it would be nearly impossible for two anycast addresses to establish a TCP connection to each other. (In general. There will be lots of local cases where it does happen to work, by coincidence.) You'll find that even anycast nodes do not make connections outbound using their anycast address, pretty much for these reasons. -Adam Adam Thompson Consultant, Infrastructure Services [1593169877849] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG on behalf of Vimal Sent: July 27, 2021 12:54 To: nanog@nanog.org Subject: Anycast but for egress (Unsure if this is the right forum to ask this question, but here goes:) >From what I understand, IP Anycast can be used to steer traffic into a server >that's close to the client. I am curious if anyone here has/encountered a setup where they use anycast IP on their gateways... to have a predictable egress IP for their traffic, regardless of where they are located? For example, a search engine crawler could in principle have the same IP advertised all over the world, but it looks like they don't... I wonder why? -- Vimal
Re: "Tactical" /24 announcements
Yes, it is bad practice. Yes, it's polluting the route table. If the # of /24s involved is not ridiculously large (say, <64?) them I would go ahead, as long as IRR and/or RPKI are also updated. Obviously if everyone did it (i.e. advertising /24s exclusively) then our FIBs would collectively balloon to a grotesquely un-manageable size, at least on platforms that can't auto-aggregate that back down. Thankfully, everyone isn't doing it. I, too, would vastly prefer no-one did this, but I have two customers that demand it from time to time... and we've even done it for our own allocation sometimes, and there's no robust, never mind bullet-proof, technical argument why I can't do that for them (or for ourselves). OTOH robust arguments exist for why it's a good thing to do - sometimes, and temporarily. ¯\_(ツ)_/¯ -Adam Adam Thompson Consultant, Infrastructure Services [1593169877849] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG on behalf of Billy Croan Sent: August 9, 2021 10:47 To: nanog list Subject: "Tactical" /24 announcements How does the community feel about using /24 originations in BGP as a tactical advantage against potential bgp hijackers? All of our allocations are larger and those prefixes we announce for clients as well usually are. But we had a request recently to originate everything as distinct /24 prefixes, to reduce the effect of a potential bgp hijack. It seemed a little bit like a tragedy of the commons situation. Is this seen as route table pollution, or a necessary evil in today's world? How many routers out there today would be affected if everyone did this? Are there any big networks that drop or penalize announcements like this?
Re: PeerinDB refuses to register certain networks [was: Setting sensible max-prefix limits]
I have an example locally: BellMTS (ASNs 684, 7122, 4398), the local ILEC. To the best of my knowledge, they only peer with downstream customers (including myself) and their sole upstream, Bell Canada (AS577). Meanwhile that's a ~700k eyeball network (with some hosting, sure), roughly ~400Gbps upstream connectivity, and no public peering whatsoever. In fact, one might describe their peering model as "feudal", where they're subjugate to their corporate overlord (Bell Canada). It's unfortunate, I know there are some smart people working there, but either they don't understand the value of sub-1ms access to root nameservers (*cough* MBIX *cough*), or they're prevented from doing anything about it. [Disclaimer: I'm on the MBIX board. But I also used to work for MTS, and tried to setup the first peering relationship but got shot down for "marketing" reasons, something about "legitimizing the competition". Very monopolistic thinking, IMO.] Meanwhile, MTS still has a PeeringDB record, even though it documents quite nicely the fact that perhaps that record shouldn't exist, or at least doesn't need to. FWIW, their upstream, Bell Canada, is a very different story. And also mostly ~8msec away. -Adam Adam Thompson Consultant, Infrastructure Services [1593169877849] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG on behalf of Eric Kuhnke Sent: August 19, 2021 10:32 To: Ben Maddison ; nanog@nanog.org list Subject: Re: PeerinDB refuses to register certain networks [was: Setting sensible max-prefix limits] I agree with you in the utility of that, but sort of as a side topic... I wonder how many ASes are out there that have any significant volume of traffic/multi-site presences, but are exclusively 100% transit customers, do not have any PNIs at major carrier hotels, and are not members of any IX. What would be a good example of such an AS and how big of a network would it be? Undoubtedly there are some enterprise end user type customers set up like that, but I can't imagine they receive a very large volume of unsolicited peering requests. On Thu, Aug 19, 2021 at 6:32 AM Ben Maddison via NANOG mailto:nanog@nanog.org>> wrote: Hi Patrick, On 08/18, Patrick W. Gilmore wrote: > > Of course! Including headers to show authenticity. I was very amused by the > > explanation of the "chicken and egg" problem. Who's creating that? The > > networks > > who refuse to peer with non-peeringdb registered ASNs, or peeringdb who > > won't > > recognize ASNs that are not peering with anyone because nobody wants to peer > > with them because they are not registered in peeringdb because nobody wants > > to > > peer with them? You get the idea. > > First, most networks do not require a PDB record to peer. (Silly of > them, I know, but still true.) > > Second, you do not need to have a PDB record to get a link to an IXP. > Even membership in a free IXP is sufficient for an account in PDB, as > Grizz points out below. > > Third, if you have an agreement, even just an email, saying a network > will peer with you once you have a record, that may well suffice. Have > you asked any network to peer? Private peering (because you are not on > an IXP) is usually reserved for networks with more than a modicum of > traffic. If your network is large enough to qualify for private > peering, I have trouble believing you cannot get another network to > agree to peer so you can get a record. > > I guess you are right, the _Peering_DB does not register “certain” > networks. Those networks would be ones that do not peer. Which seems > pretty obvious to me - it is literally in the name. > A PDB record for an Internet-connected ASN, listing no IXPs or facilities, but with a note saying approximately "We only use transit, and don't peer" has some utility: it saves prospective peers from finding contacts to ask and sending emails, etc. I'd argue this is in scope for PDB. But perhaps there was additional context to the original decision that I'm missing? Cheers, Ben
Re: Newbie Questions: How-to monitor/control unauthorized uses of our IPs and DNS zones?
I just had a conversation with John Curran (of ARIN) about this, in fact...
You don't own IP addresses. But you also don't rent IP addresses, either.
IP addresses are not a thing, good, or object, not even an intangible good.
They are an address, or an index, if you will. (You might think of an IP
address as the index on a giant, internet-wide, shared array... that we call
"the routing table".)
Your annual fee purchases registration services, specifically, the service of
ARIN entering your IP addresses into their master copy of a database that other
people use. (And some ancillary services that ARIN provides to you.) That's
it.
The closest analogy I have are either phone numbers or street addresses. You
don't own either of those things, nor do you rent them. In the case of phone
numbers, the phone company isn't renting you the phone#, they're renting you
the POTS service that gives you the ability to make outgoing, and answer
incoming, calls. Your ILEC also typically adds your name and # into a phone
book, as part of the service. (Yeah, VoIP providers have mangled this analogy
beyond recognition.) They can (and have) changed your phone number at will.
At least ARIN doesn't do that.
In the case of a street address, you own the property. The address is just an
index to a giant, irregular 2D array called "the streets in your city". Again,
when you buy or rent the property, you aren't buying or renting the address
itself from anyone, much less the city. But there are all sorts of directories
("databases") you can register your business in so that people know who
occupies such-and-such a property, and marketing folks do this all the time
(even in 2021). When you pay those companies money, you aren't renting the
property from them, you're registering your property with them.
Here's the problematic part: there's absolutely nothing saying you have to
register your addreses with an RIR to get them into the global routing table.
You could probably find an ISP somewhere willing to overlook all the rules and
conventions and advertise new address space that just happens to overlap with
someone else's registered addresses, or maybe you found some that aren't
currently advertised. In fact, I'd say it's 100% possible to do so.
However, nearly everyone agrees to play by a common set of rules, in order that
the Internet, well... works. As expected. Almost 100% of the time, taken as a
whole. Those rules include requiring you to register with an RIR, to ensure
there are no overlaps, and law enforcement can find you if necessary.
Again, you aren't buying or renting IP addresses - you're paying an admission
fee of sorts, in order to play in the global routing table. The fact your RIR
assigned you a block of addresses is part of good internet governance, and is
not actually the commercial aspect of the transaction (even though we all think
of it that way anyway, including me).
Ultimately, almost everyone thinks of it the way you do, but it's technically
quite wrong. (My statements may not be correct in jurisdictions deriving from
systems other than English common law.)
Beyond this, this is a discussion for ARIN-DISCUSS not NANOG-L. Or perhaps in
your case, whatever discussion list APNIC runs, since ARIN rules don't apply in
Thailand. But I expect APNIC will tell you almost the same thing as I just did.
-Adam
P.S. If you feel this is B.S. and it shouldn't work this way, most of the RIRs
are always looking for participants in their policy process - I know ARIN is.
Well, I don't know what's up with AfriNIC, that unfortunately seems to be a
rolling dumpster fire, but I suppose they'll need new people to put the pieces
all back together, too.
Adam Thompson
Consultant, Infrastructure Services
[1593169877849]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>
From: NANOG on behalf of
Pirawat WATANAPONGSE via NANOG
Sent: August 19, 2021 13:32
To: nanog@nanog.org
Subject: Re: Newbie Questions: How-to monitor/control unauthorized uses of our
IPs and DNS zones?
Huh.
And I thought that I did lay down information (and questions) pretty clearly,
but as you correctly pointed out, I didn't.
So, here goes the second version:
Background Information Section (v2):
We are a Registrant and already registered a zone/domain with a Registry, we
are also a LIR and have been allocated an IP block straight from RIR.
[What I meant to say is that they all keep saying that we don’t “own” those
resources and we also have to pay the annual fee so, even though we are a
Registrant and a LIR, it’s still practically a form of rent anyway.]
Zayo BGP filter update contact
Hi, It was requested on July 7 that Zayo build our inbound prefix filter from our as-set object in RADB. As of today, six weeks or so later, after beating them up for updates, all we get back from support is “we have engaged our engineering team on this” Anybody around willing and able to assist getting this done and verified? If so, please drop me a note off list. Thanks! --Adam
sflow -> aggregated aspath visualization?
I’m looking for product recommendations: We’ve noticed that about 20% of our traffic here lately has decamped from the free (or, at least, flat-rate) connection to CANARIE (our R&E network) and its various connected content-delivery networks, and onto our commercial provider. While this is presumptively a legitimate shift, we’d like to better understand these changes when they occur, in a way that our executive can understand at a glance. We do have sFlow (et al.) going to an Arbor PeakFlow box for analysis, but it’s lacklustre at best at understanding changes like this. I want: * Top #n ASNs by traffic volume, per router/interface, stacked chart * Some way to visualize large jumps in that dataset, e.g. if Cloudflare ditched their CANARIE connection and now that traffic all goes commercial, I don’t know what sort of graphic would be useful, maybe a stacked polar chart so you could see when an AS jumped from one sector to another? Even stacked bar charts could be useful. If anyone knows of tools capable of generating easy-to-understand reports, dashboards, including historical “what changed this week”-type data, please let me know. For that matter, if you have a technique of collecting this data and using Excel to do the reporting, that would work too. (Yes, I could theoretically build this off of existing open source tools… eventually) Thanks, -Adam Adam Thompson Consultant, Infrastructure Services [[MERLIN LOGO]]<https://www.merlin.mb.ca/> 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/>
RE: WIKI documentation Software?
[Disclaimer: former Atlassian Reseller and Certified Confluence Administrator here.] Atlassian Confluence. It’s not cheap, and it certainly has its flaws, but it incorporates one feature that most (all?) other wikis don’t – hierarchy. You can organize information (pages) hierarchically like a directory structure. The key here is that some people think hierarchically, and some people don’t. For the hierarchical thinkers, a free-form wiki (i.e. most of them) is absolute hell to navigate, and you can still cross-link pages and use tags and categories, so the non-hierarchical thinkers are still just as much at home as with other products. Another plus, despite the cost, is you can host it on-site or in the cloud, depending on your needs. -Adam Adam Thompson Consultant, Infrastructure Services [[MERLIN LOGO]]<https://www.merlin.mb.ca/> 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG On Behalf Of Craig Sent: Saturday, March 14, 2020 7:08 AM To: nanog group Subject: WIKI documentation Software? Wanted to ask what WIKI software teams are using to save documentation to / how to's for staff, etc. pro's con's We have an older wiki bare-metal wiki server, that I want to get replaced before it kicks the bucket and was looking into various ones. thanks; CPV
RE: COVID-19 vs. peering wars
Every large ISP does this (or rather, doesn't) at every IX in Canada. Bell isn't unique by any stretch. It's not in their economic interest to peer at a local IX, because from their perspective, the IX takes away business (Managed L2 point-to-point circuits, at the very least) from them. Don't expect the dominant wireline ISP(s) in any region to join local IXes anytime soon, sadly, no matter how much it would benefit their customers. After all, the customer is always free to purchase service to the IX and join the IX, right??? *grumble* In my local case, if BellMTS joined MBIX, un-cached DNS resolution times could potentially drop by 15msec. That's HUGE. But the end-user experience is not their primary goal. Their primary goal is profit, as always. -Adam Thompson Founding member, MBIX (once upon a time) Adam Thompson Consultant, Infrastructure Services MERLIN 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca www.merlin.mb.ca > -Original Message- > From: NANOG On Behalf Of Sadiq Saif > Sent: Friday, March 20, 2020 9:38 AM > To: nanog@nanog.org > Subject: Re: COVID-19 vs. peering wars > > On Fri, 20 Mar 2020, at 10:31, Steve Mikulasik via NANOG wrote: > > > > In Canada the CRTC really needs to get on Canadian ISPs about peering > > very liberally at IXs in each province. I know of one major > > institution right now that would have a major work from home issue > > resolved if one big ISP would peer with one big tier 1 in the IX they > > are both located at in the same province. Instead traffic needs to > > flow across the country or to the USA to get back to the same city. > > **cough** Bell Canada **cough**. > > -- > Sadiq Saif > https://sadiqsaif.com/
Re: interesting troubleshooting
On 20/03/2020 21:33, Nimrod Levy wrote: I was contacted by my NOC to investigate a LAG that was not distributing traffic evenly among the members to the point where one member was congested while the utilization on the LAG was reasonably low. I don't know how well-known this is, and it may not be something many people would want to do, but Enterasys switches, now part of Extreme's portfolio, allow "round-robin" as a load-sharing algorithm on LAGs. see e.g. https://gtacknowledge.extremenetworks.com/articles/How_To/How-to-configure-LACP-Output-Algorithm-as-Round-Robin This may not be the only product line supporting this.
RE: COVID-19 vs. peering wars
Worldwide, I don’t know. In Canada, peering is pretty messed up, i.e. it simply doesn’t happen at scale. At all. Even where it should. The overwhelmingly vast majority of Canadian traffic, even when nominally in-country, still transits the USA somewhere. If we had “ideal” full-mesh peering (i.e. setting aside all commercial considerations) at, say, regional IXes, including various popular CDNs, then service would take a giant step for the better for everyone who isn’t a big-4 (Bell, Telus, Shaw or Rogers) customer. Which admittedly would be an improvement for “only” about 30%-40% of the population… negligible, really, we’re only a country of 10M after all :-/. FYI, we have 4 big ISPs because none of them cover the entire country: they all* descend from local/regional monopolies or duopolies. *Mostly, that’s an approximation. -Adam Adam Thompson Consultant, Infrastructure Services [[MERLIN LOGO]]<https://www.merlin.mb.ca/> 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: Matthew Petach Sent: Friday, March 20, 2020 2:31 PM To: Adam Thompson Cc: Sadiq Saif ; nanog@nanog.org Subject: Re: COVID-19 vs. peering wars I'm curious; would people say that fixing peering inefficiencies could have a bigger impact on service performance than asking that Netflix, Amazon Prime, Youtube, Hulu, and other video streaming services cut their bit rates down? https://www.bbc.com/news/technology-51968302 https://arstechnica.com/tech-policy/2020/03/netflix-and-youtube-cut-streaming-quality-in-europe-to-handle-pandemic/ It seems that perhaps the fingers, and the regulatory hammer, are being pointed in the wrong direction at the moment. ^_^; Matt staying safely under the saran-wrap blanket for the next few weeks On Fri, Mar 20, 2020 at 9:31 AM Adam Thompson mailto:athomp...@merlin.mb.ca>> wrote: Every large ISP does this (or rather, doesn't) at every IX in Canada. Bell isn't unique by any stretch. It's not in their economic interest to peer at a local IX, because from their perspective, the IX takes away business (Managed L2 point-to-point circuits, at the very least) from them. Don't expect the dominant wireline ISP(s) in any region to join local IXes anytime soon, sadly, no matter how much it would benefit their customers. After all, the customer is always free to purchase service to the IX and join the IX, right??? *grumble* In my local case, if BellMTS joined MBIX, un-cached DNS resolution times could potentially drop by 15msec. That's HUGE. But the end-user experience is not their primary goal. Their primary goal is profit, as always. -Adam Thompson Founding member, MBIX (once upon a time) Adam Thompson Consultant, Infrastructure Services MERLIN 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca> > -Original Message- > From: NANOG mailto:nanog-boun...@nanog.org>> On > Behalf Of Sadiq Saif > Sent: Friday, March 20, 2020 9:38 AM > To: nanog@nanog.org<mailto:nanog@nanog.org> > Subject: Re: COVID-19 vs. peering wars > > On Fri, 20 Mar 2020, at 10:31, Steve Mikulasik via NANOG wrote: > > > > In Canada the CRTC really needs to get on Canadian ISPs about peering > > very liberally at IXs in each province. I know of one major > > institution right now that would have a major work from home issue > > resolved if one big ISP would peer with one big tier 1 in the IX they > > are both located at in the same province. Instead traffic needs to > > flow across the country or to the USA to get back to the same city. > > **cough** Bell Canada **cough**. > > -- > Sadiq Saif > https://sadiqsaif.com/
Practical guide to predicting latency effects?
I’m looking for a practical guide – i.e. specifically NOT an academic paper,
thanks anyway – to predicting the effect of increased (or decreased) latency on
my user’s applications.
Specifically, I want to estimate how much improvement there will be in
{bandwidth, application XYZ responsiveness, protocol ABC goodput, whatever} if
I decrease the RTT between the user and the server by 10msec, or by 20msec, or
by 40msec.
My googling has come up with lots of research articles discussing theoretical
frameworks for figuring this out, but nothing concrete in terms of a calculator
or even a rule-of-thumb.
Ultimately, this goes into MY calculator – we have the usual north-american
duopoly on last-mile consumer internet here; I’m connected directly to only one
of the two. There’s a cost $X to improve connectivity so I’m peered with both,
how do I tell if it will be worthwhile?
Anyone got anything at all that might help me?
Thanks in advance,
-Adam
Adam Thompson
Consultant, Infrastructure Services
[[MERLIN LOGO]]<https://www.merlin.mb.ca/>
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>
Re: xplornet contact or any experience with their satellite service?
Although I don't know of a way to solve this for videoconferencing, historically one way to mitigate the radio/vsat "batchiness" issue and its effect on end-to-end latency was to use a caching proxy server connected to a, er, "real" network somewhere, preferably as near as possible to the headend/uplink station. The modern web's move to TLS also means this technique is becoming pointless even for HTTP/S (although MITMing remains a way around that - many a HOWTO abounds, describing how to do this with Squid). FWIW, some very old radio systems behaved similarly... albeit not with 600+msec latency :-/. Some of the really old asymmetric TV systems (dial-up for uplink, CATV for downlink) exhibited similar characteristics and were similarly difficult to mitigate. Good luck! Adam Thompson Consultant, Infrastructure Services [[MERLIN LOGO]] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG on behalf of Mel Beckman Sent: Tuesday, April 21, 2020 9:05:20 AM To: Brian J. Murrell Cc: nanog@nanog.org Subject: Re: xplornet contact or any experience with their satellite service? Brian, Satellite services are shared bandwidth broadcast systems. The behavior you’re seeing is pretty common at times when you’re competing for access with other users. Just like the regular Internet, there are times of day when people tend to move more data, and because of latency and other limitations on bidirectional traffic, packets get delivered in batches. It’s not possible to interleave bytes, or even packets themselves. So when you see low or no throughput, it’s because the transponder is addressing packets to other users. -mel beckman > On Apr 21, 2020, at 6:53 AM, Brian J. Murrell wrote: > > A friend of mine just recently got Xplornet satellite service at his > rural home. I'm well aware of the latency issues with satellite > although frankly his latency is much better than I had feared it would > be and is around 600-700ms. > > But what seems to be worse than the latency is the "burstiness" of the > traffic and I am just wondering if that is normal/expected for > satellite service in general, and/or expected from Xplornet's service, > or if what I am seeing is not expected at all (i.e. not an artifact of > the satellite signal but rather a network management issue). > > Here's iperf3 for 30 seconds sending data (i.e. upload speed): > > [ ID] Interval Transfer Bitrate > [ 5] 0.00-1.21 sec 12.9 KBytes 87.4 Kbits/sec > [ 5] 1.21-2.00 sec 6.47 KBytes 67.2 Kbits/sec > [ 5] 2.00-3.00 sec 22.0 KBytes 180 Kbits/sec > [ 5] 3.00-4.00 sec 41.4 KBytes 339 Kbits/sec > [ 5] 4.00-5.00 sec 41.4 KBytes 339 Kbits/sec > [ 5] 5.00-6.00 sec 55.6 KBytes 456 Kbits/sec > [ 5] 6.00-7.00 sec 69.9 KBytes 572 Kbits/sec > [ 5] 7.00-8.00 sec 89.3 KBytes 731 Kbits/sec > [ 5] 8.00-9.00 sec 120 KBytes 986 Kbits/sec > [ 5] 9.00-10.00 sec 86.7 KBytes 710 Kbits/sec > [ 5] 10.00-11.00 sec 133 KBytes 1.09 Mbits/sec > [ 5] 11.00-12.00 sec 184 KBytes 1.51 Mbits/sec > [ 5] 12.00-13.00 sec 186 KBytes 1.53 Mbits/sec > [ 5] 13.00-14.00 sec 159 KBytes 1.30 Mbits/sec > [ 5] 14.00-15.00 sec 0.00 Bytes 0.00 bits/sec > [ 5] 15.00-16.00 sec 0.00 Bytes 0.00 bits/sec > [ 5] 16.00-17.00 sec 93.2 KBytes 763 Kbits/sec > [ 5] 17.00-18.00 sec 264 KBytes 2.16 Mbits/sec > [ 5] 18.00-19.00 sec 124 KBytes 1.02 Mbits/sec > [ 5] 19.00-20.00 sec 157 KBytes 1.28 Mbits/sec > [ 5] 20.00-21.00 sec 120 KBytes 986 Kbits/sec > [ 5] 21.00-22.00 sec 86.7 KBytes 710 Kbits/sec > [ 5] 22.00-23.00 sec 369 KBytes 3.02 Mbits/sec > [ 5] 23.00-24.00 sec 197 KBytes 1.61 Mbits/sec > [ 5] 24.00-25.00 sec 90.6 KBytes 741 Kbits/sec > [ 5] 25.00-26.00 sec 193 KBytes 1.58 Mbits/sec > [ 5] 26.00-27.00 sec 192 KBytes 1.57 Mbits/sec > [ 5] 27.00-28.00 sec 189 KBytes 1.55 Mbits/sec > [ 5] 28.00-29.00 sec 193 KBytes 1.58 Mbits/sec > [ 5] 29.00-30.00 sec 179 KBytes 1.46 Mbits/sec > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-32.20 sec 4.41 MBytes 1.15 Mbits/sec 388 sender > [ 5] 0.00-30.00 sec 3.57 MBytes 998 Kbits/sec receiver > > which averaged the overall prescribed "upload" speed, but notice that > it's not 1Mb/s in any kind of a steady stream but rather bursts of > higher than 1Mb/s speed followed by low/no speed. At one point it was >
Re: Switch for SFP+
Have you actually looked at Mikrotik switches? I don't like the OS, but the
hardware does what you want it to.
https://mikrotik.com/products/group/switches?filter&s=c&r={%22sfp_plus_interface%22:{%22s%22:%223%22,%22e%22:%2224%22}}#!
If necessary, buy your SFP modules from FS.com and get them coded as Mikrotik
modules at the factory - that's what we do for Cisco, Brocade, Juniper,
Extreme, etc.
Even the top-of-the-line Mikrotik only costs US$899.
-Adam
Adam Thompson
Consultant, Infrastructure Services
[[MERLIN LOGO]]
100 - 135 Innovation Drive
Winnipeg, MB, R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca>
www.merlin.mb.ca<http://www.merlin.mb.ca/>
From: NANOG on behalf of Mauro Gasparini
Sent: Thursday, May 14, 2020 8:46:21 AM
To: Mehmet Akcin
Cc: nanog
Subject: Re: Switch for SFP+
Thank you. The problem is that to get a price lower than U$D 3000 I have to
resort to a used device.
El 14/5/20 a las 01:08, Mehmet Akcin escribió:
Used Juniper QFX5100-48T will do it. Probably overkill but you can grab one
cheap @ebay
On Wed, May 13, 2020 at 16:36 Mauro Gasparini
mailto:mjgaspar...@gmail.com>> wrote:
Good afternoon.
I'm looking for a switch with the following capabilities:
. transport for more than 20 gbps
. link aggregation LACP
. slots for SFP+
. seamlessly when trunking vlans through the link aggregation.
And essentially that doesn't exceed US$D 2000 and is compatible with
10GBASE-ER and/or 10GBASE-ZR modules that are not from the vendor itself
(e.g. SPFs: Huawei, Mikrotik, Sumitomo, OEMs).
If any of you have a good experience with a device that meets these
requirements (which are minimal with the exception of price and
compatibility) ?
Regards.
Mauro Gasparini
--
Mehmet
+1-424-298-1903
Re: Ensuring RPKI ROAs match your routing intent
So in the ARIN world, Krill only works with "delegated" RPKI, not "hosted" RPKI - do I understand that correctly? If so, are there any plans to allow Krill's analytics and rules to monitor ARIN Hosted RPKI ROAs? -Adam Adam Thompson Consultant, Infrastructure Services [[MERLIN LOGO]] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG on behalf of Alex Band Sent: Thursday, June 25, 2020 8:31:52 AM To: Nanog Subject: Ensuring RPKI ROAs match your routing intent Hi everyone, Over the last two years NLnet Labs has been working on free, open source RPKI software and research for the community, supported by the RIPE NCC Community Projects Fund, Brazilian NIR NIC.br and Asia Pacific RIR APNIC. I have an update that we’d like to share. When creating a ROA in RPKI, it can have an effect on one or more BGP announcements, making them either Valid, Invalid or NotFound. Understanding what exactly determines these three states is not immediately obvious, especially in the beginning. At times, this can make creating ROAs a bit of a shot in the dark. I’ve seen several examples in the past where an operator created a ROA in their RIR Portal, waited for it to be published and then checked in services like BGPMon or the HE BGP Toolkit to see if everything turned out as expected. This is why, during my time at the RIPE NCC, we put a lot of work into making it immediately obvious what the effect of a ROA is going to be on the BGP announcements with your address space. Several RIRs have followed in these footsteps since. I presented on this journey at NANOG 63 in 2015: https://archive.nanog.org/meetings/abstract?id=2500 Now, in my new adventure at NLnet Labs, we’ve gotten the same team together to make simple, intuitive ROA management for Delegated RPKI available for everyone, seamlessly across RIR regions. With Krill 0.7.1 ‘Sobremesa’ you can easily create and maintain ROAs in a user interface that incorporates all of the best practices and lessons learned over the last 10 years and monitor them in ways never before possible, such as through Prometheus. Blog post with details: http://link.medium.com/1SsTJSAvB7 All the best, Alex
RE: L2VPN/L2transport, Cumulus Linux & hardware suggestion
Good luck with tunnelling LACP, no matter what boxes you have - LACP has (de facto) hard jitter requirements of under 1msec, or you'll be getting TCP resets coming out your ears due to mis-ordered packets. For your requirements, although I hesitate to recommend them for enterprise/carrier use, Miktotik's EoIP protocol does a much better job of this than most "carrier-grade" implementations. Otherwise, Juniper and Arista both come to mind, Juniper has the EX4650 that matches your h/w specs, and Arista has, oh, at least half a dozen boxes of various spec that comply, too. Not 100% sure the Juniper EX does 25G, now that I think of it. Adam Thompson Consultant, Infrastructure Services MERLIN 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca www.merlin.mb.ca > -Original Message- > From: NANOG On Behalf Of > Jürgen Jaritsch > Sent: Tuesday, July 7, 2020 3:15 PM > To: nanog@nanog.org > Subject: L2VPN/L2transport, Cumulus Linux & hardware suggestion > > Dear folks, > > have anyone already tried to run VXLAN/EVPN + “Bridge Layer 2 Protocol > Tunneling” on Cumulus Linux as an replacement for classic MPLS L2VPN/VPWS > (“xconnect”, l2circuit, VLL) ? > > I need to provide transparent Ethernet P2P virtual leased lines to my > customers and these have to support stuff like LLDP, STP, LACP, etc. The > transport L2 network is not THAT big: max hops between VTEP is 4. > > Anyone have suggestions for the below hardware request? > #) 1-3U L2/L3 box > #) 48x SFP28 / 1/10/25G > #) 6x QSFP28 / 100G > #) VXLAN/EVPN with L2 tunneling support > or > #) MPLS VPWS/l2circuit > #) Dual PSU > > > thanks & best regards > Jürgen >
Re: L2VPN/L2transport, Cumulus Linux & hardware suggestion
If jitter were defined anywhere vis-à-vis LACP, it would be _de jure_, not _de facto_ as I said. Yes, if you have *guaranteed* that TCP sessions hash uniquely to a single link in your network, you might be able to successfully tunnel LACP (or EtherChannel, or any other L1 link-bonding technique). The last time I attempted to do this on my network, I discovered that guarantee wasn't nearly as ironclad as I expected. I don't remember the gory details, at this remove, sorry. Maybe it wasn't TCP? Maybe it wasn't the default hashing algorithm? Dunno. -Adam On Jul. 8, 2020 00:48, Saku Ytti wrote: Hey Adam, On Wed, 8 Jul 2020 at 00:11, Adam Thompson wrote: > Good luck with tunnelling LACP, no matter what boxes you have - LACP has (de > facto) hard jitter requirements of under 1msec, or you'll be getting TCP > resets coming out your ears due to mis-ordered packets. Can you elaborate on this? Where is LACP jitter defined and for what purpose? We push packets around the globe in sub 200us jitter on any given day, so 1000us isn't for us a particularly hard goal. Only reason why I could imagine someone would care about jitter here is if protocol measures delay (LACP doesn't) and relies on delay to remain static and then balances per-packet or per-byte or otherwise between multiple links. However we of course put all packets from given TCP session to always same LACP interface, so from TCP session POV, each LACP is exactly a 1 interface. Per-packet balancing on LACP is possible via a special configuration, but anyone who does it, doesn't care about reordering, no matter of jitter, because even in very stable jitter, the paths may be unequal length and cause reordering. LACP hellos are sent every 1s when in fast mode with 3s keepalive, which also isn't particularly tight. We do have customers running LACP over MPLS pseudowires over great distances. -- ++ytti
Re: L2VPN/L2transport, Cumulus Linux & hardware suggestion
I do run the 7280SR2-48YC6, but I don't do VPLS or pseudowires on them right now so I can't help directly with that. Based on my experience with Arista so far, it'll be perfectly-well documented, just for a different platform, and in a blog post instead of in the user manual. :-( (Note to anyone from Arista lurking on the list: your User Manual sucks rocks because it's wildly incomplete. Please put some of the effort that goes into those EOS Central blog posts, into the manual instead.) As to the Juniper, I'm a client on a Juniper-based VPLS system, and the only thing it consistently intercepts is LLDP... which I'm actually OK with, mostly. Other BPDUs and other Ethernet protocols get passed through (that we've tested, so far). We have heard of some feature limitations on the EX4650, no CCC is unfortunate. I don't have any experience with the QFX series as an operator or customer so can't comment. Adam Thompson Consultant, Infrastructure Services [1593169877849] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG on behalf of Jürgen Jaritsch Sent: Tuesday, July 7, 2020 5:05:03 PM To: nanog@nanog.org Subject: AW: L2VPN/L2transport, Cumulus Linux & hardware suggestion Dear Adam, yeah, forget about LACP - the bigger problem is all the LLDP and STP stuff, that gets interpreted at the UNI port. LACP is a bad example - but there are many other frames and protocols, which must work. Could be that a customer wants to run MPLS+LDP on his VLL (for whatever reason ...). > For your requirements, although I hesitate to recommend them for enterprise/carrier use, Miktotik's EoIP protocol does a much better job of this than most "carrier-grade" implementations. Not at wirespeed ... and not without causing other issues (single thread load, etc). > Juniper has the EX4650 that matches your h/w specs,... Not 100% sure the Juniper EX does 25G, now that I think of it. Yeah, EX4650 it does: 48x 1/10/25G + 6x 100G + MPLS It also supports Ethernet over MPLS (at least they say here: https://www.juniper.net/documentation/en_US/junos/topics/topic-map/mpls-over view.html#id-mpls-feature-support-on-qfx-series-and-ex4600-switches) but at some of their sites they mention, that MPLS-based CCC are not support: https://www.juniper.net/documentation/en_US/junos/topics/topic-map/mpls-over view.html#jd0e2531 " ... MPLS-based circuit cross-connects (CCC) are not supported—only circuit-based pseudowires are supported. ..." There is also the QFX5120-48Y - 48x 1/10/25G + 8x 100G + MPLS In the past QFX wasn't the best idea for MPLS topics ... has this changed? > and Arista has, oh, at least half a dozen boxes of various spec that comply, too. Yeah, I already know them (do have some older 7050S). The call it "VXLAN P2P Pseudowire", but there is absolutely nothing in there CLI documentation :(. Looks like the feature is only support on the 7280 platform. Possible options: 7280SR2-48YC6 Do you have any experience with what they call "VXLAN P2P Pseudowire"? I can't even find a config example on the net :( thanks & best regards Jürgen -Ursprüngliche Nachricht- Von: Adam Thompson [mailto:athomp...@merlin.mb.ca] Gesendet: Dienstag, 7. Juli 2020 23:09 An: Jürgen Jaritsch ; nanog@nanog.org Betreff: RE: L2VPN/L2transport, Cumulus Linux & hardware suggestion Good luck with tunnelling LACP, no matter what boxes you have - LACP has (de facto) hard jitter requirements of under 1msec, or you'll be getting TCP resets coming out your ears due to mis-ordered packets. For your requirements, although I hesitate to recommend them for enterprise/carrier use, Miktotik's EoIP protocol does a much better job of this than most "carrier-grade" implementations. Otherwise, Juniper and Arista both come to mind, Juniper has the EX4650 that matches your h/w specs, and Arista has, oh, at least half a dozen boxes of various spec that comply, too. Not 100% sure the Juniper EX does 25G, now that I think of it. Adam Thompson Consultant, Infrastructure Services MERLIN 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) mailto:athomp...@merlin.mb.ca http://www.merlin.mb.ca > -Original Message- > From: NANOG <mailto:nanog-bounces+athompson=merlin.mb...@nanog.org> On Behalf > Of Jürgen Jaritsch > Sent: Tuesday, July 7, 2020 3:15 PM > To: mailto:nanog@nanog.org > Subject: L2VPN/L2transport, Cumulus Linux & hardware suggestion > > Dear folks, > > have anyone already tried to run VXLAN/EVPN + “Bridge Layer 2 Protocol > Tunneling” on Cumulus Linux as an replacement for classic MPLS > L2VPN/VPWS (“xconnect”, l2
RE: favorite network troubleshooting tools (online)
I see NLNOG’s IRRexplorer has been mentioned, but what about the NLNOG RING<https://ring.nlnog.net/> ? There’s a publicly-reachable LG (lg.ring.nlnog.net) but you have to sign up for access to the rest. -Adam Adam Thompson Consultant, Infrastructure Services [[MERLIN LOGO]]<https://www.merlin.mb.ca/> 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG On Behalf Of Matt Harris Sent: Thursday, July 16, 2020 9:49 AM To: Mehmet Akcin Cc: nanog Subject: Re: favorite network troubleshooting tools (online) Not one specific tool, but I often use various large SP looking glasses to determine what their routing tables look like when necessary. RIPE Atlas is another cool project that has also yet to be mentioned. Cloudflare's RPKI tools may also be of use. [https://netfire.net/logo_sig_gen2.png] Matt Harris | Infrastructure Lead Engineer 816‑256‑5446 | Direct Looking for something? Helpdesk Portal<https://help.netfire.net/> | Email Support<mailto:h...@netfire.net> | Billing Portal<https://my.netfire.net/> [https://netfire.net/Flag-United-States-of-America.jpg] We build and deliver end‑to‑end IT solutions. On Thu, Jul 16, 2020 at 9:08 AM Marcos Manoni mailto:marcosman...@gmail.com>> wrote: https://stat.ripe.net/ has lots of widgets https://stat.ripe.net/widget/list http://irrexplorer.nlnog.net/ for detailed IRR info (https://bgp.he.net also have some) El mié., 15 jul. 2020 a las 14:41, Mehmet Akcin (mailto:meh...@akcin.net>>) escribió: > > hey there, > > I recently have come across this http://ping.pe/ website, I have no > association with this but it's pretty awesome. This made me wonder what other > tools out there which I do not know about it. > > what are your favorite network troubleshooting tools? > > In addition to ping.pe<http://ping.pe>, I like https://bgp.he.net but would > love to hear your thought about other tool recommendations as especially the > ones that are distributed. > > Mehmet
Re: Never push the Big Red Button (New York City subway failure)
Now I'm curious... in all of the DCs and COs I've worked in - to the best of my knowledge, I haven't personally tested this! - the EPO button does not switch to emergency power. It turns off ALL equipment power in the space - no lights, no klaxons, nothing. In simpler setups, the EPO is connected to the UPS so anything plugged in to the UPS does dark instantly. In one DC I'm familiar with, the EPO switch kills all the UPS output and uses several relays to kill commercial power at the same time. In some, the room lights were not covered by the EPO switch, in some they were. Emergency exit lamps will continue to be lit, as they have internal batteries, and are required by building/fire code. Is it (somewhat) common for an EPO switch to only disconnect commercial power and leave local redundant power live? What sort of facilities would have this? -Adam Adam Thompson Consultant, Infrastructure Services [1593169877849] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG on behalf of Jay R. Ashworth Sent: September 11, 2021 22:23 To: nanog Subject: Re: Never push the Big Red Button (New York City subway failure) - Original Message - > From: "Sean Donelan" > NEW YORK CITY TRANSIT RAIL CONTROL CENTER POWER > OUTAGE ISSUE ON AUGUST 29, 2021 > Key Findings > September 8, 2021 > > https://www.governor.ny.gov/sites/default/files/2021-09/WSP_Key_Findings_Summary-for_release.pdf > > Key Findings > [...] > > 3. Based on the electrical equipment log readings and the manufacturer’s > official assessment, it was determined that the most likely cause of RCC > shutdown was the “Emergency Power Off” button being manually activated. I don't even *do* datacenter for a living, and I know that when you hit the Molly button, 1) A Klaxon goes off in the Data Center -- one that sounds *different* from the Halon Klaxon, in both cadence and tone (just for a couple bursts), and 2) Yellow rotating beacons turn on, and stay on while you're on Emergency Power. Yes, real honest-to-ghod *rotating mechanical beacons*, none of this flashing LED crap. Clearly, it's important that the use of Emergency Power be annoyingly noticeable. Cheers, -- jra -- Jay R. Ashworth Baylink j...@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274
Re: uPRF strict more
We just ran into a typical case where uRPF caused a partial outage for one of my customers: the customer is multi-homed, with another provider that I'm also connected to. Customer advertised a longer-prefix to the other guy, so I started sending traffic destined for Customer to the Other Provider... who then promptly dropped it because they had uRPF enabled on the peering link, and they were seeing random source IPs that weren't mine. Well... yeah, that can happen (semi-legitimately) anytime you have a topological triangle in peering. I've concluded over the last 2 years that uRPF is only useful on interfaces pointing directly at non-multi-homed customers, and actively dangerous anywhere else. -Adam Adam Thompson Consultant, Infrastructure Services [1593169877849] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG on behalf of Amir Herzberg Sent: September 28, 2021 20:06 To: Randy Bush Cc: North American Network Operators' Group Subject: Re: uPRF strict more Randy, great question. I'm teaching that it's very rarely, if ever, used (due to high potential for benign loss); it's always great to be either confirmed or corrected... So if anyone replies just to Randy - pls cc me too (or, Randy, if you could sum up and send to list or me - thanks!) Amir -- Amir Herzberg Comcast professor of Security Innovations, Computer Science and Engineering, University of Connecticut Homepage: https://sites.google.com/site/amirherzberg/home `Applied Introduction to Cryptography' textbook and lectures: https://sites.google.com/site/amirherzberg/applied-crypto-textbook<https://sites.google.com/site/amirherzberg/applied-crypto-textbook> On Tue, Sep 28, 2021 at 8:50 PM Randy Bush mailto:ra...@psg.com>> wrote: do folk use uPRF strict mode? i always worried about the multi-homed customer sending packets out the other way which loop back to me; see RFC 8704 §2.2 do vendors implement the complexity of 8704; and, if so, do operators use it? clue bat please randy
Re: uPRF strict more
Yeah, but loose mode is inherently useless on any router carrying full tables. (Ok, it can spot bogons, but that's a side effect and I have other ways to catch those.) Point being that MANRS implementation in the "simple" case is (or, at least, CAN be) almost trivially easy, but in the "complex" case is quite difficult - I'm still not even sure I know how to do it 100% correctly with multi-homed downstreams clients. "Just turn on RPF" is starting to feel more like an article of faith rather than genuine technical guidance. :-( -Adam Get Outlook for Android<https://aka.ms/AAb9ysg> From: Brian Johnson Sent: Friday, October 1, 2021 8:31:15 AM To: Adam Thompson Cc: Amir Herzberg ; Randy Bush ; North American Network Operators' Group Subject: Re: uPRF strict more For strict-mode... Completely agree. As has been previously said, this is a tool that all players involved need to understand. This is no different than everyone correctly using BGP in their application for their outcomes. On Sep 29, 2021, at 12:07 PM, Adam Thompson mailto:athomp...@merlin.mb.ca>> wrote: We just ran into a typical case where uRPF caused a partial outage for one of my customers: the customer is multi-homed, with another provider that I'm also connected to. Customer advertised a longer-prefix to the other guy, so I started sending traffic destined for Customer to the Other Provider... who then promptly dropped it because they had uRPF enabled on the peering link, and they were seeing random source IPs that weren't mine. Well... yeah, that can happen (semi-legitimately) anytime you have a topological triangle in peering. I've concluded over the last 2 years that uRPF is only useful on interfaces pointing directly at non-multi-homed customers, and actively dangerous anywhere else. -Adam Adam Thompson Consultant, Infrastructure Services [1593169877849] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG mailto:nanog-bounces+athompson=merlin.mb...@nanog.org>> on behalf of Amir Herzberg mailto:amir.li...@gmail.com>> Sent: September 28, 2021 20:06 To: Randy Bush mailto:ra...@psg.com>> Cc: North American Network Operators' Group mailto:nanog@nanog.org>> Subject: Re: uPRF strict more Randy, great question. I'm teaching that it's very rarely, if ever, used (due to high potential for benign loss); it's always great to be either confirmed or corrected... So if anyone replies just to Randy - pls cc me too (or, Randy, if you could sum up and send to list or me - thanks!) Amir -- Amir Herzberg Comcast professor of Security Innovations, Computer Science and Engineering, University of Connecticut Homepage: https://sites.google.com/site/amirherzberg/home `Applied Introduction to Cryptography' textbook and lectures: https://sites.google.com/site/amirherzberg/applied-crypto-textbook<https://sites.google.com/site/amirherzberg/applied-crypto-textbook> On Tue, Sep 28, 2021 at 8:50 PM Randy Bush mailto:ra...@psg.com>> wrote: do folk use uPRF strict mode? i always worried about the multi-homed customer sending packets out the other way which loop back to me; see RFC 8704 §2.2 do vendors implement the complexity of 8704; and, if so, do operators use it? clue bat please randy
Re: [External] Re: uPRF strict more
IMHO, no, it's not worth it... at least, not unless you have a larger budget than mine, a larger department than mine, and possibly more skilled operators than I am. I don't even grok how this is supposed to work - the only place I "peer" in the classical sense is my local IX; all my other "peers" are ALSO either downstream or upstream networks for me. (e.g. my NREN regional affiliate, who is a lateral peer for many prefixes, but also an upstream access network to reach the national, and then global, REN[s]) If a router doesn't have a default route, and also doesn't have full tables, I can't use it for downstream customers even if they're BGP peers; they're expecting me to either provide full tables, or act as a default gateway. Do people in other parts of the world have access (both physical and logical) to enough bilateral peering (and budgets...) that it makes sense to deploy a router per peer? For the NREN case, it's not full tables, and it's not default routes, but it's still a pretty big table. And they're the location of the "triangle" routing where I have several downstream clients who also peer directly with them. I'm both a lateral "peer" AND a downstream customer to them... so they tried to turn on uRPF on the L3 interfaces towards me and, well, bad things happened to our mutual customers' traffic. Admittedly this was triggered by the downstream customer doing questionable things with different-length prefixes, but the fact remains uRPF causes (sometimes partial) outages anywhere you have multi-path "downstream" clients. And based on the topology, I cannot conceive of any set of ACLs that I could feasibly apply to inbound traffic on the peering link with my NREN affiliate, which makes it... more difficult to be BCP/MARNS-compliant. Commercial traffic regularly transits R&D unexpectedly, and vice-versa: path asymmetry is common here. -Adam P.S. the topology in question was as simple as this. Cust. advertised /N to me, but /N+1 to the R&D side, so traffic started taking an unanticipated detour via the R&D side. IRR/RPKI does not solve this: it was a legitimate advertisement. ┌─┐ ┌───┐ ┌►│MRnet├►│R&D│ │ └─┘ └───┘ │▲ │ │ ┌─┴───┐ ┌──┴───┐┌──┐ │Cust.├►│MERLIN├───►│Commercial│ └─┘ └──┘└──┘ Adam Thompson Consultant, Infrastructure Services [1593169877849] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG on behalf of Randy Bush Sent: October 1, 2021 12:28 To: Mark Tinka Cc: North American Network Operators' Group Subject: Re: [External] Re: uPRF strict more > A partial table with no default is perfectly fine for a peering router. > > As long as your peering router knows how to get to your prefixes and > those of your customers, as well as the prefixes of the networks you > peer with, you're good to go. in fact, this seems to be the modern conservative style for some years. i sometimes wonder if it is worth the config pain. randy
Increase bandwidth usage in partial-mesh network?
Looking for recommendtions or suggestions... I've got a downstream customer asking for help; they have a private internal network that I've taken to calling the "partial-mesh network from hell": it's got two partially-overlapping radio networks, mixed with islands of isolated fiber connectivity. Dynamic routing protocols (IS-IS, OSPF, EIGRP, etc.) generally will only select the _best_ path, they won't spread the load unless all paths are equal - and they are very unequal in this network, ECMP would likely fail horribly. The network is becoming bandwidth-limited, so they're wanting to make use of all available paths, not just the single "best" path. It's also remote and spread out, so adding new links or upgrading existing links is difficult and expensive. Oh, and their routers are overdue for a refresh, so acquiring replacement h/w is now possible. Has anyone come across any product or technology that can handle the multi-path-ness and the private-network-ness like a regular router, but also provides the intelligent per-flow path steering based on e.g. latency, like an SD-WAN device (and/or some firewalls)? Here's hoping, -Adam Adam Thompson Consultant, Infrastructure Services [1593169877849] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/>
Re: Increase bandwidth usage in partial-mesh network?
Hah, no not your client 🙂. Their existing network is actually surprisingly stable, but it is bandwidth-constrained. As well as the various other replies I've seen here and off-list (THANKS!), the only commercial product I've found so far that might have a hope of handling this is HPE/Aruba's Silverpeak line. We'll see what else comes out of the woodwork, though - if nothing else, it's a very interesting exercise! Adam Thompson Consultant, Infrastructure Services [1593169877849] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: Fletcher Kittredge Sent: October 13, 2021 12:59 To: Adam Thompson Cc: nanog Subject: Re: Increase bandwidth usage in partial-mesh network? Hey! From the description it must be one of our clients! Just beware if you go this route, a network that is probably already unstable and unreliable will become at least an order of magnitude worse. You can't fix ten lbs of stuff into a 4 lb stuff bag. The internet protocols do not tolerate congestion well. On Wed, Oct 13, 2021 at 1:31 PM Adam Thompson mailto:athomp...@merlin.mb.ca>> wrote: Looking for recommendtions or suggestions... I've got a downstream customer asking for help; they have a private internal network that I've taken to calling the "partial-mesh network from hell": it's got two partially-overlapping radio networks, mixed with islands of isolated fiber connectivity. Dynamic routing protocols (IS-IS, OSPF, EIGRP, etc.) generally will only select the _best_ path, they won't spread the load unless all paths are equal - and they are very unequal in this network, ECMP would likely fail horribly. The network is becoming bandwidth-limited, so they're wanting to make use of all available paths, not just the single "best" path. It's also remote and spread out, so adding new links or upgrading existing links is difficult and expensive. Oh, and their routers are overdue for a refresh, so acquiring replacement h/w is now possible. Has anyone come across any product or technology that can handle the multi-path-ness and the private-network-ness like a regular router, but also provides the intelligent per-flow path steering based on e.g. latency, like an SD-WAN device (and/or some firewalls)? Here's hoping, -Adam Adam Thompson Consultant, Infrastructure Services [1593169877849] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> -- Fletcher Kittredge GWI 207-602-1134 www.gwi.net<http://www.gwi.net>
Anybody out there from Suddenlink AS19108?
Ping me off-list if so. Please and thank you. --Adam
Re: PCH Peering Survey 2021
Question: if I have a written contract with a peer that covers the link and IP service in general, but that contract does not specifically discuss BGP or peering, is that a Yes or No? Also, how should I indicate "unknown" , particularly for the Written Contract field? -Adam Adam Thompson Consultant, Infrastructure Services [1593169877849] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG on behalf of Bill Woodcock Sent: Friday, October 29, 2021 06:47 To: NANOG Subject: PCH Peering Survey 2021 Background: Five and ten years ago PCH conducted comprehensive global surveys characterizing Internet peering agreements. They are the only ones of their kind, and are relied upon by legislators and regulators throughout the world to understand the Internet interconnection landscape. Our write-ups of the prior surveys can be found here: https://www.pch.net/resources/Papers/peering-survey/PCH-Peering-Survey-2011.pdf https://www.pch.net/resources/Papers/peering-survey/PCH-Peering-Survey-2016/PCH-Peering-Survey-2016.pdf …and video of the NANOG presentation of the 2016 results is here: https://www.youtube.com/watch?v=lPuoBmxyXMc At the time of the 2011 survey, we committed to repeating the survey every five years, to provide time-series data about the direction peering trends take. We’re now conducting the third iteration of the survey. Among other things, the surveys have helped establish a better understanding of trends in: • The increasingly uniform global norms of interconnection • National preferences within the network operator community for country of governing law • The long tail of small networks which don’t yet support IPv6 routing • The significance of multilateral peering agreements in the density of the interconnection mesh These findings, particularly the first, have been invaluable in giving regulators in the vast majority of the world’s countries a data-driven basis for refraining from prescriptively regulating Internet interconnection. They have demonstrated in objective terms that the Internet self-regulates in a way that’s more globally uniform and closely harmonized than any coordination of national regulatory bodies could accomplish. Participation: The survey is global in scope, and our goal is to reflect the diversity of peering agreements in the world. Your participation ensures that your norms and ways of doing business are represented accurately and proportionately in the dataset. If you don’t participate, the way you do business will be less well-represented in the data, and will seem less normal to regulators and policy-makers. We’re interested in large ISPs and small ISPs, ISPs in Afghanistan and in Zimbabwe, bilateral agreements and multilateral, private and public. Our intent is to be as comprehensive as possible. In 2011, the responses we received represented 4,331 networks in 96 countries, or 86% of the world’s ISPs at that time. In 2016, responses represented 10,794 networks in 148 countries, or 60% of the world’s ISPs in 2016. Our aim is to be equally inclusive this year. Since our principal method of soliciting participation is via NOG mailing lists, I’m afraid many of you will see this message several times, on different lists, for which we apologize. Privacy: In 2011 and 2016, we promised to collect the smallest set of data necessary to answer the questions, to perform the analysis immediately, and not to retain the data after the analysis was accomplished. In that way, we ensured that the privacy of respondents was fully protected. We did as we said, no data was leaked, and the whole community benefited from the trust that was extended to us. We ask for your trust again now as we make the same commitment to protect the privacy of all respondents, using the same process as was successfully employed the last two times. We are asking for no more data than is absolutely necessary. We will perform the analysis immediately upon receiving all of the data. We will delete the data once the analysis has been performed. The Survey: We would like to know your Autonomous System Number, and the following five pieces of information relative to each other AS you peer with: • Your peer’s ASN (peers only, not upstream transit providers or downstream customers) • Whether a written and signed peering agreement exists (the alternative being a less formal arrangement, such as a "handshake agreement") • Whether the terms are roughly symmetric (the alternative being that they describe an agreement with different terms for each of the two parties, such as one compensating the other, or one receiving more or fewer than full customer routes) • Whether a jurisdiction of governing law is defined • Whether IPv6 routes are being exchange
Validating multi-path in production?
Hello all. Over time, we've run into occurrences of both bugs and human error, both in our own gear and in our partner networks' gear, specifically affecting multi-path forwarding, at pretty much all layers: Multi-chassis LAG, ECMP, and BGP MP. (Yes, I am a corner-case magnet. Lucky me.) Some of these issues were fairly obvious when they happened, but some were really hard to pin down. We've found that typical network monitoring tools (Observium & Smokeping, not to mention plain old ping and traceroute) can't really detect a hashing-related or multi-path-related problem: either the packets get through or they don't. Can anyone recommend either tools or techniques to validate that multi-path forwarding either is, or isn't, working correctly in a production network? I'm looking to add something to our test suite for when we make changes to critical network gear. Almost all the scenarios I want to test only involve two paths, if that helps. The best I've come up with so far is to have two test systems (typically VMs) that use adjacent IP addresses and adjacent MAC addresses, and test both inbound and outbound to/from those, blindly trusting/hoping that hashing algorithms will probably exercise both paths. Some of the problems we've seen show that merely looking at interface counters is insufficient, so I'm trying to find an explicit proof, not implicit. Any suggestions? Surely other vendors and/or admins have screwed this up in subtle ways enough times that this knowledge exists? (My Google-fu is usually pretty good, but I'm striking out - maybe I'm using the wrong terms.) -Adam Adam Thompson Consultant, Infrastructure Services [1593169877849] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/>
Re: Questions about IRR best practices
Job Snijders via NANOG wrote: > Dear Lee, Hi Job, > *ring ring* - "IRR/RPKI helpdesk how may I help you today?" :-) What a fantastic helpdesk it is, too! ;-) > Another way to satisfy this request is to ask the organization's > provider to create an AS-SET (preferably RIR-operatored IRR such as > ARIN, RIPE, etc), and then reference their AS-SET on your own AS-SET. > IRR AS-SETs permit both referencing AS Numbers and AS-SETs as 'members:'. This brings up a question I've been mulling over lately - if AS64500 has AS64500:AS-CUSTOMERS and has a customer AS64510, and AS64510 only ever plans to originate routes themselves, with no downstreams, is there any reason other than "future flexibility" for AS64510 to create an AS-SET, e.g.: as-set: AS64500-CUSTOMERS |-> members: AS64510:AS-AS64510 |-> members: AS64510 versus simply the following: as-set: AS64500-CUSTOMERS |-> members: AS64510 It seems to me both are entirely functionally equivalent, aside that if AS64510 ever wants to change things they would need to get AS64500 to update their AS64500-CUSTOMERS members. > The industry trend (very noticable the last 3 years) is that the ability > to create proxy route object registrations is slowly fading away. > > At at first glance proxy registrations seem better than 'no > registration', the downside is that anyone can create proxy > registrations for any prefix: proxies are not very safe! I wonder what this means (eventually, anyway) for RADB, NTTCOM, LEVEL3, etc. Though LEVEL3 can hardly be considered on the bleeding edge of IRR... > The recommendation is that each and every IP resource holder creates IRR > and/or RPKI objects themselves, or delegates the authority to do so to > their service provider. What is the mechanism for such a delegation? It seems to me that with enough RPKI adoption, IRR will eventually go away, but you mention "and/or" - is that just for current interoperability? Taking it a step futher, let's say I have valid ROAs for all my prefixes today, because I do. But I don't maintain my objects in ARIN's IRR. Do I need to migrate, and if so, assuming my upstreams still "reject RPKI invalid but permit rpki unknown AND require an IRR AS-SET to build BGP filters in the first place" it seems to me that maintaining IRR is necessary because otherwise the upstream won't listen to the announcements at all, much less verify RPKI. > Technically this is likely to work, but the downside is that you end up > with a hard dependency on another ISP's proxy registration. If for > whatever reason that registration lapses (failure to pay bills, M&A, who > knows) ... you might end up with a hard to troubleshoot situation where > it is not immediately clear "it was working yesterday, but not today?!". FWIW, Lee, I ran into this exact scenario this week. Everything was working, but the route was only being permitted by the customer's upstream because of a stale object proxy registered by someone else. My two cents is that this is an exceedling common occurence. > The best course of action is to ensure that objects are either managed > by yourself, or by the customer, so the responsibilities and object > ownership are clear to everyone involved. And yet, there's probably double-digits of people that fully grok IRR, despite the years and years of NOG presenations and tutorials and everything else. > A great tool to gain some insight into various IRR/BGP/RPKI data sources > and what the registration status of various objecst might mean can be > found at this awesome tool: https://irrexplorer.nlnog.net/ There's also a command-line irrtree that's pretty slick, and if you want to verify AS-SET recursion and ultimately expansion into prefixes, point your whois client toward filtergen.level3.net [1] and/or filtergen.dan.me.uk [2] --Adam [1] But be careful because L3 filtergen doesn't recurse across registries and will ignore data that's not source: LEVEL3 +unless* you put 'remarks: Level3 members: RADB::AS-FOO' in the AS-SET. [2] If you instead want to filtergen against RADB data (and objects which RADB mirrors from others) see https://www.dan.me.uk/filtergen
Re: Questions about IRR best practices
Jay Hennigan wrote: > Long story short, we consolidated acquisitions into a single AS, returned > the old AS to ARIN, and a 2006 RADB entry that looks to have been > auto-generated by Level 3 with the old AS is still hanging around, causing > other to question it. If it's source: LEVEL3, drop a note to ipad...@centurylink.com - it works some of the time. --Adam
Re: Validating multi-path in production?
The problem I'm looking to solve is the logical opposite, I think: I want to demonstrate that no links are malfunctioning in such a way that packets on a certain path are getting silently dropped. Which has some "proving a negative" aspects to it, unfortunately. I think the only way I can demonstrate it is to determine that every single multi-path/hashed-member link is working, which is... hard. Especially if I need to deal with the combinatoric explosion - I *think* I can skip that part. -Adam Get Outlook for Android<https://aka.ms/AAb9ysg> From: James Bensley Sent: Sunday, November 14, 2021 5:29:25 AM To: Adam Thompson ; nanog Subject: Re: Validating multi-path in production? On Fri, 12 Nov 2021 at 16:54, Adam Thompson mailto:athomp...@merlin.mb.ca>> wrote: The best I've come up with so far is to have two test systems (typically VMs) that use adjacent IP addresses and adjacent MAC addresses, and test both inbound and outbound to/from those, blindly trusting/hoping that hashing algorithms will probably exercise both paths. If the goal is to test that traffic *is* being distributed across multiple links based on traffic headers, then you can definable roll your own. I think the problem is orchestrating it (feeding your topology data into the tool, running the tool, getting the results out, and interpreting the results etc). A coupe of public examples: https://github.com/facebookarchive/UdpPinger https://www.youtube.com/watch?v=PN-4JKjCAT0 If you do roll your own, you need to taylor the tests to your topology and your equipment. For example, you can have two VMs as you mentioned, each at opposite ends of the network. Then, if your network uses a 5-tuple for ECMP inside the core for example, you could send many flows between the two VMs, rotating the sauce port for example, to ensure all links in a LAG or all ECMP paths are used. It's tricky to know the hashing algo for every type of device you have in your network, and for each traffic type for each device type, if you have a multi vendor network. Also, if your network carries a mix of IPv4, IPv6, PPP, MPLS L3 VPNs, MPLS L2 VPNs, GRE, GTP, IPSEC, etc. The number of permutations of tests you need to run and the result sets you need to parse, grows very rapidly. Cheers, James.
anyone use fbtracert successfully?
The tool fbtracert (http://github.com/facebookarchive/fbtracert) was mentioned here recently as a way to get visibility into multi-pathing. Has anyone here ever used this tool successfully? Supposedly Facebook uses this tool internally, but… that doesn’t help much. I’ve tried it on 4 different platforms/OSes (WSL Ubuntu; RedHat; Debian; OpenBSD), and versions of Go (v1.10 through v1.16), in three very different environments (on-prem public IP; on-prem NAT’d; cloud public IP), and I’ve yet to see it produce any meaningful output – each run/iteration/thread only detects one, single, hop out of the entire chain of routers, making it less than useful. Granted, that’s not a full regression test by any means, but if anyone here has ever used it successfully, could you please let me know what sort of environment you ran it in/on? Thanks, -Adam Adam Thompson Consultant, Infrastructure Services [1593169877849] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/>
RE: anyone use fbtracert successfully?
Thank you!! Some of those tools are proving much more useful for me than fbtracert. (In particular, traceflow has been updated recently enough that it “just works” in common environments that have Python3. And while it may not be perfect, it’s good enough to show what I need.) -Adam (who apparently has lost the skills needed to Google usefully, in his decrepitude) Adam Thompson Consultant, Infrastructure Services [1593169877849] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: Hugo Slabbert Sent: Thursday, November 25, 2021 10:39 AM To: Thomas Scott Cc: Adam Thompson ; nanog Subject: Re: anyone use fbtracert successfully? What about some other options? https://paris-traceroute.net/ https://dublin-traceroute.net/ https://github.com/rucarrol/traceflow -- Hugo Slabbert On Wed, Nov 24, 2021 at 9:54 AM Thomas Scott mailto:mr.thomas.sc...@gmail.com>> wrote: Ha, my apologies, I thought I was writing this for a Linux User Group, not a NOG. Ignore my simplistic explanations. - Thomas Scott | mr.thomas.sc...@gmail.com<mailto:mr.thomas.sc...@gmail.com> On Wed, Nov 24, 2021 at 12:47 PM Thomas Scott mailto:mr.thomas.sc...@gmail.com>> wrote: I have used it successfully in a test environment that I was using ECMP in. Most of the public networks that I've worked with don't use ECMP as often as other methods for steering traffic (LAGs, BGP MEDs, etc). What I have seen it fantastically useful for was troubleshooting a transit provider, or for when they were congested or had a flapping core link. Granted I think it's still subject to ICMP deprioritization (most SP's use it prodigiously), and most MPLS cores don't decrement TTL, but it was still useful to be able to show them "no, at this IP, I always drop traffic, when..." - Thomas Scott | mr.thomas.sc...@gmail.com<mailto:mr.thomas.sc...@gmail.com> On Wed, Nov 24, 2021 at 12:23 PM Adam Thompson mailto:athomp...@merlin.mb.ca>> wrote: The tool fbtracert (http://github.com/facebookarchive/fbtracert) was mentioned here recently as a way to get visibility into multi-pathing. Has anyone here ever used this tool successfully? Supposedly Facebook uses this tool internally, but… that doesn’t help much. I’ve tried it on 4 different platforms/OSes (WSL Ubuntu; RedHat; Debian; OpenBSD), and versions of Go (v1.10 through v1.16), in three very different environments (on-prem public IP; on-prem NAT’d; cloud public IP), and I’ve yet to see it produce any meaningful output – each run/iteration/thread only detects one, single, hop out of the entire chain of routers, making it less than useful. Granted, that’s not a full regression test by any means, but if anyone here has ever used it successfully, could you please let me know what sort of environment you ran it in/on? Thanks, -Adam Adam Thompson Consultant, Infrastructure Services 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/>
RE: IP tracking system
> This may have been asked and answered, but I couldn’t find the answer. The answer changes every year, anyway. > What are people recommending these days for IP tracking systems? I’m > looking for something to track the used/available IP addresses in my new > lab. FWIW, we use TeemIP/iTop, although I can't say I would recommend it, it does work. We've looked at NetBox, which is quite nice, but getting our data *out* of TeemIP/iTop proved to be much more difficult than expected, so we haven't migrated yet. CANARIE (Canadian NREN) and all the subsidiary RANs here are adopting NetBox, AFAIK. -Adam Adam Thompson Consultant, Infrastructure Services MERLIN 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca www.merlin.mb.ca
RE: BGP Route Monitoring
Most monitoring products allow you to monitor custom SNMP OIDs, and your entire BGP RIB is – usually – exposed via SNMP. Most monitoring products also treat “missing” OIDs specially, and can alert on that fact. At least, that’s how I would start doing it. We use Observium here, and it can do what you want, albeit with a little bit of futzing around in the Custom OID and Alerts sections. Cisco does weird things with getting SNMP data from VRFs, though, so… YMMV. I know there used to be a Cisco-proprietary way to select which VRF you were polling common OIDs from, but don’t remember the details. -Adam Adam Thompson Consultant, Infrastructure Services [MERLIN] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG On Behalf Of Sandoiu Mihai Sent: Thursday, January 6, 2022 4:35 AM To: nanog@nanog.org Subject: BGP Route Monitoring Hi I am looking for a route monitoring product that does the following: -checks if a specific bgp route from a specific neighbor is present the BGP table (in some vrf, not necessarily internet routed vrf) of an ASR9K running IOS XR -sends a syslog message or an alarm if the route goes missing The use case is the following: we are receiving same routes over 2 or more bgp peerings, due to best route we cannot really see at the moment if one of the routes ceased to be received over a certain peering. Alternative approach: a product that measures the number of bgp received prefixes from a certain peer. Do you know of such product that is readily available and does not require ssh sessions to the routers and parsing the outputs? I am trying to find a solution that does not require much scripting or customization. Many thanks. Regards Mihai
RE: SRv6 Capable NOS and Devices
My question is, why do you think you need Segment Routing at all? Is your network so enormously large and/or complex that IS-IS (and/or MPLS-TE) isn't capable of handling it? So far, SR looks like a solution in search of a problem, at least to me. I'm not saying you don't have a need for it, but I am questioning whether you do, or whether you're just being sucked in by all the latest sizzle (i.e. sales & marketing materials). (After all, that's what the sizzle is *designed* to do!) -Adam Adam Thompson Consultant, Infrastructure Services MERLIN 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca www.merlin.mb.ca > -Original Message- > From: NANOG On > Behalf Of Colton Conor > Sent: Tuesday, January 11, 2022 9:17 AM > To: NANOG > Subject: SRv6 Capable NOS and Devices > > I know the SRv6 is a fairly new technology. I am wondering which > vendors and network operating systems fully support SRv6 today? Has > anyone deployed this new technology? > > If building a greenfield regional ISP network, would SRv6 be a requirement? > > My understanding is that because it's using IPv6 in the dataplane, not > all devices have to have SRv6 enabled. The in-between core devices > just have to support IPv6, but not necessarily support SRv6. This is > much different than traditional MPLS networks today where all devices > have to support MPLS/LDP correct?
Useful ping targets for end-users?
Before you start reading, yes, I fully understand how silly this question is. But I need to give _something_ to a customer who has the ability to run ping/traceroute but nothing else. (And they have an intermittent latency problem that we haven’t been able to isolate yet.) Does anyone curate a list of “useful” ICMP responders that are at least kinda-sorta reliable/expected to continue responding? For example, all the major anycast DNS cloud providers respond to ICMP, but I don’t really want to tell my customer to ping an anycast IP address because the RTT results will be useless data (for comparative purposes). I’m also not excited about providing random router IP address for what should be obvious reasons. There are some IPs that my routing paths that should be stable, but between routing changes and control-plane policing, those aren’t awesome. I’m looking for IPs I can suggest that are well outside my network. Restatement: yes, there are much better ways to diagnose problems, but my customer can only run ping & traceroute (and pathping, I suppose) and is capable enough to run those tools and self-assess before calling me. It sounds foolish to even ask, but maybe there’s a resource out there I don’t know about… -Adam Adam Thompson Consultant, Infrastructure Services [MERLIN] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/>
RE: Long hops on international paths
Peering connection, I think, can explain this. With some notable exceptions (all of whom participate here, I think), carriers still don’t throw around 100G+ peering routers around like sprinkles on a donut. (Even those big networks don’t, it just looks like they do because they’re freaking huge.) I suspect what you may be seeing is NOT “international carriers all concentrate on a single router” at all, but rather a) “large carriers tend to interconnect at only a few points” and b) “the best path from North America to Europe will therefore tend to always go through the same small set of inter-AS peering routers on this side”. What you’ve described, purely in the public-visible layer 3 internet, is normal in my experience. I would fully expect upwards of 90% of my daily traffic crosses one of 3 peering routers in my network, no surprise there, but ALSO I estimate at least 80% of it crosses one of no more than 5 or 6 routers even as I get 5, 6, 7, 8 hops away from me. The internet isn’t as diversely-pathed as it once was. In the MPLS carrier case, I’m aware of a couple that use MPLS to tunnel peering traffic from their edge back to a centralized “core” router that speaks BGP. Not sure how common this is, I have a very small sample set. But in this example, no matter how diverse the carrier is at L1/L2, your L3 investigation will always hit the same much-smaller set of routers. To directly answer your question, the cost/benefit is driven by the fact that running BGP is (relatively) complicated, error-prone and expensive, compared to not running BGP. And those routers running BGP are (broadly speaking) the routers controlling inter-carrier traffic. So the “chokepoints” naturally occur as an emergent property of each carrier controlling their own operational and financial risk. Very much depends on the carrier and their operational philosophy. I know nearly nothing about Telia, but Zayo doesn’t (didn’t? did they get bought?) have a lot of peering routers – they were historically more of an L2 and/or Private-network operator, as I understand it. (I was never a customer, so that’s hearsay.) -Adam Adam Thompson Consultant, Infrastructure Services [MERLIN] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG On Behalf Of PAUL R BARFORD Sent: Tuesday, January 18, 2022 8:49 AM To: davidbass...@gmail.com Cc: nanog@nanog.org Subject: Re: Long hops on international paths Hello David, Understanding the physical topology of the network is not our objective. What we're trying to understand is the logical topology revealed by traceroute (we are well-aware of traceroute limitations) and why a relatively small set of routers in different countries tend to have the majority of the international connections. Our expectation was that layer 3 connectivity revealed in traceroute to be relatively evenly spread out along coasts and near submarine landing points. We're not seeing that. So, the question is what is the cost/benefit to providers to configure/maintain routes (that include long MPLS tunnels) that tend to concentrate international connectivity at a relatively small number of routers? Regards, PB From: davidbass...@gmail.com<mailto:davidbass...@gmail.com> mailto:davidbass...@gmail.com>> Sent: Tuesday, January 18, 2022 8:22 AM To: PAUL R BARFORD mailto:p...@cs.wisc.edu>> Cc: morrowc.li...@gmail.com<mailto:morrowc.li...@gmail.com> mailto:morrowc.li...@gmail.com>>; nanog@nanog.org<mailto:nanog@nanog.org> mailto:nanog@nanog.org>> Subject: Re: Long hops on international paths I think a large part of your problem is that you’re using trace route to try and determine the full topology of a large complex network. It won’t show the full topology. On Mon, Jan 17, 2022 at 7:43 PM PAUL R BARFORD mailto:p...@cs.wisc.edu>> wrote: What we're considering specifically are consecutive (layer 3) hops as identified by traceroute. Thus, TTL is decremented by 1 and no more than 1 (i.e., we have to get full information (not *) from consecutive hops to consider the link). I have asked my colleague to put together a set of examples. We assume that there are multiple layer 1 and 2 links, and possibly layer 3 hops masked from traceroute by MPLS. But what we're seeing in terms of hops exposed by traceroute make it look like a single (TTL decremented by 1) hop. I'll post the examples when I get them. PB From: morrowc.li...@gmail.com<mailto:morrowc.li...@gmail.com> mailto:morrowc.li...@gmail.com>> Sent: Monday, January 17, 2022 5:13 PM To: PAUL R BARFORD mailto:p...@cs.wisc.edu>> Cc: Pengxiong Zhu mailto:pzhu...@ucr.edu>>; nanog@nanog.org<mailto:n
RE: Simplified BGP peering solution
4 upstream ISP connections (2 commercial, 1 NREN, 1 partial feed from HE) 1 local IX (MBIX), so 5 more ASNs there (including HE, above) plus all the open-peering ASes accessible through the IX route servers 3 downstream customers with their own ASNs The two commercial ISPs are because end-user internet access here is the usual telco/cableco near-total-duopoly, and those two %$#@!!s only peer ~30ms away from me. (Interestingly, I have one ILEC but two cablecos, geographically separated, within my service area.) The NREN should be self-evident… that’s usually over 1/3 of my bandwidth, in fact, thanks to CANARIE’s CDN network. MBIX is to cover all the non-duopoly ISPs in the province. (Guess who doesn’t show up at the IX… first infinity guesses don’t count.) So… I actually am an end-user trying to load balance. But I’m also looking for the best paths. And I have three POPs now where I meet other carriers delivering L2 circuits. Finally, I’m an ISP but not for the general public. I don’t think you can categorize the customer type effectively based on either the peering counts OR the number of “ISPs” they deal with. (Not to mention, you need to define “ISP” in order for this question to make sense in the first place.) Adam Thompson Consultant, Infrastructure Services [MERLIN] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG On Behalf Of Josh Saul Sent: Friday, February 4, 2022 8:02 PM To: nanog@nanog.org Subject: Simplified BGP peering solution How many active ISPs are most of the people on this list dealing with? 1-2 - I'm an end user just trying to load balance 3-5 - I'm aggressively looking for the best paths for my "customer" traffic 6-20 - I have a meet-me POP room or a specific business need for so many connections 21+ - I'm an ISP Thank you!
Cloudflare human?
If anyone from Cloudflare lurks here, could you please reach out to me directly? My messages to peering@ about one specific long-standing issue (and only this issue) are going unanswered with no explanation why. Thanks, -Adam Adam Thompson Consultant, Infrastructure Services [MERLIN] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/>
Proofpoint woes?
I realize this is a more networking-focused forum, but I'm wondering if anyone has contacts inside Proofpoint? They're an anti-spam service provider who has suddenly begun rejecting metric s***tons of email for (seemingly) bogus DMARC failures within the last week or so. -Adam Get Outlook for Android<https://aka.ms/AAb9ysg>
More product suggestions: small/cheap IS-IS or VXLAN devices?
At the risk of sounding like a broken record, I’m asking for product suggestions yet again: We’re wondering if anything small & cheap (think CPE-grade) exists that supports either IS-IS or VXLAN? If IS-IS, total route count it would have to carry would be small, probably in the ~500 range. If VXLAN, it needs to interoperate with Arista. If both… yay! When I say CPE-grade, I mean under C$1k (~US$800, ~€700), and can be emplaced at a customer site without any unusual infrastructure (e.g. no -48VDC power, or DIN rail mounting, non-business-office-typical). Thanks in advance, everyone. -Adam Adam Thompson Consultant, Infrastructure Services [MERLIN] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/>
RE: DMARC ViolationAS21299 - 46.42.196.0/24 ASN prepending 255 times
Tom, how exactly does someone “ride the 0/0” train in the DFZ? I’m connected to both commercial internet and NREN, and unfortunately-long paths are not uncommon in this scenario, in order to do traffic steering. If there’s another solution that affects global inbound traffic distributions, I’d love to hear about it (and so would a lot of my peers in edu). If there were a usable way to “dump” the excessively-long path only as long as a better path was already known by at least one edge router, that might be workable, but you’d have to keep track of it somewhere to reinstall it if the primary route went away… at which point you may as well have not dropped it in the first place. -Adam Adam Thompson Consultant, Infrastructure Services [MERLIN] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG On Behalf Of Tom Beecher Sent: Friday, March 25, 2022 4:13 PM To: Paschal Masha Cc: nanog Subject: Re: DMARC ViolationAS21299 - 46.42.196.0/24 ASN prepending 255 times The best practice with regards to as_path length is to have an edge filter that dumps any prefix with a length longer than say 10. Depending on the situation, might even be able to go smaller. At a certain point, keeping that route around does nothing for you, just shoot it and ride the 0/0 train. On Fri, Mar 25, 2022 at 4:09 AM Paschal Masha mailto:paschal.ma...@ke.wananchi.com>> wrote: :) probably the longest prepend in the world. A thought though, is it breaking any standard or best practice procedures? Regards Paschal Masha | Engineering Skype ID: paschal.masha - Original Message - From: "Erik Sundberg" mailto:esundb...@nitelusa.com>> To: "nanog" mailto:nanog@nanog.org>> Sent: Friday, March 25, 2022 6:43:38 AM Subject: DMARC ViolationAS21299 - 46.42.196.0/24<http://46.42.196.0/24> ASN prepending 255 times If anyone from AS21299 is lurking on Nanog. Please reduce your AS prepends for 46.42.196.0/24<http://46.42.196.0/24> from 255 prepends to a more reasonable number of prepends let's say 20. Thanks! This is a Kazakhstan register IP Block and ASN Network Next Hop Metric LocPrf Weight Path *> 46.42.196.0/24<http://46.42.196.0/24> x.x.x.x 0 100 0 2914 174 3216 3216 35168 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21 299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 21299 i CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files or previous e-mail messages attached to it may contain confidential information that is legally privileged. If you are not the intended recipient, or a person responsible for delivering it to the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of any of the information contained in or attached to this transmission is STRICTLY PROHIBITED. If you have received this transmission in error please notify the sender immediately by replying to this e-mail. You must destroy the original transmission and its attachments without reading or saving in any manner. Thank you.
RE: DMARC ViolationAS21299 - 46.42.196.0/24 ASN prepending 255 times
d? Or the 519 at 11? Remember, those are already the de-prepended paths. I don’t want to have to police the RIB that tightly, deciding which routes I will and won’t accept and adjusting the limit periodically. Unless your intent is to eliminate prepend-based traffic engineering from the internet altogether, which case 10 is a perfectly reasonable choice, but in the absence of any other globally-usable tool/knob, that’s a hill I WILL die on. If there’s broad consensus that a path length limit is a good thing, I would suggest a value of no less than 32, based on the data I’ve got in my RIB right now. I think, based only on random sampling of longer AS paths in my RIB, that 32 would still give operators the latitude to perform AS-path-based traffic engineering at origin, during transit, and locally upon receipt, without routes getting inexplicably blackholed anywhere. I’ve heard tonight that a path length limit of 32 is already commonly implemented. Regardless of whether I think that’s a good idea, the spectre of the stack-breakingly-long path seems to be already mitigated in many places, but perhaps not widely? To sum up, Tom’s conclusions as expressed in his email (below) may be qualitatively correct, but they are quantitatively wrong, at least on the matter of the numeric threshold. And… I don’t really want to be the next Sprint(?) in BGP history just to protect myself from newbies on Mikrotiks[1], do you? -Adam [1] and others, yes, I know it’s not purely a Mikrotik issue. Adam Thompson Consultant, Infrastructure Services [MERLIN] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: Tom Beecher Sent: Saturday, March 26, 2022 11:35 AM To: Adam Thompson Cc: Paschal Masha ; nanog Subject: Re: DMARC ViolationAS21299 - 46.42.196.0/24 ASN prepending 255 times Mostly what Matt said. ( I should have also said 'ride the 0/0 train INTO the DFZ, my mistake.) Essentially, if ASN X is announcing a prefix with an excessive number of prepends, they are saying to the world 'This path exists , but it is hot garbage.' I'm more than happy to oblige those instructions and just drop that announcement completely. If ASN X announces that prefix with a reasonable number of prepends, happy to take it and use it. If I get a prefix with an as-path longer than 10, (regardless of the state of prepends), I interpret that as : 1. They have made a mistake. 2. Someone else made a mistake. 3. They think that's a good idea, and have some things yet to learn. ( The old clue by four as Matt put it.) 4. It's malicious. 5. They actually are somehow more than 10 ASNs away from me. ( I don't even know if this is possible anymore without extreme effort. ) In any of those scenarios , I don't really care about optimized routing to that destination. Perfectly happy to just follow 0/0 to a DFZ upstream and let it go how it's going to go, or not. If there is a traffic impact such that an exception HAS to be made, that can be addressed as needed, but I can't think of a single circumstance I have hit where the correction involved accepting an obnoxiously long as_path announcement. ( I don't mean to imply none exist ; I'm sure someone has had to work though that. ) Maybe a length of 10 doesn't work for some environments and use cases, but I still am a firm believer that at a certain point, there is a length that becomes straight 'really don't care'. I'd rather not discover a BGP implementation problem or acid trip memory pointer party by accepting announcements that are useless. On Fri, Mar 25, 2022 at 5:56 PM Adam Thompson mailto:athomp...@merlin.mb.ca>> wrote: Tom, how exactly does someone “ride the 0/0” train in the DFZ? I’m connected to both commercial internet and NREN, and unfortunately-long paths are not uncommon in this scenario, in order to do traffic steering. If there’s another solution that affects global inbound traffic distributions, I’d love to hear about it (and so would a lot of my peers in edu). If there were a usable way to “dump” the excessively-long path only as long as a better path was already known by at least one edge router, that might be workable, but you’d have to keep track of it somewhere to reinstall it if the primary route went away… at which point you may as well have not dropped it in the first place. -Adam Adam Thompson Consultant, Infrastructure Services [MERLIN] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG mailto:merlin.mb...@nanog.org>> On Behalf Of Tom Beecher Sent: Friday, March 25, 2022 4:13 PM To: Paschal Masha
RE: DMARC ViolationAS21299 - 46.42.196.0/24 ASN prepending 255 times
> -Original Message- > From: NANOG On > Behalf Of Joe Maimon > Sent: Thursday, March 31, 2022 6:20 PM > Subject: Re: DMARC ViolationAS21299 - 46.42.196.0/24 ASN prepending 255 > times > > [...] > I think more and perhaps different knobs were and still are needed. YES. YESYESYES. Having said that, I am currently able to express my local traffic policies (roughly 1/3 due to one particular upstream hamstrung by their parent company, 1/3 due to NREN, 1/3 due to local IXP) using prepends of no more than +2 (both outbound and inbound, which I gather might be somewhat unusual). That includes dealing with a "special" issue because both my local NREN RAN and I are both connected to the local IXP, and to a mutual downstream, which produces an interesting double-triangle topology, and this topology produces surprising emergent behaviours that I haven't seen described anywhere. Prior to some reorganization (and several compromises) I needed +5 on both inbound and outbound (usually not at the same time) to adequately steer normal traffic. Prepending is, IMHO literally, the abuse of a mechanism not initially designed to express complex policies. It's a 1-dimensional tool in a space that really needs a 2- or 3-dimensional tool. LOCAL_PREF is not useful for me and my upstreams, downstreams and peers: I have not yet found a scenario where it fixes any of my problems without creating even greater ones. My traffic-steering issues ~all exist n>3 hops away, not directly with my BGP peers. So prepending remains the only useful, usable knob I have. And it's not a great knob for this, which I think might be the one thing everyone here can agree on! > through nowadays, how about a long call back to each AS in the path I'm not really loving that solution more than prepends, but at least it's something different? > Would be nice to be able to publish your community scheme as simply > conforming with RFCX and the to configure peers with process-rfcX > statement and be mostly done. That's a LONG way off! Not that any other solution isn't equally far off, so... Gotta start somewhere, I guess! I don't have any better suggestions, other than I wish the LOCAL_PREF crowd could understand that it doesn't solve all the world's problems. (Neither of my commercial upstreams currently admit to supporting communities that control it, anyway!) Frustrated at the state of the world today, -Adam Adam Thompson Consultant, Infrastructure Services MERLIN 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca www.merlin.mb.ca
RE: Opinions on Arista for BGP?
TL;DR: Yes, go ahead, they’re good, we like them. I won’t say they’re perfect, but we’re using them at the edge (two of them in a hybrid core/edge model right now, even!) and I would happily endorse them for edge routers. Their BGP stack hasn’t put up any major roadblocks for us so far (at least, that weren’t, ahem, self-inflicted). We’ve had 1 incident in the last ~2 years where a stuck route on one router needed a full reboot to clear out, following a partial outage - that’s the worst thing I can remember right now. Don’t know if you know this already or not, so making it clear: the one thing to beware of IMO, compared to e.g. a high-end Juniper MX960-style system where you can turn every single feature on without caring, is that the Aristas can do almost anything you can dream of… but not necessarily all at the same time on the same box, no matter which model you’re looking at. So if you use it as an edge router? Fine. As a VXLAN gateway? Fine. As a core router or switch with every kind of accounting turned on? Fine. All of those things simultaneously? Maybe. It’ll be decision time for which specific, individual sub-features you can live without. But you’re paying 1/10th (probably less!) of what you would for an MX960, so there you go. If this helps, they’re similar to the Cisco Nexus platform in this regard, e.g. if you enable and use every single “Feature” on the fixed-configuration Nexuses you’ll start running out of hardware configuration resources to enable them long before you can finish configuring or using all those features. This is something your Arista SE can go through with you in excruciating detail (keyword: “TCAM Profile”), if you think you might be veering into that territory. After lots of iterations, and a new software release or two, our all-in-one boxes (7280SR2K) do more or less everything we want them to. (Apparently we aren’t typical Arista customers. Go figure.) If you want to do BGP and MLAG at the same time on the same box, get your SE involved from the start. For anyone not trying to overload the platform or do too much “weird” stuff, it should be a quick and easy deployment producing much happiness. -Adam Adam Thompson Consultant, Infrastructure Services [MERLIN] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG On Behalf Of David Hubbard Sent: Thursday, March 31, 2022 8:10 AM To: nanog@nanog.org Subject: Opinions on Arista for BGP? Hi all, would love to get any current opinions (on or off list) on the stability of Arista’s BGP implementation these days. Been many years since I last looked into it and wasn’t ready for a change yet. Past many years have been IOS XR on NCS5500 platform and Arista everywhere but the edge. I’ve been really happy with them in the other roles, so am thinking about edge now. I do like and use XR’s RPL, and prefix/as/community/object sets, but we can live without via our own config management if there aren’t easy equivalents. No fancy needs at all, just small web server networks, so just need reliable eBGP and internal OSPF/OSPFv3. Thanks, David
RE: More product suggestions: small/cheap IS-IS or VXLAN devices?
Probably not an option for us, but thank you – I wasn’t aware VyOS included VXLAN. Merci, -Adam Adam Thompson Consultant, Infrastructure Services [MERLIN] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: Pierre LANCASTRE Sent: Tuesday, April 5, 2022 11:23 AM To: Adam Thompson Cc: nanog Subject: Re: More product suggestions: small/cheap IS-IS or VXLAN devices? Le mer. 23 févr. 2022 à 00:44, Adam Thompson mailto:athomp...@merlin.mb.ca>> a écrit : At the risk of sounding like a broken record, I’m asking for product suggestions yet again: We’re wondering if anything small & cheap (think CPE-grade) exists that supports either IS-IS or VXLAN? If IS-IS, total route count it would have to carry would be small, probably in the ~500 range. If VXLAN, it needs to interoperate with Arista. If both… yay! Hi Adam, A very late answer on that topic. You could look at VyOS devices compatible, VXLAN is working on the 1.4 rolling release. No idea about the timeline for this version to become the main train. We've shared an article on our lab tests of VyOS/Arista/Cumulus VX interop in a EVPN VXLAN bgp/bgp fabric, it could help you in your research : https://wiki.nesevo.com/index.php/VyosAristaCVX-VXLAN-lab When I say CPE-grade, I mean under C$1k (~US$800, ~€700), and can be emplaced at a customer site without any unusual infrastructure (e.g. no -48VDC power, or DIN rail mounting, non-business-office-typical). You should find something which fits your needs here : https://vyos.io/hcl/?vendor=edgecore-networks&vendor=lanner-electronics-inc&vendor=dell&vendor=aaeon BR, Pierre
RE: Fwd: Fw: HOST IRR Retirement
>From https://www.irr.net/docs/list.html Registry Name (Source): HOST IP address or DNS name: rr.host.net Ftp site: ftp://ftp.host.net/host/dbase Databases Mirrored: ALTDB, AOLTW, APNIC, ARIN, EPOCH, LEVEL3, PANIX, RADB, REACH, RIPE Mirror Port and Info: rr.host.net, port 43 Whois Location: rr.host.net Type of Primary Data: Host.net customers and general Internet community Contact Info: db-ad...@host.net NOC Info: n...@host.net The better question is, how do you authenticate Ross' email, and how do you authenticate the... not the contents, but the authenticity & accuracy of the meaning of the email? I suppose you don't have to - at some point they'll just turn it off and then we'll all know the email was legit :-). As to Ross' question: IIRC there exists, or existed at one time, a private(?) mailing list for IRR operators. I would start with r...@merit.edu, if that address still works. I can also reach out to the folks running the CANARIE IRR to see if they know, if necessary. -Adam Adam Thompson Consultant, Infrastructure Services MERLIN 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca www.merlin.mb.ca > -Original Message- > From: NANOG On > Behalf Of babydr DBA James W. Laferriere > Sent: Monday, April 11, 2022 3:50 PM > To: Ross Tajvar > Cc: North American Network Operators' Group > Subject: Re: Fwd: Fw: HOST IRR Retirement > > Hello Ross , I do not see a host name or IP4or6 in the below . > Hth , JimL > > On Mon, 11 Apr 2022, Ross Tajvar wrote: > > I tried sending the below message from my work account, but it's not a > > nanog subscriber, so the email was rejected. If anyone doubts the > > authenticity, feel free to reach out to me at rtaj...@365datacenters.com. > > > > > > -- > > *From:* Ross Tajvar > > *Sent:* Monday, April 11, 2022 3:53 PM > > *To:* nanog@nanog.org > > *Subject:* HOST IRR Retirement > > > > Hi all, > > > > We (365 Datacenters) inherited the HOST IRR. We have removed all stale > > objects from it, and moved all valid objects to other IRRs. We will > > eventually (hopefully soon) turn it off altogether. Please, if you are > > mirroring it, stop doing that. If you maintain documentation that lists > > IRRs, please update it to reflect that HOST is no longer in use. > > > > Thanks! > > > > P.S. If anyone thinks I should also announce this somewhere else, please > > let me know. > > > > *Ross Tajvar* > > > > Network Engineer > > > > Office: (571)-341-8899 > > > > Support: (866)-365-6246 > > > > -- > +-+ > | James W. Laferriere| SystemTechniques | Give me VMS | > | Network & System Engineer | 3237 Holden Road | Give me Linux | > | j...@system-techniques.com | Fairbanks, AK. 99709 | only on AXP | > +-+
RE: fs.com Ethernet switches
One of my clients deployed S3900s, both 24- and 48-port copper models, across half a dozen sites, and I did 99% of the config. Theoretically the 5800s are just a faster/beefier version but haven’t seen them in person. They… work, more or less. Some of the hardcoded limits are just stupid, like max 32 DHCP-relayed devices per L3 interface/VLAN and the 33rd client just doesn’t get DHCP. Either the intermediate carrier out there is stripping VLAN tags, or there’s something really weird with their trunking, not sure which yet. Both the GUI and CLI are required to configure a switch in practice – perhaps you can use the CLI exclusively if you’re an expert, but holy cow some of the config language is radically unintuitive. OTOH, even basic models have some advanced features like EAPS/ERPS in the base system. It’s very clear to me from the capabilities and language used in the original OS release that this model, at least, was originally targeting ILECs almost exclusively (e.g. console port == craft interface). Newer software releases have made them a little less obscure or difficult to work work. I can’t quite say “don’t buy them” but I sure wouldn’t recommend them, either. Broadly put: you get what you pay for! -Adam Adam Thompson Consultant, Infrastructure Services [MERLIN] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> www.merlin.mb.ca<http://www.merlin.mb.ca/> From: NANOG On Behalf Of Richard Angeletti Sent: Wednesday, April 13, 2022 2:11 PM To: nanog@nanog.org Subject: fs.com Ethernet switches Wondering if anyone on the list has any experiences with fs.com<http://fs.com> Ethernet switches that they are willing to share (good or bad)? We're looking for some cost effective L2 only 10Gb-T switches and their S58XX switches have come up as a potential option. Thanks, Rich
RE: Announcement of Experiments
I am not claiming any of this is official MERLIN position on the matter, these are merely my thoughts so far based on the incomplete knowledge & data I have: IMHO, it's somewhat the same as if I made public statements that started with "Well, I talked to Randy Bush and he said ". I'm clearly the one articulating that sentence, but I'm nonetheless attributing to you something that is (presumably) false. This will, I think, taint historical time-series data (e.g. RIPEStat) for any ASNs the experimenters use, and I could easily see in my organization being called upon to ask "Why were we transiting x.x.x.x/y in May 2022?" and not having any answer. The operational impact will probably be somewhere between zero and negligible, assuming the experiment is run correctly, but operational impacts aren't the only impacts: reputational risks are very important to some organizations. In addition to people not fully understanding AS_PATH, which even here will be a non-zero number, there will also be a number of people (myself included in this number) who have no idea what the PEERING testbed is, nor how it works, nor the effects it can produce. I'm in alignment with several other commenters in that I should not have to go spend time to learn about Yet Another Piece of Technology just to assess the risks, operational and reputational, I now face. >From my limited understanding of the experiment, I agree that opt-in would >kind of defeat the purpose, but at the same time, the opt-out email bordered >on insulting/careless: "hey, we're going to simulate a crime scene with your >fingerprints unless you tell us not to within a week" wouldn't fly most >places. If they had run their experiment without telling anyone, possibly 5 >or 10 people/orgs worldwide would have noticed, assumed someone was doing >something naughty (or incompetent), and gone on with their lives. But no >notice would arguably have been even more wrong than the notice we did get >here. Is it possible to run such an experiment ethically without tainting the data in advance by announcing it? I don't know. Adam Thompson Consultant, Infrastructure Services MERLIN 100 - 135 Innovation Drive Winnipeg, MB R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) https://www.merlin.mb.ca Chat with me on Teams: athomp...@merlin.mb.ca > -Original Message- > From: NANOG On > Behalf Of Randy Bush > Sent: Monday, May 2, 2022 3:50 PM > To: Alexandros Milolidakis > Cc: nanog@nanog.org > Subject: Re: Announcement of Experiments > > > We are a group of researchers from the KTH Royal Institute of Technology > > (Sweden). > > > > Starting from May 9 until May 31, we plan to conduct a research study > > involving AS-PATH poisoning to measure how reliable route collectors > > are to report BGP poisoned routes. > > > > We will use the PEERING Testbed [1] to announce the following two > > prefixes: > > > > - 184.164.236.0/24 > > > > - 184.164.237.0/24 > > > > for our AS-path poisoning experiments. > > > > The above experimental prefixes do not host any production services, > > hence user traffic will *not* be affected. > > > > Furthermore, we will always start the AS-PATH with the correct ASN as the > > origin. > > > > Lastly, to keep the AS-PATH short, we will announce no more than four > > Poisoned ASNs per announcement. The frequency of the announcements > > will not exceed four per hour. > > seems quite harmless. though i am sure folk who do not really > understand AS_PATH will get their nickers in a twist. > > randy
RE: 10 Do's + Don'ts for Visiting Québec + Register Now for N85!
I think I have actually heard « tire-toi une bûche » before! But it was as a child, visiting our annual Fête du Voyageur historical re-enactment, and certainly not in any normal day-to-day setting. I’m just happy that an American author (who quite likely has never been to Montreal), writing for an overwhelmingly-American audience, recognized that we have separate cultures in the first place. Meanwhile, one thing I haven’t seen mentioned anywhere yet is ArriveCAN[1] and/or eTAs[2]. If you’re trying to enter Canada right now you must use the ArriveCAN system before you get to the border, or you’ll likely be denied entry. That means doing it before you get on the plane, not after you land. Anyone entering Canada using a Canadian or American passport should not need an eTA (Electronic Travel Authorization, sort of an e-visa), but pretty much everyone else does (e.g. passports from Mexico or anywhere in the Caribbean, in this case). Absolutely check this out for yourself, links are below, I am not guaranteeing in any way the accuracy, nor the durability, of what I’ve written here. References: [1] COVID-19: Use ArriveCAN to enter Canada - Canada.ca<https://www.canada.ca/en/public-health/services/diseases/coronavirus-disease-covid-19/arrivecan.html> [2] Electronic Travel Authorization (eTA) - Canada.ca<https://www.canada.ca/en/immigration-refugees-citizenship/services/visit-canada/eta.html> General info: Visit Canada - Canada.ca<https://www.canada.ca/en/immigration-refugees-citizenship/services/visit-canada.html?outside> Official landing page: Welcome / Bienvenue | Foreign Affairs, Trade and Development Canada / Affaires Étrangères, Commerce et Développement Canada (canadainternational.gc.ca)<https://www.canadainternational.gc.ca/> I already wrote all of this up for a conference based in Ottawa, that sees a large qty. of int’l visitors from around the world: BSDCan 2022 - Travel<https://www.bsdcan.org/2022/travel.php#mozTocId145341> Travelers from the US generally don’t have the kind of issues at customs I’ve described there. -Adam P.S. ArriveCAN is a pain to Canadians, too, I can’t just pop across the border for shopping trips whenever I want any more without planning for it in advance. Yeah, I know, first-world problems… Adam Thompson Consultant, Infrastructure Services [MERLIN] 100 - 135 Innovation Drive Winnipeg, MB R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) https://www.merlin.mb.ca<https://www.merlin.mb.ca/> [cid:image002.png@01D8614A.053D7EA0]Chat with me on Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca> From: NANOG On Behalf Of J EMail Sent: Friday, May 6, 2022 8:00 AM To: Nanog News Cc: nanog@nanog.org; nanog-annou...@nanog.org Subject: Re: 10 Do's + Don'ts for Visiting Québec + Register Now for N85! On Thu, 5 May 2022 at 08:57, Nanog News mailto:n...@nanog.org>> wrote: 10 Do's + Don'ts for Visiting Québec NANOG 85 Meeting Will Take Place Jun. 6 - 8 in Montréal We are delighted to cross international borders in our mission to grow, inspire + profoundly build the Internet of tomorrow! Montréal is Canada's second-largest city and is known for its melting pot of diverse culture, established universities, enthralling art, food, history + festivals. It has been called one of the world's "happiest locations" as an estimated 45,000 immigrants relocate to the city every year. For those who don't call Québec home, we have prepared a list of cultural "Do's and Don'ts" to help you quickly acclimate + thrive in this foreign destination. READ MORE <https://www.nanog.org/stories/10-dos-dont-of-visting-québec/> I have lived ten minutes from Quebec and two hours from Montreal for a long time, I have never encountered either item 1 or item 2. Of course, there might be a place that won't take a credit card, but your credit card company will charge you a fee and be happy to use a terrible exchange rate as will restaurants if you pay in US cash. Consider getting cash from your bank account at an ATM at a bank once you land although there are fees there as well. If you are a TD bank customer, TD is a Canadian Bank and that will eliminate a fee. As for culture, smoked meat, bagels (they are different) and poutine should be on this list.
Juniper MX204 allow oversubscription?
Hi all, Hoping some Juniper-using folks know: On the MX204, which comes with 4x100G + 8x10G ports, you can only use 3 of the 4 100G ports if you want to use any of the 10G ports at all. Supposedly this is to prevent oversubscription on what is a 400G-rated router. However, I’m perfectly fine with oversubscription with a 400G aggregate throughput cap. Is there a way to stop the automatic “oh, I’ll disable these other ports for you so you don’t oversubscribe the box” behaviour and let all the front-panel ports be used at once? -Adam Adam Thompson Consultant, Infrastructure Services [MERLIN] 100 - 135 Innovation Drive Winnipeg, MB R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) https://www.merlin.mb.ca<https://www.merlin.mb.ca/> [cid:image002.png@01D8691C.50B29030]Chat with me on Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca>
RE: BGP Javascript Map/Visualization
Neat. Any idea who to ask questions of, regarding the incorrectness of the data? I would have assumed Job, but he's long gone from NTT, is this abandonware or maintained? Anyone know? Adam Thompson Consultant, Infrastructure Services MERLIN 100 - 135 Innovation Drive Winnipeg, MB R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) https://www.merlin.mb.ca Chat with me on Teams: athomp...@merlin.mb.ca > -Original Message- > From: NANOG On Behalf > Of b...@uu3.net > Sent: Thursday, May 26, 2022 3:50 PM > To: nanog@nanog.org > Subject: Re: BGP Javascript Map/Visualization > > You were close... I think you mean this one? > https://as2914.net/ > > -- Original message -- > > From: Brian Johnson > To: nanog@nanog.org > Subject: BGP Javascript Map/Visualization > Date: Thu, 26 May 2022 20:07:51 + > > Hey all, > > Sorry for the noise. Years ago someone here built and shared a > javascript > visualization of what their routers saw for the state of BGP and paths > to > get to various ASs. One could use WSAD and other keys to fly around > and > examine various other ASs. I thought it was as2814.net, but that > seems to > not be a thing anymore. Can someone refresh my memory where that > might be? > Or did it get taken down? > > Thanks in advance, > - brian > > > >
RE: Upstream bandwidth usage
On DOCSIS systems, upload/download ratios are frequency-mapped timeslot ratios that are not adjustable in real-time. On xDSL systems, upload/download ratios are - VERY roughly speaking - a function of how much spectrum is allocated to each direction based on each individual line's characteristics, and also not adjustable in real-time. Most fixed-wireless systems have similar limitations to one or the other above, although they vary in the details. Anecdote: I used to maintain/sell/support a system that could automatically "tune" DSL service for the prevailing line conditions (as these change with age, weather, interference, etc.) and I recall learning from one customer that auto-tuning any more often than every 24hrs became *severely* counter-productive, because the connection had to drop and re-negotiate every time a change was made because of the way DSL modems work; our product had to incorporate a fall-back where we reverted to ADSL 1M rates if the line was still down an hour later, in case the remote modem just refused to renegotiate at what should have been a perfectly valid profile (speed) for some reason or other. So the short answer is: because the harder limitation to solve is the last-mile technology, not the choke-points at the network edges where shaping happens. All that rate-shaping in the core is generally about preventing the downstream packet(s) that would overload the last-mile from ever reaching the last-mile device in the first place. However, if you're talking about fiber service, it's pretty much pure marketing-dept-driven BS, combined with some vague justification of not letting TOR nodes or copyright-ignoring seeders/Warez-providers/etc. overwhelm the network in unexpected ways. -Adam (who acknowledges he runs a very unusual SP network where rate-limiting is unheard of) Adam Thompson Consultant, Infrastructure Services MERLIN 100 - 135 Innovation Drive Winnipeg, MB R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) https://www.merlin.mb.ca Chat with me on Teams: athomp...@merlin.mb.ca > -Original Message- > From: NANOG On Behalf > Of Michael Thomas > Sent: Thursday, June 9, 2022 3:46 PM > To: nanog@nanog.org > Subject: Re: Upstream bandwidth usage > > > On 6/9/22 1:26 PM, Mel Beckman wrote: > > With 430 GB versus 32 GV average down versus up usage today, > according > > to your article, this is still not a case for symmetrical consumer > > bandwidth. Yes, the upstream usage increased slightly more than the > > downstream usage. But the ratio was still so big that it would take > > decades for them to join. I doubt they ever will. Consumers just > don’t > > have that much days up to push yet, and probably never will. > > > > Also, a lot of that Usage can be explained by video conferencing > > during Covid, which has dropped off significantly already. > > > > > If it's so tiny, why shape it aggressively? Why shouldn't I be able to > burst to whatever is available at the moment? I would think most users > would be happy with that. > > Mike
Re: Upstream bandwidth usage
Ah, I did miss that, you're right. We don't have very much GPON up where I am. -Adam Get Outlook for Android<https://aka.ms/AAb9ysg> From: Mel Beckman Sent: Thursday, June 9, 2022 6:31:34 PM To: Adam Thompson Cc: Michael Thomas ; nanog@nanog.org Subject: Re: Upstream bandwidth usage Adam, Your point on asymmetrical technologies is excellent. But you may not be aware that residential optical fiber is also asymmetrical. For example, GPON, the latest ITU specified PON standard, and the most widely deployed, calls for a 2.4 Gbps downstream and a 1.25 Gbps upstream optical line rate. -mel > On Jun 9, 2022, at 3:08 PM, Adam Thompson wrote: > However, if you're talking about fiber service, it's pretty much pure > marketing-dept-driven BS, combined with some vague justification of not > letting TOR nodes or copyright-ignoring seeders/Warez-providers/etc. > overwhelm the network in unexpected ways.
RE: Serious Juniper Hardware EoL Announcements
[Not specific to the Juniper EoLs...] I sort of agree with Mark: I've been sampling a fairly wide variety of sources in various parts of the global supply chain, and my synthesis of what they're saying is that we probably won't *consistently* have the ready availability of "stuff" (both electronic and not) we had pre-pandemic, for the rest of my career (10-15yrs), and maybe not in the lifetimes of anyone reading this today, either. Whether those sources are accurate, their interpretation is accurate, my synthesis is accurate, whether I'm listening to the right people in the first place... all debatable. I sure hope the above conclusion is wrong. One possible upside: it might slow down the incessant upgrade hamster-wheel we're all running on? Imagine having enough time to do your job thoroughly and properly... Yes, I know I'm dreaming :-). Adam Thompson Consultant, Infrastructure Services MERLIN 100 - 135 Innovation Drive Winnipeg, MB R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) https://www.merlin.mb.ca Chat with me on Teams: athomp...@merlin.mb.ca > -Original Message- > From: NANOG On Behalf > Of Mark Tinka > Sent: Tuesday, June 14, 2022 11:19 AM > To: nanog@nanog.org > Subject: Re: Serious Juniper Hardware EoL Announcements > > > > On 6/14/22 18:06, JASON BOTHE via NANOG wrote: > > > Saw this coming a mile away. With chips and technology progressing > despite ability to manufacture, I’m certain many are going to do this. > > All this will do is keep these boxes off the open market, which will > simply bump up open market prices, with no incentive for the majority > of > folk to buy directly from the OEM. > > I suspect supply chain will improve within the next 12 months, but > then > regress and hit a massive crunch from around Q4'23 onward. How long > for, > I can't say... > > Mark.
GoDaddy intermittent unreachability?
Does anyone from GoDaddy lurk here? We’re getting multiple reports of GoDaddy-hosted websites being intermittently unreachable, but inconsistently – one IP can load a site, the next IP can’t, even two different outbound NAT addresses on the same link will have different results. This smells like a hashing-involved problem to me, which will be (already is) difficult to troubleshoot. (There’s my shot in the dark for today!) Thanks, -Adam Adam Thompson Consultant, Infrastructure Services [MERLIN] 100 - 135 Innovation Drive Winnipeg, MB R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) https://www.merlin.mb.ca<https://www.merlin.mb.ca/> [cid:image002.png@01D88172.DE1E4450]Chat with me on Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca>
RE: Re:
I run both OpenBSD + OpenBGPd + OpenBSD/OpenBGPd’s LG, and BIRD + xddxdd/bird-lg-go<https://github.com/xddxdd/bird-lg-go> (on two different servers, because I value my sanity) because they do a few things differently, and neither can show me everything I want. -Adam Adam Thompson Consultant, Infrastructure Services [MERLIN] 100 - 135 Innovation Drive Winnipeg, MB R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) https://www.merlin.mb.ca<https://www.merlin.mb.ca/> [cid:image002.png@01D88572.EF7703F0]Chat with me on Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca> From: NANOG On Behalf Of Michel Blais Sent: Monday, June 20, 2022 7:45 PM To: North American Network Operators' Group Subject: Re: Several seems to use OpenBSD with OpenBGP and BGPLG. Le lun. 20 juin 2022 à 17:08, J. Hellenthal via NANOG mailto:nanog@nanog.org>> a écrit : It's not about what you use as aposed more of where it's used from. -- J. Hellenthal The fact that there's a highway to Hell but only a stairway to Heaven says a lot about anticipated traffic volume. On Jun 20, 2022, at 13:47, Josh Luthman mailto:j...@imaginenetworksllc.com>> wrote: I use Cogent: https://www.cogentco.com/en/looking-glass and HE which is easier to remember: https://lg.he.net/ On Mon, Jun 20, 2022 at 9:56 AM Glenn Kelley wrote: Good Monday Morning Everyone. Quick Question: What is everyone's favorite software for running a looking glass. A friend asked me this over the weekend - and while there are others available on the internet to use - it would be helpful for them to run one within their own network. It has been a while since i have played setting one up so figured might as well ask Glenn S. Kelley, Connectivity.Engineer Text and Voice Direct: 740-206-9624 Error! Filename not specified.
RE: Papers/analysis on network equipment pricing since pandemic/banning foreign competition
If you look at the NANOG archives, this was discussed just a month or so ago, somewhere in the middle of the Juniper MX204 EOL discussion, IIRC. I’ve been seeing a number of analysts with models which show a brief reprieve in 2023 then a regression (i.e. back to where we are today, or even worse) from there out for another 4-5yrs, and no-one’s putting much credibility in models much past 5yrs right now. Which is good, because some of those models say we’re screwed for the rest of our lifetimes. I don’t know the details of why, although I do recall that some of it is based on queueing theory, so the ultra-short hyper-efficient JIT pipelines our society has spent the last 50yrs building likely play a part. FWIW, I saw something about vehicle manufacturing where a factory rep was proudly proclaiming that they’d optimized the JIT process to where they needed some specific part – and engine, I think? – delivered from the part manufacturer twice daily... seemingly completely oblivious to the supply-chain risks involved. -Adam From: NANOG On Behalf Of Drew Weaver Sent: July 18, 2022 8:51 AM To: 'Mel Beckman' ; 'Forrest Christian (List Account)' Cc: 'nanog list' Subject: RE: Papers/analysis on network equipment pricing since pandemic/banning foreign competition Okay so I suppose how many years are we allocating mentally for them to take any action? From: Mel Beckman mailto:m...@beckman.org>> Sent: Saturday, July 16, 2022 8:37 PM To: Forrest Christian (List Account) mailto:li...@packetflux.com>> Cc: Drew Weaver mailto:drew.wea...@thenap.com>>; nanog list mailto:nanog@nanog.org>> Subject: Re: Papers/analysis on network equipment pricing since pandemic/banning foreign competition Drew, The YouTube channel Asianometry has some good insights into the underlying supply chain problems: https://youtu.be/YJrOuBkYCMQ<https://urldefense.proofpoint.com/v2/url?u=https-3A__youtu.be_YJrOuBkYCMQ&d=DwMGaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=350jBt4lV5__sRM-o8xd0-5dZj5eKjEXHN6-U7rt6d0&s=td-RBYh4ifeLsrJAI4k6BLe9JWP8QXc97lGK0lWbKJM&e=> Deposits that the issue isn’t with leading as chips, as you might think, but with so-called “trailing edge“ chips: microprocessors and support circuits that make up the bulk of electronic components, including routers and switches. Asianometry points out that trailing edge components account for 50% of sales in the chip market, and given their much lower price of just a few dollars each, they represent many times as many units. This makes them a much larger factor in the supply chain problem. Everything is finally keyed to “just in time“ manufacturing, with very little component supply in the pipelines. When Covid hit, those pipelines dried up and everything collapsed. -mel beckman On Jul 16, 2022, at 4:19 PM, Forrest Christian (List Account) mailto:li...@packetflux.com>> wrote: The underlying problem is silicon Fab capacity. It has nothing to do with the actual manufacturing of the products once all the components arrive. If you don't have components for your product, a new order for components will take a year to arrive just because the factories that turn raw silicon wafers into computer chips are backlogged 12 to 18 months. Note that all of the existing factories are running around the clock, and although additional facilities are being built it takes time for them to be completed. I'm also assuming that there has been some question about whether this is short term or long term demand which would influence whether you spend a billion dollars or more building a new fab. On Fri, Jul 15, 2022, 11:57 AM Drew Weaver mailto:drew.wea...@thenap.com>> wrote: Has anyone seen any interesting write ups or analysis on what has been happening with network equipment pricing and availability in the United States over the last couple of years? Everyone (or at least I did) knew by March 2020 that what manufacturers were doing wasn’t really going to work anymore going forward and yet it’s 2 1/3 years later and we’re still looking at a 1+ year lead time on basic products. Not to be glib but I am pretty sure I could’ve devised a process to build ethernet switches by hand with a soldering iron by now. Not to mention how many additional supply chains could’ve been established since then. Did Huawei actually serve an important purpose in the market? Did we put too many eggs in the Broadcom basket? Did the industry come together and “agree” that the time is nigh to charge $16,000 for a Dell S4148T but also that it would take 9 months to get it at that price? I think it would be fascinating to learn more about what is happening in this market. Please share any resources on or off-list. Thanks and have a great weekend! -Drew
RE: End of Cogent-Sprint peering wars?
Well, yes, that goes hand-in-hand with "...expects hefty charge". To me, this just says T-Mobile wants out of the POTS business at almost any cost. All those poor people stuck with Cogent now, I feel sorry for them! Adam Thompson Consultant, Infrastructure Services MERLIN 100 - 135 Innovation Drive Winnipeg, MB R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) https://www.merlin.mb.ca Chat with me on Teams: athomp...@merlin.mb.ca > -Original Message- > From: NANOG On > Behalf Of Jawaid Bazyar > Sent: September 7, 2022 5:04 PM > To: Dave Taht ; Sean Donelan > Cc: NANOG > Subject: Re: End of Cogent-Sprint peering wars? > > $1 deals usually come with an operation in the red, or assumption of > significant debts. > > On 9/7/22, 2:55 PM, "NANOG on behalf of Dave Taht" bounces+jbazyar=verobroadband@nanog.org on behalf of > dave.t...@gmail.com> wrote: > > On Wed, Sep 7, 2022 at 1:48 PM Sean Donelan > wrote: > > > > > > Are Sprint AS1239 and Cogent AS174 finally going to settle > their peering > > disputes? > > > > T-Mobile sells legacy Sprint wireline business to Cogent for > $1, expects > > hefty charge > > https://www.reuters.com/markets/deals/cogent-communications- > acquire-t-mobiles-wireline-business-2022-09-07/ > > > > 1,400 customers > > 1,300 employees > > > > 19,000 long-haul route miles > > 1,300 metro route miles > > 16,800 leased route miles > > That's a dollar well spent. It also explains the layoffs. > > > > > -- > FQ World Domination pending: > https://blog.cerowrt.org/post/state_of_fq_codel/ > Dave Täht CEO, TekLibre, LLC >
Re: End of Cogent-Sprint peering wars?
1-800-FTC-HELP (Doesn't exist but should) -Adam Get Outlook for Android<https://aka.ms/AAb9ysg> From: Lou D Sent: Wednesday, September 7, 2022 5:49:25 PM To: Adam Thompson Cc: Dave Taht ; Jawaid Bazyar ; NANOG ; Sean Donelan Subject: Re: End of Cogent-Sprint peering wars? Is there a hotline for those of us who have Cogent and Sprint….. sighs On Wed, Sep 7, 2022 at 6:17 PM Adam Thompson mailto:athomp...@merlin.mb.ca>> wrote: Well, yes, that goes hand-in-hand with "...expects hefty charge". To me, this just says T-Mobile wants out of the POTS business at almost any cost. All those poor people stuck with Cogent now, I feel sorry for them! Adam Thompson Consultant, Infrastructure Services MERLIN 100 - 135 Innovation Drive Winnipeg, MB R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) https://www.merlin.mb.ca Chat with me on Teams: athomp...@merlin.mb.ca<mailto:athomp...@merlin.mb.ca> > -Original Message- > From: NANOG > mailto:merlin.mb...@nanog.org>> > On > Behalf Of Jawaid Bazyar > Sent: September 7, 2022 5:04 PM > To: Dave Taht mailto:dave.t...@gmail.com>>; Sean Donelan > mailto:s...@donelan.com>> > Cc: NANOG mailto:nanog@nanog.org>> > Subject: Re: End of Cogent-Sprint peering wars? > > $1 deals usually come with an operation in the red, or assumption of > significant debts. > > On 9/7/22, 2:55 PM, "NANOG on behalf of Dave Taht" bounces+jbazyar=verobroadband@nanog.org<mailto:verobroadband@nanog.org> > on behalf of > dave.t...@gmail.com<mailto:dave.t...@gmail.com>> wrote: > > On Wed, Sep 7, 2022 at 1:48 PM Sean Donelan > mailto:s...@donelan.com>> > wrote: > > > > > > Are Sprint AS1239 and Cogent AS174 finally going to settle > their peering > > disputes? > > > > T-Mobile sells legacy Sprint wireline business to Cogent for > $1, expects > > hefty charge > > https://www.reuters.com/markets/deals/cogent-communications- > acquire-t-mobiles-wireline-business-2022-09-07/ > > > > 1,400 customers > > 1,300 employees > > > > 19,000 long-haul route miles > > 1,300 metro route miles > > 16,800 leased route miles > > That's a dollar well spent. It also explains the layoffs. > > > > > -- > FQ World Domination pending: > https://blog.cerowrt.org/post/state_of_fq_codel/ > Dave Täht CEO, TekLibre, LLC >
NANOG 86 - Hotel Group Rate
The group rate of $250 USD/night is full and no longer available. Online availability is $355 USD/night. But if you call, wait on hold, and ask for the AAA discount, the rate is $305 USD/night. Thought I would share as it may help others save a little money. AK
RE: [External] Normal ARIN registration service fees for LRSA entrants after 31 Dec 2023 (was: Fwd: [arin-announce] Availability of the Legacy Fee Cap for New LRSA Entrants Ending as of 31 December 20
I believe that’s known as “saying the quiet part out loud”, Tom 😊. Without stating my own opinion, I merely observe that I know a great many people who agree with you, and a not-insignificant number who vehemently disagree with you. Most of the conclusions to be drawn there are obvious, but humans can be infinitely surprising… Whether ultimately good or bad, IANA decreeing “there’s no such thing as legacy any more” would absolutely simplify the future governance and administration of the RIRs, especially ARIN. I don’t see it happening, albeit for political reasons instead of legal. And I, as a Canadian citizen, have approximately zero influence over IANA and the organizations to which it is beholden, as they’re still, ultimately, mostly, arms of the U.S. government in one for or another. I’d even be happy if ARIN were able to implement a validation/verification process for legacy assignments, as I’m aware of a decent number of abandoned legacy blocks currently being squatted on by [usually] WISPs who very definitely do NOT have the right to do so, but I cannot provide any proof of that so nothing can be done. My own, VERY informal research suggests that while legacy blocks are around 34% of 0/0, between 1/5 and 1/3 of legacy blocks themselves (mostly the old Class-C blocks) are abandoned, and frequently being squatted-on. Some of those blocks are assigned to my clients, most of whom would probably be happy to transfer, or (gasp) lease those IPs to the current, probably-illegitimate, users – public schools can always use a bit more cash! Absent a clean-up effort, however, with appropriate policy supporting it, we’re stuck with the status-quo. -Adam Adam Thompson Consultant, Infrastructure Services [MERLIN] 100 - 135 Innovation Drive Winnipeg, MB R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) https://www.merlin.mb.ca<https://www.merlin.mb.ca/> [cid:image002.png@01D8CAF7.9540E2A0]Chat with me on Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca> From: NANOG On Behalf Of Tom Beecher Sent: September 17, 2022 10:19 AM To: John Curran Cc: North American Network Operators' Group Subject: Re: [External] Normal ARIN registration service fees for LRSA entrants after 31 Dec 2023 (was: Fwd: [arin-announce] Availability of the Legacy Fee Cap for New LRSA Entrants Ending as of 31 December 2023) I would honestly love it if IANA was able to say "As of X date, all LEGACY IPv4 allocations are transferred to the RIRs . Assignees will not change, but will now need to comply with each RIRs policies." Of course this will never happen, because it would just be a flood of billable hours, lawsuits, and injunctions, where companies will claim 'intellectual property' over something they didn't develop. It's exhausting to watch this two tiered system where the legacy holders bleat about what the rules should be for the rest of us, while they can do whatever the heck they want, simply because they had the foresight to exist at the right time. On Sat, Sep 17, 2022 at 10:41 AM John Curran mailto:jcur...@arin.net>> wrote: On 16 Sep 2022, at 10:11 PM, John Gilmore mailto:g...@toad.com>> wrote: John Curran mailto:jcur...@arin.net>> wrote: ... the long-term direction is to provide the same services to all customers under the same agreement and fees – anything else wouldn’t be equitable. There are many "anything else"s that would indeed be equitable. It is equitable for businesses to sell yesterday's bread at a lower price than today's bread. Or to rent unused hotel rooms to late-night transients for lower prices than those charged to people who want pre-booked certainty about their overnight shelter. ARIN could equitably charge different prices to people in different situations; it already does. And ARIN could equitably offer services to non-members, by charging them transaction fees for services rendered, rather than trying to force them into a disadvantageous long term contract. Please don't confuse "seeking equity" with "forcing everyone into the same procrustean bed". John - ARIN can most certainly charge different fees for different customers – we’re actually doing exactly that today for all of the legacy resource holders who have entered an agreement with ARIN already or who choose to do so in the coming year. Rather than paying the same registration service fees as everyone else, they have a cap on their total registry maintenance fees (presently $150 per year, subject to an increase $25 per year) which is a unique fee benefit that’s been provided only to the legacy resource holders. The announcement just made is that we will cease offering this fee cap for legacy resource holders who sign an agreement after 31 Dec 2023; i.e. they will pay the same fees as everyone else based on total resources held. As others have already
RE: any dangers of filtering every /24 on full internet table to preserve FIB space ?
I can't find the original message, so replying to the wrong spot in the thread, but... no, filtering /24s is a bad idea if you want (more or less) all your packets to get to their destinations. If you filter all /24s you will lose reachability to 4x /24s I publish that have no covering route because they are not contiguous and not part of any larger logical aggregate. Then there's the 10-20 legacy /24s I *don't* currently publish - if I start advertising them, you won't be able to reach them, either, because they're in the same boat: discontiguous singletons. There are a LOT of legacy discontiguous IPv4 singletons assigned out of the old Class-C space to small/medium businesses, schools, etc. in the pre-ARIN days, and I would guess that the vast majority of them do not have a correct covering /23 or larger - certainly none of the ones I'm currently working with/aware of do. I believe there's at least a couple of DNS servers running in my /24s, so you could potentially lose access to much more than those /24s. Your packet will *probably* hit a next-hop carrier who happens to have the more-specific /24, and it will *probably* eventually reach me, but I thought everyone more-or-less agreed that internet router was already nondeterministic enough as it is? IMHO, if you don't want all the /24s in your FIB (or even RIB!), just pick a carrier, set a default route, and stop worrying about all the headaches BGP provides. Alternately, a valid technique is to have a default route AND a partial BGP feed (a filtered full feed is by definition a partial feed). That helps optimize outbound routing a little bit, you still get the advantage - mostly - of multiple inbound carriers; but you still have to pick one carrier to do the heavy lifting for you. And you are paying them to route for you, so that's not an unfair shifting of the routing burden, unlike relying on covering routes. Note that this approach does NOT provide any redundancy, unlike having full BGP feeds. Separately, I don't know if Geoff has produced such a survey/article, but if not he can probably type it from memory by now :-). Adam Thompson Consultant, Infrastructure Services MERLIN 100 - 135 Innovation Drive Winnipeg, MB R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) https://www.merlin.mb.ca Chat with me on Teams: athomp...@merlin.mb.ca > -Original Message- > From: NANOG On Behalf Of > Stephane Bortzmeyer > Sent: October 10, 2022 10:21 AM > To: Edvinas Kairys > Cc: NANOG Operators' Group > Subject: Re: any dangers of filtering every /24 on full internet table to > preserve FIB space ? > > On Mon, Oct 10, 2022 at 05:58:45PM +0300, > Edvinas Kairys wrote > a message of 35 lines which said: > > > But theoretically every filtered /24 could be routed via smaller > > prefix /23 /22 /21 or etc. > > I don't think this is true, even in theory, specially for legacy > prefixes. There is probably somewhere a Geoff Huston survey on /24 > without a covering route.
RE: jon postel
The book, being written by an actual credentialed historian, contains their complete sources as footnotes/endnotes. That section was overwhelming, I mostly skipped it... Adam Thompson Consultant, Infrastructure Services MERLIN 100 - 135 Innovation Drive Winnipeg, MB R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) https://www.merlin.mb.ca Chat with me on Teams: athomp...@merlin.mb.ca > -Original Message- > From: NANOG On Behalf Of > Carsten Bormann > Sent: October 17, 2022 11:54 AM > To: Grant Taylor > Cc: nanog@nanog.org > Subject: Re: jon postel > > On 2022-10-17, at 16:57, Grant Taylor via NANOG wrote: > > > > In my not so humble opinion, Where Wizards Stay Up Late should be required > reading for anyone wanting to learn about the history / development of the > ARPAnet and the Internet. > > That said, it would be a worthwhile project to collect the places in which > this source can be supplemented with additional information (a.k.a. grains > of salt). > > Grüße, Carsten
RE: any dangers of filtering every /24 on full internet table to preserve FIB space ?
I can't believe that never occurred to me in all the time I was doing that, 'way back when... Thanks for pointing that out! -Adam Adam Thompson Consultant, Infrastructure Services MERLIN 100 - 135 Innovation Drive Winnipeg, MB R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) https://www.merlin.mb.ca Chat with me on Teams: athomp...@merlin.mb.ca > -Original Message- > From: NANOG On > Behalf Of Brandon Martin > Sent: October 21, 2022 4:30 PM > To: nanog@nanog.org > Subject: Re: any dangers of filtering every /24 on full internet > table to preserve FIB space ? > > On 10/20/22 17:50, Adam Thompson wrote: > > Alternately, a valid technique is to have a default route AND a > partial BGP feed (a filtered full feed is by definition a partial > feed). That helps optimize outbound routing a little bit, you still > get the advantage - mostly - of multiple inbound carriers; but you > still have to pick one carrier to do the heavy lifting for you. And > you are paying them to route for you, so that's not an unfair > shifting of the routing burden, unlike relying on covering routes. > Note that this approach does NOT provide any redundancy, unlike > having full BGP feeds. > > As a note, you can get redundancy (but still none of the best-path > advantages of having multiple transits) by asking your transits to > originate default in their BGP feed and then selectively accepting > it. > You can either ECMP it or pick priority with localpref. > > You need multiple full-view transits for this to work, though. > > -- > Brandon Martin
RE: BCP38 For BGP Customers
We’ve seen Juniper EX9ks implement uRPF in such a way that if I have two (load-balanced) BGP connections to the EX9k, and uRPF is turned on facing me, I immediately experience ~50% outbound packet loss. Methinks the EX9ks apply uRPF a little too close to the hardware and ignore the RIB. -Adam Adam Thompson Consultant, Infrastructure Services [MERLIN] 100 - 135 Innovation Drive Winnipeg, MB R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) https://www.merlin.mb.ca<https://www.merlin.mb.ca/> [cid:image002.png@01D8FE60.27B812C0]Chat with me on Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca> From: NANOG On Behalf Of Mike Hammett Sent: November 8, 2022 2:29 PM To: William Herrin Cc: nanog@nanog.org; Grant Taylor Subject: Re: BCP38 For BGP Customers "Reverse path filtering literally says don't accept a packet from somewhere that isn't currently the next hop for that packet's source address." FIB or RIB? I knew of uRPF as available over an interface, per the routing table, not best path available. Or is that implementation dependent? - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com From: "William Herrin" mailto:b...@herrin.us>> To: "Grant Taylor" mailto:gtay...@tnetconsulting.net>> Cc: nanog@nanog.org<mailto:nanog@nanog.org> Sent: Tuesday, November 8, 2022 2:01:49 PM Subject: Re: BCP38 For BGP Customers On Tue, Nov 8, 2022 at 8:40 AM Grant Taylor via NANOG mailto:nanog@nanog.org>> wrote: > Maybe it's the lack of caffeine, but would someone please remind / > enlighten me as to why uRPF is a bad idea on downstream interfaces? Hi Grant, Two words: asymmetric routing. If the downstream network is architected in such a way that there's more than one path in and out of their network then there's no way to guarantee that any particular router believes the forward path to that network goes to a particular next hop. It can currently map to any next hop that goes in the direction of one of the valid paths. That routing is completely independent of how next hops are selected in the other direction. Packets can travel in one path and out another. Reverse path filtering literally says don't accept a packet from somewhere that isn't currently the next hop for that packet's source address. When it's possible for the forward route to point a different direction than the one from which valid packets are received, that is the wrong thing to do. In fact, it breaks the network. Useful automated reverse path filtering can ONLY be used when there is exactly ONE valid path to which and from which packets can be received. This is where strict mode uRPF actually works. > N.B. Maybe I'm thinking more a varient of uRPF wherein if I have a route > to the source from the interface in question. Thus not uRPF-strict > (active route) nor uRPF-loose (route on any interface). Even if a particular router happens to have RIB entries for all the valid paths to a host (a sketchy proposition at best), only one such entry will be stored in the FIB where uRPF looks to make its filtering decision. As for loose mode, it's basically useless in a BCP38 discussion. Loose mode only filters bogons. It doesn't prevent impersonation of any IP address currently routed in the system and doesn't do anything at all on a router with a default route. Regards, Bill Herrin -- For hire. https://bill.herrin.us/resume/
Re: Google Speed Test
https://www.speedtest.cloud are also GCP endpoints. Adam On Wed, Dec 28, 2022 at 4:55 PM Dave Taht wrote: > Waveform leverages cloud flare's CDN. > > I have a worldwide fleet of iperf, netperf, flent and irtt servers folk > are welcome to use, as I don't trust the web best tests > > On Wed, Dec 28, 2022, 11:44 AM Douglas Fischer > wrote: > >> I have recollection of something like embeded quality testing on youtube. >> I don't remember if it was a speed test or a latency/jitter test. >> >> I looked quickly to see if I could find it... But I couldn't find it. >> >> Em qua., 28 de dez. de 2022 às 13:43, Mike Hammett >> escreveu: >> >>> Does AS15169 have a speed test? It would be nice to gauge the capacity >>> to a particular network that's something laypeople could do. I could host >>> something in GCP myself, but cloud is expensive. >>> >>> - >>> Mike Hammett >>> [ http://www.ics-il.com/ | Intelligent Computing Solutions ] >>> [ https://www.facebook.com/ICSIL ] [ >>> https://plus.google.com/+IntelligentComputingSolutionsDeKalb ] [ >>> https://www.linkedin.com/company/intelligent-computing-solutions ] [ >>> https://twitter.com/ICSIL ] >>> [ http://www.midwest-ix.com/ | Midwest Internet Exchange ] >>> [ https://www.facebook.com/mdwestix ] [ >>> https://www.linkedin.com/company/midwest-internet-exchange ] [ >>> https://twitter.com/mdwestix ] >>> [ http://www.thebrotherswisp.com/ | The Brothers WISP ] >>> [ https://www.facebook.com/thebrotherswisp ] [ >>> https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg ] >>> >> >> >> -- >> Douglas Fernando Fischer >> Engº de Controle e Automação >> >
AS3257 / AS4436 / GTT Contact re route hijack
Can someone from GTT contact me off-list about this? GTT is not an upstream carrier for me and the NOC doesn't seem to understand the issue Thank for your help. :) Adam On Thu, Jan 12, 2023 at 8:47 AM GTT NOC wrote: > Hello Team, > > As part of our ongoing improvements to enhance security measures for our > customers, we no longer accept Request tickets via email. We request you to > please log into our EtherVision <https://ethervision.gtt.net/> portal to > open your ticket, it will then be picked up accordingly by our engineers. > You can view a quick tutorial on our Ethervision portal here > <https://learn.gtt.net/EV_Focus_On_Ticketing.html>. > > > > EtherVision <https://ethervision.gtt.net/> also allows > you to check the status of existing tickets as well as access our Service > Assurance contact information. > > > If you do not have an EtherVision <https://ethervision.gtt.net/> login, > please contact > your GTT account administrator or submit a request through our website > <https://ethervision.gtt.net/sign-in>. > > > > Thanks & Regards, > > Danish Aga > > Service Desk Analyst > > Client Portal: https://ethervision.gtt.net/ > > GTT NOC: n...@gtt.net > > [image: image] > > -- > *From:* LionLink NOC > *Sent:* 12 January 2023 07:02 > *To:* GTT NOC > *Subject:* Route Hijack - 23.29.58.0/24 > > *NOTE:* This is an external message. Please use caution when replying, > opening attachments or clicking on any links in this e-mail. > Hello GTT NOC, > > I am unsure why, but it appears you are advertising a netblock that I am > assigned (199.116.84.0/24). Can you please stop advertising my netbook > <https://whois.arin.net/rest/net/NET-199-116-84-0-2/pft?s=199.116.84.1>? > > > > [image: Screen Shot 2023-01-12 at 1.50.35 AM.png] > [image: Screen Shot 2023-01-12 at 1.51.07 AM.png] > > > *NOTICE: This e-mail is only intended for the person(s) to whom it is > addressed and may contain confidential information. Unless stated to the > contrary, any opinions or comments are personal to the writer and do not > represent the official view of GTT Communications Inc or any of its > affiliates. If you have received this e-mail in error, please notify us > immediately by reply e-mail and then delete this message from your system. > Please do not copy it or use it for any purposes, or disclose its contents > to any other person. All quotes, offers, proposals and any other > information in the body of this email is subject to, and limited by, the > terms and conditions, signed service agreement and/or statement of work* > Very Best Regards, Adam *Adam Blackington, PSM, NSA* Managing Partner *LionLink Networks | *Infrastructure Solutions That Power The Enterprise ablacking...@lionlink.net email *| * (571) 482.1490 direct Service Desk: (844) DATA-CENTER, Option 2 or supp...@lionlink.net
RE: Can I do this in EVPN? (Multihome to more different CEs)
The solution we've deployed is to use a VXLAN termination device at each location requiring multi-path redundancy. Run VXLAN over isolated L3 domains, let IS-IS or OSPF handle path selection, including ECMP if desired. If multi-chassis redundancy is required, pick a platform that can do MLAG or similar. So for example, I have two sites with multiple VLANs needing to be interconnected, and for whatever reason I can't just use a LAG (distance, lack of transparent L2 service, whatever). We would put an Arista 7k-series pizzabox at each end, one end could be an MLAG pair. Terminate two L2 or L3 services on the singleton box, terminate each service onto one half of the MLAG pair at the other site. Run an IGP (ideally IS-IS with BFD, but YMNV) and ECMP and happens automatically, as does handling single-path failures. This could equally be a MLAG-to-MLAG setup if you have too much money and need to use some up. Cisco vPC does essentially the same thing, as does Juniper's VC. Extreme has something similar, too. STP does not get transported across the VXLAN transport, so you now avoid all the inherent problems with long-distance or multi-site STP bridging. -Adam Adam Thompson Consultant, Infrastructure Services MERLIN 100 - 135 Innovation Drive Winnipeg, MB R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) https://www.merlin.mb.ca Chat with me on Teams: athomp...@merlin.mb.ca > -Original Message- > From: NANOG On > Behalf Of Jason R. Rokeach via NANOG > Sent: February 9, 2023 1:11 PM > Cc: nanog@nanog.org > Subject: Re: Can I do this in EVPN? (Multihome to more different CEs) > > VPLS doesn't handle loop avoidance. At least, not apart from split > horizon rules. > > I assume that them properly connecting routers only and doing dynamic > routing over your service is out of the question? (Even _just_ doing > this doesn't completely solve the challenge though.) > > It sounds to me like your customer is needing two separate services. > One to provide connectivity to other sites at layer 2, and another to > provide backup connectivity within their single campus at layer 2. I > would suggest that you treat these as two separate services, because > there's nothing in EVPN that's going to notice on the PE side of the > equation that the customer has a break in the middle of their > network. > Maybe consider offering these two services in combination: > 1) layer 2 VPN service (VPWS / single pseudowire) between the two > sides of their campus. You would need to ensure L2CP transparency (or > tunneling) for STP and they would need to run STP across the link to > keep their campus whole > 2) EVPN with ESI in single-active mode, as you had mentioned. > > > > > --- Original Message --- > On Thursday, February 9th, 2023 at 11:56 AM, Simon Lockhart > wrote: > > > > > > > > > > On Thu Feb 09, 2023 at 11:54:28AM -0500, Shawn L wrote: > > > > > > You should be able to setup a VPLS between 3 (or more) devices. > Something like this -- > > > > > > > > [snip] > > > > > Thanks - I'm not committed to EVPN, so VPLS could work too. Would > VPLS > > handle loop avoidance for me? (i.e. if I have two VPLS PE > connections into > > the same broadcast domain on the customer side) > > > > > Simon > > ___ > Jason R. Rokeach > m: 603.969.5549 > e: ja...@rokea.ch > tg: jasonrokeach > > > Sent with ProtonMail secure email. Get my PGP Public Key.
RE: Standard DC rack rail distance, front to back question
Fascinating. I’ve never had an ASR-1001 come with two sets of ears, and I also note that the text of the instruction manual doesn’t reference the rear set at all. I’ve never seen rear ears on any Cisco gear of my own, nor on anything the local ILEC has installed either. I think the diagram is in error here. However, the “optional” step 1 is a pretty solid hint (i.e. pretty much a clue-by-four upside the head, here!) that you really should use a shelf. As in you REALLY SHOULD USE A SHELF of some kind. It doesn’t even have to be a full shelf – any rail kit that relies on an “L”-shaped profile instead of interlocking sliding bits should support an ASR-1001 just fine, e.g. Tripp-Lite’s 4POSTRAILKIT1U. RackSolutions’ Universal Fixed Server Rack Rails<https://www.rack-solutions.ca/rack-rails.html> shows an example of a slightly different design that some prefer – it all works about the same way. The other thing I’ve done is used a shallow cantilever shelf to support the rear end of equipment that only comes with ears, if it’s deep enough – something like StarTech’s CABSHELFV1U; the trick is finding a shelf that simultaneously doesn’t have the structural fold at the rear in the way AND doesn’t interfere with the device immediately below. You’d think there’re only 2 geometries of product to worry about, but there are actually more b/c there’s no standard – so test-fit first, or examine photos really carefully. This is usually more of a hack than a permanent, supportable solution, but sometimes it can work very well and very cheaply. Or, just make sure you’re installing the ASR immediately above something that does have proper 4-post mounting rails. This is probably the single most common way to safely & securely mount “eared” devices in a 4-post rack that I’ve seen – that Dell PowerEdge server in the rack suddenly starts doing double-duty as a shelf! (Or the UPS, or the KVM, or the ethernet switch, or…) -Adam Adam Thompson Consultant, Infrastructure Services [cid:image002.png@01D9790D.8F568C90] 100 - 135 Innovation Drive Winnipeg, MB R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) https://www.merlin.mb.ca<https://www.merlin.mb.ca/> [cid:image003.png@01D9790B.395F2C40]Chat with me on Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca> From: NANOG On Behalf Of Chuck Church Sent: Thursday, April 27, 2023 10:36 AM To: 'Mark Stevens' ; nanog@nanog.org Subject: RE: Standard DC rack rail distance, front to back question Hey all, sorry I did mean to say ASR1001 (an X model to be exact). The 4 post mounting they show in a hardware mounting doc uses front and back ears, which I’ve never done: https://www.cisco.com/c/en/us/td/docs/routers/asr1000/install/guide/asr1routers/asr-1000-series-hig/asr-hig-1001.html#task_1205646 see figure 16 slightly down from there. I do see some generic rails from TrippLite that probably would work, as well as shelves. I was hoping a standard depth that most vendors honored for 4 post existed, but it doesn’t seem likely. We’ll have a variety of PaloAlto, Cisco, Checkpoint, and others co-habiting. Chuck From: NANOG mailto:nanog-bounces+chuckchurch=gmail@nanog.org>> On Behalf Of Mark Stevens Sent: Thursday, April 27, 2023 11:17 AM To: nanog@nanog.org<mailto:nanog@nanog.org> Subject: Re: Standard DC rack rail distance, front to back question Lucky you with a 19" data rack. All I have are 23" telco racks but I will say, the 23" extension ears from Cisco are serious and my router chassis' don't sag. Mark On 4/27/2023 10:04 AM, Chris Marget wrote: On Thu, Apr 27, 2023 at 9:53 AM Chuck Church mailto:chuckchu...@gmail.com>> wrote: for a Cisco ASA1001, there aren’t rails, but rather front and back ‘ears’ you use to hit both front and back posts. Front *and* back ears? I'm not sure what an ASA 1001 is (ASR?) but my experience with these boxes is that they have a single pair of ears which can be mounted front OR back. The heavier / deeper 1RU devices do tend to sag alarmingly. Is there a ‘standard’ distance between front and back rails that devices usually adhere to? If you're thinking of setting the front/back distance to accommodate a specific device, table 2 might be of some interest: https://i.dell.com/sites/doccontent/business/solutions/engineering-docs/en/Documents/rail-rack-matrix.pdf
Cisco Nexus VXLAN w/vlan-aware-bundle ?
Does anyone here use VXLAN on Cisco Nexus gear in the "vlan-aware-bundle" mode (RFC7432 §6.3), not the "vlan-based" mode (§6.1)? If so, could you please contact me directly to explain how you did so? Much appreciated, -Adam Adam Thompson Consultant, Infrastructure Services [cid:image004.png@01D97DB7.68236EA0] 100 - 135 Innovation Drive Winnipeg, MB R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) https://www.merlin.mb.ca<https://www.merlin.mb.ca/> [cid:image003.png@01D97DB7.67506730]Chat with me on Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca>
10G CPE w/VXLAN - vendors?
Hello, all. I’m having difficulty finding vendors, never mind products, that fit my need. We have a small but growing number of L2 (bridged) customers that have diverse fiber paths available, and, naturally, want to make use of them. We have a solution for this: we extend the edge of our EVPN VXLAN fabric right to the customer premise. The customer-prem device needs 4x10G SFP+ cages (2 redundant paths, plus LAG to customer), and the switches we currently use, Arista 7020Rs, are quite expensive if I’m deploying one one per customer. (Nice switches, but overkill here – I don’t need 40/100G, and I don’t need 24 SFP+ ports. And they still take forever to ship.) We use RFC7438 §6.3 “vlan-aware-bundle” mode, not §6.1 “vlan-based” mode, which limits our choices somewhat. I might be willing to entertain spinning up a separate VXLAN mesh using RFC7438 §6.1 (“vlan-based”) and static VTEPs if it saves me a lot of pain. However, I’m having trouble finding small & cheaper 1U (or even desktop/wallmount) devices that have 4 SFP+ cages, and can do VXLAN, in the first place. Who even makes CPE gear with SFP+ ports? (Other than Mikrotik CRS309-1G-8S+IN / CRS317-1G-16S+RM, which are nice, but our policy requires vendor support contracts, so… no-go.) Vendors? Model#s, if you happen to know any? Reply here or privately, whatever floats your boat – any pointers appreciated! Adam Thompson Consultant, Infrastructure Services [[MERLIN logo]] 100 - 135 Innovation Drive Winnipeg, MB R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) https://www.merlin.mb.ca<https://www.merlin.mb.ca/> [cid:image002.png@01D99EC2.B891B0A0]Chat with me on Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca>
RE: 10G CPE w/VXLAN - vendors?
The redundant links to the customer site that traverse independent underlay carriers, and in some cases, equal-cost paths that we want to load-balance across, are the hard part. I’m not going to trust STP for that, and we aim for <3sec failover where we do have redundant paths. ERPS can handle the failover, but not the load-balancing. Any L2-over-L3 encapsulation protocol can handle the failover + ECMP features, but I need to do it at ~10G (~20G if ECMP) wire speed. We provide IaaS services to our customers, which is why we’re stretching VLANs to them in the first place. Viewed from the IaaS perspective, this is a bunch of DC-DC connections… but relative to the overall network, the customer-prem devices fall into the traditional “CPE” category. (Most customers either just plug in bare fiber, or they connect to an intermediate carrier’s CPE.) Adam Thompson Consultant, Infrastructure Services [[MERLIN logo]] 100 - 135 Innovation Drive Winnipeg, MB R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) https://www.merlin.mb.ca<https://www.merlin.mb.ca/> [cid:image002.png@01D99ECB.C1CEDE50]Chat with me on Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca> From: Joe Freeman Sent: Wednesday, June 14, 2023 2:16 PM To: Adam Thompson ; nanog Subject: Re: 10G CPE w/VXLAN - vendors? I think you’re probably overthinking this a bit. Why do you need to extend your vxlan/evpn to the customer premise? There are a number of 1G/10G even 100G CPE demarc devices out there that push/pop tags, even q-in-q, or 802.1ad. Assuming you have some type of aggregation node you bring these back to, tie those tags to the appropriate EVPN instance at the aggregation point. Don’t extend anything but a management tag and an S-tag essentially to the device at the customer premise. You can even put that management tagged vlan in it’s own L3 segment, or a larger L3 network and impose security. This way you’re not exposing your whole service infrastructure to a bad actor that might unplug your cpe device and plug into your network directly. From: NANOG mailto:nanog-bounces+joe=netbyjoe@nanog.org>> on behalf of Adam Thompson mailto:athomp...@merlin.mb.ca>> Date: Wednesday, June 14, 2023 at 2:52 PM To: nanog mailto:nanog@nanog.org>> Subject: 10G CPE w/VXLAN - vendors? Hello, all. I’m having difficulty finding vendors, never mind products, that fit my need. We have a small but growing number of L2 (bridged) customers that have diverse fiber paths available, and, naturally, want to make use of them. We have a solution for this: we extend the edge of our EVPN VXLAN fabric right to the customer premise. The customer-prem device needs 4x10G SFP+ cages (2 redundant paths, plus LAG to customer), and the switches we currently use, Arista 7020Rs, are quite expensive if I’m deploying one one per customer. (Nice switches, but overkill here – I don’t need 40/100G, and I don’t need 24 SFP+ ports. And they still take forever to ship.) We use RFC7438 §6.3 “vlan-aware-bundle” mode, not §6.1 “vlan-based” mode, which limits our choices somewhat. I might be willing to entertain spinning up a separate VXLAN mesh using RFC7438 §6.1 (“vlan-based”) and static VTEPs if it saves me a lot of pain. However, I’m having trouble finding small & cheaper 1U (or even desktop/wallmount) devices that have 4 SFP+ cages, and can do VXLAN, in the first place. Who even makes CPE gear with SFP+ ports? (Other than Mikrotik CRS309-1G-8S+IN / CRS317-1G-16S+RM, which are nice, but our policy requires vendor support contracts, so… no-go.) Vendors? Model#s, if you happen to know any? Reply here or privately, whatever floats your boat – any pointers appreciated! Adam Thompson Consultant, Infrastructure Services [[MERLIN logo]] 100 - 135 Innovation Drive Winnipeg, MB R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) https://www.merlin.mb.ca<https://www.merlin.mb.ca/> [cid:image002.png@01D99EC2.B891B0A0]Chat with me on Teams<https://teams.microsoft.com/l/chat/0/0?users=athomp...@merlin.mb.ca>
Re: 44/8
On 07/18/2019 at 23:08, Job Snijders wrote: > A potential upside is that hamnet operators maybe have access to some RPKI > services now! OK, I'll bitehow do you mean? --Adam
