3549 1273
3549<->1273 seem to be generating a lot of BGP updates between each other, is anyone else seeing this or noticed an adverse impact? - Jared
AT&T Wireless Issue
Good Afternoon, I am looking to get in touch with an AT&T wireless switch tech in the NY/NJ region. If someone from AT&T could reach out to me offline it would be great. Thanks Mark
Reach for a Verizon "Mobility" Network Contact
Please contact me offline at b...@fiberinternetcenter.com NOT looking for verizon a cell phone dealer - NOT looking for a verizon business multi-phone plan sales person. Looking for the verizon mobility department , someone that can generate a contract for this specific service and has contacts within that part of the organization and knows the individuals by name. Thank You Bob Evans CTO
tcp md5 bgp attacks?
so we started to wonder if, since we started protecting our bgp sessions with md5 (in the 1990s), are there still folk trying to attack? we were unable to find bgp mib counters. there are igp interface counters, but that was not our immediate interest. we did find that md5 failures are logged. looking at my logs for a few years, i find essentially nothing; two 'attackers,' one my own ibgp peer, and one that noted evildoer rob thomas, bgprs01.ord08.cymru.com. we would be interested in data from others. note that we are neither contemplating nor suggesting removing md5 from [y]our bgp sessions. randy
Re: tcp md5 bgp attacks?
On 08/14/2018 03:38 PM, Randy Bush wrote: so we started to wonder if, since we started protecting our bgp sessions with md5 (in the 1990s), are there still folk trying to attack? n00b response here I thought using ACLs or otherwise protecting the BGP endpoint was best practice. Thus it's really hard to even try break an MD5 protected BGP session if you can't even establish the TCP connection. Everything that I've seen or set up had an ACL to only allow the peer(s) to be able to connect to (from memory) TCP port 179. Is there something that I've missed the boat on? #learningOpportunity -- Grant. . . . unix || die smime.p7s Description: S/MIME Cryptographic Signature
Re: tcp md5 bgp attacks?
On Tue, Aug 14, 2018 at 05:28:13PM -0600, Grant Taylor via NANOG wrote: > On 08/14/2018 03:38 PM, Randy Bush wrote: > > so we started to wonder if, since we started protecting our bgp > > sessions with md5 (in the 1990s), are there still folk trying to > > attack? > > n00b response here > > I thought using ACLs or otherwise protecting the BGP endpoint was best > practice. Thus it's really hard to even try break an MD5 protected > BGP session if you can't even establish the TCP connection. > > Everything that I've seen or set up had an ACL to only allow the > peer(s) to be able to connect to (from memory) TCP port 179. > > Is there something that I've missed the boat on? > > #learningOpportunity To further harden your setup, consider using GTSM https://tools.ietf.org/html/rfc5082 Kind regards, Job
Re: tcp md5 bgp attacks?
On 15 Aug 2018, at 6:28, Grant Taylor via NANOG wrote: > Is there something that I've missed the boat on? No - it's a belt-and-suspenders sort of thing, along with GTSM. --- Roland Dobbins
Re: tcp md5 bgp attacks?
On Tue, 14 Aug 2018 21:38:35 + Randy Bush wrote: > we would be interested in data from others. My data is coarse, but with 'show system statistics tcp | match auth' I see sometimes thousands of rcv packets dropped on BGP routers. I doubt they are attacks, but simply badly configured or stale peer sessions over the course of time the counters initialized from. John
Re: tcp md5 bgp attacks?
> My data is coarse, but with 'show system statistics tcp | match auth' > I see sometimes thousands of rcv packets dropped on BGP routers. I > doubt they are attacks, but simply badly configured or stale peer > sessions over the course of time the counters initialized from. thanks john for the one (so far) answer to my question instead of telling me how to run my routers what i see also looks like config as opposed to attack --- follow-on question: anyone using the timed key-chain stuff? randy
Re: tcp md5 bgp attacks?
> On Aug 14, 2018, at 8:04 PM, Randy Bush wrote: > > follow-on question: > > anyone using the timed key-chain stuff? I’ve looked at it, hear it works, but not been willing to take the hit for any transition. I talked about some of this and other challenges at SAAG WG at IETF 101. Transport area has some possible interesting things, but similar to what Haas said, TCP-AO isn’t really viable yet, and we need something that’s stable enough to last 5-7 years, which is very different from a HTTP transaction that may live only a few seconds. We have some places where we could transition non-BGP protocols and rotate the key, but last I recall it was only there on a single vendor so multi-vendor posed some challenges. - Jared
Re: tcp md5 bgp attacks?
[ again, thanks for an answer to the question asked ] >> anyone using the timed key-chain stuff? > > I’ve looked at it, hear it works, but not been willing to take the hit > for any transition. and i am not sure it meets my needs. i am not seeking privacy or pfs. i want roll-if-compromise. (and no, i do not want automated compromise heuristics, a recipe for death). > > we need something that’s stable enough to last 5-7 years, which is > very different from a HTTP transaction that may live only a few > seconds. something such as, or close to, rfc 4808? randy
Re: tcp md5 bgp attacks?
> On Aug 14, 2018, at 8:12 PM, Randy Bush wrote: > > [ again, thanks for an answer to the question asked ] > >>> anyone using the timed key-chain stuff? >> >> I’ve looked at it, hear it works, but not been willing to take the hit >> for any transition. > > and i am not sure it meets my needs. i am not seeking privacy or pfs. > i want roll-if-compromise. (and no, i do not want automated compromise > heuristics, a recipe for death). >> >> we need something that’s stable enough to last 5-7 years, which is >> very different from a HTTP transaction that may live only a few >> seconds. > > something such as, or close to, rfc 4808? It provides some capability, but for example if I have a large iBGP mesh and need to change methods of securing it and have automation involved, it can often be a one-shot change unless I can zone some routers to different versions of templating to have a smooth transition. Basically the negative side of using peer-groups can be quite catastrophic with how you transition from the router software without good update packing/replication to one with a good system. It doesn’t need the groups to optimize the leadership replication, but you still use them to minimize configuration duplication. I’m not sure if in 2018 which is the right path from an automation perspective, if you can have software specify and iterate everything, do you continue to use apply-groups, or just rely upon the automation to output the full configuration? Most systems (including the ones at present and past employers) tend to be a variation on template driven in some language(s) where the original problem/engineer creep doesn’t account for the proper abstract models to allow zoned changes/rollover of subsets of the network. Ie: rolling the key is an all-or-nothing operation. Similar to JTK most of the log messages we see are from people who forgot the MD5 key, or at an IX where we did poor relationship management so the IP is now reused by others and nobody cleaned up the older session(s). I have heard (but not personally seen) of well formed TCP session attacks where md5 may have helped, but since the punt path tends to be the weak link and most folks don’t do GTSH/GTSM (or worse, have hardware that can’t filter based on this) you still incur the expensive punt operation only to have the RP/RE kernel then drop the packet. IOS-XR also has very good/robust defaults with LPTS which helps significantly. I’ve seen quite large attacks against a router be mitigated by LPTS and not require the mpp/control plane filters to be involved. Basically, once you roll md5 you may be at risk for having rolled it to need a way to undo and that pathway may not be easy, with or without automation. - Jared
Re: tcp md5 bgp attacks?
>> something such as, or close to, rfc 4808? > > It provides some capability, but for example if I have a large iBGP > mesh and need to change methods of securing it and have automation > involved, it can often be a one-shot change unless I can zone some > routers to different versions of templating to have a smooth > transition. Basically the negative side of using peer-groups can be > quite catastrophic with how you transition from the router software > without good update packing/replication to one with a good system. It > doesn’t need the groups to optimize the leadership replication, but > you still use them to minimize configuration duplication. > > I’m not sure if in 2018 which is the right path from an automation > perspective, if you can have software specify and iterate everything, > do you continue to use apply-groups, or just rely upon the automation > to output the full configuration? > > Most systems (including the ones at present and past employers) tend > to be a variation on template driven in some language(s) where the > original problem/engineer creep doesn’t account for the proper > abstract models to allow zoned changes/rollover of subsets of the > network. Ie: rolling the key is an all-or-nothing operation. > > Similar to JTK most of the log messages we see are from people who > forgot the MD5 key, or at an IX where we did poor relationship > management so the IP is now reused by others and nobody cleaned up the > older session(s). > > I have heard (but not personally seen) of well formed TCP session > attacks where md5 may have helped, but since the punt path tends to be > the weak link and most folks don’t do GTSH/GTSM (or worse, have > hardware that can’t filter based on this) you still incur the > expensive punt operation only to have the RP/RE kernel then drop the > packet. > > IOS-XR also has very good/robust defaults with LPTS which helps > significantly. I’ve seen quite large attacks against a router be > mitigated by LPTS and not require the mpp/control plane filters to be > involved. > > Basically, once you roll md5 you may be at risk for having rolled it > to need a way to undo and that pathway may not be easy, with or > without automation. one or both of us needs to reread 4808 randy
Re: tcp md5 bgp attacks?
On 8/14/18 2:38 PM, Randy Bush wrote: > so we started to wonder if, since we started protecting our bgp > sessions with md5 (in the 1990s), are there still folk trying to > attack? To recap for the purpose of my own edification and because hopefully someone will relieve me of my assumptions. The purpose of of rfc 2385 tcp md5 digests is to keep in-window, tcp segments that are spoofed from being ingested into the tcp stack. At the time of it's writing (1998) some popular network operating systems did not check that the sequence number was in fact inside the window (so that any tcp packet matching the 4 tupple would be ingested whether it was in-window or not. Variously this improvement was supplemented with the checking the TTL (https://tools.ietf.org/html/draft-gill-btsh-02), checking whether the packet is actually in window, by ACLs that would limit the impacts of spoofing from off path attackers (you can't target my multihop infracture sessions from outside my network for example), and by filters that would limit the sort of thing you could inject into bgp (rendering prefix hijacking moot) ). I see broad evidence that MD5 values are extensively shared between sessions and effectively never rekeyed (including cases where I've changed employers and the same asn is using the same values for new peers). given the existance of effective mitigations for the ibgp case, I've need seen a reason to employ it internally or to explore support for rfc 4808 mechnisms since key rolling is effectively an external coordination problem. Due to window checking and the ttl hack, the best vantage point for launching an attack against a single hop ebgp sessions is as an on path attacker (such that you would be able to identify source port and window), layer-2 exchanges which flood unicast traffic (a hub I guess or any public exchange with broken mac learning) would seem particularly vulnerable since there are many on path neighbors. That is no longer a normal topology. :/ > we were unable to find bgp mib counters. there are igp interface > counters, but that was not our immediate interest. we did find > that md5 failures are logged. I can't quite get there either. md5 failures I see quite a lot of, as peers that formerly have it configured fail either temporarily or over longer timescales. md5 failures for unestablished connections aren't very interesting in this case. I have thousands of establish connections that last a very long time at public exchange points, so the threat of tcp rsts to sessions is clearly not being realized. > > looking at my logs for a few years, i find essentially nothing; > two 'attackers,' one my own ibgp peer, and one that noted evildoer > rob thomas, bgprs01.ord08.cymru.com. > > we would be interested in data from others. > > note that we are neither contemplating nor suggesting removing md5 > from [y]our bgp sessions. > > randy >
Re: tcp md5 bgp attacks?
my memory is that seq num guessing and sending rst was the core problem motivating tcp/md5 for bgp, and btsh came some years later. but no big deal. i think that, indeed, md5 keys are shared across many links *within* an op's infrastructure. but, since integrity, and not privacy, is the goal, this does not seem risky. carrying keys to new networks seems a bit risky as does re-use with multiple external parties. > given the existance of effective mitigations for the ibgp case, I've > need seen a reason to employ it internally or to explore support for > rfc 4808 mechnisms since key rolling is effectively an external > coordination problem. if i need to roll keys on ibgp, i suspect i have a far more serious problem than if it is ebgp, twice as serious at a minimum :) < rathole > i am not much worried about a mesh which floods unicast. can you even buy devices which support that any more? a while back, i had to really dig in the closet to find one at 100mbps so i could shark mid-stream. > I have thousands of establish connections that last a very long time > at public exchange points, so the threat of tcp rsts to sessions is > clearly not being realized. my theory is that, as the attacks were mitigated the attackers moved on to other things. after all, the non-nuisance benefit i get by resetting your bgp session with margaret is shifting your traffic past some place i can mitm or to a more expensive, to you, link. the attackers moved on to more lucrative endeavors. randy
Re: tcp md5 bgp attacks?
On 15 Aug 2018, at 9:27, Randy Bush wrote: my theory is that, as the attacks were mitigated the attackers moved on to other things. With regards to BGP, the MD5 thing was promulgated to counter what was a largely theoretical threat. iACLs, and later GTSM and CoPP and LPTS and so forth really obviated the need for it. For IGPs, MD5 was belt-and-suspenders against someone deliberately or accidentally bringing up a new router and manipulating traffic internally. Passiving the IGP on non-core links was the BCP, but often was honored in the breach; pushing an additional feature for 'security' purposes got some folks' attention when the passiving BCP was ignored. We still see DDoS attacks against routers, of course. But the goal there is disruption of availability, not trying to move traffic onto some alternate path which would somehow benefit the attacker. --- Roland Dobbins
Re: tcp md5 bgp attacks?
On 8/14/18 7:27 PM, Randy Bush wrote: > > < rathole > > i am not much worried about a mesh which floods unicast. can you even > buy devices which support that any more? a while back, i had to really > dig in the closet to find one at 100mbps so i could shark mid-stream. I'm not actually worried about it because it is rare, and not a feature, that said, unicast flooding is in fact something we detect on exchanges with a fair amount of frequency e.g. 2-3 a week across the exchanges were we are present. That traffic gets discarded on our ingress but you can count dport 179 packets in there that aren't yours. I certainly wouldn't build a business model around gaining insight from that information leakage (and the bulk of the traffic is whatever the neighbor is exchanging, with someone else, from looking at mac's that sort of thing tends to be one sided unless for example it's a whole switch). >> I have thousands of establish connections that last a very long time >> at public exchange points, so the threat of tcp rsts to sessions is >> clearly not being realized.
