Yep, except that
A: Bandwidth had to know this is a when not an if. In today's internet
if your company can be considered critical infra, you will be attacked.
The more likley scenario is the technical staff knew this but the MBA
types said they were paranoid delusions and denied the project budget.
B: I believe they need to be drawing national attention to this to
highlight what a steaming dumpster fire much of the critical infra
really is. Mostly because its designed to maximize quarterly earnings,
not stay working in the face of adversity.
C: I'm absolutely sympathetic to their plight having been through a
crippling DDOS in a past life which spurred the complete redesign of the
entire network into sacrificial pods with more robust transport, and a
triage runbook to keep the most things available in the face of an
insurmountable onslaught.
D: Why is the discussion not yet turning to the fact that all major
eyeball networks in the US still don't implement BCP38 as a matter of
laziness (or above MBA reasons), and this is what allows these attacks
to happen. The telco guys are being held to the STIR/SHAKEN standard
over robocalling but for decades the major US ISP's could have
implemented network policies that would break the chain of DDOS
escalation and don't because they cant be bothered to.
I once gave a talk on DDOS at a Carrier fraud association task force
meeting (cfca.org) and had representatives from every major US eyeball
network in the room and asked the above question and the overwhelming
answer I got is "leadership doesn't feel its a worthwhile risk/reward to
implement".
-Ryan
On 9/27/2021 7:17 PM, Peter Beckman wrote:
On Mon, 27 Sep 2021, Ryan Delgrosso wrote:
Nothing meaningful other than the normal public party line.
I too have heard unofficially that its DDOS, which makes sense given
the recurring nature.
4.5hrs down Sat
Our monitoring showed 2 hours 47 minutes of actual service affecting
outages across Voice (Inbound and Outbound), Messaging, and API/Portal.
The issue started at 3pm and recovered at 5:47pm EDT. We reported it to
the TAC at 3:07pm, they did not post on Status until 3:31pm.
Some small downtime Sun
Now deep into Monday with problems.
Its not a good look, but id like some more transparency.
DDoS attacks are real and hard to null route. You've got millions of IP
addresses slamming you with data. Your router has a capacity, and your
router cannot handle all of that extra crap data along with all of our
traffic too.
I'm sure BW will be investing in some beefy hardware that will be
able to
better handle DDoS attacks, as well as working more closely with their
peering providers. I have to assume that they were getting gigabits of
traffic, overwhelming their links in addition to their edge routers.
Cloudflare details how they do it here:
https://support.cloudflare.com/hc/en-us/articles/200172676-Understanding-Cloudflare-DDoS-protection
Not much to be transparent about. The Internet is an unfriendly
place, and
bad actors can rain hell upon any public IP they want. Unsecured
laptops,
desktops, TVs, IOT devices, etc, all contribute just a little tiny bit,
and all focus on one single point, kinda like those giant solar farms
with
the mirrors and single tower in the middle to boil the molten salt.
Well, Bandwidth is the molten salt, and the mirrors are a bunch of
unsecured devices on the Internet.
---------------------------------------------------------------------------
Peter Beckman Internet Guy
[email protected] https://www.angryox.com/
---------------------------------------------------------------------------
_______________________________________________
VoiceOps mailing list
[email protected]
https://puck.nether.net/mailman/listinfo/voiceops
_______________________________________________
VoiceOps mailing list
[email protected]
https://puck.nether.net/mailman/listinfo/voiceops
