On Sun, Feb 09, 2025 at 04:41:05PM +1100, Geoff Huston wrote:
> Hi Romain
>
> We are seeing in RIS data a constant flow of update messages from a few
> ASes, here is the list of the top prefixes:
>
> ┌─────────────────────┬────────────┬──────────────┐
> │ prefix │ origin_asn │ num_announce │
> │ varchar │ varchar │ int64 │
> ├─────────────────────┼────────────┼──────────────┤
> │ 169.145.140.0/23 │ 6979 │ 843376 │
> │ 2a03:eec0:3212::/48 │ 22616 │ 435608 │
> │ 172.224.198.0/24 │ 36183 │ 380117 │
> │ 172.226.208.0/24 │ 36183 │ 374040 │
> │ 172.226.148.0/24 │ 36183 │ 367083 │
We did a bunch of internal research and such and found a
mismatch in behavior which triggered this, which was unexpected.
We did some mitiigation work immediately and the bigger fix
should be complete very soon. It takes some time to update thousands of
devices.
I also raised the issue again with my management about the
prioritization of some monitoring project and helped the teams that
needed to do the fix with an internal dashboard so they could more
immediately see the issue internally.
As is the usual with this, the bigger fix is always larger than
one thinks, and having the "doesn't run on a server under someones desk"
which now really is a VM in some location but is still tied to that
person or could be orphaned project/code ... but that's the story of
many systems :-)
Please reach out if you see any bad routing behavior.
- Jared
--
Jared Mauch | pgp key available via finger from ja...@puck.nether.net
clue++; | http://puck.nether.net/~jared/ My statements are only mine.
