See traffic on Twelve99 re-routed via LAX now. Seems to be stable, for the moment:
My traceroute [v0.95] VONMVPCVXYVFD (172.20.0.219) -> 216.147.3.209 (216.147.3.209) 2024-08-15T11:29:35+0100 Keys: Help Display mode Restart statistics Order of fields quit Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. 172.20.0.1 0.0% 68 0.7 1.0 0.7 1.9 0.2 2. lo0-0.bng2.ixn-lon.zen.net.uk 0.0% 67 12.3 8.4 5.1 43.1 6.1 3. lag-5.p2.ixn-lon.zen.net.uk 0.0% 67 5.6 5.9 5.4 7.8 0.4 4. ldn-b13-link.ip.twelve99.net 45.5% 67 5.6 5.5 4.9 6.1 0.3 5. ldn-bb2-link.ip.twelve99.net 0.0% 67 6.1 6.3 5.4 8.9 0.6 6. nyk-bb1-link.ip.twelve99.net 0.0% 67 74.7 75.0 74.1 77.0 0.5 7. rest-bb1-link.ip.twelve99.net 1.5% 67 81.5 81.5 80.8 83.9 0.5 8. lax-b22-link.ip.twelve99.net 0.0% 67 137.5 138.4 136.7 175.7 5.4 9. (waiting for reply) 10. (waiting for reply) 11. 15.230.16.155 0.0% 67 141.6 143.9 140.3 167.9 5.4 12. (waiting for reply) 13. 52.93.14.118 0.0% 67 216.6 178.8 165.8 267.3 20.9 14. 52.93.14.105 0.0% 67 171.5 171.4 170.2 175.7 1.0 15. 52.93.15.28 0.0% 67 169.2 179.0 169.0 238.9 15.5 16. 52.93.246.191 0.0% 67 166.1 167.8 165.3 194.0 4.4 17. 52.93.246.81 0.0% 67 172.3 172.4 171.3 179.7 1.4 18. (waiting for reply) 19. (waiting for reply) 20. (waiting for reply) 21. (waiting for reply) 22. (waiting for reply) 23. (waiting for reply) 24. 216.147.3.209 0.0% 67 166.4 167.1 166.0 169.1 0.6 > On 15 Aug 2024, at 11:09, Phil Lavin <phil.la...@vonage.com> wrote: > > Fwiw, still ongoing > > I raised an AWS ticket and they’re looking into it. If there’s any friendly > AWS Net Engineers on list, your help removing Twelve99 from route would be > appreciated. Cross-posting to Nanog for wider reach. > > >> On 15 Aug 2024, at 09:44, Matt Almgren <malmg...@backblaze.com> wrote: >> >> There's a planned SJC maintenance happening now. Expected end time is 2am >> PST. >> >> On Thu, Aug 15, 2024, 1:38 AM Phil Lavin via Outages <outa...@outages.org >> <mailto:outa...@outages.org>> wrote: >>> It’s a good week for the Internet. Third outages list post this week :( >>> >>> Seeing major loss on Twelve99 around SJO: >>> >>> UK to AWS USW2: >>> >>> My traceroute [v0.95] >>> VONMVPCVXYVFD (172.20.0.14) -> 216.147.3.209 (216.147.3.209) >>> 2024-08-15T09:29:13+0100 >>> Keys: Help Display mode Restart statistics Order of fields quit >>> Packets Pings >>> Host Loss% Snt Last Avg Best >>> Wrst StDev >>> 1. 172.20.0.1 0.0% 9 1.1 1.2 1.0 >>> 1.7 0.2 >>> 2. lo0-0.bng2.ixn-lon.zen.net.uk <http://lo0-0.bng2.ixn-lon.zen.net.uk/> >>> 0.0% 9 5.5 6.8 5.4 11.2 1.9 >>> 3. lag-5.p2.ixn-lon.zen.net.uk <http://lag-5.p2.ixn-lon.zen.net.uk/> >>> 0.0% 8 5.5 6.0 5.2 6.5 0.5 >>> 4. ldn-b13-link.ip.twelve99.net <http://ldn-b13-link.ip.twelve99.net/> >>> 25.0% 8 5.8 5.5 5.0 6.0 0.4 >>> 5. ldn-bb2-link.ip.twelve99.net <http://ldn-bb2-link.ip.twelve99.net/> >>> 0.0% 8 6.6 6.8 5.9 7.5 0.7 >>> 6. nyk-bb1-link.ip.twelve99.net <http://nyk-bb1-link.ip.twelve99.net/> >>> 0.0% 8 75.2 75.2 74.5 75.9 0.5 >>> 7. sjo-b23-link.ip.twelve99.net <http://sjo-b23-link.ip.twelve99.net/> >>> 57.1% 8 296.2 312.5 296.2 343.7 27.0 >>> 8. (waiting for reply) >>> 9. 15.230.28.67 62.5% 8 294.4 295.7 294.4 >>> 296.8 1.2 >>> 10. 54.240.242.119 25.0% 8 337.1 305.7 294.6 >>> 337.1 16.2 >>> 11. (waiting for reply) >>> 12. 52.93.15.36 57.1% 8 398.2 400.8 395.4 >>> 408.8 7.1 >>> 13. 52.93.15.31 42.9% 8 388.7 395.5 388.7 >>> 403.3 6.2 >>> 14. 52.93.14.34 28.6% 8 431.8 432.2 393.2 >>> 488.2 35.8 >>> 15. 52.93.246.177 42.9% 8 397.7 399.3 397.7 >>> 401.1 1.4 >>> 16. 52.93.246.81 25.0% 8 393.4 396.9 393.2 >>> 400.6 3.0 >>> 17. (waiting for reply) >>> 18. (waiting for reply)19. (waiting for re 28.6% 8 394.3 397.5 394.3 >>> 400.2 3.0 >>> >>> AWS USW2 to UK: >>> >>> Host Loss% Snt >>> Last Avg Best Wrst StDev >>> 1. ip-10-96-50-153.us-west-2.compute.internal 0.0% 87 >>> 0.2 0.1 0.1 0.2 0.0 >>> 2. ec2-44-233-118-55.us-west-2.compute.amazonaws.com >>> <http://ec2-44-233-118-55.us-west-2.compute.amazonaws.com/> 0.0% >>> 87 7.3 11.9 1.0 66.9 16.4 >>> ec2-34-221-151-47.us-west-2.compute.amazonaws.com >>> <http://ec2-34-221-151-47.us-west-2.compute.amazonaws.com/> >>> ec2-44-233-117-187.us-west-2.compute.amazonaws.com >>> <http://ec2-44-233-117-187.us-west-2.compute.amazonaws.com/> >>> 3. 240.0.212.4 0.0% 87 >>> 0.5 0.4 0.3 1.7 0.2 >>> 240.0.72.7 >>> 4. 242.2.110.95 0.0% 87 >>> 0.8 1.8 0.4 20.6 3.4 >>> 5. 240.2.144.13 0.0% 87 >>> 5.8 5.8 5.7 5.9 0.1 >>> 6. amazon-ic-375418.ip.twelve99-cust.net >>> <http://amazon-ic-375418.ip.twelve99-cust.net/> 0.0% >>> 87 5.7 5.6 5.6 6.1 0.1 >>> 7. port-b4-link.ip.twelve99.net <http://port-b4-link.ip.twelve99.net/> >>> 0.0% 87 6.2 6.1 5.9 6.5 0.1 >>> 8. port-b3-link.ip.twelve99.net <http://port-b3-link.ip.twelve99.net/> >>> 0.0% 86 5.6 6.3 5.6 18.5 1.8 >>> 9. palo-b24-link.ip.twelve99.net <http://palo-b24-link.ip.twelve99.net/> >>> 0.0% 86 21.4 21.3 21.1 21.6 0.1 >>> 10. nyk-bb2-link.ip.twelve99.net <http://nyk-bb2-link.ip.twelve99.net/> >>> 19.8% 86 165.3 172.2 165.3 206.1 4.7 >>> 11. ldn-bb1-link.ip.twelve99.net <http://ldn-bb1-link.ip.twelve99.net/> >>> 68.2% 86 391.5 401.2 387.3 411.8 6.3 >>> 12. ldn-b3-link.ip.twelve99.net <http://ldn-b3-link.ip.twelve99.net/> >>> 57.0% 86 397.1 401.4 395.5 409.0 2.7 >>> 13. zeninternet-ic-363994.ip.twelve99-cust.net >>> <http://zeninternet-ic-363994.ip.twelve99-cust.net/> 40.7% >>> 86 397.0 412.3 397.0 473.3 18.0 >>> 14. lag-1.bng2.ixn-lon.zen.net.uk <http://lag-1.bng2.ixn-lon.zen.net.uk/> >>> 63.5% 86 387.2 402.5 382.5 422.8 9.0 >>> 15. 62-3-100-30.dsl.in-addr.zen.co.uk >>> <http://62-3-100-30.dsl.in-addr.zen.co.uk/> 55.8% >>> 86 391.6 404.5 391.6 412.5 6.1 >>> 16. (waiting for reply) >>> _______________________________________________ >>> Outages mailing list >>> outa...@outages.org <mailto:outa...@outages.org> >>> https://puck.nether.net/mailman/listinfo/outages >> >> This email, including its contents and any attachment(s), may contain >> confidential and/or proprietary information and is solely for the review and >> use of the intended recipient(s). If you have received this email in error, >> please notify the sender and permanently delete this email, its content, and >> any attachment(s). Any disclosure, copying, or taking of any action in >> reliance on an email received in error is strictly prohibited. >