Nobody should ever be forced to peer to get someone to address abusive traffic originating from networks under their control.
On Thu, Nov 8, 2018 at 4:29 PM John <j...@nuclearfallout.net> wrote: > Zach, > > As mentioned before, I am open to peering (where possible) but have not > received a response. > > My goal is to connect with someone at Amazon and work with them on a > technical solution, which is why I posted asking for that here. Various > factors mean that we can't just upgrade our way out of this one, and > manual whac-a-mole on the sources has shown to have limited use. > > These attacks also impact Amazon and the networks in between the sources > and targets, and they take time to handle by the abuse teams, so there > are good reasons to investigate them further and find better ways to > mitigate or prevent them. > > -John > > On 11/8/2018 1:12 PM, Zach Puls wrote: > > Makes sense, that's understandable. Do you peer with AWS? If not, maybe > opening up a peering agreement will give you a better contact, and a bit > more pull when attacks occur? I know someone with a peering agreement with > AWS, and they have been able to get resolutions fairly quickly when issues > arise. > > > > Other than that, I'm not sure of a solution other than more IP transit. > > > > Thanks, > > > > Zach Puls > > Network Engineer | MEF-CECP > > KsFiberNet > > > > -----Original Message----- > > From: John Weekes <j...@nuclearfallout.net> > > Sent: Thursday, November 08, 2018 15:03 > > To: Zach Puls <zp...@ksfiber.net> > > Cc: nanog@nanog.org > > Subject: Re: Amazon network engineering contact? re: DDoS traffic > > > > Zach, > > > > Yes, RTBH is used to distribute the null-routes that I mentioned. > > > > Unfortunately, even brief saturation events lasting just 5-10 seconds (a > typical amount of time to detect the loss, issue the null-route, and see > the traffic start to fall off as it is distributed upstream) can cause real > damage to those customers who are sensitive to latency and packet loss. So > while null-routes limit the duration of the impact, they can't eliminate it > entirely. And, of course, the actual target of the attack > > -- the now-null-routed IP address -- becomes unreachable, which was > presumably the goal of the attacker. > > > > -John > > > > On 11/8/2018 12:54 PM, Zach Puls wrote: > >> No idea about an Amazon abuse contact, but do you have RTBH communities > enabled with your upstream provider(s)? As a general practice, when you > detect a (D)DoS attack in progress, it would help to automatically > advertise that prefix to your upstream(s) with the black-hole community. > This would at least help mitigate the effects of the attacks when they do > occur, even if they come from a different source than AWS. > >> > >> Thanks, > >> > >> Zach Puls > >> Network Engineer | MEF-CECP > >> KsFiberNet > >> > >> -----Original Message----- > >> From: NANOG <nanog-boun...@nanog.org> On Behalf Of John Weekes > >> Sent: Thursday, November 08, 2018 14:44 > >> To: nanog@nanog.org > >> Subject: Amazon network engineering contact? re: DDoS traffic > >> > >> We've been seeing significant attack activity from Amazon over the last > two months, involving apparently compromised instances that commonly send > 1-10G of traffic per source and together generate Nx10G of total traffic. > Even when our overall upstream capacity exceeds an attack's overall size, > the nature of load-balancing over multiple 10G upstream links means that an > individual link can be saturated by multiple large flows, forcing our > systems to null-route the target to limit impact. > >> > >> We've sent an abuse notification about every traffic source to Amazon, > and specific sources seem to stop their involvement over time (suggesting > that abuse teams are following up on them), but there is an endless parade > of new attackers, and each source participates in many damaging attacks > before it is shut down. > >> > >> Is there anyone at Amazon who can help with an engineering solution in > terms of programmatically detecting and rate-limiting attack traffic > sources, to our networks or overall? Or applying the kludge of a rate-limit > for all Amazon traffic to our networks? Or working with us on some other > option? > >> > >> At least one other large cloud provider has an automatic rate-limiting > system in place that is effective in reducing the damage from repeat > high-volume attacks. > >> > >> Emails to the Amazon NOC, peering contacts (since that would be another > possible solution), and abuse department have not connected me with anyone. > >> > >> Thanks, > >> John > >> > > > >
