Service provider story about tracking down TCP RSTs

2018-09-01 Thread frnkblk
I want to share a little bit of our journey in tracking down the TCP RSTs that impacted some of our customers for almost ten weeks. Almost immediately after we turned up two new Arista border routers in late July we started receiving a trickle of complaints from customers regarding their inabil

Re: Service provider story about tracking down TCP RSTs

2018-09-01 Thread William Herrin
On Sat, Sep 1, 2018 at 2:51 PM, wrote: > pointing out that a > single traceroute to a Fastly site was hitting two of their POPs (they use > anycast) and because they don’t sync state between POPs the second POP would > naturally issue a TCP RST (sidebar: fascinating blog article on Fastly’s > inf

Re: Service provider story about tracking down TCP RSTs

2018-09-01 Thread Garrett Skjelstad
I would love this as a blog post to link folks that are not nanog members. -Garrett On Sat, Sep 1, 2018, 11:52 wrote: > I want to share a little bit of our journey in tracking down the TCP RSTs > that impacted some of our customers for almost ten weeks. > > > > Almost immediately after we turne

Re: Service provider story about tracking down TCP RSTs

2018-09-01 Thread William Herrin
On Sat, Sep 1, 2018 at 4:00 PM, William Herrin wrote: > On Sat, Sep 1, 2018 at 2:51 PM, wrote: >> pointing out that a >> single traceroute to a Fastly site was hitting two of their POPs (they use >> anycast) and because they don’t sync state between POPs the second POP would >> naturally issue a

Re: Service provider story about tracking down TCP RSTs

2018-09-01 Thread Ryan Landry
Glad we could help, Frank. On Sat, Sep 1, 2018 at 11:54 wrote: > I want to share a little bit of our journey in tracking down the TCP RSTs > that impacted some of our customers for almost ten weeks. > > > > Almost immediately after we turned up two new Arista border routers in > late July we sta

Re: Service provider story about tracking down TCP RSTs

2018-09-01 Thread Lee
On 9/1/18, William Herrin wrote: > On Sat, Sep 1, 2018 at 4:00 PM, William Herrin wrote: >> On Sat, Sep 1, 2018 at 2:51 PM, wrote: >>> pointing out that a >>> single traceroute to a Fastly site was hitting two of their POPs (they >>> use >>> anycast) and because they don’t sync state between PO

Re: TekSavvy (Canada) contact

2018-09-01 Thread Eric Kuhnke
Hey all, It was not my intention to cause any unwarranted concern related to the TekSavvy network. There are zero issues with their network. Every service I have ever purchased from them is rock solid and reliable. I'm in contact with Paul and others there directly. The topic of discussion is rel

Re: automatic rtbh trigger using flow data

2018-09-01 Thread Baldur Norddahl
fre. 31. aug. 2018 17.16 skrev Hugo Slabbert : > > > I would love an upstream that accepts flowspec routes to get granular > about > drops and to basically push "stateless ACLs" upstream. > > _keeps dreaming_ > > > We just need a signal to drop UDP for a prefix. The same as RTBH but only for UDP.

Re: Service provider story about tracking down TCP RSTs

2018-09-01 Thread William Herrin
On Sat, Sep 1, 2018 at 6:11 PM, Lee wrote: > On 9/1/18, William Herrin wrote: >> On Sat, Sep 1, 2018 at 4:00 PM, William Herrin wrote: >>> Better yet, do the job right and build an anycast TCP stack as >>> described here: https://bill.herrin.us/network/anycasttcp.html > > An explosion in state m

RE: automatic rtbh trigger using flow data

2018-09-01 Thread Ryan Hamel
No ISP is in the business of filtering traffic unless the client pays the hefty fee since someone still has to tank the attack. I also don’t think there is destination prefix IP filtering in flowspec, which could seriously cause problems. From: NANOG On Behalf Of Baldur Norddahl Sent: Saturday

Re: automatic rtbh trigger using flow data

2018-09-01 Thread Roland Dobbins
On 1 Sep 2018, at 1:35, Aaron Gould wrote: I may mark internet-sourced-udp with a certain marking dscp/exp so that as it travels through my internet network, it will be the first to get dropped (? Wred ? work well for udp?) during congestion when an attack gets through You can use flow tele

Re: automatic rtbh trigger using flow data

2018-09-01 Thread Roland Dobbins
On 1 Sep 2018, at 1:20, Lotia, Pratik M wrote: Arbor report mentions volumetric attacks using DNS, NTP form 75+% of the attacks. I'm well aware of what's mentioned in the Arbor report, thanks! ;> Then QoSing certain ports and protocols is the best way to start with. The point is that wh

Re: automatic rtbh trigger using flow data

2018-09-01 Thread Roland Dobbins
On 1 Sep 2018, at 1:43, Hugo Slabbert wrote: Generally on the TCP side you can try SYN or ACK floods, but you're not going to get an amplified reflection. Actually, TCP reflection/amplification has been on the increase; the attacker is guaranteed at least 4:1 amplification in most circumsta

Re: automatic rtbh trigger using flow data

2018-09-01 Thread Hugo Slabbert
On Sun 2018-Sep-02 10:09:32 +0700, Roland Dobbins wrote: On 1 Sep 2018, at 1:43, Hugo Slabbert wrote: Generally on the TCP side you can try SYN or ACK floods, but you're not going to get an amplified reflection. Actually, TCP reflection/amplification has been on the increase; the attacke

Re: automatic rtbh trigger using flow data

2018-09-01 Thread Hugo Slabbert
On Sun 2018-Sep-02 00:39:40 +, Ryan Hamel wrote: No ISP is in the business of filtering traffic unless the client pays the hefty fee since someone still has to tank the attack. If I can tag an RTBH community on a /32, what's the additional lost revenue in letting me be more granular and

RE: automatic rtbh trigger using flow data

2018-09-01 Thread Michel Py
> Roland Dobbins wrote : > I'm well aware of what's mentioned in the Arbor report, thanks! I would not have guessed :P > Ryan Hamel wrote : > No ISP is in the business of filtering traffic unless the client pays the > hefty fee since someone still has to tank the atack. I agree. In the end, it

Re: Service provider story about tracking down TCP RSTs

2018-09-01 Thread Lee
On 9/1/18, William Herrin wrote: > On Sat, Sep 1, 2018 at 6:11 PM, Lee wrote: >> On 9/1/18, William Herrin wrote: >>> On Sat, Sep 1, 2018 at 4:00 PM, William Herrin wrote: Better yet, do the job right and build an anycast TCP stack as described here: https://bill.herrin.us/network/any