Re: TCP and anycast (was Re: ECN)

2019-11-16 Thread Scott Weeks
--- ra...@psg.com wrote: lots of good research lit on catchment topology of anycasted dns, which is very non-local. --- For the others here that didn't know what that is and are curious. I couldn't take it and just had to know... :) https://tools.ietf.org/ht

Re: TCP and anycast (was Re: ECN)

2019-11-14 Thread William Herrin
On Thu, Nov 14, 2019 at 1:10 AM Bill Woodcock wrote: > > On Nov 14, 2019, at 7:39 AM, Anoop Ghanwani wrote: > > RFC 7094 (https://tools.ietf.org/html/rfc7094) describes the pitfalls & risks of using TCP with an anycast address. It recognizes that there are valid use cases for it, though. > > Spe

Re: TCP and anycast (was Re: ECN)

2019-11-14 Thread Randy Bush
>>> RFC 7094 (https://tools.ietf.org/html/rfc7094) describes the pitfalls >>> & risks of using TCP with an anycast address. >> >> and two decades of operational experience are that prudent deployments >> just work. > > I agree with Bill/Randy here... this does just work if you understand > your lo

Re: TCP and anycast (was Re: ECN)

2019-11-14 Thread Christopher Morrow
On Fri, Nov 15, 2019 at 1:54 AM Randy Bush wrote: > > > RFC 7094 (https://tools.ietf.org/html/rfc7094) describes the pitfalls > > & risks of using TCP with an anycast address. > > and two decades of operational experience are that prudent deployments > just work. I agree with Bill/Randy here... t

Re: TCP and anycast (was Re: ECN)

2019-11-14 Thread Randy Bush
> RFC 7094 (https://tools.ietf.org/html/rfc7094) describes the pitfalls > & risks of using TCP with an anycast address. and two decades of operational experience are that prudent deployments just work. randy

Re: ECN

2019-11-14 Thread Toke Høiland-Jørgensen via NANOG
Owen DeLong writes: > Like it or not (and I really don’t), the majority of modern CDNs are > using TCP over Anycast. > > It’s ugly and it’s prone to problems like this. It’s nice to see a > customer with know-how actually publicizing and digging into the > problem. Thanks. I do plan to write thi

Re: ECN

2019-11-14 Thread Toke Høiland-Jørgensen via NANOG
Baldur Norddahl writes: > I am testing disabling our use of ECMP as it is not strictly necessary > and we are moving to a new platform anyway. Waiting for feedback from > the customer to hear if this fixes the issue. Which I can confirm that it does. Thank you for the speedy resolution! :) -Tok

Re: ECN

2019-11-14 Thread Saku Ytti
On Wed, 13 Nov 2019 at 22:57, Lukas Tribus wrote: > In fact I believe everything beyond the 5-tuple is just a bad idea to > base your hash on. Here are some examples (not quite as straight > forward than the TOS/ECN case here): ACK. > TTL: > https://mailman.nanog.org/pipermail/nanog/2018-Septe

Re: TCP and anycast (was Re: ECN)

2019-11-14 Thread Bill Woodcock
> On Nov 14, 2019, at 7:39 AM, Anoop Ghanwani wrote: > RFC 7094 (https://tools.ietf.org/html/rfc7094) describes the pitfalls & risks > of using TCP with an anycast address. It recognizes that there are valid use > cases for it, though. > Specifically, section 3.1 says this: >Most statefu

Re: ECN

2019-11-13 Thread Tore Anderson
* Saku Ytti > Not true. Hash result should indicate discreet flow, more importantly > discreet flow should not result into two unique hash numbers. Using > whole TOS byte breaks this promise and thus breaks ECMP. > > Platforms allow you to configure which bytes are part of hash > calculation, wh

TCP and anycast (was Re: ECN)

2019-11-13 Thread Anoop Ghanwani
RFC 7094 (https://tools.ietf.org/html/rfc7094) describes the pitfalls & risks of using TCP with an anycast address. It recognizes that there are valid use cases for it, though. Specifically, section 3.1 says this: >>> Most stateful transport protocols (e.g., TCP), without modification, do

Re: ECN

2019-11-13 Thread Warren Kumari
On Thu, Nov 14, 2019 at 12:25 AM Matt Corallo wrote: > > This sounds like a bug on Cloudflare’s end (cause trying to do anycast TCP > is... out of spec to say the least), not a bug in ECN/ECMP. Err. I really don't think that there is any sort of spec that covers that :-P Using Anycast for T

Re: ECN

2019-11-13 Thread Lukas Tribus
Hello, On Wed, Nov 13, 2019 at 8:35 PM Saku Ytti wrote: > > On Wed, 13 Nov 2019 at 18:27, Matt Corallo wrote: > > > This sounds like a bug on Cloudflare’s end (cause trying to do anycast TCP > > is... out of spec to say the least), not a bug in ECN/ECMP. > > Not true. Hash result should indicat

Re: ECN

2019-11-13 Thread William Herrin
On Wed, Nov 13, 2019 at 11:36 AM Saku Ytti wrote: > On Wed, 13 Nov 2019 at 18:27, Matt Corallo wrote: > > This sounds like a bug on Cloudflare’s end (cause trying to do anycast > TCP is... out of spec to say the least), not a bug in ECN/ECMP. > > Not true. Hash result should indicate discreet fl

Re: ECN

2019-11-13 Thread Owen DeLong
Like it or not (and I really don’t), the majority of modern CDNs are using TCP over Anycast. It’s ugly and it’s prone to problems like this. It’s nice to see a customer with know-how actually publicizing and digging into the problem. Until now, I believe an unknown number of customers have been

Re: ECN

2019-11-13 Thread Saku Ytti
On Wed, 13 Nov 2019 at 18:27, Matt Corallo wrote: > This sounds like a bug on Cloudflare’s end (cause trying to do anycast TCP > is... out of spec to say the least), not a bug in ECN/ECMP. Not true. Hash result should indicate discreet flow, more importantly discreet flow should not result into

Re: ECN

2019-11-13 Thread Toke Høiland-Jørgensen via NANOG
On 13 November 2019 17:20:18 CET, Matt Corallo wrote: >This sounds like a bug on Cloudflare’s end (cause trying to do anycast >TCP is... out of spec to say the least), not a bug in ECN/ECMP. Even without anycast, an ECMP shouldn't hash on the ECN bits. Doing so will split the flow over multip

Re: ECN

2019-11-13 Thread Baldur Norddahl
ZTE M6000-S V3.00.20(3.40.1) We are moving away from this platform so I can not be bothered with requesting a fix. In the past they have made fixes for us, so I believe they would also fix this issue if we asked them to do so. Also I would like to state that I have not personally verified that th

Re: ECN

2019-11-13 Thread Mikael Abrahamsson via NANOG
On Wed, 13 Nov 2019, Baldur Norddahl wrote: In any case, is it not recommended that users of anycast proxy packets that arrive at the wrong place? To avoid this kind of issue. In typical anycast deployments there is no feasible way to figure out where the "right place" is. It would be very

Re: ECN

2019-11-13 Thread Baldur Norddahl
I am testing disabling our use of ECMP as it is not strictly necessary and we are moving to a new platform anyway. Waiting for feedback from the customer to hear if this fixes the issue. In any case, is it not recommended that users of anycast proxy packets that arrive at the wrong place? To avoid

Re: ECN

2019-11-13 Thread Jon Lewis
It does when the split flows land in different anycast origin POPs. Making a few assumptions from the traceroutes, the ECMP paths are sending some packets to Hamburg and some to Denmark. Each POP may be getting parts of what should be a single TCP stream, and I doubt they have anything to cope

Re: ECN

2019-11-13 Thread Todd Underwood
as one of the authors of that talk, it definitely is "a thing", has been for years and years and years, and indeed, mostly works. t On Wed, Nov 13, 2019 at 12:18 PM Hunter Fuller wrote: > It is certainly odd, but it's definitely a "thing." > > https://archive.nanog.org/meetings/nanog37/presenta

Re: ECN

2019-11-13 Thread Anoop Ghanwani
Not to condone what cloudflare is doing, but... An ECN connection will have different bits on various packets for the duration of the connection -- pure ACKs (ACKs not piggybacking on data) will have the ECN bits as 00b, while all other packets will have either 01b, 10b (when no congestion was exp

Re: ECN

2019-11-13 Thread Hunter Fuller
It is certainly odd, but it's definitely a "thing." https://archive.nanog.org/meetings/nanog37/presentations/matt.levine.pdf On Wed, Nov 13, 2019 at 10:24 AM Matt Corallo wrote: > > This sounds like a bug on Cloudflare’s end (cause trying to do anycast TCP > is... out of spec to say the least),

Re: ECN

2019-11-13 Thread Matt Corallo
Not ideal, sure, but if it’s only for the SYN (as you seem to indicate), splitting the flow shouldn’t have material performance degradation? > On Nov 13, 2019, at 11:51, Toke Høiland-Jørgensen wrote: > >  > >> On 13 November 2019 17:20:18 CET, Matt Corallo wrote: >> This sounds like a bug o

Re: ECN

2019-11-13 Thread Matt Corallo
This sounds like a bug on Cloudflare’s end (cause trying to do anycast TCP is... out of spec to say the least), not a bug in ECN/ECMP. > On Nov 13, 2019, at 11:07, Toke Høiland-Jørgensen via NANOG > wrote: > >  >> >> Hello >> >> I have a customer that believes my network has a ECN problem.

Re: ECN

2019-11-13 Thread Toke Høiland-Jørgensen via NANOG
> Hello > > I have a customer that believes my network has a ECN problem. We do > not, we just move packets. But how do I prove it? > > Is there a tool that checks for ECN trouble? Ideally something I could > run on the NLNOG Ring network. > > I believe it likely that it is the destination that

Re: ECN

2019-11-11 Thread Owen DeLong
> On Nov 11, 2019, at 05:01 , Baldur Norddahl wrote: > > Hello > > I have a customer that believes my network has a ECN problem. We do not, we > just move packets. But how do I prove it? Are you saying that none of your routers support ECN or that you think ECN only applies to endpoints?

Re: ECN, DNS and Firewalls

2018-12-27 Thread Mark Andrews
> On 28 Dec 2018, at 2:49 pm, valdis.kletni...@vt.edu wrote: > > On Fri, 28 Dec 2018 13:35:04 +1100, Mark Andrews said: >> There are major operators that still have STUPID firewall settings >> in front of DNS servers that drop SYN packets with ECE and CWR set >> 17 years after ECN was specified

Re: ECN, DNS and Firewalls

2018-12-27 Thread valdis . kletnieks
On Fri, 28 Dec 2018 13:35:04 +1100, Mark Andrews said: > There are major operators that still have STUPID firewall settings > in front of DNS servers that drop SYN packets with ECE and CWR set > 17 years after ECN was specified. Time to name-n-shame?

Re: ECN

2008-11-08 Thread Hank Nussbacher
On Fri, 7 Nov 2008, [EMAIL PROTECTED] wrote: On Fri, 07 Nov 2008 08:27:58 +0100, Mikael Abrahamsson said: for ECN to actually be useful, we (the ISPs) have to turn this option on in the routers as well. Is anyone doing this today? What vendors support it? The only thing that's *required* for

Re: ECN

2008-11-07 Thread Valdis . Kletnieks
On Fri, 07 Nov 2008 08:27:58 +0100, Mikael Abrahamsson said: > for ECN to actually be useful, we (the ISPs) have to turn this option on > in the routers as well. Is anyone doing this today? What vendors support > it? The only thing that's *required* for it to help is that the routers and firewal

Re: ECN

2008-11-07 Thread Mikael Abrahamsson
On Fri, 7 Nov 2008, David Freedman wrote: Implementing this in an MPLS core is not an easy task, you can really only do this on the edge, when the MPLS labelled packet arrives at an LSR, we don't know if it contains a TCP segment or not (fancy deep h/w implementations excluded), all we know is

Re: ECN

2008-11-07 Thread David Freedman
Interesting , I hadn't followed this since draft-ietf-mpls-ecn-00, , I eagerly await a vendor implementation :) Dave. Bjørn Mork wrote: > David Freedman <[EMAIL PROTECTED]> writes: > >> Implementing this in an MPLS core is not an easy task, you can really >> only do this on the edge, when the MP

Re: ECN

2008-11-07 Thread Bjørn Mork
David Freedman <[EMAIL PROTECTED]> writes: > Implementing this in an MPLS core is not an easy task, you can really > only do this on the edge, when the MPLS labelled packet arrives at an > LSR, we don't know if it contains a TCP segment or not (fancy deep h/w > implementations excluded), all we kn

Re: ECN

2008-11-07 Thread David Freedman
> When I thought about it, the IP core (10G links etc) first came to mind, > and there it's fairly easy to roll out (since I guess a lot of us do > WRED already), but what about on slower links? Would it make sense to > have our DSLAMs do this? What about DSL/cable modems (well, vendors > should fi