On 5/20/20 11:24 AM, Christian Brauner wrote: > On Wed, May 20, 2020 at 10:54:21AM -0600, David Ahern wrote: >> On 5/20/20 8:58 AM, Christian Brauner wrote: >>> During NorthSec (cf. [1]) a very large number of unprivileged >>> containers and nested containers are run during the competition to >>> provide a safe environment for the various teams during the event. Every >>> year a range of feature requests or bug reports come out of this and >>> this year's no different. >>> One of the containers was running a simple VPN server. There were about >>> 1.5k users connected to this VPN over ipv6 and the container was setup >>> with about 100 custom routing tables when it hit the max_sizes routing >>> limit. After this no new connections could be established anymore, >>> pinging didn't work anymore; you get the idea. >>> >> >> should have been addressed by: >> >> commit d8882935fcae28bceb5f6f56f09cded8d36d85e6 >> Author: Eric Dumazet <eduma...@google.com> >> Date: Fri May 8 07:34:14 2020 -0700 >> ipv6: use DST_NOCOUNT in ip6_rt_pcpu_alloc() >> We currently have to adjust ipv6 route gc_thresh/max_size depending >> on number of cpus on a server, this makes very little sense. >> >> >> Did your tests include this patch? > > No, it's also pretty hard to trigger. The conference was pretty good for > this. > I tested on top of rc6. I'm probably missing the big picture here, could > you briefy explain how this commit fixes the problem we ran into? >
ipv6 still has limits on the number of dst_entry's that can be created. Eric traced the overflow to per-cpu caches in each FIB entry. Larger systems (lots of cpus) x lots of unique connections = overflow Eric's change removes the per-cpu dst caches from the counting, so only exceptions (mtu, redirect) are now counted towards the limit.
