Jim Westfall <jwestf...@surrealistic.net> wrote [01.11.18]:
> Hi
>
> I'm seeing some weird behavior related to NOARP devices and NOARP
> entries in the arp cache.
>
> I have a couple gre tunnels between a linux box and a upstream router that
> send/recv a large amount of packets with unique ips. On the order of 10k+
> unique ips per second seen by the linux box.
>
> Each one of the ips ends up getting added to the arp cache as
>
> <ip> dev tun1234 lladdr 0.0.0.0 NOARP
>
> This of course makes the arp cache grow extremely fast and overflow.
> While I can tweak gc_thresh1/2/3 to make arp cache size huge, it doesn't
> seem like the right answer as the kernel is spinning its wheels having to
> adding/expunging entries for the high rate of unique ips.
>
> I'm unclear why the kernel is even tracking them in the arp cache. If
> routing table says to route the packet out a NOARP interface then there is
> no arp, why involve the arp cache at all?
>
> You can see the behavior with the following
>
> [root@jwestfall:~]# uname -a
> Linux jwestfall.jwestfall.net 4.14.10_1 #1 SMP PREEMPT Sun Dec 31 20:23:29
> UTC 2017 x86_64 GNU/Linux
>
> [root@jwestfall:~]# ip neigh show nud noarp
> 10.0.0.172 dev lo lladdr 00:00:00:00:00:00 NOARP
> 10.70.50.5 dev tun0 lladdr 08 NOARP
> 127.0.0.1 dev lo lladdr 00:00:00:00:00:00 NOARP
>
> Setup a bogus gre tunnel, the remote ip doesn't matter
> [root@jwestfall:~]# ip tunnel add tun1234 mode gre local 10.0.0.172 remote
> 10.0.0.156 dev enp4s0
> [root@jwestfall:~]# ip link set up dev tun1234
>
> Route a bogus network to the tunnel
> [root@jwestfall:~]# ip route add 192.168.111.0/24 dev tun1234
>
> Ping ips on the bogus network
> [root@jwestfall:~]# nmap -sP 192.168.111.0/24
>
> Starting Nmap 7.60 ( https://nmap.org ) at 2018-01-11 12:06 PST
> ...
>
> [root@jwestfall:~]# ip neigh show nud noarp
> 192.168.111.18 dev tun1234 lladdr 0.0.0.0 NOARP
> 192.168.111.4 dev tun1234 lladdr 0.0.0.0 NOARP
> 192.168.111.28 dev tun1234 lladdr 0.0.0.0 NOARP
> 192.168.111.17 dev tun1234 lladdr 0.0.0.0 NOARP
> 192.168.111.14 dev tun1234 lladdr 0.0.0.0 NOARP
> 192.168.111.34 dev tun1234 lladdr 0.0.0.0 NOARP
> 192.168.111.3 dev tun1234 lladdr 0.0.0.0 NOARP
> 192.168.111.20 dev tun1234 lladdr 0.0.0.0 NOARP
> 10.0.0.172 dev lo lladdr 00:00:00:00:00:00 NOARP
> 192.168.111.6 dev tun1234 lladdr 0.0.0.0 NOARP
> 192.168.111.27 dev tun1234 lladdr 0.0.0.0 NOARP
> 192.168.111.13 dev tun1234 lladdr 0.0.0.0 NOARP
> 192.168.111.33 dev tun1234 lladdr 0.0.0.0 NOARP
> ...
>
> Also somewhat interesting is that on older kernels (3.2 time range) these
> NOARP entries didn't get added for ipv4, but they did for ipv6 if you
> pushed ipv6 through the ipv4 tunnel.
>
> 2804:14c:f281:a1d8:61a2:a30:989f:3eb1 dev tun1 lladdr 0.0.0.0 NOARP
> 2607:8400:2122:4:e9f9:dbb8:2d44:75d1 dev tun2 lladdr 0.0.0.0 NOARP
>
> Thanks
> Jim Westfall
>
>
Digging into this a bit in older kernels there was the following
static struct neighbour *ipv4_neigh_lookup(const struct dst_entry *dst, const
void *daddr)
{
static const __be32 inaddr_any = 0;
struct net_device *dev = dst->dev;
const __be32 *pkey = daddr;
struct neighbour *n;
if (dev->flags & (IFF_LOOPBACK | IFF_POINTOPOINT))
pkey = &inaddr_any;
which was forcing the hash key to be 0.0.0.0 for tunnels. This was removed as
part of a263b3093641fb1ec377582c90986a7fd0625184 which was part of a larger set
that "Disconnect neigh from dst_entry"
Would there be any aversion to me submitting a patch to mimic this older
behavior?
Thanks
jim
