Hello,

I have EC2 instance with a custom FreeBSD kernel (9.2-RELEASE) with two 
interfaces – nic0 & nic1. Both have elastic IPs associated  - E0 & E1.

I noticed that when I ping E1, the ICMP response packets go out of nic1 instead 
of nic0 & thus, from the ping source perspective, host is down. To workaround 
the asymmetric routing, I added a default route via nic1 to FIB 1 & added an 
IPFW rule to ensure FIB 1 is used for all packets received via nic1.

Now, when nic1 tries to respond to an ICMP request, the first thing it does is 
to ARP for the router’s IP. The router responds with its MAC address but nic1 
doesn’t seem to be processing it. The entry for default router’s IP in the ARP 
table remains marked ‘incomplete’ and never gets updated -

*********************

# tcpdump -ni nic1 -p arp

07:29:04.922209 ARP, Request who-has 172.31.16.1 tell 172.31.30.133, length 28

07:29:04.922316 ARP, Reply 172.31.16.1 is-at 02:18:c9:53:d0:2f, length 42


# arp -a

ip-172-31-16-1.us-west-2.compute.internal (172.31.16.1) at (incomplete) on nic1 
expired [ethernet]

ip-172-31-30-133.us-west-2.compute.internal (172.31.30.133) at 
02:6e:32:90:0e:ad on nic1 permanent [ethernet]

ip-172-31-16-1.us-west-2.compute.internal (172.31.16.1) at 02:18:c9:53:d0:2f on 
nic0 expires in 1199 seconds [ethernet]

ip-172-31-22-113.us-west-2.compute.internal (172.31.22.113) at 
02:ea:39:ce:5a:49 on nic0 permanent [ethernet]

*********************
But… if I delete the arp entry (arp –d 172.31.16.1), ARP entry for the default 
router IP gets updated properly -


*********************

# arp -a

ip-172-31-16-1.us-west-2.compute.internal (172.31.16.1) at 02:18:c9:53:d0:2f on 
nic1 expires in 1195 seconds [ethernet]

…

*********************

But but… this only lasts until the entry expires. After that, the entry gets 
marked ‘incomplete’ again.


I also notice these ARP requests start being received from the router around 
the same time I delete the entry (may the source address of these packets are 
being used to update the ARP entry?) -

*********************

07:32:41.411229 ARP, Request who-has 172.31.16.1 tell 172.31.30.133, length 28

07:32:41.411297 ARP, Reply 172.31.16.1 is-at 02:18:c9:53:d0:2f, length 42

07:33:04.482719 ARP, Request who-has 172.31.30.133 tell 172.31.16.1, length 42 
<<<<<<

07:33:04.482734 ARP, Reply 172.31.30.133 is-at 02:6e:32:90:0e:ad, length 28

*********************


Any idea why the ARP replies are not being processed by nic1? Any known 
workarounds?


Thanks,

Rohit



PS –

FIB 1:

Internet:

Destination        Gateway            Flags    Refs      Use  Netif Expire

default            172.31.16.1        UGS         0     1985   nic1

127.0.0.1          link#2             UH          0        0    lo0

172.31.16.0/20     link#4             U           0        0   nic1


IFPW -

00002 setfib 1 ip from any to any via nic1

65535 allow ip from any to any
_______________________________________________
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

Reply via email to