This query here talks about a similar scenario - https://forums.freebsd.org/threads/26050/ Only difference being that in my case, ARP replies are actually received.
Thanks, Rohit On 07/12/16, 1:12 PM, "owner-freebsd-...@freebsd.org on behalf of Rohit Atri (ratri)" <owner-freebsd-...@freebsd.org on behalf of ra...@cisco.com> wrote: >Hello, > >I have EC2 instance with a custom FreeBSD kernel (9.2-RELEASE) with two >interfaces – nic0 & nic1. Both have elastic IPs associated - E0 & E1. > >I noticed that when I ping E1, the ICMP response packets go out of nic1 >instead of nic0 & thus, from the ping source perspective, host is down. To >workaround the asymmetric routing, I added a default route via nic1 to FIB 1 & >added an IPFW rule to ensure FIB 1 is used for all packets received via nic1. > >Now, when nic1 tries to respond to an ICMP request, the first thing it does is >to ARP for the router’s IP. The router responds with its MAC address but nic1 >doesn’t seem to be processing it. The entry for default router’s IP in the ARP >table remains marked ‘incomplete’ and never gets updated - > >********************* > ># tcpdump -ni nic1 -p arp > >07:29:04.922209 ARP, Request who-has 172.31.16.1 tell 172.31.30.133, length 28 > >07:29:04.922316 ARP, Reply 172.31.16.1 is-at 02:18:c9:53:d0:2f, length 42 > > ># arp -a > >ip-172-31-16-1.us-west-2.compute.internal (172.31.16.1) at (incomplete) on >nic1 expired [ethernet] > >ip-172-31-30-133.us-west-2.compute.internal (172.31.30.133) at >02:6e:32:90:0e:ad on nic1 permanent [ethernet] > >ip-172-31-16-1.us-west-2.compute.internal (172.31.16.1) at 02:18:c9:53:d0:2f >on nic0 expires in 1199 seconds [ethernet] > >ip-172-31-22-113.us-west-2.compute.internal (172.31.22.113) at >02:ea:39:ce:5a:49 on nic0 permanent [ethernet] > >********************* >But… if I delete the arp entry (arp –d 172.31.16.1), ARP entry for the default >router IP gets updated properly - > > >********************* > ># arp -a > >ip-172-31-16-1.us-west-2.compute.internal (172.31.16.1) at 02:18:c9:53:d0:2f >on nic1 expires in 1195 seconds [ethernet] > >… > >********************* > >But but… this only lasts until the entry expires. After that, the entry gets >marked ‘incomplete’ again. > > >I also notice these ARP requests start being received from the router around >the same time I delete the entry (may the source address of these packets are >being used to update the ARP entry?) - > >********************* > >07:32:41.411229 ARP, Request who-has 172.31.16.1 tell 172.31.30.133, length 28 > >07:32:41.411297 ARP, Reply 172.31.16.1 is-at 02:18:c9:53:d0:2f, length 42 > >07:33:04.482719 ARP, Request who-has 172.31.30.133 tell 172.31.16.1, length 42 ><<<<<< > >07:33:04.482734 ARP, Reply 172.31.30.133 is-at 02:6e:32:90:0e:ad, length 28 > >********************* > > >Any idea why the ARP replies are not being processed by nic1? Any known >workarounds? > > >Thanks, > >Rohit > > > >PS – > >FIB 1: > >Internet: > >Destination Gateway Flags Refs Use Netif Expire > >default 172.31.16.1 UGS 0 1985 nic1 > >127.0.0.1 link#2 UH 0 0 lo0 > >172.31.16.0/20 link#4 U 0 0 nic1 > > >IFPW - > >00002 setfib 1 ip from any to any via nic1 > >65535 allow ip from any to any >_______________________________________________ >freebsd-net@freebsd.org mailing list >https://lists.freebsd.org/mailman/listinfo/freebsd-net >To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org" _______________________________________________ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
