Well. A reply post for the archives.... It seems like putting real IP addresses on the physical interfaces solved the whole problem. Before this, a dirty workaround was to have a script with "arp pings" in crontab.
Conclusion: Don't use CARP addresses only if these are connected against a Cisco HSRP gateway address. If you only have the CARP:s on no real IPs on each host you will see MAC/ARP issues. A little bit strange as it in general works very well with IP less interfaces together with CARP. /Per-Olov On Tuesday 21 March 2006 18.19, you wrote: > Hi misc > > > We have a firewall pair (A1 and B1) that is connected to the Internet by > talking to two Cisco routers that uses HSRP (A2 and B2). A small /28 > network connect it all together. A1 and B1 has a gw to the HSRP address on > the Cisco routers (A2 and B2). So my end is CARP and the other end (my > outgoing gateway) is Cisco HSRP... > > This is the overview config for the BSD firewall pair: > OpenBSD 3.8-STABLE (from late mars). All NIC:s are dual Intel server NIC:s > (em). GW in both servers are 1. > The outside switch is a brand new HP procurve gig switch. > A1 - No external IP > B1 - No external IP > external carp0 - IP 2 > external carp1 - IP 3 > external carp 26 - IP 7 > external carp 27 - IP 9 > external carp 28 - IP 13 > external carp 29 - IP 14 > The carp master/backup failover works ok. > > > This is the config I know for the cisco router pair: > A2 - IP 5 > B2 - IP 6 > HSRP IP - 1 > All our public IP ranges are routed from the cisco switches to carp IP 2 > and 3 on the BSD firewalls. > > > > Two times I have seen the following. I couple of hundreds of these show up. > And then then it took 4 hours and a new storm of these in the messages > log... Mar 21 10:42:15 A1 /bsd: arp: attempt to add entry for x.x.x.x.5 on > carp0 by 00:0a:8a:45:ed:00 on carp29 Mar 21 10:42:15 A1 /bsd: arp: attempt > to add entry for x.x.x.x.5 on carp0 by 00:0a:8a:45:ed:00 on carp28 Mar 21 > 10:42:15 A1 /bsd: arp: attempt to add entry for x.x.x.x.5 on carp0 by > 00:0a:8a:45:ed:00 on carp27 Mar 21 10:42:15 A1 /bsd: arp: attempt to add > entry for x.x.x.x.5 on carp0 by 00:0a:8a:45:ed:00 on carp26 Mar 21 10:42:15 > A1 /bsd: arp: attempt to add entry for x.x.x.x.5 on carp0 by > 00:0a:8a:45:ed:00 on carp1 Mar 21 10:42:17 A1 /bsd: arp: attempt to add > entry for x.x.x.x.6 on carp0 by 00:0a:b7:24:b3:00 on carp29 Mar 21 10:42:17 > A1 /bsd: arp: attempt to add entry for x.x.x.x.6 on carp0 by > 00:0a:b7:24:b3:00 on carp28 Mar 21 10:42:17 A1 /bsd: arp: attempt to add > entry for x.x.x.x.6 on carp0 by 00:0a:b7:24:b3:00 on carp27 Mar 21 10:42:17 > A1 /bsd: arp: attempt to add entry for x.x.x.x.6 on carp0 by > 00:0a:b7:24:b3:00 on carp26 Mar 21 10:42:17 A1 /bsd: arp: attempt to add > entry for x.x.x.x.6 on carp0 by 00:0a:b7:24:b3:00 on carp1 Mar 21 10:43:15 > A1 /bsd: arp: attempt to add entry for x.x.x.x.5 on carp0 by > 00:0a:8a:45:ed:00 on carp29 Mar 21 10:43:15 A1 /bsd: arp: attempt to add > entry for x.x.x.x.5 on carp0 by 00:0a:8a:45:ed:00 on carp28 Mar 21 10:43:15 > A1 /bsd: arp: attempt to add entry for x.x.x.x.5 on carp0 by > 00:0a:8a:45:ed:00 on carp27 Mar 21 10:43:15 A1 /bsd: arp: attempt to add > entry for x.x.x.x.5 on carp0 by 00:0a:8a:45:ed:00 on carp26 > > > And when the above happens all traffic to the internet stops for a while. > But before, between and after these four hour storms everything worked > perfect.... > > > I have double checked overlapping networks - no errors... > I have checked CVS for possible fixes of carp and em - nothing found... > I have double checked my carp configs that I have done many of before - > nothing found... > > > > Do I for any reason have to add IP:s to the A1 and B1 OpenBSD firewalls and > avoid using just the carp addresses? > > These BSD servers replace two Linux machines with iptables and VRRP. The > old setup did not have these issues. But Linux with VRRP inherited the > physical MAC which is not true for the carp interfaces... We probably have > to revert to Linux (no no no no arrgghhh) if we don't find this problem > fast. This as we cannot have problems like this with 70 Mbit throughput and > 25000 sessions.... > > > > > Any clues? > Cisco or OpenBSD errors? Or maybe brain damage of the configurator ;-) > > Thanks in advance > Per-Olov -- GPG keyID: 4DB283CE GPG fingerprint: 45E8 3D0E DE05 B714 D549 45BC CFB4 BBE9 4DB2 83CE
