I have two firewalls (FW1 & FW2) with multiple carp interfaces on an external interface (carp1, carp12, carp14, carp15, carp16, carp17, carp18, carp19, carp20). FW1 has all carp interfaces set with advbase 1 advskew 0 and FW2 has all carp interfaces with advbase 1 advskew 180. Frequently FW2 thinks it is the master for some of the carp interfaces. Here is a tcpdump (-ni fxp0 proto carp) from FW2. As you can see, even though FW2 sees the advertisement for carp16, carp17, carp18, carp19 and carp20 from FW1 it sometimes takes over as master for those interfaces and advertises. To find these events look for advskew=180 in the tcpdump below.
The event at 19:19:05.023848 seemed to be from lost packets. The event at 19:19:10.013844 is very odd since FW2 saw the carp20 advertisement from FW1 at 19:19:09.074444. This should be enough time for a failover, should it? Any pointers would be appreciated (relevant pf rules below.) -Steve S. 19:19:02.290779 CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:02.290807 CARPv2-advertise 36: vhid=12 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:02.290828 CARPv2-advertise 36: vhid=14 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:02.290849 CARPv2-advertise 36: vhid=15 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:02.290869 CARPv2-advertise 36: vhid=16 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:02.290887 CARPv2-advertise 36: vhid=17 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:02.290914 CARPv2-advertise 36: vhid=18 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:02.290936 CARPv2-advertise 36: vhid=19 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:02.290957 CARPv2-advertise 36: vhid=20 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:02.890823 CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:02.890849 CARPv2-advertise 36: vhid=12 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:02.890871 CARPv2-advertise 36: vhid=14 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:02.890892 CARPv2-advertise 36: vhid=15 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:02.890912 CARPv2-advertise 36: vhid=16 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:02.890933 CARPv2-advertise 36: vhid=17 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:02.890962 CARPv2-advertise 36: vhid=18 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:02.890986 CARPv2-advertise 36: vhid=19 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:02.891010 CARPv2-advertise 36: vhid=20 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:03.880791 CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:03.880818 CARPv2-advertise 36: vhid=12 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:03.880839 CARPv2-advertise 36: vhid=14 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:03.880860 CARPv2-advertise 36: vhid=15 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:03.880881 CARPv2-advertise 36: vhid=16 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:03.880901 CARPv2-advertise 36: vhid=17 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:03.880932 CARPv2-advertise 36: vhid=18 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:03.880955 CARPv2-advertise 36: vhid=19 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:03.880979 CARPv2-advertise 36: vhid=20 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:05.023848 CARPv2-advertise 36: vhid=17 advbase=1 advskew=180 (DF) [tos 0x10] 19:19:05.024936 CARPv2-advertise 36: vhid=18 advbase=1 advskew=180 (DF) [tos 0x10] 19:19:05.026003 CARPv2-advertise 36: vhid=19 advbase=1 advskew=180 (DF) [tos 0x10] 19:19:05.027069 CARPv2-advertise 36: vhid=20 advbase=1 advskew=180 (DF) [tos 0x10] 19:19:05.341023 CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:05.341047 CARPv2-advertise 36: vhid=12 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:05.341068 CARPv2-advertise 36: vhid=14 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:05.341088 CARPv2-advertise 36: vhid=15 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:05.341109 CARPv2-advertise 36: vhid=16 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:05.341129 CARPv2-advertise 36: vhid=17 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:05.341154 CARPv2-advertise 36: vhid=18 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:05.341176 CARPv2-advertise 36: vhid=19 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:05.341199 CARPv2-advertise 36: vhid=20 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:06.295736 CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:06.295760 CARPv2-advertise 36: vhid=12 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:06.295782 CARPv2-advertise 36: vhid=14 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:06.295802 CARPv2-advertise 36: vhid=15 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:06.295822 CARPv2-advertise 36: vhid=16 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:06.297299 CARPv2-advertise 36: vhid=17 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:06.297318 CARPv2-advertise 36: vhid=18 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:06.297335 CARPv2-advertise 36: vhid=19 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:06.297352 CARPv2-advertise 36: vhid=20 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:06.900831 CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:06.900876 CARPv2-advertise 36: vhid=12 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:06.900925 CARPv2-advertise 36: vhid=14 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:06.900973 CARPv2-advertise 36: vhid=15 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:06.901021 CARPv2-advertise 36: vhid=16 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:06.901069 CARPv2-advertise 36: vhid=17 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:06.901112 CARPv2-advertise 36: vhid=18 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:06.901158 CARPv2-advertise 36: vhid=19 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:06.901205 CARPv2-advertise 36: vhid=20 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:07.990804 CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:08.000806 CARPv2-advertise 36: vhid=12 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:08.010812 CARPv2-advertise 36: vhid=14 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:08.010832 CARPv2-advertise 36: vhid=15 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:08.010853 CARPv2-advertise 36: vhid=16 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:08.010874 CARPv2-advertise 36: vhid=17 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:08.010894 CARPv2-advertise 36: vhid=18 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:08.010913 CARPv2-advertise 36: vhid=19 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:08.010940 CARPv2-advertise 36: vhid=20 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:09.060841 CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:09.070814 CARPv2-advertise 36: vhid=12 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:09.074312 CARPv2-advertise 36: vhid=14 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:09.074332 CARPv2-advertise 36: vhid=15 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:09.074353 CARPv2-advertise 36: vhid=16 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:09.074373 CARPv2-advertise 36: vhid=17 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:09.074394 CARPv2-advertise 36: vhid=18 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:09.074414 CARPv2-advertise 36: vhid=19 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:09.074444 CARPv2-advertise 36: vhid=20 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:10.013844 CARPv2-advertise 36: vhid=20 advbase=1 advskew=180 (DF) [tos 0x10] 19:19:10.103204 CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:10.103228 CARPv2-advertise 36: vhid=12 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:10.103249 CARPv2-advertise 36: vhid=14 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:10.103268 CARPv2-advertise 36: vhid=15 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:10.103286 CARPv2-advertise 36: vhid=16 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:10.103305 CARPv2-advertise 36: vhid=17 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:10.103331 CARPv2-advertise 36: vhid=18 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:10.103353 CARPv2-advertise 36: vhid=19 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:10.103374 CARPv2-advertise 36: vhid=20 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:10.613850 CARPv2-advertise 36: vhid=19 advbase=1 advskew=180 (DF) [tos 0x10] 19:19:11.000985 CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:11.001009 CARPv2-advertise 36: vhid=12 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:11.001030 CARPv2-advertise 36: vhid=14 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:11.001050 CARPv2-advertise 36: vhid=15 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:11.001069 CARPv2-advertise 36: vhid=16 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:11.001087 CARPv2-advertise 36: vhid=17 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:11.001116 CARPv2-advertise 36: vhid=18 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:11.001138 CARPv2-advertise 36: vhid=19 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:11.001160 CARPv2-advertise 36: vhid=20 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:11.733838 CARPv2-advertise 36: vhid=20 advbase=1 advskew=180 (DF) [tos 0x10] 19:19:12.040852 CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:12.040877 CARPv2-advertise 36: vhid=12 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:12.040898 CARPv2-advertise 36: vhid=14 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:12.040920 CARPv2-advertise 36: vhid=15 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:12.040940 CARPv2-advertise 36: vhid=16 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:12.040960 CARPv2-advertise 36: vhid=17 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:12.040987 CARPv2-advertise 36: vhid=18 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:12.041011 CARPv2-advertise 36: vhid=19 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:12.041035 CARPv2-advertise 36: vhid=20 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:12.333855 CARPv2-advertise 36: vhid=19 advbase=1 advskew=180 (DF) [tos 0x10] 19:19:12.793851 CARPv2-advertise 36: vhid=17 advbase=1 advskew=180 (DF) [tos 0x10] 19:19:12.794962 CARPv2-advertise 36: vhid=18 advbase=1 advskew=180 (DF) [tos 0x10] 19:19:13.110832 CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:13.110857 CARPv2-advertise 36: vhid=12 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:13.110878 CARPv2-advertise 36: vhid=14 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:13.110900 CARPv2-advertise 36: vhid=15 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:13.110920 CARPv2-advertise 36: vhid=16 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:13.110941 CARPv2-advertise 36: vhid=17 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:13.110967 CARPv2-advertise 36: vhid=18 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:13.110991 CARPv2-advertise 36: vhid=19 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:13.111014 CARPv2-advertise 36: vhid=20 advbase=1 advskew=0 (DF) [tos 0x10] 19:19:13.453851 CARPv2-advertise 36: vhid=20 advbase=1 advskew=180 (DF) [tos 0x10] OpenBSD 3.8-stable (GENERIC.MP) #0: Thu Jan 5 03:55:53 EST 2006 [EMAIL PROTECTED] pfctl -sr |grep -i carp pass quick on fxp0 proto carp all keep state pass quick on fxp1 proto carp all keep state pass quick on fxp2 proto carp all keep state pass quick on fxp3 proto carp all keep state pass quick on tl0 proto carp all keep state pass quick on tl1 proto carp all keep state
