On Thu, Nov 10, 2005 at 09:31:15PM -0500, Nick Holland wrote: > I'd have prefered that a more experienced person answer this one, but > they don't seem to have, so be forewarned: everything I say here might > be wrong. However, through the glory of mail lists, if I say something > wrong, fifty people will jump all over me, and Google will put it at the > top of the list when people google for my name. :)
Consider it done! > > I set up two OpenBSD 3.7 -stable firewalls using carp. Everything works > > except preemption. > > > > When only one interface on the master side fails (pull the Cable) the > > regarding carp0 interface on the backup side becomes master. But not > > carp1. > > Right. Nothing's wrong with the master carp1, why should it demote > itself and have the backup take over? Because that is what preemption is supposed to do. When one interface on the carp master goes into BACKUP state (or is it any state that is not MASTER?), the others should become BACKUPs too. My experience is *sometimes* this is not instantaneous. At a minimum, the advskew should change and they should become BACKUPs in short order. > > I waited some minutes, but carp1 keeps being backup until I do a simple > > ifconfig(8) on the master side. Then it changes immediately. > > yep. > (though I'm not entirely sure I know what command you are typing by > "simple ifconfig(8)".) > > > I can reproduce it, waiting some minutes, or only a fiew seconds. Once I > > do an ifconfig on the master side, the backup side becomes master on all > > carp's. Strange...? > > not really, if you understand the modular approach here. > > > My config: > ... > > > Can anybody reproduce it, and has a solution for this problem? > > Any help would be very nice! :-) > > Look at the pieces here: > * CARP gives you redunancy on your INTERFACES...not your entire firewall. > * pfsync keeps your firewall state tables in sync, so either machine can > take over. > > If you lose a box completely, your system is fine. If you lose one > cable or one NIC or so on, you have a problem. That is definitely not true. Preemption is the answer here. If one carp interface fails, they all fail. Without preemption you either have a really good reason to be not using it or have a way to deal with such a situation. Imagine the typical situation: $wan_if, $lan_if, and $sync_if. Your run of the mill two legged failover setup. With preemption, if one or more of $wan_if/$lan_if fails, all other carp interfaces fail. Without preemption, if $wan_if fails, $lan_if is still the master and you've got a situation on your hands -- if all of $lan_if:network is using the current LAN master as their gateway, how is that host going to get out? Unless you play some tricks with ospf, bgpd or heck, even ifstated like I've done in the past, routing will fail. This is why preemption is a good choice in many cases. > What you need is something that will watch all interfaces and shut down > ALL (forcing a COMPLETE fail-over) if something goes wrong with any. > > That's a third part of the CARP toolset: ifstated(8) and ifstated(5). > > Yes, that's missing from the PF FAQ, though I just tossed a couple links > in faq/pf/carp.html. More will get added when I get more knowledge of > the topic (or Joel writes it :) Yes, you *can* do this with ifstated, but I'm not sure how recommended it is. I think the stock example that comes with ifstated is going down this path, but I'm not 100% sure. My suggestion would be to see that the advskew changes on the other carp interfaces when carp0 becomes a backup. If they do, that means preemption is definitely turned on and should work. -jon
