Yesterday, while troubleshooting a rdr on the pair of openBSD 4.3
firewalls we use here I discovered there was a rule that required a
particular IP to be listed as an alias on the WAN interface. I used
ifconfig to add the alias to the interface and this brought our network
down. I didn't realize that the IP I added as the alias was already
being used as an the IP of the physical WAN interface of the BACKUP
firewall.
Here is where things started to get wonky: I then removed the alias from
the firewall. The box failed over to the secondary at this point, and
when that happened, about 10% of our packets started dropping. I tried
to bring the primary back as the main firewall, but it didn't seem to
want to respond. I rebooted out of desperation, and when the main box
came back, the CARP LAN interface remained in an INIT state, which meant
the secondary, which drops 10% of its packets, was still acting as the
gateway. I was able to get it to accept the Carp IP, and after taking
down the secondary, things went back to stable. I booted the secondary,
and for some reason it tried to take over as the MASTER, while its CARP
LAN interface would also not go beyond the INIT state. I had to shut it
down and give the main fw back its priority.
Anyway, the state of things now is that when I bring either machine up,
the CARP LAN interface will not move from its INIT state. The secondary
firewall dropping packets might be unrelated. I guess I'm looking for a
direction toward which to start troubleshooting. I was going to try to
upgrade to 4.4, but I wanted to get some advice first. I'll include a
dmesg and the carp interface configs.
*Main FW dmesg:
OpenBSD 4.3 (GENERIC) #1368: Wed Mar 12 11:05:31 MDT 2008
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC
real mem = 468250624 (446MB)
avail mem = 442597376 (422MB)
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.4 @ 0xf0000 (67 entries)
bios0: vendor Phoenix Technologies, LTD version "3.09" date 06/14/2006
bios0: Compaq Presario 061 EX310AA-ABA SR1910NX NA630
acpi0 at bios0: rev 0
acpi0: tables DSDT FACP SSDT MCFG APIC
acpi0: wakeup devices HUB0(S5) XVRA(S5) XVRB(S5) XVRC(S5) USB0(S3)
USB2(S3) AZAD(S5) MMAC(S5) MMCI(S5) UAR1(S5) PS2M(S4) PS2K(S4)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 3 (HUB0)
acpicpu0 at acpi0: PSS
acpitz0 at acpi0: critical temperature 75 degC
acpibtn0 at acpi0: PWRB
cpu0 at mainbus0: (uniprocessor)
cpu0: AMD Sempron(tm) Processor 3200+, 1804.01 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SSE3,NXE,MMXX,FFXSR,LONG,3DNOW2,3DNOW
cpu0: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 256KB
64b/line 16-way L2 cache
cpu0: ITLB 32 4KB entries fully associative, 8 4MB entries fully associative
cpu0: DTLB 32 4KB entries fully associative, 8 4MB entries fully associative
cpu0: AMD erratum 89 present, BIOS upgrade may be required
cpu0: Cool'n'Quiet K8 1804 MHz: speeds: 1800 1000 MHz
pci0 at mainbus0 bus 0: configuration mode 1
"NVIDIA C51 Host" rev 0xa2 at pci0 dev 0 function 0 not configured
"NVIDIA C51 Memory" rev 0xa2 at pci0 dev 0 function 1 not configured
"NVIDIA C51 Memory" rev 0xa2 at pci0 dev 0 function 2 not configured
"NVIDIA C51 Memory" rev 0xa2 at pci0 dev 0 function 3 not configured
"NVIDIA C51 Memory" rev 0xa2 at pci0 dev 0 function 4 not configured
"NVIDIA C51 Memory" rev 0xa2 at pci0 dev 0 function 5 not configured
"NVIDIA C51 Memory" rev 0xa2 at pci0 dev 0 function 6 not configured
"NVIDIA C51 Memory" rev 0xa2 at pci0 dev 0 function 7 not configured
ppb0 at pci0 dev 2 function 0 "NVIDIA C51 PCIE" rev 0xa1
pci1 at ppb0 bus 1
ppb1 at pci0 dev 4 function 0 "NVIDIA C51 PCIE" rev 0xa1
pci2 at ppb1 bus 2
vga1 at pci0 dev 5 function 0 "NVIDIA GeForce 6150 LE" rev 0xa2
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
"NVIDIA MCP51 Host" rev 0xa2 at pci0 dev 9 function 0 not configured
pcib0 at pci0 dev 10 function 0 "NVIDIA MCP51 ISA" rev 0xa3
nviic0 at pci0 dev 10 function 1 "NVIDIA MCP51 SMBus" rev 0xa3
iic0 at nviic0
adt0 at iic0 addr 0x2e: sch5017 rev 0x8a
spdmem0 at iic0 addr 0x50: 256MB DDR SDRAM non-parity PC3200CL3.0
spdmem1 at iic0 addr 0x51: 256MB DDR SDRAM non-parity PC3200CL3.0
iic1 at nviic0
"NVIDIA MCP51 Memory" rev 0xa3 at pci0 dev 10 function 2 not configured
ohci0 at pci0 dev 11 function 0 "NVIDIA MCP51 USB" rev 0xa3: couldn't
map interrupt
ehci0 at pci0 dev 11 function 1 "NVIDIA MCP51 USB" rev 0xa3: couldn't
map interrupt
pciide0 at pci0 dev 13 function 0 "NVIDIA MCP51 IDE" rev 0xa1: DMA,
channel 0 configured to compatibility, channel 1 configured to compatibility
pciide0: channel 0 disabled (no drives)
atapiscsi0 at pciide0 channel 1 drive 0
scsibus0 at atapiscsi0: 2 targets
cd0 at scsibus0 targ 0 lun 0: <LITE-ON, COMBO SOHC-4836K, SPJ2> SCSI0
5/cdrom removable
cd0(pciide0:1:0): using PIO mode 4, Ultra-DMA mode 2
pciide1 at pci0 dev 14 function 0 "NVIDIA MCP51 SATA" rev 0xa1: DMA
pciide1: using irq 11 for native-PCI interrupt
wd0 at pciide1 channel 0 drive 0: <ST3120213AS>
wd0: 16-sector PIO, LBA48, 114473MB, 234441648 sectors
wd0(pciide1:0:0): using PIO mode 4, Ultra-DMA mode 5
pciide2 at pci0 dev 15 function 0 "NVIDIA MCP51 SATA" rev 0xa1: DMA
pciide2: using irq 10 for native-PCI interrupt
ppb2 at pci0 dev 16 function 0 "NVIDIA MCP51 PCI-PCI" rev 0xa2
pci3 at ppb2 bus 3
fxp0 at pci3 dev 8 function 0 "Intel 8255x" rev 0x08, i82559: irq 7,
address 00:02:b3:36:4d:df
inphy0 at fxp0 phy 1: i82555 10/100 PHY, rev. 4
xl0 at pci3 dev 9 function 0 "3Com 3c905B 100Base-TX" rev 0x24: irq 5,
address 00:01:02:2c:05:19
exphy0 at xl0 phy 24: 3Com internal media interface
xl1 at pci3 dev 10 function 0 "3Com 3c905C 100Base-TX" rev 0x6c: irq 11,
address 00:50:da:16:2a:89
bmtphy0 at xl1 phy 24: Broadcom 3C905C internal PHY, rev. 4
azalia0 at pci0 dev 16 function 1 "NVIDIA MCP51 HD Audio" rev 0xa2: irq 5
azalia0: codec[s]: Realtek ALC883
audio0 at azalia0
nfe0 at pci0 dev 20 function 0 "NVIDIA MCP51 LAN" rev 0xa3: irq 7,
address 00:17:31:da:9e:93
rlphy0 at nfe0 phy 13: RTL8201L 10/100 PHY, rev. 1
pchb0 at pci0 dev 24 function 0 "AMD AMD64 HyperTransport" rev 0x00
pchb1 at pci0 dev 24 function 1 "AMD AMD64 Address Map" rev 0x00
pchb2 at pci0 dev 24 function 2 "AMD AMD64 DRAM Cfg" rev 0x00
pchb3 at pci0 dev 24 function 3 "AMD AMD64 Misc Cfg" rev 0x00
isa0 at pcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
midi0 at pcppi0: <PC speaker>
spkr0 at pcppi0
softraid0 at root
root on wd0a swap on wd0b dump on wd0b
arp_rtrequest: bad gateway value
arp_rtrequest: bad gateway value
arp_rtrequest: bad gateway value
arp_rtrequest: bad gateway value
arp_rtrequest: bad gateway value
arp_rtrequest: bad gateway value
arp_rtrequest: bad gateway value
arp_rtrequest: bad gateway value
carp: pfsync0 demoted group carp to 129
carp: pfsync0 demoted group pfsync to 1
carp: pfsync0 demoted group carp to 0
carp: pfsync0 demoted group pfsync to 0
cat /etc/hostname.carp1
inet 74.8.19.30 255.255.255.240 NONE
inet alias 74.8.19.19 255.255.255.255
inet alias 74.8.19.20 255.255.255.255
#inet alias 74.8.19.21 255.255.255.255
inet alias 74.8.19.22 255.255.255.255
inet alias 74.8.19.23 255.255.255.255
inet alias 74.8.19.24 255.255.255.255
inet alias 74.8.19.25 255.255.255.255
#inet alias 74.8.19.26 255.255.255.255
inet alias 74.8.19.27 255.255.255.255
inet alias 74.8.19.28 255.255.255.255
vhid 1
advskew 1
carpdev xl0
pass 0...@ntun!
# cat /etc/hostname.carp2
inet 192.168.33.3 255.255.255.0 NONE media autoselect
! route add 192.168.34.0/24 192.168.33.7
vhid 2
advskew 1
carpdev xl1
pass d31...@nu@n!
net.inet.gre.allow=1
net.inet.carp.allow=1 # Allow the firewall to accept CARP packets
net.inet.carp.preempt=1 # Allow firewalls to failover when one
goes down
net.inet.ip.forwarding=1 # Allow packet forwarding through the
firewalls
*
Thanks,
Jon
