Thanks ... dmesg attached for both firewalls. Basic network configuration is: * 10Gb LR to external-1 ( default route ) * 1Gb to external-2 ( specific routes ) * 10Gb to dmz * 10Gb to internal network 1 * 1Gb to internal network 2 * 1Gb pfsync ( currently x-over cable ) * 10Gb trunked to 9 other internal networks
On Wed, Oct 17, 2012 at 12:23 PM, Luis Coronado <lcoron...@ticoit.com> wrote: > > Devs/gurus will need for other information as well, at least you should > include dmesg output and how your network is arranged. > > -luis > > > On Wed, Oct 17, 2012 at 9:51 AM, Bennett Samowich <bsamow...@gmail.com> > wrote: >> >> I just had an event that I'm having trouble identifying the root cause. >> I'm hoping that someone might have encountered this or might be able to >> point me toward some things to check. >> >> Yesterday we had an event where our primary firewall would stop passing >> traffic. The only thing short of a reboot that would restore service was >> to run 'sh /etc/netstart pfsync0'. Resetting pfsync's physical interface >> or pulling that cable didn't produce results. Only resetting the pfsync0 >> virtual interface would restore service. I'm not even sure what >> information would be helpful to provide or what other questions to ask. >> I >> also found it odd that the two servers did not show the same number of >> state entries by a difference of anywhere from 100 to 1000s. Is this >> typical? >> >> Thanks, >> Bennett OpenBSD 4.9 (GENERIC.MP) #819: Wed Mar 2 06:57:49 MST 2011 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 3475410944 (3314MB) avail mem = 3368873984 (3212MB) mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.6 @ 0xcf49c000 (84 entries) bios0: vendor Dell Inc. version "3.0.0" date 01/31/2011 bios0: Dell Inc. PowerEdge R710 acpi0 at bios0: rev 2 acpi0: sleep states S0 S4 S5 acpi0: tables DSDT FACP APIC SPCR HPET DM__ MCFG WD__ SLIC ERST HEST BERT EINJ SRAT TCPA SSDT acpi0: wakeup devices PCI0(S5) acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat cpu0 at mainbus0: apid 16 (boot processor) cpu0: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.39 MHz cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu0: 256KB 64b/line 8-way L2 cache cpu0: apic clock running at 133MHz cpu1 at mainbus0: apid 0 (application processor) cpu1: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu1: 256KB 64b/line 8-way L2 cache cpu2 at mainbus0: apid 18 (application processor) cpu2: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu2: 256KB 64b/line 8-way L2 cache cpu3 at mainbus0: apid 2 (application processor) cpu3: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu3: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu3: 256KB 64b/line 8-way L2 cache cpu4 at mainbus0: apid 20 (application processor) cpu4: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu4: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu4: 256KB 64b/line 8-way L2 cache cpu5 at mainbus0: apid 4 (application processor) cpu5: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu5: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu5: 256KB 64b/line 8-way L2 cache cpu6 at mainbus0: apid 22 (application processor) cpu6: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu6: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu6: 256KB 64b/line 8-way L2 cache cpu7 at mainbus0: apid 6 (application processor) cpu7: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu7: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu7: 256KB 64b/line 8-way L2 cache cpu8 at mainbus0: apid 17 (application processor) cpu8: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu8: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu8: 256KB 64b/line 8-way L2 cache cpu9 at mainbus0: apid 1 (application processor) cpu9: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu9: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu9: 256KB 64b/line 8-way L2 cache cpu10 at mainbus0: apid 19 (application processor) cpu10: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu10: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu10: 256KB 64b/line 8-way L2 cache cpu11 at mainbus0: apid 3 (application processor) cpu11: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu11: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu11: 256KB 64b/line 8-way L2 cache cpu12 at mainbus0: apid 21 (application processor) cpu12: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu12: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu12: 256KB 64b/line 8-way L2 cache cpu13 at mainbus0: apid 5 (application processor) cpu13: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu13: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu13: 256KB 64b/line 8-way L2 cache cpu14 at mainbus0: apid 23 (application processor) cpu14: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu14: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu14: 256KB 64b/line 8-way L2 cache cpu15 at mainbus0: apid 7 (application processor) cpu15: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu15: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu15: 256KB 64b/line 8-way L2 cache ioapic0 at mainbus0: apid 0 pa 0xfec00000, version 20, 24 pins ioapic1 at mainbus0: apid 1 pa 0xfec80000, version 20, 24 pins ioapic1: misconfigured as apic 0, remapped to apid 1 acpihpet0 at acpi0: 14318179 Hz acpimcfg0 at acpi0 addr 0xe0000000, bus 0-255 acpiprt0 at acpi0: bus 0 (PCI0) acpiprt1 at acpi0: bus 1 (PEX1) acpiprt2 at acpi0: bus 2 (PEX3) acpiprt3 at acpi0: bus 3 (PEX4) acpiprt4 at acpi0: bus 4 (PEX5) acpiprt5 at acpi0: bus 5 (PEX6) acpiprt6 at acpi0: bus 6 (PEX7) acpiprt7 at acpi0: bus 7 (PEX9) acpiprt8 at acpi0: bus -1 (PEXA) acpiprt9 at acpi0: bus -1 (SBEX) acpiprt10 at acpi0: bus 8 (COMP) acpicpu0 at acpi0: C3, C1 acpicpu1 at acpi0: C3, C1 acpicpu2 at acpi0: C3, C1 acpicpu3 at acpi0: C3, C1 acpicpu4 at acpi0: C3, C1 acpicpu5 at acpi0: C3, C1 acpicpu6 at acpi0: C3, C1 acpicpu7 at acpi0: C3, C1 acpicpu8 at acpi0: C3, C1 acpicpu9 at acpi0: C3, C1 acpicpu10 at acpi0: C3, C1 acpicpu11 at acpi0: C3, C1 acpicpu12 at acpi0: C3, C1 acpicpu13 at acpi0: C3, C1 acpicpu14 at acpi0: C3, C1 acpicpu15 at acpi0: C3, C1 ipmi at mainbus0 not configured pci0 at mainbus0 bus 0 pchb0 at pci0 dev 0 function 0 "Intel 5520 Host" rev 0x13 ppb0 at pci0 dev 1 function 0 "Intel X58 PCIE" rev 0x13 pci1 at ppb0 bus 1 bnx0 at pci1 dev 0 function 0 "Broadcom BCM5709" rev 0x20: apic 1 int 4 (irq 15) bnx1 at pci1 dev 0 function 1 "Broadcom BCM5709" rev 0x20: apic 1 int 16 (irq 14) ppb1 at pci0 dev 3 function 0 "Intel X58 PCIE" rev 0x13 pci2 at ppb1 bus 2 bnx2 at pci2 dev 0 function 0 "Broadcom BCM5709" rev 0x20: apic 1 int 0 (irq 15) bnx3 at pci2 dev 0 function 1 "Broadcom BCM5709" rev 0x20: apic 1 int 10 (irq 14) ppb2 at pci0 dev 4 function 0 "Intel X58 PCIE" rev 0x13 pci3 at ppb2 bus 3 mpi0 at pci3 dev 0 function 0 "Symbios Logic SAS1068E" rev 0x08: apic 1 int 1 (irq 15) scsibus0 at mpi0: 112 targets sd0 at scsibus0 targ 0 lun 0: <Dell, VIRTUAL DISK, 1028> SCSI3 0/direct fixed sd0: 237952MB, 512 bytes/sec, 487325696 sec total sd1 at scsibus0 targ 2 lun 0: <Dell, VIRTUAL DISK, 1028> SCSI3 0/direct fixed sd1: 237952MB, 512 bytes/sec, 487325696 sec total ses0 at scsibus0 targ 8 lun 0: <DP, BACKPLANE, 1.07> SCSI3 13/enclosure services fixed ppb3 at pci0 dev 5 function 0 "Intel X58 PCIE" rev 0x13: apic 1 int 21 (irq 0) pci4 at ppb3 bus 4 ix0 at pci4 dev 0 function 0 "Intel 10GbE SR Dual (82598AF)" rev 0x01: apic 1 int 2 (irq 15), address $REDACTED ix1 at pci4 dev 0 function 1 "Intel 10GbE SR Dual (82598AF)" rev 0x01: apic 1 int 12 (irq 14), address $REDACTED ppb4 at pci0 dev 6 function 0 "Intel X58 PCIE" rev 0x13: apic 1 int 21 (irq 0) pci5 at ppb4 bus 5 ppb5 at pci0 dev 7 function 0 "Intel X58 PCIE" rev 0x13: apic 1 int 21 (irq 0) pci6 at ppb5 bus 6 ix2 at pci6 dev 0 function 0 "Intel 10GbE SR Dual (82598AF)" rev 0x01: apic 1 int 6 (irq 15), address $REDACTED ix3 at pci6 dev 0 function 1 "Intel 10GbE SR Dual (82598AF)" rev 0x01: apic 1 int 13 (irq 14), address $REDACTED ppb6 at pci0 dev 9 function 0 "Intel X58 PCIE" rev 0x13: apic 1 int 21 (irq 0) pci7 at ppb6 bus 7 ix4 at pci7 dev 0 function 0 "Intel 10GbE LR (82598EB)" rev 0x01: apic 1 int 8 (irq 15), address $REDACTED "Intel X58 Misc" rev 0x13 at pci0 dev 20 function 0 not configured "Intel X58 GPIO" rev 0x13 at pci0 dev 20 function 1 not configured "Intel X58 RAS" rev 0x13 at pci0 dev 20 function 2 not configured uhci0 at pci0 dev 26 function 0 "Intel 82801I USB" rev 0x02: apic 0 int 17 (irq 14) uhci1 at pci0 dev 26 function 1 "Intel 82801I USB" rev 0x02: apic 0 int 18 (irq 11) ehci0 at pci0 dev 26 function 7 "Intel 82801I USB" rev 0x02: apic 0 int 19 (irq 10) usb0 at ehci0: USB revision 2.0 uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1 uhci2 at pci0 dev 29 function 0 "Intel 82801I USB" rev 0x02: apic 0 int 21 (irq 6) uhci3 at pci0 dev 29 function 1 "Intel 82801I USB" rev 0x02: apic 0 int 20 (irq 5) ehci1 at pci0 dev 29 function 7 "Intel 82801I USB" rev 0x02: apic 0 int 21 (irq 6) usb1 at ehci1: USB revision 2.0 uhub1 at usb1 "Intel EHCI root hub" rev 2.00/1.00 addr 1 ppb7 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0x92 pci8 at ppb7 bus 8 vga1 at pci8 dev 3 function 0 "Matrox MGA G200eW" rev 0x0a wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) wsdisplay0: screen 1-5 added (80x25, vt100 emulation) pcib0 at pci0 dev 31 function 0 "Intel 82801IB LPC" rev 0x02 pciide0 at pci0 dev 31 function 2 "Intel 82801I SATA" rev 0x02: DMA, channel 0 configured to native-PCI, channel 1 configured to native-PCI pciide0: using apic 0 int 23 (irq 14) for native-PCI interrupt atapiscsi0 at pciide0 channel 0 drive 0 scsibus1 at atapiscsi0: 2 targets cd0 at scsibus1 targ 0 lun 0: <TEAC, DVD-ROM DV28SV, D.0K> ATAPI 5/cdrom removable cd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 5 usb2 at uhci0: USB revision 1.0 uhub2 at usb2 "Intel UHCI root hub" rev 1.00/1.00 addr 1 usb3 at uhci1: USB revision 1.0 uhub3 at usb3 "Intel UHCI root hub" rev 1.00/1.00 addr 1 usb4 at uhci2: USB revision 1.0 uhub4 at usb4 "Intel UHCI root hub" rev 1.00/1.00 addr 1 usb5 at uhci3: USB revision 1.0 uhub5 at usb5 "Intel UHCI root hub" rev 1.00/1.00 addr 1 isa0 at pcib0 isadma0 at isa0 com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo pckbc0 at isa0 port 0x60/5 pckbd0 at pckbc0 (kbd slot) pckbc0: using irq 1 for kbd slot wskbd0 at pckbd0: console keyboard, using wsdisplay0 pcppi0 at isa0 port 0x61 spkr0 at pcppi0 mtrr: Pentium Pro MTRR support uhub6 at uhub0 port 3 "Standard Microsystems product 0x2514" rev 2.00/0.00 addr 2 uhidev0 at uhub2 port 1 configuration 1 interface 0 "Avocent Dell 03R874" rev 1.10/1.00 addr 2 uhidev0: iclass 3/1 ukbd0 at uhidev0: 8 modifier keys, 6 key codes, country code 33 wskbd1 at ukbd0 mux 1 wskbd1: connecting to wsdisplay0 uhidev1 at uhub2 port 1 configuration 1 interface 1 "Avocent Dell 03R874" rev 1.10/1.00 addr 2 uhidev1: iclass 3/1, 3 report ids ums0 at uhidev1 reportid 1: 5 buttons, Z dir wsmouse0 at ums0 mux 0 uhid0 at uhidev1 reportid 2: input=2, output=0, feature=0 uhid1 at uhidev1 reportid 3: input=1, output=0, feature=0 vscsi0 at root scsibus2 at vscsi0: 256 targets softraid0 at root root on sd0a swap on sd0b dump on sd0b bnx0: address $REDACTED brgphy0 at bnx0 phy 1: BCM5709 10/100/1000baseT PHY, rev. 8 bnx1: address $REDACTED brgphy1 at bnx1 phy 1: BCM5709 10/100/1000baseT PHY, rev. 8 bnx2: address $REDACTED brgphy2 at bnx2 phy 1: BCM5709 10/100/1000baseT PHY, rev. 8 bnx3: address $REDACTED brgphy3 at bnx3 phy 1: BCM5709 10/100/1000baseT PHY, rev. 8 carp2: state transition: BACKUP -> MASTER carp3: state transition: BACKUP -> MASTER carp4: state transition: BACKUP -> MASTER carp500: state transition: BACKUP -> MASTER carp51: state transition: BACKUP -> MASTER carp510: state transition: BACKUP -> MASTER carp520: state transition: BACKUP -> MASTER carp530: state transition: BACKUP -> MASTER carp560: state transition: BACKUP -> MASTER carp61: state transition: BACKUP -> MASTER carp810: state transition: BACKUP -> MASTER carp820: state transition: BACKUP -> MASTER carp540: state transition: BACKUP -> MASTER carp2: state transition: MASTER -> BACKUP carp3: state transition: MASTER -> BACKUP carp4: state transition: MASTER -> BACKUP carp51: state transition: MASTER -> BACKUP carp500: state transition: MASTER -> BACKUP carp510: state transition: MASTER -> BACKUP carp520: state transition: MASTER -> BACKUP carp540: state transition: MASTER -> BACKUP carp560: state transition: MASTER -> BACKUP carp61: state transition: MASTER -> BACKUP carp530: state transition: MASTER -> BACKUP carp810: state transition: MASTER -> BACKUP carp820: state transition: MASTER -> BACKUP carp0: state transition: BACKUP -> MASTER carp0: state transition: MASTER -> BACKUP pfsync: failed to receive bulk update carp0: state transition: BACKUP -> MASTER carp2: state transition: BACKUP -> MASTER carp3: state transition: BACKUP -> MASTER carp51: state transition: BACKUP -> MASTER carp4: state transition: BACKUP -> MASTER carp500: state transition: BACKUP -> MASTER carp510: state transition: BACKUP -> MASTER carp520: state transition: BACKUP -> MASTER carp540: state transition: BACKUP -> MASTER carp560: state transition: BACKUP -> MASTER carp61: state transition: BACKUP -> MASTER carp530: state transition: BACKUP -> MASTER carp810: state transition: BACKUP -> MASTER carp820: state transition: BACKUP -> MASTER carp500: state transition: MASTER -> BACKUP carp810: state transition: MASTER -> BACKUP carp51: state transition: MASTER -> BACKUP carp560: state transition: MASTER -> BACKUP carp61: state transition: MASTER -> BACKUP carp520: state transition: MASTER -> BACKUP carp820: state transition: MASTER -> BACKUP carp530: state transition: MASTER -> BACKUP carp510: state transition: MASTER -> BACKUP carp0: state transition: MASTER -> BACKUP carp540: state transition: MASTER -> BACKUP carp3: state transition: MASTER -> BACKUP carp4: state transition: MASTER -> BACKUP carp2: state transition: MASTER -> BACKUP carp0: state transition: BACKUP -> MASTER carp500: state transition: BACKUP -> MASTER carp540: state transition: BACKUP -> MASTER carp2: state transition: BACKUP -> MASTER carp560: state transition: BACKUP -> MASTER carp61: state transition: BACKUP -> MASTER carp3: state transition: BACKUP -> MASTER carp810: state transition: BACKUP -> MASTER carp510: state transition: BACKUP -> MASTER carp530: state transition: BACKUP -> MASTER carp51: state transition: BACKUP -> MASTER carp520: state transition: BACKUP -> MASTER carp820: state transition: BACKUP -> MASTER carp4: state transition: BACKUP -> MASTER OpenBSD 4.9 (GENERIC.MP) #819: Wed Mar 2 06:57:49 MST 2011 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 3478753280 (3317MB) avail mem = 3372130304 (3215MB) mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.6 @ 0xcf79c000 (83 entries) bios0: vendor Dell Inc. version "1.2.6" date 07/17/2009 bios0: Dell Inc. PowerEdge R710 acpi0 at bios0: rev 2 acpi0: sleep states S0 S4 S5 acpi0: tables DSDT FACP APIC SPCR HPET DM__ MCFG WDAT SLIC ERST HEST BERT EINJ SRAT TCPA SSDT acpi0: wakeup devices PCI0(S5) acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat cpu0 at mainbus0: apid 16 (boot processor) cpu0: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.39 MHz cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu0: 256KB 64b/line 8-way L2 cache cpu0: apic clock running at 133MHz cpu1 at mainbus0: apid 0 (application processor) cpu1: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu1: 256KB 64b/line 8-way L2 cache cpu2 at mainbus0: apid 18 (application processor) cpu2: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu2: 256KB 64b/line 8-way L2 cache cpu3 at mainbus0: apid 2 (application processor) cpu3: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu3: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu3: 256KB 64b/line 8-way L2 cache cpu4 at mainbus0: apid 20 (application processor) cpu4: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu4: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu4: 256KB 64b/line 8-way L2 cache cpu5 at mainbus0: apid 4 (application processor) cpu5: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu5: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu5: 256KB 64b/line 8-way L2 cache cpu6 at mainbus0: apid 22 (application processor) cpu6: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu6: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu6: 256KB 64b/line 8-way L2 cache cpu7 at mainbus0: apid 6 (application processor) cpu7: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu7: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu7: 256KB 64b/line 8-way L2 cache cpu8 at mainbus0: apid 17 (application processor) cpu8: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu8: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu8: 256KB 64b/line 8-way L2 cache cpu9 at mainbus0: apid 1 (application processor) cpu9: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu9: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu9: 256KB 64b/line 8-way L2 cache cpu10 at mainbus0: apid 19 (application processor) cpu10: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu10: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu10: 256KB 64b/line 8-way L2 cache cpu11 at mainbus0: apid 3 (application processor) cpu11: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu11: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu11: 256KB 64b/line 8-way L2 cache cpu12 at mainbus0: apid 21 (application processor) cpu12: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu12: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu12: 256KB 64b/line 8-way L2 cache cpu13 at mainbus0: apid 5 (application processor) cpu13: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu13: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu13: 256KB 64b/line 8-way L2 cache cpu14 at mainbus0: apid 23 (application processor) cpu14: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu14: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu14: 256KB 64b/line 8-way L2 cache cpu15 at mainbus0: apid 7 (application processor) cpu15: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz cpu15: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG cpu15: 256KB 64b/line 8-way L2 cache ioapic0 at mainbus0: apid 0 pa 0xfec00000, version 20, 24 pins ioapic1 at mainbus0: apid 1 pa 0xfec80000, version 20, 24 pins ioapic1: misconfigured as apic 0, remapped to apid 1 acpihpet0 at acpi0: 14318179 Hz acpimcfg0 at acpi0 addr 0xe0000000, bus 0-255 acpiprt0 at acpi0: bus 0 (PCI0) acpiprt1 at acpi0: bus 1 (PEX1) acpiprt2 at acpi0: bus 2 (PEX3) acpiprt3 at acpi0: bus 3 (PEX4) acpiprt4 at acpi0: bus 4 (PEX5) acpiprt5 at acpi0: bus 5 (PEX6) acpiprt6 at acpi0: bus 6 (PEX7) acpiprt7 at acpi0: bus 7 (PEX9) acpiprt8 at acpi0: bus -1 (PEXA) acpiprt9 at acpi0: bus -1 (SBEX) acpiprt10 at acpi0: bus 8 (COMP) acpicpu0 at acpi0: C3, C1 acpicpu1 at acpi0: C3, C1 acpicpu2 at acpi0: C3, C1 acpicpu3 at acpi0: C3, C1 acpicpu4 at acpi0: C3, C1 acpicpu5 at acpi0: C3, C1 acpicpu6 at acpi0: C3, C1 acpicpu7 at acpi0: C3, C1 acpicpu8 at acpi0: C3, C1 acpicpu9 at acpi0: C3, C1 acpicpu10 at acpi0: C3, C1 acpicpu11 at acpi0: C3, C1 acpicpu12 at acpi0: C3, C1 acpicpu13 at acpi0: C3, C1 acpicpu14 at acpi0: C3, C1 acpicpu15 at acpi0: C3, C1 ipmi at mainbus0 not configured pci0 at mainbus0 bus 0 pchb0 at pci0 dev 0 function 0 "Intel 5520 Host" rev 0x13 ppb0 at pci0 dev 1 function 0 "Intel X58 PCIE" rev 0x13 pci1 at ppb0 bus 1 bnx0 at pci1 dev 0 function 0 "Broadcom BCM5709" rev 0x20: apic 1 int 4 (irq 15) bnx1 at pci1 dev 0 function 1 "Broadcom BCM5709" rev 0x20: apic 1 int 16 (irq 14) ppb1 at pci0 dev 3 function 0 "Intel X58 PCIE" rev 0x13 pci2 at ppb1 bus 2 bnx2 at pci2 dev 0 function 0 "Broadcom BCM5709" rev 0x20: apic 1 int 0 (irq 15) bnx3 at pci2 dev 0 function 1 "Broadcom BCM5709" rev 0x20: apic 1 int 10 (irq 14) ppb2 at pci0 dev 4 function 0 "Intel X58 PCIE" rev 0x13 pci3 at ppb2 bus 3 mpi0 at pci3 dev 0 function 0 "Symbios Logic SAS1068E" rev 0x08: apic 1 int 1 (irq 15) scsibus0 at mpi0: 112 targets sd0 at scsibus0 targ 0 lun 0: <Dell, VIRTUAL DISK, 1028> SCSI3 0/direct fixed sd0: 237952MB, 512 bytes/sec, 487325696 sec total ses0 at scsibus0 targ 8 lun 0: <DP, BACKPLANE, 1.07> SCSI3 13/enclosure services fixed ppb3 at pci0 dev 5 function 0 "Intel X58 PCIE" rev 0x13: apic 1 int 21 (irq 0) pci4 at ppb3 bus 4 ppb4 at pci0 dev 6 function 0 "Intel X58 PCIE" rev 0x13: apic 1 int 21 (irq 0) pci5 at ppb4 bus 5 ix0 at pci5 dev 0 function 0 "Intel 10GbE SR Dual (82598AF)" rev 0x01: apic 1 int 3 (irq 15), address $REDACTED ix1 at pci5 dev 0 function 1 "Intel 10GbE SR Dual (82598AF)" rev 0x01: apic 1 int 14 (irq 14), address $REDACTED ppb5 at pci0 dev 7 function 0 "Intel X58 PCIE" rev 0x13: apic 1 int 21 (irq 0) pci6 at ppb5 bus 6 ix2 at pci6 dev 0 function 0 "Intel 10GbE SR Dual (82598AF)" rev 0x01: apic 1 int 6 (irq 15), address $REDACTED ix3 at pci6 dev 0 function 1 "Intel 10GbE SR Dual (82598AF)" rev 0x01: apic 1 int 13 (irq 14), address $REDACTED ppb6 at pci0 dev 9 function 0 "Intel X58 PCIE" rev 0x13: apic 1 int 21 (irq 0) pci7 at ppb6 bus 7 ix4 at pci7 dev 0 function 0 "Intel 10GbE LR (82598EB)" rev 0x01: apic 1 int 8 (irq 15), address $REDACTED "Intel X58 Misc" rev 0x13 at pci0 dev 20 function 0 not configured "Intel X58 GPIO" rev 0x13 at pci0 dev 20 function 1 not configured "Intel X58 RAS" rev 0x13 at pci0 dev 20 function 2 not configured uhci0 at pci0 dev 26 function 0 "Intel 82801I USB" rev 0x02: apic 0 int 17 (irq 14) uhci1 at pci0 dev 26 function 1 "Intel 82801I USB" rev 0x02: apic 0 int 18 (irq 11) ehci0 at pci0 dev 26 function 7 "Intel 82801I USB" rev 0x02: apic 0 int 19 (irq 10) usb0 at ehci0: USB revision 2.0 uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1 uhci2 at pci0 dev 29 function 0 "Intel 82801I USB" rev 0x02: apic 0 int 21 (irq 6) uhci3 at pci0 dev 29 function 1 "Intel 82801I USB" rev 0x02: apic 0 int 20 (irq 5) ehci1 at pci0 dev 29 function 7 "Intel 82801I USB" rev 0x02: apic 0 int 21 (irq 6) usb1 at ehci1: USB revision 2.0 uhub1 at usb1 "Intel EHCI root hub" rev 2.00/1.00 addr 1 ppb7 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0x92 pci8 at ppb7 bus 8 vga1 at pci8 dev 3 function 0 "Matrox MGA G200eW" rev 0x0a wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) wsdisplay0: screen 1-5 added (80x25, vt100 emulation) pcib0 at pci0 dev 31 function 0 "Intel 82801IB LPC" rev 0x02 pciide0 at pci0 dev 31 function 2 "Intel 82801I SATA" rev 0x02: DMA, channel 0 configured to native-PCI, channel 1 configured to native-PCI pciide0: using apic 0 int 23 (irq 14) for native-PCI interrupt atapiscsi0 at pciide0 channel 0 drive 0 scsibus1 at atapiscsi0: 2 targets cd0 at scsibus1 targ 0 lun 0: <TEAC, DVD-ROM DV28SV, D.0K> ATAPI 5/cdrom removable cd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 5 usb2 at uhci0: USB revision 1.0 uhub2 at usb2 "Intel UHCI root hub" rev 1.00/1.00 addr 1 usb3 at uhci1: USB revision 1.0 uhub3 at usb3 "Intel UHCI root hub" rev 1.00/1.00 addr 1 usb4 at uhci2: USB revision 1.0 uhub4 at usb4 "Intel UHCI root hub" rev 1.00/1.00 addr 1 usb5 at uhci3: USB revision 1.0 uhub5 at usb5 "Intel UHCI root hub" rev 1.00/1.00 addr 1 isa0 at pcib0 isadma0 at isa0 com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo pckbc0 at isa0 port 0x60/5 pckbd0 at pckbc0 (kbd slot) pckbc0: using irq 1 for kbd slot wskbd0 at pckbd0: console keyboard, using wsdisplay0 pcppi0 at isa0 port 0x61 spkr0 at pcppi0 mtrr: Pentium Pro MTRR support uhub6 at uhub0 port 3 "Standard Microsystems product 0x2514" rev 2.00/0.00 addr 2 uhidev0 at uhub4 port 1 configuration 1 interface 0 "Avocent Dell 03R874" rev 1.10/1.00 addr 2 uhidev0: iclass 3/1 ukbd0 at uhidev0: 8 modifier keys, 6 key codes, country code 33 wskbd1 at ukbd0 mux 1 wskbd1: connecting to wsdisplay0 uhidev1 at uhub4 port 1 configuration 1 interface 1 "Avocent Dell 03R874" rev 1.10/1.00 addr 2 uhidev1: iclass 3/1, 3 report ids ums0 at uhidev1 reportid 1: 5 buttons, Z dir wsmouse0 at ums0 mux 0 uhid0 at uhidev1 reportid 2: input=2, output=0, feature=0 uhid1 at uhidev1 reportid 3: input=1, output=0, feature=0 uhidev2 at uhub4 port 2 configuration 1 interface 0 "Avocent USB Composite Device-0" rev 1.10/0.00 addr 3 uhidev2: iclass 3/1 ukbd1 at uhidev2: 8 modifier keys, 6 key codes wskbd2 at ukbd1 mux 1 wskbd2: connecting to wsdisplay0 uhidev3 at uhub4 port 2 configuration 1 interface 1 "Avocent USB Composite Device-0" rev 1.10/0.00 addr 3 uhidev3: iclass 3/1 ums1 at uhidev3 ums1: X report 0x0002 not supported vscsi0 at root scsibus2 at vscsi0: 256 targets softraid0 at root root on sd0a swap on sd0b dump on sd0b bnx0: address $REDACTED brgphy0 at bnx0 phy 1: BCM5709 10/100/1000baseT PHY, rev. 8 bnx1: address $REDACTED brgphy1 at bnx1 phy 1: BCM5709 10/100/1000baseT PHY, rev. 8 bnx2: address $REDACTED brgphy2 at bnx2 phy 1: BCM5709 10/100/1000baseT PHY, rev. 8 bnx3: address $REDACTED brgphy3 at bnx3 phy 1: BCM5709 10/100/1000baseT PHY, rev. 8 carp2: state transition: BACKUP -> MASTER carp3: state transition: BACKUP -> MASTER carp4: state transition: BACKUP -> MASTER carp500: state transition: BACKUP -> MASTER carp51: state transition: BACKUP -> MASTER carp510: state transition: BACKUP -> MASTER carp520: state transition: BACKUP -> MASTER carp530: state transition: BACKUP -> MASTER carp560: state transition: BACKUP -> MASTER carp61: state transition: BACKUP -> MASTER carp810: state transition: BACKUP -> MASTER carp820: state transition: BACKUP -> MASTER carp540: state transition: BACKUP -> MASTER carp2: state transition: MASTER -> BACKUP carp3: state transition: MASTER -> BACKUP carp51: state transition: MASTER -> BACKUP carp4: state transition: MASTER -> BACKUP carp500: state transition: MASTER -> BACKUP carp510: state transition: MASTER -> BACKUP carp520: state transition: MASTER -> BACKUP carp540: state transition: MASTER -> BACKUP carp530: state transition: MASTER -> BACKUP carp560: state transition: MASTER -> BACKUP carp61: state transition: MASTER -> BACKUP carp810: state transition: MASTER -> BACKUP carp820: state transition: MASTER -> BACKUP carp0: state transition: BACKUP -> MASTER carp0: state transition: MASTER -> BACKUP carp500: state transition: BACKUP -> MASTER carp0: state transition: BACKUP -> MASTER carp540: state transition: BACKUP -> MASTER carp2: state transition: BACKUP -> MASTER carp3: state transition: BACKUP -> MASTER carp560: state transition: BACKUP -> MASTER carp61: state transition: BACKUP -> MASTER carp51: state transition: BACKUP -> MASTER carp810: state transition: BACKUP -> MASTER carp510: state transition: BACKUP -> MASTER carp520: state transition: BACKUP -> MASTER carp530: state transition: BACKUP -> MASTER carp4: state transition: BACKUP -> MASTER carp820: state transition: BACKUP -> MASTER carp0: state transition: MASTER -> BACKUP carp500: state transition: MASTER -> BACKUP carp2: state transition: MASTER -> BACKUP carp560: state transition: MASTER -> BACKUP carp61: state transition: MASTER -> BACKUP carp810: state transition: MASTER -> BACKUP carp510: state transition: MASTER -> BACKUP carp530: state transition: MASTER -> BACKUP carp51: state transition: MASTER -> BACKUP carp520: state transition: MASTER -> BACKUP carp540: state transition: MASTER -> BACKUP carp4: state transition: MASTER -> BACKUP carp3: state transition: MASTER -> BACKUP carp820: state transition: MASTER -> BACKUP