Hi,
I'm trying to upgrade a HA carped firewall cluster to 5.9 but run into
issues.
The current cluster runs old 5.2, so it's more than time to do so.
I've done a couple of carp setups so far, but don't seem to get this one
right.
I reinstalled the backup node of the cluster with OpenBSD 5.9.
I have two em interfaces, that have a few VLANs on them, with
carp on top, and two ix interfaces, that are bundled in a trunk with
vlans and carp on top of the trunk. In total I have 19 carp interfaces.
All carp interfaces have the same vhid.
All the carp interfaces see the traffic from the MASTER node, and
when started up, they stay in BACKUP MODE.
A few of the VLAN interfaces have IP addresses, most of them don't.
This is /etc/sysctl.conf:
kern.bufcachepercent=80
net.inet6.ip6.forwarding=0
net.inet.ip.forwarding=1
net.inet.carp.preempt=1
net.inet.carp.allow=1
below I just show hostname.if of the stack of the interface that
has the default route. This sits on the ix -> trunk -> vlan -> carp:
cat /etc/hostname.ix0
description "PHYS_INT_TRK"
up
cat /etc/hostname.ix1
description "PHYS_INT_TRK"
up
cat /etc/hostname.trunk0
description "INTERNAL_TRUNK"
trunkproto lacp trunkport ix0 trunkport ix1
inet 192.168.9.252 255.255.255.0 NONE
up
cat /etc/hostname.vlan90
description "NKE_IPB_VLAN"
vlan 90 vlandev trunk0
up
cat /etc/hostname.carp90
description "NKE_IPB"
vhid 1 pass mypassword carpdev vlan90 advskew 100
inet 172.16.99.1 255.255.255.0 NONE
up
ifconfig ix0
ix0:
flags=18b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST,MPSAFE>
mtu 1500
lladdr 90:e2:ba:2c:b5:08
description: PHYS_INT_TRK
priority: 0
trunk: trunkdev trunk0
media: Ethernet autoselect
status: no carrier
ifconfig ix1
ix1:
flags=18b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST,MPSAFE>
mtu 1500
lladdr 90:e2:ba:2c:b5:08
description: PHYS_INT_TRK
priority: 0
trunk: trunkdev trunk0
media: Ethernet autoselect (10GSFP+Cu full-duplex)
status: active
ifconfig trunk0
trunk0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
lladdr 90:e2:ba:2c:b5:08
description: INTERNAL_TRUNK
priority: 0
trunk: trunkproto lacp
trunk id: [(8000,90:e2:ba:2c:b5:08,404D,0000,0000),
(FFFF,00:00:00:00:00:00,0000,0000,0000)]
trunkport ix1 active,collecting,distributing
trunkport ix0
groups: trunk
media: Ethernet autoselect
status: active
inet 192.168.9.252 netmask 0xffffff00 broadcast 192.168.9.255
ifconfig vlan90
vlan90: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
lladdr 90:e2:ba:2c:b5:08
description: NKE_IPB_VLAN
priority: 0
vlan: 90 parent interface: trunk0
groups: vlan egress
ifconfig carp90
carp90: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
lladdr 00:00:5e:00:01:01
description: NKE_IPB
priority: 15
carp: BACKUP carpdev vlan90 vhid 1 advbase 1 advskew 100
groups: carp
status: backup
inet 172.16.99.1 netmask 0xffffff00 broadcast 172.16.99.255
I first tried to run on the master:
ifconfig -g carp carpdemote XX
to demote the carp on the master, but that then kicked me off from the
network. With keyboard/monitor I saw, that only some of the interfaces
changes state on the MASTER/BACKUP server. Both were in kind of mixed
mode. That explained me, why I got kicked out.
Then I just switched off the old master.
The new backup node switched all interfaces into MASTER mode.
however, after the switch, I was not able to ping my default
gateway, the 172.16.99.2. On the default gateway itself, an OpenBSD
HA carp firewall cluster running 5.8, I saw in dmesg at the time I was
trying to switch the cluster:
Apr 25 05:05:19 srv93 /bsd: arp info overwritten for 172.16.99.1 by
90:e2:ba:2c:b5:08 on trunk0
Apr 25 05:05:26 srv93 /bsd: arp info overwritten for 172.16.99.1 by
00:00:5e:00:01:01 on trunk0
Apr 25 05:19:20 srv93 /bsd: arp info overwritten for 172.16.99.1 by
90:e2:ba:2c:b5:08 on trunk0
Apr 25 05:19:21 srv93 /bsd: arp info overwritten for 172.16.99.1 by
00:00:5e:00:01:01 on trunk0
Apr 25 05:27:40 srv93 /bsd: arp info overwritten for 172.16.99.1 by
90:e2:ba:2c:b5:08 on trunk0
Apr 25 05:27:40 srv93 /bsd: arp info overwritten for 172.16.99.1 by
00:00:5e:00:01:01 on trunk0
The MAC 90:e2:ba:2c:b5:08 is from the BACKUP node, that is apparently the MAC
of the vlan90
interface underlying the carp90 interface. The 00:00:5e:00:01:01 is coming
from the old 5.2
cluster node that is the master. Actually I did not expected to see these
switched, shouldn't
the BACKUP use also the carp MAC address?
On networks, where the carp parent interfaces have IPs assigned in the given
networks, I
was able to ping other nodes in the network, on carp interfaces where the
parent doesn't have
IPs assigned, trying to ping in these networks failed.
As shown with hostname.if and ifconfig output, the parent of the carp
interface with the default route attached, doesn't have an IP address
assigned.
There I was not able to ping the default gateway. I made a test and assigning
a free IP in that 172.16.99/24 network, and was able to ping the default
gateway.
The pf rules are the same that I use on the old master as well as on the
new backup node.
Seems I need a hit with a cluestick.
What I most wonder is if the MAC address of the carp interface really
is supposed to use the MAC of the trunk?
Currently, the old 5.8 master is up and running again as MASTER, and
the 5.9 is up and running again as BACKUP.
If more information about the setup is needed, please let me know,
from the current state in BACKUP is easy, for the other, I have to find
another maintenance window.
thanks,
Sebastian
The new BACKUP node:
OpenBSD 5.9 (GENERIC.MP) #1888: Fri Feb 26 01:20:19 MST 2016
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 34172940288 (32589MB)
avail mem = 33133068288 (31598MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0x7ffeb000 (181 entries)
bios0: vendor HP version "J03" date 07/02/2012
bios0: HP ProLiant DL160 Gen8
acpi0 at bios0: rev 2
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP SPCR MCFG HPET FFFF SPMI ERST APIC SRAT FFFF BERT HEST
DMAR FFFF SSDT SSDT SSDT SSDT SSDT SSDT SSDT SSDT
acpi0: wakeup devices PCI0(S5) IPT1(S5) IPT2(S5) IPT3(S5) IPT4(S5) IPT5(S5)
IPT6(S5) IPT7(S5) IPT8(S5) PT1A(S4) PT2A(S4) PT2C(S4) PT3A(S4)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimcfg0 at acpi0 addr 0x80000000, bus 0-255
acpihpet0 at acpi0: 14318179 Hz
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz, 2394.53 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLI
NE,AES,XSAVE,AVX,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1.2, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz, 2394.24 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLI
NE,AES,XSAVE,AVX,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz, 2394.24 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLI
NE,AES,XSAVE,AVX,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz, 2394.24 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLI
NE,AES,XSAVE,AVX,NXE,PAGE1GB,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 3, package 0
ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins
ioapic1 at mainbus0: apid 0 pa 0xfec10000, version 20, 24 pins
acpiprt0 at acpi0: bus 11 (IPT1)
acpiprt1 at acpi0: bus -1 (IPT2)
acpiprt2 at acpi0: bus -1 (IPT3)
acpiprt3 at acpi0: bus -1 (IPT4)
acpiprt4 at acpi0: bus -1 (IPT5)
acpiprt5 at acpi0: bus -1 (IPT6)
acpiprt6 at acpi0: bus -1 (IPT7)
acpiprt7 at acpi0: bus 1 (IPT8)
acpiprt8 at acpi0: bus 24 (VRP1)
acpiprt9 at acpi0: bus 5 (PT1A)
acpiprt10 at acpi0: bus 18 (PT1B)
acpiprt11 at acpi0: bus 4 (PT2A)
acpiprt12 at acpi0: bus 19 (PT2B)
acpiprt13 at acpi0: bus 2 (PT2C)
acpiprt14 at acpi0: bus 3 (PT2D)
acpiprt15 at acpi0: bus 8 (PT3A)
acpiprt16 at acpi0: bus 20 (PT3B)
acpiprt17 at acpi0: bus 21 (PT3C)
acpiprt18 at acpi0: bus 22 (PT3D)
acpiprt19 at acpi0: bus 0 (PCI0)
acpicpu0 at acpi0: C2(350@96 mwait.1@0x20), C1(1000@1 mwait.1)
acpicpu1 at acpi0: C2(350@96 mwait.1@0x20), C1(1000@1 mwait.1)
acpicpu2 at acpi0: C2(350@96 mwait.1@0x20), C1(1000@1 mwait.1)
acpicpu3 at acpi0: C2(350@96 mwait.1@0x20), C1(1000@1 mwait.1)
acpitz0 at acpi0: critical temperature is 31 degC
ipmi at mainbus0 not configured
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel E5 Host" rev 0x07
ppb0 at pci0 dev 1 function 0 "Intel E5 PCIE" rev 0x07
pci1 at ppb0 bus 5
"Broadcom BCM57810" rev 0x10 at pci1 dev 0 function 0 not configured
"Broadcom BCM57810" rev 0x10 at pci1 dev 0 function 1 not configured
ppb1 at pci0 dev 1 function 1 "Intel E5 PCIE" rev 0x07
pci2 at ppb1 bus 18
ppb2 at pci0 dev 2 function 0 "Intel E5 PCIE" rev 0x07
pci3 at ppb2 bus 4
ciss0 at pci3 dev 0 function 0 "Hewlett-Packard Smart Array" rev 0x01: apic 0
int 8
ciss0: 1 LD, HW rev 1, FW 3.04/3.04, 64bit fifo rro
scsibus1 at ciss0: 1 targets
sd0 at scsibus1 targ 0 lun 0: <HP, LOGICAL VOLUME, 3.04> SCSI3 0/direct fixed
sd0: 476908MB, 512 bytes/sector, 976707632 sectors
ppb3 at pci0 dev 2 function 1 "Intel E5 PCIE" rev 0x07
pci4 at ppb3 bus 19
ppb4 at pci0 dev 2 function 2 "Intel E5 PCIE" rev 0x07
pci5 at ppb4 bus 2
em0 at pci5 dev 0 function 0 "Intel I350" rev 0x01: msi, address
b4:b5:2f:5f:13:14
em1 at pci5 dev 0 function 1 "Intel I350" rev 0x01: msi, address
b4:b5:2f:5f:13:15
ppb5 at pci0 dev 2 function 3 "Intel E5 PCIE" rev 0x07
pci6 at ppb5 bus 3
unknown vendor 0x1590 product 0x005f (class memory subclass miscellaneous, rev
0x00) at pci6 dev 0 function 0 not configured
ppb6 at pci0 dev 3 function 0 "Intel E5 PCIE" rev 0x07
pci7 at ppb6 bus 8
ix0 at pci7 dev 0 function 0 "Intel 82599" rev 0x01: msi, address
90:e2:ba:2c:b5:08
ix1 at pci7 dev 0 function 1 "Intel 82599" rev 0x01: msi, address
90:e2:ba:2c:b5:09
ppb7 at pci0 dev 3 function 1 "Intel E5 PCIE" rev 0x07
pci8 at ppb7 bus 20
ppb8 at pci0 dev 3 function 2 "Intel E5 PCIE" rev 0x07
pci9 at ppb8 bus 21
ppb9 at pci0 dev 3 function 3 "Intel E5 PCIE" rev 0x07
pci10 at ppb9 bus 22
"Intel E5 DMA" rev 0x07 at pci0 dev 4 function 0 not configured
"Intel E5 DMA" rev 0x07 at pci0 dev 4 function 1 not configured
"Intel E5 DMA" rev 0x07 at pci0 dev 4 function 2 not configured
"Intel E5 DMA" rev 0x07 at pci0 dev 4 function 3 not configured
"Intel E5 DMA" rev 0x07 at pci0 dev 4 function 4 not configured
"Intel E5 DMA" rev 0x07 at pci0 dev 4 function 5 not configured
"Intel E5 DMA" rev 0x07 at pci0 dev 4 function 6 not configured
"Intel E5 DMA" rev 0x07 at pci0 dev 4 function 7 not configured
"Intel E5 Address Map" rev 0x07 at pci0 dev 5 function 0 not configured
"Intel E5 Error Reporting" rev 0x07 at pci0 dev 5 function 2 not configured
"Intel E5 I/O APIC" rev 0x07 at pci0 dev 5 function 4 not configured
ppb10 at pci0 dev 17 function 0 "Intel C600 Virtual PCIE" rev 0x05
pci11 at ppb10 bus 24
ehci0 at pci0 dev 26 function 0 "Intel C600 USB" rev 0x05: apic 8 int 21
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb11 at pci0 dev 28 function 0 "Intel C600 PCIE" rev 0xb5
pci12 at ppb11 bus 11
ppb12 at pci0 dev 28 function 7 "Intel C600 PCIE" rev 0xb5
pci13 at ppb12 bus 1
"Hewlett-Packard iLO3 Slave" rev 0x05 at pci13 dev 0 function 0 not
configured
vga1 at pci13 dev 0 function 1 "Matrox MGA G200eH" rev 0x00
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
"Hewlett-Packard iLO3 Management" rev 0x05 at pci13 dev 0 function 2 not
configured
uhci0 at pci13 dev 0 function 4 "Hewlett-Packard USB" rev 0x02: apic 8 int 16
usb1 at uhci0: USB revision 1.0
uhub1 at usb1 "Hewlett-Packard UHCI root hub" rev 1.00/1.00 addr 1
ehci1 at pci0 dev 29 function 0 "Intel C600 USB" rev 0x05: apic 8 int 20
usb2 at ehci1: USB revision 2.0
uhub2 at usb2 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb13 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0xa5
pci14 at ppb13 bus 23
pcib0 at pci0 dev 31 function 0 "Intel C600 LPC" rev 0x05
"Intel C600 RAID" rev 0x05 at pci0 dev 31 function 2 not configured
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
com1: probed fifo depth: 0 bytes
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
uhub3 at uhub0 port 1 "Intel Rate Matching Hub" rev 2.00/0.00 addr 2
uhidev0 at uhub3 port 5 configuration 1 interface 0 "Cherry Mikroschalter
My3000 keyboard" rev 1.00/9.08 addr 3
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 variable keys, 6 key codes
wskbd1 at ukbd0 mux 1
wskbd1: connecting to wsdisplay0
axe0 at uhub3 port 6 configuration 1 interface 0 "ASIX Electronics AX88772B"
rev 2.00/0.01 addr 4
axe0: AX88772B, address 00:60:6e:d5:a1:eb
ukphy0 at axe0 phy 16: Generic IEEE 802.3u media interface, rev. 1: OUI
0x000ec6, model 0x0008
uhub4 at uhub2 port 1 "Intel Rate Matching Hub" rev 2.00/0.00 addr 2
uhub5 at uhub4 port 3 "Standard Microsystems product 0x2660" rev 2.00/8.01
addr 3
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
root on sd0a (019f0076fea89071.a) swap on sd0b dump on sd0b
The old MASTER node:
OpenBSD 5.2 (GENERIC.MP) #368: Wed Aug 1 10:04:49 MDT 2012
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 34188668928 (32604MB)
avail mem = 33256202240 (31715MB)
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0x7ffeb000 (181 entries)
bios0: vendor HP version "J03" date 07/02/2012
bios0: HP ProLiant DL160 Gen8
acpi0 at bios0: rev 2
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP SPCR MCFG HPET FFFF SPMI ERST APIC SRAT FFFF BERT HEST
DMAR FFFF SSDT SSDT SSDT SSDT SSDT SSDT SSDT SSDT
acpi0: wakeup devices PCI0(S5) IPT1(S5) IPT2(S5) IPT3(S5) IPT4(S5) IPT5(S5)
IPT6(S5) IPT7(S5) IPT8(S5) PT1A(S4) PT2A(S4) PT2C(S4) PT3A(S4)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimcfg0 at acpi0 addr 0x80000000, bus 0-255
acpihpet0 at acpi0: 14318179 Hz
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz, 2394.60 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,PCLMUL,MWAIT,DS-CPL,VMX,SMX,ES
T,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,AES,XSAVE,AVX,NXE,
LONG,LAHF
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: apic clock running at 99MHz
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz, 2394.23 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,PCLMUL,MWAIT,DS-CPL,VMX,SMX,ES
T,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,AES,XSAVE,AVX,NXE,
LONG,LAHF
cpu1: 256KB 64b/line 8-way L2 cache
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz, 2394.23 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,PCLMUL,MWAIT,DS-CPL,VMX,SMX,ES
T,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,AES,XSAVE,AVX,NXE,
LONG,LAHF
cpu2: 256KB 64b/line 8-way L2 cache
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz, 2394.23 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,PCLMUL,MWAIT,DS-CPL,VMX,SMX,ES
T,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,AES,XSAVE,AVX,NXE,
LONG,LAHF
cpu3: 256KB 64b/line 8-way L2 cache
ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins
ioapic1 at mainbus0: apid 0 pa 0xfec10000, version 20, 24 pins
acpiprt0 at acpi0: bus 11 (IPT1)
acpiprt1 at acpi0: bus -1 (IPT2)
acpiprt2 at acpi0: bus -1 (IPT3)
acpiprt3 at acpi0: bus -1 (IPT4)
acpiprt4 at acpi0: bus -1 (IPT5)
acpiprt5 at acpi0: bus -1 (IPT6)
acpiprt6 at acpi0: bus -1 (IPT7)
acpiprt7 at acpi0: bus 1 (IPT8)
acpiprt8 at acpi0: bus 24 (VRP1)
acpiprt9 at acpi0: bus 5 (PT1A)
acpiprt10 at acpi0: bus 18 (PT1B)
acpiprt11 at acpi0: bus 4 (PT2A)
acpiprt12 at acpi0: bus 19 (PT2B)
acpiprt13 at acpi0: bus 2 (PT2C)
acpiprt14 at acpi0: bus 3 (PT2D)
acpiprt15 at acpi0: bus 8 (PT3A)
acpiprt16 at acpi0: bus 20 (PT3B)
acpiprt17 at acpi0: bus 21 (PT3C)
acpiprt18 at acpi0: bus 22 (PT3D)
acpiprt19 at acpi0: bus 0 (PCI0)
acpicpu0 at acpi0: C2, C1
acpicpu1 at acpi0: C2, C1
acpicpu2 at acpi0: C2, C1
acpitz0 at acpi0: critical temperature is 31 degC
ipmi at mainbus0 not configured
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel E5 Host" rev 0x07
ppb0 at pci0 dev 1 function 0 "Intel E5 PCIE" rev 0x07
pci1 at ppb0 bus 5
vendor "Broadcom", unknown product 0x168e (class network subclass ethernet,
rev 0x10) at pci1 dev 0 function 0 not configured
vendor "Broadcom", unknown product 0x168e (class network subclass ethernet,
rev 0x10) at pci1 dev 0 function 1 not configured
ppb1 at pci0 dev 1 function 1 "Intel E5 PCIE" rev 0x07
pci2 at ppb1 bus 18
ppb2 at pci0 dev 2 function 0 "Intel E5 PCIE" rev 0x07
pci3 at ppb2 bus 4
ciss0 at pci3 dev 0 function 0 "Hewlett-Packard Smart Array" rev 0x01: apic 0
int 8
ciss0: 1 LD, HW rev 1, FW 3.04/3.04, 64bit fifo rro
scsibus0 at ciss0: 1 targets
sd0 at scsibus0 targ 0 lun 0: <HP, LOGICAL VOLUME, 3.04> SCSI3 0/direct fixed
sd0: 476908MB, 512 bytes/sector, 976707632 sectors
ppb3 at pci0 dev 2 function 1 "Intel E5 PCIE" rev 0x07
pci4 at ppb3 bus 19
ppb4 at pci0 dev 2 function 2 "Intel E5 PCIE" rev 0x07
pci5 at ppb4 bus 2
em0 at pci5 dev 0 function 0 "Intel I350" rev 0x01: msi, address
b4:b5:2f:5f:03:78
em1 at pci5 dev 0 function 1 "Intel I350" rev 0x01: msi, address
b4:b5:2f:5f:03:79
ppb5 at pci0 dev 2 function 3 "Intel E5 PCIE" rev 0x07
pci6 at ppb5 bus 3
unknown vendor 0x1590 product 0x005f (class memory subclass miscellaneous, rev
0x00) at pci6 dev 0 function 0 not configured
ppb6 at pci0 dev 3 function 0 "Intel E5 PCIE" rev 0x07
pci7 at ppb6 bus 8
ix0 at pci7 dev 0 function 0 "Intel 10GbE SFP+ (82599)" rev 0x01: msi, address
90:e2:ba:2f:95:80
ix1 at pci7 dev 0 function 1 "Intel 10GbE SFP+ (82599)" rev 0x01: msi, address
90:e2:ba:2f:95:81
ppb7 at pci0 dev 3 function 1 "Intel E5 PCIE" rev 0x07
pci8 at ppb7 bus 20
ppb8 at pci0 dev 3 function 2 "Intel E5 PCIE" rev 0x07
pci9 at ppb8 bus 21
ppb9 at pci0 dev 3 function 3 "Intel E5 PCIE" rev 0x07
pci10 at ppb9 bus 22
"Intel E5 DMA" rev 0x07 at pci0 dev 4 function 0 not configured
"Intel E5 DMA" rev 0x07 at pci0 dev 4 function 1 not configured
"Intel E5 DMA" rev 0x07 at pci0 dev 4 function 2 not configured
"Intel E5 DMA" rev 0x07 at pci0 dev 4 function 3 not configured
"Intel E5 DMA" rev 0x07 at pci0 dev 4 function 4 not configured
"Intel E5 DMA" rev 0x07 at pci0 dev 4 function 5 not configured
"Intel E5 DMA" rev 0x07 at pci0 dev 4 function 6 not configured
"Intel E5 DMA" rev 0x07 at pci0 dev 4 function 7 not configured
"Intel E5 Address Map" rev 0x07 at pci0 dev 5 function 0 not configured
"Intel E5 Error Reporting" rev 0x07 at pci0 dev 5 function 2 not configured
"Intel E5 I/O APIC" rev 0x07 at pci0 dev 5 function 4 not configured
ppb10 at pci0 dev 17 function 0 "Intel C600 Virtual PCIE" rev 0x05
pci11 at ppb10 bus 24
ehci0 at pci0 dev 26 function 0 "Intel C600 USB" rev 0x05: apic 8 int 21
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb11 at pci0 dev 28 function 0 "Intel C600 PCIE" rev 0xb5
pci12 at ppb11 bus 11
ppb12 at pci0 dev 28 function 7 "Intel C600 PCIE" rev 0xb5
pci13 at ppb12 bus 1
"Hewlett-Packard iLO3 Slave" rev 0x05 at pci13 dev 0 function 0 not
configured
vga1 at pci13 dev 0 function 1 vendor "Matrox", unknown product 0x0533 rev
0x00
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
"Hewlett-Packard iLO3 Management" rev 0x05 at pci13 dev 0 function 2 not
configured
uhci0 at pci13 dev 0 function 4 "Hewlett-Packard USB" rev 0x02: apic 8 int 16
usb1 at uhci0: USB revision 1.0
uhub1 at usb1 "Hewlett-Packard UHCI root hub" rev 1.00/1.00 addr 1
ehci1 at pci0 dev 29 function 0 "Intel C600 USB" rev 0x05: apic 8 int 20
usb2 at ehci1: USB revision 2.0
uhub2 at usb2 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb13 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0xa5
pci14 at ppb13 bus 23
pcib0 at pci0 dev 31 function 0 "Intel C600 LPC" rev 0x05
"Intel C600 RAID" rev 0x05 at pci0 dev 31 function 2 not configured
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
com1: probed fifo depth: 0 bytes
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
mtrr: Pentium Pro MTRR support
uhub3 at uhub0 port 1 "Intel Rate Matching Hub" rev 2.00/0.00 addr 2
uhidev0 at uhub3 port 5 configuration 1 interface 0 "Cherry Mikroschalter
My3000 keyboard" rev 1.00/9.08 addr 3
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 variable keys, 6 key codes
wskbd1 at ukbd0 mux 1
wskbd1: connecting to wsdisplay0
axe0 at uhub3 port 6 configuration 1 interface 0 "ASIX Electronics AX88772B"
rev 2.00/0.01 addr 4
axe0: AX88772B, address 00:60:6e:d5:9f:16
ukphy0 at axe0 phy 16: Generic IEEE 802.3u media interface, rev. 1: OUI
0x000ec6, model 0x0008
uhub4 at uhub2 port 1 "Intel Rate Matching Hub" rev 2.00/0.00 addr 2
uhub5 at uhub4 port 3 "Standard Microsystems product 0x2660" rev 2.00/8.01
addr 3
vscsi0 at root
scsibus1 at vscsi0: 256 targets
softraid0 at root
scsibus2 at softraid0: 256 targets
root on sd0a (c15d21bda6ebe005.a) swap on sd0b dump on sd0b
