Hi!
During installing a pair of OpenBSD 4.6 amd64 (patched with current
patches i.e. up to 004 and included) firewalls on IBM 3550 M2 computers
i was evaluating different options to have pfsync traffic carried
between them. Although i intend to use separate vlan for pfsync i tried
out also how does pfsync over ipsec sound and encountered that firewalls
reboot or panic so to say to the ddb> prompt with varios messages.
It happens every time sooner or later (instanly after bringing up pfsync
interfaces or some minutes later, there is practically no traffic on
firewalls, its test env.). It happens in my case only when i use pfsync
over ipsec which in turn runs over vlan and which in turn runs over one
em adapter. (I saw someone said week ago on misc@ they experience
something related to pfsync and reboots).
Otherwise these computers behave normally with OpenBSD as far as i can
tell (having used them for testing and other projects running without
any flaw for weeks). pfsync over vlan interfaces works, and just ipsec
over vlan interface works too. There aint running any services but
default after-install daemons and carp is configured as normal
master/slave on a interface different from where ipsec+pfsync events happen.
PF part:
set skip on vlan607 # there happens my ipsec
pass on enc0
IPsec part:
fw2 # ifconfig enc0 up
fw1 # ifconfig enc0 up
fw2 # cat ipsec.conf
ike passive esp from 10.1.7.108 to 10.1.7.107
fw1 # cat /etc/ipsec.conf
ike esp from 10.1.7.107 to 10.1.7.108
fw2 # isakmpd -Kvd
072201.832359 Default isakmpd: phase 1 done: initiator id 10.1.7.107,
responder id 10.1.7.108, src: 10.1.7.108 dst: 10.1.7.107
072201.840201 Default isakmpd: quick mode done: src: 10.1.7.108 dst:
10.1.7.107
fw1 # isakmpd -Kvd
072202.830382 Default isakmpd: phase 1 done: initiator id 10.1.7.107,
responder id 10.1.7.108, src: 10.1.7.107 dst: 10.1.7.108
072202.837902 Default isakmpd: quick mode done: src: 10.1.7.107 dst:
10.1.7.108
and i bring it up like this, first at fw2 and then fw1
# ipsecctl -f /etc/ipsec.conf
PFSYNC part:
fw2# ifconfig pfsync0 syncpeer 10.1.7.107 syncdev enc0
fw1# ifconfig pfsync0 syncpeer 10.1.7.108 syncdev enc0
fw2# ifconfig pfsync0 up
fw1# ifconfig pfsync0 up
(using /etc/netstart script i get not all the times this message but
most of the times
fw1 # sh /etc/netstart pfsync0
ifconfig: SIOCSETPFSYNC: Can't assign requested address)
So, sometimes i it is works for some minutes and it is possible to
confirm with tcpdump that pfsync traffic actually goes over enc0 interface.
When it panics i usually get on the console these messages (ddb> prompt
isnt responding, it might be because these are java management consoles
not serial)
1. sometimes (needs manual powercycle)
panic: tcp_output: template len != hdrlen - optlen
Stopped at Debuuger+0x5: leave
RUN AT LEAST 'trace' ..
ddb>
2. most often, at both sides (needs manual powercycle)
kernel: protection fault trap, code=0
Stopped at ether_input+0x66: testb $0x1,0(%r14)
ddb>
3. rearely, computer went to reboot and i couldnt take a note or reproduce
kernel: panic smash stack
...
ddb>
I intend to try it out on a different hardware but it would be helpful
to me to know if other people encounter something similar or it rather
is related to my hardware platform.
Best regards,
Imre
And this is my dmesg
OpenBSD 4.6 (GENERIC) #0: Fri Nov 13 12:42:15 MST 2009
r...@rtm2.avalik.kit:/usr/src/sys/arch/amd64/compile/GENERIC
real mem = 2135814144 (2036MB)
avail mem = 2061455360 (1965MB)
RTC BIOS diagnostic error 80<clock_battery>
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.5 @ 0x7f6bd000 (74 entries)
bios0: vendor IBM Corp. version "-[D6E126AUS-1.02]-" date 06/26/2009
bios0: IBM 49Y6498
acpi0 at bios0: rev 2
acpi0: tables DSDT FACP TCPA APIC MCFG SLIC HPET SSDT ERST BERT DMAR
acpi0: wakeup devices UHC1(S4) UHC2(S4) UHC3(S4) UHC4(S4) UHC5(S4)
EHC1(S4) EHC2(S4)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E5520 @ 2.27GHz, 2267.07 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: apic clock running at 133MHz
cpu at mainbus0: not configured
cpu at mainbus0: not configured
cpu at mainbus0: not configured
cpu at mainbus0: not configured
cpu at mainbus0: not configured
cpu at mainbus0: not configured
cpu at mainbus0: not configured
ioapic0 at mainbus0 apid 8 pa 0xfec00000, version 20, 24 pins
ioapic1 at mainbus0 apid 9 pa 0xfec80000, version 20, 24 pins
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 11 (PCI1)
acpiprt2 at acpi0: bus 21 (PCI3)
acpiprt3 at acpi0: bus -1 (PCI6)
acpiprt4 at acpi0: bus 26 (PCI7)
acpiprt5 at acpi0: bus -1 (PCI9)
acpicpu0 at acpi0
ipmi at mainbus0 not configured
cpu0: unknown i686 model 0x1a, can't get bus clock
cpu0: EST: unknown system bus clock
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 vendor "Intel", unknown product 0x3406
rev 0x13
ppb0 at pci0 dev 1 function 0 "Intel X58 PCIE" rev 0x13
pci1 at ppb0 bus 11
bnx0 at pci1 dev 0 function 0 "Broadcom BCM5709" rev 0x20: apic 9 int 4
(irq 11)
bnx1 at pci1 dev 0 function 1 "Broadcom BCM5709" rev 0x20: apic 9 int 16
(irq 10)
ppb1 at pci0 dev 2 function 0 "Intel X58 PCIE" rev 0x13
pci2 at ppb1 bus 16
ppb2 at pci0 dev 3 function 0 "Intel X58 PCIE" rev 0x13: apic 9 int 0
(irq 11)
pci3 at ppb2 bus 21
ppb3 at pci0 dev 7 function 0 "Intel X58 PCIE" rev 0x13: apic 9 int 6
(irq 11)
pci4 at ppb3 bus 26
ppb4 at pci4 dev 0 function 0 "IDT 89HPES12N3A" rev 0x0e
pci5 at ppb4 bus 27
ppb5 at pci5 dev 2 function 0 "IDT 89HPES12N3A" rev 0x0e
pci6 at ppb5 bus 28
em0 at pci6 dev 0 function 0 "Intel PRO/1000 QP (82571EB)" rev 0x06:
apic 9 int 14 (irq 5), address 00:15:17:c4:4a:c1
em1 at pci6 dev 0 function 1 "Intel PRO/1000 QP (82571EB)" rev 0x06:
apic 9 int 15 (irq 11), address 00:15:17:c4:4a:c0
ppb6 at pci5 dev 4 function 0 "IDT 89HPES12N3A" rev 0x0e
pci7 at ppb6 bus 29
em2 at pci7 dev 0 function 0 "Intel PRO/1000 QP (82571EB)" rev 0x06:
apic 9 int 13 (irq 10), address 00:15:17:c4:4a:c3
em3 at pci7 dev 0 function 1 "Intel PRO/1000 QP (82571EB)" rev 0x06:
apic 9 int 6 (irq 11), address 00:15:17:c4:4a:c2
"Intel X58 QuickPath" rev 0x13 at pci0 dev 16 function 0 not configured
"Intel X58 QuickPath" rev 0x13 at pci0 dev 16 function 1 not configured
"Intel X58 QuickPath" rev 0x13 at pci0 dev 17 function 0 not configured
"Intel X58 QuickPath" rev 0x13 at pci0 dev 17 function 1 not configured
"Intel X58 Misc" rev 0x13 at pci0 dev 20 function 0 not configured
"Intel X58 GPIO" rev 0x13 at pci0 dev 20 function 1 not configured
"Intel X58 RAS" rev 0x13 at pci0 dev 20 function 2 not configured
"Intel X58 Throttle" rev 0x13 at pci0 dev 20 function 3 not configured
vendor "Intel", unknown product 0x342f (class system subclass interrupt,
rev 0x13) at pci0 dev 21 function 0 not configured
vendor "Intel", unknown product 0x3430 (class system subclass
miscellaneous, rev 0x13) at pci0 dev 22 function 0 not configured
vendor "Intel", unknown product 0x3431 (class system subclass
miscellaneous, rev 0x13) at pci0 dev 22 function 1 not configured
vendor "Intel", unknown product 0x3432 (class system subclass
miscellaneous, rev 0x13) at pci0 dev 22 function 2 not configured
vendor "Intel", unknown product 0x3433 (class system subclass
miscellaneous, rev 0x13) at pci0 dev 22 function 3 not configured
vendor "Intel", unknown product 0x3429 (class system subclass
miscellaneous, rev 0x13) at pci0 dev 22 function 4 not configured
vendor "Intel", unknown product 0x342a (class system subclass
miscellaneous, rev 0x13) at pci0 dev 22 function 5 not configured
vendor "Intel", unknown product 0x342b (class system subclass
miscellaneous, rev 0x13) at pci0 dev 22 function 6 not configured
vendor "Intel", unknown product 0x342c (class system subclass
miscellaneous, rev 0x13) at pci0 dev 22 function 7 not configured
uhci0 at pci0 dev 26 function 0 "Intel 82801JI USB" rev 0x00: apic 8 int
17 (irq 10)
uhci1 at pci0 dev 26 function 1 "Intel 82801JI USB" rev 0x00: apic 8 int
18 (irq 11)
ehci0 at pci0 dev 26 function 7 "Intel 82801JI USB" rev 0x00: apic 8 int
19 (irq 5)
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb7 at pci0 dev 28 function 0 "Intel 82801JI PCIE" rev 0x00: apic 8 int
16 (irq 11)
pci8 at ppb7 bus 1
mfi0 at pci8 dev 0 function 0 "Symbios Logic SAS1078" rev 0x04: apic 8
int 16 (irq 11), 0x03641014
mfi0: logical drives 1, version 11.0.1-0014, 256MB RAM
scsibus0 at mfi0: 1 targets
sd0 at scsibus0 targ 0 lun 0: <IBM, ServeRAID-MR10i, 1.40> SCSI3
0/direct fixed
sd0: 139236MB, 512 bytes/sec, 285155328 sec total
ppb8 at pci0 dev 28 function 4 "Intel 82801JI PCIE" rev 0x00
pci9 at ppb8 bus 6
ppb9 at pci9 dev 0 function 0 unknown vendor 0x101b product 0x0452 rev 0x01
pci10 at ppb9 bus 7
vga1 at pci10 dev 0 function 0 vendor "Matrox", unknown product 0x0530
rev 0x00
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
uhci2 at pci0 dev 29 function 0 "Intel 82801JI USB" rev 0x00: apic 8 int
17 (irq 10)
uhci3 at pci0 dev 29 function 1 "Intel 82801JI USB" rev 0x00: apic 8 int
18 (irq 11)
uhci4 at pci0 dev 29 function 2 "Intel 82801JI USB" rev 0x00: apic 8 int
19 (irq 5)
ehci1 at pci0 dev 29 function 7 "Intel 82801JI USB" rev 0x00: apic 8 int
17 (irq 10)
usb1 at ehci1: USB revision 2.0
uhub1 at usb1 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb10 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0x90
pci11 at ppb10 bus 31
pcib0 at pci0 dev 31 function 0 "Intel 82801JIB LPC" rev 0x00
pciide0 at pci0 dev 31 function 2 "Intel 82801JI SATA" rev 0x00: DMA,
channel 0 configured to native-PCI, channel 1 configured to native-PC
pciide0: using apic 8 int 16 (irq 11) for native-PCI interrupt
ichiic0 at pci0 dev 31 function 3 "Intel 82801JI SMBus" rev 0x00: apic 8
int 22 (irq 11)
iic0 at ichiic0
iic0: addr 0x2e 00=40 words 00=4040 01=0000 02=0000 03=0000 04=0000
05=0000 06=0000 07=0000
pciide1 at pci0 dev 31 function 5 "Intel 82801JI SATA" rev 0x00: DMA,
channel 0 wired to native-PCI, channel 1 wired to native-PCI
pciide1: using apic 8 int 21 (irq 11) for native-PCI interrupt
usb2 at uhci0: USB revision 1.0
uhub2 at usb2 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb3 at uhci1: USB revision 1.0
uhub3 at usb3 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb4 at uhci2: USB revision 1.0
uhub4 at usb4 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb5 at uhci3: USB revision 1.0
uhub5 at usb5 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb6 at uhci4: USB revision 1.0
uhub6 at usb6 "Intel UHCI root hub" rev 1.00/1.00 addr 1
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
midi0 at pcppi0: <PC speaker>
spkr0 at pcppi0
mtrr: Pentium Pro MTRR support
uhidev0 at uhub1 port 1 configuration 2 interface 0 "IBM IBM Composite
Device-0" rev 2.00/0.00 addr 2
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 modifier keys, 6 key codes
wskbd1 at ukbd0 mux 1
wskbd1: connecting to wsdisplay0
uhidev1 at uhub1 port 1 configuration 2 interface 1 "IBM IBM Composite
Device-0" rev 2.00/0.00 addr 2
uhidev1: iclass 3/1
ums0 at uhidev1
ums0: X report 0x0002 not supported
uhidev2 at uhub1 port 1 configuration 2 interface 2 "IBM IBM Composite
Device-0" rev 2.00/0.00 addr 2
uhidev2: iclass 3/1
ums1 at uhidev2: 3 buttons, Z dir
wsmouse0 at ums1 mux 0
cdce0 at uhub4 port 2 configuration 1 interface 0 "IBM RNDIS/CDC ETHER"
rev 2.00/2.15 addr 2
cdce0: address 02:21:5e:ca:33:33
softraid0 at root
root on sd0d swap on sd0b dump on sd0b
WARNING: / was not properly unmounted
bnx0: address 00:21:5e:c6:3b:74
brgphy0 at bnx0 phy 1: BCM5709 10/100/1000baseT PHY, rev. 8
bnx1: address 00:21:5e:c6:3b:76
brgphy1 at bnx1 phy 1: BCM5709 10/100/1000baseT PHY, rev. 8
Sidenote:
To get ibm 3550 m2 going in the firsh place i did these changes to
default bios
1. System Settings -> Devices and I/O ports -> Configure IDE mode
-> <native mode>
2. Processors -> Processor Performance
-> Proc Performance States: <Disabled>
(Otherwise i couldn't boot for couple of reasons, if somebody would like
to look at those reasons, i took srceenshots at the time of 4.6-20090813
snapshot, they are at http://kuutorvaja.eenet.ee/wiki/IBM_System_x3550_M2)
