DRAM/NVRAM cfg match
Disks have good cfg but they do not match DRAM cfg
Firmware cannot flush cache
After resaving RAID configuration it started OK and everything seems
nominal... Nothing on the logs. It was very alike to the other problem,
but both could have been crappy Dell hardware, just mentioning this for
if anyone gets any idea...
Something funny to mention, this machine was carped but it was sending
advertises over the carp interface, even thought it had no disk, so the
other never took the MASTER
Thanks,
Alejandro.
dmesg:
OpenBSD 3.9-stable (GENERIC) #5: Thu Jan 4 19:36:23 GMT 2007
[EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: Intel(R) Xeon(TM) CPU 2.80GHz ("GenuineIntel" 686-class) 2.80 GHz
cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,CNXT-ID
real mem = 2146807808 (2096492K)
avail mem = 1952804864 (1907036K)
using 4278 buffers containing 107442176 bytes (104924K) of memory
mainbus0 (root)
bios0 at mainbus0: AT/286+(00) BIOS, date 01/09/06, BIOS32 rev. 0 @ 0xffe90
pcibios0 at bios0: rev 2.1 @ 0xf0000/0x10000
pcibios0: PCI IRQ Routing Table rev 1.0 @ 0xfb600/320 (18 entries)
pcibios0: PCI Interrupt Router at 000:31:0 ("Intel 82801EB/ER LPC" rev 0x00)
pcibios0: PCI bus #9 is the last bus
bios0: ROM list: 0xc0000/0xb000! 0xcb000/0x1000 0xcc000/0x800
0xcc800/0x1000 0xcd800/0x2600 0xec000/0x4000!
ipmi0 at mainbus0: version 1.5 interface KCS iobase 0xca8/8 spacing 4
cpu0 at mainbus0
pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
pchb0 at pci0 dev 0 function 0 "Intel E7520 MCH" rev 0x09
ppb0 at pci0 dev 2 function 0 "Intel MCH PCIE" rev 0x09
pci1 at ppb0 bus 1
ppb1 at pci1 dev 0 function 0 "Intel IOP331 Channel 0" rev 0x06
pci2 at ppb1 bus 2
mpt0 at pci2 dev 5 function 0 "Symbios Logic 53c1030" rev 0x08: irq 7
scsibus0 at mpt0: 16 targets
mpt1 at pci2 dev 5 function 1 "Symbios Logic 53c1030" rev 0x08: irq 3
scsibus1 at mpt1: 16 targets
ppb2 at pci1 dev 0 function 2 "Intel IOP331 Channel 1" rev 0x06
pci3 at ppb2 bus 3
ami0 at pci3 dev 11 function 0 "Symbios Logic MegaRAID" rev 0x01: irq 3
Dell 518 64b/lhc
ami0: FW 352A, BIOS v1.10, 128MB RAM
ami0: 2 channels, 0 FC loops, 1 logical drives
scsibus2 at ami0: 40 targets
sd0 at scsibus2 targ 0 lun 0: <AMI, Host drive #00, > SCSI2 0/direct fixed
sd0: 279800MB, 279800 cyl, 64 head, 32 sec, 512 bytes/sec, 573030400 sec
total
scsibus3 at ami0: 16 targets
safte0 at scsibus3 targ 6 lun 0: <PE/PV, 1x6 SCSI BP, 1.0> SCSI2
3/processor fixed
scsibus4 at ami0: 16 targets
ppb3 at pci0 dev 4 function 0 "Intel MCH PCIE" rev 0x09
pci4 at ppb3 bus 4
ppb4 at pci0 dev 5 function 0 "Intel MCH PCIE" rev 0x09
pci5 at ppb4 bus 5
ppb5 at pci5 dev 0 function 0 "Intel PCIE-PCIE" rev 0x09
pci6 at ppb5 bus 6
em0 at pci6 dev 7 function 0 "Intel PRO/1000MT (82541GI)" rev 0x05: irq
11, address 00:18:8b:34:86:bd
ppb6 at pci5 dev 0 function 2 "Intel PCIE-PCIE" rev 0x09
pci7 at ppb6 bus 7
em1 at pci7 dev 8 function 0 "Intel PRO/1000MT (82541GI)" rev 0x05: irq
3, address 00:18:8b:34:86:be
ppb7 at pci0 dev 6 function 0 "Intel MCH PCIE" rev 0x09
pci8 at ppb7 bus 8
uhci0 at pci0 dev 29 function 0 "Intel 82801EB/ER USB" rev 0x02: irq 11
usb0 at uhci0: USB revision 1.0
uhub0 at usb0
uhub0: Intel UHCI root hub, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
uhci1 at pci0 dev 29 function 1 "Intel 82801EB/ER USB" rev 0x02: irq 10
usb1 at uhci1: USB revision 1.0
uhub1 at usb1
uhub1: Intel UHCI root hub, rev 1.00/1.00, addr 1
uhub1: 2 ports with 2 removable, self powered
uhci2 at pci0 dev 29 function 2 "Intel 82801EB/ER USB" rev 0x02: irq 7
usb2 at uhci2: USB revision 1.0
uhub2 at usb2
uhub2: Intel UHCI root hub, rev 1.00/1.00, addr 1
uhub2: 2 ports with 2 removable, self powered
ehci0 at pci0 dev 29 function 7 "Intel 82801EB/ER USB2" rev 0x02: irq 5
usb3 at ehci0: USB revision 2.0
uhub3 at usb3
uhub3: Intel EHCI root hub, rev 2.00/1.00, addr 1
uhub3: 6 ports with 6 removable, self powered
ppb8 at pci0 dev 30 function 0 "Intel 82801BA AGP" rev 0xc2
pci9 at ppb8 bus 9
vga1 at pci9 dev 13 function 0 "ATI Radeon VE QY" rev 0x00
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
ichpcib0 at pci0 dev 31 function 0 "Intel 82801EB/ER LPC" rev 0x02
pciide0 at pci0 dev 31 function 1 "Intel 82801EB/ER IDE" rev 0x02: DMA,
channel 0 configured to compatibility, channel 1 configured to compatibility
atapiscsi0 at pciide0 channel 0 drive 0
scsibus5 at atapiscsi0: 2 targets
cd0 at scsibus5 targ 0 lun 0: <HL-DT-ST, CDRW/DVD GCC4244, B101> SCSI0
5/cdrom removable
cd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2
pciide0: channel 1 disabled (no drives)
isa0 at ichpcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pmsi0 at pckbc0 (aux slot)
pckbc0: using irq 12 for aux slot
wsmouse0 at pmsi0 mux 0
pcppi0 at isa0 port 0x61
midi0 at pcppi0: <PC speaker>
spkr0 at pcppi0
npx0 at isa0 port 0xf0/16: using exception 16
pccom0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
biomask efed netmask efed ttymask ffef
pctr: user-level cycle counter enabled
uhub4 at uhub3 port 3
uhub4: Dell product 0xa001, rev 2.00/0.00, addr 2
uhub4: 2 ports with 2 removable, self powered, multiple transaction
translators
dkcsum: sd0 matches BIOS drive 0x80
root on sd0a
rootdev=0x400 rrootdev=0xd00 rawdev=0xd02
WARNING: / was not properly unmounted
Marco Peereboom wrote:
>I honestly have no clue. I have banged on my mfis as much as I could
>and have never seen anything like this. I am doing some investigation
>into this. If you find a way to repro this let me know please.
>
>On Mon, Feb 12, 2007 at 01:12:56PM -0500, Alejandro Lozanoff wrote:
>
>
>>I was planning on running bonnie tonight (This is a production server)
>>to get some IO and see if i can reproduce the problem.
>>We have 3 of this servers with OBSD 4.0 stable and the same RAID card
>>running for about 2 months now, this is the first problem we
>>encounter. If you have any idea of how to reproduce this i would be
>>glad to test it in one of the servers that we are not using right now.
>>
>>Thanks for replying
>>Alejandro.-
>>
>>Quoting Marco Peereboom <[EMAIL PROTECTED]>:
>>
>>
>>
>>>I have never seen this but I am very interested in this particular
>>>instance. Apparently there is an issue with read ahead on mfi that I
>>>have never seen before on OpenBSD but other OS' have run into.
>>>
>>>Is this reproducible? If so can you try to disable read ahead in
>>>CTRL-R (bios)?
>>>
>>>Thanks,
>>>/marco
>>>
>>>On Mon, Feb 12, 2007 at 11:31:26AM -0500, Alejandro Lozanoff wrote:
>>>
>>>
>>>>Sorry for the message without body, im a little sleepy and hitted the
>>>>wrong button... :p
>>>>
>>>>Ok,
>>>>
>>>>I had this problem last night on one of our shiny Dell PowerEdge 2950s
>>>>with RAID 10 and SAS disks.
>>>>For no reason it started screaming "sd0: not queued: error 5" on the
>>>>console, the server didnt crash (no core, no trace) but was in an
>>>>unresponsive state, i couldnt log on or anything. I was wondering if
>>>>this error could be a hardware problem (RAID card maybe?).
>>>>The logs dont show anything notorious, there were no heavy IO at the
>>>>time of the problem. I'm kind of clueless at what could have been the
>>>>problem, a quick search showed some old messages with SCSI cards
>>>>problems only.
>>>>
>>>>This is 4.0 Stable as shown below.
>>>>
>>>>dmesg follows:
>>>>
>>>>OpenBSD 4.0-stable (GENERIC) #1: Mon Nov 27 16:23:49 GMT 2006
>>>> [EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/GENERIC
>>>>cpu0: Intel(R) Xeon(TM) CPU 3.00GHz ("GenuineIntel" 686-class) 3 GHz
>>>>cpu0:
>>>>
>>>>
>>>>
>>FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
>>
>>
>>H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,CNXT-ID,C
>>
>>
>>>>X16
>>>>cpu0: Enhanced SpeedStep disabled by BIOS
>>>>real mem = 1072955392 (1047808K)
>>>>avail mem = 970735616 (947984K)
>>>>using 4256 buffers containing 53751808 bytes (52492K) of memory
>>>>mainbus0 (root)
>>>>bios0 at mainbus0: AT/286+(00) BIOS, date 06/21/06, BIOS32 rev. 0 @
>>>>0xffe90, SMBIOS rev. 2.4 @ 0x3ffbc000 (62 entries)
>>>>bios0: Dell Inc. PowerEdge 2950
>>>>pcibios0 at bios0: rev 2.1 @ 0xf0000/0x10000
>>>>pcibios0: PCI IRQ Routing Table rev 1.0 @ 0xfade0/384 (22 entries)
>>>>pcibios0: PCI Interrupt Router at 000:31:0 ("Intel 6321ESB LPC" rev 0x00)
>>>>pcibios0: PCI bus #16 is the last bus
>>>>bios0: ROM list: 0xc0000/0x9000! 0xc9000/0x1000 0xca000/0x1800
>>>>0xcb800/0x5200 0xec000/0x4000!
>>>>ipmi0 at mainbus0: version 2.0 interface KCS iobase 0xca8/8 spacing 4
>>>>cpu0 at mainbus0
>>>>pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
>>>>pchb0 at pci0 dev 0 function 0 "Intel 5000X Host" rev 0x12
>>>>ppb0 at pci0 dev 2 function 0 "Intel 5000 PCIE" rev 0x12
>>>>pci1 at ppb0 bus 6
>>>>ppb1 at pci1 dev 0 function 0 "Intel 6321ESB PCIE" rev 0x01
>>>>pci2 at ppb1 bus 7
>>>>ppb2 at pci2 dev 0 function 0 "Intel 6321ESB PCIE" rev 0x01
>>>>pci3 at ppb2 bus 8
>>>>ppb3 at pci3 dev 0 function 0 "ServerWorks PCIE-PCIX" rev 0xc2
>>>>pci4 at ppb3 bus 9
>>>>bnx0 at pci4 dev 0 function 0 "Broadcom BCM5708" rev 0x11: irq 5,
>>>>address 00:18:8b:72:c0:fd
>>>>brgphy0 at bnx0 phy 1: BCM5708C 10/100/1000baseT PHY, rev. 5
>>>>ppb4 at pci2 dev 1 function 0 "Intel 6321ESB PCIE" rev 0x01
>>>>pci5 at ppb4 bus 10
>>>>ppb5 at pci1 dev 0 function 3 "Intel 6321ESB PCIE-PCIX" rev 0x01
>>>>pci6 at ppb5 bus 11
>>>>ppb6 at pci0 dev 3 function 0 "Intel 5000 PCIE" rev 0x12
>>>>pci7 at ppb6 bus 1
>>>>ppb7 at pci7 dev 0 function 0 "Intel IOP333 PCIE-PCIX" rev 0x00
>>>>pci8 at ppb7 bus 2
>>>>mfi0 at pci8 dev 14 function 0 "Dell PERC 5" rev 0x00: irq 6
>>>>mfi0: logical drives 1, version 5.0.1-0030, 256MB RAM
>>>>scsibus0 at mfi0: 1 targets
>>>>sd0 at scsibus0 targ 0 lun 0: <DELL, PERC 5/i, 1.00> SCSI3 0/direct fixed
>>>>sd0: 278784MB, 278784 cyl, 64 head, 32 sec, 512 bytes/sec, 570949632 sec
>>>>total
>>>>ppb8 at pci7 dev 0 function 2 "Intel IOP333 PCIE-PCIX" rev 0x00
>>>>pci9 at ppb8 bus 3
>>>>ppb9 at pci0 dev 4 function 0 "Intel 5000 PCIE" rev 0x12
>>>>pci10 at ppb9 bus 12
>>>>ppb10 at pci0 dev 5 function 0 "Intel 5000 PCIE" rev 0x12
>>>>pci11 at ppb10 bus 13
>>>>ppb11 at pci0 dev 6 function 0 "Intel 5000 PCIE" rev 0x12
>>>>pci12 at ppb11 bus 14
>>>>ppb12 at pci0 dev 7 function 0 "Intel 5000 PCIE" rev 0x12
>>>>pci13 at ppb12 bus 15
>>>>pchb1 at pci0 dev 16 function 0 "Intel 5000 Error Reporting" rev 0x12
>>>>pchb2 at pci0 dev 16 function 1 "Intel 5000 Error Reporting" rev 0x12
>>>>pchb3 at pci0 dev 16 function 2 "Intel 5000 Error Reporting" rev 0x12
>>>>pchb4 at pci0 dev 17 function 0 "Intel 5000 Reserved" rev 0x12
>>>>pchb5 at pci0 dev 19 function 0 "Intel 5000 Reserved" rev 0x12
>>>>pchb6 at pci0 dev 21 function 0 "Intel 5000 FBD" rev 0x12
>>>>pchb7 at pci0 dev 22 function 0 "Intel 5000 FBD" rev 0x12
>>>>ppb13 at pci0 dev 28 function 0 "Intel 6321ESB PCIE" rev 0x09
>>>>pci14 at ppb13 bus 4
>>>>ppb14 at pci14 dev 0 function 0 "ServerWorks PCIE-PCIX" rev 0xc2
>>>>pci15 at ppb14 bus 5
>>>>bnx1 at pci15 dev 0 function 0 "Broadcom BCM5708" rev 0x11: irq 5,
>>>>address 00:18:8b:72:c0:fb
>>>>brgphy1 at bnx1 phy 1: BCM5708C 10/100/1000baseT PHY, rev. 5
>>>>uhci0 at pci0 dev 29 function 0 "Intel 6321ESB USB" rev 0x09: irq 11
>>>>usb0 at uhci0: USB revision 1.0
>>>>uhub0 at usb0
>>>>uhub0: Intel UHCI root hub, rev 1.00/1.00, addr 1
>>>>uhub0: 2 ports with 2 removable, self powered
>>>>uhci1 at pci0 dev 29 function 1 "Intel 6321ESB USB" rev 0x09: irq 10
>>>>usb1 at uhci1: USB revision 1.0
>>>>uhub1 at usb1
>>>>uhub1: Intel UHCI root hub, rev 1.00/1.00, addr 1
>>>>uhub1: 2 ports with 2 removable, self powered
>>>>uhci2 at pci0 dev 29 function 2 "Intel 6321ESB USB" rev 0x09: irq 11
>>>>usb2 at uhci2: USB revision 1.0
>>>>uhub2 at usb2
>>>>uhub2: Intel UHCI root hub, rev 1.00/1.00, addr 1
>>>>uhub2: 2 ports with 2 removable, self powered
>>>>ehci0 at pci0 dev 29 function 7 "Intel 6321ESB USB" rev 0x09: irq 11
>>>>usb3 at ehci0: USB revision 2.0
>>>>uhub3 at usb3
>>>>uhub3: Intel EHCI root hub, rev 2.00/1.00, addr 1
>>>>uhub3: 6 ports with 6 removable, self powered
>>>>ppb15 at pci0 dev 30 function 0 "Intel 82801BA AGP" rev 0xd9
>>>>pci16 at ppb15 bus 16
>>>>vga1 at pci16 dev 13 function 0 "ATI ES1000" rev 0x02
>>>>wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
>>>>wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
>>>>ichpcib0 at pci0 dev 31 function 0 "Intel 6321ESB LPC" rev 0x09: PM
>>>>
>>>>
>>disabled
>>
>>
>>>>pciide0 at pci0 dev 31 function 1 "Intel 6321ESB IDE" rev 0x09: DMA,
>>>>channel 0 configured to compatibility, channel 1 configured to
>>>>compatibility
>>>>atapiscsi0 at pciide0 channel 0 drive 0
>>>>scsibus1 at atapiscsi0: 2 targets
>>>>cd0 at scsibus1 targ 0 lun 0: <HL-DT-ST, CD-ROM GCR-8240N, 1.10> SCSI0
>>>>5/cdrom removable
>>>>cd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2
>>>>pciide0: channel 1 ignored (disabled)
>>>>isa0 at ichpcib0
>>>>isadma0 at isa0
>>>>pckbc0 at isa0 port 0x60/5
>>>>pckbd0 at pckbc0 (kbd slot)
>>>>pckbc0: using irq 1 for kbd slot
>>>>wskbd0 at pckbd0: console keyboard, using wsdisplay0
>>>>pcppi0 at isa0 port 0x61
>>>>midi0 at pcppi0: <PC speaker>
>>>>spkr0 at pcppi0
>>>>npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
>>>>pccom0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
>>>>pccom1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
>>>>biomask ffc5 netmask ffe5 ttymask ffe7
>>>>pctr: user-level cycle counter enabled
>>>>uhub4 at uhub3 port 5
>>>>uhub4: Cypress Semiconductor USB2 Hub, rev 2.00/0.0b, addr 2
>>>>uhub4: 4 ports with 4 removable, self powered, multiple transaction
>>>>translators
>>>>uhidev0 at uhub1 port 1 configuration 1 interface 0
>>>>uhidev0: CHESEN PS2 to USB Converter, rev 1.10/0.10, addr 2, iclass 3/1
>>>>ukbd0 at uhidev0: 8 modifier keys, 6 key codes
>>>>wskbd1 at ukbd0 mux 1
>>>>wskbd1: connecting to wsdisplay0
>>>>uhidev1 at uhub1 port 1 configuration 1 interface 1
>>>>uhidev1: CHESEN PS2 to USB Converter, rev 1.10/0.10, addr 2, iclass 3/1
>>>>uhidev1: 3 report ids
>>>>ums0 at uhidev1 reportid 1: 5 buttons and Z dir.
>>>>wsmouse0 at ums0 mux 0
>>>>uhid0 at uhidev1 reportid 2: input=1, output=0, feature=0
>>>>uhid1 at uhidev1 reportid 3: input=3, output=0, feature=0
>>>>dkcsum: sd0 matches BIOS drive 0x80
>>>>root on sd0a
>>>>rootdev=0x400 rrootdev=0xd00 rawdev=0xd02
>>>>WARNING: / was not properly unmounted
>>>>
>>>>
>
>
>__________ NOD32 2032 (20070202) Information __________
>
>This message was checked by NOD32 antivirus system.
>http://www.eset.com