Hi, Just an update on this issue.
Quick summary: I fixed the BIOS issues, the hardware monitor issues, and the rl0/rl1 watchdog timeout issues (it seems). However I'm still having problems with my SATA drives (or at least one of them). More info below.
BIOS:I flashed my BIOS to the latest version about a year ago, and never noticed that there was any problem, but it turns out there was. I never reset the BIOS to default factory settings after the upgrade, and it seems the settings were corrupt. After having reset the BIOS to the "default optimized factory settings" it stopped crashing when I go into the H/W monitor and also when using healthd -d (output below):
Temp.= 40.0, 36.0, 66.0; Rot.= 0, 0, 0 Vcore = 1.44, 3.12; Volt. = 3.34, 5.00, 1.95, -0.11, -1.54 Temp.= 40.0, 36.0, 66.0; Rot.= 0, 0, 0 Vcore = 1.44, 3.14; Volt. = 3.33, 4.97, 1.95, -0.11, -1.54 Temp.= 40.0, 36.0, 66.0; Rot.= 0, 0, 0 Vcore = 1.44, 3.12; Volt. = 3.34, 4.97, 1.95, -0.11, -1.54 Temp.= 40.0, 36.0, 66.0; Rot.= 0, 0, 0 Vcore = 1.44, 3.12; Volt. = 3.34, 5.00, 1.95, -0.11, -1.54 Temp.= 40.0, 36.0, 66.0; Rot.= 0, 0, 0 Vcore = 1.44, 3.12; Volt. = 3.34, 5.00, 1.95, -0.11, -1.54This also seems to have fixed the rl0 watchdog timeout problems. I no longer see those in my logs.
SATA DRIVES: I'm still having problems with the SATA drives.I tried connecting the 1TB Samsung drives to my mainboard, but then the box hangs when booting with the "Detecting IDE drives" message. The regular (PATA) IDE drives are detected first, and then it repeats the "Detecting IDE drives" message to detect the sata drives, and hangs. When I connect my 250GB SATA drives to my mainboard they detect fine, and the box boots normally.
I did another rsync of my old mirror (the 250GB disks) to the new mirror (1TB disks), but again one of the disks got detached. This time there are no other messages in the log, the only thing I see is the following:
Aug 13 14:35:27 piglet su: sebster to root on /dev/ttyp5 Aug 13 14:55:38 piglet kernel: ad6: FAILURE - device detached Aug 13 14:55:38 piglet kernel: subdisk6: detached Aug 13 14:55:38 piglet kernel: ad6: detachedAug 13 14:55:38 piglet kernel: GEOM_MIRROR: Device gm1: provider ad6 disconnected.
Aug 13 15:00:00 piglet newsyslog[1800]: logfile turned over due to size>100K(unfortunate that the log file just got rotated, but in the new log file there is nothing execpt the one expected line:
Aug 13 15:00:00 piglet newsyslog[1800]: logfile turned over due to size>100K So, nothing after the disconnect... The questions I have now is:1) Could an upgrade to FreeBSD 7-STABLE fix the issue (it's a LOT of work for me, but I'll do it if there are SATA driver issues fixed). 2) What is the next step? Should I repeat the tests to see if it's always the same drive that disconnects?
3) Is there any way to get more info about what is causing the disconnect? Regards, Sebastiaan Jeremy Chadwick wrote:
On Wed, Aug 06, 2008 at 02:57:48AM -0700, Jeremy Chadwick wrote:vmstat -i output should help clear that up, or dmesg output.Sebastiaan has included vmstat -i output in another part of this thread, as well as dmesg output for the ATA disks and controllers: atapci0: <SiI SiI 3512 SATA150 controller> port 0xd200-0xd207,0xd300-0xd303,0xd400-0xd407,0xd500-0xd503,0xd600-0xd60f mem 0xf6081000-0xf60811ff irq 18 at device 10.0 on pci0 ata2: <ATA channel 0> on atapci0 ata3: <ATA channel 1> on atapci0 atapci1: <VIA 6420 SATA150 controller> port 0xd700-0xd707,0xd800-0xd803,0xd900-0xd907,0xda00-0xda03,0xdb00-0xdb0f,0xdc00-0xdcff irq 20 at device 15.0 on pci0 ata4: <ATA channel 0> on atapci1 ata5: <ATA channel 1> on atapci1 atapci2: <VIA 8237 UDMA133 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xdd00-0xdd0f at device 15.1 on pci0 ata0: <ATA channel 0> on atapci2 ata1: <ATA channel 1> on atapci2 ad0: 286188MB <Maxtor 6L300R0 BAH41G10> at ata0-master UDMA133 ad1: 239372MB <Maxtor 6L250R0 BAH41G10> at ata0-slave UDMA133 acd0: DVDR <LITE-ON DVDRW SHW-1635S/YS0N> at ata1-master UDMA33 ad4: 953869MB <SAMSUNG HD103UJ 1AA01112> at ata2-master SATA150 ad6: 953869MB <SAMSUNG HD103UJ 1AA01112> at ata3-master SATA150 ad8: 239372MB <Maxtor 6L250S0 BANC1G10> at ata4-master SATA150 ad10: 239372MB <Maxtor 6L250S0 BANC1G10> at ata5-master SATA150 interrupt total rate irq6: fdc0 10 0 irq14: ata0 645057 7 irq15: ata1 58 0 irq16: rl0 7168276 82 irq17: rl1 914667 10 irq18: atapci0 30072876 347 irq20: atapci1 1126099 12 irq21: uhci0 uhci* 308 0 irq23: vr0 3265771 37 cpu0: timer 173289011 1999 Total 216482133 2498 Here's a breakdown, so no one gets confused: ad0 = 300GB Maxtor disk, attached to on-board VIA IDE controller ad1 = 250GB Maxtor disk, attached to on-board VIA IDE controller ad4 = 1TB Samsung disk, attached to Silicon Image SATA controller ad6 = 1TB Samsung disk, attached to Silicon Image SATA controller ad8 = 250GB Maxtor disk, attached to on-board VIA SATA controller ad10 = 250GB Maxtor disk, attached to on-board VIA SATA controller IRQ 14 -- atapci2 = On-board VIA IDE controller (primary) IRQ 15 -- atapci2 = On-board VIA IDE controller (slave) IRQ 16 -- rl0 = Realtek NIC IRQ 17 -- rl1 = Realtek NIC IRQ 18 -- atapci0 = Silicon Image SATA controller IRQ 20 -- atapci1 = On-board VIA SATA controller An APIC is obviously in use here. The problem reported is with disks ad4 and ad6, residing on the Silicon Image controller. When the problem happens, rl1 emits watchdog timeouts, and disks ad4 and ad6 fall off the bus. SMART statistics on both ad4 and ad6 show no signs of the disks being power-cycled, sector errors, or anything that would indicate either disk is bad. His PSU is 450W, brand unknown. Sebastiaan, do you know if the BIOS on this system has a Health monitor, showing voltages and temperatures of things? If so, can you reboot the system, go into that part of the BIOS, and write down (or take a photo) of the voltages/temperatures? I'm wondering if maybe one of the voltages is too low or high, or fluxuates severely with so many devices. It could be enough to cause some of the ASICs to malfunction, possibly multiples... I cleared the possibility of this being a PSU problem, but that was before I knew you were seeing watchdog timeouts on one of your NICs at the *exact same time* as disks off the Silicon Image controller.
smime.p7s
Description: S/MIME Cryptographic Signature
