After another crash I checked the syslog and there were some different errors than the ones I saw previously during operation:
Nov 30 20:26:11 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 20:26:11 the-vault Disconnected command timeout for Target 10 Nov 30 20:59:12 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 20:59:12 the-vault mpt_send_handshake_msg task 3 failed Nov 30 20:59:13 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 20:59:13 the-vault LSI PCI device (1000,ffff) not supported. Nov 30 20:59:13 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 20:59:13 the-vault mpt_config_space_init failed Nov 30 20:59:15 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 20:59:15 the-vault LSI PCI device (1000,ffff) not supported. Nov 30 20:59:15 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 20:59:15 the-vault mpt_config_space_init failed Nov 30 20:59:15 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 20:59:15 the-vault mpt_restart_ioc failed Nov 30 21:32:17 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:17 the-vault mpt_send_handshake_msg task 4 failed Nov 30 21:32:18 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:18 the-vault LSI PCI device (1000,ffff) not supported. Nov 30 21:32:18 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:18 the-vault mpt_config_space_init failed Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault LSI PCI device (1000,ffff) not supported. Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault mpt_config_space_init failed Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault mpt_restart_ioc failed Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault Rejecting future commands Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): Nov 30 21:32:19 the-vault Disconnected command timeout for Target 14 Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked Nov 30 21:32:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:19 the-vault rejecting command, throttle choked (this keeps flooding like that for a _long_ time, skipping most of it) Nov 30 21:32:21 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:21 the-vault rejecting command, throttle choked Nov 30 21:32:21 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:21 the-vault rejecting command, throttle choked Nov 30 21:32:21 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): Nov 30 21:32:21 the-vault rejecting command, throttle choked Nov 30 21:33:02 the-vault fmd: [ID 377184 daemon.error] SUNW-MSG-ID: PCIEX-8000-8R, TYPE: Fault, VER: 1, SEVERITY: Major Nov 30 21:33:02 the-vault EVENT-TIME: Mon Nov 30 21:33:02 PST 2009 Nov 30 21:33:02 the-vault PLATFORM: System-Product-Name, CSN: System-Serial-Number, HOSTNAME: the-vault Nov 30 21:33:02 the-vault SOURCE: eft, REV: 1.16 Nov 30 21:33:02 the-vault EVENT-ID: 7886cc0d-4760-60b2-e06a-8158c3334f63 Nov 30 21:33:02 the-vault DESC: The transmitting device sent an invalid request. Nov 30 21:33:02 the-vault Refer to http://sun.com/msg/PCIEX-8000-8R for more information. Nov 30 21:33:02 the-vault AUTO-RESPONSE: One or more device instances may be disabled Nov 30 21:33:02 the-vault IMPACT: Loss of services provided by the device instances associated with this fault Nov 30 21:33:02 the-vault REC-ACTION: Ensure that the latest drivers and patches are installed. Otherwise schedule a repair procedure to replace the affected device(s). Us e fmadm faulty to identify the devices or contact Sun for support. Nov 30 21:33:10 the-vault fmd: [ID 377184 daemon.error] SUNW-MSG-ID: ZFS-8000-FD, TYPE: Fault, VER: 1, SEVERITY: Major Nov 30 21:33:10 the-vault EVENT-TIME: Mon Nov 30 21:33:09 PST 2009 Nov 30 21:33:10 the-vault PLATFORM: System-Product-Name, CSN: System-Serial-Number, HOSTNAME: the-vault Nov 30 21:33:10 the-vault SOURCE: zfs-diagnosis, REV: 1.0 Nov 30 21:33:10 the-vault EVENT-ID: af2c0212-2915-4eee-e9b3-e2a8fe70efba Nov 30 21:33:10 the-vault DESC: The number of I/O errors associated with a ZFS device exceeded Nov 30 21:33:10 the-vault acceptable levels. Refer to http://sun.com/msg/ZFS-8000-FD for more information. Nov 30 21:33:10 the-vault AUTO-RESPONSE: The device has been offlined and marked as faulted. An attempt Nov 30 21:33:10 the-vault will be made to activate a hot spare if available. Nov 30 21:33:10 the-vault IMPACT: Fault tolerance of the pool may be compromised. Nov 30 21:33:10 the-vault REC-ACTION: Run 'zpool status -x' and replace the bad device. Nov 30 21:33:10 the-vault genunix: [ID 846333 kern.warning] WARNING: constraints forbid retire: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 Nov 30 22:05:18 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): Nov 30 22:05:18 the-vault mpt_send_handshake_msg task 3 failed Nov 30 22:05:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): Nov 30 22:05:19 the-vault LSI PCI device (1000,ffff) not supported. Nov 30 22:05:19 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): Nov 30 22:05:19 the-vault mpt_config_space_init failed Nov 30 22:05:20 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): Nov 30 22:05:20 the-vault LSI PCI device (1000,ffff) not supported. Nov 30 22:05:20 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): Nov 30 22:05:20 the-vault mpt_config_space_init failed Nov 30 22:05:20 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): Nov 30 22:05:20 the-vault mpt_restart_ioc failed Nov 30 22:38:20 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): Nov 30 22:38:20 the-vault mpt_send_handshake_msg task 4 failed Nov 30 22:38:21 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): Nov 30 22:38:21 the-vault LSI PCI device (1000,ffff) not supported. Nov 30 22:38:21 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): Nov 30 22:38:21 the-vault mpt_config_space_init failed Nov 30 22:38:22 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): Nov 30 22:38:22 the-vault LSI PCI device (1000,ffff) not supported. Nov 30 22:38:22 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): Nov 30 22:38:22 the-vault mpt_config_space_init failed Nov 30 22:38:46 the-vault sshd[636]: [ID 800047 auth.crit] monitor fatal: protocol error during kex, no DH_GEX_REQUEST: 254 Nov 30 22:38:46 the-vault sshd[637]: [ID 800047 auth.crit] fatal: Protocol error in privilege separation; expected packet type 254, got 20 Nov 30 23:11:23 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): Nov 30 23:11:23 the-vault mpt_send_handshake_msg task 3 failed Nov 30 23:11:23 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): Nov 30 23:11:23 the-vault LSI PCI device (1000,ffff) not supported. Nov 30 23:11:23 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): Nov 30 23:11:23 the-vault mpt_config_space_init failed Nov 30 23:11:25 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): Nov 30 23:11:25 the-vault LSI PCI device (1000,ffff) not supported. Nov 30 23:11:25 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): Nov 30 23:11:25 the-vault mpt_config_space_init failed Nov 30 23:11:25 the-vault scsi: [ID 107833 kern.warning] WARNING: /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): Nov 30 23:11:25 the-vault mpt_restart_ioc failed (and that's the last message before I hit the reset button. Host was unpingable, and just moving the mouse around on the screen was extremely delayed) Nov 30 23:32:05 the-vault genunix: [ID 540533 kern.notice] ^MSunOS Release 5.11 Version snv_127 64-bit Nov 30 23:32:05 the-vault genunix: [ID 943908 kern.notice] Copyright 1983-2009 Sun Microsystems, Inc. All rights reserved. Also, it says it resilvered some data; this is the first time I've seen any notes next to a devices. Still no zpool errors though. # zpool status vault pool: vault state: ONLINE scrub: resilver completed after 0h0m with 0 errors on Mon Nov 30 23:33:16 2009 config: NAME STATE READ WRITE CKSUM vault ONLINE 0 0 0 raidz3-0 ONLINE 0 0 0 c1t6d0 ONLINE 0 0 0 c1t7d0 ONLINE 0 0 0 c1t8d0 ONLINE 0 0 0 c1t9d0 ONLINE 0 0 0 c1t11d0 ONLINE 0 0 0 c1t12d0 ONLINE 0 0 0 c1t13d0 ONLINE 0 0 0 c1t14d0 ONLINE 0 0 0 c2t3d0 ONLINE 0 0 0 c2t4d0 ONLINE 0 0 0 c2t5d0 ONLINE 0 0 0 11.5K resilvered c2t6d0 ONLINE 0 0 0 c2t7d0 ONLINE 0 0 0 c2t8d0 ONLINE 0 0 0 c2t9d0 ONLINE 0 0 0 c2t10d0 ONLINE 0 0 0 errors: No known data errors # On Mon, Nov 30, 2009 at 06:46:13PM -0800, Chad Cantwell wrote: > Hi, > > Sorry for not replying to one of the already open threads on this topic; > I've just joined the list for the purposes of this discussion and have > nothing in my client to reply to yet. > > I have an x86_64 opensolaris machine running on a Core 2 Quad Q9650 > platform with two LSI SAS3081E-R PCI-E 8 port SAS controllers, with > 8 drives each. The LSI cards are flashed with IT firmware from Feb 2009 > (I think, I can double check if it's important). The drives are Samsung > HD154UI 1.5TB disks. I was using for quite awhile OpenSolaris 2009.06 > with the opensolaris-provided mpt driver to operate a zfs raidz2 pool of > about ~20T and this worked perfectly fine (no issues or device errors > logged for several months, no hanging). A few days ago I decided to > reinstall with the latest OpenSolaris in order to take advantage of > raidz3. I hadn't known at the time about the current mpt issues, or I > may have held off on upgrading. I installed Solaris Nevada build 127 > from the DVD image. I then proceed to setup a raidz3 pool with the > same disks as before, of a slightly smaller size (obviously) than the > former raidz2 pool. I started a moderately long-running and heavy > load rsync to copy my data back to the pool from another host. Several > times during the day (sometimes a couple times an hour, or it could go up > to a few hours with no errors), I get several syslog errors and warnings > about mpt, similiar but not identical to what I've seen reported here by > others. Also, iostat -en shows several hw and trn errors of varying > amounts for all the drives (in OpenSolaris 2009.06 I never had any iostat > errors). After awhile the machine will hang in a variety of ways. The > first time it was pingable, and I could authenticate through ssh but it > would never spawn a shell. The second time it crashed it was unpingable > from the network, and the display was black, although the numlock key was > still toggling properly the numlock light on the console. Here's a > sample of my errors. I've included the complete series of errors from > one timestampe, and a few lines from a subsequent series of errors a > couple minutes later: > > (if there's any other info I can provide or more things to test just let me > know. Thanks, --Chad ) > > Nov 29 04:42:55 the-vault scsi: [ID 243001 kern.warning] WARNING: > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): > Nov 29 04:42:55 the-vault mpt_handle_event_sync: IOCStatus=0x8000, > IOCLogInfo=0x31120200 > Nov 29 04:42:55 the-vault scsi: [ID 243001 kern.warning] WARNING: > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): > Nov 29 04:42:55 the-vault mpt_handle_event: IOCStatus=0x8000, > IOCLogInfo=0x31120200 > Nov 29 04:42:55 the-vault scsi: [ID 243001 kern.warning] WARNING: > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): > Nov 29 04:42:55 the-vault mpt_handle_event_sync: IOCStatus=0x8000, > IOCLogInfo=0x31120200 > Nov 29 04:42:55 the-vault scsi: [ID 243001 kern.warning] WARNING: > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): > Nov 29 04:42:55 the-vault mpt_handle_event: IOCStatus=0x8000, > IOCLogInfo=0x31120200 > Nov 29 04:42:55 the-vault scsi: [ID 243001 kern.warning] WARNING: > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): > Nov 29 04:42:55 the-vault mpt_handle_event_sync: IOCStatus=0x8000, > IOCLogInfo=0x31120200 > Nov 29 04:42:55 the-vault scsi: [ID 243001 kern.warning] WARNING: > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): > Nov 29 04:42:55 the-vault mpt_handle_event: IOCStatus=0x8000, > IOCLogInfo=0x31120200 > Nov 29 04:42:55 the-vault scsi: [ID 243001 kern.warning] WARNING: > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): > Nov 29 04:42:55 the-vault mpt_handle_event_sync: IOCStatus=0x8000, > IOCLogInfo=0x31120200 > Nov 29 04:42:55 the-vault scsi: [ID 243001 kern.warning] WARNING: > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): > Nov 29 04:42:55 the-vault mpt_handle_event: IOCStatus=0x8000, > IOCLogInfo=0x31120200 > Nov 29 04:42:55 the-vault scsi: [ID 243001 kern.warning] WARNING: > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): > Nov 29 04:42:55 the-vault mpt_handle_event_sync: IOCStatus=0x8000, > IOCLogInfo=0x31120200 > Nov 29 04:42:55 the-vault scsi: [ID 243001 kern.warning] WARNING: > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): > Nov 29 04:42:55 the-vault mpt_handle_event: IOCStatus=0x8000, > IOCLogInfo=0x31120200 > Nov 29 04:42:55 the-vault scsi: [ID 243001 kern.warning] WARNING: > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): > Nov 29 04:42:55 the-vault mpt_handle_event_sync: IOCStatus=0x8000, > IOCLogInfo=0x31120200 > Nov 29 04:42:55 the-vault scsi: [ID 243001 kern.warning] WARNING: > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): > Nov 29 04:42:55 the-vault mpt_handle_event: IOCStatus=0x8000, > IOCLogInfo=0x31120200 > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 7. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 7. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 7. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 7. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 7. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 6. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 6. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 6. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 6. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 9. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 9. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 9. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 9. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 9. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 9. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 9. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 9. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 9. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 9. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 9. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 9. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 7. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 7. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 7. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 7. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 8. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 8. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 8. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 8. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 8. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:42:55 the-vault scsi: [ID 365881 kern.info] > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): > Nov 29 04:42:55 the-vault Log info 0x31120200 received for target 8. > Nov 29 04:42:55 the-vault scsi_status=0x0, ioc_status=0x804b, > scsi_state=0xc > Nov 29 04:44:25 the-vault scsi: [ID 243001 kern.warning] WARNING: > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): > Nov 29 04:44:25 the-vault mpt_handle_event_sync: IOCStatus=0x8000, > IOCLogInfo=0x31120200 > Nov 29 04:44:25 the-vault scsi: [ID 243001 kern.warning] WARNING: > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): > Nov 29 04:44:25 the-vault mpt_handle_event: IOCStatus=0x8000, > IOCLogInfo=0x31120200 > Nov 29 04:44:25 the-vault scsi: [ID 243001 kern.warning] WARNING: > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): > Nov 29 04:44:25 the-vault mpt_handle_event_sync: IOCStatus=0x8000, > IOCLogInfo=0x31120200 > Nov 29 04:44:25 the-vault scsi: [ID 243001 kern.warning] WARNING: > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@1/pci1000,3...@0 (mpt1): > Nov 29 04:44:25 the-vault mpt_handle_event: IOCStatus=0x8000, > IOCLogInfo=0x31120200 > Nov 29 04:44:25 the-vault scsi: [ID 243001 kern.warning] WARNING: > /p...@0,0/pci8086,2...@3/pci111d,8...@0/pci111d,8...@0/pci1000,3...@0 (mpt0): > > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
