just FYI this is from intel http://www.intel.com/support/motherboards/server/sb/CS-031831.htm
Another observation: Oracle/Sun has move away from SATA to SAS in ZFS storage/Appliance If you want to go deeper take look these presentations http://www.scsita.org/sas_library/tutorials/ and other presentations on the site regards On 5/31/2012 12:45 PM, "Antonio S. Cofiño" wrote:
Markus, After Jim's answer I have started to read bout the well known issue.Is it just mpt causing the errors or also mpt_sas?Both drivers are causing the reset storm (See my answer to Jim's e-mail).General consensus from various people: don't use SATA drives on SAS back-planes. Some SATA drives might work better, but there seems to be no guarantee. And even for SAS-SAS, try to avoid SAS1 backplanes.In the Paul Kraus's answer it mentions that Oracle support says (among other things)4. the problem happens with SAS as well as SATA drives, but is much less frequentThat means, that using SAS drives it will reduce the probability of the issue but no guarantee exists.General consensus from various people: don't use SATA drives on SAS back-planes. Some SATA drives might work better, but there seems to be no guarantee. And even for SAS-SAS, try to avoid SAS1 backplanes.Yes, may be the 'general consensus' is right but 'general consensus' said me to use hardware based raid solutions. But I started to do 'risky business' (as some vendors told me) using ZFS and have ended discovering how robust is ZFS for this kind of protocol errors.From my complete naive point of view it appears more a issue with the HBA's FW than a issue with SATA drives.With you answers I have make a lot of re-search helping me to learn new things.Please more comments and help are welcome (from some SAS expert?). Antonio -- Antonio S. Cofiño El 31/05/2012 18:04, Weber, Markus escribió:Antonio S. Cofiño wrote:[...] The system is a supermicro motherboard X8DTH-6F in a 4U chassis (SC847E1-R1400LPB) and an external SAS2 JBOD (SC847E16-RJBOD1). It makes a system with a total of 4 backplanes (2x SAS + 2x SAS2) each of them connected to a 4 different HBA (2x LSI 3081E-R (1068 chip) + 2x LSI SAS9200-8e (2008 chip)). This system is has a total of 81 disk (2x SAS (SEAGATE ST3146356SS)+ 34 SATA3 (Hitachi HDS722020ALA330) + 45 SATA6 (Hitachi HDS723020BLA642))The issue arise when one of the disk starts to fail making long time accesses. After some time (minutes, but I'm not sure) all the disks, connected to the same HBA, start to report errors. This situation produce a general failure on the ZFS making the whole POOL unavailable. [...]Have been there and gave up at the end[1]. Could reproduce (even though it took a bit longer) under most Linux versions (incl. using latest LSI drivers) and LSI 3081E-R HBA. Is it just mpt causing the errors or also mpt_sas? In a lab environment the LSI 9200 HBA behaved better - I/O only dropped shortly and then continued on the other disks without generating errors. Had a lengthy Oracle case on this, but all proposed "workarounds" did not worked for me at all, which had been (some also from other forums) - disabling NCQ - allow-bus-device-reset=0; to /kernel/drv/sd.conf - set zfs:zfs_vdev_max_pending=1 - set mpt:mpt_enable_msi=0 - keep usage below 90%- no fmservices running and did temporarily did fmadm unload disk-transportor other disk access stuff (smartd?) - tried changing retries-timeout via sd-conf for the disks without any success and ended it doing via mdb At the end I knew the bad sector of the "bad" disk and by simply dd this sector once or twice to /dev/zero I could easily bring down the system/pool without any load on the disk system.General consensus from various people: don't use SATA drives on SAS back-planes. Some SATA drives might work better, but there seems to be no guarantee. And even for SAS-SAS, try to avoid SAS1 backplanes. Markus[1] Search for "What's wrong with LSI 3081 (1068) + expander + (bad) SATAdisk?"_______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
--
<<attachment: laotsao.vcf>>
_______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
