Re: [zfs-discuss] Help identify failed drive

Mon, 19 Jul 2010 10:09:35 -0700 

A few things:
1.) did you move your drives around or change which controller each one was connected to sometime after installing and setting up OpenSolaris? If so, a pool export and re-import may be in order. 


2.) are you sure the drive is failing?  Does the problem only affect 
this drive or are other drives randomly affect too?  If you've run 
'zpool clear' and the problem comes back, something is wrong but it 
could also be RAM, CPU, motherboard, controller, or power supply 
problems.  Smartmontools can read the drive SMART data and device error 
logs...run it from an Ubuntu 10.04 Live CD (sudo apt-get install 
smartmontools) or from a PartedMagic Live CD if you have trouble getting 
Smartmontools working on OpenSolaris with your hardware.



3.) on some systems I've found another version of the iostat command to 
be more useful, particularly when iostat -En leaves the serial number 
field empty or otherwise doesn't read the serial number correctly.  Try 
this:


iostat -Eni


This should give you a list of drives showing their name in the cXtYdZsN 
format, and their Device ID which may contain the drive serial numbers 
concatenated with the model.  Compare that list with your 'zpool status 
tank' output, which in your case means looking for 'c2t3d0'.  Once you 
find the serial number, you can look at labels printed on your drives 
and verify which one it is.



One tip: if your server case is hard to work in or it is otherwise 
difficult to remove drives to read the serial numbers (lots of screws, 
cables in the way, tight fits, etc.), create additional serial number 
labels for the drives and stick them on the drive in a place you can 
read them without removing the drive from the drive bay.  This will make 
it easier to find a particular drive next time you need to replace or 
upgrade one.  This problem most significant on hardware/OS combinations 
that don't provide a way to signal where a particular drive is 
physically installed.  (This includes a lot of whitebox and small server 
hardware and OSes.)




iostat relevant man page entries:


http://docs.sun.com/app/docs/doc/816-5166/iostat-1m?l=en&n=1&a=view 
<http://docs.sun.com/app/docs/doc/816-5166/iostat-1m?l=en&n=1&a=view>


-E
Display all device error statistics.

-i

In -E output, display the Device ID instead of the Serial No. The Device 
Id is a unique identifier registered by a driver through 
ddi_devid_register(9F).


-n

Display names in descriptive format. For example, cXtYdZ, rmt/N, 
server:/export/path.



By default, disks are identified by instance names such as ssd23 or 
md301. Combining the -n option with the -x option causes disk names to 
display in the cXtYdZsN format which is more easily associated with 
physical hardware characteristics. The cXtYdZsN format is particularly 
useful in FibreChannel (FC) environments where the FC World Wide Name 
appears in the t field.







Cindy Swearingen wrote:

Hi--

A google search of ST3500320AS turns up Seagate Barracuda drives.

All 7 drives in the pool tank are ST3500320AS. The other two c1t0d0
and c3d0 are unknown, but are not part of this pool.

You can also use fmdump -eV to see how long c2t3d0 has had problems.

Thanks,

Cindy

On 07/19/10 09:29, Yuri Homchuk wrote:

Thanks Cindy,

But format shows exactly same thing:
All of them appear as Seagate, no WD at all...
How could it be ???

# format
Searching for disks...done


AVAILABLE DISK SELECTIONS:
       0. c1t0d0 <DEFAULT cyl 60798 alt 2 hd 255 sec 63>
          /p...@0,0/pci15d9,a...@5/d...@0,0
       1. c1t1d0 <ATA-ST3500320AS-SD15-465.76GB>
          /p...@0,0/pci15d9,a...@5/d...@1,0
       2. c2t0d0 <ATA-ST3500320AS-SD15-465.76GB>
          /p...@0,0/pci10de,3...@a/pci15d9,a...@0/s...@0,0
       3. c2t2d0 <ATA-ST3500320AS-SD15-465.76GB>
          /p...@0,0/pci10de,3...@a/pci15d9,a...@0/s...@2,0
       4. c2t3d0 <ATA-ST3500320AS-SD15-465.76GB>
          /p...@0,0/pci10de,3...@a/pci15d9,a...@0/s...@3,0
       5. c2t4d0 <ATA-ST3500320AS-SD15-465.76GB>
          /p...@0,0/pci10de,3...@a/pci15d9,a...@0/s...@4,0
       6. c2t5d0 <ATA-ST3500320AS-SD15-465.76GB>
          /p...@0,0/pci10de,3...@a/pci15d9,a...@0/s...@5,0
       7. c2t7d0 <ATA-ST3500320AS-SD15-465.76GB>
          /p...@0,0/pci10de,3...@a/pci15d9,a...@0/s...@7,0
       8. c3d0 <GIGABYTE-100336D9CC244B01-0001-2.00GB>
          /p...@1,0/pci1022,7...@2/pci-...@1/i...@1/c...@0,0
Specify disk (enter its number): ^C


Thanks again.


-----Original Message-----

From: Cindy Swearingen [mailto:cindy.swearin...@oracle.com] Sent: 
Monday, July 19, 2010 9:16 AM

To: Yuri Homchuk
Cc: zfs-discuss@opensolaris.org
Subject: Re: [zfs-discuss] Help identify failed drive

Hi--


I don't know what's up with iostat -En but I think I remember a 
problem where iostat does not correctly report drives running in 
legacy IDE mode.


You might use the format utility to identify these devices.

Thanks,

Cindy
On 07/18/10 14:15, Alxen4 wrote:

This is a situation:

I've got an error on one of the drives in 'zpool status' output:

 zpool status tank

  pool: tank
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An

        attempt was made to correct the error.  Applications are 
unaffected.
action: Determine if the device needs to be replaced, and clear the 
errors

        using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://www.sun.com/msg/ZFS-8000-9P
 scrub: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        tank        ONLINE       0     0     0
          raidz2    ONLINE       0     0     0
            c1t1d0  ONLINE       0     0     0
            c2t0d0  ONLINE       0     0     0
            c2t2d0  ONLINE       0     0     0
            c2t3d0  ONLINE       1     0     0
            c2t4d0  ONLINE       0     0     0
            c2t5d0  ONLINE       0     0     0
            c2t7d0  ONLINE       0     0     0

So I would like to replace 'c2t3d0'.


I know for a fact the pool has 7 physical drives : 5 of Seagate and 
2 of WD.


I want to know if 'c2t3d0' Seagate or WD.


If I run 'iostat -En' it shows that all  c*t*d0 drives are Seagate 
and sd11/sd12 are WD.


This totally confuses me...

Why there are two different types of drives in iostat output : 
c*t*d0 and sd* ???
How come all c*t*d0 appear as Seagate.I know for sure two of them 
are WD.

Why WD drives appears as sd* and not as c*t*d0 ?

Please help.


----------------------------------------------------------

# iostat -En


c1t1d0           Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA      Product: ST3500320AS      Revision: SD15 Serial No:
Size: 500.11GB <500107862016 bytes>

Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0 
Illegal Request: 54 Predictive Failure Analysis: 0


c2t0d0           Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA      Product: ST3500320AS      Revision: SD15 Serial No:
Size: 500.11GB <500107862016 bytes>

Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0 
Illegal Request: 0 Predictive Failure Analysis: 0


c2t1d0           Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA      Product: ST3500320AS      Revision: SD15 Serial No:
Size: 500.11GB <500107862016 bytes>

Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0 
Illegal Request: 0 Predictive Failure Analysis: 0


c2t2d0           Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA      Product: ST3500320AS      Revision: SD15 Serial No:
Size: 500.11GB <500107862016 bytes>

Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0 
Illegal Request: 0 Predictive Failure Analysis: 0


c2t3d0           Soft Errors: 0 Hard Errors: 9 Transport Errors: 9
Vendor: ATA      Product: ST3500320AS      Revision: SD15 Serial No:
Size: 500.11GB <500107862016 bytes>

Media Error: 7 Device Not Ready: 0 No Device: 2 Recoverable: 0 
Illegal Request: 0 Predictive Failure Analysis: 0


c2t4d0           Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA      Product: ST3500320AS      Revision: SD15 Serial No:
Size: 500.11GB <500107862016 bytes>

Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0 
Illegal Request: 0 Predictive Failure Analysis: 0


c2t5d0           Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA      Product: ST3500320AS      Revision: SD15 Serial No:
Size: 500.11GB <500107862016 bytes>

Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0 
Illegal Request: 0 Predictive Failure Analysis: 0


c2t6d0           Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA      Product: ST3500320AS      Revision: SD15 Serial No:
Size: 500.11GB <500107862016 bytes>

Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0 
Illegal Request: 0 Predictive Failure Analysis: 0


c2t7d0           Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA      Product: ST3500320AS      Revision: SD15 Serial No:
Size: 500.11GB <500107862016 bytes>

Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0 
Illegal Request: 0 Predictive Failure Analysis: 0


[b]sd11             Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA      Product: WDC WD5001AALS-0 Revision: 1D05 Serial No:
Size: 500.11GB <500107862016 bytes>

Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0 
Illegal Request: 0 Predictive Failure Analysis: 0


sd12             Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA      Product: WDC WD5001AALS-0 Revision: 0K05 Serial No:
Size: 500.11GB <500107862016 bytes>

Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0 
Illegal Request: 0 Predictive Failure Analysis: 0[/b]






Thanks a lot.

_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss



_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss






















					Reply via email to