Solution 203375 : Unable to recover a metadevice in a "Needs maintenance"

state.

Symptoms:

There may be times when trying a metareplace or metasync fails and the metadevice
is still left in a "Needs Maintenance" state. One of the disks maybe experiencing
a problem on a sector.

In this example, the mirror d10 is in a "Need Maintenance" state for both
submirrors:

# metastat d10
d10: Mirror
Submirror 0: d0
State: Needs maintenance
Submirror 1: d1
State: Needs maintenance
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 69078879 blocks

d0: Submirror of d10

State: Needs maintenance
Invoke: after replacing "Maintenance" components:
metareplace d10 c1t0d0s0 <new device>
Size: 69078879 blocks
Stripe 0:
Device Start Block Dbase State Hot Spare
c1t0d0s0 0 No Last Erred

d1: Submirror of d10

State: Needs maintenance
Invoke: metasync d10
Size: 69078879 blocks
Stripe 0:
Device Start Block Dbase State Hot Spare
c1t1d0s0 0 No Okay

When trying to run the metasync command, the c1t0d0s0 device reported errors in
/var/adm/messages:

Sep 15 09:11:17 bobbob scsi: WARNING:

/pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w2100002037f396c9,0 (ssd1):
Sep 15 09:11:17 bobbob Error for Command: read(10) Error Level: Retryable
Sep 15 09:11:17 bobbob scsi: Requested Block: 4057844 Error Block: 4057969
Sep 15 09:11:17 bobbob scsi: Vendor: SEAGATE Serial Number: 0107D1MVCF
Sep 15 09:11:17 bobbob scsi: Sense Key: Media Error
Sep 15 09:11:17 bobbob scsi: ASC: 0x11 (unrecovered read error), ASCQ: 0x0,
FRU: 0xe4
Sep 15 09:11:19 bobbob scsi: WARNING:
/pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w2100002037f396c9,0 (ssd1):
Sep 15 09:11:19 bobbob Error for Command: read(10) Error Level: Retryable
Sep 15 09:11:19 bobbob scsi: Requested Block: 4057844 Error Block: 4057969
Sep 15 09:11:19 bobbob scsi: Vendor: SEAGATE Serial Number: 0107D1MVCF
Sep 15 09:11:19 bobbob scsi: Sense Key: Media Error
Sep 15 09:11:19 bobbob scsi: ASC: 0x11 (unrecovered read error), ASCQ: 0x0,
FRU: 0xe4

In this case, the same block is being reported as having problems.

Resolution

Resolution and Determination steps:

The bad block can be fixed by running format --> analyze --> read on the c1t0d0
disk.

# format
Searching for disks...done

AVAILABLE DISK SELECTIONS:

0. c1t0d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107>
/pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w2100002037f396c9,0
1. c1t1d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107>
/pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w2100002037f8c663,0
Specify disk (enter its number): 0
selecting c1t0d0
format> analyze
analyze> read
Ready to analyze (won't harm SunOS). This takes a long time,
but is interruptable with CTRL-C. Continue? y

pass 0
Medium error during read: block 4057969 (0x3deb71) (1404/16/101)
ASC: 0x11 ASCQ: 0x0
Sep 15 09:26:59 bobbob scsi: WARNING:
/pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w2100002037f396c9,0 (ssd1):
Sep 15 09:26:59 bobbob Error for Command: read(10) Error Level: Retryable
Sep 15 09:26:59 bobbob scsi: Requested Block: 4057969 Error Block: 4057969
Sep 15 09:26:59 bobbob scsi: Vendor: SEAGATE Serial Number: 0107D1MVCF
Sep 15 09:26:59 bobbob scsi: Sense Key: Media Error
Sep 15 09:26:59 bobbob scsi: ASC: 0x11 (unrecovered read error), ASCQ: 0x0,
FRU: 0xe4
Repairing hard error on 4057969 (1404/16/101)...ok.

24619/26/53

pass 1
24619/26/53

Total of 1 defective blocks repaired.

Now running metasync completes.

# metasync d10
# metastat d10
d10: Mirror
Submirror 0: d0
State: Needs maintenance
Submirror 1: d1
State: Okay
Pass: 1
 Read option: roundrobin (default)
Write option: parallel (default)
Size: 69078879 blocks

d0: Submirror of d10

State: Needs maintenance
Invoke: after replacing "Maintenance" components:
metareplace d10 c1t0d0s0 <new device>
Size: 69078879 blocks
Stripe 0:
Device Start Block Dbase State Hot Spare
c1t0d0s0 0 No Last Erred

d1: Submirror of d10

State: Okay
Size: 69078879 blocks
Stripe 0:
Device Start Block Dbase State Hot Spare
c1t1d0s0 0 No Okay

And then, metareplace can be executed.

# metareplace -e d10 c1t0d0s0

# metastat d10
d10: Mirror
Submirror 0: d0
State: Okay
Submirror 1: d1
State: Okay
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 69078879 blocks

d0: Submirror of d10

State: Okay
Size: 69078879 blocks
Stripe 0:
Device Start Block Dbase State Hot Spare
c1t0d0s0 0 No Okay

d1: Submirror of d10

State: Okay
Size: 69078879 blocks
Stripe 0:
Device Start Block Dbase State Hot Spare
c1t1d0s0 0 No Okay

Product

Solstice DiskSuite 4.0

Solstice DiskSuite 4.0 (Japanese)
Solstice DiskSuite 4.1 (Japanese)
Solstice DiskSuite 4.1
Solstice DiskSuite 4.2
Solstice DiskSuite 4.2.1