Friday, October 5, 2012

Solaris 10 x86 SVM failed root disk replacement


Check the disk for errors with iostat -en, format etc.



 +$ metadb
        flags           first blk       block count
     a m  p  luo        16              8192            /dev/dsk/c3t3d0s7
     a    p  luo        8208            8192            /dev/dsk/c3t3d0s7
     a    p  luo        16400           8192            /dev/dsk/c3t3d0s7
     W    p  luo        16              8192            /dev/dsk/c3t2d0s7
     W  p  luo        8208            8192            /dev/dsk/c3t2d0s7
     W   p  luo        16400           8192            /dev/dsk/c3t2d0s7
[02:57:16]

If you are seeing your metadb with 'W', then it has write errors.

$metastat -c
d60              m  120GB d61 d62 (maint)
    d61          s  120GB c3t3d0s6
    d62          s  120GB c3t2d0s6 (maint)
d40              m  2.0GB d41 d42 (maint)
    d41          s  2.0GB c3t3d0s4
    d42          s  2.0GB c3t2d0s4 (maint)
d30              m  4.0GB d31 d32 (maint)
    d31          s  4.0GB c3t3d0s3
    d32          s  4.0GB c3t2d0s3 (maint)
d10              m  2.0GB d11 d12 (maint)
    d11          s  2.0GB c3t3d0s1
    d12          s  2.0GB c3t2d0s1 (maint)
d0               m  8.0GB d1 d2 (maint)
    d1           s  8.0GB c3t3d0s0
    d2           s  8.0GB c3t2d0s0 (maint)
[02:31:42] charihar@thunderbird:/home/charihar



Now we need to replace the disk.

Procedure for disk replacement. 



Get the disk replaced with a same capacity disk.

Delete the metadb from faulty disk.

$metadb -d  c3t2d0s7

Detach and clear faulty sub-mirrors

$metadetach -f d0 d1
$metadetach -f d10 d11
$metadetach -f d30 d31
$metadetach -f d40 d41
$metadetach -f d60 d61

$metaclear d1
$metaclear d11
$metaclear d31
$metaclear d41
$metaclear d61

Re-Boot your system with Good Disk:

If your First disk failed, and second disk is in Good condition; follow the below procedure.

1. Check you have active boot enabled on Disk2; (format; fdisk (select option 2 to make disk active)
2. Install grub if you have not installed alreday on Disk2
    $/usr/sbin/installgrub /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c3t3d0s0
4. Edit /boot/solaris/bootenv.rc file and point  bootpath to Good disk.sd@3,0,a

setprop bootpath '/pci@0,0/pci1022,7450@2/pci1000,3060@3/sd@3,0:a'
 ls -l /dev/dsk/c3t3d0s0
lrwxrwxrwx   1 root     root          62 Dec 20  2007 /dev/dsk/c3t3d0s0 -> ../../devices/pci@7b,0/pci1022,7458@11/pci1000,3060@2/sd@3,0:a

3. Reboot and press F2 - Go to BIOS and Change First boot disk as Good Disk, Disk2

4. Now you are good to boot with Second disk.



Once you are able to boot to OS with Good Disk.

Create similar partitions on New Disk.


$prtvtoc /dev/rdsk/c3t3d0s0|fmthard -s - /dev/rdsk/c3t2d0s0

Create metadb
$metadb -afc3 c3t2d0s7

Add metainit to new disk slices.

$metainit -f d1 1 1 c3t2d0s0
$metainit -f d11 1 1 c3t2d0s1
$metainit -f d31 1 1 c3t2d0s3
$metainit -f d41 1 1 c3t2d0s4
$metainit -f d61 1 1 c3t2d0s6

Now Attach New sub mirrors to Mirror.

$metattach d0 d1
$metattach d10 d11
$metattach d30 d31
$metattach d40 d41
$metattach d60 d61

Check the status with metastat -c

+$ metastat -c
d60              m  120GB d61 d62 (resync-0%)
    d61          s  120GB c3t3d0s6
    d62          s  120GB c3t2d0s6
d40              m  2.0GB d41 d42 (resync-64%)
    d41          s  2.0GB c3t3d0s4
    d42          s  2.0GB c3t2d0s4
d30              m  4.0GB d31 d32 (resync-33%)
    d31          s  4.0GB c3t3d0s3
    d32          s  4.0GB c3t2d0s3
d10              m  2.0GB d11 d12 (resync-75%)
    d11          s  2.0GB c3t3d0s1
    d12          s  2.0GB c3t2d0s1
d0               m  8.0GB d1 d2 (resync-20%)
    d1           s  8.0GB c3t3d0s0
    d2           s  8.0GB c3t2d0s0
[00:39:55] charihar:/home/charihar



$Do a final restart after completing sync.




Live command examples.



bash-3.00# metadetach -f  d10 d11
d10: submirror d11 is detached
bash-3.00# metaclear d11
d11: Concat/Stripe is cleared
bash-3.00#

 bash-3.00# metainit d10 -m d11
d10: Mirror is setup
bash-3.00# metainit -f d31 1 1 c3t3d0s3
d31: Concat/Stripe is setup
bash-3.00# metainit d30 -m d31
d30: Mirror is setup
bash-3.00# metainit d41     -f d41 1 1 c3t3d0s4
d41: Concat/Stripe is setup
bash-3.00# metainit d40 -m d41
d40: Mirror is setup
bash-3.00# metainit f   -f d61 1 1 c3t3d0s6
d61: Concat/Stripe is setup
bash-3.00# metainit d60 d -m d61
d60: Mirror is setup
bash-3.00#

bash-3.00# metattach d0 d2     d10 d12
d10: submirror d12 is attached
bash-3.00# metattach d10 d1230 d1232
d30: submirror d32 is attached
bash-3.00# metattach d30 d3240 d3242
d40: submirror d42 is attached
bash-3.00# metattach d40 d4260 d4262
d60: submirror d62 is attached
bash-3.00#