This is one of the very common problem which a lot of system/linux administrators face on regular basis and this blog is my effort to save precious time of those peoples.
Take an example that you have a raid five array with four disks sda, sdb, sdc, sdd.
This will be what mdstat will going to show:

# cat /proc/mdstat

Personalities : [raid5]
md0 : inactive raid5 sdd1[3] sdb1[1] sdc1[2] sda1[0]
95425536 blocks level 5, 128k chunk, algorithm 2 [4/4] [UUUU]
Imaging a situation that one of your disk goes bad due to some reasons (like power failure, improper shutdown) . This will be what mdstat going to show you after failure.

# cat /proc/mdstat

Personalities : [raid5]
md0 : inactive raid5 sdd1[3] sdc1[2] sda1[0]
95425536 blocks level 5, 128k chunk, algorithm 2 [4/4] [UUUU]
This is the state of a inactive raid 5 array who earlier have four disks sda1, sdb1, sdc1, sdd1 but now have one disk bad (sdb1)
So to fix the issue follow the following steps:
  1. Replace the bad hard drive.
  2. Create a fresh partition in the hard drive and
    3. [root@napster ~]# fdisk /dev/sdc
    The number of cylinders for this disk is set to 91201.
    There is nothing wrong with that, but this is larger than 1024,
    and could in certain setups cause problems with:
    1) software that runs at boot time (e.g., old versions of LILO)
    2) booting and partitioning software from other OSs
    (e.g., DOS FDISK, OS/2 FDISK)
    Command (m for help): p
    Disk /dev/sdc: 750.1 GB, 750156374016 bytes
    255 heads, 63 sectors/track, 91201 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes
    Device Boot Start End Blocks Id System
    Command (m for help): n
    Command action
    e extended
    p primary partition (1-4)
    p
    Partition number (1-4): 1
    First cylinder (1-91201, default 1):
    Using default value 1
    Last cylinder or +size or +sizeM or +sizeK (1-91201, default 91201):
    Using default value 91201
    Command (m for help): p
    Disk /dev/sdc: 750.1 GB, 750156374016 bytes
    255 heads, 63 sectors/track, 91201 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes
    Device Boot Start End Blocks Id System
    /dev/sdc1 1 91201 732572001 83 Linux
    Command (m for help): t
    Selected partition 1
    Hex code (type L to list codes): fd
    Changed system type of partition 1 to fd (Linux raid autodetect)
    Command (m for help): wq
    Remember to assign the type as Linux raid autodetect.
  3. Stop the inactive array

    # mdadm –stop /dev/md0

  4. Now if you will try to start the array then it will give you error that “sdb had no superblock info”.
    # mdadm –assemble /dev/md0
    mdadm: looking for devices for /dev/md0
    mdadm: no RAID superblock on /dev/sdb1
    5. This is because this is new drive and don’t have sufficient info for raid array. Now u can do two things to solve this problem.
  • Re-create the array – This sound counter-intuitive: if we recreate a raid array over an existing one, it will be erased …… Right ?? ….. No, it’s wrong. mdadm is smart enough to “see” that HDD of the new array were elements of a previous one. Knowing that, mdadm will try to do its best (i.e. if parameters match the previous array configuration) and rebuild the new array upon the previous one in a non-destructive way, by keeping HDD content.
    # mdadm –create /dev/md0 –level=5 –raid-devices=4 /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1
    mdadm: layout defaults to left-symmetric
    mdadm: chunk size defaults to 64K
    mdadm: size set to 312568576K
    mdadm: array /dev/md0 started.
    But this looks risky. Although logic says that it will not delete the array but still i was scared and don’t opt for this method. Rather i go for another one.
  • Assemble the array with less disks – This looks much safer to me so i opted this. By this method i will start the array with 3 disks and then will add the fourth disk in the running raid array, which will start the recovery process and will do the needful for me.
    # mdadm -A /dev/md0 -f –update=summaries /dev/sda1 /dev/sdc1 /dev/sdd1
    mdadm: array /dev/md0 started.
    This will start the raid array with three disks.After that i add new disk to the array

    # mdadm /dev/md0 -a /dev/sdb1

    This will add the disk to the array, will write the superblock info and will start the recovery process. U can see the status of the recovery process in the /proc/mdstat file. Try this command to see the runtime status of recovery.

    # watch -n 1 cat /proc/mdstat

Personalities : [raid5]md0 : active raid5 sdd1[3] sdb1[1] sdc1[2] sda1[0]
127872 blocks [2/1] [U_]
[======>…………..] recovery = 34.4% (44608/127872) finish=2.9min speed=461K/sec
If you enjoyed this post, make sure you subscribe to my RSS feed!!!!

No related content found.