DokuWiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Linux Software Raid ======

Software RAID in Linux is realised using the dm-raid kernel module and can be
controlled by the ''mdadm'' utility. Information and status of known raid
arrays can be gathered by reading ''/proc/mdstat''. Array details are
available using ''mdadm -D'':
  # mdadm -D /dev/md1
  /dev/md1:
          Version : 1.2
    Creation Time : Fri Mar 16 01:13:24 2012
       Raid Level : raid1
       Array Size : 1048564 (1024.16 MiB 1073.73 MB)
    Used Dev Size : 1048564 (1024.16 MiB 1073.73 MB)
     Raid Devices : 2
    Total Devices : 2
      Persistence : Superblock is persistent
  
      Update Time : Fri Mar 16 01:13:36 2012
            State : clean, resyncing 
   Active Devices : 2
  Working Devices : 2
   Failed Devices : 0
    Spare Devices : 0
  
    Resync Status : 25% complete
  
             Name : base:1  (local to host base)
             UUID : c527d58e:bfd9cd43:ebb8a4f6:a812c75b
           Events : 4
  
      Number   Major   Minor   RaidDevice State
         0     254       10        0      active sync   /dev/dm-10
         1     254       11        1      active sync   /dev/dm-11

The following shows some more or less chaotic playing around with software
raid arrays of various levels.

===== Creating a Level 1 Raid =====

Create the raid1 device ''/dev/md1'' consisting of the two disks
''/dev/lvg/raid1'' and ''/dev/lvg/raid2'':
  # mdadm --create -l 1 -n 2 /dev/md1 /dev/lvg/raid1 /dev/lvg/raid2
Check results:
  # cat /proc/mdstat
  0/0 n0-1@pts/13
  Personalities : [raid6] [raid5] [raid4] [raid1]
  md1 : active raid1 dm-11[1] dm-10[0]
        1048564 blocks super 1.2 [2/2] [UU]

===== Merging an existing Level 1 Array to Level 5 (the hard way) =====

The idea behind all this is having an existing raid1 array, and after adding a
third disk of same size to the system the wish for converting to raid5. So
after safely degrading the raid1 above, there are two disks available for
initial setup of the new raid5 array.

==== Removing one of the Raid1 Devices ====

First, set the device faulty:
  # mdadm /dev/md1 -f /dev/lvg/raid2
  mdadm: set /dev/lvg/raid2 faulty in /dev/md1
Then finally remove it:
  # mdadm /dev/md1 -r /dev/lvg/raid2
  mdadm: hot removed /dev/lvg/raid2 from /dev/md1

==== Creating a (degraded) Level 5 Raid ====

The idea behind all this is having an existing raid1 array, and after adding a
third disk of same size to the system the wish for converting to a raid5. So
after safely degrading the raid1 above, there are two disks available for
initial setup of the new raid5 array.
  # mdadm --create -l 5 -n 3 /dev/md5 missing /dev/lvg/raid2 /dev/lvg/raid3
  mdadm: Defaulting to version 1.2 metadata
  mdadm: array /dev/md5 started.
The word ''missing'' is used as a special device string.
  # cat /proc/mdstat
  Personalities : [raid6] [raid5] [raid4] [raid1]
  md5 : active raid5 dm-12[2] dm-11[1]
        2096128 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [_UU]
  
  md1 : active raid1 dm-10[0]
        1048564 blocks super 1.2 [2/1] [U_]

==== Completing the Transition from Raid1 to Raid5 ====

Now one would create a filesystem on ''/dev/md5'', mount it and then copy over
all data from ''/dev/md1''. Afterwards, the old raid1 array can be completely
destructed and the single disk it consisted of added to the new array:
  # mdadm --stop /dev/md1
  mdadm: stopped /dev/md1
  # mdadm /dev/md5 --add /dev/lvg/raid1
  mdadm: added /dev/lvg/raid1
  # cat /proc/mdstat
  Personalities : [raid6] [raid5] [raid4] [raid1] 
  md5 : active raid5 dm-10[3] dm-12[2] dm-11[1]
        2096128 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [_UU]
        [=>...................]  recovery =  9.2% (97064/1048064) finish=1.1min speed=13866K/sec

===== Merging an existing Level 1 Array to Level 5 (the easy way) =====

Since dm-raid is actually able to change raid levels on the go, all the above
can be reduced to just a single command. Given that resizing the filesystem is
possible, which has to be done afterwards, of course:
  # mdadm /dev/md1 --grow -n 3 -l 5 -a /dev/lvg/raid3
  mdadm: level of /dev/md1 changed to raid5
  mdadm: added /dev/lvg/raid3
  mdadm: Need to backup 128K of critical section..
  # cat /proc/mdstat
  Personalities : [raid6] [raid5] [raid4] [raid1] 
  md1 : active raid5 dm-12[2] dm-11[1] dm-10[0]
        2097128 blocks super 1.2 level 5, 4k chunk, algorithm 2 [3/3] [UUU]

Ok, one limitation here is the device name. In order to change it, the array
has to be brought offline (of course, since the different device name has to
be mounted again anyway):
  # mdadm --stop /dev/md1
  mdadm: stopped /dev/md1
  # mdadm --assemble /dev/md5 -U name /dev/lvg/raid1 /dev/lvg/raid2 /dev/lvg/raid3
  mdadm: /dev/md5 has been started with 3 drives.
The result can be verified using ''mdadm -E'':
  # mdadm -E /dev/lvg/raid3
  /dev/lvg/raid3:
            Magic : a92b4efc
          Version : 1.2
      Feature Map : 0x0
       Array UUID : c527d58e:bfd9cd43:ebb8a4f6:a812c75b
             Name : base:5  (local to host base)
    Creation Time : Fri Mar 16 01:13:24 2012
       Raid Level : raid5
     Raid Devices : 3
  
   Avail Dev Size : 2097128 (1024.16 MiB 1073.73 MB)
       Array Size : 4194256 (2048.32 MiB 2147.46 MB)
      Data Offset : 24 sectors
     Super Offset : 8 sectors
            State : clean
      Device UUID : 9e065712:0eca28ca:c45ce117:020de68c
  
      Update Time : Fri Mar 16 01:29:36 2012
         Checksum : 5371875b - correct
           Events : 55
  
           Layout : left-symmetric
       Chunk Size : 4K
  
     Device Role : Active device 2
     Array State : AAA ('A' == active, '.' == missing)
Note the ''Name'' field which was ''base:1'' before.

===== Various Adjustment Knobs =====

There is lots of interesting stuff in ///sys/block/mdX/md///:

==== Sync Speed =====

When a raid array has to resync, it's speed may be controlled using the
attributes //sync_speed_min// and //sync_speed_max//. The maximum
synchronisation speed can be changed using a statement like:
  # echo 1000000 > /sys/block/md1/md/sync_speed_max
The unit is kibibytes. In order to set the value for all raid devices in the
system, one may use ///proc/sys/dev/raid/speed_limit_{min,max}// instead.

In //sync_speed//, the current speed is given in K/sec, 30s average.

===== Further Documentation =====

See the file //Documentation/md.txt// in your current kernel's sources.