Software RAID in Linux is realised using the dm-raid kernel module and can be
controlled by the mdadm utility. Information and status of known raid
arrays can be gathered by reading /proc/mdstat. Array details are
available using mdadm -D:
# mdadm -D /dev/md1
/dev/md1:
Version : 1.2
Creation Time : Fri Mar 16 01:13:24 2012
Raid Level : raid1
Array Size : 1048564 (1024.16 MiB 1073.73 MB)
Used Dev Size : 1048564 (1024.16 MiB 1073.73 MB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent
Update Time : Fri Mar 16 01:13:36 2012
State : clean, resyncing
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Resync Status : 25% complete
Name : base:1 (local to host base)
UUID : c527d58e:bfd9cd43:ebb8a4f6:a812c75b
Events : 4
Number Major Minor RaidDevice State
0 254 10 0 active sync /dev/dm-10
1 254 11 1 active sync /dev/dm-11
The following shows some more or less chaotic playing around with software raid arrays of various levels.
Create the raid1 device /dev/md1 consisting of the two disks
/dev/lvg/raid1 and /dev/lvg/raid2:
# mdadm --create -l 1 -n 2 /dev/md1 /dev/lvg/raid1 /dev/lvg/raid2
Check results:
# cat /proc/mdstat
0/0 n0-1@pts/13
Personalities : [raid6] [raid5] [raid4] [raid1]
md1 : active raid1 dm-11[1] dm-10[0]
1048564 blocks super 1.2 [2/2] [UU]
The idea behind all this is having an existing raid1 array, and after adding a third disk of same size to the system the wish for converting to raid5. So after safely degrading the raid1 above, there are two disks available for initial setup of the new raid5 array.
First, set the device faulty:
# mdadm /dev/md1 -f /dev/lvg/raid2 mdadm: set /dev/lvg/raid2 faulty in /dev/md1
Then finally remove it:
# mdadm /dev/md1 -r /dev/lvg/raid2 mdadm: hot removed /dev/lvg/raid2 from /dev/md1
The idea behind all this is having an existing raid1 array, and after adding a third disk of same size to the system the wish for converting to a raid5. So after safely degrading the raid1 above, there are two disks available for initial setup of the new raid5 array.
# mdadm --create -l 5 -n 3 /dev/md5 missing /dev/lvg/raid2 /dev/lvg/raid3 mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md5 started.
The word missing is used as a special device string.
# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] [raid1]
md5 : active raid5 dm-12[2] dm-11[1]
2096128 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [_UU]
md1 : active raid1 dm-10[0]
1048564 blocks super 1.2 [2/1] [U_]
Now one would create a filesystem on /dev/md5, mount it and then copy over
all data from /dev/md1. Afterwards, the old raid1 array can be completely
destructed and the single disk it consisted of added to the new array:
# mdadm --stop /dev/md1
mdadm: stopped /dev/md1
# mdadm /dev/md5 --add /dev/lvg/raid1
mdadm: added /dev/lvg/raid1
# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] [raid1]
md5 : active raid5 dm-10[3] dm-12[2] dm-11[1]
2096128 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [_UU]
[=>...................] recovery = 9.2% (97064/1048064) finish=1.1min speed=13866K/sec
Since dm-raid is actually able to change raid levels on the go, all the above can be reduced to just a single command. Given that resizing the filesystem is possible, which has to be done afterwards, of course:
# mdadm /dev/md1 --grow -n 3 -l 5 -a /dev/lvg/raid3
mdadm: level of /dev/md1 changed to raid5
mdadm: added /dev/lvg/raid3
mdadm: Need to backup 128K of critical section..
# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] [raid1]
md1 : active raid5 dm-12[2] dm-11[1] dm-10[0]
2097128 blocks super 1.2 level 5, 4k chunk, algorithm 2 [3/3] [UUU]
Ok, one limitation here is the device name. In order to change it, the array has to be brought offline (of course, since the different device name has to be mounted again anyway):
# mdadm --stop /dev/md1 mdadm: stopped /dev/md1 # mdadm --assemble /dev/md5 -U name /dev/lvg/raid1 /dev/lvg/raid2 /dev/lvg/raid3 mdadm: /dev/md5 has been started with 3 drives.
The result can be verified using mdadm -E:
# mdadm -E /dev/lvg/raid3
/dev/lvg/raid3:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : c527d58e:bfd9cd43:ebb8a4f6:a812c75b
Name : base:5 (local to host base)
Creation Time : Fri Mar 16 01:13:24 2012
Raid Level : raid5
Raid Devices : 3
Avail Dev Size : 2097128 (1024.16 MiB 1073.73 MB)
Array Size : 4194256 (2048.32 MiB 2147.46 MB)
Data Offset : 24 sectors
Super Offset : 8 sectors
State : clean
Device UUID : 9e065712:0eca28ca:c45ce117:020de68c
Update Time : Fri Mar 16 01:29:36 2012
Checksum : 5371875b - correct
Events : 55
Layout : left-symmetric
Chunk Size : 4K
Device Role : Active device 2
Array State : AAA ('A' == active, '.' == missing)
Note the Name field which was base:1 before.
There is lots of interesting stuff in /sys/block/mdX/md/:
When a raid array has to resync, it's speed may be controlled using the attributes sync_speed_min and sync_speed_max. The maximum synchronisation speed can be changed using a statement like:
# echo 1000000 > /sys/block/md1/md/sync_speed_max
The unit is kibibytes. In order to set the value for all raid devices in the system, one may use /proc/sys/dev/raid/speed_limit_{min,max} instead.
In sync_speed, the current speed is given in K/sec, 30s average.
See the file Documentation/md.txt in your current kernel's sources.