Manual Software RAID Creation

Manual software RAID configuration from the command line can be easier than using a UI-guided configuration tool.

RAID At Boot Time

The best practice is to avoid placing boot loaders and /boot on disks that will be used in software RAID arrays. If this isn't an option, the next-best step is to partition off space at the beginning of each drive in the RAID array for use as /boot. The space required varies between distributions. Arch can create very small initram images that help /boot fit inside 10MB. Ubuntu likes to keep older versions of the kernel until you specifically delete them, which can make /boot weigh in over 128MB during the LTS life span.

As for root-on-RAID, you need two things:

• The initramdisk has the raid tools necessary to discover and load RAID arrays. Arch needs to have mdadm specified as a HOOK in /etc/mkinitcpio.conf.
• You may need to specify md=[x],[device],[device]... for each RAID array at the end of you kernel line of your boot loader.

Stopping Existing Arrays

The Arch Linux installer automatically detects and starts RAID arrays. If you want to change the partition setup, you have to stop the arrays first.

mdadm --stop /dev/md0

The Ubuntu Linux installer does not load RAID arrays unless you specify that you want to set up RAID arrays during the partitioning section.

Clearing Existing Superblocks

If you're re-using a drive that was already part of a software RAID array, most UI tools will choke (in that the UI won't let you do what you want) when they start mdadm and find superblocks. Keep in mind that simple formatting only over-writes the beginning of the disk with filesystem data, plus special blocks spread throughout the device.

mdadm --zero-superblock [device]

If you used the entire drive, [device] will be something like /dev/sda. If you used partitions, [device] will be something like /dev/sda1.

If mdadm fails to find the superblock, but you know it's there, you can use dd to zero out all possible locations

dd if=/dev/zero of=[device] bs=512 count=265
dd if=/dev/zero of=[device] bs=512 count=265 seek=[device-sectors-less-265]

Whole Device vs Partitions

There's no difference in performance -- Linux software RAID doesn't care. You, on the other hand, may get annoyed when fdisk and parted report that member devices and/or RAID arrays themselves don't have valid partition tables. Then again, it may be a trade-off you're willing to make to not have to worry about partition alignment.

Of course, you'll have no choice but to use partitions if you can't put /boot on a non-RAID device.

Chunk Sizes

With RAID0 and RAID1, chunk size doesn't really make a difference so you can use any chunk size you want. With RAID4 - RAID6. chunk size does make a difference and you'll most likely want to use a 32K chunk size.

Less typing version:

mdadm -Cve 1.2 /dev/md0 -l [0|1|4|5|6|10] -c [chunk-size] -n [num-raid-devs] \
[raiddev1] [raiddev1] etc. {-x [num-spare-devs] [sparedev1] etc.}

Easier to remember version:

mdadm --create --verbose --metadata 1.2 /dev/md0 --level=[0|1|4|5|6|10] \
--chunk=[chunk-size] --raid-devics=[num-raid-devs] [device1] [device2] etc. \
{--spare-devices [num-spare-devs] [sparedev1] etc.}

Block Sizes

RAID or not, you'll most likely want 4K block sizes, which is the default for most filesystems. (Some default to the "architecture page size default" which ends up being 4K — exact size can be had by issuing the command getconf PAGESIZE.)

On modern Linux distributions, EXT and XFS filesystems should automatically detect and set proper stride and stripe parameters upon creation to help optimize placement of special blocks. You can see exactly how a filesystem was mounted by checking the output from cat /proc/mounts.

• EXT

  mkfs -t ext[2|3|4] -L [label] -b 4096 -E stride=[stride],stripe-width=[stripe] [device]

  tune2fs -E stride=[stride],stripe-width=[stripe] [device]

  • stride = chunk size ÷ block size
  • stripe = stride × non-parity non-spare disks
• XFS

  mkfs -t xfs -b 4096 -d su=[chunk-size],sw=[non-parity non-spare devices] /dev/md0

  xfs_db -c unit=[chunk-size] width=[non-parity non-spare disks]