PumpAction
August 4th, 2007, 10:53 AM
I
write this guide because i found a lot of info on the subject but still
i had some problems because a lot of the info was very out-of-context
it dealt with very specific problems and no clue if I had/should do
anything before or after. With this guide I try to cover the whole
process, thats why I've included a plan over what I have to to before I
actually comes with the specific commands to type in terminal. This
guide got some bloat that you might not need, to skip it; search this
page for "blabla" and take it from there. For more information about
what those commands are suppose to do read the bloat:
I make this guide while experimenting to get the process right for creating and expanding a raid 5 array.
My situation is: I got three 320gb sata disks which is empty, and another three which is full, I want to combine all six sata disks in one raid 5 array.
I got no way to back it all up and format all disk and set up a 6 disk raid-5 from scratch, so if my idea was to set up a raid5 with 3 disks move over some data and, format the now empty disk and expand the array to cover the new disk. Then repeat the process until its all six disks in one array. This is of course not critical data that, that why I don't got backups. Loosing it is only inconvenient, I am aware of this SO SHOULD YOU BE IF YOU FOLLOW THIS GUIDE. IF DATA ARE IMPORTANT HAVE BACKUPS AND DONT BLAME ME IF YOU LOOSE DATA. I'm no Ubuntu/Linux/RAID expert, this guide is more of a collection of other guides aimed at solving one specific set of problems. I write this guide while testing everything i do on six small partitions spread over the three empty disks, it is my advise that you to "try-it-before-you-buy-it".
In advance I have made some preparations which i won't cover in this guide. This is:
Prepare some files with a checksum (sfv). which i use as a quick check to see if my files are ok after expansion.
I also know my partitions very well, almost to the point that i could point them out on the platters :P
Also my system is up and running with a working internet connection and has access to apt-get
This howto will try to cover the whole process of setting up a raid5 array and expanding/growing it on a system already up and running. This guide will NOT show you how to use a softraid5 array as basis for your operating system or anything like that. This simply shows you how to take a bunch of disks turn them into a raid5 and how to expand it later without loosing data.
I write this guide because i found a lot of info on the subject but still i had some problems because a lot of the info was very out-of-context it dealt with very specific problems and no clue if I had/should do anything before or after. With this guide I try to cover the whole process, thats why I've included a plan over what I have to to before I actually comes with the specific commands to type in terminal.
What I've got:
Three full 320gb harddisk
Three empty 320gb harddisk
Ubuntu Feisty Fawn (7.04) 32-bit installed on some other completely separate disk.
Access to gparted, cksfv, mdadm either already installed or via apt-get install and a working internet connection
The todo list human mode:
Create raid5 with 3 empty disks/partitions
Copy data to raid5 array
Expand raid5 array and its filesystem with one (or more) disk/partition
This simple plan is a bit more complicated for the computer. Here is it's version:
The todo list computer mode:
----Make softraid section----
Prepare partitions to be used in array and set raid flag, format and set raid flag
Create raid array with mdadm
Format (make filesystem) raid array
Mount the brand new and flashingly good looking supercool new ready to be used raid 5 array
Copy files
----Expand section----
Unmount the brand new and flashingly good looking supercool new ready to be used raid 5 array
Prepare and the new partition to be added
Put the new partition under softraiddriver control
Tell driver to grow on to the new disk
Check filesystem integrity
Expand filesystem so you get more space to play with
mount the expanded array
Check the integrety of your files
The terminal commands, no more blabla:
This is where it get serious, these are the commands I've used. MAKE SURE YOU CHANGE EVERYTHING THAT NEEDS TO BE CHANGED like device names and path, number of raid devices and such. tore is my username, md0 is the raid-array I use, level=5 is raid5 vs. for example raid1 which is level=1, raid-devices=3, the number of partitions dedicated for this specific array. /home/tore/raid is where i mount my array.
REMEMBER THIS GUIDE COME WITH NO WARRANTY, IF YOU LOOSE DATA DON'T BLAME ME. Try-it-before-you-buy-it if you can.
EDIT: I've edited the guide, I split the guide into three sections. (Prepare partition - make raid5 and format it - expand array & filesystem)
This guide have been tested with success on Ubuntu Feisty Fawn (7.04) 32-bit.
If you test it on another system, tell me and i add a line above.
---Prepare the disk/partition---
What preparations needs to be done?
First you must know which partition/disk to use. I got no specific way of helping you here, best tips is to use gparted to get a graphic view of disks and partitions.
gparted can be installed by:
sudo apt-get install gparted
To run gparted type:
sudo gparted
When you have sorted out which partitions/disks to use there are four things to do:
1.remove it from fstab
2.format it to ext3
3.set raid flag
4.unmount partitions
When you have figured out which partitions to use you can proceed to removing it from fstab. I assume you know how to do this, but a quick refresher if you don't remember the details:
Back up your fstab:
sudo cp /etc/fstab /etc/fstab.backup
edit your original fstab with gedit:
sudo gedit /etc/fstab
put a # in front of the line containing the partition you want to use. Save&Exit
Now its time for format the disks/partition to be used in the array. This is also a very straight forward kind of operation. Just run gparted again.
sudo gparted
Since gparted is graphical I cant give you a command line guide. But if you could find your way in fstab you should be able to navigate you way to your partitions in gparted with ease. Once there just right click it and format to ext3. Remember to be sure to pick the right partition, if you pick the wrong one there is no going back.
After the partition has been formated right click it and “manage flags”, then hook-on the raid flag. Click "OK" & Exit.
The last step, make sure to unmount the partitions to be used in the array. Now they should be out of the fstab, this could be done via a reboot or:
sudo umount name_of_device
eksample:
sudo umount /dev/sda1
When this is done on all disks/partitions that are suppose to be a part of the initial array you are finished with preparing the disks/partitions.
----Make softraid section----
Install mdadm (softraid driver/application):
sudo apt-get install mdadm (awnser a few questions, i choose: none - yes)
Create raid device with partitions you prepared in gparted(modify this line for correct number of raid-devices and make sure to use the right devices):
sudo mdadm --create /dev/md0 --level=5 --raid-devices=3 /dev/sda1 /dev/sdb1 /dev/sdc1
The prosess of creating the array takes time, monitor it and do not continue this guide until its done(type it over again to update):
cat /proc/mdstat
Make a ext3 filesystem that spans the array:
sudo mkfs.ext3 /dev/md0
Mount the array (remember to choose were):
sudo mount -t ext3 /dev/md0 /home/tore/raid
Take ownership (replace tore with your username and correct the path)
sudo chown -R tore:tore /home/tore/raid
The copy some files with valid sfv and check to see if everything is ok, and to have a refrence point after expansion.
cd /home/tore/raid
cksfv -r -q
----Expand section----
Unmount the array, i don't know if this is necessary, but doing so means less stuff to go wrong.
sudo umount /dev/md0
Then you need to go and prepare the disk(s) to be added. (The prepare section of the guide)
Then tell mdadm that it can play with it:
sudo mdadm --add /dev/md0 /dev/sda2
Then tell mdadm to stop playing and use it to expand/grow your raid5 array(make sure that the number of raid devices is correct, it should be the number of disks/partitions the grown/expanded array will have):
sudo mdadm --grow /dev/md0 --raid-devices=4
View and wait for the new array to reshape:
cat /proc/mdstat
Check the filesystem (required to resize it):
sudo e2fsck -f /dev/md0
Resize the filesystem on the new array so that you can use it for something useful.
sudo resize2fs /dev/md0
Mount the expanded/grown array with it's resized filesystem(remember to change the mount point):
sudo mount -t ext3 /dev/md0 /home/tore/raid
Go check your files:
cd /home/tore/raid
cksfv -r -q
_____________________
Need to expand more? Repeat the expand section. Remember to change number of raid devices and choose the correct device to add. Also, you can add more then one disk at the time, I jump strait from 3 to 6.
_____________________
Was this not the right guide for you? Try one of these (my guide is composed from some of these):
EDIT1: Fix a buch of typos.
EDIT2: Split the guide into three sections, added some info on fstab, fix more typos.
I make this guide while experimenting to get the process right for creating and expanding a raid 5 array.
My situation is: I got three 320gb sata disks which is empty, and another three which is full, I want to combine all six sata disks in one raid 5 array.
I got no way to back it all up and format all disk and set up a 6 disk raid-5 from scratch, so if my idea was to set up a raid5 with 3 disks move over some data and, format the now empty disk and expand the array to cover the new disk. Then repeat the process until its all six disks in one array. This is of course not critical data that, that why I don't got backups. Loosing it is only inconvenient, I am aware of this SO SHOULD YOU BE IF YOU FOLLOW THIS GUIDE. IF DATA ARE IMPORTANT HAVE BACKUPS AND DONT BLAME ME IF YOU LOOSE DATA. I'm no Ubuntu/Linux/RAID expert, this guide is more of a collection of other guides aimed at solving one specific set of problems. I write this guide while testing everything i do on six small partitions spread over the three empty disks, it is my advise that you to "try-it-before-you-buy-it".
In advance I have made some preparations which i won't cover in this guide. This is:
Prepare some files with a checksum (sfv). which i use as a quick check to see if my files are ok after expansion.
I also know my partitions very well, almost to the point that i could point them out on the platters :P
Also my system is up and running with a working internet connection and has access to apt-get
This howto will try to cover the whole process of setting up a raid5 array and expanding/growing it on a system already up and running. This guide will NOT show you how to use a softraid5 array as basis for your operating system or anything like that. This simply shows you how to take a bunch of disks turn them into a raid5 and how to expand it later without loosing data.
I write this guide because i found a lot of info on the subject but still i had some problems because a lot of the info was very out-of-context it dealt with very specific problems and no clue if I had/should do anything before or after. With this guide I try to cover the whole process, thats why I've included a plan over what I have to to before I actually comes with the specific commands to type in terminal.
What I've got:
Three full 320gb harddisk
Three empty 320gb harddisk
Ubuntu Feisty Fawn (7.04) 32-bit installed on some other completely separate disk.
Access to gparted, cksfv, mdadm either already installed or via apt-get install and a working internet connection
The todo list human mode:
Create raid5 with 3 empty disks/partitions
Copy data to raid5 array
Expand raid5 array and its filesystem with one (or more) disk/partition
This simple plan is a bit more complicated for the computer. Here is it's version:
The todo list computer mode:
----Make softraid section----
Prepare partitions to be used in array and set raid flag, format and set raid flag
Create raid array with mdadm
Format (make filesystem) raid array
Mount the brand new and flashingly good looking supercool new ready to be used raid 5 array
Copy files
----Expand section----
Unmount the brand new and flashingly good looking supercool new ready to be used raid 5 array
Prepare and the new partition to be added
Put the new partition under softraiddriver control
Tell driver to grow on to the new disk
Check filesystem integrity
Expand filesystem so you get more space to play with
mount the expanded array
Check the integrety of your files
The terminal commands, no more blabla:
This is where it get serious, these are the commands I've used. MAKE SURE YOU CHANGE EVERYTHING THAT NEEDS TO BE CHANGED like device names and path, number of raid devices and such. tore is my username, md0 is the raid-array I use, level=5 is raid5 vs. for example raid1 which is level=1, raid-devices=3, the number of partitions dedicated for this specific array. /home/tore/raid is where i mount my array.
REMEMBER THIS GUIDE COME WITH NO WARRANTY, IF YOU LOOSE DATA DON'T BLAME ME. Try-it-before-you-buy-it if you can.
EDIT: I've edited the guide, I split the guide into three sections. (Prepare partition - make raid5 and format it - expand array & filesystem)
This guide have been tested with success on Ubuntu Feisty Fawn (7.04) 32-bit.
If you test it on another system, tell me and i add a line above.
---Prepare the disk/partition---
What preparations needs to be done?
First you must know which partition/disk to use. I got no specific way of helping you here, best tips is to use gparted to get a graphic view of disks and partitions.
gparted can be installed by:
sudo apt-get install gparted
To run gparted type:
sudo gparted
When you have sorted out which partitions/disks to use there are four things to do:
1.remove it from fstab
2.format it to ext3
3.set raid flag
4.unmount partitions
When you have figured out which partitions to use you can proceed to removing it from fstab. I assume you know how to do this, but a quick refresher if you don't remember the details:
Back up your fstab:
sudo cp /etc/fstab /etc/fstab.backup
edit your original fstab with gedit:
sudo gedit /etc/fstab
put a # in front of the line containing the partition you want to use. Save&Exit
Now its time for format the disks/partition to be used in the array. This is also a very straight forward kind of operation. Just run gparted again.
sudo gparted
Since gparted is graphical I cant give you a command line guide. But if you could find your way in fstab you should be able to navigate you way to your partitions in gparted with ease. Once there just right click it and format to ext3. Remember to be sure to pick the right partition, if you pick the wrong one there is no going back.
After the partition has been formated right click it and “manage flags”, then hook-on the raid flag. Click "OK" & Exit.
The last step, make sure to unmount the partitions to be used in the array. Now they should be out of the fstab, this could be done via a reboot or:
sudo umount name_of_device
eksample:
sudo umount /dev/sda1
When this is done on all disks/partitions that are suppose to be a part of the initial array you are finished with preparing the disks/partitions.
----Make softraid section----
Install mdadm (softraid driver/application):
sudo apt-get install mdadm (awnser a few questions, i choose: none - yes)
Create raid device with partitions you prepared in gparted(modify this line for correct number of raid-devices and make sure to use the right devices):
sudo mdadm --create /dev/md0 --level=5 --raid-devices=3 /dev/sda1 /dev/sdb1 /dev/sdc1
The prosess of creating the array takes time, monitor it and do not continue this guide until its done(type it over again to update):
cat /proc/mdstat
Make a ext3 filesystem that spans the array:
sudo mkfs.ext3 /dev/md0
Mount the array (remember to choose were):
sudo mount -t ext3 /dev/md0 /home/tore/raid
Take ownership (replace tore with your username and correct the path)
sudo chown -R tore:tore /home/tore/raid
The copy some files with valid sfv and check to see if everything is ok, and to have a refrence point after expansion.
cd /home/tore/raid
cksfv -r -q
----Expand section----
Unmount the array, i don't know if this is necessary, but doing so means less stuff to go wrong.
sudo umount /dev/md0
Then you need to go and prepare the disk(s) to be added. (The prepare section of the guide)
Then tell mdadm that it can play with it:
sudo mdadm --add /dev/md0 /dev/sda2
Then tell mdadm to stop playing and use it to expand/grow your raid5 array(make sure that the number of raid devices is correct, it should be the number of disks/partitions the grown/expanded array will have):
sudo mdadm --grow /dev/md0 --raid-devices=4
View and wait for the new array to reshape:
cat /proc/mdstat
Check the filesystem (required to resize it):
sudo e2fsck -f /dev/md0
Resize the filesystem on the new array so that you can use it for something useful.
sudo resize2fs /dev/md0
Mount the expanded/grown array with it's resized filesystem(remember to change the mount point):
sudo mount -t ext3 /dev/md0 /home/tore/raid
Go check your files:
cd /home/tore/raid
cksfv -r -q
_____________________
Need to expand more? Repeat the expand section. Remember to change number of raid devices and choose the correct device to add. Also, you can add more then one disk at the time, I jump strait from 3 to 6.
_____________________
Was this not the right guide for you? Try one of these (my guide is composed from some of these):
EDIT1: Fix a buch of typos.
EDIT2: Split the guide into three sections, added some info on fstab, fix more typos.