Quote:
A *lot* of people seem to prefer RAID 1+0 .... Or the other way around (0+1)

Quote:

Quote:
Wow! Thanks Mark!

Well, on a related note: the Feisty bug database is presently littered with RAID roadkill. Lots of people seem to have seen similar installer issues to what I ran into, and my fix for it (intervene in the boot process to get it up, and then just update the packages to repair it all) doesn't work for most of them.

Okay, the latest word on this is that those fixes have now made it into the installer disks as of 14-April-2007, and RAID installs now seem to be working for everyone.

Woo-Hoo!

Quote:
But still no extra 250GB

In an ideal world I would just go out any buy 4 x 500Gb SATA drives, a local supplier has Seagate drives at under £70 now. With this solution 1Tb would be more than enough so RAID10 would be the perfect solution.

Although speed isn't a major issue at present, I'm running and ever increasing Gigabit network at home. At the moment it's just the Macbook and one of the servers as my switch only has 2 Gigabit ports. I have noticed that even though the spec of the new NAS box is much higher than my current clarkconnect firewall, the CC box transfers are much quicker. This could be down to the intel gigabit nic I suppose, the one in the NAS is a £15 D-Link (nasty!).

If only I had put a bet on Ascot I could be straight down to pick up my SATA drives

Cheers

Cris.

Quote:
Okay, the latest word on this is that those fixes have now made it into the installer disks as of 14-April-2007, and RAID installs now seem to be working for everyone.

Quote:
I do have enough spare HD's to run 4 x 250Gb and 3 x 120Gb arrays

Quote:

Am I right in assuming I can't boot from RAID5? Or could I do a complex setup of striping and mirroring a 1Gb partition and then RAID5 the rest? Putting a 5th drive in the system isn't something I really want to do at this stage if I can help it.

Quote:

The standard way is to have a smallish RAID1 partition for /boot (or the rootfs), and a RAID5 partition for the rest. On each drive. Just like in the article I linked earlier.

Quote:

With the two O/S partitions, you can experiment with a newer O/S version in the future, and yet still have your original working O/S version intact until all is well with the new one.

Quote:
Oh, one tip if you go software side. Don't RAID the hard disk. Create partitions on the disks and RAID those, making the partition a bit smaller then the hard disk. That way, if one of your Brand X 120gig drives fails, and only a Brand Y 120g can be found later. There is a possibility Brand Y 120 gig will be a smidge smaller then Brand X, and by using a smaller setup of space on the old disk, the new one should still work into the array.

Quote:
My problem is that I'm not a Linux god.

Me too!

I have only had the chance to test the MegaRAID i4 once so far. What I did notice is that if you reboot it does start the rebuild right from the start again. ie if you were at 5% and need to reboot your machine for some reason, when it all comes back up the MegaRAID starts again from 0%, but I did find it started again automatically.

I'm really glad I hve given "hardware" RAID a try. I can now really see Mark's point, you do stand more of a chance of getting your data back with software RAID if you know what you are doing.

My problem is I don't know what I am doing, yet

The other problem with the MegaRAID i4 is that it keeps the disc spun up 100% I left the system on over the weekend while I was away from home, nobody as been using the NAS and the drives are a full temp. For this reason alone the i4 is going back on eBay. This can't be good for the type of drive I am using here!

Cheers

Cris.

Quote:
suppose there are some new updates to the RAID implementation and I end up screwing up my data. Maybe I just don't choose the right compile flags to ensure compatibility or something like that.

Quote:
Also what if I have a catastrophic hardware failure (fried motherboard or something) and have to rebuild everything. I KNOW that it is possible to rebuild everything to get the data off the drives, but maybe I'm not capable of doing it without making a mistake somewhere along the way.

Quote:

Quote:
My problem is that I'm not a Linux god.

I'm really glad I hve given "hardware" RAID a try. I can now really see Mark's point, you do stand more of a chance of getting your data back with software RAID if you know what you are doing.

My problem is I don't know what I am doing, yet

That's the thing about 'community'

The linux-raid list has regular 'help' emails and they almost all get sorted - I think the only caveat is "don't mess if you don't know what you're doing - just ask".

Also, you mentioned wanting to power down the disks in your raid. I'm not sure how well that works 'out of the box' under linux. You may need to do some fiddling.

Quote:
I'm not throwing out FUD

Quote:
I see posts about using the force flag and various "pucker" factors in this thread and hence some of my fears. I figure plugging the array into the new box would require forcing it online.

That's the problem with being fully honest - I did try to emphasise that the --force stuff was to do recovery in situations where you were already royally FUBARed.

Let me put it this way, if and only if your system burns to the ground and is eaten by ants then linux raid *may* need to use the --force command Better?

Moving to another machine - business as usual.
Disk failure - business as usual.
Switch from RedHat to SUSE to Debian - business as usual.
Upgrade kernel - business as usual.
etc etc

One of my biggest problems with linux RAID way back when was when a disk failed and I didn't notice for months - the thing just kept going.

Quote:
Also, you mentioned wanting to power down the disks in your raid. I'm not sure how well that works 'out of the box' under linux. You may need to do some fiddling.

In FreeNAS it was one feature that worked really well, it must cache part/all of the file table (or whatever?) as you could pretty much browse the NAS and the drives spun up when a file was requested, but I never did get to the point of mave many files on it.

The only slight problem was that the WD drive sins up much slower than the seagate driver, that wa causing some retry errors in the log files, but from my point of view it worked pretty well.

I think the Software RAID argument is winning the battle at the moment, like you said if I get stuck all I need to do is ask

Cheers

Cris.

Quote:
I'm not throwing out FUD

Just trying to prevent it from being interpreted as such when somebody stumbles over this thread via Google.

You actually sound to me like a pretty competent user of this stuff -- more power to ya!

Cheers

Quote:
Periodic backups, there is no substitute.

Quote:

Considering Linux as a software RAID solution, I wonder how many people in the world have intimate knowledge of Linux on a par with some of the contributors of this forum? I’ll bet not many. This is the reason why software RAID5 is not an option for me. Some of you make it sound so easy just to do whatever needs to be done to fix whatever problems might occur.

Quote:
Again, the issues with administrating a RAID array are not unique to Linux. RAID5 failures will likely be just as complex to recover from regardless of the underlying O/S, or whether it's "hardware" or "software" RAID.

Just trying to keep things clear here for posterity (aka. "google").

Quote:

When I was using RAID5 it was the total repository of all of my media. There was no other backup. At the time I considered RAID5 to be sufficient unto itself. This was a mistake. How are you planning on monitoring the status of your drives? I used the Promise Superswap 1000 drive bays that gave constant condition status via LED’s. Even with this I lost my data once. You think it’s a hassle copying 700GB? How about re-ripping a TB? I guess I have been burned too many times. What I purpose is probably extreme.

There's a lot being said here; some is linux specific but most, I think, is generic.

Clearly if you are experienced in one OS and move to another then there's a learning curve - that said:

Setting up:
Linux RAID is simple and generally just works.
BIOS or windows RAID is probably a touch simpler and generally just works.
Not much in it.

Monitoring:
Linux RAID is simple and generally just works via email. [My machines talk to me if a disk fails.]
BIOS or windows RAID may use email, LEDs, beep or just notify you on reboot (not good for a server) .
Not much in it.

Disaster:
For an average user it is feasible to recover a Linux RAID systems when things go badly wrong (and I'm talking about massive hardware failure here!) but it needs to be dug into very deeply and can become very complex indeed. [However a few hours of hair pulling may be a lot easier than a 1Tb restore!]

For an average user if you have an unexpected hardware failure with windows or BIOS RAID then you are screwed. You would need to spend real cash (many hundreds of $) with a forensic data recovery firm to stand a chance of getting your data back - or just start restoring.

Huge difference.


Another problem area is how people thing about risk, backups and resilience.

In other words : do I bother with RAID or do I just have backups.

It's complex; but in general RAID reduces the likelihood of needing to restore from backups - Neutrino you are 100% spot on when you say it is not sufficient to replace backups. However it does almost eliminate the need to restore from backups for disk failures.

That's worth restating:
* I'll (almost) never need to restore from backup due to disk failure.

Here, I think the problem you will face Neutrino is that you are almost *certain* to lose all your data and have to restore from backup. (I assume you have 1Tb of data on 3x320Gb disks = no resilience). Your system is 3 times more fragile than a single disk system. Assuming drives are 95% likely to die in 6 years (even spread) then your system is 95% likely to die within 2 years - I hope you have an OCD tendency to backup!



FWIW I have multiple data stores.
One is for music and films that I have recorded or ripped. The backup is the original media. When (not if) a drive fails, I just replace it. I doubt I will ever have to re-rip. If I didn't have a resilient setup then I'd have to re-rip (or restore) every time a drive failed.

Another store is a simple striped array.
I hold important data on there and it backs up incrementally onto a separate striped array using snapshots.
Having the backup online all the time means I can monitor the health and replace drives.


I hope this comes across in the spirit it's intended

A few years ago I had a disk failure on the disk with all my photos on it.
I bought a replacement hard disk and did a disk copy from my backup to my new disk.
Sadly I swapped the device names and overwrote my backup with a blank disk. I felt a horrible cold chill and the hairs on the back of my neck stood up as I realised that I'd lost years worth of photos.
Luckily I was a sysadmin at the time and by 'habit', as soon as the disk failed I made a copy of the backup to some free space. It took me 10 seconds to remember that I had the copy - they were not pleasant seconds.
I did however just do a quick 'copy' and the timestamps on all those files show the date and time of the near-disaster.

So the reason I bother with this RAID stuff is because I really worry about the 'normal' people who are going to lose their digital memories and I try to help out
I would like to see Linux RAID get a nicer frontend (for home use) and better disaster recovery etc - it will happen.

Quote:

So the reason I bother with this RAID stuff is because I really worry about the 'normal' people who are going to lose their digital memories and I try to help out

Quote:

Quote:

So the reason I bother with this RAID stuff is because I really worry about the 'normal' people who are going to lose their digital memories and I try to help out

And I was agreeing with your post until I read that.
...
I'll get off my soap box now. But I won't hesitate to get back on it if someone starts implying raid is a backup solution.

Matthew
PS: Perhaps I'm reading your post wrong, and you mean preserving memories is the reason you help other people with their RAID recovery. I still stand by my rant about backups, even if it doesn't apply to you.

You did misunderstand - but it was a long post and they're not always the clearest things to read or write.

For "RAID stuff" substitute "RAID and backup stuff".

RAID is bloody useful but it is not a backup solution.

PS Somehow it has more of an "Oh f*ck, did I really do that" feeling than "Kodak moments"