Raid systems: Proprietary?

Fortunately, it's not me, it's Tod.

Thanks for the information, everyone, I think it will help him out a lot.

As others have said, it's a proprietary hack. IF you run Linux, the hack can be undone after a catastrophe, but for MS and other OSs, it'll be SOL.

In the longer term, a new standard for RAID headers is emerging, know as "DDF". But support is patchy at present.

Cheers

One advantage of having a OS-based, software RAID system is in fact that you may change hardware completely and the OS will recognise the stripe/mirror set.

Disadvantage is of course performance.

On a small server here I have a W2K-based Raid-5 system with three disks, and I've gone through mobo replacement (changed brand/model) and OS reinstallation and the software based raid-5 set survived.

Of course the OS was on a different, independent disk, since the OS needs to be up and running to detect a raid set, import it and enable it correctly.

True!
And supposedly Widnows implementation of raid is good too.

I was actually referring to the performance of the overall system, since the CPU is now in charge of RAID as well as all the rest. On the other hand, I personally never spent time in finding out how much more resources would be devoted to that, and by simply looking at it, the small server I referred to in my previous post does not seem to show any more CPU time devoted to disk management. It's not a heavily used server, though, so I don't know how it would behave under stress...

No to mention that hardware RAID has to go through a single IO channel, while software can take advantage of multiple channels. What we need is a RAID accelerator of some nature to offload the RAID tasks from the CPU.

Code:

---

 11:42am  up 37 days,  1:35,  3 users,  load average: 0.10, 0.20, 0.42
161 processes: 160 sleeping, 1 running, 0 zombie, 0 stopped
CPU states:  1.1% user,  4.7% system,  0.0% nice, 94.1% idle
Mem:   387308K av,  378620K used,    8688K free,  117560K shrd,   74840K buff
Swap:  995944K av,   66612K used,  929332K free                   92108K cached
  PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM  CTIME COMMAND
 7208 genixia   16   0  1184 1184   876 R     5.0  0.3   0:01 top
 2034 genixia    2   0  2672 2008  1236 S     0.5  0.5   2:42 gnome-terminal
 1773 root       3   0  117M 109M  5552 S     0.3 29.0  3420m X
    1 root       0   0   132   76    60 S     0.0  0.0  7764m init
    2 root       0   0     0    0     0 SW    0.0  0.0   0:42 kflushd
    3 root       0   0     0    0     0 SW    0.0  0.0   0:13 kupdate
    4 root       0   0     0    0     0 SW    0.0  0.0   0:00 kpiod
    5 root       0   0     0    0     0 SW    0.0  0.0   5:50 kswapd
    6 root     -20 -20     0    0     0 SW<   0.0  0.0   0:00 mdrecoveryd
    7 root     -20 -20     0    0     0 SW<   0.0  0.0   0:00 raid1d
    8 root      19  19     0    0     0 SWN   0.0  0.0   0:08 raid1syncd
    9 root     -20 -20     0    0     0 SW<   0.0  0.0   0:00 raid1d
   10 root      19  19     0    0     0 SWN   0.0  0.0   0:02 raid1syncd
   11 root     -20 -20     0    0     0 SW<   0.0  0.0   0:00 raid1d
   12 root      19  19     0    0     0 SWN   0.0  0.0   0:08 raid1syncd
   13 root     -20 -20     0    0     0 SW<   0.0  0.0   0:00 raid1d
   14 root      19  19     0    0     0 SWN   0.0  0.0   0:19 raid1syncd
   15 root     -20 -20     0    0     0 SW<   0.0  0.0   0:00 raid1d
   16 root      19  19     0    0     0 SWN   0.0  0.0   0:02 raid1syncd
   17 root     -20 -20     0    0     0 SW<   0.0  0.0   0:00 raid1d
   18 root      19  19     0    0     0 SWN   0.0  0.0   2:41 raid1syncd
  149 root     -20 -20     0    0     0 SW<   0.0  0.0   0:00 raid1d
  150 root      19  19     0    0     0 SWN   0.0  0.0   0:00 raid1syncd

---

7 RAID1 devices, not one of which has used a whole second of CPU time in 37 days of usage. The CTIME for raid1syncd is the time taken reconstructing the arrays after a power failure.

Yeah, well, RAID1 is easy, but all you gain is redundancy (not that that's bad). Higher RAID levels can also improve IO performance, assuming that your RAID controller (be it hardware or software) can keep up, by striping as well. Of course, RAID 0+1 and RAID 10 aren't real difficult either, but they do waste drive space. Of course, most workstation users don't have enough disks to make RAID 5 usable. You, on the other hand, do. You currently are only able to use 50% of your drive space due to mirroring. That could be improved to over 85% with RAID5, even if you leave one drive unused as a hot spare. Yet you haven't. Why? Write performance may be an issue for you, I suppose, which is where RAID5 often loses.

Two drives...many partitions.

You're right though - calculating parity would add significantly to the burden, especially for large writes on a celeron 500. The other factor is that my mobo only has 2 IDE channels so a third disk would have to share a channel which could be nasty. Next upgrade...

Note that the accounting there does not show interrupt handler CPU usage.. which is often the greatest chunk of kernel time spent in a driver.

Cheers

Promise cards are known to change raid formating (even for raid1) between revisions of the same model. true story a friend of mine swaped 2 of the same promise cards, and it lost all his data with mirroring.

Crap.

That's good information, thanks!

Follow-up. Plugging into the third-party Promise controller did not work, it thinks the drives are empty.

I thought promise cards were also known as 'wing and a prayer' controllers.

speaking of RAID controllers, although i've installed a few, whats the processes for replacing a RAID controller, like adaptec or intel, if the controller fails. Is it just a matter of plugging in the new card and it discovers the RAID configuration automatically? Or does it have to scan the drives?

Based on this I've recently installed software RAID1 on two 80GB disks on a Fedora Core 3 test 3 server. While I don't have any actual performance statistics, the array seems just as fast as I'd expect it to be, and the RAID1 process has 0 seconds CPU usage in over four days, including several software installs. That and the idea that there's much more chance of getting the RAID array working if we have to change hardware underneath it sold me.

Thanks for the idea, genixia!

Paul

Again, ps isn't going to accurately show the CPU usage of the RAID processing because it's all done in the kernel.

Much of the RAID's CPU usage will be drawn against processes that actually perform the I/O -- such is the nature of a multi-threaded monolithic style OS.

But Linux s/w RAID does indeed outperform nearly(?) all h/w RAID solutions, so it's kinda hard to go wrong by it. Especially if combined with some form of hotswap capable hardware (SATA, Firewire, USB2, etc..).

Cheers

Back from the dead with some recent real-life numbers.

3.2GHz, P4 Xeon, 9 SATA disk array.

3ware HW RAID5: 32Mb/s write, 159Mb/s read
Linux SW RAID5: 112Mb/s write, 138Mb/s read
Areca HW RAID5: 113Mb/s write, 365Mb/s read

Happy birthday!

While you're all talking about RAID...

I am running (er, not for long) an Abit BP6 with 2x Celeron 500's as my fileserver, router, openvpn server and asterisk play machine under Gentoo.

On the hardware setup before this, I had a linux software RAID1 arrangement with a pair of 120GB Seagates on a Promise 20269 Ultra TX2 controller. It was sweet, and never missed a beat.

Now when I moved to the BP6 that I have atm, I thought I'd just use the Highpoint HPT366 that is onboard. It worked a-ok for a while, and last night it just spazzed out on me. The array resynced and the filesystem (reiserfs. why? i don't know) was saying it needed a fsck.

It was like, bleep bleep bleep bleep bleep bleep and then, like, my filesystem wouldn't mount, and I was like, unngh. It devoured my FLACs, and then I had to rip them again...

The PC had been crashing recently, the logs were filling with

Code:

---

ide: failed opcode was: unknown

hdg: dma_timer_expiry: dma status == 0x21

hdg: DMA timeout error

hdg: dma timeout error: status=0x58 { DriveReady SeekComplete DataRequest }

---

and

Code:

---

Sep 13 07:03:10 peaches APIC error on CPU1: 02(02)

Sep 13 07:04:55 peaches APIC error on CPU0: 08(08)

---

and one drive losing DMA which made resyncing take forever. reiserfsck --rebuild-tree says there is a hardware error.

Basically I'm fed up with this BP6 and its very strange stability (it will be fine for 6 months... and then go crazy... and then miraculously fix itself). Tomorrow it will be replaced with a Abit/VIA/Sempron setup and my old Promise PCI IDE controller.

My Bp6 had many stability issues related to the capacitor problems and APIC stuff. The apic stuff made redhat and fedora go goofy. It was a cool board, but the frustration made me stop using it.

I always confuse the two, but one of them is a power and resources management thing ( I think). It had lots of problems with certain linux distributions. It would just more or less halt. I didn't every bother to try and figure out the problem fully.

Lots of people reporting similar problems (exact same hd messages) on a variety of systems these days. I think somebody broke something deep in the IDE layer in the kernel..

-ml

Most likely, an incompatible memory module. It happened exactly the same thing to me. No beep and no error code of any kind. Try replacing memory with some other brand, and don't thing that having the same specs is enough. I found that theoretically identical dimms as to specifications could still behave differently, unfortunately.

Thanks taym.

I've been using a Samsung original 512MB stick of PC3200 that is new, and I tried a stick of TwinMOS el-cheapo PC3200 from my old PC that I know works. The Samsung stick seems to work alright in my other PC.

The only other stuff that is plugged into the board is an old Matrox G400 16MB AGP which I know works, and nothing else. I think its basically either the chip or the board which is dud, and I have no way of knowing for sure which.

Meh, I took a chance and filed a DOA RMA on the board. Fingers crossed it will get sorted sharpish.

Does the board have one of the 4 pin AUX12V connectors? If thats not connected properly then some boards don't work properly.

Thanks tman

Yep, it has one of those and I have it hooked up. Just to check, I unplugged it and the uGuru chip displayed the correct error code on the little LED display and the speaker beeped a crazed tune at me. Thats how I knew that the speaker worked

Modern mobos should all totally have surface mount speakers and diagnostic displays.

Sorry. Had to ask I've got a AMD Athlon board which needed the AUX12V connected otherwise it would mysteriously crash even when idling.

Having the POST numbers or even diagnostic LEDs are great. The old system of beeps drove me nuts. Was that long long short long or long short long long?

Hehee, totally been there!

Anyway, I think maybe I found the problem, see this thread. Basically I think I need a BIOS upgrade for the CPU I have.