Reiserfs...

I just patched a kernel up to the 2.0beta and saw reiserfs being patched in... :)

Any clues as to whether the empeg software is going to move across to reiserfs in up coming releases...? or is it something we could use ourselves by manually converting partitions... (would then have to make modifications to the base install after every update to change the mount types before the empeg would be useable again)

Cheers

Kim

Any clues as to whether the empeg software is going to move across to reiserfs in up coming releases...? or is it something we could use ourselves by manually converting partitions...

ReiserFS was added as research work for a different project (which ended up using the 2.4 kernel). I think in theory 2.0 should just deal with the music partitions (/drive0 and /drive1) being ReiserFS. But I haven't tried it for a while and it's certainly not officially supported. Also, there's no way of converting an ext2 partition into a ReiserFS one in-place: you'd have to wipe and resynchronise all your music.

(would then have to make modifications to the base install after every update to change the mount types before the empeg would be useable again)

Not if only the music partitions are ReiserFS; you would, though, always have to stick a new kernel on, as stock car2 kernels don't have ReiserFS enabled.

Peter

What are the benefits of using ReiserFS on the empeg instead of ext2?

­­______________
Edwin de Vaan

Since it's a journalling filesystem, you eliminate the need for the extremely long and tedious fsck's of the disks (the data partitions that is, the smaller partitions for the rest of the system don't take all that long to fsck).

/Michael

You might want to consider ext3 because it does have a migration path from ext2.

-Kit

You might want to consider ext3 because it does have a migration path from ext2.

True. If the Different Project in question were being started today, we'd probably use ext3. At the time, ReiserFS was far more stable.

I'm not sure whether the player would "just cope" if you tried to run it on an ext3 filesystem -- we've never tried it, and it depends how well it pretends to be ext2. Currently we assume that anything that doesn't statfs to REISER_SUPER_MAGIC is ext2.

Peter

Ext3 is ext2, just with a journal file added. At any point one can stop using it as ext3 and revert to ext2 (or the other way around) and everybody is still happy. Very convenient. I'd be particularly suprised if the player software even knew or cared significantly about it, other than perhaps whatever scheme is used to "recognise" the music partitions (use f/s "labels"?).

Newer distributions include userland support for fstype="auto", where if the kernel has ext3 support it mounts as ext3, otherwise as ext2. Very useful for us kernel developers as we see-saw between Alan's (with ext3) and Linus's (no ext3) versions.

I'd be particularly suprised if the player software even knew or cared significantly about it

It needs to know whether to run fsck on sync, and it needs to know what fstype to pass to mount(2).

Peter

You might want to consider ext3 because it does have a migration path from ext2.

True, but there are more advantages to ReiserFS than just being a journaled filesystem.
ReiserFS is exceptionally fast in opening files in huge directories.
This makes it an excellent choice for mail and news spool directories ... and fids directories.

Hmm, well based off this, and a new unit with enough space to hold all the music on one of the two drives, I am going to try it. I'll let everyone know what I find.

>Since it's a journalling filesystem, you eliminate the need
>for the extremely long and tedious fsck's of the disks

Or instead you could just install Hijack and disable "periodic" (a.k.a "nuisance") filesystem checks from the menu. It will still have to do fsck in the event of a lockup any time the disks are mounted rw, but that is exceedingly rare..

-ml

Except when I manage to lock out the shell, have the display off, and can't do squat except reboot with the disks RW.

Does hijack try to make the drives ro before rebooting? If so, I'll have to kick the display out of standby so I can use Hijack to reboot when it happens.

So far, I have a hijack kernel compiled with ReiserFS enabled, and the tools compiled and ready to go. Right now I am just copying the music to the drive I don't intend to format yet, then its on with the conversion.

>Does hijack try to make the drives ro before rebooting?

No, but I suppose I could just hook into the "Magic SysRQ Key" code which already knows how to do that..

I'd like to see this featuer used in your kernel because I have already asked you if you could.

Yes, it would be more secured to ro when rebooting after a shell crash (problem, it happened to me).

Well I finally saved all my music on the second drive, and uploaded the Reiser kernel plus tools. I used the mkresierfs tool, it seems to go fine. But if I try to write anything to the drive, the unit becomes unresponsive on the shell, and hijack shows a load of 4. Here is what I did, with the error being the last line:

sh-2.03# mkreiserfs /dev/hda4





<-----------MKREISERFS, 1999----------->

ReiserFS version 3.5.18

Block size 4096 bytes

Block count 7304848

First 16 blocks skipped

Super block is in 16

Bitmap blocks are :

        17, 32768, 65536, 98304, 131072, 163840, 196608, 229376, 262144, 294912, 327680, 360448, 393216, 425984, 458752, 491520, 524288,

557056, 589824, 622592, 655360, 688128, 720896, 753664, 786432, 819200, 851968, 884736, 917504, 950272, 983040, 1015808, 1048576, 1081344

, 1114112, 1146880, 1179648, 1212416, 1245184, 1277952, 1310720, 1343488, 1376256, 1409024, 1441792, 1474560, 1507328, 1540096, 1572864,

1605632, 1638400, 1671168, 1703936, 1736704, 1769472, 1802240, 1835008, 1867776, 1900544, 1933312, 1966080, 1998848, 2031616, 2064384, 20

97152, 2129920, 2162688, 2195456, 2228224, 2260992, 2293760, 2326528, 2359296, 2392064, 2424832, 2457600, 2490368, 2523136, 2555904, 2588

672, 2621440, 2654208, 2686976, 2719744, 2752512, 2785280, 2818048, 2850816, 2883584, 2916352, 2949120, 2981888, 3014656, 3047424, 308019

2, 3112960, 3145728, 3178496, 3211264, 3244032, 3276800, 3309568, 3342336, 3375104, 3407872, 3440640, 3473408, 3506176, 3538944, 3571712,

 3604480, 3637248, 3670016, 3702784, 3735552, 3768320, 3801088, 3833856, 3866624, 3899392, 3932160, 3964928, 3997696, 4030464, 4063232, 4

096000, 4128768, 4161536, 4194304, 4227072, 4259840, 4292608, 4325376, 4358144, 4390912, 4423680, 4456448, 4489216, 4521984, 4554752, 458

7520, 4620288, 4653056, 4685824, 4718592, 4751360, 4784128, 4816896, 4849664, 4882432, 4915200, 4947968, 4980736, 5013504, 5046272, 50790

40, 5111808, 5144576, 5177344, 5210112, 5242880, 5275648, 5308416, 5341184, 5373952, 5406720, 5439488, 5472256, 5505024, 5537792, 5570560

, 5603328, 5636096, 5668864, 5701632, 5734400, 5767168, 5799936, 5832704, 5865472, 5898240, 5931008, 5963776, 5996544, 6029312, 6062080,

6094848, 6127616, 6160384, 6193152, 6225920, 6258688, 6291456, 6324224, 6356992, 6389760, 6422528, 6455296, 6488064, 6520832, 6553600, 65

86368, 6619136, 6651904, 6684672, 6717440, 6750208, 6782976, 6815744, 6848512, 6881280, 6914048, 6946816, 6979584, 7012352, 7045120, 7077

888, 7110656, 7143424, 7176192, 7208960, 7241728, 7274496

Journal size 8192 (blocks 18-8210 of device 0x3:0x4)

Root block 8211

Used 8434 blocks

ATTENTION: ALL DATA WILL BE LOST ON '/dev/hda4'! (y/n)y

Initializing journal - 0%....20%....40%....60%....80%....100%

Syncing..



ReiserFS core development sponsored by SuSE Labs (suse.com)



Journaling sponsored by MP3.com.



Item handlers sponsored by Ecila.com



To learn about the programmers and ReiserFS, please go to

http://www.devlinux.com/namesys



Have fun.



sh-2.03# mount /dev/hda4 /drive0

clm-2030: reading through journal entries

clm-2036: No valid transactions found

clm-2038: Replayed 0 transactions in 13 seconds, mount_id now 10

ReiserFS version 3.5.18

sh-2.03# cd /drive1/musicback/

sh-2.03# tar -cplf - . | (cd /drive0 && tar xpf -)

vs-7015: linear_search_in_dir_item: item must be 0-th item in block (1)

Yeah.. that test is still in the latest 2.4.xx reiserfs, too.

Stuff like that scares me: the filesystem code should NEVER panic and halt the kernel, and there's way too much of it in reiserfs.

My guess is that it is having trouble with your fids directory, right? You know, the place with upwards of 4000 entries.. it doesn't suprise me that (2.2.xx) resierfs might croak on something like that.

Perhaps I'll move Hijack into the 2.4.xx kernel stream someday.. gotta get the latest patches from the Rio folks for that, though. Rob/Hugo?

-ml

Perhaps I'll move Hijack into the 2.4.xx kernel stream someday.. gotta get the latest patches from the Rio folks for that, though.

Although the HSX-109 uses 2.4 (and uses ReiserFS for its music partitions, for those of you who haven't put two and two together yet), we haven't yet written 2.4 drivers for all the car player hardware.

I think I got ReiserFS working on a car player out of that 2.2 kernel code, but I'm not sure. It certainly involved a lot of faffing about making sure that structures were aligned, added up to the right size, etc. on non-x86 architectures.

Peter

EDIT No, hang on, I didn't. I remember now. I eventually got ReiserFS working on a later 2.2.17 kernel, by tremendous amounts of de-x86isation, but that binary (and thus that source) was never released. I don't think I've even still got a copy: once we decided to move to 2.4 for the HSX, it was obsolete. So anyone trying to build the 2.2.14 Reiser code and having it work, is hosed. Possibly even S.O.L. (question to Americans: what does S.O.L. means?)

My guess is that it is having trouble with your fids directory, right?

Nope, it just craps out the second anything even tries to touch it. No files are on drive0 whatsoever. mkdir, cat, cp, or anything else that would touch the disk spawn that error.

Reading peters note below, it looks like it's not going to work without a ton of tinkering. I'll post my kernel and compiled utilities, and leave it at "still insanely hard" to get working.

(question to Americans: what does S.O.L. means?)

While not american, it's always been my belief that it stood for "sh!t outta luck"

Supporting evidence, for instance: http://www.solscape.com/chat/acronyms.html and http://www.acronymfinder.com/af-query.asp?String=exact&Acronym=SOL

/Michael

Reading peters note below, it looks like it's not going to work without a ton of tinkering.

Yes, sorry about that. As I say, I think I got it working, but I didn't start from those 2.2.14 sources. I had to patch it up to 2.2.17, resolve all the conflicts with the -empeg changes, and then commence on eliminating all the x86 structure size assumptions from its headers (which incidentally made the on-disk format incompatible with x86 2.2 ReiserFS).

For C portability voyeurs, the issue is that ReiserFS assumed that, given struct foo { short x; int y; }, then sizeof(struct foo) is 6. On ARM, whether with GCC or Norcroft, it's 8. You can make it 6 by packing the structure, but then you can't read y with a word load because it isn't aligned. Plus ReiserFS's disc structure packs these things after filenames, the ends of which are byte-aligned but not word-aligned. This is admirably summarised by Drakino's phrase "insanely hard". It took a whole week. Mind you, Hans Reiser wanted $50,000 for the job, which is somewhat more than my weekly salary.

Going to 2.4, in which Reiser et al. had already made similar changes for DEC Alpha compatibility, made life a lot simpler.

Peter

Any chance of the empeg receiving a 2.4 kernel, later on down the road? An ext3 or Reiser file system would receive a warm welcome.

Is it much bulkier in terms of memory hogging than 2.2 is?

Greg

question to Americans: what does S.O.L. mean?

Shite Outta Luck.

(Note how I cleverly translated that back to UK English for you?)

i just wanted to bring back this thread from the dead. i just changed my gentoo box's fs from ext3 to reiserfs, and i had an incredible increase in performance. no one ever made an attempt after mcomb patched up the hijack kernel source to 2.2.17, which according to this post, fixed whatever drakino encountered assuming that all the x86 code was also modified.

is it feasable to attempt this again with hijack installed? would be awesome to be able to take advantage of this awesome fs. time to bust out the crosscompiler.

It's feasable but as noted above by Peter, you'll need to go through and change a lot of the code that assumes certain things about the architecture it's running under.

Just having 2.2.x isn't enough. 2.4.x has most of these changes already done for you but 2.4.x doesn't work on the car player unfortunately. I guess you could back port the 2.4.x ReiserFS code but that is also a lot of work.

We can get most of the gains of using ReiserFS like the journaling by using the Ext3 code and the increased performance when dealing with large directories can be fudged by using Mark Lord's fidshift app along with v3.0a

The other issue is figuring out what performance problems you're having with your empeg. I mean, sure, ReiserFS may increase filesystem performance and free up some IO wait or something, but what are you going to use that extra CPU time or IO throughput for? So far, I don't think anyone has encountered a dearth of CPU time, even running multiple third-party apps. The problem is much more related to physical RAM. So I'm not sure what you'd gain.

That being said, I'll never touch ReiserFS again after it decided to zero out a large portion of the files I had under it. Absolutely ruined a Linux installation.

ReiserFS may increase filesystem performance and free up some IO wait or something, but what are you going to use that extra CPU time or IO throughput for?

Just to dissuade you a little more, you aren't going to get any noticeable IO speed improvements either. The IO is limited by the empegs bus bandwidth and the lack of DMA. Another fs won't change that.

-Mike

boo! even with limited IO, i can't see why the database rebuild won't be hastened with the filesystem's b-tree design (4 lvl depth). iirc, most of the time is serching for a certain fid, then reading it and writing to the database. the whole point of fidsifting was so that the whole search process wouldn't take as long, by partitioning the big directory of fids into smaller chunks. so, even though the empeg's IO is limited by its bus, shaving off time on how long it takes to search for a fid so that it gets into memory faster will probably help. its ok tho if it doesnt. i've always wanted to just mess with the kernel source code. might as well start with this. =)

on a tangent, reiserfs4 seems exciting, and its an interesting read on the whole philosophy on its design. and the preliminary benchmarking is blowing away all the current filesystems (reiserfs3, xfs, jfs).

Just my 2 cents:

After having so much trouble with ReiserFS (I paid my Internetprovider for months just for connectiong me to namesys.com...) I sweared myself never to go back to reiserfs. This system is so complicated, in case of trouble you have to be a filesystem-god. Just the opposite to ext3. So I use ext3 now for some time and am happy.

A filesystem is just as good as it is in case of an failure. Only then you may see if it was a good choice. A little more speed is now quite unimportant for me...

This is an opinion of a man who never programmed such things as kernels or filesystems, I'm just user and programmer, no filesystem-crack.



Rolf