do it, do it, do it, yeah yeah yeah!

...proud to have one of the first Mark I units

I don't think I could be convinced to write a Windows interface, but if you wanted to do that I could supply the back-end. The only catch is all my code will be licensed under GPL.

The idea I have to come up with a normalization offset is to look at the requantized frequency values coming out of the Huffman decoder to determine the absolute peak -- and then compute the gain offset needed to normalize it, applying this gain to the entire bitstream. As it turns out, the way the requantization formula works makes this almost trivial to compute. Each global_gain offset multiple of 4 will affect the frequency value amplitudes by a factor of 2.

I think you already said this, but the only time I could see this not working well is when the bitstream is already "normalized" but the peaks do not correspond with maximum frequency amplitudes the way it is indended to be heard. For example, you may not want to peak-normalize something intended to be quiet with very low peaks throughout. In this case I think some user discretion is needed.

I'll try writing some code to do this anyhow and you can see how well it works.

-v

...If the processing is to be done on the linux side, it might not be a bad idea to build an undo feature into your program; a little script that undoes the changes in case something goes wrong..
-mark

...proud to have one of the first Mark I units

The idea I have to come up with a normalization offset is to look at the requantized frequency values coming out of the Huffman decoder to determine the absolute peak.

Normalization is great, but I don't think the scaling factor should be based on the peak level, as this is a very "local" criterion. RMS (the average energy in the signal, measured over the whole song) sounds much more useful to me. And it shouldn't be much harder to implement than just peak detection.

And if you provide the back-end, I would be willing to do a simple Windows front-end, also GPLed.

Daniel

Normalization is great, but I don't think the scaling factor should be based on the peak level, as this is a very "local" criterion. RMS (the average energy in the signal, measured over the whole song) sounds much more useful to me. And it shouldn't be much harder to implement than just peak detection.

Could you elaborate on this a bit more?

The problem I see is that you don't want to increase the gain so much that the peaks are outside the (-1.0, +1.0) normalized range, otherwise you'll get clipping. How would you propose to calculate an appropriate gain?

And if you provide the back-end, I would be willing to do a simple Windows front-end, also GPLed.

Sounds like we'll get a Windows version one way or another. I have this idea that it could also be run batch-style on the empeg itself...

-v

Could you elaborate on this a bit more?

What Tadzio was saying is: If you just measure the peaks, you're not going to get the desired result.

Technically, the definition of normalizing the file is measuring the highest peak in the song and then bringing up the level of the whole song so that the one peak hits 100%.

But that won't get the desired result. The desired result is to have all the songs at a similar apparent volume. But the only way to do this properly is either with a) compression, or b) getting a more average volume from the track rather than just measuring the peaks.

You're right in that you don't want to adjust the gain so much that it clips the peaks. So no matter what algorithm you choose to implement, you need to measure the peaks so that you can notify the user if they choose to exceed 100% so that they can decide if a few clipped peaks are OK.

Let me elaborate further and give a concrete example. This comes from direct recent experience:

I have an 80's-era album by Yes titled "Big Generator". It's a really good album. But I don't listen to it as much as I should because it's mastered poorly. Its apparent volume level is very quiet. Even on the really loud rocking songs (such as "Almost Like Love", which includes some full-blast Tower Of Power-style horn section work), it seems dull, quiet, and lifeless.

On the other hand, a copy of Madonna's recent "Ray of Light" is mastered in such a way that it seems many many times louder. I'm not talking just a few notches on the volume knob. I mean that the quietest song on Ray of Light seems at least 15db louder than the loudest song on Big Generator. On my Empeg, I have to crank Big Generator all the way up to 0db to make it sound good. But when I listen to Ray of Light, that level would be damaging to my speakers. So for Ray of Light, I have to crank it back to at least -10 or -15 before things are OK again.

But if I take the raw .WAV data from those albums and look at them in a wave editor, I find that they both are already normalized. The peaks on Big Generator hit 100%, and the peaks on Ray of Light also hit 100%.

So what's the difference? Compression.

Compression, normalization, and equalization are among the last things done to an album before it gets pressed onto a CD. The process is known as "Disc Mastering" and there are companies that specialize in doing just that. They know how to take the artist's master tapes and tweak them so that they sound good in the real world.

Ray of Light is mastered like a TV commercial: Highly compressed, so that everything seems loud. Big Generator is mastered more like a classical album, without any attempt to compress the audio before the final master.

Surely it's better to have music without the compression right? No, not necessarily. Even music with lots of dynamics can benefit from compression. The trick is to do it well. The production on Ray of Light is very crisp, and the album, sonically, is amazing to listen to. Big Generator's mix sounds muddy and dull in comparison.

I believe that if you were to apply a standard peak-detection algorithm to either of these albums, there would be little or no change made to the apparent volume of either one.

But if you tried to glean some kind of an average volume from them (not an easy task!), you would probably severly clip the Big Generator album's peaks in an attempt to make it sound as loud as Ray of Light.

I don't see an easy algorithmic solution to this problem. The only thing I can think of is that you can let the user decide themselves how much to adjust the global gain. For instance, in my case, I might try adding just a couple of DB to Big Generator and then listen to see if the clipped peaks are noticeable. But overall, what I'd rather do is reduce the gain of the loud albums instead of adding gain to the quiet albums. Of course, there's probably an issue with a sonic "floor" there, too, and it might be just as bad of a problem. We'll have to experiment and see, I guess.

What a fascinating problem. This really cuts to the root of the issues of digital audio production. Why are recent CDs mastered so differently than older CDs? What is it about the mastering process that makes a disc sound good or bad? Proper compression is truly an art form.



Tony Fabris
Empeg #144

Could you elaborate on this a bit more?

The problem I see is that you don't want to increase the gain so much that the peaks are outside the (-1.0, +1.0) normalized range, otherwise you'll get clipping. How would you propose to calculate an appropriate gain?

Well, to avoid clipping totally, we would have to actually compress the song, as Tony described. This would, however, be much more complicated than just increasing a 'gain' field, so it isn't feasable.

But clipping a few samples doesn't affect sound quality nearly as much as many think. Just to make sure I know what I say, I loaded an already normalized song (Uriah Heep - Lady In Black) into CoolEdit and amplified it by 10dB (factor 3). Clipping, of course, occured all over the place - nearly every beat was clipped. But listening to it, it still sounded good. Of course, I could hear a difference, but this is rather an extreme example.

Thinking about it, I now see that we actually have two questions here:

1. What's the right 'metric' to measure the 'loudness' of a song?

The whole idea of normalization is that you don't have to fumble with the volume control all the time. That means that all the songs should have the same subjective 'loudness'. When we want to write a program that lets the user decide how much he wants to amplify a certain song, then we need to tell him how loud it actually is. If this has to be just one value, then it really should be averaged over the whole song, not just a few samples. The average energy in the sound signal (expressed in dB) is not a bad metric for how loud a human ear senses music. CoolEdit uses RMS, expressed in dBFS (Sine or Square Wave) in its Statistics panel.

2. How can we automatically find the optimum gain?

I agree that this is not trivial. But just using the peak value could even achieve an adverse effect to what we intend. Imagine two songs, both with the same subjective loudness. The first is highly compressed (little dynamic), whereas the second uses all available dynamic. All the samples in the first will be very close to its average value, none of them reaching 100%. Peak normalizing that song would make it subjectively much louder. On the other hand, the uncompressed song will most likely already contain some 100% samples, so that normalizing would not change its loudness at all. Net effect: normalizing actually introduces a difference in loudness instead of eliminating it. (In a nutshell: peak normalization will generally make compressed songs appear louder than uncompressed.)

So, peak detection is not really a good criterion. We want the subjective loudness to be equal, so calculations could be based on what I described under 1. However, I agree that this alone is not good enough, as it could lead to some clipping distortion. What I suggest is that we let the user enter a desired loudness, then check to see how much clipping would occur. If this is over a certain limit, then we can either warn the user, or decrease the target loudness until we fall under that limit.

The definition of this limit needs some more experimentation and discussion, I think. Some ideas:
- not more than x samples - which is ~x/44000 seconds - in a row are clipped
- samples are clipped by no more than y%, i.e. no sample is over (100+y)% fullscale
- clipped energy (max(sample-fullscale, 0), summed up over the whole waveform) is below z.

Oh, and btw: CoolEdit 2000 can already do peak normalization of MP3 files. :-)

Daniel

What a fascinating problem. This really cuts to the root of the issues of digital audio production. Why are recent CDs mastered so differently than older CDs? What is it about the mastering process that makes a disc sound good or bad? Proper compression is truly an art form.

Indeed. I have a live album by Pete Seeger (actually, quite a few of them, but one is afflicted by this problem more than others). It is 'peak normalised', but the peaks are the sound made by Pete's chair being pulled across the floor as he changed his position between songs. Songs themselves are almost inaudible.

But there is a problem with properly mastered albums, as well. A look at Car&Driver tells me that a car with noise level at highway cruising speed much lower than 70 dBA is a quiet car. That means we have some 50 dB between background noise level and bursting of speaker membranes / car windshield / our eardrums (while in a concert hall or a quiet listening room we have almost 100 dB). I am sure that intended difference between barely audible choir at the beginning of Tchaikowsky's 1812 and cannons later on is more than 50 dB. (There's a lot of classical pieces with great dynamic range; Dvorak's 'New World' symphony is one of my favourites I cannot listen to in the car without frequent volume adjustments.)

So, what do we do? Does anybody know of a 'compressor' for wav files? Maybe we should put real-time dynamics compression using empeg's DSP on the wish list?

Comments?

Cheers!

Dragi "Bonzi" Raos
Zagreb, Croatia
#5196

Oh, and btw: CoolEdit 2000 can already do peak normalization of MP3 files.

Without decompressing and recompressing them? Jeez, why am I still using Cool Edit 96, then??!?!?!?!



Tony Fabris
Empeg #144

It is 'peak normalised', but the peaks are the sound made by Pete's chair being pulled across the floor as he changed his position between songs. Songs themselves are almost inaudible.

Too classic. I love it.

Dvorak's 'New World' symphony is one of my favourites I cannot listen to in the car without frequent volume adjustments.

Your point about the car being a bad environment for listening to highly dynamic music is a good one. It makes me rethink my position on Dynamat. Originally, my opinion was that it was a waste of time and money: It's easier and cheaper to buy more powerful amps and speakers than it is to strip down your interior and apply Dynamat. Now I'm not so sure. Highly dynamic music can be a big issue in a noisy car, no matter how loud your system can crank. It's too bad that car noise supression is always such a trade-off (weight/cost vs. supression).

The funny thing is that radio stations have known about this for many years. When you listen to the radio, all that music has been run through a compressor before it's broadcast. They do it because they know that people mostly listen to the radio in their cars, and that they need to compress the dynamics for that environment.

And then there's the issue of television broadcasts. Everyone always complains because the commercials are so much louder than the TV shows. Actually, the comercials aren't louder, they're just more dynamically compressed than the shows are. Some new TV's come with a DSP that does compression/limiting to take care of that problem for you. It's funny how something we dislike about television broadcasts becomes desirable in the car.

So, what do we do? Does anybody know of a 'compressor' for wav files?

Of course. It's easy to compress the dynamics of a .WAV file before encoding it into .MP3. Cool Edit has a very nice compression interface. Although in my version (96), it's tricky to adjust the parameters correctly. There's no preset just for making a CD sound louder-- The presets assume you're working with one voice or one instrument rather than an entire album. So unless you adjust the parameters carefully, you'll get artifacts such as zipper effects and snare drums that sound gated. Like I said, good compression is an art form all by itself.

There's no way to compress the dynamics an MP3 after it's been encoded, of course...

Maybe we should put real-time dynamics compression using empeg's DSP on the wish list?

As I recall, someone already brought that up. But I've read the specs on the DSP, and if I remember correctly, it doesn't include a compressor. Any dynamics compression would have to be done realtime by the Empeg player software. So don't hold your breath.



Tony Fabris
Empeg #144

Without decompressing and recompressing them? Jeez, why am I still using Cool Edit 96, then??!?!?!?!

Well, I didn't say "without decompressing and recompressing".... in fact, it includes the Fraunhofer encoder/decoder, and it decodes the file first, then performes whatever operation you want, then encodes it again. I also didn't say this was lossless...

Daniel

Cool Edit has a very nice compression interface.

Ah, professionals! I meant free compression tool

Anyway, do you think Cool Edit 2000 is worth the price (for a definite amateur - I won't be making any music of my own, but might like to transfer my vinyl records)? Fraunhofer encoder, various filtering options etc. tempt me, and 'grown up' version is certainly too much for me.

Out of curiosity, do you know any site where I could find out about dynamic range compression algorithms?

Thanks!

Dragi "Bonzi" Raos
Zagreb, Croatia
#5196

Tweaking the mp3 files themselves sounds like a definite option, but I don't like the idea of the clipping and thus quality loss that might occur.

I would think that the gain should really be adjusted at the final amplification stage of the empeg itself.
Each file could be tagged with a +ve or -ve adjustment in whatever the granularity output stage allows. This way you could subjectively adjust each file until you get your whole collection just how you want it.
Additionally any of the suggested algorithms for finding the optimum gain could be run over the collection and used as a starting point.

This way you should only have a problem if your adjustment factor for a track pushes the total gain off the top or bottom of the scale.