I'll second Rob's Ripplehost recommendation, with a couple reservations. They don't have any kind of shell access via telnet or SSH, so you need to be happy with using FTP to manage your site. They also go down more than the more expensive hosts, I've had 4-5 outages in the last year, most have been short, but one lasted 12 hours, because the whole operation is run by one guy. Also, I had the owner disable my account once because he got a complaint that spam was coming from my domain, which was completely false.

But for $5/year (maybe it's $10 now), you can't beat the price.