What is a "respectable rate"? Real time? ?x real time? 0.?x real time? You also need to define the typical MPEG2 input and desired h.264 output. With those three known, your choices will be pretty defined. So far your only clearly defined requirements are "open source software" and "cheap", though cheap is very subjective