If you are using VBR, and the headers are wrong or missing (as with some encoders), you will see this kind of thing happen. See if it is the same file all the time, and verify that the file does not have any encoder problems.