It sounds to me like the format is set up to purposely cause these problems, but maybe not.
Next I'd try using "fseek()" and "fgetc()" to see if you can read from after the third byte on, or even start from the end and read backwards.
Also, if you have a hex editor you could open the file in that to see just what the fourth, fifth, etc. bytes are, and what bytes you'd need to add to the front of the file once you get the rest of it read.