ErrorDocument 404 /404.php
Try that. It should not matter but yea.
If this is the whole .htaccess than that is weird never seen that before. I would try that first. If it does not work, well I am out of ideas there. Make sure there isn't something else that could be overriding the error document line.
--FrosT