So let's say in a file with .php extension I have no php code,
apache first passes it to php parser,
php looks for <? tags, finds none,
then returns processing back to apache.
In that case, the only delay is between the time apache surrenders processing, and the time it regains it. That time would be close to zero because php merely does a quick scan for <? opening tag.
I often use .php by default because I don't want to create a purely html file, then discover down the road that I need php coding in there. In which case one has to hunt down all references to that file and modify the links from .html to .php
From your answer, and my analysis, I would say the performance hit is negligible.