PHP could very well be "installed"; however, if you don't have the server recognize that ".php" is a server-parsed extension, then this could show up.
So if your Apache config (if you're using Apache) doesn't have a line similar to:
AddType application/x-httpd-php .php
Then you have an issue ...
Other than easyPHP there are other packages out there:
XAMPP
Server2Go
WOS
NuSphere Tech Platform
Any one of those will work just as good as the next. It's just a matter of preference for which you use.