Ok...
PHP is a server side language unlike Javascript which runs in the browser hence making it client side scripting. This means user does not have to have anything besides a browser and a internet connection to execute a PHP website.
The server does need PHP though as he will be doing all the work. Most hosting companies does have PHP installed with their Apache (Web Server), so as soon as Apache sees it is a .php extension it calls PHP to handle it and spit out the parsed HTML.
You can copy the .php file you created but you need to run it on a machine with HTTP Server (Apache and PHP) installed. So taking your flashdisk with you is not a good idea, rather have a temporary hosting location to upload your PHP file to using a FTP client like FileZilla, then you can tell your client to browse to that certain location and he will see what you have done.
Google some hosting companies, when you sign on with them they will give you FTP details and with the domain you chose, you will then upload your php files to the ftp server and simply browse to your domains location, the php file should now run.
When you browse to http://localhost this means that you turned your local machine into a http server that is why you needed to install all those XAMPP/LAMP stuff, this is because your machine (server side) is responsible for parsing the PHP code...
Hope this helps...