I would always reccomend against installing XAMPP, because those all-in-one solutions don't teach you anything.
But that's just me. I'm pretty sure PHP doesn't install anything to the registry (as I have installed PHP 4 to C:\PHP, and then swapped that directory with just the files from PHP 5, and it worked fine), so you can easily and simply just DEL C:\PHP* /y