This is a matter of "your mileage may vary". Personally, on Windows, I prefer to use the separate installers for everything except PHP, and then install PHP using the zip archive. Some people have faced problems with this method, but I only had it once with MySQL, but that is because I overlooked some documentation - and in fact this method helps you to learn more than you would by using a third party installer. I have seen people having problems with third party installers like XAMPP and WAMP, so they are not necessary fool proof either.
On Linux I use a distribution with satisfactory software packages, so I do not do anything more than install my desired packages.
Anyway, whichever you choose, feel free to ask for help if you run into problems, but do so in the Install forum.