Hi,
You don't have to install each one separately. xampp is what I use and it comes with an installer, just depends on your operating system. I have Windows 2000 Pro. You can do a search at http://sourceforge.net for web servers and lots will come up. They are all free open source packages there. I have had an excellent go with xampp.
I have to connect to the internet, start the Apache server and MySQL server, then I disconnect to work offline. xampp has batch files set up for you to do that very easily. Just read their instructions and you should do fine.
Good luck!
Barb