Let's start simple. What OS platform are you using and what OS platform do you plan to be hosting your site on? For example, personally I have a box running Windows NT4 SP5 with IIS4 installed. This is the box I do my coding on. At some point my code might be moved to a Linux box (most likely running Apache). So in the interests of portability over performance, on my NT box I use IIS (since it's there) as my webserver, I have PHP installed, I wrote the database in MS Access, I have configured the ODBC drivers in NT to hook into that database, and I access the database via ODBC calls in PHP.
If I decide to move my setup to a Linux/Apache system, the only thing I will need to do is convert the Access data to some Linux-based database system (PostgreSQL, Interbase, and MySQL come immediately to mind). Then I can configure the Linux box to do ODBC, in which case none of my code needs to change.
Others who also have Windows boxes as their desktops do all their work via telnet/ssh sessions to nix (Unix, Linux, Solaris, FreeBSD, etc.) boxes, in which case they're likely working with Apache & some nix database like MySQL or PostgreSQL.
If you are just starting out, and you don't really care which webserver & database you work with, the large majority of examples out there assume a MySQL/Apache configuration. I personally will not use MySQL until it can stand up to the ACID test, but for me data integrity is more important than performance (MySQL's real strength seems to be its performance). Whether you are using some form of Linux (RedHat Linux comes with pretty much everything you need if you get it on CD, though you may wish to pull down the latest versions of various software packages), or you are using some form of Windows (there are now Windows PHP installers available--see news section of this site--which include basic installation of all the above...PHP, Apache, MySQL...all in one installer), getting things setup is not too difficult.
But first determine where you will be doing your work, and then use that information to help you determine what configuration you should setup.
P.S. And if all you wish to do is learn basic PHP and not do database integration (though this is what I assume the vast majority of PHP coders do), then all you need is a webserver and a coyp of PHP to go with it.