You don't need much. If you have Win95, 98, ME, etc. and want to get started quick, try here:
http://badblue.com/helpphps.htm
(small footprint web server and instructions on installing PHP).
If you want to use Apache, try here:
http://www.phpgeek.com
(this will get you PHPtriad including Apache, PHP and MySQL).
Either way, you can start developing stuff locally and then upload it to your hosting provider once you've gotten a working set of PHP files going.
For authoring, you can use any capable text editor. Some people really like HomeSite as well, but I haven't used it.