Forgive me if I say something obvious. I've made all the mistakes I'm about to mention.
I'm not sure why it will not work. Your httpd.conf file should contain all of the following, not surrounded by any conditional structures (i.e., <IfModule.....):
LoadModule php4_module modules/libphp4.so [your "modules/" path may differ from the above]
AddModule mod_php4.c
AddType application/x-httpd-php .php .phtml
Also, if there is some old "php3" language in the httpd.conf, comment it out.
And finally, an rpm version of php4 (which did not work) inserted a link to an add-on httpd.conf file that I had to comment out. Check to make sure you don't have any extra php4 related language lurking in your httpd.conf.
Hope this helps.