Odd, since I just now did a:
su - root
cd /usr/local/src/php-4.0.4
./configure --with-ftp
make clean
make
and it worked...
Of course, the output from the .configure script clearly stated that ftp support was not being compiled in, but it just chugged away and built the php executable no complaints. But it built just fine.