Well, I tried to get rid of formail, but I realized it is in a directory that I do not have in my site directory even though the URL says it is. (I guess the ISP has this above my directory and redirects all calls to it?)
I doubt the ISP will remove it for me (I will ask them) but I have tried some other things.
I tried creating a directory with the same name actually in my site. No effect.
I tried adding a file to my directory called "formmail.pl." No effect (Even other files I add to this directory are not accessible.
I tried adding a redirect to the .htaccess file in my top level directory. No effect.
BTW, I did find out this is FormMail-clone, a clone of Matt Wright's Original FormMail.cgi. However, I haven't been convinced that this is any less vulnerable to attack.
Any more ideas?