In general, I agree with you completely, never rely on an external program when PHP can do the same thing natively.
However, GPG uses PKI which makes it do things that mcrypt can't. If you need PKI, then you need PKI, end of story. You use GPG.
As for the availability of an external program, when a client is paying me multiple thousands of dollars to write a security system so that he can get his XML encrypted and delivered, I know that GPG will be be available for the following three reasons: (1) It's on every Linux machine known to man, (2) I can install it manually if it's absent, (3) if I can't install it manually, we move to a different web hosting company because the client asked for PKI and so GPG is a requirement for the project at hand.
mcrypt is great for what it is. And if that's what you need, then I would definitely agree - I suggest using mcrypt over GPG. But if you need PKI, then you make some compromises in your life and you rely on an external program.