Performance wise, it's not a lot slower. Safety wise, it keeps the OTHER PHP developers on that box from trashing your stuff.
It doesn't really limit you that much. Basically, you can't change a few things like max script execution time or max file upload size I believe. There's a bit on it in the php docs, but that's mostly it.
It forces all your scripts to run as YOU instead of as the web server user, so if you have a database on that box, and only want your apps to see it, you can configure it to only answer your web scripts requests.