In order to be sure your application continues to function under many different configurations, you must continue to test it under those configs.
If those configs are more than one, I find that unacceptable, however I can imagine situations where you have multiple test servers / virtual hosts for testing the application under different configs.
You could reasonably, set up an automated tester which deploys the app in lots of different configurations and runs the test suite on each one in turn to be (reasonably) sure that it will work on different setups.
I imagine that people who make PHP apps for resale / redistribution have to do this, however, I don't. All my apps only run on the configuration they were designed for, which I mandate for the hosting (In most cases I host it too so it's not a major problem).
This is effectively a problem of PHP- there are simply TOO MANY php.ini config options (not to mention compile time ones) too take account of. Testing every possible combination is not even feasible under the setup described above, some shortcuts will be necessary.
This is similar to Win32 software, for example, where many developers have whole networks of machines with different releases of Windows for automated testing (How many versions of Vista are there? And how many languages are they all available in?)
Mark