It is indeed possible to recompile PHP without some of the larger modules installed. I personally don't have much experience with compiling it on Windows, but I'm fairly sure you'll need to get a copy of Cygwin's compiler, and for sure a copy of the PHP source.
As to the difference in size, its probably due to the number of bundled modules, and the DRASTIC changes the the Zend engine that PHP is built on.