But the security holes thing is a red herring.
1: The holes are patched MUCH more quickly than the equivalent holes in IE. When one compares the severity of a hole and the number of days of exposure since it was discovered (NOT since it was disclosed by the vendor, like a lot of sites seem to want to use) until it was patched, Firefox / Mozilla come out WAY ahead of IE. And a fair number of the "holes" in it only affect win32 platforms, so I would put some of the blame on that OS and it's semi-retarded MIME type handling.
2: It's not the number of holes that are found and fixed that matter, it's the ones that are found by black hats, not reported, and never fixed I worry about. And that, sadly, IS not something we can measure easily.
But I'm willing to bet if there's a nasty, exploitable hole that a black hat knows and isn't reporting, it's likely it's in IE, just because IE is a bigger target, as well as the negative feelings a lot of black hatters would have against IE. However, this is all conjecture.