There are several possibilities. Chiefly amongst these are:
1. You're using mysql_pconnect (or some other persistent connection mechanism)
2. Your Apache MaxClients or equivalent is set too high
3. You have some queries which take far too long.
if you're using persistent connections, don't.
Be sure that each page connects only once to the database.
Make sure MaxClients is not too high - this is VERY counter-productive, and its default setting (maybe 100) is often much too high. Most servers can handle a lot less than this, 20-40 is a better setting (depending on traffic).
Try to reproduce it on your dev server, consider getting some specialised hardware / software to generate fake load.
Mark